Modules

flexCALC includes three modules: process, analyze and batch.

Various preprocessing utilities are available via modules process and analyze. Module batch allows to build a pipeline for processing large amounts of data on a single workstation.

flexcalc.process module

@author: Alex Kostenko This module contains calculation routines for processing of the data.

flexcalc.process.affine(array, matrix, shift)

Apply 3x3 rotation matrix and shift to a 3D arrayset.

flexcalc.process.allign_moments(array, axis=0)

Compute orientations of the volume intensity moments and allign them with XYZ. Align the primary moment with vertical axis - use axis == 0.

flexcalc.process.append_tile(array, geom, tot_array, tot_geom)

Append a tile to a larger arrayset. Args:

array: projection stack geom: geometry descritption tot_array: output array tot_geom: output geometry

flexcalc.process.append_volume(array, geom, tot_array, tot_geom, ramp=10)

Append a volume array to a larger arrayset. Args:

array: projection stack geom: geometry descritption tot_array: output array tot_geom: output geometry

flexcalc.process.autocrop(array, geom=None)

Auto_crop the volume and update the geometry record.

flexcalc.process.equalize_intensity(master, slave, mode='percentile')

Compute 99.99th percentile of two volumes and use it to renormalize the slave volume.

flexcalc.process.equivalent_density(projections, geometry, energy, spectr, compound, density, preview=False)

Transfrom intensity values to projected density for a single material array

flexcalc.process.expand_medipix(array)

Get the correct image size for a MEDIPIX data (fill in extra central lines)

flexcalc.process.find_shift(volume_m, volume_s)

flexcalc.process.generate_stl(array, geometry)

Make a mesh from a volume.

flexcalc.process.hard_threshold(array, mode='histogram', threshold=0)

Returns a binary map based on the threshold value. Args:

array (ndarray) : data array (implicit) mode (str) : ‘histogram’, ‘otsu’ or ‘constant’ threshold (float): threshold value if mode = ‘constant’

flexcalc.process.interpolate_holes(array, mask2d, kernel=[1, 1])

Fill in the holes, for instance, saturated pixels.

Args:

mask2d: holes are zeros. Mask is the same for all projections. kernel: size of the interpolation kernel

flexcalc.process.interpolate_lines(proj)

Interpolate values of the horizontal read out lines of the flexray flat panel detector.

flexcalc.process.interpolate_zeros(array, kernel=[1, 1], epsilon=1e-09)

Fill in zero volues, for instance, blank pixels.

Args:

kernel: Size of the interpolation kernel epsilon: if less than epsilon -> interpolate

flexcalc.process.norm(array, type='L2')

Compute L2 norm of the array.

flexcalc.process.optimize_modifier(values, projections, geometry, samp=[1, 1, 1], key='axs_tan', metric='correlation', update=True, preview=False)

Optimize a geometry modifier using a particular sampling of the projection array.

flexcalc.process.optimize_modifier_multires(projections, geometry, step, guess=None, subscale=1, key='axs_tan', metric='correlation', update=True, preview=False)

flexcalc.process.optimize_rotation_center(projections, geometry, guess=None, subscale=1, centre_of_mass=False, metric='highpass', preview=False)

Find a center of rotation. If you can, use the center_of_mass option to get the initial guess. If that fails - use a subscale larger than the potential deviation from the center. Usually, 8 or 16 works fine!

flexcalc.process.preprocess(array, flats=None, darks=None, mode='sides', dim=1)

Apply flatfield correction based on availability of flat- and dark-field.

Args:

flats (ndarray): divide by flats darks (ndarray): subtract darks mode (str): “sides” to use maximum values of the detector sides to estimate the flat field or a mode of intensity distribution with “single”. dim (int): dimension that represents the projection number

flexcalc.process.process_flex(path, *, sample=1, skip=1, memmap=None, index=None, proj_number=None, correct, correct_vol_center=True)

Read and process the array.

Args:

path: path to the flexray array sample: skip: memmap: index: proj_number (int): force projection number (treat lesser numbers as missing)

Return:

proj: min-log projections meta: meta array

flexcalc.process.register_astra_geometry(proj_fix, proj_mov, geom_fix, geom_mov, subsamp=1)

Compute a rigid transformation that makes sure that two reconstruction volumes are alligned. Args:

proj_fix : projection data of the fixed volume proj_mov : projection data of the fixed volume geom_fix : projection data of the fixed volume geom_mov : projection data of the fixed volume

Returns:

geom : geometry for the second reconstruction volume

flexcalc.process.register_volumes(fixed, moving, subsamp=2, use_moments=True, use_CG=True, use_flips=False, threshold='otsu')

Registration of two 3D volumes.

Args:

fixed (array): reference volume moving (array): moving/slave volume subsamp (int): subsampling of the moments computation use_itk (bool): if True, use congugate descent method after aligning the moments treshold (str): can be None, ‘otsu’ or ‘histogram’ - defines the strategy for removing low intensity noise

Returns:

flexcalc.process.residual_rings(array, kernel=[3, 3])

Apply correction by computing outlayers .

flexcalc.process.rotate(array, angle, axis=0)

Rotates the volume via interpolation.

flexcalc.process.scale(array, factor, order=1)

Scales the volume via interpolation.

flexcalc.process.soft_threshold(array, mode='histogram', threshold=0)

Removes values smaller than the threshold value. Args:

array (ndarray) : data array (implicit) mode (str) : ‘histogram’, ‘otsu’ or ‘constant’ threshold (float): threshold value if mode = ‘constant’

flexcalc.process.translate(array, shift, order=1)

Apply a 3D tranlation.

flexcalc.analyze module

@author: Alex Kostenko This module contains data analysis routines.

flexcalc.analyze.binary_threshold(data, mode='histogram', threshold=0)

Compute binary threshold. Use ‘histogram, ‘otsu’, or ‘constant’ mode.

flexcalc.analyze.bounding_box(array)

Find a bounding box for the volume based on intensity (use for auto_crop).

flexcalc.analyze.calibrate_spectrum(projections, volume, geometry, compound='Al', density=2.7, threshold=None, iterations=1000, n_bin=10)

Use the projection stack of a homogeneous object to estimate system’s effective spectrum. Can be used by process.equivalent_thickness to produce an equivalent thickness projection stack. Please, use conventional geometry.

flexcalc.analyze.centre(array)

Compute the centre of the square of mass.

flexcalc.analyze.find_marker(array, geometry, d=5)

Find a marker in 3D volume by applying a circular kernel with an inner diameter d [mm].

flexcalc.analyze.get_background(array, mode='histogram')

Get the background intensity.

flexcalc.analyze.histogram(data, nbin=256, rng=[], plot=True, log=False)

Compute histogram of the data.

flexcalc.analyze.intensity_range(data)

Compute intensity range based on the histogram.

Returns:

a: position of the highest spike (typically air) b: 99.99th percentile c: center of mass of the histogram

flexcalc.analyze.moment2(array, power, dim, centered=True)

Compute 2D image moments (weighed averages) of the data.

sum( (x - x0) ** power * data )

Args:

power (float): power of the image moment dim (uint): dimension along which to compute the moment centered (bool): if centered, center of coordinates is in the middle of array.

flexcalc.analyze.moment3(array, order, center=array([0., 0., 0.]), subsample=1)

Compute 3D image moments $mu_{ijk}$.

Args:

data(array): 3D dataset order(int): order of the moment center(array): coordinates of the center subsample: subsampling factor - 1,2,4…

Returns:

float: image moment

flexcalc.analyze.moments_orientation(data, subsample=1)

Find the center of mass and the intensity axes of the image.

Args:

data(array): 3D input subsample: subsampling factor to to make it faster

Returns:

T, R: translation vector to the center of mass and rotation matrix to intensity axes

flexcalc.batch module

class flexcalc.pipeline.Buffer(path, writer_node, shape=(1, 1, 1), dtype='float32')

Bases: object

Each node has an input and output buffer. It will be in read-only or write-only state. Buffer can store a memmap data and a metadata record.

property dshape

property dtype

get_data()

Read the data array.

get_geom()

Read the meta record.

get_misc()

Read the meta record.

set_data(data)

Write the data array.

set_geom(geom)

Write the meta record.

set_misc(misc)

Write the misc record.

suicide()

Remove file from disk and delete variables.

switch_offline()

Switch buffers to offline to be able to serialize node tree and save them on disk.

switch_online()

Switch buffers to online after loading the node tree from disk or after backing it up.

switch_readonly()

Switch the buffer into reading mode.

switch_writeonly()

Switch the buffer into writing mode.

class flexcalc.pipeline.Node(scheduler, arguments, inputs)

Bases: object

Class responsible for processing of a single block of data. It has two buffers: input and output. Three states: waiting, active, ready Three methods: start, action, finish

activate()

Switch input buffers to read-only mode and output buffers to write-only.

cleanup()

Remove files:

deactivate()

Kill all inputs. Switch all outputs to read-only mode.

get_children()

Get children nodes.

get_inputs(index)

Get data, geom and misc from a buffer using index

get_outputs(index)

Get data, geom and misc from a buffer using index

get_parents()

Get parent nodes.

init_outputs(count)

Create output buffers.

initialize()

Initializtion callback. Override this in sub-classes

isready()

Check if the node is ready.

node_name = 'Default node'

node_type = 0

offline()

Switch node to offline (switch its buffers to offline). This is needed for backing up nodes.

online()

Switch node to online (switch its buffers to online). This is needed for backing up nodes.

runtime()

Runtime callback. Override this in sub-classes

set_outputs(index, data, geom=None, misc=None)

Set data, geom and misc from a buffer using index

state2str()

Report my state.

class flexcalc.pipeline.adjust_geometry_node(scheduler, arguments, inputs)

Bases: Node

Adjust geometry using a user-specified callback.

NB: Geometries may get reset or adjusted further by certain types of nodes later in the pipeline.

node_name = 'Adjust Geometry'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.autocrop_node(scheduler, arguments, inputs)

Bases: Node

Apply autocrop.

node_name = 'Autocrop'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.batch_node(scheduler, arguments, inputs)

Bases: Node

A standard batch node based on a given callback function. Callback function is saved as the first argument in the argument list.

node_name = 'batch'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.beamhardening_node(scheduler, arguments, inputs)

Bases: Node

Apply beam hardening based on a single material approximation and an estimated spectrum.

node_name = 'Beam-hardening correction'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.bin_node(scheduler, arguments, inputs)

Bases: Node

Apply binning.

node_name = 'Bin'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.cast2type_node(scheduler, arguments, inputs)

Bases: Node

Apply autocrop.

node_name = 'Cast to type'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.crop_node(scheduler, arguments, inputs)

Bases: Node

Apply crop.

node_name = 'Crop'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.derotate_node(scheduler, arguments, inputs)

Bases: Node

Derotate projections by geom[‘det_roll’].

node_name = 'Derotate projections'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.display_node(scheduler, arguments, inputs)

Bases: Node

Display data.

node_name = 'Display'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.fdk_node(scheduler, arguments, inputs)

Bases: Node

Feldkamp reconstruction.

node_name = 'FDK'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.flatlog_node(scheduler, arguments, inputs)

Bases: Node

Apply flat-field and dark-field correction. Take -log(x).

node_name = 'Flatlog'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.info_node(scheduler, arguments, inputs)

Bases: Node

Print data info.

node_name = 'Buffer Info'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.markernorm_node(scheduler, arguments, inputs)

Bases: Node

Find marker and normalize data using its intensity.

node_name = 'Marker normalization'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.optimize_node(scheduler, arguments, inputs)

Bases: Node

Use auto-focusing to optimize geometry parameters. Its a group node - it will wait untill all previous nodes are ready before activating.

initialize()

Initializtion callback. Override this in sub-classes

node_name = 'Optimize'

node_type = 1

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.pad_node(scheduler, arguments, inputs)

Bases: Node

Apply autocrop.

node_name = 'Pad'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.proj_merge_node(scheduler, arguments, inputs)

Bases: Node

Merge projections node.

initialize()

Initializtion callback. Override this in sub-classes

node_name = 'Merge projections'

node_type = 1

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.reader_node(scheduler, arguments, inputs)

Bases: Node

initialize()

Initiallization callback of read_data.

node_name = 'Reader'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.registration_node(scheduler, arguments, inputs)

Bases: Node

Register volumes.

initialize()

Initializtion callback. Override this in sub-classes

node_name = 'Registration'

node_type = 1

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.scheduler(_scratch_path_, clean_scratch=True)

Bases: object

Class responsible for scheduling tasks by creating tree of processing nodes connected via buffers. Scheduler links to the root_node that provides an entry point to the node tree.

FDK(vol_shape=None, sirt=0)

Schedule FDK reconstruction.

Args:

vol_shape : force reconstruction volume shape sirt : run a few iterations of SIRT with non-negativity constraint

adjust_geometry(callback)

Adjust geometry.

Args: callback (callable): user-provided function that takes a list of geometry and a list of shapes, and returns a list of new geometries

autocrop()

Schedule autocrop operation.

backup()

Save the node tree on disk.

beamhardening(file, compound, density)

Single material beamhardening based on a file with an effective spectrum record.

Args:

file : filepath of the spectrum record compound: chemical formula of the specimen material density : density in g / cm3

bin(dim)

Schedule a bin operation.

buffer_info()

Print data and meta info.

cast2type(dtype, bounds=None)

Schedule a cast to type operation.

Args:

cleanup()

Remove files after a succesfull run.

crop(dim, width)

Schedule a crop operation.

derotate(ang=None)

Schedule a to derotate the detector.

Args:

ang: angle to derotate with, if None - get it from geom[‘det_roll’].

display(display_type, **argin)

draw_nodes()

Draw the node tree.

flatlog(usemax=False, flats='', darks='', sample=1, transpose=[1, 0, 2], updown=True)

Read flats and darks and apply them to projection data or use ‘usemax’ mode to perform a data-driven correction.

generic(callback, **arguments)

Schedule a generic batch node with one input and one output using any given callback function.

markernorm(norm, size=5)

Find a marker and normalize density of that marker to match the given value

Args:

norm : value used for normalization size : size of the marker (diametre in mm)

merge(mode='projections')

Schedule a data merge operation.

Args:

mode(str): use ‘projections’ or ‘volume’, depending on the type of the input.

optimize(values, key='axs_hrz', tile_index=None, sampling=[5, 1, 1], metric='highpass')

Optimize a parameter using parameter range, geometry key, tile number and sub-sampling.

pad(width, dim, mode='linear')

Schedule padding operation.

read_data(paths, name, *, sampling=1, shape=None, dtype='float32', format=None, transpose=[1, 0, 2], updown=True, proj_number=None, correct, correct_vol_center=True)

Schedule an image stack reader. Often will be the first node in the queue.

Args:

registration(subsamp=1, use_moments=False)

Register volumes.

report()

Print the node tree.

restore_nodes()

Load the node tree from disk.

run()

Run scheduled nodes.

schedule(node_class, arguments=[])

Schedule nodes.

soft_threshold(mode, threshold=None)

Removes values smaller than the threshold value.

Args:

mode (str) : ‘histogram’, ‘otsu’ or ‘constant’ threshold (float): threshold value if mode = ‘constant’

write_data(path, name, dim=0, skip=1, compress=True)

Schedule an image stack writer.

Args:

class flexcalc.pipeline.threshold_node(scheduler, arguments, inputs)

Bases: Node

Apply a soft threshold.

node_name = 'Soft threshold'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.vol_merge_node(scheduler, arguments, inputs)

Bases: Node

Merge volumes node.

initialize()

Initializtion callback. Override this in sub-classes

node_name = 'Merge volumes'

node_type = 1

runtime()

Runtime callback. Override this in sub-classes

class flexcalc.pipeline.writer_node(scheduler, arguments, inputs)

Bases: Node

Write data on disk.

node_name = 'Writer'

node_type = 0

runtime()

Runtime callback. Override this in sub-classes

Module contents