tmlib.workflow.corilla package

Module contents

Workflow step for correction of illumination artifacts.

Microscopic images generally display inhomogeneous illumination. Correction of these artifacts is important for visualization and even more so for quantitative analysis of pixel intensities. To this end, the corilla step calculated illumination statistics across all acquisition sites. These statistics are stored and can later be applied to individual images to correct them for illumination artifacts. The step further computes global intensity statistics, which can be useful to uniformly rescale intensities across images.

Submodules

tmlib.workflow.corilla.api module

class tmlib.workflow.corilla.api.IllumstatsCalculator(experiment_id)

Bases: tmlib.workflow.api.WorkflowStepAPI

Class for calculating illumination statistics.

Parameters:

experiment_id: int

ID of parent experiment
collect_job_output(obj, *args, **kwargs)

Not implemented.

create_run_batches(args)

Creates job descriptions for parallel computing.

Parameters:

args: tmlib.workflow.corilla.args.CorillaBatchArguments

step-specific arguments
Returns:

generator

job descriptions
delete_previous_job_output()

Deletes all tmlib.models.file.IllumstatsFile instances of the processed experiment.

has_collect_phase = False
run_job(batch, assume_clean_state=False)

Calculates illumination statistics.

Parameters:

batch: dict

job description

assume_clean_state: bool, optional

assume that output of previous runs has already been cleaned up

tmlib.workflow.corilla.args module

class tmlib.workflow.corilla.args.CorillaBatchArguments(**kwargs)

Bases: tmlib.workflow.args.BatchArguments

Parameters:

**kwargs: dict, optional

keyword arguments to overwrite
class tmlib.workflow.corilla.args.CorillaSubmissionArguments(**kwargs)

Bases: tmlib.workflow.args.SubmissionArguments

Parameters:

**kwargs: dict, optional

keyword arguments to overwrite

tmlib.workflow.corilla.cli module

class tmlib.workflow.corilla.cli.Corilla(api_instance, verbosity)

Bases: tmlib.workflow.cli.WorkflowStepCLI

Calculation of illumination statistics over a set of images belonging to the same channel. The resulting statistics can subsequently be applied to individual images to correct them for illumination artifacts.

Parameters:

api_instance: tmlib.workflow.corilla.api.IllumstatsCalculator

instance of API class to which processing is delegated

verbosity: int

logging level
collect(obj, *args, **kwargs)

Not implemented.

corilla

Calculation of illumination statistics over a set of images belonging to the same channel. The resulting statistics can subsequently be applied to individual images to correct them for illumination artifacts.
experiment_id

ID of the experiment that should be processed

-h, --help

show this help message and exit

--verbosity, -v

increase logging verbosity

corilla experiment_id cleanup

cleans up the output of a previous submission, i.e. removes files and database entries created by previously submitted jobs
-h, --help

show this help message and exit

corilla experiment_id info

prints the description of a given batch job to the console
-h, --help

show this help message and exit

--job, -j

ID of the job for which information should be displayed

--phase {collect,run}, -p {collect,run}

phase of the workflow step to which the job belongs

corilla experiment_id init

creates batches for parallel processing and thereby defines how the computational task should be distrubuted over the cluster (also cleans up the output of previous submissions)
-h, --help

show this help message and exit

corilla experiment_id log

prints the log output of a given batch job to the console
-h, --help

show this help message and exit

--job, -j

ID of the job for which log output should be shown

--phase {collect,init,run}, -p {collect,init,run}

phase of the workflow step to which the job belongs

corilla experiment_id resubmit

resubmits previously created jobs for “run” and “collect” phases to the cluster and monitors their status upon processing
-h, --help

show this help message and exit

--depth <index>, -d <index>

number of child tasks that should be monitored (default: 1)

--interval <seconds>, -i <seconds>

seconds to wait between monitoring iterations (default: 10)

corilla experiment_id run

runs an invidiual batch job on the local machine
-h, --help

show this help message and exit

--assume-clean-state

assume that previous outputs have been cleaned up

--job <job_id>, -j <job_id>

ID of the job that should be run

corilla experiment_id submit

creates batch jobs for the “run” and “collect” phases, submits them to the cluster and monitors their status upon processing (requires a prior “init”)
-h, --help

show this help message and exit

--depth <index>, -d <index>

number of child tasks that should be monitored (default: 1)

--interval <seconds>, -i <seconds>

seconds to wait between monitoring iterations (default: 10)

--cores <number>

number of cores that should be allocated to each “run” job (may be increased in case memory requirements of a job exceed resources of a single core) (default: 1)

--duration <hh:mm:ss>

walltime that should be allocated to a each “run” job in the format “HH:MM:SS” (may need to be adapted depending on the choice of batch size) (default: 06:00:00)

--memory <mb>

amount of memory in megabytes that should be allocated to each “run” job (default: 2147)

tmlib.workflow.corilla.stats module

Calculation of illumination statistics, which can subsequently be applied to individual images in order to correct them for illumination artifacts [1] .

References

[R7]Stoeger T, Battich N, Herrmann MD, Yakimovich Y, Pelkmans L. 2015. “Computer vision for image-based transcriptomics”. Methods.
[R8]Welford BP. 1962. “Note on a method for calculating corrected sums of squares and products”. Technometrics 4(3):419-420.
class tmlib.workflow.corilla.stats.OnlineStatistics(image_dimensions, decimals=3)

Bases: object

Class for calculating online statistics (mean and variance) element-by-element on a series of numpy arrays based on Welford’s method [2] . For more information see Wikipedia article “Algorithms for calculating variance”.

Parameters:

image_dimensions: Tuple[int]

dimensions of the pixel array

decimals: int

precision after the comma that determines the number of percentiles that will be calculated
mean

tmlib.image.IllumstatsImage: mean values

percentiles

Dict[float, int]: calculated percentiles (rounded to integer values)

std

tmlib.image.IllumstatsImage: standard deviation values

update(image, log_transform=True)

Update statistics with additional image.

Parameters:

image: tmlib.image.ChannelImage

additional image

log_transform: bool, optional

log10 transform image (default: True)
var

numpy.ndarray[float]: variance