tmlib.workflow.jterator package

Module contents

Workflow step for building and running image analysis pipelines.

The objective of image analysis is to identify meaningful objects (e.g. “cells”) in the images and extract features for the identified objects. The jterator step provides users an interface to combine individual modules available via the JtModules repository into custom image analysis pipelines in a CellProfiler-like manner. Outlines of segmented objects and extracted feature values can be stored for further analyis or interactive visualization in the viewer.

Submodules

tmlib.workflow.jterator.api module

class tmlib.workflow.jterator.api.ImageAnalysisPipelineEngine(experiment_id, pipeline_description=None, handles_descriptions=None)

Bases: tmlib.workflow.api.WorkflowStepAPI

Class for running image analysis pipelines.

Parameters:

experiment_id: int

ID of the processed experiment

pipeline_description: tmlib.workflow.jterator.description.PipelineDescription, optional

description of pipeline, i.e. module order and paths to module source code and descriptor files (default: None)

handles_descriptions: Dict[str, tmlib.workflow.jterator.description.HandleDescriptions], optional

description of module input/output (default: None)

collect_job_output(batch)

Computes the optimal representation of each SegmentationLayer on the map for zoomable visualization.

Parameters:

batch: dict

job description

create_debug_run_jobs(user_name, job_collection, batches, verbosity, duration, memory, cores)

Creates debug jobs for the parallel “run” phase of the step.

Parameters:

user_name: str

name of the submitting user

job_collection: tmlib.workflow.job.RunPhase

empty collection of run jobs that should be populated

batches: List[dict]

job descriptions

verbosity: int

logging verbosity for jobs

duration: str

computational time that should be allocated for a single job; in HH:MM:SS format

memory: int

amount of memory in Megabyte that should be allocated for a single

cores: int

number of CPU cores that should be allocated for a single job

Returns:

tmlib.workflow.jobs.RunPhase

run jobs

create_debug_run_phase(submission_id)

Creates a job collection for the debug “run” phase of the step.

Parameters:

submission_id: int

ID of the corresponding Submission

Returns:

tmlib.workflow.job.RunPhase

collection of debug “run” jobs

create_run_batches(args)

Creates job descriptions for parallel computing.

Parameters:

args: tmlib.workflow.jterator.args.BatchArguments

step-specific arguments

Returns:

generator

job descriptions

delete_previous_job_output()

Deletes all instances of MapobjectType that were generated by a prior run of the same pipeline as well as all children instances for the processed experiment.

figures_location

str: location where figure files are stored

has_collect_phase = True

pipeline

List[tmlib.jterator.module.ImageAnalysisModule]: pipeline built based on :class`PipelineDescription <tmlib.workflow.jterator.description.PipelineDescription>` and HandleDescriptions.

remove_previous_pipeline_output()

Removes all figure files.

run_job(batch, assume_clean_state)

Runs the pipeline, i.e. executes modules sequentially. After successful completion of the pipeline, instances of MapobjectType, Mapobject, SegmentationLayer, MapobjectSegmentation, Feature and FeatureValues are created and persisted in the database for subsequent visualization and analysis.

Parameters:

batch: dict

job description

assume_clean_state: bool, optional

assume that output of previous runs has already been cleaned up

start_engines()

Starts engines required by non-Python modules in the pipeline. This should be done only once, since engines may have long startup times, which would otherwise slow down the execution of the pipeline.

tmlib.workflow.jterator.args module

class tmlib.workflow.jterator.args.JteratorBatchArguments(**kwargs)

Bases: tmlib.workflow.args.BatchArguments

Parameters:

**kwargs: dict, optional

keyword arguments to overwrite

batch_size

int: number of sites that should be processed per job

plot

bool: whether plotting should be activated

class tmlib.workflow.jterator.args.JteratorSubmissionArguments(**kwargs)

Bases: tmlib.workflow.args.SubmissionArguments

Parameters:

**kwargs: dict, optional

keyword arguments to overwrite

tmlib.workflow.jterator.description module

class tmlib.workflow.jterator.description.HandleDescriptions(version, input, output)

Bases: object

Description of a jterator module’s parameters and return value.

Parameters:

version: str

module version

input: List[dict]

description of module input parameters

output: List[dict]

description of module return value

input

List[Union[tmlib.workflow.jterator.handles.InputHandle, tmlib.workflow.jterator.handles.PipeHandle]]: input handles

output

List[Union[tmlib.workflow.jterator.handles.OutputHandle, tmlib.workflow.jterator.handles.PipeHandle]]: output handles

to_dict()

Returns attributes “version”, “input” and “output” as key-value pairs.

Returns:dict

version

str: version of the module

class tmlib.workflow.jterator.description.PipelineChannelInputDescription(name, correct=True)

Bases: object

A Channel that should be made available to the pipeline.

Parameters:

name: str

name of the channel

correct: bool, optional

whether images should be corrected (default: True)

correct

bool: whether images should be corrected for illumination artifacts

name

str: name of the channel

to_dict()

Returns attributes “name” and “correct” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.description.PipelineDescription(input, pipeline, output)

Bases: object

Description of a jterator pipeline.

Parameters:

input: Dict[str, List[Dict[str, str]]]

description of pipeline input

pipeline: List[Dict[str, str]]

description of pipeline

output: Dict[str, List[Dict[str, str]]]

description of pipeline output

input

tmlib.workflow.jterator.description.PipelineInputDescription: input that should be available to modules in the pipeline

output

tmlib.workflow.jterator.description.PipelineOutputDescription: output of a pipeline that should be persisted

pipeline

List[tmlib.workflow.jterator.description.PipelineModuleDescription]: modules that should be run

to_dict()

Returns attributes “input”, “pipeline” and “output” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.description.PipelineInputDescription(channels=[], objects=[])

Bases: object

Input of a jterator pipeline.

Parameters:

channels: List[dict], optional

description of channels input

objects: List[dict], optional

description of objects input

channels

List[tmlib.workflow.jterator.description.PipelineChannelInputDescription]: pipeline channels input

objects

List[tmlib.workflow.jterator.description.PipelineObjectInputDescription]: pipeline objects input

to_dict()

Returns attributes objects”, and “channels” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.description.PipelineModuleDescription(handles, source, active=True)

Bases: object

Parameters:

handles: str

path to the .handles.yaml file

source: str

name of the module source code file in the jtmodules package

active: bool, optional

whether the module should be run (default: True)

active

bool: whether module should be run

handles

str: path to handles file

name

source

str: name of the source

to_dict()

Returns attributes “handles”, “source” and “active” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.description.PipelineObjectInputDescription(name)

Bases: object

A MapobjectType that should be made available to the pipeline.

Parameters:

name: str

name of the object type

name

str: name of the objects type

to_dict()

Returns attribute “name” as key-value pair.

Returns:dict

class tmlib.workflow.jterator.description.PipelineObjectOutputDescription(name, as_polygons=True)

Bases: object

Registered SegmentedObjects that should be persisted as MapobjectType.

Parameters:

name: str

name of the object type

as_polygons: bool, optional

whether objects should be represented as polygons (if False only centroid coordinates will be stored; default: True)

as_polygons

bool: whether object should be represented as polygons

name

str: name of the objects type

to_dict()

Returns attributes “name” and “as_polygons” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.description.PipelineOutputDescription(objects)

Bases: object

Description of the output of a jterator pipeline.

Parameters:

objects: List[dict]

description of objects input

objects

List[tmlib.workflow.jterator.description.PipelineObjectOutputDescription]: pipeline objects input

to_dict()

Returns attributes “name” and “objects” as key-value pairs.

Returns:dict

tmlib.workflow.jterator.cli module

class tmlib.workflow.jterator.cli.Jterator(api_instance, verbosity)

Bases: tmlib.workflow.cli.WorkflowStepCLI

Application of a sequence of algorithms to a set of images to segment the images and extract features for the identified objects.

Parameters:

api_instance: tmlib.workflow.jterator.api.ImageAnalysisPipelineEngine

instance of API class to which processing is delegated

verbosity: int

logging verbosity

check()

Checks pipeline and module descriptor files

create(repo_dir, skel_dir)

Creates a new project on disk

Parameters:

repo_dir: str

path to local copy of jtlib repository

skel_dir: str

path to a directory that should serve as a project skeleton

debug(site_id, plot)

Runs an invidiual site on the local machine for debugging

Parameters:

plot: bool

whether figures should be generated by modules

site_id: int

ID of the site that should be run

remove()

Removes an existing project

run(job_id, assume_clean_state)

Runs an invidiual job on the local machine

Parameters:

assume_clean_state: bool

assume that previous outputs have been cleaned up

job_id: int

ID of the job that should be run

jterator

Application of a sequence of algorithms to a set of images to segment the images and extract features for the identified objects.

experiment_id

ID of the experiment that should be processed

-h, --help

show this help message and exit

--verbosity, -v

increase logging verbosity

jterator experiment_id check

checks pipeline and module descriptor files

-h, --help

show this help message and exit

jterator experiment_id cleanup

cleans up the output of a previous submission, i.e. removes files and database entries created by previously submitted jobs

-h, --help

show this help message and exit

jterator experiment_id collect

collects the output of run jobs, i.e. performs a post-processing operation that either cannot be parallelized or needs to be performed afterwards

-h, --help

show this help message and exit

jterator experiment_id create

creates a new project on disk

-h, --help

show this help message and exit

--repo_dir <repo_dir>

path to local copy of jtlib repository

--skel_dir <skel_dir>

path to a directory that should serve as a project skeleton

jterator experiment_id debug

runs an invidiual site on the local machine for debugging

-h, --help

show this help message and exit

--plot

whether figures should be generated by modules

--site <site_id>, -s <site_id>

ID of the site that should be run

jterator experiment_id info

prints the description of a given batch job to the console

-h, --help

show this help message and exit

--job, -j

ID of the job for which information should be displayed

--phase {collect,run}, -p {collect,run}

phase of the workflow step to which the job belongs

jterator experiment_id init

creates batches for parallel processing and thereby defines how the computational task should be distrubuted over the cluster (also cleans up the output of previous submissions)

-h, --help

show this help message and exit

--batch-size <batch_size>, -b <batch_size>

number of sites that should be processed per job (default: 100)

--plot, -p

whether plotting should be activated

jterator experiment_id log

prints the log output of a given batch job to the console

-h, --help

show this help message and exit

--job, -j

ID of the job for which log output should be shown

--phase {collect,init,run}, -p {collect,init,run}

phase of the workflow step to which the job belongs

jterator experiment_id remove

removes an existing project

-h, --help

show this help message and exit

jterator experiment_id resubmit

resubmits previously created jobs for “run” and “collect” phases to the cluster and monitors their status upon processing

-h, --help

show this help message and exit

--depth <index>, -d <index>

number of child tasks that should be monitored (default: 1)

--interval <seconds>, -i <seconds>

seconds to wait between monitoring iterations (default: 10)

jterator experiment_id run

runs an invidiual job on the local machine

-h, --help

show this help message and exit

--assume-clean-state

assume that previous outputs have been cleaned up

--job <job_id>, -j <job_id>

ID of the job that should be run

jterator experiment_id submit

creates batch jobs for the “run” and “collect” phases, submits them to the cluster and monitors their status upon processing (requires a prior “init”)

-h, --help

show this help message and exit

--depth <index>, -d <index>

number of child tasks that should be monitored (default: 1)

--interval <seconds>, -i <seconds>

seconds to wait between monitoring iterations (default: 10)

--cores <number>

number of cores that should be allocated to each “run” job (may be increased in case memory requirements of a job exceed resources of a single core) (default: 1)

--duration <hh:mm:ss>

walltime that should be allocated to a each “run” job in the format “HH:MM:SS” (may need to be adapted depending on the choice of batch size) (default: 06:00:00)

--memory <mb>

amount of memory in megabytes that should be allocated to each “run” job (default: 2147)

tmlib.workflow.jterator.handles module

A Handle describes a key-value pair which is either passed as an argument to a Jterator module function or is returned by the function. The approach can be considered a form of metaprogramming, where the object extends the code of the actual module function via its properties and methods. This is used to assert the correct type of arguments and return values and enables storing data generated by modules to make it accessible outside the scope of the module or retrieving data from the store when required by modules. The object’s attributes are specified as a mapping in a handles YAML module descriptor file.

class tmlib.workflow.jterator.handles.BinaryImage(name, key, help='')

Bases: tmlib.workflow.jterator.handles.MaskImage

Class for an image handle, where pixels of the image are either background (False) or foreground (True).

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str

help message

value

numpy.ndarray[numpy.bool]: pixels/voxels array

class tmlib.workflow.jterator.handles.Boolean(name, value, help='', options=[True, False])

Bases: tmlib.workflow.jterator.handles.Scalar

Handle for a boolean input argument.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value: bool

value of the item, i.e. the actual argument of the function parameter

help: str, optional

help message (default: "")

options: List[bool]

possible values for value

class tmlib.workflow.jterator.handles.Character(name, value, help='', options=[])

Bases: tmlib.workflow.jterator.handles.Scalar

Handle for a character input argument.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value: basestring

value of the item, i.e. the actual argument of the function parameter

help: str, optional

help message (default: "")

options: List[basestring]

possible values for value

class tmlib.workflow.jterator.handles.Figure(name, help='')

Bases: tmlib.workflow.jterator.handles.OutputHandle

Handle for a figure whose value is a JSON string representing a figure created by a module, see Plotly JSON schema.

Parameters:

name: str

name of the item, which must match a parameter of the module function

key: str

name that should be given to the objects

help: str, optional

help message (default: "")

value

str: JSON representation of a figure

class tmlib.workflow.jterator.handles.Handle(name, help)

Bases: object

Abstract base class for a handle.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

help: str

help message

type

str: handle type

class tmlib.workflow.jterator.handles.Image(name, key, help)

Bases: tmlib.workflow.jterator.handles.PipeHandle

Base class for an image handle.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str

help message

iter_planes()

Iterates over 2D pixel planes of the image.

Returns:

Generator[Tuple[Tuple[int], numpy.ndarray]]

pixels at each time point and z-level

iter_volumes()

Iterates over 3D voxel volumes of the image.

Returns:

Generator[Tuple[int, numpy.ndarray]]

voxels at each time point

value

numpy.ndarray: pixels/voxels array

class tmlib.workflow.jterator.handles.InputHandle(name, value, help)

Bases: tmlib.workflow.jterator.handles.Handle

Abstract base class for a handle whose value is used as an argument for a module function.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value:

the actual argument of the module function parameter

help: str

help message

to_dict()

Returns attributes “name”, “type”, “help” and “value” as key-value pairs.

class tmlib.workflow.jterator.handles.IntensityImage(name, key, help='')

Bases: tmlib.workflow.jterator.handles.Image

Class for an intensity image handle, where image pixel values encode intensity.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str, optional

help message (default: "")

value

Returns:numpy.ndarray[Union[numpy.uint8, numpy.uint16]]: pixels/voxels array

class tmlib.workflow.jterator.handles.LabelImage(name, key, help='')

Bases: tmlib.workflow.jterator.handles.MaskImage

Class for an image handle, where connected pixel components of the image are labeled such that each component has a unique positive integer label and background is zero.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str

help message

value

numpy.ndarray[numpy.int32]: pixels/voxels array

class tmlib.workflow.jterator.handles.MaskImage(name, key, help='')

Bases: tmlib.workflow.jterator.handles.Image

Class for an image handle, where pixels of the image with zero values represent background and pixels with values greater than zero represent foreground.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str

help message

value

numpy.ndarray: pixels/voxels array

class tmlib.workflow.jterator.handles.Measurement(name, objects, objects_ref, channel_ref=None, help='')

Bases: tmlib.workflow.jterator.handles.OutputHandle

Handle for a measurement whose value is a list of two-dimensional labeled arrays with n rows and p columns, where n is the number of segmented objects and p the number of features that were measured for the referenced segmented objects.

Parameters:

name: str

name of the item, which must match a parameter of the module function

objects: str

object type to which features should are assigned

objects_ref: str

reference to object type for which features were extracted (may be the same as objects)

channel_ref: str, optional

reference to channel from which features were extracted (default: None)

help: str, optional

help message (default: "")

to_dict()

Returns attributes “name”, “type”, “help”, “objects”, “objects_ref” and “channel_ref” as key-value pairs.

value

List[pandas.DataFrame]: features extracted for segmented objects at each time point

class tmlib.workflow.jterator.handles.Numeric(name, value, help='', options=[])

Bases: tmlib.workflow.jterator.handles.Scalar

Handle for a numeric input argument.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value: int or float

value of the item, i.e. the actual argument of the function parameter

help: str, optional

help message (default: "")

options: List[int or float]

possible values for value

class tmlib.workflow.jterator.handles.OutputHandle(name, help)

Bases: tmlib.workflow.jterator.handles.Handle

Abstract base class for a handle whose value is returned by a module function.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

help: str

help message

to_dict()

Returns attributes “name”, “type” and “help” as key-value pairs.

value

value returned by module function

class tmlib.workflow.jterator.handles.PipeHandle(name, key, help)

Bases: tmlib.workflow.jterator.handles.Handle

Abstract base class for a handle whose value can be piped between modules, i.e. returned by one module function and potentially passed as argument to another.

Parameters:

name: str

name of the item, which must either match a parameter of the module function in case the item represents an input argument or the key of a key-value pair of the function’s return value

key: str

unique and hashable identifier; it serves as lookup identifier to retrieve the actual value of the item

help: str

help message

to_dict()

Returns attributes “name”, “type”, “help” and “key” as key-value pairs.

value

Data that’s returned by module function and possibly passed to other module functions.

class tmlib.workflow.jterator.handles.Plot(name, value=False, help='', options=[True, False])

Bases: tmlib.workflow.jterator.handles.InputHandle

Handle for a plot that indicates whether the module should generate a figure or rather run in headless mode.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value: bool, optional

whether plotting should be activated (default: False)

help: str, optional

help message (default: "")

options: List[bool]

possible values for value

class tmlib.workflow.jterator.handles.Scalar(name, value, help='', options=[])

Bases: tmlib.workflow.jterator.handles.InputHandle

Abstract base class for a handle for a scalar input argument.

Parameters:

name: str

name of the item, which must match a parameter of the module function

value: str or int or float or bool

value of the item, i.e. the actual argument of the function parameter

help: str, optional

help message (default: "")

options: List[str or int or float or bool]

possible values for value

class tmlib.workflow.jterator.handles.SegmentedObjects(name, key, help='')

Bases: tmlib.workflow.jterator.handles.LabelImage

Class for a segmented objects handle, which represents a special type of label image handle, where pixel values encode segmented objects that should ultimately be visualized by TissueMAPS and for which features can be extracted.

Parameters:

name: str

name of the item

key: str

name that should be assigned to the objects

add_measurement(measurement)

Adds an additional measurement.

Parameters:

measurement: tmlib.workflow.jterator.handles.Measurement

measured features for each segmented object

add_polygons(polygons, y_offset, x_offset, dimensions)

Creates a label image representation of segmented objects based on global map coordinates of object contours.

Parameters:

polygons: List[List[Tuple[Union[int, shapely.geometry.polygon.Polygon]]]]

label and polygon geometry for segmented objects at each z-plane and time point

y_offset: int

global vertical offset that needs to be subtracted from y-coordinates

x_offset: int

global horizontal offset that needs to be subtracted from x-coordinates

dimensions: Tuple[int]

y, x dimensions of image pixel planes

Returns:

numpy.ndarray[numpy.int32]

label image

is_border

Dict[Tuple[int], pandas.Series[bool]]: True if object lies at the border of the image and False otherwise

iter_points(y_offset, x_offset)

Iterates over point representations of segmented objects. The coordinates of the centroid points are relative to the global map, i.e. an offset is added to the image site specific coordinates.

Parameters:

y_offset: int

global vertical offset that needs to be subtracted from y-coordinates (y-axis is inverted)

x_offset: int

global horizontal offset that needs to be added to x-coordinates

Returns:

Generator[Tuple[Union[int, shapely.geometry.point.Point]]]

time point, z-plane, label and point geometry

iter_polygons(y_offset, x_offset)

Iterates over polygon representations of segmented objects. The coordinates of the polygon contours are relative to the global map, i.e. an offset is added to the image site specific coordinates.

Parameters:

y_offset: int

global vertical offset that needs to be subtracted from y-coordinates (y-axis is inverted)

x_offset: int

global horizontal offset that needs to be added to x-coordinates

Returns:

Generator[Tuple[Union[int, shapely.geometry.polygon.Polygon]]]

time point, z-plane, label and polygon

labels

List[int]: unique object identifier labels

measurements

List[pandas.DataFrame]: features extracted for segmented objects at each time point

represent_as_polygons

bool: whether objects should be represented as polygons

save

bool: whether objects should be saved

class tmlib.workflow.jterator.handles.Sequence(name, value, help='')

Bases: tmlib.workflow.jterator.handles.InputHandle

Class for a sequence input argument handle.

Parameters:

name: str

name of the item, which must match a parameter of the module function

mode: str

mode of the item, which defines the way it can be handled by the program

value: List[str or int or float]

value of the item, i.e. the actual argument of the function parameter

help: str, optional

help message (default: "")

tmlib.workflow.jterator.handles.create_handle(type, **kwargs)

Factory function to create an instance of an implementation of the tmlib.workflow.jterator.handles.Handle abstract base class.

Parameters:

type: str

type of the handle item; must match a name of one of the implemented classes in tmlib.workflow.jterator.handles

**kwargs: dict

keyword arguments that are passed to the constructor of the class

Returns:

tmlib.jterator.handles.Handle

Raises:

AttributeError

when type is not a valid class name

TypeError

when an unexpected keyword is passed to the constructor of the class

tmlib.workflow.jterator.module module

class tmlib.workflow.jterator.module.CaptureOutput

Bases: dict

Class for capturing standard output and error and storing the strings in dictionary.

Examples

with CaptureOutput() as output:

foo()

class tmlib.workflow.jterator.module.ImageAnalysisModule(name, source_file, handles)

Bases: object

Class for a Jterator module, the building block of an image analysis pipeline.

Parameters:

name: str

name of the module

source_file: str

name or path to program file that should be executed

handles: tmlib.workflow.jterator.description.HandleDescriptions

description of module input/output as provided

build_figure_filename(figures_dir, job_id)

Builds name of figure file into which module will write figure output of the current job.

Parameters:

figures_dir: str

path to directory for figure output

job_id: int

one-based job index

Returns:

str

absolute path to the figure file

keyword_arguments

dict: name and value of each input handle as key-value pairs

language

str: language of the module (e.g. “Python”)

run(engine=None)

Executes a module, i.e. evaluate the corresponding function with the keyword arguments provided by tmlib.workflow.jterator.handles.

Parameters:

engine: matlab_wrapper.matlab_session.MatlabSession, optional

engine for non-Python languages, such as Matlab (default: None)

update_handles(store, headless=True)

Updates values of handles that define the arguments of the module function.

Parameters:

store: dict

in-memory key-value store

headless: bool, optional

whether plotting should be disabled (default: True)

Returns:

List[tmlib.jterator.handles.Handle]

handles for input keyword arguments

update_store(store)

Updates store with key-value pairs that were returned by the module function.

Parameters:

store: dict

in-memory key-value store

Returns:

store: dict

updated in-memory key-value store

tmlib.workflow.jterator.project module

class tmlib.workflow.jterator.project.AvailableModules

Bases: object

Container for information about Jterator modules available in a local copy of the JtModules repository.

See also

tmlib.config.LibraryConfig.modules_home

handles

Handles files are assumed to reside in a subfolder called “handles” and have the suffix “.handles.yml”.

Returns:

List[dict]

name and description for each handles file

module_files

List[str]: absolute paths to module files

module_languages

List[str]: languages of the modules (determined from file suffixes)

module_names

List[str]: names of the modules (determined from file names)

pipe_registration

Build pipeline elements for registration in the UI in the format excepted in the “pipeline” section in the pipeline file.

Returns:

List[dict]

pipeline elements

to_dict()

Returns attributes as key-value pairs

Returns:dict

class tmlib.workflow.jterator.project.Handles(name, description)

Bases: object

Representation of the content of a .handles.yaml file.

Parameters:

name: str

module name

description: tmlib.workflow.jterator.description.HandleDescriptions

module description

description

tmlib.workflow.jterator.description.HandleDescriptions: module description

name

str: module name

to_dict()

Returns attributes “name” and “description” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.project.Pipe(description)

Bases: object

Representation of the content of the pipeline.yaml file.

Parameters:

description: tmlib.workflow.jterator.description.PipelineDescription

pipeline description

description

tmlib.workflow.jterator.description.PipelineDescription: pipeline description

to_dict()

Returns attribute “description” as key-value pairs.

Returns:dict

class tmlib.workflow.jterator.project.Project(location, pipeline_description=None, handles_descriptions=None)

Bases: object

A project is defined as a folder containing a pipeline.yaml file and a handles subfolder with zero or more .handles.yaml files. The class holds information about the project, in particular on the content of the pipeline.yaml and .handles.yaml module descriptor files and provides methods for retrieving, updating and removing the project.

Parameters:

location: str

path to the project folder

pipeline_description: tmlib.workflow.jterator.description.PipelineDescription, optional

pipeline description (default: None)

handles_descriptions: Dict[str, tmlib.workflow.jterator.description.HandleDescriptions], optional

module descriptions (default: None)

create(skel_dir=None)

Creates a Jterator project: Creates an empty “handles” subfolder as well as a skeleton pipeline file, i.e. a pipeline descriptor file with all required main keys but an empty module list. When skel_dir is provided, pipeline and handles files are copied.

Parameters:

skel_dir: str, optional

path to repository directory that represents a project skeleton, i.e. contains a pipeline and one or more handles files in a handles directory.

handles

List[tmlib.workflow.jterator.project.Handles]: module descriptions

pipe

tmlib.workflow.jterator.project.Pipe: pipeline description

pipe_file

str: absolute path to the pipeline descriptor file

remove()

Removes a Jterator project.

save()

Saves a Jterator project: Updates the content of pipeline and handles files on disk according to modifications to the pipeline and module descriptions.

to_dict()

Returns attributes “pipe” and “handles” as key-value pairs.

Returns:dict

tmlib.workflow.jterator.utils module

tmlib.workflow.jterator.utils.determine_language(filename)

Determines language of a module from filename suffix.

Parameters:

filename: str

name of a module file

Returns:

str