User guide

TissueMAPS uses the client-server model, where clients can make requests to the server via a REST API using the Hypertext Transfer Protocol (HTTP).

Most users will interact with the server via the browser-based interface. However, additional HTTP client implementations are provided via the tmclient package, which allows users to interact more programmatically with the server. These use cases are covered in the section interacting with the server.

The server handles client requests, but delegates the actual processing to the tmlibrary package. The TissueMAPS library provides active programming (API) and command line interfaces (CLI), which can also be used directly, i.e. in a server-independent way. This is covered in the section using the library.

Interacting with the server

This section demonstrates different ways of interacting with a TissueMAPS server. All of them of course require access to a running server instance. This may either be a production server that you access via the internet or a development server running on your local machine. The following examples are given for localhost, but they similarly apply a server running on a remote host.


Here, we connect to the server using URL http://localhost:8002. The actual IP address of localhost is (by convention). It’s possible to use the name localhost, because this host is specified in /etc/hosts. So when you are running the TissueMAPS server on a remote host in the cloud instead of your local machine, you can use the same trick and assign a hostname to the public IP address of that virtual machine. To this end, add a line to /etc/hosts, e.g.   tmaps. You will then be able to connect to the server via http://tmaps. This can be convenient, because you don’t have to remember the exact IP address (which may also be subject to change in case you don’t use a static IP address). Note that you don’t need to provide the port for the production server, because it will listen to port 80 by default (unlike the development server, who listens to port 8002).

User interface

Enter the IP address (and optionally the port number) of the server in your browser. This directs you to the index site and you are asked for your login credentials.


Login prompt.

Enter username and password into the provided forms.

User panel

After successful authorization, you will see an overview of your existing experiments.


Experiment overview.

Empty list because no experiments have been created so far.
Creating an experiment

To create a new experiment, click on create_new_exp_button.


Experiment creation.

Provide information about the image acquisition process of the experiment.

When you click on create_exp_button, the experiment gets created and you get directed back to the overview.


Experiment overview.

The created experiment is listed.


By default, experiments can only be viewed and modified by the user who created them, but they can be shared with other users. However, this functionality is currently only available via the API (see ExperimentShare).

Next, you can upload images and process them. To this end, click on modify_button, which directs you to the workflow manager.

Workflow manager


Workflow manager.

Interface for uploading and processing images. At the top of the page there is a button for upload and one for each stage of the canonical workflow.
Uploading image files

To begin with, add a new plate, by clicking on create_plate_button.


Plate creation.

Provide a name and optionally a description for the plate.

Plate overview.

The created plate is now listed. It is not yet ready for processing, because it doesn’t contain any acquisitions yet.

Select the created plate, by clicking on the link plate_link.


Acquisition overview.

Empty list because no acquisitions have been added so far for the selected plate.

Add a new acquisition, by clicking on create_acq_button.


Acquisition creation.

Provide a name and optionally a description for the acquistion.

Acquisition overview.

The created acquisition is now listed. It has status “WAITING” because no images have yet been uploaded.

Select the created acquisition, by clicking on the link acq_link.



Either drag and drop files or folders into the dedicated area or select them from your local filesystem by clicking on select_files_button. Then click on upload_files_button to start the upload process.

Upload in process.

You can monitor the upload status of individual files.


File upload via the user interface works reliable for serveral thousand images. When uploading tens or hundreds of thousands of images, we recomment uploading files via the command line instead. To this end, you can use the tm_client tool provided by the tmclient package.


The upload process will be interrupted when the page is reloaded. However, you can simply add the files afterwards again and restart uploading. The server keeps track of which files have already been uploaded and won’t upload them again.


Plate overview.

The loading bar in the top right corner indicates that the upload was successful and is “TERMINATED”. The plate is now ready for processing.

You can add additional acquisitions and plates to the experiments by repeating the steps described above. Once you have uploaded all files, you can continue to process them.

Processing images

Once you have uploaded all files, you can proceed to the subsequent processing stages.


You are prevented from proceeding until upload is complete. Requesting this information from the server may take a few seconds for large experiments.


Workflow control panel.

You can toggle between different stages and steps. The view applies to the currently active combination of stage and step - in this case “image convesion” and “metaextract”, respectively.

The loading bars indicate the progress of workflow, stage and step. They are green by default, but turn red as soon as a single job failed. Above the loading bars, you can see the current processing state (e.g. “SUBMITTED”, “RUNNING”, or “TERMINATED”). When the currently active stage or step is in state “RUNNING”, the cog wheel above the loading bar will also start spinning. The cog will also appear on the stage and step tabs to indicate the state of stages or steps, which are not selected at the moment.

In the main window you can set “batch” and “submission” arguments to control the partition of the computational task into individual jobs and the allocation of resources for each job, respectively. Upon submission, individual jobs will be listed below the argument section. If required arguments are missing for a stage or step, this will be indicated on the corresponding tab by a minus (visible for the “image analysis” stage). In this case, you cannot submit these stages without providing the missing arguments.

Since the workflow hasn’t been submitted yet, the loading bars are all set to zero and no jobs are listed.

You can click through all stages and steps and set arguments according to your needs. Once you are happy with your settings, you can either save_button the settings or submit_button the workflow for processing (settings get also automatically saved upon submission).


Submission depends on the current view. Only the currently active stage as well as stages to the left will be processed. For example, if you are in stage “image preprocessing” and you click on submit_button, only stages “image conversion”, “image preprocessing” and “pyramid creation” will be submitted.


Arguments batch_size and duration depend on each other. The larger the batch, i.e. the more images are processed per compute unit, the longer the job will typically take.


Arguements cores and memory depend upon the available compute resources, i.e. the number of CPU cores and the amount of RAM available per core. The defauls of 3800MB applies to the default machine type flavor at ScienceCloud at University of Zurich and may need to be adapted for other clouds.

For now, let’s submit the workflow from the first stage “image conversion”.


Workflow submission in progress.

After submitting the workflow from stage “image conversion”, the state of step “metaconfig” switches to “RUNNING” and individual jobs are listed. Some of the jobs are already “TERMINATED”, while some are still “RUNNING”. Note the cogs on the tabs of the other steps indicating that they have also been submitted for processing.

Workflow submission done.

All steps of stage “image conversion” have been processed and all jobs have “TERMINATED” sucessfully, noticeable by the fully loaded green bars as well as the green checks on stage/step tabs and individual job items.


Once a workflow has been submitted, you can safely close the window or disconnect from the internet, since the jobs are processed remotely on the server in an asynchronous manner.

Once stage “image conversion” is done, you can proceed to any other stage and click on resume_button. Alternatively, you could have submitted a further downstream stage in the first place.

For the purpose of this demo, we will proceed to stage “pyramid creation” and resume the workflow.


Workflow submission resumed.

After successful termination of stage “image conversion”, the workflow has been resumed from stage “pyramid creation”, which is currently in state “NEW”.


Once stage “pyramid creation” is done, you can already view the experiment. However, you won’t be able to visualize segmented objects on the map. To this end, you first need to process stage “image analysis”.

You can further resubmit_button the workflow with modified arguments from any stage afterwards.

The image analysis stage is a bit more complex, therefore we will cover it in a separate section.

Setting up image analysis pipelines

Image analysis stage.

Notice the “extra arguments” section, which hasn’t been present in any of the other previous stages. You are required to select a “pipeline”, which should be processed by the “jterator” step. Since no pipeline has been created so far, the drop-down menue is empty.

To begin with, you need to create a pipeline. To this end, click on create_pipe_button. Give the pipeline a descriptive name, here we call it test-pipe. This will direct you to a separate interface for defining the pipeline.


Jterator interface.

On the left side in the “Available Modules” column, you find all modules implemented in the jtmodules package. “Pipeline Settings” describes the input for the pipeline in form of “channels” and lists modules that have been added to the pipeline. The module “Module Settings” section describes input arguments and expected outputs of the currently selected module.

Pipeline and module settings.

You can drag and drop modules from the list of available modules into the indicated field in the pipeline section and then click on added item to set module parameters. The order of modules in the pipeline can be rearranged by dragging them up or down.

You can further select channels in the input section to make them available to the pipeline. Additional channels can be removed when neeeded. The selected “channels” become available as an input for the selected module.


Images for all channels selected in the input section will be loaded into memory (for the acquisition site corresponding to the given batch). So remove any channel you don’t use in your pipeline to gain performance.


Module renaming.

A module can be renamed by clicking on the textfield in the respective pipeline item. Enter a new name in the provided field and press enter. Note, that the name in the “Module Settings” column remains unchanged. It continues to refer to the module source file.


Names of modules in the pipeline must be unique. When adding the same module twice, it will be automatically renamed by appending it with a number. Be aware that names of module outputs must be hashable and therefore also unique. Best practice is to use to use the module as a namespace: <module_name>.<output_argument_name>, e.g. smooth.smoothed_image for the above example. Since module names must be unique the resulting output will consequently have a unique name, too.

Add all modules to the pipeline that you need for your analysis and set parameters.


Types of input parameters are checked internally. Only inputs matching the type definition of the input argument are listed in the drop-down menue.

Here, we will first add all the modules required to segment “Nuclei” and “Cells” in the images.


Example segmentation pipeline.

This pipeline identifies primary (“Nuclei”) and secondary objects (“Cells”): The image corresponding to channel “wavelength-1” is smoothed and subsequently thresholded. The resulting mask is then labeled to define individual primary objects. The primary objects are subsequently expanded using a watershed transform of the smoothed image belonging to channel “wavelength-2”, which generates secondary objects.

The pipeline can be saved at any time by clicking on save_button. This will save the pipeline settings as well as settings of each module in the pipeline.

When all required parameters are set, the pipeline can be submitted by clicking on submit_button (submission will automatically save the pipeline as well).


Pipeline submission.

Up to ten jobs can be maximally submitted for a pipeline.

To see which acquisition sites the jobs map to, click on list_jobs_button.


Job list.

The table shows the name of “plate” and “well” as well as the “x” and “y” coordinate of each site corresponding to a particular job. This is intended to help you select job IDs for testing your pipeline, such that you include images from different wells or positions within a well.


In the workflow panel you can set a batch_size for the “jterator” step. However, when you submit the pipeline for testing in the jterator user interface, batch_size will be automatically set to 1, such that only one acquisition site will be processed per job.

Once submitted, jobs get cued and processed depending on available computational resources. If you have access to enough compute units, all jobs will be processed in parallel.


Pipeline results.

Results of individual jobs are listed in the “Results” column. figure_button is active for the currently selected module.

When clicking on figure_button, the figure for the respective job is displayed in fullscreen mode.


Module figures.

Figures are interactive. Pixels values are displayed when hovering over images. You can also zoom into plots to have a closer look. Be aware, however, that plots may have a reduced resolution.


Plotting needs to be explicitely activated for a module by selecting true for argument “plot”. This is done to speed up processing of the pipeline.

When clicking on log_button, the log output for the respective job is displayed. The messages includes the log of the entire pipeline and is the same irrespective of which module is currently active.


Pipeline log outputs.

Standard output and error are caputered for each pipeline run. The logging level is set to INFO by default.

To save segmented objects and be able to assign values of extracted features to them, objects need to be registered using the register_objects modules. From a user perspective, the registration simply assigns a name to a label image.


Object registration.

Assign a unique, but short and descriptive name to each type of segmented objects that you want to save. To this end, objects need to be provided in form of a labled image, where each object has a unique ID, as output by the label module, for example.

When we are happy with the segmentation results, we can add addtional modules for feature extraction.


All extracted features will be automatically saved. Since the resulting I/O will increase processing time, its recommended to exclude measurement modules from the pipeline for tuning segmentation parameters.


You can inactivate modules by clicking on eye_open_symbol without having to remove them from the pipeline. Just be aware that this may affect downstream modules, since the output of inactivated modules will of course no longer be produced.


You can quickly move down and up in the pipeline in a Vim-like manner using the j and k keys, respectively.


Feature extraction.

Select a previously registerd object type for which you would like to take a measurement. Some features, such as intensity require an additional raster image. Others, such as morphology measure only object size and shape and are thus independent of the actual pixel intensity values.


Feature names follow a convention: <class>_<statistic>_<channel>. In case features are intensity-independent, the name reduces to <class>_<statistic>. For the above example this would result in Intensity_mean_wavelength-2 or Morphology_area.

Once you have set up your pipeline, save your pipeline (!) and return to the workflow panel. Select the created pipeline and submit the “image analysis” stage by clicking on resume_button. In contrast to submissions in the jterator user interface, this will now submit all jobs and potentially run more than one pipeline per job in a sequential manner, depending on the specified batch_size.


Image analysis submission.

Select the created pipline in the drop-down menu. In case the pipeline doesn’t show up, you may have to reload_button the workflow settings.


Once you’ve setup your experiment, you can view it by returning to the user panel and clicking on view_button.


The interactive MAP is the centerpiece of TissueMAPS (as the name implies).


Viewer overview.

Upon initial access, the first channel is shown in the viewport at the maximally zoomed-out resultion level. You can zoom in and out using either the mouse wheel or trackpad or the + and - buttons provided at the top left corner of the viewport. The map can also be repositions within the viewport by dragging it with the mouse.

To the right of the viewport is the map sidebar and to the left the tool sidebar. Sections of the map control sidebar can be resized using the mouse and individual items can be rearranged via drag and drop. Below the viewport are sliders to zoom along the z-axis or time series for experiment comprised of images acquired at different z resolutions or time points, respectively.

The map sidebar has the following sections:

  • Channels: one raster image layer for each channel (created during the “pyramid_creation” workflow stage)
  • Objects: one vector layer for each object type (created during the “image_analysis” workflow stage)
  • Selections: tool for selecting mapobjects on the map
  • Saved results: one vector layer for each saved (previously generated) tool result
  • Current result: single vector layer for the most recent tool result

Individual sections are described in more detail below.


Map sidebar: Channels.

Channels are represented on the map in form of raster images. Individual channel layers can be toggled as well as dynamically colorized and rescaled. By default, channels are shown in grayscale. When multiple channels are active, colors are additively blended (e.g. red + green = yellow). Pixel intensities are mapped to 8-bit for map representation. However, intensities value shown below sliders reflect the original bit range.

Map sidebar: Objects.

Objects are represented on the map in form of vector graphics. Individual object layers can be toggled as well as dyncamically colorized. In addition, the opacity of object outlines can be adapted. When multiple objects are active, colors are additively blended.


Objects of type “Plates”, “Wells” and “Sites” will be auto-generated based on available image metadata. These static types are independent of parameters set in the “image_analysis” workflow stage.


Object outlines may not be represented 100% accurately on the map, because the polygon contours might have been simplified server side.


Map sidebar: Selections.

Objects can be selected and assigned to different groups. A map marker will be dropped at for selected object. An object can be unselected by clicking on it again using the same selection item. It is further possible to assign an object to more than one selection. The respective object layer will automatically be activated for the choosen mapobject type.
Data analysis tools

TissueMAPS provides a plugin framework for interactive data analysis tools. Available tools are listed in the tool sidebar to the left of the viewport.


Tool sidebar.

Each tool is associated with a separate window, which opens when the corresponding tool icon is clicked in the tool sidebar.

The window content varies between tools depending on their functionality. Typically, there is a section for selection of object types and features and a button to submit the tool request to the server. In case of the supervised classification (SVC) tool, there is also a section for assigning selections to label classes, which can be used for training of the classifier.

Let’s say you want to perform a supervised classification using the “SVC” tool based on labels provided in form of map selections (see above). To perform the classification, select an object type (e.g. Cells) and one or more features from and click on classify_button. This will submit a request to the server to perform the computation. Once the classification is done the result will appear in the “Current result” section of the map control sidebar.


Map sidebar: Current result.

Once a tool result is available a layer will appear in the “Current result” section. Similar to object layers, they are represented on the map as vector graphics. In contrast to the object layers, however, the filled objects are shown instead outlines. Result layers can also be toggled and the opacity can be changed to reveal underlying channel layers (or other tool result layers).

Map sidebar: Saved results.

When additional tool requests become available, the “Current result” moves to “Saved results” and gets replaced with the more recent result. Multiple results can be active simultaneously and their colors are additively blended. Transparency of result layers can be controlled independently. Here, we performed an additional unsupervised classification, using the same features and number of classes as in the supervised case, and can now visually compare the results of both analysis on the map.

RESTful programming

Clients use the REST API to access server side resources. In case of the user interface this is handled by the browser, but the same can be achieved in more programmatic, browser-independent way.

A request is composed of a resource specification provided in form of a Uniform Resource Locator (*URL*) and one of the following verbs: GET, PUT, POST or DELETE. The server listens to routes that catch request messages, handles them and returns a defined response message to the client. This response includes a status code (e.g. 200) and the actual content. In addition, requests and responses have headers that hold information about their content, such as the media type (e.g. application/json or image/png).

Consider the following example: Let’s say you want to GET a list of your experiments. To this end, you can send the following request to the TissueMAPS server:

GET /api/experiments

The server would handle this response via the get_experiments() view function and respond with this message (using the example given in the user interface section):

HTTP/1.1 200 OK
Content-Type: application/json

    "data": [
            "id": "MQ==",
            "name": "test",
            "description": "A very nice experiment that will get me into Nature",
            "user": "demo"

The response has status code 200, meaning there were no errors, and the content of type application/json with the list of existing experiments. In this case, there is only one experiment named test that belongs to the demo user.

The same logic also applies to more complex query strings with additional parameters.

To download an image for a specific channel you could send a request like this:

GET /experiments/MQ==/channels/dG1hcHM0/image-file?plate_name=plate01,cycle_index=0,well_name=D03,x=0,y=0,tpoint=0,zplane=0

The server would respond with a message that contains the requested image as PNG-compressed binary data, which can be written to a file client-side using the provided filename:

HTTP/1.1 200 OK
Content-Type: image/png
Content-Disposition: attachment; filename="test_D03_x000_y000_z000_t000_wavelength-1.png"


Similarly, you can download all feature values extracted for a particular type of objects:

GET /api/experiments/MQ==/mapobjects/dG1hcHMx/feature-values

In this case, the server would respond with a message containing the requested feature values as CSV-encoded binary data, which can be written to a file using the provided filename:

HTTP/1.1 200 OK
Content-Type: application/octet-stream
Content-Disposition: attachment; filename="test_Cells_feature-values.csv"


For more information about available resources and verbs, please refer to tmserver.api.


In principle, GET requests could be handled via the browser. You can try it by entering a URL into the browser address bar, e.g.:


The server will responds with an error message with status code 401 (not authorized) because no access token was provided along with the request, which is required for JWT authentication.

So to make requests in practice, we need a client interface that is able to handle authentication. This can be achieved via the command line using cURL or through any other HTTP interface. In the following, we will demonstrate how requests can be handled in Python and Matlab:

Python example
import os
import requests
import json
import cv2
from StringIO import StringIO
import pandas as pd

def authenticate(url, username, password):
    response =
        url + '/auth',
        data=json.dumps({'username': username, 'password': password}),
        headers={'content-type': 'application/json'}
    data = response.json()
    return data['access_token']

def http_get(url, api_uri, token, **params):
    response = requests.get(
        url + '/' + api_uri, params=params,
        headers={'Authorization': 'JWT ' + token}
    return response

def get_data(url, api_uri, token, **params):
    response = http_get(url, api_uri, token, params)
    data = response.json()
    return data['data']

def get_image(url, api_uri, token, **params):
    response = http_get(url, api_uri, token, params)
    data = response.content
    return cv2.imdecode(data)

def get_feature_values(url, api_uri, token, location, **params):
    response = http_get(url, api_uri, token, params)
    data = StringIO(response.content)
    return pd.from_csv(data)

if __name__ = '__main__':

    url = 'http://localhost:8002'

    # Login
    token = authenticate(url, 'demo', 'XXX')

    # GET list of existing experiments
    experiments = get_data(url, '/api/experiments', token)

    # GET image for a specific channel
    image = get_image(
        url, '/api/experiments/MQ==/channels/dG1hcHM0/image-files?

    # GET feature values for a specific objects type
    data = get_feature_values(
        url, '/api/experiments/MQ==/mapobjects/dG1hcHMx/feature-values',
Matlab example
function [] = __main__()

    url = 'http://localhost:8002';

    % Login
    token = authenticate(url, 'demo', 'XXX');

    % GET list of existing experiments
    experiments = get_data(url, '/api/experiments', token);

    % GET image for a specific channel
    image = get_image(url, '/api/experiments/MQ==/channels/dG1hcHM0/image-files?plate_name=plate01,cycle_index=0,well_name=D03,well_pos_x=0,well_pos_y=0,tpoint=0,zplane=0', token);

    % GET feature values for a specific objects type
    data = get_feature_values(url, '/api/experiments/MQ==/mapobjects/dG1hcHMx/feature-values', token);


function token = authenticate(url, username, password)
    data = struct('username', username, 'password', password);
    options = weboptions('MediaType', 'application/json');
    response = webwrite([url, '/auth'], data, options);
    token = response.access_token;

function response = http_get(url, api_uri, token, varargin):
    options = weboptions('KeyName', 'Authorization', 'KeyValue', ['JWT ', token]);
    response = webread([url, '/', api_uri], options, varargin{:});

function data = get_data(url, api_uri, token, varargin)
    repsonse = http_get(url, api_uri, token, varargin);
    data =;

function image = get_image(url, api_uri, token, varargin)
    image = http_get(url, api_uri, token, varargin);

function data = get_feature_values(url, api_uri, token, location, varagin)
    data = http_get(url, api_uri, token, varargin);

Python client

The tmclient package is a REST API wrapper that provides users the possibility to interact with the TissueMAPS server in a programmatic way. It abstracts the REST implementation and exposes objects and methods that don’t require any knowledge of RESTful programming.

Active programming interface

The TmClient class implements high-level methods for accessing resources without having to provide the actual resource indentifiers.

First, a TmClient object must be instantiated by providing the server address and login credentials:

from tmclient import TmClient

client = TmClient(
    host='localhost', port=8002, username='demo', password='XXX',

The instantiated object can then be used, for example, to download the pixels of a ChannelImageFile:

image = client.download_channel_image(
    channel_name='wavelength-1', plate_name='plate01', well_name='D03', well_pos_x=0, well_pos_y=0,
    cycle_index=0, tpoint=0, zplane=0, correct=True

The returned image object is an instance of NumPy ndarray:

# Show image dimensions
print image.shape

# Show first row of pixels
print image[0, :]

Similarly, FeatureValues for a particular MapobjectType can be downloaded as follows:

data = client.download_object_feature_values(mapobject_type='Cells')

In this case, the returned data object is an instance of Pandas DataFrame:

# Show names of features
print data.columns

# Iterate over objects
for index, values in data.iterrows():
    print index, values

# Iterate over features
for name, values in data.iteritems():
    print name, values
Command line interface

The tmclient Python package further provides the tm_client progam.

You can upload images and manage workflows entirely via the command line:

tm_client --help


You can store passwords in a ~/.tm_pass file as key-value pairs (username: password) in YAML format:

demo: XXX

This will allow you to omit the password argument in command line calls. This is not totally safe either, but at least your password won’t show up in the history and you don’t have to remember it.

The command line interface is structured according to the type of available resources, defined in tmserver.api.

To begin with, create the Experiment:

tm_client -vv -H localhost -P 8002 -u demo experiment create -n test


You may want to override default values of parameters, such as microscope-type or workflow-type, depending on your use case.

Create a new Plate plate01:

tm_client -vv -H localhost -P 8002 -u demo plate -e test create --name plate01

Create a new Acquisition acquisition01 for plate plate01:

tm_client -vv -H localhost -P 8002 -u demo acquisition -e test create -p plate01 --name acquisition01

Upload each MicroscopeImageFile and MicroscopeMetadataFile for plate plate01 and acquisition acquisition01 from a local directory:

tm_client -vv -H localhost -P 8002 -u demo microscope-file -e test upload -p plate01 -a acquisition01 --directory ...

Check whether all files have been uploaded correctly:

tm_client -vv -H localhost -P 8002 -u demo microscope-file -e test ls -p plate01 -a acquisition01

To be able to process the uploaded images, you have to provide a WorkflowDescription. You can request a template and store it in a YAML file with either .yaml or .yml extension:

tm_client -vv -H localhost -P 8002 -u demo workflow -e test download --file /tmp/workflow.yml

Modify the workflow description acoording to your needs (as you would do in the workflow manager user interface) and upload it:

tm_client -vv -H localhost -P 8002 -u demo workflow -e test upload --file /tmp/workflow.yml

In case your workflow contains the jterator step, you will also have to provide a jterator project, i.e. a directory containing:

tm_client -vv -H localhost -P 8002 -u demo jtproject -e test upload --directory ...


Handles file templates are available for each module in the JtModules repository. For additional information, please refer to tmlib.workflow.jterator.handles.

After workflow description and jtproject have been uploaded, you can submit the workflow for processing:

tm_client -vv -H localhost -P 8002 -u demo workflow -e test submit

You can subsequently monitor the workflow status:

tm_client -vv -H localhost -P 8002 -u demo workflow -e test status


You can use the program watch to periodically check the status:

watch -n 10 tm_client -H localhost -P 8002 -u demo workflow -e test status

Once the workflow is completed, you can download generated data:

Download the pixels content of a ChannelImageFile:

tm_client -vv -H localhost -P 8002 -u demo -p XXX channel-image -e test download -c wavelength-1 -p plate01 -w D03 -x 0 -y 0 -i 0 --correct

Download feature values for all objects of type Cells:

tm_client -vv -H localhost -P 8002 -u demo -p XXX feature-values -e test download -o Cells


By default, files will be downloaded to your temporary directory, e.g. /tmp (the exact location depends on your operating system settings). The program will print the location of the file to the console when called with -vv or higher logging verbosity. You can specify an alternative download location for the download command using the --directory argument.

Using the library

The tmlibrary package implements an application programming interface (API) that represents an interface between the web application (implemented in the tmserver package) and storage and compute resources. The API provides routines for distributed computing and models for interacting with data stored on disk. The server uses the library and exposes part of its functionality to users via the RESTful API. Users with access to the server can also use the library directly. It further exposes command line interfaces (CLI), which provide users the possibility to interact with implemented programs via the console, which can be convenient for development, testing, and debugging.

Application programming interface (API)

Accessing data

Data can be accessed via model classes implemented in tmlib.models. Since data is stored (or referenced) in a database, a database connection must be established. This is achieved via the MainSession or ExperimentSession, depending on whether you need to access models derived from MainModel or ExperimentModel, respectively.

Model classes are implemented in form of SQLAlchemy Object Relational Mapper (ORM). For more information please refer to the ORM tuturial.

For example, a ChannelImage can be retrieved from a ChannelImageFile as follows (using the same parameters as in the examples above):

import tmlib.models as tm

with tm.utils.MainSession() as session:
    experiment = session.query(\
    experiment_id =

with tm.utils.ExperimentSession(experiment_id) as session:
    site = session.query(\
   == 'plate01',
   == 'D03',
            tm.Site.x == 0,
            tm.Site.y == 0
    image_file = session.query(tm.ChannelImageFile).\
            tm.Cycle.index == 0,
   == 'wavelength-1',
            tm.ChannelImageFile.site_id ==,
            tm.ChannelImageFile.tpoint == 0,
            tm.ChannelImageFile.zplane == 0
    image = image_file.get()


Some experiment-specific database tables are distributed, i.e. small fractions (so called “shards”) are spread across different database servers for improved performance. Rows of these tables can still be selected via the ExperimentSession, but they can not be modified within a session (see ExperimentConnection). In addition, distributed tables do not support sub-queries and cannot be joined with standard, non-distributed tables.


Content of files should only be accessed via the provided get and put methods of the respective model class implemented in tmlib.models.file, since the particular storage backend (e.g. filesystem or object storage) may be subject to change.

Command line interface (CLI)

A Workflow and each individual WorkflowStep can also be controlled via the command line.

Managing workflow steps

Each WorkflowStep represents a separate program that exposes its own command line interface. These interfaces have are a very similar structure and provide sub-commands for methods defined in either the CommandLineInterface base class or the step-specific implementation.

The name of the step is also automatically the name of the command line program. For example, the jterator program can be controlled via the jterator command.

You can initialize the step via the init sub-command:

jterator -vv 1 init --batch_size 5

or shorthand:

jterator -vv 1 init -b 5

And then run jobs:

  • either individually on the local machine via the run sub-command:

    jterator -vv 1 run -j 1
  • or submit them for parallel processing on remote machines via the submit sub-command:

    jterator -vv 1 submit


The submit command internally calls the program with run --job <job_id> on different CPUs of the same machine or on other remote machines for each of the batch jobs defined via init.

To print the description of an individual job to the console call the info sub-command:

jterator -vv 1 info --phase run --job 1

You can further print the standard output or error of a job via the log sub-command:

jterator -vv 1 log --phase run --job 1


The detail of log messages depends on the logging level, which is specified via the --verbosity or -v argument. The more vs the more detailed the log output becomes.

The full documentation of each command line interface is available online along with the documentation of the respective cli module, e.g. tmlib.workflow.jterator.cli, or via the console by calling the program with --help or -h:

jterator -h

Managing workflows

Distributed image processing workflows can be set up and submitted via the workflow manager user interface. The same can be achieved via the command line through the tm_workflow program.

Submitting the workflow for experiment with ID 1 is as simple as:

tm_workflow -vv 1 submit

The workflow can also be resubmitted at a given stage:

tm_workflow -vv 1 resubmit --stage image_preprocessing


Names of workflow stages may contain underscores. They are stripped for display in the user interface, but are required in the command line interface.