tmlib.models package¶

Module contents¶

Database models.

A database model is an object-relational mapping (ORM) of Python objects to relational database entries. A class represents a database table, class attributes correspond to columns of that table and each instances of the class maps to an individual row of the table.

The central organizational unit of TissueMAPS is an Experiment. In the database, each experiment is represented by a separate schema, which contains tables for images and related data.

There is also a main (or “public”) schema that holds data beyond the scope of individual experiments, such as credentials of a User or the status of a submitted computational Task. The main schema further provides reference to existing experiments (see ExperimentRerefence) and information on experiment-specific user permissions (see ExperimentShare).

Main and experiment-specific database schemas can be accessed programmatically using MainSession or ExperimentSession, respectively. These sessions provide a database transaction that bundles all enclosing statements into an all-or-nothing operation to ensure that either all or no changes are persisted in the database. The transaction will be automatically committed or rolled back in case of an error. The session context exposes an instance of SQLAlchemySession and queries return instances of data model classes derived from MainModel or ExperimentModel, respectively:

import tmlib.models as tm

with tm.utils.ExperimentSession(experiment_id=1) as session:
    plates = session.query(tm.Plates).all()
    print(plates)
    print(plates[0].name)
    print(plates[0].acquisitions)

Some SQL statements cannot be performed within a transaction. In addition, the ORM comes with a performance overhead and is not optimal for inserting or updating a large number of rows. In these situations, MainConnection or ExperimentConnection can be used. These classes create individual database connections and bypass the ORM. They futher make use of autocommit mode, where each statement gets directly committed such that all changes are immediately effective. Sessions and connections are entirely different beasts and expose a different interface. While sessions use the ORM, connections require raw SQL statements. In addition, they don’t return instance of data model classes, but light-weight instances of a namedtuple. Similar to data models, columns can be accessed via attributes, but the objects only return the query result without providing any relations to other objects:

import tmlib.models as tm

with tm.utils.ExperimentConnection(experiment_id=1) as connection:
    connection.execute('SELECT * FROM plates;')
    plates = connection.fetchall()
    print(plates)
    print(plates[0].name)

The session and connection contexts automatically add the experiment-specific schema to the search path at runtime. To access data models outside the scope of a session or connection, you either need to set the search path manually or specify the schema explicitly, e.g. SELECT * FROM experiment_1.plates.

Some of the data models represent distributed table, which are sharded accross different servers to scale out the database backend over a cluster. To this end, TissueMAPS uses Citus. Distributed models are flagged with __distribution_method__ and in case of hash or range distribution additionally with __distribute_by__. This will either replicate the table (so called “reference” table) or distribute it accross available database server nodes. To this end, the extension must have been installed on all database servers and these servers (worker nodes) must have been registered on the main database server (master node). For more details on how to set up a database cluster, please refer to Installation section of the documentation.

Distributed tables can be accessed via the ORM for reading (SELECT) using ExperimentSession. However, they cannot be modified (INSERT, UPDATE or DELETE) via the ORM, mainly because multi-statement transactions are not (yet) supported. Distributed tables must therefore be modified using ExperimentConnection. There are additional SQL features that are not supported for distributed tables. Please refer to the Citus documentation for more information on how to query and modify them.

Submodules¶

tmlib.models.acquisition module¶

tmlib.models.acquisition.ACQUISITION_LOCATION_FORMAT = 'acquisition_{id}'¶: Format string for acquisition locations

class tmlib.models.acquisition.Acquisition(name, plate_id, description='')¶

Bases: tmlib.models.base.DirectoryModel, tmlib.models.base.DateMixIn, tmlib.models.base.IdMixIn

An acquisition contains all files belonging to one microscope image acquisition process. Note that in contrast to a cycle, an acquisition may contain more than one time point.

The incentive to group files this way relates to the fact that most microscopes generate separate metadata files per acquisition.

Attributes

microscope_image_files: List[tmlib.models.file.MicroscopeImageFile]	microscope image files belonging to the acquisition
microscope_metadata_files: List[tmlib.models.file.MicroscopeMetadataFile]	microscope metadata files belonging to the acquisition
channel_image_files: List[tmlib.models.file.ChannelImageFile]	channel image files belonging to the acquisition

Parameters:

name: str

name of the acquisition

plate_id: int

ID of the parent Plate

description: str, optional

description of the acquisition

belongs_to(user)¶

Determines whether the acquisition belongs to a given user.

Parameters:

user: tmlib.user.User

TissueMAPS user

Returns:

bool

whether acquisition belongs to user

created_at¶: datetime: date and time when the row was inserted into the column

description¶: str: description provided by the user

id¶

location¶: str: location were the acquisition content is stored

microscope_images_location¶: str: location where microscope image files are stored

microscope_metadata_location¶: str: location where microscope metadata files are stored

name¶: str: name given by the user

plate¶: tmlib.models.plate.Plate: plate to which the acquisition belongs

plate_id¶: int: ID of the parent plate

status¶: str: upload status based on the status of microscope files

to_dict()¶

Returns attributes as key-value pairs.

Returns:	dict

updated_at¶: datetime: date and time when the row was last updated

tmlib.models.alignment module¶

class tmlib.models.alignment.SiteShift(x, y, site_id, cycle_id)¶

Bases: tmlib.models.base.ExperimentModel

Translation of a given Site acquired at a given Cycle relative to the corresponding Site of the reference Cycle.

Parameters:

x: int

shift in pixels along the x-axis relative to the corresponding site of the reference cycle (positive value -> right, negative value -> left)

y: int

shift in pixels along the y-axis relative to the corresponding site of the reference cycle (positive value -> down, negative value -> up)

site_id: int

ID of the parent Site

cycle_id: int

ID of the parent Cycle

cycle¶: tmlib.models.cycle.Cycle: parent cycle for which the shift was calculated

cycle_id¶: int: ID of the parent cycle

site¶: tmlib.models.site.Site: parent site for which the shift was calculated

site_id¶: int: ID of the parent site

x¶: int: vertical translation in pixels relative to the corresponding site of the reference cycle (positive value -> down, negative value -> up)

y¶: int: horizontal translation in pixels relative to the corresponding site of the reference cycle (positive value -> right, negative value -> left)

tmlib.models.base module¶

Abstract base and mixin classes for database models.

Mixin classes must implement additional table columns using the declared_attr decorator.

class tmlib.models.base.DateMixIn¶

Bases: object

Mixin class to automatically add columns with datetime stamps to a database table.

created_at = Column(None, DateTime(), table=None, default=ColumnDefault(<sqlalchemy.sql.functions.now at 0x7fad69de7690; now>))¶

updated_at = Column(None, DateTime(), table=None, onupdate=ColumnDefault(<sqlalchemy.sql.functions.now at 0x7fad69de7550; now>), default=ColumnDefault(<sqlalchemy.sql.functions.now at 0x7fad69de7810; now>))¶

class tmlib.models.base.DirectoryModel(**kwargs)¶

Bases: tmlib.models.base.FileSystemModel

Abstract base class for model classes, which refer to data stored in directories on disk.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class tmlib.models.base.DistributedExperimentModel(**kwargs)¶

Bases: tmlib.models.base.ExperimentModel

Abstract base class for models of an experiment-specific schema that is partitioned and potentially distributed over multiple database servers.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

classmethod get_unique_ids(connection, n)¶

Gets unique, shard-specific values for the distribution column.

Parameters:

connection: tmlib.models.utils.ExperimentConnection

experiment-specific database connection

n: int

number of IDs that should be returned

Returns:

List[int]

unique, shard-specific IDs

class tmlib.models.base.ExperimentModel(**kwargs)¶

Bases: abc.ExperimentBase

Abstract base class for models of an experiment-specific schema.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

class tmlib.models.base.FileModel(**kwargs)¶

Bases: tmlib.models.base.FileSystemModel

Abstract base class for model classes, which refer to data stored in files on disk.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

format¶: str: file extension, e.g. “.tif” or “.jpg”

get()¶: Gets the file content.

put(data)¶: Puts data to the file.

class tmlib.models.base.FileSystemModel(**kwargs)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

Abstract base class for model classes, which refer to data stored on disk outside of the database.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

location¶

str: location on disk

Devired classes must implement location and decorate it with sqlalchemy.ext.hybrid.hyprid_property().

class tmlib.models.base.IdMixIn¶

Bases: object

Mixin class to automatically add an ID column to a database table with primary key constraint.

hash¶: str: encoded id

id = Column(None, Integer(), table=None, primary_key=True, nullable=False)¶

class tmlib.models.base.MainModel(**kwargs)¶

Bases: abc.MainBase, tmlib.models.base.IdMixIn

Abstract base class for models of the main (“public”) schema.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

tmlib.models.channel module¶

tmlib.models.channel.CHANNEL_LAYER_LOCATION_FORMAT = 'layer_{id}'¶: Format string for channel layer locations

tmlib.models.channel.CHANNEL_LOCATION_FORMAT = 'channel_{id}'¶: Format string for channel locations

class tmlib.models.channel.Channel(name, wavelength, bit_depth, experiment_id)¶

Bases: tmlib.models.base.DirectoryModel, tmlib.models.base.DateMixIn, tmlib.models.base.IdMixIn

A channel represents all images across different time points and spatial positions that were acquired with the same illumination and microscope filter settings.

Attributes

image_files: List[tmlib.models.file.ChannelImageFile]	images belonging to the channel
layers: List[tmlib.models.layer.ChannelLayer]	layers belonging to the channel

Parameters:

name: str

name of the channel

wavelength: str

name of the corresponding wavelength

bit_depth: int

number of bits used to indicate intensity of pixels

experiment_id: int

ID of the parent Experiment

bit_depth¶: int: number of bytes used to encode intensity

created_at¶: datetime: date and time when the row was inserted into the column

experiment¶: tmlib.models.experiment.Experiment: parent experiment

experiment_id¶: int: ID of the parent experiment

get_image_file_location(image_file_id)¶

id¶

illumstats_location¶: str: location where illumination statistics files are stored

image_files_location¶: str: location where image files are stored

location¶: str: location were channel content is stored

name¶: str: name given by the microscope or user

remove_image_files()¶: Removes all image files on disk

updated_at¶: datetime: date and time when the row was last updated

wavelength¶: str: name of wavelength

class tmlib.models.channel.ChannelLayer(channel_id, tpoint, zplane)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

A channel layer represents a multi-resolution overview of all images belonging to a given Channel, z-plane and time point in form of an image pyramid.

Parameters:

channel_id: int

ID of the parent Channel

tpoint: int

zero-based time series index

zplane: int

zero-based z-resolution index

base_tile_coordinate_to_image_file_map¶: Dict[Tuple[int], List[int]]: IDs of all images, which intersect with a given tile; maps coordinates of tiles at the maximal zoom level to the files of intersecting images

calc_coordinates_of_next_higher_level(z, y, x)¶

Calculates for a given tile the coordinates of the 4 tiles at the next higher zoom level that represent the tile at the current level.

Parameters:

z: int

zero-based index of the current zoom level

y: int

zero-based index of the current row

x: int

zero-based index of the current column

Returns:

List[Tuple[int]]

row, column coordinate at the next higher zoom level

calculate_max_image_size()¶

Determines dimensions of the pyramid, i.e. height, width of the image at the highest resolution level.

Returns:

Tuple[int]

number of pixels along the y, x axis of the image at the maximum zoom level

calculate_zoom_levels(height, width)¶

Calculates number of zoom levels.

Parameters:

height: int

number of pixels along vertical axis at highest resolution

width: int

number of pixels along horizontal axis at highest resolution

Returns:

int

number of zoom levels

channel¶: tmlib.models.channel.Channel: parent channel

channel_id¶: int: ID of parent channel

depth¶: int: number of pixels along horizontal axis at highest resolution level

dimensions¶: List[Tuple[int]]: number of tiles along the vertical and horizontal axis of the layer at each zoom level; levels are sorted such that the first element represents the lowest resolution (maximally zoomed out) level and the last element the highest resolution (maximally zoomed in) level

extract_tile_from_image(image, y_offset, x_offset)¶

Extracts a subset of pixels for a tile from an image. In case the area of the tile overlaps the image, pad the tile with zeros.

Parameters:

image: tmlib.image.ChannelImage

image from which the tile should be extracted

y_offset: int

offset along the vertical axis of image

x_offset: int

offset along the horizontal axis of image

Returns:

tmlib.image.PyramidTile

extracted tile

get_empty_base_tile_coordinates()¶

Gets coordinates of empty base tiles, i.e. tiles at the maximum zoom level that don’t map to an image because they fall into a spacer region, e.g. gap introduced wells.

Returns:

Set[Tuple[int]]

row, column coordinates

height¶: int: number of pixels along vertical axis at highest resolution level

id¶

map_base_tile_to_images(site)¶

Maps tiles at the highest resolution level to all image files of the same channel, which intersect with the given tile. Only images bordering site to the left and/or top are considered.

Parameters:

site: tmlib.models.Site

site whose neighbours could be included in the search

Returns:

Dict[Tuple[int], List[int]]

IDs of images intersecting with a given tile hashable by tile y, x coordinates

map_image_to_base_tiles(image_file)¶

Maps an image to the corresponding tiles at the base of the pyramid (maximal zoom level) that intersect with the image.

Parameters:

image_file: tmlib.models.ChannelImageFile

file containing the image that should be mapped

Returns:

List[Dict[str, Tuple[int]]]

mappings with y and x coordinate as well as y_offset and x_offset relative to the image in image_file for each tile whose pixels are part of the image

max_intensity¶: int: maximum intensity value at which images get clipped at original bit depth before rescaling to 8-bit

maxzoom_level_index¶: int: index of the highest resolution level, i.e. the base of the pyramid

min_intensity¶: int: minimum intensity value at which images get clipped at original bit depth before rescaling to 8-bit

n_tiles¶: int: total number of tiles across all resolution levels

tile_size¶: int: maximal number of pixels along each axis of a tile

tpoint¶: int: zero-based index in time series

width¶: int: number of pixels along horizontal axis at highest resolution level

zoom_factor¶: int: factor by which resolution increases per pyramid level

zplane¶: int: zero-based index in z stack

tmlib.models.cycle module¶

class tmlib.models.cycle.Cycle(index, experiment_id)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.DateMixIn, tmlib.models.base.IdMixIn

A cycle represents an individual image acquisition time point of a a “multiplexing” experiment.

Attributes

site_shifts: List[tmlib.models.site.SiteShift]

shifts belonging to the cycle

Parameters:

index: int

index of the cycle (based on the order of acquisition)

experiment_id: int

ID of the parent Experiment

created_at¶: datetime: date and time when the row was inserted into the column

experiment¶: tmlib.models.experiment.Experiment: parent experiment

experiment_id¶: int: ID of parent experiment

id¶

index¶: int: zero-based index in the acquisition sequence

updated_at¶: datetime: date and time when the row was last updated

tmlib.models.dialect module¶

tmlib.models.experiment module¶

tmlib.models.experiment.EXPERIMENT_LOCATION_FORMAT = 'experiment_{id}'¶: Format string for experiment locations.

class tmlib.models.experiment.Experiment(id, microscope_type, plate_format, plate_acquisition_mode, location, workflow_type='canonical', zoom_factor=2, well_spacer_size=500, vertical_site_displacement=0, horizontal_site_displacement=0)¶

Bases: tmlib.models.base.DirectoryModel

An experiment is the main organizational unit of TissueMAPS. It represents a set of images and associated data.

Attributes

plates: List[tmlib.models.plate.Plate]	plates belonging to the experiment
cycles: List[tmlib.model.cycle.Cycle]	cycles belonging to the plate
channels: List[tmlib.models.channel.Channel]	channels belonging to the experiment
mapobject_types: List[tmlib.models.mapobject.MapobjectType]	mapobject types belonging to the experiment

Parameters:

id: int

ID that should be assigned to the experiment

microscope_type: str

microscope that was used to acquire the images

plate_format: int

number of wells in the plate, e.g. 384

plate_acquisition_mode: str

the way plates were acquired with the microscope

location: str

absolute path to the location of the experiment on disk

workflow_type: str, optional

name of an implemented workflow type (default: "canonical")

zoom_factor: int, optional

zoom factor between pyramid levels (default: 2)

well_spacer_size: int

gab between neighboring wells in pixels (default: 500)

vertical_site_displacement: int, optional

displacement of neighboring sites within a well along the vertical axis in pixels (default: 0)

horizontal_site_displacement: int, optional

displacement of neighboring sites within a well along the horizontal axis in pixels (default: 0)

See also

tmlib.workflow.metaconfig.SUPPORTED_MICROSCOPE_TYPES, tmlib.models.plate.SUPPORTED_PLATE_AQUISITION_MODES, tmlib.models.plate.SUPPORTED_PLATE_FORMATS

channels_location¶: str: location where channel data are stored

get_mapobject_type(name)¶

Returns a mapobject type belonging to this experiment by name.

Parameters:

name : str

the name of the mapobject_type to be returned

Returns:

tmlib.models.MapobjectType

Raises:

sqlalchemy.orm.exc.MultipleResultsFound

when multiple mapobject types with this name were found

sqlalchemy.orm.exc.NoResultFound

when no mapobject type with this name was found

horizontal_site_displacement¶: displacement of neighboring sites within a well along the horizontal axis in pixels

id¶

location¶: str: location of the experiment

microscope_type¶: str: microscope that was used to acquire the images

persist_workflow_description(description)¶

Persists the workflow description and updates workflow_type.

Parameters:

description: tmlib.workflow.tmaps.description.WorkflowDescription

description of the workflow

plate_acquisition_mode¶: str: the order in which plates were acquired via the microscope

plate_format¶: int: number of wells in the plate, e.g. 384

plate_grid¶: numpy.ndarray[int]: IDs of plates arranged according to their relative position of the plate within the experiment overview image (sorted row-wise by plate names)

plate_spacer_size¶: int: gap between neighboring plates in pixels

plates_location¶: str: location where plates data are stored

pyramid_depth¶: int: number of zoom levels of the pyramid

pyramid_height¶: int: number of pixels along y-axis of the pyramid at highest zoom level

pyramid_width¶: int: number of pixels along x-axis of the pyramid at highest zoom level

tools_location¶: str: location where tool data are stored

vertical_site_displacement¶: displacement of neighboring sites within a well along the vertical axis in pixels

well_spacer_size¶: int: gab introduced between neighbooring wells in pixels

workflow_description¶: tmlib.workflow.tmaps.description.WorkflowDescription: description of the workflow

workflow_location¶: str: location where workflow data are stored

workflow_type¶: str: workflow type

zoom_factor¶: int: zoom factor between pyramid levels

class tmlib.models.experiment.ExperimentReference(name, user_id, root_directory, description='')¶

Bases: tmlib.models.base.MainModel, tmlib.models.base.DateMixIn

A reference to an experiment, which is stored in a separate database.

All data associated with an experiment are stored in separate, experiment-specific databases.

See also

tmlib.models.experiment.Experiment

Attributes

submissions: List[tmlib.models.submission.Submission]

submissions belonging to the experiment

Parameters:

name: str

name of the experiment

microscope_type: str

microscope that was used to acquire the images

plate_format: int

number of wells in the plate, e.g. 384

plate_acquisition_mode: str

the way plates were acquired with the microscope

user_id: int

ID of the User that owns the experiment

root_directory: str

absolute path to root directory on disk where experiment should be created in

description: str, optional

description of the experimental setup

belongs_to(user)¶

Determines whether the experiment belongs to a given user.

Parameters:

user: tmlib.user.User

TissueMAPS user

Returns:

bool

whether experiment belongs to user

can_be_accessed_by(user_id, permission='write')¶

Checks whether a user has the permissions to access the referenced experiment.

Parameters:

user_id: int

ID of User for which permissions should be checked

permission: str, optional

whether user must have "read" or "write" permission (default: "write")

Returns:

bool

True if the user can access the referenced experiment and False otherwise

created_at¶: datetime: date and time when the row was inserted into the column

description¶: str: description provided by the user

id¶

location¶: str: location of the experiment, e.g. absolute path to a directory on disk

name¶: str: name given by the user

root_directory¶: absolute path to the directory where experiments are located

updated_at¶: datetime: date and time when the row was last updated

user¶: tmlib.models.user.User: user that owns the experiment

user_id¶: int: ID of the owner

class tmlib.models.experiment.ExperimentShare(experiment_id, user_id, write_access=False)¶

Bases: tmlib.models.base.MainModel

Relationship between a User <tmlib.models.user.User and an Experiment <tmlib.models.experiment.ExperimentReference to to share an experiment with other users.

Parameters:

experiment_id: int

ID of the Experiment that should be shared

user_id: int

ID of the User with whom the experiment should be shared

write_access: bool, optional

whether the user will have write access to the shared experiment (default: False)

experiment¶: tmlib.models.experiment.ExperimentReference: shared experiment

experiment_id¶: int: ID of shared experiment

id¶

user¶: tmlib.models.user.User: user with whom the experiment is shared

user_id¶: int: ID of user with whom the experiment is shared

write_access¶: bool: whether the user has write access, i.e. can modify the experiment

tmlib.models.feature module¶

class tmlib.models.feature.Feature(name, mapobject_type_id, is_aggregate=False)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

A feature is a measurement that is associated with a particular MapobjectType. For example a feature named “Morphology_Area” would correspond to values reflecting the area of each individual Mapobject.

Parameters:

name: str

name of the feature

mapobject_type_id: int

ID of parent MapobjectType

is_aggregate: bool, optional

whether the feature is an aggregate calculated based on another feature

id¶

is_aggregate¶: bool: whether the feature is an aggregate of child object features

mapobject_type¶: tmlib.models.mapobject.MapobjectType: parent mapobject type

mapobject_type_id¶: int: id of the parent mapobject type

name¶: str: name given to the feature

class tmlib.models.feature.FeatureValues(partition_key, mapobject_id, values, tpoint=None)¶

Bases: tmlib.models.base.DistributedExperimentModel

An individual value of a Feature that was extracted for a given Mapobject.

Parameters:

partition_key: int

key that determines on which shard the object will be stored

mapobject_id: int

ID of the mapobject to which values should be assigned

values: Dict[str, float]

mapping of feature ID to value

tpoint: int, optional

zero-based time point index

mapobject_id¶: int: ID of the parent mapobject

partition_key¶

tpoint¶: int: zero-based time point index

values¶: Dict[str, str]: mapping of feature ID to value encoded as text

tmlib.models.file module¶

class tmlib.models.file.ChannelImageFile(tpoint, zplane, site_id, acquisition_id, channel_id, file_map, cycle_id=None)¶

Bases: tmlib.models.base.FileModel, tmlib.models.base.DateMixIn

A channel image file holds a single 2D pixels plane that was extracted from a microscope image file. It represents a unique combination of time point, z-level, site, and channel.

Parameters:

tpoint: int

zero-based time point index in the time series

zplane: int

zero-based z-level index in the 3D stack

site_id: int

ID of the parent Site

channel_id: int

ID of the parent Channel

acquisition_id: int

ID of the parent Acquisition

file_map: Dict[str, list]

mapping to link the file to each corresponding MicroscopeImageFile from which is was derived as well as the locations within the file (a channel image file might be linked to more than one microscope image file in case it was obtained by projection of a z-stack, for example)

cycle_id: int, optional

ID of the parent Cycle

FILENAME_FORMAT = 'channel_image_file_{id}.h5'¶: Format string for filenames

acquisition¶: tmlib.models.channel.Channel: parent channel

acquisition_id¶: int: ID of the parent acquisition

channel¶: tmlib.models.channel.Channel: parent channel

channel_id¶: int: ID of the parent channel

created_at¶: datetime: date and time when the row was inserted into the column

cycle¶: tmlib.models.cycle.Cycle: parent cycle

cycle_id¶: int: ID of the parent cycle

file_map¶

get()¶

Gets stored image.

Returns:

tmlib.image.ChannelImage

image stored in the file

id¶

location¶: str: location of the file

put(image)¶

Puts image to storage.

Parameters:

image: tmlib.image.ChannelImage

pixels data that should be stored in the image file

site¶: tmlib.models.site.Site: parent site

site_id¶: int: ID of the parent site

tpoint¶: int: zero-based index in the time series

updated_at¶: datetime: date and time when the row was last updated

zplane¶: int: zero-based index in the z-stack

class tmlib.models.file.IllumstatsFile(channel_id)¶

Bases: tmlib.models.base.FileModel, tmlib.models.base.DateMixIn

An illumination statistics file holds matrices for mean and standard deviation values calculated at each pixel position across all images of the same channel and cycle.

Parameters:

channel_id: int

ID of the parent channel

FILENAME_FORMAT = 'illumstats_file_{id}.h5'¶: Format string to build filename

channel¶: tmlib.models.channel.Channel: parent channel

channel_id¶: int: ID of parent channel

created_at¶: datetime: date and time when the row was inserted into the column

get()¶

Get illumination statistics images from store.

Returns:

Illumstats

illumination statistics images

id¶

location¶: str: location of the file

put(data)¶

Put illumination statistics images to store.

Parameters:

data: IllumstatsContainer

illumination statistics

updated_at¶: datetime: date and time when the row was last updated

class tmlib.models.file.MicroscopeImageFile(name, acquisition_id)¶

Bases: tmlib.models.base.FileModel, tmlib.models.base.DateMixIn

Image file that was generated by the microscope. The file format differs between microscope types and may be vendor specific.

Parameters:

name: str

name of the microscope image file

acquisition_id: int

ID of the parent Acquisition

acquisition¶: tmlib.models.acquisition.Acquisition: parent acquisition

acquisition_id¶: int: ID of the parent acquisition

created_at¶: datetime: date and time when the row was inserted into the column

get(obj, *args, **kwargs)¶: Not implemented.

id¶

location¶: str: location of the file

name¶: str: name given by the microscope

omexml¶: str: OMEXML metadata

put(obj, *args, **kwargs)¶: Not implemented.

status¶: str: upload status

to_dict()¶

Returns attributes “id”, “name” and “status” as key-value pairs.

Returns:	dict

updated_at¶: datetime: date and time when the row was last updated

class tmlib.models.file.MicroscopeMetadataFile(name, acquisition_id)¶

Bases: tmlib.models.base.FileModel, tmlib.models.base.DateMixIn

Metadata file that was generated by the microscope. The file format differs between microscope types and may be vendor specific.

Parameters:

name: str

name of the file

acquisition_id: int

ID of the parent Acquisition

acquisition¶: tmlib.models.acquisition.Acquisition: parent acquisition

acquisition_id¶: int: ID of the parent acquisition

created_at¶: datetime: date and time when the row was inserted into the column

get(obj, *args, **kwargs)¶: Not implemented.

id¶

location¶: str: location of the file

name¶: str: name given by the microscope

put(obj, *args, **kwargs)¶: Not implemented.

status¶: str: upload status

to_dict()¶

Returns attributes “id”, “name” and “status” as key-value pairs.

Returns:	dict

updated_at¶: datetime: date and time when the row was last updated

tmlib.models.mapobject module¶

class tmlib.models.mapobject.Mapobject(partition_key, mapobject_type_id, ref_id=None)¶

Bases: tmlib.models.base.DistributedExperimentModel

A mapobject represents a connected pixel component in an image. It has one or more 2D segmentations that can be used to represent the object on the map and may also be associated with measurements (features), which can be queried or used for further analysis.

Parameters:

partition_key: int

key that determines on which shard the object will be stored

mapobject_type_id: int

ID of parent MapobjectType

ref_id: int, optional

ID of the referenced record

classmethod delete_objects_with_invalid_segmentation(connection)¶

Deletes all instances with invalid segmentations as well as all “children” instances of MapobjectSegmentation FeatureValues, LabelValues.

Parameters:

connection: tmlib.models.utils.ExperimentConnection

experiment-specific database connection

classmethod delete_objects_with_missing_feature_values(connection)¶

Deletes all instances that don’t have FeatureValues as well as their “children” instances of MapobjectSegmentation and LabelValues.

Parameters:

connection: tmlib.models.utils.ExperimentConnection

experiment-specific database connection

classmethod delete_objects_with_missing_segmentations(connection)¶

Deletes all instances that don’t have a MapobjectSegmentation as well as their “children” instances of FeatureValues and LabelValues.

Parameters:

connection: tmlib.models.utils.ExperimentConnection

experiment-specific database connection

id¶

mapobject_type_id¶: int: ID of parent mapobject type

partition_key¶

ref_id¶: int: ID of another record to which the object is related. This could refer to another mapobject in the same table, e.g. in order to track proliferating cells, or a record in another reference table, e.g. to identify the corresponding record of a “Well”.

class tmlib.models.mapobject.MapobjectSegmentation(partition_key, geom_polygon, geom_centroid, mapobject_id, segmentation_layer_id, label=None)¶

Bases: tmlib.models.base.DistributedExperimentModel

A segmentation provides the geometric representation of a Mapobject.

Parameters:

partition_key: int

key that determines on which shard the object will be stored

geom_polygon: shapely.geometry.polygon.Polygon

polygon geometry of the mapobject contour

geom_centroid: shapely.geometry.point.Point

point geometry of the mapobject centroid

mapobject_id: int

ID of parent Mapobject

segmentation_layer_id: int

ID of parent SegmentationLayer

label: int, optional

label assigned to the segmented object

geom_centroid¶: str: EWKT POINT geometry

geom_polygon¶: str: EWKT POLYGON geometry

label¶: int: label assigned to the object upon segmentation

mapobject_id¶: int: ID of parent mapobject

partition_key¶

segmentation_layer_id¶: int: ID of parent segmentation layer

class tmlib.models.mapobject.MapobjectType(name, experiment_id, ref_type=None)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

A mapobject type represents a conceptual group of mapobjects (segmented objects) that reflect different biological entities, such as “cells” or “nuclei” for example.

Attributes

records: List[tmlib.models.result.ToolResult]	records belonging to the mapobject type
features: List[tmlib.models.feature.Feature]	features belonging to the mapobject type

Parameters:

name: str

name of the map objects type, e.g. “cells”

experiment_id: int

ID of the parent Experiment

ref_type: str, optional

name of another reference type (default: None)

classmethod delete_cascade(connection, static=None)¶

Deletes all instances as well as “children” instances of Mapobject, MapobjectSegmentation, Feature, FeatureValues, ToolResult, LabelLayer and LabelValues.

Parameters:

connection: tmlib.models.utils.ExperimentConnection

experiment-specific database connection

static: bool, optional

if True static types (“Plates”, “Wells”, “Sites”) will be deleted, if False non-static types will be delted, if None all types will be deleted (default: None)

experiment¶: tmlib.models.experiment.Experiment: parent experiment

experiment_id¶: int: ID of parent experiment

get_feature_values_per_site(site_id, tpoint, feature_ids=None)¶

Gets all FeatureValues for each Mapobject where the corresponding MapobjectSegmentation intersects with the geometric representation of a given Site.

Parameters:

site_id: int

ID of a Site for which objects should be spatially filtered

tpoint: int

time point for which objects should be filtered

feature_ids: List[int], optional

ID of each Feature for which values should be selected; by default all features will be selected

Returns:

pandas.DataFrame[numpy.float]

feature values for each mapobject

get_label_values_per_site(site_id, tpoint)¶

Gets all LabelValues for each Mapobject where the corresponding MapobjectSegmentation intersects with the geometric representation of a given Site.

Parameters:

site_id: int

ID of a Site for which objects should be spatially filtered

tpoint: int, optional

time point for which objects should be filtered

Returns:

pandas.DataFrame[numpy.float]

label values for each mapobject

get_segmentations_per_site(site_id, tpoint, zplane, as_polygons=True)¶

Gets each MapobjectSegmentation that intersects with the geometric representation of a given Site.

Parameters:

site_id: int

ID of a Site for which objects should be spatially filtered

tpoint: int

time point for which objects should be filtered

zplane: int

z-plane for which objects should be filtered

as_polygons: bool, optional

whether segmentations should be returned as polygons; if False segmentations will be returned as centroid points (default: True)

Returns:

Tuple[Union[int, geoalchemy2.elements.WKBElement]]

label and geometry for each segmented object

get_site_geometry(site_id)¶

Gets the geometric representation of a Site. in form of a MapobjectSegmentation.

Parameters:

site_id: int

ID of the Site

Returns:

geoalchemy2.elements.WKBElement

id¶

identify_border_objects_per_site(site_id, tpoint, zplane)¶

Determines for each Mapobject where the corresponding MapobjectSegmentation intersects with the geometric representation of a given Site, whether the objects is touches at the border of the site.

Parameters:

site_id: int

ID of a Site for which objects should be spatially filtered

tpoint: int

time point for which objects should be filtered

zplane: int

z-plane for which objects should be filtered

Returns:

pandas.Series[numpy.bool]

True if the mapobject touches the border of the site and False otherwise

name¶: str: name given by user

ref_type¶: str: name of another type that serves as a reference for “static” mapobjects, i.e. objects that are pre-defined through the experiment layout and independent of image segmentation (e.g. “Plate” or “Well”)

class tmlib.models.mapobject.SegmentationLayer(mapobject_type_id, tpoint=None, zplane=None)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

Parameters:

mapobject_type_id: int

ID of parent MapobjectType

tpoint: int, optional

zero-based time point index

zplane: int, optional

zero-based z-resolution index

calculate_zoom_thresholds(maxzoom_level, represent_as_polygons)¶

Calculates the zoom level below which mapobjects are represented on the map as centroids rather than polygons and the zoom level below which mapobjects are no longer visualized at all. These thresholds are necessary, because it would result in too much network traffic and the client would be overwhelmed by the large number of objects.

Parameters:

maxzoom_level: int

maximum zoom level of the pyramid

represent_as_polygons: bool

whether the objects should be represented as polygons or only as centroid points

Returns:

Tuple[int]

threshold zoom levels for visualization of polygons and centroids

centroid_thresh¶: int: zoom level threshold below which centroids will not be visualized

get_segmentations(x, y, z, tolerance=2)¶

Get outlines of each Mapobject contained by a given pyramid tile.

Parameters:

x: int

zero-based column map coordinate at the given z level

y: int

zero-based row map coordinate at the given z level (negative integer values due to inverted y-axis)

z: int

zero-based zoom level index

tolerance: int, optional

maximum distance in pixels between points on the contour of original polygons and simplified polygons; the higher the tolerance the less coordinates will be used to describe the polygon and the less accurate it will be approximated and; if 0 the original polygon is used (default: 2)

Returns:

List[Tuple[int, str]]

GeoJSON representation of each selected mapobject

classmethod get_tile_bounding_box(x, y, z, maxzoom)¶

Calculates the bounding box of a layer tile.

Parameters:

x: int

horizontal tile coordinate

y: int

vertical tile coordinate

z: int

zoom level

maxzoom: int

maximal zoom level of layers belonging to the visualized experiment

Returns:

Tuple[int]

bounding box coordinates (x_top, y_top, x_bottom, y_bottom)

id¶

mapobject_type¶: tmlib.models.mapobject.MapobjectType: parent mapobject type

mapobject_type_id¶: int: ID of parent channel

polygon_thresh¶: int: zoom level threshold below which polygons will not be visualized

tpoint¶: int: zero-based index in time series

zplane¶: int: zero-based index in z stack

tmlib.models.plate module¶

tmlib.models.plate.PLATE_LOCATION_FORMAT = 'plate_{id}'¶: Format string for plate locations

class tmlib.models.plate.Plate(name, experiment_id, description='')¶

Bases: tmlib.models.base.DirectoryModel, tmlib.models.base.DateMixIn

A plate represents a container with reservoirs for biological samples (referred to as wells). It’s assumed that all images of a plate were acquired with the same microscope settings implying that each acquisition has the same number of z-planes and channels.

The format of the plate is encode by the number of wells in the plate, e.g. 384.

Attributes

acquisitions: List[tmlib.model.Acqusition]	acquisitions belonging to the plate
wells: List[tmlib.model.Well]	wells belonging to the plate

Parameters:

name: str

name of the plate

experiment_id: int

ID of the parent Experiment

description: str, optional

description of the plate

acquisitions_location¶: str: location where acquisitions are stored

created_at¶: datetime: date and time when the row was inserted into the column

description¶: str: description provided by user

dimensions¶: Tuple[int]: number of wells in the plate along the vertical and horizontal axis, i.e. the number of rows and columns

experiment¶: tmlib.models.experiment.Experiment: parent experiment

experiment_id¶: int: ID of parent experiment

id¶

image_size¶: Tuple[int]: number of pixels along the vertical and horizontal axis

location¶: str: location were the plate is stored

n_wells¶: int: number of wells in the plate

name¶: str: name given by user

nonempty_columns¶: List[int]: indices of nonempty columns, i.e. columns of the plate where at least one well has been imaged

nonempty_rows¶: List[int]: indices of nonempty rows, i.e. rows of the plate where at least one well has been imaged

offset¶: Tuple[int]: y, x coordinate of the top, left corner of the plate relative to the layer overview at the maximum zoom level

status¶: str: upload status based on the status of acquisitions

updated_at¶: datetime: date and time when the row was last updated

well_grid¶: numpy.ndarray[int]: IDs of wells arranged according to their relative position within the plate

tmlib.models.plate.SUPPORTED_PLATE_AQUISITION_MODES = set(['multiplexing', 'basic'])¶: Supported plate acquisition modes. Mode “series” means that cycles are interpreted as separate acquisitions relating to the same marker as part of a time series experiment. Mode “multiplexing” implies that a different marker was used in each acquisition as part of a multiplexing experiment.

tmlib.models.plate.SUPPORTED_PLATE_FORMATS = set([96, 1, 384])¶: Supported plate formats (number of wells in the plate).

tmlib.models.plate.determine_plate_dimensions(n_wells)¶

Determines the dimensions of a plate given its number of wells.

Parameters:

n_wells: int

number of wells in the plate

Returns:

Tuple[int]

number of rows and column in the plate

tmlib.models.site module¶

class tmlib.models.site.Site(y, x, height, width, well_id, omitted=False)¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.IdMixIn

A site is a unique y, x position projected onto the plate bottom plane that was scanned by the microscope.

Attributes

shifts: [tmlib.models.alignment.SiteShifts]	shifts belonging to the site
channel_image_files: List[tmlib.models.file.ChannelImageFile]	channel image files belonging to the site

Parameters:

y: int

zero-based row index of the image within the well

x: int

zero-based column index of the image within the well

height: int

number of pixels along the vertical axis of the site

width: int

number of pixels along the horizontal axis of the site

well_id: int

ID of the parent well

omitted: bool, optional

whether the image file is considered empty, i.e. consisting only of background pixels without having biologically relevant information (default: False)

aligned_height¶: int: number of pixels along the vertical axis of the site after alignment between cycles

aligned_image_size¶: Tuple[int]: number of pixels along the vertical (y) and horizontal (x) axis, i.e. height and width of the aligned site

aligned_offset¶: Tuple[int]: y, x coordinate of the top, left corner of the site relative to the layer overview at the maximum zoom level after alignment for shifts between cycles

aligned_width¶: int: number of pixels along the horizontal axis of the site after alignment between cycles

bottom_residue¶: number of pixels that should be cropped at the bottom for alignment

coordinate¶: Tuple[int]: row, column coordinate of the site within the well

height¶: int: number of pixels along the vertical axis of the image

id¶

image_size¶: Tuple[int]: number of pixels along the vertical (y) and horizontal (x) axis, i.e. height and width of the site

left_residue¶: number of pixels that should be cropped at the left side for alignment

offset¶: Tuple[int]: y, x coordinate of the top, left corner of the site relative to the layer overview at the maximum zoom level

omitted¶: bool: whether the site should be omitted from further analysis

right_residue¶: number of pixels that should be cropped at the right side for alignment

top_residue¶: number of pixels that should be cropped at the top for alignment

well¶: tmlib.models.well.Well: parent well

well_id¶: int: ID of parent well

width¶: int: number of pixels along the horizontal axis of the image

x¶: int: zero-based column index of the image within the well

y¶: int: zero-based row index of the image within the well

tmlib.models.status module¶

class tmlib.models.status.FileUploadStatus¶

Bases: object

Upload status of a file.

COMPLETE = 'COMPLETE'¶: Upload is complete

FAILED = 'FAILED'¶: Upload has failed

UPLOADING = 'UPLOADING'¶: Upload is ongoing

WAITING = 'WAITING'¶: The file is registered, but upload not yet started

tmlib.models.submission module¶

class tmlib.models.submission.Submission(experiment_id, program, user_id)¶

Bases: tmlib.models.base.MainModel, tmlib.models.base.DateMixIn

A submission handles the processing of a computational task on a cluster.

Parameters:

experiment_id: int

ID of the parent experiment

program: str

name of the program that submits the tasks

user_id: int

ID of the submitting user

created_at¶: datetime: date and time when the row was inserted into the column

experiment¶: tmlib.models.experiment.Experimment: parent experiment

experiment_id¶: int: ID of the parent experiment

id¶

program¶: str: name of the program that submitted the tasks

top_task_id¶: int: ID of the top task in the submitted collection of tasks

updated_at¶: datetime: date and time when the row was last updated

user¶: tmlib.models.user.User: submitting user

user_id¶: int: ID of the submitting user

class tmlib.models.submission.Task(**kwargs)¶

Bases: tmlib.models.base.MainModel, tmlib.models.base.DateMixIn

A task represents a computational job that can be submitted to computational resources for processing. Its state will be monitored while being processed and statistics will be collected.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance’s class are allowed. These could be, for example, any mapped columns or relationships.

cpu_time¶: datetime.timedelta: total CPU time of task

created_at¶: datetime: date and time when the row was inserted into the column

data¶: Pickeled Python gc3libs.Task object

exitcode¶: int: exitcode

id¶

is_collection¶: bool: whether the task is a collection of tasks

memory¶: int: total memory in MG of task

name¶: str: name given by application

parent_id¶: int: ID of the parent task

state¶: str: procssing state, e.g. "RUNNING" or "TERMINATED"

submission_id¶: int: ID of parent submission

time¶: datetime.timedelta: total time of task

type¶: str: name of the corresponding Python object

updated_at¶: datetime: date and time when the row was last updated

tmlib.models.tile module¶

class tmlib.models.tile.ChannelLayerTile(z, y, x, channel_layer_id, pixels=None)¶

Bases: tmlib.models.base.DistributedExperimentModel

A channel layer tile is a component of an image pyramid. Each tile holds a single 2D 8-bit pixel plane with pre-defined dimensions.

Parameters:

z: int

zero-based zoom level index

y: int

zero-based row index of the tile at given zoom level

x: int

zero-based column index of the tile at given zoom level

channel_layer_id: int

ID of the parent channel pyramid

pixels: tmlib.image.PyramidTile, optional

pixels array (default: None)

channel_layer_id¶: int: ID of parent channel layer

pixels¶: tmlib.image.PyramidTile: pixel data and metadata

x¶: int: zero-based coordinate on horizontal axis

y¶: int: zero-based coordinate on vertical axis

z¶: int: zero-based zoom level index

tmlib.models.types module¶

Custom SQLalchemy data types.

class tmlib.models.types.ST_Boundary(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_Boundary.

identifier = 'ST_Boundary'¶

name = 'ST_Boundary'¶

type¶: alias of Geometry

class tmlib.models.types.ST_Expand(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_Expand.

identifier = 'ST_Expand'¶

name = 'ST_Expand'¶

type¶: alias of Geometry

class tmlib.models.types.ST_ExteriorRing(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_ExteriorRing.

identifier = 'ST_ExteriorRing'¶

name = 'ST_ExteriorRing'¶

type¶: alias of Geometry

class tmlib.models.types.ST_GeomFromText(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_GeomFromText.

identifier = 'ST_GeomFromText'¶

name = 'ST_GeomFromText'¶

type¶: alias of Geometry

class tmlib.models.types.ST_IsValid(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_IsValid.

identifier = 'ST_IsValid'¶

name = 'ST_IsValid'¶

type¶: alias of Geometry

class tmlib.models.types.ST_SimplifyPreserveTopology(*args, **kwargs)¶

Bases: geoalchemy2.functions.GenericFunction

Implementation of the Postgis function ST_SimplifyPreserveTopology.

identifier = 'ST_SimplifyPreserveTopology'¶

name = 'ST_SimplifyPreserveTopology'¶

type¶: alias of Geometry

class tmlib.models.types.XML(*args, **kwargs)¶

Bases: sqlalchemy.sql.type_api.TypeDecorator

XML data column type.

Construct a TypeDecorator.

Arguments sent here are passed to the constructor of the class assigned to the impl class level attribute, assuming the impl is a callable, and the resulting object is assigned to the self.impl instance attribute (thus overriding the class attribute of the same name).

If the class level impl is not a callable (the unusual case), it will be assigned to the same instance attribute ‘as-is’, ignoring those arguments passed to the constructor.

Subclasses can override this to customize the generation of self.impl entirely.

bind_processor(dialect)¶

get_col_spec()¶

impl¶: alias of UnicodeText

process_result_value(value, dialect)¶

type¶: alias of _Element

tmlib.models.user module¶

class tmlib.models.user.User(name, email, password)¶

Bases: tmlib.models.base.MainModel, tmlib.models.base.DateMixIn

A TissueMAPS user.

Attributes

submissions: List[tmlib.models.submission.Submission]

submissions belonging to the user

Parameters:

name: str

user name

email: str

email address

password: str

password

created_at¶: datetime: date and time when the row was inserted into the column

email¶: str: email

experiments¶: List[tmlib.models.experiment.ExperimentReferences]: references to owned experiments

id¶

name¶: str: username

password¶: str: encoded password

updated_at¶: datetime: date and time when the row was last updated

tmlib.models.utils module¶

tmlib.models.utils.DATABASE_ENGINES = {}¶: Dict[str, sqlalchemy.engine.base.Engine]: mapping of chached database engine objects for reuse within the current Python process hashable by URL

class tmlib.models.utils.ExperimentConnection(experiment_id, transaction=False)¶

Bases: tmlib.models.utils._Connection

Database connection for executing raw SQL statements for an experiment-specific database outside of a transaction context.

See also

tmlib.models.base.ExperimentModel

Examples

>>> import tmlib.models as tm
>>> with tm.utils.ExperimentConnection(experiment_id=1) as connection:
>>>     connection.execute('SELECT mapobject_id, value FROM feature_values')
>>>     print(connection.fetchall())

Parameters:

experiment_id: int

ID of the experiment that should be queried

transaction: bool, optional

whether a transaction should be began

get_partition_placement(model_cls, partition_key)¶: Finds the location of a partition of a distributed table.

get_unique_ids(model, n)¶

Gets a unique, but shard-specific value for the distribution column.

Parameters:

model: class

class derived from ExperimentModel

n: int

number of IDs that should be returned

Returns:

List[int]

unique, shard-specific IDs

Raises:

ValueError

when the table of model is not distributed

locate_partition(model, partition_key)¶

Determines the location of a table partition (shard).

Parameters:

model: class

class derived from ExperimentModel

partition_key: int

value of the distribution column

Returns:

Tuple[Union[str, int]]

host and port of the worker server and the ID of the shard

class tmlib.models.utils.ExperimentSession(experiment_id, transaction=True)¶

Bases: tmlib.models.utils._Session

Session scopes for interaction with an experiment-secific database. If transaction is set to True (default), all changes get automatically committed at the end of the interaction. In case of an error, a rollback is issued. If transaction is set to False, the session will be in autocomit mode and every query will be immediately flushed to the database.

See also

tmlib.models.base.ExperimentModel

Examples

>>> import tmlib.models as tm
>>> with tm.utils.ExperimentSession(experiment_id=1) as session:
>>>     print(session.query(tm.Plate).all())

Parameters:

experiment_id: int

ID of the experiment that should be queried

transaction: bool, optional

whether a transaction should be used; distributed tables cannot be modified within a transaction context (default: True)

class tmlib.models.utils.ExperimentWorkerConnection(experiment_id, host, port, transaction=False)¶

Bases: tmlib.models.utils._Connection

Database connection for executing raw SQL statements on a database “worker” server to target individual shards of a distributed table.

See also

tmlib.models.base.ExperimentModel

Parameters:

experiment_id: int

ID of the experiment that should be queried

host: str

IP address or name of database worker server

port: str

port to which database worker server listens

transaction: bool, optional

whether a transaction should be began

class tmlib.models.utils.MainConnection(transaction=False)¶

Bases: tmlib.models.utils._Connection

Database connection for executing raw SQL statements for the main tissuemaps database outside of a transaction context.

See also

tmlib.models.base.MainModel

Examples

>>> import tmlib.models as tm
>>> with tm.utils.MainConnection() as connection:
>>>     connection.execute('SELECT name FROM plates')
>>>     print(connection.fetchall())

Parameters:

transaction: bool, optional

whether a transaction should be began

class tmlib.models.utils.MainSession(transaction=True)¶

Bases: tmlib.models.utils._Session

Session scopes for interaction with the main tissuemaps database. All changes get automatically committed at the end of the interaction. In case of an error, a rollback is issued.

See also

tmlib.models.base.MainModel

Examples

>>> import tmlib.models as tm
>>> with tm.utils.MainSession() as session:
>>>    print(session.query(tm.ExperimentReference).all())

Parameters:

transaction: bool, optional

whether a transaction should be used (default: True)

tmlib.models.utils.POOL_SIZE = 5¶: int: number of pooled database connections

class tmlib.models.utils.Query(*args, **kwargs)¶

Bases: sqlalchemy.orm.query.Query

A custom query class.

delete()¶

Performs a bulk delete query.

Returns:

int

count of rows matched as returned by the database’s “row count” feature

tmlib.models.utils.create_db_engine(db_uri, cache=True)¶

Creates a database engine with a given pool size.

Parameters:

db_uri: str

database uri

cache: bool, optional

whether engine should be cached for reuse (default: True)

Returns:

sqlalchemy.engine.base.Engine

created database engine

tmlib.models.utils.create_db_session_factory()¶

Creates a factory for creating a scoped database session that will use Query to query the database.

Returns:	sqlalchemy.orm.session.Session

tmlib.models.utils.create_db_tables(engine)¶

Creates all tables in the public schema.

Parameters:	engine: sqlalchemy.engine

tmlib.models.utils.delete_location(path)¶

Deletes a location on disk.

Parameters:

path: str

absolute path to directory or file

tmlib.models.utils.exec_func_after_insert(func)¶

Decorator function for a database model class that calls the decorated function after an insert event.

Parameters:	func: function

Examples

@exec_func_after_insert(lambda target: do_something()) SomeClass(db.Model):

tmlib.models.utils.parallelize_query(func, args)¶

Parallelize database query. This can be useful for targeting different shards of a distributed table. The number of parallel connections depends on the value of POOL_SIZE.

Parameters:

func: function

a function that establishes a database connection and executes a given SQL query; function must return a list

args: Union[list, generator]

arguments that should be parsed to the function

Returns:

list

aggregated output of function return values

tmlib.models.utils.remove_location_upon_delete(cls)¶

Decorator function for an database model class that automatically removes the location that represents an instance of the class on the filesystem once the corresponding row is deleted from the database table.

Parameters:

cls: tmlib.models.base.DeclarativeABCMeta

implemenation of tmlib.models.base.FileSystemModel

Raises:

AttributeError

when decorated class doesn’t have a “location” attribute

tmlib.models.utils.set_pool_size(n)¶: Sets the pool size for database connections of the current Python process.

tmlib.models.well module¶

class tmlib.models.well.Well(name, plate_id, description={})¶

Bases: tmlib.models.base.ExperimentModel, tmlib.models.base.DateMixIn, tmlib.models.base.IdMixIn

A well is a reservoir for a biological sample and multiple wells are typically arranged as a grid on a Plate.

Parameters:

name: str

name of the well

plate_id: int

ID of the parent plate

description: Dict[str, Union[str, int, float, bool, List, Dict]], optional

description of the well content

coordinate¶: Tuple[int]: row, column coordinate of the well within the plate

created_at¶: datetime: date and time when the row was inserted into the column

description¶: dict: description of the well content in form of a mapping to store for example the name or ID of a stain, compound or gene

dimensions¶: Tuple[int]: number of sites in the well along the vertical and horizontal axis, i.e. the number of rows and columns

id¶

image_size¶: Tuple[int]: number of pixels along the vertical and horizontal axis

static map_coordinate_to_name(coordinate)¶

Maps coordinate to identifier string representation.

Parameters:

coordinate: Tuple[int]

zero-based row, column position of a given well within the plate

Returns:

str

identifier string representation of a well

Examples

>>> Well.map_coordinate_to_name((0, 1))
"A02"

static map_name_to_coordinate(name)¶

Maps identifier string representation to coordinate.

Parameters:

name: str

well name

Returns:

Tuple[int]

zero-based row, column position of a given well within the plate

Examples

>>> Well.map_name_to_coordinate("A02")
(0, 1)

name¶: str: name assinged by the application, e.g. "A01"

offset¶: Tuple[int]: y, x coordinate of the top, left corner of the site relative to the layer overview at the maximum zoom level

plate¶: tmlib.models.plate.Plate: parent plate

plate_id¶: int: ID of parent plate

site_grid¶: numpy.ndarray[int]: IDs of sites arranged according to their relative position within the well

updated_at¶: datetime: date and time when the row was last updated

x¶: int: zero-based x-coordinate (column) of the well within the plate

y¶: int: zero-based y-coordinate (row) of the well within the plate

tmlib.models package¶

Module contents¶

Submodules¶

tmlib.models.acquisition module¶

tmlib.models.alignment module¶

tmlib.models.base module¶

tmlib.models.channel module¶

tmlib.models.cycle module¶

tmlib.models.dialect module¶

tmlib.models.experiment module¶

tmlib.models.feature module¶

tmlib.models.file module¶

tmlib.models.mapobject module¶

tmlib.models.plate module¶

tmlib.models.site module¶

tmlib.models.status module¶

tmlib.models.submission module¶

tmlib.models.tile module¶

tmlib.models.types module¶

tmlib.models.user module¶

tmlib.models.utils module¶

tmlib.models.well module¶

Table Of Contents

Related Topics