ROI classification

pydetecdiv.plugins.roi_classification

An plugin for deep-learning ROI classification

class pydetecdiv.plugins.roi_classification.Results(**kwargs)

Bases: Base

The DAO defining and handling the table to store results

save(project: Project, run: Run, roi: ROI, t: int, predictions: ndarray, class_names: list[str]) → None

Save the results from a plugin run on a ROI at time t into the database

Parameters:

  • project – the current project

  • run – the current run

  • roi – the current ROI

  • t – the current frame

  • predictions – the list of prediction values

  • class_names – the class names

class pydetecdiv.plugins.roi_classification.TrainingData(**kwargs)

Bases: Base

The DAO defining and handling the table to store information about ROIs in training, validation and test datasets

save(project: Project, roi_id: int, t: int, target: int, dataset: int) → None

Save the results from a plugin run on a ROI at time t into the database

Parameters:

  • project – the current project

  • run – the current run

  • roi – the current ROI

  • t – the current frame

  • predictions – the list of prediction values

  • class_names – the class names

class pydetecdiv.plugins.roi_classification.DataProvider(h5file_name: str, indices: list, batch_size: int = 32, image_shape: tuple[int, int] = (60, 60), seqlen: int = 0, name: str | None = None, targets: bool = False, shuffle: bool = True, **kwargs)

Bases: Sequence

An extension of keras Sequence for prediction. For performance reasons, data are stored in an HDF5 file

on_epoch_end() → None

Operations to run at the end of each epoch (shuffle dataset, etc.)

close() → None

Closes the HDF5 file

class pydetecdiv.plugins.roi_classification.PredictionBatch(fov_data: DataFrame, roi_list: DataFrame, image_size: tuple[int, int] = (60, 60), seqlen: int = None, z_channels: list[int] = None)

Bases: Sequence

An extension of keras Sequence for prediction. For performance reasons, FOVs are read sequentially and all ROIs from a FOV are

placed in the same batch.

class pydetecdiv.plugins.roi_classification.Plugin

Bases: Plugin

A class extending plugins.Plugin to handle the example plugin

register() → None

Registers the plugin

update_parameters(groups: list[str] | str = None) → None

Updates parameters

Parameters:

groups – the parameter groups to update

class_names(as_string: bool = True) → list[str] | str

Returns the classes currently in use

Returns:

the class list

create_table() → None

Create the table to save results if it does not exist yet

addActions(menu: QMenu) → None

Overrides the addActions method in order to create a submenu with several actions for the same menu

Parameters:

menu (QMenu) – the parent menu

set_enabled_actions(manual_annotation: QAction, train_model: QAction, fine_tuning: QAction, predict: QAction, show_results: QAction, export_classification: QAction) → None

Enables or disables actions in menus according to the level of completion of the ROI classification workflow.

Parameters:

  • manual_annotation – Manual annotation, enabled only if there are ROIs to annotate

  • train_model – Train model, enabled only if there are annotated ROIs

  • fine_tuning – Fine-tuning, enabled only if a previously trained model is available (previous training or import)

  • predict – Prediction, enabled only if a previously trained model is available (previous training or import)

  • show_results – Showing results, enabled only if a prediction has been run

  • export_classification – Export classification, enabled only if a prediction has been run

add_context_action(data: tuple[QGraphicsRectItem, QMenu]) → None

Adds an action to annotate the ROI from the FOV viewer

Parameters:

data – the data sent by the PyDetecDiv().viewer_roi_click signal

import_annotated_rois() → None

Selects a csv file containing ROI frames annotations and open a FOV2ROIlinks window to load the data it contains into the database as FOVs and ROIs with annotations.

manual_annotation(trigger=None, roi_selection: list[ROI] = None, run: Run = None) → None

Opens a ManualAnnotator widget to annotate a list of ROIs

Parameters:

  • trigger – the data passed by the triggered action

  • roi_selection – the list of ROIs

  • run – the current Run instance

resume_manual_annotation(annotator: ManualAnnotator, roi_selection: list[ROI] = None, run: Run = None) → None

Resumes manual annotation after the creation of a new set of classes

Parameters:

  • annotator – the Annotator widget

  • roi_selection – the selection of ROIs

  • run – the current Run

show_results(trigger=None, roi_selection: list[ROI] = None) → None

Show predictions results in an Annotator widget

Parameters:

  • trigger – the data passed by the triggered action

  • roi_selection – the list of ROIs to show results for

get_classification_df(roi_selection: list[int] = None, ground_truth: bool = True, run_list: list[int] = None) → DataFrame

Gets classification in a pandas DataFrame

Parameters:

  • roi_selection – the list of ROI indices to save

  • ground_truth – if True, includes ground truth

  • run_list – the list of Run ids to include

Returns:

the pandas DataFrame with the annotations

export_classification_to_csv(trigger, filename: str = 'classification.csv', roi_selection: list[int] = None, ground_truth: bool = True, run_list: list[int] = None) → None

Exports classification of ROIs in a CSV file

Parameters:

  • trigger – the data passed by the triggered action

  • filename – the CSV file name

  • roi_selection – the list of ROI indices to include in the CSV file

  • ground_truth – if True, ground truth classification is saved in the file

  • run_list – the list of Run indices to export classification for

get_classification_runs() → dict

Gets previous annotation and prediction runs

Returns:

a dictionary containing the ids of all classification runs corresponding to a given list of classes

get_annotation_runs() → dict

Gets previously run prediction runs

return:

a dictionary containing the ids of all prediction runs corresponding to a given list of classes

get_prediction_runs() → dict

Gets previously run prediction runs

Returns:

a dictionary containing the ids of all prediction runs corresponding to a given list of classes

run_prediction() → None

Runs a prediction process. Opens the PredictionDialog widget

run_training() → None

Runs a model training process. Opens the TrainingDialog widget

run_fine_tuning() → None

Runs fine-tuning process. Opens the FineTuningDialog widget

load_models() → None

Loads available models (modules)

select_saved_parameters(weights_file: str) → None
Gets the parameters of a previous model training run, identified by the name of the saved weight file, for display in the

GUI form

Parameters:

weights_file – the weights file name

update_model_weights(project_name: str = None) → None

Update the list of model weights associated with training and fine-tuning runs

update_class_names(prediction: bool = False) → None

Update the classes associated with the currently selected model

update_channels() → None

Updates the list of available channels to display in the GUI form

update_fov_list() → None

Updates the list of available FOV to display in the GUI form

save_annotations(roi: ROI, roi_classes: list[int], run) → None

Saves manual annotation into the database

Parameters:

  • roi – the annotated ROI

  • roi_classes – the classes along time

  • run – the annotation run

get_annotated_rois(run: Run = None, ids_only: bool = False) → list[ROI] | list[int]

Gets a list of annotated ROI frames

Returns:

the list of annotated ROI frames

get_unannotated_rois()

Gets the unannotated ROIs

Returns:

list of unannotated ROIs and list of all ROIs

get_annotation(roi: ROI, as_index: bool = True) → list[int] | list[str]

Get the annotations for a ROI

Parameters:

  • roi – the ROI

  • as_index – bool set to True to return annotations as indices of class_names list, set to False to return annotations as class names

Returns:

the list of annotated classes by frame

get_classifications(roi: ROI, run_list: list[int], as_index: bool = True) → list[int] | list[str]

Get the annotations for a ROI as defined in a list of runs

Parameters:

  • roi – the ROI

  • run_list – the list of Runs where the ROI was annotated or classified

  • as_index – bool set to True to return annotations as indices of class_names list, set to False to return annotations as class names

Returns:

the list of annotated classes by frame

get_all_annotations(z_layers: tuple[int] = None) → DataFrame
Gets all annotations, i.e. Run id, ROI and FOV ids, ROI positions, time frame, channel, z layer index, class name,

file name, drift values

Parameters:

z_layers – the z layers to be used as RGB channels

Returns:

the annotations in a pandas DataFrame

get_fov_data(z_layers: tuple[int] = None, channel: int = None) → DataFrame

Gets FOV data, i.e. FOV id, time frame and the list of files with the z layers to be used as RGB channels

Parameters:

  • z_layers – the z layer files to use as RGB channels

  • channel – the channel of the original image to be loaded

Returns:

a pandas DataFrame with the FOV data

get_roi_list() → DataFrame

Gets a list of ROIs in a DataFrame

Returns:

pandas DataFrame containing ROIO id, FOV id, ROI x, y positions of top left and bottom right corners

get_annotations() → DataFrame

Gets ROI annotations from manual annotation or imported annotation runs in a DataFrame

Returns:

a pandas DataFrame containing run id, ROI and FOV ids, frame, class name

get_drift_corrections() → DataFrame

Gets the drift correction values for FOVs in a pandas DataFrame

Returns:

a pandas DataFrame with the (dx, dy) drift values for each FOV and frame

layers2channels(zfiles) → list[str]

Gets the image file names for red, green and blue channels

Parameters:

zfiles – the list of files corresponding to one z layer each

Returns:

the list of zfiles

create_hdf5_annotated_rois(hdf5_file: str, z_channels: list[int] = None, channel: int = 0) → None

Creates a HDF5 file containing the annotated ROI data and their targets

Parameters:

  • hdf5_file – the HDF5 file name

  • z_channels – the z layers to be used as channels

  • channel – the original channel to use

prepare_data_for_training(hdf5_file: str, seqlen: int = 0, train: float = 0.6, validation: float = 0.2, seed: int = 42)

Prepares data for training.

Parameters:

  • hdf5_file – the HDF5 file containing the annotated ROI data

  • seqlen – the length of ROI time sequence or 0 if the model takes a single image as input

  • train – the proportion of images or sequences used for training

  • validation – the proportion of images or sequences used for validation

  • seed – the seed used to shuffle the data before dispatching data into the datasets

Returns:

indices for training, validation, testing datasets

prepare_data_for_classification(fov_list: list[int], z_channels: tuple[int] = None) -> (<class 'pandas.core.frame.DataFrame'>, <class 'pandas.core.frame.DataFrame'>, <class 'numpy.ndarray'>)

Prepares the data for class prediction. Drift correction is automatically applied

Parameters:

  • fov_list – the list of FOV indices whose ROIs should be classified

  • z_channels – the z layers to be used as channels

Returns:

Pandas DataFrames containing FOV data, list of (ROI, frame) with positions, unique indices of ROIs

compute_class_weights() → dict

Computes class weights from their cardinality

Returns:

a dictionary with class weights

lr_decay(epoch: int, lr: float) → float

Learning rate scheduler

Parameters:

  • epoch – the current epoch

  • lr – the current learning rate

Returns:

the new learning rate

train_model() -> (<class 'str'>, <class 'str'>, <class 'keras.src.callbacks.History'>, list[int], list[int], list[int], list[int])

Launches model training: select the network, load weights (optional), define the training, validation and test sets, then run the training using training and validation sets and the evaluation on the test set.

save_training_run(finetune: bool = False) → Run

Saves the current training Run

Parameters:

finetune – False if the run is a training run (from scratch or pretrained Keras model), True if it is a fine-tuning run

Returns:

the current Run instance

save_training_datasets(hdf5_file: str, training_idx: list[int], validation_idx: list[int], test_idx: list[int]) → None

Saves in TrainingData table the (ROI, frame, class) data subsets that were used for training, validation and testing while training a model.

Parameters:

  • hdf5_file – the HDF5 file containing the annotated ROI data

  • training_idx – the list of (ROI, frame) data indices in the training dataset

  • validation_idx – the list of (ROI, frame) data indices in the validation dataset

  • test_idx – the list of (ROI, frame) data indices in the validation dataset

predict() → None

Running prediction on all ROIs in selected FOVs.

save_annotation(project: Project, run: Run, roi: ROI, frame: int, class_name: str) → None

Saves the results in database

Parameters:

  • project – the current project

  • run – the current run

  • roi – the current ROI

  • frame – the current frame

  • class_name – the class name

run_import_classifier() → None
Gets all runs with an available classifier from another project, and launches the ImportClassifierDialog for the

user to choose one

import_classifier() → None

Imports a classifier, i.e. a combination of a deep-learning network/model, weights trained on annotated data and class names

draw_annotated_rois() → None

Draw annotated ROIs as rectangles coloured according to the class

pydetecdiv.plugins.roi_classification.get_lr_metric(optimizer: Optimizer) → Callable

Get the learning rate metric for optimizer for use during training to monitor the learning rate

Parameters:

optimizer – the optimizer

Returns:

the learning rate function

pydetecdiv.plugins.roi_classification.lr_exp_decay(epoch: int, lr: float) → float

Learning rate scheduler for exponential decay

Parameters:

  • epoch – the current epoch

  • lr – the current learning rate

Returns:

the new learning rate

pydetecdiv.plugins.roi_classification.gui.classification

GUI for ROI manual annotation and class visualization

class pydetecdiv.plugins.roi_classification.gui.classification.AnnotationScene(**kwargs)

Bases: Scene

The viewer scene where images and other items are drawn

property annotation_tool

Returns the annotation tool parent of this annotation scene :return: the annotation tool

mouseMoveEvent(event: QMouseEvent) → None

Detect mouse movement and apply the appropriate method according to the currently checked drawing tool and key modifier

Parameters:

event – the mouse move event

mousePressEvent(event: QMouseEvent) → None

Detect when the left mouse button is pressed and perform the action corresponding to the currently checked drawing tool

Parameters:

event – the mouse press event

class pydetecdiv.plugins.roi_classification.gui.classification.AnnotationTool

Bases: VideoPlayer

Annotator class extending the VideoPlayer class to define functionalities specific to ROI image annotation

property class_names: list[str]

The current list of class names

Returns:

The current list of class names

property annotation_run_list

The list of annotation and/or prediction run ids available in the project

roi_class(frame: int) → str

The class name of the current ROI at the given frame

Parameters:

frame – the frame index

Returns:

the class name

setup(menubar: QMenuBar = None, plugin: Plugin = None, scene: AnnotationScene = None) → None

Sets the Annotation tool up

Parameters:

  • menubar – the menu bar

  • plugin – the plugin

  • scene – the annotation scene to display plots in

set_title(title: str) → None

Sets the title of the Annotation tool

Parameters:

title – the title to display

set_run_list() → None

Abstract method for setting the list of annotation or prediction runs that use the specified class names

change_frame(T: int = 0) → None

Change frame and display the corresponding class name below the image :param T: the frame index

display_class_name(roi_class: str = None) → None

Display the class name below the frame :param roi_class: the class name

plot_roi_classes() → None

A convenience method to plot the class annotations or predictions for a list of ROIs

select_class_names(class_names: str) → None

Select the set of classes

Parameters:

class_names – the set of classes

update_ROI_selection(class_names: str) → None

Updates the ROI selection according to a set of classes.

Parameters:

class_names – the set of classes

load_selected_ROIs() → None

Abstract method loading selected ROIs. This method is executed when the set of classes has been changed to select the ROIs that have been annotated or classified using those classes

update_roi_classes_plot() → None

Updates the chart view plot whenever there has been a modification

set_roi_list(roi_selection: list[ROI]) → None

Sets the list of ROIs to annotate as an iterator :param roi_selection: the list of ROIs

next_roi() → None

Jumps to first frame of next ROI if there is one.

previous_roi() → None

Jumps back to the first frame of previous ROI in the list

display_roi() → None

Displays a ROI image

get_roi_annotations() → None

Retrieve from the database the manual annotations for a ROI

class pydetecdiv.plugins.roi_classification.gui.classification.AnnotationChartView(parent=None, annotator=None)

Bases: ChartView

Generic Annotation viewer providing features common to chart views in Annotator and Prediction viewer

property class_names: list[str]

The list of class names

Returns:

The list of class names

plot_roi_classes(roi_classes_idx: list[int]) → None

Plots ROI classes along time for a selection of ROIs, one ROI after the other

Parameters:

roi_classes_idx – the list of ROI ids to display classes

clicked(plot: ScatterPlotItem, points: list[SpotItem]) → None

If a point representing the annotation of the ROI for a given frame has been clicked, then the time cursor is moved to the clicked frame

Parameters:

  • plot – the ScatterPlotItem displaying the annotation plot

  • points – the clicked point

class pydetecdiv.plugins.roi_classification.gui.classification.ManualAnnotator

Bases: AnnotationTool

Class to annotate ROIs along time

property annotation_run_list: list[int]

The list of annotation runs available in the project (manual annotation and annotation import)

Returns:

the list of Run ids

setup(menubar: QMenuBar = None, plugin: Plugin = None, scene: AnnotationScene = None) → None

Sets the Manual annotator up

Parameters:

  • menubar – the menu bar

  • plugin – the plugin

  • scene – the annotation scene to display plots in

keyPressEvent(event: QKeyEvent) → None

Handle actions triggered by pressing keys when the scene is in focus. Letters from azertyuiop assign a class to the current frame, and jumps to the next frame suggesting a class Space bar validates the suggested assignation and jumps to the next frame Right arrow moves one frame forward Left arrow moves one frame backwards Enter key validates the current suggestion, saves the annotations to the database and jumps to the nex ROI if there is one Escape key cancels annotations and jumps to the next ROI if there is one

Parameters:

event – the keyPressEvent

define_classes(suggestion: list[str] = None) → None

Launch the class set definition interface

Parameters:

suggestion – the default list of classes

load_selected_ROIs() → None

Loads the annotated and/or annotated ROIs and shows the first ROI classes accordingly

annotate_current(class_name: str = None) → None

Assign the class name to the current frame :param class_name: the class name

save_run() → None

Save the current ROI annotation process in the database

class pydetecdiv.plugins.roi_classification.gui.classification.PredictionViewer

Bases: AnnotationTool

Class to visualize predictions for ROIs along time

setup(menubar: QMenuBar = None, plugin: Plugin = None, scene: AnnotationScene = None) → None

Sets the Viewer up

Parameters:

  • menubar – the menu bar

  • plugin – the plugin

  • scene – the annotation scene to display plots in

property annotation_run_list: list[int]

Gets the list of available annotation runs

Returns:

the list of annotation runs

load_selected_ROIs() → None

Selects the prediction run checked in the menu

select_prediction_run(prediction_run: QAction) → None

Loads the ROIs corresponding to the selected prediction run and shows ROI class predictions accordingly

Parameters:

prediction_run – the requested QAction of prediction runs menu

keyPressEvent(event: QKeyEvent) → None

Handle actions triggered by pressing keys when the scene is in focus. Letters from azertyuiop assign a class to the current frame, and jumps to the next frame suggesting a class Space bar validates the suggested assignation and jumps to the next frame Right arrow moves one frame forward Left arrow moves one frame backwards Enter key validates the current suggestion, saves the annotations to the database and jumps to the nex ROI if there is one Escape key cancels annotations and jumps to the next ROI if there is one :param event: the keyPressEvent

class pydetecdiv.plugins.roi_classification.gui.classification.AnnotationMenuBar(parent: AnnotationTool)

Bases: QMenuBar

The menu bar for annotation tools

setup() → None

Sets the menus up

set_class_names_choice() → None

Sets the options for class names, i.e. all lists of class names that are available in the current project.

class pydetecdiv.plugins.roi_classification.gui.classification.ManualAnnotationMenuBar(parent: ManualAnnotator)

Bases: AnnotationMenuBar

The menu bar for a manual annotator widget

setup() → None

Sets the menus up

set_class_names_choice() → None

Sets the options for class names, i.e. all lists of class names that are associated to annotation runs available in the current project.

class pydetecdiv.plugins.roi_classification.gui.classification.PredictionMenuBar(parent: PredictionViewer)

Bases: AnnotationMenuBar

The menu bar for prediction viewers

setup() → None

Sets the menus up

set_class_names_choice() → None

Sets the options for class names, i.e. all lists of class names that are associated to a classifier (trained or imported) available in the current project.

set_run_choice(class_names: str) → None

Sets the menu to choose the prediction run. Runs are selected according to the class names. By default, the last prediction run is selected.

Parameters:

class_names – the class names to select prediction runs

class pydetecdiv.plugins.roi_classification.gui.classification.DefineClassesDialog(annotator: ManualAnnotator, plugin: Plugin, suggestion: list[str] = None)

Bases: Dialog

Dialog window to define a set of classes for manual annotation

setup_class_names(suggestion: list[str] = None) → None

Sets the initial list of classes. If no suggestion is made, then the current list stored in the plugin is used. If none is available, then [‘A’, ‘B’] is proposed. Otherwise, the suggestion made is used.

Parameters:

suggestion – the class list suggestion

save_new_classes() → None

Saves the new list of classes in the current annotation Run table

import_classes() → None

Selects a csv file containing ROI frames annotations and open a FOV2ROIlinks window to load the data it contains into the database as FOVs and ROIs with annotations.

class pydetecdiv.plugins.roi_classification.gui.classification.ClassListView(parent: DefineClassesDialog, parameter: Parameter = None, height: int = None, multiselection: bool = False, **kwargs: dict[str, Any])

Bases: ListView

A list view used to define a set of classes for annotation

contextMenuEvent(e: QContextMenuEvent) → None

Defines a contextual menu

Parameters:

e – the contextual menu event

add_class() → None

Adds a new line to the current ListView model and selects it for the user to type the new class name in

pydetecdiv.plugins.roi_classification.gui.ImportAnnotatedROIs

Dialog window handling the definition of patterns for FOV creation from raw data file names

Bases: QDialog, Ui_FOV2ROIlinks

A class extending the QDialog and the Ui_RawData2FOV classes. Ui_RawData2FOV was created using QTDesigner

reset()

Reset the form with default patterns and colours

get_regex()

Build the complete regular expression from the individual patterns

Returns:

the regular expression string

change_sample_style()

Change the colours of file name samples showing the pattern matches.

find_matches(fov_names, regexes)

Find a list of matches with the defined regular expressions

Parameters:

regexes – the list of regular expressions to match

Returns:

a list of matches

static get_match_spans(matches, group)

Get the list of group positions for matches

Parameters:

  • matches – the list of matches

  • group – the group index to retrieve the spans for

Returns:

a dictionary of spans for the patterns (FOV, C, T, Z)

static get_match_span(match, group=2)

Get the span of a given group in a match

Parameters:

  • match – the match

  • group – the group index

Returns:

the group match span

colourize_matches(matches)

Find matches in file name samples and colourize them accordingly. Non-matching pattern check boxes’ background is set to orange. Conflicting patterns (having overlapping matches) are shown in red.

Parameters:

matches – the list of matches to colourize

overlap(start1, end1, start2, end2)

Checks whether positions overlap

Parameters:

  • start1 – start of first span

  • end1 – end of first span

  • start2 – start of second span

  • end2 – end of second span

Returns:

True if spans overlap, False otherwise

colourize_labels(pattern, colour)

Colourize the file name samples matching the pattern with the specified colour

Parameters:

  • pattern – the pattern

  • colour – the colour

clear_colours()

Clear colours to avoid overlapping style sheets

choose_colour(object_name)

Choose colour for a given pattern specified by its object name

Parameters:

object_name – the object name

show_chosen_colours()

Show the chosen colour in the little square box on the right and the border of the pattern.

button_clicked(button)

React to clicked button

Parameters:

button – the button that was clicked

save_run()

Save current run :return: the Run instance

create_annotated_rois(regex, run)

The actual FOV creation and data annotation method

Parameters:

regex – the regular expression to use for data annotation

pydetecdiv.plugins.roi_classification.gui.training

class pydetecdiv.plugins.roi_classification.gui.training.TrainingDialog(plugin, title=None)

Bases: Dialog

update_datasets(changed_dataset=None)

Update the proportion of data to dispatch in training, validation and test datasets. The total must sum to 1 and the modifications are constrained to ensure it is the case.

Parameters:

changed_dataset – the dataset that has just been changed

class pydetecdiv.plugins.roi_classification.gui.training.FineTuningDialog(plugin, title=None)

Bases: Dialog

pydetecdiv.plugins.roi_classification.gui.training.plot_history(history, evaluation)

Plots metrics history.

Parameters:

  • history – metrics history to plot

  • evaluation – metrics from model evaluation on test dataset, shown as horizontal dashed lines on the plots

pydetecdiv.plugins.roi_classification.gui.training.plot_confusion_matrix(ground_truth, predictions, class_names)

Plot the confusion matrix normalized i) by rows (recall in diagonals) and ii) by columns (precision in diagonals)

Parameters:

  • ground_truth – the ground truth index values

  • predictions – the predicted index values

  • class_names – the class names

Returns:

the plot viewer where the confusion matrix is plotted

class pydetecdiv.plugins.roi_classification.gui.training.ImportClassifierDialog(plugin, title=None)

Bases: Dialog

pydetecdiv.plugins.roi_classification.gui.prediction

class pydetecdiv.plugins.roi_classification.gui.prediction.PredictionDialog(plugin, title=None)

Bases: Dialog