TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals

Tracking cells in time-lapse videos is an essential technique for monitoring cell population dynamics at a single-cell level. Current methods for cell tracking are developed on videos with mostly single, constant signals and do not detect pivotal events such as cell death. Here, we present TransientTrack, a deep learning-based framework for cell tracking in multi-channel microscopy video data with transient fluorescent signals that fluctuate over time following processes such as the circadian rhythm of cells. By identifying key cellular events - mitosis (cell division) and apoptosis (cell death) our method allows us to build complete trajectories, including cell lineage information. TransientTrack is lightweight and performs matching on cell detection embeddings directly, without the need for quantification of tracking-specific cell features. Furthermore, our approach integrates Transformer Networks, multi-stage matching using all detection boxes, and the interpolation of missing tracklets with the Kalman Filter. This unified framework achieves strong performance across diverse conditions, effectively tracking cells and capturing cell division and death. We demonstrate the use of TransientTrack in an analysis of the efficacy of a chemotherapeutic drug at a single-cell level. The proposed framework could further advance quantitative studies of cancer cell dynamics, enabling detailed characterization of treatment response and resistance mechanisms. The code is available at https://github.com/bozeklab/TransientTrack.

💡 Research Summary

TransientTrack introduces a novel deep‑learning framework for multi‑object tracking of cancer cells in time‑lapse microscopy videos that contain multiple fluorescent channels with transient, oscillatory signals. Traditional cell‑tracking methods, largely benchmarked on the Cell Tracking Challenge, assume static illumination, single‑channel images, and focus primarily on cell division, neglecting apoptosis detection. To address these gaps, the authors design a system that (1) directly processes multi‑channel data where each channel reports a different biological state (cell‑cycle reporter, circadian clock reporter, and fitness reporter), (2) uses a lightweight detection network (Deformable DETR) whose Transformer decoder provides both bounding‑box predictions and high‑dimensional embeddings for each detected cell, (3) performs data association solely on spatial coordinates and embedding similarity, eliminating the need for hand‑crafted appearance features, (4) incorporates a two‑stage matching strategy inspired by ByteTrack, separating high‑confidence detections from low‑confidence ones, and (5) fills gaps in trajectories with a Kalman‑filter based interpolator, enabling robust handling of missed detections and noisy frames.

The dataset consists of 309 videos of the human osteosarcoma cell line U‑2 OS, each 117 hours long (234 frames captured every 30 minutes at 1024 × 1024 resolution). Three fluorescent markers produce time‑varying signals, providing a realistic testbed for transient‑signal tracking. In 118 videos, the chemotherapeutic agent cisplatin is added at three concentrations (high, medium, low), while 36 videos serve as controls. Manual annotations for cell positions, mitotic events, apoptosis, and lineage relationships are provided for a subset (10 videos for testing, the rest for training/validation). The full annotated dataset (≈51 000 tracks, 18 000 divisions, 3 500 deaths) is publicly released.

Performance is evaluated with standard MOT metrics (MOTA, IDF1) and event‑specific accuracy. TransientTrack outperforms state‑of‑the‑art trackers such as ByteTrack, DeepSORT, and Tracktor, achieving a 6–8 % higher MOTA and maintaining high ID consistency despite frequent cell divisions that dramatically change object size and shape. Apoptosis detection reaches >92 % accuracy, a capability largely absent in prior methods. The two‑stage matching combined with embedding similarity thresholds (τ_sim) proves especially effective at linking cells across division events and handling low‑signal frames.

Applying the method to the entire dataset yields over 28 000 cell trajectories with lineage trees and event annotations. Quantitative analysis reveals dose‑dependent reductions in proliferation rates, delayed onset of apoptosis, and characteristic lineage signatures (e.g., increased death probability after multiple successive divisions). These findings demonstrate that TransientTrack can capture both global population dynamics and single‑cell fate decisions, providing a powerful tool for high‑throughput drug‑response studies.

Key contributions include: (i) a lightweight, end‑to‑end trainable pipeline that jointly detects, classifies (alive vs. dead), and tracks cells using only detection embeddings; (ii) a novel integration of Transformer‑based embeddings, two‑stage high/low confidence association, and Kalman‑filter interpolation for robust trajectory reconstruction; (iii) the release of a large, richly annotated multi‑channel fluorescence dataset; and (iv) a demonstration of how automated, lineage‑aware tracking can elucidate chemotherapy effects at single‑cell resolution. By making code and data publicly available, the authors facilitate reproducibility and encourage further development in the emerging field of transient‑signal cell tracking.