Hello! We're currently working on a benchmark of spatial TI methods across tissues, and we would love to be able to add it in Openproblems. Here is the task proposal, I'm happy to discuss it further with the OP team!
Task motivation
Spatial transcriptomics preserves each cell's physical position within a tissue section alongside its expression profile, enabling methods to predict pseudotime — a cell's position along a trajectory of differentiation, zonation, or disease progression — using this 2D spatial context. However, a growing number of spatial pseudotime prediction methods (stLearn, SpaceFlow, SpaTrack, CaSCAT, spatially-aware Monocle 3) each make different assumptions about resolution, input format, and pseudotime output, and these methods can suffer from significant challenges due to spatial batch effects in spatial transcriptomics data, which add independent sources of noise not present in dissociated single-cell data. Non-spatial TI methods were previously subject to a similar comparison problem, resolved by dynbenchmark, which benchmarked 45 methods on 110 real and 229 synthetic datasets and showed method choice should depend on dataset dimensions and trajectory topology — no equivalent standardized benchmark exists for spatial pseudotime prediction on 2D tissue data. Critically, existing spatial TI method papers are validated only against each other or qualitative marker trends, not against tissue sections where the correct trajectory is independently known. This task focuses specifically on 2D tissue datasets and benchmarks pseudotime prediction methods against tissues with well-characterized ground-truth spatial axes, providing researchers with an objective, reproducible measure of which methods actually recover known biology.
Task description
Problem: Given a spatial transcriptomics dataset from a tissue with a known ground-truth spatial gradient, infer a pseudotime trajectory per cell and evaluate how well it recovers that gradient.
Input:
- Count matrix: genes (rows) x cells (columns) with expression counts, and the spatial metadata (cell coordinates)
Output:
- A set of quantitative metric scores comparing each method's predicted pseudotime against ground truth, capturing both trajectory accuracy and preservation of relevant biological information.
Assumptions:
- Datasets are single-cell resolution, 2D spatial transcriptomics.
- Datasets contain sufficient variation in cell age/state along the trajectory to meaningfully differentiate method performance.
- Spatial coordinates are provided for every cell.
Constraints:
- Ground truth pseudotime is available, either derived from established biological knowledge (e.g., known spatial gradients, marker panels) or provided directly by simulation for synthetic datasets.
- Methods must accept a common standardized input format despite differing internal preprocessing/clustering requirements.
Proposed ground-truth in datasets
- Simulated data using
- Real biological Datasets - WIP
Initial set of methods to implement
- Spatial optimal transport: Frames trajectory inference as an optimal transport problem, computing the most efficient "mass flow" of cells between states while incorporating spatial coordinates as a cost constraint.
- Graph-based: Build a graph over cells using spatial proximity and expression similarity, then infer pseudotime by propagating or ordering cells along paths through that graph
- Embeddings: Learn a joint low-dimensional embedding that integrates expression and spatial information, from which pseudotime is derived based on position in the embedding space.
- Baseline: Classical, spatially-agnostic single-cell TI methods that infer pseudotime purely from expression similarity (diffusion maps, minimum spanning trees, or partition-based graph abstraction), included to establish whether spatially-aware methods actually outperform methods that ignore spatial information entirely.
Proposed control methods
- Positive Controls
- Simply taking the ground truth from the original dataset
- Negative Controls
Proposed Metrics
- Pseudo-time
- Spearman correlation
- Cell ordering accuracy
- Spatial Topology
- Spatial neighbour consistency
- Spatial autocorrelation
- Robustness
- Noise sensitivity
- Dropout, batch effects
- Efficiency
- Runtime, memory
- Scalability
- Usability
Hello! We're currently working on a benchmark of spatial TI methods across tissues, and we would love to be able to add it in Openproblems. Here is the task proposal, I'm happy to discuss it further with the OP team!
Task motivation
Spatial transcriptomics preserves each cell's physical position within a tissue section alongside its expression profile, enabling methods to predict pseudotime — a cell's position along a trajectory of differentiation, zonation, or disease progression — using this 2D spatial context. However, a growing number of spatial pseudotime prediction methods (stLearn, SpaceFlow, SpaTrack, CaSCAT, spatially-aware Monocle 3) each make different assumptions about resolution, input format, and pseudotime output, and these methods can suffer from significant challenges due to spatial batch effects in spatial transcriptomics data, which add independent sources of noise not present in dissociated single-cell data. Non-spatial TI methods were previously subject to a similar comparison problem, resolved by dynbenchmark, which benchmarked 45 methods on 110 real and 229 synthetic datasets and showed method choice should depend on dataset dimensions and trajectory topology — no equivalent standardized benchmark exists for spatial pseudotime prediction on 2D tissue data. Critically, existing spatial TI method papers are validated only against each other or qualitative marker trends, not against tissue sections where the correct trajectory is independently known. This task focuses specifically on 2D tissue datasets and benchmarks pseudotime prediction methods against tissues with well-characterized ground-truth spatial axes, providing researchers with an objective, reproducible measure of which methods actually recover known biology.
Task description
Problem: Given a spatial transcriptomics dataset from a tissue with a known ground-truth spatial gradient, infer a pseudotime trajectory per cell and evaluate how well it recovers that gradient.
Input:
Output:
Assumptions:
Constraints:
Proposed ground-truth in datasets
Initial set of methods to implement
Proposed control methods
Proposed Metrics