Uploaded November 2026 | Updated November 2026, 16 hours ago
ReproducibleJobs.jl enables practical and reproducible workflows in SingleCellProjections.jl by Rasmus Henningsson
PreTalx: pretalx.com/juliacon-2026/talk/8Z8UFG
Reproducibility in scientific analyses is often hampered by insufficient tooling. Large data and slow computations force users to save partial results to disk, and the actual steps to reproduce the entire chain from raw data to end results are lost. Here we present ReproducibleJobs.jl - a computational framework that enables natural workflows and fast turnaround, while still achieving reproducibility and show how it works in practice for single cell expression data in SingleCellProjections.jl.
In this talk, we will describe how ReproducibleJobs.jl is structured, and show how it is used in SingleCellProjections.jl to achieve reproducible and practical workflows for large single cell expression data.
ReproducibleJobs.jl is a framework for reproducible analyses of scientific data, that is based on the following ideas:
* The burden of reproducibility should be moved from the user to the packages they use for analysis, when possible.
* Memoization/caching is a good strategy because while the raw data can be large, the computed results are in essence much smaller.
* It is possible to create succinct specifications of how to perform analyses.
Results in ReproducibleJobs.jl are lazy. Consider a simple but standard single cell workflow that look something like this:
```
counts = load_counts(["paths/to/large/files"])
transformed = sctransform(counts)
normalized = normalize_matrix(transformed)
reduced = pca(normalized; nsv=100)
```
To actually retrieve the result of the PCA (Principal Component Analysis) computation, the user then needs to call `fetch!(reduced)`. This is what enables ReproducibleJobs.jl to work under the hood. The lazy result is in fact a *specification*, a recipe, of what to compute. And if the computation was already memoized (cached), even in an earlier Julia session, ReproducibleJobs.jl can load the result from disk directly, without accessing any data for the earlier analysis steps. Importantly, several steps are taken to standardize the specifications, such that only changes that actually affect the results cause recomputations.
The talk also describe some of the technical challenges and solutions relating to:
* Specification design
* Specification metaprogramming/preprocessing - going from "intent" to "implementation"
* Hashing and caching
* Canonical representations of specifications and why they are important
* Strategies for handling high-level data types (such as tables, DataMatrices (matrix + variable & observation annotations)) in specifications
* Stability across Julia sessions and package versions
* How specification metaprogramming enables projections of one dataset onto another in SingleCellProjections.jl
**SingleCellProjections.jl** is a package for analyzing Single Cell expression data in Julia, with support for loading, transforming, normalizing, filtering, doing dimension reduction, performing statistical tests and more. Internally, SingleCellProjections.jl uses matrix expressions built from sparse and low rank matrices, thus avoiding speed and memory problems that competing (R/python) packages face when using large dense matrices.
ReproducibleJobs.jl enables practical and reproducible workflows in SingleCellProjections.jl by Rasmus Henningsson
PreTalx: pretalx.com/juliacon-2026/talk/8Z8UFG
Reproducibility in scientific analyses is often hampered by insufficient tooling. Large data and slow computations force users to save partial results to disk, and the actual steps to reproduce the entire chain from raw data to end results are lost. Here we present ReproducibleJobs.jl - a computational framework that enables natural workflows and fast turnaround, while still achieving reproducibility and show how it works in practice for single cell expression data in SingleCellProjections.jl.
In this talk, we will describe how ReproducibleJobs.jl is structured, and show how it is used in SingleCellProjections.jl to achieve reproducible and practical workflows for large single cell expression data.
ReproducibleJobs.jl is a framework for reproducible analyses of scientific data, that is based on the following ideas:
* The burden of reproducibility should be moved from the user to the packages they use for analysis, when possible.
* Memoization/caching is a good strategy because while the raw data can be large, the computed results are in essence much smaller.
* It is possible to create succinct specifications of how to perform analyses.
Results in ReproducibleJobs.jl are lazy. Consider a simple but standard single cell workflow that look something like this:
```
counts = load_counts(["paths/to/large/files"])
transformed = sctransform(counts)
normalized = normalize_matrix(transformed)
reduced = pca(normalized; nsv=100)
```
To actually retrieve the result of the PCA (Principal Component Analysis) computation, the user then needs to call `fetch!(reduced)`. This is what enables ReproducibleJobs.jl to work under the hood. The lazy result is in fact a *specification*, a recipe, of what to compute. And if the computation was already memoized (cached), even in an earlier Julia session, ReproducibleJobs.jl can load the result from disk directly, without accessing any data for the earlier analysis steps. Importantly, several steps are taken to standardize the specifications, such that only changes that actually affect the results cause recomputations.
The talk also describe some of the technical challenges and solutions relating to:
* Specification design
* Specification metaprogramming/preprocessing - going from "intent" to "implementation"
* Hashing and caching
* Canonical representations of specifications and why they are important
* Strategies for handling high-level data types (such as tables, DataMatrices (matrix + variable & observation annotations)) in specifications
* Stability across Julia sessions and package versions
* How specification metaprogramming enables projections of one dataset onto another in SingleCellProjections.jl
**SingleCellProjections.jl** is a package for analyzing Single Cell expression data in Julia, with support for loading, transforming, normalizing, filtering, doing dimension reduction, performing statistical tests and more. Internally, SingleCellProjections.jl uses matrix expressions built from sparse and low rank matrices, thus avoiding speed and memory problems that competing (R/python) packages face when using large dense matrices.









.
For structured systems with many variables, such as minimal problems in computer vision, the resulting Vandermonde-like interpolation matrices become prohibitively large, leading to expensive nullspace computations and numerical instability. We address this by exploiting equivariance of minimal problems with respect to matrix Lie group actions. The interpolation space of bounded-degree polynomials decomposes into isotypic components as a representation of the Lie symmetry group, allowing a substantial reduction of the problem size. This representation-theoretic decomposition is implemented in the Julia package [DecomposingGroupRepresentations.jl](https://github.com/MultivariatePolynomialSystems/DecomposingGroupRepresentations.jl). Together, these tools provide a scalable Julia-based framework that integrates monodromy and computational representation theory to recover closed-form symmetries of polynomial systems. Using monodromy and representation theory to recover symmetries of polynomial systems | Korotynskiy](https://i.ytimg.com/vi/sQnEwwvZYoA/mqdefault.jpg)
. This work provides the framework for identifying optimal algorithms for refinement of protein structures.
Protein structures are determined experimentally by X‑ray crystallography, electron microscopy, or NMR, or they are predicted with AlphaFold or RoseTTAFold [1,2]. Either way, the resulting protein models often contain missing atoms, distorted bond lengths, or unrealistic side chain orientations generating severe steric clashes and steep energy gradients. Energy minimization is thus essential to achieve physically realistic conformations. While classical minimizers (e.g., conjugate gradient, quasi-Newton) are standard, we demonstrate how machine learning (ML) optimizers can be effectively adapted to this domain.
We introduce a novel mini-batching strategy for ML-based minimization (e.g., SGD, ADAM), implemented in BiochemicalAlgorithms.jl, our library for molecular analysis and simulation [3]. The library provides molecular mechanic functionalities to evaluate energy gradients and corresponding forces, which are essential for energy minimization. Our mini-batching strategy partitions the force‑field contributions by atom‑pair groups, allowing the energy and gradient calculations to be performed on small batches corresponding to random selected portions of the molecule.
Furthermore, a major contribution of this work is a systematic assessment of different optimization algorithms by leveraging the unified Optimization.jl ecosystem—an interface that connects to more than 25 optimization libraries [4].
Our approach supports the possibility of interchanging solvers including SGD and ADAM or alternative ML solvers for energy minimization of proteins without custom code per algorithm.
Using a representative set of proteins from each of the five top‑level SCOP classes, we compared runtime and convergence precision across the solvers. The results highlight which algorithms deliver the best trade‑off between speed and final energy for different structural families.
Overall, the talk will demonstrate how the proposed mini‑batching scheme opens a path towards ML‑augmented protein‑structure refinement and how a single, extensible optimization interface can streamline the evaluation of diverse minimization strategies for protein structure relaxation.
[1] J. Jumper et al., “Highly accurate protein structure prediction with AlphaFold,” Nature, vol. 596, pp. 583‑589, 2021, doi: 10.1038/s41586‑021‑03819‑2.
[2] M. Baek et al., “Accurate prediction of protein structures and interactions using a three‑track neural network,” Science, vol. 373, pp. 871‑876, 2021, doi: 10.1126/science.abj8754.
[3] J. Leclaire et al., “Structure‑based bioinformatics with BiochemicalAlgorithms.jl,” in Proceedings of the JuliaCon Conferences, vol. 7, no. 78, p. 188, 2025, doi: 10.21105/jcon.00188.
[4] V. K. Dixit and C. Rackauckas, “Optimization.jl: A unified optimization package (version v3.12.1),” Zenodo, Mar. 2024. [Online]. Available: https://doi.org/10.5281/zenodo.7738525.
[5] N. K. Fox, S. E. Brenner, and J. M. Chandonia, “SCOPe: Structural classification of proteins—extended, integrating SCOP and ASTRAL data and classification of new structures,” Nucleic Acids Res., vol. 42, no. D1, pp. D304‑D309, 2014, doi: 10.1093/nar/gkt1240. Effects of stochasticity on molecular minimization | Leclaire | JuliaCon Global 2026](https://i.ytimg.com/vi/sbtZq2jgngw/mqdefault.jpg)