Bridging the Gap between Dagger.jl and HPC Interconnects | Guimarães, Tomé | JuliaCon Global 2026 @TheJuliaLanguage
Bridging the Gap between Dagger.jl and HPC Interconnects | Guimarães, Tomé | JuliaCon Global 2026  @TheJuliaLanguage
Uploaded November 2026 | Updated November 2026, 4 days ago
Bridging the Gap between Dagger.jl and HPC Interconnects by Yan Guimarães, Felipe Tomé

PreTalx: pretalx.com/juliacon-2026/talk/AUBUSN

While Julia’s `Dagger.jl` provides a productive framework for task-based parallelism using Directed Acyclic Graphs (DAGs), its default reliance on TCP-based `Distributed.jl` limits performance on low-latency HPC interconnects. To bridge this gap, we developed `MPIAcceleration`, a strategic extension that replaces standard transport with an MPI-aware backend. By leveraging `MPI.jl` and non-blocking communication, we enable Dagger to use specialized hardware such as InfiniBand and Slingshot while maintaining a simple, high-level API.

Dagger's MPIAcceleration works seamlessly with the current scheduler, allowing task graphs to be executed across MPI ranks with minimal modifications. It only requires a single line of code: `Dagger.accelerate!(:mpi)`.

When this feature is enabled, each MPI rank is integrated into Dagger's Processor/Memory Space model, which ensures that tasks are executed close to where their data resides. This rank-aware placement helps to minimize communication overhead. Additionally, remote data transfers happen transparently, providing handles on the appropriate ranks.
Bridging the Gap between Dagger.jl and HPC Interconnects | Guimarães, Tomé | JuliaCon Global 2026TenSolver.jl: A tensor network based QUBO solver with JuMP integration | Iago Leal de FreitasHierarchical Precision and Recursion for Accelerating Symmetric Linear Solves on MXUs | CarricaStaged programming in pharmacometrics | Noack | JuliaCon Global 2026FerriteCon 2024 Chris Rackauckas: Extending the Julia SciML Ecosystem to a Backbone for PDEsEfficient Constrained Optimization using ConicSolve.jl | Leong | JuliaCon Global 2025TrixiParticles.jl on GPUs: A Deep Dive into Simulating Fluid Dynamics of Carbon Fiber FinsDeep Adaptive Experimental Design for SciML | Strouwen, Micluța-Câmpeanu | JuliaCon Global 2026Spying into Julia’s Runtime. | Krieger, Nuzhdin, Alberto Vieyra Salas | JuliaCon Global 2026Advanced Model Composition in ApplicationDrivenLearning.jl | Giovanni AmorimTrixiAtmo.jl: Advanced numerical schemes for atmospheric flows | Geihe | JuliaCon Global 2026ReproducibleJobs.jl enables practical and reproducible workflows in SingleCellProjections.jl
The Julia Programming Language |

Bridging the Gap between Dagger.jl and HPC Interconnects | Guimarães, Tomé | JuliaCon Global 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER