Trillion-Token Pretraining | Adyen | Ray Summit 2026 @anyscale
Trillion-Token Pretraining | Adyen | Ray Summit 2026  @anyscale
Uploaded September 2026 | Updated November 2026, 1 day ago
Pretraining foundation models on tabular and sequential data presents challenges that differ fundamentally from NLP or vision.

At Ray Summit 2026, Hanna van der Vlis, AI Research Engineer at Adyen, covers the key design decisions behind a payments foundation model trained on billions of transactions and trillions of tokens: tokenization of heterogeneous features, sequence construction, masking strategies and pretraining objectives for sequential tabular data, and the architectural tradeoffs between hierarchical and language-model framings.

You'll leave with transferable techniques for adapting self-supervised pretraining to non-text sequential data.

Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo

Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale

đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Trillion-Token Pretraining | Adyen | Ray Summit 2026Running Ray in Production: Google’s Guide to Operators & Observability | Ray Summit 2025From Agents to Action: Scaling Agentic AI with Ray on Intel Xeon | Intel | Ray Summit 2026Building an AI Science Factory | Andrew Beam (Lila Sciences) | Ray Summit 2026GLM-5 RL Training with vLLM | Prime Intellect | Ray Summit 2026Accelerating vLLM with LMCache | Ray Summit 2025Ray History Server: Observability for Ephemeral K8s Clusters | KubeRay | Ray Summit 2026SQL or Python? Why It’s the Wrong Question in the AI Era | Ray Summit 2025Brendan Burns: Lessons from Building Kubernetes and the Future of AI InfrastructureScaling AI the Snowflake Way: ML Workloads on Ray | Ray Summit 2025Scaling Ray on Kubernetes: Pragmatic Strategies for Every Team | Ray Summit 2025The Rise of vLLM: Building an Open Source LLM Inference Engine
Anyscale |

Trillion-Token Pretraining | Adyen | Ray Summit 2026

SHARE TO X SHARE TO REDDIT SHARE TO FACEBOOK WALLPAPER