Uploaded September 2026 | Updated November 2026, 1 day ago
Pretraining foundation models on tabular and sequential data presents challenges that differ fundamentally from NLP or vision.
At Ray Summit 2026, Hanna van der Vlis, AI Research Engineer at Adyen, covers the key design decisions behind a payments foundation model trained on billions of transactions and trillions of tokens: tokenization of heterogeneous features, sequence construction, masking strategies and pretraining objectives for sequential tabular data, and the architectural tradeoffs between hierarchical and language-model framings.
You'll leave with transferable techniques for adapting self-supervised pretraining to non-text sequential data.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com
Pretraining foundation models on tabular and sequential data presents challenges that differ fundamentally from NLP or vision.
At Ray Summit 2026, Hanna van der Vlis, AI Research Engineer at Adyen, covers the key design decisions behind a payments foundation model trained on billions of transactions and trillions of tokens: tokenization of heterogeneous features, sequence construction, masking strategies and pretraining objectives for sequential tabular data, and the architectural tradeoffs between hierarchical and language-model framings.
You'll leave with transferable techniques for adapting self-supervised pretraining to non-text sequential data.
Liked this video? Check out other Ray Summit breakout session recordings: youtube.com/playlist?list=PLZNaJyYZRPdo
Subscribe to our YouTube channel to stay up-to-date on the future of AI: youtube.com/c/anyscale
đź”— Connect with us:
LinkedIn: linkedin.com/company/joinanyscale
X: https://x.com/anyscalecompute
Website: anyscale.com










