Uploaded December 2025 | Updated September 2026, 2 weeks ago
cppcon.org
---
Building a High-Performance Binary Serialization Format with In-Place Modification - Hamish Morrison - CppCon 2025
---
There are a huge number of binary serialization formats out there: protobufs, Thrift, Avro, MessagePack, flatbuffers, BSON, CBOR, and many more. How do they differ? Which one should you use? How fast are they? Codegen vs non-codegen? Schema-based or schemaless? Zero-copy or not?
Within our systems we use many different serialization formats for encoding market data. These formats range from the simple, to more complex tagged formats with compression dictionaries. We recently needed an efficient schema-less data format with support for arrays and maps, and -- critically -- the ability to mutate the data quickly in-place, without re-encoding the whole buffer every time. Essentially, a compact, binary version of JSON, with in-place modification.
This talk walks through the design and implementation of this binary serialization format. We'll cover:
- Creating a simple binary serialization using tags and values
- Implementing arrays using offset tables and indirection
- Using SIMD for fast searches in offset tables
- Building more complex data structures in contiguous buffers, in order to implement maps
- Minimizing copies during in-place updates
- Benchmarking against other serialization formats and standard library containers
- Exploring the trade-offs of our format
Throughout the talk we will be developing an ergonomic C++ interface for the serialization format, allowing users to interact with the encoded data similarly to how they would interact with ordinary vectors and maps.
While the focus is on serialization, the talk should be interesting to anyone who is interested in data structures and high-performance computing.
---
Slides: github.com/CppCon/CppCon2025/blob/main/Presentations/Building_a_High-Performance_Binary_Serialization_Format_with_In-Place_Modification.pdf
Work at Hudson River Trading (HRT): tinyurl.com/safxfctf
---
Hamish Morrison
Hamish Morrison is a software engineer at Bloomberg where he works on high performance market data processing systems.
---
CppCon is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. You will enjoy inspirational talks and a friendly atmosphere designed to help attendees learn from each other, meet interesting people, and generally have a stimulating experience. Taking place this year in Aurora, Colorado, near the Denver airport, and including multiple diverse tracks, the conference will appeal to anyone from C++ novices to experts.
Annual CppCon Conference - cppcon.org
linkedin.com/company/cppcon
https://x.com/cppcon
facebook.com/CppConference
reddit.com/r/cppcon
https://mastodon.social/@CppCon
---
Videos Filmed & Edited by Bash Films: BashFilms.com
YouTube Channel Managed by Digital Medium Ltd: events.digital-medium.co.uk
---
#cpp #cplusplus #programming #coding #softwareengineering
cppcon.org
---
Building a High-Performance Binary Serialization Format with In-Place Modification - Hamish Morrison - CppCon 2025
---
There are a huge number of binary serialization formats out there: protobufs, Thrift, Avro, MessagePack, flatbuffers, BSON, CBOR, and many more. How do they differ? Which one should you use? How fast are they? Codegen vs non-codegen? Schema-based or schemaless? Zero-copy or not?
Within our systems we use many different serialization formats for encoding market data. These formats range from the simple, to more complex tagged formats with compression dictionaries. We recently needed an efficient schema-less data format with support for arrays and maps, and -- critically -- the ability to mutate the data quickly in-place, without re-encoding the whole buffer every time. Essentially, a compact, binary version of JSON, with in-place modification.
This talk walks through the design and implementation of this binary serialization format. We'll cover:
- Creating a simple binary serialization using tags and values
- Implementing arrays using offset tables and indirection
- Using SIMD for fast searches in offset tables
- Building more complex data structures in contiguous buffers, in order to implement maps
- Minimizing copies during in-place updates
- Benchmarking against other serialization formats and standard library containers
- Exploring the trade-offs of our format
Throughout the talk we will be developing an ergonomic C++ interface for the serialization format, allowing users to interact with the encoded data similarly to how they would interact with ordinary vectors and maps.
While the focus is on serialization, the talk should be interesting to anyone who is interested in data structures and high-performance computing.
---
Slides: github.com/CppCon/CppCon2025/blob/main/Presentations/Building_a_High-Performance_Binary_Serialization_Format_with_In-Place_Modification.pdf
Work at Hudson River Trading (HRT): tinyurl.com/safxfctf
---
Hamish Morrison
Hamish Morrison is a software engineer at Bloomberg where he works on high performance market data processing systems.
---
CppCon is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. You will enjoy inspirational talks and a friendly atmosphere designed to help attendees learn from each other, meet interesting people, and generally have a stimulating experience. Taking place this year in Aurora, Colorado, near the Denver airport, and including multiple diverse tracks, the conference will appeal to anyone from C++ novices to experts.
Annual CppCon Conference - cppcon.org
linkedin.com/company/cppcon
https://x.com/cppcon
facebook.com/CppConference
reddit.com/r/cppcon
https://mastodon.social/@CppCon
---
Videos Filmed & Edited by Bash Films: BashFilms.com
YouTube Channel Managed by Digital Medium Ltd: events.digital-medium.co.uk
---
#cpp #cplusplus #programming #coding #softwareengineering
