Meta Open Source
Building your multiplayer VR experience : Platform SDK and Unity SharedSpaces Sample
updated
Why it matters: Concrete production accounts for 8% of all global CO2 emissions and is a major contributor to the embodied carbon emissions in data center construction. Meta’s aim is to scale the use of sustainable concrete in data centers and advance its goal of net zero emissions in 2030.
Learn more here: engineering.fb.com/2025/07/16/data-center-engineering/ai-make-lower-carbon-faster-curing-concrete
Presenters include Constantin Patsch, Boyu Han, Wei-Jin Huang, Michele Mazzamuto, Maia Stiber, and Sean Andrist.
HoloAssist website: holoassist.github.io/#HoloAssist
- EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Vision arxiv.org/abs/2409.02224: 06:15
- Estimating Body and Hand Motion in an Ego-sensed World arxiv.org/abs/2410.03665: 14:05
- Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory arxiv.org/abs/2411.16934: 19:30
Arsha gives an overview on the state of foundation models, presents challenges, and proposes future directions for the field.
- Julia Romero presents Keystep Recognition using Graph Neural Networks: 11:30
- Lorenzo Mur-Labadia presents O-MaMa Learning Object Mask Matching between Egocentric and Exocentric Views: 16:01
- Luigi Seminara presents Ego-Exo4D Procedure Understanding Challenge 2025: 20:32
- Md Mohaiminul Islam presents BIMBA: Selective-Scan Compression for Long Range Video Question Answering : 26:15
- Yisen Feng presents OSGNet for Video Localization: 33:55
- AssemblyHands Towards Egocentric Activity Understanding via 3D Hand Pose Estimation arxiv.org/abs/2304.12301: 3:55
- Ego-4D Goal-Step: Toward Hierarchical Understanding of Procedural Activities neurips.cc/virtual/2023/poster/73705: 7:14
- EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exocentric View of Procedural Activities in Real World arxiv.org/abs/2403.16182: 11:20
- HierVL: Learning Hierarchical Video-Language Embeddings arxiv.org/abs/2301.02311: 13:40
- It’s Just Another Day: Unique Video Captioning by Discriminative Prompting arxiv.org/abs/2410.11702: 15:12
- LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning arxiv.org/abs/2312.03849: 17:15
- Video Recap: Recursive Captioning of Hour Long Videos arxiv.org/abs/2402.13250: 19:49
Learn more about Project Aria: projectaria.com
Learn more about Aria Gen 2: ai.meta.com/blog/aria-gen-2-research-glasses-under-the-hood-reality-labs
Learn more about Reading in the Wild: projectaria.com/news/introducing-reading-recognition-in-the-wild
Dr. Tang discusses how multimodal foundation models will help us predict human behaviors for downstream tasks like intelligent embodied devices or human-robot interaction.
Timestamps:
- Intro: 00:00
- Jiangkang Yang presents EgoLife: 00:19
- Jinglei Zhang presents HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos: 06:35
- Hanzi Chen presents VidBod: Learning Generalizable 3D Actions from in-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation: 12:05
- Satoshi Tsutsui presents Discovering Hidden Visual Concepts Beyond LInguistic Input in Infant Learning: 19:05
1st Place: Angese Taluzzi presents From Pixels to Graphs: Using Scene and Knowledge Graphs for HD-EPIC VQA Challenge
2nd Place: Optimizing Multimodal LLMs for Egocentric Video Understanding: A solution for the HD-EPIC VQA Challenge
Pyrefly is a static type checker that analyzes Python code to ensure type consistency and help you catch errors throughout your codebase before your code runs.
Learn more in an accompanying blog post: engineering.fb.com/2025/05/15/developer-tools/introducing-pyrefly-a-new-type-checker-and-ide-experience-for-python
----- Resources ------
Pyrefly Website: pyrefly.org
Pyrefly Github: github.com/facebook/pyrefly
Pyrefly Discord: discord.com/invite/Cf7mFQtW7W
Pyrefly VSCode Extension: marketplace.visualstudio.com/items?itemName=meta.pyrefly
------ Social Accounts ------
Meta Open Source X Account: http://twitter.com/metaOpenSource
Meta Open Source Threads Account: threads.com/@metaopensource
Meta Open Source Facebook Page: http://www.facebook.com/metaOpenSource
Meta Open Source Website: http://opensource.facebook.com
Meta Open Source LinkedIn Page: linkedin.com/showcase/78457036
In collaboration with Lighthouse SF, Envision is exploring the integration of its Ally AI assistant and spatial audio using Aria Gen 2 glasses to improve navigation for individuals who are blind and have low vision.
The team leverages the on-device SLAM capabilities of Aria Gen 2, along with spatial audio features via onboard speakers to give a user spatialized audio cues to guide them to products they need in a grocery store. This work is still in an exploratory phase, but exemplifies how partners can leverage the new capabilities of Aria Gen 2 glasses for prototyping AI experiences.
Read the blog post to learn more about Aria Gen 2: meta.com/blog/project-aria-gen-2-next-generation-egocentric-research-glasses-reality-labs-ai-robotics
Learn more about Envision: letsenvision.com
Learn more about LightHouse SF: lighthouse-sf.org
Learn more about Ally AI: ally.me
Today, we're excited to announce the next step in this journey: the introduction of Aria Gen 2 glasses. This next generation of hardware will unlock new possibilities across a wide range of research areas including machine perception, egocentric and contextual AI, and robotics.
Read the blog post to learn more about Aria Gen 2: meta.com/blog/project-aria-gen-2-next-generation-egocentric-research-glasses-reality-labs-ai-robotics
Learn more about Project Aria: projectaria.com
Learn more about the Project Aria datasets: projectaria.com/datasets
In collaboration with Lighthouse SF, Envision is exploring the integration of its Ally AI assistant and spatial audio using Aria Gen 2 glasses to improve navigation for individuals who are blind and have low vision.
The team leverages the on-device SLAM capabilities of Aria Gen 2, along with spatial audio features via onboard speakers to give a user spatialized audio cues to guide them to products they need in a grocery store. This work is still in an exploratory phase, but exemplifies how partners can leverage the new capabilities of Aria Gen 2 glasses for prototyping AI experiences.
Read the blog post to learn more about Aria Gen 2: meta.com/blog/project-aria-gen-2-next-generation-egocentric-research-glasses-reality-labs-ai-robotics
Learn more about Envision: letsenvision.com
Learn more about LightHouse SF: lighthouse-sf.org
Learn more about Ally AI: ally.me
Read the blog post to learn more: ai.meta.com/blog/egomimic-project-aria-georgia-tech-ego4d-robotics-embodied-ai
Learn more about the Aria Research Kit at projectaria.com.
Timestamps:
- Intro: 00:00
- How Timeplus leverages RocksDB to do billions of keys streaming join and aggregation (Ken Chen - Timeplus): 03:02
- Decoupled-RocksDB and Its Automatic Tuning with LLMs (Zhichao Cao - Arizona State): 23:47
- Sally - A RocksDB Storage Management Plane (Dhanabal Ekambaram, Sushil Patil - Meta): 45:43
- JunoDB - PayPal's Distributed Key-value Store (Yaping Shi - Paypal): 01:18:09
- Multi-Column-Family Iterator in RocksDB (Jay Huh - Meta): 01:44:26
- Multi-dimensional filtering in RocksDB (Peter Dillinger - Meta): 01:54:32
- RocksDB stress/crash testing (Hui Xiao - Meta): 2:14:24
Project Links
Website: http://rocksdb.org
Github: github.com/facebook/rocksdb
Social Accounts
RocksDB Facebook: facebook.com/rocksdb
RocksDB X: https://x.com/rocksdb
Meta Open Source X: http://x.com/MetaOpenSource
Meta Open Source Facebook Page: http://www.facebook.com/MetaOpenSource
Meta Open Source Website: http://opensource.facebook.com
Meta Open Source LinkedIn: linkedin.com/showcase/meta-open-source
Meta Open Source Threads: threads.net/@metaopensource
In addition to algorithms, the workshop promotes multiple recent datasets and associated challenges to accelerate research in this field.
Learn more at: egomotion-workshop.github.io
Organizers: Lingni Ma (Meta), Richard Newcombe (Meta), Yuting Ye (Meta), Edward Miller (Meta), Ziwei Liu (NTU), Yifeng Jiang (Stanford), Vladimir Guzov (University of Tübingen)
Chapters (timecodes to be refined once uploaded)
00:00:00 Richard Newcombe (Meta) - Opening
00:13:47 Siyu Tang (Assistant Professor, ETH Zürich) - Egocentric 3D Human Estimation and Synthesis
00:35:17 Christian Theobalt (Professor, MPI-IN) - Egocentric Human Motion Capture with Head-mounted Cameras
01:04:51 Hanbyul Joo (Assistant Professor, Seoul National University) - Towards Capturing Everyday Movements to Scale Up and Enrich Human Motion Data
01:31:01 Lingni Ma (Research Scientist, Meta) Yuting Ye (Research Scientist, Meta) - Nymeria: Understanding Human Motion from Egocentric Data
02:02:12 C. Karen Liu (Professor, Stanford University), Jiaman Li (Ph.D Student, Stanford University) - Egocentric Perception for Human Motion Synthesis
02:27:14 Gerard Pons-Moll (Professor, University of Tübingen) - How and Why Should We Learn Avatars with Sensorimotor Capabilities?
02:55:24 Manuel Kaufmann (Postdoc, ETH Zurich) - Motion Capture with Electromagnetic Body-worn Sensors
03:09:18 Fangzhou Hong (Ph.D Student, NTU) - EgoLM: Multi-Modal Language Model of Egocentric Motions
03:22:02 Xinyu Yi (Ph.D Student, Tshinghua University) - Egocentric Motion Capture with Sparse Inertial/Visual Sensors
#CVPR2024 #ARIA
This workshop is a great opportunity to learn about the latest developments in egocentric perception and how Project Aria is helping to accelerate research in this field.
Learn more at: http://www.ProjectAria.com
Organizers: James Fort (Meta), Richard Newcombe (Meta), Edward Miller (Meta), Kristen Grauman (Meta)
Chapters
00:00:00 James Fort (Meta) - An Update to the Aria Research Kit and Open Science Initiatives
00:25:22 Xiaqing Pan (Meta) - Aria Live Demo
00:41:09 Yang Lou (Meta) - Aria Training and Evaluation Kit (ATEK)
00:54:52 Dr. Zhao Dong (Meta) - Digital Twin Catalog dataset
01:27:30 Dr. Prithviraj Banerjee (Meta) - HOT3D dataset for hand-object interaction understanding
01:42:02 Dr. Lingni Ma (Meta) - Nymeria dataset for egocentric human motion understanding
01:57:16 Dr. Julian Straub (Meta) - EFM3D dataset for 3D egocentric foundation models
02:10:53 Dr. Antonino Furnari (University of Catania) - Ego-Exo4D Overview
02:34:25 Prof. Dima Damien (University of Bristol) - Egocentric Recording in the Home
02:50:39 Dr. Francesco Ragusa (University of Catania) - EFM3D dataset for 3D egocentric foundation models
03:01:25 Dr. Paul-Edouard Sarlin (ETH Zurich) - LaMar Aria
3:26:16: James Fort (Meta) - Concluding Remarks
#ECCV2024 #ARIA
By working with Project Aria, CMU were able to reduce their dependence on bluetooth beacons, and bring new environments online at a faster rate, enabling NavCog to be available to more people who need it.
Learn more about Project Aria and the Aria Research Kit at projectaria.com
Organizers: Tomas Hodan (Meta), Martin Sundermeyer (Google), Van Nguyen Nguyen (ParisTech), Stephen Tyree (NVIDIA), Andrew Guo (University of Toronto), Médéric Fourmy (CTU in Prague), Anas Gouda (TU Dortmund), Taeyeop Lee (KAIST), Sungphill Moon (NAVER LABS), Hyeontae Son (NAVER LABS), Lukas Ranftl (MVTec), Jonathan Tremblay (NVIDIA), Yann Labbé (Meta), Eric Brachmann (Niantic), Bertram Drost (MVTec), Sindi Shkodrani (Meta), Carsten Steger (TUM, MVTec), Vincent Lepetit (ParisTech), Carsten Rother (Heidelberg University), Stan Birchfield (NVIDIA), Jiri Matas (CTU in Prague)
Chapters:
00:00:00 Introduction: Tomas Hodan (Meta)
00:08:20 Talk 1: Detector-free feature matching for pose estimation - Xiaowei Zhou (Zhejiang University)
00:36:57 Talk 2: Relaxing pose estimation: Unseen objects and visual challenges - Benjamin Busam (TUM)
01:05:17 Talk 3: Practical robotic scene awareness - Stan Birchfield (NVIDIA)
01:34:14 Talk 4: Towards Contextual AI: Tracking objects and our interactions with them - Jakob Engel (Meta)
02:02:07 BOP 2024: Introduction - Tomas Hodan (Meta)
02:17:00 BOP 2024: HANDAL dataset - Stephen Tyree (NVIDIA)
02:19:31 BOP 2024: HOPEv2 dataset - Stephen Tyree (NVIDIA)
02:23:10 BOP 2024: HOT3D dataset - Prithviraj Banerjee, Sindi Shkodrani (Meta)
02:32:08 BOP 2024: Estimating object visibility on HOT3D - Taeyeop Lee (KAIST)
02:36:21 BOP 2024: Challenge tracks - Van Nguyen Nguyen (ParisTech)
02:43:40 BOP 2024: Results and early-bird awards - Martin Sundermeyer (Google), Van Nguyen Nguyen (ParisTech)
02:54:10 BOP 2024: GenFlow results on BOP-H3 - Sungphill Moon (NAVER LABS)
02:58:59 BOP 2024: BOP-Robotics - Martin Sundermeyer (Google)
#ECCV2024 #BOP
Learn more about Project Aria and the Aria Research Kit at projectaria.com
Learn more about Project Aria and the Aria Research Kit at projectaria.com
Join Project Aria's academic research community, and apply for an Aria Research Kit. Learn more about Project Aria and the Aria Research Kit at projectaria.com
The time to apply for an Aria Research Kit is now, learn more about Project Aria and the Aria Research Kit at projectaria.com
Below is the list of talks:
- Our experience with using RocksDB in TiKV by PingCap (Sunny Bain)
- Key Value Unification at Pinterest (Rajath Prasad)
- RocksDB Vision Expansion by Meta (Danny Chen)
- An efficient RocksDB Checkpoint Diff Mechanism by Cloudera (Prashant Pogde)
- Tuning Optimizations on RocksDB Scans by Meta (Akanksha Mahajan)
- Dremel - Adaptive Configuration Tuning of RocksDB KV-Store by University of Washington (Frank Zhao)
Project Links
Website: http://rocksdb.org
Github: github.com/facebook/rocksdb
Social Accounts
RocksDB Facebook: facebook.com/rocksdb
RocksDB Twitter: twitter.com/rocksdb
Meta Open Source Twitter: http://twitter.com/MetaOpenSource
Meta Open Source Facebook Page: http://www.facebook.com/MetaOpenSource
Meta Open Source website: http://opensource.facebook.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Recorded June 2023 during the CVPR 2023 Tutorial entitled 'Hands-on egocentric research with Project Aria, from Meta'
Learn more at ProjectAria.com
Learn more about Project Aria at www.projectaria.com
Listen to the full podcast here: thediffpodcast.com/docs/episode-17
------ Resources ------
The Diff Podcast: thediffpodcast.com/docs/episodes
------ Social Accounts ------
Meta Open Source Twitter: http://twitter.com/metaOpenSource
Meta Open Source Facebook Page: http://www.facebook.com/metaOpenSource
Meta Open Source Website: http://opensource.facebook.com
Meta Open Source LinkedIn Page: linkedin.com/showcase/78457036
Listen to the full podcast here: thediffpodcast.com/docs/episode-17
------ Resources ------
The Diff Podcast: thediffpodcast.com/docs/episodes
------ Social Accounts ------
Meta Open Source Twitter: http://twitter.com/metaOpenSource
Meta Open Source Facebook Page: http://www.facebook.com/metaOpenSource
Meta Open Source Website: http://opensource.facebook.com
Meta Open Source LinkedIn Page: linkedin.com/showcase/78457036


