Uploaded December 2024 | Updated September 2026, 5 hours ago
Hey everyone! Thank you so much for watching the 110th episode of the Weaviate Podcast! Today we are diving into Snowflake’s Arctic Embedding model series and their newly released Arctic Embed 2.0 open-source model, additionally supporting multilingual text embeddings. The podcast covers the origin of Arctic Embed, Pre-training embedding models, Matryoshka Representation Learning (MRL), Fine-tuning embedding models, Synthetic Query Generation, Hard Negative Mining, and Single-Vector Embeddings Models in the cohort of Multi-Vector ColBERT, SPLADE, and Re-rankers.
Links:
Snowflake's Arctic Embed 2.0 Goes Multilingual: snowflake.com/engineering-blog/snowflake-arctic-embed-2-multilingual
Arxiv Paper: arxiv.org/abs/2412.04506
Arctic Embedding Model Series on HuggingFace: huggingface.co/collections/Snowflake/arctic-embed-661fd57d50fab5fc314e4c18
Arctic Embed M v1.5: snowflake.com/engineering-blog/arctic-embed-m-v1-5-enterprise-retrieval
Initial Launch of Arctic Embed: snowflake.com/en/blog/introducing-snowflake-arctic-embed-snowflakes-state-of-the-art-text-embedding-family-of-models
Arxiv Paper: arxiv.org/abs/2405.05374
Embedding and Clustering our Data can improve Contrastive Pretraining: arxiv.org/pdf/2407.18887
Google Gecko Embeddings Technical Report: arxiv.org/pdf/2403.20327
Introducing Weaviate Embeddings: weaviate.io/blog/introducing-weaviate-embeddings
Embeddings on Weaviate Workbench: weaviate.io/workbench/embeddings
Rescoring from Disk in Weaviate: weaviate.io/blog/pq-rescoring
Chapters
0:00 Welcome Luke, Puxuan, and Charles!
1:08 The Origin of Arctic Embed
4:45 Moving up the Stack at Weaviate
7:08 MTEB Benchmark
12:08 Pre-Training Embedding Models
19:15 Cost of Embedding Model Pre-Training
25:42 Pre-Training LLMs vs. Embedding Models
28:45 Matryoshka Representation Learning
37:50 Resolutions of Embeddings
44:50 Source Stratification
50:10 Hard Negative Mining
1:04:55 Synthetic Query Generation
1:14:45 Multilingual Text Embeddings
1:24:50 Thoughts on ColBERT, SPLADE, and Rerankers
Hey everyone! Thank you so much for watching the 110th episode of the Weaviate Podcast! Today we are diving into Snowflake’s Arctic Embedding model series and their newly released Arctic Embed 2.0 open-source model, additionally supporting multilingual text embeddings. The podcast covers the origin of Arctic Embed, Pre-training embedding models, Matryoshka Representation Learning (MRL), Fine-tuning embedding models, Synthetic Query Generation, Hard Negative Mining, and Single-Vector Embeddings Models in the cohort of Multi-Vector ColBERT, SPLADE, and Re-rankers.
Links:
Snowflake's Arctic Embed 2.0 Goes Multilingual: snowflake.com/engineering-blog/snowflake-arctic-embed-2-multilingual
Arxiv Paper: arxiv.org/abs/2412.04506
Arctic Embedding Model Series on HuggingFace: huggingface.co/collections/Snowflake/arctic-embed-661fd57d50fab5fc314e4c18
Arctic Embed M v1.5: snowflake.com/engineering-blog/arctic-embed-m-v1-5-enterprise-retrieval
Initial Launch of Arctic Embed: snowflake.com/en/blog/introducing-snowflake-arctic-embed-snowflakes-state-of-the-art-text-embedding-family-of-models
Arxiv Paper: arxiv.org/abs/2405.05374
Embedding and Clustering our Data can improve Contrastive Pretraining: arxiv.org/pdf/2407.18887
Google Gecko Embeddings Technical Report: arxiv.org/pdf/2403.20327
Introducing Weaviate Embeddings: weaviate.io/blog/introducing-weaviate-embeddings
Embeddings on Weaviate Workbench: weaviate.io/workbench/embeddings
Rescoring from Disk in Weaviate: weaviate.io/blog/pq-rescoring
Chapters
0:00 Welcome Luke, Puxuan, and Charles!
1:08 The Origin of Arctic Embed
4:45 Moving up the Stack at Weaviate
7:08 MTEB Benchmark
12:08 Pre-Training Embedding Models
19:15 Cost of Embedding Model Pre-Training
25:42 Pre-Training LLMs vs. Embedding Models
28:45 Matryoshka Representation Learning
37:50 Resolutions of Embeddings
44:50 Source Stratification
50:10 Hard Negative Mining
1:04:55 Synthetic Query Generation
1:14:45 Multilingual Text Embeddings
1:24:50 Thoughts on ColBERT, SPLADE, and Rerankers
