Uploaded March 2024 | Updated September 2026, 52 minutes ago
Voyage AI is the newest giant in the embedding, reranking, and search model game! I am SUPER excited to publish our latest Weaviate podcast with Tengyu Ma, Co-Founder of Voyage AI and Assistant Professor at Stanford University!
We began the podcast with a deep dive into everything embedding model training and contrastive learning theory. Tengyu delivered a masterclass in everything from scaling laws to multi-vector representations, neural architectures, representation collapse, data augmentation, semantic similarity, and more! I am beyond impressed with Tengyu's extensive knowledge and explanations of all these topics.
The next chapter dives into a case study Voyage AI did fine-tuning an embedding model for the LangChain documentation. This is an absolutely fascinating example of the role of continual fine-tuning with very new concepts (for example, very few people were talking about chaining together LLM calls 2 years ago), as well as the data efficiency advances in fine-tuning.
We concluded by discussing ML systems challenges in serving an embeddings API. Particularly the challenge of detecting if a request is for batch or query inference and the optimizations that go into either say ~100ms latency for a query embedding or maximizing throughput for batch embeddings.
Chapters
0:00 Welcome Tengyu!
0:40 Founding Voyage AI
4:30 Contrastive Learning Theory
7:35 Data Augmentation
13:55 Dataset Building for Embedding Models
19:10 Clustering and Diversity in Representation Learning
24:00 Chunking and Multi-Discourse Representations
25:53 Vector Distance Functions
28:56 Multi-Vector Search
34:15 Architecture Search for Embedding Models
40:35 Scaling Laws for Embedding Models
43:18 Case Study with LangChain Documentation
49:25 Zero-Shot versus Fine-Tuning Debate (again)
54:53 Building an Embeddings API Product
1:00:15 Directions for the Future!
Learn more about Voyage AI: voyageai.com
Here is the documentation on the module: weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-voyageai
Here is a Weaviate recipe showing how to configure Voyage in a Weaviate schema and test it! - github.com/weaviate/recipes/blob/main/search/similarity-search/similarity_search_voyageai.ipynb
Voyage AI is the newest giant in the embedding, reranking, and search model game! I am SUPER excited to publish our latest Weaviate podcast with Tengyu Ma, Co-Founder of Voyage AI and Assistant Professor at Stanford University!
We began the podcast with a deep dive into everything embedding model training and contrastive learning theory. Tengyu delivered a masterclass in everything from scaling laws to multi-vector representations, neural architectures, representation collapse, data augmentation, semantic similarity, and more! I am beyond impressed with Tengyu's extensive knowledge and explanations of all these topics.
The next chapter dives into a case study Voyage AI did fine-tuning an embedding model for the LangChain documentation. This is an absolutely fascinating example of the role of continual fine-tuning with very new concepts (for example, very few people were talking about chaining together LLM calls 2 years ago), as well as the data efficiency advances in fine-tuning.
We concluded by discussing ML systems challenges in serving an embeddings API. Particularly the challenge of detecting if a request is for batch or query inference and the optimizations that go into either say ~100ms latency for a query embedding or maximizing throughput for batch embeddings.
Chapters
0:00 Welcome Tengyu!
0:40 Founding Voyage AI
4:30 Contrastive Learning Theory
7:35 Data Augmentation
13:55 Dataset Building for Embedding Models
19:10 Clustering and Diversity in Representation Learning
24:00 Chunking and Multi-Discourse Representations
25:53 Vector Distance Functions
28:56 Multi-Vector Search
34:15 Architecture Search for Embedding Models
40:35 Scaling Laws for Embedding Models
43:18 Case Study with LangChain Documentation
49:25 Zero-Shot versus Fine-Tuning Debate (again)
54:53 Building an Embeddings API Product
1:00:15 Directions for the Future!
Learn more about Voyage AI: voyageai.com
Here is the documentation on the module: weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-voyageai
Here is a Weaviate recipe showing how to configure Voyage in a Weaviate schema and test it! - github.com/weaviate/recipes/blob/main/search/similarity-search/similarity_search_voyageai.ipynb
