Uploaded June 2024 | Updated September 2026, 2 hours ago
Josh Engels is a Ph.D. student at MIT who has published several works advancing the state of the art in Vector Search. Josh has recently developed the Window Search Tree, a new algorithm particularly targeted for improving Filtered Vector Search. Even more particularly than that, the WST algorithm targets Filtered Search with continuous-valued filters such as "price" or "date", also known as range filters. This is a huge application for Vector Databases and it was incredible getting to pick Josh's brain on how this works and the state of Approximate Nearest Neighbor Search!
Window Search Tree: arxiv.org/abs/2402.00943
Learn more about Josh Engels here: joshengels.com/!
Chapters
0:00 Welcome Josh
0:28 What lead you to ANN research?
1:45 Types of Filters
5:00 Applications of Filtered ANN
7:44 Window Search Tree: An Overview
16:16 Batch vs. Incremental ANN
17:30 Use of Parallelism and ParlayANN
18:52 Saving Memory with Window Search Trees
22:00 Thoughts on IVF^2
27:12 Optimized Postfiltering and Super Postfiltering
37:07 ANN Benchmarking
42:22 Query and Filter Correlation
45:18 ANN Algorithm per Type of Filter
49:34 Can we merge the child nodes into 1 graph?
53:30 Massively Parallel Vector Search
56:35 Vector Set Searches
Josh Engels is a Ph.D. student at MIT who has published several works advancing the state of the art in Vector Search. Josh has recently developed the Window Search Tree, a new algorithm particularly targeted for improving Filtered Vector Search. Even more particularly than that, the WST algorithm targets Filtered Search with continuous-valued filters such as "price" or "date", also known as range filters. This is a huge application for Vector Databases and it was incredible getting to pick Josh's brain on how this works and the state of Approximate Nearest Neighbor Search!
Window Search Tree: arxiv.org/abs/2402.00943
Learn more about Josh Engels here: joshengels.com/!
Chapters
0:00 Welcome Josh
0:28 What lead you to ANN research?
1:45 Types of Filters
5:00 Applications of Filtered ANN
7:44 Window Search Tree: An Overview
16:16 Batch vs. Incremental ANN
17:30 Use of Parallelism and ParlayANN
18:52 Saving Memory with Window Search Trees
22:00 Thoughts on IVF^2
27:12 Optimized Postfiltering and Super Postfiltering
37:07 ANN Benchmarking
42:22 Query and Filter Correlation
45:18 ANN Algorithm per Type of Filter
49:34 Can we merge the child nodes into 1 graph?
53:30 Massively Parallel Vector Search
56:35 Vector Set Searches
