In this workshop, we will explore efficient methods and approaches for performing queries in a billion-dimensional vector space. In recent years, we have accumulated vast amounts of data and transformed them into meaningful embedding vectors using machine learning models.
As a result, there are now numerous use-cases where we need to find the closest match from a large set of high-dimensional vectors. However, exhaustive search is not feasible due to its high computational cost and time limitations.
To address this challenge, we will focus on techniques and practices implemented in the FAISS library. We will identify the basic indexes and provide solutions to make the billion-scale problem achievable.