← ClaudeAtlas

vector-search-recall-testslisted

Use when HNSW or IVF parameters are being tuned, an embedding model is swapped, or vector search recall was never measured. Builds brute-force ground truth, measures recall@k against latency and detects embedding drift for Qdrant, Weaviate, pgvector, Pinecone, Elasticsearch k-NN and Milvus.
testland/qa-skills · ★ 0 · AI & Automation · score 72
Install: claude install-skill testland/qa-skills
# vector-search-recall-tests Per the [ANN-Benchmarks docs] and [Qdrant search docs], tests must measure recall@k against known ground truth - engine self-eval doesn't catch index corruption or parameter drift. ## When to use - Production uses vector search for retrieval (RAG, semantic search, recommendations). - HNSW / IVF / ScaNN parameter tuning - need quantitative test for the recall/latency Pareto curve. - Embedding model upgrade - verify recall doesn't drop on the existing corpus. ## Step 1 - Construct ground truth (brute-force k-NN) Ground truth = top-k closest documents under exact (brute-force) distance. Compute once for a fixed query set: ```python import numpy as np from sklearn.neighbors import NearestNeighbors def compute_ground_truth(corpus_vectors, query_vectors, k=10): nn = NearestNeighbors(n_neighbors=k, algorithm="brute", metric="cosine") nn.fit(corpus_vectors) distances, indices = nn.kneighbors(query_vectors) return indices # shape (n_queries, k) ``` Save indices alongside corpus + queries; recompute only if corpus or embedding model changes. ## Step 2 - Recall@k measurement ```python def recall_at_k(retrieved_ids, ground_truth_ids): """Both: list of doc-id arrays, length n_queries; each row top-k retrieved.""" recalls = [] for retr, gt in zip(retrieved_ids, ground_truth_ids): recalls.append(len(set(retr) & set(gt)) / len(gt)) return float(np.mean(recalls)) ``` For a given engine + parameter set, retr