vector-search-recall-testslisted
Install: claude install-skill testland/qa-skills
# vector-search-recall-tests
Per the [ANN-Benchmarks docs] and [Qdrant search docs], tests must
measure recall@k against known ground truth - engine self-eval
doesn't catch index corruption or parameter drift.
## When to use
- Production uses vector search for retrieval (RAG, semantic search,
recommendations).
- HNSW / IVF / ScaNN parameter tuning - need quantitative test for
the recall/latency Pareto curve.
- Embedding model upgrade - verify recall doesn't drop on the
existing corpus.
## Step 1 - Construct ground truth (brute-force k-NN)
Ground truth = top-k closest documents under exact (brute-force)
distance. Compute once for a fixed query set:
```python
import numpy as np
from sklearn.neighbors import NearestNeighbors
def compute_ground_truth(corpus_vectors, query_vectors, k=10):
nn = NearestNeighbors(n_neighbors=k, algorithm="brute", metric="cosine")
nn.fit(corpus_vectors)
distances, indices = nn.kneighbors(query_vectors)
return indices # shape (n_queries, k)
```
Save indices alongside corpus + queries; recompute only if corpus
or embedding model changes.
## Step 2 - Recall@k measurement
```python
def recall_at_k(retrieved_ids, ground_truth_ids):
"""Both: list of doc-id arrays, length n_queries; each row top-k retrieved."""
recalls = []
for retr, gt in zip(retrieved_ids, ground_truth_ids):
recalls.append(len(set(retr) & set(gt)) / len(gt))
return float(np.mean(recalls))
```
For a given engine + parameter set, retr