embedding-strategies

# Embedding Strategies Guide to selecting and optimizing embedding models for vector search applications. ## Do not use this skill when - The task is unrelated to embedding strategies - You need a different domain or tool outside this scope ## Instructions - Clarify goals, constraints, and required inputs. - Apply relevant best practices and validate outcomes. - Provide actionable steps and verification. - If detailed examples are required, open `resources/implementation-playbook.md`. ## Use this skill when - Choosing embedding models for RAG - Optimizing chunking strategies - Fine-tuning embeddings for domains - Comparing embedding model performance - Reducing embedding dimensions - Handling multilingual content ## Core Concepts ### 1. Embedding Model Comparison | Model | Dimensions | Max Tokens | Best For | |-------|------------|------------|----------| | **text-embedding-3-large** | 3072 | 8191 | High accuracy | | **text-embedding-3-small** | 1536 | 8191 | Cost-effective | | **voyage-2** | 1024 | 4000 | Code, legal | | **bge-large-en-v1.5** | 1024 | 512 | Open source | | **all-MiniLM-L6-v2** | 384 | 256 | Fast, lightweight | | **multilingual-e5-large** | 1024 | 512 | Multi-language | ### 2. Embedding Pipeline ``` Document → Chunking → Preprocessing → Embedding Model → Vector ↓ [Overlap, Size] [Clean, Normalize] [API/Local] ``` ## Templates ### Template 1: OpenAI Embeddings ```python from openai import OpenAI from typing import List ...