ai-cost-token-optimizer

# AI Cost & Token Optimizer [English](#english) | [Bahasa Indonesia](#bahasa-indonesia) --- <a name="english"></a> ## English ### Purpose & Overview Production-grade guidelines for FinOps in AI engineering — prompt caching (Anthropic Prompt Caching, Gemini Context Caching), dynamic model routing (routing lightweight queries to Flash/Haiku and complex reasoning to Pro/Opus), semantic caching with Redis/GPTCache, and real-time token expenditure tracking. ### Key Capabilities - **Prompt & Context Caching**: Storing static system prompts, long-context documents, and schemas in cache to reduce token costs by up to 90%. - **Model Router**: Heuristic and classifier-based routing between ultra-fast Flash models and high-reasoning Pro models. - **Semantic Caching**: Hashing query vector embeddings to serve cached responses for semantically identical user queries. ```typescript // Model Routing Strategy Example export function selectOptimalModel(promptLength: number, taskType: 'classification' | 'reasoning' | 'summary') { if (taskType === 'classification' || promptLength < 500) { return 'gemini-3.5-flash'; // High speed, ultra low cost } return 'gemini-3.1-pro'; // Complex reasoning } ``` ### Implementation Checklist - [ ] Enable Context Caching for static system prompts or documents larger than 32k tokens. - [ ] Implement a router heuristic: use `gemini-3.5-flash` for simple parsing and `gemini-3.1-pro` for deep reasoning. - [ ] Set up semantic caching (e.g., Redis + ...