groq-sdk-patterns

# Groq SDK Patterns ## Overview Production patterns for the `groq-sdk` package. The Groq SDK mirrors the OpenAI SDK interface (`chat.completions.create`), so patterns feel familiar but must account for Groq-specific behavior: extreme speed (500+ tok/s), aggressive rate limits on free tier, and unique response metadata like `queue_time` and `completion_time`. ## Prerequisites - `groq-sdk` installed - Understanding of async/await and error handling - Familiarity with OpenAI SDK patterns (Groq is API-compatible) ## Instructions ### Step 1: Typed Client Singleton ```typescript // src/groq/client.ts import Groq from "groq-sdk"; let _client: Groq | null = null; export function getGroq(): Groq { if (!_client) { _client = new Groq({ apiKey: process.env.GROQ_API_KEY, maxRetries: 3, timeout: 30_000, }); } return _client; } ``` ### Step 2: Type-Safe Completion Wrapper ```typescript import Groq from "groq-sdk"; import type { ChatCompletionMessageParam } from "groq-sdk/resources/chat/completions"; const groq = getGroq(); interface CompletionResult { content: string; model: string; tokens: { prompt: number; completion: number; total: number }; timing: { queueMs: number; totalMs: number; tokensPerSec: number }; } async function complete( messages: ChatCompletionMessageParam[], model = "llama-3.3-70b-versatile", options?: { maxTokens?: number; temperature?: number } ): Promise<CompletionResult> { const response = await groq.chat.compl...