openrouter-load-balancing

Featured

Distribute OpenRouter requests across multiple keys and models for high throughput. Use when scaling beyond single-key rate limits or building high-availability systems. Triggers: 'openrouter load balance', 'openrouter scaling', 'distribute openrouter requests', 'multiple api keys'.

AI & Automation 2,926 stars 433 forks Updated yesterday MIT

Install

View on GitHub

Quality Score: 99/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# OpenRouter Load Balancing ## Overview A single OpenRouter API key has rate limits (requests/minute and tokens/minute). To scale beyond those limits, distribute requests across multiple keys. OpenRouter also provides server-side load balancing via provider routing and the `:nitro` variant for low-latency inference. This skill covers multi-key rotation, health-based routing, circuit breakers, and concurrent request patterns. ## Multi-Key Round Robin ```python import os, itertools, time, logging from openai import OpenAI, RateLimitError from dataclasses import dataclass, field log = logging.getLogger("openrouter.lb") @dataclass class KeyPool: """Round-robin API key pool with health tracking.""" keys: list[str] _cycle: itertools.cycle = field(init=False, repr=False) _health: dict[str, dict] = field(init=False, default_factory=dict) def __post_init__(self): self._cycle = itertools.cycle(self.keys) self._health = {k: {"errors": 0, "last_error": 0, "healthy": True} for k in self.keys} def next_key(self) -> str: """Get next healthy key.""" attempts = 0 while attempts < len(self.keys): key = next(self._cycle) h = self._health[key] # Recover after 60s cooldown if not h["healthy"] and time.time() - h["last_error"] > 60: h["healthy"] = True h["errors"] = 0 if h["healthy"]: return key attempts += 1...

Details

Author
jeremylongshore
Repository
jeremylongshore/claude-code-plugins-plus-skills
Created
1 years ago
Last Updated
yesterday
Language
Python
License
MIT

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category