together-rate-limits
FeaturedTogether AI rate limits for inference, fine-tuning, and model deployment. Use when working with Together AI's OpenAI-compatible API. Trigger: "together rate limits".
AI & Automation 2,926 stars
433 forks Updated yesterday MIT
Install
Quality Score: 99/100
Stars 20%
Recency 20%
Frontmatter 20%
Documentation 15%
Issue Health 10%
License 10%
Description 5%
Skill Content
# Together AI Rate Limits
## Overview
Together AI's OpenAI-compatible inference API enforces per-key rate limits that vary by model tier and operation type. Chat completions and embeddings share a global request quota, while fine-tuning jobs and batch inference have separate concurrency caps. High-throughput workloads like embedding entire document corpora or running evaluations across 100+ prompts require client-side token bucket limiting. Together's batch inference endpoint offers 50% cost savings but has its own queue depth limits that differ from real-time inference.
## Rate Limit Reference
| Endpoint | Limit | Window | Scope |
|----------|-------|--------|-------|
| Chat completions | 600 req | 1 minute | Per API key |
| Embeddings | 300 req | 1 minute | Per API key |
| Image generation (FLUX) | 60 req | 1 minute | Per API key |
| Fine-tune jobs (concurrent) | 3 jobs | Rolling | Per API key |
| Batch inference | 100 req/batch, 10 batches | Rolling | Per API key |
## Rate Limiter Implementation
```typescript
class TogetherRateLimiter {
private tokens: number;
private lastRefill: number;
private readonly max: number;
private readonly refillRate: number;
private queue: Array<{ resolve: () => void }> = [];
constructor(maxPerMinute: number) {
this.max = maxPerMinute;
this.tokens = maxPerMinute;
this.lastRefill = Date.now();
this.refillRate = maxPerMinute / 60_000;
}
async acquire(): Promise<void> {
this.refill();
if (this.tokens...
Details
- Author
- jeremylongshore
- Repository
- jeremylongshore/claude-code-plugins-plus-skills
- Created
- 1 years ago
- Last Updated
- yesterday
- Language
- Python
- License
- MIT
Integrates with
Bundled in these plugins
Similar Skills
Semantically similar based on skill content — not just same category
AI & Automation Featured
glean-rate-limits
Glean Indexing API: ~100 requests/min per token. Trigger: "glean rate limits", "rate-limits".
2,926 Updated yesterday
jeremylongshore AI & Automation Featured
appfolio-rate-limits
Handle AppFolio API rate limits with throttling and backoff. Trigger: "appfolio rate limit".
2,926 Updated yesterday
jeremylongshore AI & Automation Featured
together-common-errors
Together AI common errors for inference, fine-tuning, and model deployment. Use when working with Together AI's OpenAI-compatible API. Trigger: "together common errors".
2,926 Updated yesterday
jeremylongshore