omni-inference

Featured

The core OpenAI-compatible inference endpoints: chat completions, embeddings, images, audio (TTS/STT), moderations, rerank, and the Responses API. The primary integration surface for AI agents.

AI & Automation 32,540 stars 4212 forks Updated today MIT

Install

View on GitHub

Quality Score: 93/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

<!-- generated by src/lib/agentSkills/generator.ts; manual edits will be overwritten --> ## Overview The core OpenAI-compatible inference endpoints: chat completions, embeddings, images, audio (TTS/STT), moderations, rerank, and the Responses API. The primary integration surface for AI agents. ## Authentication All requests require a valid Bearer token or session cookie. Obtain a token via `POST /api/auth/login` or configure `REQUIRE_API_KEY=false` for local development. ## Endpoints ### POST /api/v1/chat/completions Create chat completion OpenAI-compatible chat completions endpoint. Routes to configured providers. ```bash curl -X POST https://localhost:20128/api/v1/chat/completions \ -H "Authorization: Bearer $OMNIROUTE_TOKEN" -H "Content-Type: application/json" \ -d '{}' ``` ### GET /api/v1/ws Chat completion over WebSocket (handshake + upgrade) OpenAI-compatible chat over a WebSocket connection. `GET` with `?handshake=1` returns the connection descriptor (auth path, message protocol and live-event channels) as JSON; a plain `GET` without an Upgrade returns `426 Upgrade Required`. After upgrading, the client exchanges JSON frames — `{type:"request", id, payload:{model, messages}}` to start a completion and `{type:"cancel", id}` to abort it. A separate live channel (default port `LIVE_WS_PORT=20129`, path `/live`) streams dashboard events on the `requests`, `combo` and `credentials` topics with a 15s heartbeat. Requires an API key. ```bash curl https://loc...

Details

Author
diegosouzapw
Repository
diegosouzapw/OmniRoute
Created
5 months ago
Last Updated
today
Language
TypeScript
License
MIT

Integrates with

Similar Skills

Semantically similar based on skill content — not just same category