qdrant-horizontal-scaling

Solid

Diagnoses and guides Qdrant horizontal scaling decisions. Use when someone asks 'vertical or horizontal?', 'how many nodes?', 'how many shards?', 'how to add nodes', 'resharding', 'data doesn't fit', or 'need more capacity'. Also use when data growth outpaces current deployment.

AI & Automation 222 stars 28 forks Updated yesterday Apache-2.0

Install

View on GitHub

Quality Score: 87/100

Stars 20%
78
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# What to Do When Qdrant Needs More Capacity Vertical first: simpler operations, no network overhead, good up to ~100M vectors per node depending on dimensions and quantization. Horizontal when: data exceeds single node capacity, need fault tolerance, need to isolate tenants, or IOPS-bound (more nodes = more independent IOPS). ## Most basic distributed configuration - 3 nodes, 3 shards with `replication_factor: 2` for zero-downtime scaling Minimum of 3 nodes is important for consensus and fault tolerance. With 3 nodes, you can lose 1 node without downtime. With 2 nodes, losing 1 node causes downtime for collection operations. Replication factor of 2 means each shard has 1 replica, so you have 2 copies of data. This allows for zero-downtime scaling and maintenance. With `replication_factor: 1`, zero-downtime is not guaranteed even for point-level operations, and cluster maintenance requires downtime. ## Choosing number of shards Shards are the unit of data distribution. More shards allows more nodes and better distribution, but adds overhead. Fewer shards reduces overhead but limits horizontal scaling. For cluster of 3-6 nodes the recommended shard count is 6-12. This allows for 2-4 shards per node, which balances distribution and overhead. ## Changing number of shards Use when: shard count isn't evenly divisible by node count, causing uneven distribution, or need to rebalance. Resharding is expensive and time-consuming, it should be used as a last resort if regul...

Details

Author
qdrant
Repository
qdrant/skills
Created
6 months ago
Last Updated
yesterday
Language
Python
License
Apache-2.0

Integrates with

Bundled in these plugins

Similar Skills

Semantically similar based on skill content — not just same category