Performance Benchmarking
Here we present a comprehensive benchmark comparing our implementation of VectorChord with other vector databases, focusing on QPS and scalability. If you are interested in the details of our implementation, please refer to our blog VectorChord: Store 400k Vectors for $1 in PostgreSQL.
QPS
LAION 5M
In LAION 5M dataset, VectorChord achieved more than 3x QPS at 0.95 recall compared to different vector databases, including MyScale, pgvector, pinecone, weaviate, qdrant and zilliz(milvus).
LAION 100M on i4i.xlarge
This setup utilizes 4 CPUs and 32 GB of memory, with 937 GB of SSD storage for the 100 million vectors. We achieved a QPS of 16.2 at a recall rate of 0.95 for the top 10 results and 4.3 at 0.95 for the top 100, using a single-threaded query. The results are truly impressive.
Scalability
If you are interested VectorChord scalable capability, you can refer our blog Vector Search at 10,000 QPS in PostgreSQL with VectorChord.
LAION 100M && LAION 5M
| dataset | minimal hardware for 10000+ QPS | number of vcpu | estimate cost per month |
|---|---|---|---|
| LAION-5m | r7i.xlarge × 7 | 4 × 7 = 28 vcpu | $0.2646 × 7 × 720 = $1334 |
| LAION-100m | r7i.16xlarge × 2 | 64 × 2 = 128 vcpu | $4.2336 × 2 × 720 = $6096 |
TIP
The price benchmark is tested on December 5, 2024. We are ongoingly enhancing performance and optimizing costs. For example, VectorChord 0.2 builds on its predecessor with a 30% smaller index size and 50% faster builds. If you want to know the latest benchmark, please contact us at discord.