Benchmark Trust Model

PomaiDB benchmarks exist to earn trust, not to market speed. We adhere to a "Single Source of Truth" policy.

What We Guarantee

• Correctness: Recall matches brute-force oracle with high fidelity.
• Tail Latency: We always report P99 and P999. No hidden outliers.
• Low-end Viability: Benchmarks must run on 2-core/8GB laptops.

Running Benchmarks

We provide a comprehensive suite via the pomai-bench script.

Quick Commands

# 1. Trust benchmarks (Recall, Tail Latency, Crash Recovery)
./scripts/pomai-bench recall
./scripts/pomai-bench mixed-load
./scripts/pomai-bench crash-recovery

# 2. End-to-end Python Benchmark (CIFAR-10)
python3 benchmarks/python_cifar10_feature_bench.py --images 6000 --queries 200

Metrics to Watch

• Search Latency (µs): Target P99 < 10ms.
• Throughput (QPS): Should scale linearly with threads.
• Recall@10: Should be > 0.90 for production.
• Build Time: Time to index vectors.

Low-End Device Warning

On 2-core CPUs with 8GB RAM, avoid medium/large datasets. Multi-threading on 2 cores causes thrashing. Stick to --dataset small or use --threads 1.