Databricks Benchmarks Lakebase Search Performance

Databricks released technical benchmark data and sizing rules for Lakebase Search, its integrated hybrid search extension for Lakebase Postgres. Evaluated using Slide Hub, an internal tool indexing 23,000 slides, as well as synthetic scale tests on up to 4 million passages from the MIRACL English corpus using the gte-large-en-v1.5 embedding model, the findings establish clear operational guidelines for developers running vector and keyword searches within PostgreSQL.

The benchmarks demonstrate that aligning the Local File Cache with the vector index size keeps 95% to 98% of database pages in memory, bringing warm vector query latencies down to single-digit milliseconds. Each cache miss forces an 8 KB page fetch from storage, adding approximately 0.3 milliseconds to query execution. Reducing the candidate pool limit from 1,000 to 200 candidates cut page reads to roughly one-third—from 28,972 down to 10,401 pages—while retaining 90.5% of top-10 search results and halving cold latency.

System throughput tests revealed a structural bottleneck threshold. Below 8 Compute Units (CU), database compute limits performance, but at or above 8 CU, the application's Python worker connection pool becomes the primary constraint. At 16 CU, scaling the host app to a LARGE tier with four workers pushed throughput beyond 662 requests per second. Additionally, building the vector index with quality mode raised recall@10 from 0.74 to 0.80 at default probe counts in just 3 milliseconds, requiring a 52-minute build process for 4 million passages at 16 CU.

For practitioners, these results demonstrate that high-performance vector search does not require dedicated external vector databases. Because serverless Lakebase Postgres bills only active compute, a 14-minute load test serving 109,602 requests accrued just $0.02 in database compute costs. To avoid one-second cold-start connection penalties in production, Databricks recommends establishing a 0.5 CU compute baseline, which costs between $31 and $40 per month.