BM25 (Okapi BM25) is a lexical ranking function that scores documents by keyword overlap with a query, widely used in search and retrieval systems.
GPT-4.1 scored 93.49% on BrowseComp-Plus with documents supplied and 14.58% when BM25 searched, according to The New Stack. That 78.9-point gap sizes the room for retrieval work on one fixed corpus. Production failure rates need their own measurement.
Reality
turbopuffer says v3 cut hot and cold full-text p90 latency by up to 80% against v2 on 10-million-document namespaces. The figure is from the vendor's own nightly benchmark, and the company does not expect a gain for cold vector queries.
Reality
Databricks tests Lakebase Search on 1M to 4M rows, using a 23,000-row internal app as its baseline. A search request touches two computes, the app's and the database's, and either one can set the limit.
Reality
ParadeDB's pg_search 0.26 cut a ten-term BM25 search from 129ms to 29ms in a benchmark published on dev.to. The same storage change made single-term queries several times slower, so teams whose searches are mostly one or two words would pay for the upgrade.
Reality
Databricks made Lakebase Search generally available, pairing BM25 with a Postgres vector index it says runs 4 times cheaper than pgvector at 100M vectors. The vector index lives in object storage behind a cache, so large agent corpora no longer need RAM sized to the whole index.
Reality
LiveReview's Maneshwar says Gemini File Search cost over 50 cents a review run because a reasoning model performed each search. A local index and a cheaper model brought runs down to 4 cents.
Reality
A two-stage eval forces every shortlist to hold the correct tool plus its four strongest BM25 siblings. Selection accuracy comes in at 90 to 97 percent. That puts the 5 percent paraphrase recall on the critical path.
Reality
Deferred tool loading assumes retrieval puts the right tool in the shortlist. A BM25 harness over 100 synthetic enterprise tools and 200 tasks says it does that 5 percent of the time when the user does not use your words.
Reality
A Magic: The Gathering rules agent works out which source is entitled to settle a question before it answers. Its evaluation pits 496 structured documents against a single BM25 pass over the same corpus, graded blind.
Reality
The build log for a Python package finder shows a day-one no-key rule pushing the download data onto a public JSON file and 15,000 PyPI API calls per refresh, held to a few minutes by a semaphore of 25.
Reality
An embedding swap to bge-large made two long-failing negative tests pass because the retriever stopped surfacing the trap chunks at all. A chunk-id check in CI gives the suite a third verdict, test did not run.
Reality
The extension puts BM25 ranking behind an ordinary Postgres index, so a newly committed row is searchable without a trip to an Elasticsearch cluster. PlanetScale's own benchmark reports 199 queries per second against ParadeDB's 7.9.
Reality
The four RAGAS-lineage metrics were built to separate a retriever's failures from a generator's. Read in pairs, they also expose the case where the model skipped the context and got the answer right anyway.
Reality
Name-only BM25 matched nothing for "money we gave back to shoppers". The description channel put the right table second and embeddings put it twenty-second, and reciprocal rank fusion let the blind channel cost the answer nothing.
Reality
A dev.to post argues that IBM's chunkless RAG only helps on documents whose structure a parser can actually recover, and that on scanned PDFs and one-table wikis the agent ends up walking a tree the parser invented.
Reality
An Apache-2.0 memory layer for coding agents returns a candidate only when several signals agree, and it labels every answer STRONG, WEAK or MISS so the calling agent has to branch on confidence before reusing an old fix.
Reality
The generated descriptions were accurate and specific, and indexing them beside the table names dropped BM25's IDF for the term contact to 0.15, about what the same index gives words the tokenizer never strips out.
Reality
The same platform ships regex and BM25 ranking in front of its tool catalog, while its memory store offers only depth, limit, page, path_prefix and view, so ranking memories is work the developer supplies.
Reality
pgvector ships hnsw.ef_search at 40, the size of the candidate list its graph walk keeps in flight, and a top-20 query with a tenant filter can come back with two rows and no error to explain it.
Reality
Perplexity pairs about 190 million web pages with 69,721 agent-written queries, which is closer to production than most retrieval tests get, and keeps the corpus, queries and labels private so it stays the only party able to run it.
Reality