Open Benchmarks vs Proprietary Benchmarks for Search Infrastructure
Open benchmarks let you verify claims; proprietary ones only let vendors control the narrative.
Daniel Ellison
Section
API Benchmarks
11 stories in API Benchmarks.
Hallucination Rates as a Search API Quality Signal
Vendor hallucination rates measure different things, making them unreliable procurement signals.
Elena Fontaine
Citation Quality as a Search API Evaluation Metric
Search APIs optimize the wrong metrics, leaving citations factually unsupported.
Marcus Rossi
Building a Custom Eval Suite for Search-Augmented LLMs
Separate retrieval scoring from generation scoring to catch citation gaps most evals miss.
Daniel Ellison
Building an Internal Search API Eval Suite
Test each failure mode independently so regressions don't hide behind a single misleading score.
Daniel Ellison
Open Search API Evaluation Benchmarks Explained
Understanding what search benchmarks actually measure matters more than chasing the highest scores.
Marcus Rossi
Evaluating Search API Freshness for AI Agents
Stale data in agents' context windows causes failures that retraining won't fix.
Daniel Ellison
Search APIs ranked by SimpleQA benchmark accuracy
Retrieval-augmented systems close a 60-point accuracy gap that unaided language models cannot cross.
Elena Fontaine
How to Benchmark Search APIs for AI Applications
Evaluate search APIs on agent behavior, not human browsing patterns.
Daniel Ellison
Search API Accuracy Benchmark Methodology
Methodology decisions buried under benchmark scores matter more than the numbers themselves.
Priya Sinclair
Accuracy vs. latency trade-offs across search API providers
Bing's retirement forces teams to choose between speed and accuracy in search APIs.
Marcus Rossi