Search APIs ranked by SimpleQA benchmark accuracy

Unaided LLMs, models answering from training data alone with no search tool attached, are bad at factual recall in a way that should worry anyone building on them without retrieval. That is the whole reason the search-API benchmark category exists in the first place. Gemini 2.5 Pro leads the unaided leaderboard at 53.0%, and that number deserves a moment of attention, because it marks the ceiling for parametric memory alone: the best a frontier model can do with nothing to consult but its own weights. Earlier snapshots make the trend look worse: o1-preview landed at 42.7%, and Claude 3.5 Sonnet came in at 28.9%. Neither was a second-tier system; both were the frontier at the time they shipped, and both still got short, simple factual questions wrong more often than they got them right.

A model's training data is a compressed, lossy snapshot of the internet frozen at some point in the past. An obscure geography fact, a minor political detail, a narrow topic that only shows up in a handful of sources: this is exactly what gets flattened or dropped in that compression. So the model does not know what it does not know, yet it answers anyway. Call it the overconfidence problem, and it is the more dangerous of the two failure modes here, worse than simply being wrong. These systems almost never say "not attempted," and a wrong answer delivered with total conviction does more damage than an admitted gap, because the reader has no signal telling them to doubt it.

Here is where most builders get the calculation backwards: they treat retrieval as a feature to bolt on once the core product works, rather than the thing that makes the core product trustworthy in the first place. The distance between 28.9% and the 90-plus scores that retrieval-augmented systems post later in this piece is the strongest argument for grounded infrastructure that exists in benchmark form, the difference between a coin flip and something close to dependable. For any application where a wrong fact carries real cost, a search API is not optional. Treating it as a nice-to-have, something to layer in during a later sprint, is the first mistake builders make here, and it is a more expensive mistake than most teams budget for.

How the major search API benchmarks were run and why methodology shapes the rankings

Most published SimpleQA rankings for search APIs come from the vendors themselves, and none of them are neutral parties. Valyu, Parallel, Linkup, Tavily, and Felo have each put out their own numbers, and the methodology behind a score matters more than the score itself. Treating a vendor's leaderboard as settled fact is the second mistake builders make, right after skipping retrieval altogether. If the first section argued that grounding matters more than any single accuracy figure, this one argues that the figures themselves need interrogating before they mean anything.

A few variables move results substantially, and they move who "wins" as a result. The reader model, whichever LLM turns retrieved content into a final answer, matters enormously; swap GPT-5 for a weaker model and the numbers shift underneath it. The judge model, the LLM grading whether an answer counts as correct, carries its own error rate, since LLM-as-judge is an imperfect measuring tool by nature. Content budget, meaning how many characters of retrieved text an API can stuff into the reader's context, changes outcomes too: a generous budget lets a provider pile in more supporting evidence, which does not necessarily mean better search, just more of it. And the query set matters a great deal. The full SimpleQA corpus produces different numbers than a 100-question composite sample, which produces different numbers again than Verified SimpleQA or the tougher SealQA-0. None of these are directly comparable, even when they get reported side by side in the same blog post.

One study earns a mention for its transparency. Parallel's November 2025 comparison isolated each provider so it could only call its own search tool, and held the reader model, grader, and prompts identical across every provider tested. The only thing that varied was what each API actually returned. That design carries more weight than letting each vendor grade its own homework, worth flagging even though Parallel is itself one of the providers in the comparison.

Linkup's evaluation harness is open source, a different kind of transparency: anyone can pull it down and rerun it against a provider of their choosing. Reproducibility counts for something, even coming from an interested party, though it does not erase the fact that Linkup is still publishing its own leaderboard.

So here is the rule, kept short on purpose: before trusting any published ranking's order, check the query set, the reader model, the content budget, and whether the run was fast mode or deep mode. Skip that check and the ranking reads closer to marketing copy than to a finding.

The published SimpleQA scores across major search APIs

With that caveat in front of you, here is what the numbers look like, and they should be read as clusters, not a precise ladder. A 2 to 3 point gap between two providers is very likely noise, an artifact of which reader model got used or how the sample was drawn. A 15 to 20 point gap is signal. Confusing the two is how a vendor blog post turns noise into a marketing claim, and it is worth naming that as the single most common misread in this space.

In the high-accuracy cluster, tested mostly in fast or standard retrieval mode against Verified SimpleQA or something close to it: Valyu posted 94% in its own October 2025 study, and 79% on the harder, time-sensitive FreshQA set in that same study, a gap worth holding onto for the next section. Linkup Fast leads Linkup's own August 2026 leaderboard on Verified SimpleQA. Tavily claimed 93.3% in a June 2025 self-published evaluation using real-time web retrieval alone. Parallel posted 90% in its November 2025 composite study on a 100-question sample with GPT-5 as reader and GPT-4.1 as judge, then 91% in a separate July 2026 fast-API comparison. Felo Pro's Fast Mode reported 91.2% in a February 2025 self-published run.

The mid-accuracy cluster tells a slightly different story. OpenAI's GPT-5 with native search hit 88% in Parallel's composite study, and Perplexity landed at 74% in that same study. Tavily's Ultra Fast mode scored 72% in Parallel's latency-controlled comparison, well below Tavily's own 93.3% figure above. That gap alone is a clean demonstration of how much retrieval mode can swing a score for the same provider.

Lower down, the range widens further. One provider came in at 76.7% in Parallel's latency study, and in Valyu's October 2025 study, Google scored 38%, a full 56 percentage points below that study's top performer. That single gap, inside one controlled study, is probably the starkest illustration available of how much provider choice can swing outcomes on the exact same benchmark.

You.com shows up in Tavily's August 2026 SealQA results, named among the providers Tavily says it outperformed after rebuilding its reranking and evidence-quality pipeline. SimpleQA, though, is a narrower test than what You.com's Research API is built for. It is positioned for grounded, cited research work rather than short single-fact lookups, which is the more relevant comparison for anyone actually evaluating that product.

The caveat that has to close this section: every one of these numbers came from a different study, run at a different time, on a different query set, with a different reader model doing the synthesis. Stitching them into one table is not a valid head-to-head ranking. Read them as separate snapshots, and be suspicious of anyone who presents them as one.

How latency interacts with accuracy and what that means for agent loop design

Accuracy is only half the equation once a search API sits inside an agent loop. Parallel's latency-controlled study, equal content budget across providers, GPT-5.4 as reader, puts the tradeoff in a single table. Parallel Search Turbo posted 91% accuracy at a 240ms p50 latency; one provider followed at 87% accuracy and 475ms. Another provider came in at 76.7% accuracy with 652ms latency, while Tavily Ultra Fast was the fastest of the group at 150ms, but also the least accurate at 72%.

Notice what is missing from that pattern: a clean inverse relationship between speed and accuracy. The intuitive assumption, that fast search must mean shallow search, does not hold up here. Tavily Ultra Fast is both the fastest and the least accurate in this comparison, while Parallel Turbo sits second-fastest with the highest accuracy of the group. Speed and correctness behave like separate engineering problems, and a provider can solve one without touching the other. That is the detail worth sitting with if the plan is to pick a provider on latency alone.

That raises the real question for anyone building an agent: is the choice between fast mode and deep mode more consequential than the choice between two providers running at similar speeds? The data above says yes. Fast or turbo endpoints, running sub-second, suit conversational agents and high-concurrency loops where one call's error has a decent chance of getting caught downstream by multi-hop reasoning. Deep or research endpoints, running several seconds to minutes, suit the cases where one synthesized answer has to be right on the first pass, no do-over: finance, compliance, medical triage.

Latency compounds in ways that are easy to underestimate on paper. A five-step agentic reasoning chain, each step firing off a search call at 650ms, adds more than three seconds of wall-clock time before the model even starts composing its final answer. Multiply that across concurrent users, and sub-300ms search stops being a nice-to-have; it becomes an architectural constraint. In a pipeline where the chain is only as fast as its slowest step, tail latency carries as much weight as the headline accuracy score.

What SimpleQA misses: freshness, multi-hop reasoning, and domain depth

Here is the limitation that matters most, and it is baked into the benchmark's design rather than being a flaw in how anyone ran it. SimpleQA uses a fixed corpus of questions with fixed, static ground-truth answers. The benchmark cannot, by construction, test whether a provider's index is current, because "correct" is defined against an answer that never moves. A provider could run on an index eighteen months stale and still score well on SimpleQA, as long as the facts being asked about have not changed since. That is the single biggest reason a SimpleQA leaderboard should never be the only document a team reads before signing a contract.

FreshQA exists to close that gap, running time-sensitive queries instead, and the rankings shift hard once freshness enters the picture. In the Valyu October 2025 study, Valyu scored 79% on FreshQA, Parallel scored 52%, Google scored 39%, and the field compresses further from there. That is a genuinely different competitive order than the one SimpleQA produces on its own, and it is the clearest proof in this landscape that a single benchmark score should never stand in for "accurate." A provider posting 91% on SimpleQA and dropping to the low 20s on FreshQA is the wrong pick for a news-monitoring agent, no matter how clean its factual accuracy looks on static trivia.

SimpleQA is also single-hop by construction: one question, one answer, no requirement to chain sources together or resolve a contradiction between them. BrowseComp, a benchmark built around multi-step reasoning, produces yet another ranking entirely. Parallel leads at 58%, OpenAI's GPT-5 follows at 53%, and other providers trail well behind, down in the low twenties and thirties. For an agent that has to reconcile conflicting sources across a longer research chain, BrowseComp predicts real-world performance far better than SimpleQA does, because it tests the actual skill the agent needs.

Domain depth is the third blind spot, and maybe the most consequential one for vertical applications. A general corpus spanning science, politics, art, and geography tells a builder nothing about performance on SEC filings or macroeconomic indicators. The medical-domain benchmark inside the Valyu study makes this concrete: scores compress into a tight band, roughly 42% to 48% across the providers tested, and the general-purpose ranking advantage that looked so decisive on SimpleQA nearly disappears. Whatever separated the leaders on trivia questions does not necessarily separate them on specialized domain content, and that alone should make anyone nervous about extrapolating a SimpleQA score into a vertical use case.

SealQA-0 sits at the far end of this difficulty curve, built specifically to stress-test deep research pipelines at the exact points where standard search tends to break. Tavily claims the top spot on the SealQA-Hard variant after an August 2026 rebuild of its reranking stack, with Linkup Research also reported near the front of that harder benchmark. The rankings scramble again, because SealQA measures a different failure mode than SimpleQA does.

The practical implication follows directly, and it is worth stating without hedging: match the benchmark to the actual query distribution, or the benchmark tells you nothing useful. SimpleQA is a reasonable filter for general factual grounding, FreshQA matters for anything time-sensitive, and BrowseComp matters for multi-hop agents. SealQA matters for deep research pipelines. Domain-specific evaluations matter for anything vertical, and none of the general benchmarks substitute for them.

How to run your own accuracy evaluation before committing to a provider

The numbers above need supplementing with testing against an application's own query distribution, and that needs saying plainly before getting into method. The most useful accuracy number is the one measured on the questions an agent will actually get asked, not the one measured on SimpleQA's fixed corpus of science and geography trivia.

Start by sorting the query types an application actually generates: general factual, time-sensitive, multi-hop, domain-specific. Each calls for a different benchmark as a sanity check, for the reasons laid out above. A sample of at least 100 representative queries gets a directional read; the gap between two providers visible at 100 questions is usually stable when the sample grows toward SimpleQA's full corpus.

Hold every other variable fixed while testing: same reader model across providers, same content budget per result, same judge model or human grading rubric, same retrieval mode, fast against fast, deep against deep. Skip this and the comparison measures the test harness instead of the search API, which is exactly the mistake buried in half the vendor leaderboards discussed above.

Measure at least three things at once, never accuracy alone: accuracy under SimpleQA-style pass/fail grading, freshness of the returned content, and latency at both p50 and p99. Averages hide tail behavior, and tail behavior is what breaks an agent loop under load.

Cost deserves a second look too, because cost-per-call is a limited unit to optimize on its own. Cost-per-correct-answer is the number that actually predicts production spend: a cheaper API running 15 points below a competitor on accuracy often forces re-querying, which erases whatever savings looked good on the pricing page. Parallel's composite study makes the point directly. Tavily at $227 CPM with 59% accuracy against Parallel at $50 CPM with 90% accuracy is a starker gap once cost gets adjusted for correctness than the raw accuracy numbers suggest on their own. Anyone still comparing providers on sticker price alone is optimizing the wrong variable.

One more dimension gets missed constantly: citation integrity, tested separately from factual accuracy. An answer can pass SimpleQA's grading and still carry a citation that does not actually support the claim, or no verifiable citation at all. In finance or healthcare contexts, an uncited correct answer is still a compliance problem waiting to surface, and no accuracy score catches that on its own.

Linkup's open-source evaluation harness lowers the barrier to running all of this independently; fork it, point it at any provider. Transparency in methodology matters enough here to name directly: You.com, as a provider of web search and research APIs built for AI agents, takes part in third-party comparisons like Parallel's controlled studies, which let developers see the reader model, content budget, and query set behind a score rather than taking an ordinal ranking on faith. Its Research API and Finance Research API are reasonable candidates for an enterprise evaluation that weights citation integrity heavily, alongside whichever domain benchmarks, DeepSearchQA and FinSearchComp among them, best match the actual workload being built.

What accuracy-first infrastructure selection looks like in practice

Put the pieces together and a pattern emerges. SimpleQA scores in the low 90s are common across several leading search APIs, so the real decision rarely comes down to a 2-point gap on one general-knowledge benchmark. Anyone picking a provider off a single SimpleQA number is missing the variable that matters more: which failure mode an application can least afford, stale information, a broken multi-hop chain, a missing citation, or a slow tail latency that stalls an agent loop under concurrent load.

SimpleQA did something valuable by turning "does this system hallucinate" into a measurable, reproducible number, and the gap between unaided LLMs scoring in the 20s and 30s and retrieval-augmented systems scoring in the 90s proves that grounding delivers a real improvement. Yet no single benchmark, including SimpleQA's own harder descendants, captures every dimension that decides whether infrastructure holds up in production. Freshness, multi-hop reasoning, domain depth, latency under load, cost-per-correct-answer, citation integrity: each is a separate axis, and a provider's strength on one says surprisingly little about its strength on another.

The builders who get this right stop asking which provider wins on SimpleQA and start asking which axis matters most for the specific agent being built. Then they test against that axis directly, on a sample drawn from real queries rather than a published leaderboard. That is more work than reading a vendor's blog post, but it is also the method most likely to produce a number worth trusting.