AI Assistants Name the Right SaaS Price Only 15% of the Time

Ask a leading AI assistant what a piece of software costs and there is a 6-in-7 chance it will not give you the current price. We tested three frontier assistants on the starting paid price of 79 widely used SaaS tools, checked our own answer against each vendor within the last two weeks, and let an independent model grade the results. The assistants named the correct current price 15% of the time. When they did commit to a number, they were wrong far more often than right.

This is the single clearest reason software data belongs in a live tool, not a model's memory: prices move, plans get renamed, cheaper tiers appear, and training data freezes.

Why this matters

An assistant that invents a plausible-but-wrong price is worse than one that says "I do not know," because the user acts on it. A buyer comparing tools, an agent building a shortlist, or a spreadsheet of stack costs inherits the error silently. As AI assistants take on more purchasing research, the cost of stale pricing knowledge compounds.

The fix is not a better model. It is giving the model a live, verified source at answer time.

What we measured

We sampled 79 well-known SaaS products (editorial score 89 or higher) where Toolradar had verified the pricing within the previous ~10 days. For each, we recorded the cheapest paid monthly plan as ground truth. We then asked three frontier assistants, from memory and with no web access or tools, for the current cheapest paid plan and its price. A separate model graded each answer against our verified figure as correct, wrong (a specific price that materially differs), or hedged (declines or gives no current price).

Results by model

The three assistants disagree on style, not accuracy. Claude Sonnet 4 is the most calibrated: it hedges the most and is wrong the least. Gemini 2.5 Flash is the opposite, committing to a price four times out of five and getting most of them wrong. GPT-4o, with the oldest training cutoff, hedges nearly half the time.

The failure mode: confidently wrong

These are real answers from the test. The verified column is the current cheapest paid plan as of the check date; the assistant column is what the model stated from memory.

The direction is consistent: the model quotes a higher, older number. It learned a past price list and cannot see that the vendor added a cheaper tier or cut the entry price. No amount of reasoning fixes a stale fact.

What fixes it

Toolradar verifies pricing continuously and exposes it to AI systems through the Toolradar MCP server. Instead of recalling a price from training data, an assistant calls get_pricing or recommend_tools and receives the figure we verified this week, with the plan name and the date we last checked it. On the 79 tools in this test, that turns a 15% hit rate into the verified number, every time, with a citation.

The same server also lets an agent flag a discrepancy back to us with report_issue, so the data gets sharper the more it is used.

Methodology

Sample. 79 published SaaS products with a Toolradar editorial score of 89 or higher and a clean, monthly, per-seat starting paid price between $3 and $100, each with pricing verified within roughly 10 days of the test (verification dates span August 11-21, 2026). We deliberately restricted the set to well-known tools and simple monthly pricing so the question is fair and the ground truth is unambiguous; usage-based, contact-sales, one-time, and enterprise-only products were excluded.

Ground truth. The cheapest paid monthly plan recorded in Toolradar's database, each field verified against the vendor's own pricing page within the check window.

Baseline. OpenAI GPT-4o, Anthropic Claude Sonnet 4, and Google Gemini 2.5 Flash, queried through OpenRouter with no tools and no web access, at temperature 0, so answers reflect parametric (training) knowledge only. Reasoning-tier models were excluded because they returned empty or truncated content over this API path.

Grading. A separate model (GPT-4o mini) classified each answer against the verified figure as correct, wrong, or hedged, using a fixed rubric. As an independent cross-check we also ran a rule-based numeric match (any dollar figure within 12% of the verified price); it agreed with the grader on the overall order of magnitude (41.8% loose match vs 15.2% strict-correct).

Limitations. Some "wrong" verdicts reflect a billing-basis gap (a monthly figure vs our annual-equivalent) or a plan-scope gap (the model quotes a mid-tier when a cheaper tier exists); we count these as misses because a user would be misled either way. Prices change constantly, so this snapshot reflects mid-August 2026. Grading with a single model introduces some noise; we spot-checked a sample of verdicts by hand and found them consistent.