Methodology v3.1
How we rate 9,486+ software tools
Four weighted criteria, one named editor, and a public worked example so you can see exactly how a score gets built. This is the version of the methodology you can hold us to.
The short version
- 1.Every tool gets an internal score out of 100 built from four weighted criteria: editorial review, external reviews, media mentions, profile completeness.
- 2.That score is a sort key and indexing gate, not a star rating. We never show /100 on tool pages because a private internal number with no transparent review backing it would be misleading.
- 3.What you do see on tool pages: the external rating aggregated from G2, Capterra, App Store, and Play Store, plus our written editor's take and the verified community reviews.
- 4.Sponsorships, paid placements, and vendor outreach never move the score. The list of things that do not influence ratings is in section 6.
1. Two scores you can see, one you cannot
Toolradar surfaces two numbers on tool pages: an external rating aggregated from public review sites, and a community rating from verified Toolradar reviews. Behind the scenes there is also an internal editorial score we use to sort lists and gate which tools are eligible to appear in guides. That third number is intentionally private.
External rating
The number shown on every tool page. Aggregated from G2, Capterra, Trustpilot, the Apple App Store, the Google Play Store, and the Chrome Web Store where available. We show the source and review count next to it so you can verify.
- Backed by a public review corpus
- Source named on the tool page
Community rating
Calculated from verified Toolradar reviews. Each review is email-confirmed, manually moderated, and flagged if it shows the patterns we associate with review bombing or vendor astroturf.
- Verified by email plus moderation
- Updates as new reviews land
Internal editorial score (not shown on tool pages)
A 0 to 100 ranking input we use to decide which tools sort highest in a category, which ones qualify for the editor's top picks in a guide, and which ones are deindexed for thin coverage. Showing it on tool pages would imply a precision we cannot defend without a public review corpus behind every digit. So we keep it backstage and use the external rating in the UI instead.
2. The four weighted criteria
The internal editorial score is built from four criteria with fixed weights. Each is scored 0 to 100 from observable evidence, then combined.
Editorial review
35%Hands-on or documentation-grounded judgment of the tool itself: depth of features, quality of onboarding, pricing fairness, security posture, and documentation. The editor scores each subfactor and the weighted average becomes this leg of the score.
Subfactor breakdown: Features (30%), UX and onboarding (25%), Value and pricing (20%), Support and docs (15%), Security and reliability (10%).
External reviews
25%Aggregate of G2, Capterra, Trustpilot, Apple App Store, Google Play Store, and Chrome Web Store ratings where the tool is present. Volume-weighted: a 4.7 from 12 reviewers contributes less than a 4.4 from 4,000 reviewers. Sources with documented review-buying programs are excluded.
Media mentions
25%Coverage in the 431+ press, blog, and newsletter sources we track. Sources are tiered: tier 1 (TechCrunch, The Verge, Wired, NYT, etc.) is worth 30 points, tier 2 (Hacker News front page, established trade press) is worth 15, tier 3 (auto-discovered blogs and forums) is worth 5. Recent mentions are weighted more than historical ones. Currently 8,859 mentions logged across 431+ sources.
Profile completeness
15%Does the Toolradar profile have verified pricing, a written editor's take, structured pros and cons, at least three categories assigned, a working website, and a logo over 48x48? Incomplete profiles get less surface area because we cannot in good faith tell readers this is a fully vetted listing.
Score bands
Tools in the deindexed band are excluded from category sitemaps and guide consideration. They remain accessible via direct URL so the data we have stays public.
3. A worked example: Linear
Linear is an issue tracker for software teams. Here is how each weighted criterion reads against the same public inputs you can verify on its Toolradar profile. We keep the raw internal number backstage (see why above) and describe the evidence instead.
That ranking is what we use to decide Linear shows near the top of the issue tracker category and qualifies for the editor's top picks in the related guides. What you actually see on the Linear page is the external rating from G2, Capterra, and the App Store, with the source named next to it.
4. Review verification
The community rating shown on tool pages is computed only from reviews that pass every step below. We would rather have ten honest reviews than a thousand we cannot stand behind.
Email confirmation
Every reviewer must verify a working inbox before the review goes live.
Account age threshold
New accounts cannot review on day one. The minimum age catches the most obvious astroturf pattern.
Professional verification
A reviewer who confirms a company email domain gets a verified-buyer label next to their review.
Manual moderation
Every approved review has been read by the editor or a designated moderator before publication.
Pattern detection
Bursts of one-star reviews around a release, identical wording across accounts, and IP clustering all get flagged.
Reader reports
Any reader can flag a review. Flagged reviews get re-moderated within 48 hours, the editor signs off on the outcome.
5. How often things get refreshed
Weekly for the top 500 tools by traffic, otherwise on every guide rebuild or when a vendor flags a change.
Daily ingest from 431+ tracked sources, scored at write time, then decayed weekly so recent coverage outweighs old hits.
Re-pulled monthly from G2, Capterra, and the app stores. We do not cache stale stars.
Refreshed when a vendor ships a major release, when reader pushback forces a rethink, or on the annual category sweep.
Reviewed quarterly. 401 categories live today, pruned when overlap gets confusing for readers.
This page is reviewed every quarter. Material changes get a version bump at the top and a note in the changelog.
6. What does not influence a score
- ×Sponsorship in the Dupple newsletter network. A tool sponsoring Techpresso or any other Dupple newsletter does not get a score bump. We disclose the overlap on the relevant guide section but the number stays put.
- ×Paid placements. Sponsored slots on Toolradar are always labeled and sit outside the editorial ranking. They do not change which tools appear in the editor's top picks.
- ×Vendor outreach. Emails asking us to "take another look" or PRs pitching a new feature do not move a score. We use those as input for the next editor pass and that is it.
- ×Affiliate revenue. Some outbound links are tracked for analytics. None of them pay us per click in a way that would shift a score. Our full revenue breakdown lives at /how-we-make-money.
- ×Social media noise. A viral tweet does not produce a score. A measurable spike in tier-1 press coverage does, through the media-mentions criterion, on its own merits.
- ×Editor sympathy. If the editor likes a vendor personally, that does not move the number. The criteria are the criteria. Personal opinion goes in the editor's take where readers can see it and argue with it.
7. Methodology changelog
Added the worked example, the deindex threshold for thin coverage, and the explicit statement that the internal score is never shown on tool pages. Score weights unchanged.
Introduced the Media mentions criterion (25%) backed by the 431+ tracked source network. Editorial review weight reduced from 50% to 35%.
Split community rating from external rating. Added profile completeness so tools with thin data stop appearing in guide top picks.
Toolradar launches with the five-subfactor editorial rubric (Features, UX, Value, Support, Security).
Think a score is wrong? Tell us why.
Methodology disputes go to toolradar@dupple.com. Bring the URL plus a primary source or a concrete counter-example. We re-review and document the outcome.