Methodology

This page explains where our numbers come from, how they are checked, and how an article gets from data to publication. Each article also ends with its own methodology section listing the exact queries, packages, time windows and collection date it used.

Data sources

We use only public data that anyone can retrieve.

• Hacker News stories, from the Hacker News Algolia search API. We count stories, not comments.
• npm download counts, from the npm download counts API.
• Wikipedia page views, from the Wikimedia Pageviews API, English Wikipedia, daily views from traffic not classified as automated. Used for home and living and for work and career.

How data is collected

Code collects the data on a fixed date and stores it as a dataset for that article. For Hacker News, each query is an exact phrase that must appear in a story title, with typo tolerance turned off, and code re-checks each title for the full phrase. Results are de-duplicated. For npm, we request the download count for each listed package over the same time windows. For Wikipedia, we collect a year of daily views for a reviewed list of articles in each field, plus the desktop and mobile split of the latest 30 days. Code leaves out an article when its typical daily views are too low to compare, when its traffic is almost all desktop or almost all mobile (a common sign of automated traffic), or when it spiked in the latest 30 days. A spike can come from a news event, and we do not write about a topic the day it trends.

Our articles so far compare a current 30-day window with the previous 30 days. The exact dates are stated in each article.

Original metrics

Every article includes at least one metric we calculate ourselves by combining two independent measurements, for example a tool’s share of Hacker News stories divided by its share of npm downloads, or a topic’s Wikipedia views against the same 30 days a year earlier and against its share of attention within its field. Restating a single public number does not count as original data. The calculation for each metric is described in the article that uses it.

How numbers are checked

Every number in a draft is linked to a value in the article’s dataset. Before an article can go further, code checks that each number matches its dataset value, that no untagged data numbers appear, and that all links come from the list of sources actually collected. A draft that fails this check is not published.

We also avoid words such as “doubled”, “fastest” or “winning” as evidence. We state the measured number instead.

The small-count rule

When a count is below 10 in either of the periods being compared, we report the raw count only. We do not calculate a percentage change, a share or a ratio from it, and we do not use it in rankings or headlines. With counts that small, a single story can swing a percentage widely. Leaving a tool out of a comparison for this reason says nothing about the tool itself.

Three reviews

Each draft is reviewed three ways. The model that drafted the article never reviews it.

1. Fact check (code): the number matching, link and disclosure checks described above.
2. Policy and claims review (a second AI model, OpenAI Codex): each qualitative claim is marked as supported, insufficient or contradicted against the dataset. Any unsupported claim, missing disclosure, or uneven treatment of the compared tools fails the draft.
3. Quality review (a third AI model, GLM): scores reader value, specificity, honesty about limits, the author’s input, structure and English quality against a fixed rubric. A draft must reach a minimum score to pass.

Two human approvals

A person approves each article twice.

1. Topic approval. Before anything is written, the author reviews the proposed topic, including the scores of every candidate topic, and writes an observation that is quoted word for word in the article.
2. Publication approval. After a draft passes all three reviews, the author reads the full article as it will appear and records at least one check made or change requested. Without that approval, nothing is published.

We work on one article at a time, so the pace is set by how much the author can review carefully.

What our numbers do not show

• Correlation is not causation. When two numbers move together, or in opposite directions, that does not show that one caused the other.
• Downloads are not users. npm is one install channel among several, and its counts include automated installs, CI systems and mirrors. We never present npm downloads as total adoption or rank tools by popularity from them.
• Page views are attention, not use. A Wikipedia page view says someone looked something up. It does not say they bought it, used it or liked it. Views of a redirect title are counted separately from the article it points to, and some automated traffic is still counted.
• Story counts are not all coverage. Title search misses stories that do not name the product in the title, and Hacker News reflects one specific audience.
• Counts are not sentiment. A story count does not say whether coverage was positive or negative.

Each article states the limits that apply to its own data.

AI use and conflicts of interest

Articles are drafted with Claude, made by Anthropic. Anthropic also makes Claude Code, which appears in some of our comparisons. Every article carries a disclosure saying this. More on who does what is on the About page.

Corrections

If you report an error, we check it against the article’s dataset and sources. If the error is confirmed, we fix the article and add a dated correction note at the end that says what was changed. We do not silently edit published numbers. To report an error, use the Contact page.