How we measure CSS conformance on the web-platform-tests
We changed the unit behind our conformance headline: a reftest counts as one test, a scripted file counts as its subtests. Both scales, same dataset.
, updated , a 5-minute read
Until 11 September 2026 the conformance headline for our engine counted reftests only. On 11 September it moved to the units wpt.fyi uses, and on those it first read lower: the denominator grew and got harder.
The Rust post published the old headline at 89.48% on 27 August and at 94.40% on 4 September. On the new units a scripted file counts as its subtests instead of as one unit, anchor positioning came back into the denominator, and a tier judging computed values started reporting. In the run of 8 November 2026 the engine passes 94.6% of subtests, and 94.4% of files pass with every subtest green, the figure /conformance puts first. The reftest share, which the old headline measured, read 95.2% at the end of 11 September and reads 95.3% in the same run.
Why we changed the count
A subtest is one test() or async_test() reported by the web-platform-tests harness, and one subtest can hold many assertions. wpt.fyi, the public dashboard of browser runs of those tests, counts a reftest (a test that renders a page, renders a reference and compares them) as one unit, and a scripted file as its subtests. Until 11 September we counted reftests only.
In this run, css/CSS2/normal-flow/auto-margins-used-values-with-floats.tentative.html states 12 expected values and 8 of them match. Counted as one file it is a single failure, and those 8 matches would be recorded nowhere.
Our corpus holds 27,211 reftest units and 57,980 units in total. The published conformance numbers we know of, wpt.fyi and Servo’s (the browser engine written in Rust), count the harness unit, and a number on another scale cannot be laid beside them.
A print engine has a second reason. 1,476 files are not judged: 449 scripted reftests whose script changes the page before the comparison, 978 left out by the rule for unsettled drafts, 30 that need interaction or the network, and 19 for other reasons. On the old scale those files were absent from both halves of the fraction. They are now counted, named and shown with the reason: the third headline on the conformance page treats every unjudged file as a failure and stands at 89.9%.
What the new scale shows
Coverage against wpt.fyi. Of the 34,511 tests wpt.fyi records under css/, our corpus judges 27,381, which is 79.3%; of its 218,751 subtests we judge 55,967, which is 25.6%. The snapshot is of Chrome 155.0.8052.0, taken on 11 September 2026. The two totals differ by 2,013 units, because our corpus reaches outside css/ (html/rendering, html/semantics, mathml, svg and compat) and the snapshot covers css/ only.
Two additions to the denominator. Anchor positioning had been excluded as unshipped and came back: 223 of 285 tests and 563 of 650 subtests in this dataset. A tier, one family of tests with a unit of its own, began judging computed values, meaning getComputedStyle serialization as specified in CSSOM §6.7.2: 357 files, 3,375 of 3,928 subtests passing, and 382 subtests set aside because the serializer for those properties is not written yet. Both areas score below the corpus average.
Where the figures come from. Every number on the conformance page is read from the dataset built after the last accepted run (this page reads the run of 8 November 2026), and each test links to its own evidence: our render, the reference, and the pixels that differ.
The numbers as they stand
| Scale | Unit | Result | Dataset |
|---|---|---|---|
| Reftests only, as first published | one reftest = one unit | 89.48% | 27 August 2026 |
| Reftests only, as updated | one reftest = one unit | 94.40% | 4 September 2026 |
| Reftests only, run of 8 November 2026 | one reftest = one unit | 95.3% | 8 November 2026 |
| Files, the first headline of /conformance | a file passes when every subtest in it passes | 94.4% | 8 November 2026 |
| Subtests, as wpt.fyi counts | a reftest is one, a scripted file is its subtests | 94.6% | 8 November 2026 |
| Score, as Servo defines it | per test, the share of its subtests that pass (a reftest scores 0 or 1); the mean over all tests | 94.94 | 8 November 2026 |
| Every file, the unjudged counted as failing | one file = one unit | 89.9% | 8 November 2026 |
Where the units come from: the four tiers
| Tier | What it judges | Units | Passing |
|---|---|---|---|
| Reftests | rendering, against the test’s own reference | 27,211 | 95.3% |
| Layout assertions | element positions, one unit per element checked by check-layout-th.js | 17,094 | 96.1% |
| Parsing acceptance | which declarations survive the parser, one unit per parsing-testcommon.js assertion | 9,747 | 93.6% |
| Computed values | getComputedStyle serialization, CSSOM §6.7.2 | 3,928 | 85.9% |
Reftests, the part the old number measured, stand at 95.3% on this dataset, against 94.40% on 4 September. The conformance page charts the last run of each day under “Run by run”.
Print modules are never excused. A failing test leaves the denominator only when fewer than three of the four browsers pass it and its specification is below Candidate Recommendation, the W3C stage at which a specification is considered stable. That rule never applies to print modules: css-break, css-page and css-multicol decide where a page ends, which is the part of CSS a PDF engine exists to get right.
What moves the number next. Of the files we do not judge, 978 wait on their specifications to settle; the 449 scripted reftests wait on us. The engine runs a document’s scripts when it prints (the developer API’s javascript option), but the harness renders these files without them; joining the two is the next step.
Questions
- Why did the pass rate go down?
- Three changes to the denominator landed on 11 September 2026: a scripted file counts as its subtests rather than as one unit, anchor positioning returned to the denominator after having been excluded, and a tier judging computed values started reporting. Rendering did not get worse: the reftest share, which is what the old headline measured, read 94.40% on 4 September, and the post prints it for the current run beside the new figures.
- How is a subtest counted?
- A subtest is one test() or async_test() reported by the Web Platform Tests harness, and one subtest can contain many assertions. The layout tier counts one unit per element checked by check-layout-th.js, and the parsing tier one unit per assertion in parsing-testcommon.js, because those are the units wpt.fyi records for those files.
- Do you drop tests you fail?
- A test we pass is always judged. A test we fail leaves the denominator only under three conditions at once: fewer than three of the four browsers pass it, its specification is below Candidate Recommendation, and it is not a print module. The rule and the counts are published in the methodology.
- Can I check a single test myself?
- Every test has a page with our render, the test’s own reference and the pixels that differ, and the test source opens in the playground. A scripted file’s page lists each of its assertions, with the expected and the actual value.
Every test behind these numbers has its own page: our render, the reference and the pixels that differ.
More from the blog
Our PDF engine is now written in Rust, and measured on the web-platform-tests
Why we rewrote our HTML-to-PDF engine in Rust: the benchmark and its method, web-platform-tests results by CSS area, and what a PDF/UA validator misses., a 16-minute readPDF/UA-2 (ISO 14289-2): what changes, and how we verify it
PDF/UA-2 is written on PDF 2.0. Five changes from PDF/UA-1, how to check each, and what veraPDF does not check., a 9-minute readHow to edit a PDF on an iPhone: what is built in, and what takes an editor
An iPhone signs, annotates and fills PDFs out of the box, but cannot change the text already in them. What each built-in tool does, and how to change the text in the browser., a 2-minute read