Verification experiments
These seven experiments compare the listed affected and corrected immutable states or published artifacts. Each result reports only its own observed outcome; it does not establish that an entire historical suite passed.
The experiment records are also available in the machine-readable dataset.
The complete historical experiment archive preserves the exact environments, dependency locks, reproducers, and logs at a fixed commit. Read the reproduction instructions and dependency security policy before rerunning an experiment.
The extraction preserved the source files unchanged. Four environment files already differed from the original recorded hashes; the migration notes document those differences. The extraction did not rerun the historical experiments.
Issue #1617: demonstrated detection
The same reserved-word assertion distinguishes corrected and affected immutable states.
- Affected state
- 8ff2046f6e7ddfd2f58d75c600de0fcc2053ab09 (v2.0.0)
- Corrected state
- 59f74e1cf2374290b0499567bbe5ff9107cd011b (PR #1622 merge; first public correction v2.5.0)
- Test provenance
- Contemporaneous operation-type-coder reserved-word assertion from #1622; audit copy on affected state.
- Runtime
- Node v26.5.1; affected Jest 30.2.0; corrected Jest30.3.0. Affected dependencies installed with Yarn4.18.0, corrected with Yarn1.22.22.
Commands
affected setup: copy archived environments/1617/{package.json,yarn.lock,.yarnrc.yml} into detached affected checkout; npx --yes @yarnpkg/[email protected] install --immutable --mode=skip-buildcorrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.test.js --watchman=false --coverage=false --runInBand --testNamePattern='reserved word'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='reserved word'
Recorded observations
- logs/issue-1617-corrected.log: 3/3 reserved-word assertions passed.
- logs/issue-1617-affected.log: 3/3 failed: delete_ expected/delete received; import_ expected/import received; delete_ expected/delete!! received.
Issue #1618: demonstrated detection
The same no-schema assertion distinguishes corrected and affected immutable states.
- Affected state
- 1db0b22de2998f1abf84e536e74da624840ee0fc (v0.13.0)
- Corrected state
- 8587018a0daa9f93b934b6e0060023b954c4fb91 (PR #1621 merge; first public correction v2.5.0); corrected execution ref: 59f74e1cf2374290b0499567bbe5ff9107cd011b (first installed state containing the unchanged assertion)
- Test provenance
- Contemporaneous no-schema response assertion from #1621; audit copy on affected state.
- Runtime
- Node v26.5.1; affected Jest 28.1.2; corrected Jest30.3.0. Affected dependencies installed with Yarn4.18.0, corrected with Yarn1.22.22.
Commands
affected setup: copy archived environments/1618/{package.json,yarn.lock,.yarnrc.yml} into detached affected checkout; npx --yes @yarnpkg/[email protected] install --immutable --mode=skip-buildcorrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.test.js --watchman=false --coverage=false --runInBand --testNamePattern='no schema'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='no schema'
Recorded observations
- logs/issue-1618-corrected-containing-release.log: 1/1 assertion passed.
- logs/issue-1618-affected.log: TypeError reading isReference from an absent schema requirement.
Issue #1619: demonstrated detection
The broad v1.4.5 bound is retained; the pair isolates the defect on the immediate pre-merge state.
- Affected state
- 0c081caf351ad5c378b1da9a48a54130abf1fe62 (v1.4.5 earliest confirmed affected bound); paired immediate first parent: 241d6284ac1424669c179ca4538786b97aa11539 (v2.4.0)
- Corrected state
- 80dc0732ecb4e6e664dbedc29d1ceac7ce49a366 (PR #1620 merge; checkout package version v2.4.0; first public correction v2.5.0)
- Test provenance
- Contemporaneous script.test.js assertion from #1620, copied unchanged to script.audit.test.js; SHA-256 da594a4b4cbb306882a8cec60cb2043a162f75de79288a6f2eae154a63981684.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.4.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/script.test.js --watchman=false --coverage=false --runInBand --testNamePattern='escapes colons in import paths'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/script.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='escapes colons in import paths'
Recorded observations
- logs/issue-1619-corrected.log: 1 passed, 9 skipped, exit 0.
- logs/issue-1619-affected.log: 1 failed, 9 skipped, exit 1; expected Unicode ratio colon and received ASCII colon in import path.
Issue #1842: demonstrated detection
Path-item summary and description were treated as operations before the correction.
- Affected state
- f7b52edf77bfecc3b841f1fc33918fb29b483934 (v0.5.0 earliest confirmed affected bound); paired immediate first parent: 60a320e5c9673d5fc0cfd5ec351891869b6f1eb1 (v2.7.0)
- Corrected state
- af406dd6a4709d00fbae8b757ff43eb10ddd4bab (PR #1843 merge; checkout package version v2.7.0; first public correction v2.8.1)
- Test provenance
- Contemporaneous generate.test.ts assertion from #1843, copied unchanged to generate.audit.test.ts; SHA-256 6d3d2dc0d13dcc8cee023195a361796a5f1d982f51cb1bc3f2fc129e02e129e9.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.7.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/generate.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='ignores summary and description'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/generate.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='ignores summary and description'
Recorded observations
- logs/issue-1842-corrected.log: 1 passed, 6 skipped, exit 0.
- logs/issue-1842-affected.log: 1 failed, 6 skipped, exit 1; TypeError: Cannot use 'in' operator to search for 'operationId' in Test Summary.
Issue #1933: demonstrated detection
The pair shows both rejected exploded-object requests and absent reconstructed handler input.
- Affected state
- 3ad1e91161dae681ac208b314a3756d33d555096 (v0.13.1 earliest confirmed affected bound); paired immediate first parent: a67ccaf1f3f2067210fe67c3d8ffd9876036300a (v2.9.0)
- Corrected state
- 2997e43c12df3d4214d2302183b618d772c452e3 (PR #1935 merge; checkout package version v2.9.0; first public correction v2.10.0)
- Test provenance
- Contemporaneous dispatcher.test.ts assertions from #1935, copied unchanged to dispatcher.audit.test.ts; SHA-256 5820cff72e416f154303f1fe96880d547005a209517f427a93492a6de95f52e2.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.9.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='exploded object query parameters'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='exploded object query parameters'
Recorded observations
- logs/issue-1933-corrected.log: 4 passed, 49 skipped, exit 0.
- logs/issue-1933-affected.log: 3 failed and 1 passed, 49 skipped, exit 1; exploded fields caused HTTP 400 and handlers did not receive pageable.
Issue #1971: demonstrated detection
The pair distinguishes missing path-item-level parameter merging from corrected conversion.
- Affected state
- f7b52edf77bfecc3b841f1fc33918fb29b483934 (v0.5.0 earliest confirmed affected bound); paired immediate first parent: fd9b7dcb179b45c465fc438244272953ba1d5c8b (v2.10.0)
- Corrected state
- 695c1ca39d90d8cc26a8e5448f3209e81ef08e41 (PR #1972 merge; checkout package version v2.10.0; first public correction v2.11.0)
- Test provenance
- Contemporaneous dispatcher.test.ts assertions from #1972, copied unchanged to dispatcher.audit.test.ts; SHA-256 266ad9faed8cd7a2a45bb8cbafaf660421dd53c377f6743622638d506b47251c.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.10.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='converts path-level parameters to numbers when defined at path item level|merges path-item-level and operation-level parameters'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='converts path-level parameters to numbers when defined at path item level|merges path-item-level and operation-level parameters'
Recorded observations
- logs/issue-1971-corrected.log: 2 passed, 59 skipped, exit 0.
- logs/issue-1971-affected.log: 2 failed, 59 skipped, exit 1; intId and pathId remained strings rather than numbers.
Issue #2075: demonstrated detection
The observed artifact-level defect is the unnecessary patch lifecycle: the affected published package invokes it while publishing no patches; the corrected published package does not. This does not claim that 2.11.0 installation exited unsuccessfully.
- Affected state
- 1320d69f8a6719ea9ebb81ffeed328376ce4443f (v0.39.0 earliest confirmed affected bound); paired published artifact: [email protected]
- Corrected state
- 1cd24bdf1d965760b527f8b31f73523995126a8c (PR #2076 merge; checkout package version v2.11.0; first public correction v2.12.0); paired published artifact: [email protected]
- Test provenance
- Audit-created published-artifact lifecycle assertion: npm tarballs and separate clean consumer directories, because #2076 contains no contemporaneous packed-artifact test.
- Runtime
- Node v26.5.1; npm 11.17.0. Tarball SHA-256: 2.11.0 432a6484fa46ec6a4c02da1c319935d63f9fa4c43e654fd1974c73a527514fe6; 2.12.0 426c40f1ebc9c19a7cf2f6d1ddc0febd71a26803220f08fa09ac5b1c09c25e12.
Commands
npm pack [email protected] --pack-destination /private/tmp/counterfact-exp-2075-packsnpm pack [email protected] --pack-destination /private/tmp/counterfact-exp-2075-packsnpm install --foreground-scripts --loglevel=notice --prefix <clean-2.11-dir> /private/tmp/counterfact-exp-2075-packs/counterfact-2.11.0.tgznpm install --foreground-scripts --loglevel=notice --prefix <clean-2.12-dir> /private/tmp/counterfact-exp-2075-packs/counterfact-2.12.0.tgzaudit rerun: npm install --offline --foreground-scripts --loglevel=notice --prefix <new-clean-dir> <counterfact-2.11.0-or-2.12.0.tgz>; both exit0, actual outputs retained in logs/issue-2075-<version>-install.log
Recorded observations
- logs/issue-2075-2.11.0-package.json and logs/issue-2075-assertion.log: 2.11.0 declares postinstall: patch-package, depends on patch-package 8.0.1, and publishes no patch files.
- logs/issue-2075-2.12.0-package.json and logs/issue-2075-assertion.log: 2.12.0 declares neither postinstall nor patch-package.
- The separately observed clean installs ran the 2.11.0 hook and logged No patch files found; both npm installs exited 0. Both also emitted an unrelated optional fsevents node-gyp diagnostic under Node 26.