Verification experiments
These seven experiments compare the listed affected and corrected immutable states or published artifacts. Each result reports only its own observed outcome; it does not establish that an entire historical suite passed.
The experiment records are also available in the machine-readable dataset.
Issue #1617: demonstrated detection
The same reserved-word assertion distinguishes corrected and affected immutable states.
- Affected state
- 8ff2046f6e7ddfd2f58d75c600de0fcc2053ab09 (v2.0.0)
- Corrected state
- 59f74e1cf2374290b0499567bbe5ff9107cd011b (PR #1622 merge; first public correction v2.5.0)
- Test provenance
- Contemporaneous operation-type-coder reserved-word assertion from #1622; audit copy on affected state.
- Runtime
- Node v26.5.1; affected Jest 30.2.0; corrected Jest30.3.0. Affected dependencies installed with Yarn4.18.0, corrected with Yarn1.22.22.
Commands
affected setup: copy archived environments/1617/{package.json,yarn.lock,.yarnrc.yml} into detached affected checkout; npx --yes @yarnpkg/[email protected] install --immutable --mode=skip-buildcorrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.test.js --watchman=false --coverage=false --runInBand --testNamePattern='reserved word'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='reserved word'
Recorded observations
- logs/issue-1617-corrected.log: 3/3 reserved-word assertions passed.
- logs/issue-1617-affected.log: 3/3 failed: delete_ expected/delete received; import_ expected/import received; delete_ expected/delete!! received.
Issue #1618: demonstrated detection
The same no-schema assertion distinguishes corrected and affected immutable states.
- Affected state
- 1db0b22de2998f1abf84e536e74da624840ee0fc (v0.13.0)
- Corrected state
- 8587018a0daa9f93b934b6e0060023b954c4fb91 (PR #1621 merge; first public correction v2.5.0); corrected execution ref: 59f74e1cf2374290b0499567bbe5ff9107cd011b (first installed state containing the unchanged assertion)
- Test provenance
- Contemporaneous no-schema response assertion from #1621; audit copy on affected state.
- Runtime
- Node v26.5.1; affected Jest 28.1.2; corrected Jest30.3.0. Affected dependencies installed with Yarn4.18.0, corrected with Yarn1.22.22.
Commands
affected setup: copy archived environments/1618/{package.json,yarn.lock,.yarnrc.yml} into detached affected checkout; npx --yes @yarnpkg/[email protected] install --immutable --mode=skip-buildcorrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.test.js --watchman=false --coverage=false --runInBand --testNamePattern='no schema'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/operation-type-coder.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='no schema'
Recorded observations
- logs/issue-1618-corrected-containing-release.log: 1/1 assertion passed.
- logs/issue-1618-affected.log: TypeError reading isReference from an absent schema requirement.
Issue #1619: demonstrated detection
The broad v1.4.5 bound is retained; the pair isolates the defect on the immediate pre-merge state.
- Affected state
- 0c081caf351ad5c378b1da9a48a54130abf1fe62 (v1.4.5 earliest confirmed affected bound); paired immediate first parent: 241d6284ac1424669c179ca4538786b97aa11539 (v2.4.0)
- Corrected state
- 80dc0732ecb4e6e664dbedc29d1ceac7ce49a366 (PR #1620 merge; checkout package version v2.4.0; first public correction v2.5.0)
- Test provenance
- Contemporaneous script.test.js assertion from #1620, copied unchanged to script.audit.test.js; SHA-256 da594a4b4cbb306882a8cec60cb2043a162f75de79288a6f2eae154a63981684.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.4.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/script.test.js --watchman=false --coverage=false --runInBand --testNamePattern='escapes colons in import paths'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/script.audit.test.js --watchman=false --coverage=false --runInBand --testNamePattern='escapes colons in import paths'
Recorded observations
- logs/issue-1619-corrected.log: 1 passed, 9 skipped, exit 0.
- logs/issue-1619-affected.log: 1 failed, 9 skipped, exit 1; expected Unicode ratio colon and received ASCII colon in import path.
Issue #1842: demonstrated detection
Path-item summary and description were treated as operations before the correction.
- Affected state
- f7b52edf77bfecc3b841f1fc33918fb29b483934 (v0.5.0 earliest confirmed affected bound); paired immediate first parent: 60a320e5c9673d5fc0cfd5ec351891869b6f1eb1 (v2.7.0)
- Corrected state
- af406dd6a4709d00fbae8b757ff43eb10ddd4bab (PR #1843 merge; checkout package version v2.7.0; first public correction v2.8.1)
- Test provenance
- Contemporaneous generate.test.ts assertion from #1843, copied unchanged to generate.audit.test.ts; SHA-256 6d3d2dc0d13dcc8cee023195a361796a5f1d982f51cb1bc3f2fc129e02e129e9.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.7.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/generate.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='ignores summary and description'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/typescript-generator/generate.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='ignores summary and description'
Recorded observations
- logs/issue-1842-corrected.log: 1 passed, 6 skipped, exit 0.
- logs/issue-1842-affected.log: 1 failed, 6 skipped, exit 1; TypeError: Cannot use 'in' operator to search for 'operationId' in Test Summary.
Issue #1933: demonstrated detection
The pair shows both rejected exploded-object requests and absent reconstructed handler input.
- Affected state
- 3ad1e91161dae681ac208b314a3756d33d555096 (v0.13.1 earliest confirmed affected bound); paired immediate first parent: a67ccaf1f3f2067210fe67c3d8ffd9876036300a (v2.9.0)
- Corrected state
- 2997e43c12df3d4214d2302183b618d772c452e3 (PR #1935 merge; checkout package version v2.9.0; first public correction v2.10.0)
- Test provenance
- Contemporaneous dispatcher.test.ts assertions from #1935, copied unchanged to dispatcher.audit.test.ts; SHA-256 5820cff72e416f154303f1fe96880d547005a209517f427a93492a6de95f52e2.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.9.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='exploded object query parameters'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='exploded object query parameters'
Recorded observations
- logs/issue-1933-corrected.log: 4 passed, 49 skipped, exit 0.
- logs/issue-1933-affected.log: 3 failed and 1 passed, 49 skipped, exit 1; exploded fields caused HTTP 400 and handlers did not receive pageable.
Issue #1971: demonstrated detection
The pair distinguishes missing path-item-level parameter merging from corrected conversion.
- Affected state
- f7b52edf77bfecc3b841f1fc33918fb29b483934 (v0.5.0 earliest confirmed affected bound); paired immediate first parent: fd9b7dcb179b45c465fc438244272953ba1d5c8b (v2.10.0)
- Corrected state
- 695c1ca39d90d8cc26a8e5448f3209e81ef08e41 (PR #1972 merge; checkout package version v2.10.0; first public correction v2.11.0)
- Test provenance
- Contemporaneous dispatcher.test.ts assertions from #1972, copied unchanged to dispatcher.audit.test.ts; SHA-256 266ad9faed8cd7a2a45bb8cbafaf660421dd53c377f6743622638d506b47251c.
- Runtime
- Node v26.5.1; Jest 30.3.0 in both v2.10.0 checkouts.
Commands
corrected: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='converts path-level parameters to numbers when defined at path item level|merges path-item-level and operation-level parameters'affected audit copy: NODE_OPTIONS=--experimental-vm-modules node ./node_modules/jest/bin/jest.js test/server/dispatcher.audit.test.ts --watchman=false --coverage=false --runInBand --testNamePattern='converts path-level parameters to numbers when defined at path item level|merges path-item-level and operation-level parameters'
Recorded observations
- logs/issue-1971-corrected.log: 2 passed, 59 skipped, exit 0.
- logs/issue-1971-affected.log: 2 failed, 59 skipped, exit 1; intId and pathId remained strings rather than numbers.
Issue #2075: demonstrated detection
The observed artifact-level defect is the unnecessary patch lifecycle: the affected published package invokes it while publishing no patches; the corrected published package does not. This does not claim that 2.11.0 installation exited unsuccessfully.
- Affected state
- 1320d69f8a6719ea9ebb81ffeed328376ce4443f (v0.39.0 earliest confirmed affected bound); paired published artifact: [email protected]
- Corrected state
- 1cd24bdf1d965760b527f8b31f73523995126a8c (PR #2076 merge; checkout package version v2.11.0; first public correction v2.12.0); paired published artifact: [email protected]
- Test provenance
- Audit-created published-artifact lifecycle assertion: npm tarballs and separate clean consumer directories, because #2076 contains no contemporaneous packed-artifact test.
- Runtime
- Node v26.5.1; npm 11.17.0. Tarball SHA-256: 2.11.0 432a6484fa46ec6a4c02da1c319935d63f9fa4c43e654fd1974c73a527514fe6; 2.12.0 426c40f1ebc9c19a7cf2f6d1ddc0febd71a26803220f08fa09ac5b1c09c25e12.
Commands
npm pack [email protected] --pack-destination /private/tmp/counterfact-exp-2075-packsnpm pack [email protected] --pack-destination /private/tmp/counterfact-exp-2075-packsnpm install --foreground-scripts --loglevel=notice --prefix <clean-2.11-dir> /private/tmp/counterfact-exp-2075-packs/counterfact-2.11.0.tgznpm install --foreground-scripts --loglevel=notice --prefix <clean-2.12-dir> /private/tmp/counterfact-exp-2075-packs/counterfact-2.12.0.tgzaudit rerun: npm install --offline --foreground-scripts --loglevel=notice --prefix <new-clean-dir> <counterfact-2.11.0-or-2.12.0.tgz>; both exit0, actual outputs retained in logs/issue-2075-<version>-install.log
Recorded observations
- logs/issue-2075-2.11.0-package.json and logs/issue-2075-assertion.log: 2.11.0 declares postinstall: patch-package, depends on patch-package 8.0.1, and publishes no patch files.
- logs/issue-2075-2.12.0-package.json and logs/issue-2075-assertion.log: 2.12.0 declares neither postinstall nor patch-package.
- The separately observed clean installs ran the 2.11.0 hook and logged No patch files found; both npm installs exited 0. Both also emitted an unrelated optional fsevents node-gyp diagnostic under Node 26.