Supporting delivery and response evidence
These retrospective comparisons support the study question. They describe the observed delivery and correction record; they do not estimate a counterfactual defect total or establish a causal effect of coding agents.
Mature intake cohorts
Each intake window is recomputed from the mature-record, delivery, and window records. An unresolved chronology is displayed as incomplete rather than counted as zero.
| Intake window | Eligible event records | Reports observed for response | Non-dependency merged PRs | Published releases | Response record |
|---|---|---|---|---|---|
| 20222022-03-10 to 2022-06-06 | 0 | 0 | 0Rate not reported | 0Rate not reported | 0 corrected within follow-up; 0 censored |
| 20232023-03-10 to 2023-06-06 | Unresolved (0 confirmed; 3 origins pending) | 0 | 8Rate not reported | 3Rate not reported | 0 corrected within follow-up; 0 censoredIncomplete: issue-453, issue-487, issue-530 |
| 20242024-03-10 to 2024-06-06 | 3 | 7 | 575.26 per 100; exact 95% interval 1.09–15.38 | 1915.79 per 100; exact 95% interval 3.26–46.14 | 7 corrected within follow-up; 0 censored |
| 20252025-03-10 to 2025-06-06 | 0 | 1 | 80.00 per 100; exact 95% interval 0.00–46.11 | 20.00 per 100; exact 95% interval 0.00–184.44 | 0 corrected within follow-up; 1 censored |
| 20262026-03-10 to 2026-06-06 | 0 | 7 | 2350.00 per 100; exact 95% interval 0.00–1.57 | 140.00 per 100; exact 95% interval 0.00–26.35 | 7 corrected within follow-up; 0 censored |
The delivery denominators above come from merged pull-request and npm-publication records within each stated window. Reported rates and exact Poisson intervals describe those observed counts only; they are not a reliability model or a causal estimate. Incomplete cohorts and zero denominators suppress rate display. The full source, including classification rules, is in Download data.
Matched five-year report records
This matched-window view derives report IDs from mature records whose report timestamps fall inside each delivery window.
| Window | Reported mature records |
|---|---|
| matched-20222022-03-10 to 2022-09-04 | 0No report IDs |
| matched-20232023-03-10 to 2023-09-04 | 4issue-453, issue-487, issue-491, issue-530 |
| matched-20242024-03-10 to 2024-09-04 | 7issue-820, issue-828, issue-835, issue-866, issue-890, issue-906, issue-918 |
| matched-20252025-03-10 to 2025-09-04 | 3issue-1244, issue-1370, issue-1381 |
| matched-20262026-03-10 to 2026-09-04 | 7issue-1617, issue-1618, issue-1619, issue-1842, issue-1933, issue-1971, issue-2075 |
Pre- and post-adoption delivery record
The pre-adoption interval runs from September 1, 2024 to the recorded adoption timestamp; the post-adoption interval runs from that timestamp to September 4, 2026. The durations differ, so this table reports counts only and makes no rate or effect claim.
| Period | Non-dependency merged PRs | Published releases | Released reports by discovery channel | Pre-release reports | Unknown release status |
|---|---|---|---|---|---|
| preAdoption2024-09-01T00:00:00Z to 2026-03-10T16:38:20Z | 86 | 26 | external human: 9; internal human: 0; automated agent: 0; unknown: 5 | 1 | 3 |
| postAdoption2026-03-10T16:38:20Z to 2026-09-04T00:00:00Z | 279 | 21 | external human: 7; internal human: 0; automated agent: 0; unknown: 10 | 2 | 4 |
Response observations
A response observation records the report date, the 90-day follow-up deadline, and either a first corrected publication or the applicable censoring point. The deadline is a study boundary for comparable historical follow-up, not a repair objective, MTTR, or time-to-restore measure. It is separate from the eligible-event count because a report may be observable without satisfying that event definition.
mature-intake-2022: 0 response observations
No response observations are available for this intake window.
mature-intake-2023: 0 response observations
No response observations are available for this intake window.
mature-intake-2024: 7 response observations
| Record | Reported | 90-day deadline | Observed until | Status | Elapsed days |
|---|---|---|---|---|---|
| issue-820 | 2024-03-27 | 2024-06-25 | 2024-04-30 | corrected-within-followup | 34.49 |
| issue-828 | 2024-04-03 | 2024-07-02 | 2024-04-18 | corrected-within-followup | 15.13 |
| issue-835 | 2024-04-05 | 2024-07-04 | 2024-04-30 | corrected-within-followup | 25.02 |
| issue-866 | 2024-04-24 | 2024-07-23 | 2024-04-24 | corrected-within-followup | 0.27 |
| issue-890 | 2024-05-06 | 2024-08-04 | 2024-05-06 | corrected-within-followup | 0.27 |
| issue-906 | 2024-05-14 | 2024-08-12 | 2024-05-15 | corrected-within-followup | 1.18 |
| issue-918 | 2024-05-24 | 2024-08-22 | 2024-05-30 | corrected-within-followup | 6.58 |
mature-intake-2025: 1 response observation
| Record | Reported | 90-day deadline | Observed until | Status | Elapsed days |
|---|---|---|---|---|---|
| issue-1244 | 2025-03-28 | 2025-06-26 | 2025-06-26 | corrected-after-followup | 90.00 |
mature-intake-2026: 7 response observations
| Record | Reported | 90-day deadline | Observed until | Status | Elapsed days |
|---|---|---|---|---|---|
| issue-1617 | 2026-04-02 | 2026-07-01 | 2026-04-03 | corrected-within-followup | 1.69 |
| issue-1618 | 2026-04-02 | 2026-07-01 | 2026-04-03 | corrected-within-followup | 1.65 |
| issue-1619 | 2026-04-02 | 2026-07-01 | 2026-04-03 | corrected-within-followup | 1.64 |
| issue-1842 | 2026-04-13 | 2026-07-12 | 2026-04-14 | corrected-within-followup | 0.93 |
| issue-1933 | 2026-04-21 | 2026-07-20 | 2026-04-25 | corrected-within-followup | 3.38 |
| issue-1971 | 2026-04-30 | 2026-07-29 | 2026-05-05 | corrected-within-followup | 4.93 |
| issue-2075 | 2026-05-15 | 2026-08-13 | 2026-05-16 | corrected-within-followup | 1.09 |
Unresolved-record diagnostics
20 detailed records remain unresolved in the public-record census. They are not treated as confirmed defects, absent defects, or a zero response count.
- commit-17635fe6b5a55b8381f83eb4f81583d930ae02e3: Diff replaces unsupported Chokidar4 globs with directory watch. Parent upgrades dependency; both are committedOct2, despite older authored date. No independently documented discovery instant or affected-public-release established; remains unresolved and cannot be placed at an invented report time.
- commit-8a0483ecfd624227d19781a6005fab121de1f4b1: Diff restores extension guard disabled by preceding watch repair. No observed failure report or independently affected release established. Retained unresolved with both author and committer times; no invented discovery timestamp.
- pr-2362: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2335: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2316: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2260: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2258: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2252: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2250: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-2102: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1804: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1665: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1651: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1581: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1571: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1470: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1296: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1240: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1139: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
- pr-1081: Detailed candidate retained without enough public evidence for confirmed faulty behavior, or likely pre-release/process work.
2 unresolved direct-mainline candidates cannot be placed in a report-time month
These records remain globally unresolved. Their author and committer timestamps support the record, but neither is presented as an invented discovery time or a zero monthly count.
| Commit | Authored | Committed | Reason |
|---|---|---|---|
| 17635fe6b5a55b8381f83eb4f81583d930ae02e3 | 2024-09-16T11:44:32-04:00 | 2024-10-02T23:19:26Z | Diff replaces unsupported Chokidar4 globs with directory watch. Parent upgrades dependency; both are committedOct2, despite older authored date. No independently documented discovery instant or affected-public-release established; remains unresolved and cannot be placed at an invented report time. |
| 8a0483ecfd624227d19781a6005fab121de1f4b1 | 2024-10-02T18:47:00-04:00 | 2024-10-02T23:19:26Z | Diff restores extension guard disabled by preceding watch repair. No observed failure report or independently affected release established. Retained unresolved with both author and committer times; no invented discovery timestamp. |
Major capabilities released in 2026
1.5.0, January 22: An HTTP client inside the REPL
Developers could send requests to the simulated API without leaving the interactive session, alongside delay controls for latency testing.
2.2.0, March 19: An Admin API and agent skill
Counterfact exposed a control surface intended for tooling and coding-agent workflows, while preserving the human-facing REPL.
2.6.0, April 6: Request validation and a fluent request builder
Incoming requests could be checked against the OpenAPI contract, and route() added typed discovery, construction, autocomplete, and execution inside the REPL.
2.7.0–2.8.1, April 10–14: Composable scenarios and startup state
Named scenario functions made state transitions reusable; startup scenarios could seed a useful simulated world automatically when the server began.
2.9.0, April 17: Multiple APIs in one process
ApiRunner isolated generation and runtime state per specification and enabled grouped APIs to share one Counterfact server and REPL.
2.11.0, May 5: First-class API versioning
Version-aware generated types, derived prefixes, and $.minVersion() let handlers be shared safely across versions of an API.
2.12.0–2.14.0, May 16–30: OpenAPI 3.2 and Overlay support
QUERY, querystring parameters, streaming and server-sent events expanded protocol coverage; repeatable overlays made it possible to adapt a source document without editing it.
2.15.0–2.16.2, August 1–19: A shared store and modular public packages
A typed, hot-reload-safe store became available to routes and tooling, while focused OpenAPI, generator, runtime, client, and REPL packages opened Counterfact to embedding and reuse.