Skip to content

Controlled evaluation of v2.0.1

The thesis evaluation reports only version 2.0.1 of the harness (amendment of 2026-11-11 in the preregistration; the Haiku and Sonnet pilots ran on 2.0.0 as checks of the apparatus, not results). The campaign is in progress: its scripts, catalogue and plan are in evaluation/campaign/, and the evaluation code, fixtures and hidden tests are in evaluation/. No result is published on this page yet.

The campaign runs on version 2.0.1 installed from the wheel built from the annotated tag v2.0.1, never from a source tree; every result records the harness version, the SHA-256 of the wheel and the commit in its environment.json. The deterministic suites are taken from GitHub Actions (ubuntu-24.04); the runs with a model run on the author's Windows 10 PC, declared in each record. The deviations of the preregistration, the earlier pilot and precheck attempts (apparatus checks, not results) and the apparatus fixes made before the campaign are listed in its amendment of 2026-11-12.

Results of earlier versions are not part of this analysis. They remain in the git history and in the immutable v0.8.0 tag (see provenance).