Brownfield guide: governing a change in an existing repository¶
The flow is the same as in the greenfield guide. What changes in an existing
repository is the starting point: history, conventions, dependencies you did not install and,
often, a baseline that is already broken. This guide reproduces the brownfield case of the thesis
on a real third-party project, pallets/itsdangerous 2.2.0, and shows every output.
The whole case is scripted in
examples/brownfield-itsdangerous/reproduce.sh:
PYTHON=python3.12 examples/brownfield-itsdangerous/reproduce.sh <harness wheel or requirement> /tmp/bf
What is different in a brownfield repository¶
- Start from a clean tree, or know what the baseline contains. The baseline is captured when
the run starts. Uncommitted changes that already exist are part of the baseline and stay out of
the ChangeSet. Only files the run owns count: the patch paths in
patchmode, ormetadata.ownedPathsfor an external agent. If neither is declared, every file that changes during the run is included.harness initkeeps the tree clean: in a Git repository it writes its.harness/entry to.git/info/exclude, which Git reads like a.gitignorebut which is not part of the tree, so the first run's dirty-tree check (environment.dirtyTree) finds nothing caused by init (issue #86). An entry already in.gitignoreis left as it is;harness init --ignore-file gitignorewrites the.gitignoreinstead (a change to commit). - Install the project's test dependencies where the harness runs. Validators run the
project's own commands (
python -m pytest,npm test) with the interpreter or tools found onPATH; undertoolchain.interpreter: auto(written byharness initsince 2.0) the Python validators use the project's.venvorvenv,uv run --no-syncorpoetry runinstead. A project whose commands differ declares its own validators undertoolchain.validators(see project toolchain). A missing test dependency is not detected in advance: the tests fail to collect. - The baseline may already be broken. Under
verification.differential(written byharness init) a failing mandatory validator also runs on the baseline: a failure the baseline shares isPREEXISTING_ERRORand does not block, a new one isINTRODUCED_ERRORand blocks (see better agent results). A test run that ran no test, such as pytest stopped by a collection error from a missing test dependency, is not comparable and still blocks (#190): fix the environment or close withAPPROVE_EXCEPTION. The evidence shows which files failed. - Large repositories. Since 2.0
harness initwritesworkspace.snapshot: git,workspace.baseline: manifestandworkspace.snapshotCache: true: files that.gitignoreexcludes are never read or stored, the baseline is a manifest of digests, and unchanged files are not hashed again. See large repositories for the measurement on 10,001 files. - Respect the existing conventions. Detection is read-only: the weights of the marker files
found are added up to a confidence of at most 1.0 and reported with the files as evidence. Lock
files (
package-lock.json,pnpm-lock.yaml,yarn.lock) add to the Node.js confidence, but the profile always runsnpm; itsambiguousPackageManagerpolicy is declared and not enforced. If the project uses another package manager, treat the Node.js results with care.
The case: itsdangerous 2.2.0¶
Obtain and verify the source¶
The thesis used the tag archive published by the project on GitHub. Its SHA-256 matches the value recorded in the thesis (the sdist on PyPI is a different file with a different digest):
$ curl -sSfL -o itsdangerous-2.2.0.tar.gz \
https://github.com/pallets/itsdangerous/archive/refs/tags/2.2.0.tar.gz
$ shasum -a 256 itsdangerous-2.2.0.tar.gz
7b0c6d4186e963b88489b69603b7ab2bf7c8e9eb4135a7b13b5f21bd4b937f2b itsdangerous-2.2.0.tar.gz
Treat third-party code as untrusted: the harness is not a sandbox. Before running it, the thesis checked the archive for absolute or traversal paths, symbolic links (there are none), install scripts and network or subprocess calls in code and tests.
Environment¶
A virtual environment with the harness and pytest==9.0.3, the project installed in editable
mode, and without freezegun, which the project declares in requirements/tests.txt. The
project pins pytest==8.1.1 there; this reproduction uses 9.0.3, the first release with the fix
for GHSA-6w46-j5rx-g56g (CVE-2025-71176, vulnerable tmpdir handling), and the 298 tests of
itsdangerous 2.2.0 pass with it:
python3.12 -m venv venv
venv/bin/python -m pip install <harness wheel> "pytest==9.0.3"
venv/bin/python -m pip install -e ./itsdangerous-2.2.0 --no-deps
cd itsdangerous-2.2.0 && git init -q && git add . && git commit -qm "itsdangerous 2.2.0"
Detection and task¶
$ harness init --path .
$ harness inspect --path .
{
"workspace": "…/itsdangerous-2.2.0",
"detections": [
{
"profileId": "python_default",
"technology": "python",
"confidence": 1.0,
"evidence": [
"pyproject.toml",
"tox.ini"
],
"warnings": []
}
]
}
The task (examples/brownfield-itsdangerous/task.yaml) hardens base64_decode so that non-ASCII
input raises BadData instead of being silently dropped, and adds one test. It is a patch task,
so the ChangeSet owns exactly src/itsdangerous/encoding.py and
tests/test_itsdangerous/test_encoding.py.
First attempt: the baseline is broken¶
$ harness run start --path . --task task_itsdangerous_base64_non_ascii
$ echo $?
6
The run stops in VERIFICATION with status FAILED. environment.baseline: report (written by
harness init) ran the mandatory validators on a scratch copy of the baseline in DISCOVERY and
found that pytest already fails there. verification.differential: true compares the failure
with the baseline, but pytest stopped while collecting on both sides, so no test ran: such a run is
not comparable and keeps blocking (#190; the event verification.differential.inconclusive gives
the reason). harness findings list --path . --run <run> returns:
| Severity | Rule | Message (abridged) |
|---|---|---|
LOW |
environment.baseline-failing |
The baseline already fails before the change: python.pytest FAILED |
HIGH |
python.pytest.failed |
python.pytest failed with exit code 2 |
HIGH |
python.pytest.test-error |
tests/test_itsdangerous/test_timed.py error |
HIGH |
python.pytest.test-error |
tests/test_itsdangerous/test_url_safe.py error |
INFO |
tests.coverage-unavailable |
Coverage of the changed lines could not be measured |
The pytest output stored as evidence shows why, and that the failing files are not part of the ChangeSet:
____________ ERROR collecting tests/test_itsdangerous/test_timed.py ____________
E ModuleNotFoundError: No module named 'freezegun'
__________ ERROR collecting tests/test_itsdangerous/test_url_safe.py ___________
E ModuleNotFoundError: No module named 'freezegun'
!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!
governance.stopTheLine: restore (written by harness init) keeps the failed run's changes as a
quarantined patch (event workspace.quarantined) and restores the baseline, so the workspace is
clean again.
Repair the environment and run the task again¶
Install the declared dependency (the environment, not the ChangeSet) and start the task again. A
new run applies the same patch; harness run continue on the failed run would verify the restored
workspace, an empty ChangeSet:
$ python -m pip install "freezegun==1.4.0"
$ harness run start --path . --task task_itsdangerous_base64_non_ascii
$ echo $?
4
The new run waits in DECISION with the gate PASSED. Its pytest run passed 298 tests,
the added one included; the only finding left is the INFO tests.coverage-unavailable. Approve
with the digest of the ChangeSet:
$ harness gate decide --path . --run <new run> --decision APPROVE \
--change-set-digest <digest> --actor human.reviewer \
--rationale "Baseline repaired by installing freezegun; the latest verification passes with the same ChangeSet"
$ echo $?
0
Final state: status PASSED, 122 events, event chain valid, ChangeSet of 2 files with digest
sha256:5702c3865fad03eb8ba9c449a42269e03afd2b93794bac34be4625f252b70a07 (recorded on Windows with
Git Bash and Python 3.12.10). The failed first run stays in the record (status FAILED, 106
events).
If the baseline cannot be repaired, decide APPROVE_EXCEPTION with a rationale (an ordinary
APPROVE is refused with exit 5), or REJECT.
Difference with earlier figures and the thesis¶
The thesis evaluated version 0.8.0. There the run stopped in VERIFICATION (run start exit 6,
one HIGH finding python.pytest.failed), installing freezegun and run continue gave exit 4
and 298 passed, and the gate consolidated every validation of the digest across attempts, so the
failure of attempt 1 kept counting (issue #2): the plain APPROVE was refused with exit 5 and the
run was closed with APPROVE_EXCEPTION (46 events, ChangeSet digest
sha256:e775aa2dc82793fea6af88f63e6b01564e29398f7e0c0b563c416383e8723d46). From 0.9.0 the latest
attempt of each validator counted, and the same steps closed with a plain APPROVE (46 events).
The defaults harness init now writes change two things: the failed run's changes go to
quarantine, so the repaired run is a new run start instead of run continue, and VERIFICATION
runs more checks (for example the requirement traceability of base64_decode_rejects_non_ascii),
so there are more events. The ChangeSet digest recorded above differs from the earlier one; the
cause has not been established. The tag v0.8.0 keeps the evaluated behavior.
Checklist for your own repository¶
- [ ] Commit or stash unrelated work before
harness run start. - [ ] Install the project's test dependencies in the harness environment
(
pip install -r requirements/tests.txt,npm ci, …). - [ ] Run the project's test command once by hand to know whether the baseline passes.
- [ ] Declare the files the task may change (
patchpaths ormetadata.ownedPaths). - [ ] If the gate passed over pre-existing failures, check in the evidence which tests ran.
- [ ] If the run failed, fix what fails and run the task again (
run start; withgovernance.stopTheLine: restorethe failed run's changes are in quarantine); if it cannot be fixed, decide knowingly betweenAPPROVE_EXCEPTION(with the reason) andREJECT.