# Evidence-gated AI run starter kit

Turn an AI worker's “Done” from a chat message into an explicit state claim that another process can reject.

```text
BEFORE  worker says Done  -> human discovers production is stale
AFTER   contract + report + exact revisions + readback -> VERIFIED or non-zero exit
```

This is a small, standard-library-only reference implementation. It does **not** prove that a worker, test runner, deploy system, or readback probe told the truth. A real remote gate must capture those attestations independently and keep the gate outside the worker's write and credential scope.

## 60-second quickstart

```bash
# 1. Run the included verified example.
python3 verify_run.py \
  --contract contract.example.json \
  --report report.verified.example.json \
  --require-state VERIFIED

# exit 0: the requested state is proven by contract-bound evidence
```

Now watch the false-DONE example fail:

```bash
python3 verify_run.py \
  --contract contract.example.json \
  --report report.false-done.example.json \
  --require-state VERIFIED

# exit 2: the deploy revision contradicts the worker result
```

Exit codes:

- `0`: required state is proven by contract-bound, internally consistent evidence;
- `2`: evidence is malformed or contradictory;
- `3`: evidence is consistent but has not reached the required state.

## Put it in an agent loop

The controller—not the worker—should own the contract and decide whether the run advances:

```python
import subprocess

completed = subprocess.run(
    [
        "python3",
        "tools/evidence-gate/verify_run.py",
        "--contract", "evidence/contract.json",
        "--report", "evidence/report.json",
        "--require-state", "LANDING_READY",
    ],
    check=False,
)

if completed.returncode != 0:
    raise SystemExit("STOP: worker evidence did not satisfy the contract")
```

Minimal control flow:

```text
1. Controller writes immutable contract.json.
2. Worker produces a commit and report.json.
3. Controller or CI captures changed paths and test attestations.
4. verify_run.py must reach LANDING_READY before merge.
5. Deploy system records the exact deployed revision.
6. Independent probe records revision + contracted predicate.
7. verify_run.py must reach VERIFIED before anyone says Done.
```

The verifier binds the report to:

- schema version and `run_id`;
- base and result commit OIDs;
- canonical changed paths with slash-aware write/deny scopes;
- named acceptance tests and their exact commands;
- landed and deployed revisions;
- the exact production predicate and observed revision.

## Add a remote check

`remote-gate.github-actions.example.yml` shows the smallest GitHub Actions skeleton. Copy it to `.github/workflows/evidence-gate.yml`, copy the entire starter kit to `tools/evidence-gate/`, adapt the paths, and make the check required. It deliberately fails until a trusted CI or deploy step writes independently captured attestations to `$RUNNER_TEMP/report.json`; it never accepts a worker-authored report from the pull-request checkout.

The remote example calls `verify_run_isolated.sh`, which starts Python with `-I` so a sibling module cannot shadow standard-library imports. `CODEOWNERS.example` protects the entire executable directory, not only `verify_run.py`.

That workflow is **not a security boundary by itself**. If the worker can edit the workflow, executable directory, contract, branch-protection rule, or their credentials, it can bypass the gate. `CODEOWNERS.example` shows the minimum paths and repository settings that require an owner outside the worker fleet.

## States

```text
ACCEPTED
  -> WORKER_EXITED
  -> LANDING_READY
  -> LANDED
  -> DEPLOYED
  -> VERIFIED
```

The verifier will not claim a later state unless all earlier evidence is present. `*` matches within one path segment; `**` is recursive and must occupy a complete segment. Other wildcard syntax—including `?`, character classes, brace expansion, extglob, and negation—is rejected. Ambiguous forms such as `src/**x` or `src/***/x` are rejected. Noncanonical paths such as `src//file.py`, `src/./file.py`, absolute paths, backslashes, and parent traversal are also rejected. Production readback requires a timezone-qualified ISO 8601 `observed_at` value.

## Run regression tests

```bash
python3 -m unittest -v test_verify_run.py
```

The suite includes adversarial cases for missing or mismatched run IDs, schema drift, substituted commands, boolean exit codes, write/deny scope bypasses, noncanonical paths, revision mismatches, and substituted production predicates.

## What improves—and what is not yet proven

This changes the **claim protocol**:

- a worker message can no longer advance the state by itself;
- contradictory or incomplete evidence returns a non-zero exit;
- every state names the next required evidence;
- merge, deploy, and production become separate transitions.

It does **not** show that productivity, quality, customer outcomes, or return on investment improved. Measure those separately with a stable run ledger and denominator.

## Production hardening you still need

- one identity per worker, not a shared human account;
- least-privilege, short-lived credentials;
- changed paths captured from the actual base/result diff;
- independently captured test, build, deploy, and readback attestations;
- remote checks and branch rules the worker cannot edit or bypass;
- signed build/deploy provenance;
- a readback predicate that checks the intended business result, not only HTTP 200;
- append-only run ledger fields for model/provider/version, retry count, stop reason, human takeover, token/cost, rollback, and outcome.

Fork it, try to bypass it, and report the counterexample. The useful contribution is not another success claim; it is a failing case that the gate should have rejected.