Skip to content

Add deterministic ./verify CLI for agent loops - #419

Open
OktavianusKokoh wants to merge 2 commits into
shiyu-coder:masterfrom
OktavianusKokoh:verify/cli
Open

OktavianusKokoh wants to merge 2 commits into
shiyu-coder:masterfrom
OktavianusKokoh:verify/cli

Conversation

@OktavianusKokoh

Copy link
Copy Markdown

What

Adds a repo-root ./verify CLI so agents can close their own loop:

  • ./verify seed
  • ./verify run <flow> (smoke | regression)
  • ./verify screenshot <page> (chart)
  • ./verify check (non-zero exit if any rule in verification/rules.yaml is broken)

Why

Verification is the job. Same input must always give the same result (fixed seed, SHA256 fixture lock, Matplotlib Agg).

Proof

./verify seed
./verify run smoke
./verify screenshot chart
./verify check   # exit 0
# remove smoke marker → ./verify check exits 1

Not changed

  • Model weights / training code
  • Existing pytest suite (still available via ./verify run regression)

Unsure

  • Chart SHA256 may differ across Matplotlib minor versions; baseline is pinned for this environment.
seed, run <flow>, screenshot <page>, and check with non-zero exit on
broken rules. Smoke flow needs no model download; chart baseline is
SHA256-pinned for repeatable proof.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant