ai newsFr., 16. Okt.
EilmeldungClaude Code Updates 2.1.299 bis 2.1.301 verändern das Verhalten von Hooks

Evidence Expires: A Fail-Closed Readiness Checklist with TTLs for Free-Tier AI Builds

A gate that passed last month is not a gate. It is a receipt for a system that no longer exists. That is the failure mode this checklist targets. You wire up a readiness gate, it goes green, and three weeks later you promote a build using evidence that describes a runtime, a model default, or a quo

dev.toFreitag 07:20 · 16.10.6 Min. Lesezeit
Bild: dev.to

A gate that passed last month is not a gate. It is a receipt for a system that no longer exists.

That is the failure mode this checklist targets. You wire up a readiness gate, it goes green, and three weeks later you promote a build using evidence that describes a runtime, a model default, or a quota policy that has already changed. The gate never lied. It just never expired.

What follows is a copy-paste workflow built on evidence TTLs. Every gate artifact carries an issue time and an expiry. Anything missing, unreadable, or stale blocks promotion by default.

Model progress is currently outrunning release process. That gap shows up in the boring places, not in benchmarks.

None of these events break your code. They invalidate your evidence. A readiness gate that cannot detect that is decoration.

Keep the verdict vocabulary tiny so nobody has to interpret it during an incident.

Fail-closed means the default branch is BLOCK. You do not get a PASS because a file exists. You get a PASS because a file is current.

TTLs are policy choices, not proofs. Pick them from how fast the underlying fact can change, then write them down where reviewers can argue with them.

If a row has no owner, delete the row. An unattributed gate becomes nobody's job within two sprints.

Two files. One declares the gates and their TTLs; the other reads the evidence records and returns an exit code your CI can branch on.

# gates.yaml version: 1 gates: model_pin: evidence:.evidence/model_pin.json ttl_hours: 168 owner: " @platform" server_image: evidence:.evidence/server_image.json ttl_hours: 336 owner: " @platform" quota_headroom: evidence:.evidence/quota_headroom.json ttl_hours: 24 owner: " @platform" rollback_drill: evidence:.evidence/rollback_drill.json ttl_hours: 720 owner: " @oncall" Enter fullscreen mode Exit fullscreen mode Evidence record format The record is deliberately dumb: a timestamp, a result, the exact command that produced it, and a digest of the artifact. Do not store prose here; store the command.

{ "gate": "model_pin", "issued_at": "2026-10-16T09:00:00Z", "issued_by": "platform", "command": "<the exact command that produced this record>", "digest": "sha256:REPLACE_ME", "result": "PASS" } Enter fullscreen mode Exit fullscreen mode The 60-line checker This is a sketch, not a battle-tested tool. Confirm the paths and timestamp formats against your own repo before you wire it into a pipeline.

#!/usr/bin/env python3 """ Fail-closed gate checker. Exit codes: 0 = all gates PASS 2 = at least one gate ESCALATE (stale, human re-run required) 1 = at least one gate BLOCK (missing, unreadable, negative) """ import datetime as dt import json import sys from pathlib import Path import yaml # pip install pyyaml ROOT = Path ( __file__ ). resolve (). parent ORDER = { " BLOCK ": 0, " ESCALATE ": 1, " PASS ": 2 } def now_utc (): return dt. datetime. now ( dt. timezone. utc ) def parse_ts ( value ): return dt. datetime. fromisoformat ( value. replace ( " Z ", " +00:00 " )) def evaluate ( name, spec, clock ): path = ROOT / spec [ " evidence " ] if not path. exists (): return " BLOCK ", f " { name }: no evidence file at { spec [ ' evidence ' ] } " try: record = json. loads ( path. read_text ()) except ( json. JSONDecodeError, OSError ) as exc: return " BLOCK ", f " { name }: unreadable evidence ( { exc } ) " if record. get ( " result " )!= " PASS ": return " BLOCK ", f " { name }: last result was { record. get ( ' result ' )!r} " if not record. get ( " command " ): return " BLOCK ", f " { name }: record has no reproducible command " try: age_h = ( clock - parse_ts ( record [ " issued_at " ])). total_seconds () / 3600 except ( KeyError, ValueError ) as exc: return " BLOCK ", f " { name }: bad issued_at ( { exc } ) " if age_h < 0: return " BLOCK ", f " { name }: evidence is dated in the future " ttl = float ( spec [ " ttl_hours " ]) if age_h > ttl: return " ESCALATE ", f " { name }: { age_h:.

1 f } h old, ttl { ttl:. 0 f } h " return " PASS ", f " { name }: fresh ( { age_h:. 1 f } h / { ttl:.

0 f } h), owner { spec. get ( ' owner ' ) } " def main (): config = yaml. safe_load (( ROOT / " gates.yaml " ). read_text ()) clock = now_utc () verdicts = [ evaluate ( n, s, clock ) for n, s in config [ " gates " ]. items ()] for verdict, message in sorted ( verdicts, key = lambda v: ORDER [ v [ 0 ]]): print ( f " [ { verdict: 8 } ] { message } " ) if any ( v == " BLOCK " for v, _ in verdicts ): return 1 if any ( v == " ESCALATE " for v, _ in verdicts ): return 2 return 0 if __name__ == " __main__ ": sys. exit ( main ()) Enter fullscreen mode Exit fullscreen mode Wiring it in python check_gates.py code = $? case " $code " in 0 ) echo "promotion allowed";; 2 ) echo "re-run stale gates, then re-check"; exit 2;; 1 ) echo "do not promote"; exit 1;; esac Enter fullscreen mode Exit fullscreen mode The important detail is the middle branch. Exit code 2 is not a failure of your software; it is a signal that a fact you depend on has aged out. Treat re-running as the default path, not as an exception.

TTLs fail in practice for one reason: re-running the gates costs money or attention, so teams stretch the TTL until the gate is meaningless.

The lever that makes the loop survivable is cheap execution. This is where MonkeyCode is relevant to the workflow: the project is open source, and it publishes a free model allowance plus a free server option. Those two make a scheduled re-verify job affordable in a way that a metered runner often is not — you can re-run a prompt-contract check, capture the evidence record, and let the TTL reset without opening a budget conversation. Confirm the current terms yourself before you depend on them; free allowances are advertised figures and they change. At the time of writing the operator advertises a 10M-token grant, which is a number to verify, not a number to bake into an SLA.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Run this before every promotion. Each item is a gate, and each gate must fail closed.

Item 9 is the one teams skip. A checker that prints warnings but returns success is a dashboard, not a gate.

Skip this approach if your deployments are fully pinned and immutable with no third-party drift — the TTLs will just add ceremony. Skip it too if you have nowhere durable to store evidence, because expired records you cannot find are worse than no records.

Start with a single gate and a real TTL. Pick model_pin, give it seven days, and write the command that produces its evidence. When it goes stale, you will feel the friction immediately — and that friction is the point, because it is the moment you would otherwise have shipped on a fact that had already stopped being true.

If you want to try the loop without a budget, the open-source project's free server option is a reasonable place to run the first scheduled re-verify job.

Quelle: dev.to (dev.to). Original öffnen