ai newsDi., 27. Okt.
EilmeldungDeepMind unveils Genie 4 world model promising more realistic physics and humans

A Handoff Contract for a Shared AI Bench

Most handoffs lose evidence, not enthusiasm. A shift ends, someone types "almost done", and the next person rebuilds context from chat scrollback. The fix is a contract, not a template. This post gives one small contract you can paste into a team wiki. It covers three roles, two enforced transition

dev.toDienstag 14:29 · 27.10.6 Min. Lesezeit
Bild: dev.to

Most handoffs lose evidence, not enthusiasm. A shift ends, someone types "almost done", and the next person rebuilds context from chat scrollback. The fix is a contract, not a template.

This post gives one small contract you can paste into a team wiki. It covers three roles, two enforced transitions, and a validator that refuses handoffs with missing artifacts. The code is a reference sketch, not a battle-tested tool. Run it on a throwaway repo before you trust it.

The dispatcher owns the queue. That role decides what enters the bench and what waits outside it. A dispatcher who also builds is a dispatcher who quietly reorders the queue.

The runner owns the diff. This person runs the model, saves the output, and records the command that produced it. The artifact here is a path plus a log, not a summary.

The checker owns the verdict. This person reads the diff against the stated goal and writes pass or fail. A checker who rewrites the code has become a second runner, and the review is gone.

Three roles, three artifacts. Goal, diff, verdict. Everything else is commentary.

Because capacity is shared, the shift boundary matters more than the shift length. A free bench is not a fixed server. Latency moves, queues move, and a runner who was fast at 09:00 may crawl at 17:00. Contracts survive that variance. Verbal notes do not.

This bench runs on MonkeyCode, which the operator describes as offering free model access and a free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Treat both claims as starting assumptions and check the current limits on the site yourself before you plan a shift around them.

One JSON file per task. No database, no schema migration, no service to keep alive. The record is the handoff.

{ "id": "T-014", "state": "queued", "goal": "Add retry with backoff to the ingest client", "owner": "morgan", "deadline": "2026-10-28T10:00:00Z", "diff_path": null, "run_log": null, "reviewer": null, "verdict": null, "trail": [] } Enter fullscreen mode Exit fullscreen mode Five fields matter at the start. Goal, owner, deadline, state, trail. The rest stay null until someone earns them.

#!/usr/bin/env python3 # handoff.py — untested reference sketch, Python 3.11 # Enforces a three-role handoff contract on a JSON task record. import json import pathlib import sys FIELDS = { " queued ": { " goal ", " owner ", " deadline " }, " built ": { " goal ", " owner ", " deadline ", " diff_path ", " run_log " }, " checked ": { " goal ", " owner ", " deadline ", " diff_path ", " run_log ", " reviewer ", " verdict " }, } ORDER = [ " queued ", " built ", " checked " ] def advance ( path: str, target: str ) -> str: rec = json. loads ( pathlib. Path ( path ). read_text ()) current = rec. get ( " state ", " queued " ) if target not in ORDER: return f " refused: unknown state { target } " if ORDER. index ( target )!= ORDER. index ( current ) + 1: return f " refused: { current } -> { target } is not a legal step " missing = FIELDS [ target ] - { k for k, v in rec. items () if v } if missing: return f " refused: missing { sorted ( missing ) } " if target == " checked " and rec [ " verdict " ] not in { " pass ", " fail " }: return " refused: verdict must be pass or fail " rec [ " state " ] = target rec. setdefault ( " trail ", []). append ({ " to ": target, " by ": rec [ " owner " ]}) pathlib. Path ( path ). write_text ( json. dumps ( rec, indent = 2 )) return f " ok: { path } is now { target } " if __name__ == " __main__ ": print ( advance ( sys. argv [ 1 ], sys. argv [ 2 ])) Enter fullscreen mode Exit fullscreen mode Run it like this.

python handoff.py tasks/T-014.json built # refused: missing ['diff_path', 'run_log'] python handoff.py tasks/T-014.json checked # refused: queued -> checked is not a legal step Enter fullscreen mode Exit fullscreen mode The first refusal is the useful one. It tells the runner that a status is not an artifact. The second one keeps a tired person from marking work as reviewed by a reviewer who never existed.

Fill the fields, then rerun. The record moves forward exactly once per command, which makes the trailing trail readable in a diff.

jq -r '.state, (.trail | length)' tasks/T-014.json # built # 1 Enter fullscreen mode Exit fullscreen mode When the transition stalls A contract is only useful if the failure path is boring. Use a table, not a meeting.

Every row ends with a person and a clock. Rows without a clock become opinions.

The fourth row deserves a note. Free capacity is shared capacity, so unresponsiveness is a normal event, not an incident. Park the task with a timestamp and let the next shift decide. Do not sit and retry for an hour while the queue backs up.

Paste this into the wiki above the table.

BENCH RUN — SHARED FREE TIER 1. Dispatcher opens the shift: list queued tasks with owner and deadline. 2. Runner builds one task at a time. Save diff path and command log. 3. Runner advances to built. Refusal means the artifact is missing. 4. Checker reviews against the goal. Write pass or fail, then advance. 5. On shift end: park open tasks, note the time, hand the queue over. Enter fullscreen mode Exit fullscreen mode Five lines, one owner per line. A new teammate can follow it without asking which channel to use.

The validator checks presence, not truth. A runner can paste a log from a different task. Only the checker catches that, and only by reading.

The contract costs about two minutes per task. On a solo project that overhead is pure loss. Below roughly three people sharing a bench, skip the script and keep the five-line run.

This is not audit-grade provenance. If you need signed artifacts, use signed commits and CI attestations instead. Nothing here replaces a security review.

Free tier behavior changes. The script assumes nothing about quota, model names, or server lifetime, and neither should your plan.

Solo developers. Teams with a single reviewer and no queue. Anyone whose tasks finish in one sitting, because a handoff that never happens needs no contract.

Use it when two or more people share one bench across shifts. Use it when the queue outlives the people in it. Use it when "where did we leave off" is a real question.

A small contract plus a boring script beats a long postmortem. Hand over artifacts, not adjectives. If you want a bench to test the run on, MonkeyCode offers free model access and a free server option, and the five-line run above is enough to try it for one shift.

Quelle: dev.to (dev.to). Original öffnen