Beyond the Autopilot: Reclaiming Corporate DNA with System One AI Decision Contracts

Signup Write New Story Beyond the Autopilot: Reclaiming Corporate DNA with System One AI Decision Contracts by M Sheik Uduman Ali by M Sheik Uduman Ali | @ sheikudumanali Director, Engineering | Driving Innovation from Ideation to Execution | AI, Cloud-Native & IoT | Technology Strategy & Design Thinking

Director, Engineering | Driving Innovation from Ideation to Execution | AI, Cloud-Native & IoT | Technology Strategy & Design Thinking

I came across a really thoughtful HBR’s (Harvard Business Review) AI at work article in Economic Times Magazine that reminds us not to let AI erode the skills that give a company its edge.

The HBR article’s coolest example is a 40-year-old bank that began using an LLM for credit approvals. It is initially relying on AI for credit approval but later balanced autopilot choices on approving from its Credit Committee where AI (LLM based!) handled about 15% of reviews by starting quarterly “credit roundtables.” These chats move the conversation from “Does this fit our rules?” to “Are our rules still working?”.

The real risk is not that AI makes bad decisions. Hallucinations happen, especially when context poisoned, but they are visible and fixable. The deeper danger is that AI silently replaces the process that made an organization’s judgment its own:

There is a technical parallel. Training generative models on AI-generated data triggers “ model collapse,” a digital game of telephone. Without a steady supply of fresh human content, each generation becomes a photocopy of a photocopy, and by the ninth generation one well-known test model had lost the plot entirely. Organizations face the same loop: if your decisions feed on machine-made decisions, your competitors’ do too, and strategy and tactics converge, whatever fine-tuning you have done.

Note: You can customize LLMs to protect corporate wisdom through context injection, custom embeddings, small language models, RAG, supervised fine-tuning and preference alignment. But these are long, hard paths that consume a lot of effort and tokens.

In short: the hidden risk of generative AI at work is not only wrong decisions, but losing the skills that make an organization uniquely good at making them right. When teams rubber-stamp LLM output, expert debate fades, juniors lose chances to learn, and distinctive judgment drifts toward the generic.

Through the bank credit-committee scenario, this article explores how System One’s explicit decision contracts, confidence gates, and auditable human review can make AI-assisted decisions more disciplined while keeping expertise active. The advantage comes from more than changing models: it requires preserving the structures through which people challenge decisions, teach judgment, and revise the rules.

The bank’s two fixes target exactly those. The roundtables separate “does this meet our criteria?” from “are our criteria still right?”. The apprenticeships teach juniors to challenge scores, not just read them as the diagram below illustrates.

Where System One helps. It is better suited to this than a generic LLM, for four reasons that come straight based on the design principles explained in my recent book Mastering TypeSafe Jev ( https://books.sheikaxis.com/mastering-jev ):

Architecting deterministic, sub-second decision layers for mission-critical systems.

Don’t miss reading Appendix A: — Why Jev or Kev Instead of a Regular LLM, and How Decision Engines Work Inside in this book.

Here the examples of bank credit-committee setting, written in the same shape as the book’s FinTech contract and policy code. The criteria text and thresholds are just illustrative purpose.

A contract is the versioned set of questions the model answers, with your own definitions of each label. The roundtable debates this file. Here, a sample contract.

CONTRACT_VERSION = "credit-contract/1.3.0" # bump on every roundtable change CREDIT_QUESTIONS = { "risk_band": { "type": "choice", "instructions": "Which risk band fits `applicant`, given `financials` and `history`?", "criteria": { "LOW": "Stable income, low debt load, clean repayment history", "MEDIUM": "One weak signal (thin history or high debt load), no default flags", "HIGH": "Several weak signals, or any recent missed payment", "REFER": "Evidence is missing, conflicting or unusual for our book", }, }, "meets_policy": { "type": "noul", "instructions": "Does this application meet the written lending criteria in `policy_version`?", "criteria": { "true": "Every stated criterion is satisfied by the evidence", "false": "At least one criterion is unmet or cannot be checked", }, }, "judgment_needed": { # speculative, cheap extra question "type": "noul", "instructions": "Does this case involve circumstances our criteria do not clearly cover?", }, } Things to consider:

The gate turns the model's probabilities into a route. Thresholds differ by action because the cost of being wrong differs. Here a Python example.

from dataclasses import dataclass @dataclass(frozen=True) class Gates: approve_min: float = 0.92 # auto-approve only when very sure decline_min: float = 0.97 # declines hurt customers, so even stricter margin_min: float = 0.30 # top option must beat the runner-up by this much audit_sample: float = 0.05 # random share of confident cases sent to apprentices def route(band: str, p_top: float, margin: float, meets_policy: float, judgment_needed: float, g: Gates, rnd: float) -> str: if band == "REFER" or judgment_needed >= 0.50: return "committee" # out of criteria: humans decide if p_top < 0.60 or margin < g.margin_min: return "committee" # flat bars: model is unsure if band == "LOW" and p_top >= g.approve_min and meets_policy >= 0.90: return "apprentice_review" if rnd < g.audit_sample else "auto_approve" if band == "HIGH" and p_top >= g.decline_min: return "senior_review" # never auto-decline return "analyst_review" # middle: one human looks How different inputs route:

Note: The book covers two capstone projects (FinTech and Industrial IoT) built on foundation libraries. Source code is available at https://github.com/udooz/mastering-jev-and-kev-book

Where it makes things worse. Its strengths are also its risks:

Bottom line. System One fits the challenges on AI at work described in the article better than a generic LLM, because it makes the criteria explicit, separates judgment from computation, and routes uncertainty to people. But the HBR article’s real point is organizational: expertise comes from structures where people debate and teach. No model creates those structures. At best, a well-designed one leaves room for them and feeds them better material.

Note: This article was originally published on https://notes.sheikaxis.com/p/preserving-human-judgment-in-the.