Your AI hands users a plan, then lets them lose it

Posted on Oct 16 • Originally published at k-leumas.com

You ask an LLM to review your setup and get back five numbered issues. Good answer. You start on #1, and a few messages later the list is a screen and a half above you.

To get to #3, you scroll up, reread it, scroll back down and type "for #3…", hoping the model knows which list you mean. Later replies bring numbered lists of their own, so "#3" can mean two different things. And nothing tracks which items you actually finished.

To the interface, a list of things to work through is just formatting. It renders and scrolls exactly like five paragraphs of explanation. Some tools now show the model's own multi-step work as a checklist, but when the list is work for the user, it's still just text.

The principle: when a reply hands the user a set of things to work through, the interface should hold onto it as work, not as text.

Smallest version: a pin control on any rendered ordered list that moves it into a sticky side panel. Users check items off themselves. It's front-end only, with no model changes, and it already fixes the scrolling problem.

Solid version: the model marks a list as actionable and gives each item a stable ID. On later turns it reports which item it's addressing and whether it's resolved, so the panel can highlight and check items automatically. Clicking an item attaches its ID to your next message, so "this one" is never ambiguous.

Ambitious version: lists become part of the conversation's state. Later messages can add or split items, finishing the list closes its span, and scrolling back through history shows each list as it stood at that point.

The full post has a checklist for auditing your own product.

If you're building an AI feature that hands users steps to follow, how are you handling this?