What would make you pay for AI cost savings? A 30-line script that shows where your agent tokens go

A recent Ask HN asked: "Would you pay for saving AI cost?" and "what would you need to see before trusting to pay?" My answer: a number measured on my own logs, before and after, that I can reproduce. Not a percentage claim from a landing page.

So here is the measuring part, free. Run it on your own agent transcripts first; decide about fixes after.

Most coding-agent cost is not your prompt. It is repeated context: the same file dumps, tool output and boilerplate resent every turn. This script takes a JSONL transcript (one message per line, with a content string) and reports which chunks repeat the most.

// token-hotspots.mjs - node token-hotspots.mjs transcript.jsonl import fs from " node:fs "; const lines = fs. readFileSync ( process. argv [ 2 ], " utf8 " ). split ( " \n " ). filter ( Boolean ); const approxTokens = s => Math. ceil ( s. length / 4 ); // rough: ~4 chars per token const seen = new Map (); let total = 0; for ( const l of lines ) { let m; try { m = JSON. parse ( l ); } catch { continue; } const text = typeof m. content === " string "? m. content: JSON. stringify ( m. content?? "" ); total += approxTokens ( text ); // fingerprint each 400-char block for ( let i = 0; i < text. length; i += 400 ) { const k = text. slice ( i, i + 400 ); if ( k. length < 200 ) continue; seen. set ( k, ( seen. get ( k ) || 0 ) + 1 ); } } const repeated = [... seen. entries ()]. filter (([, n ]) => n > 1 ); const wasted = repeated. reduce (( a, [ k, n ]) => a + approxTokens ( k ) * ( n - 1 ), 0 ); console. log ({ messages: lines. length, approxTotalTokens: total, approxRepeatedTokens: wasted, repeatedShare: total? + ( wasted / total ). toFixed ( 3 ): 0 }); repeated. sort (( a, b ) => b [ 1 ] - a [ 1 ]). slice ( 0, 5 ). forEach (([ k, n ]) => console. log ( ` ${ n } x ${ k. slice ( 0, 80 ). replace ( / \n /g, " " )}...` )); Enter fullscreen mode Exit fullscreen mode The repeatedShare number is the ceiling on what a de-duplication pass could save for that transcript. If it is 3%, do not pay anyone.

If it is 40%, now you know why.

None of these needs a paid tool. The value of a paid checklist is only saving you the hour of working out the order and the thresholds.

If you want the checklist version of the fixes above (with the thresholds and the order I use), it is a pay-what-you-want download, $1 minimum: Prompt Token Diet Checklist. The script above is yours either way.

What would you need to see before paying for something like this? I am curious which number convinces you.