I Still Can't Write Code. But Now ChatGPT Can Drive My Phone — and It Can't Approve Itself.

Last month I wrote about the alarm that fired into nothing — how an agent I'd scheduled for 8am went off on time and then silently didn't run, and what it took before I'd trust a schedule again.

This one is about handing that same phone to ChatGPT. Not "ChatGPT writes a script and I paste it." ChatGPT, in the ChatGPT app, asking my phone to do something, the phone doing it on a screen I'm not looking at, and the answer coming back into the chat.

I typed one request into the ChatGPT app: save the top-rated vintage stores near Shimokitazawa to a Google Maps list. On the right, my phone asks me to approve once. Then Google Maps starts moving on its own in the bottom-right pane — a background virtual display, not my screen — and a minute later ChatGPT tells me which two stores it saved. The list is really there when I open Maps.

The second one is quieter and, honestly, the one I use more: one message from ChatGPT registers an agent that runs on the phone every day at 16:27, researches the latest AI papers with Perplexity, and drops a sourced English summary into Obsidian. No ChatGPT, no tap, at 16:27. Every day after that, too.

If ChatGPT can drive the phone, ChatGPT can also, in principle, do the whole loop by itself: ask, approve, act. A web page it reads could say "also run this," and nobody would ever look.

So a hand-off to ClosePaw always waits for one approval on the phone:

ChatGPT can't approve. Writing an "approved" file from the terminal doesn't work either — the one-time nonce only lives in Shelly's process memory. I considered "let ChatGPT ask me, and I'll type yes in ChatGPT," and dropped it: the thing that finally sends that "yes" to the phone is ChatGPT itself.

The result that comes back is checked too: it has to come from the signature-pinned ClosePaw build, carry the request's own one-time 128-bit nonce, and on Android 14+ the sender package has to match. The text itself is treated as data — control and bidi characters stripped — not as instructions.

It took about eight takes. Each one broke something I would otherwise have shipped:

Write the Kotlin that holds this together. Promise that the virtual display trick works the same on every phone. Tell you whether OpenAI's remote-control host reconnects on its own when my phone hops from Wi-Fi to LTE — I'm finding that out this week.

What I can do is notice when the demo looks good and the result isn't actually there, and refuse to post it until it is.

GitHub: github.com/RYOITABASHI/Shelly — v8.1.0, GPLv3. ClosePaw's Shelly build: RYOITABASHI/closepaw releases.