Side Quest: A Local AI Game Master That Sends You Outside

Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass Submission 🌿

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

Side Quest is a tiny game where an AI running on your own laptop gives you one small real-world mission, and the only way to win is to go outside.

The screen part is about 30 seconds. Everything else happens outside.

It's for people who want to go outside but get stuck at "and do what?": kids who need a mission, someone on a walk-to-clear-my-head break, or a friend group that wants a silly challenge. Quests are about noticing things like a heart-shaped leaf, a reflection in a puddle, or three kinds of bark. They aren't workouts, and they never ask you to photograph strangers.

Video Created with assistance of Brag Skill

The whole thing is about 180 lines of Python ( standard library only, no pip installs) plus one HTML file.

ollama pull gemma3:4b && ollama pull qwen3-vl:8b python3 app.py # then open http://<laptop-ip>:8080 on your phone Enter fullscreen mode Exit fullscreen mode How I Built It Models (all open weights, via Ollama ):

Structured outputs everywhere. Both calls pass a JSON schema in Ollama's format field, so the game master must return {title, mission, photo_proof, why} and the referee must return {passed, saw, verdict, outdoors}. No regex parsing of chatty model output. The photo_proof field matters most: the game master writes a checkable success criterion, and the referee is graded against exactly that sentence. The referee also has to confirm the photo was taken outdoors, so a photo of your screen doesn't count.

Context without tracking. The quest prompt gets:

With no internet, the weather is simply "unknown" and everything else still works.

Phone-first with no app store. The laptop serves one page over Wi-Fi. <input type="file" accept="image/*" capture="environment"> opens the rear camera directly. The photo is downscaled to 1024px on the phone before upload, which keeps the vision model fast.

The bug that ate an hour. My first referee calls all came back as JSONDecodeError. Calling Ollama directly showed why: with think: false and a JSON-schema format, qwen3-vl puts its answer into the thinking field, malformed, and leaves content empty. Leaving thinking enabled for the referee returns clean, schema-valid JSON in about 9 seconds. It's a good reminder that "structured output" depends on the model and the runtime flags. With open weights and a local runtime I could inspect the raw response and work out what was happening. A closed API would have just been a black box returning errors.