Building Mintu: An Open-Source Living Desktop Pet and Local AI Companion with Tauri 2, Rust, and TypeScript

Most desktop AI interfaces today follow a predictable pattern: an isolated browser tab, a heavy Electron window, or a generic sidebar that gets lost in a sea of open windows.

With Mintu, I wanted to explore a different paradigm: what if your AI assistant was not an application you open and close, but an ambient, living desktop companion that resides naturally in your workflow?

Mintu is an open-source living desktop pet and local AI assistant built from scratch using Tauri 2, Rust, Vite, and TypeScript. It combines the ergonomics of an Apple-style Dynamic Island with 60 FPS kinematic pet physics, 100% private local LLM execution, real-time screen intelligence, and an open plugin ecosystem.

Building an interactive desktop pet that overlays other full-screen applications without stealing input focus or causing high CPU consumption requires strict architectural separation:

Unified Click-Through Screen Overlay Window (Tauri 2 / Win32): Single transparent coordinate space across the primary monitor with a Dynamic Island using spring-physics morphing (width, height, radius), a kinematic desktop pet layer (gravity, momentum, window edge anchoring), and a 30 Hz Win32 cursor poller with dynamic DOM hit-rect synchronization (maintaining under 2% idle CPU).

Local AI Engine (llama.cpp Sidecar): Automatic hardware tier detection (RAM, VRAM, CPU cores), a resumable chunk downloader with SHA-256 hash validation, real-time token streaming with syntax-highlighted markdown, and multi-provider cloud adapters (Gemini, Groq, OpenAI, Ollama).

Ambient Intelligence & Smart Clipboard: Foreground window process inspection and security blocklist, interactive screen-region capture with offline Windows Media OCR, and a real-time clipboard classifier (code review, translation, JSON cleanup).

Extensibility & Gamification: Focus XP progression and customizable wardrobe accessories, paired with JSON-based skin templates and runtime plugin manifests.

Mintu lives inside a floating pill at the top of your screen that expands, contracts, and morphs into different states (notifications, system media, Pomodoro timer, AI chat).

When you want Mintu to explore, you can drag Mintu out or press Ctrl + Alt + M. Mintu leaves the island, sits on top of your currently active window, falls down if that window closes, and hops back into the island when thrown near the magnetic docking zone.

Whenever you copy text with Ctrl + C, Mintu analyzes the content in real time and offers context-sensitive actions directly in the island:

As you complete Pomodoro work intervals, Mintu earns focus XP and levels up. You can customize Mintu with earned accessories from the wardrobe, including reading glasses, winter beanies, a coffee mug, and noise-cancelling headphones.

Mintu v0.2.0 introduces an open architecture:

You can test the pre-compiled standalone binary or build from source: