Why We Ditched a 30MB AI Model and Built a <2.5ms Canvas Snapper in TypeScript

When building FocusFlow — a spatial storytelling canvas that turns architecture diagrams into 60fps cinematic presentations — we faced an intriguing computer vision problem: Click-to-Snap.

When a user imports a rasterized architecture diagram (PNG/JPEG/SVG) and clicks anywhere inside an element (like an ECS task card, a VPC boundary, or a database cylinder), the engine needs to extract its precise bounding box in < 5ms to smoothly anchor camera viewport frustums.

In 2026, the knee-jerk reaction for image segmentation in the browser is to reach for a machine learning model:

For a lightweight web application that users open in their desktop browser, forcing a 30MB weight download just to detect rectangles felt like engineering malpractice.

We asked ourselves: Can classical computer vision, implemented in pure TypeScript and tuned for CSS geometry, beat neural nets in speed, accuracy, and footprint?

Traditional Sobel gradient filters assume simple solid shapes on clean contrast. But modern UI diagrams look nothing like that:

To solve this without machine learning, we built a 3-tier deterministic pipeline:

Instead of single-point sampling, we sample 33 concentric points across 4 rings ($r \in [1, 8, 16, 24]\text{px}$). Each sample is weighted inversely by its local Sobel gradient flatness: $$W_i = \max(1, 35 - G_i)$$ High-gradient text pixels are suppressed, while flat card pixels dominate. If a cluster fails validation, a Top-3 candidate fallback loop recovers the true background.

Rather than comparing against an absolute seed color, our BFS tracks local step-to-step color deviation ($\Delta E_{\text{step}} \le 8$).

To seal text perforations without blurring vertical card edges, we employ an anisotropic separable kernel:

Across our 20-suite automated E2E test matrix (including nested cloud diagrams, dark-mode wireframes, and accent-bar cards):

Sometimes the most scalable engineering decision in 2026 is not to deploy a transformer model. For structured UI domains, classical computer vision combined with modern TypedArrays delivers 60x faster performance with zero bundle bloat.

Our full architecture and implementation details are open-sourced: