SneakAI: An Open-Source Architecture for Physical and Decentralized LLM Distribution
GitHub Hello everyone! 👋 I wanted to share and discuss with the community an open-source project I've been working on called SneakAI. ❓ The Problem In many regions, access to slow, unstable internet infrastructures or high traffic costs makes downloading large language models (LLMs in GGUF format
I wanted to share and discuss with the community an open-source project I've been working on called SneakAI.
❓ The Problem In many regions, access to slow, unstable internet infrastructures or high traffic costs makes downloading large language models (LLMs in GGUF format from 30B to 70B+ parameters) an absolute nightmare. Connection drops at 90% progress mean a complete waste of time and excessive data plan consumption.
💡 The Solution To bypass the dependency on high-speed internet connections, SneakAI proposes a decentralized model for physical data distribution on a national and international scale:
Distribution via Local and International Nodes: The platform connects clients to a decentralized network of local and international distributors who write LLM models directly onto physical storage media (SSDs, USB Flash Drives).
Distributor Operations: Distributors manage their storage media inventory (new or used) and set shipping rates to process and fulfill customer orders.
🛠️ Tech Stack Runtime: Bun (>= 1.x) Web Framework: Next.js 15+ (App Router) CMS & Admin: Payload CMS 3.x (with integration for searching and importing models directly from the Hugging Face API with 1 click) Database: PostgreSQL UI: Tailwind CSS + shadcn/ui
Since the project is completely open and experimental, I would love to get your technical opinion and feedback, and even build a community around this.