Train an 8B LLM on a 4GB Laptop GPU: Hands-On with Soup
Train an 8B LLM on a 4GB Laptop GPU: Hands-On with Soup TL;DR Soup eliminates the high VRAM barrier of LLM fine-tuning by introducing layer streaming, enabling you to fine-tune an 8B parameter model on a consumer GPU with as little as 4 GB VRAM. Everything—from dataset loading to training hyperpara
Soup eliminates the high VRAM barrier of LLM fine-tuning by introducing layer streaming, enabling you to fine-tune an 8B parameter model on a consumer GPU with as little as 4 GB VRAM. Everything—from dataset loading to training hyperparams—is configured inside a single, declarative YAML file, stripping away days of boilerplate setup.
pip install soup-train Enter fullscreen mode Exit fullscreen mode Create a config file named train.yaml:
model: name: " meta-llama/Meta-Llama-3-8B" quantization: " 4bit" streaming: true training: batch_size: 1 gradient_accumulation_steps: 8 learning_rate: 2e-4 epochs: 3 lora: r: 16 lora_alpha: 32 target_modules: [ " q_proj", " v_proj" ] dataset: path: " tatsu-lab/alpaca" split: " train" max_seq_length: 2048 output: dir: "./checkpoints/llama3-8b-custom" Enter fullscreen mode Exit fullscreen mode Kick off your fine-tune with a single CLI command:
soup train --config train.yaml Enter fullscreen mode Exit fullscreen mode Why It Matters Until now, fine-tuning modern 7B–8B open-weights models typically demanded expensive high-VRAM hardware (A100/H100) or at least 24 GB cards like the RTX 3090/4090 when using QLoRA.
Soup changes the economics of AI development: