Your Laptop Can Now Train a Custom AI on Your Business Data

There's a free, open-source tool that trains an AI on your own data, on your own laptop, using less GPU memory than a video game. And it's going viral for good reason.

What's happening

Fine-tuning a language model used to mean either paying for cloud compute (think hundreds of dollars) or owning server-grade hardware nobody on a small team has. The problem is VRAM: big models just don't fit on consumer GPUs.

Soup solves that with "layer streaming." Instead of loading the full base model into your GPU at once, it streams the frozen weights from RAM one decoder layer at a time. Peak VRAM stays under 3.5 GB. On a standard 4 GB laptop GPU (tested on an RTX 3050), Llama-3.1-8B trains at 119 tokens per second. The whole thing is driven from one YAML file.

Version 0.75.0 shipped September 12, 2026. It is currently the fastest-growing open-source local AI project this week, with 60 external pull requests from 22 community contributors landing in this release alone. The license is Apache-2.0: free to use, modify, and run on your own hardware.

What this means practically: a café can train a model on their menu, seasonal specials, and FAQ responses. A clinic can fine-tune one on their intake process. A nonprofit can teach one to answer donor and volunteer questions using their own language. The result runs locally in Ollama or LM Studio. No API costs. No data leaving your machine.

Try this this week

• Check whether your laptop has a 4 GB NVIDIA GPU. Even an older RTX 3050 or 2060 qualifies.
• Follow the quickstart at trysoup.dev to install the Soup CLI (Python 3.11+, about five minutes).
• Build a small training file: 50 to 200 question-and-answer pairs in JSONL format. Start with your most common customer questions and the best answers your team already gives.
• Point your soup.yaml at the file, pick a base model (Llama-3.1-8B or Qwen-3-8B are solid starting points), and run soup train.
• Load the resulting adapter in Ollama and chat with it. Tweak a few examples and retrain. Each run takes minutes, not hours.

The bigger picture

Custom AI that actually knows your business has always sounded like a budget item reserved for companies with ML teams. Soup is a reminder that the open-source community keeps closing that gap. The smallest-working-thing version is simple: write down 100 things your team says to customers, turn them into training examples, and run Soup. That is a concrete AI win for your shop, done before the end of next week, and it costs nothing but an afternoon.