Running a private AI on your laptop just got easy enough to actually do.
What's happening
On Monday, the Unsloth AI team released Unsloth Desktop: a free, open-source app for Mac, Windows, and Linux that runs over 500 AI models locally. No Python environment. No internet connection after the initial download. No monthly bill.
Unsloth is a small team that built their name on open-source tools for fine-tuning AI models at about 2x the speed and 70% less GPU memory than the standard approach. Their libraries are popular with developers, but this desktop release is the first product aimed at people who would rather not touch a terminal. It installs like any other app.
The thing that makes this immediately useful for a small business is the built-in RAG connector. Point it at a folder on your computer and the AI reads whatever is in it automatically. Drop your FAQ, service menu, or product list in there and you have a private assistant that knows your actual business. Your files do not touch any server. Nothing leaves your machine.
There is also a fine-tuning workflow inside the app if you want to go further: feed it a dataset built from your own PDFs or CSVs and create a model that has your specific knowledge baked in, not just retrieved from a folder. That is a longer project, but it runs entirely offline.
Try this this week
- Download Unsloth Desktop from github.com/unslothai/unsloth (free, Mac/Windows/Linux)
- In the model library, pull Phi-4 mini (runs on 8 GB RAM) or Qwen3-8B if you have a bit more headroom
- Create a folder and drop in your FAQ, product list, or services page as a plain text or PDF file
- Connect that folder in the RAG section of the app
- Ask it the questions a new customer would actually ask
- You paid nothing, and your files went nowhere
The bigger picture
A year ago, getting local AI working well enough to use for real business tasks took an afternoon of Python wrangling. Unsloth Desktop brings that down to about fifteen minutes of downloading. You need a computer from the last few years and a bit of patience for the first model download. That is the full price of admission. If you have been curious about running your own private AI, one that knows your documents and costs nothing per message, this is a good version to try first. Small teams should not need a cloud budget to get AI doing useful work for them.