You are halfway through typing something into a chatbot when you stop. Maybe it is a client contract, a letter about a family health issue, or a draft you are not proud of yet. The question that makes you pause is fair. Where does this text actually go once you press send?
There is a second option most beginners never hear about. You can run AI on your own computer, with no account, no internet connection and no monthly fee. The model sits on your hard drive and answers you locally. This guide covers what that means in plain English, what your laptop needs, the two free apps that do the hard part, and the limits nobody mentions.
What it means to run AI on your own computer
When you use a normal chatbot, your words travel over the internet to a company’s servers. The model lives there, does the thinking there, and sends an answer back. You are renting access to something you never touch.
Local AI flips that around. You download a model file once, and after that the calculation happens on your own processor. No internet needed. Nothing leaves the machine.
The models you can download this way are usually called open-weight models, which simply means the company has published the trained file for anyone to download and use. If the vocabulary is new, our plain-English guides to what a large language model is and the common AI terms explained will make the rest of this post easier.
This is not a fringe hobby any more. In August 2025 OpenAI released two open-weight models, gpt-oss-120b and gpt-oss-20b, under the permissive Apache 2.0 licence, and said the smaller one can run on a device with just 16 GB of memory. Google publishes its Gemma models with a similar goal, describing them as built to run anywhere from cloud servers to laptops and even phones.
Why anyone would bother
Four reasons come up again and again, and only one of them is about saving money.
- Privacy. Text you type into a local model never reaches anyone’s server, so there is no policy to read and no setting to double check.
- It works with no internet. On a plane, on a train with bad signal, or during an outage, a local model still answers.
- No message caps and no subscription. Ollama’s own pricing page states plainly that running models on your own hardware is always unlimited.
- You finally see how it works. Watching a model load into memory, and noticing what a smaller one gets wrong, teaches you more in an evening than a month of reading about AI.
What your computer actually needs
Memory is the deciding factor, not the brand of your laptop. A model has to fit into RAM to run, so the size of the model you can use is set by how much memory you have spare.
LM Studio’s own system requirements put it plainly: 16 GB or more is recommended, and on an 8 GB machine you should stick to smaller models and modest context sizes. Those small models are still useful for summarising, rewriting and simple questions. At 16 GB you get real choice, including OpenAI’s gpt-oss-20b, though that 16 GB is the model’s own requirement, so it will be tight once your operating system takes its share.
You also need disk space, since each model is a file of a few gigabytes upwards. A dedicated graphics card or an Apple Silicon chip makes answers appear noticeably faster, but neither is required. An ordinary laptop will run a small model, just more slowly.
Important tip: start with the smallest model that does your job, not the biggest one your computer can technically load. A fast small model you actually use beats an impressive large one that takes forty seconds per answer and drains your battery.
Two free apps that do the hard part
You do not need to touch Python or compile anything. Two well-known free apps handle the setup for you, and both were launch partners when OpenAI published its open models.
LM Studio is the friendlier starting point if you like buttons and menus. It is a desktop app where you browse and download models inside the app itself, then chat with them in a familiar window. Under the hood it uses the MLX and llama.cpp runtimes, so you get good local performance without configuring any of it. The free tier costs nothing and now covers offline voice transcription as well as chat.
Ollama suits people who are comfortable typing a command, though it now ships desktop apps too. Its free plan costs nothing and runs models on your own hardware, which is the part that matters here, because text a local model reads never leaves your machine in the first place.
One honest caveat that most guides skip. Both companies now also sell access to much bigger models running on their own servers, and they charge for it differently. Ollama’s paid plans start at 20 dollars a month, while LM Studio sells cloud credits you top up as you go. Price is not the thing to watch, though. Ollama’s free plan now lists cloud model access too, so free no longer means local. Those cloud modes are convenient, but they are not local, so the privacy benefit disappears the moment you switch to one. Check which mode you are in before you paste anything sensitive.
A first setup that takes about twenty minutes
- Check your available memory first, so you pick a realistic model instead of guessing.
- Install one app, not both. LM Studio if you want a normal window, Ollama if you do not mind a terminal.
- Download one small model. Resist queueing up five, because you will not compare them fairly anyway.
- Test it on three real tasks from your own week. Summarise an email you already answered, rewrite a paragraph you wrote badly, explain a term you half understand.
- Compare those answers with your usual chatbot, then decide honestly whether the local one is good enough for that kind of task.
That last step matters more than the install. Most people who give up on local AI never tested it against real work, so they had nothing to judge it by.
The limits, honestly
A small model on your laptop is not the equal of the best hosted model, and anyone who says otherwise is selling something. Expect slower answers, weaker reasoning on long problems, and confident mistakes. It also knows nothing about today’s news unless the app’s search tool is switched on, and those built-in search features usually send your query out to a web service, which quietly undoes the offline privacy you installed the thing for. The habit of checking anything that matters still applies, and our guide to using AI safely covers that in more detail.
There is a practical cost too. Running a model works your processor hard, so a laptop gets warm and the battery drops faster than usual.
Where local AI is genuinely the better choice
From my own experience building and maintaining websites for other people, the strongest case is client material. When someone sends you their unreleased copy, their pricing or a database export, running that text through a local model keeps a promise you already made about their data. It removed a decision I used to make far too casually.
The same logic applies at work, where employees often paste company information into whatever free tool they found, a habit now widely called shadow AI. A local model is one of the few answers that keeps people productive without pushing internal documents to an outside service.
It is also the obvious choice for offline study, travel, and anyone on an expensive connection. If you mostly work from a handset, our guide to using AI on your phone is a better place to start.
Common Questions
Is it really free to run AI on your own computer?
The local part is free. The apps cost nothing, the open-weight models are published for free download, and there are no message limits. You pay in disk space, a warmer laptop and some electricity. Any subscription you see advertised is for running models on the company’s cloud servers instead.
Do I need a graphics card?
No. A dedicated GPU or an Apple Silicon chip makes replies come faster, but a standard laptop can run a small model. Speed is the thing you trade away, not the ability to run it at all.
Is a local model as good as ChatGPT or Gemini?
Not in general, no. For everyday jobs like summarising, tidying up your writing, drafting a reply or explaining a concept, a good small model is often close enough. For hard reasoning, long documents and current information, the hosted models are still clearly ahead.
Does it work with no internet at all?
Yes, once the model is downloaded. You need a connection to fetch the file the first time, and after that you can switch off the wifi and keep working. LM Studio documents this directly, saying the app can operate entirely offline once you have the model files.
Final takeaway
You do not have to pick one and abandon the other. Most people end up with both, using a hosted model for heavy thinking and a local one for anything private, offline or repetitive. The useful move this week is small: install one app, download one small model, put three real tasks through it. That evening will tell you more about whether you should run AI on your own computer than any comparison table will.
0 Comments