Local LLM

intelligent automation, kept local

A local AI setup running entirely on a second-hand laptop, with an external GPU on a Thunderbolt cable. Used for automation, lookups, and quiet work patch — no subscription, no telemetry, no data leaving the device. Open-source models, an open Vulkan compute path, and an offline knowledge layer underneath.

Why build your own

Most language model tools today live in the cloud — your conversations, your data, your questions all travel to servers you don't control. This is something different: a local assistant that runs on its own hardware, answers without an internet connection, and goes quiet when the laptop closes.

What's running inside

🐧 Linux Mint as the base — open, durable, repairable 🦙 Ollama for local model inference — three commands and you have your own 🌐 Self-hosted search for live web lookups without tracking 📚 Offline knowledge layer (TonttuLibrary, see Kiwix) for verified reference material

Honest about the limits

Running models locally on modest hardware comes with real trade-offs. On CPU alone it is slow. With a dedicated GPU it becomes genuinely usable, but the bottleneck shifts to model size and context length. For tasks that demand the strongest reasoning, cloud tools still have an edge today. The value here is not raw capability — it is ownership, privacy, and the freedom to run without permission or cost per query.

A note before you try this

A large language model is a software component — a program that predicts statistically likely text. It has no awareness, no intentions, no understanding. The mythology around these tools dissolves quickly when you run one yourself and watch it confidently invent facts. What remains is a useful tool, applied with appropriate expectations. That is exactly why running your own is the most grounding thing you can do.

🔗 Curious? Start at ollama.com — three commands and you're running.