77. LM Studio

The lesson

LM Studio is a free desktop application for discovering, downloading, and running large language models locally and offline on your own computer. It is aimed at developers, privacy-conscious teams, and anyone who wants to experiment with open-weight models without sending data to cloud APIs.

The main value proposition is local inference. Your model weights, prompts, documents, and responses stay on your machine. Nothing you type into the chat leaves the device. LM Studio combines model discovery, a chat interface, document-based retrieval-augmented generation, and a local API server into one app.

Installing LM Studio on Windows is straightforward. Download the installer from lmstudio.ai/download, run it, and launch the app. Windows x64 or ARM64 is supported, AVX2 is required on x64, and 16 GB of RAM is recommended. The first launch may prompt you to download an inference runtime, which needs internet; everything else works offline once models are present.

You find models in the Discover tab. Search by name or paste a Hugging Face URL. LM Studio supports GGUF models and, on Apple Silicon, MLX models. Quantization is shown in filenames: Q4_K_M is a common 4-bit sweet spot, while Q8_0 gives higher quality at the cost of more RAM and VRAM. To load a model, pick it in the Chat tab model loader and adjust context length and GPU offload if needed.

The Chat tab gives you a ChatGPT-like interface. You can set a system prompt that defines behavior for the current conversation, save system prompts plus parameters as reusable presets, and tune sampling values such as temperature, top-p, top-k, max tokens, and repeat penalty. Lower temperatures produce more deterministic output, which is good for coding and factual tasks.

LM Studio can act as a local OpenAI-compatible API server. From the Developer tab, toggle Start server, or run `lms server start` from the terminal. The default URL is http://localhost:1234, and the OpenAI-compatible base URL is http://localhost:1234/v1. By default no authentication is required; you can enable API tokens in the server settings.

Coding assistants can connect to LM Studio by pointing their base URL at http://localhost:1234/v1 and using any loaded model identifier. Tools such as Claude Code, Codex, Continue.dev, Cursor, Cline, and VS Code extensions all support this pattern. Enable JIT loading in LM Studio so the model loads automatically on first use.

Prompt engineering tips: put role, constraints, and output-format instructions in the system prompt; save reusable presets; use 0.0-0.3 temperature for coding and 0.7-1.0 for brainstorming; match context length to the task to save memory; and use reasoning models for multi-step problems. When chatting with documents, include explicit keywords to improve retrieval.

Performance tips: choose the right quantization, offload layers to GPU, enable Flash Attention if supported, use speculative decoding with a small draft model for a free speed boost, and set idle TTL so unused models are unloaded. If you run out of memory, reduce context length, quantization, or GPU offload. Exposing the server beyond localhost is a security risk, so enable authentication first.

Quick reference: start the server with `lms server start`, list models with `lms ls`, download with `lms get <model>`, load with `lms load <model>`, and stream logs with `lms log stream`. Remember that searching the catalog and downloading models need internet; chatting, document RAG, and running the local server work offline once your models are installed.

A clinical educator at an aged care home keeps a warm, plain-language system prompt in LM Studio for training handouts and saves it as a preset for the whole team.

A junior paralegal loads a small quantised model in LM Studio to summarise her own case notes, keeping client material on the office machine.

A freelance journalist uses LM Studio offline while travelling, drafting article outlines locally and raising temperature for ideas but lowering it for clean copy.

A credit analyst tests two quantisation levels in LM Studio on the same summary task and records which one fits the office laptop memory.

A workshop owner runs a local model in LM Studio on the counter computer to draft service reminders, reading each draft before it is printed.

A secondary school teacher runs LM Studio in a classroom with no internet so students can see model size, memory use and answer speed directly.

A municipal planning officer uses a local model in LM Studio to summarise public submissions, keeping submission text on the office machine.

A practice nurse drafts appointment recall letters with a local model in LM Studio and has the practice manager check the wording before sending.

A livestock farm manager runs a local model in LM Studio to turn weigh-sheet notes into a monthly summary for the farm adviser.

A bookshop owner uses LM Studio to draft the weekly newsletter, saving a preset so the tone stays the same each week.

Check yourself