Sourced local AI setups for your Mac.
Pick your chip and RAM, get the model, server and agent config that fits. Mac-only, coding-agent focused, every claim sourced.
What can my Mac run?
Unified memory
~5.2 GB peak on mlx-lm leaves headroom for macOS, the agent and a modest context.
Fits, but tight: cap context hard and watch memory pressure. Or drop to a 9B for speed.
The default pick: only 3B params active per token, so generation is fast.
Room for both weights and a long agent context. 27B scores higher on SWE-bench but is slower.
Space to serve a coder plus embeddings/reranker side by side in oMLX.
Starting points from published model sizes and the ~0.5 GB / billion params (4-bit) rule, plus room for context. These setups haven't been hardware-tested yet. Sources →
73.4
SWE-bench Verified for Qwen3.6-35B-A3B, with only 3B params active per token. src
107.5 tok/s
mlx-lm on Qwen3.5 9B 4-bit (M3 Ultra), vs 77.7 for Ollama GGUF. src
6.6 → 15 GB
A 262k default context more than doubled one model's memory use. Cap your context. Troubleshoot →
Start here
Run Claude Code or OpenCode against a local server on your Mac
Point Claude Code and OpenCode at oMLX, Ollama or any OpenAI-compatible local server running Qwen3.6.
LoRA fine-tuning on a Mac with mlx_lm.lora
QLoRA fine-tune a model on Apple Silicon with mlx-lm: data format, training, fusing and serving, with RAM guidance.
oMLX setup and tuning on Apple Silicon
Install oMLX, serve a Qwen3.6 model over OpenAI and Anthropic APIs, and tune the KV cache for long agent sessions.
Choosing a Qwen3.6 model and quant by RAM
Which Qwen3.6 model and quantization fits your Mac's unified memory, with a simple sizing rule and per-tier picks.
Why is my Mac slow running local LLMs? A troubleshooting tree
Work out whether slowness comes from prefill, swap, bad defaults, the runtime or the agent harness, and fix it.
Config packs
Early access · $9
Mac RAM-Tier Config Pack
Copy-paste starting configs for oMLX, Ollama and mlx_lm.server, wired to Claude Code and OpenCode, organised by RAM tier (16 / 24-32 / 48-64 / 96 GB+).
Early access · $7
Local Coding Agent Prompt Pack
Compact system prompts and task templates tuned for smaller local models, written to keep prompts short (prefill is the #1 Mac bottleneck).
Early access · $12
Qwen LoRA Fine-Tuning Notebook (MLX)
A step-by-step Jupyter notebook for QLoRA fine-tuning with mlx_lm.lora on a Mac: dataset prep, training, eval, fuse and serve.
Configs are starting points built from cited sources. Verify on your Mac.
Stay current
Get the weekly Mac local-AI digest
New models, tool updates, and sourced configs. Free. Unsubscribe anytime.
Launching soon. Follow along via RSS in the meantime.