Guides · 2026-10-13

Why is my Mac slow running local LLMs? A troubleshooting tree

Work out whether slowness comes from prefill, swap, bad defaults, the runtime or the agent harness, and fix it.

Researched and drafted with AI assistance from the cited sources. Benchmarks are single-machine third-party results: treat them as indicative, not guaranteed.

The first question is always which part is slow: the wait before the first token, or the tokens themselves?

1. Long wait before the first token (slow prefill / TTFT)

This is the number one complaint from Mac users. Prefill is compute-bound. M1–M4 chips are comparatively weak at it, and M5 is a big step up. Coding agents make it worse by sending long system prompts and tool definitions.

Fixes:

• Reuse the KV cache. If your agent shifts context, many servers recompute everything. oMLX’s persistent tiered cache targets exactly this. On llama.cpp, use --cache-prompt, --cache-reuse N and slot save/restore.
• Use an MLX engine. In one M4 Pro test on Qwen3.6-35B-A3B, LM Studio’s MLX engine prefilled in ~30 s, compared with ~75 s on llama.cpp.
• Shrink the prompt. Fewer tools and a shorter system prompt mean fewer tokens to prefill.

2. Tokens stream slowly

• Check memory pressure and swap (Activity Monitor → Memory). Once a model plus context exceeds RAM, macOS swaps and speed falls off a cliff. Use a smaller quant or model, or cap the context.
• Check the context default. Ollama’s GGUF default of 262k context turned a 6.6 GB model into 15 GB in one test. Set num_ctx / OLLAMA_CONTEXT_LENGTH explicitly.
• Try a faster runtime. In one M3 Ultra test, mlx-lm ran 107.5 tok/s against 77.7 for Ollama GGUF on the same model.
• Pick an MoE. Qwen3.6-35B-A3B only activates 3B params per token.

3. Is it the harness, the runtime or the model?

Test the server directly with a simple curl request. If that’s fast but your agent is slow, the harness is the problem: huge prompts, cache-busting context edits, or many round-trips. If curl is also slow, go back to steps 1–2.

4. Still stuck

Faster experimental servers exist (rapid-mlx, MTPLX). The community describes them as faster but buggier than oMLX.

Skip the trial and error: sourced config packs →