Qwen3.8-27B on a Mac: Ollama MLX, MTPLX, and the MTP path

This is a recipe article for Qwen3.8-27B on Apple Silicon, not a measured-tok/s article. No published Qwen3.8-27B Mac benchmark exists at the time of writing. The closest published reference on the same chip + quant + stack is ~63 tok/s on M5 Max with MTPLX, measured on Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed (the Qwen3.6-27B MTPLX artifact). Qwen3.8-27B is expected to land in the same class once a Qwen3.8-MTPLX-optimized build ships. The recipe below — Ollama MLX engine + NVFP4 MLX quant + MTPLX for max-MTP — is the right starting point regardless of whether a published number exists.

What Qwen3.8-27B is

27-billion-parameter dense vision-language model from the Qwen team. Apache 2.0. Hidden dimension 5120, 64 layers (48 Gated DeltaNet linear-attention + 16 full-attention, head dim 256). Native context 262,144 tokens, extendable to 1 M via YaRN. Trained with multi-token prediction (MTP). All claims here match the Hugging Face model card.

Three things matter for the recipe:

1. Thinking is on by default. The model emits <think>...</think> blocks. You tune via reasoning_effort: xhigh (default — complex analysis), medium (balanced baseline), low (short replies). The Qwen3.8-27B card’s template raises an exception for any level outside these three. Do not set a value the card does not list — the brief included a stale none level that’s not on the card.
2. MTP is the speed lever. Vendor-stated on the model card: “Qwen3.8 was trained with multi-token prediction (MTP) … This is the single biggest inference speedup available without changing quant.” Generic MTP acceleration is 1.5–2× across hardware per the same card.
3. KV cache scales with context. Native context is 262K but the cache runs ~64 KB/token in BF16 or ~32 KB in FP8 at full context. Stay at 8K–16K unless you actually need more — long context is the silent VRAM tax.

The Mac stack in 2026

There are three runtime layers that matter on Apple Silicon; each one handles MTP differently:

• Ollama (default since 0.19, March 2026) ships an MLX engine for safetensors models and a llama.cpp Metal backend for GGUFs. Ollama auto-detects per file. MP status: no MTP support in Ollama’s llama.cpp backend as of 0.30.x; the MLX engine path is expected to gain MTP for Qwen3.8 once an MTP-enabled MLX build drops.
• oMLX — third-party GUI app, MLX-only. Same MTP status as Ollama MLX.
• MTPLX (youssofal/MTPLX) — MLX plus native MTP speculative decoding that uses Qwen’s own MTP heads as the drafter. Reference benchmark: ~63 tok/s on M5 Max with the Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed model artifact (HF page, Towards AI measured the 2.24× over plain decode on this stack, behind paywall, verified 2026-08-15).

The Ollama MLX engine swap (not MTP) is what roughly doubled decode speed on Apple Silicon for safetensors models in 0.19: 58 → 112 tok/s for Qwen3.5-35B-A3B (per Kashif Mehmood on Level Up Coding, Jul 2026; behind paywall, verified 2026-08-15). Important: only new pulls (safetensors / NVFP4 / MLX builds) get the MLX engine. Existing GGUFs on disk stay on the older llama.cpp path. Re-pull to get the speedup.

Ollama 0.31 (June 2026) added auto-tuned MTP for Gemma 4 on MLX — (inferred from the Gemma 4 MTP engine; Ollama’s blog does not name Qwen) the same engine should apply to Qwen3.8 once an MTP-enabled MLX quant lands. Ollama’s MLX performance blog (June 2026) also notes the MLX engine is “~20% faster from new optimizations” and that NVFP4 is now natively supported on Apple Silicon through Ollama’s MLX engine and yields ~20% higher tok/s than Q4_K_M with better perplexity. The reference: “Ollama’s MLX engine is now up to 20% faster from new optimizations: several operations are now fused into single Metal kernels via MLX’s just-in-time compiler features, and we’ve reworked Ollama’s GPU-backed sampling to run more efficiently.” — Ollama, MLX performance blog.

Picking the right quant for your chip

Per the Qwen3.8-27B model card and Ollama NVFP4 MLX support:

Quant	Weights	Recommended for
NVFP4 MLX	~16.5 GB	M4 Pro 48 GB, M5 Max 64 GB+, M4 Max 128 GB (~20% faster than Q4_K_M per Ollama MLX perf blog)
Q4_K_M (safetensors)	~17.1 GB	M3 Pro 36 GB (NVFP4 fits but leaves no KV cache headroom at long context); M2 Pro 32 GB (heavy KV offload past 8K)
FP8	~28.5 GB	Not yet viable for typical unified-memory chips
BF16	~55–56 GB	Reference benchmark runs only

Add ~0.93 GB if you load images for the vision projector.

Recipe 1 — Ollama MLX (the default for most users)

This is the simplest setup and gives you the MLX engine speedup without MTP. Once a Qwen3.8 MTP-enabled MLX build drops in Ollama, this same recipe will gain MTP automatically (no flag needed — Ollama auto-tunes it).

# Pull the MLX/NVFP4 build (auto-selects MLX engine)
ollama pull qwen3.8:27b-mlx

# Prevent 5-minute idle unload (so model stays resident between turns)
launchctl setenv OLLAMA_KEEP_ALIVE -1

# Verify residency — should show UNTIL Forever
ollama ps

# Set Hermes (or your agent) reasoning effort
hermes config set agent.reasoning_effort medium   # default baseline

reasoning_effort defaults to medium per the model card. Use low for short replies, xhigh for hard problems. Don’t blanket-set low — Qwen’s own card warns it backfires on multi-turn agent work.

Recipe 2 — MTPLX (the fastest path on Mac)

MTPLX is a community runtime that uses Qwen’s trained-in MTP heads as a native MLX drafter. It is OpenAI-API-compatible (serves on port 11434), which means any Hermes provider already pointing at http://127.0.0.1:11434/v1 will switch over automatically once MTPLX is up.

Reference benchmark: ~63 tok/s on M5 Max with the Qwen3.6-27B-MTPLX-Optimized-Speed model artifact (Towards AI measured 2.24× over plain MLX decode on the same hardware stack). A Qwen3.8-27B MTPLX-optimized build is expected to land in the same speed class once it ships.

# 1. Install MTPLX (one-time, follows upstream)
git clone https://github.com/youssofal/MTPLX.git ~/mtplx
pip install -e ~/mtplx

# 2. Pull the MTPLX-optimized Qwen 27B model artifact
hf download Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed \
  --local-dir ~/models/qwen3.8-mtplx-ref
# (When a Qwen3.8-27B-MTPLX-Optimized-Speed artifact is published, swap the
# repo ID above. The published Qwen3.6 artifact is the current reference.)

# 3. Serve it on port 11434 with MTP enabled
mtplx serve \
  --model ~/models/qwen3.8-mtplx-ref \
  --port 11434

# 4. Point Hermes (or your agent) at the MTPLX endpoint
hermes config set model.openai.base_url http://127.0.0.1:11434/v1
hermes config set model.openai.api_key not-needed
hermes config set model.default qwen3.8-27b-mtplx-ref

MTPLX is a community project — check the GitHub repo for current Qwen3.8 model support as the artifact gets published. MLX is Apple-only; ship a GGUF alongside for Linux/Windows fallback (Ollama auto-detects format per file).

Speed reference (not targets)

Reference numbers from prior-generation benchmarks at the same chip + quant + stack. Use them to set expectations, not as Qwen3.8-27B targets.

Chip	Quant (weights)	Stack	Reference tok/s	Source
M5 Max 64 GB	4-bit MLX, MTPLX	MTPLX (MTP-native)	~63	Towards AI MLX vs oMLX vs MTPLX, Reddit MTPLX 2.24× announcement (measured on Qwen3.6-27B-MTPLX-Optimized-Speed; Qwen3.8-27B MTPLX build expected in same class)
M5 Max 64 GB	4-bit MLX, no MTP	Ollama MLX	~28 baseline	Towards AI (measured on Qwen3.6-27B; Qwen3.8 expected ~28 in same setup)
M3 Max 64 GB	4-bit MLX + MTP	oMLX / Ollama 0.31+ MLX	~80–100 typical, ~50 worst case	HN anon373839 (behind paywall / VPN-gated; verified manually on 2026-08-15; HN thread measures Qwen3.5-35B-A3B-era 27B-class with MTP, not Qwen3.8-27B specifically)
M1 Ultra 64 GB	4-bit MLX, Qwen3.5-35B-A3B	Ollama MLX pre-/post 0.19	58 → 112 tok/s (1.93× from engine swap, MTP off)	Level Up Coding (different model class, used as an “MLX engine is fast enough on Apple Silicon” reference)
M4 Pro 48 GB (common 48 GB-class config — operator baseline)	4-bit MLX, MTPLX	MTPLX	~50–65	Brief rule of thumb from Qwen3.6-27B measurements; same engine applies to Qwen3.8 — current-gen benchmark not yet measured
M4 Pro 48 GB	4-bit MLX, no MTP	Ollama MLX	~25–35	Brief rule of thumb from Qwen3.6 measurements

Rule of thumb for chipset pick: below 48 GB unified memory, Q4_K_M (safetensors) is the only fit; M4 Pro 48 GB and above gets NVFP4 MLX for ~20% extra tok/s with no measurable quality hit. MTPLX beats plain Ollama MLX by 2.24× on the same hardware where MTP-trained-in heads can speculate; plain Q4_K_M GGUF is the path that won’t speculate on MTP.

Pitfalls worth flagging

1. Re-pull after Ollama 0.19. Existing GGUFs on disk keep the older llama.cpp engine and will not get the MLX doubling. Apply this on every new MLX build release, not just 0.19.
2. NVFP4 on Mac is the MLX path — not a Blackwell-only quirk. Pull the NVFP4 MLX build, not Q4_K_M, on M4 Pro 48 GB+ for ~20% extra tok/s and slightly better perplexity.
3. Plain Q4_K_M GGUF won’t speculate. Pick Ollama 0.31+ (auto-tuned MTP, Qwen3.8 once an MTP-enabled MLX build drops) or MTPLX if you want the MTP speedup. Plain Q4_K_M + plain Ollama MLX leaves 2× on the table.
4. Unified memory is shared with macOS. Leave 8 GB headroom or you’ll swap-thrash the moment a Safari tab or Mail window opens.
5. MTPLX is a community project. Verify current Qwen3.8 model support on GitHub before committing to it as your primary runtime; the Qwen3.6 MTPLX artifact is current, a Qwen3.8 analogue is expected.
6. MLX is Apple-only. If you need a Linux/Windows fallback, ship a GGUF alongside the MLX build; Ollama auto-detects format per file.

The recipe in one paragraph

Pull the NVFP4 MLX build (qwen3.8:27b-mlx) with Ollama 0.31+, set reasoning_effort to medium unless you have a reason not to, leave 8 GB headroom for macOS, and if the MTP class matters to you, install MTPLX and run a Qwen3.8-MTPLX-optimized build against the recipe above (currently using the Qwen3.6-27B-MTPLX-optimized-speed reference artifact at ~63 tok/s on M5 Max). Skip MTP and you halve the reference numbers; skip the MLX engine and you more than halve them.

Done means

• ollama ps (or mtplx serve) shows the model resident and serving on a port.
• A chat request returns tokens; the rate is in the same class as the MTPLX or Ollama MLX reference table above (not a Qwen3.8 target; reference only).
• reasoning_effort is set to medium (or your chosen level) and <think> blocks appear in thinking mode.
• The chosen quant fits in unified memory with 8 GB headroom left for macOS.
• When a Qwen3.8-27B-MTPLX-Optimized-Speed HF artifact appears, swap the hf download line in Recipe 2 and re-measure.

What this article does NOT cover

• Linux/Windows installs — those need GGUF plus llama.cpp or SGLang (see the RTX 3090 and DGX Spark guides).
• Fine-tuning or training Qwen3.8-27B — these are inference-only setups.
• Multi-GPU or multi-node clusters — this is a single Mac.
• The Qwen3.8-2.4T-A95B variant — different architecture, far larger memory footprint.

Sources

• Qwen3.8-27B - Hugging Face model card
• Ollama MLX performance blog (June 2026)
• Ollama - Faster Gemma 4 on MLX with MTP
• Ollama - edtorre/qwen3.6-hermes (Qwen3.6-27B MTP-enabled GGUF, Qwen3.8 analogue expected)
• youssofal/MTPLX - GitHub
• Youssofal/Qwen3.6-27B-MTPLX-Optimized-Speed HF artifact (reference benchmark)
• Kashif Mehmood, Level Up Coding - Ollama MLX speed (behind paywall; verified manually on 2026-08-15)
• Towards AI - MLX vs oMLX vs MTPLX (behind paywall; verified manually on 2026-08-15)
• Research brief this article was built from: coordinate/abs-research-briefs/qwen3.8-27b-hardware-tuning-brief.md (R2, Marvin-authored, 2026-08-14)