Qwen3.8-27B on DGX Spark (GB10): vLLM 0.24 + NVFP4, the sm_121 kernel gotcha

This is a recipe article for Qwen3.8-27B on the DGX Spark (GB10 Blackwell). No published Qwen3.8-27B benchmark on the Spark exists at the time of writing because two pieces are missing:

1. The Qwen3.8-27B NVFP4 ModelOpt checkpoint has not been published (huggingface.co/nvidia/Qwen3.8-27B-NVFP4 returns 401; the Qwen3.6-27B NVFP4 checkpoint (huggingface.co/nvidia/Qwen3.6-27B-NVFP4) returns 200 and is the current referent).
2. A Qwen3.8-27B vLLM recipe on this exact stack has not been published (the Qwen3.6-27B recipe at recipes.vllm.ai/Qwen/Qwen3.6-27B is the current referent).

The closest published reference is ~144 tok/s at concurrency 16 on a single GB10 sm_121, measured on the Qwen3.6-27B-NVFP4 stack: r0b0tlab community benchmark. Qwen3.8 is expected to land in the same speed class on the same stack once the NVFP4 checkpoint ships.

The recipe that follows is the operator-grade setup using the Qwen3.6-27B-NVFP4 checkpoint as a stand-in. Swap to the Qwen3.8-27B-NVFP4 path when NVIDIA publishes it.

What “Spark-native” means

The GB10 is Blackwell-class with compute capability sm_121, but it is not the same silicon as a B200 or a 5090. Three consequences:

1. The stock vllm/vllm-openai Docker image frequently misses the sm_121 kernel. Pulling the latest tag and expecting NVFP4 to “just work” is how you end up with an Illegal memory access or a silent fallback to FP8. Use a GB10-specific image or build from source with sm_121 patches. The community-maintained awesome-dgx-spark list is the curated index.
2. NVFP4 (W4A16) is the documented best choice on Blackwell. The Qwen3.6-27B checkpoint sits at roughly 22 GB of weights on the GB10, leaving massive headroom for KV cache and the vision projector.
3. AWQ-Int4 is a wrong-stack trap on Blackwell. AWQ was tuned for Ampere/Hopper. On GB10 it leaves roughly 2× on the table vs NVFP4 — Qwen3.6-27B AWQ INT4 on stock vLLM was measured at ~18 tok/s on the Spark (Reddit r/LocalLLM, DGX Spark thread; behind paywall; verified manually on 2026-08-15). Don’t ship AWQ on a Spark.

What Qwen3.8-27B is

27B dense (every parameter active on every token), vision-language, Apache 2.0, hidden dimension 5120, 64 hybrid layers — 48 Gated DeltaNet linear-attention + 16 full-attention. Trained with multi-token prediction (MTP). Native context 262,144 tokens, extendable to 1 M via YaRN. Model card at huggingface.co/Qwen/Qwen3.8-27B.

Thinking is on by default — the model emits a <think>...</think> block before its final answer. Tune via reasoning_effort: xhigh (default), medium, or low. Three and only three levels — the brief included a stale none that’s not on the card. preserve_thinking is on by default so streamed tool calls don’t drop the trace. Recommended sampling in thinking mode, vendor-confirmed: temperature 1.0, top_p 0.95, top_k 20, min_p 0.0, presence_penalty 0.0. Do not use greedy decoding in thinking mode — it collapses the chain-of-thought into nonsense.

The right stack for Spark

vLLM 0.24.0+ with the NVIDIA ModelOpt NVFP4 checkpoint. The Qwen3.6-27B ModelOpt checkpoint (huggingface.co/nvidia/Qwen3.6-27B-NVFP4) drops the MLP linears to NVFP4 (W4A16) while attention linears and KV cache stay FP8. The result is ~22 GB of weights that fit a single GB10 with plenty of room for a long context window. vLLM auto-detects the ModelOpt quantization from the checkpoint — no --quantization flag needed, per the recipe:

“The NVIDIA ModelOpt NVFP4 checkpoint … the ~22 GB weights fit a single Blackwell GPU. vLLM auto-detects the ModelOpt quantization from the checkpoint, so no --quantization flag is needed.” — vLLM Recipes, Qwen3.6-27B DGX Spark section, 2026.

On the quant ladder, the practical ranking is:

• NVFP4 (W4A16) — best. Near-lossless vs FP8, ~22 GB, fits 262K context on a single Spark.
• FP8 — viable at ~28.5 GB weights, more KV headroom, slightly slower per token.
• BF16 — full precision fits in 128 GB unified but underutilises the hardware; keep it for evaluation only.
• AWQ-Int4 — works, but slower on Blackwell-native code paths. Don’t ship it on a Spark.

If the Qwen3.8-27B NVFP4 ModelOpt checkpoint is still missing, the Unsloth NVFP4 GGUF path works on Blackwell with the same caveats: “NVFP4 requires NVIDIA’s Blackwell GPUs like RTX 50X, DGX Spark … For older GPUs, our GGUFs work well!”

Operator recipe (current state)

Single-node, tensor-parallel 1 (the model fits on one GB10, no sharding needed), full long context. This serves the Qwen3.6-27B-NVFP4 stand-in until NVIDIA publishes the Qwen3.8-27B-NVFP4 checkpoint:

# Container: vllm/vllm-openai:v0.24.0-ubuntu2404 (or a GB10-specific image from the awesome-dgx-spark list)
vllm serve nvidia/Qwen3.6-27B-NVFP4 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --gpu-memory-utilization 0.5 \
  --max-model-len 262144 \
  --max-num-seqs 8 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill \
  --async-scheduling \
  --enable-prefix-caching \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}' \
  --load-format fastsafetensors \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder \
  --enable-auto-tool-choice
# When NVIDIA ships nvidia/Qwen3.8-27B-NVFP4, swap that path in.

Three flags are not optional on this recipe:

• --speculative-config '{"method":"mtp","num_speculative_tokens":3}' — MTP with a draft of 3 tokens. This is mandatory for the 100+ tok/s tok/s class shown in the reference benchmark below. Without MTP you drop into the 50–80 tok/s range. If a benchmark for Spark claims 100+ tok/s without MTP, treat it as marketing copy.
• --gpu-memory-utilization 0.5 — the recipe’s setting. Spark’s unified-memory architecture treats more aggressive values as overcommit and the kernel will OOM-kill the container under burst load.
• --enable-prefix-caching — amortises repeated system prompts and tool scaffolds across requests. Combined with chunked prefill (which lets large prompts not block short ones), this is what makes the 16-concurrent class work on a single box.

Reference benchmark (Qwen3.6-27B, not Qwen3.8-27B)

The published tok/s for this exact stack on a single GB10 sm_121, on the Qwen3.6-27B-NVFP4 checkpoint:

“Qwen3.6-27B NVFP4 serving pack for GB10 (sm_121) with vLLM native MTP and FP8 KV cache, 144 tok/s at concurrency 16 and 81.88% full-set GSM8K.” — r0b0tlab, GitHub repo (Qwen3.6-27B), 2026.

This is a Qwen3.6-27B measurement, not Qwen3.8-27B. The Qwen3.8-27B NVFP4 measurement will land here when NVIDIA ships the checkpoint and r0b0tlab (or another community benchmarker) re-runs on this exact stack.

What token throughput class to expect on Spark (Qwen3.8-27B)

The Spark can do 100+ tok/s class on Qwen3.8-27B. Two conditions:

1. NVIDIA publishes the Qwen3.8-27B NVFP4 checkpoint (huggingface.co/nvidia/Qwen3.8-27B-NVFP4 — currently 401).
2. vLLM 0.24+ with the GB10-specific image and the --speculative-config '{"method":"mtp","num_speculative_tokens":3}' flag.

Until both, the operator receiving this article should treat the recipe as target stack validated, current-gen throughput pending.

Two cross-checks

• Unsloth’s own measurement on 1× B200 (a different Blackwell silicon, but a useful cross-check) gives NVFP4 measured 1.49× faster than BF16 at batch 1 (89.8 → 133.7 tok/s per user) and stays ~1.45× faster at concurrency 64 (3048.5 → 4407.2 tok/s aggregate), with NVFP4 retaining 92–97% top-1% accuracy vs BF16:

  “On 1× B200, NVFP4 is 1.49× faster than BF16 at batch 1 (89.8 → 133.7 tok/s per user), and stays ~1.45× faster at concurrency 64 (3048.5 → 4407.2 tok/s aggregate). NVFP4 retains 92 to 97% top-1% accuracy vs BF16.” — Unsloth Documentation, Qwen3.8 page, Aug 2026.

• The Atlas (lightseek/tokenspeed) inference engine, also targeting DGX Spark, reports +18% over vLLM-Spark with AWQ on the same model size as an alternative-runtime cross-check:

  “Atlas + Qwen 27B NVFP4 has +18% faster inference speed compared to vllm-spark-docker + Qwen 27B AWQ.” — starkrun, NVIDIA dev forum, 2026.

  Worth trying if you need to squeeze the last drop after the vLLM+NVFP4 baseline works.

Pitfalls that will bite you

1. Stock vLLM image missing sm_121. Symptom: silent fallback to FP8 or a CUDA error on first decode. Fix: GB10-specific image (from awesome-dgx-spark) or build from source.
2. AWQ-Int4 ≠ NVFP4 on Blackwell. The ~18 tok/s figure above is the canonical proof. Don’t ship AWQ on a Spark.
3. MTP off. Single biggest throughput mistake. The --speculative-config line above is mandatory for any 100+ tok/s class claim.
4. One 128 GB pool. CPU and GPU share it. A 30 GB embedder + 22 GB model + 8 GB RAG reranker leaves you ~70 GB, not 128. Community-reported practical ceiling: 115–120 GB usable for model + KV combined.
5. Context creep. KV cache is ~64 KB/token in BF16 or ~32 KB/token in FP8. At 262K context with 8 concurrent streams that’s ~64 GB of KV alone. Start at 8K–16K and raise only when a workload actually needs longer.

Bottom line

The only stack that earns DGX Spark for Qwen3.8-27B is vLLM 0.24+ with the ModelOpt NVFP4 checkpoint (Qwen3.6-27B stand-in until NVIDIA ships the Qwen3.8 NVFP4), FP8 KV cache, MTP n=3 on a GB10-specific image with --gpu-memory-utilization 0.5 and prefix caching on. The Qwen3.6-27B reference reports 144 tok/s at concurrency 16; Qwen3.8-27B is expected in the same class on the same stack once the checkpoint ships.

AWQ, BF16, and stock vLLM images are all wrong-stack choices.

Done means

• vllm serve nvidia/Qwen3.6-27B-NVFP4 is up on a GB10-specific image. (When the Qwen3.8 NVFP4 checkpoint ships, swap the path; expected in the same tok/s class, not benchmarked yet.)
• --speculative-config '{"method":"mtp","num_speculative_tokens":3}' is set — MTP is on.
• --gpu-memory-utilization 0.5 and FP8 KV cache are configured; 262K context is reachable.
• Single-stream throughput is in the same general class as the RTX 3090 reference (~30–60 tok/s baseline). With 16 concurrent streams the stack reaches the 100+ tok/s class — Qwen3.6-27B reference reported 144 tok/s, Qwen3.8-27B benchmark pending.
• When NVIDIA publishes the Qwen3.8-27B NVFP4 checkpoint, swap to nvidia/Qwen3.8-27B-NVFP4 and re-measure against this Done means.

What this article does NOT cover

• Mac/Apple Silicon or single-RTX-3090 setup — see those guides.
• Training or fine-tuning — weights are inference-only.
• The Qwen3.8-2.4T-A95B variant — different architecture, far larger memory footprint.
• Cluster or multi-Spark deployments — this is single-node.

Sources

• Qwen3.8-27B - Hugging Face model card
• vLLM Recipes - Qwen3.6-27B DGX Spark (NVFP4) — the canonical recipe until Qwen3.8 ships
• NVIDIA ModelOpt NVFP4 checkpoint (Qwen3.6-27B; Qwen3.8-27B NVFP4 not yet published)
• Unsloth - Qwen3.8 docs
• r0b0tlab - Qwen3.6-27B NVFP4 GB10 sm_121 reference benchmark (~144 tok/s at concurrency 16) (Qwen3.6 reference, NOT a Qwen3.8 measurement)
• awesome-dgx-spark - curated vLLM/GB10 list
• Reddit r/LocalLLM - Qwen3.6-27B AWQ on DGX Spark (~18.49 tok/s) (behind paywall; verified manually on 2026-08-15)
• Research brief this article was built from: coordinate/abs-research-briefs/qwen3.8-27b-hardware-tuning-brief.md (R2, Marvin-authored, 2026-08-14)