ddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw

mlx-serve 26.10.2 · Apple M5 Ultra · 256GB · darwin/arm64 · 2026-10-15 · mtp-tokenv3-0.95 no-attn-quant

Scores

Coverage100%core tier
Conformance100%209/210 MUST checks
Capability100%strong
Agentic88%7/8 tasks
Fidelity100%36 items

Performance

Decode166tok/smedian of 3 · 157.8–176.5
Prefill5304tok/s5280–5307 · 10406 prompt tokens
First token131ms130–131
Speculative2.03×273.4 predictable vs 134.9 novel tok/s · effective
Prefix cache6.4×402 → 63 ms first token
Concurrency0.4180.3 aggregate vs 114 tok/s alone (4 streams)
Sustained load-8.3%166 → 152.2 tok/s

Context scaling

Decode tok/s
0881775124.1k8.2k16.4kddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 512: 138.6 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 4.1k: 151.4 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 8.2k: 153.8 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 16.4k: 151.3 tok/s
First token ms
0171234255124.1k8.2k16.4kddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 512: 238 msddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 4.1k: 884 msddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 8.2k: 1571 msddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 16.4k: 2978 ms
Prefill tok/s
0314962995124.1k8.2k16.4kddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 512: 2151 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 4.1k: 4800 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 8.2k: 5245 tok/sddalcu/Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw · 16.4k: 5477 tok/s

Capability by category

tool-selection
100%6/6
tool-restraint
100%6/6
tool-args
100%6/6
multiturn
100%3/3
instructions
100%3/3
json-discipline
100%6/6
reasoning
100%3/3
knowledge
100%3/3

Conformance by surface

models
100%4/4
chat
100%71/71
responses
100%66/66
messages
98%61/62
embeddings
100%4/4
completions
100%3/3

Coverage by tier

core
100%9/9
extended
93%14/15
frontier
56%5/9

report.json · measured with llmprobe