Qwen3.8-Flash-Next Full-Stack Deployment SOP: 125B Main Model plus 51B N-gram Embeddings, Three Tiers from Hosted API to Apple Silicon
A three-tier route for taking Qwen3.8-Flash-Next from "it runs" to "it runs cheaply". The managed tier needs no ops: the QwenCloud API speaks both OpenAI and Anthropic specs, and QwenWork's Standard mode is powered by this model. For self-hosted serving, four commands quoted verbatim from the official README: transformers serve (--continuous-batching), SGLang (--tp-size 4 --context-length 262144 --reasoning-parser qwen3 --tool-call-parser qwen3_coder), vLLM (--tensor-parallel-size 4 --max-model-len 262144 --enable-auto-tool-choice) and TokenSpeed, all exposing an OpenAI-compatible API at localhost:8000/v1. Local and edge paths include GGUF builds via llama.cpp, mlx-vlm on Apple Silicon, and Unsloth. The engineering detail most worth remembering: the extra 51B of N-gram embeddings can be offloaded to host memory and overlapped with model compute through async prefetch. Because the README gives no official VRAM baseline, this SOP refuses to guess a hardware floor and marks it as "defer to the official recipe and your own measurements". Also covers the trade-offs of YaRN extrapolation to 1M, fine-tuning framework choices (Unsloth, Swift, Llama-Factory) and seven pitfalls - the first being that the GitHub repo ships no LICENSE file, so check the model page before commercial use.