Xing4.0-29B-Intent Local Deploy: 4B Active, Which Command?

A support ticket arrives with one line: "find me a flight from Beijing to Shanghai tomorrow." Two things must come out: is this one of our known intents, and what are the parameters? Historically you either hand-wrote regular expressions or handed the sentence to a general-purpose model and hoped it would not get creative. Xing4.0-29B-Intent merges the two into one forward pass: give it categories and field definitions, and it emits category + args. This manual covers weights, service startup, SDK integration, acceptance testing, retraining, and boundaries. Repo: github.com/XingChen-AGI/Xing4.0-29B-Intent

Repository: https://github.com/XingChen-AGI/Xing4.0-29B-Intent

Division of labor first: the companion piece on this site covers the release and the specialist-versus-general debate; this one answers the more elementary questions — which command you type, and how you verify the output.

Step 1: What problem this model solves

The official positioning is a "schema-guided model for intent recognition and structured information extraction." Four tasks share one output protocol:

The output is always a JSON array where each item carries a category and a set of args; args is empty when there is nothing to fill, and one sentence matching several categories or one category with several parameter instances both fit this structure.

Schema-guided means categories and fields are not the model's guesses: the caller supplies candidate category names, what each means, and the extraction rule for every field. Swapping the schema moves the protocol to another business. In the official example, "find me a flight from Beijing to Shanghai tomorrow" with a Flight Search category returns the category name plus {'origin': 'Beijing', 'destination': 'Shanghai', 'date': 'tomorrow'} — "tomorrow" survives verbatim, never resolved into a calendar date.

Step 2: Pre-flight checklist

The model repository is github.com/XingChen-AGI/Xing4.0-29B-Intent, Apache-2.0, created 2026-10-17, last pushed 10-19, 17 stars, 0 open issues (GitHub API snapshot).

Weights live in three official Hugging Face repositories (sizes from the HF API, measured 2026-10-22):

All three carry the apache-2.0 tag, ModelScope hosts mirrors under the same ids, and there is no official AWQ or GPTQ release — FP8 and GGUF are the only quantization paths.

Host requirements follow the base model's deployment tutorial: Docker 24.0+, NVIDIA driver 535+, the NVIDIA Container Toolkit, and for the SDK Python 3.8+ with httpx 0.24+. One disambiguation: the clean Python 3.12 virtual environment you may have read about is the DeepInfer framework's requirement, not this model's.

One licensing note: weights, public data, code, and the Python SDK are all Apache-2.0, friendly for commercial use — but the dataset README says the original license terms of each public source dataset still apply, so check them before going commercial.

Step 3: Starting the service, where the commands live

This is where people get stuck. The Intent repository's README contains no startup command — it points to the base model's "service-oriented inference" instructions and tells you to swap the model path, set --served-model-name to Xing4.0-29B-Intent, and configure VRAM, parallelism, and context length for your environment. The commands come from github.com/XingChen-AGI/Xing4.0-29B-A4B. Three prebuilt Docker images ship, so no framework compilation is needed: xingchen-inference-vllm:v0.29.1rc1-xing4_0, xingchen-inference-vllm-ascend:v0.26.0rc1-xing4_0 for Ascend, and xingchen-inference-sglang:v0.5.20.rc1-xing4_0 for SGLang, all under quay.io/xingchen-agi/. The vLLM pull:

docker pull quay.io/xingchen-agi/xingchen-inference-vllm:v0.29.1rc1-xing4_0

The official vLLM startup command, with the model path pointed at your Intent weight directory:

vllm serve ${MODEL_PATH} \
    --host 0.0.0.0 \
    --port 8000 \
    --served-model-name Xing4.0-29B-Intent \
    --tensor-parallel-size 2 \
    --trust-remote-code \
    --max-model-len 262144 \
    --gpu-memory-utilization 0.90 \
    --max-num-seqs 4 \
    --enable-auto-tool-choice \
    --reasoning-parser xing4_0 \
    --tool-call-parser xing4_0 \
    --speculative-config '{"method":"mtp", "num_speculative_tokens": 1}'

The model name must match the model field in client requests, or the API reports the model as missing; --reasoning-parser and --tool-call-parser must both be xing4_0.

The official SGLang command:

sglang serve --model-path ${MODEL_PATH} \
   --trust-remote-code \
   --host 0.0.0.0 \
   --port 8000 \
   --served-model-name Xing4.0-29B-Intent \
   --tp-size 2 \
   --context-length 262144 \
   --mem-fraction-static 0.90 \
   --max-running-requests 4 \
   --reasoning-parser xing4_0 \
   --tool-call-parser xing4_0 \
   --speculative-algorithm EAGLE

--tp-size is SGLang's tensor-parallel degree, the same knob as vLLM's --tensor-parallel-size; on both sides it must divide the number of visible GPUs.

If you would rather not use the prebuilt images, open-source plugins exist — XingChen-AGI/xing4-vllm-plugin and XingChen-AGI/xing4-sglang-plugin — installed with pip install -e /workspace/xing4-vllm-plugin --no-deps; verify afterward that the entry point output contains ('xing4_0', 'vllm_xing4_0_plugin:register').

What does official documentation say about VRAM? The deployment tutorial gives no hard floor. Its example runs on NVIDIA H100, 80 GB per card and 160 GB across two, explicitly labeled "the hardware environment used for demonstration and verification, not a mandatory requirement." The tunables are the reservation ratios --gpu-memory-utilization 0.90 (vLLM) and --mem-fraction-static 0.90 (SGLang), and the official out-of-memory advice is threefold: lower the reservation ratio or --max-model-len (131072), switch to FP8, or reduce --max-num-seqs. An official member has also given a capacity reference in the discussion area of the base repository (Xing4.0-29B-A4B, HF discussion 9): a single H100 80G can reach or come close to the full 256K context at one concurrent request, and 512K needs roughly double the KV cache headroom — with the official caveat that actual usage depends on quantization and framework overhead.

One inconsistency inside the official docs is worth flagging early: the vLLM plugin tutorial states that mHC mode (hc_mult > 1) only supports bfloat16 and that "FP16 or quantized paths will error out," while the out-of-memory section of the same document recommends the FP8 version. The official side does not adjudicate — reproduce both as written, then determine empirically on your own hardware.

Step 4: SDK integration

The SDK lives in the repository's XingIntentSDK directory and installs from source with python -m pip install -e . run from the repository root.

The minimal runnable example, straight from the official README:

from xingintent import Arg, Intent, IntentClient

client = IntentClient(
    base_url="http://localhost:8000/v1",
    model="Xing4.0-29B-Intent",
    max_tokens=512,
)
result = client.predict(
    text="Find me a flight from Beijing to Shanghai tomorrow.",
    intents=[
        Intent(
            "Flight Search",
            "Search for flights by date and route",
            args=[
                Arg("origin", "City of departure"),
                Arg("destination", "City of arrival"),
                Arg("date", "Travel date the user mentioned, keep the original wording"),
            ],
        )
    ],
)
print(result.category)
print(result.args)

Intent takes the category name, its description, and the argument list; Arg takes the field name and description — and that description is the extraction rule handed to the model, not decoration.

result.category returns the primary category and result.args the merged parameter dictionary. The detail people trip over: result.args covers only the first intent, and when two parameters share a name the later value wins — with multiple categories iterate result.intents, whose entries each carry their own args list:

for item in result.intents:
    print(item.category, item.args)

Practical features, briefly: Intent.fallback("Other") builds a catch-all intent reported by result.is_fallback; guided_json=True generates a JSON Schema passed through response_format for constrained decoding; async, file schemas, and multi-turn each have their own entry points. Exceptions are typed (APIError, ParseError, ValidationError), and debug=True logs the request body and raw output.

Step 5: Knowing whether it ran correctly

Start with the official evaluation protocol so you do not mistake a leaderboard number for a business metric: the official test set is 6,000 samples from 48 public datasets covering four task types, each task score is the macro average of field-level Micro-F1, the total is the equal-weight average, on a 100-point scale — and the official text stresses that this does not equal business accuracy.

For your own business, the only meaningful acceptance test is a regression set you build:

1. Sample hand-labeled sentences from real traffic: high-frequency intents, long-tail intents, ambiguous sentences, and pure chitchat.
2. Write the business schema out as JSON, load it with Schema.from_json, and run every sentence through predict.
3. Parse with json.loads and compare structure, never raw strings — string comparison counts JSON key ordering as errors.

Align with the official prompt conventions first: every request sends a single user message containing the task description, the schema pasted in verbatim, and the conversation; category and slot key names must not be rewritten, and test labels must never enter the request. Single-label tasks allow one category object per sample, multi-label tasks such as NER, RE, and EE allow several — change any of this and you must re-validate.

Step 6: Data and retraining

The companion dataset is XingChen-AGI/Xing4.0-29B-Intent-oss, Apache-2.0. Per the official data README: 2,753,000 total samples restructured from 61 public datasets, covering 19 domains, 52 languages, 4 task types (NER / RE / EE / IR-SLU), plus a standard label set with more than 170 intents and 148 slots usable as a new project's starting schema.

For business fine-tuning, everything is unified into category + args: sample from the open data, mix with your own business data at a ratio you choose, and supervise-fine-tune. The repository's data/build_examples.py assembles samples into the intermediate training format:

python3 data/build_examples.py --root data/General/test --save_path out/test/

It processes every dataset file under one split and writes one examples file per dataset; --fmt json and --max_samples N handle format and quick debugging. Details live in data/README_zh.md. For an external yardstick, the same lab built CMPA-SLU, a Chinese spoken language understanding benchmark, at github.com/XingChen-AGI/CMPA-SLU, with the paper accepted at EMNLP 2026.

Step 7: Why a 29B model activates only 4B

The MoE accounting is spelled out in config.json, read from the Intent weights:

Every layer holds 64 routed experts with one shared expert always resident, each token routes to 4, and the first two dense layers are not sparsified — 29B total parameters with roughly 4B per token.

One misreading needs correcting: four billion activated refers to compute, not VRAM occupancy. All 29B of weights stay resident — hence the 62.4 GB bfloat16 checkpoint; the only ways to cut memory are quantized weights or a shorter context. MLA low-rank projections (q 768, kv 512), 40 layers, and a hidden dimension of 3584 are what let a 256K context hold up.

Two field details matter when you reproduce this. First, config.json reports transformers_version 5.14.1 while generation_config.json reports 4.48.1, unexplained officially — suspect this first if loading misbehaves. Second, max_position_embeddings is 262144, so 256K natively, but the usable length depends on deployment configuration; --max-model-len 262144 is only an upper bound and the official fallback advice is 131072.

Generation defaults: temperature 1.0, top_p 0.95, repetition_penalty 1.05 (base README recommends 0.8 for code and agent work). The SDK sends requests with temperature 0 and chain-of-thought disabled, which fits structured extraction.

Step 8: Boundaries, pitfalls, licensing

The officially stated boundaries: the model responds quickly on key decision information and does not do long-form reasoning; business effectiveness must be validated against your own target data and schema; the published score is not business accuracy.

On licensing, weights, public data, and code including the Python SDK are all Apache-2.0, verified against the repository's LICENSE text — but remember the source-dataset license check.

On community issues, the Intent repository has zero open issues. The base repository has 11, mostly documentation and feature requests; issue 4, "want to add domestic configuration requirements," is still open — production-grade parameters for domestic accelerators such as Ascend have not been written down officially, so measure them on your own cluster. Separately, the config shows one NextN draft head layer (num_nextn_predict_layers: 1), the premise behind the --speculative-config mtp method; if your checkpoint lacks that head, remove the speculative decoding parameters.

Step 9: Pre-launch checklist

• Weights come from one of the three official repos, license apache-2.0
• Service is up: vLLM logs Uvicorn running on http://0.0.0.0:8000, SGLang logs The server is fired up and ready to roll!
• --served-model-name matches the client model field, both Xing4.0-29B-Intent
• The client sends one user message with the schema pasted in verbatim
• The regression set passes, compared by structure rather than by string
• Under VRAM pressure: lower the reservation ratio, lower --max-model-len, switch to FP8, cut concurrency
• Multi-category cases iterate result.intents, not result.args

FAQ

Q1: The Intent repository README has no startup command. Did official simply not publish one? A: It is published, just split across repositories — the official text points to the "service-oriented inference" section of Xing4.0-29B-A4B, which carries the full commands, prebuilt images, and parameter tables. On the Intent side you swap the model path and set --served-model-name to Xing4.0-29B-Intent.

Q2: 29B total parameters but only 4B active. Can a single consumer GPU run it? A: Two different things. Four billion activated saves compute, not VRAM — all weights stay resident, and the bfloat16 checkpoint is about 62.4 GB. Fitting a consumer card requires the official GGUF version at roughly 17.8 GB; the base repository's llama.cpp document tested an RTX 3090 24 GB with the quantized base model, but that tested the base model and the llama.cpp adaptation PR has not been merged.

Q3: If only 4B is activated, why does the VRAM requirement still track 29B? A: Because in MoE every expert weight must be resident in VRAM; each token merely routes to 4 of the 64 experts. The deployment documentation gives no hard VRAM floor, using H100 at 80 GB per card and 160 GB across two only as a labeled demonstration example; an official member's community reference puts a single H100 80G at roughly full 256K context for one concurrent request.

Q4: How do I know the model ran correctly instead of returning well-formed JSON with wrong content? A: Well-formed is not the same as correct. The official protocol (6,000 samples, 48 datasets, macro-averaged Micro-F1, equal-weight total) does not equal business accuracy. The only meaningful acceptance test is a regression set you build: sample real traffic, hand-label it, compare category and fields by structure, and walk result.intents for multi-category cases.

Q5: Which quantized version should I use, FP8 or GGUF, given the official docs seem to conflict? A: The docs really are inconsistent: the vLLM plugin tutorial says mHC mode only supports bfloat16 while the out-of-memory section of the same document recommends FP8, and the official side does not adjudicate. Get the pipeline running on bfloat16 first, then run the same regression set against FP8 and GGUF, and decide with business metrics rather than with the memory you saved.

Related reading

• AI agent tool calling setup SOP
• Context compression in practice
• Running MiniCPM-V 4.7: three routes from phone to desktop