LingBot-World 2.0 Local Small Model Deployment SOP

Who Should Read This

LingBot-World 2.0 (repo alias LingBot-World-Infinity) is Robbyant's open-source "world model" video system, built on Wan2.2 (Wan-Video). Its core is causal, chunk-by-chunk inference: given a first frame plus a camera/action path, it keeps generating interactive video along that path. Input is not just a prompt; it is one image plus one camera/action path plus one textual event. Output is a long video extending along the path.

This SOP targets Chinese technical practitioners who want to run this system locally, reproduce the official demo, and build on it. Keep reading if you are a researcher on world models or video generation needing a local base; an engineering lead checking feasibility before productization; or a developer already on the Wan family who wants interactive long video in an existing pipeline.

Skip this if you only want to click and play (use Reactor for international web, or LingGuang for domestic mobile); if you plan to ship a commercial product (read the license section first, CC BY-NC-SA 4.0, non-commercial); or if you have only one consumer GPU under 24G (there is no single-card real-time path in the repo). One-line judgment: this is a hands-on manual to get the official repo running and stable, not a concept primer and not a commercial plan.

Environment and Dependencies

Clarify the hardware bar first, because the repo is internally inconsistent here and I list the versions honestly.

On hardware, the 1.3B model has two official reference configs. The README example uses 4 GPUs (--nproc_per_node=4), while run_fast.sh comments state the 1.3B reference is 2 GPUs (ulysses_size=2, the 1.3B causal-ODE CP=1 recipe). The 14B model uses 8 GPUs. Media and the official demo claim "1.3B Small runs in real time on a consumer single GPU", but that has no reproducible command in the repo. Taking the repo as truth, the minimum reproducible reference for 1.3B is 2 GPUs; single-card real time is officially unconfirmed. For the easy path, use run_fast.sh, which infers GPU count from the weights directory name.

Key dependencies from requirements.txt: torch>=2.4.0, torchvision>=0.19.0, diffusers>=0.31.0, transformers>=4.49.0,<=4.51.3 (note the upper bound), flash_attn, numpy>=1.23.5,<2 (numpy 2.x breaks, pin to 1.x), opencv-python>=4.9.0.80. Use Python 3.10 and CUDA 12.1+, matching Wan2.2.

Install commands, verbatim from the official files:

git clone https://github.com/robbyant/lingbot-world-v2.git
cd lingbot-world-v2
pip install -r requirements.txt
pip install flash-attn --no-build-isolation

flash-attn needs --no-build-isolation because it compiles from source against the current torch. Install it last and separately so failures are easy to locate. If you already have a Wan2.2 environment, reuse it; for install problems, consult Wan-Video docs, since this repo is modified from Wan2.2.

Weight Download (Including 1.3B Missing-Asset Handling)

Four variants, by hardware and quality:

• lingbot-world-v2-14b-causal-fast: 14B, distilled few-step, default mode.
• lingbot-world-v2-14b-causal-pretrain: 14B, pretrained causal, higher quality, slower.
• lingbot-world-v2-14b-bid: 14B, bidirectional.
• lingbot-world-v2-1.3b-causal-fast: 1.3B, distilled few-step, small-model choice.

Download with huggingface-cli, verbatim. Note the 1.3B --local-dir carries a /transformers suffix; this is official, do not remove it:

pip install "huggingface_hub[cli]"
huggingface-cli download robbyant/lingbot-world-v2-1.3b-causal-fast --local-dir ./lingbot-world-v2-1.3b-causal-fast/transformers

The 14B download has no suffix:

huggingface-cli download robbyant/lingbot-world-v2-14b-causal-fast --local-dir ./lingbot-world-v2-14b-causal-fast

In regions where HuggingFace is slow, use ModelScope: modelscope download robbyant/... --local_dir ./..., swapping the command, with the same directory structure.

The key pitfall: the 1.3B package contains only DiT weights; T5, VAE, and the tokenizer are shared with 14B. Download a 14B directory too, then point --assets_dir at it during inference (or use the third argument of run_fast.sh). I strongly recommend small-model users keep the 14B weights as an asset library; disk for stability is worth it.

Run Your First Clip

Goal: generate one official example clip end to end to prove environment, weights, and parallel config are correct.

The simplest path is the official run_fast.sh. It infers task and GPU count from the directory name: 1.3b or 1p3b means 1.3B, NPROC=2, GPU 0,1; otherwise 14B, NPROC=8, GPU 0-7. The third argument is assets_dir, mandatory for 1.3B:

bash run_fast.sh lingbot-world-v2-1.3b-causal-fast 361 lingbot-world-v2-14b-causal-fast

This equals the full torchrun form below (README 1.3B example, 4 GPUs). Note --assets_dir points at 14B:

torchrun --nproc_per_node=4 generate.py --task i2v-1.3B --size 480*832 --ckpt_dir lingbot-world-v2-1.3b-causal-fast --assets_dir lingbot-world-v2-14b-causal-fast --image examples/03/image.jpg --action_path examples/03 --dit_fsdp --t5_fsdp --ulysses_size 4 --frame_num 361 --local_attn_size 18 --sink_size 6 --prompt "A serene lakeside scene with a lone tree standing in calm water, surrounded by distant snow-capped mountains under a bright blue sky with drifting white clouds — gentle ripples reflect the tree and sky, creating a tranquil, meditative atmosphere."

For the 14B pretrained causal model (higher quality, 40 steps per chunk with CFG), the official command is:

torchrun --nproc_per_node=8 generate.py --task i2v-A14B --infer_mode causal_pretrain --size 480*832 --ckpt_dir lingbot-world-v2-14b-causal-pretrain --image examples/03/image.jpg --action_path examples/03 --dit_fsdp --t5_fsdp --ulysses_size 8 --frame_num 81 --prompt "A serene lakeside scene with a lone tree standing in calm water, surrounded by distant snow-capped mountains under a bright blue sky with drifting white clouds — gentle ripples reflect the tree and sky, creating a tranquil, meditative atmosphere."

Success is an .mp4 under output/ whose name contains causal_fast or causal_pretrain, size, ulysses_size, and a timestamp. Import or compile errors mean check versions; an assert about attention heads means read ulysses_size below.

Parameter Tuning: Frames, Window, Chunk, Seed

Defaults below come from the argparse in generate.py.

--frame_num: frames to generate, must be 4n+1 (81, 161, 361). Sets duration with cfg.sample_fps. Longer means more memory and time.

--local_attn_size: KV-cache local attention window, default -1 (use config). Official uses 18. Controls how many past frames the model sees; too small breaks long-range consistency, too large costs memory and compute.

--sink_size: KV-cache sink size, default 0, official uses 6. With local_attn_size it shapes the window; keep the official combo for long video.

--chunk_size: frames per chunk, default 4. The streaming unit; usually keep default.

--base_seed: seed, default 42. Fix to reproduce, change for variety. Rank 0 broadcasts it under multi-GPU, so set one.

--save_dir: output directory, default output.

--size: area width*height, default 1280*720, official uses 480*832. Under i2v the aspect follows the input image, so this mainly sets total area and compute.

The critical parallel arg is --ulysses_size: Ulysses degree, default 1, must divide head count. generate.py asserts cfg.num_heads % ulysses_size == 0. 1.3B has 12 heads (use 2 or 4); 14B has 40 (official 8). Wrong value raises cannot be divided evenly. Also ulysses_size must equal world_size (nproc_per_node), so change both together.

--infer_mode: causal_fast (default, 4 steps per chunk, no CFG, fast) or causal_pretrain (40 steps per chunk, with CFG, slower, higher quality). --dit_fsdp/--t5_fsdp shard across GPUs, required multi-GPU. --t5_cpu puts T5 on CPU to save GPU memory but slower. --offload_model unloads to CPU after each forward; turn these on in order when memory is tight.

Production and Deployment Paths

The official repo states "We do NOT plan to release our deployment code". For a service, high concurrency, or low latency, build it yourself; do not wait for an official solution.

Two reference paths. First, the SGLang LingBot-World cookbook at https://docs.sglang.io/cookbook/diffusion/LingBot-World/LingBot-World-2.0, which targets diffusion deployment with a ready pattern. Second, NVIDIA's flashdreams repo. Both are references, not turnkey; you must wire weights, pre/post-processing, and scheduling yourself.

On scaling: ulysses_size is sequence parallelism bounded by head divisibility (max 12 for 1.3B, 40 for 14B). For more throughput, layer FSDP and data parallelism on top. But generate.py is single-node multi-GPU; cross-node, dynamic batching, and streaming serving are outside its scope and must be built on SGLang or flashdreams. Use generate.py for local work, a separate stack for production.

Before you provision hardware, do the arithmetic honestly. Sequence parallelism is bounded by head divisibility, so a 1.3B run tops out at 4 ranks and a 14B run at 8; adding more GPUs past that point buys nothing for a single clip and only helps if you also shard weights with FSDP or run several independent jobs. For a research loop, two to four cards is a realistic floor, and one long clip still occupies those cards for its whole duration, so you cannot batch many clips on the same rig without a queue. Treat throughput as clips per hour per GPU set, measure it once at your own resolution and frame count, and budget from that number instead of vendor framing. Because the official team ships no serving layer, the request queue, scheduler, health checks, and retry logic are all yours to write, so budget engineering time alongside GPU hours.

Pitfalls (Eight Notes)

1. The 1.3B package has only DiT weights. T5, VAE, and the tokenizer are shared with 14B, so point --assets_dir (or run_fast.sh's third arg) at the 14B directory, or it will not run. The most common newcomer pitfall.

2. Hardware bar has three conflicting versions, listed honestly. README 1.3B example uses 4 GPUs; run_fast.sh comment says 2 GPUs (12 heads, ulysses divides 12, the 1.3B causal-ODE CP=1 recipe); media and demo claim consumer single-card real time. Conclusion: repo truth is 2 GPUs minimum; single-card real time is unconfirmed in the repo.

3. ulysses_size must divide head count. 1.3B=12 heads (2 or 4), 14B=40 heads (8). Wrong value triggers the assert in generate.py.

4. causal_fast versus causal_pretrain. Fast is distilled, 4 steps per chunk, no CFG, default. Pretrain is 40 steps per chunk, with CFG, slower, higher quality. Do not demand one from the other.

5. No open-source deployment code. For production use SGLang or flashdreams yourself; do not expect generate.py as a service.

6. License is CC BY-NC-SA 4.0, non-commercial. Confirm authorization before any commercial use; derivatives must share the license. A legal red line, not a detail.

7. Built on Wan2.2. For install and compile issues, Wan-Video docs are the fastest reference, same origin.

8. Frame count and window params strongly affect long video. --frame_num sets duration; --local_attn_size 18 and --sink_size 6 are the official long-video combo; --chunk_size is the streaming unit; --base_seed controls reproducibility; --save_dir controls output. Tune these together for long video.

Pre-Launch Checklist (Ten Items)

1. Repo cloned and cded in, path correct.
2. pip install -r requirements.txt and flash-attn --no-build-isolation both succeeded.
3. transformers in >=4.49.0,<=4.51.3 and numpy in >=1.23.5,<2.
4. Weights directory exists, plus a 14B directory as the --assets_dir library.
5. 1.3B command carries --assets_dir pointing at 14B, or it fails.
6. --ulysses_size divides head count (1.3B=12, 14B=40) and equals nproc_per_node.
7. --frame_num is 4n+1 and matches the wanted duration.
8. An .mp4 exists under output/, normal size, playable.
9. Chosen infer_mode matches speed/quality expectation.
10. License assessed: continue only non-commercial; confirm authorization for commercial.

Sister Articles and Further Reading

Companion pieces, already published:

• Hotspot analysis: /en/posts/lingbot-world-2-0-hotspot
• Open-source resource roundup: /en/posts/gods-eye-view-resource
• World-model comparison review: /en/posts/open-world-model-comparison-review

What You Can and Cannot Do Today (Honest Judgment)

What it can do today: on 2-plus GPUs (1.3B) or 8 GPUs (14B), it stably reproduces official-demo-grade interactive long video; it supports research reproduction and secondary development; and it lets you wire first-frame-plus-trajectory driven video into your pipeline. Quality ceiling is visible in the examples, with 14B causal_pretrain at the top.

What it cannot do: first, it cannot run in real time on a consumer single card. That is the demo/media claim with no repo reproduction path, so do not set boss expectations with it. Second, it is not a ready service; production depends on SGLang or flashdreams you build. Third, and key, it cannot be commercial. The license is CC BY-NC-SA 4.0, non-commercial. Any productization, charging, or closed distribution must confirm authorization first and prepare derivatives to be open-sourced under the same license. Think that through before investing engineering resources.

One practical way to decide is to separate evaluation from production. Use the local, single-node path to answer whether the model's output quality clears your bar for a specific scene, then decide separately whether the serving story justifies the engineering. If the answer to the first is yes and the second is no, the honest move is to keep the model inside an offline content pipeline rather than promise an interactive feature you cannot operate. Also watch the moving parts: the repo is young, the public issue list is small, and the license forbids the commercial use most product teams quietly assume, so any plan that depends on shipping it should start from the license, not from the demo video.

FAQ

Q1: I only have one consumer GPU; can I run 1.3B? A1: Repo truth is 2 GPUs minimum (ulysses_size=2). README uses 4, run_fast.sh says 2. "Consumer single-card real time" is the demo/media claim with no repo path, marked unconfirmed. Whether single card runs is unverified, so we call it unconfirmed.

Q2: Why does 1.3B fail to find T5/VAE after download? A2: The 1.3B package has only DiT weights; T5, VAE, and tokenizer are shared with 14B. Download a 14B directory and point --assets_dir at it (or run_fast.sh's third arg). This is explicit in the README, not a broken environment.

Q3: Can I set ulysses_size to 3? A3: No. generate.py asserts ulysses_size must divide head count. 1.3B has 12 heads (2 or 4); 14B has 40 (official 8). Setting 3 raises an assert. It must also equal nproc_per_node.

Q4: causal_fast or causal_pretrain? A4: causal_fast for speed (default, 4 steps per chunk, no CFG). causal_pretrain for quality (40 steps per chunk, with CFG, slower). A speed-quality trade-off, neither better; depends on scenario.

Q5: I want to ship a product; will the official team give deployment code? A5: No. The official team does not release deployment code. For production use the SGLang cookbook or NVIDIA flashdreams and build it yourself. Also the license is CC BY-NC-SA 4.0, non-commercial; confirm authorization before commercial use, derivatives under the same license.