80GB GPU or bust: local Ming-Image deploy plus layer SOP

Last month this site tore down an on-device agent; this time we take a more direct line: getting Ant Group inclusionAI's Ming-Image-0.1-Design running on local hardware. It is not a general-purpose image-quality model. It is a design-generation line that produces complete visual designs with layout and text - UI, dashboards, infographics, posters - end to end. A companion Layer model decomposes a flat design into editable RGBA layers. This article has one thread: run it on your own GPU, without a cloud API, then generate, decompose, and ship. Commands are quoted verbatim from the GitHub README and can be copied. All speed and benchmark numbers are labeled "model-card or vendor figures, not independently verified", and anything unpublished is written as "subject to official announcements".

1. Hardware Check: 80 GiB Is the Floor, Not a Suggestion

Settle the budget before the how-to. Both Ming-Image weights are in the 6B class, but the default and minimum validated deployment is a single GPU with at least 80 GiB of memory, in BF16. That is the only validated configuration stated on the model card, and it runs both the Design and Design-Layer families end to end. In other words, it is not "80 GiB recommended"; it is "below 80 GiB is outside the official validation envelope".

The repository is about 52.88 GB and the weights about 49.25 GiB (orcarouter figures, not independently verified). Under BF16 the whole chain consumes that single card's capacity with no room to maneuver.

A 24 GiB consumer card (RTX 4090 / 3090 class) has no official guidance. Hours after open-sourcing, the community produced INT4 / INT8 / FP8 / GGUF / ComfyUI builds, but these are community routes with no official endorsement. Whether it runs on 24 GiB is community exploration, and this article does not present it as a promised path.

Practically, if your only GPU today is a 24 GiB card, the honest interim path is either rent an 80 GiB-class cloud instance for the run or follow a community quantization thread and accept unverified quality. Do not expect the official line to cover you; the model card is explicit that 80 GiB is the validated floor, not a soft target.

A decision table makes the line clearer:

One sentence: confirm an 80 GiB single card first, then proceed; if not, the main path here does not apply and community quantization is a separate matter.

2. Environment: Clone the Inference Repo, Install, Set CUDA

Ming-Image has no model_index.json. The documented path is not "pip install a package and import it"; it is git clone the inference repo, then python infer.py. That is the first pitfall; clone first.

# Pick the card to use (single 80 GiB card is usually device 0)
export CUDA_VISIBLE_DEVICES=0

# Clone the inference code repository
git clone https://github.com/inclusionAI/Ming-Image
cd Ming-Image

# Install dependencies
pip install -r requirements.txt

A few reminders. First, set CUDA_VISIBLE_DEVICES before launching to avoid grabbing the wrong card or spilling OOM onto the system disk. Second, the weights are not in the code repo; at runtime they are pulled by the --model HuggingFace name (inclusionAI/Ming-Image-0.1-Design and inclusionAI/Ming-Image-0.1-Design-Layer), so the machine must reach HuggingFace (or a ModelScope mirror). Third, with FlashAttention2 installed you can add --attn-implementation flash_attention_2 to the CLI for speed; the portable CLI defaults to eager, so do not add the flag if it is not installed.

To confirm the environment without loading weights, use --validate-only (see section 7) to check the checkpoint contract before pulling tens of GiB. Budget the first run generously: the initial weight pull alone can take minutes to tens of minutes depending on bandwidth, and only after that does inference begin, so do not abort the process mid-download or you will restart the fetch.

3. Text-to-Design Inference: Produce an Image First

Text-to-design uses the text-to-image task. The README command, quoted directly, is:

python infer.py --model inclusionAI/Ming-Image-0.1-Design \
  --task text-to-image \
  --prompt "your prompt" \
  --resolution 2048 \
  --output-dir outputs/t2i

Sampling defaults (from the GitHub README, quotable): text-to-image defaults to steps 12 and CFG 1.0; the resolution bucket is 1024 / 2048, with 2048 recommended (square). --resolution takes any positive integer and snaps to the nearest bucket; --steps and --cfg can be overridden explicitly.

RGBA transparent output: Ming-Image has a native Alpha-channel VAE and can emit transparent assets directly. To make the alpha channel behave as expected, prepend the recommended prefix that ships with the repo to your prompt - the exact prefix is published in assets/ alongside the inference repo, subject to what the repo actually ships; this article does not invent one for you.

Output lands in the directory named by --output-dir (outputs/t2i). To validate the whole contract without loading tens of GiB, skip ahead to --validate-only in section 7. After the run, sanity-check the file before shipping it anywhere: open the generated PNG and confirm the alpha channel is real (a transparent checkerboard behind the subject), not a flattened RGB with a white or black background. A missing alpha usually means the recommended prompt prefix was skipped; rerun with it prepended rather than trying to key out the background by hand.

4. Layer Decomposition: Split a Design Into Editable RGBA Layers

The Design-Layer model splits a flattened design into 2 to 9 semantically independent RGBA transparent layers (text, subject, background can be edited, moved, or swapped separately). The task name is layer; the command, again quoted directly:

python infer.py --model inclusionAI/Ming-Image-0.1-Design-Layer \
  --task layer \
  --prompt "Decompose this image into N layers ..." \
  --resolution 1024

Sampling defaults (README, quoted): the layer task defaults to steps 12 and CFG 2.0; the bucket is 512 / 1024, with 1024 recommended. Drop to 512 for speed at the cost of layer-edge completeness.

The command is not the hard part; the layer plan in the prompt is. You must tell the model explicitly how many layers to produce and what role each plays - a Type Token constrains each layer's design role, Alpha-Aware Layer Optimization handles transparent edges, and Composite-Layer Stack Consistency keeps the recomposed image faithful. A vague prompt yields unstable splits.

A sketch (write your real plan; do not copy N literally):

Decompose this image into 3 layers:
1. background - solid color / gradient panel
2. main object - the product render with transparency
3. text - all headline and body copy as a separate editable layer

5. Prompt Enhancement: Normalize the Brief With a Model First

Ming-Image consumes an "8K long structured prompt" that organizes the brief into four dimensions - copy, modules, layout, visual style. But the quality of the raw short description fed to --prompt decides the output. This step happens outside infer.py, as preprocessing.

The README's prescribed approach: use Ling-3.0-flash-VL or qwen3.8-27B to rewrite a one-line brief into a Figma-style structured JSON / layered spec, then feed that to --prompt. The system prompt ships in assets/; follow its format for the steadiest result.

A minimal workflow (pseudocode; the actual tool call depends on your model SDK):

# 1) Use a PE model to expand the short brief into a structured spec
pe_prompt="Rewrite the following into a layered design JSON: a SaaS dashboard home, dark background, left nav, three data cards in the middle."

# 2) Feed the PE output to Ming-Image as --prompt
python infer.py --model inclusionAI/Ming-Image-0.1-Design \
  --task text-to-image \
  --prompt "$PE_OUTPUT" \
  --resolution 2048 \
  --output-dir outputs/t2i

The value: a short brief ("make a nice dashboard") invites free interpretation, while a structured spec pins copy, modules, and layout up front, which lifts the text and layout stability of text-to-design. PE is not part of the inference command itself, so do not expect infer.py to do it for you. Concretely, that bare brief may land on a generic template the model favors, whereas the same intent rewritten as a structured spec - dark theme, hero headline left, three tier cards, footer CTA, all copy literal - removes the ambiguity and keeps the text layer exact. The PE step is where you spend a little compute up front to avoid regenerating the whole image later.

6. Deployment: vLLM-Omni Service and ComfyUI Integration

After single images work locally, production means wrapping it as a service. The official deployment frameworks are three:

• diffusers: the standard pipeline, continuous with the infer.py above.
• vLLM-Omni: covered by official recipes / installation guides; fits turning the model into a concurrently callable inference service.
• ComfyUI: officially supported node-based workflow; fits chaining "prompt enhancement + text-to-design + layer decomposition" into a visual pipeline.

Community-side, hours after open-sourcing, INT4 / INT8 / FP8 / GGUF / ComfyUI builds appeared; these are community routes, not officially endorsed, and whether they serve your 24 GiB card is likewise subject to community testing.

Suggested shape (subject to official announcements; this article does not guess ports or image names):

Local inference (infer.py)  -->  validate output quality
        |
        v
vLLM-Omni service           -->  concurrent calls / API integration
        |
        v
ComfyUI workflow            -->  visualize PE + text-to-design + layer decomposition

Note: the Layer model and the Design model are two separate weights. When serving, either stand up two endpoints or call each within one workflow; do not mix --model.

7. Pre-Launch Checklist and Pitfall Quick Reference

Close the previous six sections into a checklist you can tick, plus a pitfall table.

Pre-launch checklist

1. Memory confirmed: single card >= 80 GiB, BF16, and export CUDA_VISIBLE_DEVICES set.
2. Repo cloned: git clone https://github.com/inclusionAI/Ming-Image done - not just a pip package (no model_index.json).
3. Weights reachable: machine can reach HuggingFace or ModelScope to pull inclusionAI/Ming-Image-0.1-Design and -Layer.
4. Validate before pull: use --validate-only to check the checkpoint contract before pulling tens of GiB.
5. Sampling defaults known: text-to-design steps 12 / CFG 1.0 / 2048; layer steps 12 / CFG 2.0 / 1024.
6. PE wired in: short brief expanded via Ling-3.0-flash-VL or qwen3.8-27B before --prompt.
7. Framework chosen: vLLM-Omni or ComfyUI, with the two weight endpoints kept distinct.

Pitfall quick-reference table

Speed and benchmark figures (such as "about 20 seconds per image", "single inference 183s vs 795s", "about 4.3x faster") all come from ai-bot and vendor claims, labeled "model-card or vendor figures, not independently verified"; this article has not independently verified them, so validate on your own hardware before launch.

FAQ

Q1: Can an RTX 4090 with 24 GiB run Ming-Image? A1: The official documentation gives no deployment guidance for 24 GiB consumer cards. Ming-Image's default and minimum validated configuration is a single card of at least 80 GiB in BF16. Community INT4 / GGUF quantization builds appeared hours after open-sourcing, but these are community routes with no official endorsement. Whether it runs on 24 GiB is community exploration, subject to official announcements.

Q2: Why do I not find model_index.json after cloning? A2: Because Ming-Image does not follow the standard diffusers package layout. Its documented path is "git clone the inference repo, then python infer.py"; weights are pulled at runtime by the --model HuggingFace name. Absence of model_index.json is expected behavior, not a wrong download.

Q3: Are the sampling defaults the same for text-to-design and layer decomposition? A3: No. Text-to-design (text-to-image) defaults to steps 12 and CFG 1.0, with 2048 recommended. Layer decomposition defaults to steps 12 and CFG 2.0, with 1024 recommended (drop to 512 for speed). Both come from the GitHub README and can be quoted directly.

Q4: How do I make sure the RGBA alpha channel takes effect? A4: Ming-Image has a native Alpha-channel VAE and emits transparent assets directly. To make the alpha channel behave as intended, prepend the recommended prefix that ships with the repo to your prompt; the prefix is published in assets/ with the inference repo, subject to what the repo actually ships. Also, write the layer plan explicitly in the prompt for layer tasks; transparent edges are handled by Alpha-Aware Layer Optimization.

Q5: Can I trust the speed and comparison numbers? A5: Not as hard facts. The model card presents parts of its leaderboard and comparison results as images, with no quotable numbers and no reproducible scripts, and this site has not independently verified them. Figures like "about 20 seconds per image", "about 4.3x faster", and "12 of 12 wins" come from ai-bot and vendor claims and are uniformly labeled "model-card or vendor figures, not independently verified". Verify on your own hardware before launch.

Sources

• GitHub repository inclusionAI/Ming-Image: README (inference commands and sampling defaults quoted)
• GitHub API fact-check (2026-09-24 snapshot): 91 stars / 6 forks / MIT license
• HuggingFace inclusionAI/Ming-Image-0.1-Design and -Layer; ModelScope mirror
• orcarouter.ai and other evaluation figures (speed / benchmark labeled vendor figures, not independently verified)
• Related reading: Chinese version Ming-Image local deploy SOP; our SOP for running an agent on Android

Note: commands and sampling defaults are from the GitHub README and quotable; speed / benchmark figures are vendor figures and not independently verified; anything unpublished is subject to official announcements.