A 3B Model That Chews 200k Pages a Day: DeepSeek-OCR Deploy SOP

Open-source OCR models keep getting bigger, as if anything under thirty billion parameters no longer counts as a document model. DeepSeek-OCR goes the other way: its DeepEncoder squeezes an entire page into a few dozen to a few hundred vision tokens, while the decoder is a 3B MoE with only about 570M active parameters. A single GPU can chew through documents, and the official paper puts throughput at 200k+ pages of LLM/VLM training data per day on one A100-40G. The token math is even better: resolution modes from 64 vision tokens per page up to a 400+ tier, with 97% OCR accuracy at compression ratios under 10x.

This is a hands-on SOP: environment setup, Transformers inference, vLLM batching, API serving, and a troubleshooting checklist, with every command taken verbatim from the official READMEs. One boundary note first: this piece only covers "how to deploy". For how DeepSeek-OCR compares against GOT-OCR2.0, MinerU2.0 and others — and whose per-page token bill is smaller — see our companion review (opensource-ocr-models-comparison-review). Picking a model is the review's job; installing one is this article's.

One: Two Generations, Two Routes

There is a trap to clear before touching anything: DeepSeek-OCR has two official repos, and their routes are not the same.

The second-generation repo, DeepSeek-OCR-2 (github.com/deepseek-ai/DeepSeek-OCR-2), released January 27, 2026 under the Apache-2.0 license, ships both transformers and vLLM scripts. But its README contains no section on upstream vLLM support. The mature ecosystem — the nightly vLLM install, the anti-garbage logits processor, the OpenAI-compatible API — all lives in the first-generation repo, DeepSeek-OCR (github.com/deepseek-ai/DeepSeek-OCR), which gained upstream vLLM support starting October 20, 2025.

So this SOP's main line is: environment setup and Transformers inference use generation 2, while the upstream vLLM path and API serving follow the generation-1 compatible route, with differences noted. The reasoning: generation 2 is the currently recommended version, so new projects can start there without migrating later; generation 1 is the proven entry point into the vLLM ecosystem, which production serving can hardly avoid. Parameters must never be mixed across generations — every command below is labeled.

Licenses first: generation 1's LICENSE file is MIT (Copyright (c) 2025 DeepSeek, verified from the repo); generation 2's LICENSE.txt is Apache-2.0, signed "Copyright (c) 2023 DeepSeek" — the year looks like a template-reuse slip. Both are commercial-friendly, but do not lazily write "MIT" for both.

Two: The Hardware Account

The weight size comes from the ModelScope file-list API: a single shard, model-00001-of-000001.safetensors, is 6,672,547,120 bytes — about 6.68GB in bf16 — plus a roughly 9.5MB tokenizer.json and the modeling_deepseekocr.py files that trust_remote_code requires. Missing any of these and loading fails.

The architecture account (paper arXiv 2510.18234): DeepEncoder carries about 250M parameters — SAM-base windowed attention, a 16x convolutional compressor, CLIP-large global attention, and MLA strung together. The decoder is DeepSeek3B-MoE-A570M: 3B total, about 570M active. The small active count is exactly why it is cheap on VRAM and fast.

Throughput uses the paper's numbers: 200k+ pages per day on a single A100-40G in production; the README separately cites ~2500 tokens/s of concurrency on the vLLM PDF path, same card. On compression: 97% OCR accuracy below 10x compression, about 60% at 20x. Downstream, that means a page becomes a few hundred tokens before it ever reaches an LLM for summarization or extraction — the harder you compress, the cheaper everything after.

Now the cold water: the vendor has never published a VRAM requirement. Community measurements (mostly from CSDN promotional posts, verify yourself) put bf16 peak usage at roughly 19.1-22.8GB depending on resolution, concluding that 24GB cards are the safe line; an A10 24GB runs 3.2-12.5 seconds per page, with the 4090 about 2.4-2.6x faster. There is also a low-end route on a 3060 12GB using sdpa attention plus tiled preprocessing. Treat all of this as community-sourced only — some of it comes from CSDN articles bundling third-party mirror promotions — not as any official figure.

Three: Environment Setup

The README's own words: "Our environment is cuda11.8+torch2.6.0", identical for both generations. Verbatim install commands (generation 2 shown; generation 1 differs only in the repo name):

git clone https://github.com/deepseek-ai/DeepSeek-OCR-2.git   # gen1: DeepSeek-OCR.git
conda create -n deepseek-ocr2 python=3.12.9 -y
conda activate deepseek-ocr2
# 需先手动下载 vllm-0.8.5 whl: https://github.com/vllm-project/vllm/releases/tag/v0.8.5
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu118
pip install vllm-0.8.5+cu118-cp38-abi3-manylinux1_x86_64.whl
pip install -r requirements.txt
pip install flash-attn==2.7.3 --no-build-isolation

Key points: Python is pinned to 3.12.9 (community reports say 3.11 fails on flash-attn); flash-attn 2.7.3 needs --no-build-isolation and must be installed after torch; the dedicated conda environment is non-negotiable because the version pins are strict. Why the manual vllm-0.8.5 wheel: the official environment is locked to cu118, PyPI's generic build does not match, so the README has you grab the wheel from the vLLM GitHub release page. If you want vLLM and transformers in one environment, the README notes that errors like "vllm 0.8.5+cu118 requires transformers>=4.51.1" can be ignored. Summary: python 3.12.9, torch 2.6.0+cu118, transformers per requirements.txt (at least 4.51.1), flash-attn 2.7.3, and vLLM either at 0.8.5+cu118 (repo-script route) or nightly (upstream route). This cuda11.8 stack looks dated in 2026 and the docs have not been refreshed — an ecosystem lag worth knowing, but it runs.

Four: Transformers Inference, Generation 2 Verbatim

The shortest route after install. Verbatim from the generation-2 README:

from transformers import AutoModel, AutoTokenizer
import torch, os
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
model_name = 'deepseek-ai/DeepSeek-OCR-2'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, _attn_implementation='flash_attention_2', trust_remote_code=True, use_safetensors=True)
model = model.eval().cuda().to(torch.bfloat16)

# prompt = "<image>\nFree OCR. "
prompt = "<image>\n<|grounding|>Convert the document to markdown. "
image_file = 'your_image.jpg'
output_path = 'your/output/dir'

res = model.infer(tokenizer, prompt=prompt, image_file=image_file, output_path = output_path, base_size = 1024, image_size = 768, crop_mode=True, save_results = True)

Generation 1 differs in three places: model_name becomes deepseek-ai/DeepSeek-OCR, image_size=640, and there is an extra test_compress = True parameter. Never mix the two.

Unpacking infer: base_size=1024 is the global image size, image_size=768 the tile size, crop_mode=True enables dynamic tiling, and save_results=True writes output into output_path; this dynamic combo is generation 2's default. The first run downloads weights into the HF cache — on flaky networks, download manually first and point to a local path. Generation 2 keeps only two main prompts: the grounded "Convert the document to markdown." for layout-aware markdown, and "Free OCR." for raw text. The grounded variant makes the model emit markdown with layout coordinates, useful for layout reconstruction. Generation 1 is richer: "Parse the figure." for charts, general description, layout-free output, plus a <|ref|> localization mode that even accepts Chinese refs — the README's own example is a classical Chinese line.

Five: Five Resolution Modes and the Token Bill

This is the account most worth running. Generation 1's README lists five modes (its original data):

crop_mode=False means native fixed resolution; True means dynamic tiling. Generation 2 has no five-mode table — it ships a single dynamic mode: (0-6) tiles of 768×768 plus one 1024×1024 tile, i.e. (0-6)×144+256 vision tokens, effectively Gundam's idea as the only default.

How to choose: start ordinary documents at Small 640 — one hundred tokens per page is a page of paper compressed into a hundred tokens, one full tier cheaper downstream. Dense small print goes to Large; messy, image-heavy layouts go to Gundam; bulk triage starts at Tiny's 64. Concrete math on a 300-page document: Tiny costs about 19k vision tokens, Base 77k, Large 120k — a near-6x spread. On a pipeline doing thousands of pages a day, the mode choice decides how many cards you buy. If VRAM runs tight, drop base_size to 512/640 or turn off crop_mode — both community-verified.

Six: The vLLM Path

Upstream vLLM support is generation 1's story (official since October 23, 2025). Verbatim from the generation-1 README:

uv venv
source .venv/bin/activate
# Until v0.11.1 release, you need to install vLLM from nightly build
uv pip install -U vllm --pre --extra-index-url https://wheels.vllm.ai/nightly

Offline batch inference, verbatim from the README:

from vllm import LLM, SamplingParams
from vllm.model_executor.models.deepseek_ocr import NGramPerReqLogitsProcessor
from PIL import Image

llm = LLM(
    model="deepseek-ai/DeepSeek-OCR",
    enable_prefix_caching=False,
    mm_processor_cache_gb=0,
    logits_processors=[NGramPerReqLogitsProcessor]
)

image_1 = Image.open("path/to/your/image_1.png").convert("RGB")
image_2 = Image.open("path/to/your/image_2.png").convert("RGB")
prompt = "<image>\nFree OCR."

model_input = [
    {"prompt": prompt, "multi_modal_data": {"image": image_1}},
    {"prompt": prompt, "multi_modal_data": {"image": image_2}}
]

sampling_param = SamplingParams(
            temperature=0.0,
            max_tokens=8192,
            # ngram logit processor args
            extra_args=dict(
                ngram_size=30,
                window_size=90,
                whitelist_token_ids={128821, 128822},  # whitelist: <td>, </td>
            ),
            skip_special_tokens=False,
        )
model_outputs = llm.generate(model_input, sampling_param)
for output in model_outputs:
    print(output.outputs[0].text)

Three points: NGramPerReqLogitsProcessor must be passed — without it, output degenerates into repeated garbage; the whitelisted token ids 128821 and 128822 are the table start/end markers, required for table output; and per the official vLLM recipe, keep prefix caching off, use plain prompts, and tune max_num_batched_tokens to your VRAM. One more note: temperature=0.0 is the OCR standard since recognition wants deterministic output, and max_tokens=8192 is headroom for dense pages. Upstream support also means you never touch the repo's own scripts — pip vLLM alone runs generation 1, which is the prerequisite for API serving below.

Generation 2 still ships its own scripts: enter the DeepSeek-OCR2-vllm directory, set INPUT_PATH and OUTPUT_PATH in config.py, then run run_dpsk_ocr2_image.py (streaming), run_dpsk_ocr2_pdf.py (README claims speed on par with generation 1), or run_dpsk_ocr2_eval_batch.py (OmniDocBench v1.5). Generation 1's equivalents are run_dpsk_ocr_image/pdf/eval_batch.py.

Seven: API Serving and PDF Batching

For an OpenAI-compatible API, the official vLLM recipe command (generation 1):

vllm serve deepseek-ai/DeepSeek-OCR --logits_processors vllm.model_executor.models.deepseek_ocr:NGramPerReqLogitsProcessor --no-enable-prefix-caching --mm-processor-cache-gb 0

Once it is up, connect with the openai library at http://localhost:8000/v1, send images as base64 through image_url, and carry the anti-garbage parameters in the request: skip_special_tokens set to False, and vllm_xargs with ngram_size 30, window_size 90, whitelist_token_ids [128821, 128822] — the same configuration as offline inference. Whether generation 2 works behind vllm serve directly is unverified (its README has no such section), so the serving example sticks to generation 1.

PDF batching has two routes. The official one is run_dpsk_ocr_pdf.py in the repo, configured via config.py, quoted at ~2500 tokens/s on A100-40G for generation 1. The community route uses PyMuPDF to render each page via get_pixmap(dpi=150), converts to PIL Images, batches them into one generate call, and joins pages with a markdown separator — 150 dpi is the community's sweet spot; pushing to 300 inflates both VRAM and latency. Docker is a community affair: pull vllm/vllm-openai:v0.11.2 and run with --gpus all and a sane --shm-size, remembering the official repo ships no Dockerfile at all. Multi-GPU, community-sourced, is tensor_parallel_size=N on the vLLM side.

Eight: Troubleshooting Checklist

FAQ

Q1: How much VRAM does DeepSeek-OCR actually need? No official number exists. Community measurements put bf16 peaks around 19-23GB with 24GB cards as the safe line, plus a 3060 12GB sdpa route — but much of this comes from promotional CSDN posts, so treat it as reference only and benchmark your own resolution mode before scaling. On a 12GB card, start with sdpa plus reduced resolution, pinned to Small mode.

Q2: Generation 1 or generation 2? New architecture and Transformers inference: generation 2 (Apache-2.0). Upstream vLLM, API serving, mature ecosystem: generation 1 (MIT). The plain test: if you need vllm serve and upstream batch inference, take generation 1; for single-machine scripting, generation 2's code is cleaner. No official accuracy comparison between generations exists, so do not decide on version numbers alone.

Q3: flash-attn will not install. What now? Confirm python 3.12.9 and torch 2.6.0 installed first, then run the flash-attn 2.7.3 command with --no-build-isolation. Compilation eats RAM and toolchain patience — on weak machines do not grind through it; switch the load parameter to _attn_implementation='sdpa' and trade a little speed for a working setup.

Q4: Can it be served as an OpenAI-compatible API? Yes, on generation 1: vllm serve with --logits_processors and --no-enable-prefix-caching, plus the anti-garbage extra_body on the client side — see section seven; the command is verbatim from the official vLLM recipe. Generation 2 behind vllm serve is unverified. Smoke-test with two image requests before routing production traffic.

Q5: How good is it on Chinese documents and tables? There is no official Chinese-specific guidance; the <|ref|> localization prompt accepts Chinese, and the community reports mixed Chinese-English pages work, pending on-machine verification. For tables, always carry the 128821/128822 whitelist tokens or tables get truncated or garbled. Run small batches of Chinese documents at Small or Base before scaling. Vertical text, stamps, and handwriting have no official benchmarks — the companion review has same-yardstick comparisons.

Related reading

• Same Page, 26x the Tokens: Six Open-Source OCR Models Compared
• DeepSeek Ships Again: Free Weights, 60% Lighter Task Bills
• Five Open-Source Speech Models in One Drop: Four Ant Models Under MIT