Testing DeepSeek-V4-Flash Official Release with Codex: A 30-Question Hardcore Benchmark

On July 31, 2026, DeepSeek quietly released V4 Flash official (deepseek-v4-flash-0731) in its API changelog. No launch event, but it hit Zhihu's trending list that night. The official 9-benchmark suite showed code-agent benchmark DeepSWE jumping from 7.3 in the preview to 54.4 (about 7.5x), and Terminal Bench 2.1 hitting 82.7 - within striking distance of Claude Opus 4.8's 85.0. Impressive numbers, but they are the vendor's own. Can you trust them?

This article does not restate official benchmarks. I used Codex to build a pure-standard-library test harness (harness.py), and at 2026-08-02 10:51 Beijing time I made real calls to the official API, running 30 self-built questions (10 coding / 10 math / 10 Chinese). Everything was scored locally, raw data was archived, and the whole run is reproducible and auditable. Result: 30/30 correct, all 59 coding unit-test cases passed, and the total API cost for 30 questions was under 5 fen (less than one US cent).

This is a small-sample, single-run snapshot to gauge whether the "official release" lives up to its billing - not a comprehensive verdict. Below I lay out the method, data, per-test breakdown, cross-references, and a price shootout.

1. Method: How the Data Stays Honest

This article uses no "cloud benchmarks" or second-hand data. Every result comes from my own real API call to deepseek-v4-flash at 2026-08-02 10:51 Beijing time. Raw data is archived in the workspace and is reproducible and auditable.

Test setup

Three test sets, all self-built with answers pre-verified programmatically:

1. Coding: 10 classic algorithm problems (two-sum, valid-parentheses, merge-intervals, longest-increasing-subsequence, max-subarray, majority-element, rotate-array, first-unique-char, house-robber, climbing-stairs), totaling 59 unit-test cases. The model outputs only the function; a local independent process executes and scores.
2. Math: 10 exact-answer problems (divisibility sum, digit-7 counting, modular arithmetic, probability, digit-count, Vieta's, combinations, permutations, sum-of-squares, clock-angle). Answers verified by program before testing.
3. Chinese: 10 single-choice questions (literature, history, science, syllogistic logic, poetry, idioms, geography, astronomy, reading comprehension, transitive inference). Fixed answers.

Reproduction: python harness.py --selftest validates question answers locally (no API cost); python harness.py --tests coding,math,chinese --workers 3 reproduces the full run.

Limitations (stated up front): The sample is small (10 per category), single-run, and mid-difficulty. A 100% pass rate does not mean the model "has no weaknesses." Statistically significant conclusions require larger samples and multiple runs. The value of this test is using reproducible real data to confirm the direction of official benchmarks - not to replace comprehensive evaluation.

2. Test One: Coding (10/10, 59/59 Cases Passed)

Each problem requires the model to produce a solve function, executed by a local Python process against all test cases. The model never sees the test data.

Spot-check the actual generated code for C07 (unmodified):

# C07 Rotate Array (7.0s, including reasoning)
def solve(nums: list[int], k: int) -> list[int]:
    if not nums:
        return []
    k %= len(nums)
    return nums[-k:] + nums[:-k]

Clean code, solid edge-case handling (empty list, k modulo). All 10 passed on the first try - no syntax errors, no runtime exceptions. C07 took a few extra seconds because the model reasoned through the modulo relationship between k and array length in its thinking - the output itself was short, reasoning dominated.

3. Test Two: Math (10/10 Correct)

The model must provide a derivation and end strictly with "answer: xxx". A program parses and compares (integer or simplest fraction).

Two meaty problems, with the model's derivations (excerpts):

M03 (last 3 digits of 7^2026): Since 7^20 = 1 (mod 1000), and 2026 = 20x101 + 6, then 7^2026 = 7^6 = 117649 = 649 (mod 1000). Answer: 649

M02 (count of integers containing 7): Integers without 7: in 0000-2026, thousands digit 0 or 1 gives 2x9^3=1458; thousands digit 2 gives... 25 total. So 1-2026 has 1458+25-1=1482 without 7, and 2026-1482=544 with 7. Answer: 544

M02 took 25.4 seconds and output 3318 tokens - the model chose to enumerate carefully for this counting problem. Slow, but steady. This single problem pulled up the math group's average latency (5.16s); the other 9 all finished within 4 seconds.

4. Test Three: Chinese (10/10 Correct)

Covers literature, history, science, syllogistic logic, poetry, idioms, geography, astronomy, reading comprehension, and transitive inference. Fixed answers.

Chinese was the fastest of the three tests (average 1.3s) and the model gave correct reasoning. For example, Z04: "From 'all plants need photosynthesis' and 'sunflowers are plants,' by syllogism it necessarily follows that sunflowers need photosynthesis. Answer: A." These are not hard for a frontier model, but answering all correctly with valid reasoning shows the baseline is solid.

5. Summary: Score, Speed, and Cost

Three noteworthy "real" details:

• Reasoning dominates output: 8165 output tokens across 30 questions, of which 6884 were reasoning tokens - 84%. In math, reasoning hit 93%. The model "thinks a lot, speaks briefly."
• Cost is negligible: At off-peak rates (1 yuan input / 2 yuan output per million tokens), about 0.022 yuan. This run happened during peak hours (10:51 Beijing time), so at doubled rates it was about 0.044 yuan - still under 5 fen.
• Speed: From request to full response, 30 questions took about 40 seconds wall-clock (3-way parallel), median per-question about 1.8s. From M02's single request, generation speed is roughly 130 tokens/s (including reasoning).

6. Cross-Referencing Official and Third-Party Data

My 30-question results are directionally consistent with the official benchmarks - no contradictions:

Key jumps in the official 9-benchmark suite (V4-Flash-0731 vs preview): DeepSWE 7.3 to 54.4 (~7.5x), Cybergym 38.7 to 76.7, Terminal Bench 2.1 61.8 to 82.7. The post-training effect is visible. Note: these are official figures, not this article's measurements.

Architecture (official): MoE, 284B total parameters, only 13B active, 1M token context, switchable thinking/non-thinking mode (including speculative decoding). Same skeleton as the April preview - only post-training was redone. The gains come from training method, not a bigger model. Direct validation of the "small model, strong post-training" path.

7. Price Shootout: How Much Cheaper Is V4-Flash?

By output price, V4-Flash (~0.28 USD) is about 1/90 of Opus 4.8 and 1/43 of Gemini 3.1 Pro. Even at peak doubled rates (4 yuan / million tokens ~ 0.56 USD), it is still 1/45 of Opus 4.8. DeepSeek also introduced peak/off-peak billing: Beijing time 9:00-12:00 and 14:00-18:00 are peak (doubled), the rest off-peak. If your workload is latency-tolerant, avoiding peak hours halves the cost.

8. Strengths, Known Weaknesses, and Fit

Strengths

• All three dimensions green: code passes all cases, math derivations correct, Chinese Q&A fast and stable;
• Clear speed and cost advantages: sub-second-to-seconds per question, 30 questions under 5 fen, cache-hit as low as 0.02 yuan / million tokens;
• "Small model, strong post-training" validated: 13B active parameters delivering near-frontier agent scores.

Known weaknesses (from public reports, not covered by this sample)

• World knowledge still 3-6 months behind frontier closed-source (official);
• Occasional instruction-following failures, verbose output - Artificial Analysis burned 210M output tokens, 3x the same-tier median;
• Long-context (~900K) retrieval is mediocre; agent-mode reasoning is locked to English.

Who it is for: heavy coding/agent daily workloads, batch generation, cost-sensitive teams - nothing competes on price-performance right now. Who it is not for: scenarios requiring precise encyclopedic knowledge, strict short outputs, or long-document retrieval - test before committing.

Bottom line: V4 Flash official is the 2026 mid-year poster child for "capable and absurdly cheap." My 30-question test found no point where it dropped the ball - but a small-sample clean run does not mean no weaknesses. The larger the sample, the more stable the conclusion.

FAQ

Q: What does 30/30 correct actually prove? A: It proves the model made no errors on these 30 mid-difficulty coding/math/Chinese questions. But the sample is small (10 per category), single-run, and mid-difficulty. A 100% pass rate cannot be generalized to "the model has no weaknesses." Statistically significant conclusions need larger samples and multiple runs. The value of this test is using reproducible real data to confirm the direction of official benchmarks, not to replace comprehensive evaluation.

Q: How do I choose between this and Claude Opus 4.8? A: It depends on your scenario and budget. V4-Flash's output price is about 1/90 of Opus 4.8, so for daily coding and agent tasks its price-performance is unbeatable. But Opus 4.8 still leads on hard benchmarks (Terminal Bench 85.0 vs 82.7, Agents' Last Exam 25.7 vs 25.2). For complex reasoning and long-chain agent scenarios, use Opus 4.8. A common combo: Opus 4.8 for hard jobs, V4-Flash for volume, routed by task difficulty.

Q: How is the cost calculated? What is the difference between 0.022 and 0.044 yuan? A: DeepSeek introduced peak/off-peak billing for the V4 series: Beijing time 9:00-12:00 and 14:00-18:00 are peak hours with doubled prices. This test ran at 10:51 Beijing time, during peak, so it cost about 0.044 yuan. Running the same 30 questions off-peak would cost about 0.022 yuan. Both are under 5 fen.

Q: How do I reproduce this benchmark? A: Three steps. First, set the API key: $env:DEEPSEEK_API_KEY = "sk-...". Second, local validation: python harness.py --selftest (no API cost, validates question answers and parser). Third, full reproduction: python harness.py --tests coding,math,chinese --workers 3. harness.py is pure standard library with no third-party dependencies. Raw results are saved as JSONL files in the results/ directory, including full output, token usage, and latency per question - all auditable.

Q: What is peak billing? A: DeepSeek introduced peak/off-peak pricing for the V4 series: Beijing time 9:00-12:00 and 14:00-18:00 are peak, with input and output prices doubled. All other times (including nights and weekends) are off-peak. Cache-hit prices also double during peak (0.02 to 0.04 yuan) but remain extremely low. If your workload is batch-oriented and latency-tolerant, avoiding peak hours halves the cost.

References

• DeepSeek official model card DeepSeek-V4-Flash-0731 (ModelScope / HuggingFace): specs and 9 benchmarks
• DeepSeek API docs "Models & Pricing" (api-docs.deepseek.com): model name, peak/off-peak pricing
• Artificial Analysis LLM intelligence index and public benchmarks
• IT Home "DeepSeek-V4-Flash official benchmark report" (2026-07-31)
• Science and Technology Daily "US standards-body assessment..." (2026-05-05)
• This article's raw test data: harness.py + results/summary.json + raw_{coding,math,chinese}.jsonl (real API call at 2026-08-02 10:51 Beijing time)