Summary: Based on public third-party test reports, this article analyzes how Qwen4-Max-Preview performed on tasks such as completing a claw-machine challenge, fixing a Python service, and retrieving 1,600 tickets. It also explains the model’s limits when it comes to time spent and scientific judgment.
Bottom line first: the core take on Qwen4-Max-Preview’s coding capability
Public test reports show that Qwen4-Max-Preview can push multi-step development forward, add tests, and deliver projects that can be re-verified. The strongest evidence comes from a code-fix task: the model first reproduced the defect, then added tests, sent real HTTP requests, and checked data integrity — and in the end, all 57 tests passed. Its weak spots are just as clear: complex tasks take a long time — the claw-machine task took about 75 minutes — and in a research task, the model’s explanation of the computed results went beyond the evidence. So complex deliverables and scientific conclusions both need developer review.
Below, this article starts with the claw-machine task, then walks through a report covering 12 tasks, with a focus on two sets of verifiable evidence: code repair and ticket retrieval. Next, it uses a non-negative matrix factorization case to show the model’s reasoning limits, then discusses efficiency through a cross-tool comparison on the same claw-machine challenge, and finally outlines the kinds of tasks that are a good fit for it. This is an evaluation analysis of a public report; we did not rerun the author’s project, and every number is annotated with its source and test conditions.

To Know: 9 Terms for Understanding the Qwen4-Max-Preview Evaluation
| Term | Explanation |
|---|---|
| Agent | A way of working where the model plans its own steps, calls tools, and moves the task forward step by step. |
| State machine | Controls program behavior using a set of states and transition rules between them—for example, the “move—grab—lift—settle” sequence of a claw machine. |
| Regression testing | Re-verifying existing functionality after code changes to confirm that old behavior hasn’t been broken. |
| HTTP request | A request sent from a client to a server, used to check what an interface returns when it actually runs. |
| Hash verification | Computes a digest of file contents; comparing values before and after shows whether the original data or test file has been changed. |
| Token throughput | The number of tokens processed or generated per unit time; the value depends on the tools, network, and runtime environment. |
| Context retrieval | Locating and linking target information within the large amount of material carried by a single request. |
| Non-negative matrix factorization | An algorithm that approximately factors a non-negative data matrix into the product of two non-negative factor matrices. |
| Local optimum | A solution that cannot be improved within its neighboring range; determining whether a solution is locally optimal requires additional mathematical conditions or analysis. |
1 Building a Claw Machine in 75 Minutes: A Complex Programming Task for Qwen4-Max-Preview
1.1 What the Claw Machine Task Actually Tests
On November 12, 2026, 人人都是产品经理 (Everyone Is a Product Manager) published a hands-on report on Qwen4-Max-Preview built on Qoder, and the most complex item in it was a claw machine interactive system. This single project concentrates a bunch of programming challenges: mechanical interaction and force feedback, the swaying of a hanging rope, collisions between plush toys, the state machine that drives the control flow, and the scoring logic at the end.
This kind of task is hard to pull off in one shot. Physics, state transitions, and scoring rules all depend on each other, so the model has to write, run, observe, and fix things over and over. What’s really being tested here is the agent-style ability to keep moving forward across multiple steps.
1.2 The Time, Speed, and Consumption the Author Recorded
| Metric | Author’s record | Test conditions and notes |
|---|---|---|
| Total time | About 75 minutes | Qoder environment, single task |
| Output speed | About 33 tokens/s | Speed observed during that task |
| Credits consumed | About 65 | Author’s estimate, metered by Token Plan |
| Deliverable quality | Fairly solid | Author’s subjective call, considers it roughly on par with Kimi K3.1 |
These numbers apply to that one task only. Credits are the subscription billing unit for Token Plan, and actual consumption is shaped by input, cache hits, output, reasoning tokens, and tool calls together—so 65 Credits only tells you the rough order of magnitude for this run. The 33 tokens/s figure is likewise an observation made in a specific environment.
The same report also covered several other tasks. In an ink-wash painting task restricted to NumPy and PIL, the model generated more than 1,000 lines of code and drew roughly 4,019 brush strokes. It self-checked using pixel-brightness statistics and ASCII contour scanning, and after noticing that a swallow overlapped a dark block in the window opening, it adjusted the coordinates on its own. The author found its code organization and self-checking ability strong, while noting that its aesthetic output still has room to improve.
2 What Do Qwen4-Max-Preview’s Twelve Real-World Tasks Actually Cover?
2.1 A More Thorough Test Report
On November 16, Everyone Is a Product Manager published a Z Finance test report. The author says they completed 12 tasks, then spot-checked and re-verified the key results. The table below organizes the tasks the report covers:
| Category | Test task | Main capability examined |
|---|---|---|
| Visual generation | A pelican riding a bicycle (SVG) | Visual understanding, code generation |
| Financial analysis | Joint analysis of three financial reports | Long-document understanding, financial conventions |
| Chart reading | Apple product revenue bar chart | Chart reading, numerical estimation |
| Office collaboration | Three rounds of edits to meeting minutes | Context retention, edit consistency |
| Data processing | Cleaning 44 rows of sales data | Deduplication, unit conversion, formulas |
| Presentation | Pingtouge IPO deck | Real-time search, file generation |
| Long-text retrieval | 1,600 customer support tickets | Cross-record linking, conditional filtering |
| Code maintenance | Fixing a Python ticket service | Defect localization, testing and regression |
| Research reproduction | Non-negative matrix factorization algorithm | Algorithm implementation, numerical validation |
| Web development | Prospectus information display page | HTML, offline delivery, interaction |
| Simulation system | Rain garden simulation | Interaction design, numerical computation |
| Game development | Four-seasons-themed 3D mini game | 3D scenes, state management, interaction |
2.2 Five Cases Worth a Closer Look
Of these 12 tasks, the following five best demonstrate the model’s ability to deliver “usable results”:
- Financial report analysis: The inputs were Apple’s and Costco’s FY2026 annual reports plus Anthropic’s second-version prospectus, and it finished in about 8 minutes. All 6 spot-checked key figures were correct. The model distinguished operating loss from net loss, noted how changes in the fair value of convertible bonds affected net loss, and did not make up any future guidance that hadn’t been disclosed.
- Excel cleanup: The 44 rows of sales detail were seeded with duplicate records, duplicates differing only by whitespace, two units (yuan and ten-thousand yuan), three date formats, and 5 missing orders. After cleanup, 35 rows remained, the four regions totaled 215,700 yuan, and all 5 summary cells used real formulas — editing a detail row updated the summary automatically.
- PPT generation: Using real-time search, the model identified the spin-off announcement details, produced an editable PPT, and kept valuation forecasts separate from issue pricing. After a second round of audience adjustments, the original 20 key figures, dates, and names all stayed unchanged. The author also noted a missing product line.
- Ticket retrieval and Python repair: These two offer the most concrete evidence and are covered in Sections 4 and 3 respectively.
What these tasks have in common is that the deliverables can be checked: do the formulas update together, do the numbers match the source text, do the tests pass. That kind of evidence is more convincing than “it generated a chunk of code that looks right.”
3 57 Tests Passed: Qwen4-Max-Preview’s Most Convincing Engineering Evidence
3.1 The Fix Process, Fully Reconstructed
The subject under test is a ticket query service written with the Python standard library. The bugs involve combined filtering, sorting, pagination, and parameter validation. The task required fixing the bugs while preserving the interface, original data, and existing tests. The report records that this task took 8 minutes and 07 seconds, and the process went like this:
| Step | Model action | Engineering significance |
|---|---|---|
| Reproduce the bug | First run the original 9 tests and reproduce 2 failures among them | Confirm the problem exists before making changes |
| Locate and modify | Most changes are concentrated in query.py | Keep the change scope contained and reduce side effects |
| Add tests | Add 48 tests | Cover edge cases the original tests missed |
| Self-correction | Fix 3 issues in the tests it wrote itself | Notice that the tests themselves were wrong and correct them in time |
| Regression verification | In the end, all 57 tests pass | 9 original tests plus 48 newly added tests |
| Integrity check | Run hash checks on the original data and existing tests; the hashes are unchanged | Prove that it did not “pass” by tampering with data or tests |
| Real requests | Start the local service and send 13 HTTP requests, getting 7 200s, 4 400s, and 2 404s | Verify normal responses and error handling at API runtime |
| Delivery re-check | Unzip the final delivery package and run it again; the tests pass | Confirm the deliverable itself can run independently |

3.2 Why This Says More Than Generating Code
There are two common failure modes in AI coding: one is changing tests or data to make the tests “go green”; the other is code that passes in tests but fails at real runtime. The hash check in this task targets the first risk, while the real HTTP requests and the delivery-package re-check target the second. The 4 400 responses and 2 404 responses show that the model also verified error paths; parameter validation and handling of nonexistent resources were both actually exercised.
This process matches how seasoned engineers fix bugs: reproduce, fix, add tests, run regression, and deliver. It shows that Qwen4-Max-Preview can complete a full maintenance loop in a service with clear rules and manageable scope. Still, this is a record of one task; it can’t be converted into a general code-fix success rate.
The Practical Value of a 4-Million Context: Finding a Clue 741 Lines Apart in 1,600 Support Tickets
4.1 Test Design and Results
Qwen4-Max-Preview’s officially published context window is 1,000,000 tokens. Window size is just a capacity ceiling — what determines practical value is whether the model can build the right connections across a large body of material. The ticket test in the Z Finance report answers that question.
The test data consisted of 1,600 customer service tickets, roughly 500 KB in size, with the key clues distributed across two records 741 lines apart, and 5 questions were posed. The report records that all 5 questions were answered correctly. Specific behaviors included:
- Linking the two scattered tickets together to find the 1,386 yuan refund amount and the person who handled it.
- Using the approval date to determine whether the new compensation voucher policy applied to that case.
- Counting 62 tickets matching the specified conditions, consistent with the script’s count.
- Finding two sets of duplicate order numbers and distinguishing which set was normal processing and which set was a customer information conflict.
4.2 Bytes and Tokens Are Two Different Units
Roughly 500 KB is the file size. The token count depends on the tokenization method and the Chinese-to-English ratio, so the two need to be calculated separately. What this test demonstrates is “cross-record association and conditional filtering within a medium-sized body of material” — its scale is far below the 1,000,000-token window ceiling, so it can’t be treated as a stress test of the model’s maximum capacity.
The value of result #3 lies in its verifiability: 62 matches the script’s count, which shows the model’s counting can be double-checked with a deterministic program. In practice, I’d recommend the same approach — use a script to verify the model’s statistical results, then use the model for the association and interpretation work that scripts struggle to do.
5 A Shortcoming You Can’t Ignore: Qwen4-Max-Preview’s Overreach in Scientific Reasoning
5.1 It Got the Computation Right, but Its Interpretation Went Beyond the Evidence
The scientific reproduction task asked the model to implement the multiplicative update algorithm for nonnegative matrix factorization based on a specified paper. The input matrix was 48×36, with three different initializations and 500 iterations per run. The report recorded a model runtime of 10 minutes 06 seconds; it delivered code, numerical results, charts, and logs; each of the three result sets provided 52 checkpoints; the key checkpoints for Group A matched publicly available reference results; and there were also 63 self-check records.
The problem came at the conclusion stage. The model saw that the objective function kept decreasing and interpreted that as convergence to a local optimum. The authors point out that this is over-inference.
5.2 Three Concepts That Need to Be Judged Separately
This case involves three levels of judgment, and each later judgment requires stronger evidence than the one before it:
- The objective function decreases: this is a design property of the multiplicative update rule and can be observed directly in logs and charts.
- Numerical iteration convergence: this requires checking whether changes in the iterates are stabilizing and setting clear stopping criteria.
- Convergence to a local optimum: this requires verifying optimality conditions, such as the KKT conditions; a drop in the function value alone is not enough to establish it.
This shows that the model’s execution capability and the rigor of its argument need to be evaluated separately. Numerical computation can be verified with checkpoints, while scientific conclusions need review by someone with the relevant background. In research, paper reproduction, or technical report settings, the “conclusion section” produced by a model should be treated as a draft.
6 Qwen4-Max-Preview vs. Other Coding Tools on the Same Task
6.1 Comparison Data for the Claw Machine Challenge
The November 12 report also included records of the same claw machine challenge across different tools. In the original table, the Qwen row is labeled Qwen4-Max, with a test date of November 12—earlier than the official version’s November 16 launch—so it corresponds to the preview-stage model.
| Tool/Model | Time | Total tokens processed (estimated) | Output tokens | Retest access |
|---|---|---|---|---|
| Codex | 8 min | 3.5M | 28K | Corresponding official tool |
| Claude Code | 17 min | 12.15M | 130K | Official tool; Claude models can be accessed via APIYI apiyi.com |
| Gemini | 21 min | 13.94M | 81K | Official tool; Gemini models can be accessed via APIYI |
| Kimi Code | 36 min | 9.5M | 59K | Corresponding official tool |
| DSH | 46 min | 46.44M | 281K | Corresponding official tool |
| MiniMax Code | 68 min | 40M | 198K | Corresponding official tool |
| Qwen4-Max (preview stage) | 75 min | 2.69M | 55K | Token Plan (Qoder) |

6.2 How to Read This Data
The data shows a striking contrast: Qwen4-Max-Preview had the lowest total token processing volume on this task—about 2.69 million—yet the longest duration, about 75 minutes. Combined with the observed speed of about 33 tokens/s, it’s clear that the bottleneck in this task was generation speed, not the sheer number of tokens consumed.
This data isn’t suitable for ranking for three reasons: the total token counts are the author’s estimates; the tools differ in context cache share, invocation strategy, and execution approach; and the exact model versions in the table aren’t uniformly specified. Low token processing volume could come from fewer repeated reads, or from differences in tool-layer caching strategies—the existing material can’t distinguish between the two. What it can support is this judgment: in this tool environment, Qwen4-Max-Preview is suited to tasks where waiting time isn’t a concern and delivery completeness matters more.
7 Which Developers Should Try Qwen4-Max-Preview
7.1 Suitable Scenarios and Those to Approach with Caution
Based on the two reports, Qwen4-Max-Preview performs relatively consistently on the following tasks:
- Code maintenance with clear rules: fixing and adding tests when existing tests are available and the problem boundaries are well defined.
- Long-document retrieval and linking: cross-record search across tickets, logs, contracts, and financial reports, with scripts used to double-check statistical results.
- Office automation: data cleaning, Excel with formulas, editable PPT decks, and multi-round revisions.
- Interactive prototype development: single-file web pages, simulation systems, and mini-game prototypes—for example, the roughly 46KB offline demo page with no external dependencies mentioned in the report.
Three types of tasks call for caution: interactive development that’s sensitive to response time (a single task may run for more than an hour), outputs of conclusions that require rigorous mathematical or scientific justification, and production changes with high reliability requirements. The report authors also conclude that “some information will be missed in broader tasks” and that “final deliverables still require human verification.”
Before you use it, also keep the official limitations in mind: Preview can only be used interactively through Alibaba Cloud Token Plan in tools such as Claude Code, Cursor, and Qoder. It doesn’t support pay-as-you-go billing, and the preview period ends and it will be taken offline after 24:00 (Beijing Time) on November 25, 2026. The official Qwen4-Max launched on November 16, and the official recommendation is to migrate to the official version. The two have different parameters: in thinking mode, the official version has a maximum output of 262,144 Tokens, while Preview has 393,216 Tokens, so you’ll need to reset the output budget when migrating.
7.2 Run a Like-for-Like Retest with Your Own Tasks
At best, a secondhand report shows that “this model can do it.” The most direct way to decide whether it fits your team is to run a comparison against your own real code and tests. The example below sends a code repair task through an OpenAI-compatible interface and records elapsed time and token usage. Be sure to use the actual model name from the platform’s model list.
import openai
client = openai.OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.apiyi.com/v1" # APIYI unified interface, OpenAI-compatible
)
resp = client.chat.completions.create(
model="qwen4-max", # Use the actual name shown in the APIYI model list
messages=[{"role": "user", "content": "阅读以下代码与失败的测试,给出最小修复补丁:..."}]
)
print(resp.choices[0].message.content)
View the multi-model comparison retest script
import time
import openai
client = openai.OpenAI(
api_key="YOUR_API_KEY",
base_url="https://api.apiyi.com/v1" # Use one key to call multiple models
)
# Replace with the actual names from the platform's model list
MODELS = ["qwen4-max", "claude-sonnet-model-name", "gpt-model-name"]
TASK = """你将收到一个 Python 服务的源码和测试结果。
要求:1) 先说明复现到的失败原因;2) 只修改必要文件;
3) 补充覆盖边界条件的测试;4) 不得修改原始数据和已有测试。
--- 源码与测试输出 ---
{payload}
"""
def run_case(model: str, payload: str) -> dict:
start = time.time()
resp = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": TASK.format(payload=payload)}],
max_tokens=32000, # Reasoning models' thinking tokens also count against the output budget, so leave headroom
)
usage = resp.usage
return {
"model": model,
"seconds": round(time.time() - start, 1),
"prompt_tokens": usage.prompt_tokens,
"completion_tokens": usage.completion_tokens,
"answer": resp.choices[0].message.content,
}
if __name__ == "__main__":
payload = open("bug_context.txt", encoding="utf-8").read()
for m in MODELS:
r = run_case(m, payload)
print(f"{r['model']}: {r['seconds']}s, "
f"in={r['prompt_tokens']}, out={r['completion_tokens']}")
# Apply the patch in an isolated directory, run tests, verify the original data by hash, then score
The script only collects candidate solutions and usage. For the scoring step, follow the report’s approach exactly: apply the patch in an isolated directory, run the original tests and the newly added tests, and use hashes to confirm that the data and existing tests haven’t been changed. That’s the only way the conclusions will be valid for your codebase.
🚀 Quick start: If you want to compare multiple models on the same set of tasks, you can get free testing credits through APIYI at apiyi.com. The platform provides an OpenAI-compatible interface, and a single API key lets you switch between mainstream models such as OpenAI, Claude, and Gemini, making it easy to run your retest script through the same evaluation workflow.
8 Qwen4-Max-Preview Review FAQs
Q1: Does taking 75 minutes to complete the claw machine task mean it’s slow?
In that Qoder task, its duration was the longest in the comparison table, with an observed output speed of about 33 tokens/s. This is only a record from a single environment; tool orchestration and network conditions can both affect the results. If your scenario is sensitive to wait times, it’s best to test it with your own tasks.
Q2: Can 57 passing tests represent its code-fix success rate?
It represents one complete, verifiable fix process, and can’t be converted into a success rate. To evaluate performance in your own projects, you can use APIYI apiyi.com to compare multiple models with the same set of bug samples, scoring them by test pass status and hash verification results.
Q3: Did the ticket test verify the 1M-token context window?
No. The test is about 500,000 bytes; the token count depends on tokenization, so its scale is far below the context window limit. What it validated was cross-record correlation and conditional filtering.
Q4: What should I do after Preview ends?
The official stable release of Qwen4-Max is now available, and it’s recommended to use the stable version. The stable version supports a pay-as-you-go API, with a maximum output of 262,144 tokens in thinking mode, so you’ll need to adjust your output budget when migrating. Third-party platform availability and pricing change quickly, so check the currently available model names in APIYI apiyi.com’s model list before integrating.
Q5: Can I put the Token Plan Key in my own application backend?
No. Official rules restrict Token Plan to interactive use in permitted coding tools and agent tools. Batch calls from automated scripts and self-built application backends are out of scope, and violations may lead to subscription suspension or Key bans. Self-built apps should use API products that fit the permitted usage scope.
9 Qwen4-Max-Preview Review Key Takeaways
- End-to-end delivery: The code-repair task covered reproduction, fixing, adding tests, hash verification, real requests, and re-verifying the delivery — 57 tests passed.
- Long-document cross-referencing: Across 1,600 tickets, it traced clues spanning 741 lines, answered all 5 questions correctly, and its 62-item statistics matched the script output.
- Office deliverables: Excel formulas could be linked together, and after multiple rounds of PPT edits, 20 key facts remained unchanged.
- Speed is the weak spot: The claw-machine task took about 75 minutes — low token throughput and long elapsed time.
- Reasoning boundaries: Its computations in the research reproduction matched the reference results, but the conclusions went beyond the evidence and need human review.
- Nature of the evidence: All of the above are hands-on task tests by a single author and do not constitute a standardized ranking.
- How to get started: Run your own code and tests as a comparative re-test, and you can use the APIYI apiyi.com API proxy to switch between multiple models through a single endpoint.
10 Conclusion: Where Developers Should Use Qwen4-Max-Preview
The public reports paint a fairly clear picture. Qwen4-Max-Preview is good at work with well-defined rules and verifiable results: code maintenance with tests, cross-record retrieval from long materials, formula-driven data processing and editable office files, and interactive prototype development. In these tasks it can write its own tests, debug itself, and deliver results you can re-verify.
Its limitations are just as concrete: complex tasks take a long time, so it’s best run in the background; scientific conclusions easily overstep the evidence and need expert review; and the Preview is only interactively usable through Token Plan in designated tools, will be retired on November 25, and long-term use should move to the official release.
The most pragmatic approach is to treat it as an engineering assistant that can independently close the maintenance loop, while developers own the final judgment on conclusions and the merge decision. Before going live, it’s worth running your own real tasks through a multi-model comparison — with the unified APIYI apiyi.com endpoint, you can complete testing and comparison within a single workflow.
📚 References
-
Everyone Is a Product Manager: Qwen4-Max Preview Is Here! Hands-On Test and Impressions: Qoder claw machine hands-on test and cross-tool comparison, 2026-11-12
- Link:
woshipm.com/ai/6492544.html - Details: Source for the 75-minute runtime, 33 Tokens/s, 65 Credits, and comparison table
- Link:
-
Everyone Is a Product Manager: Hands-On Test of Qwen4-Max-Preview (Z Finance Test Report): Comprehensive test across 12 tasks, 2026-11-16
- Link:
woshipm.com/evaluating/6494229.html - Details: Source for code fixing, ticket retrieval, research replication, and office task data
- Link:
-
Alibaba Cloud Model Studio: Qwen4-Max-Preview Documentation: Official specs and usage instructions
- Link:
help.aliyun.com/zh/model-studio/qwen4-max-preview - Details: Context window, maximum output, preview period, and access methods
- Link:
-
Alibaba Cloud Model Studio: Qwen4-Max Documentation: General availability parameters and billing
- Link:
help.aliyun.com/zh/model-studio/qwen4-max - Details: GA output budget and pay-as-you-go information
- Link:
-
Alibaba Cloud: Token Plan Permissions and Usage Guidelines: Scope of use restrictions
- Link:
help.aliyun.com/zh/model-studio/token-plan-permissions-and-usage - Details: Interactive usage requirements and prohibited scenarios
- Link:
Author: APIYI Technical Team
Technical Discussion: Feel free to contact us through the APIYI apiyi.com homepage. For more integration resources, visit the docs.apiyi.com documentation center.
