Claude Ranks First, Gemini Costs Less Than a Third: Comparing Value for Money Across 13 Models in GDPx

Author’s note: This article breaks down the first GDPx leaderboard from SemiAnalysis, comparing top models’ scores and “average cost per task.” It also distinguishes evaluation costs from official token prices to help you make an initial choice based on capability, cost, and availability.

On November 2, 2026, SemiAnalysis released GDPx v1.0, a demanding agent benchmark. Claude Opus 5.5 ranked first with an overall score of 12.0%, followed by Gemini 4 Carbon at 10.9%. The cost difference is even more striking: Carbon’s average cost per task was only about 31.7% of Opus’s, while its score reached about 90.8% of Opus’s.

This isn’t just rumor. These figures come from the official GDPx leaderboard and SemiAnalysis’s original report. The official site also publishes 168 execution traces across 14 representative tasks. But the leaderboard is easy to misinterpret. “Average cost per task” isn’t the same as API pricing, and “12%” isn’t AI’s success rate at real-world work.

What you’ll learn: This article breaks down the cost and capability data in the first GDPx leaderboard, clearly distinguishing three pricing measures: GDPx evaluation costs, vendors’ official token prices, and prices listed in APIYI documentation. By the end, you’ll understand why overall rankings and actual purchasing decisions don’t always align.

GDPx at a Glance

GDPx stands for “Measuring Frontier Model Capabilities Across the Most Valuable Domains.” Rather than testing knowledge through Q&A, it measures whether an agent can independently complete long-running tasks using tools and software in real or highly realistic professional environments.

Importantly, GDPx measures the combined performance of the model, agent framework, tools, and environment, not the model’s reasoning ability in isolation. Because the overall score combines equal domain weighting with mixed scoring methods, it can’t be converted into a count of “how many of the 206 tasks were answered correctly.”

GDPx Leaderboard: Scores and Average Cost per Task

The table below lists the 13 models shown by default on the official website (the filter includes 20 test configurations in total). The last column indicates whether the corresponding model appears in APIYI’s public model documentation, based on a pricing snapshot from 2026-10-27. It’s provided for reference only.

The dollar amounts are GDPx’s reported average execution cost per evaluation task, including failed runs. They aren’t the price per million tokens, the cost of a user’s individual query, or the cost of completing a business task to an acceptable standard.

Three GDPx Cost Comparisons Worth Noting

Calculations based on the official data reveal three typical price-performance patterns:

These comparisons point to three choices: pursuing frontier performance, reducing costs while maintaining a similar score, and choosing a budget option for low-risk tasks. But similar overall scores don’t necessarily mean similar performance on your specific tasks.

Ranking First Overall on GDPx Doesn’t Mean Leading in Every Field

The original SemiAnalysis report notes that different models lead in different areas:

• Claude Opus 5.5: Leads in quantitative trading, hardware design, and chip design.
• Gemini 4 Carbon: Leads in biology and holds a smaller lead in software engineering.
• GPT-6 Astra: Leads in recursive self-improvement (RSI) and materials science.

The researchers also noted that safety classifiers affected some scores for Opus 5.5 and GPT-6 Astra, mainly in cybersecurity, biology, and AI alignment. In the chip physical design task vt01, Astra scored an average of 0.34, while Opus scored just 0.135. On an individual task, the relative rankings can be the opposite of the overall leaderboard.

GDPx Costs Aren’t the Same as Official Token Prices

A common mistake when comparing value is to treat GDPx’s $/task figure as an API price. The table below lists each vendor’s publicly available standard text pricing separately, in US dollars per million tokens. It also calculates a theoretical bill for 1 million input tokens and 200,000 output tokens, excluding caching and tool fees.

Price sources: Anthropic’s and OpenAI’s official pricing pages, and Google’s Carbon announcement published on October 28, 2026. Google also announced that after the introductory period, Carbon’s standard pricing will be $4 for input and $20 for output. The end date hasn’t been announced. Different pricing may apply to OpenAI’s long-context requests, and vendors also have different billing rules for cache writes, batch processing, and other features.

This table only shows differences in unit pricing. For the same task, models may use very different numbers of tokens and require different numbers of calls, so actual bills won’t necessarily rank in the same order as the theoretical bills. For example, Opus has a lower theoretical bill than Astra, but their average task costs on GDPx are close ($51.41 vs $53.99).

Gemini 4 Carbon’s Practical Limitations

Carbon is the star of the leaderboard’s value-for-money story, but it’s still difficult for most developers to access directly. When Google released Carbon on October 28, it made the model available only to select cybersecurity defense teams and vetted partners, with plans to expand access gradually. As of November 3, no announcement of a broadly available public API had been found.

The pricing snapshot in APIYI’s public model documentation also contains no entry for gemini-4-carbon, and an explanatory article published by the platform on October 29 explicitly states that Carbon doesn’t yet have a public API. For now, Carbon is better viewed as a sign of where the technology is heading than as an option you can purchase immediately. Gemini 4 Argon, from the same model family, is already available to call and is worth watching in the meantime (GDPx 7.3%, $12.79/task).

Factors Affecting the Total Cost of GDPx-Style Agent Tasks

The bill for a long-running Agent task isn’t as simple as “unit price × one call.” You can break the total cost down as follows:

Total cost = input Token cost + output Token cost + cache-related cost + tool call cost + failed retry cost + other operating costs

For businesses, a more useful metric is cost per accepted delivery = cost of all attempts ÷ number of accepted deliveries. This is a suggested calculation, not an official GDPx statistic. If a cheaper model needs repeated retries, its final cost may be higher. A more expensive model that delivers more reliably may have a lower cost per business outcome.

How to Reproduce a GDPx-Style Cost Comparison: Quick Start

Most of GDPx’s 206 tasks aren’t public, so you can’t directly reproduce the leaderboard. A more practical approach is to use real tasks from your own business: run each candidate model several times with the same inputs and acceptance criteria, then record token usage and whether each run passes.

Minimal Example

The code below calls a model through an OpenAI-compatible interface and reads the token usage for the request:

import openai

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"
)

resp = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[{"role": "user", "content": "核对这份应收账款账龄表的合计是否正确"}]
)
print(resp.choices[0].message.content)
print(resp.usage)  # prompt_tokens / completion_tokens

import openai
from typing import Callable

client = openai.OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.apiyi.com/v1"
)

# Prices: USD per million tokens, based on the 2026-10-27 snapshot of the APIYI documentation
# Actual charges are shown in the console; group and top-up discounts may affect your bill
PRICES = {
    "claude-opus-5-5":     {"in": 4.0,  "out": 20.0},
    "gpt-6.1-sol":         {"in": 2.0,  "out": 10.0},
    "gemini-4-argon":      {"in": 2.0,  "out": 10.0},
    "deepseek-v4.1-flash": {"in": 0.30, "out": 1.20},
    "gpt-6-luna":          {"in": 0.10, "out": 0.50},
}

def run_once(model: str, task: str) -> tuple[str, float]:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": task}],
    )
    u, p = resp.usage, PRICES[model]
    cost = (u.prompt_tokens * p["in"] + u.completion_tokens * p["out"]) / 1_000_000
    return resp.choices[0].message.content, cost

def evaluate(task: str, validator: Callable[[str], bool], runs: int = 3):
    report = []
    for model in PRICES:
        total_cost, passed = 0.0, 0
        for _ in range(runs):
            try:
                out, cost = run_once(model, task)
                total_cost += cost          # Failed attempts count toward the cost too
                passed += int(validator(out))
            except Exception as e:
                print(f"{model} 调用失败: {e}")
        per_pass = total_cost / passed if passed else float("inf")
        report.append((model, passed, runs, total_cost, per_pass))
    for m, ok, n, c, pp in sorted(report, key=lambda x: x[4]):
        print(f"{m:22s} 通过 {ok}/{n}  总成本 ${c:.4f}  单次合格交付成本 ${pp:.4f}")

# Example: write a validator based on your business criteria; this is only a demonstration
evaluate(
    task="根据以下条款判断 418 美元的保险审计调整是否终止原有保险义务……",
    validator=lambda out: "不终止" in out,
)

This script only covers single-turn calls. For real agent tasks, you should also add up token usage from tool calls and multi-turn context. The validator should reflect your actual delivery standards as closely as possible, rather than relying on simple keyword matching.

🚀 Quick-start tip: When comparing multiple models, one of the biggest hassles is setting up accounts with each provider and adapting to different SDKs. With APIYI at apiyi.com, you can switch between the models listed above using one API key and an OpenAI-compatible interface. New users get free test credits, making it a good way to run an initial small-sample comparison.

GDPx Impact Analysis: What It Means for Developers and Enterprise Model Selection

Impact on Developers

• Don’t rely on the overall rankings alone: Opus ranks first overall, but Carbon leads in biology and software engineering, while Astra leads in materials science and RSI. Results for individual tasks can run counter to the overall rankings.
• Cost differences can be much larger than performance differences: Astra and Sol’s scores are only 1.2 percentage points apart, but their average task costs differ by about 5.51 times. That’s a strong reason to experiment with tiered routing.
• The framework matters too: In GDPx, each model used its own native framework. A different framework could change both performance and token consumption.

Impact on Enterprises

• Judge high-risk tasks by the final deliverable: Financial and legal tasks used strict binary scoring, and all models scored 0%. In the legal case, Opus earned an average partial score of 44%, but none of its documents met the standard for use.
• Completing a task doesn’t mean completing it compliantly: In 200 cybersecurity test runs, 196 violated at least one rule of conduct. This was a high-pressure adversarial test, so it can’t establish real-world violation rates, but it does show that authorization boundaries need separate evaluation.
• Availability is a hard constraint: No matter how well a model performs in testing, you can’t deploy it if you can’t buy access to it. Carbon is a prime example.

A preliminary selection framework follows from this: first assess capability (the pass rate on your own business samples), then cost (the cost per acceptable deliverable), and finally availability (whether you can purchase access, rate limits, and data terms). All three are essential.

Frequently Asked Questions

GDPx Value for Money: Key Takeaways

• Rankings: In GDPx v1.0, Opus 5.5 leads at 12.0%, followed by Carbon at 10.9% and Astra at 9.6%.
• Cost difference: Carbon scores about 90.8% of Opus’s result, at just 31.7% of its average cost per task.
• Metric distinction: GDPx’s $/task is an average across evaluation runs, not an official token price.
• Domain differences: First place overall doesn’t mean first in every domain. Individual task results can even reverse the overall ranking.
• Availability constraint: Carbon’s public API isn’t yet widely available, so it isn’t an immediately accessible option.
• Selection method: Compare models by cost per accepted delivery. APIYI apiyi.com provides a unified interface for small-sample comparisons.

Conclusion

The first GDPx leaderboard reveals clear differences in value for money among leading AI models:

1. Capability: Claude Opus 5.5 leads overall, but each model has its own strengths. For high-risk tasks, models still have a long way to go before they can deliver independently.
2. Cost: The Carbon-versus-Opus and Sol-versus-Astra comparisons show that similar scores can come with several-fold differences in cost.
3. Availability: Carbon is still difficult to access directly. Even the best evaluation results matter only if you can deploy the model.

GDPx can point you in the right direction, but it can’t replace testing on your own business tasks. A safer approach is to choose a few frontier, balanced, and budget models, run each through several rounds of real tasks using the same acceptance criteria, and compare their cost per accepted delivery. We recommend using APIYI apiyi.com to test multiple models with a single key. Check the console for current pricing.

📚 References

1. SemiAnalysis GDPx Original Report: Benchmark methodology, domain results, and case studies
2. Link: newsletter.semianalysis.com/p/gdpx-measuring-frontier-model-capabilities

3. Notes: Includes leading models in each domain and analysis of failure modes

4. GDPx Official Leaderboard: Overall scores and average task costs

5. Link: gdpx.semianalysis.com

6. Notes: Source of the leaderboard data in this article; execution traces are available on the /rollouts page

7. Google Gemini 4 Carbon Launch Announcement: Availability and planned API pricing

8. Link: blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-carbon

9. Notes: Carbon’s introductory pricing, standard pricing, and access restrictions

10. OpenAI API Pricing Page: Official per-token pricing for the GPT-6 series

11. Link: developers.openai.com/api/docs/pricing

12. Notes: Check the special pricing for long-context requests

13. APIYI Model Pricing List: A snapshot of models and listed prices on the platform

14. Link: docs.apiyi.com/models
15. Notes: Snapshot from 2026-10-27; actual charges are shown in the console

Author: APIYI Technical Team
Technical discussions: Contact us through the APIYI website. For more model integration resources, visit the docs.apiyi.com documentation center.

Related Articles

• Gemini 4 Carbon Release Analysis: Scores 59 to Overtake Opus 5.5, Wins 15 of 17 Tests but Remains Inaccessible: Gemini 4 Carbon topped the Artificial Analysis Intelligence Index with 59 points and won 15 of 17 official comparisons, but has no API model ID or release timeline; this article breaks down the scores, explains why it is unavailable, and suggests gemini-4-argon for Chinese developers.
• Gemini 4 Carbon Released: How Much Longer Will Developers Have to Wait to Use It?: Gemini 4 Carbon was released on October 28, 2026, but has no public API yet; this article uses four timeline milestones and Fairwind’s phased access gates to predict developer availability and suggest alternatives available now.