Benchmarking GPT-6 Astra Across 6 Reasoning Effort Levels in Codex

AI engineer and researcher @bridgemindai has published empirical benchmark results evaluating OpenAI's GPT-6 Astra across all six reasoning effort levels within the terminal-based Codex CLI environment. By running an identical prompt under strictly controlled conditions, the benchmark provides concrete measurements of token consumption, execution runtime, API pricing, and final output quality across Low, Medium, High, Extra high, Max, and Ultra configurations.

Image source: @bridgemindai / X

This empirical test quantifies how different reasoning effort settings translate into tangible compute resource usage and developer costs. Rather than assuming that higher effort settings automatically produce proportional benefits, the measurements provide engineering teams with actionable baseline data for optimizing model parameters across diverse development pipelines.

Empirical Metrics Across All Six Reasoning Effort Levels

The experiment injected a single identical prompt into GPT-6 Astra via the Codex CLI, systematically iterating through all six supported effort levels (Low, Medium, High, Extra high, Max, Ultra) while logging token usage, elapsed duration, and final API expenses.

Effort Level	Token Consumption	Runtime	API Cost
Low	7,923 tokens	4 min	$0.63
Medium	9,085 tokens	5 min	$0.69
High	19,596 tokens	10 min	$1.21
Extra high	34,420 tokens	19 min	$1.95
Max	37,241 tokens	20 min	$2.09
Ultra	27,465 tokens	14 min	$1.84

The resulting measurements reveal clear performance tiers across the reasoning spectrum:

• Low to Medium Tier: Moving from Low (7,923 tokens, 4 min, $0.63) to Medium (9,085 tokens, 5 min, $0.69) represented a modest 14.6% increase in tokens and an additional $0.06 in cost. This provides a lightweight stepping stone for quick iterative tasks without substantial budget impact.
• Sharp Inflection at High: Escalating to High caused token consumption to more than double relative to Medium, jumping to 19,596 tokens over 10 minutes and pushing total execution expense past the dollar mark to $1.21.
• Compute Concentration at Extra High and Max: Extra high consumed 34,420 tokens over 19 minutes ($1.95), while Max reached the overall peak with 37,241 tokens over 20 minutes ($2.09). At the Max setting, the model utilized nearly five times the tokens required by Low.

Quality Assessment and the Non-Linear Ultra Token Drop

Beyond recording latency and expenditure metrics, @bridgemindai evaluated the qualitative merits of each generated output, highlighting several key technical observations:

First, Max secured the highest qualitative performance ("Max still won on quality"). Although Max demanded roughly 4.7 times the tokens and more than triple the budget of Low ($2.09 versus $0.63), the expanded reasoning depth delivered a noticeably more comprehensive, rigorous, and refined final result. For difficult architectural problems and complex code transformations, the extended compute budget proved justified by the caliber of the output.

Second, the Ultra tier exhibited an unexpected non-linear drop in compute consumption. While conventional intuition suggested Ultra would sit above Max as the most resource-intensive tier, the empirical logs showed a substantial decrease: Ultra consumed only 27,465 tokens (approximately 26% fewer than Max), finished in 14 minutes, and incurred a cost of $1.84. Addressing this reversal, @bridgemindai noted, "Surprised me too," demonstrating that reasoning effort parameters within the underlying model architecture do not necessarily map to a simple strictly monotonic progression.

Third, scaling efficiency remains heavily dependent on task structure ("Task dependent for sure"). The author stressed that different types of prompts and programming problems respond differently to increased reasoning depth. Simple or moderately constrained tasks may see diminishing returns past Medium or High, while deeply entangled reasoning challenges may fully capitalize on Max.

Complete Model Outputs by Reasoning Level

The complete outputs produced by GPT-6 Astra across each of the six reasoning effort tiers are documented in the original continuation threads:

• Low Tier Full Output: Read GPT 6 Astra Low output
• Medium Tier Full Output: Read GPT 6 Astra Medium output
• High Tier Full Output: Read GPT 6 Astra High output
• Extra High Tier Full Output: Read GPT 6 Astra xHigh output
• Max Tier Full Output: Read GPT 6 Astra Max output
• Ultra Tier Full Output: Read GPT 6 Astra Ultra output

Original source

The empirical token counts, runtime durations, cost calculations, and comparative insights documented in this article are sourced directly from the benchmark thread published on X by AI engineer @bridgemindai.

• @bridgemindai on X: Empirical Benchmark Thread: GPT 6 Astra in Codex at all six effort levels