Auker Optimized Itself and Surpassed GPT-5.6 Luna to Redefine a Benchmark Pareto Frontier

Most AI budgets are spent buying intelligence. The question is how to best spend it.

Auker answers that through its Self-Improving Loop: an experimental process that learns how to decompose a query into sub-tasks, which models and capabilities to select for each task, how much intelligence to allocate, and how to improve those decisions over time.

After 116 experiments, Auker cut cost 6× to $9.73 while holding accuracy at 80.35% — displacing GPT-5.6 Luna and Qwen3.7-Plus on the MMMU-Pro Pareto frontier. Among models above 80% accuracy on MMMU-Pro, Auker is the most cost-effective model.

The result points to a larger shift in AI:

Intelligence isn't the bottleneck. Allocation is.

The Real Bottleneck Is Allocation

Today's models are already remarkably capable. The harder problem is deciding how to deploy that capability.

Which model should handle this task? How much reasoning should it use? Does it need a tool? Should the task be divided among several workers — or can one worker solve it directly? How do we allocate models to each of the workers?

Most AI systems answer those questions in advance. They use a fixed model, a fixed workflow, or a fixed team. That makes every query pay for roughly the same machinery, whether the task is routine or genuinely difficult.

A routine support request should not summon the same AI resources as the review of a multimillion-dollar contract. Yet static architectures struggle to tell the difference at runtime.

Auker is designed to make that decision dynamically. It builds a multi-agent team for each query and orchestrates it. It decides how to allocate models, agents, tools, reasoning, and execution across its team. Then it studies the outcome and improves the policy used for future work.

Think of frontier models as world-class specialists. Auker's job is to become the manager who knows whom to call, when, and for how long.

We Let Auker Experiment on Its Own Playbook

Auker's Self-Improving Loop treats orchestration as an experimental problem.

In each iteration, Auker forms one or more hypotheses: change a model, add a gate, adjust a prompt, modify routing logic, change when planning begins, or redistribute work across the team. It picks the most probable hypothesis to test.

Auker is free to propose its own experiments from an openly documented set of leads and rules. It runs the experiments, measures the outcome, and then hands the verdict to an automated grader which it cannot override. Judgment decides what gets explored. Hard evidence decides what gets kept.

Most experiments do not become part of the system. The changes that do survive become executable logic governing future runs.

In total, Auker ran 116 experiments across a bounded set of possible execution policies. Experiment after experiment, it tested different models, prompts, gates, routing rules, reasoning depths, and team structures. The strongest validated changes accumulated into a more adaptive policy.

The loop is simple to describe:

Hypothesis → Experiment → Evidence → Validation → Better future execution.

But one detail makes it unusual.

The thing being trained was not the model. It was the agent's code.

Traditional model training changes weights. Auker's Self-Improving Loop changes the code that governs how the agent works.

Each validated experiment can become code-as-policy: executable logic that decides which models, tools, workers, reasoning stages, routing paths, and escalation mechanisms to activate for a given task.

In batch mode, the loop acts as a training period for the agent itself. It does not just produce answers and record scores. It uses those scores to improve the code that will produce future answers.

No single experiment creates the advantage. Accumulation does. Each validated result changes the policy inherited by the next execution.

Not a bigger model. A better decision about which model.
Not more agents. The right agents.
Not more reasoning. Reasoning when it matters.

And That Changes What A Frontier Model Looks Like

If intelligence can be allocated, the question is no longer simply which model is smartest. It is which model delivers the right level of accuracy at the right cost. Higher accuracy often comes at a higher cost.

For tasks where errors are extremely expensive, paying more for additional accuracy may be the right choice. For high-volume workloads, a small accuracy trade-off may justify a substantial efficiency gain.

That is why the Pareto frontier – the boundary formed by models that no other model can beat on both accuracy and cost – is more useful than a leaderboard alone. A leaderboard asks:

Which model has the highest score?

The Pareto frontier asks the more practical question:

What capability are you buying, and at what cost?

A model lies on the frontier when no other model is both more accurate and less expensive. Redefining the frontier means establishing a cost–accuracy trade-off that the models behind it cannot match. Models farther up the frontier achieve higher accuracy at higher cost.

Auker's result matters because it establishes an alternative class of frontier models. Frontier models are typically defined by the highest accuracy. Auker takes a different approach: rather than maximizing accuracy alone, it maximizes the value delivered for every dollar spent.

What Happened on MMMU-Pro?

MMMU bills itself as “A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI.” As shown above, it consists of college-level exam questions spanning 30 subjects across six core disciplines and 183 subfields. Each question combines a text prompt with one or more images, including circuit diagrams, radiology scans, sheet music, financial charts, chemical structures, and geometry figures.

MMMU-Pro is the robust version of that benchmark. It hardens the original two ways: it expands each question to up to 10 answer options (so lucky guessing collapses from ~25% to ~10%), and it adds a vision-only setting where the question itself is embedded in the image. We report the Standard setting on its 1,730-question test split.

We began with a straightforward objective:

Reduce cost while maintaining an accuracy floor.

That objective allowed Auker to capture the most accessible efficiency gains first. We then raised the bar:

Could it improve its position on the cost–accuracy Pareto frontier?

On the full 1,730-question MMMU-Pro benchmark, Auker's starting policy produced:

80.40% accuracy at $59.19.

After 116 experiments, the evolved policy produced:

80.35% accuracy at $9.73.

That is an approximately 6× reduction in cost with essentially unchanged measured accuracy. The 0.05-percentage-point difference is too small for this benchmark to reliably distinguish the two policies. Economically, however, the change is substantial.

The starting policy sat off the frontier. The evolved policy became its second point.

From Fig. 1, we read the accompanying chart from the top-left outward, the region that represents high accuracy at low cost. The teal line is the Pareto frontier consisting of the models that nothing else dominates.

Auker is the second point on that line, and the only model under $30 scoring above 80%. In particular:

• No model scoring 80% or above is cheaper. Only three models on the full board cost less to run, and all three score between 53% and 75%.

• Among models that reach 80%, Auker is the most cost-effective. It has the lowest cost per accuracy point — 1.4× better than the nearest such model and 15× better than the priciest model in that band.

• It dominates 18 of the 29 costed models outright, including Qwen3.7 Plus at 80% and $14, GPT-5.6 Luna at 79% and $12, and Gemini 3 Flash at 80% and $30.

• Against Gemini 3.6 Flash at 83% and $35, Auker trades approximately 2.6 accuracy points for $25.27. That is a real trade-off, not a rounding artifact.

Thus, on this MMMU-Pro cost–accuracy comparison, Auker's Self-Improving Loop optimized and evolved its execution policy from an off-frontier starting point to the second point on the frontier, displacing GPT-5.6 Luna and Qwen3.7 Plus.

A More Sophisticated Policy, Yet Fewer LLM Calls

At first glance, the efficiency gain might suggest that we stripped away the multi-agent architecture.

We did not.

The planner, worker team, routing mechanisms, models, tools, and escalation paths remain available. What changed was the policy governing when and how to use them.

On MMMU-Pro, the loop learned that many questions did not benefit from being divided into more subproblems. In those cases, one worker within the team could complete the reasoning path without activating additional workers. For harder tasks, the broader team remained available.

At the same time, the loop added gates and execution logic that made the model more responsive to the state of each task.

The result appears paradoxical:

The policy became more sophisticated while average model usage fell from roughly 7 calls per question to 4.9.

More adaptive policy. Fewer LLM calls.

To reduce the number of LLM calls, Auker did not eliminate its team. It became a better manager of that team.

The Enterprise Lesson

Enterprises do not have a shortage of capable models. They have too many models, too many possible workflows, and no durable way to decide which combination each task requires. The cost of guessing wrong is not abstract — up to 70% of AI inference spend is wasted on poor allocation in our study.

Choosing one frontier model for everything simplifies procurement, but it does not solve the runtime problem. Work varies not only across departments, but also within them. A routine risk review and an active incident are different jobs. A standard marketing brief and a reputational crisis should not receive the same team, reasoning depth, or budget.

The same inefficiency Auker eliminated for itself is what enterprises face every day.

Given the heterogeneous workloads within enterprises, the right allocation cannot be chosen once and left unchanged. It must be composed for each task at hand and continuously refined as the system gathers evidence.

That is the role of Auker's Self-Improving Loop. It detects what a task requires, activates the appropriate capabilities, observes the outcome, and feeds validated learning back into the execution policy.

The scarce skill is no longer simply building intelligence. It is knowing how much intelligence to deploy, where, and when, trained on the unique workloads of each enterprise.

One Benchmark In A Comprehensive Evaluation

The 116 experiments were conducted as part of a broader development process using composite benchmark data across multiple domains and task types. The goal is to learn policies that allocate well generally, not merely on one kind of question.

We highlight MMMU-Pro here because it gives a particularly clear view of the cost–accuracy trade-off. It is one benchmark in Auker's broader evaluation program, not the full scope of its training or evaluation. Additional cross-domain studies are underway, and we plan to report the broader methodology and results in an upcoming paper.

From Research To Production

These results come from Auker's research and experimentation environment. We are now bringing the validated findings from its Self-Improving Loop into production through our Decision Intelligence OS.

Soon, the allocation strategies that moved Auker onto the MMMU-Pro Pareto frontier will start composing and executing multi-agent teams on real-world tasks.

Next, we're bringing that policy to enterprise workloads — and if your inference budget hasn’t been spent optimally, we want to show you the gap.

Experience Auker and Follow Reasonary AI as we take Auker's self-improving technology from research into production at https://reasonary.ai/

The frontier isn't just a bigger model. It's a smarter allocation of intelligence — that keeps getting smarter.