Your Laravel Agent Is Not Running the Model You Pinned

· 7 min read
laravelaiagentslaravel-ai-sdkcost
Model Routing cover: a dark cover with a vertical ladder of five code chips in resolution order, model colon, model parens, hash bracket Model, hash bracket UseCheapest and default, with all of them struck through in grey except hash bracket Model which glows emerald as the winner, and a muted red X labelled failover beside it

A month ago you put #[UseCheapestModel] on the summariser. Every guide says to. The docs say it will use “the cheapest model (e.g., Haiku)”.

The invoice never moved.

So check it. Not the config, not the attribute. The model id on the request that actually left your app.

+-----------------------------+------------------------+---------------------+
| Agent                       | Provider / model       | Decided by          |
+-----------------------------+------------------------+---------------------+
| ConversationSummarizer      | openai / gpt-5.6-luna  | #[UseCheapestModel] |
| ConversationalResearchAgent | openai / gpt-5.6-terra | provider default    |
| ResearchAgent               | openai / gpt-5.6-terra | provider default    |
+-----------------------------+------------------------+---------------------+

That one is fine. Four configurations later, none of them are.

First, the thing you came here for does not exist

The advice everyone repeats is “use a cheap model for the boring steps”. Haiku for the extract, Sonnet for the plan, inside one agent run.

You cannot do that.

TextGenerationLoop::generate() takes string $model and hands that same string to every generateTextStep() inside its step loop. The only per-step object is TextGenerationOptions, which carries maxSteps, maxTokens, temperature, topP, toolChoice and the two caching attributes. No model. Its forStep() method changes exactly one thing: it releases a forced tool choice after step 0.

Routing granularity is one whole agent invocation. Nothing finer.

Save yourself the weekend I nearly spent looking for the hook.

Then keep reading, because the granularity you do have is broken in four places and every one of them is silent.

The order nobody wrote down

Five things can decide your model. They run in this order:

  1. The call-time model: argument
  2. A model() method on the class
  3. The #[Model] attribute
  4. #[UseSmartestModel] or #[UseCheapestModel]
  5. The provider’s own defaultTextModel()

Step 4 only runs when the model is still null. That is the whole story, and it is one guard in the source:

if (! is_array($provider) && is_null($model)) {
    // ... only now does #[Model] get read
}

So #[Model] alongside #[UseCheapestModel] makes the cheap one dead code:

#[Provider(Lab::Anthropic)]
#[Model('claude-sonnet-4-6')]
#[UseCheapestModel]
class ResearchAgent implements Agent
{
    use Promptable;
}

That agent runs claude-sonnet-4-6. Forever. No warning, no log line, no exception.

The docs show #[Provider] with #[Model] in one example and #[UseCheapestModel] in the next. They never appear together. Put them on one class and one of them quietly loses.

Your failover array threw away your model

Here is the one that costs real money.

The official guidance for resilience is to pass an array of providers, so the SDK retries the next one when the first fails.

Good advice. Nobody mentions what it does to your model.

Same agent, five ways, recorded:

pinned, no failover                                 -> anthropic / claude-sonnet-4-6
pinned + call-time model: haiku, no failover        -> anthropic / claude-haiku-4-5-20251001
pinned + failover LIST                              -> anthropic / claude-sonnet-5
pinned + failover LIST + call-time model: haiku     -> anthropic / claude-sonnet-5
pinned + failover MAP                               -> anthropic / claude-sonnet-4-6

Read row four again.

$agent->prompt('...', provider: [Lab::Anthropic, Lab::OpenAI], model: 'claude-haiku-4-5-20251001');
// runs claude-sonnet-5

You wrote the model id at the call site, in the same call, and it was discarded. formatProviderAndModelList() maps a numerically keyed list to [provider => null] for every entry, so the model never survives the format step no matter where it came from. Each provider then falls back to its own default.

claude-sonnet-5 instead of claude-haiku-4-5-20251001, on every request, because you added a fallback provider.

The fix is the map form. Name the model per provider and the string keys branch keeps it:

$agent->prompt('...', provider: [
    'anthropic' => 'claude-sonnet-4-6',
    'openai' => 'gpt-5.6-terra',
]);

One thing does survive the list form: #[UseCheapestModel] and #[UseSmartestModel], because those resolve later, per provider, inside getDefaultModelFor(). Attributes that name a role live. A model id you typed dies.

Routing per prompt instead of per class

Attributes bind one model to the class for the life of the process. If you want the decision made per prompt, there is exactly one place that sees the prompt before the request is built.

Middleware.

The destination closure inside GeneratesText::prompt() reads $prompt->model, so a middleware that hands $next() a prompt carrying a different model changes the request:

class ModelRouter
{
    public function __construct(
        private string $cheap = 'claude-haiku-4-5-20251001',
        private string $smart = 'claude-sonnet-4-6',
        private int $threshold = 280,
    ) {}

    public function handle(AgentPrompt $prompt, Closure $next): AgentResponse
    {
        return $next($this->withModel(
            $prompt,
            strlen($prompt->prompt) < $this->threshold ? $this->cheap : $this->smart,
        ));
    }

    private function withModel(AgentPrompt $prompt, string $model): AgentPrompt
    {
        return new AgentPrompt(
            $prompt->agent,
            $prompt->prompt,
            $prompt->attachments,
            $prompt->provider,
            $model,
            $prompt->timeout,
            $prompt->invocationId,
            $prompt->approvalDecisions,
            $prompt->parentInvocationId,
            $prompt->parentToolInvocationId,
            $prompt->isFinalAttempt(),
        );
    }
}

On the agent whose attribute says claude-sonnet-4-6:

short prompt   ( 27 chars) -> request used claude-haiku-4-5-20251001
long prompt    (552 chars) -> request used claude-sonnet-4-6

Two details in that method are load-bearing.

AgentPrompt::$model is readonly, and revise() copies $this->model into the new instance. There is no withModel() helper in the SDK. Reach for revise() and you get back the model you were trying to change, which looks exactly like a router that runs and does nothing.

And carry every remaining argument. They all default to null, so the four-argument version compiles and runs. It also drops the tool-approval decisions and the parent invocation ids, which breaks approval resumes and orphans sub-agent traces, on the routed path only. That bug shows up in production, in the one code path your tests skipped.

Do not route the provider there

The same middleware can swap $prompt->provider. Do not.

middleware rewrites PROVIDER -> openai
  | the event reported: openai / gpt-5.6-terra
  | the request used:   anthropic / gpt-5.6-terra

The destination closure passes $this, the provider whose prompt() method was entered, and reads the model off the prompt. So changing the provider in middleware changes what your listeners and logs say. It does not change where the request goes. You get Anthropic carrying an OpenAI model id, and every observable in your app insists otherwise.

Route the model. Choose the provider before the call.

Your router’s test passes for the wrong reason

You wrote the router. Now prove it.

assertPrompted() sees: claude-sonnet-4-6
the request used     : claude-haiku-4-5-20251001

gatherMiddlewareFor() prepends the fake’s recording closure ahead of your agent’s own middleware, so Ai::recordPrompt() captures the prompt on the way in. assertPrompted() asserts against the model your attributes resolved, not the model your router sent.

A test written against it goes green while the router is unproven.

Worse, a correct router makes a correct-looking assertion fail.

Assert on something downstream instead. The PromptingAgent event fires inside the destination, after middleware. So does $response->meta:

$response = (new ResearchAgent)->prompt('Summarise this in one line.');

$this->assertSame('claude-haiku-4-5-20251001', $response->meta->model);

The cheap model that cost more

One honest caveat before you route everything down. I ran it.

Same agent, same topic, same ten minutes. Three runs on claude-sonnet-4-6, three on claude-haiku-4-5-20251001. Published rates: Sonnet at $3 in and $15 out, Haiku at $1 and $5. The cheap model bills a third per token.

It cost more on every pair.

run                       steps   context sent   output   cost
sonnet, prefix cached       3        18,039      2,445   $0.086
haiku,  prefix cached       5        92,668      1,499   $0.100   no answer
sonnet, no cache            4        28,716      2,435   $0.123
haiku,  no cache            4       198,281      2,189   $0.209
sonnet, prefix cached       4        39,536      2,544   $0.146
haiku,  prefix cached       3       239,940      2,066   $0.250

Haiku averaged $0.187 a run. Sonnet averaged $0.118. A third of the price per token, 1.6× the bill.

The rate card is the one number in that decision you can look up. It is the one that mattered least. The bill followed the tokens sent, and the model decides how many tokens get sent.

Three Haiku runs, three different ways to send 5× the context.

The first hit the step cap. With no #[MaxSteps] the SDK allows round(tools × 1.5) steps, five for this agent. Haiku spent all five searching, nine searches and four scrapes, and was still calling a tool on step five. The loop ended. The response text is “Let me check our current post coverage more thoroughly and then compile the findings:”. Billed in full. No findings.

The second scraped the entire Laravel AI SDK docs page as one tool result. Context went from 12,000 tokens to 92,000 in a single step, and the next step re-sent it.

The third got one page back at 110,000 tokens and built the whole answer on it.

And none of them cached a byte. The cost-math post measured this agent’s cacheable prefix at 1,042 tokens. Sonnet 4.6’s minimum is 1,024, so Sonnet wrote it once and read it on every later step. Haiku 4.5’s minimum is 4,096. Zero cache writes, three runs.

Quality went the same way. All three Sonnet runs scraped the Laravel News announcement of the feature and built their angles on its API. None of the Haiku runs scraped it. One produced nothing, one led with a vendor blog about OAuth tokens, one cited a Reddit consensus from search-result titles.

Three runs per model, one topic, one agent. Not a benchmark. What travels is the mechanism: on an agent that pulls its own context in through tools, the model decides the token count. The token count decides the bill.

Route it, then read $response->usage on the routed run. Not the rate card.

You started with an attribute you trusted and an invoice that disagreed.

Now you can resolve every agent in your app without spending a token. You can see which of the five actually decided each model, and watch a failover array discard the id you typed. You can price the cheap model on your own runs instead of trusting the rate card.

You can move the decision from the class to the prompt. And write a test that fails when the router breaks.

The command that printed the tables above is in the companion repo, laravel-ai-research-agent. Point it at your own agents:

php artisan ai:route-matrix --failover=anthropic,openai

Model ids in this post are the vendor fallbacks in laravel/ai v0.11.2, which live in provider classes and move on a composer update. Which is the point.

Read the model off the request, not off the class.

All articles