Updated October 8, 2026, for laravel/ai 1.2. This post said you cannot route a model per step. laravel/ai 1.0 can, through step middleware, and laravel/ai merged that change eight days before this post went up. The router this post first showed crashed on 1.0, and the failover section missed that the docs already warn about plain lists. I rewrote both sections, and the companion repo’s tests now prove every runnable code block on 1.2.0. On October 9 I ran the new router against the API. See what changed.
A month ago you put #[UseCheapestModel] on the summariser. A production guide lists it under cost control. The docs say it will use “the cheapest model (e.g., Haiku)”.
The invoice never moved.
So check it. Not the config, not the attribute. The model id on the request your app builds. The companion repo’s ai:route-matrix prints it for every agent without spending a token. Three of its rows:
+-----------------------------+----------------------+---------------------+
| Agent | Provider / model | Decided by |
+-----------------------------+----------------------+---------------------+
| ConversationSummarizer | openai / gpt-6-luna | #[UseCheapestModel] |
| ConversationalResearchAgent | openai / gpt-6.1-sol | provider default |
| ResearchAgent | openai / gpt-6.1-sol | provider default |
+-----------------------------+----------------------+---------------------+That row is fine. Below are four places where the model you set and the model you get come apart, and none of them warns you.
Per-step routing exists now
The usual advice is “use a cheap model for the boring steps”. Haiku for the extract, Sonnet for the plan, inside one agent run.
On laravel/ai 0.11.2, the release this post first ran on, you could not do that. The loop took one model string and sent it on every step.
1.0 changed it. Agent middleware now wraps each generation step and receives a PendingStep. Call $step->withModel() and that step runs on another model.
That makes step middleware a sixth input. It runs after the other five, on every step, and overrides whatever they decided.
The router below uses it. First the five, because they decide the model before any step runs, and they fail without a warning.
The five inputs, in order
Five things can decide your model before the first step. They run in this order:
- The call-time
model:argument - A
model()method on the class - The
#[Model]attribute #[UseSmartestModel]or#[UseCheapestModel]- The provider’s own
defaultTextModel()
Step 4 only runs when the model is still null. One ?? in Promptable decides it:
yield [$provider, $model ?? $this->getDefaultModelFor($provider), --$remaining === 0];So #[Model] alongside #[UseCheapestModel] makes the cheap one dead code:
use Laravel\Ai\Attributes\Model;
use Laravel\Ai\Attributes\Provider;
use Laravel\Ai\Attributes\UseCheapestModel;
use Laravel\Ai\Contracts\Agent;
use Laravel\Ai\Enums\Lab;
use Laravel\Ai\Promptable;
#[Provider(Lab::Anthropic)]
#[Model('claude-sonnet-4-6')]
#[UseCheapestModel]
class PinnedResearchAgent implements Agent
{
use Promptable;
public function instructions(): string
{
return 'You are a content-research assistant.';
}
}That agent runs claude-sonnet-4-6. No warning, no log line, no exception.
The docs show #[Provider] with #[Model] in one example and #[UseCheapestModel] in the next. They never appear together. Put them on one class and one of them quietly loses.
If your summariser carries both, there is your invoice.
Your failover array threw away your model
Here is the one that changes what you pay.
The docs’ resilience advice is to pass an array of providers, so the SDK moves to the next one when the first is rate-limited or down.
They also say, in one sentence, what a plain list does to your model: “each provider uses its default model.”
Read that sentence literally. It covers the id you typed in the same call.
Take an agent pinned with #[Provider(Lab::Anthropic)] and #[Model('claude-sonnet-4-6')], with no role attribute, and prompt it five ways:
pinned, no failover -> anthropic / claude-sonnet-4-6
pinned + call-time model: haiku, no failover -> anthropic / claude-haiku-4-5-20251001
pinned + failover LIST -> anthropic / claude-sonnet-5-5
pinned + failover LIST + call-time model: haiku -> anthropic / claude-sonnet-5-5
pinned + failover MAP -> anthropic / claude-sonnet-4-6Read row four again.
$agent->prompt('...', provider: [Lab::Anthropic, Lab::OpenAI], model: 'claude-haiku-4-5-20251001');
// runs claude-sonnet-5-5Two lines do it. getProvidersAndModels() reads model() and #[Model] only when the provider is not an array:
if (! is_array($provider) && is_null($model)) {
// ... only now does #[Model] get read
}Then formatProviderAndModelList() maps a numerically keyed list to [provider => null] for every entry, and the call-time model: goes too. Each provider falls back to its own default.
claude-sonnet-5-5 at $2 in and $10 out per million tokens, instead of the Haiku 4.5 you typed at $1 and $5. On every request, because you added a fallback provider.
Do not file it as a bug. A May pull request made the list keep your model (#633), and Taylor Otwell reverted it within a week (#667).
The fix is the map form, the same one the docs show. Name the model per provider and the string keys keep it:
$agent->prompt('...', provider: [
'anthropic' => 'claude-sonnet-4-6',
'openai' => 'gpt-6.1-sol',
]);One thing survives the list form, and the docs’ sentence leaves it out: #[UseCheapestModel] and #[UseSmartestModel]. Those resolve later, per provider, inside getDefaultModelFor(). Attributes that name a role live. A model id you typed dies.
So the list brings PinnedResearchAgent’s dead #[UseCheapestModel] back. Under [Lab::Anthropic, Lab::OpenAI] it runs claude-haiku-5-5.
For pinned ids, use the map. Role attributes survive the list as it is, and a map would replace them with fixed ids.
Routing per step
Attributes bind one model to the class. Step middleware decides again on every step.
So pick a rule, and take it from your own runs. Mine come from the September runs below, and they argue against “Haiku for the extract”. Every expensive Haiku decision there was a tool choice, from the first step on. On step 0 it asked for eight to ten results per search, where Sonnet asked for five to seven. It requested full page text in one search, and it made three or four scrapes in a single step where Sonnet made two.
The one step without a tool choice is the answer. In the Sonnet runs it carried 46 to 60% of the context and 65 to 72% of the output.
So the tool steps stay on Sonnet, and the cheap model writes the answer with tools forbidden. laravel/ai’s own pull request for step middleware gives the idea as an example: “Reason with a strong model, finish with a cheap one.”
<?php
namespace App\Ai;
use Closure;
use Laravel\Ai\PendingStep;
class ModelRouter
{
public function __construct(
private array $cheap = ['anthropic' => 'claude-haiku-5-5', 'openai' => 'gpt-6-luna'],
private int $toolSteps = 3,
) {}
public function handle(PendingStep $step, Closure $next)
{
if ($step->number < $this->toolSteps || ! isset($this->cheap[$step->provider])) {
return $next($step);
}
return $next($step->withModel($this->cheap[$step->provider])->withToolChoice('none'));
}
}Three tool steps, because no Sonnet run in September needed more. A run that answers sooner never reaches the cheap model.
withToolChoice('none') keeps the tool definitions on the request and forbids new calls. The cheap model reads what Sonnet gathered and cannot fetch more.
The ids are keyed by provider. Failover reruns the whole prompt on the next provider, and every step of that run carries the new provider’s name. A single Claude id would go to OpenAI.
Register it on the research agent, pinned to Sonnet:
<?php
namespace App\Ai\Agents;
use App\Ai\ModelRouter;
use Laravel\Ai\Attributes\Model;
use Laravel\Ai\Attributes\Provider;
use Laravel\Ai\Contracts\HasMiddleware;
use Laravel\Ai\Enums\Lab;
#[Provider(Lab::Anthropic)]
#[Model('claude-sonnet-4-6')]
class RoutedResearchAgent extends ResearchAgent implements HasMiddleware
{
public function middleware(): array
{
return [new ModelRouter];
}
}Three faked tool calls, then an answer. The Laravel\Ai\Events\StartingStep event for each step:
step 0 -> claude-sonnet-4-6
step 1 -> claude-sonnet-4-6
step 2 -> claude-sonnet-4-6
step 3 -> claude-haiku-5-5 tool choice: noneThe saving is in the answer step. In September’s two Sonnet runs that used three tool steps, it was 52% and 62% of the bill. Its input text is fixed by the time the cheap model sees it. Haiku 5.5 bills $0.10 in and $0.50 out per million under 100,000 tokens, a thirtieth of Sonnet 4.6’s rate. Anthropic says its tokenizer counts the same text as about 30% more tokens than the previous one, which Sonnet 4.6 uses. I ran this agent’s answers and tool traces through the token-counting endpoint on both models, and Haiku counted 36% more, so the same input costs about a twenty-second. On the October runs, Haiku’s whole answer step cost about a twenty-fifth of Sonnet’s.
Every property on PendingStep is readonly. withModel() returns a copy, so pass the copy to $next(). Call it and drop the result, and the router runs and changes nothing.
Do not route the provider there
PendingStep has a withModel() and no withProvider(). Build a step by hand with another provider’s name, and the loop ignores the name:
middleware hands $next a step naming openai / gpt-6.1-sol
the request used: anthropic / gpt-6.1-solThe loop calls the provider it already holds, uses the step’s model and ignores its provider name. You get Anthropic carrying an OpenAI model id.
Route the model. Choose the provider before the call.
Your router’s test passes for the wrong reason
You wrote the router. Now prove it.
Three observables look like the obvious assertion. On the routed agent:
assertPrompted() sees : claude-sonnet-4-6
PromptingAgent reports : claude-sonnet-4-6
$response->meta reports : claude-haiku-5-5
steps 0 to 2 ran : claude-sonnet-4-6assertPrompted() and PromptingAgent see the prompt before any step runs. Delete the router and both still say Sonnet.
$response->meta comes from the final step alone. Break the router so every step runs on Haiku, tools included, and it still reports claude-haiku-5-5.
Any of them goes green on a broken router. Assert on every step instead:
use App\Ai\Agents\RoutedResearchAgent;
use Laravel\Ai\Responses\Data\Step;
use Laravel\Ai\Responses\Data\ToolCall;
public function test_the_answer_step_runs_on_the_cheap_model(): void
{
RoutedResearchAgent::fake([
new ToolCall('call_1', 'DatabaseQueryTool', ['query' => 'select title from posts']),
new ToolCall('call_2', 'DatabaseQueryTool', ['query' => 'select slug from posts']),
new ToolCall('call_3', 'DatabaseQueryTool', ['query' => 'select tags from posts']),
'Three angles.',
]);
$response = (new RoutedResearchAgent)->prompt('Find an angle on model routing.');
$this->assertSame(
['claude-sonnet-4-6', 'claude-sonnet-4-6', 'claude-sonnet-4-6', 'claude-haiku-5-5'],
$response->steps->map(fn (Step $step) => $step->meta->model)->all(),
);
}A faked ToolCall runs the real tool, so its name must match a tool the agent registers. DatabaseQueryTool reads the posts table, which RefreshDatabase migrates on the companion’s test class, so the test spends nothing.
The cheap model that cost more
One honest caveat before you route anything down. In September I ran the cheap model on whole runs.
Same agent, same topic, same ten minutes. Three runs on claude-sonnet-4-6, three on claude-haiku-4-5-20251001. They ran on laravel/ai 0.11.2, so each run used one model for every step. Published rates: Sonnet 4.6 at $3 in and $15 out, Haiku 4.5 at $1 and $5, with cache writes at 1.25× and reads at 0.1×. The cheap model bills a third per token.
It cost more on every pair.
run steps context sent output cost
sonnet, prefix cached 3 18,039 2,445 $0.086
haiku, prefix cached 5 92,668 1,499 $0.100 no answer
sonnet, no cache 4 28,716 2,435 $0.123
haiku, no cache 4 198,281 2,189 $0.209
sonnet, prefix cached 4 39,536 2,544 $0.146
haiku, prefix cached 3 239,940 2,066 $0.250Haiku averaged $0.187 a run. Sonnet averaged $0.118. A third of the price per token, 1.6× the bill.
The rate card is the one number in that decision you can look up. It is the one that mattered least. The bill followed the tokens sent, and the model decides how many tokens get sent.
Three Haiku runs, three different ways to send 5× the context.
The first hit the step cap. With no #[MaxSteps] the SDK allows round(tools × 1.5) steps, five for this agent. Haiku spent all five on tool calls: seven searches, four scrapes and two database queries. It was still calling a tool on step five. The loop ended. The response text is “Let me check our current post coverage more thoroughly and then compile the findings:”. Billed in full. No answer.
The second scraped three pages in one step, the entire Laravel AI SDK docs page among them. Context went from 12,000 tokens to 92,000, and the next step re-sent it.
The third asked one search to return the full text of ten results. The next step’s context was 112,000 tokens, before the run had scraped a single page.
And none of them cached a byte. The cost-math post measured this agent’s cacheable prefix at 1,042 tokens. Sonnet 4.6’s minimum is 1,024, so Sonnet wrote it once and read it on every later step. Haiku 4.5’s minimum is 4,096. Zero cache writes on both prefix-cached Haiku runs.
That floor belongs to Haiku 4.5. On laravel/ai 1.2, #[UseCheapestModel] resolves Anthropic to claude-haiku-5-5, and its minimum is 512 tokens, under this agent’s prefix. My October runs used Haiku 5.5 only for the answer step, with caching off, so they do not test that floor.
Quality went the same way. All three Sonnet runs scraped the Laravel News announcement of the feature and built their angles on its API. None of the Haiku runs scraped it. One produced nothing, one led with a vendor blog about OAuth tokens, one cited a Reddit consensus from search-result titles.
Three runs per model, one topic, one agent. Not a benchmark. What travels is the mechanism: on an agent that pulls its own context in through tools, the model decides the token count. The token count decides the bill.
That mechanism is why the router above keeps the cheap model off every step that picks a tool. The rule comes from these six runs.
The router against the API
In October I ran the router: three routed runs and three without it, interleaved, on the same agent and topic as September.
It never fired. Every run made two tool steps and answered on step 2. Each one scraped the Laravel News announcement on its first scrape step, and none took a third tool step.
So I set toolSteps to 2 for the routed runs and ran three fresh pairs. Haiku 5.5 wrote every routed answer:
run steps context sent output cost
routed, toolSteps 2 3 20,546 2,853 $0.034
no router 4 38,819 2,285 $0.151
routed, toolSteps 2 3 18,227 2,746 $0.029
no router 3 15,456 2,012 $0.077
routed, toolSteps 2 3 18,311 2,902 $0.029
no router 3 20,289 2,468 $0.098The routed runs averaged $0.031. The runs without the router averaged $0.108, or $0.087 without the four-step run. That run took a third tool step, which toolSteps: 2 forbids.
Haiku’s answer step cost $0.0023 to $0.0024. On the eight October Sonnet runs with two tool steps, the answer step cost $0.050 to $0.066, about 25 times as much. Haiku spent about 1,000 tokens thinking before it wrote, and its answers ran about half the length of Sonnet’s.
To check quality, I gave the six answers and their tool calls, shuffled and labelled only A to F, to a fresh Claude agent as judge. It got the agent’s instructions and September’s criteria: SDK names taken from pages the run actually read, and claims built on those pages rather than on search-result titles.
It ranked the three Haiku answers first, second and third. All six said something about a page they never opened, but the Haiku answers did it less. Each Haiku answer said its Reddit points came from a search snippet, because the page returned 403. No Sonnet answer said so. One Sonnet answer spliced words into a Reddit quote, and one used $this->agent->user(), which is in no source and not in the SDK.
That is three runs per arm, on one topic, with one judge. Haiku’s answers were about half the length, and a shorter answer has fewer sentences to get wrong. Haiku 5.5 also thinks by default, and Sonnet 4.6 did not think on these runs, so this compares two model generations as much as two sizes. Still, the cheap model wrote the answer from Sonnet’s research and did not do worse.
Take the step count from your own logs. Twelve of my Sonnet runs had nothing stopping a third tool step, and three took one. In each of those three, a first scrape hit a Reddit 403, and the third step fetched the announcement. toolSteps: 2 would have made Haiku answer those runs without it.
Set it too high and the router never fires. Set it too low and the cheap model answers without the page the run went back for. I kept three. I would rather pay Sonnet’s rate on an answer step, which shows on the invoice, than have Haiku answer without a page the run needed, which nothing flags.
Route it, then log each step’s usage on the routed run:
use Illuminate\Support\Facades\Log;
foreach ($response->steps as $i => $step) {
Log::info('agent step', [
'step' => $i,
'model' => $step->meta->model,
'input' => $step->usage->inputTokens,
'cache_read' => $step->usage->cacheReadInputTokens,
'cache_write' => $step->usage->cacheWriteInputTokens,
'output' => $step->usage->outputTokens,
]);
}input includes the cached tokens. Price each line at its own model’s rates, cache reads and writes included.
Print it before you ship it
You started with an attribute you trusted and an invoice that disagreed.
Now you can resolve every agent in your app without spending a token. You can see which of the five inputs decided each model, and watch a failover list discard the id you typed. You can price the cheap model on your own runs instead of trusting the rate card.
You can move the decision from the class to the step. And write a test that fails when the router breaks.
The command that printed the first table is app/Console/Commands/RouteMatrixCommand.php in the companion repo, laravel-ai-research-agent. Copy it into your app. It resolves every class in app/Ai/Agents, and --failover adds a second table under that provider list:
php artisan ai:route-matrix --failover=anthropic,openaiIt fakes a single step, so it shows step 0. The router above leaves step 0 alone, so read $response->steps for the rest. tests/Feature/ModelRoutingTest.php in the same repo runs every class, call and test in this post on fakes.
The fallback ids in this post are laravel/ai 1.2.0’s. Between 0.11.2 and 1.2.0, all six text fallbacks for Anthropic and OpenAI changed, and nothing in the app did. Which is the point.
Read the model off the step, not off the class.
What changed
October 9, 2026, after running the router against the API:
- The post said I had not run the router or judged its answers. I ran twelve runs on September’s topic, and a new section reports them. At three tool steps the router never fired. At two, the routed runs averaged $0.031 against $0.108, and a blind judge ranked Haiku’s three answers above Sonnet’s three.
- “About a twenty-third” came from Anthropic’s 30% tokenizer figure. This agent’s text measured 36%, so the router section now says about a twenty-second.
- The caching paragraph said I had not run Haiku 5.5. The October runs used it for the answer step only, with caching off, and the paragraph now says so.
October 8, 2026, for laravel/ai 1.2:
- The post said you cannot route a model per step, and that the SDK has no
withModel(). Both were true of laravel/ai 0.11.2, the release this post ran on. But laravel/ai merged per-step middleware withPendingStep::withModel()to its 1.x branch on September 10, and 1.0.0 shipped it on September 23. I rewrote the opening and the router section for it. - The old router type-hinted
AgentPrompt. On 1.0 and later it throws aTypeErroron the first step. ItsAgentResponsereturn type also broke streamed calls on 0.11.2. The new router is step middleware: Sonnet runs the tool steps and Haiku 5.5 writes the answer. The companion repo’s tests prove it on 1.2.0, on fakes. - The test section told you to assert on
PromptingAgentor$response->meta. On 1.x,PromptingAgentfires before step middleware runs, and$response->metareports only the final step. The test now checks every step’s model. - The failover section said nobody mentions what a plain list does to your model. The docs had said it since the end of May. The section now quotes them and links the May pull request and its revert. It also names its agent: the September probe used one without
#[UseCheapestModel], and with it the list runs the cheap model. - The provider section described middleware that rebuilt the whole prompt. It also said every observable hid the real provider, which was wrong on publish day:
$response->metareported it. The section now covers step middleware. - Smaller errors, wrong on publish day: the guard excerpt sat under the wrong claim, and the example agent had no
instructions()method, so it did not run. Run 02 made seven searches, not nine. Run 04’s jump came from three scrapes in one step. Run 06’s 112,000-token step came from one search that asked for full page text, not from a scrape. - The fallback ids now match laravel/ai 1.2.0. The six September runs used pinned model ids on 0.11.2, and their numbers are unchanged.