There’s a quiet architectural shift happening in production AI. Teams are moving away from “pick one model and use it everywhere” toward “route each request to the right model for the job.” The reason is simple: a single model is always the wrong choice for part of your traffic — too expensive for the easy calls, or not capable enough for the hard ones. Routing fixes both. And a good routing setup needs a strong, cheap default tier to carry the bulk of the load.
That default-tier role is exactly what Google’s newest budget model was built for. Released July 21, 2026, Gemini 3.5 Flash-Lite is the most cost-efficient Gemini – multimodal, with a 1M-token context and about a fifth the price of 3.6 Flash – which makes it an ideal workhorse under a routing layer. A router like OrcaRouter makes the pattern concrete, fronting it and stronger models behind one OpenAI-compatible endpoint so each request lands on the right tier.
What model routing actually is
Model routing means deciding, per request, which model handles it. The decision can be as simple as a rule (“PDFs go to the extraction model”) or as dynamic as a classifier that scores difficulty and picks a tier. The goal is always the same: send each call to the cheapest model that will do it well, and no cheaper.
The canonical setup is two tiers. A cheap default tier handles the high-volume, simple majority. A frontier tier handles the hard minority. Requests flow to the default first and escalate only when needed.
Why the default tier should be a lite model
The default tier carries the most traffic, so its price and capability set the economics of the whole system. That’s why a lite model with real features — not a stripped-down one — matters. Gemini 3.5 Flash-Lite keeps native multimodality, the 1M-token context, tool calling, and structured outputs, so it can handle a wide slice of tasks directly rather than punting them upward. The more the default tier can absorb, the less traffic hits the expensive tier, and the lower your bill.
Escalation: when to send it up a tier
Good routing needs a clear escalation trigger. Common ones: a difficulty classifier flags the request; the lite model returns low confidence or fails a validation check; the task type is on a “always escalate” list (legal, medical, high-stakes). Start simple — route by task type — and add a confidence-based fallback once you have data on where the lite tier struggles.
Why OpenAI compatibility makes this easy
Routing is only practical if switching models is cheap. Because Gemini 3.5 Flash-Lite speaks the OpenAI chat-completions format, moving a request between it and another model is a change of model name, not a change of integration. You can lean on this further with a gateway: OrcaRouter fronts many models — lite and frontier — behind a single OpenAI-compatible endpoint, so your routing logic just picks a model string and the plumbing stays the same. That turns a two-tier architecture into a configuration detail rather than a build project.
Measure before you route
Set your routing thresholds with data. Run representative traffic through the lite tier, grade the results, and find the boundary where quality starts to slip — that’s your escalation line. Vendor benchmarks (74.0% OSWorld-Verified, 72.2% on 128k retrieval) are a starting hypothesis, not the answer; your own task distribution decides where the line sits.
A minimal routing sketch
You don’t need a fancy framework to start routing. The simplest version is a dispatch function that looks at a request and returns a model name. Version one can be pure rules: if the task type is “classify” or “extract” or “summarize,” return the lite model; if it’s on your high-stakes list, return the frontier model. Ship that, and you’ve already captured most of the benefit.
Version two adds a safety net. After the lite model responds, run a cheap validation — does the JSON parse, does the answer pass a schema or a confidence check? If it fails, retry once on the frontier model and log the case. Over a few weeks those logs tell you exactly where the lite tier struggles, and you fold that knowledge back into version one’s rules. Because Gemini 3.5 Flash-Lite is OpenAI-compatible, both the lite and frontier calls use the same client code with a different model string, so the dispatch function stays tiny. The discipline is to keep the router boring and data-driven: default cheap, escalate on a clear signal, and let production logs — not intuition — move the boundary over time. A boring router that you actually measure beats a clever one you can’t reason about.
Frequently asked questions
What is model routing in one sentence? Deciding, per request, which model handles it — so each call goes to the cheapest model that will do it well, and no cheaper.
Why should the default tier be a lite model? The default tier carries the most traffic, so its price and capability set the economics of the whole system. A lite model that keeps real features — multimodality, 1M context, tool calling, JSON — absorbs more work directly, so less traffic reaches the expensive tier.
What triggers an escalation to a frontier model? Common triggers are a difficulty classifier, a low-confidence or failed-validation response from the lite model, or an “always escalate” task list for high-stakes categories. Start with routing by task type and add confidence-based fallback once you have data.
Does routing require heavy engineering? Not if switching models is cheap. Because the lite model is OpenAI-compatible, moving a request between models is a change of model name; a gateway that fronts many models behind one endpoint turns the two-tier design into a configuration detail.
How do I set the thresholds? With data. Run representative traffic through the lite tier, grade it, and put the escalation line where quality starts to slip — not where a public benchmark suggests.
Bottom line
Model routing is how mature teams get frontier quality at budget prices, and it lives or dies on the strength of its default tier. Gemini 3.5 Flash-Lite is a strong default: cheap, multimodal, long-context, and OpenAI-compatible. Put it under a simple two-tier router — cheap by default, frontier on escalation — and let measured quality set the threshold.