Enterprises are overpaying for simple AI queries — Snowflake's gateway now auto-routes to cut costs up to 3x

Enterprise teams running AI agents at scale are finding that a single model handles every task poorly — either the model is too expensive for simple questions or not capable enough for hard ones. Model routing, which picks the right model for each task automatically, is becoming the fix. Snowflake’s Cortex AI Gateway now offers dynamic model routing to address that: enterprises can select “auto” instead of a fixed model, and the system routes each task to whichever model offers the best combination of quality and cost. Snowflake said the capability can cut token costs by as much as 3x on some workloads — a figure from the company’s own internal testing — after finding that simple questions were often handled by its most capable model, making responses more expensive and slower than necessary. The move lands amid a broader industry shift toward automated model routing. Databricks, AWS, Google Cloud and Nvidia have all announced some form of model routing technology. Snowflake argues that model routing is more complex than just price and performance, it's also about governance and context. "For high quality, enterprise grade agents to be built, it's crucial to get the context and the governance right," Baris Gultekin, vice president of AI at Snowflake, told VentureBeat. “Context, trust and model choice all go hand in hand." Two mechanisms decide where a task goes The capability builds on Cortex AI Gateway, which Snowflake launched in July 2026 as a governance layer for agent and model traffic. Before dynamic routing, model selection ran off a static list per task rather than a true fallback system, Gultekin said. Dynamic routing itself runs on two mechanisms, according to Gultekin. A small model tries first. Under what Snowflake calls an advisor pattern, a smaller model attempts a task first. If it cannot finish the job, it calls a larger model as a tool and continues from there. A classifier sorts by task history. A separate classifier, trained on past queries, automatically routes straightforward questions to simpler models. Customers can still pin a model. Auto routing is optional. Customers can restrict routing to one model or a defined set of models, and the system routes only within that boundary. There is no separate fee. Snowflake prices AI purely on token usage. Routing to a cheaper model produces a cheaper bill, with no additional charge for the routing decision itself. Access controls follow the task, not just the data Snowflake ties routing to the same access controls it already uses for data governance. Governance starts at the data level with role-based access controls. It extends to models next, where customer roles map to buckets of approved models. It extends again to agents, where an agent can be restricted to narrower privileges than the user invoking it. Open models can run from a customer's own region to satisfy data residency requirements. Gultekin said all inference, open and proprietary alike, stays inside Snowflake's security boundary rather than routing out to an external provider. That regional and perimeter setup matters specifically for open models with non-U.S. origins, including DeepSeek-V4-Flash and GLM-5.3, both developed in China. Snowflake's recent acquisition of Natoma adds another layer. The deal brings more than 100 MCP connectors with scoped, governed access. An agent could get read-only access to a connected tool like email, for example, rather than broader permissions. Context lets a cheaper model do the work Snowflake recently announced its Horizon Context and Cortex Sense tools that provide context capabilities. Without good context, a model has to do the exploratory work itself, writing and testing SQL, searching through data and retrying when something does not work. Gultekin explained that the process is expensive, and getting it right typically requires a more capable model. Packaging the context in advance removes that exploratory step, which means a simpler, cheaper model can often handle the same task. Snowflake also builds agent memory into that context. As an agent is used repeatedly, its memory updates and gets folded back into future queries. The system does not re-solve the same problem from scratch each time. Memory becomes part of the context passed to the model. OpenRouter, Databricks and Nvidia are chasing the same problem There is no shortage of technologies in the model routing space. OpenRouter is one of the most widely known options, providing a platform that enables organizations to route based on cost and performance. Nvidia on August 11 announced Switchyard as a technology layer to help route AI model choice . Databricks has an offering as well with Smart Routing for its Unity AI Gateway. "The interesting part is what it says about where differentiation has moved," Sanjeev Mohan, Principal and Founder, SanjMo, told VentureBeat. "Snowflake isn't really selling routing, it's selling routing that never leaves the governed data boundary, with access controls, tagging, and cost attribution already attached." Mohan added that for a company whose data and compliance already center on Snowflake, routing that keeps data in place and attributes spend by team is a real lever on that problem. For a company without that center of gravity, a neutral gateway may route across more models with less friction. Mohan frames the market as three distinct camps rather than one competitive field. Databricks approaches governance from data engineering and ML lineage. Its Unity Catalog governs data, models and pipelines for teams building and training models. Snowflake approaches governance from analytics and access control, governing who can touch which data and attributing usage across business units. A third camp includes neutral gateways such as OpenRouter, LiteLLM, Portkey and hyperscaler routers like Azure AI Foundry. These compete on model breadth and avoiding lock-in rather than deep governance. Choosing a router means choosing a governance model Model routing is now table stakes for enterprises. The decision that matters is which governance model already fits how their data and teams are organized, not which vendor’s router is fastest or cheapest. Manual model selection is becoming a cost liability at agent scale. What worked when a team ran a handful of agents breaks down at scale. Hundreds of agents making routine model calls with no automated cost check in place adds up fast. Evaluate the governance model, not the router's feature list. The real question, per Mohan, is which governance model matches the data estate already in place, and which one gives the cost visibility needed to avoid an unpleasant surprise. The right starting point depends on where an enterprise's data already lives. A Snowflake shop gets more value from in-platform routing that respects its existing access model and bills back to cost centers than from raw model breadth, according to Mohan. A Databricks-centric team worried about lineage across training and deployment is better served by a gateway built around that same lineage. A multi-platform or model-first team that wants maximum choice with minimal lock-in fits better with a neutral gateway, the same pitch behind OpenRouter's valuation. "For a practitioner, don't start with the router, start with where your governed data and platform commitment already live, and with how exposed your margins are to inference cost," Mohan said.