Everyone's telling you to innovate faster with AI. Move quicker. Ship more features. Deploy more agents. But before we sprint headlong into the AI revolution, we need to have a proper dollars-and-cents conversation that most companies are avoiding.
*Disclaimer: Statistics cited in this post were reported in the *[*2025 State of AI Cost Governance Report*](https://www.mavvrik.ai/state-of-ai-cost-governance-report/)*2025 State of AI Cost Governance Report** by Mavvrik.*
The data is stark: **84% of companies report more than a 6% hit to gross margin from AI costs**.
Within that alarming statistic, 58% see a 6–15% reduction, and 26% report erosion of 16% or more. This isn't a rounding error: this is a fundamental business problem threatening the viability of AI-powered products and services.
According to the report, AI costs are already eroding the margins that businesses have worked years to build. Yet most organizations are racing ahead without understanding the true cost of their AI infrastructure.
Why — with all of the promise that AI brings — is this happening at such an alarming rate?
Here's a sobering reality: only about 35% of companies include on-premises components in their AI cost reporting. Even more concerning, roughly half don't include LLM API costs in their tracking — even when AI is a core product component.
When teams are asked what would most improve their cost management, the answer is clear: unified visibility across environments is the top priority, followed closely by clear cost attribution. You can't optimize what you can't see.
Only 15% of companies can forecast AI costs within ±10% accuracy. Let that sink in. The majority (56%) miss their forecasts by 11–25%, and nearly one in four companies miss by more than 50%.
For CFOs and budget owners trying to protect gross profit targets, this level of unpredictability is a nightmare. As AI grows as a share of cost of goods sold (COGS), the inability to accurately forecast becomes an existential threat to financial planning.
A full 61% of companies run AI workloads across a combination of public and private environments and different tools. This hybrid pattern spans all company sizes, including small businesses, and creates exponentially greater difficulty in achieving unified cost reporting and governance.
All of this — mostly driven by lack of a unified approach to managing AI resources and AI connectivity — results in the** hidden AI fragmentation tax**. Let’s break it down.
*Source: 2025 State of AI Cost Governance Report by Mavvrik*
Let’s take an MCP-enabled agentic workflow as our example.
Each MCP client connects to MCP servers. Those servers interact with LLMs. Your end consumers trigger these interactions. And every single one of those touchpoints generates LLM API token consumption that quietly eats into your margins.
The 2025 State of AI Cost Governance Report reveals a critical insight: "Even companies that do not charge for AI-enabled products are heavy users of third-party LLMs (73%), meaning token-based costs are quietly reducing gross margins without being offset by direct revenue."
Think about that. Nearly three-quarters of companies are hemorrhaging margin to LLM providers without generating corresponding revenue.
LLMs aren’t the only resource that AI consumes, and they certainly aren’t the only driver of excess networking / egress charges, which is the second most common unexpected AI cost according to the report.
*Source: 2025 State of AI Cost Governance Report by Mavvrik*
Whether the AI application itself directly consumes them, or the AI application consumes an MCP server that consumes them, there is a massive amount of agentic consumption of both APIs and real-time data that organizations must be aware of.
So, when you map out the full architecture of modern AI applications, you see a sprawling web of connectivity:
And of course, all of this exists — for many businesses — across multiple on-prem, hybrid, cloud, and multi-cloud vendor environments.
Are you starting to see the looming problem? It’s time for a reality check.
The typical AI infrastructure today suffers from three critical problems:
This is why — when the report explores “Tactics for improvement,” it finds:
But, given this mess of connectivity to manage, how is this possible?
The solution isn't to stop innovating or slow down. It's to build a unified AI cost governance layer that can unify real-time cost visibility & enforcement.
From a technical implementation perspective, this means building a single cost control plane that provides:
This governance layer needs to span your entire AI infrastructure:
*This* is why we are [so excited about OpenMeter and what it brings to the Konnect platform](https://konghq.com/blog/product-releases/konnect-metering-and-billing)so excited about OpenMeter and what it brings to the Konnect platform. By combining the existing LLM, MCP, API, Eventing, and microservices runtime infrastructure already in Konnect with the real-time metering, billing, and usage attribution of OpenMeter (now powering [Konnect Metering and Billing](https://konghq.com/products/kong-konnect/features/usage-based-metering-and-billing)Konnect Metering and Billing), you have a single platform that can:
And all of this is, of course, on top of the security, reliability, performance, and developer experience benefits of using Konnect to build, run, discover, and govern the resources that drive AI, API, eventing, and microservices innovation.
Today, [Konnect Metering & Billing](https://konghq.com/blog/product-releases/konnect-metering-and-billing)Konnect Metering & Billing is in early access. If you’d like to inquire into the early access program, [fill out the form here](https://docs.google.com/forms/d/e/1FAIpQLSeMk-CTemciPmzDB5ULBhUB8sxHy_COLwKfS1iq-zEYVZ0C8g/viewform)fill out the form here. We’ll get back to you soon.
Take it from the report itself: "As AI transforms from 'nice to have' to 'must have,' the companies that master cost visibility and control will protect their margins while competitors watch profits disappear into untracked infrastructure costs."
Here's what this means practically:
In a world where everyone has access to similar AI capabilities, competitive advantage comes from operational excellence. The companies that can deliver AI-powered experiences profitably will win.
This isn't about choosing between speed and control. Modern AI Connectivity platforms must deliver both — fast deployment cycles with comprehensive cost governance built in from day one.
Seeing your costs isn't enough. You need real-time enforcement mechanisms that prevent overruns before they impact margins, not months later in a financial review.
Every day without unified AI cost governance is another day of margin erosion. The longer you wait, the more deeply embedded these costs become in your COGS structure, making them exponentially harder to unwind.
AI innovation is critical. Speed matters. But sustainable AI businesses are built on a foundation of cost visibility and control. Before you accelerate, make sure you can see where you're going—and how much it's actually costing you to get there.
The companies that figure this out will thrive. The ones that don't will find themselves with impressive AI capabilities and vanishing margins.
At Kong, we’re here to help you win the AI cost governance battle. [Just give us a ring](https://konghq.com/contact-sales)Just give us a ring.
Bring Financial Accountability to Enterprise LLM Usage with Konnect Metering and Billing Showback and chargeback are not the same thing. Most organizations conflate these two concepts, and that conflation delays action. Understanding the LLM showb
The first pillar is enablement. Developers need tools that reduce friction when building AI-powered applications and agents. This means providing: Native MCP support for connecting agents to enterprise tools and data sources SDKs and frameworks op
Introduction: It’s a great report for us here at Kong, and it further validates the changes happening in the larger market The 2025 Gartner Magic Quadrant for API Management report was a great one for us here at Kong. We continue to move “up and to
Ever feel like you're fighting an uphill battle with your API strategy? You're building APIs faster than ever, but somehow everything feels harder. Wasn’t API-first supposed to make all this easier? Well, you're not alone. And now industry analys
Survey Says: Google LLMs See Usage Surge, Most OK with DeepSeek in the Workplace Enterprise adoption of large language models (LLMs) is surging. According to Gartner , more than 80% of enterprises will have deployed generative AI (GenAI) applicatio
AI is quickly transforming the way businesses operate, turning what was once futuristic into everyday reality. However, we're still in the early innings of AI, and there are still several key limitations with AI that organizations should remain awa
LLMs are powerful, but not inherently privacy-aware LLMs operate as highly capable, non-deterministic pattern matchers. But they come with two significant privacy challenges: They don’t automatically distinguish between sensitive and non-sensitive
Get a personalized walkthrough of Kong's platform tailored to your architecture, use cases, and scale requirements.