As organizations transition from experimental AI to production-grade systems, they often face a fragmented landscape of unmanaged LLM providers, complex tool integrations, and escalating security risks. Without a centralized orchestration layer, developers struggle to maintain consistent governance, observability, and cost control across disparate models and the [Model Context Protocol (MCP) ](https://konghq.com/blog/learning-center/what-is-mcp)Model Context Protocol (MCP) ecosystem. This infrastructure gap leaves AI applications vulnerable to sophisticated threats like prompt injection and data exfiltration, necessitating a unified stack that secures the edge while streamlining the data plane.
As organizations move from simple large language model (LLM) experiments to production-grade multi GenAI model AI systems, a new infrastructure stack is emerging where scalability and security become even more critical. This blog post describes how Kong and Akamai are providing a new combined offering to build a modern AI-based infrastructure for new agents.
Together, the following components represent the three layers of the new AI platform:
Here's the fundamental diagram with the three main components:
A basic LLM request flow goes through the following steps:
Multiple variations also exist. Depending on the request, Kong AI Gateway may send queries to the Vector Database to solve, for example, Semantic Routing policies.
Similarly, the Agent might send MCP requests which will be also handled by the Data Plane which is responsible for enforcing policies specifically defined for the MCP Servers and Tools, like, for example OAuth2 Authorization.
As part of the Kong Konnect product, the Kong AI Gateway is designed to solve several challenges in AI adoption. In fact, modern AI applications rarely rely on a single model. Instead, they combine multiple GenAI providers, Agent frameworks, external MCP (Model Context Protocol) Servers and REST based APIs.
Kong AI Gateway is a powerful set of features built on top of Kong Gateway, designed to help developers and organizations effectively adopt AI capabilities quickly and securely. Logically speaking it provides capabilities to play both roles LLM (or GenAI) and MCP Gateway.
At its core, Kong AI Gateway provides:
The Kong Konnect platform provides a Cloud Management Control Plane (CP), which manages all service configurations. Through specific communication channels, it propagates those configurations to all runtime Data Planes (DP) where the Kong AI Gateway resides.
Underneath the Kong Konnect AI Gateway Data Plane sits the compute infrastructure. This is where Akamai Linode Kubernetes Engine (LKE) comes in.
Besides being totally based on Kubernetes, Akamai LKE provides Virtual Machines based on CPU/GPU, Load Balancers, Storage, Global Edge infrastructure, and Built-in Network Security.
In this architecture, security is not an afterthought—it is embedded directly into the flow of the AI interaction. Akamai Firewall for AI is deployed here as a high-performance, specialized security layer designed specifically for the nuances of Large Language Models (LLMs).
By sitting alongside the Kong AI Gateway on Akamai’s Edge, FAI provides real-time inspection of the "intent" behind every request. It doesn't just look for malicious code; it understands the context of the conversation. While Kong orchestrates the LLM traffic and MCP flows, Akamai secures the intelligence being exchanged.
The Akamai Firewall for AI introduces sophisticated security policies to address the unique vulnerabilities of the GenAI era:
Here's a screenshot of a Firewall for AI policy definition:
By unifying Kong AI Gateway with Akamai’s secure cloud infrastructure, organizations can finally move beyond the "experimental" phase of GenAI with total confidence. This "Secure by Design" architecture that effectively decouples your AI logic from the underlying model providers, shielding your business from vendor lock-in and volatile API costs.
For the enterprise, this means accelerated time-to-market for new AI agents, strengthened compliance through rigorous edge filtering, and operational clarity via a single pane of glass for all AI and MCP traffic. Instead of managing fragmented tools, you gain a resilient, high-performance foundation that turns AI from a security risk into a scalable competitive advantage.
**Ready to Build?**
Explore the documentation from [Akamai](https://techdocs.akamai.com/home)Akamai and [Kong](https://developer.konghq.com/)Kong to find the right deployment path for your environment.
Claude Code is Anthropic's agentic coding and agent harness tool. Unlike traditional code-completion assistants that suggest the next line in an editor, Claude Code operates as an autonomous agent that reads entire codebases, edits files across mult
[](https://konghq.com/blog/engineering/claude-code-governance-with-an-ai-gateway)
In today's digital landscape, APIs are the backbone of modern applications, and AI is the engine of innovation. As organizations increasingly rely on microservices and AI-powered features, the API gateway has become the critical control point for man
[](https://konghq.com/blog/engineering/prisma-airs-kong-ai-gateway)
MCP servers expose all tools by default. There are two problems with this: security (agents get capabilities they shouldn't have) and performance (too many tools degrade LLM tool selection). The solution? Put a gateway between agents and MCP server
[](https://konghq.com/blog/engineering/mcp-tool-governance-security-meets-context-efficiency)
The example below shows how an AI agent can be built using Volcano SDK with minimal code, while still interacting with backend services in a controlled way. The agent is created by first configuring an LLM, then defining an MCP (Model Context Prot
[](https://konghq.com/blog/engineering/secure-ai-agents-volcano-sdk-kong-mcp-proxy)
Agent-to-agent communication is the next frontier of AI infrastructure. As teams decompose monolithic AI workflows into specialized agents — a research agent, a booking agent, a summarization agent — the calls between those agents become as importa
[](https://konghq.com/blog/product-releases/kong-ai-gateway-3-14)
The Anatomy of Architectural Complexity Modern architectures now juggle three distinct traffic patterns. Each brings unique demands. Traditional approaches treat them separately. This separation creates unnecessary complexity. North-South API Traf
[](https://konghq.com/blog/enterprise/microservices-to-ai-traffic-kong-as-the-unified-control-plane)
Executive Summary AI adoption has moved past the "honeymoon phase" and into the "operational chaos" phase. As enterprises juggle multiple LLM providers, skyrocketing token costs, and "Shadow AI" usage, the need for a centralized control plane has be
[](https://konghq.com/blog/enterprise/ai-gateways-for-scalable-ai-connectivity)