Blog
  • AI Gateway
  • AI Security
  • AIOps
  • API Security
  • API Gateway
    • API Management
    • API Development
    • API Design
    • Automation
    • Service Mesh
    • Insomnia
  1. Home
  2. Blog
  3. Enterprise
  4. Multi-Cluster & Multi-Cloud Service Meshes With CNCF’s Kuma and Envoy
Enterprise
September 10, 2020
7 min read

Multi-Cluster & Multi-Cloud Service Meshes With CNCF’s Kuma and Envoy

Marco Palladino
CTO and Co-Founder of Kong
Topics
Multi CloudCNCFKuma
Share on Social

More on this topic

eBooks

Why API Initiatives and Strategies Fail: Guide to Common Pitfalls

eBooks

How to Sell Kong's API Platform to your CIO

See Kong in action

Accelerate deployments, reduce vulnerabilities, and gain real-time visibility. 

Get a Demo

When we first created Kuma - which means "bear" in Japanese - we dreamed of creating a service mesh that could run across every cluster, every cloud and every application. These are all requirements that large organizations must implement to support their application teams across a wide variety of architectures and platforms: VMs, Kubernetes, AWS, GCP and so on.

With Kuma - now a CNCF project and at the time of this writing, the only Envoy-based service mesh donated and accepted by the foundation - we have relentlessly executed on this vision with the community.

Kuma Multi Zone Capability

Starting from Kuma 0.6, which was released this summer with a new advanced multi-zone capability, Kuma now supports every cloud vendor, every architecture and every platform together in a multi-mesh control plane. When deployed in a multi-zone deployment, Kuma abstracts away both the synchronization of the service mesh policies across multiple zones and the service connectivity (and service discovery) across those zones. A zone can be a Kubernetes cluster, a data center, a cloud region, a VPC, a subnet and so on - we can slice and dice zones to our liking, and we can also mix Kubernetes and VM workloads into one distributed mesh deployment.

Kuma creates a service connectivity overlay across hybrid infrastructure to discover and connect services automatically, including hybrid Kubernetes + VM services.

Multi Service Mesh Support

This multi-zone capability has been added in addition to the multi-mesh support that Kuma introduced since day one to create multiple isolated meshes on the same cluster (with dedicated mTLS CAs) in order to reduce team coordination, increase isolation and improve security rather than one large service mesh that everybody is sharing. Also, since multi-zone leverages the first-class K8s + VM support that shipped since the first version of Kuma, all teams and workloads in the organizations can benefit from service mesh and not just our greenfield initiatives.

Kuma has been donated to CNCF, making it the first and only Envoy-based service mesh to be accepted in the foundation.

A Kuma service mesh distributed across every cloud, cluster and workload that the teams are using can therefore be managed from one individual cluster of Kuma itself. Meanwhile, multiple service meshes can be virtually provisioned on one Kuma control plane (horizontally scalable) to simplify mesh management across the organization - very similar to how Kubernetes and its namespaces work.

Kuma supports multiple virtual meshes on the same Kuma deployment, removing the requirement of having multiple service mesh clusters for each application and therefore lowering the ops costs significantly.

Envoy xDS and Kuma KDS

In Kuma, we can deploy a distributed service mesh running across multiple clusters, clouds or regions by leveraging the "multi-zone" deployment mode. The "multi-zone" deployment is a new way of running Kuma that has been introduced in v0.6+ in addition to the "standalone" deployment mode (one service mesh per zone).

In a multi-zone deployment, there are a few important features that Kuma provides:

  1. There are two control plane modes: global and remote. This is quite unique in the service mesh landscape.
  2. There is a new DNS “.mesh” zone for service discovery.
  3. There is a new “Ingress” data plane proxy type that enables connectivity between zones within a Kuma mesh.

In a distributed deployment, the "global" control plane will be in charge of accepting Kuma resources to determine the behavior of the service mesh via either native Kubernetes CRDs or vanilla YAML in VM-based deployments. It will be in charge of propagating these resources to the "remote" control planes - one for each zone that we want to support.

The "remote" control planes are going to be exchanging Kuma resources with the "global" control plane over an extension of the Envoy xDS API that we called KDS (Kuma Discovery Service) over a gRPC protocol, and therefore, over HTTP/2. The "remote" control planes are also going to be accepting requests from the Envoy-based data plane proxies that belong to the same zone over traditional xDS.

The remote control planes also embed a DNS service discovery that can be used to discover services across different zones. The above architecture can be easily installed in just a few steps using either the Kuma CLI “kumactl” or HELM charts.

In Kuma, we abstract away the Envoy proxy process in a "kuma-dp" process so that the user never directly configures or starts "envoy", therefore making the whole process of starting a data plane proxy much easier. Advanced users can still access the underlying "envoy" process if they want to.

By leveraging the native xDS API of Envoy to connect "kuma-dp" with "remote" control planes in every zone as well as leveraging KDS to connect the "remote" control planes to the global control plane, effectively we have gRPC communication enabled across the entire service mesh infrastructure stack in a consistent way.

The "global" and "remote" architecture has a few benefits:

  1. We can scale each zone independently by scaling the “remote” control planes and achieve HA in case one zone experiences issues.
  2. There is no single point of failure: even if the “global” control plane goes down, we can still register and deregister data plane proxies on the “remote” ones, and the most updated addresses of every service can still be propagated to other Envoy-based sidecars.
  3. The “global” control plane allows us to automatically propagate the state across every zone, while at the same time making sure that the "remote" control planes are aware of each zone's Kuma ingress in order to enable cross-zone connectivity.

The control plane separation governs how Kuma resources (like meshes and policies) are synchronized across the zones, but it requires two other components in order to enable discovery and connectivity at the data plane layer across our zones in an automated way: service discovery and the "ingress" data plane proxy mode.

Cross-Zone Discovery and Connectivity

Like we described, a multi-zone deployment can be used to deploy a distributed service mesh across multiple clouds and clusters, as well as to enable seamless communication between services running in different zones, effectively providing cross-zone service discovery and connectivity.

Among other use cases, cross-zone connectivity is useful for:

  • Enabling high availability across multiple Kubernetes clusters, regions and availability zones in both single and multi-cloud environments
  • Enabling traffic shifting from one zone to another for disaster recovery
  • Enabling a step-by-step transition of our applications from VMs to Kubernetes - or from physical data centers to the cloud - by leveraging traffic control policies to determine the conditions under which service traffic should be shifted from one zone to another.

Kuma's multi-zone deployment enables cross-zone connectivity by providing two important features:

  1. A new “ingress” data plane proxy mode processes incoming traffic into a zone. There will be one Kuma ingress deployment per zone, that can be scaled horizontally as the traffic increases. The “ingress” data plane mode is being added in addition to the default proxying one and the “gateway” one (to support third-party API gateways like Kong). Because of the new “ingress” mode, Kuma doesn't require a flat networking topology between zones and can support more complex infrastructure.
  2. A built-in service discovery DNS server resolves the address of a service to either an IP address of a replica in the same zone or the address of an ingress proxy in another zone.

Likewise with the "global" and "remote" control planes, the ingress and the DNS service discovery can also be installed in one click by following the multi-zone instructions on Kuma.

When it comes to service discovery, Kuma creates a ".mesh" DNS entry on the built-in DNS resolver that can be used to resolve services across the same zone or in other zones, effectively "flattening" the discovery of services across a complex infrastructure. Kuma will - accordingly to the traffic routing policies that have been configured - determine if we should be consuming a replica of the service in the local zone or if we should resolve the request to the IP address of a Kuma ingress in another zone, which will then leverage SNI to determine what service has been requested and route the request accordingly.

In this example, we have three services (users, invoices and billing). Requests to “invoices.mesh” will be proxied to an IP address within the same zone, whereas requests to “billing.mesh” will be automatically proxied to an ingress of another zone.

Since SNI resolution is mandatory for cross-zone communication, the mTLS policy must be enabled on the mesh. Also, since Kuma already knows where all the services are running, cross-zone discovery and connectivity happen automatically.

When a new service is registered into Kuma, a new "kuma.io/zone" tag is added to the data plane definition so that we can use the attribute-based policy selectors to configure Kuma policies like Traffic Route to determine the behavior of cross-zone traffic (blue/green or canary across different zones, weighted traffic, as well as traffic shifting).

When consuming any "{service-name}.mesh" on default port 80 (even if the service is not listening on port 80), the DNS resolver - in addition to resolving the address of the service - will also automatically resolve the port of the destination service and inject it into the connection in order to keep the uptime of the overall connectivity even when a team decides to re-assign ports of a service that other teams may be using. This feature reduces the team coordination required to maintain a large number of services and connections in a Kuma mesh.

Conclusion

Thanks to the new multi-zone capability that Kuma provides since v0.6+, we can now easily run a service mesh across multiple Kubernetes clusters, clouds and regions. Since Kuma natively supports both containerized and VM workloads, this functionality can also be used to create service connectivity across hybrid architectures.

By providing one-click installation steps to automate the installation of new zones as well as features like global/remote control planes, built-in service discovery and a native Kuma ingress, Kuma abstracts away service connectivity by creating a network overlay that effectively flattens out how services can discover and consume each other across complex network topologies. This makes it a great fit for any enterprise or distributed environment.

Topics
Multi CloudCNCFKuma
Share on Social
Marco Palladino
CTO and Co-Founder of Kong

Recommended posts

Kong Mesh 2.12: SPIFFE/SPIRE Support and Consistent XDS Resource Names

Kong Logo
Product ReleasesSeptember 18, 2025

We're very excited to announce Kong Mesh 2.12 to the world! Kong Mesh 2.12 delivers two very important features: SPIFFE / SPIRE support, which provides enterprise-class workload identity and trust models for your mesh, as well as a consistent Kuma R

Justin Davies

API Management as a Central Security Hub

Kong Logo
EnterpriseSeptember 11, 2025

While many organizations mistakenly believe a single tool can solve all their API security woes, the truth is far more complex. This blog post will dismantle the myth of the "silver bullet" and demonstrate how a comprehensive, defense-in-depth strat

Veena Rajarathna

You Might Be Doing API-First Wrong, New Analyst Research Suggests

Kong Logo
EnterpriseSeptember 3, 2025

Ever feel like you're fighting an uphill battle with your API strategy? You're building APIs faster than ever, but somehow everything feels harder. Wasn’t  API-first  supposed to make all this easier?  Well, you're not alone. And now industry analys

Heather Halenbeck

Kong's Dedicated Cloud Gateways: A Deep Dive

Kong Logo
Product ReleasesJune 18, 2025

In case you missed it, we recently made a big announcement around beta GCP support for Kong’s Dedicated Cloud Gateways (DCGWs) . There’s a lot of good stuff in there, but TL;DR DCGWs now support all three of the major cloud service providers (CSPs)

Michael Field

72% Say Enterprise GenAI Spending Going Up in 2025, Study Finds

Kong Logo
EnterpriseJune 18, 2025

Survey Says: Google LLMs See Usage Surge, Most OK with DeepSeek in the Workplace Enterprise adoption of large language models (LLMs) is surging. According to Gartner , more than 80% of enterprises will have deployed generative AI (GenAI) applicatio

Eric Pulsifer

5 Steps to Immediately Reduce Kafka Cost and Complexity

Kong Logo
EnterpriseJune 24, 2025

Kafka delivers massive value for real-time businesses — but that value comes at a cost. As usage grows, so does complexity: more clusters, more topics, more partitions, more ACLs, more custom tooling. But it doesn’t have to be that way. If your tea

Umair Waheed

Is Ambient Mesh the Future of Service Mesh?

Kong Logo
EnterpriseJune 30, 2025

A Practical Look at When (and When Not) to Use Ambient Mesh The word on the street is that ambient mesh is the obvious evolution of service mesh technology — leaner, simpler, and less resource-intensive. But while ambient mesh is an exciting develop

Umair Waheed

Ready to see Kong in action?

Get a personalized walkthrough of Kong's platform tailored to your architecture, use cases, and scale requirements.

Get a Demo
Powering the API world

Increase developer productivity, security, and performance at scale with the unified platform for API management, AI gateways, service mesh, and ingress controller.

Sign up for Kong newsletter

Platform
Kong KonnectKong GatewayKong AI GatewayKong InsomniaDeveloper PortalGateway ManagerCloud GatewayGet a Demo
Explore More
Open Banking API SolutionsAPI Governance SolutionsIstio API Gateway IntegrationKubernetes API ManagementAPI Gateway: Build vs BuyKong vs PostmanKong vs MuleSoftKong vs Apigee
Documentation
Kong Konnect DocsKong Gateway DocsKong Mesh DocsKong AI GatewayKong Insomnia DocsKong Plugin Hub
Open Source
Kong GatewayKumaInsomniaKong Community
Company
About KongCustomersCareersPressEventsContactPricing
  • Terms•
  • Privacy•
  • Trust and Compliance•
  • © Kong Inc. 2025