Mutual TLS (mTLS) and Workload Identity

Service meshes provide comprehensive security for internal traffic, through the capabilities mentioned below.

Mutual TLS (mTLS): Mutual TLS (mTLS) automatically encrypts all east-west traffic while using certificates to authenticate services and verify cryptographic identities. This approach enables a zero-trust security model, ensuring that every service interaction is authenticated and protected.

Certificate management: Service meshes simplify certificate management by automating rotation and renewal while providing SPIFFE-compliant workload identities. With a built-in certificate authority, they eliminate the need for manual intervention and reduce the risk of expired or misconfigured certificates.

Traffic Management Primitives

Sophisticated traffic control includes:

• Load balancing - Service meshes provide intelligent load balancing to distribute traffic evenly across service instances, improving availability and performance within distributed environments. Algorithms such as round-robin and least-request help optimize resource utilization, while consistent hashing enables predictable routing for stateful services. Session affinity can also be maintained when required, ensuring request continuity for workloads that depend on persistent connections.
• Resilience patterns - Service meshes enhance application resilience by embedding fault-tolerance directly into service-to-service communication. Circuit breakers help prevent cascading failures by isolating unhealthy services, while automatic retries with exponential backoff increase the likelihood of successful requests during transient outages. Configurable timeouts prevent resource exhaustion, and fault injection allows teams to simulate failure scenarios, strengthening operational readiness and validating system behavior before issues occur in production.
• Traffic splitting - Service meshes enable advanced traffic control by intelligently routing requests between service versions. Teams can direct a small percentage of traffic, such as 1 – 5%, to new releases for canary deployments, minimizing risk before broader adoption. Blue-green deployments support near-seamless cutovers, while A/B testing allows teams to evaluate changes under real production conditions. Progressive rollout capabilities further help introduce updates gradually, improving stability and giving operators time to detect and resolve issues before they propagate across the system.
• Observability and telemetry - Service meshes deliver detailed operational visibility without requiring changes to application code, giving teams a clearer understanding of how services behave in production. Built-in distributed tracing tracks requests as they move across services, while golden signals — latency, errors, traffic, and saturation — provide a reliable foundation for monitoring system health. Service dependency mapping helps operators visualize communication paths and quickly identify bottlenecks, and performance profiling surfaces inefficiencies before they escalate. Custom metrics collection further allows organizations to tailor observability to their specific operational and business needs.

Sidecar Proxy Model

The traditional sidecar model deploys a proxy alongside each service instance, managing inbound and outbound traffic while enabling consistent security, policy enforcement, and observability — all without modifying application code. According to [Istio service mesh](https://konghq.com/blog/learning-center/what-is-istio-service-mesh)Istio service mesh's official performance benchmarks, the Envoy proxy uses 0.35 vCPU and 40 MB memory per 1000 requests per second going through the proxy ([Istioldie 1.8 / Performance and Scalability](https://istio.io/v1.8/docs/ops/deployment/performance-and-scalability/)Istioldie 1.8 / Performance and Scalability) ([Istioldie 1.14 / Performance and Scalability](https://istio.io/v1.14/docs/ops/deployment/performance-and-scalability/)Istioldie 1.14 / Performance and Scalability) in baseline configurations. However, memory consumption can vary significantly based on cluster configuration state, potentially reaching 700MB to 1.2 GB in large clusters with many services ([Watch Out for This Istio Proxy Sidecar Memory Pitfall](https://medium.com/geekculture/watch-out-for-this-istio-proxy-sidecar-memory-pitfall-8dbd99ea7e9d)Watch Out for This Istio Proxy Sidecar Memory Pitfall) when namespace isolation is not properly configured.

The key characteristics of this model are:

• Maximum control and isolation
• Fine-grained policy enforcement
• Resource overhead varies by environment

Examples: Istio with Envoy, Linkerd

Ambient Mesh (Sidecar-less)

Newer deployment models are significantly reducing the resource overhead traditionally associated with sidecars. Research indicates that Istio’s dataplane in a ztunnel-only ambient configuration can use roughly 1% of the memory and CPU required in sidecar-based setups, representing up to a 90% redambuction in allocated resources at Layer 4. This efficiency makes ambient architectures have an been attractive option for organizations looking to scale service mesh capabilities while controlling infrastructure costs.

There key benefits:

• Node-level proxies replace per-pod sidecars
• Dramatically reduced resource consumption
• Simplified operations

Examples: Istio ambient mode, Cilium eBPF

However, for most enterprise production environments with diverse services, high compliance needs, or multiple teams, sidecar-based service meshes are still the right approach and provide the clarity, control, and maturity our customers can count on. Learn more about [ambient mesh vs sidecar-based mesh] (https://konghq.com/blog/enterprise/ambient-mesh-vs-sidecar-based-mesh).

NIST SP 800-204A Standards Reference

The increasing trend in building microservices-based applications calls for addressing security in all aspects of service-to-service interactions due to their unique characteristics. The distributed cross-domain nature of microservices needs secure token service (STS), key management and encryption services for authentication and authorization, and secure communication protocols. The ephemeral nature of clustered containers (by which microservices are implemented) calls for secure service discovery. The availability requirement calls for: (a) resiliency techniques, such as load balancing, circuit breaking, and throttling, and (b) continuous monitoring (for the health of the service). For more information, refer to [Istio / Performance and Scalability](https://istio.io/latest/docs/ops/deployment/performance-and-scalability/)Istio / Performance and Scalability..

The NIST framework emphasizes the following.

• Zero-trust network principles
• Secure service discovery
• Resilience through redundancy
• Continuous security monitoring
• Policy-based access control

Coexistence architecture

Many organizations use both technologies in tandem to ensure comprehensive coverage. This layered strategy provides security and control across a range of traffic patterns

Integration points

1. Identity Propagation: Gateway validates external credentials and passes claims to the mesh
2. Unified Telemetry: Correlate traces from edge to backend
3. Policy Synchronization: Consistent security across layers
4. Traffic Handoff: Seamless transition between gateway and mesh

Platform-specific examples

• AWS Architecture - In an AWS-based architecture, API Gateway typically manages external traffic, serving as the entry point for client requests while enforcing security and routing policies. Internal service-to-service communication is handled by a service mesh such as AWS App Mesh or Istio running on Amazon EKS, enabling consistent traffic control, encryption, and policy enforcement across workloads. Unified observability is supported through services like Amazon CloudWatch and AWS X-Ray, which provide meaningful insights into performance and dependencies. At the edge, AWS WAF adds an additional layer of protection by helping detect and block malicious traffic before it reaches your applications.
• Azure implementation
  In Azure environments, Azure API Management operates at the edge to handle external traffic, enforce policies, and provide centralized API governance. Internal service-to-service communication is typically managed by Open Service Mesh running on Azure Kubernetes Service (AKS), enabling secure and reliable traffic flow across applications. Azure Monitor centralizes logging and telemetry for improved operational visibility, while Azure Key Vault securely manages certificates, secrets, and encryption keys.
• Google Cloud Platform
  On Google Cloud, Apigee delivers comprehensive API management, acting as the primary interface for external consumers while supporting security, analytics, and lifecycle governance. Anthos Service Mesh manages internal communication, providing consistent policy enforcement, traffic control, and observability across services. Cloud Trace and Cloud Monitoring offer deep performance insights, and Cloud Armor strengthens security by helping defend applications against DDoS attacks and other external threats.

Egress and ingress patterns

• Egress Control - Service meshes manage outbound traffic by providing centralized control over how services communicate with external dependencies. Dedicated egress gateways route traffic through defined paths, improving security and simplifying policy enforcement. External service authentication helps verify trusted destinations, while traffic filtering and monitoring increase visibility into outbound requests. Circuit breakers further protect the environment by preventing cascading failures when external dependencies become slow or unavailable.
• Mesh Ingress Controller - Some teams rely on service mesh ingress to expose internal APIs, simplifying architecture by consolidating traffic management within the mesh itself. This approach promotes consistent policy enforcement across services while reducing the number of infrastructure components required to operate the environment. However, mesh ingress is typically designed for internal use cases and may offer fewer capabilities for external traffic management, making it less suitable for scenarios that require advanced edge features.
  

When to use an API gateway

Deploy an API gateway when your architecture requires secure, scalable access for external consumers and stronger control at the edge. Typical scenarios include:

• Public API exposure to external clients
• Developer experience with documentation and portals
• Monetization capabilities for API products
• Partner integration with SLA requirements
• Protocol translation for legacy systems
• External security enforcement

API gateways are especially effective at managing business logic where traffic enters the system, providing the capabilities organizations need to support API-as-a-product strategies and scale them with confidence.

When to use a service mesh

Adopt a service mesh when your architecture demands secure, reliable communication between services and greater operational visibility across distributed environments. Common indicators include:

• Zero-trust security between all services
• Advanced traffic management with circuit breakers
• Deep observability including distributed tracing
• Compliance with NIST SP 800-204A
• Multi-cluster communication across regions
• Chaos engineering capabilities

Service meshes excel at securing and observing internal traffic, delivering infrastructure-level capabilities that operate independently of application code. This allows teams to strengthen security, improve reliability, and scale operations without requiring service-level changes.

When to use both

Organizations running distributed microservices architectures often gain the most value from deploying both technologies together, creating a layered foundation for security, visibility, and traffic control. Key benefits include:

• Complete security coverage from edge to backend
• End-to-end observability across all traffic
• Clear separation of concerns between external and internal communication
• Best-of-breed capabilities from each technology
• Stronger alignment with regulatory and compliance requirements

Migration Considerations

Successfully adopting both an API gateway and a service mesh often requires a deliberate, phased approach that minimizes disruption while strengthening security and operational maturity.

• Phase 1: Deploy API gateway - Begin by establishing control at the edge. Implement security policies to protect external traffic, launch a developer portal to streamline onboarding, and configure rate limiting to stabilize consumption patterns. This foundation helps standardize access while preparing the environment for deeper architectural improvements.
• Phase 2: Add service mesh - Introduce the service mesh gradually, starting with observability-only mode to gain visibility into service interactions without impacting traffic flow. From there, enable mTLS in stages to strengthen service-to-service security, and implement traffic policies to improve reliability and control.
• Phase 3: Full integration - Once both layers are operational, focus on integration. Connect identity systems to enforce consistent authentication and authorization, unify telemetry for end-to-end visibility, and adopt advanced traffic and resilience patterns to optimize performance across the architecture.

Resource projections

Consider these factors based on implementation benchmarks:

• Sidecar memory consumption can vary widely from roughly 40 MB in baseline setups to over 1 GB in more complex environments
• Latency may increase by about 1.7 ms at the 90th percentile and up to 2.7 ms at the 99th percentile when two proxies sit in the data path ([Istioldie 1.8 / Performance and Scalability](https://istio.io/v1.8/docs/ops/deployment/performance-and-scalability/)Istioldie 1.8 / Performance and Scalability) ([Istioldie 1.14 / Performance and Scalability](https://istio.io/v1.14/docs/ops/deployment/performance-and-scalability/)Istioldie 1.14 / Performance and Scalability)
• Operational complexity tends to grow as environments scale
• Training needs differ based on the chosen architecture and tooling
• Licensing costs vary depending on the vendor and feature set

Kubernetes Gateway API

The [Kubernetes Service Mesh](https://konghq.com/blog/engineering/using-service-mesh-in-kubernetes-enviroment)Kubernetes Service Mesh Gateway API standardizes configuration across vendors, creating a more consistent approach to traffic management. It provides:

• Vendor-agnostic policies
• A unified resource model
• Multi-cluster support
• Progressive delivery capabilities

By establishing consistent configuration patterns, the API helps bridge traditional gateways and service meshes, simplifying operations across modern architectures.

Envoy as universal proxy

Envoy has seen widespread adoption in cloud-native environments. According to the Cloud Native Computing Foundation (CNCF), 54% of enterprises use Envoy as their proxy solution, underscoring its growing role in modern infrastructure ([Envoy Gateway is Emerging as the Enterprise Standard for Ingress](https://tetrate.io/blog/envoy-gateway-emerging-as-enterprise-standard)Envoy Gateway is Emerging as the Enterprise Standard for Ingress). It serves as the foundation for many API gateways and service meshes, offering a flexible and extensible data plane for traffic management.

• Powers many leading gateway and service mesh platforms
• The xDS protocol enables dynamic, real-time configuration
• WebAssembly (Wasm) extends functionality through custom filters
• An active open-source community continues to drive innovation

Sidecar-less architectures

Sidecar-less approaches such as ambient mesh and eBPF are gaining traction for their ability to reduce infrastructure overhead while simplifying operations. Research indicates that Istio’s ambient mode can lower allocated resource usage by up to 90%, making these models an increasingly efficient alternative to traditional sidecar deployments. The key benefits include:

• Simplified operations through node-level deployment
• Easier integration with legacy workloads without requiring pod modifications
• Lower total cost of ownership

Together, these advancements are making service mesh adoption more practical and accessible for organizations with smaller teams or limited platform resources.

AI and automation

AI is increasingly influencing both API gateways and service meshes, introducing smarter ways to manage security, traffic, and operations across distributed architectures. These capabilities help teams respond to changing conditions more quickly while reducing manual effort.

Emerging use cases include AI-powered threat detection at the edge and within service-to-service communication, automated policy generation to strengthen governance, and intelligent traffic routing that dynamically adjusts based on performance signals. Natural language configuration is lowering the barrier to managing complex environments, while self-healing systems can detect failures and trigger corrective actions, resulting in improved resilience across both gateway and mesh layers.

Security considerations

Defense in depth
A defense-in-depth strategy strengthens overall security by applying protective controls at multiple layers of the architecture. Organizations should secure both the perimeter and internal communication paths by combining edge protections with zero-trust principles, ensuring that every request is authenticated and authorized regardless of origin.

Regular security audits help identify gaps before they become risks, while automated vulnerability scanning enables teams to detect and remediate threats more efficiently. Establishing a well-defined incident response plan further ensures that organizations can act quickly and minimize impact when security events occur.

Certificate management
Industry guidance increasingly recommends using short-lived certificates to strengthen security and limit risk exposure. Rotating certificates every 90 days, especially when supported by automated renewal pipelines, helps reduce the impact of compromised credentials while maintaining uninterrupted operations. ([CI/CD Secrets for TLS certificate rotation that meet zero-downtime goals - UMA Technology](https://umatechnology.org/ci-cd-secrets-for-tls-certificate-rotation-that-meet-zero-downtime-goals/)CI/CD Secrets for TLS certificate rotation that meet zero-downtime goals - UMA Technology)

Organizations should also consider integrating hardware security modules (HSMs) where appropriate to safeguard cryptographic keys. Certificate transparency logging improves visibility into certificate issuance, while well-defined backup and recovery procedures ensure continuity during unexpected events. Maintaining comprehensive audit trails further supports compliance efforts and enhances overall security governance.

Performance optimization

Latency reduction

Minimizing latency is critical for maintaining responsive, high-performing applications across distributed environments. Techniques such as connection pooling help reduce the overhead of establishing new connections, while strategic caching limits repeated backend calls and accelerates response times.

Optimizing proxy configurations can further streamline traffic flow, and regional deployments place services closer to users to reduce network distance. Integrating a content delivery network (CDN) adds another performance layer by caching content at the edge, improving speed and delivering a more consistent user experience.

Resource efficiency

Right-sizing proxy resources based on actual usage patterns helps prevent overprovisioning while ensuring consistent performance under load. Horizontal autoscaling further strengthens resilience by allowing infrastructure to expand or contract in response to real-time demand.

Using efficient serialization formats such as Protocol Buffers can significantly reduce payload size and processing overhead, while compression improves transfer speeds for large responses. Connection multiplexing adds another layer of efficiency by enabling multiple requests to share a single connection, reducing latency and optimizing network utilization.

Operational excellence

Monitoring and alerting

Defining service level objectives (SLOs) and service level indicators (SLIs) gives teams clear targets for reliability and performance. Setting proactive thresholds helps identify potential issues before they affect users, while automated runbooks enable faster, more consistent responses when incidents occur. Proactive capacity planning ensures the infrastructure can support future demand, and continuously tracking costs helps organizations stay aligned with budget expectations as systems scale.

Deployment automation

Adopting GitOps for configuration management brings consistency and traceability to infrastructure changes by treating configuration as code. Automated testing pipelines help catch issues early, while progressive delivery enables teams to roll out updates gradually and reduce deployment risk. Built-in rollback capabilities allow for quick recovery if problems arise, and configuration validation ensures changes meet defined policies before reaching production.

Complexity management

Combining a service mesh with an API gateway can increase operational complexity, requiring teams to manage additional layers of infrastructure, policies, and tooling.

Solutions
Organizations can reduce this complexity by starting with one technology before expanding their architecture. Using managed services early on helps offload operational overhead, while investing in team training builds the expertise needed to support long-term success. Implementing changes gradually allows teams to adapt without disrupting existing workflows, and thorough documentation ensures clarity as the environment continues to evolve.

Performance overhead

Proxies introduce additional hops in the data path, which can increase latency and drive higher resource consumption across the environment.

Solutions
Teams can mitigate this impact by considering sidecar-less architectures in resource-constrained environments and optimizing proxy configurations to align with specific workload requirements. Strategic caching at the right layers can further reduce unnecessary traffic, while continuous performance monitoring helps identify bottlenecks early. Right-sizing resources based on actual usage ensures infrastructure remains efficient without sacrificing reliability.

Debugging distributed systems

As architectures grow and more components are introduced, troubleshooting can become increasingly complex. Teams can improve visibility by implementing distributed tracing to follow requests across services and centralizing logging to create a single source of operational insight. Service maps help visualize dependencies and identify failure points, while correlation IDs make it easier to connect events across systems. Building dedicated debugging dashboards further streamlines investigation, enabling faster and more efficient issue resolution.

The service mesh versus API gateway discussion often misses the bigger picture: these technologies are designed to work together, not compete. API gateways protect and manage the front door, handling external access and policy enforcement, while service meshes secure the internal pathways that connect your services. When used in tandem, they create a layered architecture that delivers stronger security, deeper observability, and more consistent traffic control across the entire environment.

Key takeaways:

• API gateways are purpose-built for managing external traffic and enforcing policies at the edge
• Service meshes focus on securing and controlling internal service-to-service communication
• In production environments, the two technologies often deliver the most value when deployed together
• Emerging standards such as the Kubernetes Gateway API are helping drive greater convergence across these layers
• Modern architectures benefit from adopting both to achieve more complete security, visibility, and traffic management

The microservices architecture market continues to expand rapidly, with projections estimating it will reach $13.1 billion by 2033, growing at a CAGR of 12.7%. This growth is fueled by rising demand for scalability, ongoing digital transformation efforts, the expansion of e-commerce, and continuous technological innovation, according to [Microservices Architecture Market Share, Size 2025-2033](https://www.imarcgroup.com/microservices-architecture-market)Microservices Architecture Market Share, Size 2025-2033. As architectures become more distributed, organizations increasingly rely on both API gateways and service meshes to address distinct security, traffic management, and operational requirements.

Start by defining clear requirements and identifying your most pressing pain points. From there, adopt these technologies incrementally to minimize disruption and allow teams to build operational maturity over time. Modern distributed systems often benefit from both an API gateway and a service mesh, but the right implementation should always align with your architecture, scale, and business goals.

Looking ahead, the lines between these technologies will continue to blur as deeper integration becomes the norm. Unified control planes, shared data planes, and standardized configuration models are already simplifying adoption and day-to-day management. While gateways and meshes will continue to serve distinct roles, the experience of operating them together is expected to become far more seamless.

Next Steps

Ready to implement these patterns? Follow the steps below to guide your approach:

• Assess your architecture — Identify north-south and east-west traffic patterns to understand where each technology delivers the most value.
• Start with the gateway — Secure the perimeter first by establishing strong edge controls.
• Add mesh capabilities — Begin with observability to gain visibility, then introduce security features, such as mTLS.
• Integrate gradually — Connect identity and telemetry systems to create a more unified operational model.
• Optimize continuously — Monitor performance and costs to ensure the architecture remains efficient as it scales.

Whether you're building new microservices or modernizing existing systems, knowing when and how to use API gateways and service meshes is central to long-term success. Together, these technologies provide the foundation for secure, scalable, and highly observable distributed architectures.

References

• NIST Special Publication 800-204A, Building Secure Microservices-based Applications Using Service-Mesh Architecture. National Institute of Standards and Technology. May 2020.
• OWASP API Security Top 10 2023. Open Web Application Security Project Foundation. 2023.
• Istio Performance and Scalability Benchmarks. Istio Documentation. 2024.
• IMARC Group Microservices Architecture Market Report. IMARC Group. 2024.
• Cloud Native Computing Foundation Survey. CNCF. 2024.