Owning Infrastructure vs Owning Policies: Balancing Engineering Autonomy and Platform Control of APIs
How to empower engineers and increase velocity without shadow IT with fast and secure API infrastructure strategy
We want our engineering teams to move fast and be agile, yet many organizations confuse ownership of the infrastructure with ownership of the policies. The latter is what engineering teams want to rapidly iterate and develop their APIs. The former is the responsibility of the platform team.
In this post, we'll talk about how to empower engineers with a secure API infrastructure strategy. This is part of a series on becoming a secure API-first company. For a deeper dive, check out the eBook Best Practices for Becoming a Secure API-First Company.
What the engineering team should and shouldn't own
The engineering team’s responsibility is to build captivating products and user experiences while being able to iterate fast on API policies and ship them out quickly. Typically, they need a subset of capabilities that a modern API infrastructure can provide:
- Traffic routing configuration to new versions of the services and APIs, including generic split traffic, traffic mirroring, blue/green deployments and canary releases, and traffic introspection and request injection
- Feature flagging to route across different versions of an API
- Access to observability metrics and logs and traces as well as other traffic capabilities such as retries, timeouts, and circuit-breaking
On the other hand, there are some components that engineering teams shouldn’t own:
- Overall deployment of the infrastructure
- Ops of the infrastructure, including no-downtime upgrades
- Security and encryption configuration of the traffic
- AuthN/Z, which they can influence, but it should comply with the broader security stance of the organization
- Logging and debugging infrastructure, including tracing infrastructure (they should have access but not be running these capabilities)
- Firewall rules
- Cross-cloud and cross-datacenter connectivity
As the buyers of API infrastructure technology, it’s important to assess that the technology allows for both infrastructure and configuration segregation to simplify the deployments across the organization. This ensures simplified deployments across the organization with a unified control plane, while still compartmentalizing the data plane infrastructure and the applied configurations:
Without a solid strategy in place, engineering teams will go ahead and develop their own customized solutions for managing API infrastructure. In doing this, teams will unknowingly promote shadow IT bad practices, a lack of control and scale, inefficiencies across the organization, and an increased risk for security vulnerabilities. It’s a bad move.
Obviously, we can all agree we don’t want our teams to be slowed down by inefficiencies and bottlenecks. But it doesn’t have to be this way. The solution? The platform team gives engineering teams some autonomy to apply policies while still managing the underlying infrastructure on their behalf (at the API management and service mesh layers). This requires planning. And sometimes it may seem easier to just delegate the whole setup to the engineering teams, but that will inevitably lead to catastrophe.
It’s our corporate responsibility to fully own the API infrastructure without delegating to our people a massive area of responsibility that they won’t be able to properly address in conjunction with their other tasks.
As our applications evolve from monolithic to microservices, networking requirements become more critical for our applications. We’re essentially replacing the reliability of the CPU in monolithic applications with the unreliability and security issues of the network in microservices. Of course, this is in exchange for better scalability, resilience, and agility.
The evolution of our applications to microservices has created more API traffic than ever before, at an unprecedented scale.
Kong provides both infrastructure and configuration segregation for hundreds of top Fortune 500 and Global 2000 organizations that have deployed a single pane of glass to manage API infrastructure across every team, while still allowing developers to be quick and agile in their rapid iterations.
Want to see how Kong can help your organization balance agility and security? Get a demo today.