See what makes Kong the fastest, most-adopted API gateway
Check out the latest Kong feature releases and updates
Single platform for SaaS end-to-end connectivity
Enterprise service mesh based on Kuma and Envoy
Collaborative API design platform
How to Scale High-Performance APIs and Microservices
Call for speakers & sponsors, Kong API Summit 2023!
9 MIN READ
As more and more companies move to a multi-cloud strategy and increase usage of a cloud native infrastructure, API providers are under a lot of pressure to deliver APIs at scale in multi-cloud environments. At the same time, APIs should follow each company’s security requirements and best practices, no matter the cloud platform. These reasons explain why many providers have such complex API authorization requirements.
Let’s assume in a company that multiple teams from different lines of business are building and deploying APIs in the Azure cloud. Different teams use different technologies to build these APIs (e.g., Azure Functions, Node.js). A company might host applications that consume the APIs on the same network, such as a company’s AWS account or external SaaS applications.
Now let’s take a closer look at the challenges API providers face when deploying and running these APIs.
As mentioned earlier, multiple teams use different technologies to build and deploy APIs in a multi-cloud environment. We need a way to validate the APIs and make sure they follow best practices. Some companies leave this responsibility to individual teams and expect them to come up with these best practices. This approach normally leads to chaos. Teams either follow different standards and best practices or completely ignore them to speed up the delivery process. Another approach is a Center of Excellence (COE) to review all the APIs and ensure they follow best practices. This approach is not scalable; it might work for delivering a few APIs, but the COE team becomes a bottleneck when dealing with thousands of APIs because it takes a long time to review them.
Companies are looking to automate this as part of their CI/CD process and automatically validate the APIs. Let’s assume every API should be secured using API key authentication and authorization. This means we need a way to ensure developers have applied the right security policies on top of APIs before deploying them.
Enforcing authorization can get complex in multi-cloud environments, and it can cause some serious challenges for the API providers. Let’s assume an API provider wants to control applications, users and the IP ranges allowed to call their APIs—a typical use case.
One of the common approaches for storing authorization data is to store it in the IDP and use JWT tokens to pass it to the backend APIs. This approach is not a good fit for multi-cloud environments due to the distributed nature of these environments and the likelihood of using different authorization technologies. For example, users coming from AWS might use different IDP vs. Azure vs. SaaS users.
Replicating the user’s authorization data across many IDPs and keeping them in sync is complex and inefficient. There might also be some restrictions on the data which API providers can store in each IDP. For example, in our case, the AWS IDP admin is questioning why they need to store authorization data for Azure APIs.
To solve this problem, we can use declarative configuration with APIOps, Kong Gateway and OPA. Before I dive into how these solutions work together, I’ll cover some basics. If you’re already familiar, you may skip ahead.
APIOps brings DevOps and GitOps principles to the API and microservices lifecycle to automate these stages as much as possible. You can achieve a big part of this automation using declarative configuration. If you have been using Kubernetes or DevOps, you are familiar with declarative configuration concepts. APIOps uses declarative configuration to automate most of the API lifecycle.
OPA is an open source policy engine that lets users define policy as code. OPA uses Rego language to define these policies.
You can use OPA to create policies as code and enforce them across different applications in your organization. You can create OPA policies that can answer questions such as:
The declarative nature of OPA makes it a great fit for APIOps. You can create these policies, store them in your code repository and use them through different stages of the API lifecycle. For more information about OPA, please refer to the OPA website.
You can integrate OPA with Kong Gateway. Let’s look at the interactions between the API consumer, Kong Gateway, OPA server and a backend API.
An important part of OPA policy architecture is decoupling the data from the policy definition. In this scenario, there are two ways for the policy to access the information it needs to decide:
There are other approaches for injecting external data into OPA policy. For more information, please refer to the OPA website.
One of the great things about OPA is that it can integrate with most technologies as long as they generate and consume JSON. Most CI/CD tools these days have either out-of-the-box libraries or third-party add-ons to support JSON. During deployment, we can use OPA to validate the:
To implement the declarative authorization as part of the continuous deployment:
You can implement this solution by following these steps:
In the next sections, we discuss the implementation and walk you through the details. We are using GitHub Actions to automate the API deployment. You can adjust these scripts based on the CI/CD tool you are using.
The first part of the solution ensures developers have included the right policies on top of APIs. This means Key Authentication and OPA plugins should exist in the API specification.
API developers can use “x-kong-plugin-” prefix to add the required plugins to their APIs. The OAS spec for the Customer API looks like this:
openapi: "3.0.0" info: version: 1.0.1 title: customer api license: name: Kong servers: - url: ${UPSTREAM_SERVER_URL}$ description: Upstream server URL. # Key Auth plugin enables authentication for this service x-kong-plugin-key-auth: enabled: true config: key_names: - apikey x-kong-plugin-opa: enabled: true config: opa_host: ${OPA_HOST}$ opa_port: ${OPA_PORT}$ opa_path: /v1/data/userAuthz/allowUser include_consumer_in_opa_input: true include_route_in_opa_input: true include_service_in_opa_input: true
We can simply create an OPA policy to look for these plugins in the API specification.
package apiCDAuthz default allow = false allow { input["x-kong-plugin-key-auth"].enabled == true input["x-kong-plugin-opa"].enabled == true }
As you can see, this policy looks for these two plugins in the OAS specification (e.g., input) and checks if they are enabled. It returns true if both policies exist and are enabled; otherwise, it returns false.
You can create this policy by calling OPA’s create policy API:
curl -XPUT http://<opa-server:port>/v1/policies/apiCDAuthz --data-binary @./opa/apiCDAuthz.rego
The next step is to send the OAS specification to the OPA policy and check the result. In this example, we wrote our OpenAPI Specification in YAML, and OPA can only process JSON. The first thing we need to do is to export the customer specification and transform it to JSON, as shown in this GitHub Action workflow:
# Export customer yaml - name: Export customer yaml run: inso export spec customer-opa --output customer.yaml # Transform OAS to JSON - uses: fabasoad/yaml-json-xml-converter-action@main id: yaml2json with: path: 'customer.yaml' from: 'yaml' to: 'json' - name: Prepare OPA input run: | cat <<EOF > ./v1-data-input.json { "input": ${{ steps.yaml2json.outputs.data }} } EOF
And then, we can call the OPA policy and validate the JSON data.
# Check deployment authorisation - name: Call OPA to Check Deployment Authorisation id: call-opa run: | result=$(curl ${{ secrets.OPA_HOST }}/v1/data/apiCDAuthz -d @./v1-data-input.json \ -H 'Content-Type: application/json' | jq '.result.allow') echo "::set-output name=result::$result" - name: Should I continue? run: | if ${{ steps.call-opa.outputs.result }}; then echo "Call OPA Result: ${{ steps.call-opa.outputs.result }}" else echo "Call OPA Result: ${{ steps.call-opa.outputs.result }}" exit 1 fi
If the result is false (i.e., the API spec does not conform to the configured best practice), the deployment process will automatically fail, and the system will notify the developer.
This is a very simple example to demonstrate the role which OPA can play in the deployment process. You can extend it to:
As discussed earlier, API providers want to expose their APIs to:
At the same time, they want to control the applications, users and the networks calling their APIs. For example, an API provider might only allow users to call GET customers using the CRM application hosted in the company’s AWS account.
We are confident that the API developer has already configured the OPA plugin for this API at runtime. Otherwise, it would fail during the deployment. Below is the configuration for the OPA plugin:
x-kong-plugin-opa: enabled: true config: opa_host: ${OPA_HOST}$ opa_port: ${OPA_PORT}$ opa_path: /v1/data/userAuthz/allowUser include_consumer_in_opa_input: true include_route_in_opa_input: true include_service_in_opa_input: true
The OPA plugin sends the request, consumer, service and route information to the userAuthz policy in the OPA server. Adding service, consumer and route information on top of the current request helps us implement a more granular access level. For example, the policy can restrict the users to only certain routes.
Now let’s take a look at the userAuthz policy.
package userAuthz default allowUser = false allowUser = response { # Validate JWT token v := input.request.http.headers.authorization startswith(v, "Bearer ") t := substring(v, count("Bearer "), -1) io.jwt.verify_hs256(t, data.userAuthz.jwt_signiture) [_, payload, _] := io.jwt.decode(t) # Validate consumer app kong_consumer := input.consumer.username some j data.userAuthz.apps[j] == kong_consumer # Validate network net.cidr_contains(data.userAuthz.networkcidr, input.client_ip) # Validate user's access to api role := data.userAuthz.users[payload.username].role serviceName := input.service.name access := data.userAuthz.role_service_access[role][serviceName].access tier := data.userAuthz.role_service_access[role][serviceName].tier some i access[i] == input.request.http.method response := { "allow": true, "headers": { "x-user-tier": tier, }, } }
This policy:
As you can see, this policy uses data variables. This data is injected into this policy using the OPA API.
curl -XPUT http://<opa-server:port>/v1/data/userAuthz --data-binary @./opa/userAuthz.json
If there is a change in data, we can simply update the JSON file and call the OPA API again.
{ "jwt_signiture": "46546B41BD5F462719C6D6118E673A2389", "networkcidr": "178.10.0.0/24", "apps": ["insomnia"], "users": { "john.smith@myco.com": { "role": "admin" } }, "role_service_access": { "admin": { "customer_api": { "access": ["GET"], "tier": "Gold" }, "employee_api": { "access": ["GET","POST"], "tier": "Silver" } } } }
By using this policy and the data combined, the API provider only allows the requests with the following attributes to call GET customers API:
Another beneficial feature of OPA policies is the ability to return useful information to Kong Gateway. This case returns the user’s access tier (i.e., Gold) for each API. Kong Gateway can use this information to do things like tier-based rate limiting.
Using OPA and declarative policies is becoming very popular, especially in APIOps. That’s because they’re:
Building and managing APIs in a multi-cloud environment can be difficult, especially regarding security and authorization. Using policy-as-code with APIOps is a great way to build an automated and scalable solution. Try it out and see for yourself!
Share Post
Learn how to make your API strategy a competitive advantage.