See what makes Kong the fastest, most-adopted API gateway
Check out the latest Kong feature releases and updates
Single platform for SaaS end-to-end connectivity
Enterprise service mesh based on Kuma and Envoy
Collaborative API design platform
API and Microservices Security for Gateways, Service Mesh, and Beyond
Call for speakers & sponsors, Kong API Summit 2023!
5 MIN READ
This post highlights how you could use Kong Gateway to implement a solution for the Australian Consumer Data Standards (CDS), which is part of the Consumer Data Right legislation introduced by the Australian Government in November 2017.
As detailed on the Australian ACCC website:
CDR will give consumers greater access to and control over their data and will improve consumers’ ability to compare and switch between products and services. It will encourage competition between service providers, leading not only to better prices for customers but also more innovative products and services. CDR will first apply to the banking sector, followed by the energy sector. The telecommunications sector is currently proposed to follow.
Kong Gateway covers many of the CDS requirements using out-of-the-box features in conjunction with using Kong plugins, such as the OpenID Connect (OIDC) and CORS.
This blog post will focus on specific features provided by Kong to support CDS and CDR, including CDS Versioning and CDS Traffic Thresholds. These implementation details target the initial rollout of CDS to the banking sector. Some of Kong’s banking customers in Australia are currently using these CDS artifacts in production. We can also utilize this solution to support telecommunications and energy sectors in the future.
Two major CDS requirements that are the focus of this blog post are Traffic Thresholds and Versioning, as detailed in the Standards and Non-functional Requirements sections of the Consumer Data Standards.
Calls above the CDS Traffic Thresholds will be freely throttled or rejected. Limits are based on the number of individual sessions per day, transactions per second (TPS) and calls for a specified duration. These limits are applied independently for different consumer types, as explained below.
Data Recipient is a configured application that is presented in the register metadata. This acknowledges that a single accredited entity may register multiple independent services (or apps) that can obtain authorizations from consumers independently of each other.
A Session is defined as the lifespan of a unique Access Token. Multiple API requests made with a single valid Access Token would be considered part of a single Session.
Customer Present are authenticated API requests made in direct response to interactions by the end customer using the digital services of the data recipient. Technically, a data holder will define an API request as “Customer Present” if the “x-fapi-customer-ip-address” header is populated with a valid IP address of the end customer’s device.
Customer Not Present is authenticated API requests that are not deemed to be “Customer Present.”
Unattended is a synonym of “Customer Not Present.”
Kong’s CDS Traffic Thresholds plugin extends the enterprise functionality of the Advanced Rate Limiting plugin to cater to the CDS Traffic Threshold requirements. This plugin implements the sliding window algorithm for high accuracy rate limits. See this blog post for more information on the benefits of the sliding window algorithm.
The high-level logic for determining both rate and session limits is detailed below. Note that for security, the plugin expects HTTP headers to be passed in as internal variables using kong.ctx.shared, and in the case of the session ID, hashed so that it never appears as clear text.
The following table displays the available plugin schema properties applicable to the CDS Session and rate limits described above:
CDS Traffic Thresholds defines the following limits:
Customer present and authorization traffic thresholds:
Unattended traffic thresholds will apply for low traffic periods:
For Unattended traffic during high-traffic periods, only best-effort support is required.
Secure traffic (both Customer Present and Unattended) thresholds:
Public traffic (i.e., traffic to unauthenticated endpoints) thresholds:
Here’s how easy it is to apply the above limits to the plugin:
curl -s http://localhost:8001/services/example-service/plugins/ \ -d name=cds-traffic-thresholds \ -d config.max_attended_customer_tps=10 \ -d config.max_data_recipient_tps=50 \ -d config.max_unattended_sessions_low=100 \ -d config.max_unattended_per_session_low=20 \ -d config.max_unattended_session_tps_low=5 \ -d config.max_private_tps=300 \ -d config.max_public_tps=300 \ -d config.sync_rate=0 \ -d config.redis.host=kong-ee-redis \ -d config.redis.port=6379 \
The CDS has adopted a two-level versioning strategy. The high-level standards, including principles, Uniform Resource Identifier (URI) structure, endpoint versioning, payload naming conventions, etc., can be versioned. Each API endpoint will have an additional version specific to that endpoint. For endpoint versioning, each endpoint will have multiple versions independent of other endpoints. A consumer will request a specific endpoint using an HTTP header:
x-v (Mandatory)
x-min-v (Optional)
The plugin works similarly to the Kong Route By Header plugin in that it overrides the normal upstream selection logic based on request HTTP headers.
The CDS requirement is to support multiple major and minor versions. In facilitate request routing, major (x-v) and minor (x-min-v) version header values will determine the exact application route based on a logic, such as, if x-min-v > x-v is an error case, while invalid x-v and valid x-min-v responds with resource supporting x-min-v. This logic is built into the CDS versioning plugin.
The following table displays the available plugin schema properties applicable to the CDS Versioning requirements described above:
This curl command shows how straightforward it is to configure the plugin:
curl -i http://localhost:8001/routes/example-v1/plugins \ -H 'Content-Type: application/json' \ -d '{"name": "cds-versioning", "config": {"versions":[{"major": 1, "minor": 3, "upstream_name": "myapi"}]}}'
The specific requirements for Traffic Thresholds and Versioning are complex. Traffic Thresholds, in particular, require two types of limits for request rate limiting, tracking daily sessions and logic for the various request classifications. Using the Kong CDS plugins hides this complexity, and together with Kong Gateway Enterprise’s features, provides a real-world solution to the Australian Consumer Data Standards.
If you are an existing customer and want more information on the CDS plugins, contact your Kong customer experience manager. If you’re not a customer, then reach out to your local Kong Sales Team.
Share Post