How and Why to Migrate From Kong Open Source to Kong Enterprise API Gateway

Kong’s open-source API Gateway is a highly successful project with over 33k stars on GitHub and 293 contributors. Kong provides a powerful platform that can be extended using hundreds of plugins provided by Kong, ecosystem partners, and the community. Kong’s Enterprise subscription provides an extended set of capabilities over and above the core foundation of Kong OSS. We will explore how to migrate an installed system from OSS to Enterprise here today.

Why Kong Gateway Enterprise?

There are quite a few features available in Kong Enterprise that extend the basic capabilities of the Kong OSS project to provide a Production grade API Management platform for suitable use. Let's review some of these capabilities.

1. Kong Manager exposes a graphical user interface for configuration and operational monitoring and is a foundational platform to expose more Enterprise features.
2. Secrets Management lets us store sensitive information such as passwords, certificates and API keys in a vault. Supported vaults include AWS Secrets Manager, GCP Secrets Manager, and HashiCorp Vault.
3. Kong Developer Portal provides a central location for developers to discover published API specifications, try them out, view example code snippets in common languages such as javascript, python, or ruby and subscribe to APIs.
4. Kong Vitals exposes access to monitoring and analytics of services, routes, and application usage directly within Kong Manager.
5. Kong Workspaces and Role-Based Access Control (RBAC) creates a user and role structure to allow multiple teams to share an instance of Kong Gateway with separation of concerns (I.e., multi-tenancy). Kong Enterprise can be integrated with an existing Identity Management System to allow roles and rights to be driven by the existing identity and entitlement workflows.

The features listed above really transform Kong from “hey, neat I can forward API requests and insert some basic security features and controls” into “wow, I have a production-ready API gateway with built-in visibility, monitoring, and administrative features that is ready for Enterprise use.” Beyond that, however, is a key differentiator of the Enterprise version of Kong, Enterprise Plugins. Kong Enterprise provides 29 additional plugins that provide production grade security, transformation, traffic control and observability capabilities. For some examples...

Let’s say a developer wants to provide basic Mocking for APIs in development, aligned with the Open API Specification. The Kong Mocking plugin adds this capability to the gateway under the Enterprise license and is supported by Kong. https://docs.konghq.com/hub/kong-inc/mocking/

Or if an organization would like to enforce authentication using OpenID Connect (OIDC) using a third party Identity Provider. Kong’s OpenID Connect plugin is another Enterprise licensed plugin that is supported by Kong. https://docs.konghq.com/hub/kong-inc/openid-connect/

See the full list of plugins at Plugin Hub and check out the ones tagged with “Enterprise” on their tile that apply to your API Gateway use cases: https://docs.konghq.com/hub/

Finally, there is the benefit of access to Kong Enterprise Support and Services that comes with licensing Kong Enterprise. While community collaboration helps drive the direction of the foundational code base of the project, establishing a partnership with Kong will ensure success, maximize uptime and usage of the platform to get the highest return on using Kong.

A full feature matrix of the delta between OSS and Enterprise versions of Kong is available at the following link: https://docs.konghq.com/gateway/latest/#features.

Migration Scenarios

Kong is a highly flexible product that can be deployed in multiple ways. The base Kong executable includes both control plane and data plane capabilities which can be enabled or disabled based on the specific configuration. Deployment architecture options from the Traditional mode to a highly scalable Hybrid approach are described in the documentation at this link:

https://docs.konghq.com/gateway/latest/production/deployment-topologies/

_NOTE: The Konnect implementation of Kong offloads the access, support and ongoing maintenance of the Kong Gateway control plane and configuration store to a multi-tenant Kong-supported public cloud-based SaaS implementation. For this article, we will focus on migrating to Kong Enterprise, self-managed. In a future article we will cover how to migrate from Kong Gateway OSS to Konnect. _

Fundamental across the migration scenarios are the common elements of the Kong executable itself, and a store for configuration data. The store of data may be one of the following:

• Contained in a statically defined declarative yaml file (referred to as a db-less install), which is loaded into memory at the time Kong starts. The file format for Enterprise includes additional options such as Workspaces not supported in the OSS version. This approach however does not manage other Enterprise-only entities such as admins, RBAC permissions/roles, or anything related to Dev Portal.
• Derived from an external source of authoritative state such as the etcd key-value store in the Kong Kubernetes Ingress Controller (KIC) implementation (which can run with or without a database). The Ingress Controller container within the KIC pod reads the relevant objects through the K8s API server and translates that into the desired state for routes, services, and plugins within Kong.
• Stored in a PostgreSQL database. This approach is used in the Traditional and Hybrid Kong architectures. To support Enterprise, the database structure must be updated and extended to support the additional capabilities offered by Enterprise features and plugins. The “kong migrations” command set is used to update, and then commit the state of the updates to PostgreSQL.

The overall Kong migration process boils down to both updating the running version of the Kong executable and ensuring the data store format is updated to match additional features available in the Enterprise version of Kong. The declarative file format update needed would primarily be to add the “_workspace” value to the file, if using a Workspace name other than default. The Kubernetes Ingress Controller update steps would be based on the tooling used for the original K8s installation, either a kubectl apply based install, or an update using Helm to manage the process.

When using a database, performing the migration of the changes to the database is automated via the “kong migrations” command set, covered in the final section of this article. In this blog, we will be focusing on this migration pattern.

Installing, then Migrating a Kong OSS Installation to Enterprise

The best way to understand and see how easy the migration process is, execute the steps yourself. This can even be done without having an Enterprise license of Kong. Simply load up a Linux host with a basic Kong OSS installation and a PostgreSQL database. Create an example service/route configuration. Then remove the OSS Kong executable, and replace it with an Enterprise version executable of Kong. Follow the instructions to migrate the database and start up Kong. An Enterprise license can be applied after the fact using an API call, but it is not needed for Kong Enterprise to start. Let’s walk through the steps to do that now. We will provide just the commands necessary to build up the OSS setup first, followed by full examples with response output for the migration process itself.

NOTE: These steps are provided only as a minimal functional example of a Kong OSS to Enterprise migration using database migration tools. They are not intended to be general applicable as-is to all topologies and deployment modes possible with Kong API Gateway. For a production environment, it is recommended to simulate the process using a Kong topology in a lab setup that is built like what you are running first to understand the commands and possible approaches that could work best for your specific needs.

OSS Install

My test environment consists of an AWS instance running Ubuntu focal-20.04. Some of the syntax will be specific to the version and release of Linux. Here is the sequence of commands used to install and start up Kong.

NOTE Open source community Kong Gateway official install instructions are located here:

https://konghq.com/install#kong-community.

Prep the Ubuntu OS:

I like running current code when if possible:

Run this to resolve a Kong dependency in newer versions of Ubuntu:

Install jq utility to help with viewing json responses later:

Install and prep PostgreSQL:

Change to postgres user to prep the db:

Launch postgres interactive prompt:

You should be back at ubuntu command prompt now:

Download and Install Kong:

Edit Kong Configuration File:

Update these fields in the DATASTORE section of the file to uncomment lines and look like the following, starting around line 1097:

Run initial “kong migrations” database command, and start Kong:

Create an example service and route:

Test the route:

Ensure you can see it in the gateway configuration, and also that you see a “200 OK” header response and output when accessing the path for the route:

This displays route config.

This tests actual route request.

At this point you should have a complete running instance of Kong OSS Gateway, with an example route and service. This creates a baseline environment to ensure that once a migration to Enterprise is complete, we still have a functional route.

Enterprise Migration

To migrate this installation to Enterprise, we will first shut down and remove the existing OSS mode Kong, leaving the configuration saved in the database. Then we will install the Enterprise version of Kong and use the “kong migrations” command set to update the PostgreSQL database to support Enterprise. There are many variations of this process, depending on your installed architecture of Kong which can involve building a new virtual host to run the new release, or even cloning the existing running database first, and using the cloned copy to build the Enterprise installation (ie. blue/green deployment). These options can allow a gradual transition. The approach here does an in-place upgrade which if taken requires some downtime while the migration takes place and therefore should be planned accordingly.

Note: Any upgrade between numbered Kong release versions should be taken care of separately from migrating from OSS mode to Enterprise mode. Migration should be done laterally from the same release version number of Kong OSS to Kong Enterprise. There are specific requirements for migrating to the Kong 3.x release which is current as of this article. For overall details on the Kong upgrade process see the link in the documentation. https://docs.konghq.com/gateway/latest/upgrade/

Stop and uninstall Kong OSS:

Download and install Kong Enterprise:

NOTE: Enterprise Kong Gateway official install instructions are located here:

​​https://docs.konghq.com/gateway/latest/install/linux/ubuntu/

Run database migrations:

NOTE: To consider options for staging a migration in a production environment for the various deployment topologies supported by Kong, see the following documentation:

https://docs.konghq.com/gateway/latest/production/deployment-topologies/

This command will update the database in a non-destructive way. Meaning that if this were a larger production system in traditional or hybrid topologies, it would allow the OSS version to continue operating, leaving some of the updates pending. It will produce many lines of output, with various updates being applied. At the end you should see totals for the updates, and a warning that the updates are pending:

In our example, we have already upgraded our single node Kong. For a production environment, once all Kong OSS nodes have been updated to Kong Enterprise, we can apply the command to complete the database migration process.

The last line of this output shows no pending migrations. Kong Enterprise could have been started and run against the database after just the “migrations up” command, but the “migrations finish” command is needed to support full functionality of the new capabilities.

Run Kong Enterprise:

Now we can repeat the test commands that we used against the OSS mode installation to ensure everything is working. Here is the expected result, up and running on Kong Enterprise 3.0.

Conclusion

As we have shown here, migration from Kong OSS to Kong Enterprise is easily and safely achievable while keeping your configuration and gaining the benefits of the Kong Enterprise Gateway platform. Kong Enterprise brings great new administrative and operational features, as well as access to the full set of Enterprise plugins as well as industry leading support and services. Try this exercise out on a simple virtual machine today or contact Kong to discuss licensing or getting help from Kong professional services to plan your migration.