Use ChatGPT to Develop a SOAP/XML Custom Kong Konnect Plugin

Today, APIs are based on modern communication patterns: REST, GraphQL, or gRPC. But two decades ago, the majority of Web Services were developed with SOAP/XML.

In this blog, we’ll explain how Kong Konnect can manage SOAP/XML Web Services by creating custom plugins and by using ChatGPT. We’ll cover using ChatGPT to develop a Lua custom plugin and how to deploy and test a SOAP/XML custom plugin on Kong Konnect and Kong Enterprise.

Why manage SOAP/XML Web Services in Kong Konnect?

As we’ve evolved from monolith applications to microservices on Kubernetes and moved to the cloud, new APIs are developed with the latest framework, which is mainly REST.

But there are many types of APIs. Before the massive usage of REST APIs, organizations developed their Web Service by using SOAP (for Simple Object Access Protocol) and XML (for Extensible Markup Language). While many have moved on from SOAP and XML, due to legacy, a large number of organizations still have many SOAP Web Services that manage critical missions.

At Kong, we believe you need a central point for managing, governing, and securing modern APIs and historical Web Services too. And you should have all capabilities in the same place, which is Kong Konnect. The purpose of these plugins is to help achieve this goal.

Benefits of SOAP/XML custom plugins

The governance of SOAP Web Services relies on several key aspects, including:

• Validating the request and response to prevent corrupted calls and enhance security.
• Transforming the request and response of the Web Service through mediation, catering to application clients that expect different results.
• Dynamically rerouting the Web Service, enabling a flexible and adaptable architecture.

XML plays a crucial role in providing technical solutions for that governance in the following ways:

• Validity of a SOAP/XML message is ensured by checking it against an XML Schema Definition (XSD) schema. This helps verify the structure and integrity of the message.
• XML documents can be transformed into different XML documents using Extensible Stylesheet Language Transformations (XSLT). This allows for flexible data transformations and formatting.
• Routing from one Web Service to another can be achieved by evaluating the value provided by an XPath request in the XML document. XPath utilizes path expressions to select specific nodes or node-sets in an XML document, enabling seamless routing between services.

Basically, we want two custom plugins managing the Request and the Response, as we see on the schema below. We include an XSLT transformation, before and after, the XSD validation for greater flexibility.

How to use ChatGPT to develop plugins

Why ChatGPT? To address the challenge of XSD validation, XPath routing, and XSLT transformation, we have found a perfect solution in the GNOME C libraries: libxml2 and libxslt. These libraries have been developed 20 years ago and are still actively maintained, making them highly popular and robust choices.

The GNOME C libraries are already included in the kong/kong-gateway Docker image, so you don't need to rebuild a Kong Gateway image.

Many functions are already bound in the XMLua/libxml2 library. These capabilities are included in Kong Enterprise (but not in Kong OSS) are great.

The challenge is that these libraries have about 2,000 functions and definitions, and they’re not exactly well documented. So the question is: Which function do we use for XSD, XSLT, and XPath?

This is where ChatGPT provides a quick and efficient answer.

It’s easy to tell ChatGPT: “Give me an example of Lua script that uses the libxml2 library to validate an XML document against an XSD schema”

ChatGPT works its magic, and we discover the required functions of the GNOME C / XMLua/libxml2 libraries to use among the 2,000.

See below for the Lua code managing XSD validation given by ChatGPT:

By refining the request, you’re able to improve the error management code. Tell ChatGPT: “Improve the error management”.

We’ve now enhanced this Lua code with better error management:

Finally, ask ChatGPT to: “Include the XSD validation of the XML document on the body request in a Kong plugin”

ChatGPT provides an awesome starting point to develop our Kong custom plugin in Lua. But as a developer, our job is not yet finished. ChatGPT has done a great deal, but we have to keep working to improve the code in terms of robustness and reliability.

See this GitHub repository for the finalized version of the custom plugins.

Deploy the custom plugins on Konnect

There are two steps to achieve this: deploy the custom plugins on the Konnect Control Plane and on the Kong Konnect Data Plane (i.e., Kong Gateway).

Every custom plugin consists of two mandatory modules:

• schema.lua: holds the configuration entered by the user (like XSD schema and XSLT definition) => deployed on Konnect Control Plane and Data Plane
• handler.lua: the core of the plugin => deployed on the Konnect Data Plane

See here for more information.

In our case, we’ll first copy the custom plugins locally. Do a Git Clone of the following projects containing the different SOAP/XML Custom Plugins developed in Lua.

git clone https://github.com/jeromeguillaume/kong-plugin-soap-xml-handling.git

Deploy the plugins on Konnect Control Plane

Deploying a custom plugin on Konnect is pretty simple: send the schema.lua to kong@support.com and, a couple of hours later, your custom plugin will be available in the Konnect Control Plane. This process will be improved in a few months, and you’ll be able to do it by yourself from the Konnect Control Plane UI.

Note:

Two schema.lua located under:

• ./kong-plugin-soap-xml-handling/kong/plugins/soap-xml-request-handling/schema.lua
• ./kong-plugin-soap-xml-handling/kong/plugins/soap-xml-response-handling/schema.lua

And the xmldefinition.lua located under:

• ./kong-plugin-soap-xml-handling/kong/plugins/soap-xml-handling-lib/xmldefinition.lua

After confirmation from the Kong support team that it has been deployed, you’ll be able to see our SOAP/XML plugins for the Request and the Response in the Konnect Control Plane.

Deploy the plugins on Konnect Data Plane (Kong Gateway)

We have to update the Data Plane deployment, based for instance on helm Values file (yaml) including the SOAP/XML custom plugins definition. The Data Plane is also called Runtime Instance. These properties rely on a configMap containing the Lua code of the custom plugins.

1. Login to Konnect
2. Click on Runtime Manager
3. Select a Runtime Group or create a new one
4. Click on New Runtime Instance and select Kubernetes
5. Follow the procedure and copy the yaml description into a values.yaml file

6. Clone the GitHub repository of the custom plugins (if it’s not done yet):

git clone https://github.com/jeromeguillaume/kong-plugin-soap-xml-handling.git

7. Go in the custom plugins directory.

cd ./kong-plugin-soap-xml-handling/kong/plugins

8. Create a configMap for the custom plugins (Request and Response).

kubectl -n kong create configmap soap-xml-request-handling --from-file=./soap-xml-request-handling

kubectl -n kong create configmap soap-xml-response-handling --from-file=./soap-xml-response-handling

9. Create a configMap for the shared library

kubectl -n kong create configmap soap-xml-handling-lib --from-file=./soap-xml-handling-lib

Include subdirectories of the library.

cd soap-xml-handling-lib

kubectl -n kong create configmap libxml2ex --from-file=./libxml2ex
kubectl -n kong create configmap libxslt --from-file=./libxslt

10. Add the following properties to the values.yaml description:

env:
  plugins: bundled,soap-xml-request-handling,soap-xml-response-handling
...
plugins:
  configMaps:
  - pluginName: soap-xml-request-handling
    name: soap-xml-request-handling
  - pluginName: soap-xml-response-handling
    name: soap-xml-response-handling
  - pluginName: soap-xml-handling-lib
    name: soap-xml-handling-lib
    subdirectories:
    - name: libxml2ex
      path: libxml2ex
    - name: libxslt
      path: libxslt

See below a complete example of values.yaml

11. Install the new Runtime instance by executing the helm command:

helm install my-kong kong/kong -n kong --values ./values.yaml

Try the custom plugin

Prepare the test: Create a Gateway Service and Route of a SOAP/XML Web Service

Let’s create a Gateway Service based on a SOAP/XML Web Service that adds or subtracts two numbers.

1. Login to Konnect
2. Click on Runtime Manager
3. Select the Runtime Group where we deployed the custom plugins
4. Create a Gateway Service named calcWebService with this URL: https://ecs.syr.edu/faculty/fawcett/Handouts/cse775/code/calcWebService/Calc.asmx
5. Create a Route on the Gateway Service calcWebService with the path value /calcWebService
6. Call the calcWebService through the Kong Gateway Route by using httpie tool.

Note: Replace localhost:8000 with the IP address and the port of your Kong Gateway.

The expected result is 12:

Example 1: Do an XSLT Transformation on the Request

The plugin applies an XSLT Transformation on the XML request before the XSD Validation. In this example, the XSLT adds the value <b>8</b> that will not be present in the request.

Add soap-xml-request-handling plugin on calcWebService Gateway Service and configure the plugin with: XsltTransformBefore property with this XSLT definition:

Use this command where the <b>7</b> is no longer present.

The expected result is no longer 12 but 13 because the XSLT transforms the XML document by adding <b>8</b>

Example 2: Calling incorrectly calcWebService and detecting issue on the Request with XSD schema

We incorrectly call the Service by injecting a SOAP error; the plugin detects it, sends an error message to the Consumer, and Kong doesn't call the SOAP backend API.

Open soap-xml-request-handling plugin and configure the plugin with:

• VerboseRequest: enabled
• XsdApiSchema property with this value:

Use the command defined at example 1, change <soap:Envelope> to <soap:EnvelopeKong> and change </soap:Envelope> to </soap:EnvelopeKong> and Kong says:

Use the command defined at example 1, remove <a>5</a> => there is an error because the <a> tag has the minOccurs="1" XSD property and Kong says:

Example 3: Do an XSLT Transformation on the Response and change the SOAP message to an XML message

In this example the XSLT removes all tags and converts the response from SOAP to XML.

Add soap-xml-response-handling plugin and configure the plugin with: XsltTransformAfter property with this value:

Use the command defined at example 1. There is no longer a <soap> tag and the expected result is:

Conclusion

Other examples, like Request routing by XPath or Response XSD validation, are available in the GitHub repository.

By using ChatGPT we were able to drastically reduce the time needed to develop a Kong custom plugin; and Kong Konnect can provide governance, mediation, and security of SOAP/XML Web Services.

[cta-banner banner=konnect]