Building Dynamic Aggregate APIs with GraphQL

Domain-driven designs are popular in organizations that have complex domain models and wish to organize engineering around them. REST-based architectures are a common choice for implementing the API entry point into these domains. REST-based solutions are straightforward for the API builder and for API consumers concerned with data from a single API. But what about developers tasked with aggregating information across domains? Developers building end-user websites, mobile apps, and internal dashboards are faced with the difficult task of orchestrating data across these domain APIs.

These developers need access to a wide range of data to build a consistent experience for the end-user. Often these developers resort to anti-patterns, like building hand-crafted "back-end for front-end" (BFF) APIs that do complex domain aggregation. These custom BFF APIs are maintenance burdens as they multiply when clients require different aggregate views of the domain information. BFF APIs are often authored in REST and directly expose clients to custom static specifications which are vulnerable to upstream API changes. What these developers need is a dynamic aggregate API to build their applications on top of.

GraphQL offers a solution to this problem by providing a flexible query design along with a layer of decoupling between clients and the domains that service their data needs.

If you're new to GraphQL, you may want to review the basics of GraphQL and the base concepts in our previous post before proceeding. In this post, we're going to take the next step and look at a functioning GraphQL service that aggregates existing domain APIs and exposes them into a dynamic API for these common use cases.

Domain REST APIs

In this post, we're going to use a fictitious airline named KongAir as our example business use case.

KongAir is a domain-driven technology organization where individual development teams build services and applications focused on their business domains. Teams are free to work in their own languages and technology stacks. APIs are delivered using the REST model, and each service contains an OpenAPI Specification detailing the API behavior. All the source code discussed in this post is available in the public KongAir GitHub repository.

KongAir deploys the domain services on Kong Konnect, which provides a full API management platform including an API Developer Portal we can use to explore the REST APIs. You can view the developer portal at https://portal.kong-air.com/.

The KongAir flight-data team builds APIs that serve public, unsecured data for the airline’s flight routing and schedule. This team builds services in GoLang for its straightforward syntax and high-performance runtime characteristics. The team provides the following services:

• Routes API (spec, source): Provides the KongAir routing information including flight origin and destination codes.
• Flights API (spec, source): Provides the KongAir flight schedule information including flight number and other details.

The airline's sales development team builds services that serve customer-specific needs including personal information and flight booking systems. The sales development team uses NodeJS for its rapid development capabilities and extensive library ecosystem. The team provides these services:

• Customer Information API (spec, source): Provides the KongAir customer information service including payment methods, contact info, frequent fliers, etc…
• Bookings API (spec, source): Provides the customer bookings service that is used to secure and report customer flight bookings.

The APIs are also hosted at api.kong-air.com, which means we can test the APIs with an API development tool like Kong Insomnia.

We can also query the Routes API on the command line with tools like curl:

Now that we have some REST APIs to build on, let's see how to compose them up into a dynamic experience API for our application developers.

GraphQL Experience APIs with Apollo Server

The experience API is built on top of the popular Apollo Server, which provides us with a complete GraphQL SDK. The full source code for this API is available in the experience team's folder in the KongAir repository. In the following sections, we'll tease apart the important sections of the code and see how the REST APIs are integrated with a GraphQL service.

Schemas

As we learned in the first part of this series, GraphQL services start with a schema that defines the queries and types available to clients. The KongAir experience API is coded from the perspective of the end user, with a single query named me that returns an object of type Me. The Me object defines all of the information that can be provided about a user including their personal information and current bookings. When making a query to the GraphQL service, applications will provide a bearer token in the Authorization header. This token contains a username claim in the token body which will be used to identify the user throughout the KongAir system.

schema.js

The full schema for the KongAir experience API is defined in the schema.js file and includes the rest of the types available to query in the API.

Data Sources

GraphQL services work in partnership with upstream data sources. In greenfield projects, you may choose to connect GraphQL directly to databases. Here, we're layering GraphQL onto the existing KongAir domain services. The Apollo SDK provides us with a REST-specific data source library that aids in the integration of upstream APIs by providing convenience functions, error handling, and caching layers. In the datasources folder you will find a separate file for each KongAir domain API we are integrating. For example, the routes API datasource is defined as follows:

The JavaScript class extends the RESTDataSource and includes a configurable baseURL value as well as implementation functions for each resource and HTTP verb combination. At runtime the baseURL is paired with the path provided in each implementation function argument to construct the full request URL. You can see how the RoutesAPI class aligns with the routes service REST paths from the OpenAPI snippet below (openapi.yaml):

In the KongAir system, users are identified by a JSON Web Token (JWT) provided in the request Authorization header. KongAir runs Kong Gateway as the API gateway in front of all services, and uses the JWT Plugin to authorize requests. The gateway passes the token to the GraphQL service, so when we code the data sources for any secured APIs, we must pass the token through to the upstream REST APIs. More details on token handling in the API will be discussed later in this post. For now, the BookingsAPI class shows the passing through of request authorization headers to the upstream API.

booking-api.js

The Apollo documentation on fetching from REST APIs provides more details on using the library.

Resolvers

The Apollo documentation on Resolvers provides us with a precise definition of their purpose:

A resolver is implemented as a JavaScript object with properties that correspond to the GraphQL schema types. The property values are functions that retrieve the relevant data, and as the Apollo SDK fulfills a query request, it traverses the schema and invokes the resolver function for each field. When using data sources as described above, we define a resolver that maps the schema type to the corresponding data sources. Here is what the resolver code looks like for the KongAir Me experience API:

resolvers.js

The magic in the Apollo resolver SDK is that for any field that does not have a resolver definition, the SDK defines a default resolver for it. In cases where the data source returns an object with property names that matches the schema property names, the server can map these fields automatically.

For example, in the Me schema, a field is defined as username, which matches the returned field from the Customer Information API exactly allowing the field to be mapped automatically. Resolving data for more complex objects and relationships between them requires additional logic. For example, let's look at how a user’s Bookings are returned from the GraphQL service.

A Booking contains a Flight, but the Bookings API only returns some of the flight information, not the full flight details. FlightDetails are provided by the Flights API, and the Flight and FlightDetails are related using the Flight.number field. Similarly, a Flight contains a Route, but the Route information is provided by the Routes API. A Flight and a Route are linked by the Flight.route_id field.

When a request comes into the GraphQL service, the Apollo SDK traverses the resolvers' structure to populate the data, this is called the resolver chain. Initially, the SDK will invoke the resolver defined for the specific Query that was invoked, me in our example above. From that resolver's return value, the SDK will traverse each field and either invoke a defined resolver function or attempt to use the default resolver as explained earlier. The resolver functions accept arguments, the first of which is parent, which contains the previous result in resolver in the chain. The parent instance is analogous to a parent in an object-oriented data model. This allows us to obtain related information from different APIs using parent / child relationships in the data. In our example, the flight details are returned from the Flights API using the parent.number field, which is defined as the Booking.flight.number field in the GraphQL schema.

The full code for the resolver is defined in the resolvers.js file.

Bringing it together

The entry point of the GraphQL service is defined in the index.js file. We bring together the schema and the resolvers to configure the Apollo Server. Additionally, the resolvers need access to instances of the data sources. This is accomplished using the Apollo server context function, which is provided as an argument to the startStandaloneServer function.

index.js

The context function is executed for every incoming GraphQL request, and within this function we define new data source instances. In the cases of the CustomerAPI and BookingsAPI, we also pass in the incoming request object which allows the data sources to forward authorization headers to the upstream API calls.

Run and query the experience API

We've covered the basics of the experience API design. Let's run the GraphQL service and see it in action.

The server is a NodeJS application and requires node and npm to be installed and available in the path prior to running. The service has been tested with Node version 17.9.1 and npm version 8.11.0. You will also require git to obtain the source code in the instructions below.

From the terminal, clone the KongAir repository and change into the experience directory:

You may have noticed above that the data source base URLs are configured using the dotenv configuration library. The KongAir repository provides a default .env file that configures the base URLs to point to the hosted api.kong-air.com services. This will allow the GraphQL service to run and route requests to the hosted APIs without running the domain services yourself. Alternatively, you can run each of the domain API services locally following the instructions in the experience API documentation.

To continue, run the Apollo server:

The running server reports it's listening endpoint:

Before running GraphQL queries against the server, we have to understand the method the KongAir system uses to verify user identity.

JWTs provide an easy way to transfer identity between services. The domain REST APIs require a JWT with a payload that contains a username field which is used to identify API users.

All KongAir services require the JWT to be provided in the request Authorization header as a bearer token. Here is an example header with a bearer token:

The KongAir repository provides a helper file (JWT.env) with some valid example JWTs that you can use to make queries to the experience API.

From the experience folder, source in the JWT.env file which loads the variables into your environment:

Now we can query the GraphQL API with curl, providing a query string as well as the user’s JWT in the request:

And the server will respond with the user's information:

You may wonder why the GraphQL query is sent as a POST request. GraphQL query expressions can be complicated nested JSON objects, and it may be simplest to pass them to the server as a JSON formatted body in an HTTP request. Idiomatically, GET requests do not support a request body, so POST is commonly used. However, GraphQL servers should also support accepting a query in a GET request via the query parameter. Here is the same example via GET:

From the perspective of the client, what exactly makes the GraphQL API different from the upstream domain REST APIs? The dynamic nature of the GraphQL query capabilities puts more power in the hands of the client making requests. Imagine KongAir is building a mobile application with a screen that shows the user their personal information including their username, address, and frequent flier number. The following client query provides this information:

And the server responds with these specific fields:

The same application has a different screen that shows the user their current bookings. This requires a different upstream domain API, but the GraphQL service can compose it together with the customer information for us. We don't have to create a new BFF API; we just construct a different query on the client side. Here is an example request for the user's bookings:

And the user's bookings are returned:

Designing GraphQL Queries with the Apollo Sandbox

In development mode, the Apollo SDK provides an embedded GraphQL IDE called the Apollo Sandbox. To use the sandbox with the server running, open your browser to http://localhost:4000 and you should see an interactive interface to build GraphQL queries.

You may find this easier to use than a command line for exploring the capabilities of the GraphQL service. To successfully make requests to the KongAir API, you will need to configure the Sandbox with the bearer tokens. Open the IDE Settings, then Connection Settings and provide the token information in the Shared headers section.

Summary

GraphQL offers a compelling solution for building dynamic APIs and is particularly useful for clients that present data to end users. These types of experience APIs are beneficial to clients that need to avoid the rigidity of traditional REST-based APIs and GraphQL provides a key abstraction layer towards that goal.

But once you've integrated GraphQL into your API ecosystem, the job isn't complete. GraphQL APIs need management, as all API technologies do. Protecting the API is paramount as you expose it to end-user applications and the full outside world.

In the next entry of our series, we'll look at using Kong Konnect with Kong Gateway to apply critical governance to our KongAir experience API using a declarative and modern approach (aka APIOps). See you there!