API Gateways vs ESB: Mediation

With the advent of API Gateway solutions to the Enterprise Integration World, Integration Architects have a new solution option for solving integration issues. But how should we position this solution. We already have Enterprise Service Busses, B2B Gateways, so where do these API Gateways fit into the picture?

In this blog series, we want to explore some of the features of the API Gateway solutions and compare them with the more traditional ESB offerings. At first glance there seems to be some feature overlap between the two platforms, which leads to the natural question which tool to use?

Each blog in the series will focus on a specific functional topic and will discuss how this topic is addressed on ESBs vs API Gateways.

As a reference point, we looked at 3 common API Gateway solution offerings: API Connect from IBM, API Manager from Axway and the API Management Solution from Software AG. For the ESB reference we did not focus on a specific product offering as these products are more mature and therefore more alike.

In this first installment, we will address mediation. Mediation is a typical ESB functionality that often returns in API Gateway solutions. As mediation is a very broad term, we turned to the very common mediation pattern “VETRO” and analyzed the different aspects of this pattern and how they are addressed in the different products.

The following table provides a brief overview of our conclusion, please refer to the next sections for a detailed explanation on how we reached those conclusions

Validation of incoming messages has 2 aspects:

• Format validation: Validates if the incoming message is well formed. Typically, this type of validation is handled by XSD or JSON schema validation
• Content Validation: Validates the actual content of the message. A well-formed message can still contain invalid data. This kind of validation requires custom validation logic and often access to a data store.

Format validation is equally well handled by ESBs as API Gateways. All of the products we looked at provided “policies” to either validate XML or JSON messages and are very straight forward to implement. Software AG only supported XSD validation, but it included the option to call an “Integration Server” script, which basically means calling the ESB.

Content validation can also be implemented on API gateways. In an API message flow, it is possible to retrieve data elements from the incoming message and use it to invoke additional services to validate these elements. However, we see 2 advantages ESBs have over API gateways. Typically, it is easier to extract information from incoming messages and ESB flows have the possibility to call different validation services in parallel, whereas in an API management flow these calls are usually done sequentially.

In conclusion, limit the validation on the API Gateway to format validation and push the content validation back to the ESB or the implementation which provides you with better support and avoids the risk that business logic is introduced in the API gateway layer.

Format validation is equally well handled by ESBs as API Gateways. All of the products we looked at provided “policies” to either validate XML or JSON messages and are very straight forward to implement. Software AG only supported XSD validation, but it included the option to call an “Integration Server” script which basically means calling the ESB.

Enrichment means adding information to an incoming message that is needed for further processing. This additional information can be either static or dynamic. Static information is data that hardly changes over time and is something that can be looked up in the data dictionary or can easily be derived from the incoming information.

Example of static data: We have an API that will automatically create a new virtual machine. When calling the API, the client only needs to provide the name and type of the machine. It is not necessary to specify the environment. This information can be derived from the incoming message. A request coming from the test environment (identified by ip range) will result in the creation of the machine in the test environment. If the request originates from production, the machine is created in production.

Dynamic data requires access to a data store to retrieve the extra information. An example use case could be an API that allows to create orders. The API only specifies that the customer id is needed. But to create the actual order the customer type (gold, silver, bronze) must also be specified. This additional information can be “enriched” by looking up the information (calling a service) using the incoming customer id.

All the products we looked into provided this functionality with varying degrees of support. Both API Connect and the Axway API Gateway allow to include logic in their API message flows to retrieve data from incoming messages and use that information to call additional services and subsequently inject that information into the incoming message (dynamic data). It is also possible to execute logic in the API flow to derive additional information and inject that into the incoming message. Axway allows you to define database connections and has a key property store functionality build-in which helps with the retrieval of static data. On the Software AG platform, implementing this functionality is more complex as this also requires a callout to an “Integration Server” script which means calling the “ESB”.

In this section, we will address 2 aspects of transformation:

• Message transformation: Converts incoming data structures to the format required in downstream processing. Conversions between XML and JSON are also considered message transformations.
• Protocol transformation: Converting the protocol used to send the incoming message to the required protocol needed by the implementation. In the API world, this traditionally means converting SOAP based services to REST based services

All investigated products provide tools to support these use cases. When we look at message transformation, the different gateways offer “policies” for doing XSLT transformations and IBM and Axway also have XML to JSON conversion policies. Be aware, however, that converting XML to JSON and vice versa isn’t as simple as it seems. You have to resolve issues like namespaces, XML attributes etc. These policies use the badgerfish convention and will require additional processing to get the format right. We feel that ESBs have a better support for this feature, as they often provide visual mapping tools to do format conversion and thus bypass these issues. Only API Connect from IBM offers a similar tooling.

When looking at protocol conversion, we focused on the SOAP to REST conversion, and again all products supported this functionality. The Software AG solution produced an RPC style REST interface. So, for example, creating a customer resulted in the following REST endpoint “Post to customer/createCustomer”.

On Axway, creating the connection between REST and SOAP was very tedious as it required some low-level setup such as adding the correct HTTP headers and setting HTTP verbs. The best experience we had on API Connect, which provided an advanced tool set to perform the conversion. In all, the tooling is comparable to the tooling provided on ESBs.

Also for transformation, our advice is to use this functionality prudently and push this to either the ESB or the implementation, as this provides you with a more extensive tool set. The main reason being when using API gateways, you might not get an optimal result, and/or you are left with complex implementation logic on your Gateway. Certainly for gateways that are running in a DMZ this might not be a desirable situation. Use the transformation functionality of the gateway to overcome simple integration hurdles (combining or splitting fields, etc…).

Routing means delivering the incoming request to the correct implementation service. Determining where to deliver the request is based on the information derived from the incoming message.

Here our advice is simple, the functionality is very similar to what you have on an ESB so if you have use cases for it, go ahead to exploit the functionality of your API Gateway.