Clik here to view.
Amazon EventBridge — a gem!
Many in the industry hailed the availability of Amazon EventBridge as the next big thing after lambda functions. They indeed have every reason to trumpet their appreciation and adoration for a technology that was made available to use for everyone by Amazon.
You probably know that EventBridge isn’t something completely new from Amazon, but is actually making the technology behind the CloudWatch events publicly available, so that everyone can now create ‘custom’ event buses alongside the existing ‘default’ event bus. The think tank behind this must be lauded for opening up a tonne of new possibilities that offer power, flexibility and most importantly the acceleration towards a ‘functionless’ integration of services!
What is EventBridge?
Amazon EventBridge is a serverless event bus that makes it easier to connect applications with data from a variety of sources. These event sources can be custom applications, AWS services and partner SaaS applications.
It provides a flexible way of filtering to allow events to be published to the event bus, and then, based on the target routing rules, sends the eligible events to the configured target applications or services.
Here is a high level diagram of EventBridge from AWS docs.
Clik here to view.
At the centre of EventBridge is the event bus. In addition, here are the main parts as I see them-
- Event publishing — variety of supported applications or services can send (publish) events to an event bus.
- Event filtering — this is the first pass filtering template (condition) that lets the events through and make them available for routing to consuming target applications.
- Event routing rule — this is like the exit criteria or gate for events before the qualified ones can be sent to the interested targets. This area is the focus of this article. We will see how we can make the best use of the different options available to us.
- Event targets or consumers — these are the applications or services that receive the eligible events that they are interested in and qualified to get through the routing rules.
Why look under the bridge?
At a high level, you will see the event producers that sent events to the bus and then on the other side there are different targets that can receive (subscribe to) events from EventBridge. All well and good.
In addition, you will also see a flexible way of setting up patterns to match incoming events to formulate efficient filtering rules. This is so very cool. However, majority of the early adopters and those who just scratch the surface stop right there.
In order to benefit from the extra power that EventBridge offers, we need to go under the bridge and look at the features available to us! I am merely bringing out certain features of EventBridge that we often overlook or think they are irrelevant. In fact, these are the ones that we will be thankful for, depending on the event routing requirements we will be working with in real life scenarios.
Event dispatch patterns
Before we go through the routing patterns, lets start with a simple event structure. Please note that for the purpose of this article and the points discussed here, I am focusing on the custom events. However, the same principles can be applied to the events that AWS services emit as well.
Sample event
To set the context, here is a simplified version of an event published to the event bus. Lets say this event is published by a microservice ‘service-checkout’ whenever an order is submitted. I’ve kept the structure brief for the purpose of the discussion.
{
"version": "0",
"id": "0c90d427-dd5d-0f19-12cd-c90bb8ae53fd",
"detail-type": "event",
"source": "service-checkout",
"account": "1234567890",
"time": "2019-12-17T10:29:48Z",
"region": "eu-central-1",
"resources": [],
"detail": {
"metadata": {
"domain": "SHOP",
"service": "service-checkout",
"type": "ORDER",
"status": "SUBMITTED"
},
"data": {
"orderNumber": "T123123123",
"customerId": "23hdfjdf-34ff-34ghj",
"totalValue": 29.99,
"items": 5
}
}
}As you know, the “detail” attribute represents the custom payload data of the event. It is up to the event producing or publishing application to decide what is included here, as long as it is a valid JSON.
The top part of the structure is specific to AWS. Few of those attributes are mandatory and others not. Ignoring those for now and focusing on the “detail” part for our discussion.
As you can see from the structure, I have split the “detail” into two parts — metadata and data.
The “metadata” section can be standard and a mandatory section for all the microservices that produce an event, where as the “data” section can be specific to each service and the content can vary depending on the service.
Note that this is not an EventBridge requirement to have this ‘metadata’ and ‘data’ subsections under ‘detail’ but it helps to maintain some order, especially in a typical environment where there can be a number of microservices and each sending different types of custom events to the bus.
Event filtering
A simple event filtering pattern to catch all of the orders that are in submitted state could look like this for the event structure mentioned above.
{
"detail": {
"metadata": {
"type": [
"ORDER"
],
"status": [
"SUBMITTED"
]
}
}
}Now lets focus on the event dispatch configuration options, which I think can be quite powerful depending on the way we handle events.
It is worth keeping in mind that the following configurations apply to most of the event targets that are commonly used. These may not be applicable for certain target services.
Pattern 1: Static event dispatch (Constant JSON text)
Imagine a situation where a monitoring dashboard continuously updates the number of orders customers placed in real time or the number of payments successfully authorised by a payment system. For this purpose, the consuming application only requires a notification or a prompt to tell that something has happened. Beyond that there is no real interest in the data or values associated with those events. This can be easily achieved by the target input configuration, as shown below.
Clik here to view.
The beauty here, as shown above, is that the original event data is never passed on to the consuming target. Instead, a predefined constant JSON text is sent as the event to the consumer. This is static and defined at the target configuration. The consuming service can act based on this static data it receives and will never get to see the original event data. Simple and secure!
Pattern 2: Filtered event dispatch (Part of the matched event)
With the static event dispatch above, we saw the ability to hide the entire event from the consuming targets. But, how about if the consumer requires part of the event data in order to perform its business logic? Well, in such situations we can filter in just the data that is needed to be sent to the targets. Nothing more, nothing less!
A typical scenario could be that a consumer application that keeps track of all the different types of payments made by customers and the amount of each payment. Or, it could even be an application that is monitoring the valid and failed payments for proactive status reporting. In such cases, the target applications need only the data they can act upon and nothing beyond that.
In the following example, the target consumer is receiving just the “data” section from the event. No other part from the original event is passed on to the targets.
Clik here to view.
Pattern 3: Transformed event dispatch (Input transformer)
This is an interesting event flow pattern. Here we get the benefit of both static events and filtered events. This is best suited for cases where the target application dictates the structure of the event to be delivered or the dispatch rule stipulates the abstraction of the original event data and standardises on the delivery format.
Another usecase could be where the target application itself does not perform data extraction and manipulation on the event data but relies on the EventBridge to deliver the data that can be used out of the box. For example, an user friendly descriptive message that incorporates values from the original event.
As shown below, the first part is where the required data values are extracted from the event payload. The second part is the template where the outgoing event data is created by incorporating the previously extracted values. One thing to note here is that the outgoing event data doesn’t need to be a JSON!
Clik here to view.
Pattern 4: Pass through event dispatch (Matched events)
If you want to deliver the original event payload and there is nothing to be hidden away from the targets, then this is the pattern to go for. In this case, the entire event data that satisfies the filtering rule is delivered to the targets without any meddling in the event data.
As you can guess, this is the default option and (due to that reason) probably the most commonly used one as well. Personally I would not recommend to go with this option unless the targets are within the same domain or account and there are valid reasons to send the original data as such.
Clik here to view.
Conclusion
This is a brief write up on EventBridge focusing mainly on the event routing rule configurations. If used wisely, it certainly can bring more versatility to the entire event ingestion and the delivery mechanism that we benefit from Amazon EventBridge.
When it comes to security, we promote granular level policies and least permissions. In a similar vein, we can also apply the least data exposure concept when it comes to sharing events across many services, applications and accounts. This is achievable with Amazon EventBridge and is a good practice to follow as well.
I hope you found the details I shared here useful. I will be bringing more on EventBridge in the coming weeks and months. Until then, happy eventing!
Image may be NSFW.Clik here to view.
The power of Amazon EventBridge is in its detail was originally published in LEGO Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.