Less is more?
During my school days I used to get confused by the saying, “less is more”. It puzzled me for a very long time. I remember trying to analyse the statement and asking myself “how is 2 more than 5?”. Those days I feared to ask my English teacher the meaning as I thought she might think I sounded stupid. Many years later, I am still baffled by the same saying, especially when I read statements such as, “… in serverless less is more”. So what does it exactly mean? I was determined to find out.
The definition for the phrase “less is more” varied depending on where I looked. One definition said,
‘less is more’ is used to express the view that a minimalist approach to artistic or aesthetic matters is more effective.
To my humble brain, that sounded complicated to digest. So I searched again and found the following —
‘less is more’ means that having just the essential things is better than having way too much of superfluous things. It allows you to focus on what matters.
Though not convincing, I was getting closer but then I found the following that made me to feel at ease. I halted my search there!
Simplicity is better than elaborate embellishment; Sometimes something simple is better than something advanced or complicated.
The Functionless fantasy
In recent months, especially after the popular adoption of serverless, experts have been prophesying and urging the service providers to get to a state of Functionless. This is like the ‘Lambda Nirvana’; where you no longer hand code your functions and instead somehow knit the services using out of the box features to make things work the way we want them to work.
Though it sounds interesting and certainly a fantasy, the term Functionless, as the next evolutionary stage of Serverless, is far from being becoming a 100% reality. Similar to a natural evolutionary process, this will also take time before serverless as a technology matures and gets somewhat closer to being fully functionless.
Should that disappoint anyone? Absolutely not! Why, you may ask. Because, we have within our limited powers to being functionless in certain areas and in certain ways, rather than endlessly waiting for the vanity state of being Functionless.
But how?
We as engineers often get carried away with technology and the affinity we have towards it. The same is true when it comes to serverless and writing lambda functions. Once in my talk, I showed the ‘Lambda Hammer’ and termed it as the architects’ best tool (not to mention the noise it created among the architects present there).
In the same vein, we often walk around with a ‘Lambda Vision’. Because, all of a sudden, we tend to think that every engineering problem can be solved or is solvable only with a lambda function. This narrows our thinking and we try to fit lambda functions everywhere, irrespective of whether one is needed or not.
Why ‘too many’ is too many?
Why am I exhibiting so much of a concern on having plentiful lambda functions? The reason is simple. Why have one when not needed in the first place? After all, one definition that we saw earlier said “simple is better than complicated” or in other words “less is more”!
Think about these benefits of having less functions:
- less code to write, test, deploy and run
- less functions to maintain
- less points of failures
- less IAM policies, less permissions to set
- less security worry
- less lambda concurrency
- less monthly spend on lambdas
If you are a start-up and trying to come up with cool business ideas then you don’t need 100s of lambdas to spend your money on. Rather, you need just a few good lambdas to bring the money in. It is crucial that, whether it is a start-up or an established enterprise, the mindset to reduce waste and awareness around the serverless costing could make a considerable amount of saving.
Ways to make less more
There are a number of ways we can make this happen and achieve less more often. Here are a few but not restricted to-
- AWS API Gateway integration
- Fat vs lean lambdas
- StepFunctions integrations
- Data expiry mechanisms
- Third-party notifications and alerts
- Event filtering and routing
- Key rotation
- … and many more
The intention of this light-hearted write-up is to raise the awareness and not to delve deep into technical matters. Nevertheless, I will briefly cover few areas to express my views and share my limited knowledge.
API Gateway integration
This is one place where most engineers overlook and miss the trick. Mainly happens to those who are new to AWS and API Gateway and often get carried away with the assumption that there must be a lambda function behind the API in order to fulfil a service. This need not be the case!
Look at the pictures below that show a portion of the non-lambda AWS services that can be integrated directly with API Gateway.
The most common services such as S3, SQS, DynamoDB, etc are so easy to be integrated directly with an API. There are decent AWS documentation available and also there are numerous blog posts written by enthusiastic serverless engineers as well.
I published a post recently on “Sequence Numbering in Serverless via API Gateway” that follows this Functionless Pattern to integrate DynamoDB with API Gateway to generate sequence numbers without the help of any lambda functions.
Sequence Numbering in Serverless via API Gateway
Fat vs lean lambdas
This is more of a development and engineering practice than anything else. At various stages during the serverless development, teams will be at cross roads arguing whether to have a single lambda doing bulk of the work or to split things to fit many smaller lambdas. The decision depends on each situation as there is no right or wrong answer here.
I poured my thinking on this in a recent article, aptly titled as “Lambda vs Lambdas”.
StepFunctions integration
Though most of us are now up to date with all the recent enhancements to StepFunctions, I just thought of highlighting here if anyone still out of date with the updates.
In the past, if we wanted to send a message to SQS or to perform a read/write action on a DynamoDB table, we had to rely on having a lambda function as part of the work flow to have these external communications with SQS or DynamoDB. This has now changed, and StepFunctions provide out of the box integration support for SQS and DynamoDB. This eliminates the need for lambda functions to act as communication agents, thus enabling us to reduce the lambda footprint.
Data expiry mechanisms
As we move from traditional form of development and applications, often influenced with the closeness to relational data storage, we tend to rely on some form of timestamp to know the age of the data that we store and measure its validity beyond a certain point. There are different ways of making sure whether to retain such aged or expired data or to remove them completely or to move them to a different location. One common approach traditionally practiced is to have some sort of batch job or jobs that get triggered as needed to carry out the inspection and removal action.
The above often influences our thinking to follow the same path when it comes to serverless and handling data held in DynamoDB and S3. In order to achieve the above, there must be some logic wrapped in as a lambda function and triggered by a CloudWatch scheduler rule. This will work fine but there are better options that we can do without having to rely on lambda functions.
Enter DynamoDB TTL (Time to Live). TTL offers us the built-in functionality to expire/remove unwanted data based on a timestamp. By enabling the TTL feature and by adding a TTL attribute with the required time to retain the data, DynamoDB takes care of removing that data item once its period is up. Of course there are few other little details to understand but that’s pretty much the functionless way of expiring data.
S3 offers a similar feature with its data retention policy setup where it is possible to configure the number of days before expiring the objects based on their creation timestamp.
Many other services also have ways to express how long the data to be retained within their stipulated maximum data holding duration. Making appropriate use of these built-in capabilities aid in reducing unwanted functions being written.
Event filtering and routing
Those who have worked with SNS (Simple Notification Service) topics and multiple subscriptions must have experienced the difficulty in setting up subscription filtering. Unless there are custom attributes to rely on, this is a challenging problem to solve, especially when the messages are received from external applications that are beyond our control where we have no say in the content of the messages or the attributes being present.
One way to ease the above is to implement some kind of a Message Relay Pattern where a dedicated lambda will inspect the incoming messages before adding the necessary filtering attributes and then pushing to another topic and/or other destinations. This adds unwanted complexity and lambda footprint. Good news is that there is now a much better way to handle this situation.
Bring on Amazon EventBridge! In my view, EventBridge adds order and aura to event-driven computing. The event filtering and routing capabilities available with EventBridge are just awesome and must be explored. In most places this eliminates the need for unwanted lambda function to carry out the filtering and routing tasks.
With the above sample event structure, the routing filter can be easily setup based on any part of the event data. For example, if only the events with detail:event:type as CHECKOUT to be sent to a subscribing lambda function, then the filtering pattern can be as simple as –
Moreover, EventBridge filtering rules also allow to transform the outgoing event to its subscribers. Say for example, a lambda function needs to be notified when an order is placed but does not need to receive any event data for its functioning. In this case, a rule can be configured to send a custom input data to the subscriber which can be completely unrelated to the incoming event bus data.
So, making good use of EventBridge has the potential to eliminate or reduce the lambda footprint to a great extent.
Other areas
There are other areas and possibilities to limit the need to implement lambda functions. Key rotation is one area where we can make use of the out of the box capabilities of Secrets Manager, for example.
Another common area is where we interact with third-party applications and their APIs to send monitoring alerts and similar interactions. In such cases, we are most likely to have SNS topics that receive the notifications. Rather than having a lambda function, we can use the HTTP/HTTPS based subscription model to perform the external interaction. Again without the need of any lambda function.
The ever popular AppSync and GraphQL area is yet another place where external API interactions as well as resources in DynamoDB can be managed without needing a lambda function as a mediator.
Conclusion
The areas that I covered here are probably the most common ones and constrained to my AWS and serverless knowledge. I would encourage you to identify other areas and other means of reducing the use of lambda functions. Please share your thoughts for the benefit of all of us!
We live in a world where everyone wants more share of material things such as money, wealth, power, etc., but for some strange reason when it comes to technology we all try very hard to make less more appealing, which is definitely a good thing. And I do hope we all agree that we should become increasingly aware of reducing complexity in serverless. After all, something simple is better than something complicated.
Go Build Server-less-Functions!
Sheen Brisals is a Senior Application Engineer at The LEGO Group.
Don’t wait for Functionless. Write less Functions instead was originally published in LEGO Engineering on Medium, where people are continuing the conversation by highlighting and responding to this story.