Enterprise Serverless 🚀 Hints & Tips
This section in the series on Enterprise Serverless specifically covers some hints and tips based off working on large scale serverless projects. You can read the ideas behind the series by clicking on the ‘Series Introduction’ link below.
The series is split over the following sections:
- Series Introduction 🚀
- Tooling 🚀
- Architecture 🚀
- Databases 🚀
- Hints & Tips 🚀
- AWS Limits & Limitations 🚀
- Security 🚀
- Useful Resources 🚀
Hints & Tips
YAML vs JSON
When working with the Serverless Framework you have the option of using JSON or YAML when defining your resources, both of which are fairly limited when you need to perform some ultra bespoke logic (on large projects this happens a lot in my experience!).
One of the key things to consider and monitor with distributed event driven serverless architectures is cost — and how to link that cost to particular workflows or larger domain services through the system (from API through to datastores such as databases or S3 for example). Tagging your AWS resources is a best practice and allows you too quickly monitor and determine where associated services have incurred overall costs.
Split out CloudFormation
The serverless framework is fantastic when managing serverless resources, but on any large scale serverless project you will no doubt quickly find yourself using the resources section in CloudFormation to manage other resources such as AWS Parameter Store, S3, IAM etc
One key consideration is to split each resource type, such as IAM, into its own file per serverless project, which can then be imported into the main JSON/YAML. If not you will soon find that your main serverless file grows massively and becomes hard to manage, and you will constantly have merge conflicts when working with multiple scrum teams.
One of the most difficult things to do in a distributed system is to track through logs how an API call, for example hitting API Gateway or AppSync, has made its way via various SQS queues, Step Functions and lambdas, and how far it has got before potentially failing or being successful. Times this by 500–200m+ API calls per month and its an issue tracking individual calls..
A key approach to all large scale serverless projects I have worked on is to generate a correlation ID at the earliest possible opportunity, and ensuring that this correlation ID is passed throughout the entire flow (including SQS messages for example). This allows collating the log entries per correlation ID using search as if it was one synchronous call, rather than trying to search through individual logs over many services for one API call which would be a nightmare!
💡 I typically set the correlation ID to the request ID when using API Gateway, or by generating a UUID in the before step of a pipeline in AppSync using a shared VTL.
Share expensive resources
Based on the first example of exporting JSON dynamically above; one key approach I have used in the past for sharing expensive resources across many ephemeral environments and QA is to export the resources dynamically based on stage.
Examples of these resources have been AWS Elasticsearch, VPCs and Nat Gateways, and AWS DocumentDB for example where it made no sense for each short lived environment to have its own dedicated resources, as well as the slow down in deploying them individually each time.
💡 The key with this approach is to ensure that you build the DocumentDB database names or Elasticsearch indexes dynamically per stage so this works seamlessly on the fly.
Lets connect on any of the following:
If you found the articles inspiring or useful please feel free to support me with a virtual coffee https://www.buymeacoffee.com/leegilmore and either way lets connect and chat! ☕️
If you enjoyed the posts please follow my profile Lee James Gilmore for further posts/series, and don’t forget to connect and say Hi 👋
I consider myself a serverless evangelist with a love of all things AWS, innovation, software architecture and technology.”
** The information provided are my own personal views and I accept no responsibility on the use of the information.