Continuous Deployment and Lambda
Continuous Deployment and Lambda
Continuous Deployment (CD) is the practice of building, packaging, and deploying to a piece of code — or a system — every time a change is made to that code or system. It’s kind of the ultimate in agile development, with near real-time release of new software.
AWS Lambda functions are small and (from a software coupling perspective) independent pieces of code used to construct a serverless system. Because of their small, decoupled nature — and because they normally have limited, well-defined contracts — they inherently lend themselves to CD.
One of the challenges to using CD with Lambda functions is that Lambda functions, from a solution perspective, rarely stand completely on their own. They’re often initiated by other system components (including other Lambda functions), and they interact with — or modify — other system components. This means that, to effectively perform CD with Lambda, you must also provision the system components they use.
We’ve seen three different methods of doing this, and we’ve used two of them. In this post, I’ll cover all three, as well as their pros and cons — then detail the one we use currently.
Complete configuration during the build process
This method of CD specifies not just the build, packaging and deployment of the Lambda function itself, but the full integration of all of the required surrounding system components as well. The lambda-maven-plugin is an excellent example of this methodology. When specifying a function using this plugin, you give all of the information required for that function to run in AWS. This includes items like the security groups, subnets, role, environment variables and integrations (triggers) that the function will require.
PROS: One-stop shopping to get your lambda function up and going, even creating it for the first time. Developers have full control over the configuration of the function.
CONS: Spreads your DevOps solution architecture across many different code projects. Requires cutting and pasting information from AWS (like the role ARN) into your build scripts. Developers have full control over the configuration of the function.
VERDICT: We believe this approach creates a relatively brittle system overall, where a change in any number of related components will cause builds to fail and CD to cease.
Complete configuration during DevOps provisioning
This method places all of the burden of releasing new lambda functions on the provisioning tool (e.g. Terraform). Using this method, we will re-run Terraform at the end of the build process to deploy our newly built/packaged Lambda function. The Terraform code itself lives in a different repository, and can be run independently from the Lambda build.
PROS: All of your system configuration is in one place. Changes to system components other than your Lambda functions are automatically managed by Terraform (if you’ve written good Terraform).
CONS: Your build process must have admin rights to your AWS account (Terraform requires it). You’ll run Terraform a lot, and only one execution can be active at any time. Each Lambda code repository must have Read Access to the Terraform repository. Lambda function builds may have unintended side-effects, because Terraform will make systemic changes.
VERDICT: We think this is a little heavy-handed for simply deploying a new version of a single Lambda function. What’s more, the admin rights and system changes it brings with it make the hairs stand up on our necks.
Configure with DevOps provisioning, but deployment during the Lambda build
This is a compromise between the two previous methods, and is the one we currently use.
A solution-provisioning tool (like Terraform) is used to provision and configure all of the components in the solution, including the Lambda functions, then link them together. The Lambda function build itself builds and packages the Lambda function, and if an actual named Lambda function exists in AWS, updates (i.e. – deploys) the new package to that function(s). A good example of this for Python is the lambda-setuptools project.
PROS: Centralizes all solution configuration into Terraform, absolving the Lambda build from having to know these items. Removes the need for admin privileges to deploy the Lambda package. Provides for ultra-fast CI/CD for your Lambda functions.
CONS: Initial Lambda creation is a two-step process (build/package Lambda, then run Terraform). Changes to Lambda function requirements (environment variables mostly) involves a two-step process (run Terraform, then build the Lambda function).
VERDICT: We find this process to be the least problematic of the ones that we’ve seen and used —providing the best solution control, while still enabling full CD.
Final thoughts
Assuming you want to do CD (and you should want to), you still don’t want chaos ensuing. We don’t deploy new Lambda function code automatically to production. Hats off to you if you’re that bold, and have the rigorous automated testing architecture to support it.
We use gitflow. It’s an awesome process that you should use to control git branching madness (CD aside) to manage where things get deployed, as well as the normal gitflow checks and balances for the progression of code changes.
Develop branches are automatically deployed to our development systems, release branches are automatically deployed to our staging systems, and new code only gets deployed to our production systems when it’s pulled to our master branches. With this approach, we’re able to impose checks and balances on CD, while still being able to roll something quickly and (mostly) automatically to production.
An experienced C-Level executive with a breadth of experience in operations, sales, marketing, technology, and product development. Joe specializes in innovation and change leadership, having had success as both an entrepreneur and intrapreneur, across numerous industries including defense, insurance, telecommunications, information services and healthcare industries.
Joe relishes the opportunity to make a real difference, both in the success of the corporation and in the lives of the people he interacts with. A self-starter that neither requires nor desires large amounts of oversight having succesfully built numerous DevOps teams and Solution Architected some of the most advanced and secure utility computing applications.