Three Tips For Lambda Layers

The Three Tips

1. Consider and manage your layer lifecycle
2. Don’t prematurely optimise
3. Consider different ways to decompose

What Is A Lambda Layer?

Layers are a mechanism by which you can have common code available to an array of AWS lambda functions. They’re a simple enough thing to set up:

MyLayer:
    Type: "AWS::Lambda::LayerVersion"
    Properties:
        Description: Common code for reuse
        LayerName: CommonCodeLayer
        CompatibleRuntimes:
            - python3.7
        Content: layer

Then you can use it in a serverless function by:

lambdaFunction:
    Type: "AWS::Lambda::Function"
    Properties:
        Code: lambda
        FunctionName: FunctionWithLayer
        Handler: function.handler
        MemorySize: 128
        Role: !Ref LambdaRole
        Runtime: python3.7
        Layers:
            - !Ref MyLayer

Dunning-Kruger & Lambda Layers

When getting started is so quick and simple as it often is with true serverless development, the speed at which you move can come with debts accrued through the best intentions.

A common journey to discovering layers is where you’ve written a collection of functions that operate as a microservice, and all of sudden you hit copy paste one too many times and you know there must be a better way. You find the notion of layers and a lightbulb goes off, you split your code off into a layer, drop the resource into your CloudFormation and hey presto, the dream of reuse has become reality. Day 1 is a wonderful thing, the problem feels solved and you move onto the next.

Then comes day 2, you need to make a change the layer, but now it’s shared across 10, 20, 30 functions, Hyrum’s Law has become manifest. You want to make a change but only have the change affect a subset of the functions, the speedy solution is to reach for the trust control-c control-v solutions to all life’s coding problems.

Having had this exact situation happen to me, hopefully the lessons I learnt may save you from the pain.

Consider And Manage Your Layer Lifecycle

Action: Separate the layer into its own CloudFormation stack

Give yourself the option of managing it independently.

Lambda layers introduce coupling between functions, in the example given above changes to the layer now have a large potential blast radius. If we need to make a breaking change to the layer, either we’ve now got to perform code updates across a huge swathe of functions, or we have to maintain multiple parallel layer versions.

How onerous a task the function updates are depends in a large part on your automated testing suite. If you have a robust testing culture, it may well be possible to make all the requisite changes and have confidence no regressions have been introduced. Google’s Beyoncé rule applies well here, “If you liked it, you should have put a CI test on it”

For managing multiple versions, by operating as an independent stack you can now export the layer reference with versioning, potentially semver. Now stacks can !ImportValue the layer they’re expecting and you can bring functions up to date independently of the layer update.

Don’t Prematurely Optimise

One of the most oft quoted truisms in software development:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. Knuth and Hoare

You may be thinking that managing the lifecycle is a non-trivial amount of work, and you’d be right. Managing shared code comes with overhead, that you’re trading off against the effort of managing the duplication. Like all trade offs this should be conscious.

Action: Regularly review the code in the layer to consciously decide what is worth the trade off

Consider Other Options

I’m not trying to make out that layers are evil, far from it. They’re a very useful tool to have. However, they are but one solution to code reuse in lambdas.

1. Packaging

Now CodeArtifact is finally released by AWS, packaging code has become a more realistic option. You still have to manage the lifecycle, but now the code has potentially wider reuse angles outside of lambda itself.

Packages are necessary in most non-trivial functions, potentially managing everything as packages is less cognitive load than a mixture of packages and layers.

Local Testing Side Note

When integrating with language standard testing frameworks, layers can be hard or impossible to import as needed. Packages are much easier to handle in this regard.

2. Functional Decomposition

If you have common code functions, could they potentially be served as a lambda function in their own right? This kind of decomposition has interesting benefits from many angles, such as:

• Allows for design options to better leverage AWS technologies, e.g. push data to SQS so you can manage throughput to a Dynamo table
• Finer grained IAM permissions, e.g. only give KMS key permissions to a much smaller subset of functions
• Better user experience, e.g. respond back to the client once work has been asynchronously queued rather than wait for the entire process to finish

See serverlesspatterns.io for many more ways functional decomposition can improve your architectures

Action: Reevaluate whether layers are the right choice for your current goals

Conclusion

Layers when used with due care and for the right reasons are an essential part of a serverless developers toolkit. Like every technology they suffer from the Dunning-Kruger effect, hopefully my battle scars will prove to be pedagogical and these lessons will help make you a happier developer.