CDK Lambda Deployment takes about a minute - how about sub second Function Code Deployment?



CDK Lambda Deployment takes about a minute - how about sub second Function Code Deployment?

Creation of Lambda infrastructure with the CDK is really powerful. Updating the Function code is really slow. Here is a fix for that to get to a sub-second Lambda function deployment time.

Setup

Let`s assume we have a simple lambda function we want to test in the cloud by invoking it from the AWS console. We deploy the code to Lambda often, so the deployment should be as fast as possible.

We start with using a cool CDK Lambda Infrastructure Construct, assuming we have a Lambda function called “warmkalt” - the german word for weather in April, “warm and cold”:

  const image = DockerImage.fromRegistry("golang:latest")
  const backend=new Function(this, 'warmkalt', {
    runtime: Runtime.GO_1_X,
    handler: 'main',
    code: Code.fromAsset(join(__dirname, '../code'),
      {
        bundling: {
          user: "root",
          //included GO version to old
          //image: Runtime.GO_1_X.bundlingDockerImage,
          image,
          command: [
            'bash','-c', [
              'go build -mod=mod -o /asset-output/main main/main.go' ,
            ].join(' && '),
          ],
          environment: environment,
        }

      }),
    memorySize: 1024,
  });

As we like the newest (1.16 at this time) GO version, we use a custom image with the latest GO version. The bundle version with the CDK is older.

We have the following Lambda code in code/main.go. For your own deployment you can use any code and language, you just have to have a build script.

Function Code for Function “warmkalt”

package main

import (
        "fmt"
        "github.com/aws/aws-lambda-go/lambda"
        "os"
)

func hello() (string, error) {

        _, err := os.Stat("/tmp/test.txt")
        if os.IsNotExist(err) {
                fmt.Println(err)
                fmt.Println("File does not exist, creating file.")
                f, _ := os.Create("/tmp/test.txt")
                defer f.Close()
                _, err := f.WriteString("Hello World")
                if err != nil {
                        fmt.Println(err)
                }

        } else {
                fmt.Println("File exists.")
        }

        return "Hello ƛ", nil
}

func main() {
        lambda.Start(hello)
}

The code shows that a file created during the Lambda cold start is still there with a warm start.

We deploy it the first time, the resources are created, everything fine.

Update Code in a long Minute

We now change only a single line and deploy again with the cdk:

cdk deploy --force --require-approval "never"  

This takes about a minute.

The time output:

 5,86s user 0,78s system 10% cpu 1:04,33 total

That takes too long!

Next step - using the AWS cli.

Update code in 4 seconds

Now we build the code separately, so that CDK does not have to build it.

GOOS=linux go build -o dist/main main.go

This is valid for a real life development cycle, because we only deploy pre-build function code. If the build does not work, we would not deploy it. If you work inside a CI/CD pipeline, just stick with the CDK..

Now we can deploy the code build into the static binary directly with the update-function-call:

zip function.zip main
aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip

The median from 5 iterations gives a 3.61 seconds deploy time, which is about 18 times faster.

7307f7b7.png

(This is a german excel, so “1,01” means “1.01”)

Update Code in 3 seconds

How to speed this up even more?

We could zip the build (or build the zip) in the background after each build cycle!

With Task and other build systems it is possible to watch for file changes, here with --watchas a parameter.

  build-zip:
    desc: Builds lambda
    cmds:
      - clear && date
      - GOOS=linux go build -o dist/main main.go
      - cd dist && zip function.zip main
    sources:
      - main.go    
    generates:
      - dist/main
      - dist/function.zip
    silent: true
    ignore_error: true

When we start task with:

task build-zip --watch

It will update the dist/funktion.zip each time the main.go is changed.

So we change the deploy code (also in Taskfile.yml) as follows to deploy the pre-build zip:

  deploy2:
    dir: dist
    deps: [ build-zip ]  
    desc: Deploy from local disc
    cmds:
      - aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip

85361adf.png

Update Code in 2 seconds

Updating from a S3 Bucket should be even faster. So we upload the zip file to s3 also in the background:

  build-zip-s3:
    desc: Builds lambda
    cmds:
      - clear && date
      - GOOS=linux go build -o dist/main main.go
      - cd dist && zip function.zip main
      - cd dist && aws s3 cp function.zip s3://cdktoolkit-stagingbucket/functions/warmkalt.zip --quiet && echo "Upload complete"
    sources:
      - main.go    
    generates:
      - dist/main
      - dist/function.zip
    silent: true
    ignore_error: true

and run

task build-zip-s3 --watch

To deploy we use:

  deploy3:
    dir: dist
    deps: [ build-zip-s3 ]  
    desc: Deploy from s3
    cmds:
      - aws lambda update-function-code --function-name "warmkalt" --s3-bucket cdktoolkit-stagingbucket --s3-key functions/warmkalt.zip 

And now we are down to 1,56 seconds, about 40 times faster.

Sub 1 second

The remove the last slow step, we replace the aws-cli with a less than 30 lines GO programm:

package main

import (
	"context"
	"fmt"
	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/config"
	"github.com/aws/aws-sdk-go-v2/service/lambda"
)
func main() {
	cfg, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		panic(err)
	}
	client := lambda.NewFromConfig(cfg)
	params := &lambda.UpdateFunctionCodeInput{
	  FunctionName: aws.String("warmkalt"),
	  S3Bucket: aws.String("cdktoolkit-stagingbucket"),
	  S3Key: aws.String("functions/warmkalt.zip"),
	}
	response, err := client.UpdateFunctionCode(context.TODO(), params);
	if err != nil {
		fmt.Println("Error: ", err)
	}else{
		fmt.Println("Deploy:",response.LastUpdateStatus)
	}
}

This just calls the UpdateFunctionCode Lambda api with the parameters, where to find the zip on the bucket.

We compile it in the binary tools/deploylf with go build -o deploylf main.go

The task changes to:

  deploy4:
    deps: [ build-zip]
    desc: Deploy with go programm
    cmds:
      - tools/deploylf

Now we are down to 0.6 seconds for a Lambda Function deployment, wich is about 100times faster than the CDK deployment.

4bec5636.png

Fast Edit setup

So when i save the code:

Edit

The build is started, zipped & uploaded in the background in the build&upload window.

Upload

Now i can deploy the Lambda function code in less than one second:

667d62d6.png

If the update process is not running we “slow back” to 6 seconds:

3bf21538.png

Taskfile as whole, try yourself!

# https://taskfile.dev

version: '3'

tasks:
  build:
    desc: Builds lambda
    cmds:
      - clear && date
      - GOOS=linux go build -o dist/main main.go
    sources:
      - main.go    
    generates:
      - dist/main
    silent: true
    ignore_error: true


  deploy1:
    dir: dist
    deps: [ build ]  
    desc: Deploy from local disc
    cmds:
      - zip function.zip main
      - aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip

  build-zip:
    desc: Builds lambda
    cmds:
      - clear && date
      - GOOS=linux go build -o dist/main main.go
      - cd dist && zip function.zip main
    sources:
      - main.go    
    generates:
      - dist/main
      - dist/function.zip
    silent: true
    ignore_error: true

  build-zip-s3:
    desc: Builds lambda
    cmds:
      - clear && date
      - GOOS=linux go build -o dist/main main.go
      - cd dist && zip function.zip main
      - cd dist && aws s3 cp function.zip s3://cdktoolkit-stagingbucket/functions/warmkalt.zip --quiet && echo "Upload complete"
    sources:
      - main.go    
    generates:
      - dist/main
      - dist/function.zip
    silent: false
    ignore_error: true



  deploy2:
    dir: dist
    deps: [ build-zip ]  
    desc: Deploy from local disc
    cmds:
      - aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip

  deploy3:
    dir: dist
    deps: [ build-zip ]  
    desc: Deploy from s3
    cmds:
      - aws lambda update-function-code --function-name "warmkalt" --s3-bucket cdktoolkit-stagingbucket --s3-key functions/warmkalt.zip 

  deploy4:
    deps: [ build-zip-s3]
    desc: Deploy with go programm
    cmds:
      - tools/deploylf

Thanks to

Photo by Mélody P on Unsplash

Similar Posts You Might Enjoy

Cloud Driven Development Workshop@devopenspace

This is a live Blog from the workshop “Cloud Driven Development” on https://devopenspace.de/. Forget a lot of what you know about classic full-stack development. Together, we’ll dive into cloud-driven software development and build a sample serverless application in AWS. This blog was build live during the workshop on November 2021. So it`s not a complete reference, just a few hints to test and deploy the infrastructure and the applications. - by Gernot Glawe

Lambda Container Deployment with CDK: Using Arm based Lambda with GO

End of September 2021, AWS announced Graviton 2 powered Lambda Functions. The announcement post says “All Lambda runtimes built on top of Amazon Linux 2, including the custom runtime, are supported on Arm…”. Not all languages out of the box, for using GO as fast language on a fast Graviton processor, you have to use Docker based deployment. Here I show you a simple - CDK powered way to do that. - by Gernot Glawe

CDK Infrastructure Testing - Part 2b - Unit, Integration and Application Test for Serverless Lambda Functions

After describing the context of the test pyramid for Infrastructure as Code in part 1, and the Web Application in Part 2a - let`s apply that to some Lambda function. - by Gernot Glawe