Create a CI/CD pipeline for .NET Lambda functions using AWS CDK pipelines

The AWS Cloud Development Kit (AWS CDK) is an open-source development platform that defines cloud infrastructure in familiar programming languages and makes it available via AWS CloudFormation.
The following article will explore the process of creating a Continuous Integration/Continuous Deployment (CI/CD) pipeline for .NET AWS Lambda features using CDK pipelines. The authors will discuss all the necessary steps to automate the deployment of .NET Lambda features, including configuring the development environment, creating a pipeline with AWS CDK, configuring pipeline stages, and publishing test reports. Additionally, they will show how to promote a deployment from a lower environment to a higher environment with manual validation.

Context

AWS CDK makes deploying a stack that delivers infrastructure to AWS from a workstation easy by simply running CDK Deploy. This is useful during initial development and testing. However, in most real-world scenarios, multiple environments exist, such as programming, testing, moving, and production. Deploying CDK applications in all these environments using the CDK tool may not be the best approach. Deployment in these environments should be done through more reliable, automated pipelines. CDK Pipelines makes configuring a continuous deployment pipeline for CDK applications supported by AWS CodePipeline easy.

The AWS CDK Developer Guide's Continuous Integration and Delivery (CI/CD) using CDK Pipelines page shows how CDK Pipelines can deploy Node.js-based Lambda functions. However, .NET-based Lambda functions differ from Node.js or Python-based Lambda functions because the .NET code must first be compiled to create the deployment package. As a result, this article was written as a step-by-step guide to help .NET customers implement their Lambda functions using CDK pipelines.

In this article, you and the authors will explore creating a pipeline that runs compilation and unit tests and deploys a .NET Lambda function in one or more environments.

Architecture

CDK Pipelines is a design library that allows you to share a CodePipeline pipeline. The pipeline created by CDK pipelines mutates itself. This means that you have to run cdk once to run the pipeline. The pipeline automatically updates itself if you add new application stages or stacks in the source code.

The diagram below shows the architecture of a CI/CD pipeline created using CDK pipelines. Take a high-level look at this architecture before delving into the details.

The solution creates a CodePipeline with the AWS CodeCommit repository as the source (CodePipeline Source Stage). When the code is curated in CodeCommit, the pipeline automatically starts and downloads the code from the CodeCommit repository branch to proceed to the compile stage.

• The Compile stage compiles the CDK application code and generates the assembly in the cloud.
• The Update pipeline stage updates the pipeline (if required).
• The Publish Resource stage uploads the CDK resources to Amazon S3.

Once the published resource is complete, the pipeline deploys the Lambda function in both development and production environments. The architecture includes a manual approval stage for versions destined for the production environment to provide additional control.

Prerequisites

For this tutorial, you should have the following:

1. An AWS account
2. Visual Studio 2022
3. AWS Toolkit for Visual Studio
4. Node.js 18.x or later
5. AWS CDK v2 (2.67.0 or later required)
6. Git

Bootstrapping

Before using AWS CDK to deploy CDK pipelines, you must load the AWS environments where you want to deploy the Lambda feature. The environment is the target AWS account and the region where the stack will be deployed.

In this post, you are deploying the Lambda function in a development environment and optionally in a production environment. This requires both environments to be loaded. However, deployment in the production environment is optional; you can skip loading this environment for now, as the authors will take care of this later.

This is a one-time action per environment for each environment where you want to deploy CDK applications. To start the development environment, run the following command: replace your AWS account ID with your developer account, the region you will use in the development environment, and the locally configured AWS CLI profile you want to use for that account. Please refer to the documentation for additional information.

BASH

cdk bootstrap aws://<DEV-ACCOUNT-ID>/<DEV-REGION> \
    --profile DEV-PROFILE \ 
    --cloudfor

 --profile specifies the AWS CLI credential profile that will be used to organize the environment. If not specified, the default profile will be used. The profile should have sufficient permissions to provide resources to the AWS CDK during the initial organizing process.

--cloudformation-execution-policies specifies the ARN of managed policies that should be attached to the deployment role adopted by AWS CloudFormation when deploying your stacks.

Note: By default, stacks are deployed with full administrator privileges using the Administrator Access policies, but for real-world use, you should define more restrictive IAM policies and use them, customizing bootstrapping in AWS CDK documentation and Secure CDK deployments with IAM permission boundaries to see how to do this.

Create a Git repository in AWS CodeCommit

In this article, you will use CodeCommit to store your source code. First, create a git repository named dotnet-lambda-cdk-pipeline in CodeCommit by following the steps in the CodeCommit documentation.

Once the repository is created, generate git credentials to access it from your local computer if you don't already have them. Follow the steps below to create git credentials.

1. Log in to the AWS management console and open the IAM console.
2. Create an IAM user (for example, git-user).
3. Once the user has been created, attach the AWSCodeCommitPowerUser policy to the user.
4. Next, open the user details page, select the Security Credentials tab, and under HTTPS Git credentials for AWS CodeCommit, select Generate.
5. Download the credentials to retrieve this information as a .CSV file.

Clone the recently created repository to your workstation, then cd to the dotnet-lambda-cdk-pipeline directory.

GIT

git clone 
cd dotnet-lambda-cdk-pipeline

Alternatively, you can use the git-remote-codecommit command to clone the repository using the git clone codecommit::://@ command, replacing the replacement symbols with their original values. Using git-remote-codecommit does not require creating additional IAM users to manage git credentials. To learn more, see the AWS CodeCommit documentation page with git-remote-codecommit.

Initialize the CDK project

At the command line in the dotnet-lambda-cdk-pipeline directory, initialize the AWS CDK project by running the following command.

Git

cdk init app --language csharp

Open the generated C# solution in Visual Studio, right-click the DotnetLambdaCdkPipeline project, and select Properties. Set the target platform to .NET 6.

Create a CDK stack to share the CodePipeline

Your CDK Pipelines application contains at least two stacks: one representing the pipeline itself and one or more stacks representing the applications deployed through the pipeline. In this step, you create the first stack that deploys the CodePipeline pipeline to your AWS account.

In Visual Studio, open the solution by opening the .sln solution file (in the src/ folder). Once the solution is loaded, open the DotnetLambdaCdkPipelineStack.cs file and replace its contents with the following code. Note that the file name, namespace, and class name assume that you have named your Git repository, as shown earlier.

Note: remember to replace the "" in the code below with the name of your CodeCommit repository (this blog post uses dotnet-lambda-cdk-pipeline).

C#

using Amazon.CDK;
using Amazon.CDK.AWS.CodeBuild;
using Amazon.CDK.AWS.CodeCommit;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.Pipelines;
using Constructs;
using System.Collections.Generic;

namespace DotnetLambdaCdkPipeline 
{
    public class DotnetLambdaCdkPipelineStack : Stack
    {
        internal DotnetLambdaCdkPipelineStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
        {
    
            var repository = Repository.FromRepositoryName(this, "repository", "");
    
            // This construct creates a pipeline with 3 stages: Source, Build, and UpdatePipeline
            var pipeline = new CodePipeline(this, "pipeline", new CodePipelineProps
            {
                PipelineName = "LambdaPipeline",
                SelfMutation = true,
    
                // Synth represents a build step that produces the CDK Cloud Assembly.
                // The primary output of this step needs to be the cdk.out directory generated by the cdk synth command.
                Synth = new CodeBuildStep("Synth", new CodeBuildStepProps
                {
                    // The files downloaded from the repository will be placed in the working directory when the script is executed
                    Input = CodePipelineSource.CodeCommit(repository, "master"),
    
                    // Commands to run to generate CDK Cloud Assembly
                    Commands = new string[] { "npm install -g aws-cdk", "cdk synth" },
    
                    // Build environment configuration
                    BuildEnvironment = new BuildEnvironment
                    {
                        BuildImage = LinuxBuildImage.AMAZON_LINUX_2_4,
                        ComputeType = ComputeType.MEDIUM,
    
                        // Specify true to get a privileged container inside the build environment image
                        Privileged = true
                    }
                })
            });
        }
    }
}

In the code above, you are using CodeBuildStep instead of ShellStep because ShellStep does not provide a property to specify BuildEnvironment. You must select a build environment to set a privileged mode that allows access to the Docker daemon to build container images in the build environment. This is necessary to use the CDK linking function, as explained later in this article.

Open the src/DotnetLambdaCdkPipeline/Program.cs file and edit its contents to reflect the following. Remember to replace the placeholders with the AWS account ID and region for the development environment.

C#

using Amazon.CDK;

namespace DotnetLambdaCdkPipeline
{
    sealed class Program
    {
        public static void Main(string[] args)
        {
            var app = new App();
            new DotnetLambdaCdkPipelineStack(app, "DotnetLambdaCdkPipelineStack", new StackProps
            {
                Env = new Amazon.CDK.Environment
                {
                    Account = "",
                    Region = ""
                }
            });
            app.Synth();
        }
    }
}

Note: Instead of validating your account ID and region for source control, you can set and use environment variables in the CodeBuild agent; see the Environment in AWS CDK documentation for more information. Because the CodeBuild agent is also configured in your CDK code, you can use the BuildEnvironmentVariableType property to store environment variables in the AWS Systems Manager Parameter Store or AWS Secrets Manager.

After making changes to your code, compile the solution to ensure there are no compilation issues. Then validate and push out all the changes you have just made. Run the following commands (or use Visual Studio's built-in Git function to commit and push out changes):

Git

git add --all .
git commit -m 'Initial commit'
git push

Then, navigate to the repository's root directory, where the cdk.json file is located, and run the cdk deploy command to deploy the initial version of CodePipeline. Note that deployment may take a few minutes.

The pipeline created by CDK Pipelines mutates itself. This means that to deploy the pipeline, you only need to run cdk once. The pipeline automatically updates itself if you add new CDK applications or steps in the source code.

Once the deployment is complete, a CodePipeline will be created and automatically run. The pipeline includes three stages, as shown below.

• Source - Retrieves the source of your AWS CDK application from the CodeCommit repository and starts the pipeline each time you upload new approvals to it.
• Build - This step compiles your code (if necessary) and executes the cdk synthesizer. The output of this step is a set in the cloud.
• UpdatePipeline - This step runs the cdk Deploy command on the cloud assembly generated in the previous step. It modifies the pipeline if necessary. For example, if you update your code to add a new deployment stage to the application pipeline, the pipeline is automatically updated to reflect the changes.

Define the CodePipeline step to implement the .NET Lambda function

In this step, you create a stack containing a simple Lambda function and place this stack on the stage. You then add the stage to the pipeline to be deployed.

To create a Lambda project, do the following:

1. In Visual Studio, right-click the solution, select Add, and then select New Project.
2. Select the AWS Lambda Project (.NET Core - C#) template in the New Project window, and then select OK or Next.
3. In the Project Name field, type SampleLambda, then select Create.
4. In the Select Blueprint dialog box, select Empty Function, and then choose Finish.

Next, create a new file in the CDK project at src/DotnetLambdaCdkPipeline/SampleLambdaStack.cs to define the application stack containing the Lambda function. Update the file with the following content (adjust the namespace if necessary):

C#

using Amazon.CDK;
using Amazon.CDK.AWS.Lambda;
using Constructs;
using AssetOptions = Amazon.CDK.AWS.S3.Assets.AssetOptions;

namespace DotnetLambdaCdkPipeline 
{
    class SampleLambdaStack: Stack
    {
        public SampleLambdaStack(Construct scope, string id, StackProps props = null) : base(scope, id, props)
        {
            // Commands executed in a AWS CDK pipeline to build, package, and extract a .NET function.
            var buildCommands = new[]
            {
                "cd /asset-input",
                "export DOTNET_CLI_HOME=\"/tmp/DOTNET_CLI_HOME\"",
                "export PATH=\"$PATH:/tmp/DOTNET_CLI_HOME/.dotnet/tools\"",
                "dotnet build",
                "dotnet tool install -g Amazon.Lambda.Tools",
                "dotnet lambda package -o output.zip",
                "unzip -o -d /asset-output output.zip"
            };
                
            new Function(this, "LambdaFunction", new FunctionProps
            {
                Runtime = Runtime.DOTNET_6,
                Handler = "SampleLambda::SampleLambda.Function::FunctionHandler",
    
                // Asset path should point to the folder where .csproj file is present.
                // Also, this path should be relative to cdk.json file.
                Code = Code.FromAsset("./src/SampleLambda", new AssetOptions
                {
                    Bundling = new BundlingOptions
                    {
                        Image = Runtime.DOTNET_6.BundlingImage,
                        Command = new[]
                        {
                            "bash", "-c", string.Join(" && ", buildCommands)
                        }
                    }
                })
            });
        }
    }
}

Building inside a Docker container

The code above uses the bundling function to build the Lambda function inside the docker container. Bundling starts a new docker container, copies the Lambda source code to the container's /asset-input directory, and runs the specified commands that save the bundle files in the /asset-output directory. The files in /asset-output are copied as resources to the kit directory in the stack cloud. At a later stage, these files are packaged and uploaded to S3 as a CDK resource.

Building Lambda functions in Docker containers is better than creating them locally because it reduces host-machine dependencies, resulting in a more consistent and reliable compilation process.

Bundling requires the creation of a docker container on the compilation machine. For this purpose, the setting privileged: true on the build machine has already been configured.

Adding a development stage

Create a new file in the CDK project in src/DotnetLambdaCdkPipeline/DotnetLambdaCdkPipelineStage.cs to hold the stage. This class will create the pipeline development stage.

Edit the src/DotnetLambdaCdkPipeline/DotnetLambdaCdkPipelineStack.cs file to add a stage to the pipeline. Add the bold line from the code below to your file.

Then, the solution will be compiled, and the changes will be validated and released to the CodeCommit repository. This will trigger the start of the CodePipeline.

Once the pipeline is running, the UpdatePipeline stage detects the changes and updates the pipeline based on the code found there. Once the UpdatePipeline stage has been completed, the pipeline is updated with additional stages.

It is time to observe the changes:

1. A resource stage is added. In this stage, all the resources you use in your application are uploaded to Amazon S3 (the S3 tray created during the initial load) to be used in other deployment stages further down the pipeline. For example, the CloudFormation template used in the development stage contains references to these resources, so the resources are first transferred to S3 and then referenced in later stages.
2. A development stage has been added with two actions. The first action is to create a set of changes and the second is to Deploy it.

After the Deploy stage, you can find the newly deployed Lambda function by visiting the Lambda console, selecting "Functions" from the left menu and filtering the list of functions with "LambdaStack". Note: the runtime environment is .NET.

Running unit test cases in CodePipeline

You will then add unit test cases to your Lambda function and run them in a pipeline to generate a test report in CodeBuild.

To create a unit test project, do the following:

1. Right-click the solution, select Add, and then choose New Project.
2. In the New Project dialog box, select the xUnit test project template, and then choose OK or Next.
3. In the Project Name field, type SampleLambda.Tests, and then select Create or Next.
4. Depending on the version of Visual Studio you are using, you may be prompted to select the .NET platform version. Select the .NET 6.0 platform (long-term support), and then choose Create.
5. Right-click on the SampleLambda.For the test project, select Add and then choose Project Reference. Select the SampleLambda project, and then select OK.

Then edit the src/SampleLambda.Tests/UnitTest1.cs file to add the unit test. You can use the code below, which checks that the Lambda function returns the input string as uppercase.

You can add pre-implementation or post-implementation actions to a step by calling its AddPre() or AddPost() method. The authors will use a pre-implementation action to perform the above test cases.

To add a pre-implementation action, edit the src/DotnetLambdaCdkPipeline/DotnetLambdaCdkPipelineStack.cs file in the CDK project after adding the code to generate test reports.

To run unit tests and publish the test report in CodeBuild, construct a BuildSpec for our CodeBuild project. The authors also provide IAM policy statements to attach to the CodeBuild service role, granting it permission to run tests and generate reports. Update the file by adding new code (starting with "// Add this code to test reports") under the devStage statement added earlier:

C#

using Amazon.CDK; 
using Amazon.CDK.Pipelines;
...

namespace DotnetLambdaCdkPipeline 
{
    public class DotnetLambdaCdkPipelineStack : Stack
    {
        internal DotnetLambdaCdkPipelineStack(Construct scope, string id, IStackProps props = null) : base(scope, id, props)
        {
            // ...
            // ...
            // ...
            var devStage = pipeline.AddStage(new DotnetLambdaCdkPipelineStage(this, "Development"));
            
            
            
            // Add this code for test reports
            var reportGroup = new ReportGroup(this, "TestReports", new ReportGroupProps
            {
                ReportGroupName = "TestReports"
            });
           
            // Policy statements for CodeBuild Project Role
            var policyProps = new PolicyStatementProps()
            {
                Actions = new string[] {
                    "codebuild:CreateReportGroup",
                    "codebuild:CreateReport",
                    "codebuild:UpdateReport",
                    "codebuild:BatchPutTestCases"
                },
                Effect = Effect.ALLOW,
                Resources = new string[] { reportGroup.ReportGroupArn }
            };
            
            // PartialBuildSpec in AWS CDK for C# can be created using Dictionary
            var reports = new Dictionary<string, object>()
            {
                {
                    "reports", new Dictionary<string, object>()
                    {
                        {
                            reportGroup.ReportGroupArn, new Dictionary<string,object>()
                            {
                                { "file-format", "VisualStudioTrx" },
                                { "files", "**/*" },
                                { "base-directory", "./testresults" }
                            }
                        }
                    }
                }
            };
            // End of new code block
        }
    }
}

Finally, add CodeBuildStep as a pre-deployment action to the development stage with the necessary CodeBuildStepProps to configure the reports. Add this after the new code added above.

C#

devStage.AddPre(new Step[]
{
    new CodeBuildStep("Unit Test", new CodeBuildStepProps
    {
        Commands= new string[]
        {
            "dotnet test -c Release ./src/SampleLambda.Tests/SampleLambda.Tests.csproj --logger trx --results-directory ./testresults",
        },
        PrimaryOutputDirectory = "./testresults",
        PartialBuildSpec= BuildSpec.FromObject(reports),
        RolePolicyStatements = new PolicyStatement[] { new PolicyStatement(policyProps) },
        BuildEnvironment = new BuildEnvironment
        {
            BuildImage = LinuxBuildImage.AMAZON_LINUX_2_4,
            ComputeType = ComputeType.MEDIUM
        }
    })
});

Compile the solution, then validate and push the changes to the repository. Pushing the changes triggers the pipeline, runs the test cases, and publishes a report in the CodeBuild console. Once the pipeline is complete, navigate to TestReports in the CodeBuild report groups to view the report, as shown below.

Deploy in a production environment with manual validation

CDK Pipelines make it very easy to deploy additional stages with different accounts. You must initialize the accounts and regions where you want to deploy, and they must have a trust relationship added to the pipeline account.

To load an additional production environment where AWS CDK applications will be deployed through the pipeline, run the following command, substituting the AWS account ID for the production account, the region you will use in the production environment, the AWS CLI profile to use with the prod account and the AWS account ID where the pipeline is already deployed (the account you initialized at the beginning of this article).

Bash

cdk bootstrap aws://<PROD-ACCOUNT-ID>/<PROD-REGION>
    --profile <PROD-PROFILE> \
    --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess \
    --trust <PIPELINE-ACCOUNT-ID>

The—-trust option indicates which other account should have permission to deploy AWS CDK applications in this environment. For this option, specify the pipeline's AWS account ID.

Use the code below to add a new production deployment step with manual approval. Add this code under the "devStage.AddPre(...)" code block added in the previous section, and remember to replace the placeholders with the AWS account ID and region for the production environment.

C#

var prodStage = pipeline.AddStage(new DotnetLambdaCdkPipelineStage(this, "Production", new StageProps
{
    Env = new Environment
    {
        Account = "",
        Region = ""
    }
}), new AddStageOpts
{
    Pre = new[] { new ManualApprovalStep("PromoteToProd") }
});

To support deploying a CDK application to another account, the artifact trays must be encrypted, so add the CrossAccountKeys property to the CodePipeline at the top of the pipeline stack file and set the value to true (see the bold line in the code snippet below). This will create a KMS key for the artifact tray, allowing multiple accounts to be implemented.

C#

var pipeline = new CodePipeline(this, "pipeline", new CodePipelineProps
{
   PipelineName = "LambdaPipeline",
   SelfMutation = true,
   CrossAccountKeys = true,
   EnableKeyRotation = true, //Enable KMS key rotation for the generated KMS keys
   
   // ...
}

Once the changes have been approved and pushed out to the repository, a new manual approval step called PromoteToProd will be added to the production stage of the pipeline. The pipeline stops at this stage and waits for manual approval, as shown in the screenshot below.

When you click the Review button, the following dialogue box will appear. Here, you can approve or reject and add comments if necessary.

Once approved, the pipeline resumes, the remaining steps are performed, and the deployment is completed in the production environment.

Ordering

To avoid future charges, log into the AWS console of the various accounts you used, navigate to the AWS CloudFormation console of the regions where you selected the deployment, select and click Delete on the stacks created for this activity, and then delete them. Alternatively, you can delete the CloudFormation stacks using the destroy cdk command. This will not delete the CDKToolkit stack created by the initial load command. If you want to delete this, you can do so from the AWS console.

Conclusions

This article teaches you how to use CDK pipelines to automate the deployment of .NET Lambda functions. The intuitive and flexible architecture makes it easy to set up a CI/CD pipeline covering the entire application lifecycle, from compilation and testing to deployment. With CDK Pipelines, you can streamline your development workflow, reduce errors, and ensure consistent and reliable deployments.

For more information on CDK pipelines and all the ways you can use them, see the CDK Pipelines reference documentation.