Terraform Jenkins AWS ECS Fargate

Terraform stack to deploy Jenkins on ECS Fargate with Jenkins configuration stored in EFS and agents on Fargate. It can be used as a starting point to build a production ready Jenkins on AWS.

More details can be found on my blog post (in French).

How it works

It uses a docker image based on the official Jenkins. See docker/ folder.

The following main resources will be created:

• An application load balancer in front of Jenkins.
• A network load balancer for Agent -> Controller communication. For more information about how Controller <-> Agents
  communication works, see this page.
• An EFS to store Jenkins configuration.
• An S3 bucket used by the Jenkins Configuration as code plugin to get the configuration generated by Terraform.
• Two log groups for the Controller and agents logs

Prerequisites

• A VPC with public and private subnets configured properly (route table, nat gateways...)
• An IAM user with the proper policies to run Terraform on the following services: EC2, ECS, IAM, S3, Cloudwatch, EFS,
  Route53 et ACM.
• A recent version of Terraform ( > 0.12.20)

The only required Terraform variables are:

• vpc_id : the VPC ID
• public_subnets : public subnets IDs
• private_subnets : private subnets IDs

See variables.tf for all the possible variables to override.

AWS authentication:

export AWS_PROFILE=...
# or
export AWS_SECRET_ACCESS_KEY=""
export AWS_ACCESS_KEY_ID=""

Deployment:

export TF_VAR_vpc_id="vpc-123456789"
export TF_VAR_private_subnets='["private-subnet-a", "private-subnet-b", "private-subnet-c"]'
export TF_VAR_public_subnets='["public-subnet-a", "public--subnet-b", "public-subnet-c"]'
terraform init
terraform apply

Get the default admin credentials and connect to the controller url in the output jenkins_public_url:

terraform output jenkins_credentials

The first time you access the controller, the Getting started guide will ask you to install the recommended plugins. Install them and restart the controller.

Faster startup of tasks with SOCI (Optional)

To speed up the startup of the controller and the agents, you can use the SOCI feature (Seekable OCI image config) by setting the input variable soci.enabled to true (see below for more details about the input variables).

This requires a recent version of the Fargate platform (>= 1.4.0). When you set this to true, the index builder will build the SOCI indexes locally and push them to ECR. This can take a while (around ~5 minutes) and requires Docker to be installed on your machine and be able to run it in privileged mode.

To compare the startup time of the tasks, a local module modules/ecs-events-capture is used to capture the relevant ECS Task events in a CloudWatch Log Group. After some runs, you can run the Python script (check the README of the module).

Here are some numbers in the tables below for the controller and the agent with the following images built from here:

• Controller version 2.433: 1.12 GB
• Agent version 3192.v713e3b_039fb_e-4-alpine-jdk17: 0.315 GB
• The times are in seconds and represent the difference between the creation and the start time of the task.
• Number of runs: 16.

The controller:

The agent:

Note that SOCI only works with the private ECR repositories at the moment.

In a nutshell, on average, the start time of the controller and the agent are reduced by 36.5% and 19.4% respectively.

For more information about SOCI, see the following links:

• Under the hood: Lazy Loading Container Images with Seekable OCI and AWS Fargate
• https://docs.aws.amazon.com/AmazonECS/latest/userguide/container-considerations.html
• AWS Fargate Enables Faster Container Startup using Seekable OCI

Docs

Requirements

Providers

Modules

Resources

Inputs

Outputs

References:

• Jenkins Master official docker image
• Jenkins Agent official docker image
• EFS support in fargate
• EFS IAM Authorization and access point
• https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-nfs-permissions.html
• Jenkins's architecture for scale