Terraform Enterprise HVD on AWS EC2

Terraform module aligned with HashiCorp Validated Designs (HVD) to deploy Terraform Enterprise (TFE) on Amazon Web Services (AWS) using EC2 instances with a container runtime. This module defaults to deploying TFE in the active-active operational mode, but external is also supported. Docker and Podman are the supported container runtimes.

Prerequisites

General

• TFE license file (e.g. terraform.hclic)
• Terraform CLI >= 1.9 installed on clients/workstations that will be used to deploy TFE
• General understanding of how to use Terraform (Community Edition)
• General understanding of how to use AWS
• git CLI and Visual Studio Code editor installed on workstations are strongly recommended
• AWS account that TFE will be deployed in with permissions to provision these resources via Terraform CLI
• (Optional) AWS S3 bucket for S3 remote state backend that will be used to manage the Terraform state of this TFE deployment (out-of-band from the TFE application) via Terraform CLI (Community Edition)

Networking

• AWS VPC ID and the following subnets:
  • Load balancer subnet IDs (can be the same as EC2 subnets if desirable)
  • EC2 (compute) subnet IDs
  • RDS (database) subnet IDs
  • Redis subnet IDs (can be the same as RDS subnets if desirable)
• (Optional) S3 VPC endpoint configured within VPC
• (Optional) AWS Route53 hosted zone for TFE DNS record creation
• Chosen fully qualified domain name (FQDN) for your TFE instance (e.g. tfe.aws.example.com)

📝 Note: It is recommended to specify a minimum of two subnets for each subnet input to enable high availability.

Security groups

• This module will automatically create the necessary security groups and attach them to the applicable resources
• Identify CIDR range(s) that will need to access the TFE application (managed via cidr_allow_ingress_tfe_443 input variable)
• Identify CIDR range(s) that will need to access the shell of the TFE EC2 instances (managed via cidr_allow_ingress_ec2_ssh input variable)
• Be familiar with the TFE ingress requirements
• Be familiar with the TFE egress requirements

TLS certificates

• TLS certificate (e.g. cert.pem) and private key (e.g. privkey.pem) that matches your chosen fully qualified domain name (FQDN) for TFE
  • TLS certificate and private key must be in PEM format
  • Private key must not be password protected
• TLS certificate authority (CA) bundle (e.g. ca_bundle.pem) corresponding with the CA that issues your TFE TLS certificates
  • CA bundle must be in PEM format
  • You may include additional certificate chains corresponding to external systems that TFE will make outbound connections to (e.g. your self-hosted VCS, if its certificate was issued by a different CA than your TFE certificate).

📝 Note: All three of these files will be created as secrets in AWS Secrets Manager per the next section.

Secrets management

The following bootstrap secrets stored in AWS Secrets Manager in order to bootstrap the TFE deployment and installation:

• TFE license file - raw contents of license file stored as a plaintext secret (e.g. cat terraform.hclic)
• TFE encryption password - random characters stored as a plaintext secret (used to protect internally-managed Vault unseal key and root token)
• TFE database password - used to create RDS Aurora (PostgreSQL) database cluster; random characters stored as a plaintext secret; value must be between 8 and 128 characters long and must not contain @, ", or / characters
• TFE Redis password - used to create Redis (Elasticache Replication Group) cluster; random characters stored as a plaintext secret; value must be between 16 and 128 characters long and must not contain @, ", or / characters
• TFE TLS certificate - file in PEM format, base64-encoded into a string, and stored as a plaintext secret
• TFE TLS certificate private key - file in PEM format, base64-encoded into a string, and stored as a plaintext secret
• TFE TLS CA bundle - file in PEM format , base64-encoded into a string, and stored as a plaintext secret

📝 Note: See the TFE Bootstrap Secrets doc for more details on how these secrets should be stored in AWS Secrets Manager.

Compute

Connecting to shell of EC2 instances

One of the following mechanisms for shell access to TFE EC2 instances:

• EC2 SSH key pair
• AWS SSM (can be enabled by setting ec2_allow_ssm boolean input variable to true)

Log forwarding (optional)

One of the following logging destinations:

• AWS CloudWatch log group
• AWS S3 bucket
• A custom fluent bit configuration that will forward logs to custom destination

Usage

1. Create/configure/validate the applicable prerequisites.

2. Nested within the examples directory are subdirectories containing ready-made Terraform configurations for example scenarios on how to call and deploy this module. To get started, choose the example scenario that most closely matches your requirements. You can customize your deployment later by adding additional module inputs as you see fit (see the Deployment-Customizations doc for more details).

3. Copy all of the Terraform files from your example scenario of choice into a new destination directory to create your Terraform configuration that will manage your TFE deployment. This is a common directory structure for managing multiple TFE deployments:

   .
   └── environments
       ├── production
       │   ├── backend.tf
       │   ├── main.tf
       │   ├── outputs.tf
       │   ├── terraform.tfvars
       │   └── variables.tf
       └── sandbox
           ├── backend.tf
           ├── main.tf
           ├── outputs.tf
           ├── terraform.tfvars
           └── variables.tf
   

   📝 Note: In this example, the user will have two separate TFE deployments; one for their sandbox environment, and one for their production environment. This is recommended, but not required.

4. (Optional) Uncomment and update the S3 remote state backend configuration provided in the backend.tf file with your own custom values. While this step is highly recommended, it is technically not required to use a remote backend config for your TFE deployment.

5. Populate your own custom values into the terraform.tfvars.example file that was provided (in particular, values enclosed in the <> characters). Then, remove the .example file extension such that the file is now named terraform.tfvars.

6. Navigate to the directory of your newly created Terraform configuration for your TFE deployment, and run terraform init, terraform plan, and terraform apply.

7. After your terraform apply finishes successfully, you can monitor the installation progress by connecting to your TFE EC2 instance shell via SSH or AWS SSM and observing the cloud-init (user_data) logs:

   Connecting to EC2 instance

   SSH when ec2_os_distro is ubuntu:

   ssh -i /path/to/ec2_ssh_key_pair.pem ubuntu@<ec2-private-ip>
   

   SSH when ec2_os_distro is rhel or al2023:

   ssh -i /path/to/ec2_ssh_key_pair.pem ec2-user@<ec2-private-ip>
   

   Viewing the logs

   View the higher-level logs:

   tail -f /var/log/tfe-cloud-init.log
   

   View the lower-level logs:

   journalctl -xu cloud-final -f
   

   📝 Note: The -f argument is to follow the logs as they append in real-time, and is optional. You may remove the -f for a static view.

   Successful install log message

   The log files should display the following log message after the cloud-init (user_data) script finishes successfully:

   [INFO] tfe_user_data script finished successfully!
   

8. After the cloud-init (user_data) script finishes successfully, while still connected to the TFE EC2 instance shell, you can check the health status of TFE:

   cd /etc/tfe
   sudo docker compose exec tfe tfe-health-check-status
   

9. Follow the steps to here to create the TFE initial admin user.

Docs

Below are links to various docs related to the customization and management of your TFE deployment:

• Deployment Customizations
• TFE Version Upgrades
• TFE TLS Certificate Rotation
• TFE Configuration Settings
• TFE Bootstrap Secrets

Requirements

Providers

Resources

Inputs

Outputs