Using HashiCorp Boundary, Vault, and Consul with Amazon ECS
This example demonstrates how HashiCorp tools run on AWS, including:
It uses the following AWS services:
To run this example, you need Terraform Cloud to set up a series of workspaces. The workspaces need to be set up as follows, with the appropriate working directory, secrets, and remote workspace sharing.
Workspace Name | Working Directory for VCS | Variables | Remote State Sharing |
---|---|---|---|
hcp | hcp/ |
name, trusted_role_arn, bootstrap AWS access keys, HCP credentials | infrastructure, consul, boundary, vault-aws |
vault-aws | vault/aws/ |
name, AWS access keys (for AWS secrets engine) | |
infrastructure | infrastructure/ |
name, client_cidr_block, HCP service principal credentials, database_password, boundary_database_password, key_pair_name. [FROM VAULT] AWS access keys | boundary, apps, vault-products |
vault-products | vault/products/ |
name, HCP service principal credentials | boundary |
boundary | boundary/ |
name. [FROM VAULT] db_password, db_username, AWS access keys | |
apps | apps/ |
name, client_cidr_block. [FROM VAULT] db_password, db_username, AWS access keys |
You need to run plan and apply for each workspace in the order indicated.
Imagine you want to issue AWS access keys for each group that runs Terraform. You can use Vault's AWS secrets engine to generate access keys for each group.
For example, you set up an initial AWS access and secret key for Vault to issue new credentials. The AWS access and secret key assume a role with sufficient permissions for Terraform to configure infrastructure on AWS.
Run terraform apply
for the hcp
workspace. It creates:
Sets the Vault address, token, and namespace for you to get a new set of AWS access keys from Vault in your CLI.
source set.sh
Next, generate a set of AWS access keys for the Vault secrets engine. These should be different than the ones you used to bootstrap HCP and the AWS IAM role!
Add the new AWS access keys to vault-aws
workspace.
Run terraform apply
for the vault-aws
workspace. It creates:
terraform/aws
hashicups
)Run make vault-aws
. This retrieves a new set of AWS access keys from Vault via
the secrets engine and saves it to the secrets/
directory locally.
make vault-aws
Use the AWS access and secret keys from secrets/aws.json
and add them to the
infrastructure
, boundary
, and apps
workspaces.
Run terraform apply
for the infrastructure
workspace. It creates:
We need to generate a few things for the products API (and Boundary).
product-api
and Boundary)vault-agent
in HashiCups product-api
To configure this, you need to add HCP service credentials with the Vault address, token, and
namespace to vault-products
.
You have two identities that need to access the application's database:
product-api
) to read from the databaseops
or dev
team) to update the database using BoundaryConfigure the following.
terraform apply
for the vault-products
workspace. It creates:
hashicups/database
product
)boundary
)Boundary needs a set of organizations and projects. You have two projects:
core_infra
: ECS container instance. Allow ops
team to SSH into it.product_infra
: Application database. Allow ops
or dev
team to configure it.Configure the following.
terraform apply
for the boundary
workspace. It creates:
core_infra
and the other for product_infra
.jeff
for the ops
team, rosemary
for the dev
team,taylor
for the security
team.Run source set.sh
to set your Boundary address.
Run make boundary-host-catalog
to configure the host catalog for the ECS container instances.
This uses dynamic host catalog plugins in Boundary to auto-discover AWS EC2 instances with the cluster
tag.
You can also SSH into the ECS container instance as the ops
team. Run make ssh-ecs
.
make configure-db
to log into Boundary as the dev
team and configure the database -You may need to control network policy between services on ECS and other services registered to Consul. You can use intentions to secure service-to-service communication.
Run terraform apply
for the apps
workspace. It creates three ECS services:
frontend
(Fargate launch type)public-api
(Fargate launch type)product-api
(EC2 launch type)Run terraform apply
for the vault-products
workspace. It adds:
Run make products
to mark the product-api
to be recreated.
Run terraform apply
for the apps
workspace. It should redeploy the product-api
.
Try to access the frontend via the ALB. You might get an error! We need to enable traffic between the services registered to Consul.
Try to access the frontend via the ALB. You'll get a Packer Spiced Latte
!