KF_EnvOrchestatorTool
This tool is used to serve an automated environment in local or cloud platform to easily develop applications monolith or any architecture, it doesn't matter.
Index
Systems
- Jenkins set up with docker-compose
- Automate Jenkins implantation on AWS with Terraform
- Automate Jenkins jobs with Python
- Execute local Jenkins with preconfigured plugins with a custom image on Docker Hub with Docker Compose.
- Github Actions to CI/CD with Jenkins.
- Second service to initialize Jenkins with Python and Shellscript.
Getting Started
Local installation
Configure Jenkins
Config Jenkins on your local machine:
- Initial password located at
/var/jenkins_home/secrets/initialAdminPassword
- Execute
docker-compsoe.yml
from .docker/local
folder.
- Go to
localhost:8080
and follow the instructions to configure Jenkins. (Create an initialAdminPassword)
- Install Git plugin for Jenkins. (This is necessary to trigger pipelines with SCM option enabled)
Configure Environment Variables to execute main.py
- Create a
local.env
file at .env
folder.
Add the following variables:
JENKINS_URL=<YOUR_JENKINS_URL>
JENKINS_USER=<YOUR_JENKINS_USER>
JENKINS_PASS=<YOUR_JENKINS_PASSWORD>
ACCESS_TOKEN=<YOUR_GITHUB_ACCESS_TOKEN>
Configure SSH
- Install OpenSSH Server on your local machine.
- Start the service.
# Windows
Start-Service sshd
- (Optional) Try to connect from container to your local machine with SSH.
docker exec -it jenkins-git bash
ssh -i /var/jenkins_home/.ssh/id_rsa [email protected] -vvv
AWS Configuration
Implantation of Jenkins automated with Terraform on AWS.
Requirements:
- Create your AWS account.
- Create your Access Key in the Security Credentials section.
- Take an AMI valid for your region.
- Configure SSH key pair in your AWS account for EC2 instances.
- Configure VPC.
- Configure Subnet.
- Configure AWS CLI with your credentials:
aws configure
# AWS Access Key ID [None]: YOUR_ACCESS_KEY
# AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY
- Go to AMI Catalog and take an AMI ID for your region.
Put your AMI ID in main.tf
file.
- Configure your SSH key pair in
main.tf
file.
aws ec2 create-key-pair --key-name my-ssh-key --query 'KeyMaterial' --output text > my-ssh-key.pem
Trigger Terraform pipeline
Project has different .tf files decoupled by behaviour. Terraform will treat all files as an unique project.
- Init Terraform:
terraform init
- Plan Terraform:
terraform plan
- Apply Terraform:
terraform apply
- Destroy Terraform:
terraform destroy
Connect to EC2 instance
Here we've different ways to connect to EC2 instance:
- Using SSH command:
# Create your SSH key pair previously in the EC2 AWS section.
ssh -i my-ssh-key.pem ec2-user@YOUR_EC2_PUBLIC_IP
Configuration
In this project, a Jenkins docker image was built and pushed to Docker Hub to store a basic archetypal Jenkins config
with Git, Docker and Pipeline plugins.
You can pulled it from Docker Hub with:
docker pull kolmanfreecss/jenkins-git
(Process to build the image and push it to Docker Hub)
- Commit the current status container
docker commit YOUR_CONTAINER_ID kolmanfreecss/jenkins-git
- Login to Docker Hub
docker login
- Push the image to Docker Hub
docker push kolmanfreecss/jenkins-git
Dependencies
- Jenkins API
- Plugins:
- Git (Check Configuration section)
- Pipeline (Check Configuration section)
- Docker (Check Configuration section)
Tech stacks CI/CD
- Jenkins
- Docker & Docker Compose
- AWS
- Python
- Shellscript
- Terraform
Tips
- Check Event Viewer on Windows to see if SSH Server is running properly.
Applications and Services Logs > OpenSSH > Operational
- Remember that Jenkins needs SSH private key and the local machine needs its public key to validate the connection stored in
the authorized_keys
file.
Troubleshoting
Jenkins
- Script to install Jenkins not working properly.
- Alternative Solution: Connect through SSH to the EC2 instance and install Jenkins
manually. (https://mirrors.jenkins.io/redhat-stable/)
- After that connect to the IPv4 Public EC2 instance with HTTP protocol and port 8080.
- Example: http://YOUR_EC2_PUBLIC_IP:8080
- Check EC2 system log from AWS section to see if Jenkins is running properly or installed.
- BIG Problems installing plugins https://community.jenkins.io/t/issue-while-upgrading-plugins-on-latest-jenkins/9846
- It takes its time to start even if the instance is running. Be patient. :)
- Check logs with
aws ec2 get-console-output --instance-id YOUR_INSTANCE_ID --output text
Scripts
- Use
dos2unix
to convert the scripts to Unix format.
- Create SSH credentials on Jenkins through Python with Jenkins API. Problem with the XML tag using incorrect format for the implementation
- Solution: Use
com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey
AWS
- Check SSH key permissions to connect to EC2 instance.
chmod 400 my-ssh-key.pem
- Remove permissions to other group users or another users because AWS won't let you connect to the EC2 instance if
the permissions are too permissive.
Settings
- Install SSH Server on local machine.
- If you have a Windows Server you can follow Microsoft official documentation to install OpenSSH Server. Or check https://github.com/PowerShell/Win32-OpenSSH/releases
- Also you could use WSL to install OpenSSH Server.
- Configure permissions to the id_rsa file to not be too permissive.
- Create an authorized_keys file in the .ssh folder with the public key of the local machine.
- Create an sshd_config file in the .ssh folder with the following IMPORTANT configurations UNCCOMMENTED.:
Shield:
This work is licensed under
a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.