Security solution to find secrets in a git repository using Gitleaks, generate a JSON report based on the findings from Gitleaks by extracting only the relevant information, find the commit id and commit author for each finding, update an Atlassian Confluence page with the secrets found based on that generated report and send an alert on Slack.
APACHE-2.0 License
A security solution that finds secrets in a git repository using Gitleaks, generates a JSON report based on the findings from Gitleaks by extracting only the relevant information, finds the commit id and commit author for each finding, updates an Atlassian Confluence page with the secrets found based on that generated report and finally sends an alert on Slack.
❓ Where I can run this?
This solution can be executed on any macOS or Linux system either locally or on a remote server. It can also be executed on a CI/CD tool like on GitHub Actions, GitLab CI, etc, in a pipeline.
Below you can find an example of the JSON report generated:
[
{
"Description": "Detected a Generic API Key, potentially exposing access to various services and sensitive operations.",
"File": "scripts/main.py",
"Line No.": "11",
"Link": "https://gitlab.com/my-projects/my-repo/-/blob/master/scripts/main.py#L11",
"Secret Type": "generic-api-key",
"Commit": "__REDACTED__",
"Author": "__REDACTED__"
},
{
"Description": "Identified a HashiCorp Terraform password field, risking unauthorized infrastructure configuration and security breaches.",
"File": "configurations/main.tf",
"Line No.": "6",
"Link": "https://gitlab.com/my-projects/my-repo/-/blob/master/configurations/main.tf#L6",
"Secret Type": "hashicorp-tf-password",
"Commit": "__REDACTED__",
"Author": "__REDACTED__"
}
...
]
Note: In the actual execution, you will see the actual values instead of __REDACTED__
values.
Below you can find an example of the Slack notification messages in case of both no secrets found and 1 or more secrets found:
Following are the prerequisites to be met once before you begin:
./installation/linux_install_packages.sh
script or by installing them manually:
git
jq
bash
make
wget
python3
py3-pip
golang
gitleaks
atlassian-python-api
pip
pytz
pip
requests
pip
./installation/macos_install_packages.sh
script or by installing them manually:
git
jq
bash
python
python@3
gitleaks
atlassian-python-api
pip
pytz
pip
requests
pip
Once all the prerequisites are met, set the following environment variables:
LOCAL_PATH_TO_GIT_REPO
/Users/Abdullah.Khawer/Desktop/my-projects/my-repo
REMOTE_PATH_TO_GIT_REPO
https://gitlab.com/my-projects/my-repo
BRANCH_NAME
/Users/Abdullah.Khawer/Desktop/myrepo
CONFLUENCE_ENABLED
1
1
or 0
CONFLUENCE_SITE
https://mydomain.atlassian.net
CONFLUENCE_ENABLED
is set to 1
)CONFLUENCE_USER_EMAIL_ID
[email protected]
CONFLUENCE_ENABLED
is set to 1
)CONFLUENCE_USER_TOKEN
CONFLUENCE_ENABLED
is set to 1
)CONFLUENCE_PAGE_TITLE
Secrets Detected in the Git Repositories
CONFLUENCE_ENABLED
is set to 1
)CONFLUENCE_PAGE_SPACE
docs
CONFLUENCE_ENABLED
is set to 1
)SLACK_ENABLED
1
1
or 0
SLACK_WEBHOOK_URL
[https://mydomain.atlassian.net](https://hooks.slack.com/services/__REDACTED__/__REDACTED__/__REDACTED__)
SLACK_ENABLED
is set to 1
)And then simply run the following 2 commands:
bash gitleaks.sh
python3 main.py TIME_ZONE REPOSITORY_NAME BRANCH_NAME [JSON_REPORT_URL]
python3 main.py Europe/Amsterdam my-projects/my-repo master
In order to run it on any GitHub repository, add the following in the .github-workflow.yml
file under the .github/workflows/
directory in the repository:
name: find-and-report-secrets-in-code
on:
push:
branches:
- master
jobs:
find-and-report-secrets-in-code:
uses: abdullahkhawer/find-and-report-secrets-in-code/.github/workflows/.github-workflow.yml@master
with:
CONFLUENCE_ENABLED: "1"
CONFLUENCE_PAGE_TITLE: ${{ vars.CONFLUENCE_PAGE_TITLE }}
CONFLUENCE_PAGE_SPACE: ${{ vars.CONFLUENCE_PAGE_SPACE }}
SLACK_ENABLED: "1"
secrets:
CONFLUENCE_SITE: ${{ secrets.CONFLUENCE_SITE }}
CONFLUENCE_USER_EMAIL_ID: ${{ secrets.CONFLUENCE_USER_EMAIL_ID }}
CONFLUENCE_USER_TOKEN: ${{ secrets.CONFLUENCE_USER_TOKEN }}
SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}
In the on
section, you specify events can cause the workflow to run. In the above example, the job is only allowed to execute if something is pushed to the master
branch.
The variables referred using $
are supposed to be created on the repository under Repository secrets
and Repository variables
depending on the type of variable from here: Settings > Security > Secrets and variables > Actions
.
In order to run it on any GitLab repository, add the following in the .gitlab-ci.yml
file on root level in the repository:
include:
- remote: 'https://raw.githubusercontent.com/abdullahkhawer/find-and-report-secrets-in-code/master/.gitlab/.gitlab-ci.yml'
stages:
- scan
find-and-report-secrets-in-code:
stage: scan
extends:
- .find-secrets:scan
variables:
CONFLUENCE_ENABLED: "1"
CONFLUENCE_SITE: $CONFLUENCE_SITE
CONFLUENCE_USER_EMAIL_ID: $CONFLUENCE_USER_EMAIL_ID
CONFLUENCE_USER_TOKEN: $CONFLUENCE_USER_TOKEN
CONFLUENCE_PAGE_TITLE: $CONFLUENCE_PAGE_TITLE
CONFLUENCE_PAGE_SPACE: $CONFLUENCE_PAGE_SPACE
SLACK_ENABLED: "1"
SLACK_WEBHOOK_URL: $SLACK_WEBHOOK_URL
retry:
max: 2
rules:
- if: $CI_PIPELINE_SOURCE == "schedule" && $CI_COMMIT_REF_NAME == "master"
when: always
allow_failure: false
In the rules
section, you specify rules for execution as if
conditions. In the above example, the job is only allowed to execute if it is a scheduled job for the master
branch.
The variables referred using $
are supposed to be created on the repository under CI/CD Variables
from here: Settings > CI/CD > Variables
.
The Docker image used is built using the Dockerfile that is present in this repository here: Dockerfile
Following build command is used on the root level in the GitHub repository: docker buildx build --platform linux/amd64 -t "abdullahkhawer/find-and-report-secrets-in-code:latest" --no-cache -f ./docker/Dockerfile .
The image used is publicly available here: Docker - find-and-report-secrets-in-code
For more details, check out its README.
.gitleaks.toml
View
and Add
permissions in the space on it and the Can edit
permission on the page in that space. Also, you need to create an API token as the password won't work.This project is licensed under the Apache License - see the LICENSE file for details.