A GitHub App built with Probot that provide more flexible GitHub Actions workflow for monorepo repositories.
Currently, GitHub Actions does not support monorepo repositories natively.
You can define workflows in .github/workflows
directory, but if you have a monorepo repository, you might want to run different workflows for different subdirectories.
This can be achieved by using paths
filter in the workflow definition, but it is not very flexible and can be hard to maintain.
/validate param=value
).gha.yaml
files by adding label gha-conductor:load
to PRpr-satus
GitHub checkpr-status
check to branch protection rules).gha.yaml
files w/o merge into main branch by open PR with changes.gha.yaml
files against JSON schema and check name uniqueness with error messages in PR diff.github/gha-conductor-config.yaml
This app provides a way to define which workflows should be run for each event. During the workflow run, the app will create corresponding GitHub checks.
Currently, it supports the following events:
Event | GitHub check name | Description |
---|---|---|
onPullRequest |
pr-status |
opened , rereopened , synchronize - when a pull request is opened, reopened or synchronized |
onBranchMerge |
pr-merge |
merged - when a branch is merged into another branch |
onPullRequestClose |
pr-close |
closed - when a pull request is closed and not merged |
onSlashCommand |
pr-slash-command |
issue_comment.created , issue_comment.edited - when a comment with slash command is created or edited |
It uses .gha.yaml
files to define which workflows should be run for each event.
Json schema for .gha.yaml
files can be found in schemas/gha_yaml_schema.json
directory.
Example of .gha.yaml
file:
moduleName: example-c
teamNamespace: domain-b
sharedParams:
ROOT_DIR: "namespaces/domain-b/projects/example-c"
defaultFileChangeTrigger: &defaultFileChangeTrigger
- "namespaces/domain-b/projects/example-c/**"
onPullRequest:
- name: build
pipelineRef:
name: common-job
pipelineRunValues:
params:
COMMAND: make build
triggerConditions:
fileChangesMatchAny: *defaultFileChangeTrigger
- name: test
pipelineRef:
name: common-job
pipelineRunValues:
params:
COMMAND: make test
triggerConditions:
fileChangesMatchAny:
- "namespaces/domain-b/projects/example-c/tests/test.sh"
onBranchMerge:
- name: release
pipelineRef:
name: common-job
pipelineRunValues:
params:
COMMAND: >-
make release
triggerConditions:
destinationBranchMatchesAny:
- 'main'
fileChangesMatchAny: *defaultFileChangeTrigger
onPullRequestClose:
- name: cleanup
pipelineRef:
name: common-job
pipelineRunValues:
params:
COMMAND: >-
make clean
triggerConditions:
destinationBranchMatchesAny:
- 'main'
fileChangesMatchAny: *defaultFileChangeTrigger
onSlashCommand:
- name: validate-before-merge
pipelineRef:
name: generic-job
pipelineRunValues:
params:
COMMAND: make ${command} ${args} # ${command} and ${args} will be replaced with values from the comment slash command
triggerConditions:
slashCommands:
- "validate"
fileChangesMatchAny: *defaultFileChangeTrigger
Files can be places in any directory in the repository.
App uses worfklow_dispatch
event to trigger GitHub Actions workflows.
GitHub Actions workflows should be defined in .github/workflows
directory and should have workflow_dispatch
trigger.
Example of GitHub Actions workflow:
name: "Common job"
run-name: "${{ inputs.PIPELINE_NAME }}"
on:
workflow_dispatch: # This is required to be able to trigger the workflow from the app
inputs: # extra inputs can be added to the workflow and will be provided by the app from context, params or sharedParams
PIPELINE_NAME: # Required to be able to differentiate between jobs
required: true
SERIALIZED_VARIABLES: # workaround the 10 input limit by serializing the variables into a JSON string
required: true
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
jobs:
Execute_Task:
name: "${{ github.event.inputs.PIPELINE_NAME }}" # Required to be able to differentiate between jobs
runs-on: ubuntu-latest
timeout-minutes: 5 # This is the maximum time the job can run for
env:
SERIALIZED_VARIABLES: ${{ github.event.inputs.SERIALIZED_VARIABLES }}
steps:
- name: Load Serialized Variables
run: |
variables=$(echo $SERIALIZED_VARIABLES | jq -r 'to_entries|map("\(.key)=\(.value|tostring)")|.[]')
while IFS= read -r line; do
echo "$line" >> $GITHUB_ENV
done <<< "$variables"
- name: Check out Code
uses: actions/checkout@v4
with:
fetch-depth: 0 # We need to fetch all history so that we can checkout the PR merge commit
# We check out github.event.pull_request.merge_commit_sha
# to ensure we are testing the exact code that will be merged into the base branch
ref: ${{ env.PR_MERGE_SHA }} # Provided via SERIALIZED_VARIABLES
- name: Execute Task
env:
USER_HOME: ${{ github.workspace }}
working-directory: ${{ env.ROOT_DIR }}
run: ${{ env.COMMAND }}
.gha.yaml
files in your repository.github/workflows
directorygha-conductor:load
to trigger the app to load all .gha.yaml
files and create corresponding hooks in the database.gha.yaml
files.gha.yaml
filepr-status
check before merginggha-conductor
can be found in mdolinin/mono-repo-example repository.Sequence diagram of how the app works for PR event:
sequenceDiagram
participant GitHub
participant App
participant Workflows
GitHub->>App: Send pull request event
App->>App: Check what files changed in PR
App->>App: Find what workflows needs to be triggered
App->>Workflows: Trigger all requested workflows jobs
App->>GitHub: Create pr-status check
loop until all triggered jobs completed
Workflows->>App: Notify workflow job queued
App->>GitHub: Create check for triggered workflow job
Workflows->>App: Notify workflow job in_progress
App->>GitHub: Update workflow job check in_progress
App->>GitHub: Update pr-status check in_progress
Workflows->>App: Notify workflow job completed
App->>GitHub: Update workflow run check completed
end
App->>App: Calculate conclusion
App->>GitHub: Update pr-status check with conclusion
App uses PostgreSQL database to store information about which workflows should be triggered for each event and workflow executions that were triggered.
@databases/pg
is used to interact with the database.pg-migrations
is used to manage database schema. Migrations are located in migrations
directory.@databases/pg-typed
and @databases/pg-schema-cli
are used to generate TypeScript types and JSON schemas from the database schema. Generated types and schema are located in src/__generated__
directory..github/gha-conductor-config.yaml
file to define repo specific configuration.gha_hooks_file: .gha.yaml
workflow_file_extension: .yaml
APP_CONFIG_FILE
- name of the configuration file (default: gha-conductor-config.yaml
)DEFAULT_GHA_HOOKS_FILE_NAME
- name of the file that contains hooks configurations file (default: .gha.yaml
)DEFAULT_WORKFLOW_FILE_EXTENSION
- extension of the GitHub Actions workflow files inside .github/workflows
folder (default: .yaml
)# Install dependencies
yarn
# Generate schemas
yarn generate
# Apply db migrations
yarn db:migrate
# Generate db schema
yarn db:generate
# Run the bot
yarn start
# Run tests
yarn test
# 1. Build container
docker build -t gha-conductor .
# 2. Start container
docker run -e APP_ID=<app-id> -e PRIVATE_KEY=<pem-value> -e WEBHOOK_SECRET=<secret-value> -e LOG_LEVEL=info -e DATABASE_URL=<postgres://user:pass@localhost:5432/postgres> gha-conductor
export DATABASE_URL=<your-database-url>
yarn db:migrate
APP_ID
- the ID of the GitHub AppPRIVATE_KEY
- the private key of the GitHub App, base64 encodedWEBHOOK_SECRET
- the secret used to secure webhooksLOG_LEVEL
- the log level (default: info
)DATABASE_URL
- the URL of the PostgreSQL database (format: postgres://user:pass@localhost:5432/postgres
)docker run -e APP_ID=<app-id> -e PRIVATE_KEY=<pem-value> -e WEBHOOK_SECRET=<secret-value> -e LOG_LEVEL=info -e DATABASE_URL=<db-url> ghcr.io/mdolinin/gha-conductor:latest
pr-status
check before mergingIf you have suggestions for how gha-conductor could be improved, or want to report a bug, open an issue! We'd love all and any contributions.
For more, check out the Contributing Guide.
ISC © 2024 mdolinin