To create a Azure Devops Pipeline, you must perform the following steps depending on whether you're starting from scratch or extending an existing pipeline.
New Azure Devops Project:
Log in to your Azure account at .
Sign in to your organization (https://dev.azure.com/{Your_Organization}).
Create and configure a Project -
Create and configure a Dev Center -
Add a git repository to your project -
Create and configure an environment type to be associated with your Dev Center -
Create a new pipeline and review your existing pipelines -
Existing Azure Devops Project
Follow these steps for an existing Azure Devops Project:
Review your existing pipelines and select which one you would like to integrate the Gomboc YAML into -
Follow the Azure Pipeline Editor guide to modify an existing pipeline -
When editing an existing pipeline it is important to consider that some of the pipeline definition such as the trigger ,pr , and jobs sections may already exist, including the possibility of a nested structure. In this case, be sure to use appropriate indentation in the YAML file so that the pipeline is configured properly. The Azure tooling should help with the validation, but it is important to pay attention as you can create valid YAML that may not trigger if the indentation is incorrect when editing an existing file.
Pull request pipeline
Azure devops pipeline can use the following yaml to run the Gomboc CLI on every pull request to main. Be sure to replace the {ORGANIZATION_NAME} fields within the yaml file with your your Azure DevOps organization name.
Gomboc CLI Client Configuration
To configure Gomboc client execution, modify the following line:
The following table outlines the configuration parameter options and their meanings:
Configuration Option
Options
Description
REVIEW_TYPE
submit-for-review
preview
This is the type of review that Gomboc will perform on the code.
"preview" means that there will be no subsequent pull request created, only a record of the scan in the pipeline logs and in the Gomboc Portal UI.
"submit-for-review" means that a pull request (PR) will be created with the fixes that Gomboc discovers are applicable to the code.
Please note that with preview, the pipelines will pass, regardless of any remediations we find.
TRIGGER_TYPE
on-pull-request
on-commit
This is the type of trigger that was used to initial the Gomboc client.
"on-pull-request" means a PR was submitted by a user for review
"on-commit" means that the notification came on a commit to a particular branch
IAC_TYPE
terraform
cloudformation
The IaC types which Gomboc should consider when analyzing the code. If providing multiple values, it must be specified as a comman separated list. For example:
Example Pipeline Definition
Let's break down each section of this YAML file:
Trigger - here you must include a list of all branches which you want the build to be executed when there is a change on that branch
PR - this indicates the branches that should have a build run when a pull request is created.
Pool - this is the resource pool definition to specify which type of container should be used to execute the build
Jobs - this is where the real work happens in two main parts
CheckBranchPattern - this job skips re-running Gomboc analysis on branches that Gomboc created.
Gomboc_CLI - executes the script to run the Gomboc API client.
# Example of a pipeline that runs on ever Pull Request
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
pool:
vmImage: ubuntu-latest
jobs:
- job: CheckBranchPattern
steps:
- script: |
branchName="$(System.PullRequest.SourceBranch)"
regexPattern="^refs/heads/[A-Za-z]+-remediated-[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}-[0-9]+$"
echo $branchName
echo $branchName =~ $regexPattern
if [[ $branchName =~ $regexPattern ]]; then
echo "Branch matches the pattern."
echo "##vso[task.setvariable variable=branchMatchesPattern;isOutput=true]true"
else
echo "Branch does not match the pattern."
echo "##vso[task.setvariable variable=branchMatchesPattern;isOutput=true]false"
fi
name: 'match'
- job: 'Gomboc_CLI'
dependsOn: CheckBranchPattern
condition: eq(dependencies.CheckBranchPattern.outputs['match.branchMatchesPattern'], 'false') # Checks to see if the action was initiated by a bot, or the user
container:
image: gombocai/cli:1.0.11
steps:
- checkout: self
fetchDepth: 0
- script: |
if [ $(Build.Reason) == "PullRequest" ]; then
TARGET_BRANCH_NAME=$(echo $(System.PullRequest.TargetBranch) | sed 's/refs\/heads\///')
SOURCE_BRANCH_NAME=$(echo $(System.PullRequest.SourceBranch) | sed 's/refs\/heads\///')
before=$(git rev-parse origin/$TARGET_BRANCH_NAME)
after=$(git rev-parse origin/$SOURCE_BRANCH_NAME)
target_directories=$(for i in $(git diff --name-only --diff-filter=ACMRT $before $after) ; do dirname $i ; done | sort -u | xargs)
if [ -z "$target_directories" ]; then
echo -e "\033[0;31mNo changes detected\033[0m"
exit 0
fi
gomboc submit-for-review on-pull-request --auth-token $(System.AccessToken) --azdo-collection-uri $(System.TeamFoundationCollectionUri) --azdo-organization-name {ORGANIZATION_NAME} --target-directories $target_directories --pull-request 0 --iac terraform
exit_code=$?
exit $exit_code
fi
if [ $(Build.Reason) == "IndividualCI" ]; then
SOURCE_BRANCH_NAME=$(echo $(Build.SourceBranch) | sed 's/refs\/heads\///')
before=$(git rev-parse origin/$SOURCE_BRANCH_NAME^1)
after=$(git rev-parse origin/$SOURCE_BRANCH_NAME)
target_directories=$(for i in $(git diff --name-only --diff-filter=ACMRT $before $after) ; do dirname $i ; done | sort -u | xargs)
if [ -z "$target_directories" ]; then
echo -e "\033[0;31mNo changes detected\033[0m"
exit 0
fi
gomboc submit-for-review on-pull-request --auth-token $(System.AccessToken) --azdo-collection-uri $(System.TeamFoundationCollectionUri) --azdo-organization-name {ORGANIZATION_NAME} --target-directories $target_directories --pull-request 0 --iac terraform
else
echo -e "\033[0;31mUnsupported event type: $CI_PIPELINE_SOURCE. Only the \"merge_request_event\" and \"push\" sources are currently supported\033[0m"
exit_code=$?
exit $exit_code
fi
Scheduled pipeline
If more flexibility is desired, the Docker image can be used by using gombocai/cli:{TAG} for the image value in your pipeline yaml file. From the image, you may use the gomboc command directly.
trigger:
branches:
include:
- main
pr:
branches:
include:
- main
pool:
vmImage: ubuntu-latest
# scheduled CI job execution
schedules:
- cron: '0 0 * * *' # Runs the job at midnight
displayName: "test-scheduled-pipeline"
branches:
include:
- main
always: false # should be made to true if runs should be started regardless if changes were made to the repo
jobs:
- job: CheckBranchPattern
steps:
- script: |
branchName="$(System.PullRequest.SourceBranch)"
regexPattern="^refs/heads/[A-Za-z]+-remediated-[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}-[0-9]+$"
echo $branchName
echo $branchName =~ $regexPattern
if [[ $branchName =~ $regexPattern ]]; then
echo "Branch matches the pattern."
echo "##vso[task.setvariable variable=branchMatchesPattern;isOutput=true]true"
else
echo "Branch does not match the pattern."
echo "##vso[task.setvariable variable=branchMatchesPattern;isOutput=true]false"
fi
name: 'match'
- job: 'Gomboc_CLI'
dependsOn: CheckBranchPattern
condition: eq(dependencies.CheckBranchPattern.outputs['match.branchMatchesPattern'], 'false') # Checks to see if the action was initiated by a bot, or the user
container:
image: gombocai/cli:1.0.11
steps:
- checkout: self
fetchDepth: 0
- script: |
if [ $(Build.Reason) == "PullRequest" ]; then
TARGET_BRANCH_NAME=$(echo $(System.PullRequest.TargetBranch) | sed 's/refs\/heads\///')
SOURCE_BRANCH_NAME=$(echo $(System.PullRequest.SourceBranch) | sed 's/refs\/heads\///')
before=$(git rev-parse origin/$TARGET_BRANCH_NAME)
after=$(git rev-parse origin/$SOURCE_BRANCH_NAME)
target_directories=$(for i in $(git diff --name-only --diff-filter=ACMRT $before $after) ; do dirname $i ; done | sort -u | xargs)
if [ -z "$target_directories" ]; then
echo -e "\033[0;31mNo changes detected\033[0m"
exit 0
fi
gomboc submit-for-review on-schedule --auth-token $(System.AccessToken) --azdo-collection-uri $(System.TeamFoundationCollectionUri) --azdo-organization-name {ORGANIZATION_NAME} --recurse true --iac terraform
exit_code=$?
exit $exit_code
fi
if [ $(Build.Reason) == "IndividualCI" ]; then
SOURCE_BRANCH_NAME=$(echo $(Build.SourceBranch) | sed 's/refs\/heads\///')
before=$(git rev-parse origin/$SOURCE_BRANCH_NAME^1)
after=$(git rev-parse origin/$SOURCE_BRANCH_NAME)
target_directories=$(for i in $(git diff --name-only --diff-filter=ACMRT $before $after) ; do dirname $i ; done | sort -u | xargs)
if [ -z "$target_directories" ]; then
echo -e "\033[0;31mNo changes detected\033[0m"
exit 0
fi
gomboc submit-for-review on-schedule --auth-token $(System.AccessToken) --azdo-collection-uri $(System.TeamFoundationCollectionUri) --azdo-organization-name {ORGANIZATION_NAME} --recurse true --iac terraform
else
echo -e "\033[0;31mUnsupported event type: $CI_PIPELINE_SOURCE. Only the \"merge_request_event\" and \"push\" sources are currently supported\033[0m"
exit_code=$?
exit $exit_code
fi
The Gomboc pipeline also recognizes scheduled execution to support detecting remediations in code that happen due to changes in IaC modules or improvements in Gomboc remediation coverage. You can view the to setup scheduled pipelines properly.
