gitleaks

What's new

• Stricter regex for the Github rule https://github.com/zricethezav/gitleaks/issues/531
• Add ability to append repo configurations, and user supplied configurations to the specified or default configuration https://github.com/zricethezav/gitleaks/pull/487

What's fixed

• Go-diff panic https://github.com/zricethezav/gitleaks/issues/503

What's fixed:

• --no-git was reporting abs paths in the file field in reports when running inside Docker. The file field should be relative to --path for all scan types now.

What's fixed:

• Better ignoring .git folders https://github.com/zricethezav/gitleaks/pull/507
• Invalid usage or flags provided results in 0 exit code https://github.com/zricethezav/gitleaks/issues/500
• Add file type checking for unstaged commits https://github.com/zricethezav/gitleaks/issues/499

What's new:

• Added a -q/--quiet flag for json only logging https://github.com/zricethezav/gitleaks/pull/491

What's fixed:

• Took care of some gosec warnings https://github.com/zricethezav/gitleaks/pull/490
• .git matching for --no-git scans: https://github.com/zricethezav/gitleaks/issues/486. Thank you @ramimac

What's Fixed

• --file-at-commit suffered still suffered from #482. Big thanks to @ramimac for surfacing this

What's Fixed

• Allowlist seems broken post v7.0.* https://github.com/zricethezav/gitleaks/issues/482. Big thanks to @ramimac for surfacing this

What's New:

• https://github.com/zricethezav/gitleaks/pull/480 I forked go-git and made some specific operations thread safe. See https://github.com/zricethezav/go-git/releases/tag/v5.2.1. This allows for concurrent patch generation which speeds up scans significantly. As a large repo benchmark, I often use https://github.com/rails/rails.

Below are some results:
v7.1.0: gitleaks --repo=https://github.com/rails/rails --threads=12

INFO[0000] cloning... https://github.com/rails/rails
INFO[0134] scan time: 1 minute 51 seconds 828 milliseconds 144 microseconds
INFO[0134] commits scanned: 89801
WARN[0134] leaks found: 4

v7.0.2: gitleaks --repo=https://github.com/rails/rails --threads=12

INFO[0000] cloning... https://github.com/rails/rails
INFO[0239] scan time: 3 minutes 36 seconds 45 milliseconds 232 microseconds
INFO[0239] commits scanned: 89801
WARN[0239] leaks found: 4

• https://github.com/zricethezav/gitleaks/pull/481 Let's talk exit codes... again. I'm trying to follow the convention defined here: https://tldp.org/LDP/abs/html/exitcodes.html. But it seems that people expect exit codes to behavior differently. So, I'm adding a --leaks-exit-code flag:

--leaks-exit-code=  Exit code when leaks have been encountered (default: 1)

The default codes:

0 - no leaks found
1 - leaks or errors encountered 

What's New:

• Config package exposes some rule/allowlist functions

What's Fixed:

• --no-git scans .git directory, finds false positives https://github.com/zricethezav/gitleaks/issues/474
• --path + --repo-config-path + --no-git = segmentation violation https://github.com/zricethezav/gitleaks/issues/473
• File-based Rules + Deduplication logic removal in v7.0.0 https://github.com/zricethezav/gitleaks/issues/472

What's Fixed:

• There was an error when extracting line numbers from leaks when the line containing a leak exceeded 200 characters. https://github.com/zricethezav/gitleaks/commit/389d45e38026f2b6a594e1109db4360216f5149b

What's new:

A lot. v7.0.0 might piss some people off but hey, that's why v6.2.0 exists. I kinda hated the way Gitleaks
was structured which resulted in a creeping dread when even thinking about maintaining this project. So I did what
any good software engineer would do (the following is a joke) -- completely rewrote gitleaks (okay okay some of the utils and algos stayed the same). So you may be wondering, why did I do this? Well, I'm hoping this will
help the longevity of the project and make it easier to contribute now that the code follows a
factory pattern (see the scan package). Perhaps what folks will be most upset about is the removal of the hosts option, no more
scanning groups, projects, owners, users, or PRs/MRs directly using gitlab/github's APIs. This is something I just don't want to maintain anymore so feel free to use gitleaks as a library and create your own github/gitlab gitleaks scanner or use this script. On a lighter note, v7.0.0 includes some new features.

• Updated exit codes. 0 if gitleaks ran successfully (leaks found or no leaks found), 1 if error has occurred.
• Removal of --pretty. Pretty printing leaks is now the default
• Removal of deduplication for leaks. If two different rules detect the same offender they will be both reported.
• Removal of --timeout option
• Replaced --repo-config with --repo-config-path. This will load a repo's config if available.
• Add remote leak resource with LeakURL
• Added no-git options for directory and file scanning (maybe someone will develop a vim/IDE plugin for gitleaks??). --no-git also supports threaded execution for speedy scans
• Replaced --repo-path and --owner-path with -p,--path. Gitleaks will determine if --path is a git repo or directory containing git repos. If supplied with a --no-git option gitleaks will scan all the contents of --path. This could be a directory or file.
• Replaced --repo with --repo-url
• Replaced --disk with --clone-path. No more cloning to tmp dir
• Scans that require a git clone and have --branch or --depth set are speedier as both --branch and --depth are set as clone options.
• Replaced --uncommited with --unstaged

What's fixed:

• Line number extraction bug

What's new:

• reportGroup field for rules in gitleaks configs. https://github.com/zricethezav/gitleaks/pull/456

What's fixed

• off by one error w/r/t line number extraction introduced in v6.1.1

What's fixed:

• Line extraction with leaks with the same offender within the same patch: https://github.com/zricethezav/gitleaks/pull/436

What's new:

Solution to #432 by adding a commits and commits-file option.

--commits=<comma separated list of commits>
--commits-file=path to file containing a list of commits separated by newlines

NOTE

commits or commits-file should be used in favor of commit-to and commit-from as there is a reachability issue in that commit-from is not guaranteed to reach commit-to so you risk scanning many more commits than originally intended.

What's New

SARIF support:

This release adds Oasis SARIF support with --report-format=sarif which will write your report with a sarif schema. Note this is the first implementation of gitleaks+sarif so if anyone has any suggestions or wanted changes please open a PR.

Config update:

Users can now include a regular expressions in a global allow list. This is potentially useful if your code is riddled with example credentials and secrets. Below is a sample config which demonstrates how this can be used:

[[rules]]
	description = "AWS Manager ID"
	regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
	tags = ["key", "AWS"]

[allowlist]
	description = "global allowlists"
	files = [
		'''(.*?)(jpg|gif)$''',
		'''(.*?)(doc|pdf|bin)$''',
		'''(.*?)(gitleaks.toml)$'''
		]
	regexes = [
	    '''(.*?)gitleaks:allow'''
	]

This example will consider the line aws_access_key_id='AKIAIO5FODNN7EXAMPLE' #example credential a leak whereas aws_access_key_id='AKIAIO5FODNN7EXAMPLE' #gitleaks:allow" would not be considered a leak as it has the comment #gitleaks:allow which matches on the global allowlist presented above.

Breaking Changes

This PR also includes some breaking changes to the configuration file hence the bump to v6.

v5 config:

[[rules]]
  description = "a string describing one of many rule in this config"
  regex = '''one-go-style-regex-for-this-rule''' 
  filenameregex = '''a-file-name-regex'''
  filepathregex = '''a-file-path-regex'''
  tags = ["tag","another tag"]
  [[rules.entropies]] # note these are strings, not floats
    Min = "3.5"
    Max = "4.5"
    Group = "1"
  [[rules.allowlist]]
    description = "a string"
    file = '''one-file-name-regex'''
    path = '''one-file-path-regex'''
    regex = '''one-regex-within-the-already-matched-regex'''
[allowlist]
  description = "a description string for only _ONE_ allowlist config"
  commits = [ "commit-A", "commit-B"]
  files = [ '''file-regex-a''', '''file-regex-b''']
  paths = [ '''path-regex-a''', '''path-regex-b''']
  repos = [ '''repo-regex-a''', '''repo-regex-b''']

v6 config:

[[rules]]
  description = "a string describing one of many rule in this config"
  regex = '''one-go-style-regex-for-this-rule''' 
  file = '''a-file-name-regex''' # this changed!
  path = '''a-file-path-regex'''  # this changed!
  tags = ["tag","another tag"]
  [[rules.entropies]] # note these are strings, not floats
    Min = "3.5"
    Max = "4.5"
    Group = "1"
  [rules.allowlist]  # this changed! No longer an array of tables. It's just a table.
    description = "a string"
    files = ['''one-file-name-regex''']  # this changed!
    paths = ['''one-file-path-regex''']  # this changed!
    regexes = ['''one-regex-within-the-already-matched-regex''']  # this changed!

[allowlist]
  description = "a description string for a global allowlist config"
  commits = [ "commit-A", "commit-B"]
  files = [ '''file-regex-a''', '''file-regex-b''']
  paths = [ '''path-regex-a''', '''path-regex-b''']
  repos = [ '''repo-regex-a''', '''repo-regex-b''']
  regexes = ['''one-regex-within-the-already-matched-regex''']  # this added!

What's Fixed:

• Fixed two issues described here: https://github.com/zricethezav/gitleaks/issues/413. There was a bug when scanning commits that were the result of a merge request, i.e. when there are two or more commits. Prior to this release gitleaks would iterate each parent of the commit in question and generate a patch. This could result in bloated an unnecessary patches. We want to pick the FIRST parent and ignore the other parents as those commits are reachable with git log. The second issue fixed was a bug regarding the line extraction feature released in v5.0.0. If the patch content generate was greater than the default bufio.Scanner's max, then all patch lines would not be scanned. Both these issues fixed here in this PR: https://github.com/zricethezav/gitleaks/pull/414
• Validate Configs https://github.com/zricethezav/gitleaks/pull/412

What's new:

• Change whitelist to allowlist. This is a breaking change. v5.0.0 custom configs will need to be updated.
• Line numbers have been added to the report. This has been a requested feature by many and I'm happy to include it in this release. There is a slight performance hit if leaks are present due to the fact that in order to extract the line number Gitleaks has to do a reverse search on a file/commit/patch depending on the scan conditions.
• Git additions only are now scanned by default. Scanning additions by defaults makes the most sense since users mainly care about when a leak has been introduced, not necessarily removed. Deletions can be included with the --include-deletion option.
• Operation added to the report. This is the git operation (addition/equal/deletion) associated with a leak.
• Bumped gitleaks module to v5.
• go-git has been bumped to v5.10.0

What's fixed:

• The patch direction was wrong. Gitleaks was patching currentCommit.Patch(parent) rather than the correct parent.Patch(currentCommit). This had the side effect of reporting the wrong commit sometimes. In v5.0.0 this has been fixed.

What's fixed:

• There was some bad option guarding around commit options that has been removed. Fixes https://github.com/zricethezav/gitleaks/issues/392

What's new:

• --commit-since= Audit commits more recent than a specific date. Ex: '2006-01-02' or '2006-01-02T15:04:05-0700' format. https://github.com/zricethezav/gitleaks/pull/385
• --commit-until= Audit commits older than a specific date. Ex: '2006-01-02' or '2006-01-02T15:04:05-0700' format. https://github.com/zricethezav/gitleaks/pull/385

Shoutout to @steeve85 for the PR

What's fixed:

• Whitelisted root commits (commits without a parent) specified in the config were not getting whitelisted. This has been fixed: https://github.com/zricethezav/gitleaks/pull/390

What's new

• repo whitelisting in config: https://github.com/zricethezav/gitleaks/pull/378
  example:

   description = "AWS Manager ID"
    regex = '''(A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}'''
    tags = ["key", "AWS"]
    [whitelist]
        description = "whitelist repo"
        repos = [
            '''test_repo_1'''
        ]

• bumped go-git and go-github versions

What's fixed:

• Incorrect results with the --redact option: https://github.com/zricethezav/gitleaks/pull/368
• Removing generic rule from the default configuration: https://github.com/zricethezav/gitleaks/pull/379