Skip to main content

Defining custom patterns for secret scanning

You can extend secret scanning to detect secrets beyond the default patterns.

Secret scanning is available for organization-owned repositories in GitHub Enterprise Server if your enterprise has a license for GitHub Advanced Security. 有关详细信息,请参阅“关于 GitHub Advanced Security”。

About custom patterns for secret scanning

You can define custom patterns to identify secrets that are not detected by the default patterns supported by secret scanning. For example, you might have a secret pattern that is internal to your organization. For details of the supported secrets and service providers, see "Secret scanning patterns."

You can define custom patterns for your enterprise, organization, or repository. Secret scanning supports up to 500 custom patterns for each organization or enterprise account, and up to 100 custom patterns per repository.

Regular expression syntax for custom patterns

You can specify custom patterns for secret scanning as one or more regular expressions.

  • Secret format: an expression that describes the format of the secret itself.
  • Before secret: an expression that describes the characters that come before the secret. By default, this is set to \A|[^0-9A-Za-z] which means that the secret must be at the start of a line or be preceded by a non-alphanumeric character.
  • After secret: an expression that describes the characters that come after the secret. By default, this is set to \z|[^0-9A-Za-z] which means that the secret must be followed by a new line or a non-alphanumeric character.
  • Additional match requirements: one or more optional expressions that the secret itself must or must not match.

For simple tokens you will usually only need to specify a secret format. The other fields provide flexibility so that you can specify more complex secrets without creating complex regular expressions. For an example of a custom pattern, see "Example of a custom pattern specified using additional requirements" below.

Secret scanning uses the Hyperscan library and only supports Hyperscan regex constructs, which are a subset of PCRE syntax. Hyperscan option modifiers are not supported. For more information on Hyperscan pattern constructs, see "Pattern support" in the Hyperscan documentation.

Defining a custom pattern for a repository

Before defining a custom pattern, you must ensure that secret scanning is enabled on your repository. For more information, see "Configuring secret scanning for your repositories."

  1. On your GitHub Enterprise Server instance, navigate to the main page of the repository.

  2. 在存储库名称下,单击 “设置”。 “存储库设置”按钮

  3. In the "Security" section of the sidebar, click Code security and analysis.

  4. 在“代码安全性和分析”下,查找“GitHub Advanced Security”。

  5. 在“Secret scanning”下的“自定义模式”下,单击“新建模式”。

  6. Enter the details for your new custom pattern:

    1. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.
    2. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    3. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Create a custom secret scanning pattern form

  7. When you're ready to test your new custom pattern, to identify matches in the repository without creating alerts, click Save and dry run.

  8. 试运行完成后,你将看到结果示例(最多 1000 个)。 查看结果并确定任何误报结果。 显示试运行结果的屏幕截图

  9. 编辑新的自定义模式以修复结果的任何问题,然后测试更改,单击“保存并试运行”。

    注意:试运行功能目前为 beta 版本,可能会有变动。

  10. When you're satisfied with your new custom pattern, click Publish pattern

After your pattern is created, secret scanning 将在 GitHub 仓库中存在的所有分支上扫描整个 Git 历史记录的任何密钥。 For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Example of a custom pattern specified using additional requirements

A company has an internal token with five characteristics. They use the different fields to specify how to identify tokens as follows:

CharacteristicField and regular expression
Length between 5 and 10 charactersSecret format: [$#%@AA-Za-z0-9]{5,10}
Does not end in a .After secret: [^\.]
Contains numbers and uppercase lettersAdditional requirements: secret must match [A-Z] and [0-9]
Does not include more than one lowercase letter in a rowAdditional requirements: secret must not match [a-z]{2,}
Contains one of $%@!Additional requirements: secret must match [$%@!]

These tokens would match the custom pattern described above:

a9@AAfT!         # Secret string match: a9@AAfT
ee95GG@ZA942@aa  # Secret string match: @ZA942@a
a9@AA!ee9        # Secret string match: a9@AA

These strings would not match the custom pattern described above:

a9@AA.!
a@AAAAA
aa9@AA!ee9
aAAAe9

Defining a custom pattern for an organization

Before defining a custom pattern, you must ensure that you enable secret scanning for the repositories that you want to scan in your organization. To enable secret scanning on all repositories in your organization, see "Managing security and analysis settings for your organization."

  1. 在 GitHub Enterprise Server 的右上角,单击你的个人资料照片,然后单击“你的组织”。 贵组织在配置文件菜单中

  2. 在组织旁边,单击“设置”。 设置按钮

  3. In the "Security" section of the sidebar, click Code security and analysis.

  4. 在“代码安全性和分析”下,查找“GitHub Advanced Security”。

  5. 在“Secret scanning”下的“自定义模式”下,单击“新建模式”。

  6. Enter the details for your new custom pattern:

    1. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.
    2. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    3. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Create a custom secret scanning pattern form

  7. When you're ready to test your new custom pattern, to identify matches in select repositories without creating alerts, click Save and dry run.

  8. 搜索并选择最多 10 个要在其中执行试运行的存储库。 显示为试运行选择的存储库的屏幕截图

  9. 准备好测试新的自定义模式后,请单击“试运行”。

  10. 试运行完成后,你将看到结果示例(最多 1000 个)。 查看结果并确定任何误报结果。 显示试运行结果的屏幕截图

  11. 编辑新的自定义模式以修复结果的任何问题,然后测试更改,单击“保存并试运行”。

    注意:试运行功能目前为 beta 版本,可能会有变动。

  12. When you're satisfied with your new custom pattern, click Publish pattern

After your pattern is created, secret scanning scans for any secrets in repositories in your organization, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found and can review the alert in the repository where the secret is found. For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Defining a custom pattern for an enterprise account

Before defining a custom pattern, you must ensure that you enable secret scanning for your enterprise account. For more information, see "Enabling GitHub Advanced Security for your enterprise."

Notes:

  • At the enterprise level, only the creator of a custom pattern can edit the pattern, and use it in a dry run.
  • Enterprise owners can only make use of dry runs on repositories that they have access to, and enterprise owners do not necessarily have access to all the organizations or repositories within the enterprise.
  1. 在 GitHub Enterprise Server 的右上角,单击你的个人资料照片,然后单击“企业设置”。 GitHub Enterprise Server 上个人资料照片下拉菜单中的“企业设置”

  2. 在企业边栏中,单击 “策略”。 企业帐户边栏中的“策略”选项卡

  3. “策略”下,单击“高级安全”。 边栏中的“高级安全”策略

  4. 在“GitHub Advanced Security”下,单击“安全功能”选项卡。

  5. Under "Secret scanning custom patterns", click New pattern.

  6. Enter the details for your new custom pattern:

    1. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.
    2. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    3. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Create a custom secret scanning pattern form

  7. When you're ready to test your new custom pattern, to identify matches in the enterprise without creating alerts, click Save and dry run.

  8. 搜索并选择最多 10 个要在其中执行试运行的存储库。 显示为试运行选择的存储库的屏幕截图

  9. 准备好测试新的自定义模式后,请单击“试运行”。

  10. 试运行完成后,你将看到结果示例(最多 1000 个)。 查看结果并确定任何误报结果。 显示试运行结果的屏幕截图

  11. 编辑新的自定义模式以修复结果的任何问题,然后测试更改,单击“保存并试运行”。

    注意:试运行功能目前为 beta 版本,可能会有变动。

  12. When you're satisfied with your new custom pattern, click Publish pattern

After your pattern is created, secret scanning scans for any secrets in repositories within your enterprise's organizations with GitHub Advanced Security enabled, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found, and can review the alert in the repository where the secret is found. For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Editing a custom pattern

When you save a change to a custom pattern, this closes all the secret scanning alerts that were created using the previous version of the pattern.

  1. Navigate to where the custom pattern was created. A custom pattern can be created in a repository, organization, or enterprise account.
  2. Under "Secret scanning", to the right of the custom pattern you want to edit, click .
  3. When you're ready to test your edited custom pattern, to identify matches without creating alerts, click Save and dry run.
  4. When you have reviewed and tested your changes, click Save changes.

Removing a custom pattern

  1. Navigate to where the custom pattern was created. A custom pattern can be created in a repository, organization, or enterprise account.

  2. To the right of the custom pattern you want to remove, click .

  3. Review the confirmation, and select a method for dealing with any open alerts relating to the custom pattern.

  4. Click Yes, delete this pattern.

    Confirming deletion of a custom secret scanning pattern