Skip to main content

Defining custom patterns for secret scanning

You can define your own custom patterns to extend the capabilities of secret scanning by generating one or more regular expressions.

Who can use this feature?

Secret scanning alerts for partners runs automatically on public repositories and public npm packages to notify service providers about leaked secrets on GitHub.com.

Secret scanning alerts for users are available for user-owned public repositories for free. Organizations using GitHub Enterprise Cloud with a license for GitHub Advanced Security can also enable secret scanning alerts for users on their private and internal repositories. Additionally, secret scanning alerts for users are available and in beta on user-owned repositories for GitHub Enterprise Cloud with Enterprise Managed Users. For more information, see "About secret scanning" and "About GitHub Advanced Security."

For information about how you can try GitHub Advanced Security for free, see "Setting up a trial of GitHub Advanced Security."

About custom patterns for secret scanning

You can define custom patterns to identify secrets that are not detected by the default patterns supported by secret scanning. For example, you might have a secret pattern that is internal to your organization. For details of the supported secrets and service providers, see "Secret scanning patterns."

You can define custom patterns for your enterprise, organization, or repository. Secret scanning supports up to 500 custom patterns for each organization or enterprise account, and up to 100 custom patterns per repository.

You can also enable push protection for custom patterns. For more information about push protection, see "Push protection for repositories and organizations."

About using regular expressions for custom patterns

You can specify custom patterns for secret scanning as one or more regular expressions.

Secret scanning uses the Hyperscan library and only supports Hyperscan regex constructs, which are a subset of PCRE syntax. Hyperscan option modifiers are not supported. For more information on Hyperscan pattern constructs, see "Pattern support" in the Hyperscan documentation.

Regular expressions can be entered manually or generated using the regular expression generator.

Regular expression syntax for manually defining custom patterns

The More options section in the UI helps you write regular expressions manually.

  • Secret format: an expression that describes the format of the secret itself.
  • Before secret: an expression that describes the characters that come before the secret. By default, this is set to \A|[^0-9A-Za-z] which means that the secret must be at the start of a line or be preceded by a non-alphanumeric character.
  • After secret: an expression that describes the characters that come after the secret. By default, this is set to \z|[^0-9A-Za-z] which means that the secret must be followed by a new line or a non-alphanumeric character.
  • Additional match requirements: one or more optional expressions that the secret itself must or must not match.

For simple tokens you will usually only need to specify a secret format. The other fields provide flexibility so that you can specify more complex secrets without creating complex regular expressions. For an example of a custom pattern, see "Example of a custom pattern specified using additional requirements" below.

Using the regular expression generator

The generator uses a generative AI model where you input a text description of the type of pattern you would like to detect, including optional example strings that should be detected. The model returns up to three regular expressions for you to review. For more information, see "About the regular expression generator for custom patterns and "Generating regular expressions for custom patterns with AI."

Defining a custom pattern for a repository

Before defining a custom pattern, you must ensure that secret scanning is enabled on your repository. For more information, see "Configuring secret scanning for your repositories."

  1. On GitHub.com, navigate to the main page of the repository.

  2. Under your repository name, click Settings. If you cannot see the "Settings" tab, select the dropdown menu, then click Settings.

    Screenshot of a repository header showing the tabs. The "Settings" tab is highlighted by a dark orange outline.

  3. In the "Security" section of the sidebar, click Code security and analysis.

  4. Under "Code security and analysis", find "GitHub Advanced Security."

  5. Under "Secret scanning", under "Custom patterns", click New pattern.

  6. Enter the details for your new custom pattern. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.

    1. In the "Pattern name" field, type a name for your pattern.
    2. In the "Secret format" field, type a regular expression for the format of your secret pattern. Alternatively, you can use the generator to generate a regular expression for you. For more information, see "Generating regular expressions for custom patterns with AI."
    3. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    4. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Screenshot of a filled custom secret scanning pattern form.

  7. When you're ready to test your new custom pattern, to identify matches in the repository without creating alerts, click Save and dry run.

  8. When the dry run finishes, you'll see a sample of results (up to 1000). Review the results and identify any false positive results. Screenshot showing results from dry run.

  9. Edit the new custom pattern to fix any problems with the results, then, to test your changes, click Save and dry run.

  10. When you're satisfied with your new custom pattern, click Publish pattern.

  11. Optionally, to enable push protection for your custom pattern, click Enable.

    Note: The "Enable" button isn't available until after the dry run succeeds and you publish the pattern.

    For more information about push protection, see "Push protection for repositories and organizations."

After your pattern is created, secret scanning scans for any secrets in your entire Git history on all branches present in your GitHub repository. For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Example of a custom pattern specified using additional requirements

A company has an internal token with five characteristics. They use the different fields to specify how to identify tokens as follows:

CharacteristicField and regular expression
Length between 5 and 10 charactersSecret format: [$#%@AA-Za-z0-9]{5,10}
Does not end in a .After secret: [^\.]
Contains numbers and uppercase lettersAdditional requirements: secret must match [A-Z] and [0-9]
Does not include more than one lowercase letter in a rowAdditional requirements: secret must not match [a-z]{2,}
Contains one of $%@!Additional requirements: secret must match [$%@!]

These tokens would match the custom pattern described above:

a9@AAfT!         # Secret string match: a9@AAfT
ee95GG@ZA942@aa  # Secret string match: @ZA942@a
a9@AA!ee9        # Secret string match: a9@AA

These strings would not match the custom pattern described above:

a9@AA.!
a@AAAAA
aa9@AA!ee9
aAAAe9

Defining a custom pattern for an organization

Before defining a custom pattern, you must ensure that you enable secret scanning for the repositories that you want to scan in your organization. To enable secret scanning on all repositories in your organization, see "Managing security and analysis settings for your organization."

  1. In the upper-right corner of GitHub.com, select your profile photo, then click Your organizations.

    Screenshot of the dropdown menu under @octocat's profile picture. "Your organizations" is outlined in dark orange.

  2. Next to the organization, click Settings.

  3. In the "Security" section of the sidebar, click Code security and analysis.

  4. Under "Code security and analysis", find "GitHub Advanced Security."

  5. Under "Secret scanning", under "Custom patterns", click New pattern.

  6. Enter the details for your new custom pattern. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.

    1. In the "Pattern name" field, type a name for your pattern.
    2. In the "Secret format" field, type a regular expression for the format of your secret pattern. Alternatively, you can use the generator to generate a regular expression for you. For more information, see "Generating regular expressions for custom patterns with AI."
    3. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    4. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Screenshot of a filled custom secret scanning pattern form.

  7. When you're ready to test your new custom pattern, to identify matches in select repositories without creating alerts, click Save and dry run.

  8. Select the repositories where you want to perform the dry run.

    • To perform the dry run across the entire organization, select All repositories in the organization.
    • To specify the repositories where you want to perform the dry run, select Selected repositories, then search for and select up to 10 repositories.
  9. When you're ready to test your new custom pattern, click Run.

  10. When the dry run finishes, you'll see a sample of results (up to 1000). Review the results and identify any false positive results. Screenshot showing results from dry run.

  11. Edit the new custom pattern to fix any problems with the results, then, to test your changes, click Save and dry run.

  12. When you're satisfied with your new custom pattern, click Publish pattern.

  13. Optionally, to enable push protection for your custom pattern, click Enable. For more information, see "Push protection for repositories and organizations."

    Notes:

    • Push protection for custom patterns will only apply to repositories in your organization that have secret scanning as push protection enabled. For more information, see "Push protection for repositories and organizations."
    • Enabling push protection for commonly found custom patterns can be disruptive to contributors.

After your pattern is created, secret scanning scans for any secrets in repositories in your organization, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found and can review the alert in the repository where the secret is found. For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Defining a custom pattern for an enterprise account

Before defining a custom pattern, you must ensure that you enable secret scanning for your enterprise account. For more information, see "Enabling GitHub Advanced Security for your enterprise."

Notes:

  • At the enterprise level, only the creator of a custom pattern can edit the pattern, and use it in a dry run.
  • You can only perform a dry run on repositories that you have administration access to. If an enterprise owner wants access to perform dry runs on any repository in an organization, they must be assigned the organization owner role. For more information, see "Managing your role in an organization owned by your enterprise."
  1. In the top-right corner of GitHub.com, click your profile photo, then click Your enterprises.

  2. In the list of enterprises, click the enterprise you want to view.

  3. In the enterprise account sidebar, click Policies.

  4. Under "Policies", click Code security and analysis.

  5. Under "Code security and analysis", click Security features.

  6. Under "Secret scanning custom patterns", click New pattern.

  7. Enter the details for your new custom pattern. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.

    1. In the "Pattern name" field, type a name for your pattern.
    2. In the "Secret format" field, type a regular expression for the format of your secret pattern. Alternatively, you can use the generator to generate a regular expression for you. For more information, see "Generating regular expressions for custom patterns with AI."
    3. You can click More options to provide other surrounding content or additional match requirements for the secret format.
    4. Provide a sample test string to make sure your configuration is matching the patterns you expect.

    Screenshot of a filled custom secret scanning pattern form.

  8. When you're ready to test your new custom pattern, to identify matches in the enterprise without creating alerts, click Save and dry run.

  9. Search for and select up to 10 repositories where you want to perform the dry run.

  10. When you're ready to test your new custom pattern, click Run.

  11. When the dry run finishes, you'll see a sample of results (up to 1000). Review the results and identify any false positive results. Screenshot showing results from dry run.

  12. Edit the new custom pattern to fix any problems with the results, then, to test your changes, click Save and dry run.

  13. When you're satisfied with your new custom pattern, click Publish pattern.

  14. Optionally, to enable push protection for your custom pattern, click Enable. For more information, see "Push protection for repositories and organizations."

    Notes:

    • To enable push protection for custom patterns, secret scanning as push protection needs to be enabled at the enterprise level. For more information, see "Push protection for repositories and organizations."
    • Enabling push protection for commonly found custom patterns can be disruptive to contributors.

After your pattern is created, secret scanning scans for any secrets in repositories within your enterprise's organizations with GitHub Advanced Security enabled, including their entire Git history on all branches. Organization owners and repository administrators will be alerted to any secrets found, and can review the alert in the repository where the secret is found. For more information on viewing secret scanning alerts, see "Managing alerts from secret scanning."

Editing a custom pattern

When you save a change to a custom pattern, this closes all the secret scanning alerts that were created using the previous version of the pattern.

  1. Navigate to where the custom pattern was created. A custom pattern can be created in a repository, organization, or enterprise account.

  2. Under "Secret scanning", to the right of the custom pattern you want to edit, click .

  3. When you're ready to test your edited custom pattern, to identify matches without creating alerts, click Save and dry run.

  4. When you have reviewed and tested your changes, click Publish changes.

  5. Optionally, to enable push protection for your custom pattern, click Enable.

    Note:

    • Push protection for custom patterns will only apply to repositories that have secret scanning as push protection enabled. For more information about enabling push protection, see "Push protection for repositories and organizations."
    • Enabling push protection for commonly found custom patterns can be disruptive to contributors.

    Screenshot of custom pattern page with the button to enable push protection emphasized.

  6. Optionally, to disable push protection for your custom pattern, click Disable.

    Screenshot of the custom pattern page with the button to disable push protection highlighted with a dark orange outline.

Removing a custom pattern

  1. Navigate to where the custom pattern was created. A custom pattern can be created in a repository, organization, or enterprise account.
  2. To the right of the custom pattern you want to remove, click .
  3. Review the confirmation, and select a method for dealing with any open alerts relating to the custom pattern.
  4. Click Yes, delete this pattern.

Metrics for custom patterns

Organization owners and people with admin permissions can see an overview of the activity for custom patterns. The overview includes alert and push protection activity for the custom pattern during the last 30 days.

Note: Metrics for custom patterns are in public beta and subject to change.

Viewing metrics for custom patterns

  1. Navigate to where the custom pattern was created. A custom pattern can be created in a repository, organization, or enterprise account.
  2. Under "Secret scanning", click the custom pattern you want to view.

The metrics are displayed under the custom pattern's name.