Enterprise Server 3.13 release notes

Packages have been updated to the latest security versions.

On an instance in a cluster configuration, ghe-repl-promote failed if the primary node was unavailable.

In a high availability configuration, with GitHub Actions, replication would fail on nodes where MSSQL was not configured to run.

The --no-async flag was not implemented for the ghe-cluster-support-bundle command, leading to a potentially increased load.

Pre-receive hook environments with shared memory enabled could not access shared memory at runtime.

For instances hosted on Azure, if a pre-upgrade check failed due to insufficient user disk size, the Management Console displayed an internal server error.

The Enterprise Overview page incorrectly displayed a Beta label, even though it is generally available.

After a user made changes to the isolated subdomain setting, some user assets did not display properly.

On an instance with secret scanning enabled, when selecting repositories for a dry run of an enterprise-level custom pattern, searches for full repository names (ORGANIZATION/REPOSITORY) did not return results.

When adding bypass permissions to a ruleset, the dropdown menu failed to load if one of the suggested actors was an invalid integration.

When creating a pre-receive hook environment, attempts to include an image URL over 255 characters failed with a database error. The maximum length is still 255 characters, but the URL length is now validated before the process starts.

Performing a browser back navigation to a pull request now displays up-to-date status checks.

Jekyll-build tooling for GitHub pages could fail when using the jekyll-relative-links plugin, see Failure details.

Subversion services were non-functional in some cases.

When exporting repositories to blob storage using the migrations REST API endpoint to start an organization migration, the maximum compressed archive size is limited to 90 GB. This is an increase from 30 GB.

Removes the minimum date for the new commit filter bar.

When exporting repositories using the migrations REST API, prior to blob storage upload the tarball is staged in the root volume. For more disk capacity, the tarball will now be staged in the data volume.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

LOW: Instance administrators could see tokens used to authenticate against gitauth in plaintext in/var/log/github-audit.log.

Packages have been updated to the latest security versions.

Pull request review requests for teams were sometimes not assigned to users correctly, because of a limit on manual review requests.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Attempting to stop replications after stopping GitHub Actions on a GitHub Enterprise Server instance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl

Customers performing a feature version upgrade to 3.13.6 or 3.14.3 may experience issues with database migrations due to data issues during database conversions.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Attempting to stop replications after stopping GitHub Actions on a GHES instanstance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl.

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

Elasticsearch packages have been updated to the latest security versions.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This is a follow up fix for CVE-2024-9487 to further harden the encrypted assertions feature against this type of attack. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document to exploit this vulnerability.

HIGH: An attacker with Enterprise Administrator access to the GitHub Enterprise Server instance could escalate privileges to SSH root access. This is achieved by exploiting the pre-receive hook environment to bypass symlink checks in the ghe-firejail path and execute malicious scripts. GitHub has requested CVE ID CVE-2024-10007 for this vulnerability, which was reported via the GitHub Bug Bounty program. [Updated: 2024-11-07]

A missing configuration value prevented Dependabot from creating group update pull requests.

When saving settings in the Management Console, the configuration run would stop if the enterprise-manage process was restarted.

On an instance with GitHub Actions enabled, some maintenance tasks could fail due to incomplete upgrade steps during previous upgrades to new releases of GitHub Enterprise Server.

The initial setup certificate generation in AWS took longer than expected due to fallback to private IPs. The time for this fallback has been reduced.

The ghe-support-bundle generation would fail when the aqueduct-lite service is down.

If the primary instance was unreachable, running ghe-repl-stop --force on a replica would fail during the config apply run.

For instances that use the mandatory message feature logging in to certain URLs may have caused a 500 error.

When restoring from a backup, repositories that had been deleted in the last 90 days were not completely restored.

Restoring Git repositories using backup-utils occasionally failed.

Enterprise installations experienced unpredictable repository search results due to the default 4,000 repository limit. A relaxed repository filter mode, which includes all single-tenant organization repositories and bypasses the limit, has been introduced. Administrators can enable this mode using ghe-config app.github.enterprise-repo-search-filter-enabled true && ghe-config-apply.

Organizations were limited to using 100 Actions organization variables instead of 1,000.

Running config-apply became stuck under certain circumstances due to a misconfiguration with Packages and Elasticsearch.

Some customers upgrading to 3.13 may experience issues with undecryptable records during the upgrade. This issue has now been resolved. We recommend you read Undecryptable records.

When connecting to an appliance via SSH, a notification about upcoming root disk changes displays.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. See Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track GitHub Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Customers doing feature version upgrade to 3.13.6 may experience issues with database migrations due to data issues during database conversions. [Added: 2024-11-08]

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

MEDIUM: Malicious URLs for SVG assets provided information about a victim user who clicked the URL, allowing an attacker to retrieve metadata belonging to the user and use it to generate a convincing phishing page. This required the attacker to upload malicious SVGs and phish a victim user to click the URL for the uploaded asset. GitHub has requested CVE ID CVE-2024-9539. This vulnerability was reported via the GitHub Bug Bounty program.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This was a regression introduced as part of follow-up remediation from CVE-2024-4985, which resulted in a new variant of the vulnerability. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document. GitHub has requested CVE ID CVE-2024-9487. This vulnerability was reported via the GitHub Bug Bounty program.

HAProxy reloading was failure prone, which could lead to failed Git operations. This reloading process has been replaced with a more resilient Systemd process.

On an instance with secret scanning enabled, internal jobs were created and not processed, which could contribute to performance issues.

This error message mbind: Operation not permitted was repeatedly showing in the /var/log/mysql/mysql.err MySQL logs.

The backup of audit log could take longer after upgrading to Elasticsearch 8.

An unhandled nil value when configuring Actions storage with AWS S3 via OIDC configuration in the terminal could cause an error.

Users were unable to sign out from gist pages.

On an instance with secret scanning enabled, the custom pattern page would not load because dry run results were tied to a deleted repository.

The "List teams" API endpoint returning duplicate results when paginating.

A model with no URL could cause a ghe-migrator import to fail.

Restore could fail when restoring MySQL using backup-utils.

The ghe-remove-node command will display the log file location when running in quiet mode.

Pre-receive hook environments can use the clone3() system call.

The creation, deletion, or change in visibility of a gist has been added to the audit log.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

MEDIUM: An attacker could steal sensitive information by exploiting a Cross-Site Scripting vulnerability in the repository transfer feature. This exploitation would require social engineering. GitHub has requested CVE ID CVE-2024-8770 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could push a commit with changes to a workflow using a PAT or OAuth app that lacks the appropriate workflow scope by pushing a triple-nested tag pointing at the associated commit. GitHub has requested CVE ID CVE-2024-8263 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: A GitHub App installed in organizations could upgrade some permissions from read to write access without approval from an organization administrator. An attacker would require an account with administrator access to install a malicious GitHub App. GitHub has requested CVE ID CVE-2024-8810 for this vulnerability, which was reported via the GitHub Bug Bounty Program. [Updated: 2024-11-07]

For instances deployed on AWS with IMDSv2 enforced, fallback to private IPs was not successful.

A config apply run may not have been properly applied due to calls being made to Nomad before it was ready to accept connections. When this occurred, the Error querying agent info: failed querying self endpoint: Get "http://127.0.0.1:4646/v1/agent/self" error was written to the /data/user/common/ghe-config.log file.

ghe-storage-find was sometimes unable to identify a data disk.

Replication could be stuck in an loop running ghe-repl-start because GHE_REPL_SSH_RETRY_COUNT was set to 60 by default for the whole scope of ghe-repl-start which will retry config apply (up to 60 times).

After upgrading the relevant GHES version, the resolvconf service failed to start due to a missing directory.

When configuring a high availability replica and during the database seeding of a MySQL replica node, restarting the nomad service could time out. Consequently, when MySQL replication attempted to start an error was reported, and setting up replication failed.

Some pre-receive hooks using the faccessat2 system call, such as those using Alpine Linux as the base, failed unexpectedly.

Placing Nomad jobs would not allow retries in cases when Nomad wasn't available yet.

A repeated error message concerning connectivity to port 6002 was emitted to the system logs when Actions was enabled.

On an instance in a cluster configuration, the ghe-cluster-status command returned an error if a soft-deleted repository had a checksum mismatch.

Some repositories could miss spokes information after restoring in a clustering topology due to unrescued exceptions.

In organizations with a large number of repositories, when an administrator used repository properties to target repositories in an organization ruleset, the ruleset index page timed out.

After a user created a Projects Insights chart with time as the X-axis, the chart became hidden and inaccessible.

The CommandPalette component no longer displays repository information on 404 pages, preventing the leakage of private repository information for users without access.

A bug introduced in 3.12 which prevented the search input in the global navigation from displaying a dropdown of search suggestions has been fixed. The search input functionality prior to 3.12 has been restored, and users are once again able to see and submit suggested search queries, including scope suggestions.

Custom links to other repositories displayed incorrect breadcrumbs.

Some custom pattern matches were incorrectly filtered during post-scan filtering and outdated alerts were sometimes published. You may want to edit and republish your custom patterns. You can manually republish custom patterns with the following command: ghe-secret-scanning jobs queue custom-patterns republish --custom-pattern-id=?.

On an instance with secret scanning enabled, a banner indicated that secret scanning was running on pull request comments and discussions. This feature is not available in this version of GitHub Enterprise Server.

Memory utilization would sometimes exceed levels comparable to GitHub Enterprise Server 3.12.

Some custom pattern matches were incorrectly filtered during post-scan filtering and outdated alerts were sometimes published. You may want to edit and republish your custom patterns. You can manually republish custom patterns with the following command: ghe-secret-scanning jobs queue custom-patterns republish --custom-pattern-id=?.

For instances deployed on Amazon Web Services (AWS), site administrators can configure regional AWS STS endpoints for OIDC from the Management Console.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

For customers using Secret Scanning, internal jobs were created and not worked that could contribute to performance issues.

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

The Known issues section previously indicated that Instance setup in AWS with IMDSv2 enforced fails if no public IP is present is still an issue. The issue is resolved and is documented in the Bug fixes section. [Updated: 2024-09-30]

Users can view the app state of gists, networks, and wikis in the spokesctl info output, enhancing visibility into the status of these elements. Additionally, spokesctl check can diagnose and, in most cases, fix empty repository networks, improving network management.

CRITICAL: On GitHub Enterprise Server instances that use SAML single sign-on (SSO) authentication with specific IdPs utilizing publicly exposed signed federation metadata XML, an attacker could forge a SAML response to provision and/or gain access to a user account with site administrator privileges. GitHub has requested CVE ID CVE-2024-6800 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could update the title, assignees, and labels of any issue inside a public repository. This was only exploitable inside a public repository, and private/internal repositories were not affected. GitHub has requested CVE ID CVE-2024-7711 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the issue contents from a private repository using a GitHub App with only contents: read and pull requests: write permissions. This was only exploitable via user access token, and installation access tokens were not impacted. GitHub has requested CVE ID CVE-2024-6337 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

During hotpatching and sometimes when applying configuration changes, a configuration run to upgrade the GitHub Actions service was unnecessarily triggered. The GitHub Actions service will only be upgraded in GitHub Enterprise Server feature releases.

On an instance with GitHub Actions enabled, during a hotpatch upgrade, a race condition could block various upgrade activities.

The ghe-config-apply process made an unnecessary number of connections to Redis.

Upgrading the Dependency Graph sometimes failed due to outdated data from go.sum manifests.

Restarting the resolvconf service would not correctly update the contents of /etc/resolv.conf.

The configuration log at /data/user/common/ghe-config.log was no longer rotated to /data/user/config-apply/logs/ after each config apply run. This was because a regular expression failed to match after timestamps were added to the config apply log.

Empty lines were inserted into the configuration log at /data/user/common/ghe-config.log.

Instances installed on Google Cloud Platform (GCP) could have their hostname overwritten by GCP when a hotpatch was applied.

The minimum password requirements for Management Console users and the root site administrator required an upper case character when providing a password with a minimum of 8 characters, contradicting the documentation and password hint.

The ghe-migrations utility for visualizing migrations did not work due to a regression. Administrators can now run ghe-migrations to view the progress and status of github migrations, or run ghe-migrations --all to view progress on all services.

On an instance with subdomain isolation enabled, configuration runs created subdomains for ChatOps services, such as slack.HOSTNAME and teams.HOSTNAME, regardless of whether the service was enabled.

Audit log data migration failed on instances using a legacy Elasticsearch data directory.

When clicking the help link under the Authentication header in enterprise-manage, the user would be redirected to /admin/managing-accounts-and-repositories instead of /admin/managing-iam/understanding-iam-for-enterprises/about-identity-and-access-management.

During support bundle generation or when running ghe-diagnostics, filesystem usage for the Elasticsearch data directory was not be included.

On an instance with GitHub Actions enabled, due to an insufficient wait time, MS SQL and MySQL replication could fail with the error message Failed to start nomad service!.

Site administrators could not switch maintenance mode directly from "scheduled" to "on," or vice versa.

Some users were unable to delete project views.

On the repository settings page for GitHub Pages, users saw an option to upgrade to GitHub Enterprise to use GitHub Pages with private visibility.

When importing using ghe-migrator, team URLs containing dots were imported as-is, leading to 404s when attempting to view the imported teams. Dots in imported team URLs are now escaped to dashes.

In the file tree on the "Files changed" tab of a pull request, users could not collapse or expand directories.

Due to a regression introduced in a previous patch, for enterprises that use encrypted SAML assertions, SSO attempts failed with a digest mismatch error if the entire SAML response was signed, rather than just the assertions.

Administrators sometimes saw an error message when visiting the administrative search page.

On an instance with subdomain isolation enabled, images served from a subdomain or external source did not render correctly in issues opened in the Projects side panel.

Running go get for a Golang repository with a directory structure that overlaps with GitHub UI routes failed

The wrong help link was displayed when push protection blocked a secret from the CLI.

Embedded images in wiki pages were broken.

For repositories with issues disabled, issue links were redirected to pull requests.

In custom pre-receive hooks, the paths stored in environment variables that allow for newly pushed objects to be in a quarantine directory could be incorrectly interpreted as relative to a worktree instead of the Git directory, causing certain commands to fail to read from the repository. The variables now use absolute paths.

A corrupted entry in the Git audit log could cause out of memory errors.

Fixes and improvements for the git core module.

When enabling GitHub Advanced Security for an organization, active committers in other organizations were not accounted for.

Following an upgrade, Elasticsearch search migrations are sometimes incorrectly reported as failing in the audit log, even though the migrations completed successfully. [Updated: 2024-09-27]

Actions KPI logs are disabled by default to reduce log size.

When running ghe-support-bundle, the support bundle includes the Elasticsearch config.

In the site admin dashboard, administrators have more granular options for the maximum object size in repositories.

Users can set their styling preference for link underlines in the web interface, on their "Accessibility" settings page.

Audit log events related to audit log streaming are available in the enterprise audit log page, and via audit log streaming.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

TokenScanningServiceMetricsApiError errors may appear after the upgrade.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Memory utilization may increase after the upgrade. During periods of high traffic, interruptions in service may occur due to insufficient memory allocations for internal components.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

Including ../ when editing a file name does not move the file up a directory level.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the elasticsearch indices need to be reindexed before some of the data will show up. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

The global search bar does not have suggestions enabled due to the redesigned navigation and pending new search experience.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Instance setup in AWS with IMDSv2 enforced fails if no public IP is present.

On boot, the resolvconf service may fail to start because the /run/resolvconf directory does not exist when the service attempts to touch a file there, with the error:

/bin/touch: cannot touch '/run/resolvconf/postponed-update': No such file or directory

If this occurs, workaround this issue with the following commands — this change will persist on reboots, but not upgrades:

sudo sed -i.bak \
'/\[Service\]/a ExecStartPre\=\/bin\/mkdir \-p \/run\/resolvconf' \
/etc/systemd/system/resolvconf.service.d/local.conf

sudo systemctl daemon-reload
sudo systemctl start resolvconf 

[Updated: 2024-08-26]

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

These release notes previously indicated as a known issue that on GitHub Enterprise Server 3.13.3 when log forwarding is enabled, some forwarded log entries may be duplicated. The fix for this problem was already included in GitHub Enterprise Server 3.13.2. [Updated: 2024-09-16]

HIGH: An attacker could cause unbounded resource exhaustion on the instance by sending a large payload to the Git server. To mitigate this issue, GitHub has limited the count of "have" and "want" lines for Git read operations. GitHub has requested CVE ID CVE-2024-5795 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An improper privilege management vulnerability allowed users to migrate private repositories without having appropriate scopes defined on the related personal access token. GitHub has requested CVE ID CVE-2024-5566 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized access in a public repository using a suspended GitHub App via a scoped user access token. This was only exploitable in public repositories while private repositories were not impacted. GitHub has requested CVE ID CVE-2024-5816 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could execute a Cross Site Request Forgery (CSRF) attack to perform write operations on a victim-owned repository in GitHub Enterprise Server by exploiting incorrect request types. A mitigating factor is that the attacker has to be a trusted user and the victim has to visit a tag in the attacker's fork of their own repository. GitHub has requested CVE ID CVE-2024-5815 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the name of a private repository on the GitHub Enterprise Server appliance when the private repository has a deploy key associated to it. GitHub has requested CVE ID CVE-2024-6395 for this vulnerability, which was reported via the GitHub Bug Bounty program.

LOW: Instance administrators could see fine-grained personal access tokens in plaintext in the babeld and gitauth logs.

LOW: An attacker with read access to a project could use the REST API to view a list of all members in an organization, including members who had made their membership private. This vulnerability was reported via the GitHub Bug Bounty program.

LOW: An attacker could include MathJax syntax in Markdown to bypass GitHubs normal restrictions on CSS properties in Markdown. This vulnerability was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized read access to issue content inside an internal repository via GitHub projects. This attack required attacker access to the corresponding project board. GitHub has requested CVE ID CVE-2024-5817 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could gain unauthorized access to secret scanning alert data because the REST API secret scanning endpoint did not properly verify whether the user account has the business owner role. Only organization members can exploit this vulnerability, requiring a personal access token (PAT) with repo or security_events scopes, limiting exposure to internal actors. Exploitation also required secret scanning to be enabled on user-owned repositories. GitHub has requested CVE ID CVE-2024-10824 for this vulnerability. [Updated: 2024-11-07]

An attacker could access previously executed private required workflows by changing the repository visibility from private to public. This occurred despite the repositories with the required workflows remaining private. This vulnerability was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

When an instance hosted on Azure was upgraded with a hotpatch, the upgrade failed with an rsync error.

On an instance with GitHub Actions enabled, remote blob storage could fill up with large amounts of data because cleanup jobs were skipped on old hosts.

The ghe-cluster-repl-status command could be run on instance configurations other than high-availability clusters, resulting in an incorrect or incomplete status.

The threshold set by server_rejoin_age_max for single-node GHES deployments was too low.

On an instance in a cluster configuration, former primary nodes were able to access the newly promoted nodes after failover.

In some cases, commands run in an administrative SSH shell were not written to the audit log.

When an administrator submitted support data to GitHub Support, spokesd keys were incorrectly sanitized.

When log forwarding was enabled, some specific service logs, including babeld, gitauth, unicorn, and resqued, were duplicated.

During the initial boot of an instance, a data disk attached as /dev/sdb may not have been recognized as an available disk.

In a high availablity configuration, running ghe-repl-node multiple times from a node that didnt have replication running had the potential to overwrite the configuration on the primary node.

Configuration history is only generated for instances in a cluster, high availability (HA) cluster, or standalone HA configuration. The current node must be a primary or replica node with replication running.

In some cases, the HAProxy kill_timeout setting caused service outages during upgrades or large transactions.

The ssh-audit-log.sh script did not effectively log SSH commands, and the ghe-sanitize-log.psed script inadequately sanitized password-related logs.

For an instance running on Microsoft Azure, the user disk service failed to start because the attached volume could not be found.

When analyzing a repository with code scanning, the extractor logs only contained warnings and errors for some languages.

The GitHub Desktop option in the Open with... edit menu was not shown unless github.dev was also enabled.

When transferring a repository, the required properties for one organization continued to be displayed even after a user chose a different owner.

Establishing a new GitHub Connect connection could fail with a 500 error.

When using ghe-migrator to migrate a repository, the links for pull requests merge commits were not imported.

When a user used the REST API endpoints that returned secret scanning alerts at the repository or organization level with non-cursor-based pagination (for example, without before or after query parameters), the REST API endpoints for secret scanning returned incorrect Link headers.

On certain branch names, the branch info bar was causing frozen string errors.

On instances with SAML authentication configured, users were unable to sign out and became stuck in an infinite SAML SSO loop.

On instances with SCIM enabled, the administrator was unable to view users without an external identity record (for example, because they were provisioned before SCIM was enabled on the instance) in stafftools.

On instances enrolled in the SCIM private beta, built-in authentication users can be added to organizations and teams. Organization owners will no longer see the misleading message that the organization membership is managed by the SAML identity provider when updating organization memberships.

Enterprise owners managed by an identity provider were asked to authenticate within GitHub when performing privileged actions.

On an instance that restricts emails to verified domains, secret scanning emails would sometimes be sent to an unverified domain.

In some cases, on the "Files" tab of a pull request, a comment on the first line did not render.

Some organizations were not recognized as part of an instance's enterprise account.

Some users would encounter an error when navigating to their personal security settings page at https://HOSTNAME/settings/security.

The SpokesSyncCacheReplicaJob could not initialize in some cases, resulting in an exception when handling the error.

In the sidebar menu that is displayed when a user clicks their profile picture, users who are not enterprise owners saw an "Enterprise settings" option, linking to the main page of an enterprise. This option is now labeled "Your enterprise".

On the "Code scanning" page of a repository, the branch filter did not correctly display all branches.

The video player did not load a video that was uploaded to an issue.

The warning message irb: warn: cant alias delete from irb_delete would appear during Support Bundle creation and upload.

When including a .gitignore or README.md file on repository creation failed due to a ruleset or pre-receive hook, no error message displayed.

On an instance with a GitHub Advanced Security license, requests to the /enterprises/{enterprise}/settings/billing/advanced-security REST API endpoint could fail due to timeout.

The global enterprise overview page contained a "Give feedback" link that was only intended for GitHub Enterprise Cloud.

Organizations named "C" were incorrectly routed to the GitHub Enterprise Server contact page instead of their organization page.

On an instance with a GitHub Advanced Security license, commits made by users who do not belong to an organization were not counted.

Due to a regression, adding ../ when editing a files name did not result in the file being moved up a directory level.

When servers responded with unsupported characters, webhook deliveries were not displayed in the UI.

Chat integrations required frequent reauthentication, as a result of new app installations overwriting previous ones.

On an instance in a cluster configuration, the ghe-spokesctl ssh command did not select the correct Nomad container when running a command within a git repository.

On an instance with a GitHub Advanced Security license, contributions were not tracked on public repositories.

The "Adjust configuration" step failed when enabling code scanning with default setup on self-hosted Windows runners.

Migration of the issue_edits table caused intermittent failures during the upgrade to GitHub Enterprise Server version 3.13, resulting in the error message ActiveRecord::ConcurrentMigrationError: Failed to release advisory lock. [Updated: 2024-08-14]

In a high availability configuration, users can only run ghe-config-apply or ghe-cluster-config-apply on a replica node if replication is already running (from ghe-repl-start). If replication isnt running on the node, the user will be instructed to start replication.

Configuration history has been extended. When ghe-config-apply, ghe-cluster-config-apply, or ghe-config-archive is run: secrets.conf is captured, a sha256sum for each of the current configuration files is included, the existing patch that is generated includes secrets.conf, and an additional sanitized patch that excludes secrets.conf is also generated.

The timeout for requests made to the REST API endpoints for secret scanning has been extended.

A more specific error message is shown when a non-provisioned user tried to sign in to an instance with SCIM enabled.

A more specific error message is shown when a deprovisioned user attempts signing into an instance with SCIM enabled.

In the audit logs, administrators can see more context for failed user authentication attempts using LDAP.

The system logs provide more context for authentication failures related to multi-factor authentication.

When using the ghe-webhook-logs utility, webhook delivery logs can be filtered by event and action. Users can use ghe-webhook-logs --event issues to filter by event, or ghe-webhook-logs --event issues.opened to filter by event and action.

To avoid excessive log volume and associated disk pressure, requests for GetCacheKey are no longer logged. Previously, the high frequency of these requests caused significant log accumulation.

When restoring data originally backed up from a 3.13 appliance onto a 3.13 appliance, the Elasticsearch indices need to be reindexed before some data will appear. This happens via a nightly scheduled job. It can also be forced by running /usr/local/share/enterprise/ghe-es-search-repair.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

TokenScanningServiceMetricsApiError errors may appear after the upgrade.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Memory utilization may increase after the upgrade. During periods of high traffic, interruptions in service may occur due to insufficient memory allocations for internal components.

Running a config apply as part of the steps for Replacing a node in an emergency may fail with errors if the node being replaced is still reachable. If this occurs, shutdown the node and repeat the steps.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

Following an upgrade, Elasticsearch search migrations are sometimes incorrectly reported as failing in the audit log, even though the migrations completed successfully. [Updated: 2024-08-02]

Images embedded in wiki pages may stop rendering shortly after being published. [Updated: 2024-10-16]

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

Instance administration

• The root navigational experience for enterprise accounts lands all users on an "Enterprise Overview". From this page, enterprise owners can create a README for their enterprise, which will be visible internally to all enterprise members. The "Organization" page still exists and can be accessed from the left sidebar of the enterprise account.

• To improve the pre-flight checks experience, all pre-flight checks run even if one check fails. A consolidated report of the results is shown in the UI.

• The editor role for a Management Console user has been deprecated in the Manage GitHub Enterprise Server API.

• People deploying a GitHub Enterprise Server instance in AWS can now deploy in an environment that uses Instance Metadata Service Version 2 (IMDSv2).

• As part of the upgrade to GitHub Enterprise Server 3.13, Elasticsearch (ES) is upgraded from version 5.6.16 to 8.7.0. Upgrading platform components improves performance and security posture. For important upgrade considerations, see Preparing for the Elasticsearch upgrade in GitHub Enterprise Server 3.13.

• To improve existing tooling for license handling, the ghe-license script handles all operations regarding the active license. Commands can be performed on new licenses without importing them first. The script allows direct application of the license without a full configuration run and avoids restarting the instance to reduce downtime. See Command-line utilities.

  Administrators can upload the license to their instance using multiple interfaces, including the Management Console, Manage GHES API, CLI, or SSH. See Uploading a new license to GitHub Enterprise Server.

Audit logs

• Enterprise and organization audit log events include the applicable SAML and SCIM identity data associated with the user. This data provides increased visibility into the identity of the user and enables logs from multiple systems to quickly and easily be linked using a common corporate identity. The SAML identity information displays in the external_identity_nameid field and the SCIM identity data displays in the external_identity_username field within the audit log payloads. For more information, see Reviewing the audit log for your organization.

GitHub Actions

• For self-hosted GitHub Actions runners on this GitHub Enterprise Server release, the minimum required version of the GitHub Actions Runner application is 2.314.1. See the release notes for this version in the actions/runner repository. If your instance uses ephemeral self-hosted runners and you've disabled automatic updates, you must upgrade your runners to this version of the Runner application before upgrading your instance to this GitHub Enterprise Server release.

• To ensure Actions runners are truly ephemeral and more secure, execution timeouts on self-hosted jobs are limited to 5 days. If a job reaches this limit, the job is terminated and fails to complete. For more information, see About self-hosted runners.

Repositories

• Users can use repository properties to add meaningful metadata to repositories that simplifies repository classification, enhances discoverability, and seamlessly integrates with rulesets. For more information, see Managing custom properties for repositories in your organization.

• Users can browse and view code in a revamped experience for GitHub repositories, providing a tree pane for browsing files, fuzzy search for files, sticky code headers, and more.

• Users can migrate existing tag protection rules into repository rules. For more information, see Configuring tag protection rules.

Projects

• Users can post status updates on their projects to share the current status, start date, and target date of the project itself. For more information, see Sharing project updates.

• Users can migrate their projects (classic) to the new Projects experience. For more information, see Migrating from projects (classic).

Pull requests

• Rebase commits are now created using the merge-ort strategy.

Secret scanning

• In the secret scanning list view, users can apply a filter to display alerts that are the result of having bypassed push protection. For more information, see Managing alerts from secret scanning.

• To increase coverage of secret scanning across an instance, users can enable secret scanning in repositories owned by their personal account. Enterprise owners can disable this feature, or automatically enable it for all new user-owned repositories, in the enterprise settings. See Managing GitHub Advanced Security features for your enterprise.

Code scanning

• Users can enable code scanning on repositories even if they don’t contain any code written in the languages currently supported by CodeQL. Default setup will automatically trigger the first scan when a supported language is detected on the default branch. For more information, see Configuring default setup for code scanning.

• Users can use CodeQL threat model settings for Java to adapt CodeQL's code scanning analysis to detect the most relevant security vulnerabilities in their code. This feature is in public beta and subject to change. For more information, see Customizing your advanced setup for code scanning.

• The CodeQL action for code scanning analysis uses version 2.16.5 of the CodeQL CLI by default, an upgrade from 2.15.5 compared to the previous GitHub Enterprise Server feature release. For a detailed list of changes included in each version, see the CodeQL change logs. Significant changes include:

  • Support for Swift 5.9.2, C# 12 / .NET 8, and Go 1.22.
  • Installation of Python dependencies is disabled for all Python scans by default. See the GitHub Blog post.
  • A new python_executable_name option for the Python extractor. This allows you to select a non-default Python executable installed on the system running the scan (such as py.exe on Windows machines). See the changelog in the CodeQL documentation.
  • A fix for CVE-2024-25129, a low-severity data exfiltration vulnerability that could be triggered by processing untrusted databases or CodeQL packs.
  • The code scanning UI now includes partially extracted files. See the GitHub Blog post.
  • 2 new C/C++ queries: cpp/use-of-unique-pointer-after-lifetime-ends and cpp/incorrectly-checked-scanf
  • 6 new Java queries: java/insecure-randomness , java/exec-tainted-environment , java/android/sensitive-text, java/android/sensitive-notification, java/android/insecure-local-authentication, and java/android/insecure-local-key-gen
  • 2 new Swift queries: swift/weak-password-hashing and swift/unsafe-unpacking

Code security

• On the security overview dashboard, users can find detailed insights for the security alerts in an organization or enterprise, including trending data that tracks alert counts and activity over time and snapshot data that reflects the current state of the security landscape. Alerts are displayed for both GitHub's security features and third-party tools. Filters are available for the type and visibility of alerts, date range, repository custom properties, and more. The overview dashboard is in public beta and subject to change. For more information, see Viewing security insights.

• Users can view trending data for the enablement of security features in an organization. In security overview for an organization, the "Enablement trends" view shows historical data for the activation of security features including Dependabot updates, code scanning alerts, and secret scanning alerts. This feature is in public beta and subject to change. For more information, see Assessing adoption of code security features.

• For users who use devcontainer.json files to define development containers for repositories, Dependabot version updates can keep "features" defined for the dev container up to date. Once configured in dependabot.yml, Dependabot will open pull requests on a specified schedule to update the listed features to the latest version. Dependabot security updates for dev containers are not currently supported. For more information, see About Dependabot version updates.

Authentication

• For enterprises or organizations that use an SSH certificate authority (CA) to provide SSH certificates to members, to protect against a security risk involving user renames, new SSH CAs that are uploaded to a GitHub Enterprise Server 3.13 instance can only be used to sign certificates that are set to expire. For new CAs, you must use the -V parameter with ssh-keygen to generate a certificate with a valid-after claim.

  The valid-after claim allows GitHub to validate that the user named in the SSH certificate hasn't been renamed since the certificate was signed. CAs uploaded prior to version 3.13 are exempt from this requirement and can be used to sign certificates that do not expire. However, when you've ensured that your certificate signing process uses the -V flag, GitHub encourages you to upgrade existing certificates to enforce the expiration requirement. For more information, see Managing your organization's SSH certificate authorities or Enforcing policies for security settings in your enterprise.

TCP port 9103 is opened for future administrative features related to support for Prometheus scraping. The port has been open since GitHub Enterprise Server 3.12, but this change wasn't communicated at the time release notes for version 3.12 were first published.

Upcoming change: In version 3.14 and later of GitHub Enterprise Server, for instances with GitHub Actions and GitHub Connect enabled, self-hosted runners that download actions from GitHub.com via GitHub Connect will need to allow access to the following new hosts.

• ghcr.io
• *.actions.githubusercontent.com

Please update the outbound firewall rules on your self-hosted runners to allow requests to these services. You can make this change on version 3.13, or on a previous version of GitHub Enterprise Server. For a smooth upgrade to version 3.14, we recommend you make changes to your firewall rules now, as failing to do so will result in your runners being unable to download certain actions in version 3.14 and later.

The "Create a reference" REST API endpoint is restricted from accepting POSTs from users and apps that only have permission to read and write packages. Previously, this endpoint accepted updates to both tags and branches.

To ensure security updates are applied correctly regardless of your repository's configuration settings, Dependabot uses private registry configurations specified in the dependabot.yml file as expected, even if there is a configuration with target-branch. Security updates still do not support target-branch configuration. For more information, see Configuring access to private registries for Dependabot.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

When enabling log forwarding, specific service logs, including babeld, are duplicated. For more information, see Log forwarding.

Repositories originally imported using ghe-migrator do not correctly track committers for GitHub Advanced Security billing.

When log forwarding is enabled, some forwarded log entries may be duplicated.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

TokenScanningServiceMetricsApiError errors may appear after the upgrade.

The log entry irb: warn: can't alias delete from irb_delete may appear during creation and upload of support bundles.

The admin stats REST API endpoints may time out on appliances with many users or repositories. Retrying the request until data is returned is advised.

When following the steps for Replacing the primary MySQL node, step 14 (running ghe-cluster-config-apply) might fail with errors. If this occurs, re-running ghe-cluster-config-apply is expected to succeed.

Running ghe-cluster-config-apply as part of the steps for Replacing a node in an emergency might fail with errors if the node being replaced has not first been turned off. If this occurs, turn the node off and repeat the steps.

For an instance in a cluster configuration and with GitHub Actions enabled, restoring a cluster from backup requires targeting the primary DB node.

Memory utilization may increase after the upgrade. During periods of high traffic, interruptions in service may occur due to insufficient memory allocations for internal components.

Following an upgrade, Elasticsearch search migrations are sometimes incorrectly reported as failing in the audit log, even though the migrations completed successfully. [Updated: 2024-08-02]

Images embedded in wiki pages may stop rendering shortly after being published. [Updated: 2024-10-16]

When operating in a high availability configuration, running ghe-repl-promote on a replica node will fail if the original primary cannot be reached by the replica node. This is because the ghe-repl-promote script attempts to decommission all Elasticsearch nodes other than the promoted node, however these requests are made to the original primary node which is no longer reachable. The error message written to the terminal will be similar to:

Maintenance mode has been enabled for active replica <REPLICA_HOSTNAME>
{"message": "No server is currently available to service your request. Sorry about that. Please try resubmitting your request and contact your local GitHub Enterprise site administrator if the problem persists."}
jq: error (at :3): Cannot index string with string "node"

If this occurs, workaround this issue by running the following command — this changes the ghe-repl-promote script in place:

sudo sed -i.bak -e '/for node_hostname in/i if ! $forced; then' -e '/^  done/a fi' /usr/local/bin/ghe-repl-promote

Then re-run the updated ghe-repl-promote script.

[Updated: 2024-11-29]

As part of sunsetting Subversion compatibility, Subversion support is now disabled by default. Subversion can be re-enabled in the 3.13 release series by setting app.svnbridge.enabled = true. In 3.14, subversion support will be permanently removed. For more information, see Sunsetting Subversion support on the GitHub blog.

The Manage GHES API reached feature parity with the Management Console API in GHES 3.12. As a result, we will remove the Management Console API in GitHub Enterprise Server 3.15. For information about updating tooling that relies on the Management Console API, see REST API endpoints for Management Console.

From November 19, 2024, references to v1 and v2 of artifacts actions in GitHub Actions will not resolve. GitHub deprecated v1 and v2 of actions/upload-artifact, actions/download-artifact, and related npm packages on June 30, 2024. You can read more about this deprecation on the GitHub Blog. GitHub Enterprise Server instances configured to use GitHub Connect to download these actions will need to store cached copies locally for workflows to continue working. If your local copy of these actions has been removed, use GitHub Actions Sync to manually re-download the actions. [Updated: 2024-18-20]

The deprecated v1 and v2 versions of artifacts actions will be removed from GitHub Enterprise Server 3.15 onwards. Users should update their workflows to use v3 or later versions of artifacts actions. [Updated: 2024-18-20]

The Deprecations section previously indicated that the Management Console API would be deprecated in GitHub Enterprise Server 3.14. Instead, the Management Console API will be removed in GitHub Enterprise Server 3.15. [Updated: 2024-07-08]