Enterprise Server 3.12 release notes

Packages have been updated to the latest security versions.

The --no-async flag was not implemented for the ghe-cluster-support-bundle command, leading to a potentially increased load.

In a high availability configuration, with GitHub Actions, replication would fail on nodes where MSSQL was not configured to run.

The audit log cluster rebalance script incorrectly proceeded before all shards were ready. This caused the script to exit before the necessary data was available, potentially leading to issues with the audit log migration.

For instances hosted on Azure, if a pre-upgrade check failed due to insufficient user disk size, the Management Console displayed an internal server error.

On an instance with secret scanning enabled, when selecting repositories for a dry run of an enterprise-level custom pattern, searches for full repository names (ORGANIZATION/REPOSITORY) did not return results.

When creating a pre-receive hook environment, attempts to include an image URL over 255 characters failed with a database error. The maximum length is still 255 characters, but the URL length is now validated before the process starts.

Performing a browser back navigation to a pull request now displays up-to-date status checks.

Subversion services were non-functional in some cases.

When exporting repositories to blob storage using the migrations REST API endpoint to start an organization migration, the maximum compressed archive size is limited to 90 GB. This is an increase from 30 GB.

Pull request merges are handled more efficiently, allowing more Git objects to be created before timeout. Additionally, loose objects created by merges that time out are now discarded, limiting the accumulation of these objects.

When exporting repositories using the migrations REST API, prior to blob storage upload the tarball is staged in the root volume. For more disk capacity, the tarball will now be staged in the data volume.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation. When regenerating the certificates with management console, the subdomain reply.[hostname] is missing from the ssl certification.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Some customers upgrading from 3.11.x or 3.12.x may experience a bug with the feature "Automatic update checks", filling the root disk with logs causing a system degradation. To prevent this, you can turn off the feature Enable automatic update check in the management console.

Packages have been updated to the latest security versions.

Creating a new comment on a pull request could incorrectly emit a pull_request_review_comment.edited webhook event.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation. When regenerating the certificates with management console, the subdomain reply.[hostname] is missing from the ssl certification.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Attempting to stop replications after stopping GitHub Actions on a GitHub Enterprise Server instance would fail, reporting that MSSQL was not responding. The can be avoided by start MSSQL prior to stopping replication /usr/local/share/enterprise/ghe-nomad-jobs queue /etc/nomad-jobs/mssql/mssql.hcl

Some customers upgrading from 3.11.x or 3.12.x may experience a bug with the feature "Automatic update checks", filling the root disk with logs causing a system degradation. To prevent this, you can turn off the feature Enable automatic update check in the management console.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This is a follow up fix for CVE-2024-9487 to further harden the encrypted assertions feature against this type of attack. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document to exploit this vulnerability.

HIGH: An attacker with Enterprise Administrator access to the GitHub Enterprise Server instance could escalate privileges to SSH root access. This is achieved by exploiting the pre-receive hook environment to bypass symlink checks in the ghe-firejail path and execute malicious scripts. GitHub has requested CVE ID CVE-2024-10007 for this vulnerability, which was reported via the GitHub Bug Bounty program. [Updated: 2024-11-07]

This error message mbind: Operation not permitted was repeatedly showing in the /var/log/mysql/mysql.err MySQL logs.

When saving settings in the Management Console, the configuration run would stop if the enterprise-manage process was restarted.

A missing configuration value prevented Dependabot from creating group update pull requests.

On an instance with GitHub Actions enabled, some maintenance tasks could fail due to incomplete upgrade steps during previous upgrades to new releases of GitHub Enterprise Server.

The initial setup certificate generation in AWS took longer than expected due to fallback to private IPs. The time for this fallback has been reduced.

If the primary instance was unreachable, running ghe-repl-stop --force on a replica would fail during the config apply run.

When restoring from a backup, repositories that had been deleted in the last 90 days were not completely restored.

Restoring Git repositories using backup-utils occasionally failed.

Organizations were limited to using 100 Actions organization variables instead of 1,000.

Some customers upgrading from 3.12 to 3.13 or to 3.14 may experience issues with undecryptable records during the upgrade. This issue has now been resolved. We recommend you read Undecryptable records.

For instances deployed on AWS, the default settings for Chrony NTP synchronization have been aligned with AWS's suggested default configurations.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. See Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

Repositories originally imported using ghe-migrator will not correctly track GitHub Advanced Security contributions.

The reply.[HOSTNAME] subdomain is falsely always displaying as having no SSL and DNS record, when testing the domain settings via the Management Console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

MEDIUM: Malicious URLs for SVG assets provided information about a victim user who clicked the URL, allowing an attacker to retrieve metadata belonging to the user and use it to generate a convincing phishing page. This required the attacker to upload malicious SVGs and phish a victim user to click the URL for the uploaded asset. GitHub has requested CVE ID CVE-2024-9539. This vulnerability was reported via the GitHub Bug Bounty program.

HIGH: An attacker could bypass SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, allowing unauthorized provisioning of users and access to the instance, by exploiting an improper verification of cryptographic signatures vulnerability in GitHub Enterprise Server. This was a regression introduced as part of follow-up remediation from CVE-2024-4985, which resulted in a new variant of the vulnerability. Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO, or utilizing SAML SSO authentication without encrypted assertions, are not impacted. Additionally, an attacker would require direct network access as well as a signed SAML response or metadata document. GitHub has requested CVE ID CVE-2024-9487. This vulnerability was reported via the GitHub Bug Bounty program.

HAProxy reloading was failure prone, which could lead to failed Git operations. This reloading process has been replaced with a more resilient Systemd process.

An unhandled nil value when configuring Actions storage with AWS S3 via OIDC configuration in the terminal could cause an error.

On an instance with secret scanning enabled, the custom pattern page would not load because dry run results were tied to a deleted repository.

The "List teams" API endpoint returning duplicate results when paginating.

A model with no URL could cause a ghe-migrator import to fail.

Restore could fail when restoring MySQL using backup-utils.

The ghe-remove-node command will display the log file location when running in quiet mode.

Pre-receive hook environments can use the clone3() system call.

The creation, deletion, or change in visibility of a gist has been added to the audit log.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation.

The admin stats REST API endpoints may time out on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

MEDIUM: An attacker could steal sensitive information by exploiting a Cross-Site Scripting vulnerability in the repository transfer feature. This exploitation would require social engineering. GitHub has requested CVE ID CVE-2024-8770 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could push a commit with changes to a workflow using a PAT or OAuth app that lacks the appropriate workflow scope by pushing a triple-nested tag pointing at the associated commit. GitHub has requested CVE ID CVE-2024-8263 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: A GitHub App installed in organizations could upgrade some permissions from read to write access without approval from an organization administrator. An attacker would require an account with administrator access to install a malicious GitHub App. GitHub has requested CVE ID CVE-2024-8810 for this vulnerability, which was reported via the GitHub Bug Bounty Program. [Updated: 2024-11-07]

For instances deployed on AWS with IMDSv2 enforced, fallback to private IPs was not successful.

A config apply run may not have been properly applied due to calls being made to Nomad before it was ready to accept connections. When this occurred, the Error querying agent info: failed querying self endpoint: Get "http://127.0.0.1:4646/v1/agent/self" error was written to the /data/user/common/ghe-config.log file.

ghe-storage-find was sometimes unable to identify a data disk.

After upgrading the relevant GHES version, the resolvconf service failed to start due to a missing directory.

When configuring a high availability replica and during the database seeding of a MySQL replica node, restarting the nomad service could time out. Consequently, when MySQL replication attempted to start an error was reported, and setting up replication failed.

Placing Nomad jobs would not allow retries in cases when Nomad wasnt available yet.

On an instance in a cluster configuration, the ghe-cluster-status command returned an error if a soft-deleted repository had a checksum mismatch.

Some repositories could miss spokes information after restoring in a clustering topology due to unrescued exceptions.

After a user created a Projects Insights chart with time as the X-axis, the chart became hidden and inaccessible.

The CommandPalette component no longer displays repository information on 404 pages, preventing the leakage of private repository information for users without access.

Custom links to other repositories displayed incorrect breadcrumbs.

A bug introduced in 3.12 which prevented the search input in the global navigation from displaying a dropdown of search suggestions has been fixed. The search input functionality prior to 3.12 has been restored, and users are once again able to see and submit suggested search queries, including scope suggestions.

When a GitHub App installation had all repositories installed individually, it was not possible to remove the repositories from the selection.

Some custom pattern matches were incorrectly filtered during post-scan filtering. You may want to edit and republish your custom patterns. You can manually republish custom patterns with the following command: ghe-secret-scanning jobs queue custom-patterns republish --custom-pattern-id=?. Outdated alerts caused by edits during custom pattern backfills have been fixed in version 3.13 and above.

For instances deployed on Amazon Web Services (AWS), site administrators can configure regional AWS STS endpoints for OIDC from the Management Console.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

Users can view the app state of gists, networks, and wikis in the spokesctl info output, enhancing visibility into the status of these elements. Additionally, spokesctl check can diagnose and, in most cases, fix empty repository networks, improving network management.

CRITICAL: On GitHub Enterprise Server instances that use SAML single sign-on (SSO) authentication with specific IdPs utilizing publicly exposed signed federation metadata XML, an attacker could forge a SAML response to provision and/or gain access to a user account with site administrator privileges. GitHub has requested CVE ID CVE-2024-6800 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could update the title, assignees, and labels of any issue inside a public repository. This was only exploitable inside a public repository, and private/internal repositories were not affected. GitHub has requested CVE ID CVE-2024-7711 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the issue contents from a private repository using a GitHub App with only contents: read and pull requests: write permissions. This was only exploitable via user access token, and installation access tokens were not impacted. GitHub has requested CVE ID CVE-2024-6337 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

During hotpatching and sometimes when applying configuration changes, a configuration run to upgrade the GitHub Actions service was unnecessarily triggered. The GitHub Actions service will only be upgraded in GitHub Enterprise Server feature releases.

On an instance with GitHub Actions enabled, during a hotpatch upgrade, a race condition could block various upgrade activities.

The ghe-config-apply process made an unnecessary number of connections to Redis.

Upgrading the Dependency Graph sometimes failed due to outdated data from go.sum manifests.

Restarting the resolvconf service would not correctly update the contents of /etc/resolv.conf.

Instances installed on Google Cloud Platform (GCP) could have their hostname overwritten by GCP when a hotpatch was applied.

The minimum password requirements for Management Console users and the root site administrator required an upper case character when providing a password with a minimum of 8 characters, contradicting the documentation and password hint.

The ghe-migrations utility for visualizing migrations did not work due to a regression. Administrators can now run ghe-migrations to view the progress and status of github migrations, or run ghe-migrations --all to view progress on all services.

On an instance with subdomain isolation enabled, configuration runs created subdomains for ChatOps services, such as slack.HOSTNAME and teams.HOSTNAME, regardless of whether the service was enabled.

During support bundle generation or when running ghe-diagnostics, filesystem usage for the Elasticsearch data directory was not be included.

On an instance with GitHub Actions enabled, due to an insufficient wait time, MS SQL and MySQL replication could fail with the error message Failed to start nomad service!.

Site administrators could not switch maintenance mode directly from "scheduled" to "on," or vice versa.

Some users were unable to delete project views.

On the repository settings page for GitHub Pages, users saw an option to upgrade to GitHub Enterprise to use GitHub Pages with private visibility.

When importing using ghe-migrator, team URLs containing dots were imported as-is, leading to 404s when attempting to view the imported teams. Dots in imported team URLs are now escaped to dashes.

Due to a regression introduced in a previous patch, for enterprises that use encrypted SAML assertions, SSO attempts failed with a digest mismatch error if the entire SAML response was signed, rather than just the assertions.

On an instance with subdomain isolation enabled, images served from a subdomain or external source did not render correctly in issues opened in the Projects side panel.

In tag input fields, such as when adding topics to a repository, pressing space did not start a new tag.

Running go get for a Golang repository with a directory structure that overlaps with GitHub UI routes failed

The wrong help link was displayed when push protection blocked a secret from the CLI.

For repositories with issues disabled, issue links were redirected to pull requests.

Fixes and improvements for the git core module.

In custom pre-receive hooks, the paths stored in environment variables that allow for newly pushed objects to be in a quarantine directory could be incorrectly interpreted as relative to a worktree instead of the Git directory, causing certain commands to fail to read from the repository. The variables now use absolute paths.

A corrupted entry in the Git audit log could cause out of memory errors.

Actions KPI logs are disabled by default to reduce log size.

When running ghe-support-bundle, the support bundle includes the Elasticsearch config.

Users can set their styling preference for link underlines in the web interface, on their "Accessibility" settings page.

Audit log events related to audit log streaming are available in the enterprise audit log page, and via audit log streaming.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

The reply.HOSTNAME subdomain is falsely displayed as having no SSL and DNS record, when testing the domain settings via the Management Console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

The global search bar does not have suggestions enabled due to the redesigned navigation and pending new search experience.

When restoring from a backup snapshot, a large number of mapper_parsing_exception errors may be displayed.

Services may respond with a 503 status due to an out of date haproxy configuration. This can usually be resolved with a ghe-config-apply run.

On boot, the resolvconf service may fail to start because the /run/resolvconf directory does not exist when the service attempts to touch a file there, with the error:

/bin/touch: cannot touch '/run/resolvconf/postponed-update': No such file or directory

If this occurs, workaround this issue with the following commands — this change will persist on reboots, but not upgrades:

sudo sed -i.bak \
'/\[Service\]/a ExecStartPre\=\/bin\/mkdir \-p \/run\/resolvconf' \
/etc/systemd/system/resolvconf.service.d/local.conf

sudo systemctl daemon-reload
sudo systemctl start resolvconf 

[Updated: 2024-08-26]

These release notes previously indicated as a known issue that on GitHub Enterprise Server 3.12.8 when log forwarding is enabled, some forwarded log entries may be duplicated. The fix for this problem was already included in GitHub Enterprise Server 3.12.7. [Updated: 2024-09-16]

HIGH: An attacker could cause unbounded resource exhaustion on the instance by sending a large payload to the Git server. To mitigate this issue, GitHub has limited the count of "have" and "want" lines for Git read operations. GitHub has requested CVE ID CVE-2024-5795 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An improper privilege management vulnerability allowed users to migrate private repositories without having appropriate scopes defined on the related personal access token. GitHub has requested CVE ID CVE-2024-5566 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized access in a public repository using a suspended GitHub App via a scoped user access token. This was only exploitable in public repositories while private repositories were not impacted. GitHub has requested CVE ID CVE-2024-5816 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could execute a Cross Site Request Forgery (CSRF) attack to perform write operations on a victim-owned repository in GitHub Enterprise Server by exploiting incorrect request types. A mitigating factor is that the attacker has to be a trusted user and the victim has to visit a tag in the attacker's fork of their own repository. GitHub has requested CVE ID CVE-2024-5815 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose the name of a private repository on the GitHub Enterprise Server appliance when the private repository has a deploy key associated to it. GitHub has requested CVE ID CVE-2024-6395 for this vulnerability, which was reported via the GitHub Bug Bounty program.

LOW: Instance administrators could see fine-grained personal access tokens in plaintext in the babeld and gitauth logs.

LOW: An attacker with read access to a project could use the REST API to view a list of all members in an organization, including members who had made their membership private. This vulnerability was reported via the GitHub Bug Bounty program.

LOW: An attacker could include MathJax syntax in Markdown to bypass GitHubs normal restrictions on CSS properties in Markdown. This vulnerability was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could disclose sensitive information from a private repository exploiting organization ruleset features. This attack required an organization member to explicitly change the visibility of a dependent repository from private to public. GitHub has requested CVE ID CVE-2024-6336 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could have unauthorized read access to issue content inside an internal repository via GitHub projects. This attack required attacker access to the corresponding project board. GitHub has requested CVE ID CVE-2024-5817 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

When an instance hosted on Azure was upgraded with a hotpatch, the upgrade failed with an rsync error.

On an instance with GitHub Actions enabled, remote blob storage could fill up with large amounts of data because cleanup jobs were skipped on old hosts.

The ghe-cluster-repl-status command could be run on instance configurations other than high-availability clusters, resulting in an incorrect or incomplete status.

The threshold set by server_rejoin_age_max for single-node GHES deployments was too low.

In some cases, commands run in an administrative SSH shell were not written to the audit log.

When an administrator submitted support data to GitHub Support, spokesd keys were incorrectly sanitized.

When log forwarding was enabled, some specific service logs, including babeld, gitauth, unicorn, and resqued, were duplicated.

During the initial boot of an instance, a data disk attached as /dev/sdb may not have been recognized as an available disk.

In a high availablity configuration, running ghe-repl-node multiple times from a node that did not have replication running had the potential to overwrite the configuration on the primary node.

Configuration history is only generated for instances in a cluster, high availability (HA) cluster, or standalone HA configuration. The current node must be a primary or replica node with replication running.

In some cases, the HAProxy kill_timeout setting caused service outages during upgrades or large transactions.

The ssh-audit-log.sh script did not effectively log SSH commands, and the ghe-sanitize-log.psed script inadequately sanitized password-related logs.

The default MSSQL timeout of 8 seconds sometimes caused issues during administrator activities. The default timeout has been increased to 30 seconds.

For an instance running on Microsoft Azure, the user disk service failed to start because the attached volume could not be found.

Establishing a new GitHub Connect connection could fail with a 500 error.

When using ghe-migrator to migrate a repository, the links for pull requests merge commits were not imported.

When a user used the REST API endpoints that returned secret scanning alerts at the repository or organization level with non-cursor-based pagination (for example, without before or after query parameters), the REST API endpoints for secret scanning returned incorrect Link headers.

On certain branch names, the branch info bar was causing frozen string errors.

On instances with SAML authentication configured, users were unable to sign out and became stuck in an infinite SAML SSO loop.

On instances with SCIM enabled, the administrator was unable to view users without an external identity record (for example, because they were provisioned before SCIM was enabled on the instance) in stafftools.

After navigating to a discussion, the link underline for the Discussions tab in the GitHub UI incorrectly appeared under the Settings tab heading.

On instances enrolled in the SCIM private beta, built-in authentication users can be added to organizations and teams. Organization owners will no longer see the misleading message that the organization membership is managed by the SAML identity provider when updating organization memberships.

Enterprise owners managed by an identity provider were asked to authenticate within GitHub when performing privileged actions.

On an instance that restricts emails to verified domains, secret scanning emails would sometimes be sent to an unverified domain.

In some cases, on the "Files" tab of a pull request, a comment on the first line did not render.

Some organizations were not recognized as part of an instance's enterprise account.

Some users would encounter an error when navigating to their personal security settings page at https://HOSTNAME/settings/security.

The SpokesSyncCacheReplicaJob could not initialize in some cases, resulting in an exception when handling the error.

In the sidebar menu that is displayed when a user clicks their profile picture, users who are not enterprise owners saw an "Enterprise settings" option, linking to the main page of an enterprise. This option is now labeled "Your enterprise".

On the "Code scanning" page of a repository, the branch filter did not correctly display all branches.

When including a .gitignore or README.md file on repository creation failed due to a ruleset or pre-receive hook, no error message displayed.

On an instance with a GitHub Advanced Security license, requests to the /enterprises/{enterprise}/settings/billing/advanced-security REST API endpoint could fail due to timeout.

On some instances, users were unable to save historical insights charts for Projects.

The setting to enable or view non-provider patterns was not available for public repositories.

Users viewing the alerts index page experienced inconsistencies in rendering the closed alert state.

Organizations named "C" were incorrectly routed to the GitHub Enterprise Server contact page instead of their organization page.

On an instance with a GitHub Advanced Security license, commits made by users who do not belong to an organization were not counted.

When servers responded with unsupported characters, webhook deliveries were not displayed in the UI.

Chat integrations required frequent reauthentication, as a result of new app installations overwriting previous ones.

On an instance in a cluster configuration, the ghe-spokesctl ssh command did not select the correct Nomad container when running a command within a Git repository.

On an instance with a GitHub Advanced Security license, disabling and re-enabling GitHub Advanced Security for an organization resulted in redundant scans of some repositories.

On an instance with a GitHub Advanced Security license, contributions were not tracked on public repositories.

On an instance with a GitHub Advanced Security license, the "adjust configuration" step failed when enabling code scanning with the default setup on self-hosted Windows runners.

Migration of the issue_edits table caused intermittent failures during the upgrade to GitHub Enterprise Server version 3.12, resulting in the error message ActiveRecord::ConcurrentMigrationError: Failed to release advisory lock. [Updated: 2024-08-14]

In a high availability configuration, users can only run ghe-config-apply or ghe-cluster-config-apply on a replica node if replication is already running (from ghe-repl-start). If replication isnt running on the node, the user will be instructed to start replication.

Configuration history has been extended. When ghe-config-apply, ghe-cluster-config-apply, or ghe-config-archive is run: secrets.conf is captured, a sha256sum for each of the current configuration files is included, the existing patch that is generated includes secrets.conf, and an additional sanitized patch that excludes secrets.conf is also generated.

The timeout for requests made to the REST API endpoints for secret scanning has been extended.

A more specific error message is shown when a non-provisioned user tried to sign in to an instance with SCIM enabled.

When a user changes a repository's visibility to public, the user is now warned that previous Actions history and logs will become public as well.

A more specific error message is shown when a deprovisioned user attempts signing into an instance with SCIM enabled.

In the audit logs, administrators can see more context for failed user authentication attempts using LDAP.

The system logs provide more context for authentication failures related to multi-factor authentication.

When using the ghe-webhook-logs utility, webhook delivery logs can be filtered by event and action. Users can use ghe-webhook-logs --event issues to filter by event, or ghe-webhook-logs --event issues.opened to filter by event and action.

To avoid excessive log volume and associated disk pressure, requests for GetCacheKey are no longer logged. Previously, the high frequency of these requests caused significant log accumulation.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

Due to a known regression, operators will not be able to use the ghe-migrations visualizer to view the status of migrations during an upgrade. Instead, the operator can inspect the log files in /var/log/dbmigration to see the status and progress of migrations.

The reply.[hostname] subdomain is falsely always displaying as having no ssl and dns record, when testing the domain settings via management console without subdomain isolation.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

HIGH: An attacker with the site administrator role could gain arbitrary code execution capability on the GitHub Enterprise Server appliance when configuring audit log streaming. GitHub has requested CVE ID CVE-2024-5746 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

On an instance with GitHub Actions and External MySQL enabled, a validation step in the config apply could fail.

Users would see an error message from the server while pushing to a gist (the push would still complete).

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

ghe-migrations visualizer is not working due to a known regression. As a results, users will not be able to use ghe-migrations to view the status of migrations during an upgrade. Instead you can inspect the log files in /var/log/dbmigration to get the status/progress of migrations.

When enabling log forwarding, specific services logs (babeld and some more) are duplicated.

The reply.[hostname] subdomain is falsely always displaying as having no SSL and DNS record, when testing the domain settings via management console without subdomain isolation.

When log forwarding is enabled, some forwarded log entries may be duplicated.

Admin stats REST API endpoints may timeout on appliances with many users or repositories. Retrying the request until data is returned is advised.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected.

CRITICAL: On instances that use SAML single sign-on (SSO) authentication with the optional encrypted assertions feature, an attacker could forge a SAML response to provision and/or gain access to a user with administrator privileges.

Please note that encrypted assertions are not enabled by default. Instances not utilizing SAML SSO or utilizing SAML SSO authentication without encrypted assertions are not impacted. Exploitation of this vulnerability would allow unauthorized access to the instance without requiring prior authentication. GitHub has requested CVE ID CVE-2024-4985 for this vulnerability, which was reported via the GitHub Bug Bounty program.

For more information, see Configuring SAML single sign-on for your enterprise and Enabling encrypted assertions.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Firewall port 9199, which linked to a static maintenance page used when enabling maintenance mode with an IP exception list, was opened unnecessarily.

As a result of a security vulnerability, the editor role for a Management Console user has been deprecated in the Manage GitHub Enterprise Server API.

Packages have been updated to the latest security versions.

Running ghe-repl-node -d did not validate value length in order to prevent values longer than 20 characters.

On an instance in a cluster configuration with high availability enabled, ghe-repl-setup did not successfully complete on a replica due to a missing key.

For an instance in a cluster configuration, during the migration phase of a configuration run, the process of copying configuration updates to all nodes would fail.

Admins in the actions organization were excluded from license consumption, causing incorrect license counts.

An LDAP-related error message was incorrectly displayed at the enterprise and organization levels.

An incorrect job queue mapping caused the hydro_advanced_security_archived_status_changed queue to constantly grow.

External collaborators with read-only access were able to run workflows on their pull requests from private forks without approval.

On an instance with a GitHub Advanced Security license, custom pattern matches were incorrectly filtered during post-scan filtering.

To aid in understanding the CPU/memory utilization of secret scanning processes, the binary names of nomad workers were updated to differentiate between the different types of secret scanning jobs.

A more specific error message is shown when the ghe-repl-node command is run on an instance not configured for high availability.

The SCIM private beta has resumed with support from GitHub engineering in GitHub Enterprise Server version 3.11 and later. Site administrators can provision users and groups on a GitHub Enterprise Server instance automatically with SCIM. SCIM for GitHub Enterprise Server is in private beta and subject to change. For more information, see Configuring user provisioning with SCIM on GitHub Enterprise Server and REST API endpoints for SCIM in the REST API documentation.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker with the editor role in the Management Console could gain administrative SSH access to the appliance by command injection when configuring the chat integration. GitHub has requested CVE ID CVE-2024-3646 for this vulnerability, which was reported via the GitHub Bug Bounty program. The editor role has been deprecated. For more information, see the "Changes" section of these release notes.

HIGH: An attacker with an editor role in the Management Console could gain SSH access to the instance by command injection when configuring Artifact & Logs and Migrations Storage. GitHub has requested CVE ID CVE-2024-3684 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker with a deploy key for an organization-owned repository could bypass a ruleset that specified organization administrators as bypass actors. Exploitation would require an attacker to already have access to a valid deploy key for a repository. GitHub has requested CVE ID CVE-2024-3470 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could maintain admin access to a detached repository in a race condition by making a GraphQL mutation to alter repository permissions while the repository is detached. GitHub has requested CVE ID CVE-2024-2440 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

When configuring audit log streaming to Datadog or Splunk on an instance with custom CA certificates, the connection failed with the error There was an error trying to connect.

Disk usage, utilization, and latency for data devices could render incorrectly in Grafana.

On an instance in a cluster configuration with high availability replication enabled, Git operations for existing repositories would fail after failover to the replica cluster.

On an instance in a cluster configuration, former primary nodes were able to access the newly promoted nodes after failover. The ghe-cluster-failover command has been updated to block access from the old cluster, and four new command-line utilities have been introduced to manually block IP addresses: ghe-cluster-block-ips, ghe-cluster-block-ip, ghe-cluster-unblock-ips, and ghe-cluster-unblock-ip. For more information, see Command-line utilities. [Updated: 2024-05-01]

A Redis job had a memory limit that was too low in some cases, leading the process to run out of memory.

The ghe-update-check command did not clean up .tmp files in /var/lib/ghe-updates/, which could lead to full disk issues.

On an instance that failed a configuration run, when attempting to repeat the restore step of a backup, the audit log restore step returned error lines even though audit logs were being fully restored.

The / keyboard shortcut did not display the search field in the web UI.

On an instance where Dependabot alerts are or were enabled, upgrades to GitHub Enterprise Server version 3.12 could fail and require intervention from GitHub Support.

In some cases, Treelights timeouts caused pull requests to return a 500 error.

Administrators could get a 500 error when trying to access the "File storage" section of the site admin dashboard.

Setting a maintenance message failed if the message contained a multibyte character.

On an instance where user avatars had been deleted directly from the database, an identicon avatar was not correctly displayed for affected users, and administrators may have observed a relatively high number of application exceptions.

On an instance with repository caching configured, adding new repositories to a cache node sometimes failed.

On an instance with a GitHub Advanced Security license, after enabling secret scanning for the first time for an organization or the instance, the historical backfills for alerts in existing repositories issues did not appear.

On an instance with a GitHub Advanced Security license, alert counts for secret type on the secret scanning alerts page, as well as metrics for custom patterns, were incorrect.

On an instance with code scanning enabled, on the tool status page for code scanning, outdated upload errors were still displayed after a successful upload.

On an instance hosted on Azure, administrators can set and reset SSH keys and passwords via the Azure Agent.

As a result of a security vulnerability, the editor role for a Management Console user has been deprecated. For details, see the "Security fixes" section of these release notes. Existing users with the editor role will be unable to log in to the Management Console, and should contact their site administrator requesting that access be reinstated by updating the user to the operator role if appropriate.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

Repositories originally imported using ghe-migrator will not correctly track Advanced Security contributions.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

HIGH: An attacker with an Administrator role in GitHub Enterprise Server could gain SSH root access via remote code execution. GitHub has requested CVE ID CVE-2024-2469 for this vulnerability, which was reported via the GitHub Bug Bounty program.

HIGH: An attacker with an editor role in the Management Console could gain SSH access to the instance by command injection when configuring GeoJSON settings. GitHub has requested CVE ID CVE-2024-2443 for this vulnerability, which was reported via the GitHub Bug Bounty program.

MEDIUM: An attacker could execute CSRF attacks to perform unauthorized actions on behalf of an unsuspecting user, using the GraphQL mutations. A mitigating factor is that user interaction is required. GitHub has requested CVE ID CVE-2024-2748 for this vulnerability, which was reported via the GitHub Bug Bounty program.

Packages have been updated to the latest security versions.

On an instance in a high availability or cluster configuration, configuring fluent-bit on a primary node returned an empty primary_host value.

When an administrator performed certain operations related to an instance's storage, the user disk would fail to unmount.

In some cases, storage initialization on a new instance launch could cause EBS-backed data volumes to not be detected correctly.

Redundant messages caused an increase in the volume of events logged in /var/log/syslog.

On an instance in a cluster configuration with high availability enabled, the ghe-spokesctl command failed when run on a replica node.

On an instance in a cluster configuration, the ghe-remove-node utility allowed accidental removal of a node with the redis-master role.

If an administrator lost SSH access to an instance, authentication from the hypervisor console using the password for the root site administrator would fail.

On an instance with GitHub Actions enabled, GitHub Actions workflows that deployed GitHub Pages sites failed with the following error: Error: Deployment failed, try again later.

On an instance in a cluster configuration, Jupyter notebooks did not render correctly.

After an administrator runs gh es config apply using the GitHub CLI, the output includes a run ID.

On an instance in a cluster configuration with many nodes, requests to the REST API for managing GitHub Enterprise Server would exceed the instances HTTP timeouts.

Administrators could initiate an SSH audit that unknowingly unverified all SSH keys.

Attributes used to debug LDAP issues were not included in system logs.

Some API endpoints for projects did not properly filter target repositories based on the users access.

Improved error handling for domain verification.

On an instance with a GitHub Advanced Security license, some searches for secret scanning alerts resulted in a 500 error.

Organizations using projects (classic) returned an error log about a soon-to-be deprecated MySQL feature when viewing a project.

When an administrator set a policy to require two-factor authentication (2FA) for an enterprise, a message incorrectly indicated that users without 2FA enabled on their account would be removed from the enterprise. These users will be removed from repositories and organizations in the enterprise, but not from the enterprise itself.

On an instance with a GitHub Advanced Security license, viewing a secret scanning alert as a user without the security manager role would return a 500 error if the alert was generated from a Git tag instead of a normal commit.

When using GitHub Enterprise Importer to import repositories, ghost users in archive metadata files would cause an error when generating a list of migration conflicts using ghe-migrator conflicts.

After an administrator ran ghe-saml-mapping-csv, the output did not include the corresponding SQL query.

On an instance with a GitHub Advanced Security license, the security overview did not display updated alert counts for code scanning immediately after the completion of analysis.

The web UI presented inapplicable fine-grained permissions for assignment to custom repository roles. The permissions were also displayed as implicitly included in certain base roles.

Unauthenticated requests to the REST APIs /search/code endpoint returned erroneous rate-limit values.

On an instance with SAML authentication configured, users with a SAML mapping were able to configure two-factor authentication (2FA).

The profile settings for organizations displayed a warning about profile images that does not apply to organizations on a GitHub Enterprise Server instance.

Some pages in the settings for the instances enterprise account contained a link that responded with a 404 Not Found error.

When viewing a file in the instance's web interface, the "Copy lines" and "Copy permalink" interactions did not copy content to the clipboard.

The landing page for the site admin dashboard did not render details about the instance's enterprise account.

During a configuration run prompted by the delayed restart of the notebooks service, a container validation warning appeared in system logs.

On an instance in a cluster configuration, rebuilds of GitHub Pages sites failed if no replicas of the GitHub Pages data were available (for example, on a newly restored cluster).

In some cases, manual repository maintenance using ghe-spokesctl would fail with the following error: panic: runtime error: invalid memory address or nil pointer dereference.

On an instance with a GitHub Advanced Security license, repositories with generic secret detection or non-provider pattern scanning enabled wouldn't see the results of scans for secrets.

On an instance with a GitHub Advanced Security license, in some cases, when a user deleted a custom pattern for secret scanning, GitHub Enterprise Server failed to close or delete the patterns alerts.

On an instance with a GitHub Advanced Security license, the speed of migration for code scanning analyses is increased during an upgrade from GitHub Enterprise Server 3.10 or earlier.

On an instance with a GitHub Advanced Security license, in some cases, weekly scheduled runs for code scanning's default setup might not occur.

On an instance with a GitHub Advanced Security license, a user with read-only access to a repository could adjust severity settings for code scanning.

People deploying a GitHub Enterprise Server instance in AWS can now deploy in an environment that uses Instance Metadata Service Version 2 (IMDSv2).

On an instance in a cluster configuration, MySQL replica nodes can be configured to skip database seeding. For more information, see Deferring database seeding. Gists can be deleted using the Purge Gist button on the Deleted Gists page in Staff Tools.

The payload for the push webhook event is now limited to 2,048 commits. If there are more than 2,048 commits in a push, the payload for the push webhook will not contain serialized diff information for each commit. If you need to fetch commit information, you can use the Commits endpoints of the REST API. For more information, see Webhook events and payloads and REST API endpoints for commits.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

On an instance in a cluster configuration, restoration of a backup using ghe-restore will exit prematurely if Redis has not restarted properly.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Instance administration

• To ensure an instance's readiness for an upgrade to a new feature release of GitHub Enterprise Server, administrators can ensure that background tasks from a previous upgrade are complete using the ghe-check-background-upgrade-jobs command-line utility. For more information, see Upgrading with an upgrade package and Command-line utilities.

• When backing up an instance in a cluster configuration using GitHub Enterprise Server Backup Utilities, the pre-flight routine includes a health check for all nodes and notifies administrators of any issues before the backup begins.

• The REST API's /manage/v1 endpoints have been expanded to include all the same operations as the /setup/api endpoints. The /setup/api endpoints will be deprecated in a future release of GitHub Enterprise Server. For more information, see the following articles in the REST API documentation.

  • REST API endpoints for managing GitHub Enterprise Server
  • REST API endpoints for Management Console

• On an instance in a cluster configuration, administrators can use the ghe-remove-node command-line utility to remove a node from a cluster. This command evacuates data from the node's data services, marks the node as offline, and stops traffic being routed to the node, replacing the manual steps previously required to remove a node. For more information, see Command-line utilities. [Updated: 2024-02-28]

• On an instance in a cluster configuration, administrators can more easily configure or tear down a high availability replica of the cluster. For more information, see the documentation for the following utilities in the "Command-line utilities" article:

  • ghe-cluster-repl-bootstrap
  • ghe-cluster-repl-teardown

  [Updated: 2024-03-25]

Authentication

• To manage work across different accounts and GitHub products, users can authenticate to the GitHub CLI with multiple accounts, then use the gh auth switch command to switch between active accounts. For more information, see gh auth login in the GitHub CLI manual.

GitHub Advanced Security

• The GitHub Advanced Security billing REST API and CSV download includes the email addresses for active committers. This provides information for insights into Advanced Security license usage across your business. For more information, see REST API endpoints for enterprise billing and Viewing your GitHub Advanced Security usage.

• To make it easier for users to secure repositories, default setup for code scanning automatically attempts to analyze all languages supported by CodeQL. Users no longer need to manually include analysis of C, C++, C#, Java, or Kotlin when enabling default setup, and organization owners and security managers can enable analysis of these languages for multiple repositories in an organization. For more information about the languages and versions supported by CodeQL and code scanning, see Configuring default setup for code scanning and Supported languages and frameworks in the CodeQL documentation.

• Customers who use both GitHub Enterprise Server and GitHub Enterprise Cloud can ensure license usage for GitHub Advanced Security is calculated correctly by synchronizing license usage across deployments. Synchronization can be performed automatically, using GitHub Connect, or manually, using an export file. For more information, see Syncing license usage between GitHub Enterprise Server and GitHub Enterprise Cloud.

• For code scanning, this release of GitHub Enterprise Server uses version 2.15.5 of CodeQL by default. This version of CodeQL includes more up-to-date support for various languages, including C# 12, .NET 8, TypeScript 5.3, Java 21, and Python 3.12. For more information, see the changelog for CodeQL 2.15.5 in the CodeQL documentation.

• Code scanning with CodeQL has improved support for detecting vulnerabilities in C and C++ code, with queries available for detecting common memory-corruption vulnerabilities. These queries are in beta and subject to change. For more information, see ICYMI: improved C++ vulnerability coverage and CodeQL support for Lombok on the GitHub Blog.

• For repositories migrated to GitHub Enterprise Server from other platforms, to calculate active committers for GitHub Advanced Security license usage, GitHub only considers commits made after the migration date. Previously, historic commits were included in the calculation, and users needed to intervene manually to avoid consuming licenses unnecessarily. For more information, see About billing for GitHub Advanced Security.

• To make the language overview on the tool status page more informative, users can directly specify Kotlin, C, and TypeScript as languages to be analyzed using the language property of a codeql.yml file. For example: language: [ 'kotlin' ]. These languages were already supported by CodeQL, but were previously treated as being part of the Java, CPP, and JavaScript languages respectively.

• To increase the coverage of secret scanning without needing to maintain custom patterns, users can configure secret scanning to detect non-provider patterns. Non-provider patterns are patterns such as private keys that tend to have a higher rate of false positives than high-confidence patterns. GitHub displays non-provider alerts in a different list from high-confidence alerts, making triaging a better experience for users. This feature is in beta and subject to change. For more information, see Managing alerts from secret scanning.

Dependabot

• To debug issues with Dependabot, users can view logs for Dependabot job runs associated with version updates, security updates, and rebase updates. For more information, see Viewing Dependabot job logs.

• Users can choose how to respond to Dependabot alerts automatically by setting up custom auto-triage rules in repositories or organizations. Auto-triage rules provide control over whether an alert is ignored, is snoozed, or triggers a pull request for a security update. Users can also use a rule created by GitHub to automatically dismiss low-impact issues in npm dependencies. Auto-triage rules are in public beta and subject to change. For more information, see About Dependabot auto-triage rules.

• Dependabot version updates have improved support for dependencies in NuGet, the package manager for .NET. Improvements include better support for implicit dependencies and peer dependencies. For more information about supported package managers, see About Dependabot version updates.

GitHub Actions

• For self-hosted GitHub Actions runners on this GitHub Enterprise Server release, the minimum required version of the GitHub Actions Runner application is 2.311.0. See the release notes for this version in the actions/runner repository. If your instance uses ephemeral self-hosted runners and you've disabled automatic updates, you must upgrade your runners to this version of the Runner application before upgrading your instance to this GitHub Enterprise Server release. [Updated: 2024-04-25]

• Users can set up organization-wide rules to enforce their CI/CD workflows, ensuring workflows pass before pull requests can be merged into target repositories. You can fine-tune your rule by selecting a specific branch, tag, or SHA, and provide maximum control over the version expected to run. To reduce risk, you can "evaluate" workflow rules to validate rules are working correctly. For more information, see Available rules for rulesets.

• GitHub Actions developers can use GitHub Actions Importer to plan, forecast, and automate the migration of existing CI/CD pipelines from Bamboo Server, Bamboo Data Center, and Bitbucket. Developers can migrate their Bamboo and Bitbucket pipelines to GitHub Actions using the GitHub CLI or IssueOps. For more information, see Migrating from Bitbucket Pipelines with GitHub Actions Importer and Migrating from Bamboo with GitHub Actions Importer.

• Actions environments support defining selected tag patterns to restrict deployments. Administrators who want to have more secure and controlled deployments can specify selected tags or tag patterns on their protected environments. For more information, see Managing environments for deployment.

Community experience

• To tailor information to users' needs, users are prompted to sign in to access the GitHub Support portal. For customers with an enterprise account on GitHub.com, we encourage users to sign in to an account with support privileges for the enterprise. For more information, see Managing support entitlements for your enterprise. Users who cannot sign in to an account on GitHub.com can still access the portal by verifying an email address.

• To help users find answers to their questions more quickly, GitHub Copilot is integrated into GitHub Support. Users can choose to chat with Copilot instead of creating a ticket on the Get help with GitHub contact form. Copilot has been trained on the GitHub Enterprise Server documentation on GitHub Docs. This feature is in public beta and subject to change.

Projects

• Project templates for organizations are generally available. Users in an organization can create a template to share a pre-configured project with other people in your organization as the base for their projects. For more information, see Managing project templates in your organization.

• Users can access Projects from from the global navigation menu. This page can be used to find projects you've recently viewed or created, regardless of the organization or where they are located. For more information, see Finding your projects.

GitHub Discussions

• Users can filter for answered or unanswered discussions with the GraphQL API by using the answered property. For more information, see Using the GraphQL API for Discussions.

Pull requests

• Users can merge pull requests without needing to wait for status checks to pass by adding a pull request to a merge queue. The merge queue ensures that the changes in the pull request will pass all required status checks when applied to the latest version of the target branch. A pull request is merged automatically once it reaches the front of the queue. This feature is particularly useful on branches where pull requests are merged frequently. For more information, see Managing a merge queue.

Markdown

• Users can highlight information using Markdown alerts. Alerts are displayed with distinctive colors and icons, and include notes, tips, warnings, and more. For more information, see Basic writing and formatting syntax.

Accessibility

• The web interface for GitHub Enterprise Server has been redesigned to provide a more intuitive, responsive, and accessible navigation experience. Changes include:

  • Breadcrumbs to help users navigate the site more efficiently
  • Menus to quickly access a user's top repositories and teams
  • A more accessible navigation experience, including more consistent keyboard navigation and improvements to code search

  For more information, see Exploring GitHub with the redesigned navigation on the GitHub Blog. Note that the redesigned navigation is now generally available.

• The comment field in issues, discussions, and pull requests has been redesigned for easier use across different screen sizes, and for better integration with assistive technology such as keyboard navigation and screen readers.

Field names for some service logs on GitHub Enterprise Server have changed as part of GitHub's gradual migration to internal semantic conventions for OpenTelemetry. Additional field names were changed in GitHub Enterprise Server 3.9, 3.10, and 3.11. If any tooling or processes in your environment rely on specific field names within logs, or log entries in specific files, the following changes may affect you.

• level is now SeverityText.
• log_message, msg, or message is now Body.
• now is now Timestamp.
• Custom field names such as gh.repo.id or graphql.operation.name use semantic names.
• Log statements that the instance would previously write to auth.log, ldap.log, or ldap-sync.log now appear in containerized logs for github-unicorn if the statement originated from a web request, or in logs for github-resqued if the statement originated from a background job. For more information about containerized logs, see About system logs.

For a full list of mappings, download the OpenTelemetry attribute mapping CSV for GitHub Enterprise Server 3.9, 3.10, 3.11, and 3.12.

On an instance with GitHub Advanced Security and code scanning enabled, the bot that posts comments and annotations for code scanning alerts on pull requests has been renamed from github-code-scanning to github-advanced-security.

The REST API's /rate_limit endpoint is now subject to rate limits. Requests will not consume the primary rate limit quotas for the authenticated user. However, making a very high number of requests in a short period of time will trigger the secondary rate limits if secondary rate limits are enabled on your instance. For more information, see REST API endpoints for rate limits in the REST API documentation and Configuring rate limits.

TCP port 9103 is opened for future administrative features related to support for Prometheus scraping.

Custom firewall rules are removed during the upgrade process.

During the validation phase of a configuration run, a No such object error may occur for the Notebook and Viewscreen services. This error can be ignored as the services should still correctly start.

If the root site administrator is locked out of the Management Console after failed login attempts, the account does not unlock automatically after the defined lockout time. Someone with administrative SSH access to the instance must unlock the account using the administrative shell. For more information, see Troubleshooting access to the Management Console.

If an instance is configured to forward logs to a target server with TLS enabled, certificate authority (CA) bundles that a site administrator uploads using ghe-ssl-ca-certificate-install are not respected, and connections to the server fail.

The mbind: Operation not permitted error in the /var/log/mysql/mysql.err file can be ignored. MySQL 8 does not gracefully handle when the CAP_SYS_NICE capability isn't required, and outputs an error instead of a warning.

On an instance hosted in AWS, system time may lose synchronization with Amazon's servers after an administrator reboots the instance.

On an instance with the HTTP X-Forwarded-For header configured for use behind a load balancer, all client IP addresses in the instance's audit log erroneously appear as 127.0.0.1.

In some situations, large .adoc files stored in a repository do not render properly in the web UI. The raw contents are still available to view as plaintext.

After failing over an instance in a cluster configuration, Git pushes to the instance will fail. This issue impacts pushes from the command line as well as the web interface. To resolve this issue, contact GitHub Support.

Restoring backups with ghe-restore on a GHES cluster will exit prematurely if redis has not restarted properly.

If a hotpatch upgrade requires the haproxy-frontend service to be restarted, the restart will hang if there are existing long-lived connections, such as browser web sockets or Git operations. No new connections will be accepted for up to 5 minutes. Any existing unfinished connections at this time will be disconnected. [Updated: 2024-06-17]

Upgrading GitHub Enterprise Server Backup Utilities using a Git repository is no longer supported

• The ability to upgrade the GitHub Enterprise Server Backup Utilities tool using a Git repository has been removed following changes to the github/backup-utils repository. For more information about the repository changes, see Repository updates - November 2023 in the project's README.

  Instead, to upgrade the tool using a compressed archive, download the relevant GitHub Enterprise Server Backup Utilities release from the Releases page of the github/backup-utils repository. For more information, see Configuring backups on your instance.

  This deprecation does not affect the functionality of the GitHub Enterprise Server Backup Utilities tool or your ability to backup or restore your GitHub Enterprise Server instance. [Updated: 2024-03-15]