Upgrading from GitHub Enterprise 2.x

To upgrade from GitHub Enterprise versions prior to 2.9, follow the steps in the "Preparing to upgrade" section.

Patch releases (e.g. 2.0.1 to 2.0.2) typically require less than five minutes of downtime. However, feature releases (e.g. 2.0.2 to 2.1.0) that include data migrations may take longer. For more information on expected upgrade times, see the release notes for the relevant version.

Upgrading from GitHub Enterprise versions between 11.10.348 and 11.10.354.

To upgrade from GitHub Enterprise versions 11.10.348 through 11.10.354, you must:

1. Migrate to GitHub Enterprise 2.1.

2. Upgrade to GitHub Enterprise 2.3.

3. Upgrade to GitHub Enterprise 2.5.
4. Upgrade to GitHub Enterprise 2.7.
5. Upgrade to GitHub Enterprise 2.9.

Upgrading from a version older than 11.10.348

To upgrade from a version older than GitHub Enterprise 11.10.348, you must:

1. Upgrade to GitHub Enterprise 11.10.354. This may require several passes of the previous upgrade procedure.

2. Migrate to GitHub Enterprise 2.1.

3. Upgrade to GitHub Enterprise 2.3.

4. Upgrade to GitHub Enterprise 2.5.
5. Upgrade to GitHub Enterprise 2.7.
6. Upgrade to GitHub Enterprise 2.9.

Preparing to upgrade

1. Create a fresh backup of your primary instance with the GitHub Enterprise Backup Utilities.

2. Plan to take a snapshot of your existing data immediately after placing the instance in maintenance mode.

3. Determine if you should follow the basic upgrade procedure or the primary/replica pair upgrade procedure.

   • If you only have one instance of GitHub Enterprise, follow the basic upgrade procedure.
   • If you have configured a replica instance for High Availability, follow the primary / replica pair upgrade procedure.

4. Schedule a maintenance window for end users of your GitHub Enterprise instance, as the instance will be unavailable for normal use while it is restarting and performing post-restart tasks.

Taking a snapshot

A snapshot is a checkpoint of a virtual machine at a point in time. We highly recommend taking a snapshot prior to upgrading your virtual machine. If an upgrade fails, you should revert back to a snapshot to restore the state of your VM prior to the upgrade, because irreversible database migrations may have run during the failed upgrade.

There are two types of snapshots:

• A VM snapshot, which saves your entire VM state, including user data and configuration data.
• A data disk snapshot, which saves only your user data.

Taking a snapshot of your entire VM can be a time-consuming operation and requires a significant amount of disk space. For the upgrade process, we recommend taking a snapshot of just your data disk. Note that some platforms don't allow you to take a snapshot of just your data disk, requiring you to take a snapshot of the entire VM.

Platform	Snapshot method	Snapshot documentation URL
Amazon AWS	Disk	https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-creating-snapshot.html
Azure	VM	https://azure.microsoft.com/en-us/documentation/articles/backup-azure-vms/
Hyper-V	VM	https://technet.microsoft.com/en-us/library/dd851843.aspx
VMware	VM	https://pubs.vmware.com/vsphere-50/index.jsp#com.vmware.vsphere.vm_admin.doc_50/GUID-9720B104-9875-4C2C-A878-F1C351A4F3D8.html
XenServer	VM	http://docs.citrix.com/en-us/xencenter/6-2/xs-xc-vms-snapshots.html

Performing a basic upgrade

1. Connect to the GitHub Enterprise administrative shell over SSH as the "admin" user on port 122:

   ssh -p 122 admin@hostname
   

   See Administrative Shell (SSH) Access if you need to authorize shell access for your account.

2. Visit the GitHub Enterprise Download Page and copy the URL for the upgrade package (.pkg file) to the clipboard.

3. Download the upgrade package to your GitHub Enterprise instance using curl. For example, if you're upgrading from GitHub Enterprise version 2.0.0 to 2.0.1 on the Amazon Web Services platform:

   admin@hostname:~$ curl -L -O https://github-enterprise.s3.amazonaws.com/ami/updates/github-enterprise-ami-2.0.1.pkg
   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                        Dload  Upload   Total   Spent    Left  Speed
   100  675M  100  675M    0     0  18.2M      0  0:00:37  0:00:37 --:--:-- 25.0M
   

4. Do one of the following:

   • If you are upgrading a High Availability pair, follow the instructions in "Performing an upgrade with a primary / replica pair (High Availability)".
   • Otherwise, enable Maintenance Mode and wait for all active processes to complete on the GitHub Enterprise instance.

5. Run the ghe-upgrade command, providing the package file name downloaded in the previous step:

   admin@hostname:~$ ghe-upgrade github-enterprise-ami-2.0.1.pkg
   *** verifying upgrade package signature...
   676MB 0:00:05 [ 127MB/s] [=============================================================================>] 100%
   gpg: Signature made Sun 09 Nov 2014 12:20:29 AM UTC using RSA key ID 0D65D57A
   gpg: checking the trustdb
   gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
   gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
   gpg: Good signature from "GitHub Enterprise (Upgrade Package Key) "
   

   After the package signature is verified, information on the current and target root partition is displayed and you will be prompted to confirm whether to continue with the upgrade and restart:

   *** applying update...
   This package will upgrade your installation to version 2.0.1 (5661634)
   Current root partition: /dev/xvda1 [2.0.0 (0019198)]
   Target root partition:  /dev/xvda2
   Proceed with installation? [y/N]
   

   Once you've accepted the upgrade details, the new root filesystem is written to the secondary partition and the instance is automatically restarted:

   *** extracting new root filesystem...
   676MB 0:01:04 [10.5MB/s] [=============================================================================>] 100%
   *** setting up bootloader configuration...
   *** Rebooting!
   Broadcast message from admin@hostname
   (/dev/pts/3) at 20:11 ...
   The system is going down for reboot NOW!
   Connection to hostname closed.
   

   Upgrades to GitHub Enterprise 2.1.4 and later will reboot back into maintenance mode. Earlier revisions will not and your instance will be immediately accessible to end users after you restart it and run any data migrations.

6. If your instance is in maintenance mode, make it accessible to your users again.

Performing an upgrade with a primary / replica pair (High Availability)

Upgrading the primary instance

1. On the primary instance, enable Maintenance Mode and wait for all active processes to finish.

2. Connect to the replica instance over SSH as the "admin" user on port 122:

   ssh -p 122 admin@replica-host
   

3. On the replica instance, run ghe-repl-stop to stop replication.

4. On the primary instance, perform the basic upgrade procedure.

Because replication is stopped, any work users do before the replica is upgraded and replication begins again will be lost if the primary fails.

Upgrading the replica instance

1. On the replica instance, follow the basic upgrade procedure. There is no need to enable maintenance mode and save the settings after upgrading.

2. After the replica instance reboots, connect to it over SSH on port 122 as the "admin" user.

   ssh -p 122 admin@replica-host
   

3. Verify the upgrade by running:

   cat /etc/github/enterprise-release
   

   RELEASE_VERSION should reflect the version you have upgraded to.

4. On the replica instance, replication will automatically start. To ensure all replication services are running appropriately, run ghe-repl-status. This command will return OK for all services when a successful replication is in progress.

   Note: While the resync is in progress ghe-repl-status may return messages indicating that replication is behind. For example: CRITICAL: git replica is lagging 1481.744s behind primary. These are expected.

   If the command returns Replication is not running, the replication may still be starting. Wait about one minute before running ghe-repl-status again.

   If replication started correctly, the replica is successfully upgraded and you're finished. Otherwise, continue with the next steps to manually start the replication.

5. On the replica instance, run ghe-repl-setup <primary-instance-ip> again.

6. On the replica instance, to begin replication, run ghe-repl-start.

7. On the replica instance, to ensure all replication services are running appropriately, run ghe-repl-status. This command will return OK for all services when successful replication is in progress.

Restoring from a failed upgrade

If an upgrade fails or is interrupted, you should revert your instance back to its previous state. The process for completing this depends on whether you're using a VM snapshot or a data disk snapshot.

Using a data disk snapshot

1. Enable Maintenance Mode and wait for all active processes to complete on the GitHub Enterprise instance.
2. To restore your boot partition, run ghe-rollback-rootfs. This command will turn the VM off when it's finished.
3. To restore your instance, load your previous user data snapshot.
4. After the restoration finishes, start your VM.

Using a VM snapshot

Follow the instructions for your platform to restore your VM snapshot.

Restoring corrupt Elasticsearch indexes

GitHub Enterprise 2.3 included an upgrade to Elasticsearch 1.5. This new version provides improved validations for data integrity. If your search index was previously corrupted, you won't be able to upgrade until you've repaired your search indexes.

For help performing an upgrade, contact GitHub Enterprise Support.