Warning: The replacement node must use a hostname that has not been used previously in the cluster to avoid conflicts.
Replacing a functional node
-
Provision and install GitHub Enterprise Server with a unique hostname on the replacement node.
-
Using the administrative shell or DHCP, only configure the IP address of the replacement node. Don't configure any other settings.
-
To add the newly provisioned replacement node, on any node, modify the
cluster.conf
file to remove the failed node and add the replacement node. For example, this modifiedcluster.conf
file replacesghe-data-node-3
with the newly provisioned node,ghe-replacement-data-node-3
:[cluster "ghe-replacement-data-node-3"] hostname = ghe-replacement-data-node-3 ipv4 = 192.168.0.7 # ipv6 = fd12:3456:789a:1::7 git-server = true pages-server = true mysql-server = true elasticsearch-server = true redis-server = true memcache-server = true metrics-server = true storage-server = true
-
From the administrative shell of the node with the modified
cluster.conf
, runghe-cluster-config-init
. This will initialize the newly added node in the cluster. -
From the same node, run
ghe-cluster-config-apply
. This will validate the configuration file, copy it to each node in the cluster, and configure each node according to the modifiedcluster.conf
file. -
If you're taking a node offline that has
git-server = true
set in cluster.conf, evacuate the node. For more information, see "Evacuating a cluster node." -
To mark the failed node offline, on any node, modify the cluster configuration file (
cluster.conf
) in the relevant node section to include the textoffline = true
.For example, this modified
cluster.conf
will mark theghe-data-node-3
node as offline:[cluster "ghe-data-node-3"] hostname = ghe-data-node-3 offline = true ipv4 = 192.168.0.6 # ipv6 = fd12:3456:789a:1::6
-
From the administrative shell of the node where you modified
cluster.conf
, runghe-cluster-config-apply
. This will validate the configuration file, copy it to each node in the cluster, and mark the node offline. -
If you're replacing the primary MySQL or Redis node, in
cluster.conf
, modify themysql-master
orredis-master
value with the replacement node name.For example, this modified
cluster.conf
file specifies a newly provisioned cluster node,ghe-replacement-data-node-1
as the primary MySQL and Redis node:mysql-master = ghe-replacement-data-node-1 redis-master = ghe-replacement-data-node-1
Replacing a node in an emergency
-
Provision and install GitHub Enterprise Server with a unique hostname on the replacement node.
-
Using the administrative shell or DHCP, only configure the IP address of the replacement node. Don't configure any other settings.
-
To mark the failed node offline, on any node, modify the cluster configuration file (
cluster.conf
) in the relevant node section to include the textoffline = true
.For example, this modified
cluster.conf
will mark theghe-data-node-3
node as offline:[cluster "ghe-data-node-3"] hostname = ghe-data-node-3 offline = true ipv4 = 192.168.0.6 # ipv6 = fd12:3456:789a:1::6
-
From the administrative shell of the node where you modified
cluster.conf
, runghe-cluster-config-apply
. This will validate the configuration file, copy it to each node in the cluster, and mark the node offline. -
To add the newly provisioned replacement node, on any node, modify the
cluster.conf
file to remove the failed node and add the replacement node. For example, this modifiedcluster.conf
file replacesghe-data-node-3
with the newly provisioned node,ghe-replacement-data-node-3
:[cluster "ghe-replacement-data-node-3"] hostname = ghe-replacement-data-node-3 ipv4 = 192.168.0.7 # ipv6 = fd12:3456:789a:1::7 git-server = true pages-server = true mysql-server = true elasticsearch-server = true redis-server = true memcache-server = true metrics-server = true storage-server = true
-
If you're replacing the primary MySQL or Redis node, in
cluster.conf
, modify themysql-master
orredis-master
value with the replacement node name.For example, this modified
cluster.conf
file specifies a newly provisioned cluster node,ghe-replacement-data-node-1
as the primary MySQL and Redis node:mysql-master = ghe-replacement-data-node-1 redis-master = ghe-replacement-data-node-1
-
From the administrative shell of the node with the modified
cluster.conf
, runghe-cluster-config-init
. This will initialize the newly added node in the cluster. -
From the same node, run
ghe-cluster-config-apply
. This will validate the configuration file, copy it to each node in the cluster, and configure each node according to the modifiedcluster.conf
file. -
If you're taking a node offline that has
git-server = true
set in cluster.conf, evacuate the node. For more information, see "Evacuating a cluster node."