GitHub Enterprise clustering relies on proper DNS name resolution, load balancing, and communication between nodes to operate properly.

Network considerations

The simplest network design for Clustering is to place the nodes on a single LAN. If a redundant cluster must span across subnets, then appropriate routes should be available between subnets and latency should be less than 1ms.

Application (End User) ports

These ports provide primary application access to end-users:

Port Description Encrypted
22/TCP Git over SSH Yes
25/TCP SMTP Requires STARTTLS
80/TCP HTTP No
(When SSL is enabled this port redirects to HTTPS)
443/TCP HTTPS Yes
9418/TCP Simple Git protocol port
(Disabled in private mode)
No

Administrative ports

These ports are used for administrative purposes and are not required for basic end-user application use:

Port Description Encrypted
ICMP ICMP Ping No
122/TCP Administrative SSH Yes
161/UDP SNMP No
8080/TCP Management Console HTTP No
(When SSL is enabled this port redirects to HTTPS)
8443/TCP Management Console HTTPS Yes

Cluster communication ports

If a network level firewall is in place between nodes, these ports will need to be accessible. The communication between nodes is not encrypted. These ports should not be accessible externally.

Port Description
1336/TCP Internal API
3033/TCP Internal SVN access
3037/TCP Internal SVN access
3306/TCP MySQL
5115/TCP Storage backend
5208/TCP Internal SVN access
6379/TCP Redis
8000/TCP Graphite
8149/TCP GitRPC fileserver access
9000/TCP Git Daemon
9102/TCP Pages fileserver
9200/TCP Elasticsearch
9300/TCP Elasticsearch
11211/TCP Memcache
161/UDP SNMP
8125/UDP Statsd
25827/UDP Collectd

Configuring a load balancer

We recommend an external TCP load balancer to distribute traffic across nodes. Consider these load balancer configurations:

  • TCP ports (shown below) should be forwarded to nodes running the web-server service. These are the only nodes that serve external client requests.
  • Sticky sessions shouldn't be enabled.

Warning: Terminating SSL at a load balancer is not supported. When using SSL (which is recommended), HTTPS traffic must be forwarded directly to the appliance without modification.

Handling client connection information

Because client connections to the cluster come from the load balancer, the client IP address can be lost. To properly capture the client connection information, additional consideration is required.

If your load balancer can support it, the preferred solution is the PROXY protocol. When no PROXY support is available, it is also possible to load balance the HTTP and HTTPS ports using the X-Forwarded-For header.

Security Warning: When either PROXY support or HTTP forwarding is enabled, it is critical that no external traffic can directly reach the GitHub Enterprise appliances. If external traffic is not properly blocked, it can allow the source IP addresses to be forged.

Enabling PROXY support on GitHub Enterprise

Enable PROXY support for both your instance and the load balancer.

  • For your instance, use this command:

    ghe-config 'loadbalancer.proxy-protocol' 'true' && ghe-cluster-config-apply
    
  • For the load balancer, use the instructions provided by your vendor.

PROXY protocol TCP port mappings
Source Port Destination Port Service Description
22 23 Git over SSH
80 81 HTTP
443 444 HTTPS
8080 8081 Management Console HTTP
8443 8444 Management Console HTTPS
9418 9419 Git

Enabling X-Forwarded-For support on GitHub Enterprise

The X-Fowarded-For header will only work with HTTP and HTTPS. The IP address reported for Git over SSH connections will show the load balancer IP.

To enable the X-Fowarded-For header, use this command:

ghe-config 'loadbalancer.http-forward' 'true' && ghe-cluster-config-apply
Protocol TCP port mappings for use without PROXY support
Source Port Destination Port Service Description
22 22 Git over SSH
25 25 SMTP
80 80 HTTP
443 443 HTTPS
8080 8080 Management Console HTTP
8443 8443 Management Console HTTPS

Configuring Health Checks

Health checks allow a load balancer to stop sending traffic to a node that is not responding if a pre-configured check fails on that node. If a cluster node fails, health checks paired with redundant nodes provides high availability.

Configure the load balancer to check one of these URLs:

  • https://<hostname>/status if HTTPS is enabled (default).
  • http://<hostname>/status if HTTPS is disabled

The check will return status code 200 (OK) if the node is healthy and available to service end-user requests.

Note: The https://<hostname>/status url will return status code 503 (Service Unavailable) when the appliance is in maintenance mode.

DNS Requirements

DNS lookups for the GitHub Enterprise hostname should resolve to the load balancer. If subdomain isolation is enabled (recommended), an additional wildcard record (*.[hostname]) should also resolve to the load balancer. For more information, see "DNS, Hostname, Subdomain Isolation, and SSL".