Network considerations
The simplest network design for clustering is to place the nodes on a single LAN. If a cluster must span subnets, we do not recommend configuring any firewall rules between the networks. The latency between nodes should be less than 1 millisecond.
Application ports for end users
Application ports provide web application and Git access for end users.
| Port | Description | Encrypted |
|---|---|---|
| 22/TCP | Git over SSH | Yes |
| 25/TCP | SMTP | Requires STARTTLS |
| 80/TCP | HTTP | No (When SSL is enabled this port redirects to HTTPS) |
| 443/TCP | HTTPS | Yes |
| 9418/TCP | Simple Git protocol port (Disabled in private mode) | No |
Administrative ports
Administrative ports are not required for basic application use by end users.
| Port | Description | Encrypted |
|---|---|---|
| ICMP | ICMP Ping | No |
| 122/TCP | Administrative SSH | Yes |
| 161/UDP | SNMP | No |
| 8080/TCP | Management Console HTTP | No (When SSL is enabled this port redirects to HTTPS) |
| 8443/TCP | Management Console HTTPS | Yes |
Cluster communication ports
If a network level firewall is in place between nodes, these ports will need to be accessible. The communication between nodes is not encrypted. These ports should not be accessible externally.
| Port | Description |
|---|---|
| 1336/TCP | Internal API |
| 3033/TCP | Internal SVN access |
| 3037/TCP | Internal SVN access |
| 3306/TCP | MySQL |
| 4486/TCP | Governor access |
| 5115/TCP | Storage backend |
| 5208/TCP | Internal SVN access |
| 6379/TCP | Redis |
| 8001/TCP | Grafana |
| 8090/TCP | Internal GPG access |
| 8149/TCP | GitRPC fileserver access |
| 8300/TCP | Consul |
| 8301/TCP | Consul |
| 8302/TCP | Consul |
| 9000/TCP | Git Daemon |
| 9102/TCP | Pages fileserver |
| 9105/TCP | LFS server |
| 9200/TCP | Elasticsearch |
| 9203/TCP | Semantic code service |
| 9300/TCP | Elasticsearch |
| 11211/TCP | Memcache |
| 161/UDP | SNMP |
| 8125/UDP | Statsd |
| 8301/UDP | Consul |
| 8302/UDP | Consul |
| 25827/UDP | Collectd |
Configuring a load balancer
We recommend an external TCP-based load balancer that supports the PROXY protocol to distribute traffic across nodes. Consider these load balancer configurations:
- TCP ports (shown below) should be forwarded to nodes running the
web-serverservice. These are the only nodes that serve external client requests. - Sticky sessions shouldn't be enabled.
Warning: When terminating HTTPS connections on a load balancer, the requests from the load balancer to GitHub Enterprise Server also need to use HTTPS. Downgrading the connection to HTTP is not supported.
Handling client connection information
Because client connections to the cluster come from the load balancer, the client IP address can be lost. To properly capture the client connection information, additional consideration is required.
If your load balancer can support it, we strongly recommend implementing the PROXY protocol. When no PROXY support is available, it is also possible to load balance the HTTP and HTTPS ports using the X-Forwarded-For header.
Security Warning: When either PROXY support or HTTP forwarding is enabled, it is critical that no external traffic can directly reach the GitHub Enterprise Server appliances. If external traffic is not properly blocked, the source IP addresses can be forged.
Enabling PROXY support on GitHub Enterprise Server
We strongly recommend enabling PROXY support for both your instance and the load balancer.
-
For your instance, use this command:
$ ghe-config 'loadbalancer.proxy-protocol' 'true' && ghe-cluster-config-apply -
For the load balancer, use the instructions provided by your vendor.
PROXY protocol TCP port mappings
| Source port | Destination port | Service description |
|---|---|---|
| 22 | 23 | Git over SSH |
| 80 | 81 | HTTP |
| 443 | 444 | HTTPS |
| 8080 | 8081 | Management Console HTTP |
| 8443 | 8444 | Management Console HTTPS |
| 9418 | 9419 | Git |
Enabling X-Forwarded-For support on GitHub Enterprise Server
Use the X-Forwarded-For protocol only when the PROXY protocol is unavailable. The X-Forwarded-For header only works with HTTP and HTTPS. The IP address reported for Git connections over SSH will show the load balancer IP.
To enable the X-Fowarded-For header, use this command:
$ ghe-config 'loadbalancer.http-forward' 'true' && ghe-cluster-config-applyProtocol TCP port mappings for use without PROXY support
| Source port | Destination port | Service description |
|---|---|---|
| 22 | 22 | Git over SSH |
| 25 | 25 | SMTP |
| 80 | 80 | HTTP |
| 443 | 443 | HTTPS |
| 8080 | 8080 | Management Console HTTP |
| 8443 | 8443 | Management Console HTTPS |
Configuring Health Checks
Health checks allow a load balancer to stop sending traffic to a node that is not responding if a pre-configured check fails on that node. If a cluster node fails, health checks paired with redundant nodes provides high availability.
Configure the load balancer to check one of these URLs:
https://HOSTNAME/statusif HTTPS is enabled (default)http://HOSTNAME/statusif HTTPS is disabled
The check will return status code 200 (OK) if the node is healthy and available to service end-user requests.
Note: When the appliance is in maintenance mode, the https://HOSTNAME/status URL will return status code 503 (Service Unavailable). For more information, see "Enabling and scheduling maintenance mode."
DNS Requirements
DNS lookups for the GitHub Enterprise Server hostname should resolve to the load balancer. We recommend that you enable subdomain isolation. If subdomain isolation is enabled, an additional wildcard record (*.HOSTNAME) should also resolve to the load balancer. For more information, see "Enabling subdomain isolation."