Using nginx as a load balancer

Load balancing technique is commonly used by websites or web applications with high traffic not only to maximize throughput but also to ensure service availability and reducing latency.


This technique is not limited for HTTP protocol and is the only way to handle millions of visitors, database queries, emails, data transfers or every kind of scalable service.


In this tutorial we will use nginx as HTTP load balancer to distribute the traffic between multiple web servers. The modules required by nginx as a load balancer in the examples below are:

  • HTTP core module
  • HTTP headers module
  • HTTP proxy module
  • HTTP rewrite module
  • HTTP ssl module

Nginx support these 3 methods of load balancing:

Round robin

Many of you are familiar with round robin DNS service. And yes, this is the same round robin method. Essentially is a simple mechanism that respond to requests in a rotational basis, for the first request responds the first available server, for the second request responds the second available server and so on in a continuous loop. This is the default nginx load balancing method.



Least connected

In this method the next request is assigned to the server with less number of active connections. This method will distribute the requests to the less busy server.



IP Hash
A hash based on client IP address function is used to determine the responsible server for next request. This will guarantee the session persistence, the requests from the same client will always be forwarded to the same server except when this server is unavailable.

Let's assume we have 4 servers (dedicated or VPS) and we want to distribute the web traffic between them. To identify the servers we may use IP address or a dedicated A record in DNS for each one. Example:

node1.example.com => First server
node2.example.com => Second server
node3.example.com => Third server
node4.example.com => Fourth server

A typical configuration is described with this scheme:

nginx as a load balancer

To configure nginx for this scenario will use the nginx main configuration file:

vi /etc/nginx/nginx.conf

Here is a completed configuration example:

user nginx;
worker_processes 8;
worker_rlimit_nofile 50000;
pid /run/nginx.pid;
error_log /var/log/nginx/error.log crit;

http {
upstream cluster {
#ip_hash;
#least_conn;
server node1.example.com:8080 max_fails=1 fail_timeout=8 weight=2;
server node2.example.com:8080 max_fails=2 fail_timeout=8 weight=1;
server node3.example.com:8080 max_fails=2 fail_timeout=8 weight=1;
server node4.example.com:8080 max_fails=2 fail_timeout=8 weight=1;
}

server {
listen 203.0.113.1:80;
server_name example.com;
server_name example.net;
return 301 https://$server_name$request_uri;
}

server {
listen 203.0.113.1:443 ssl;
server_name example.com;
server_name example.net;
ssl_certificate /certs/example.com.bundle.crt;
ssl_certificate_key /certs/example.com.key;
ssl_session_cache shared: SSL:20m;
ssl_session_timeout 10m;
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS;
add_header Strict-Transport-Security "max-age=31536000";
location / {
proxy_pass http://cluster;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}


user

Defines user and group credentials used by worker process. If group is absent, a group with equal name as user is used.



worker_processes

Defines the number of worker processes. Although the optimal value depends on many factors ( CPU core, storage and load pattern ) a good practice is to set 1 worker process per core.



To check the number of CPU cores you can grep the cpuinfo:



grep processor /proc/cpuinfo | wc -l

worker_connections

Defines the number of maximum of simultaneous connections handled by a worker process. In this number are included all connections like connections with clients, proxied servers Etc. The number of worker_connections cannot exceed the current limit of maximum open files.



worker_rlimit_nofile

Changes the limit of maximum open files for worker process.



pid

Like in all linux applications, defines a file that store the process ID on main process.



http

Provides the configuration file context in which the HTTP server directives are specified.



upstream

Defines a group of servers (defined with server, do not be confused with virtual host) listen on specific port or unix domain socket (inter-process communication). Here you can specify the load balancing method. As we mentioned before, the default method is round robin.



If you add the directive least_conn the method Least connected will be used.
If you add the directive ip_hash the IP hash method will be used.

max_fails specifies the number of unsuccessful attempts to communicate with server within the interval fail_timeout. The default value of max_fails is 1.

fail_timeout, as mentioned above, specifies the amount of time for server to be considered unavailable if max_fails is reached within this interval or considered available (The worker will try to communicate) after this amount of time. The default value for this parameter is 10 (seconds).

weight specifies the weight of server. In our example the loop will completed after 5 requests, 2 for the first server and one for each remaining servers. Default value for weight is 1.



server

Creates virtual host for domains specified with server_name (example.com) listening (listen) on specific IP and port. In our configuration example we have 2 virtual hosts, one listen on port 80 and one listen on port 443 for domains example.com and example.net.



server {
listen 203.0.113.1:80;
server_name example.com;
server_name example.net;
return 301 https://$server_name$request_uri;
}


This virtual server handle the HTTP connections on port 80 and redirects (return 301 moved permanently, the status code used for permanent URL redirection) the traffic to HTTPS $server_name with uri $request_uri (Practically to the virtual server responsible for HTTPS connections, on port 443)

The second virtual server listen on port 443 handle the HTTPS traffic and maps the location with a proxied server, in our example http://cluster. In additional to http virtual server, here the SSL parameters must be specified (like certificate, protocols, ciphers etc.)

location
Sets configuration depending on request URI. You can set specific configuration if URI match with a string pattern (like /images) or with a regular expression (like ~* \.(gif|jpg|jpeg)$ ).



proxy_pass

Maps the location with a proxied server. Sets the protocol, address of a proxied server and an optional URI to which a location should be mapped.



proxy_set_header

Set or redefine the header fields



ssl_certificate and ssl_certificate_key

As the name describes, are certificate and certificate key location for specified domain in virtual host (example.com and example.net)



ssl_session_cache

Sets the types and sizes of caches that store session parameters.



ssl_session_timeout

Specifies a time during which a client may reuse the session parameters stored in a cache



ssl_prefer_server_ciphers

If on, the server ciphers should be preferred over client ciphers when using the SSLv3 and TLS protocols.



ssl_protocols

Enable the specified SSL protocols



ssl_ciphers

Enable the specified ciphers



add_header

Adds the specified field to a response header provided. This works only when the response code is 200, 201, 204, 206, 301, 302, 303, 304 or 307.




In order to use a system with load balanced resources you have to modify a little bit your application (or website).



  • First, you have to copy your web application in all servers. Of course, specific configuration may needed for each server.
  • Second, you have to use the same database (or clustered database for high traffic) for all servers.
  • Third, you have to share the sessions (details about client) between servers, perhaps storing them to database.