FOSSGuides

Practical Linux and Free Open Source Software Guides

Useful HAProxy Tips and Tricks – Part 1

by · Published · Updated

Overview:

This guide explains some key tips and tricks for deploying and configuring HAProxy as a load balancer and reverse proxy in a production environment.

Preface

HAProxy offers several basic, standard, and advanced features for HTTP reverse proxying, TCP proxying, and load balancing with SSL termination. These features can be enabled using various categorized configuration directives and parameters. These directives may be added in one or more of the four essential sections of an HAProxy configuration.

The four sections are:

  • global – which contains settings that define process-wide security and performance tunings that affect HAProxy at a low level,
  • defaults – which contains settings that apply to all of the frontend and backend sections that come after it,
  • frontend – this section defines the IP addresses and ports that clients can connect to, usually the * symbol is used to define all IP addresses of the server where HAProxy is installed, and
  • backend – this section defines a group of upstream servers that will be load-balanced and assigned to handle requests.

Another section is the listen section. It can be used to combine the functionality of a frontend and a backend into one. However, to make the configuration file more readable, most users separate the frontend and backend sections.

Importantly, not all directives can work in all sections. Some directives work in specific sections. The HAProxy documentation (link under references) clearly defines all sections in which each directive is valid.

Okay, let’s get started with the tips and tricks.

1. Define Common Settings in the Defaults Section

To avoid repeating settings in many sections, if a setting you have added is supposed to apply to all other frontend and backend sections, it is better you add it in a default section. Note that you can have many default sections. Subsequent defaults sections will override those that came before and reset all options to their default values.

2. Set the Correct HAProxy Working Mode

By default, HAProxy can operate in two modes: HTTP mode and TCP mode. You need to know when to use what mode when deploying HAProxy.

For more information, read: When To Run HAProxy in HTTP or TCP Modes

3. Enable SSL Termination

HAProxy only supports PEM-formatted SSL certificates which should contain all necessary SSL files including the public certificate and the private key. Once you have your certificate file and the key, you can create a .prm file like this:

#cat  example1.com.key  example1.com.pem >/etc/ssl/certs/example1.com.pem

To enable SSL termination, add the sslparameter to the bind directive listener. And use the crt parameter to identify the location of the SSL certificates. You can add multiple crt parameters to enable multiple SSL certificates as shown.

frontend http_in
  bind   *:80
  bind *:443 ssl crt /etc/ssl/certs/example1.com.pem crt /etc/ssl/certs/example2.com.pem crt /etc/ssl/certs/example3.com.pem
4. Limit supported SSL versions

HAProxy allows for limiting supported versions of SSL to be negotiated. This allows you to disable usage of SSL versions that are known to have vulnerabilities in them. Simply use the ssl-default-bind-options ssl-min-ver setting in the global section as shown. Possible values are SSLv3, TLSv1.0, TLSv1.1, TLSv1.2, and TLSv1.3.

In this example, I have set the minimum accepted SSL version to TLSv1.2, meaning versions below it TLSv1.1, TLSv1.0, and SSLv3, will not be negotiated.

global
  ssl-default-bind-options ssl-min-ver TLSv1.2
5. Redirect HTTP to HTTPS

To redirect HTTP to HTTPS, use the redirect scheme directive as shown:

frontend http_in
  bind   *:80
  bind *:443 ssl crt /etc/ssl/certs/example1.com.pem crt /etc/ssl/certs/example2.com.pem crt    /etc/ssl/certs/example3.com.pem  alpn h2,http/1.1
  redirect scheme https code 301 if !{ ssl_fc }
6. Redirect www to non-www

To redirect www to non-www, add the redirect prefix directive as follows. Replace example.com with your domain:

redirect prefix https://example.com code 301 if { hdr(host) -i www.example.com }
7. Use a Specific Backend for a Domain

To use a specific backend for all requests to a specific domain name, use the use_backend directive as follows. Replace example1.com and example2.com with your domains:

use_backend example1_backend if { hdr(host) -m dom example1.com } 
use_backend example2_backend if { hdr(host) -m dom example2.com }
8. Set Load Balancing Algorithm

If you have several upstream servers defined in a backend or listen section, you can set the load balancing algorithm that HAProxy should use, using the balance directive. The default algorithm is roundrobin and possible values are leastconn, first, static-rr, hash, source, uri, url_param, hdr, random, and rdp-cookie.

This setting can only be applied in the default, listen, and backend sections.

balance leastconn
9. Delete HTTP Header in Response

To remove an HTTP header from a response, use the http-response del-header directive like this:

http-response  del-header Server
http-response  del-header Via
10. Add or Set HTTP Header in Response

To add an HTTP header to a response, use the http-response add-header directive like this:

http-response  add-header Server “FOSSGuides Load Balancer”

To set an HTTP header to a response, use the http-response set-header directive. It works just like the http-response add-header directive except that the header name is first removed if it exists. This example shows how to add HTTP security headers in HAProxy.

http-response set-header X-Frame-Options DENY
http-response set-header X-XSS-Protection 1;mode=block
http-response set-header X-Content-Type-Options nosniff
http-response set-header Strict-Transport-Security max-age=31536000;includeSubDomains;preload
http-response set-header Referrer-Policy no-referrer-when-downgrade
http-response set-header Content-Security-Policy "default-src self *.example1.com example2.com fonts.googleapis.com fonts.gstatic.com"
http-response set-header Expect-CT enforce,\ max-age=86400
http-response set-header Referrer-Policy strict-origin-when-cross-origin
http-response add-header X-Permitted-Cross-Domain-Policies none
http-response set-header Permissions-Policy "accelerometer=(), autoplay=(), camera=(), cross-origin-isolated=(), display-capture=(), encrypted-media=(), fullscreen=(self), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), sync-xhr=(), usb=(), xr-spatial-tracking=(), clipboard-read=(self), clipboard-write=(self), gamepad=()"
11. Set a Backen Server Cookie

To enable cookie-based persistence in a backend, use the cookie directive. In this example, the cookie name is SVR_ID (remember to set your custom cookie name) and the arguments are insert, indirect, nocache, dynamic, secure, and httponly.

cookie SVR_ID insert indirect nocache dynamic secure httponly
dynamic-cookie-key  KSNDKNklsndkn89jnwedn
12. Define Maximum Number of Connections

To set the maximum number of connections use the maxconn directive. If defined in the global section, it defines the maximum per-process number of concurrent connections.

global
  maxconn 80000

Besides, you can define a maxconn setting on each server line in a backend or listen section of your configuration. This allows HAProxy to limit the number of concurrent connections that it will relay to the upstream server.

backend  webservers
  server  nginx_svr1 10.10.1.5:80 maxconn 100
  server  nginx_svr1 10.10.1.6:80 maxconn 100
  server  nginx_svr1 10.10.1.7:80 maxconn 100
13. Enable Health Checks

If you are running many upstream servers, it’s recommended to enable health checks so that if one or even several servers fail, clients can still use your application as long as other servers are still running. HAProxy will perform health checks to know which servers are alive and ready to receive requests. HAProxy supports active and passive health checks.

Below are several examples of how to enable active health checks. This configuration snippet shows the simple form of active health checks in HAProxy. Here, HAProxy will try to establish a TCP connection every two seconds (the default inter parameter). After three failed connections (default fall parameter), the server is removed temporarily, until HAProxy gets at least two successful connections (default rise parameter), reinstating the server into the backend.

backend  webservers
  server  nginx_svr1 10.10.1.5:80 maxconn 100  check
  server  nginx_svr1 10.10.1.6:80 maxconn 100  check
  server  nginx_svr1 10.10.1.7:80 maxconn 100  check

You can customize the health check time intervals using the inter parameter (changes the interval between checks), the fall parameter (number of failed checks allowed after which server is marked down), and the rise parameter (sets how many passing checks there must be before the server is marked alive again).

backend  webservers
  server  nginx_svr1 10.10.1.5:80 maxconn 100  check inter 10s fall 5 rise 5
  server  nginx_svr1 10.10.1.6:80 maxconn 100  check inter 10s fall 5 rise 5
  server  nginx_svr1 10.10.1.7:80 maxconn 100  check inter 10s fall 5 rise 5

Alternatively, for web applications, you can use HTTP health checks using the option httpck directive like this:

backend  webservers
  option httpchk 
  server  nginx_svr1 10.10.1.5:80 maxconn 100  check
  server  nginx_svr1 10.10.1.6:80 maxconn 100  check
  server  nginx_svr1 10.10.1.7:80 maxconn 100  check

To instruct HAProxy to expect a certain status code, you can use the http-check expect directive as shown:

backend  webservers
  option httpchk 
  http-check send meth GET uri /health
  http-check expect status 200
  server  nginx_svr1 10.10.1.5:80 maxconn 100  check
  server  nginx_svr1 10.10.1.6:80 maxconn 100  check
  server  nginx_svr1 10.10.1.7:80 maxconn 100  check

You can enable passive health checks like this.

backend  webservers
  option httpchk 
  http-check send meth GET uri /health
  http-check expect status 200
  server  nginx_svr1 10.10.1.5:80 maxconn 100  check observe layer7 error-limit 50 on-error mark-down
  server  nginx_svr1 10.10.1.6:80 maxconn 100  check  observe layer7 error-limit 50 on-error mark-down
  server  nginx_svr1 10.10.1.7:80 maxconn 100  check  observe layer7 error-limit 50 on-error mark-down
14. Fix Timeout Errors

To fix timeout errors in HAProxy, you need to adjust the timeout parameters. One of the most common errors is 504 gateway timeout. To understand more about the timeout directives and how to fix the 504 gateway timeout error, read this guide.

15. Always Check HAProxy Configuration for Syntax Correctness After Making Changes

Always remember to check your HAProxy configuration to ensure that it is valid (and the syntax is correct). This helps to ensure that a change to your configuration file does not unintentionally stop your load balancer and bring down your services.

To check your HAProxy configuration, run the following haproxy command where the -c flag implies check mode where the haproxy command checks the config file(s) and exits. The -f flag specifies the config file to be checked.

$sudo haproxy -c -f /etc/haproxy/haproxy.cfg
OR
#haproxy -c -f /etc/haproxy/haproxy.cfg
Check HAProxy configuration to ensure it is valid
Sample HAProxy Configuration File

Sample HAProxy configuration file showing all the above tips and tricks defined:

global
    maxconn 100000
    log	127.0.0.1 local2
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock user haproxy group haproxy mode 660 level admin expose-fd listeners
    stats timeout 30s
    user haproxy
    group haproxy
    daemon
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
    ssl-default-bind-options ssl-min-ver TLSv1.2
    ssl-dh-param-file /etc/ssl/dhparam

defaults
    log	global
    mode http 
    option httplog
    option dontlognull
    option forwardfor
    timeout connect 30s
    timeout check   30s
    timeout client  20m
    timeout server  30m
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

listen stats
    bind *:8500
    stats enable
    stats hide-version
    stats uri /monitor
    stats realm Haproxy\ Statistics
    stats refresh 5s
    stats admin if TRUE
    stats auth admin:f9394tp@1ss

frontend rq_receiver
      mode http
      bind *:80
   bind *:443 ssl crt /etc/ssl/certs/example1.com.pem crt /etc/ssl/certs/example2.com.pem crt    /etc/ssl/certs/example3.com.pem  alpn h2,http/1.1
      redirect scheme https code 301 if !{ ssl_fc }
      redirect prefix https://example.com code 301 if { hdr(host) -i www.example.com }
      use_backend example1_backend if { hdr(host) -m dom example1.com } 
      use_backend example2_backend if { hdr(host) -m dom example2.com }
      # HSTS (63072000 seconds)
      http-response set-header Strict-Transport-Security max-age=63072000
      default_backend   webservers

backend webservers
      balance roundrobin
      option httpchk GET /health
      http-check expect status 200

      http-response  del-header Server
      http-response  del-header Via
      http-response  del-header X-Kong-Proxy-Latency
      http-response  del-header X-Kong-Upstream-Latency
      http-response  add-header Server "Entryxit Server"
      http-response set-header X-Frame-Options DENY
      http-response set-header X-XSS-Protection 1;mode=block
      http-response set-header X-Content-Type-Options nosniff
http-response set-header Strict-Transport-Security max-age=31536000;includeSubDomains;preload
      http-response set-header Referrer-Policy no-referrer-when-downgrade
      http-response set-header Expect-CT enforce,\ max-age=86400
      http-response set-header Referrer-Policy strict-origin-when-cross-origin
      http-response add-header X-Permitted-Cross-Domain-Policies none
      http-response set-header Permissions-Policy "accelerometer=(), autoplay=(), camera=(), cross-origin-isolated=(), display-capture=(), encrypted-media=(), fullscreen=(self), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), picture-in-picture=(), publickey-credentials-get=(), screen-wake-lock=(), sync-xhr=(), usb=(), xr-spatial-tracking=(), clipboard-read=(self), clipboard-write=(self), gamepad=()"
      cookie SVR_ID insert indirect nocache dynamic secure httponly
      dynamic-cookie-key kldf03434JKSNDVS
      server nginx_svr1 10.30.1.10:8080 check maxconn 500  check inter 10s fall 5 rise 5
      server nginx_svr2 10.30.1.11:8080 check maxconn 500  check inter 10s fall 5 rise 5
      server nginx_svr3 10.30.1.12:8080 check maxconn 500  check inter 10s fall 5 rise 5
      server nginx_svr4 10.30.1.13:8080 check maxconn 500  check inter 10s fall 5 rise 5




backend  example1_backend
      option httpchk HEAD /
      http-response  del-header Server
      http-response  del-header Via
      cookie SVR_ID insert indirect nocache dynamic secure httponly
      dynamic-cookie-key JKSBJD934903jksndksd
      server svr1 10.10.3.60:80 check check inter 10s fall 5 rise 5

backend  example2_backend
      balance leastconn
      http-response  del-header Server
      http-response  del-header Via
      cookie SVR_ID insert indirect nocache dynamic secure httponly
      dynamic-cookie-key KLSND93lsdkldkl30940934
      server svr1 10.10.2.69:80 check check inter 10s fall 5 rise 5
      server svr2 10.10.2.69:80 check check inter 10s fall 5 rise 5
Final words

That is all I have prepared for you in this guide. In upcoming guides, I will be expounding on many of these tips and tricks to show more advanced configuration options. I would like to hear from you. If you have any tips or tricks to add here or any comments to share, use the feedback form below.

References
1. https://www.haproxy.com/blog/the-four-essential-sections-of-an-haproxy-configuration
2. https://www.haproxy.com/blog/haproxy-ssl-termination
3. https://www.haproxy.com/blog/how-to-enable-health-checks-in-haproxy
4. https://docs.haproxy.org/

LinkedInPinShares

Tags:

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

Page Contents

→ Page Contents