RousselTMLEARNING

Proxy & networking

HAProxy: adding and removing backends dynamically

Learn how to use the HAProxy Runtime API to create, configure and delete backends and servers on the fly, without any restart.

HAProxy: adding and removing backends dynamically

1. Introduction to dynamic management #

Traditionally, changing the structure of backends required reloading the configuration file. The Runtime API now lets you make these changes in memory.
Warning: Adding and removing backends and servers dynamically through the Runtime API requires HAProxy 2.8 or later.

Basic configuration #

To use these commands, your HAProxy instance must expose an administration socket with an admin level in the global section of your configuration:

Classic setup:
stats socket /run/haproxy/admin.sock user haproxy group haproxy mode 660 level admin
and/or through an IP (remote access, but beware of the security risks if you use an IP other than localhost):
stats socket ipv4@127.0.0.1:9999  level admin  expose-fd listeners

2. Glossary of key commands #

To change the infrastructure on the fly, it is essential to understand the role of each Runtime API verb.

help #

The help command displays all the available commands. The syntax is as follows; only the endpoint changes depending on how you enabled the socket:

The path to use with socat may vary depending on your configuration
  • Classic:
    echo 'help' | socat stdio /var/run/haproxy.stat
  • IP/PORT:
    echo 'help' | socat stdio tcp4-connect:127.0.0.1:9999

add (backend / server) #

The add command creates a new object (backend or server) in memory. A backend can be created from scratch or by inheriting the settings of an existing defaults section (essential for timeouts).

Warning: You may run into resolution problems when adding a server. You can see it by running this command, in which a server has no IP:
$ echo "show servers conn default" | socat stdio /run/haproxy/admin.sock
# bkname/svname bkid/svid addr port - purge_delay served used_cur used_max need_est idle_sess unsafe_nb safe_nb idle_lim idle_cur idle_per_thr[10]
default/web1 4/1 192.168.48.3 80 - 5000 0 0 1 1 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0
default/web2 4/2 192.168.48.2 80 - 5000 0 0 1 1 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0
default/web3 4/3 - 80 - 5000 0 0 0 0 0 0 0 -1 0 0 0 0 0 0 0 0 0 0 0
You can avoid this by precomputing the IP:
IP=$(getent hosts web3 | awk '{print $1}'); echo "add server default/web3 $IP:80 check" | socat stdio /run/haproxy/admin.sock

enable (server / health) #

By default, a dynamically added server is in maintenance mode (maint). enable server makes it operational. enable health specifically enables health checks for that server.

publish (backend) #

This is a validation step. As long as a backend is not published, it is invisible to frontends, even though it exists in memory. This lets you fully configure a pool before exposing it to traffic.

map (add / show / del) #

Maps are lookup tables (e.g. URL -> backend) stored in memory. add map adds a routing rule, del map removes it, and show map lets you debug the table's content without reloading HAProxy.

3. Adding a backend and a server #

Here is the workflow to create a new server pool and make it available.

Creation and activation #

The process breaks down into 4 key steps: adding the backend, adding the server, enabling health checks and publishing.

# 1. Ajouter le backend en héritant d'une section defaults
echo "add backend test-backend from mydefaults mode http" | socat stdio unix-connect:/run/haproxy/admin.sock

# 2. Ajouter un serveur au backend
echo "add server test-backend/server1 127.0.0.1:3000 check" | socat stdio unix-connect:/run/haproxy/admin.sock

# 3. Activer le serveur et ses health checks
echo "enable server test-backend/server1" | socat stdio unix-connect:/run/haproxy/admin.sock
echo "enable health test-backend/server1" | socat stdio unix-connect:/run/haproxy/admin.sock

# 4. Publier le backend pour le rendre utilisable par les frontends
echo "publish backend test-backend" | socat stdio unix-connect:/run/haproxy/admin.sock

Dynamic routing with map files #

To send traffic to this new backend without changing the frontend, use in-memory map files:

# Associer le chemin /test au nouveau backend
echo "add map virt@paths.map /test test-backend" | socat stdio unix-connect:/run/haproxy/admin.sock

4. Removing a backend cleanly #

Removal requires following a precise order so as not to break in-flight connections.

Removal procedure #

You must put the server into maintenance and wait until it is 'removable' before physically deleting it.

# 1. Passer le serveur en maintenance
echo "set server test-backend/server1 state maint" | socat stdio unix-connect:/run/haproxy/admin.sock

# 2. Attendre et supprimer le serveur
echo "wait 2s srv-removable test-backend/server1; del server test-backend/server1" | socat stdio unix-connect:/run/haproxy/admin.sock

# 3. Dé-publier et supprimer le backend
echo "unpublish backend test-backend" | socat stdio unix-connect:/run/haproxy/admin.sock
echo "wait 2s be-removable test-backend; del backend test-backend" | socat stdio unix-connect:/run/haproxy/admin.sock

5. Important points to watch #

A few behavior rules of the HAProxy engine you should know.

Publishing and force switch #

A backend referenced by a use_backend is ignored if it is not published. To force its use even when 'unpublished', use the force-be-switch directive.

Memory management #

To allow dynamic additions, HAProxy keeps the defaults sections in memory. If you don't use dynamic backends, you can free this memory with the following global directive:

global
    tune.defaults.purge on

Conclusion #

Towards full automation #

This approach is ideal to couple HAProxy with tools such as Consul or custom controllers, letting you scale your infrastructure without ever causing micro-outages from process reloads.

Related articles

Anatomy of HAProxy configuration, clustering and high availability

Understand the structure of an haproxy.cfg file: the global, defaults, frontend (high performance), backend a…

HAProxy in Kubernetes: the high-performance Ingress Controller

Discover in depth how HAProxy integrates with Kubernetes: direct-to-pod architecture, hot update options (DNS…

What's new in HAProxy: OpenTelemetry, native JWT and performance optimizations

Explore HAProxy's latest innovations: native OpenTelemetry support for observability, JWT validation at the e…

Keep learning

Support us

♥ SUPPORT US

Support RousselTM Learning

Courses, articles, labs and certificates stay free with no intrusive ads. A voluntary contribution, of any amount you choose, pays for hosting, lab environments and new content.

Contribute with PayPal Any amount · card or PayPal accountLearn more
QR code PayPal
Or scan this QR code with your phone

Thank you, every contribution counts!