Cloud Resource
Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Toggle Dark/Light/Auto mode Back to homepage

HAProxy LB

When you’re working with a Kubernetes cluster, having a separate external proxy that enables different public IP addresses for each application can be beneficial. In this post, we delve into setting up HAProxy as a frontend load balancer for Kubernetes cluster ingress, focusing on Layer 7 load balancing, SSL offloading, and maintaining separate public IPs via an external proxy.

Understanding the Topology

To give you a visual understanding of what we aim to achieve, consider the following flowchart, where Users are directed to their respective clusters based on the URLs they access:

flowchart LR
A[User] ---> | https://aaa.com | B(Internet)
0[User] ---> | https://bbb.com | B(Internet)
B --> | https URL | C{HAProxy}
C --->|http://aaa.com| D[Cluster A Ingress worker-01]
C --->|http://aaa.com| E[Cluster A Ingress worker-02]
C --->|http://aaa.com| F[Cluster A Ingress worker-03]

C --->|http://bbb.com| G[Cluster B Ingress worker-01]
C --->|http://bbb.com| H[Cluster B Ingress worker-02]
C --->|http://bbb.com| I[Cluster B Ingress worker-03]

To facilitate this, we will use a HAProxy configuration based on ACL (Access Control Lists)…

global
        log /dev/log    local0
        log /dev/log    local1 notice
        chroot /var/lib/haproxy
        stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd listeners
        stats timeout 30s
        user haproxy
        group haproxy
        daemon

        # Default SSL material locations
        ca-base /etc/ssl/certs
        crt-base /etc/ssl/private

        # See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
        ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
        ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
        ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        timeout connect 5000
        timeout client  50000
        timeout server  50000
        errorfile 400 /etc/haproxy/errors/400.http
        errorfile 403 /etc/haproxy/errors/403.http
        errorfile 408 /etc/haproxy/errors/408.http
        errorfile 500 /etc/haproxy/errors/500.http
        errorfile 502 /etc/haproxy/errors/502.http
        errorfile 503 /etc/haproxy/errors/503.http
        errorfile 504 /etc/haproxy/errors/504.http

cache mycache
        total-max-size 4095   # MB
        max-object-size 10000 # bytes
        max-age 30            # seconds

frontend http-in
        bind :443 ssl crt /usr/local/etc/haproxy/ssl-bundle.crt
        mode http
        option httplog
        redirect scheme https if !{ ssl_fc }
        acl http ssl_fc,not
        acl https ssl_fc
        http-request set-header X-Forwarded-Protocol http if http
        http-request set-header X-Forwarded-Protocol https if https
        log global
        acl from_cf src -f /etc/haproxy/CF_ips.lst
        acl cf_ip_hdr req.hdr(CF-Connecting-IP) -m found
        http-request set-header X-Forwarded-For %[req.hdr(CF-Connecting-IP)] if from_cf cf_ip_hdr
        http-request set-src hdr(x-forwarded-for) if from_cf
        http-request set-var(sess.cloudflare) always_true if { http_first_req } from_cf

        acl is_aaa_com hdr_end(host) -i aaa.com
        use_backend aaa_com if is_aaa_com

        acl is_bbb_com hdr_end(host) -i bbb.com
        use_backend bbb_com if is_bbb_com

backend aaa_com
        log global
        mode http
        option forwardfor
        http-request cache-use mycache
        http-response cache-store mycache
        compression algo gzip
        compression type text/html "text/html; charset=utf-8" text/html;charset=utf-8 text/plain text/css text/javascript application/x-javascript application/javascript application/ecmascript application/rss+xml application/atomsvc+xml application/atom+xml application/atom+xml;type=entry application/atom+xml;type=feed application/cmisquery+xml application/cmisallowableactions+xml application/cmisatom+xml application/cmistree+xml application/cmisacl+xml application/msword application/vnd.ms-excel application/vnd.ms-powerpoint image/svg+xml
        balance roundrobin
        option httpchk

        http-check send hdr User-Agent "HAproxy-health-check" meth HEAD uri / hdr Host "aaa.com"

        cookie SERVERUID insert indirect nocache httponly secure
        server a-worker-01 192.168.0.2:80 cookie ingress-01 check
        server a-worker-02 192.168.0.3:80 cookie ingress-02 check
        server a-worker-03 192.168.0.4:80 cookie ingress-03 check

backend bbb_com
        log global
        mode http
        option forwardfor
        http-request cache-use mycache
        http-response cache-store mycache
        compression algo gzip
        compression type text/html "text/html; charset=utf-8" text/html;charset=utf-8 text/plain text/css text/javascript application/x-javascript application/javascript application/ecmascript application/rss+xml application/atomsvc+xml application/atom+xml application/atom+xml;type=entry application/atom+xml;type=feed application/cmisquery+xml application/cmisallowableactions+xml application/cmisatom+xml application/cmistree+xml application/cmisacl+xml application/msword application/vnd.ms-excel application/vnd.ms-powerpoint image/svg+xml
        balance roundrobin
        option httpchk

        http-check send hdr User-Agent "HAproxy-health-check" meth HEAD uri / hdr Host "bbb.com"

        cookie SERVERUID insert indirect nocache httponly secure
        server b-worker-01 192.168.0.5:80 cookie ingress-01 check
        server b-worker-02 192.168.0.6:80 cookie ingress-02 check
        server b-worker-03 192.168.0.7:80 cookie ingress-03 check

frontend stats
        bind 192.168.0.61:8404
        mode http
        option httpclose
        http-request use-service prometheus-exporter if { path /metrics }
        stats enable
        stats uri /stats
        stats refresh 10s

listen statistic
        bind 192.168.0.61:9000
        mode http
        stats enable
        stats hide-version
        stats realm Haproxy\ Statistics
        stats uri /
        stats auth haproxy:<password>

By following these steps, you can successfully set up HAProxy as a frontend load balancer for your Kubernetes cluster ingress. This setup will not only enhance your load balancing capabilities on Layer 7 but also facilitate SSL offloading. Further, it allows you to maintain separate public IPs for each of your applications via an external proxy, adding an extra layer of security and manageability to your Kubernetes deployments.

Optimizing Web Performance with HAProxy Caching

Caching is an integral part of any high-performing web infrastructure, and HAProxy, a powerful and flexible load balancer, doesn’t shy away from providing robust caching capabilities. HAProxy’s in-memory cache, aptly named mycache in our configuration, improves response times by storing and delivering frequently accessed data directly to the client, thereby reducing the load on backend servers.

The mycache configuration operates on three key parameters: total-max-size, max-object-size, and max-age. Let’s delve into these parameters to understand their roles and how you can optimize them.

cache mycache
        total-max-size 4095   # MB
        max-object-size 10000 # bytes
        max-age 30            # seconds

  1. total-max-size: Defines the maximum size of the cache in megabytes (MB). In our case, it’s set to 4095 MB, allocating about 4GB of RAM for the cache. Be cautious while setting this value as it directly impacts your server’s memory consumption.

  2. max-object-size: Specifies the maximum size of an individual object that can be stored in the cache. It’s measured in bytes, and we’ve set it to 10000 bytes or roughly 10KB. Any object larger than this size won’t be cached.

  3. max-age: Determines how long an object stays in the cache before it’s considered stale and is subsequently removed. The value is given in seconds, with our configuration set to hold objects for 30 seconds.

By understanding and effectively managing these parameters, you can tailor your HAProxy cache to align with your application needs, significantly boosting web performance and ensuring a smoother user experience.

A deep dive into HAProxy’s frontend configuration. Learn how to implement SSL, set HTTP headers, and use ACLs for efficient traffic management.

In the intricate web of network traffic management, HAProxy’s frontend configuration plays a pivotal role. It serves as the first point of contact for client requests and decides how to direct traffic to backend servers. The frontend configuration, encompassing SSL implementation, HTTP headers management, and Access Control Lists (ACLs), can significantly enhance your server’s security and efficiency.

frontend http-in
        bind :443 ssl crt /usr/local/etc/haproxy/ssl-bundle.crt
        mode http
        option httplog
        redirect scheme https if !{ ssl_fc }
        acl http ssl_fc,not
        acl https ssl_fc
        http-request set-header X-Forwarded-Protocol http if http
        http-request set-header X-Forwarded-Protocol https if https
        log global
        acl from_cf src -f /etc/haproxy/CF_ips.lst
        acl cf_ip_hdr req.hdr(CF-Connecting-IP) -m found
        http-request set-header X-Forwarded-For %[req.hdr(CF-Connecting-IP)] if from_cf cf_ip_hdr
        http-request set-src hdr(x-forwarded-for) if from_cf
        http-request set-var(sess.cloudflare) always_true if { http_first_req } from_cf

        acl is_aaa_com hdr_end(host) -i aaa.com
        use_backend aaa_com if is_aaa_com

        acl is_bbb_com hdr_end(host) -i bbb.com
        use_backend bbb_com if is_bbb_com

  1. bind :443 ssl crt /usr/local/etc/haproxy/ssl-bundle.crt: This line instructs HAProxy to bind to port 443 (the default HTTPS port) and implement SSL using the provided certificate bundle. Key file name ssl-bundle.crt.key .

  2. redirect scheme https if !{ ssl_fc }: Here, HAProxy checks if the incoming connection uses SSL (ssl_fc). If not, it redirects the client to the HTTPS version of the requested URL, enhancing your application’s security.

  3. http-request set-header X-Forwarded-Protocol ...: These lines set the X-Forwarded-Protocol header to either http or https based on the scheme of the incoming request. This information can be helpful for your backend servers to understand the client’s original protocol.

  4. acl from_cf src -f /etc/haproxy/CF_ips.lst and subsequent lines: These Access Control Lists (ACLs) and associated actions deal with traffic from Cloudflare, setting appropriate headers and source IPs based on Cloudflare’s connecting IPs.

  5. acl is_aaa_com hdr_end(host) -i aaa.com and acl is_bbb_com hdr_end(host) -i bbb.com: These ACLs determine which backend to use based on the host header of the incoming HTTP request. If the host header ends with aaa.com, the request is forwarded to aaa_com backend, and if it ends with bbb.com, the request is directed to bbb_com backend.

By leveraging HAProxy’s robust frontend configuration, you can ensure not only the secure transmission of data but also efficient and intelligent traffic management, leading to an optimized and secure web environment.

Explore HAProxy’s backend configuration and learn how to balance loads, optimize traffic, and maintain high availability for your web services.

When it comes to serving client requests, the configuration of the backend servers in HAProxy plays a crucial role. This section covers several features of a typical HAProxy backend configuration, including load balancing, caching, compression, health checking, and session persistence.

backend aaa_com
        log global
        mode http
        option forwardfor
        http-request cache-use mycache
        http-response cache-store mycache
        compression algo gzip
        compression type text/html "text/html; charset=utf-8" text/html;charset=utf-8 text/plain text/css text/javascript application/x-javascript application/javascript application/ecmascript application/rss+xml application/atomsvc+xml application/atom+xml application/atom+xml;type=entry application/atom+xml;type=feed application/cmisquery+xml application/cmisallowableactions+xml application/cmisatom+xml application/cmistree+xml application/cmisacl+xml application/msword application/vnd.ms-excel application/vnd.ms-powerpoint image/svg+xml
        balance roundrobin
        option httpchk

        http-check send hdr User-Agent "HAproxy-health-check" meth HEAD uri / hdr Host "aaa.com"

        cookie SERVERUID insert indirect nocache httponly secure
        server a-worker-01 192.168.0.2:80 cookie ingress-01 check
        server a-worker-02 192.168.0.3:80 cookie ingress-02 check
        server a-worker-03 192.168.0.4:80 cookie ingress-03 check

  1. option forwardfor: The ‘forwardfor’ option enables HAProxy to include the client’s IP address in the request forwarded to the backend server, which is often required for client tracking or blacklisting purposes.

  2. http-request cache-use mycache and http-response cache-store mycache: These directives instruct HAProxy to use the ‘mycache’ cache for serving and storing HTTP requests and responses, respectively. This can significantly speed up response times for cacheable resources.

  3. compression algo gzip and compression type ...: These lines enable and configure gzip compression for various types of responses, which can reduce bandwidth usage and improve response times, especially for text-based resources.

  4. balance roundrobin: The ‘roundrobin’ balancing algorithm distributes incoming requests evenly across the available servers, helping to maintain high availability and prevent any single server from becoming a bottleneck.

  5. http-check send hdr ...: This directive configures HAProxy to send health check requests to the backend servers. If a server does not respond to the health check, HAProxy will stop sending requests to it until it becomes responsive again.

  6. cookie SERVERUID insert indirect nocache httponly secure: This line manages session persistence through cookies. It tells HAProxy to insert a cookie named ‘SERVERUID’ into the response if the request didn’t initially contain this cookie. This way, subsequent requests from the same client will be directed to the same backend server, which is crucial for session-dependent applications.

  7. server a-worker-01 192.168.0.2:80 cookie ingress-01 check: These lines define the backend servers. The ‘check’ option enables health checking for these servers.

By understanding these various aspects of HAProxy’s backend configuration, you can tailor it to your specific needs, ensuring efficient traffic management, optimal server performance, and high availability for your services.

Dive into the details of HAProxy’s statistics and metrics exposition configuration, exploring the significance of different directives in gathering and viewing vital performance data.

Maintaining a high-performance infrastructure requires constant monitoring and analysis of your HAProxy load balancer. By exposing essential statistics and metrics, HAProxy provides visibility into your application’s health and performance. In this post, we will discuss some crucial HAProxy configuration parts that enable statistics exposition and how they contribute to a more transparent and manageable system.

frontend stats
        bind 192.168.0.61:8404
        mode http
        option httpclose
        http-request use-service prometheus-exporter if { path /metrics }
        stats enable
        stats uri /stats
        stats refresh 10s

listen statistic
        bind 192.168.0.61:9000
        mode http
        stats enable
        stats hide-version
        stats realm Haproxy\ Statistics
        stats uri /
        stats auth haproxy:<password>

  1. bind 192.168.0.61:8404 and bind 192.168.0.61:9000: These lines tell HAProxy to bind the stats frontend and the ‘statistic’ listen block to specific IP addresses and ports. The bind directive determines where HAProxy listens for incoming connections.

  2. mode http: This directive sets the mode to HTTP, meaning HAProxy will treat all incoming connections as HTTP requests. This is vital because the statistics interface is a web page that provides real-time data in an HTTP-friendly format.

  3. http-request use-service prometheus-exporter if { path /metrics }: This configuration line is specifically tailored for integration with Prometheus, a popular open-source monitoring and alerting toolkit. It instructs HAProxy to use the prometheus-exporter service if the request path is ‘/metrics’. This service converts the statistics from HAProxy’s native format into a format that Prometheus can understand.

  4. stats enable, stats uri /stats, and stats refresh 10s: These directives are related to HAProxy’s built-in statistics module. The ‘stats enable’ directive turns on the statistics module, the ‘stats uri’ defines the URL where the statistics page can be accessed, and ‘stats refresh’ sets the refresh rate of the statistics page to 10 seconds.

  5. stats enable, stats hide-version, stats realm Haproxy\ Statistics, stats uri /, and stats auth haproxy:<password>: In the ’listen statistic’ block, these directives enable the stats, hide the HAProxy version from the stats page, set a realm for HTTP authentication, define the URI for accessing the stats page, and set up authentication credentials, respectively. These settings provide both a layer of security and a user-friendly interface to access your HAProxy statistics.

Through these settings, HAProxy allows you to monitor its performance and gain insights into your application’s behavior, helping you make informed decisions about resource allocation, troubleshooting, and system improvements.

In conclusion ,configuring HAProxy appropriately is pivotal to the optimal functioning and management of your infrastructure. Every directive and line of code in the configuration plays a crucial role in maintaining the balance between resource allocation, security, and performance. Understanding and effectively implementing these elements, from setting up frontends and backends, managing caching and compression, to exposing detailed statistics for monitoring, can make a significant difference in how your system performs under varying loads.

However, the configuration of HAProxy is not a one-time task. It requires constant monitoring, tweaking, and tuning according to the evolving requirements of your system. The robust and flexible nature of HAProxy allows it to adapt to these changes, making it a highly reliable solution for load balancing and proxying HTTP and TCP applications.

In the end, the power of HAProxy is as vast as your ability to harness its features and capabilities. The more familiar you become with its intricacies, the better you can utilize it to create a robust, resilient, and highly available system. Don’t be afraid to delve deeper into the documentation, experiment with different configurations, and optimize the setup to meet your specific needs. Remember, a well-configured HAProxy is an invaluable asset to any high-performance application infrastructure.

Strengthening Security with HAProxy in Your Kubernetes Environment

In today’s digital landscape, the protection and security of data and services have never been more critical. Implementing robust security measures in a Kubernetes environment can sometimes be challenging due to the complex nature of the system. However, using HAProxy as a frontline barrier offers an efficient way to enhance security significantly.

HAProxy acts as a sturdy shield, intercepting all incoming traffic and protecting the Kubernetes cluster from direct access. This approach has a twofold benefit. Firstly, it controls and secures the flow of traffic entering the Kubernetes ecosystem, minimizing the exposure to potential external threats. Secondly, it liberates the cluster from handling client connections directly, thereby reserving more resources for processing requests internally and improving overall efficiency.

One standout feature of HAProxy is its ability to provide SSL termination. In a time when data security is paramount, the SSL termination feature ensures that all data is encrypted when in transit, providing a secure communication channel and enhancing data privacy.

Moreover, HAProxy can enforce Access Control Lists (ACLs) and rate-limiting policies to mitigate common web threats, such as DDoS attacks. By defining and applying appropriate ACLs, the traffic reaching your services can be adequately controlled, mitigating risks and potential vulnerabilities. Rate limiting can also be used to limit the number of requests a client can make to your service within a certain timeframe, protecting your services from being overwhelmed by an influx of traffic.

HAProxy’s extensive logging capability offers another layer of security. Monitoring traffic patterns in real-time helps to identify and mitigate potential threats before they can impact your services. Regular log analysis can also provide insights into usage patterns and performance, offering valuable data for capacity planning and infrastructure optimization.

However, while HAProxy offers many security enhancements, it’s crucial to remember that no single tool or approach can offer complete protection. Regular audits of your HAProxy and Kubernetes configurations, staying updated with the latest software versions, and following best security practices are all critical components of maintaining a secure system.

In conclusion, the integration of HAProxy into your Kubernetes architecture provides a vital line of defense, allowing your Kubernetes cluster to focus on efficiently orchestrating your containers and running your applications smoothly. With HAProxy in place, you can enjoy a more secure, controlled, and performant Kubernetes environment.