GHSA-CHX6-46F5-W4VP: Uncontrolled Resource Consumption in Tornado CurlAsyncHTTPClient

Tornado is an asynchronous networking library for Python that provides high-performance HTTP client implementations. The library features two distinct client classes: SimpleAsyncHTTPClient, written in pure Python, and CurlAsyncHTTPClient, which wraps the libcurl engine using pycurl bindings. The default configuration of these clients enables automatic decompression of response bodies when they are compressed using gzip or deflate transfer encodings.

While SimpleAsyncHTTPClient historically integrated safety parameters to limit unbounded memory consumption during response parsing, CurlAsyncHTTPClient was implemented without equivalent constraints. This architectural omission exposes systems to uncontrolled resource consumption vulnerabilities when interacting with untrusted upstream servers. Specifically, the client fails to restrict the volume of decompressed data processed in memory.

An upstream attacker can exploit this weakness by serving a highly compressed response, commonly known as a decompression bomb or zip bomb. Because the library performs decompression transparently in its event loop, the system memory is exhausted, causing an Out-of-Memory (OOM) event that terminates the application process. This vulnerability is cataloged as GHSA-CHX6-46F5-W4VP and represents a high-severity denial-of-service vector.

The primary flaw resides in the handling of response bodies within CurlAsyncHTTPClient. When HTTP response decompression is active (decompress_response=True), libcurl automatically manages the decoding of compressed transfer-codings. As the network buffer is filled, libcurl continuously triggers the registered write callback function (pycurl.WRITEFUNCTION) to forward the decoded data to Python.

In buffered mode (where no streaming_callback is defined), Tornado historically stored incoming response bytes directly in an unbounded io.BytesIO buffer. Because BytesIO expands dynamically without enforcing capacity limits, libcurl could stream gigabytes of uncompressed data into Python's heap. The engine had no mechanism to detect that the decompressed payload exceeded acceptable limits, as the wire-level compressed size remained extremely small.

In streaming mode (where a streaming_callback is provided), a different resource exhaustion vector occurs. The callback registered with pycurl immediately enqueued each decompressed chunk to the Tornado IOLoop event queue using IOLoop.add_callback(). Because libcurl processes the socket stream synchronously inside curl_multi_socket_action, it enqueues raw byte chunks much faster than the single-threaded event loop can schedule and execute the application-level callbacks, resulting in a rapid accumulation of memory objects.

The vulnerability is clearly demonstrated by inspecting the original implementation of the writing routines inside tornado/curl_httpclient.py. In older versions, when a streaming callback was configured, the write function registered with pycurl was bound to io_loop.add_callback without any backpressure checks or queue size evaluations.

# Vulnerable configuration in older versions
write_function = buffer.write
curl.setopt(pycurl.WRITEFUNCTION, write_function)

The patch introduced in version 6.5.9 mitigates these vectors by enforcing size limits in both buffered and streaming execution paths. In buffered mode, the write function now checks the current buffer offset plus the new chunk length against self.max_body_size (defaulting to 100MB). If this threshold is exceeded, the function returns zero, which explicitly forces libcurl to abort the transfer with a CURLE_WRITE_ERROR exception.

# Patched Buffered Write Function
def write_function(chunk: bytes) -> int:
    if buffer.tell() + len(chunk) > self.max_body_size:
        curl.info["body_too_large"] = True
        # Returning zero forces libcurl to abort with CURLE_WRITE_ERROR
        return 0
    return buffer.write(chunk)

For streaming mode, Tornado introduced _CurlStreamingBuffer to act as an intermediate queue with backpressure capability. When the size of the queued chunks waiting to be delivered to the application exceeds 1MB (max_buffer_size), the class pauses the transfer by returning pycurl.WRITEFUNC_PAUSE. This instructs libcurl to halt reading from the underlying network socket until the IOLoop drains the existing chunks and invokes curl.pause(pycurl.PAUSE_CONT).

# Patched Streaming Buffer Logic
class _CurlStreamingBuffer:
    max_buffer_size = 1024 * 1024  # 1MB high-water mark
 
    def write(self, chunk: bytes) -> int:
        if self.total + len(chunk) > self.client.max_body_size:
            self.body_too_large = True
            return 0  # Abort transfer
 
        if self.size >= self.max_buffer_size:
            self.paused = True
            return pycurl.WRITEFUNC_PAUSE  # Pause libcurl processing
 
        self.chunks.append(chunk)
        self.size += len(chunk)
        self.total += len(chunk)
        self._schedule_flush()
        return len(chunk)

Exploiting GHSA-CHX6-46F5-W4VP requires an attacker to control or influence the destination URL requested by the target's CurlAsyncHTTPClient. This is highly relevant in systems performing automated web scraping, content fetching, or webhook delivery. Once the client initiates a request to the malicious URL, the attacker's server responds with highly compressed data.

The server responds with standard HTTP headers indicating compression, such as Content-Encoding: gzip or Content-Encoding: deflate. The response payload contains a decompression bomb constructed using recursive compression techniques. For example, a file containing 4GB of null bytes can be compressed to approximately 1MB or less, which easily bypasses wire-level size checks or timeout limits.

As libcurl decompresses the data, the memory footprint of the Tornado application expands exponentially. If the application is operating in buffered mode, the process heap memory is depleted within seconds, triggering the operating system's Out-Of-Memory (OOM) killer. If the application is operating in streaming mode, the callback queue inflates to hundreds of megabytes before the callbacks can be scheduled, culminating in an identical process termination.

The primary impact of this vulnerability is a complete Denial of Service (DoS) of the affected application. Because Tornado is commonly used to build API gateways, web crawlers, and asynchronous microservices, the termination of the main application process halts all concurrent operations. This results in service disruption for all users relying on the affected node.

While the vulnerability resides on the client side, it represents a substantial threat because client applications routinely connect to dynamic, user-supplied, or third-party endpoints. In environments where the client is automated, an attacker can trigger crashes repeatedly, preventing recovery and maintaining a persistent denial of service. The vulnerability does not require any authentication to exploit.

Although this flaw leads primarily to resource exhaustion, it does not allow for direct Remote Code Execution (RCE) or Information Disclosure. However, the resulting instability can be leveraged in chained attacks to bypass monitoring tools, force failovers to less secure nodes, or disrupt critical synchronization services across distributed systems.

The most effective remediation is upgrading the tornado package to versions 6.5.9 (if using the 6.5.x release branch) or 6.6.0 (if upgrading to the next major version branch). These releases introduce the max_body_size enforcement and the backpressure buffering logic for CurlAsyncHTTPClient, establishing a predictable memory ceiling of 100MB plus the 1MB streaming high-water mark.

In scenarios where immediate package upgrades are unfeasible, administrators can mitigate the risk by modifying their application configuration. Developers can explicitly switch the HTTP client backend from CurlAsyncHTTPClient to the pure-Python SimpleAsyncHTTPClient. The pure-Python implementation inherently respects the max_body_size limitation, protecting the host from unbounded decompression attacks.

Alternatively, developers can disable automatic response decompression by setting the decompress_response property to False in their HTTPRequest configurations. While this prevents transparent decompression-based memory exhaustion, any subsequent manual decompression of user-supplied payloads must be implemented with strict input length checks to prevent similar resource exhaustion conditions.