CVE-2024-29857: Denial of Service via Algorithmic Complexity in Bouncy Castle ECC

Bouncy Castle provides foundational cryptographic APIs and serves as the underlying TLS and certificate validation provider for a vast array of Java and C# .NET enterprise applications. CVE-2024-29857 is an algorithmic complexity vulnerability residing in the library's Elliptic Curve Cryptography (ECC) implementation. The flaw manifests exclusively when the library processes ECC parameters defined over binary finite fields ($F_{2^m}$).

The vulnerability stems from an uncontrolled resource consumption condition (CWE-400). The computational cost of instantiating binary fields scales quadratically or is directly proportional to the field degree parameter. Attackers exploit this mathematical property by injecting exceptionally large degree values into explicit curve definitions.

Upon encountering these malicious parameters during certificate parsing or curve initialization, the vulnerable application initiates execution loops that process massive polynomial structures. This results in synchronous thread blocking and continuous 100% CPU utilization. Downstream applications depending on Bouncy Castle for untrusted input validation, such as Jenkins, Keycloak, and WebLogic, inherit this exposure directly.

In Elliptic Curve Cryptography over binary finite fields, the field structure is mathematically defined by a reduction polynomial of degree $m$. Foundational field operations require constructing these polynomials and solving quadratic equations to execute point decompression. The algorithmic complexity of these foundational operations is directly proportional to the magnitude of the $m$ parameter.

Prior to the patch, Bouncy Castle implementations in ECCurve.java and ECCurve.cs accepted arbitrary integer values for $m$. The library lacked upper bound validation during the initialization phase of $F_{2^m}$ curves. An attacker could specify a maliciously large value for $m$, up to the maximum 32-bit integer limit.

When point decompression is mandated by the cryptographic payload, the library invokes the solveQuadraticEquation method to resolve the equation $z^2 + z = \beta$. This method implements the Half-Trace approach for odd values of $m$ and Trace-based approaches for even values. Both algorithms utilize iterative loops that execute exactly $m$ times. Supplying an extreme upper-bound integer value forces the thread to execute billions of complex polynomial operations, locking the CPU core indefinitely.

Network-based exploitation requires the target application to parse an untrusted X.509 certificate containing malicious explicit curve parameters. The attacker does not need authentication to trigger the vulnerability, provided the endpoint accepts and evaluates client certificates. The vulnerability triggers during the initial parsing phase, long before cryptographic validation or signature verification occurs.

The attack vector was formalized in USENIX Security 2025 research detailing the X.509DoSTool. The researchers automated the generation of malicious certificates by injecting an exceptionally large $m$ parameter into the binary field definition of the X.509 ASN.1 structure. When a vulnerable service receives this certificate, the ASN.1 parser extracts the parameters and passes them directly to the Bouncy Castle ECCurve.F2m constructor.

The exploit results in synchronous thread blocking. Applications handling concurrent connections will rapidly exhaust their thread pools if multiple malicious certificates are submitted. This leads to a complete denial of service for the affected cryptographic module and the host application.

The remediation strategy focuses on implementing a strict fail-fast boundary check during field instantiation. Bouncy Castle developers modified the buildField method to validate the degree parameter before allocating resources or initiating mathematical operations. This prevents the execution flow from reaching the expensive loop structures inside solveQuadraticEquation.

The Java implementation patch in commit fee80dd230e7fba132d03a34f1dd1d6aae0d0281 introduces a default upper limit of 1142 for the $m$ parameter. This limit accommodates standard secure curve sizes (such as B-571) while preventing denial-of-service conditions.

private static FiniteField buildField(int m, int k1, int k2, int k3) {
    if (m > Properties.asInteger("org.bouncycastle.ec.max_f2m_field_size", 1142)) {
        throw new IllegalArgumentException("field size out of range: " + m);
    }
    // Field construction proceeds safely
}

The C# .NET implementation patch in commit 56daa6eac526f165416d17f661422d60de0dfd63 applies an identical logic flow in ECCurve.cs. The implementation retrieves the maximum threshold from an environment variable, throwing an ArgumentException if the input exceeds the safe boundary.

private static IFiniteField BuildField(int m, int k1, int k2, int k3) {
    int maxM = ImplGetInteger("Org.BouncyCastle.EC.F2m_MaxSize", 1142);
    if (m > maxM)
        throw new ArgumentException("F2m m value out of range");
    // Field construction proceeds safely
}

The primary remediation is upgrading the Bouncy Castle library to the officially patched releases. Java applications must upgrade to version 1.78 or LTS version 2.73.6. C# .NET applications must upgrade to version 2.3.1. Bouncy Castle FIPS (BC-FJA) users must upgrade to version 1.0.2.5.

Administrators who cannot immediately deploy patched binaries can utilize configuration-based workarounds introduced in the backported fixes. The patched versions allow runtime configuration of the maximum $m$ parameter via system properties or environment variables. This enables organizations to adjust the default limit of 1142 if specific legacy applications require larger field sizes.

In Java environments, administrators can set the org.bouncycastle.ec.max_f2m_field_size system property. In .NET environments, the equivalent setting is Org.BouncyCastle.EC.F2m_MaxSize. Network-layer mitigation via Web Application Firewalls (WAF) is generally ineffective, as deep packet inspection of ASN.1 structures within encrypted TLS handshakes is computationally prohibitive and prone to false negatives.