The One That Got Away: Crashing Rust RSA with a Single Integer

Rust is the golden child of modern systems programming. We love it for its borrow checker, its memory safety guarantees, and its ability to stop us from shooting our own feet with buffer overflows. But here’s the thing about Rust: it can stop memory corruption, but it can't fix bad math.

Meet the rsa crate. It's the de facto standard for pure Rust RSA implementations, widely used in everything from JWT signing services to TLS terminators. It handles the heavy lifting of big integer arithmetic so developers don't have to.

However, in early 2026, a vulnerability surfaced that proved even the safest languages yield to basic logic flaws. It turns out that if you politely ask the library to create a private key using the number 1 as a prime factor, it doesn't just reject you—it accepts the input, tries to do the impossible, and promptly dies in a panic. It’s a classic Denial of Service, delivered via a single malformed integer.

To understand this bug, we have to look at the math of RSA. An RSA private key consists of a modulus $n$ which is the product of two (or more) large prime numbers, usually denoted as $p$ and $q$. Crucially, for the math to work, these numbers must actually be prime. By definition, a prime number must be greater than 1.

When the rsa crate constructs a private key from components (RsaPrivateKey::from_components), it performs a sanity check on the provided primes. It iterates through the list of primes and checks if they are valid. This is where the logic took a nap.

The code checked if a prime was strictly less than 1. In the world of BigUint (unsigned big integers), the only integer strictly less than 1 is 0. So, if you passed 0, the library caught it. But if you passed 1, the check 1 < 1 evaluated to false. The validation passed, and the library happily accepted 1 as a "prime" number.

Why is this fatal? Later in the RSA lifecycle—specifically during the calculation of Euler's totient or CRT parameters—the code needs to perform operations involving $(p - 1)$. If $p = 1$, then $(p - 1) = 0$. In modular arithmetic, calculating an inverse modulo 0 or dividing by 0 is undefined behavior mathematically, and in Rust, it results in an immediate thread panic.

The vulnerability lived in src/key.rs. It’s a textbook example of an off-by-one error, but applied to cryptographic primitives rather than array indices.

Here is the vulnerable code before the patch:

// src/key.rs (Vulnerable)
for prime in &self.primes {
    // logic error: only checks for 0
    if *prime < BigUint::one() {
        return Err(Error::InvalidPrime);
    }
    m *= prime;
}

And here is the fix in commit 2926c91:

// src/key.rs (Fixed)
for prime in &self.primes {
    // fixed: checks for 0 AND 1
    if *prime <= BigUint::one() {
        return Err(Error::InvalidPrime);
    }
    m *= prime;
}

The fix is a single character change: adding an equals sign (<=). This ensures that 1 is treated as an invalid prime, returning a proper Error::InvalidPrime result rather than letting the code proceed to the execution phase where the panic occurs.

Exploiting this is trivial if you can feed a private key to a target application. Imagine a "Bring Your Own Key" (BYOK) scenario where a user uploads a PEM file or raw key components to configure a signing service.

An attacker constructs a malformed key where $p=1$ and $q$ is a valid prime. The application parses this, calls RsaPrivateKey::from_components, and since the validation check fails to catch the 1, the object is created. However, the crate often pre-computes optimization values (like CRT exponents) immediately or upon first use.

When those computations hit the underlying num-bigint-dig library, they attempt to calculate the modular inverse of e modulo p-1. Since p-1 is 0, the underlying math library hits a hardware or software divide-by-zero trigger and panics.

In Rust, a panic unwinds the stack. If the application handles requests in the main thread or doesn't use catch_unwind boundaries (which is common in async runtimes for fatal errors), the entire process crashes. The service goes dark.

The impact here is purely Denial of Service (DoS). There is no memory corruption allowing for Remote Code Execution (RCE), and there is no leakage of private key material (Confidentiality).

However, don't underestimate DoS. If this library is used in an authentication gateway that processes client certificates or JWT signing keys, a single malicious packet could take down the auth service for everyone. If the service automatically restarts, the attacker can keep it in a boot loop.

The CVSS score is 5.3 (Medium), reflecting the lack of integrity/confidentiality impact, but for a high-availability system, this is a critical stability flaw.

The remediation is straightforward: Update your dependencies.

Run cargo update -p rsa to ensure you are pulling version 0.9.7 or later. If you are pinning versions in Cargo.toml, bump the version number manually.

[!TIP] This is a reminder to always validate cryptographic inputs at the application boundary, even if you trust your libraries. If you are accepting raw RSA components, ensure $p$ and $q$ are roughly the same bit length and definitely larger than 1.