CVE-2026-34544: Signed Integer Overflow and Out-of-Bounds Write in OpenEXRCore B44/B44A Compression

OpenEXR is a high dynamic range (HDR) image file format developed by the Academy Software Foundation. It is widely used in computer imaging, rendering, and visual effects pipelines. The library introduces an OpenEXRCore C library in version 3, which handles fundamental encoding and decoding operations.

CVE-2026-34544 identifies a vulnerability in this core library, specifically within the B44 and B44A compression modules. These compression formats are utilized to achieve lossy compression for 16-bit float data. The vulnerability is classified primarily as CWE-190 (Integer Overflow or Wraparound), which directly cascades into CWE-787 (Out-of-bounds Write).

When an application parses an untrusted EXR image file, the underlying OpenEXRCore library processes the pixel data. An attacker can construct an image file with dimensions explicitly designed to trigger a mathematical overflow during internal memory calculations. This results in memory corruption that destabilizes the host application.

The root cause lies in how the OpenEXRCore library calculates memory offsets for row pointers within an internal decoding buffer. The vulnerability resides in src/lib/OpenEXRCore/internal_b44.c and affects both the uncompress_b44_impl and compress_b44_impl functions. During compression or decompression, the library iterates over the rows of an image.

The variables nx (representing the image width) and y (representing the current row index) are defined as 32-bit signed integers. To locate the specific row within an allocated scratch buffer, the code multiplies y by nx and adds the result to a base pointer. Because both operands are 32-bit signed integers, the multiplication operation is constrained by INT_MAX (2,147,483,647).

If an attacker supplies an EXR file with excessively large dimensions, such as a width nx of 268,435,456 and a row index y of 8, the product y * nx equals 2,147,483,648. This value exceeds INT_MAX and wraps around to a negative 32-bit integer. When this negative offset is applied to the base pointer, the application generates a pointer referencing an address outside the boundaries of the scratch buffer.

Subsequent write operations using these corrupted pointers write data to unintended memory locations. This behavior constitutes an out-of-bounds write, enabling an attacker to manipulate adjacent heap allocations depending on the current process layout.

The vulnerability manifests in the direct manipulation of pointers using unsafe 32-bit integer arithmetic. The affected code defines multiple row pointers that map directly to the scratch buffer memory.

uint16_t *row0, *row1, *row2, *row3;
row0 = (uint16_t*) scratch;
row0 += y * nx;

In this vulnerable implementation, the expression y * nx is evaluated as a 32-bit signed integer prior to being added to the row0 pointer. The C standard dictates that signed integer overflow results in undefined behavior. In practice, most compilers will generate code that wraps the value negatively, causing row0 to point to a memory address preceding the actual scratch buffer allocation.

The official patch addresses this by explicitly casting the operands to 64-bit unsigned integers before performing the multiplication. This ensures the resulting offset accurately reflects the required memory boundary without overflowing.

/* Corrected calculation */
uint64_t row_off = (uint64_t)(y) * (uint64_t)(nx);
row0 = (uint16_t*) scratch + row_off;

By enforcing 64-bit arithmetic via uint64_t, the maximum possible value of y * nx is safely contained within the data type limits. The pointer arithmetic then operates correctly, advancing the row0 pointer to the exact required address within the boundaries of the scratch buffer. This remediation eliminates the out-of-bounds write primitive entirely.

Exploiting CVE-2026-34544 requires an attacker to deliver a maliciously crafted EXR file to a vulnerable application. The target must utilize the OpenEXR library versions 3.4.0 through 3.4.7. The vulnerability requires local access or user interaction, reflecting the AV:L and UI:A components of the CVSS vector.

The attacker must modify the EXR header to specify B44 or B44A compression. They must also define image dimensions that are large enough to force the y * nx calculation to exceed INT_MAX. The attacker then provides this file to the victim application, which initiates the exr_decoding_run() function upon loading the file.

When the application processes the file, the internal uncompress_b44_impl function triggers the out-of-bounds write. The attacker does not need to bypass authentication to trigger this vulnerability. Any pipeline that automatically processes user-uploaded EXR files is heavily exposed to this attack vector.

The primary and most reliable impact of this vulnerability is a denial of service (DoS). When the application writes data to an invalid memory location via the corrupted pointer, the operating system typically intercepts the memory access violation. This generates a segmentation fault, crashing the host process immediately.

Beyond denial of service, the out-of-bounds write condition introduces the risk of heap corruption. If the attacker can predict or manipulate the heap memory layout, they may force the application to overwrite critical adjacent memory structures. These structures could include function pointers, object vtables, or application state variables.

Successful exploitation of the heap corruption could lead to arbitrary code execution. Code execution would occur within the context of the user or service running the vulnerable application. The CVSS v4.0 score of 8.4 reflects the high severity of the confidentiality, integrity, and availability impacts should successful code execution be achieved.

The vulnerability is fundamentally resolved in OpenEXR version 3.4.8. Organizations must update their dependencies to this patched version. The fix is implemented in commit 35e7aa35e22c1975606be86e859f31cc1fc598ee by the Academy Software Foundation.

If upgrading the OpenEXR library is not immediately feasible, developers can implement defensive workarounds within their applications. The application should parse the EXR header and independently validate the image dimensions before passing the file to the OpenEXR decoder. This validation must verify that the product of the width and height cannot exceed the limits of a 32-bit integer.

Security teams should also review their threat models regarding automated image processing pipelines. Running such processing within isolated sandboxes or unprivileged containers reduces the potential impact of a successful exploit. These defense-in-depth measures mitigate the risk of arbitrary code execution stemming from memory corruption vulnerabilities.