CVE-2022-46292: Heap-Based Out-of-Bounds Write in Open Babel MOPAC Parser

Open Babel is an open-source chemical toolbox designed for the translation, search, analysis, and storage of molecular modeling data. The tool parses a variety of solid-state and crystallographic formats, making it a critical component in scientific data pipelines. The software handles crystal lattice structures by storing their spatial translation vectors, which represent periodic boundary conditions.

The vulnerability resides in the parsing of translation vectors inside the MOPAC output file parser. This component is responsible for processing scientific output files generated by the MOPAC (Molecular Orbital Package) semi-empirical quantum chemistry program. The file parser exposes an attack surface when processing untrusted molecular data.

Specifically, the vulnerability belongs to the heap-based out-of-bounds write class (CWE-787). An attacker can trigger this flaw by submitting a malformed MOPAC output file containing an excessive number of translation vectors. The parsing process fails to restrict the written indexes, leading to memory corruption and potential system takeover.

In crystallographic modeling, three-dimensional unit cells are defined by a maximum of three lattice translation vectors, traditionally labeled as a, b, and c. To store these properties, the developers of Open Babel defined a static array of three vector3 structures, named translationVectors. The state of the parsing process tracks the number of currently processed vectors through the integer counter numTranslationVectors.

The parser in src/formats/mopacformat.cpp processes lines sequentially within the UNIT CELL TRANSLATION block. When it encounters a row designated as a translation vector, it parses the three-dimensional floating-point coordinates and assigns them to the index pointed to by the numTranslationVectors counter. After assignment, the counter is incremented.

Because the parser does not implement a check on the value of numTranslationVectors before executing the write operation, it can increment the index beyond the bounds of the array. If the input file declares four or more translation vectors, the assignment targets index positions greater than or equal to 3. This leads to an out-of-bounds write into adjacent heap-allocated variables or internal object structures.

The vulnerable version of the code in MOPACFormat::ReadMolecule lacks any validation check on the vector index before writing. The program reads coordinate strings, converts them using the standard library atof function, and directly modifies the array structure:

x = atof((char*)vs[2].c_str());
y = atof((char*)vs[3].c_str());
z = atof((char*)vs[4].c_str());
 
// Vulnerable write operation without index boundary validation
translationVectors[numTranslationVectors++].Set(x, y, z);

The fix committed in patch 40e852138f21d586b7ccdce6329e7b23a87168bb introduces an explicit conditional check that prevents memory modification beyond the allocated boundary. The comparison limits the execution of the storage function to cases where the index is strictly less than 3:

x = atof((char*)vs[2].c_str());
y = atof((char*)vs[3].c_str());
z = atof((char*)vs[4].c_str());
 
// Patched logic incorporating index safety validation
if (numTranslationVectors < 3)
  translationVectors[numTranslationVectors++].Set(x, y, z);

Although the patch resolves the primary heap-based write issue, the parsing mechanism still displays vulnerabilities to out-of-bounds vector reads if the input line is malformed. If the tokenized vector vs contains fewer than five elements, accessing indices 2, 3, and 4 causes undefined behavior or a null-pointer dereference inside std::vector functions. Security teams must ensure that input files are validated for structural integrity prior to passing them to the parser logic.

To exploit this vulnerability, an attacker must construct a malformed MOPAC output file (.out) containing a UNIT CELL TRANSLATION section. The attacker appends four or more vector declarations within this block, exceeding the physical constraints of a three-dimensional model. This forces the parser to write coordinate values beyond the boundaries of the statically allocated three-element array.

The exploitation flow depends on the memory layout of the target system. Because the object holding translationVectors resides on the heap, successive writes overwrite contiguous heap memory fields. These fields can include virtual table pointers, object pointers, or structural parsing metadata associated with the active OBFormat instance.

By carefully tuning the coordinate floating-point values, an attacker can overwrite these pointers with shellcode addresses or addresses of existing system functions. The attack requires user interaction or an automated backend processing pipeline to open the file. For example, a web application that offers online chemical format conversions could trigger this vulnerability when processing an uploaded file, executing the payload in the context of the daemon process.

The security consequences of CVE-2022-46292 depend on the environment where the Open Babel library is deployed. When integrated into web-based file-conversion servers, cloud computing clusters, or scientific repositories, the vulnerability allows remote code execution without authentication. The attack vector is classified as network-based if the application processes uploads automatically.

If the library is executed as a command-line tool on a workstation, the vulnerability behaves as a local exploit requiring user interaction. The victim must open the malicious file locally, leading to code execution with the permissions of the local user account. If the local user possesses elevated permissions, the entire workstation may be compromised.

The severity of this flaw is reflected in its CVSS score of 9.8 from the NVD and 7.8 from the OSV database. The vulnerability results in a total loss of confidentiality, integrity, and availability. Because Open Babel does not sandbox file parsing by default, a compromise allows arbitrary file access, local network probing, or remote shell spawning.

The primary remediation strategy is upgrading Open Babel to a version that includes the safety check in the translation vector parsing loop. Users who compile the application from source must apply the fix from commit 40e852138f21d586b7ccdce6329e7b23a87168bb or fetch the latest master branch. Pre-compiled binaries distributed by operating systems must be updated via their respective package managers.

When immediate updates are not possible, administrators can implement workarounds to reduce exposure. Since the vulnerability is triggered by parsing specific crystallographic headers, automated ingress validation can scan incoming files. Deploying the YARA rule provided in this report on file upload endpoints can block malformed files before they reach the parser.

Additionally, organizations should run Open Babel inside restricted, low-privilege execution containers. Restricting network access for the conversion process reduces the risk of reverse shell payloads executing successfully. Implementing standard memory-safety hardening mechanisms, such as address space layout randomization (ASLR) and data execution prevention (DEP), raises the barrier for reliable exploitation.