GHSA-chgr-c6px-7xpp: Thread-Safety Data Race in PyO3 Closure Wrapping

PyO3 is a widely adopted Rust library that provides safe bindings for the Python interpreter, allowing developers to write high-performance native Python extensions in Rust. A central component of PyO3 is the bridging layer that permits Rust functions and closures to be registered as native Python callables. When a closure is registered with Python via the PyCFunction::new_closure or PyCFunction::new_closure_bound interfaces, it exposes an attack surface where execution control transitions from the dynamic Python runtime into compiled Rust code. Under standard usage, PyO3 relies on Rust's strong compile-time type invariants to ensure that these transitions are safe and free from memory hazards.

However, a thread-safety flaw exists in versions prior to 0.29.0 due to a missing Sync trait constraint on the generic closure parameter passed to PyCFunction::new_closure and PyCFunction::new_closure_bound. This omission corresponds to the weakness class defined as CWE-362 (Concurrent Execution using Shared Resource with Improper Synchronization). Because the PyO3 API failed to enforce this safety boundary, the Rust compiler permitted the compilation of code that wraps thread-unsafe Rust closures into Python callables. Consequently, when a multithreaded Python runtime executes these callables concurrently, the underlying Rust closures are run without necessary synchronization, leading to concurrent shared mutation.

The impact of this vulnerability ranges from localized data corruption to potential arbitrary code execution. When thread-unsafe types, such as Cell or RefCell, are concurrently mutated, the internal safety assumptions of these types are violated, resulting in undefined behavior. This issue is particularly critical in environments using the free-threaded Python configuration (PEP 703 / "nogil") which does not serialize execution through a Global Interpreter Lock. However, the flaw is also exploitable on standard GIL-protected Python runtimes if the closure releases the GIL during its execution path, allowing multiple threads to interleave execution within the same closure state.

To analyze the root cause of this vulnerability, one must examine the interface between Rust's concurrency model and Python's multi-threaded execution environment. Rust enforces thread safety at compile time using the built-in Send and Sync marker traits. A type T is Send if ownership of its data can be safely transferred to a different thread. A type T is Sync if references to the data (&T) can be safely shared and accessed concurrently across multiple threads. A closure in Rust automatically implements these traits depending on the types of the variables it captures from its surrounding scope; it is Sync only if all captured variables are also Sync.

In the vulnerable versions of PyO3, the signature of PyCFunction::new_closure only required the closure type F to implement Send + 'static. The omission of the Sync bound meant that PyO3 assumed a closure that was merely Send could be safely wrapped in a Python-callable object. This assumption is structurally flawed. A PyCFunction represents a persistent object in the Python heap that can be referenced and invoked by multiple Python threads simultaneously. When Python executes a callable, it calls the internal Rust callback wrapper, which executes the closure via its Fn::call implementation using a shared reference (&self). Therefore, the closure is subjected to concurrent access from multiple threads, necessitating a Sync constraint.

By omitting the Sync bound, PyO3 allowed developer code to pass closures capturing !Sync types like std::cell::Cell or std::cell::RefCell to Python. These types are explicitly designed for single-threaded "interior mutability" and do not contain any atomic operations or locking primitives to serialize concurrent updates. Under a concurrent execution pattern, multiple threads read and write to the same memory address without coordination. In the case of RefCell, concurrent modification bypasses the dynamic borrow checker's state tracking, allowing simultaneous mutable and immutable borrows, which is a direct violation of Rust's fundamental aliasing invariants.

The severity of this flaw is compounded by the evolution of the Python interpreter. Under standard GIL-enabled Python configurations, the Global Interpreter Lock prevents multiple Python bytecode instructions from executing in parallel on separate OS threads. However, if the Rust closure invokes Python::allow_threads or any external blocking operation, the GIL is released. This action permits the Python scheduler to run other threads, which can then re-enter the same closure. Under the newer free-threaded Python variant (PEP 703), the GIL is completely removed, meaning that standard Python multi-threading results in actual parallel execution of the closure on multiple CPU cores, triggering immediate hardware-level data races.

The vulnerable implementation of new_closure in PyO3 prior to version 0.29.0 is defined as follows:

// Vulnerable implementation in PyO3 < 0.29.0
pub fn new_closure<'py, F, R>(
    py: Python<'py>,
    name: Option<&'static CStr>,
    doc: Option<&'static CStr>,
    closure: F,
) -> PyResult<Bound<'py, Self>>
where
    // The closure F is bound by Send but lacks Sync
    F: Fn(&Bound<'_, PyTuple>, Option<&Bound<'_, PyDict>>) -> R + Send + 'static,
    for<'p> R: crate::impl_::callback::IntoPyCallbackOutput<'p, *mut ffi::PyObject>,
{
    let name = name.unwrap_or(c"pyo3-closure");
    // ... internal closure registration ...
}

In the implementation shown above, the trait bounds on the generic type parameter F specify Send + 'static. Because Sync is not required, the Rust compiler permits a closure that captures a RefCell to satisfy these bounds. When this function is called, PyO3 allocates the closure on the heap and creates a Python C-extension function object whose wrapper function executes the closure.

The patch introduced in PyO3 version 0.29.0 corrects this omission by appending the Sync trait to the bounds of the closure F:

// Patched implementation in PyO3 >= 0.29.0
pub fn new_closure<'py, F, R>(
    py: Python<'py>,
    name: Option<&'static CStr>,
    doc: Option<&'static CStr>,
    closure: F,
) -> PyResult<Bound<'py, Self>>
where
    // The closure F must now implement both Send and Sync
    F: Fn(&Bound<'_, PyTuple>, Option<&Bound<'_, PyDict>>) -> R + Send + Sync + 'static,
    for<'p> R: crate::impl_::callback::IntoPyCallbackOutput<'p, *mut ffi::PyObject>,
{
    let name = name.unwrap_or(c"pyo3-closure");
    // ... internal closure registration ...
}

By adding the Sync bound, the Rust compiler will statically reject any attempt to pass a closure that captures a non-Sync type. For example, if a closure captures a std::cell::Cell<i32>, the compiler detects that Cell<i32> does not implement Sync, propagates this failure to the closure type itself, and raises a compile-time error. This compile-time check prevents the vulnerability from being introduced into production binaries.

Additionally, the test suite of PyO3 itself had to be refactored because it contained a test (test_closure_counter) that captured a RefCell inside a closure, thus demonstrating the unsoundness. The patch replaced the RefCell with an atomic type:

// Vulnerable test code in PyO3 test suite:
let counter = std::cell::RefCell::new(0);
let counter_fn = move |_args: &Bound<'_, types::PyTuple>,
                       _kwargs: Option<&Bound<'_, types::PyDict>>|
      -> PyResult<i32> {
    let mut counter = counter.borrow_mut();
    *counter += 1;
    Ok(*counter)
};
 
// Patched test code using thread-safe AtomicI32:
let counter = AtomicI32::new(0);
let counter_fn = move |_args: &Bound<'_, types::PyTuple>,
                       _kwargs: Option<&Bound<'_, types::PyDict>>|
      -> PyResult<i32> {
    let prev_count = counter.fetch_add(1, Ordering::SeqCst);
    Ok(prev_count + 1)
};

This code modification highlights the structural difference between thread-unsafe interior mutability and thread-safe atomic state transitions. The atomic approach guarantees serialized access, preventing the data race.

Exploitation of this vulnerability requires an application architecture where a PyO3-wrapped closure is exposed to concurrent execution paths. A typical scenario involves a multi-threaded web application or an asynchronous processing pipeline where user-controlled inputs or concurrent tasks trigger the execution of Python callbacks. The prerequisite for the exploit is that the target Rust library or application has been compiled with PyO3 versions prior to 0.29.0 and contains a closure that captures a thread-unsafe !Sync state.

To trigger the vulnerability, an attacker must generate high-concurrency requests or tasks that target the specific API endpoint or callback registration. As multiple worker threads process the incoming tasks, they simultaneously execute the Python callback. If the application runs under standard Python (with the GIL), the exploit must rely on the closure releasing the GIL (for instance, via a database query, network request, or an explicit call to Python::allow_threads within the closure or adjacent code). This release allows the scheduler to context-switch to another thread, which then enters the same closure, resulting in interleaved access to the non-Sync captured memory.

In a free-threaded Python environment, the exploitation process is more direct. Because there is no GIL, the worker threads execute the underlying Rust closure in parallel across multiple physical CPU cores. This concurrent execution immediately leads to memory corruption as both threads write to the same memory region without locking. In the case of RefCell, concurrent modification disrupts the internal reference counter tracking mutable and immutable borrows. This disruption causes the borrow tracker to allow multiple mutable references to the underlying data, violating Rust's aliasing rules and leading to a use-after-free or double-free condition.

The consequence of a successful exploitation attempt is undefined behavior at the CPU level. Depending on the memory layout and the specific data structures captured by the closure, an attacker can manipulate the race window to corrupt adjacent heap structures, overwrite function pointers inside objects, or corrupt virtual tables. This memory manipulation can be leveraged to hijack control flow, escalate privileges within the application process, or cause a segmentation fault resulting in a denial-of-service condition.

The concrete security impact of this vulnerability is severe, as it bypasses the core safety guarantees of the Rust language. Rust's primary value proposition is the elimination of data races and memory safety issues at compile time without the overhead of a garbage collector. This vulnerability undermines that model, introducing runtime memory hazards into code that developers compiled under the assumption of absolute safety. Because the compiler did not emit any warnings, developers had no indication that their concurrent state modifications were structurally unsafe.

From an impact standpoint, the vulnerability can result in arbitrary code execution or local privilege escalation. When an attacker corrupts the state of captured variables, they can overwrite pointers, modify bounds check values, or trigger use-after-free conditions on the heap. If the application process runs with elevated privileges, a successful exploit could allow an attacker to escape any application-level sandboxes and execute code directly on the host operating system. Alternatively, the continuous corruption of the internal reference state will inevitably lead to application crashes (segmentation faults), causing a persistent denial of service.

The vulnerability is rated as Medium to High depending on the deployment environment. In modern cloud infrastructures deploying PEP 703 (free-threaded Python) to achieve maximum performance, the exploitability increases substantially because execution serialization is entirely removed. Although the CVSS score is categorized as moderate due to the high complexity required to reliably exploit memory race conditions, the absolute hazard to application integrity and availability remains a significant concern for security teams managing high-throughput services.

The primary remediation strategy is to upgrade the pyo3 dependency to version 0.29.0 or higher. This version updates the type signatures of new_closure and new_closure_bound to require the Sync trait on the closure type parameter. When building the application with the updated dependency, the Rust compiler will automatically audit the codebase, identifying and rejecting any closures that capture non-thread-safe variables. This static enforcement eliminates the vulnerability at compile time, ensuring that no unsound code can be deployed.

For development teams unable to immediately upgrade PyO3 due to dependency constraints or API compatibility issues, a manual audit of all closure registrations is necessary. Developers must review every instance of PyCFunction::new_closure and inspect the captured variables. If any closure captures a Cell, RefCell, Rc, or other !Sync container, these types must be replaced with thread-safe alternatives. Specifically, RefCell should be replaced with Mutex or RwLock, Cell should be replaced with atomic primitives (such as AtomicI32), and Rc must be replaced with Arc.

The fix implemented in PyO3 version 0.29.0 is fully complete and structurally robust. By integrating the constraint into the type signature as a trait bound, the library delegates the enforcement of thread safety back to the Rust compiler. Because the compiler performs this verification statically, there are no runtime performance overheads or potential logical bypasses in the fix. Variant attacks targeting the same code path are impossible because the compiler guarantees that any closure passed to Python can be safely shared across threads.