GHSA-MF7Q-R4RV-JV94: Time-of-Check to Time-of-Use (TOCTOU) Signature Verification Bypass in Crossplane Runtime

Crossplane relies on OCI (Open Container Initiative) registries to host, distribute, and manage packages (Providers and Configurations) that extend control plane capabilities. To ensure supply chain security, administrators configure signature verification using Cosign via Crossplane's ImageConfig architecture. This control ensures that only packages cryptographically signed by authorized keys are loaded and executed within the Kubernetes cluster.

However, the implementation of this verification pipeline within xpkg.CachedClient introduces a critical flaw. The underlying package manager evaluates dynamic OCI tag references (such as :v1.2.3 or :latest) sequentially over separate HTTP network requests. Because these steps are decoupled and depend on a remote, untrusted server to resolve the same mutable tag multiple times, the mechanism is exposed to a classic Time-of-Check to Time-of-Use (TOCTOU) race condition.

This flaw allows a malicious or compromised OCI registry to undermine the trust model. By manipulating the resolution of a tag pointer during the brief window between signature verification and package extraction, an attacker can trick Crossplane into installing arbitrary, unsigned code while registering the execution as verified and trusted.

The root cause of GHSA-MF7Q-R4RV-JV94 is the failure to enforce content immutability across the verification and retrieval boundaries. In a secure OCI distribution workflow, a dynamic reference tag must be resolved to a static cryptographic digest (the SHA-256 hash) immediately. All subsequent operations, such as signature verification, metadata analysis, and layer retrieval, must execute against this immutable digest.

Historically, the Crossplane package manager client performed these steps in isolation. When a user submitted a package manifest reference, Crossplane queried the registry to find the signature mapping associated with the human-readable tag. It verified this signature payload. It did not, however, lock the underlying package retrieval phase to the exact digest returned by the first query.

This structural design allows a malicious OCI registry to track state across inbound API requests. The registry answers the signature payload request with verified data linked to a clean, signed image. When the immediate subsequent request arrives seeking the image layers for installation, the registry dynamically updates its pointer, mapping the same tag reference to a completely different, unsigned, and malicious image digest. The package manager, still executing in the context of the same logical task, extracts the unsigned container layers without detecting the switch.

Before the remediation patch, the package retrieval and verification phases in the package client relied on separate calls to the container registry library. The xpkg.CachedClient structured its fetch loop conceptually as follows:

// Conceptual representation of the vulnerable xpkg.CachedClient design
func (c *CachedClient) PullAndVerify(ctx context.Context, ref name.Reference) (v1.Layer, error) {
    // Step 1: The 'Check' Phase
    // The registry resolves the dynamic tag 'ref' to fetch the signature
    desc, err := remote.Head(ref, c.options...)
    if err != nil {
        return nil, err
    }
    if err := c.verifyCosignSignature(ctx, ref); err != nil {
        return nil, fmt.Errorf("signature verification failed: %w", err)
    }
 
    // Step 2: The 'Use' Phase
    // The registry resolves the dynamic tag 'ref' AGAIN during the actual fetch call
    img, err := remote.Image(ref, c.options...)
    if err != nil {
        return nil, err
    }
    return img.Layers()
}

The vulnerability is fixed by reorganizing the flow to perform a single, authoritative resolution of the tag to its immutable digest before any operations begin. Once the digest is acquired, all subsequent steps use this resolved digest rather than the dynamic tag.

// Conceptual representation of the patched and secure implementation
func (c *CachedClient) PullAndVerifyPatched(ctx context.Context, ref name.Reference) (v1.Layer, error) {
    // Step 1: Resolve tag to an immutable digest immediately
    desc, err := remote.Head(ref, c.options...)
    if err != nil {
        return nil, err
    }
 
    // Build an immutable reference mapping using the absolute SHA-256 digest
    digestRef, err := name.NewDigest(ref.Context().Name() + "@" + desc.Digest.String())
    if err != nil {
        return nil, err
    }
 
    // Step 2: Run verification against the immutable digest
    if err := c.verifyCosignSignature(ctx, digestRef); err != nil {
        return nil, fmt.Errorf("signature verification failed: %w", err)
    }
 
    // Step 3: Fetch the image using the same immutable digest
    img, err := remote.Image(digestRef, c.options...)
    if err != nil {
        return nil, err
    }
    return img.Layers()
}

By forcing the use of digestRef instead of ref in both the verification and fetch functions, the client eliminates the second registry query for the tag. If the remote registry tries to serve a different image during the fetch step, the request fails because the client specifically requests the content by its SHA-256 digest hash rather than its symbolic tag pointer.

To exploit this vulnerability, an attacker must have administrative control over an OCI registry (or manipulate an upstream registry) and convince a victim to reference a package hosted on it. The exploitation pipeline proceeds systematically:

First, the attacker crafts an OCI registry server with dynamic request routing. The registry maintains a state machine for incoming HTTP connections based on client IP addresses and user-agent strings. When a target initiates a package pull request, the registry detects the initial requests seeking signature layers (e.g., sha256-<digest>.sig). It serves the metadata and signature payload for a benign, fully compliant, and signed package, satisfying the local Crossplane Cosign validation checks.

Second, the registry tracks the request lifecycle. Immediately after delivering the signature manifest, the registry's tag resolution engine modifies the mapping for the targeted tag. When the client executes the follow-up HTTP GET request for the container's image manifest config or layers, the registry redirects the request to point to an unsigned, malicious package containing custom controllers or resources.

Finally, the target cluster extracts and installs the dynamic payload. Because the client believes the signature step succeeded for the requested logical unit, the malicious resources are registered directly into the Kubernetes cluster as trusted extensions.

The execution of unauthorized packages in a Kubernetes cluster represents a critical control bypass. Crossplane Providers typically run with highly privileged service accounts to manage cloud infrastructure, meaning a compromised package can result in full control-plane takeover or unauthorized cloud resource creation.

The CVSS v4.0 score is calculated at 8.2, reflecting a high-integrity impact with low attack complexity. Because this vulnerability bypasses supply chain protection controls, organizations that rely on code signing to permit third-party packages are exposed to silent, unauthenticated arbitrary code execution if they pull packages from external registries.

Fortunately, because the vulnerability requires active collaboration with or control over the host registry, exploitation is limited to scenarios where victims are tricked into referencing packages on untrusted registries, or where an upstream registry has been compromised. There are no known instances of this vulnerability being exploited in the wild, and it is not listed in CISA's Known Exploited Vulnerabilities catalog.

The permanent remediation is to upgrade the crossplane-runtime dependency and the Core Crossplane installation to the patched releases. The maintenance teams have backported the single-resolution fix to all active stable releases.

If you cannot apply the patch immediately, you can remediate this flaw entirely by modifying your package deployment manifests. You must reference packages exclusively by their cryptographic digest (@sha256:...) instead of dynamic version tags (:v1.0.0). Because digest references are inherently immutable, they force the client libraries to request identical data during both the verification and fetch steps, removing the TOCTOU window entirely.

Additionally, security teams should implement Kubernetes admission controllers (such as OPA Gatekeeper or Kyverno) to enforce policies that block Provider or Configuration specifications that use tag-based references, requiring a strict SHA-256 digest format for all packages.