CVE-2026-118759: Cross-Tenant Pod Log Exposure via Authorization Bypass in Kubeflow Pipelines

Kubeflow Pipelines (KFP) is a platform for building and deploying portable, scalable machine learning workflows on Kubernetes. The KFP backend API server (ml-pipeline) exposes both gRPC and RESTful HTTP interfaces to manage pipeline definitions, experiments, runs, and execution artifacts. In multi-user deployments, pipeline runs are isolated within specific Kubernetes namespaces assigned to individual users or organizational tenants, with access governed by Kubernetes Role-Based Access Control (RBAC) and SubjectAccessReview (SAR) checks.

CVE-2026-118759 (GHSA-wg5q-x4j9-qm54) is a Missing Authorization (CWE-862) and Exposure of Sensitive Information (CWE-200) vulnerability in the KFP API server affecting all versions prior to 2.18.0. Specifically, the HTTP endpoint responsible for streaming execution logs from pipeline workflow steps—GET /apis/v1alpha1/runs/{run_id}/nodes/{node_id}/log—retrieves container logs without authenticating or authorizing the caller against the target run.

In addition to missing caller authorization, the endpoint acts as a confused deputy when querying the Kubernetes API for live container logs. The server uses the target run_id solely to resolve the target Kubernetes namespace, and then passes the caller-controlled node_id path parameter directly to the Kubernetes core client as the target pod name. Because the server does not verify whether the specified pod was actually created by the given pipeline run, an attacker who knows any valid run_id in a victim namespace can read the standard output and standard error streams of the main container of any arbitrary pod within that namespace.

The vulnerability stems from two distinct architectural and implementation flaws in how the KFP API server handles HTTP log streaming requests compared to standard gRPC-backed REST endpoints.

First, most RESTful endpoints in the KFP API server are generated via grpc-gateway, a reverse-proxy layer that translates incoming HTTP requests into gRPC requests. During this translation, grpc-gateway automatically extracts HTTP authentication and identity headers (such as Google Identity-Aware Proxy headers, bearer tokens, or Istio authentication headers) and populates them into the incoming gRPC metadata.MD context. Downstream service methods then extract this metadata from the context.Context object to perform Kubernetes SubjectAccessReview evaluations.

However, to support chunked HTTP log streaming directly to clients, the ReadRunLogV1 handler in backend/src/apiserver/server/run_log_server.go was registered directly onto the underlying Gorilla mux HTTP router, bypassing grpc-gateway entirely. As a consequence, r.Context() inside ReadRunLogV1 contained an empty gRPC metadata structure. Furthermore, RunLogServer did not invoke canAccessRun or any equivalent authorization check before processing the request; instead, it immediately wrote an HTTP 200 OK status header (w.WriteHeader(http.StatusOK)) and initiated log retrieval.

Second, a secondary logic flaw existed in the RBAC verb model for multi-user deployments configured with shared read access (MultiUserModeSharedReadAccess). When shared read access was enabled, the authorization engine automatically approved standard get and list verbs prior to issuing a Kubernetes SubjectAccessReview. Because log retrieval did not define a dedicated RBAC verb distinct from get, restricting log access independently of basic run metadata was not supported by the policy model.

Third, inside backend/src/apiserver/resource/resource_manager.go, the ReadLog and readRunLogFromPod functions failed to validate resource ownership. Once ReadLog looked up the runId in the database and extracted its Kubernetes namespace via getPodNamespace(run), it invoked r.k8sCoreClient.PodClient(namespace).GetLogs(nodeId, &logOptions) using the API server's own privileged Kubernetes ServiceAccount. The code assumed nodeId was a valid Argo workflow pod belonging to runId, without inspecting the pod's labels (pipelines.kubeflow.org/run_id) to confirm ownership.

In versions prior to 2.18.0, backend/src/apiserver/server/run_log_server.go implemented ReadRunLogV1 without embedding *BaseRunServer or validating caller permissions. Additionally, committing w.WriteHeader(http.StatusOK) prior to calling ReadLog meant that even if an internal error occurred, the HTTP status code had already been sent as 200 OK.

// Vulnerable implementation in backend/src/apiserver/server/run_log_server.go (pre-2.18.0)
func (s *RunLogServer) ReadRunLogV1(w http.ResponseWriter, r *http.Request) {
	vars := mux.Vars(r)
	runId, ok := vars[RunKey]
	if !ok {
		s.writeErrorToResponse(w, http.StatusBadRequest, util.NewInvalidInputError("%s not found in request", RunKey))
		return
	}
	nodeId, ok := vars[NodeKey]
	if !ok {
		s.writeErrorToResponse(w, http.StatusBadRequest, util.NewInvalidInputError("%s not found in request", NodeKey))
		return
	}
	follow := vars[Follow] == "true"
 
	// VULNERABILITY 1: No authentication or authorization check is performed.
	// Status 200 OK is committed immediately before attempting to read logs.
	w.WriteHeader(http.StatusOK)
	w.Header().Set("Content-Type", "text/plain")
	w.Header().Set("Cache-Control", "no-cache, private")
 
	flushWriter, err := util.NewFlushWriter(w)
	if err != nil {
		s.writeErrorToResponse(w, http.StatusInternalServerError, util.Wrap(err, "Failed to create flush writer."))
		return
	}
 
	err = s.resourceManager.ReadLog(r.Context(), runId, nodeId, follow, flushWriter)
	if err != nil {
		s.writeErrorToResponse(w, s.getErrorStatus(err), util.Wrap(err, "Failed to read a run log."))
	}
}

In backend/src/apiserver/resource/resource_manager.go, the pre-patch readRunLogFromPod method directly passed the user-supplied nodeId into the Kubernetes client's GetLogs call:

// Vulnerable implementation in backend/src/apiserver/resource/resource_manager.go (pre-2.18.0)
func (r *ResourceManager) readRunLogFromPod(ctx context.Context, namespace string, nodeId string, follow bool, dst io.Writer) error {
	logOptions := corev1.PodLogOptions{
		Container:  "main",
		Timestamps: false,
		Follow:     follow,
	}
	// VULNERABILITY 2: nodeId is used directly as the Pod name without verifying
	// whether the pod actually belongs to the requested runId.
	req := r.k8sCoreClient.PodClient(namespace).GetLogs(nodeId, &logOptions)
	podLogs, err := req.Stream(ctx)
	if err != nil {
		return util.NewInternalServerError(err, "Failed to get pod log stream")
	}
	defer podLogs.Close()
 
	return copyAndFlush(dst, podLogs)
}

Commit be4723e33727c6dad71a95cba61e8f61562ab20f (PR #13684) remediates both issues. First, it introduces a dedicated RbacResourceVerbReadLog = "readLog" constant in backend/src/apiserver/common/const.go and updates the multi-user cluster roles to grant readLog on pipelines.kubeflow.org/runs. Second, RunLogServer embeds *BaseRunServer and introduces an authorize helper that copies HTTP request headers into incoming gRPC metadata and evaluates canAccessRun with the readLog verb:

// Patched implementation in backend/src/apiserver/server/run_log_server.go
func (s *RunLogServer) ReadRunLogV1(w http.ResponseWriter, r *http.Request) {
	// ... [parameter extraction omitted] ...
	follow := vars[Follow] == "true"
 
	// FIX 1: Authorize request against the target run using the 'readLog' verb
	if err := s.authorize(r, runId); err != nil {
		s.writeErrorToResponse(w, http.StatusForbidden, util.Wrap(err, "Failed to authorize the request"))
		return
	}
 
	// FIX 2: Set headers without calling w.WriteHeader(http.StatusOK) prematurely,
	// allowing downstream errors in ReadLog to return proper non-200 HTTP status codes.
	w.Header().Set("Content-Type", "text/plain")
	w.Header().Set("Cache-Control", "no-cache, private")
 
	flushWriter, err := util.NewFlushWriter(w)
	// ...
}
 
func (s *RunLogServer) authorize(r *http.Request, runID string) error {
	md := metadata.MD{}
	for key, values := range r.Header {
		md.Set(key, values...)
	}
	ctx := metadata.NewIncomingContext(r.Context(), md)
	return s.canAccessRun(ctx, runID, &authorizationv1.ResourceAttributes{Verb: common.RbacResourceVerbReadLog})
}

Third, readRunLogFromPod in backend/src/apiserver/resource/resource_manager.go now fetches the pod object from the Kubernetes API first and verifies that its pipelines.kubeflow.org/run_id label (util.LabelKeyWorkflowRunId) matches the requested runID:

// Patched implementation in backend/src/apiserver/resource/resource_manager.go
func (r *ResourceManager) readRunLogFromPod(ctx context.Context, runID string, namespace string, nodeID string, follow bool, dst io.Writer) error {
	// FIX 3: Fetch pod metadata and verify the pod was created by this specific run
	pod, err := r.k8sCoreClient.PodClient(namespace).Get(ctx, nodeID, v1.GetOptions{})
	if err != nil {
		if !apierrors.IsNotFound(err) {
			glog.Errorf("Failed to get pod %v: %v", nodeID, err)
		}
		return util.NewInternalServerError(err, "Failed to read logs from pod %v due to error fetching the pod", nodeID)
	}
	if pod == nil || pod.Labels[util.LabelKeyWorkflowRunId] != runID {
		return util.NewInvalidInputError("Pod %v does not belong to run %v", nodeID, runID)
	}
 
	logOptions := corev1.PodLogOptions{
		Container:  "main",
		Timestamps: false,
		Follow:     follow,
	}
	req := r.k8sCoreClient.PodClient(namespace).GetLogs(nodeID, &logOptions)
	// ...
}

The fix is complete with respect to both route authorization and pod ownership validation. By explicitly binding readLog to SubjectAccessReview and enforcing exact equality on pod.Labels[util.LabelKeyWorkflowRunId] == runID, callers can neither bypass tenant isolation nor pivot to unrelated pods within an authorized namespace.

Exploiting CVE-2026-118759 requires network connectivity to the Kubeflow Pipelines API server (ml-pipeline service, typically listening on TCP port 8888 for HTTP) and knowledge of a valid run_id associated with the target namespace, along with a target pod/node identifier. No authentication credentials are required by the handler itself, though external API gateways or service mesh policies may require the attacker to be an authenticated cluster tenant.

In a multi-tenant Kubeflow deployment where MultiUserModeSharedReadAccess is enabled, any authenticated user can list runs across namespaces to obtain valid run_id UUIDs. Even when shared read access is disabled, an attacker may obtain a run_id through shared experiment metadata, webhook notifications, or CI/CD logs.

Scenario 1: Reading Pipeline Step Logs Cross-Tenant

If the attacker knows a victim run_id (e.g., 6b6858e2-f791-4e76-8041-55bb59582236) and the corresponding Argo workflow step pod name (which typically follows deterministic naming patterns based on the workflow name, or is retrieved from archived workflow manifests), the attacker issues a standard HTTP GET request:

GET /apis/v1alpha1/runs/6b6858e2-f791-4e76-8041-55bb59582236/nodes/training-pipeline-abcde-123456789/log?follow=true HTTP/1.1
Host: ml-pipeline.kubeflow.svc.cluster.local:8888

If the pod has already terminated and its logs were archived to object storage (such as MinIO or S3), ReadLog falls back to readRunLogFromArchive, retrieving the archived log file from the storage bucket using the API server's object storage credentials and streaming it to the caller.

Scenario 2: Arbitrary Pod Log Exfiltration via Confused Deputy

Because readRunLogFromPod only uses run_id to look up the namespace (e.g., tenant-finance) and passes node_id directly as the pod name to the Kubernetes API with Container: "main", an attacker can target non-pipeline pods running in tenant-finance provided they have a container named main:

GET /apis/v1alpha1/runs/6b6858e2-f791-4e76-8041-55bb59582236/nodes/credentials-sync-worker-0/log HTTP/1.1
Host: ml-pipeline.kubeflow.svc.cluster.local:8888

Upon receiving this request, the KFP API server uses its ml-pipeline-api-server Kubernetes ServiceAccount to execute GET /api/v1/namespaces/tenant-finance/pods/credentials-sync-worker-0/log?container=main against the Kubernetes API server and pipes the output back to the attacker.

CVE-2026-118759 is rated High severity with a CVSS v3.1 base score of 7.5 (CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N). The vulnerability impacts confidentiality exclusively (C:H), with no direct impact on resource integrity (I:N) or service availability (A:N).

In machine learning environments, container execution logs frequently contain high-value sensitive information:

• Credentials and Secrets: Database connection strings, cloud provider access keys, API bearer tokens, and internal service passwords printed during initialization or debugging.
• Proprietary Data and Intellectual Property: Training dataset samples, feature engineering parameters, hyperparameter configurations, and model evaluation metrics.
• Infrastructure Telemetry: Internal service endpoints, storage bucket paths, and debugging stack traces that facilitate lateral movement within the Kubernetes cluster.

Because the API server executes the log retrieval using its own cluster-wide ServiceAccount (ml-pipeline-api-server), the attacker bypasses Kubernetes namespace isolation boundaries. As of the latest threat intelligence assessment, CVE-2026-118759 is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog and holds an EPSS score of 0.00336 (27.01st percentile), with no publicly weaponized exploit modules tracked in ExploitDB or Metasploit.

Official Fix and Upgrade Path

Organizations running Kubeflow Pipelines should upgrade to version 2.18.0 or later, which incorporates commit be4723e33727c6dad71a95cba61e8f61562ab20f (PR #13684).

> [!IMPORTANT] > When upgrading multi-user installations of Kubeflow Pipelines, administrators must also apply the updated Kubernetes RBAC manifests (manifests/kustomize/base/installs/multi-user/view-edit-cluster-roles.yaml). Version 2.18.0 requires the readLog verb on the pipelines.kubeflow.org/runs resource; if the ClusterRole definitions (kubeflow-pipelines-edit and kubeflow-pipelines-view) are not updated, legitimate users will receive 403 Forbidden responses when attempting to view run logs.

Workarounds and Mitigations

If an immediate upgrade to 2.18.0 is not feasible, apply the following compensating controls:

1. Kubernetes NetworkPolicies: Restrict network ingress to the ml-pipeline service (ports 8888 and 8887) using a NetworkPolicy so that only authorized reverse proxies, Istio ingress gateways, or verified internal components can establish connections to the API server.
2. Service Mesh / API Gateway Route Filtering: In environments using Istio or an external API gateway, configure an AuthorizationPolicy or WAF rule to block or strictly authenticate requests matching the path pattern /apis/v1alpha1/runs/*/nodes/*/log.
3. Least-Privilege ServiceAccount Scoping: Ensure the ml-pipeline-api-server ServiceAccount is only granted pods/log read permissions in dedicated pipeline execution namespaces rather than cluster-wide (ClusterRoleBinding) across administrative or control-plane namespaces.