CVE-2026-88983: Authenticated Server-Side Request Forgery in Kubeflow Pipelines API Server

Kubeflow Pipelines (kubeflow/pipelines) is a platform for building and deploying portable, scalable machine learning workflows on Kubernetes. A core component of this platform is the backend API server (ml-pipeline), which exposes HTTP REST and gRPC endpoints to manage pipeline definitions, versions, experiments, and runs. Users can register new pipelines either by uploading a packaged specification file directly or by instructing the API server to fetch the specification from an external URI via the pipelineURL or PipelineSpecURI parameters.

In Kubeflow Pipelines versions prior to 2.17.0, the API server's URL retrieval mechanism contains a Server-Side Request Forgery (CWE-918) and Confused Deputy (CWE-441) vulnerability, tracked as CVE-2026-88983 (and internally under audit identifier ADA-KUBEFL-01). When processing a pipeline registration request containing a remote URL, the API server directly passes the user-supplied string to Go's standard http.DefaultClient without performing destination validation, hostname allowlisting, or network boundary checks.

Because the ml-pipeline API server pod executes within the Kubernetes cluster network namespace, it possesses ambient network access to cluster-internal DNS, Kubernetes service IP ranges, pod subnets, loopback interfaces, and link-local cloud provider instance metadata services. An authenticated attacker with basic RBAC permissions to create a pipeline or pipeline version can supply arbitrary internal destinations as the pipelineURL, coercing the API server into initiating outbound TCP connections and HTTP GET requests on their behalf.

Furthermore, the API server's error handling logic distinguishes between TCP-level connection failures and non-200 HTTP responses, returning distinct error messages and status codes to the caller. While the response body of a non-pipeline endpoint is not directly returned to the attacker, this differential error reporting establishes a reliable HTTP status-code and port-scanning oracle across the internal Kubernetes topology.

The root cause of CVE-2026-88983 lies in the initialization and unguarded invocation of the HTTP client within backend/src/apiserver/server/pipeline_server.go. Both the v1 (PipelineServerV1) and v2 (PipelineServer) server implementations initialized their internal httpClient struct field using http.DefaultClient. The vulnerable code paths reside within the createPipelineAndPipelineVersion and createPipelineVersion methods, which handle pipeline ingestion over both REST and gRPC interfaces.

When a caller provides a remote URI for a pipeline specification, the server extracts the URL string and executes s.httpClient.Get(pipelineURL.String()). Prior to the patch in version 2.17.0, the application performed zero validation on the parsed URL components. Specifically, there were no checks to restrict the URL scheme (allowing plain http://), no restrictions on destination ports (allowing requests to arbitrary internal ports such as :6379, :8080, or :22), and no filtering of the target hostname or resolved IP addresses against private or reserved CIDR blocks.

In addition to missing pre-request validation, reliance on http.DefaultClient introduced secondary transport-layer weaknesses. By default, Go's http.Client automatically follows up to 10 HTTP redirects (301, 302, 307, 308) without re-evaluating the target destination. Even if superficial string checks had existed at the API boundary, an attacker could bypass them using an external server that responds with a redirect to an internal IP address.

Finally, standard HTTP client resolution is susceptible to DNS rebinding (Time-of-Check to Time-of-Use, or TOCTOU). If an application validates a hostname by resolving its IP prior to calling http.Client.Get(), the underlying network dialer performs a second DNS lookup when establishing the TCP socket. An attacker controlling an authoritative DNS server can return a public IP during the validation check with a Time-To-Live (TTL) of zero, and subsequently return a loopback (127.0.0.1) or metadata (169.254.169.254) IP address when the dialer connects.

In versions prior to 2.17.0, backend/src/apiserver/server/pipeline_server.go retrieved remote pipeline specifications directly using the server's default HTTP client and reflected the resulting transport or HTTP errors back to the client:

// Vulnerable code in backend/src/apiserver/server/pipeline_server.go (prior to v2.17.0)
resp, err := s.httpClient.Get(pipelineURL.String())
if err != nil {
    // Reflects low-level dial/network errors as HTTP 500
    return nil, nil, util.NewInternalServerError(err, "error downloading the pipeline spec from %v", pipelineURL.String())
} else if resp.StatusCode != http.StatusOK {
    // Reflects exact remote HTTP status string (e.g., "403 Forbidden") as HTTP 400
    return nil, nil, util.NewInvalidInputError("error fetching pipeline spec from %v - request returned %v", pipelineURL.String(), resp.Status)
}

To remediate this vulnerability in Pull Request #12750 (commit 504d7797c1e061327ddb2bcb8bc1c5dd4940cfc3), the maintainers introduced a dedicated validation and transport package at backend/src/apiserver/validation/pipeline_url.go. First, the patch establishes an explicit IP/CIDR blocklist covering loopback interfaces, RFC 1918 private networks, IPv6 Unique Local Addresses (ULA), link-local subnets, and cloud metadata endpoints:

// Patched code in backend/src/apiserver/validation/pipeline_url.go
var blockedCIDRs = []string{
    // Loopback
    "127.0.0.0/8",
    "::1/128",
 
    // Private Subnets (RFC 1918 & IPv6 ULA)
    "10.0.0.0/8",
    "172.16.0.0/12",
    "192.168.0.0/16",
    "fc00::/7",
 
    // Link-local
    "169.254.0.0/16",
    "fe80::/10",
 
    // Cloud Metadata Endpoints (AWS, GCP, Azure, OpenStack)
    "169.254.169.254/32",
    "fd00:ec2::254/128",
 
    // Invalid / Unspecified
    "0.0.0.0/8",
    "::/128",
}

Second, the patch implements ValidatePipelineURL, which enforces scheme restrictions (https only by default unless PIPELINE_URL_ALLOW_HTTP is set), blocks non-standard ports (permitting only port 443 for HTTPS and 80 for HTTP), and verifies the hostname against a strict allowlist (defaulting to storage.googleapis.com, s3.amazonaws.com, github.com, and raw.githubusercontent.com, extensible via PIPELINE_URL_ALLOWED_DOMAINS).

Third, to prevent DNS rebinding (TOCTOU) and redirect-based bypasses, the patch replaces http.DefaultClient with SafePipelineHTTPClient. This custom client disables ambient environment proxies (transport.Proxy = nil), implements safeDialContext to validate resolved IP addresses at TCP connection time before dialing the verified IP directly, and attaches a CheckRedirect handler:

// Redirect validation in SafePipelineHTTPClient
CheckRedirect: func(req *http.Request, via []*http.Request) error {
    if len(via) >= 10 {
        return util.NewInvalidInputError("too many redirects")
    }
    // Re-validate every redirect target against scheme, port, domain allowlist, and CIDR blocklist
    return ValidatePipelineURL(req.URL.String())
},

Fix Completeness Assessment

The remediation implemented in PR #12750 is comprehensive and addresses both direct SSRF vectors and common bypass techniques. By combining a strict domain allowlist with socket-level IP verification inside safeDialContext, the fix eliminates the TOCTOU window required for DNS rebinding. Re-running ValidatePipelineURL on every redirect hop prevents open-redirect pivoting from allowed domains (such as GitHub or S3) to internal addresses. One operational caveat is the PIPELINE_URL_VALIDATION_ENABLED environment variable: while it defaults to true, administrators must ensure it is never set to false in production deployments, as doing so completely disables these controls.

Exploiting CVE-2026-88983 requires network access to the Kubeflow Pipelines API server (either directly or through the Kubeflow central dashboard/Istio ingress gateway) and an authenticated session with RBAC privileges to create a pipeline or pipeline version (pipelines.kubeflow.org create verbs). In standard multi-user Kubeflow deployments, any authenticated data scientist or developer assigned to a user namespace possesses these permissions.

To execute the attack, the adversary submits an HTTP POST request to the v1beta1 (/apis/v1beta1/pipelines) or v2beta1 (/apis/v2beta1/pipelines) endpoint, or invokes the corresponding CreatePipeline / CreatePipelineVersion gRPC methods. In the request payload, the attacker sets the pipeline_url field to the target internal resource.

Scenario 1: Probing Cloud Instance Metadata Services (IMDS)

An attacker attempting to discover whether IMDSv1 is accessible from the ml-pipeline pod submits the following payload:

{
  "pipeline": {
    "name": "metadata-probe",
    "url": {
      "pipeline_url": "http://169.254.169.254/computeMetadata/v1/"
    }
  }
}

On Google Cloud Platform (GCP), requests to /computeMetadata/v1/ without the Metadata-Flavor: Google header are rejected by the metadata server with an HTTP 403 Forbidden status. The Kubeflow API server processes this response and returns: 400 Invalid Input: error fetching pipeline spec from http://169.254.169.254/computeMetadata/v1/ - request returned 403 Forbidden. This confirms to the attacker that the pod has network routing to the metadata endpoint.

Scenario 2: Internal Kubernetes Service and Port Discovery

An attacker can systematically enumerate internal cluster services and open TCP ports by targeting Kubernetes DNS names or ClusterIP ranges:

{
  "pipeline": {
    "name": "internal-service-scan",
    "url": {
      "pipeline_url": "http://metadata-grpc-service.kubeflow.svc.cluster.local:8080/healthz"
    }
  }
}

By analyzing the returned error codes, the attacker differentiates the state of internal targets:

• Closed Port / Unreachable Host: The API server returns 500 Internal Server Error containing the Go net.OpError string (e.g., dial tcp 10.96.x.x:8080: connect: connection refused or no such host).
• Open Port with Non-200 HTTP Response: The API server returns 400 Bad Request / InvalidArgument along with the exact HTTP status line (e.g., 401 Unauthorized, 404 Not Found, 405 Method Not Allowed).
• Open Port with 200 OK Response: The API server attempts to parse the response body as a pipeline archive or YAML specification, failing with a deserialization error if the content is not a valid pipeline spec.

CVE-2026-88983 is assigned a CVSS v3.1 base score of 5.0 (Medium) with the vector CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:L/I:N/A:N. The Attack Vector is Network (AV:N) and Attack Complexity is Low (AC:L), as exploitation requires only standard HTTP/gRPC API calls without race conditions. Privileges Required is Low (PR:L), reflecting the requirement for an authenticated user account capable of registering pipelines.

The Scope metric is Changed (S:C) because the vulnerable component is the Kubeflow Pipelines API server, whereas the impacted resources are separate microservices within the Kubernetes cluster network, underlying host loopback services, or cloud infrastructure metadata planes. The Confidentiality impact is rated Low (C:L), while Integrity (I:N) and Availability (A:N) are rated None.

The primary concrete impact of this vulnerability is unauthorized internal network reconnaissance and topology mapping. Because the API server expects the remote response body to conform to a valid Kubeflow pipeline package or YAML structure, arbitrary raw response bodies from internal services (such as credentials from an IMDSv1 endpoint) are not directly echoed back to the client if they fail pipeline deserialization. However, the verbose error reporting acts as a reliable oracle that reveals internal DNS records, active IP addresses, open TCP ports, and HTTP endpoint existence.

Furthermore, if any internal microservice within the cluster or local pod network exposes state-changing operations via unauthenticated HTTP GET requests, an attacker can trigger those actions through this SSRF primitive. As of publication, the Exploit Prediction Scoring System (EPSS) score for CVE-2026-88983 is 0.00199 (9.82nd percentile), and the vulnerability is not listed in the CISA Known Exploited Vulnerabilities (KEV) catalog.

Patching Guidance

Organizations running Kubeflow Pipelines must upgrade the ml-pipeline API server component to version 2.17.0 or later. Version 2.17.0 incorporates the URL validation engine, CIDR blocklist, and hardened HTTP dialer introduced in Pull Request #12750.

Configuration Hardening (v2.17.0+)

When deploying Kubeflow Pipelines 2.17.0 or later, administrators should review the environment variables passed to the ml-pipeline deployment to ensure defensive defaults remain active:

• PIPELINE_URL_VALIDATION_ENABLED: Must be set to "true" (the default). Never disable this setting in non-test environments.
• PIPELINE_URL_ALLOW_HTTP: Keep set to "false" (the default) to enforce TLS (https://) for all remote pipeline specification downloads.
• PIPELINE_URL_ALLOWED_DOMAINS: By default, only storage.googleapis.com, s3.amazonaws.com, github.com, and raw.githubusercontent.com are permitted. If an organization stores pipeline specifications in an internal artifact repository (such as an enterprise GitLab instance or MinIO gateway), add only the specific FQDN to PIPELINE_URL_ALLOWED_DOMAINS rather than disabling URL validation.
• PIPELINE_URL_TIMEOUT: Defaults to 30 seconds; can be lowered to reduce connection hold times.

Workarounds and Defense-in-Depth

For environments that cannot immediately upgrade to version 2.17.0, administrators should apply Kubernetes NetworkPolicy resources to restrict egress traffic from the ml-pipeline API server pod:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: restrict-ml-pipeline-egress
  namespace: kubeflow
spec:
  podSelector:
    matchLabels:
      app: ml-pipeline
  policyTypes:
    - Egress
  egress:
    # Allow DNS resolution
    - to:
        - namespaceSelector: {}
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    # Allow external HTTPS while blocking RFC 1918 and Link-Local/IMDS
    - to:
        - ipBlock:
            cidr: 0.0.0.0/0
            except:
              - 10.0.0.0/8
              - 172.16.0.0/12
              - 192.168.0.0/16
              - 169.254.0.0/16
      ports:
        - protocol: TCP
          port: 443

> [!NOTE] > When applying an egress NetworkPolicy to ml-pipeline, ensure explicit egress rules are added for required cluster-internal dependencies, including the relational database (mysql or postgres), the object store (minio-service), and the ML Metadata (metadata-grpc-service) endpoints on their respective ports.