5. Security & Threat Risk Assessment¶
← Operational Flows | Back to index | Next: Implementation Phases →
The two-tier architecture introduces both stronger isolation guarantees and new attack surfaces. Threats are grouped by which tier they affect.
5.1. GMC-Level Threats (Cluster-Scoped)¶
| Threat Vector | Impact | Mitigation Strategy |
|---|---|---|
| GMC Privilege Escalation (Blast Radius: All Tenants) | Critical | Cluster-wide provisioning writes are confined to marked tenant namespaces by two ValidatingAdmissionPolicy objects (namespace-psa-guard, gmc-tenant-resource-guard); the honest residual is cluster-wide Secret reads, offset by metadata-only caching, uncached direct reads, and the Q29 audit policy. Full blast radius and compensating controls: see below. |
| Admission Webhook Unavailability or Bypass | Medium | The reserved-namespace validating webhook serves as a safety check, not a security boundary — namespace isolation is enforced by RBAC and NetworkPolicy regardless of webhook state. The webhook uses failurePolicy: Fail so requests are rejected when the webhook pod is unhealthy rather than silently bypassed. Serving certificates are managed by cert-manager with automatic rotation; the CA bundle is injected via caBundle from the cert-manager-managed Secret. Webhook pod runs replicas: 2 behind a Service with podAntiAffinity to survive single-node loss without stalling tenant onboarding. |
| Metrics Scrape Interception (MITM) | Low | The :8443 metrics endpoint is TLS + TokenReview/SAR-authorized, served with a cert-manager metrics cert the ServiceMonitor verifies (secure default), with a documented insecureSkipVerify fallback only when cert-manager is absent; the manager NetworkPolicy admits :8443 only from metrics: enabled namespaces. Full TLS/network posture: see below. |
| Tenant Namespace Escape via Overpermissioned AGC | Critical | Each AGC's ServiceAccount is bound by a RoleBinding limited to its own namespace. The AGC cannot list or touch resources in any other tenant namespace. |
| Cross-Tenant GitHub App Credential Leakage | High | ActionsGateway is namespace-scoped so a gitHubAppRef cannot cross namespaces, and the credential Secret is mounted into the AGC pod only (never worker pods). Secret writes are confined by gmc-tenant-resource-guard; cluster-wide reads are offset by two-layer cache isolation (metadata-only informer + DisableFor uncached Get). Full detail: see below. |
ActionsGateway CR in Reserved Namespace |
Medium | An admission webhook rejects ActionsGateway CRs created in reserved namespaces: the universal kube-system and kube-public, the GMC's default install namespace gmc-system, and the namespace the GMC pod is actually running in (read from the POD_NAMESPACE downward-API env var, so custom installs are protected too). The same guard applies to the v2 kinds that make the GMC provision workloads into their namespace — the v2 ActionsGateway (AGC control plane) and EgressProxy (proxy Deployment + NetworkPolicies) — via the shared validation package (Q323); a v2 RunnerSet/RunnerTemplate needs no guard of its own because it is inert without a same-namespace gateway. Since the CR is namespace-scoped, a tenant can only affect their own namespace — the risk is self-harm or collision with operator-owned resources, not cross-tenant impact. |
GMC privilege-escalation blast radius and compensating controls¶
The GMC's ClusterRole grants cluster-wide write (create/update/delete) on the tenant provisioning kinds — Deployments, Services, ServiceAccounts, RoleBindings, Roles, NetworkPolicies, HPAs, PodDisruptionBudgets, RunnerGroups, and Secret create/update — because RBAC cannot express "only namespaces that carry a marker label". Two ValidatingAdmissionPolicy objects close that gap by confining the GMC ServiceAccount at admission: namespace-psa-guard confines namespaces:patch to the six pod-security.kubernetes.io/* keys on marked namespaces, and gmc-tenant-resource-guard confines every create/update/delete of the kinds above to namespaces an administrator has marked actions-gateway.github.com/tenant: "true" (Q121 write path / Q122). So a compromised GMC cannot create a Deployment or RoleBinding in kube-system, write a Secret into an arbitrary namespace, or relabel kube-system PSA to privileged (see §5.3). The grant RBAC and admission cannot confine is Secret reads: admission never runs on get/list/watch, resourceNames cannot scope list/watch, and tenant Secret names are dynamic so get cannot be name-scoped either — therefore the ClusterRole grants cluster-wide Secret get/list/watch, and this is an honestly-stated residual, not a confined property (Q121; compensating controls below). Explicit blast radius if compromised: a compromised GMC can (a) enumerate every ActionsGateway CR in the cluster, learning each tenant's gitHubAppRef name and namespace; (b) list/watch and get the full .data of any Secret in the cluster, including GitHub App private keys (the metadata-only informer and uncached reads below are client-side hygiene, not authorization); (c) create/update/delete workloads and Secrets only in marked tenant namespaces (the gmc-tenant-resource-guard VAP denies writes elsewhere). It CANNOT exec into pods, create new namespaces, or write resources into unmarked namespaces such as kube-system. Secret-read compensating controls: the GMC uses WatchesMetadata so the informer cache holds only Secret ObjectMeta (no .data), client.Cache.DisableFor[*corev1.Secret] forces r.Get() to hit the API server directly (key material is never cache-resident), in practice the GMC only gets the named credential Secret in the CR's own namespace, and the Q29 audit policy (sample, runbook) is the detective complement that surfaces any out-of-pattern Secret read. GMC pod runs with non-root user, read-only root filesystem, no host mounts, and seccompProfile: {type: RuntimeDefault}. Image is digest-pinned, enforced at chart render time — the Helm chart refuses to render an unpinned gmc.image. Treat the GMC pod as a Tier-0 workload for monitoring and access.
Metrics scrape interception and the mTLS posture¶
The GMC metrics endpoint (:8443) is served over TLS and authorized by TokenReview/SubjectAccessReview. By default the chart issues a dedicated metrics serving cert via cert-manager (metrics-serving-cert → metrics-server-cert Secret, minted from the same selfsigned-issuer as the webhook), the GMC serves it with --metrics-cert-path, and the rendered ServiceMonitor verifies it against the issuing CA (tlsConfig.ca + serverName) — so an in-cluster attacker cannot impersonate the metrics endpoint. This is the secure default (metrics.tls.certManager.enabled=true), gated on certManager.enabled=true since it reuses the webhook Issuer. When cert-manager is absent (certManager.enabled=false) or the toggle is off, the GMC falls back to controller-runtime's auto-generated self-signed cert and the ServiceMonitor scrapes with tlsConfig.insecureSkipVerify: true — an explicit, documented opt-out that loses server verification (the bearer token still authenticates the scraper to the server, but not the server to the scraper). As network-layer defence in depth, the manager NetworkPolicy (networkPolicy.enabled=true, default; Q34/E5) flips the controller-manager pod to default-deny ingress and re-admits :8443 only from namespaces labelled metrics: enabled, while leaving the webhook port :9443 open (no source restriction) so admission keeps working under failurePolicy: Fail. This deny-unlabelled / allow-labelled / admission-still-works posture was verified end-to-end at runtime on a Calico kind cluster on 2026-06-18 (Q83): a scrape from an unlabelled namespace timed out (no route to :8443), a scrape from a metrics: enabled namespace reached the endpoint (HTTP 401 — TLS/auth layer, not network), and the validating webhook on :9443 returned its verdict (admission unaffected). The check is codified as a Calico-gated Tier-A e2e spec (Manager NetworkPolicy). See observability.md § Verifying the metrics scrape TLS.
Both TLS servers the manager exposes — the :8443 metrics endpoint and the :9443 admission webhook — are configured with HTTP/2 disabled (TLSOpts force NextProtos: ["http/1.1"]). This closes the HTTP/2 Rapid Reset denial-of-service class (CVE-2023-44487 / CVE-2023-39325), where a client cheaply cancels a flood of streams to exhaust server resources. The endpoints carry low request volume, so losing HTTP/2 multiplexing costs nothing; keeping it would expose a DoS surface for no benefit. This matches the controller-runtime scaffold's own secure default.
Cross-tenant GitHub App credential leakage and cache isolation¶
ActionsGateway is namespace-scoped, so a tenant's gitHubAppRef defaults to their own namespace — another tenant cannot reference it. The GMC mounts credentials into the AGC Pod only; worker pods never have access to the Secret object. Secrets are immutable; rotation creates a new Secret and updates the CR reference, producing a clean Deployment rollout. Old Secrets are not readable by running Pods once the rollout completes. The GMC's ClusterRole grants Secret get/list/watch cluster-wide (not name-scoped — resourceNames cannot scope list/watch, and tenant Secret names are dynamic so it cannot scope get either); Secret writes (create/update) are confined to marked tenant namespaces by the gmc-tenant-resource-guard ValidatingAdmissionPolicy, but reads cannot be confined at the authorization layer (admission does not run on read verbs — Q121). Two-layer cache isolation (the compensating control for reads): the GMC uses WatchesMetadata (not a full Secret watch) so the in-process informer cache holds only Secret ObjectMeta (name, namespace, resourceVersion — no .data); and client.Cache.DisableFor[*corev1.Secret] ensures r.Get() calls bypass the cache entirely and hit the API server directly, so actual key material is never resident in memory beyond the duration of a single reconcile call.
5.2. AGC & Proxy-Level Threats (Namespace-Scoped)¶
Several mitigations below rest on the per-tenant NetworkPolicies the GMC reconciles (workload egress restricted to DNS + proxy, with DNS itself confined to the cluster DNS service rather than any resolver — Q105; only AGC-labelled pods get apiserver egress; workload pods default-deny all ingress — Q128). The ingress default-deny matters because worker pods run untrusted GitHub Actions job code and are outbound-only by design (they long-poll/dial out to GitHub via the proxy and to the AGC); nothing legitimately initiates a connection to a worker, so the workload NP declares policyTypes: [Ingress, Egress] with an empty ingress rule set. Without it, worker pods were default-allow ingress and any pod in the cluster could open connections to untrusted job code — a lateral-movement / cross-tenant channel. NetworkPolicy objects are inert unless the cluster CNI enforces them — kind's default kindnet does not drop traffic, so production clusters must run a policy-enforcing CNI (Calico, Cilium, or equivalent). Runtime enforcement of the egress negatives was observed on a Calico kind cluster on 2026-06-11 (Q7b; see network-architecture.md § How to Validate Network Isolation).
Secure-by-default trade for the optional egress proxy (Q168, signed off). In v2 the egress proxy is optional (appendix-h §H.10): a gateway/runner set with no defaultProxyRef/proxyRef egresses directly to GitHub. This is a deliberate, reviewed trade and it splits the two properties the proxy used to bundle. It drops egress identity (per-tenant IP attribution): direct egress leaves a workload with no stable per-tenant source IP, surfaced explicitly as proxyMode: Direct plus an advisory EgressUnattributed condition so the operator sees the property they opted out of. It does not drop egress restriction: the GMC still provisions the mandatory, on-by-default default-deny egress NetworkPolicy, now allowing DNS (cluster DNS only) + the GitHub CIDR allowlist for workers, plus the kube API server for the AGC — there is no proxy-less mode in which a worker or AGC can reach arbitrary internet, and the GMC's IPRangeReconciler keeps the direct-egress GitHub allowlist current as GitHub rotates ranges. Restriction stays mandatory and default-on; only identity is the opt-in (attach an EgressProxy). Defaulting off the restriction would be a security regression and is out of scope. The DNS-exfiltration containment below (port-53 confined to cluster DNS) is unchanged in direct mode; what direct egress removes is only the per-tenant attribution of the GitHub-bound leg, which is exactly the EgressUnattributed-flagged property.
CNI-native FQDN egress is an opt-in, fail-closed, never a default (Q208 / Q245). How the GitHub allowlist is expressed is operator-selectable on a v2
EgressProxy, split across two roles (Q245): the tenant expresses intent viaspec.egressPolicyMode(FQDN= "by hostname"; defaultCIDR= the standard NetworkPolicy + 24hIPRangeReconcilerabove, works on every CNI), and the operator picks the enforcement mechanism via the GMC--fqdn-policy-backendflag (none|cilium|calico|gke). The GMC then emits a CNI-native DNS-aware policy (CiliumNetworkPolicytoFQDNs/ CalicoNetworkPolicydomains/ GKEFQDNNetworkPolicymatches) scoped to the GitHub hostnames, dropping the GitHub-CIDR rule from the standard NetworkPolicy and skipping the IP-range reconcile for that proxy. The deprecatedCiliumFQDN/CalicoFQDNenum values still work (each pins its namesake backend) but the webhook warns and steers toFQDN+ a backend. This does not relax restriction: the standard NetworkPolicy still default-denies GitHub egress, so if the chosen backend cannot enforce the FQDN policy (wrong CNI / CRD absent) the apply fails loudly (EgressProxy→Degraded) and GitHub egress stays denied rather than opening wide — selecting an FQDN mode can only ever keep or tighten the posture, never weaken it. The secure default--fqdn-policy-backend=nonerejects anFQDN-intentEgressProxyat admission rather than guessing a mechanism.The
gkebackend is additive-allow, so the base default-deny NetworkPolicy is a required invariant. Unlike Cilium/Calico FQDN policies (self-default-denying), a GKEFQDNNetworkPolicycomposes with a NetworkPolicy on the same pod as a union — it only adds an allow, it denies nothing on its own. GAG's fail-closed guarantee forgketherefore depends on the base standard NetworkPolicy (always emitted; GitHub-CIDR rule dropped, DNS-only allow kept) staying present alongside it: theFQDNNetworkPolicyonly widens the union to permit GitHub, and if it is absent or unenforced GitHub egress stays denied by the base NP. A future refactor must not drop the base NP for thegkebackend, or it opens egress. (gke-backend enforcement is not yet live-validated; see the Q245 plan.) See network-architecture.md § CNI-native FQDN egress mode and the operator runbook security-operations.md § Expressing GitHub egress by FQDN.Pod placement is CR-author-settable, so per-tenant egress-IP attribution is a property of the cluster's policy layer, not of GAG alone (Q282). A per-tenant egress IP is realized by pinning a tenant's pods to a node pool whose egress path (cloud NAT gateway, dedicated subnet) owns that IP — so the pod's node decides which IP its traffic leaves by. GAG passes placement through to the CR author in three places:
RunnerTemplate.spec.podTemplate(a fullPodTemplateSpec, always has), and — since Q282 —EgressProxy.spec.schedulingandActionsGateway.spec.scheduling, each carryingnodeSelector/tolerations/affinity. This is deliberate and not admin-gated: choosing an egress path is a feature (tenants should not share a rate limit or a block radius). Stated precisely, since the symmetry with worker pods is only partial — inproxyMode: Directworker placement already selects the egress IP with no gate, but in the proxied default it does not (NetworkPolicy forces worker traffic through the proxy, so the proxy pool's placement selects the IP).EgressProxy.spec.schedulingtherefore does extend the capability to the default posture; gating it would still leave the direct-egress path open, and — see below — no validating gate on these fields can be sound anyway. What this means precisely: a CR author who retargets its pods onto another pool's egress path makes both tenants' traffic leave via one IP, so attribution no longer holds. Isolation, RBAC, and the egress choke point are unchanged — the pods still traverse the same default-deny NetworkPolicy and the same proxy. Operators who attribute traffic by source IP (a downstream firewall allowlist, an audit trail) and do not fully trust their tenant CR authors must constrain placement in the pod admission layer. Note that a validating allowlist ofnodeSelectorvalues is unsound —affinity.nodeAffinitysupportsNotIn/DoesNotExist, so "any pool but mine" is expressible — whereas pinningnodeSelectorby mutation is sound, since Kubernetes ANDs it withnodeAffinityand affinity can only narrow. Ready-to-apply Kyverno and Gatekeeper samples: admission-policies.md § Governing where GAG pods schedule. The same layer is where a cluster that grants tenantspods: createmust constrain placement anyway — GAG's NetworkPolicies select GAG-labelled pods, so an unlabelled tenant-created pod is selected by none of them.Worker egress beyond GitHub is a platform-gated, deny-by-default opt-in (Q242 G.1). By default the proxy carries GitHub only. A platform admin can allow a small, explicit set of non-GitHub destinations a v2
EgressProxymay forward to — by DNS host suffix (spec.destinationFQDNs) and CIDR (spec.destinationCIDRs) — so CI jobs reach their build dependencies without forfeiting per-tenant egress-IP attribution or the DNS-exfil containment. Because theEgressProxyis tenant-authorable, the destinations are governed by a platform-owned allowlist, not the CR's ownership: the GMC--allowed-egress-fqdns(suffix match) /--allowed-egress-cidrs(subnet containment) flags — optionally augmented by a watched ConfigMap — plus a validating webhook that rejects any destination not on the allowlist. Both empty ⇒ deny-all-non-GitHub, identical in spirit to the empty--allowed-priority-classesdefault (Q132/Q188). The destinations widen one source into two enforcement surfaces (CIDRipBlock/ FQDNtoFQDNson the pod-egress policy, and the proxy CONNECT allowlist as defense-in-depth with DNS-rebinding revalidation on the CIDR path); GitHub stays implicit. This is a deliberate, bounded relaxation of the GitHub-only posture, accepted with the trade-offs recorded — the residual is that a compromised worker can exfiltrate to an allowlisted destination, which the allowlist bounds but does not eliminate; the docs therefore steer operators to an in-cluster caching mirror first and reserve the allowlist for what a mirror can't proxy. See the threat row below, network-architecture.md § Worker egress to allowlisted non-GitHub destinations, the operator runbook security-operations.md § Worker egress destinations, and the design Q242 plan.A
noProxyCIDRsentry must never route GitHub around the proxy.noProxyCIDRs(v1ActionsGateway.spec.proxy, v2EgressProxy.spec) is threaded verbatim into the AGC/workerNO_PROXYenv var, where a hostname entry is a domain-suffix match — so a tenant-authored entry likegithub.com(or an over-broad.com) would silently exclude GitHub from the proxy and defeat per-tenant egress-IP attribution. Both GMC validating webhooks reject any entry that NO_PROXY-matches a protected GitHub host (shared guard:gmc/internal/webhook/noproxy), while still admitting the legitimate internal-destination entries (CIDRs, bare IPs, non-GitHub domain suffixes such assvc.cluster.local). On the v2EgressProxy— which carries nogitHubURLof its own — the protected set is the public GitHub hosts plus thegitHubURLhost (a GitHub Enterprise Server host included) of every referrer, resolved through the uncached API reader from both directions (Q322): theEgressProxywrite checks the gateways/RunnerSets that reference it, and theActionsGateway/RunnerSetwrite checks the referenced proxy'snoProxyCIDRs, so the bypass pair is rejected regardless of creation order (gmc/internal/webhook/v2alpha1/noproxy_referrers.go; unresolvable reads fail closed). One accepted residual: a CIDR/IP entry covering GitHub's rotating published ranges is not detected on either version (an in-tree IP blocklist would rot; operator responsibility). The runbook for the rejection: troubleshooting.md §proxy.noProxyCIDRsrejected.
| Threat Vector | Impact | Mitigation Strategy |
|---|---|---|
| Host Namespace Escape via Malicious Workflow (Container Breakout) | Critical | Enforced in three layers. (1) The AGC unconditionally sets hostPID: false, hostNetwork: false, hostIPC: false, and automountServiceAccountToken: false on every worker pod, overwriting tenant PodTemplate values at pod-creation time. (2) The GMC stamps pod-security.kubernetes.io/enforce on the tenant namespace at provisioning time, with the level chosen by ActionsGateway.spec.securityProfile — see §5.3. The default baseline blocks privileged containers, hostPath, dangerous capabilities, and host namespaces via the in-tree PodSecurity admission plugin; no external policy engine is required. (3) Sandboxed container runtimes (Kata Containers, gVisor) are supported via runtimeClassName in the PodTemplate — optional but strongly recommended for tenants who select the privileged profile. See Appendix B for tradeoffs. |
| Supply-Chain Compromise of Worker Image | High | WorkerImage should be digest-pinned; the GMC-injected AGC/proxy images are digest-pinning-enforced at startup and the GMC's own image at chart-render time. CI runs govulncheck + trivy on every PR; release images are multi-arch, keyless-cosign-signed with per-architecture SBOM and SLSA build-provenance attestations. Full scanning / signing / provenance detail: see below. |
| Cross-Job Code Contamination | High | Enforce absolute 1-Job-Per-Pod isolation. Avoid reusing volumes or host paths between worker pods. Use ephemeral, emptyDir volumes for workspace storage. |
| AGC Token Compromise | High | The AGC never saves plaintext keys to disk. GitHub App private keys are mounted as read-only volumes with restrictive file permissions (0400). |
| Token Exchange over a Plaintext Channel | High | Minting a GitHub App installation access token POSTs an App-JWT (signed with the tenant's RSA/Ed25519 private key) to the token-exchange endpoint and receives a short-lived installation token in the response — both are credential material that a non-HTTPS channel would expose to any on-path observer. The endpoint host is GITHUB_API_BASE_URL (defaulting to https://api.github.com); historically any value was accepted, including a plaintext http:// URL. The token provider now rejects a non-HTTPS GITHUB_API_BASE_URL by default (validated at AGC/probe startup, so a misconfiguration fails fast rather than leaking on first mint) — TLS for this leg is secure-by-default, not opt-in. The legitimate dev/test case (the e2e suite points the AGC at an in-cluster fakegithub over plaintext) is preserved via an explicit opt-in: the AGC permits a plaintext base URL only when the stub env (STUB_AUTH_URL) is set — a signal a production AGC never carries (it reaches a GMC-provisioned AGC only via AGC_EXTRA_* under the testing-only --allow-agc-extra-env flag) — and logs the relaxation at startup. The probe and the live egress-IP test pass no opt-in, so they always require HTTPS. The error names the offending URL but never any token/JWT material. See security-operations.md § GitHub API base URL must be HTTPS. |
| GitHub-App-Key Exfiltration via AGC apiserver Egress | Medium | The AGC apiserver egress rule allows ports 443/6443 with no to: restriction because the post-DNAT apiserver IP is unpredictable — any-destination is the secure default. Blast radius is bounded (read-only 0440 key mount, no worker apiserver egress, digest-pinned non-root AGC); operators with a stable apiserver CIDR can opt into --apiserver-cidrs scoping. Rationale, per-cloud guidance, and the auto-narrowing verdict: see below. |
| Credential Leak via Logged Error Bodies | Medium | The AGC, broker client, and probe interpolate upstream GitHub HTTP response bodies into errors that callers log. Some of these bodies carry credential material — the runner-token endpoint's 200 body holds an access token, and generate-jitconfig's body holds the runner JIT registration credential plus RSA key. Before any upstream body is placed into an error or log line it passes through a single shared redactor (githubapp.SanitizeBody) that strips credential-shaped substrings (GitHub gh*_/github_pat_ tokens, JWTs, access_token/encoded_jit_config/private_key/secret JSON values, and long opaque base64 blobs) and caps the result. Redaction runs before capping so a secret straddling the cap boundary cannot survive in the truncated tail. No secret is ever logged directly; this control hardens the indirect path. |
| Eviction-Retry API Misuse | Medium | The AGC calls POST /repos/{owner}/{repo}/actions/runs/{run_id}/rerun-failed-jobs using the tenant's installation access token when a worker pod is evicted. The blast radius is bounded: the installation token is scoped to the GitHub App's installation on a specific organization or repository, so the AGC cannot re-run jobs belonging to other tenants or organizations. The run_id is extracted from the job payload delivered by GitHub's broker — the AGC cannot fabricate or substitute a run ID for a run it did not acquire. To prevent abuse of the retry path (e.g. a compromised AGC looping re-runs), maxEvictionRetries caps the number of automatic retries per job and is enforced before the API call is made. Operators should monitor actions_gateway_eviction_retries_exhausted_total to detect abnormal eviction patterns. |
| DNS Exfiltration Side-Channel (Unattributed Egress) | Medium | All three per-tenant NetworkPolicies confine port-53 egress to cluster DNS only — three OR'd peers (kube-dns, node-local-dns, and the 169.254.0.0/16 link-local block) — closing the unattributed any-resolver side-channel while preserving legitimate resolution. Enforced only by a policy-aware CNI. Full peer breakdown and Q229 verification: see below. |
| Proxy as Traffic Interception Point | Medium | The proxy only handles CONNECT tunneling and does not terminate TLS. It cannot inspect or modify the encrypted payload between the AGC/worker and GitHub. Proxy pods run with a read-only root filesystem and no elevated capabilities. |
| Worker Exfiltration via an Allowlisted Non-GitHub Destination (deliberate, bounded relaxation — Q242 G.1) | Medium | Exists only when a platform admin opts in to non-GitHub worker egress; the tenant-authorable destinations are gated by a platform-owned allowlist (both empty = deny-all-non-GitHub) enforced by webhook plus dual-surface NetworkPolicy/CONNECT gates, with per-tenant attribution and in-cluster DNS containment preserved. Full trade-off record: see below. |
| Cross-Tenant Proxy CA Trust | Medium | The egress proxy's TLS cert is signed by a cert-manager-issued self-signed CA stored in the per-tenant actions-gateway-proxy-tls Secret. The AGC pins this CA explicitly (via its trust pool) rather than trusting the cluster's root store, and worker pods install the same CA into a combined SSL_CERT_FILE bundle so Runner.Worker's .NET HttpClient accepts the proxy handshake. The cert (tls.crt) is projected into both AGC and worker pods via an Items: [tls.crt] Secret volume; the private key (tls.key) is mounted only into the proxy pod itself, so a runner compromise does not yield the ability to forge a proxy cert. Trust is tenant-scoped: each tenant's CA is independent, so a compromised CA in one namespace cannot mint a cert trusted by another tenant's AGC or workers. The proxy's CONNECT listener pins a TLS 1.2 minimum explicitly (matching the metrics listener) rather than inheriting the Go default — the worker→proxy leg carries every tenant's GitHub-bound traffic, so its TLS floor is a tenant-isolation boundary and is stated in code, not left implicit (Q127). |
| Per-Tenant Metrics Scrape Interception / Unauthorized Read | Low | The per-tenant AGC and proxy serve /metrics over mutual TLS on :8443 (Q69): the listener requires a client certificate signed by that tenant's metrics CA, and there is no bearer-token or insecureSkipVerify fallback. The GMC generates a per-tenant metrics PKI (CA → server leaf with the proxy/AGC Service DNS names as SANs, client leaf for the scraper) and writes two Secrets — the server bundle (actions-gateway-metrics-tls, mounted into the AGC/proxy pods) and the scraper client bundle (actions-gateway-metrics-client, published for the monitoring stack, never mounted into the workload pods so a runner compromise cannot read it). Each tenant's CA is independent, so a client cert from one tenant cannot scrape another's endpoints, and an in-cluster attacker without the client cert cannot read a tenant's metrics or impersonate the endpoint. The metrics port is reachable on ingress only from namespaces labelled metrics: enabled (the per-tenant NetworkPolicy metrics-scrape rule, L-8). Scrape wiring is opt-in (metrics.serviceMonitor.enabled, default off): when on, the GMC creates a per-tenant ServiceMonitor for the proxy and AGC presenting the client bundle and verifying the server cert via serverName + ca — no insecureSkipVerify. The metrics Services exist regardless of the toggle; only the scrape config is gated. See observability.md § Scraping per-tenant AGC and proxy metrics (mTLS). |
| Egress IP Change Mid-Session | Low–Unknown | GitHub's broker protocol is token-based, not IP-bound. Session IDs and bearer tokens carry no IP affinity, so rotating across proxy pods mid-job is expected to work. The Twirp log stream is naturally sticky (long-lived HTTP/2 connection stays on one proxy pod once open). Impact is unknown because GitHub's abuse detection heuristics are undocumented. Early mitigation: the Milestone 1 wire protocol probe explicitly tests broker API calls routed through a multi-pod proxy pool to confirm GitHub does not reject or flag IP variance across CreateSession → GetMessage → AcquireJob. If the probe surfaces a problem, ClientIP session affinity on the proxy Service is the low-effort fallback; explicit per-goroutine proxy assignment is the higher-fidelity option if needed. |
| Proxy Pool Exhaustion (DoS via proxy saturation) | Medium | HPA minReplicas ensures a floor of available capacity. The PodDisruptionBudget prevents draining all replicas simultaneously. The platform-owned namespace ResourceQuota caps proxy pod count so a misconfigured maxReplicas cannot consume cluster CPU. The GMC also surfaces two non-blocking status conditions (Q82): ProxyQuotaPressure (warning — the pool cannot scale to maxReplicas within the quota's remaining headroom) and ProxyQuotaExceeded (error — replica creates are being rejected by the quota now). They turn a silent ceiling (the HPA wedging at FailedCreate under load) into operator-visible signals — each also exported as a gauge for alerting — without rejecting the config (the quota stays the enforcement point). |
| Denial of Service via Resource Exhaustion | Medium | The namespace ResourceQuota is platform-owned — the platform admin sets it on the tenant namespace and GAG operates within it without ever creating or mutating it (Q130). It is the hard cap: a tenant-authored quota was removed pre-1.0 because the tenant could simply raise it. CPU/Memory limits are also defined in the RunnerGroup CRD spec. Rogue workflows cannot exceed the tenant quota. Both the GMC and the AGC hold read-only (get/list/watch) access to resourcequotas so they can compute the proxy and worker quota conditions respectively (Q82); both deliberately hold no quota-write verb, keeping least privilege intact (partially subsumes Q122) — they observe the quota but can never create or relax it. The AGC additionally surfaces WorkerQuotaPressure (warning — workers can't scale to maxWorkers within headroom) and WorkerQuotaExceeded (error — the quota can't admit another worker pod) on each RunnerGroup, complementing Q59's configured-ceiling admission gate. |
| Cross-Tenant Pod Preemption via PriorityClass | High | priorityClassName stamps a cluster-scoped PriorityClass, so the platform owns which classes a tenant may reference: the GMC webhook rejects any name not on --allowed-priority-classes (empty = forbid all, the secure default; optional additive, fail-safe ConfigMap source). Both tenant-reachable routes are gated on every tenant-authored kind — priorityTiers[] (v1 ActionsGateway, v2 RunnerSet) and podTemplate.spec (v1 ActionsGateway, v2 RunnerTemplate) — the latter otherwise admitting system-cluster-critical from any namespace. Platform should create allowlisted classes preemptionPolicy: Never unless cross-tenant preemption is intended. Infra pods (EgressProxy/ActionsGateway spec.scheduling.priorityClassName) are gated by a separate, disjoint allowlist --allowed-infra-priority-classes (Q284). Closed (Q132 v1 tiers, Q289 podTemplate + v2 RunnerSet tiers; Q284 infra allowlist); ConfigMap source Q188. Full detail: see below. |
Supply-chain compromise of the worker image¶
WorkerImage SHOULD reference an immutable digest, not a floating tag (see §3.1). Digest pinning eliminates the "update the tag, get a different binary" attack class. Operators are expected to restrict the set of permitted registries via cluster admission policy (e.g. Kyverno, OPA Gatekeeper) — the GMC does not enforce this itself because registry policy is a cluster-wide concern. The gateway's own CI runs two supply-chain gates on every PR (security-scan.yml): govulncheck across all Go modules and trivy image scans of all five built images — see testing.md § Security scanning. The four images built from a minimal/distroless base block on fixable HIGH/CRITICAL findings; the default worker image (built FROM the upstream actions-runner) is scanned report-only because its CVEs live in upstream components, with base bumps automated via dependabot. Tenants supplying their own WorkerImage are still expected to scan it themselves. imagePullPolicy: IfNotPresent (digest) or Always (tag) ensures the kubelet does not serve a stale, possibly tampered local copy. For the four first-party images the chart resolves (gmc, and the AGC_IMAGE/PROXY_IMAGE/WRAPPER_IMAGE refs the GMC injects), digest pinning is enforced, not advisory, and it is enforced at chart render time: the Helm chart fails to render — naming the offending image — while any of the four digests is empty (the allowFloatingImageTags value is the shared dev/test opt-out), and the CI manifest-validate gate asserts each image's render is rejected, so the check cannot regress to fail-open (Q307). Render-time enforcement is the primary layer because the GMC's own image has no runtime guard at all (nothing validates the image the GMC runs from), and because catching an unpinned AGC/proxy/wrapper ref at render surfaces it as one clear error rather than a later GMC crash-loop. The AGC/proxy/wrapper images are additionally re-checked by the GMC at startup — it rejects any injected reference not in image@sha256:<digest> form (the same --allow-floating-image-tags opt-out) — a second layer that also covers deployments that bypass the chart. All five first-party images additionally carry org.opencontainers.image.* provenance labels (source/revision/version/title/description, with revision/version stamped from the build's git SHA via docker-bake.hcl) so SBOM scanners can trace an image back to its commit, and their Go binaries are compiled with -trimpath -ldflags=-buildid= for path-free, reproducible output (a reproducible-build input). On a v* release tag, publish.yml pushes those five images to GHCR as multi-arch OCI indexes (linux/amd64 + linux/arm64; the Go builder stages cross-compile on $BUILDPLATFORM, and the digest operators pin is the index digest) and signs each one keyless with cosign (sigstore/Fulcio via GitHub Actions OIDC — no long-lived signing key, no stored secret), recursively over the index and every per-arch manifest, and attaches an SPDX-JSON SBOM per architecture as a cosign attestation on that architecture's manifest, so a downstream operator can cosign verify the publish-workflow identity before deploying and enforce it cluster-wide via an admission policy. Each image also carries a signed SLSA build-provenance attestation (actions/attest-build-provenance, Q103) on the index digest via the same keyless Fulcio/Rekor path — authenticated provenance recording the workflow, repo, commit, and trigger (SLSA Build L2; buildx's unsigned default provenance is disabled in favour of it, and full Build L3 would require an isolated reusable-workflow builder). The PR-time security-scan.yml already generates each SBOM as a build artifact so that path can't silently break. The publish pipeline itself is hardened against upstream-tag hijack: every uses: across .github/workflows/ is pinned to a full commit SHA (the publish job holds id-token: write, so a mutable action tag repointed at malicious code could otherwise keyless-sign images as the release identity), runtime tool downloads are version-pinned (cosign via cosign-installer, syft via syft-version), and Dependabot's github-actions ecosystem bumps the SHA pins so they don't rot (Q123). The keyless signing identity is tags-only: publish.yml refuses to run from a non-tag ref and make verify-release anchors the cosign --certificate-identity-regexp to …/publish.yml@refs/tags/v.*$, so a signature minted from a branch (e.g. a workflow_dispatch from a scratch branch overwriting a released tag) is both prevented and rejected (Q124). See security-operations.md § Image provenance for the operator verification runbook and release.md § Supply-chain integrity of the pipeline for the SHA-pin / signing-identity policy.
GitHub-App-key exfiltration via AGC apiserver egress¶
The AGC pod holds the tenant's GitHub App private key, and its NetworkPolicy (buildAGCNetworkPolicy) admits egress on the Kubernetes API server ports (443 and 6443) with no to: destination restriction — so a compromised AGC could in principle exfiltrate the key over port 443 to an arbitrary external HTTPS endpoint, not just the apiserver. This breadth is the default, not an oversight: the kubernetes Service ClusterIP is DNAT'd by kube-proxy to provider-specific node/apiserver IPs before NetworkPolicy is evaluated, so a precise ipBlock is not portable and a wrong one silently severs the AGC's apiserver access (the post-DNAT trap that bit the proxy NP in PR #59) — so any-destination must stay the secure default where the post-DNAT apiserver IP is unpredictable. The blast radius is bounded by compensating controls: the App key is file-mounted read-only at 0440 (never an env var), worker pods carry no apiserver egress at all (only DNS + proxy), the AGC image is digest-pinned and the pod runs non-root with a read-only root filesystem, and all GitHub-bound AGC traffic still routes through the per-tenant proxy (the apiserver egress rule is scoped to apiserver ports, not GitHub). Operators whose platform exposes a stable apiserver endpoint CIDR can now close this residual with a first-class opt-in (Q145): set the GMC's --apiserver-cidrs flag (Helm value apiServerCIDRs) to that CIDR set and the AGC NetworkPolicy's 443/6443 rule is scoped to those CIDRs via ipBlock — a strict tightening, validated as CIDRs at GMC startup, ports unchanged. Leaving it empty preserves the any-destination default for clusters where the apiserver IP is unpredictable. Why this stays an opt-in and not an auto-tightened default (Q183 feasibility verdict): the GMC could in principle read the kubernetes Service endpoints and scope the rule itself, but those post-DNAT IPs rotate on managed control planes without notice — a snapshot goes stale and breaks the AGC; keeping it live needs an endpoint watch with an unavoidable race window during which the AGC (and the GMC, on the same path) can be locked out of the apiserver it needs to repair the policy; and some CNIs rewrite the target again. So automatic default-on tightening is not safe or portable, and narrowing stays operator-confirmed and per-cluster. See security-operations.md § Tightening AGC apiserver egress for per-cloud (EKS/GKE/AKS/kubeadm/kind) guidance and the full verdict, and appendix-g §G.10 for the deferred auto-narrowing enhancement.
DNS exfiltration side-channel containment¶
The per-tenant egress-IP attribution that isolates tenants rests on all real egress traversing the tenant proxy, whose source IPs are attributable. An unrestricted port-53 egress rule (to: [] ≡ any resolver) would defeat that: any pod — including untrusted worker job code — could smuggle data out by encoding it into DNS queries aimed at an attacker-controlled authoritative server, an unattributed side-channel that never touches the proxy. All three per-tenant NetworkPolicies (workload, AGC, proxy) therefore confine port-53 egress to cluster DNS only, via three OR'd peers: (1) the kube-dns / CoreDNS Service in kube-system (matched by namespaceSelector on kubernetes.io/metadata.name: kube-system plus podSelector on k8s-app: kube-dns); (2) the NodeLocal DNSCache pods k8s-app: node-local-dns in kube-system (same namespace+pod AND-selector), the redirect backend on CNIs that rewrite the kube-dns ClusterIP to a per-node cache pod — GKE Dataplane V2 (Cilium) does this via a RedirectService / Cilium Local Redirect, and enforces egress against that backend's identity (node-local-dns, not kube-dns), so without this peer DNS is silently dropped and the tenant's AGC crash-loops on its first GitHub token fetch (Q229); and (3) for clusters running NodeLocal DNSCache in the classic link-local mode, where pods send DNS to a per-node hostNetwork cache at a link-local address no selector can match, the link-local block 169.254.0.0/16 (ipBlock, Q136). All three peers preserve the attribution property: kube-dns/node-local-dns recurse upstream on the pod's behalf so legitimate resolution (including the proxy's own GitHub-hostname lookups) is unaffected, each is port-53 cluster DNS — never an arbitrary external resolver — and 169.254.0.0/16 is non-routable and node-scoped, so the exfiltration channel stays closed while the "any resolver" breadth remains removed (Q105/Q136/Q229). Like the other egress negatives, this is enforced only by a policy-aware CNI; the reliable CI guard is the authoring-level test TestBuildNetworkPolicy_DNSEgressRestrictedToKubeDNS, which asserts every policy's DNS rule selects kube-dns, node-local-dns, and the link-local block and is never open. The node-local-dns peer's runtime effect was verified end-to-end on a live GKE Dataplane V2 cluster (Q229).
Worker exfiltration via an allowlisted non-GitHub destination¶
By default the proxy carries GitHub only, so this risk does not exist; it appears only when a platform admin opts in to widening worker egress to specific non-GitHub destinations (EgressProxy.spec.destinationFQDNs / destinationCIDRs). The residual is honestly stated: a compromised worker (malicious dependency, job RCE) can exfiltrate to any allowlisted destination — the allowlist bounds the hole, it does not eliminate it; this is the core trade the feature accepts. It is never tenant-openable: the EgressProxy is tenant-authorable, so the destinations are gated by a platform-owned allowlist (GMC --allowed-egress-fqdns suffix match / --allowed-egress-cidrs subnet containment, optionally a watched ConfigMap) enforced by a validating webhook that rejects any off-allowlist entry — both empty = deny-all-non-GitHub, the secure default (mirrors --allowed-priority-classes, Q132/Q188). The properties the cheap alternatives throw away are preserved: traffic still exits the per-tenant proxy IPs (attribution intact) and DNS still resolves on the in-cluster path (no new exfil channel). Enforcement is dual-surface from one source of truth — CIDR ipBlock / FQDN toFQDNs on the pod-egress policy (the hard gate; fail-closed if the CNI can't enforce an FQDN rule, same as Q208) and the proxy CONNECT allowlist as defense-in-depth (with resolve-and-dial-the-validated-IP closing the DNS-rebinding window on the CIDR path; denials counted on actions_gateway_proxy_connect_denied_total). The admin footgun (a too-broad suffix/CIDR — *.googleapis.com, a CDN, the IMDS endpoint) is bounded by guidance, not code: the docs lead with an in-cluster caching mirror as the recommended path and reserve the allowlist for what a mirror genuinely can't proxy. See network-architecture.md § Worker egress to allowlisted non-GitHub destinations, security-operations.md § Worker egress destinations, and the Q242 plan for the full trade-off record.
Cross-tenant pod preemption via PriorityClass¶
A cluster-scoped PriorityClass carries a priority value and a preemptionPolicy (Kubernetes default PreemptLowerPriority), so an unvalidated tenant-chosen class would let a tenant name a high-priority, preempting class and have the scheduler evict other tenants' running worker pods — and their egress proxies — to schedule its own, defeating per-tenant isolation.
A tenant can reach a worker pod's priorityClassName by two routes, and the allowlist gates both — on every tenant-authored kind that carries them:
| Route | Kind | Gated by |
|---|---|---|
priorityTiers[].priorityClassName |
v1 ActionsGateway / RunnerGroup (Q132); v2 RunnerSet (Q289) |
v1 ActionsGateway webhook; v2 RunnerSet webhook |
podTemplate.spec.priorityClassName |
v1 ActionsGateway.runnerGroups[], v2 RunnerTemplate (Q289) |
v1 ActionsGateway webhook; v2 RunnerTemplate webhook |
The podTemplate route existed because it is a full corev1.PodTemplateSpec that the AGC copies verbatim into the worker pod, overriding priorityClassName only when a priorityTiers tier matches. Gating only the v1 tiers therefore left the guarantee unenforced: Kubernetes ships system-cluster-critical (value 2000000000, preemptionPolicy: PreemptLowerPriority) in every cluster and does not restrict it to kube-system — verified against a real apiserver — so a tenant could preempt every other tenant with no setup at all. The same applied to the v2 RunnerSet's own priorityTiers: the RunnerSet is tenant-authored (it is the v2 front door) and its matched tier is stamped over the template, so its tiers are gated in the RunnerSet webhook exactly as the v1 tiers are in the ActionsGateway webhook. All routes are now rejected at admission unless the class is on the platform allowlist.
The platform owns which classes a tenant may reference: the platform admin pre-creates the PriorityClass objects (the GMC never creates cluster-scoped objects — consistent with the Q121/Q122/Q130 platform-ownership model) and lists their names in the GMC --allowed-priority-classes flag; the GMC validating webhook rejects any priorityClassName not on the allowlist. The allowlist can additionally be sourced from a watched ConfigMap in the GMC namespace (--priority-class-allowlist-configmap, Q188) so an admin can add a class without a flag edit + restart — the ConfigMap is additive to the flag and fail-safe (a missing/malformed ConfigMap falls back to the static flag allowlist and never silently widens it), so the guardrail is preserved. An empty allowlist forbids every named class (secure default), so out of the box no tenant can set a PriorityClass at all; an unset podTemplate.spec.priorityClassName stays admissible, since the secure default must not forbid ordinary unprioritized workers.
The cluster-scoped ClusterRunnerTemplate is exempt: a tenant cannot create cluster-scoped objects, so gating it would only bind the platform against itself — the same reasoning that lets it declare privileged containers (§H.4/§H.6).
Because PriorityClass is global, the platform should create allowlisted classes with preemptionPolicy: Never unless cross-tenant preemption is genuinely intended for that tier — see security-operations.md § Priority classes. The dead tenant-settable per-tier preemptionPolicy field was removed pre-1.0 (it was never wired to pods and was a tenant-controlled preemption lever the platform must own). Closed (Q132 v1 tiers, Q289 podTemplate + v2 RunnerSet tiers); ConfigMap source added Q188. The webhook residual — RunnerGroup has no webhook of its own, so direct runnergroups RBAC bypassed the allowlist, and webhooks never re-validate stored objects — is closed by the priorityclass-allowlist-guard ValidatingAdmissionPolicy (G.7, Q289): it gates every runnergroups write from any principal against a paramKind ConfigMap allowlist and re-validates existing objects on update. The policy also matches the v2 carriers of the same two routes — runnersets (priorityTiers) and runnertemplates (podTemplate.spec.priorityClassName), both v2alpha1 and v2beta1 (Q323): those kinds have failurePolicy: Fail webhooks, so for them the policy is the webhook-outage/bypass backstop plus stored-object re-validation, restoring the defense-in-depth v1 had (ClusterRunnerTemplate stays exempt as platform-authored, matching its webhook exemptions). What remains outside any admission gate is a stored off-allowlist object that is never written again — the upgrade sweep finds those.
The two PriorityClass allowlists must stay disjoint (worker vs. infra, Q284)¶
priorityClassName is reachable from a second family of surfaces: spec.scheduling.priorityClassName on the tenant-authorable EgressProxy and v2 ActionsGateway, which prioritizes the infra pods (the per-tenant egress proxy pool and the AGC control plane). This is gated too — but by a separate allowlist, --allowed-infra-priority-classes, and the separation is a durable governance invariant, not an implementation detail:
- The two allowlists must be disjoint, and the GMC enforces it at startup (a boot-time intersection check; a shared class fails the GMC to boot). Reusing the worker allowlist is the non-obvious trap. Infra pods must sit above workers — that is the point of prioritizing them — so an operator would add a high, preempting class to let an
EgressProxyname it. An allowlist has one namespace of meaning, so that class would immediately be nameable from a worker pod (RunnerTemplate.podTemplate.spec.priorityClassName) as well. Any tenant could then lift its workers to infra priority and preempt other tenants' proxy pods: the gate meant to protect the proxy becomes the mechanism for evicting it, and the priority ordering inverts. Two disjoint allowlists make an infra class unreachable from any worker surface by construction. priorityClassNameis the onePodSchedulingfield behind a gate, and the asymmetry is principled. The rest of the block (nodeSelector/tolerations/affinity/topologySpreadConstraints) is tenant-settable and ungated because it is a choice about the tenant's own traffic — it weakens attribution (two tenants may egress via one IP), not isolation, and cannot preempt another tenant. Priority is a cluster-wide, cross-tenant preemption lever: distinct in kind, so gated where placement is not. And becausesystem-*classes are nameable from any namespace (above), an ungated infrapriorityClassNamewould be the same cluster-wide preemption escape the worker gate closes — reopened on a new path. Any futurePodTemplateSpec/priority passthrough on a tenant-writable CR must be audited for this.
The infra webhook carries the same residual as the worker one (G.7): direct RBAC on the underlying resource bypasses the webhook, and stored objects are never re-validated. A ValidatingAdmissionPolicy backstop for the infra allowlist, following the G.7 paramKind-ConfigMap pattern, is the planned closure. Closed (Q284): flag, disjointness check, and both webhooks (EgressProxy extended; a new v2 ActionsGateway webhook stood up, since none existed).
5.3. Security Profiles and the Privileged Opt-In¶
Worker pod security is defense-in-depth: PSA enforcement at the API server, AGC-enforced invariants on the PodSpec, and an optional sandbox runtime layer. The default posture is secure; tenants opt into looser policy explicitly.
The three profiles¶
ActionsGateway.spec.securityProfile is one of three values; the GMC
stamps the corresponding label on the tenant namespace.
| Profile | PSA label | Container escape risk | Typical use |
|---|---|---|---|
baseline (default) |
pod-security.kubernetes.io/enforce: baseline |
Low — privileged/host namespaces/hostPath/dangerous caps all blocked | Normal CI: builds, tests, integration runs |
restricted |
pod-security.kubernetes.io/enforce: restricted |
Very low — adds runAsNonRoot, drop ALL caps, seccomp RuntimeDefault | High-isolation tenants; compliance workloads |
privileged |
pod-security.kubernetes.io/enforce: privileged |
High — admission imposes no restrictions | DinD, Buildah-without-sandbox, kernel-module workflows |
The default is baseline. A tenant must explicitly set
securityProfile: privileged on the ActionsGateway to allow
privileged worker pods — there is no silent path to it. And setting it
is necessary but not sufficient: privileged is only eligible in a
namespace a platform administrator has opted in — see
Privileged eligibility is a platform decision
below.
In-runner image builds (docker build, the most common heavyweight runner
workload) are where this profile choice bites hardest. The
In-runner image builds operator
guide maps each build approach — BuildKit-rootless, Kaniko, Sysbox, and
privileged Docker-in-Docker (DinD) — to the profile it needs, so most
builds land on baseline rather than privileged.
Privileged eligibility is a platform decision¶
The tenant owns the ActionsGateway CR, so a tenant can self-select
securityProfile: privileged at create — only downgrades of an
existing CR are otherwise gated (see
No silent profile downgrades below).
That left a self-granted-escalation gap: privileged makes the GMC
stamp the namespace PSA to privileged, the cluster's
least-restrictive pod-security posture, so a tenant who could create an
ActionsGateway could grant their own namespace that posture at
will (Q133).
Whether a namespace may run privileged workers is a platform
decision, not a tenant one. The GMC validating webhook therefore
rejects any ActionsGateway requesting securityProfile: privileged
— at create or update — unless its namespace carries the label
applied by a platform administrator. This is the same trust model as
the actions-gateway.github.com/tenant marker (see
Constraining the GMC's PSA-stamping privilege):
a label on the namespace, an object the tenant does not own and
cannot edit, set by a trusted identity. The GMC never sets it itself.
The granting value is the enum keyword allowed, not true,
deliberately: a boolean-looking label value invites the YAML coercion
footgun (privileged-profile: true parses as a boolean, which a string
label value then rejects or mishandles — and YAML 1.1 coerces
yes/no/on/off too), so a non-boolean keyword is both safer to
author and self-documenting. The value is matched exactly. This is the
project-wide convention for new operator-set labels and annotations —
see
Kubernetes API conventions.
The gate is fail-closed: an absent label, any value other than
allowed, or a namespace the webhook cannot read all leave privileged
ineligible and the request is rejected. Non-privileged profiles
(baseline, restricted, and the empty default) never consult the
label and are unaffected.
The two privileged gates are independent and both must pass: the
namespace must be labelled eligible (this check), and on an update the
rank-downgrade guard still requires the
actions-gateway.github.com/allow-profile-downgrade annotation
(anything → privileged is a downgrade in rank). The label is the
platform's eligibility decision; the annotation is the tenant's
deliberate, auditable act of relaxing their own profile.
This is a webhook check rather than a CRD CEL XValidation rule
because the decision depends on a label of a different object (the
namespace), which a spec-scoped CEL rule cannot read. The webhook reads
the namespace's current label through the uncached API reader, so a
tenant cannot smuggle eligibility in through the CR. Operators grant
eligibility per
Tenant Onboarding; a tenant who
hits the rejection is pointed at
troubleshooting: privileged profile rejected.
Constraining the GMC's PSA-stamping privilege¶
Stamping a PSA label on a namespace requires patch on namespaces,
which is cluster-scoped — RBAC cannot express "only namespaces the GMC
manages". Left unconstrained, a compromised GMC pod could relabel
kube-system (or any namespace) to privileged. Two controls confine
the grant:
- A trusted marker label. A namespace is eligible for GMC
management only if an administrator has labelled it
actions-gateway.github.com/tenant: "true". This is a tenant onboarding pre-condition (see Tenant Onboarding). The GMC never sets this label itself — doing so would defeat the control. - The
namespace-psa-guardValidatingAdmissionPolicy. Scoped to the GMC ServiceAccount only, it denies any namespaceUPDATEunless the existing namespace already carries the marker (read fromoldObject, which the requester cannot forge), and denies any change to a namespace label other than the sixpod-security.kubernetes.io/*keys or to any annotation. It ships in the Helm chart (charts/actions-gateway/templates/namespace-psa-guard.yaml, gated onadmissionPolicy.enabled).
The policy deliberately does not ban writing privileged outright,
because securityProfile: privileged is a supported per-tenant opt-in
and the GMC legitimately stamps it on those tenants' namespaces. The
marker scope confines the blast radius to GMC-managed tenant
namespaces.
No silent profile downgrades¶
Separately from the GMC's stamping privilege, the GMC validating
webhook (ValidateUpdate) prevents a tenant's profile from being
silently weakened. The profiles are ranked
privileged(0) < baseline(1) < restricted(2); on update the webhook
compares the old and new rank:
- An upgrade (
baseline → restricted) — hardening — is always allowed. So is a no-op change. - A downgrade (
restricted → baseline, or anything →privileged) is rejected unless the object carries the annotationactions-gateway.github.com/allow-profile-downgrade: "true".
This closes the accidental path without trapping operators. A stray
kubectl apply of an older manifest — or one that drops the field and
lets it re-default to baseline — does not carry the annotation, so it
is refused rather than quietly relaxing isolation (an empty value is
treated as its baseline default for the comparison, so dropping the
field cannot sneak a downgrade through). But a deliberate relaxation —
for example rolling back a baseline → restricted hardening attempt that
turned out to break the tenant's pods at admission — needs only a
two-field edit (set the annotation, set the profile), not a destructive
recreate of the whole ActionsGateway. Requiring the explicit annotation
keeps the relaxation auditable while keeping the safe direction
(hardening) cheap and the unsafe direction (relaxing) intentional.
The check is a webhook rather than a CRD CEL XValidation rule because
the decision depends on metadata.annotations, which a spec-scoped CEL
rule cannot read. (gitHubAppRef.name is deliberately left mutable:
changing it is the supported credential-rotation mechanism — see §3.2.)
This is a guard against accidental/silent downgrade, not an absolute
boundary: an operator who holds the allow-profile-downgrade annotation
write (i.e. edit access to the CR) is trusted to relax the profile on
purpose, and one with direct namespace patch rights could edit the PSA
labels regardless. A compromised GMC relabelling namespaces is a
separate threat, constrained by the namespace-psa-guard
ValidatingAdmissionPolicy above, not by this rule.
Mixing privileged and non-privileged workloads¶
PSA enforcement is namespace-scoped: every pod in a namespace is
evaluated against the same profile. A tenant that needs both
privileged and non-privileged workloads deploys two
ActionsGateway CRs in two namespaces — for example,
myteam-builds with securityProfile: privileged for DinD jobs and
myteam-tests with the default baseline for everything else.
Workflows route to the appropriate gateway via runs-on: labels
matching RunnerGroups in each.
This is the same separation operators already use to assign different quotas, priority tiers, and node selectors to different workload classes — the security profile rides on the existing namespace boundary rather than introducing a new sub-namespace concept.
If finer granularity (per-RunnerGroup profile within one
ActionsGateway) becomes necessary, the path forward is documented
in Appendix G as a future
enhancement.
Pairing privileged with a sandbox runtime¶
Selecting privileged removes the API-server-side admission guard
but does not remove the option of sandbox-based isolation. For
tenants who need privileged semantics (a real Docker daemon, full
syscall surface) but don't trust the workload code, the recommended
pattern is:
apiVersion: actions-gateway.github.com/v1alpha1
kind: ActionsGateway
metadata:
name: builds
namespace: myteam-builds
spec:
securityProfile: privileged
runnerGroups:
- runnerLabels: [self-hosted, dind]
podTemplate:
spec:
runtimeClassName: kata-containers # or gvisor
containers:
- name: runner
securityContext:
privileged: true
runtimeClassName: kata-containers runs the worker pod inside a
lightweight VM. Privileged-inside-Kata grants the workload full
control of a microVM kernel, not the host kernel — a container escape
lands in a throwaway guest kernel rather than on the node. See
Appendix B for the full tradeoff
between runc, gvisor, and kata-containers. (The RuntimeClass
name is whatever the platform admin registered; the Q226 reference
architecture registers kata → handler kata-qemu.)
Kata bounds the kernel, not the pod network. A micro-VM does not
isolate the cloud metadata server: Q226 measured the node's GCE
service-account token endpoint returning HTTP 200 from inside a Kata
guest on GKE. Escaping the container is no longer a path to the node, but
minting node credentials over the pod network still is. Kata must
therefore be paired with a metadata control (GKE Workload Identity, AWS
IMDSv2 hop-limit 1, or a NetworkPolicy denying 169.254.169.254/32) and
automountServiceAccountToken: false — which the AGC already sets
unconditionally on worker pods. Kata is one layer, not a posture; the
capabilities a DinD workload still needs inside the guest approach
privileged, so the whole guarantee rests on the VM boundary. See
Running DinD workloads under Kata § What Kata does not buy you.
This pairing is a tenant-level decision: the platform team can
recommend it via policy and documentation, but cannot enforce it
from the GMC alone (the runtimeClassName field lives in the
PodTemplate, owned by the tenant).
Privileged worker containers are admitted only under the privileged profile¶
The GMC validating webhook (validateRunnerGroups) rejects a
securityContext.privileged: true container or init-container in any
RunnerGroup's PodTemplate — except when the ActionsGateway has
explicitly set securityProfile: privileged. Under baseline (the
default) and restricted, a privileged container is refused at admission,
secure by default with no silent opt-in. The check is keyed on the
effective profile, so an omitted/empty securityProfile is treated as
baseline and still rejects.
This makes the webhook coherent with the rest of the profile model: the
privileged profile stamps the namespace PSA to privileged so the pod
is actually admittable, and the documented Kata/DinD pattern above (a
privileged: true runner under securityProfile: privileged) is now
accepted end to end rather than being blocked by the webhook while PSA
would have allowed it. The webhook covers only the GMC-managed
ActionsGateway path; a directly-applied RunnerGroup CR bypasses the
webhook entirely, so Pod Security Admission — stamped per the
namespace's profile — is the real enforcement backstop for both paths.
Operators who hit the rejection are pointed at
troubleshooting: privileged worker container rejected.
Floor invariants apply at every profile¶
The AGC enforces the following on every worker pod regardless of profile, by overwriting the merged PodSpec before submission:
Spec.HostPID = falseSpec.HostNetwork = falseSpec.HostIPC = falseSpec.AutomountServiceAccountToken = falseSpec.ServiceAccountName = <worker SA>(no K8s API credentials projected)- Reserved env vars (
HTTP_PROXY,HTTPS_PROXY,NO_PROXY, the payload mount path) are stamped by the controller
A tenant who sets securityProfile: privileged still cannot enable
host namespace sharing or expose Kubernetes API credentials inside
the worker pod. These invariants are non-negotiable across all
profiles.
Secure-by-default pod SecurityContext and resources¶
Floor invariants above are non-negotiable. On top of them, the AGC
stamps a secure-by-default SecurityContext and resource requests/limits
onto every worker pod. These defaults gap-fill only — an explicit
value in the tenant PodTemplate always wins, so a tenant can opt out of
any individual default (e.g. runAsNonRoot: false for a root-based image)
without escalating to the privileged profile.
The hardening scales to the namespace's PSA profile (propagated to the AGC
via the SECURITY_PROFILE env var the GMC sets on the AGC Deployment):
| Profile | SecurityContext defaults stamped |
|---|---|
baseline (default) |
Pod-level runAsNonRoot: true + runAsUser: 1001 + seccompProfile: RuntimeDefault. Deliberately not allowPrivilegeEscalation: false or capability drop — baseline PSA permits in-job privilege escalation (sudo) and many CI jobs rely on it. |
restricted |
The above, plus the per-container PSA-restricted floor: allowPrivilegeEscalation: false and capabilities.drop: [ALL]. Without these the namespace's PodSecurity admission would reject the pod, so the AGC stamps them to make the profile usable rather than self-blocking. |
privileged |
None — this profile exists precisely so DinD / host-capability workloads can opt in via their own PodTemplate. |
Why runAsUser: 1001 accompanies runAsNonRoot: true. kubelet's
runAsNonRoot enforcement can only prove a container is non-root against a
numeric UID. The default worker image
(ghcr.io/actions/actions-runner, and the cmd/worker image built from it)
declares its user by name — USER runner — which kubelet cannot resolve to
a UID at admission. With runAsNonRoot: true but no numeric UID, kubelet
rejects the pod outright (CreateContainerConfigError: container has
runAsNonRoot and image has non-numeric user), so an unmodified RunnerGroup
would fail every job. The AGC therefore gap-fills the runner image's own UID
(1001) whenever non-root is being enforced, letting kubelet verify non-root
without changing which user the runner actually runs as. The gap-fill is skipped
when a tenant sets runAsNonRoot: false (a root-based image), so it never
contradicts an explicit opt-out, and an explicit runAsUser always wins. (Q115)
readOnlyRootFilesystem is not defaulted on any profile: the GitHub
Actions runner writes to its work, diagnostics, and home directories at
runtime, so a read-only root would break essentially every job, and it is
not part of the PSA restricted floor. Tenants who can run with a
read-only root may set it (plus the writable emptyDir mounts the runner
needs) explicitly in their PodTemplate.
Resource requests and limits default to 500m CPU / 1Gi memory on
every profile when the tenant container declares neither. This moves a
worker pod off Best-Effort QoS — the first thing the kubelet evicts under
node pressure, which otherwise burns the eviction-retry budget fast. A
single-container worker pod with the defaults is Guaranteed QoS.
5.7. Workload identity: the no-PEM delegation model¶
A gateway authenticates to GitHub as a GitHub App by signing a short-lived App
JWT and exchanging it for an installation token. There are two trust models for
where the App private key lives, expressed as the two members of the
spec.credentials discriminated union (appendix-h §H.15):
githubApp— the possession model (default, Q196). The App's RSA private key lives at rest in a namespaceSecret, mounted into the AGC, which signs the JWT in-process. This is the secure-by-default option: the key is confined to the tenant namespace under the GMC's Secret-read controls, and admission/RBAC are unchanged from v1.workloadIdentity— the delegation model (Q197). No App private key is ever in the cluster. The AGC holds only the non-secret App identity (appId/installationId) and a reference to an external signer. It proves its own pod identity to that signer's trust anchor and the anchor signs the App JWT on its behalf. The AGC never reads, holds, mounts, or logs the App key — there is noprivateKeyfield on this member by construction.
workloadIdentity is on the strict-upgrade direction of the secure-by-default
principle (it removes a stored credential), so it is offered as an explicit
opt-in member; the in-cluster-PEM githubApp stays the default. Choosing
workloadIdentity regresses no property: the GitHub token-exchange channel still
requires HTTPS, the App JWT still carries a jti replay defense and a 10-minute
expiry, and the union CEL still enforces exactly one credential member.
MVP signer — HashiCorp Vault transit + Vault Kubernetes auth. The first external signer is Vault transit, chosen because it is kind-validatable end to end without a cloud account. The flow holds no long-lived secret in the cluster:
- The AGC reads its kubelet-projected ServiceAccount token fresh from disk —
a short-lived token minted by the kubelet, not a stored
Secret. - It presents that token to Vault Kubernetes auth (
POST auth/<mount>/loginwith{role, jwt}); Vault verifies it via the clusterTokenReviewAPI and returns a short-lived Vault client token, cached only in memory and re-fetched on lease expiry. - It asks Vault transit (
POST <transit>/sign/<key>) to sign the App JWT'sheader.payloadwith an RSA key Vault holds — RSASSA-PKCS1-v1_5 over SHA-256, i.e. JWSRS256. The private key never leaves Vault.
The signing material's location is abstracted behind a githubapp.Signer
interface (JWTAlg/Sign), so cloud KMS/HSM signers (AWS/GCP/Azure) add as new
implementations + new signer.provider union members without another breaking
change. No code path logs, returns in an error, or env-passes the projected
ServiceAccount token, the Vault client token, or the produced signature — Vault
error responses surface only operational messages (permission denied) and HTTP
status. The Vault address is HTTPS-by-default (the ServiceAccount token transits
it at login); a plaintext address is permitted only under an explicit dev/test
opt-in, mirroring the GitHub-API-base-URL rule above.
Operator configuration is in
tenant-onboarding.md § Workload-identity credentials.
The GMC provisions a workload-identity AGC end to end (Q201): it stamps the signer
config env, projects a Vault-audience-scoped ServiceAccount token volume (mounted
read-only; never an env var), and runs the AGC under its per-gateway ServiceAccount —
the in-cluster identity the operator binds to a Vault Kubernetes-auth role out of
band. The AGC's vaultsigner provider then logs in to Vault with that projected
token and delegates the App-JWT sign to Vault transit, so the gateway authenticates
to GitHub with no App key in the cluster. The live no-PEM round-trip is covered by an
in-cluster (dev-mode) Vault kind e2e.
NetworkPolicy egress to Vault (Q202). The per-tenant AGC NetworkPolicy default-denies egress except DNS + GitHub + the kube API. Vault's
addressis an opaque URL, not a selectable peer, so the GMC takes the peer fromsigner.vault.networkPolicy: a pod/namespace selector (in-cluster Vault) or a CIDR (external Vault). When set, the GMC adds a scoped AGC→Vault egress rule — that one peer on the Vault API port parsed fromaddress— to the per-gateway AGC NetworkPolicy only (worker pods never get it). It is emitted only forcredentials.type: WorkloadIdentitywith a Vault signer; a possession-model (githubApp) gateway keeps strict default-deny and the rule is a strict tightening that is only ever added, never a broaden-to-all-egress. LeavenetworkPolicyunset on a non-enforcing CNI (kindnet) or to manage the rule out of band — the egress posture is unchanged in that case. As with every egress negative this is enforced only by a policy-aware CNI.
signer.vault.networkPolicyis a sharedEgressPeer(selector | CIDR + optional explicitport) — the one descriptor future tenant-delegated egress holes (cloud KMS signers, telemetry endpoints) reuse so the v2 API freezes one consistent shape (Q204). For Vault theportis left unset and derived fromaddress; the secure default is unchanged — still a single scoped peer, never a broaden-to-all-egress. See appendix-g §G.9.
← Operational Flows | Back to index | Next: Implementation Phases →