Skip to content

Tenant Onboarding Checklist

Audience: Platform engineer

This checklist walks from pre-conditions through first successful job. For the full setup reference, see Getting Started. For day-2 operations after onboarding, see the Runbook. Coming from Actions Runner Controller (ARC)? Start with the Migrating from ARC guide — it maps ARC scale-set concepts onto the steps below and walks one runner group across.

New tenants: onboard on the v2 API

Steps 0–3 (GitHub App, Secret, ResourceQuota) are the same for every tenant. At the gateway step, the recommended shape for a new tenant is the v2 API (actions-gateway.com/v2alpha1) — see v2 API below and the getting-started v2 walkthrough. The single-CR v1alpha1 flow in Step 2 is still fully supported but deprecated; already on v1? gag-migrate moves you across.

Pre-Conditions

Before beginning, confirm all of the following:

  • [ ] Namespace exists and is marked as a managed tenant. The tenant's Kubernetes namespace has been created and carries the marker label actions-gateway.github.com/tenant=true: 
    kubectl create namespace <tenant-namespace>   # if it does not exist yet
kubectl label namespace <tenant-namespace> actions-gateway.github.com/tenant=true
    This label is what authorizes the GMC to operate in the namespace at all. Two admission policies key on it: namespace-psa-guard denies the GMC any namespace patch (the PSA-stamping step) it has not been marked for, and gmc-tenant-resource-guard denies the GMC any create/update/delete of tenant resources (Deployments, Secrets, RoleBindings, NetworkPolicies, …) outside marked namespaces. So an unlabelled namespace leaves the ActionsGateway stuck with a NamespaceMarkerMissing warning event and no provisioned resources. Apply the label with a trusted (administrator) identity — the GMC must never set it itself. Verify: kubectl get namespace <tenant-namespace> -o jsonpath='{.metadata.labels.actions-gateway\.github\.com/tenant}' → true.
  • [ ] Cluster CNI enforces egress NetworkPolicy. The tenant isolation model (workers restricted to DNS + the per-tenant proxy; no direct GitHub or Kubernetes API egress) is implemented as NetworkPolicy egress rules, which are inert unless the cluster's Container Network Interface (CNI) plugin enforces them. Production clusters must run an egress-enforcing CNI such as Calico or Cilium — kind's default kindnet, for example, accepts NetworkPolicy objects without enforcing egress. Verify with your CNI's documentation, or run the negative probes in network-architecture.md § How to Validate Network Isolation after onboarding: the "blocked" probes must actually time out.
  • [ ] Service mesh is accounted for (if the cluster runs one). A mesh (Istio, Linkerd, Cilium Service Mesh, Kuma) that injects a sidecar into the tenant namespace will strand completed worker pods and fight the per-tenant egress proxy. The supported posture is to opt the GAG tenant namespace out of the mesh; if mesh membership is mandatory, use native sidecars or an ambient/sidecar-less data plane plus egress exclusions. Decide this before provisioning — see Running GAG Alongside a Service Mesh.
  • [ ] GMC is running. The Gateway Manager Controller (GMC) is deployed and healthy: kubectl get deploy -n gmc-system gmc-controller-manager. Install it with the actions-gateway Helm chart (helm install gag charts/actions-gateway -n gmc-system --create-namespace …).
  • [ ] CRDs are installed. kubectl get crd actionsgateways.actions-gateway.github.com && kubectl get crd runnergroups.actions-gateway.github.com.
  • [ ] GitHub App is registered. The GitHub App is registered in the target GitHub organization with at least Actions: Read and Administration: Read permissions. The platform team has the appId, installationId, and private key .pem file. First time? Step 0 walks through creating the App and capturing all three.
  • [ ] GitHub App is installed. The App is installed on the organization (or specific repos): Settings → Developer settings → GitHub Apps → <app> → Install App.
  • [ ] GitHub URL is known. The org/enterprise/repo URL the runners register against — https://github.com/<org>, https://github.com/<org>/<repo>, or a GitHub Enterprise Server URL https://ghes.example.com/<org>. It goes in spec.gitHubURL (Step 2) and must match where the App is installed. It is a required field — there is no default.
  • [ ] Quota is provisioned (platform-owned). The tenant's resource requirements have been reviewed and the platform has created a ResourceQuota (and any LimitRange) on the tenant namespace — CPU, memory, and pod count. This is the real, tenant-uncontrollable cap; the gateway operates within it but never creates or mutates it. See Step 1b. (If you provision namespaces and quotas via a GitOps or tenant-operator stack — Capsule, HNC, vCluster, kiosk — the quota comes from there instead.)
  • [ ] PriorityClass objects exist and are allowlisted (priority-tiered tenants only). Any priorityClassName a tenant references in priorityTiers must (1) be pre-created at the cluster level by the platform (kubectl get priorityclass) and (2) appear on the GMC --allowed-priority-classes flag. The GMC validating webhook rejects any priorityClassName not on the allowlist (an empty allowlist rejects all of them) — this stops a tenant naming a high-priority, preempting class and evicting other tenants' worker pods. Create allowlisted classes with preemptionPolicy: Never unless cross-tenant preemption is genuinely intended for that tier; see security-operations.md § Priority classes.
  • [ ] Cluster service CIDR is known. Needed if the tenant's noProxyCIDRs must be customized: kubectl cluster-info dump | grep -m1 service-cluster-ip-range.
  • [ ] Security profile decided. Default baseline is correct for normal CI workloads (builds, tests). Confirm with the tenant whether they need restricted (compliance / high-isolation) or privileged (docker-in-docker, kernel-module workflows). Tenants with both needs deploy two ActionsGateway CRs in two namespaces. You can harden a profile later in place (baseline → restricted) freely, but relaxing it (a downgrade) is rejected by admission unless you set the actions-gateway.github.com/allow-profile-downgrade: "true" annotation — see troubleshooting: securityProfile downgrade rejected. See §5.3 — Security Profiles.

v2 API (actions-gateway.com): the security profile is no longer a field on the ActionsGateway CR — it is a property of the namespace, because Pod Security Admission is namespace-scoped (appendix-h §H.16 #7). Instead of spec.securityProfile, label the namespace:

kubectl label namespace <tenant-namespace> actions-gateway.com/security-profile=restricted   # baseline | restricted | privileged; absent ⇒ baseline

The GMC stamps the pod-security.kubernetes.io/* labels from it. The downgrade and privileged-eligibility rules are identical to v1 but enforced by the gmc-namespace-security-profile-guard ValidatingAdmissionPolicy on the namespace (downgrade needs actions-gateway.com/allow-profile-downgrade: allowed as a namespace annotation; privileged needs actions-gateway.com/privileged-profile: allowed as a namespace label). Co-located v2 gateways share the one namespace profile; tenants needing different postures use different namespaces. - [ ] Privileged eligibility granted (only if the tenant needs privileged). securityProfile: privileged is a platform decision, not tenant-settable: the GMC validating webhook rejects it (at create and update) unless the namespace carries the eligibility label, applied by a trusted administrator:

kubectl label namespace <tenant-namespace> actions-gateway.github.com/privileged-profile=allowed

The granting value is the enum keyword allowed, not true — a boolean-looking label value is a YAML footgun. This is a separate gate from the actions-gateway.github.com/tenant marker (which authorizes GMC management at all): without privileged-profile, the tenant can still run baseline/restricted but not privileged. The gate is fail-closed — absent the label (or any value other than allowed), privileged is refused — so a tenant cannot self-grant the cluster's least-restrictive PSA posture by creating a CR. Apply this only for tenants approved for docker-in-docker / kernel-module workloads, ideally paired with a sandbox runtimeClassName. Verify: kubectl get namespace <tenant-namespace> -o jsonpath='{.metadata.labels.actions-gateway\.github\.com/privileged-profile}' → allowed. To revoke, remove the label (…/privileged-profile-). See troubleshooting: privileged securityProfile rejected and §5.3.

Step 0: Create and Install the GitHub App

First-time setup. Skip this step if the platform team already handed you an appId, an installationId, and the private-key .pem file (the "GitHub App is registered" pre-condition). Otherwise, this is where those three values come from. The output of this step feeds directly into Step 1.

GitHub Apps are the gateway's only credential model on the v1 API — there is no Personal Access Token (PAT) path. This is deliberate: an App yields short-lived, auto-rotating installation tokens scoped to the installation and to the App's declared permissions (Actions: Read + Administration: Read here), so a compromise has a far smaller blast radius than a long-lived PAT carrying its owner's full account access. The App is also an automation-owned identity — it does not break when a user leaves the org or loses access — and installation-level rate/concurrency budgets are what let one tenant scale to thousands of sessions (and shard across installations; see Appendix E §E.6). See security §5 for the full credential trust model.

There is no gh command to create a GitHub App (the GitHub CLI has no gh app create), so the App is created in the web UI; the GitHub CLI (gh) is used afterwards to read back the IDs.

0a. Create the App (web UI)

  1. Go to the org's App settings: https://github.com/organizations/<org>/settings/apps → New GitHub App. (For an enterprise, use the enterprise settings path; for a user-owned App, Settings → Developer settings → GitHub Apps → New GitHub App.)
  2. GitHub App name — any unique name, e.g. acme-actions-gateway.
  3. Homepage URL — any valid URL (e.g. the repo URL); it is not used by the gateway.
  4. Webhook — uncheck Active. The gateway polls GitHub; it does not receive webhooks, so no webhook URL or secret is needed.
  5. Permissions — grant only the two read-only permissions the gateway needs to register and observe self-hosted runners:
  6. Repository permissions → Actions: Read-only
  7. Organization permissions → Administration: Read-only (labelled Self-hosted runners: Read-only on some plans)
  8. These are the Actions: Read and Administration: Read permissions referenced throughout this guide. Leave Contents, Pull requests, and everything else at No access — the gateway never reads tenant code or writes to repositories.
  9. Where can this GitHub App be installed? — Only on this account is the typical choice for a single-org deployment.
  10. Click Create GitHub App.

Least privilege. The installation tokens the gateway mints inherit exactly these App permissions. Granting more than Actions: Read + Administration: Read widens the blast radius of a key compromise for no functional gain — see security §5 and the runbook key-compromise scope assessment.

0b. Capture the appId

On the App's General page, copy the numeric App ID (top of the page). This is appId — a small integer, not the App name or the client ID.

0c. Install the App and capture the installationId

  1. On the App's Install App tab, install it onto the target organization (or specific repositories). The org/repos you install it on must match the gitHubURL you set in Step 2.
  2. After installing, the browser lands on the installation's settings page; the installationId is the trailing number in the URL: https://github.com/organizations/<org>/settings/installations/<installationId>.

You can also read it back with the GitHub CLI authenticated as the App (advanced — requires an App JWT, not your user token). The web-UI URL above is the reliable path; the gh api /app/installations endpoint is available once you can present an App JWT.

0d. Generate and download the private key (.pem)

On the App's General page → Private keys → Generate a private key. The browser downloads a .pem file once — GitHub never shows it again, so store it safely and treat it as a high-value secret.

The exact PEM format the controller expects. The AGC parses the key with Go's standard library and accepts exactly two PEM block types (githubapp/auth.go):

First line of the .pem Format Accepted?
-----BEGIN RSA PRIVATE KEY----- PKCS#1 (RSA) ✅ — this is what GitHub downloads
-----BEGIN PRIVATE KEY----- PKCS#8 (RSA or Ed25519) ✅ — e.g. after openssl pkcs8 conversion
-----BEGIN OPENSSH PRIVATE KEY-----, -----BEGIN EC PRIVATE KEY-----, anything else other ❌ — rejected with unsupported PEM block type

GitHub's downloaded key is already PKCS#1 — use it byte-for-byte; no conversion is required or recommended.

PEM pitfalls — the #1 first-day failure. The strict PEM format makes hand-editing the key the most common onboarding error. The controller surfaces it as private key: RSA key parse error / no PEM block found and AGCAvailable=False (reason CredentialError). Avoid all of these by never opening or retyping the key:

  • Do not copy-paste the key into a terminal or YAML by hand. Pasting can drop the trailing newline, re-wrap the base64 body, insert spaces/blank lines, or substitute smart-quotes — any of which breaks parsing. Always feed GitHub's file directly (Step 1's --from-file).
  • Do not open it in an editor that rewrites line endings. A CRLF (\r\n) conversion or a stripped final newline corrupts the block. Keep the file exactly as downloaded.
  • Do not base64-encode it yourself. kubectl ... --from-file and stringData encode the value for you; pre-encoding double-encodes it.
  • Header/footer must be intact and exact. The -----BEGIN …----- / -----END …----- lines must be present and unaltered, with no leading/trailing whitespace or extra blank lines.

You now have the three values Step 1 needs: appId, installationId, and the .pem file on disk (referred to below as app.pem).

Step 1: Create the GitHub App Secret

Create this in the tenant's namespace. Use a stable, versioned name (e.g. github-app-v1) to enable clean credential rotation later. The Secret holds three keys — appId, installationId, and privateKey — which the GMC projects into the AGC pod as files the controller reads at startup (cmd/agc/main.go).

Recommended — create from the downloaded file (secure). Pass the .pem straight into kubectl with --from-file; this preserves the key byte-for-byte (sidestepping every PEM pitfall above) and never exposes it in a shell argument, an environment variable, or your shell history. Delete the file as soon as the Secret exists:

# app.pem is the file downloaded in Step 0d — never echo, cat, or paste its contents.
kubectl create secret generic github-app-v1 \
  --namespace <tenant-namespace> \
  --from-literal=appId='<GitHub App ID>' \
  --from-literal=installationId='<Installation ID>' \
  --from-file=privateKey=app.pem

# Remove the private key from disk now that it lives only in the Secret.
rm -f app.pem

Why not paste the key into YAML? The declarative form below requires putting the PEM body into a file on disk that is easy to accidentally commit to git, and inviting a hand-paste that mangles the key. Prefer --from-file. If you must use YAML (e.g. GitOps), keep the manifest out of version control or supply the privateKey via your secrets manager / sealed-secrets tooling — never commit a plaintext key.

Alternative — declarative manifest. Equivalent to the command above; the --from-file key name (privateKey) maps to the stringData key here:

apiVersion: v1
kind: Secret
metadata:
  name: github-app-v1
  namespace: <tenant-namespace>
type: Opaque
stringData:
  appId: "<GitHub App ID>"
  installationId: "<Installation ID>"
  privateKey: |
    -----BEGIN RSA PRIVATE KEY-----
    <contents of the .pem file>
    -----END RSA PRIVATE KEY-----

Both PKCS#1 (-----BEGIN RSA PRIVATE KEY-----, the format GitHub downloads) and PKCS#8 (-----BEGIN PRIVATE KEY-----, RSA or Ed25519) are accepted — paste whichever your .pem file contains, header and footer lines included.

kubectl apply -f secret.yaml

Verify: 

kubectl get secret github-app-v1 -n <tenant-namespace>
kubectl get secret github-app-v1 -n <tenant-namespace> \
  -o jsonpath='{.data.privateKey}' | base64 -d | head -1
# Expected: -----BEGIN RSA PRIVATE KEY----- (PKCS#1)
#      or: -----BEGIN PRIVATE KEY----- (PKCS#8, RSA or Ed25519)

If the AGC later reports CredentialError / RSA key parse error, the key was altered in transit — see troubleshooting: GitHub App Secret misconfiguration.

Step 1b: Set the Platform-Owned ResourceQuota

The namespace ResourceQuota (and any LimitRange) is platform-owned — it is not a field on the ActionsGateway CR. The platform admin creates and manages it on the tenant namespace, and the gateway operates within it but never creates or mutates it. This is deliberate: a tenant-authored quota would be no real cap (the tenant could raise it in their own CR), and owning quotas would force broad, cluster-wide write RBAC on the GMC. Apply it with a trusted (administrator) identity:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: <tenant>-quota
  namespace: <tenant-namespace>
spec:
  hard:
    requests.cpu: "20"
    requests.memory: "40Gi"
    pods: "50"

kubectl apply -f resourcequota.yaml

If you already provision namespaces and quotas through a GitOps pipeline or a tenant operator (Capsule, HNC, vCluster, kiosk), set the quota there instead — the gateway will not fight it.

Size the quota for both pools at full scale. The quota must leave room for the proxy pool at spec.proxy.maxReplicas and worker pods up to maxWorkers (each × its per-pod requests/limits, plus pod count). When the remaining headroom can't cover scaling to those ceilings, the gateway flags it without blocking provisioning: the GMC raises ProxyQuotaPressure/ProxyQuotaExceeded on the ActionsGateway and the AGC raises WorkerQuotaPressure/WorkerQuotaExceeded on each RunnerGroup (each also exported as a gauge for alerting). See troubleshooting: Proxy Pool Not Scaling and Jobs Failing Due to Namespace ResourceQuota Exhaustion.

Sizing calculator

The full-scale footprint is the proxy pool plus every RunnerGroup's worker pool, each at its configured ceiling. The same arithmetic the gateway uses for the quota-pressure conditions is what you size against:

pods            = proxy.maxReplicas + Σ_groups(maxWorkers) + 1   # +1 = AGC control-plane pod
requests.cpu    = proxy.maxReplicas × proxyReq.cpu    + Σ_groups(maxWorkers × Σ_containers req.cpu)
requests.memory = proxy.maxReplicas × proxyReq.memory + Σ_groups(maxWorkers × Σ_containers req.memory)
limits.cpu      = proxy.maxReplicas × proxyLim.cpu    + Σ_groups(maxWorkers × Σ_containers lim.cpu)
limits.memory   = proxy.maxReplicas × proxyLim.memory + Σ_groups(maxWorkers × Σ_containers lim.memory)

Where the inputs come from:

Term Source Default (if unset)
proxy.maxReplicas spec.proxy.maxReplicas on the ActionsGateway 10
proxyReq.cpu / proxyReq.memory spec.proxy.resources.requests (per proxy pod) 10m / 32Mi
proxyLim.cpu / proxyLim.memory spec.proxy.resources.limits (per proxy pod) 500m / 64Mi
maxWorkers spec.runnerGroups[].maxWorkers (per group) unbounded — set it, or the worker pool has no ceiling to size against
Σ_containers req.* / lim.* sum of requests/limits across every container in that group's podTemplate.spec.containers none — workers carry only what the pod template sets

Notes:

  • The + 1 pod is the AGC control-plane pod. On the v1 (actions-gateway.github.com) API it stamps no CPU/memory requests or limits, so it consumes one pods slot but contributes nothing to the cpu/memory rows. It still has to be admitted, though — see the LimitRange caveat below.
  • Σ over groups, not just the first. Each runnerGroups[] entry has its own maxWorkers and pod template; total worker demand is the sum across all of them. A GPU group with large per-pod requests can dominate the total even at a low maxWorkers.
  • A term with no value drops out. If a group's pod template sets requests but no limits, its limits.* contribution is 0 (the gateway sums only what's declared).

Worked example

A gateway with the default proxy pool and two runner groups:

Pool Count Per-pod requests Per-pod limits
Proxy (defaults) maxReplicas: 10 cpu 10m, mem 32Mi cpu 500m, mem 64Mi
default group maxWorkers: 20 cpu 1, mem 2Gi cpu 2, mem 4Gi
gpu group maxWorkers: 4 cpu 4, mem 16Gi cpu 8, mem 32Gi
Quota key Proxy default (×20) gpu (×4) AGC Total Rounded up
pods 10 20 4 1 35 35
requests.cpu 100m 20 16 — 36.1 37
requests.memory 320Mi 40Gi 64Gi — ~104.3Gi 105Gi
limits.cpu 5 40 32 — 77 77
limits.memory 640Mi 80Gi 128Gi — ~208.6Gi 209Gi

The proxy pool's footprint is tiny next to the worker pools — round the totals up to leave headroom for the next scale-out and for any system pods the platform schedules into the namespace.

Copy-pasteable template

apiVersion: v1
kind: ResourceQuota
metadata:
  name: <tenant>-quota
  namespace: <tenant-namespace>
spec:
  hard:
    # From the worked example above — recompute for your own proxy.maxReplicas,
    # maxWorkers, and per-pod requests/limits.
    pods: "35"
    requests.cpu: "37"
    requests.memory: "105Gi"
    # limits.* are optional — include them only if you want to cap aggregate
    # limits as well as requests. If you DO constrain limits.*, every pod in the
    # namespace must declare limits (see the LimitRange caveat).
    limits.cpu: "77"
    limits.memory: "209Gi"

LimitRange caveat. A ResourceQuota that constrains requests.cpu/requests.memory (or limits.*) makes those declarations mandatory for every pod in the namespace — Kubernetes rejects any pod that omits a constrained resource. The v1 AGC pod stamps none, so pair the quota with a LimitRange that supplies default requests (and limits, if constrained); otherwise the AGC Deployment's pods are rejected with a must specify requests.cpu admission error. The proxy always carries requests, and workers carry whatever their pod template sets, but the AGC pod relies on the LimitRange default. Constraining only pods avoids this entirely (at the cost of no cpu/memory cap).

Step 2: Create the ActionsGateway Resource

Apply the ActionsGateway CR in the tenant's namespace. Adjust proxy and runnerGroups for the tenant's workload. The namespace quota is set separately on the namespace (Step 1b), not on this CR.

One ActionsGateway per namespace. The admission webhook rejects a second ActionsGateway in a namespace that already has one — every per-tenant resource has a fixed, namespace-scoped name, so two CRs would contend over them and flap the namespace's PSA labels. To run a second logical gateway (e.g. a privileged profile alongside a baseline one), give it its own namespace. See troubleshooting: second ActionsGateway rejected.

apiVersion: actions-gateway.github.com/v1alpha1
kind: ActionsGateway
metadata:
  name: <tenant>-gateway
  namespace: <tenant-namespace>
spec:
  gitHubAppRef:
    name: github-app-v1
  # GitHub org/enterprise/repo URL the runners register against (required). Use an
  # org URL (https://github.com/my-org) for org-wide runners, a repo URL
  # (https://github.com/my-org/my-repo) to scope to one repo, or your GitHub
  # Enterprise Server URL (https://ghes.example.com/my-org). The App referenced by
  # gitHubAppRef must be installed on this same org/enterprise.
  gitHubURL: https://github.com/my-org
  # Default: blocks privileged containers, host namespaces, hostPath, dangerous caps.
  # Set to "restricted" for stricter isolation, or "privileged" only if the workload
  # genuinely needs an unrestricted PodSpec (DinD, Buildah without sandbox, kernel modules).
  # "privileged" requires the namespace to be eligible: a platform admin must label it
  # actions-gateway.github.com/privileged-profile=allowed (see Pre-Conditions), else the webhook
  # rejects the CR at create/update. It is deliberately not tenant-settable.
  # A privileged worker container (securityContext.privileged: true in a podTemplate)
  # is ONLY admitted under securityProfile: privileged — under baseline/restricted the
  # webhook rejects it. See troubleshooting: privileged worker container rejected.
  securityProfile: baseline
  # Log verbosity for this tenant's AGC and egress proxy: info (default) or debug.
  # Leave at info; flip to debug only for a bug repro (see "Per-tenant log level"
  # below). Changing it is a rolling restart of the AGC and proxy, not a hot reload.
  logLevel: info
  proxy:
    # minReplicas is a floor, not a replica count: the GMC applies it when the pool
    # is first created (or has been scaled to zero) and passes it to the pool's HPA.
    # Above the floor, .spec.replicas belongs to the HPA — raising minReplicas on a
    # running pool takes effect through the HPA, which needs a healthy metrics-server.
    # See troubleshooting: "Proxy Pool Never Scales Out".
    minReplicas: 2
    maxReplicas: 10
    # Optional: noProxyCIDRs excludes internal destinations from the egress proxy.
    # Entries may be CIDRs (10.0.0.0/8), bare IPs, or NO_PROXY domain suffixes
    # (svc.cluster.local, internal.example.com). Admission rejects any entry that
    # would route this tenant's GitHub traffic around the proxy — a hostname
    # matching the gitHubURL host or the public GitHub domains (github.com,
    # githubusercontent.com, ghcr.io) — since that breaks egress-IP attribution.
    # Never list GitHub here. Cluster-internal defaults are appended automatically.
    # noProxyCIDRs: ["10.0.0.0/8"]
  # The namespace ResourceQuota is platform-owned and set on the namespace in
  # Step 1b — it is not a field on this CR.
  # A runner group has no `name` field — the RunnerGroup CR name is derived from
  # the gateway name + the group's first runnerLabel (here "linux").
  runnerGroups:
    - runnerLabels: ["linux", "self-hosted"]
      maxListeners: 10
      maxWorkers: 20
      podTemplate:
        spec:
          containers:
            - name: runner
              resources:
                requests:
                  cpu: "1"
                  memory: "2Gi"

kubectl apply -f actionsgateway.yaml

Optional — worker-pod lifecycle. Each runnerGroups[] entry accepts two cleanup knobs. completedPodTTL (default 5m) is how long a finished worker pod (Succeeded/Failed) is kept before the AGC deletes it — the retention window is your chance to kubectl logs/describe a failed pod; "0s" deletes pods immediately on completion. pendingPodDeadline (default 10m, minimum 1s) is how long a worker pod may sit Pending (unpullable image, unschedulable constraints) before the AGC deletes it and frees the concurrency slot it was holding — raise it above your worst-case node-autoscaling time for GPU pools, e.g.:

  runnerGroups:
    - runnerLabels: ["self-hosted", "gpu"]
      completedPodTTL: "30m"      # longer debugging window for failed jobs
      pendingPodDeadline: "30m"   # GPU node provisioning can exceed the 10m default

A reaped Pending pod emits a WorkerPodStuckPending Warning Event on the RunnerGroup and cancels the job (it never started); see troubleshooting: worker pod reaped while Pending.

Optional — worker scale-up rate limit (scaleUp, opt-in, default-off). Each runnerGroups[] entry accepts an optional scaleUp token bucket that caps the rate at which the AGC creates new worker pods. It is off by default — omit it and provisioning stays immediate (GAG's zero-idle default). It is not the same as maxWorkers: maxWorkers caps how many worker pods run at once (a ceiling), while scaleUp caps how fast they start (a ramp). Reach for it only when a burst of simultaneously-acquired jobs stampedes a shared, rate-sensitive egress path — a NAT/SNAT gateway, a stateful firewall's connection-tracking table, or a site-to-site VPN — where the onset of connections (not the steady-state count) is what causes damage. It deliberately delays already-claimed jobs, trading time-to-pickup for a gentler ramp.

  runnerGroups:
    - runnerLabels: ["self-hosted", "linux"]
      maxWorkers: 200            # ceiling: at most 200 concurrent worker pods
      scaleUp:
        maxPerSecond: 10         # ramp: sustained ≤10 new worker pods/second…
        burst: 20                # …after an initial instantaneous batch of 20
  • maxPerSecond (required, ≥1): the sustained pod-creation rate once the burst is spent.
  • burst (optional, ≥1): the largest instantaneous batch before throttling engages; defaults to maxPerSecond (one second's worth) when omitted.

When the bucket is empty an acquired job waits for a token before its pod is created — holding its GitHub job lock (renewed in the background), so it composes with the namespace-quota retry backoff rather than dropping the job. Each throttled creation increments actions_gateway_worker_scaleup_throttled_total{namespace, runner_group}; a sustained non-zero rate means the ramp is actively smoothing a burst (see observability: metrics). Set maxPerSecond low enough to protect the shared resource but high enough that already-claimed jobs are not held long — a very low rate against a large burst can hold locks for the ramp's full duration.

Pick the right tool for the stampede. A scale-up rate limit is the wrong fix for some bursts:

Burst symptom Use instead
Every cold node re-pulls the large runner image Peer-to-peer image mirror / pre-pull DaemonSet (p2p-image-distribution.md) — a ramp still pulls N times, just spread out
Workers stampede a shared egress path (NAT/firewall/VPN) at onset scaleUp (this knob), often alongside workflow-level concurrency:
Sustained saturation of egress bandwidth/ports for the whole run maxWorkers ceiling (a ramp only defers the cliff), plus more capacity
One workflow drains the whole shared quota and starves others maxWorkers / priority tiers (a fairness ceiling, already shipped)

It coexists with the cluster autoscaler / Karpenter: bounding pod-admission rate eases the node-scale-up burst they react to, and they keep their own independent rate controls.

v2 (RunnerSet) carries the same spec.scaleUp field with identical semantics.

Changing runnerGroups later. Editing spec.runnerGroups on an existing ActionsGateway reconciles to the desired set: added entries create new RunnerGroup CRs, and removing an entry deletes its RunnerGroup (which stops its listeners and cascades to its worker pods). Reordering entries is safe — the GMC keys pruning on owner labels, not list position, so a reorder never deletes or recreates a group. Removing an entry is the way to retire a runner group; maxListeners has a minimum of 1, so there is no in-place scale-to-zero.

Worker image — the default works out of the box. A plain install runs jobs with no workerImage set: the AGC injects GAG's wrapper into every worker pod — a read-only OCI image volume on Kubernetes ≥ 1.33, an initContainer below that — so the runner image itself can be the unmodified upstream ghcr.io/actions/actions-runner (the default) or any actions/runner-derived image. The wrapper is what feeds the mounted job payload + JIT config into Runner.Worker; injecting it means the runner image no longer has to carry it. Set workerImage only to use a custom image (your own tools, a pinned digest) — the wrapper is injected into that too. The upstream actions-runner (and GAG's images built on it) run as UID 1001, so on every profile except privileged the AGC stamps runAsNonRoot: true and gap-fills runAsUser: 1001 automatically. If you point workerImage at a custom image whose user is not UID 1001 — a different named user, or one that runs as root — set securityContext.runAsUser (or runAsNonRoot: false for a root-based image) on the runner container in the podTemplate; otherwise kubelet rejects the pod with CreateContainerConfigError. See troubleshooting: worker pod fails to start after secure-by-default SecurityContext.

Building a build-capable workerImage. The default upstream actions-runner is deliberately minimal — it ships the runner agent but no build toolchain (make, a C compiler, language SDKs). A job that shells out to make on it fails exit 127: make: command not found, where the GitHub-hosted ubuntu-latest image would have had those tools preinstalled. The fix is the same as on Actions Runner Controller (ARC): build your own image FROM the upstream runner with the tools your jobs need, and set it as workerImage. Because the AGC injects the wrapper on top of any base, your image carries only your toolchain — nothing GAG-specific:

FROM ghcr.io/actions/actions-runner:<pinned version>@sha256:<digest>
USER root
RUN apt-get update \
    && apt-get install -y --no-install-recommends build-essential \
    && rm -rf /var/lib/apt/lists/*
USER runner   # keep the non-root UID 1001 the AGC expects

A working reference you can copy and extend lives at scripts/dogfood/runner/Dockerfile (built by scripts/dogfood/runner-build.sh); it adds just enough to run a make-based Go CI. It is a reference example, not an officially supported image — GAG signs and CVE-scans only its five first-party images, so a runner image you ship (or copy from the example) is yours to pin by digest and scan. Keep the runner version in step with the default the AGC would otherwise inject (it is the version GitHub validates at session creation); a stale runner surfaces as the RunnerGroup VersionTooOld condition.

Optional — distributed tracing. To send the AGC's OpenTelemetry traces to a collector, add a spec.tracing block. Setting endpoint is what turns tracing on; leave the block out to keep it off (the default). sampler is a fixed enum — an unrecognized value is rejected by admission (see troubleshooting: tracing sampler rejected).

spec:
  tracing:
    endpoint: https://otel-collector.observability:4317
    sampler: parentbased_traceidratio   # optional
    samplerArg: "0.1"                    # optional — sample 10% of traces
    resourceAttributes:                  # optional
      deployment.environment: prod
    # insecure: true   # only for a plaintext in-cluster collector; TLS is the default

There is no field for OTLP auth headers: collector authentication is a network-layer concern (in-cluster collector, mutual TLS, or a service mesh), not a CR secret. See observability — enabling tracing on GMC-managed AGCs.

Optional — per-tenant log level. spec.logLevel sets the log verbosity of this tenant's AGC and egress proxy: info (the default) or debug. The GMC threads it to both workloads as the LOG_LEVEL environment variable, so you can crank one gateway to debug for a bug repro without redeploying the GMC or touching any other tenant:

kubectl patch actionsgateway -n <tenant-namespace> <name> \
  --type merge -p '{"spec":{"logLevel":"debug"}}'
# ...reproduce the issue, read the debug logs, then revert:
kubectl patch actionsgateway -n <tenant-namespace> <name> \
  --type merge -p '{"spec":{"logLevel":"info"}}'
  • The default is info, never debug. A CR that omits the field — or sets it back to info — runs at info. At thousands of concurrent sessions the per-session/per-job debug lines dominate log volume, so debug is a deliberate, temporary opt-in, not a steady state.
  • Changing it is a rolling restart, not a hot reload. The new level takes effect once the AGC and proxy pods roll (the value is part of their pod templates). Expect the AGC's listener pool to drain and re-establish; in-flight jobs finish on the old pod within its termination grace period.
  • debug surfaces the AGC's per-session → per-job → per-pod lifecycle lines (the listener/multiplexer/provisioner traces, each carrying namespace/group/sessionId/podName correlation fields) and the proxy's per-CONNECT detail. The grep anchors are in observability — debug diagnostics.
  • Only info and debug are accepted; admission rejects any other value.
  • v2 splits the knob per kind. On the v2 (actions-gateway.com) API, ActionsGateway.spec.logLevel covers the AGC only; a proxy pool carries its own EgressProxy.spec.logLevel (same info | debug values and default, same rolling-restart semantics), so a shared pool can be flipped to debug without touching any gateway:
kubectl patch egressproxy -n <tenant-namespace> <name> \
  --type merge -p '{"spec":{"logLevel":"debug"}}'

Step 3: Validate Provisioning

The GMC provisions all tenant resources within ~30 seconds of CR creation.

# Check the ActionsGateway conditions
kubectl get actionsgateway -n <tenant-namespace> <name> \
  -o jsonpath='{.status.conditions}' | jq .

# Expected conditions:
#   Ready=True
#   AGCAvailable=True
#   ProxyAvailable=True
#   ProxyQuotaPressure=False  (True warns the proxy can't scale to maxReplicas within the ResourceQuota)
#   ProxyQuotaExceeded=False  (True means proxy replica creates are being rejected by the ResourceQuota)

# Confirm the AGC Deployment is running
kubectl get deploy -n <tenant-namespace> actions-gateway-controller
# Expected: READY 1/1

# Confirm the proxy pool is running
kubectl get deploy,hpa -n <tenant-namespace>
# Expected: proxy Deployment READY >= minReplicas, HPA TARGETS shows a percentage (not <unknown>)

# Confirm RunnerGroup CRs were created
kubectl get runnergroup -n <tenant-namespace>

# Confirm RBAC was created
kubectl get serviceaccount,role,rolebinding -n <tenant-namespace> | grep actions-gateway

# Confirm NetworkPolicies and ResourceQuota were applied
kubectl get networkpolicy,resourcequota -n <tenant-namespace>
# Expected NetworkPolicies (3):
#   actions-gateway-workload — restricts AGC and worker pods to proxy + DNS
#   actions-gateway-controller      — adds Kubernetes API server egress for the AGC only
#   actions-gateway-proxy    — restricts proxy pods to GitHub CIDRs + DNS

# Confirm the Pod Security Admission label matches the chosen securityProfile
kubectl get namespace <tenant-namespace> \
  -o jsonpath='{.metadata.labels.pod-security\.kubernetes\.io/enforce}{"\n"}'
# Expected: baseline (default), or restricted / privileged if explicitly chosen

If TARGETS: <unknown> on the HPA: resources.requests.cpu is not set on proxy pods. Add it to spec.proxy.resources.requests.cpu in the ActionsGateway spec. See Troubleshooting — Proxy Pool Not Scaling.

Step 4: Validate Listener Sessions

The AGC should begin polling GitHub within seconds of starting.

# Check AGC logs for session registration
kubectl logs -n <tenant-namespace> deploy/actions-gateway-controller --tail=30
# Look for: "session registered" or "starting listener goroutine"

# Check the active sessions metric
# Metric: actions_gateway_active_sessions{namespace="<tenant-namespace>"}
# Expected: 1 per RunnerGroup (e.g. 1 if one RunnerGroup is defined)

If sessions are not appearing: - Check for token errors: kubectl logs ... | grep "token refresh". - Check proxy connectivity: see Troubleshooting — AGC CrashLoopBackOff.

Step 5: Run a Test Job

Have the tenant run a workflow in their repository targeting the registered labels.

Example workflow: 

name: Runner connectivity test
on: workflow_dispatch
jobs:
  test:
    runs-on: [self-hosted, linux]
    steps:
      - run: echo "Runner is healthy. Host $(hostname)"

Trigger from the GitHub Actions UI or: 

gh workflow run "Runner connectivity test" --repo <org>/<repo>

Watch for the job to be acquired and a worker pod to appear: 

# Watch for worker pod creation
kubectl get pods -n <tenant-namespace> -w

# Check jobs acquired metric
# Metric: actions_gateway_jobs_acquired_total{namespace="<tenant-namespace>"}
# Expected: increments by 1

# Check pod creation latency
# Metric: actions_gateway_pod_creation_latency_seconds
# Expected: well under the 15s p95 SLO

Success Criteria

Onboarding is complete when:

  • [ ] ActionsGateway has Ready=True condition.
  • [ ] HPA TARGETS shows a CPU percentage (not <unknown>).
  • [ ] actions_gateway_active_sessions is ≥ 1 per RunnerGroup.
  • [ ] At least one test job has completed successfully in the GitHub Actions UI.
  • [ ] Worker pod was created and deleted after job completion.
  • [ ] No errors in AGC logs during the test job.

Common First-Day Mistakes

Symptom Cause Fix
ActionsGateway condition AGCAvailable=False, logs show RSA key parse error Private key has trailing whitespace or incorrect PEM format Recreate the Secret; ensure the key starts with -----BEGIN RSA PRIVATE KEY----- (PKCS#1) or -----BEGIN PRIVATE KEY----- (PKCS#8, RSA or Ed25519) and has no extra blank lines or spaces
HPA TARGETS: <unknown> proxy.resources.requests.cpu not set Add requests.cpu: "10m" under spec.proxy.resources.requests
Worker pods stuck Pending ResourceQuota exhausted or no schedulable nodes Check kubectl describe resourcequota -n <namespace> and node capacity
RunnerGroup condition VersionTooOld Worker image contains a runner version below GitHub's minimum Update workerImage in the RunnerGroup spec
Test job stays queued in GitHub for >2 minutes active_sessions = 0 — listener goroutines are not running Check AGC logs for credential or proxy errors
HPA present but proxy doesn't scale up maxReplicas too low or HPA metric is <unknown> Check both the HPA spec and that requests.cpu is set
Proxy stuck below maxReplicas; FailedCreate ... exceeded quota events proxy.maxReplicas exceeds the namespace ResourceQuota Check the ProxyQuotaPressure condition (kubectl describe actionsgateway …); raise the quota or lower maxReplicas
Jobs acquired but pods not appearing priorityClassName referenced in priorityTiers does not exist kubectl get priorityclass <name> — create it if missing
ActionsGateway apply rejected: priorityClassName … is not in the platform allowlist The named PriorityClass is not on the GMC --allowed-priority-classes flag (the allowlist is empty by default) Have the platform admin create the PriorityClass and add its name to --allowed-priority-classes; see security-operations.md § Priority classes
EgressProxy/ActionsGateway apply rejected: spec.scheduling.priorityClassName … is not in the platform infra allowlist The class named for an infra pod is not on the GMC --allowed-infra-priority-classes flag (a separate allowlist from the worker one, empty by default) Have the platform admin create the PriorityClass and add its name to --allowed-infra-priority-classes, kept disjoint from --allowed-priority-classes; see security-operations.md § Infra pods

v2 API (alpha): multiple gateways per namespace

Audience: Platform engineer adopting the v2alpha1 (actions-gateway.com) API. This is an alpha, early-adopter API served beside v1alpha1 — everything above (the v1alpha1, actions-gateway.github.com flow) stays fully supported. Install the opt-in actions-gateway-crds-v2 chart first; see Getting Started — Deploy the GMC.

The biggest onboarding change in v2 is that a single namespace may hold multiple ActionsGateways, lifting the v1 one-gateway-per-namespace rule (Step 2). What that changes when onboarding a v2 tenant:

  • Per-gateway resource naming. Every resource a gateway derives is prefixed with the gateway name — <gateway>-agc (AGC Deployment / ServiceAccount / RoleBinding / Service), <gateway>-worker (worker ServiceAccount), <gateway>-workload (workload NetworkPolicy), and so on — so two gateways in one namespace never contend over a fixed name. List one gateway's resources with kubectl get all,networkpolicy,secret -n <tenant-namespace> -l actions-gateway.com/gateway=<gateway>.
  • 52-character name cap. Any v2 CR (ActionsGateway, RunnerSet, RunnerTemplate, ClusterRunnerTemplate, EgressProxy) whose metadata.name exceeds 52 characters is rejected at admission. The cap reserves room for the derived <name>-<suffix> so a label value / Service name stays under RFC 1123's 63-character ceiling (appendix-h §H.6). Pick short gateway names.
  • Kubernetes ≥ 1.31 required. Each AGC reconciles only the RunnerSets whose spec.gatewayRef.name targets it, via a server-side CRD field selector (KEP-4358) that is alpha-off on 1.30. On a 1.30 cluster a v2 AGC's RunnerSet informer fails to sync (field label not supported) and the pod never becomes ready. Confirm the cluster is ≥ 1.31 before onboarding any v2 gateway.
  • Co-located gateways share one namespace security profile. In v2 the Pod Security level is a property of the namespace, not the gateway (see the v2 callout in Pre-Conditions) — so all gateways in a namespace run under the same actions-gateway.com/security-profile label. Tenants needing different postures (e.g. baseline vs privileged) still use separate namespaces, exactly as in v1.

For the full reference — the naming table, per-gateway garbage-collection behavior, the CRD chart prerequisite, and the failure modes — see Troubleshooting — Multiple v2 gateways in one namespace and Appendix H — v2 API decomposition.

Proxy-less onboarding (direct egress)

In v2 the egress proxy is optional. A gateway with no spec.defaultProxyRef and a RunnerSet with no spec.proxyRef egress directly to GitHub, collapsing the minimal onboarding to three objects — one ActionsGateway, one RunnerTemplate, one RunnerSet, with no EgressProxy at all:

apiVersion: actions-gateway.com/v2alpha1
kind: ActionsGateway
metadata: { name: acme, namespace: team-a }
spec:
  credentials:
    type: GitHubApp                       # discriminated union; workload identity is the additive 2nd member
    githubApp: { name: acme-github-app }   # name-only Secret ref, same namespace
  githubURL: https://github.com/acme
  # no defaultProxyRef ⇒ direct egress
---
apiVersion: actions-gateway.com/v2alpha1
kind: RunnerTemplate
metadata: { name: default, namespace: team-a }
spec:
  podTemplate:
    spec:
      containers:
        - name: runner
          resources: { requests: { cpu: "1", memory: 2Gi } }
---
apiVersion: actions-gateway.com/v2alpha1
kind: RunnerSet
metadata: { name: linux, namespace: team-a }
spec:
  gatewayRef:  { name: acme }
  templateRef: { name: default }
  runnerLabels: [gag-linux]   # exactly one label: the ScaleSet default's runs-on name (see below)
  maxWorkers: 50
  # acquisitionProtocol omitted ⇒ ScaleSet (the default); no proxyRef / defaultProxyRef ⇒ direct egress

What you trade, and what you do not:

  • Egress is still restricted — this is mandatory and default-on. Direct egress is not open egress. The GMC still provisions the default-deny egress NetworkPolicy; it allows only DNS (cluster DNS) + the GitHub CIDR allowlist for workers, plus the Kubernetes API server for the AGC. A worker still cannot reach an arbitrary internet destination. The GMC's IP-range refresh keeps the GitHub allowlist current. (As with all egress rules, this is enforced only by a policy-aware CNI — see Pre-Conditions.)
  • You lose per-tenant egress IP attribution. Without a proxy there is no stable per-tenant source IP, so GitHub IP-allowlisting (common with Enterprise Managed Users), incident attribution by source IP, and avoiding shared-NAT throttling are not available. This is the property you opt into by attaching an EgressProxy.
  • The trade is surfaced in status. A proxy-less gateway and runner set report status.proxyMode: Direct (visible as the Egress print column) plus an advisory EgressUnattributed condition (True). The condition is informational — it does not make the object NotReady. Check it with kubectl get actionsgateway,runnerset -n <ns> (the Egress column) or kubectl describe.

To add attribution later, create an EgressProxy and set spec.defaultProxyRef on the gateway (every RunnerSet under it inherits the proxy unless it sets its own proxyRef). A proxyRef/defaultProxyRef that names a missing EgressProxy is treated as an error and fails closed (Ready=False/ProxyNotFound) — it does not silently fall back to direct egress; only an entirely-unset reference means direct.

Acquisition protocol (spec.acquisitionProtocol)

A RunnerSet acquires jobs from GitHub with one of two protocols, selected by spec.acquisitionProtocol:

  • ScaleSet (the default). The runner-scale-set message-queue protocol: one listener session per set, capacity-gated assignment, and a full-runner worker. It is the default because it removes a many-acquirers job-assignment race the classic protocol was subject to under high burst (validated end-to-end before the default flip). A ScaleSet set must declare exactly one runnerLabel — the scale set's name is its single runs-on match target at GitHub — and that label must be unique across the ScaleSet sets under one gateway (a second set claiming it is rejected at admission). Omitting the field selects ScaleSet, so a runner set that does not name a protocol must carry a single label.
  • Classic (deprecated). The per-runner broker protocol. Select it explicitly only to keep a classic-only capability during the migration window — chiefly multi-label matching, which ScaleSet cannot express:
apiVersion: actions-gateway.com/v2alpha1
kind: RunnerSet
metadata: { name: linux, namespace: team-a }
spec:
  gatewayRef:  { name: acme }
  templateRef: { name: default }
  acquisitionProtocol: Classic       # deprecated; required for a multi-label set
  runnerLabels: [self-hosted, linux]
  maxListeners: 10                   # honored under Classic; ignored under ScaleSet
  maxWorkers: 50

Classic is scheduled for removal one minor release after this deprecation (Q264); new runner sets should prefer ScaleSet (a single label) and split a multi-label group into one ScaleSet set per label.

The field is immutable — switching a live set's protocol is a re-registration storm, so change it by creating a new RunnerSet, not by editing an existing one. Existing sets are unaffected by the default flip: the value is recorded at admission, so a set created while the default was Classic keeps Classic. And gag-migrate writes acquisitionProtocol: Classic onto every set it emits, so a migrated tenant's multi-label groups keep working unchanged (opt a migrated set into ScaleSet later by creating a fresh single-label set).

maxListeners has no effect on a ScaleSet set (there is one session per set; concurrency is governed by maxWorkers/priorityTiers, advertised to GitHub as the scale set's capacity). It remains meaningful only for Classic.

Optional templateRef (a default worker pod shape)

RunnerSet.spec.templateRef is optional. A RunnerSet that omits it resolves a worker pod shape through a fallback chain, so a tenant does not have to name a template in every runner set. The chain (resolved at runtime, fail-closed):

  1. RunnerSet.spec.templateRef — the explicit reference (unchanged: a set that sets it behaves exactly as before).
  2. else ActionsGateway.spec.defaultTemplateRef — a per-gateway default the platform or tenant sets on the gateway; inherited by every RunnerSet under it that omits templateRef. It may name a namespaced RunnerTemplate or a cluster-scoped ClusterRunnerTemplate (kind: ClusterRunnerTemplate).
  3. else the single cluster-default ClusterRunnerTemplate — the one a platform admin has marked with the annotation actions-gateway.com/is-default-template: "true" (the same pattern as Kubernetes' default StorageClass).
  4. else the set fails closed Ready=False/TemplateNotFound — the controller never synthesizes a worker pod without a real pod shape.

Which rung resolved is reported in status.templateSource (TemplateRef / GatewayDefault / ClusterDefault), visible as the -o wide Template print column — so you can audit whether a set runs on an explicit template or a default.

Marking the cluster-default (platform admin). Annotate exactly one ClusterRunnerTemplate:

kubectl annotate clusterrunnertemplate <name> actions-gateway.com/is-default-template=true
  • The marker is honored only on the cluster-scoped ClusterRunnerTemplate (platform-authored). A tenant cannot self-elect a namespaced RunnerTemplate as the cluster-wide default.
  • At most one may be marked. If two are marked, any RunnerSet relying on the cluster-default rung fails closed Ready=False/AmbiguousDefault (the message names the conflicting templates) rather than silently picking one — demote the extra (kubectl annotate clusterrunnertemplate <name> actions-gateway.com/is-default-template-) and the set recovers automatically.
  • A defaultTemplateRef/templateRef that names a missing template still fails closed (TemplateNotFound); only an entirely-unset reference falls through to the next rung.

The minimal proxy-less onboarding above can therefore drop templateRef from the RunnerSet once a defaultTemplateRef or a cluster-default exists.

Tuning AGC control-plane resources

ActionsGateway.spec.agcResources is an optional per-gateway override for the CPU/memory requests and limits stamped on this gateway's AGC control-plane container. Most tenants never set it — the AGC ships with a sensible platform default sized for the worst-case listener burst.

The platform default (applied when agcResources is omitted):

CPU Memory
request 500m 2Gi
limit 2 4Gi

This is the Appendix A capacity sizing: the 2Gi memory request is a generous reservation for the ~1,000-goroutine peak burst with headroom for Go runtime overhead; the 4Gi memory limit sits well above the working set so transient bursts don't trigger an OOMKill; the 2-core CPU limit absorbs reconcile/token-refresh spikes on an otherwise I/O-bound (long-poll-blocked) workload.

When to tune. Override only on real signal:

  • Memory — raise requests.memory and limits.memory if you run many RunnerGroups with high maxListeners and observe container OOMKilled events or high GC pressure (go_gc_duration_seconds). Budget roughly sum(maxListeners) × 60 KiB of working set plus the default headroom.
  • CPU — raise limits.cpu only if container_cpu_throttled_seconds_total shows sustained throttling during peak reconcile churn.

The override is per key: set only the request/limit entries you want to change and every other entry keeps its platform default. For example, raising just the memory limit leaves the CPU request/limit and memory request at their defaults:

apiVersion: actions-gateway.com/v2alpha1
kind: ActionsGateway
metadata:
  name: my-gateway
  namespace: my-tenant
spec:
  credentials:
    type: GitHubApp
    githubApp:
      name: my-tenant-github-app
  githubURL: https://github.com/my-org
  agcResources:
    limits:
      memory: 8Gi   # CPU request/limit and memory request stay at the platform default

Changing agcResources rolls the AGC Deployment (a rolling restart, not a hot reload); in-flight listener sessions deregister and re-register within GitHub's redelivery window.

Recommended floor / footguns. The AGC is a single pod holding all listener state in memory — size it generously rather than tight:

  • Do not set limits.memory below ~512Mi, and keep it above your observed working set; a limit under the working set OOMKills the control plane (a CrashLoopBackOff, not a clear error). The platform default 4Gi is a safe starting point.
  • Do not set requests larger than a single node can schedule, or larger than the namespace ResourceQuota leaves free — an over-large request leaves the AGC pod Pending (unschedulable). Check kubectl describe pod for FailedScheduling if the AGC never starts after an agcResources change.

There is no admission-time floor enforced on the values — the guidance above is operator-owned. When in doubt, leave agcResources unset and let the platform default apply.

Workload-identity credentials (external signer)

spec.credentials is a discriminated union (keyed by credentials.type) with two members. The default, GitHubApp (every example above), is the possession model: the App's RSA private key lives in a namespace Secret (Step 1) and the AGC signs the App JWT in-process. WorkloadIdentity is the opt-in delegation model: no App private key is ever stored in the cluster — an external signer signs the App JWT, and the AGC proves its own pod identity to that signer. Use it when policy forbids a long-lived signing key at rest in the cluster and you run a signer (HashiCorp Vault in the MVP) the AGC can reach. See security §5.7 for the trust model.

There is no githubApp Secret for this method — you do not run Step 1. Instead you put the non-secret App identity (appId/installationId) inline and reference a Vault transit key:

apiVersion: actions-gateway.com/v2alpha1
kind: ActionsGateway
metadata: { name: acme, namespace: team-a }
spec:
  credentials:
    type: WorkloadIdentity          # the no-PEM delegation member
    workloadIdentity:
      appId: 12345                  # non-secret; the JWT issuer
      installationId: 67890         # non-secret
      signer:
        provider: Vault             # HashiCorp Vault transit (cloud KMS providers follow)
        vault:
          address: https://vault.vault.svc:8200   # HTTPS required (dev/test plaintext is an explicit AGC opt-in)
          keyName: github-app       # the RSA transit key Vault signs with (signed as RS256)
          transitMount: transit     # optional; defaults to "transit"
          auth:
            role: agc-acme          # the Vault Kubernetes-auth role bound to this gateway's AGC ServiceAccount
            mount: kubernetes       # optional; defaults to "kubernetes"
          # Optional (Q202): on a policy-enforcing CNI, identify Vault as a NetworkPolicy
          # egress peer so the GMC opens a scoped AGC→Vault egress rule automatically.
          # Set exactly one form — a pod/namespace selector (in-cluster Vault) OR a cidr
          # (external Vault). Omit on a non-enforcing CNI (kindnet) or to manage it by hand.
          networkPolicy:
            namespaceSelector:
              matchLabels: { kubernetes.io/metadata.name: vault }
            podSelector:
              matchLabels: { app.kubernetes.io/name: vault }
            # external Vault instead: cidr: 10.0.5.7/32
            # port: 8200            # optional override; default is the port from `address`
  githubURL: https://github.com/acme

Operator prerequisites in Vault (configured out of band, once per gateway):

  • A transit key (keyName) of an RSA type — GitHub App keys are RSA, and transit signs it as RS256 (pkcs1v15 + sha2-256). Import the App's existing private key into transit, or generate a new key in transit and register its public half as the App's key in GitHub.
  • A Kubernetes auth role (auth.role) bound to this gateway's AGC ServiceAccount (named <gateway-name>-agc) and namespace, granting update on transit/sign/<keyName>. The AGC logs in with its projected ServiceAccount token; Vault verifies it via the cluster TokenReview API. The GMC projects that token with the audience vault, so configure the role with audience=vault (or leave it unset to skip the audience check) — e.g. vault write auth/kubernetes/role/<role> bound_service_account_names=<gateway-name>-agc bound_service_account_namespaces=<namespace> token_policies=<policy> audience=vault.
  • The Vault address must be HTTPS — the ServiceAccount token transits it at login. A plaintext address is rejected unless the AGC carries an explicit dev/test opt-in.

NetworkPolicy egress to Vault (Q202). The GMC's per-tenant AGC NetworkPolicy default-denies egress except DNS, GitHub, and the kube API server. Vault's address is not itself a NetworkPolicy-expressible peer, so set signer.vault.networkPolicy (above) to identify Vault — a namespaceSelector/podSelector for an in-cluster Vault, or a cidr for an external one. The GMC then opens a scoped AGC→Vault egress rule (that peer, on the Vault API port from address) on the AGC NetworkPolicy automatically — you no longer add it by hand. If you leave networkPolicy unset on a policy-enforcing CNI (Calico/Cilium — the production recommendation), the AGC's Vault login will be dropped, so set it; on a non-enforcing CNI (kindnet) it is inert and may be omitted. As with all egress rules, it is enforced only by a policy-aware CNI. The field is a shared EgressPeer type (Q204) — the optional port overrides the address-derived port and is normally left unset; the same type will back future egress peers (cloud KMS, telemetry) so the example shape stays stable.

The GMC provisions the workload-identity AGC the same as a GitHubApp gateway — minus the credential Secret mount (there is none), plus the projected Vault-audience ServiceAccount token and the signer config env. A WorkloadIdentity gateway reaches Ready=True once its AGC mints its first installation token through Vault.

Handing Off to the Tenant

Once onboarding is complete, share with the tenant team:

  • The namespace name and the ActionsGateway CR name they own.
  • The runner labels to use in their workflow runs-on fields.
  • A link to Getting Started for self-service changes (RunnerGroup config, quota requests, credential rotation).
  • A link to Observability for the metrics they can watch.
  • The on-call contact for platform-level issues (AGC crashes, GMC failures).

Tenants can manage their own RunnerGroup configuration, credential rotation, and maxListeners tuning without platform team involvement after this handoff.