Skip to main content
Modulo

Kubernetes

Modulo runs on any conformant Kubernetes cluster. This page has two halves.

  1. Install the stack: deploy the Modulo backend, queue workers, and frontend with the Helm chart.
  2. Run agent workspaces in your cluster: enable the Kubernetes runtime provider so each agent workspace becomes a long-lived pod that runs under a ServiceAccount you create, in a namespace your admission policy governs.

The Kubernetes tier exists because the control plane then holds a namespaced Kubernetes credential instead of a Docker Engine API socket, and your own admission policy validates every workspace pod spec before it can run.

Install the stack

Prerequisites

  • A Kubernetes 1.25+ cluster: Amazon Elastic Kubernetes Service (EKS), Azure Kubernetes Service (AKS), Google Kubernetes Engine (GKE), or any conformant distribution
  • kubectl and Helm 3.10+
  • Python 3 with the cryptography package, used only to generate the Fernet key in the install commands below (Linux distributions usually ship it as python3, not python)
  • An external PostgreSQL 16 database the pods can reach. The chart never deploys a database.
  • Redis 7+: external, or embedded by the chart (redis.embedded=true)
  • Pull access to the Modulo container images. The GitHub Container Registry (GHCR) images are private, so create a pull secret and reference it in imagePullSecrets.

The chart needs two PostgreSQL roles. Provision the admin role yourself (superuser, or at minimum CREATEROLE plus CREATEDB); the restricted runtime role is created automatically by the backend on startup.

What the chart deploys

What the chart deploys
Workload Role
backend FastAPI API server
saq-runner Queue workers that execute pipeline jobs
saq-system Scheduler, reconcile, and system cron jobs
frontend nginx-served single-page app
redis Queue backend, only when redis.embedded=true

The chart also provisions a ResourceQuota and a LimitRange for the namespace, an optional HorizontalPodAutoscaler (HPA) for the backend and frontend, an optional ingress, and the namespace itself with Pod Security Admission labels (see Pod Security below).

Install

Clone the product repository and install the chart. This example creates a modulo namespace (via namespace.name), embeds Redis, and points at your own database:

# Replace the uppercase placeholders below with your own values before running.
git clone https://github.com/farnalabs/modulo
cd modulo

helm install modulo ./deploy/helm/modulo \
  --set namespace.name=modulo \
  --set redis.embedded=true \
  --set postgres.host=your-db.example.com \
  --set postgres.username=modulo_app \
  --set postgres.adminUsername=modulo \
  --set postgres.password=DATABASE_PASSWORD \
  --set imagePullSecrets[0].name=ghcr-secret \
  --set backend.env.SECRET_KEY=$(openssl rand -base64 32) \
  --set backend.env.FERNET_KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())") \
  --set backend.env.MODULO_USERS="admin:$(openssl passwd -6 ADMIN_PASSWORD)" \
  --set backend.env.SAQ_AUTH_USERNAME=admin \
  --set backend.env.SAQ_AUTH_PASSWORD=$(openssl rand -hex 16)

Four of those values are mandatory, and helm install fails without them:

Install
Value Why
backend.env.SECRET_KEY Signs every API token. Any string of 32 bytes or more.
backend.env.FERNET_KEY Encrypts connector credentials at rest. A URL-safe Fernet key of 32 bytes or more.
backend.env.SAQ_AUTH_USERNAME and SAQ_AUTH_PASSWORD The saq-system worker fail-closes without them: it crash-loops, the scheduler never runs, and backend readiness never passes. Empty strings count as missing.
postgres.host (or a full DATABASE_URL) The chart does not deploy a database.

backend.env.MODULO_USERS is optional but recommended: without it no login exists, and the app warns at boot instead of crashing.

To store credentials in a pre-created Kubernetes Secret instead of inline values, use postgres.existingSecret (and the equivalent keys for Redis). See the chart README for the full values reference.

Managed-cluster notes

  • EKS: the shipped values.eks.example.yaml wires the chart to Amazon RDS for PostgreSQL, Amazon ElastiCache for Redis, and an AWS Load Balancer Controller ingress. This combination was validated on a live EKS cluster in September 2026: every workload became Ready across two nodes and the backend readiness endpoint returned HTTP 200.
  • AKS and GKE: the chart is vendor-neutral. Point postgres.host and redis.host at your managed database and cache, set ingress.className to the ingress controller you run, and set imagePullSecrets if your images are private. No cloud-specific example ships for these two yet.
  • Any other cluster: the same chart and the same values. Nothing in it depends on a cloud provider.
helm install modulo ./deploy/helm/modulo \
  -f deploy/helm/modulo/values.eks.example.yaml \
  --set postgres.password=DATABASE_PASSWORD \
  --set backend.env.SECRET_KEY=$(openssl rand -base64 32) \
  --set backend.env.FERNET_KEY=$(python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())") \
  --set backend.env.SAQ_AUTH_USERNAME=admin \
  --set backend.env.SAQ_AUTH_PASSWORD=$(openssl rand -hex 16)

If you would rather let Helm use an existing namespace than have the chart create one, install with --namespace <your-namespace> --create-namespace --set namespace.create=false, then apply the Pod Security labels yourself.

Verify

kubectl get pods -n modulo
kubectl port-forward svc/modulo-frontend 8080:80 -n modulo

Open http://localhost:8080 and sign in with the MODULO_USERS credentials. Use /healthz/ready (not /healthz) to judge readiness: it checks the database, Redis, the queue workers, the scheduler, and the other subsystems, and returns HTTP 200 only once they are all up. /healthz is a shallow liveness probe and always answers immediately.

Backend and frontend health probes are wired into the Deployments by default, so a stuck subsystem shows up as a non-Ready pod rather than a silent outage.

Upgrade and uninstall

helm upgrade modulo ./deploy/helm/modulo -f <your-values.yaml>

helm uninstall modulo
kubectl delete namespace modulo   # only if the chart created it

Run agent workspaces in your cluster

With the stack installed, the Kubernetes runtime provider runs each agent workspace as a pod instead of as a process on the host or a container in a Docker engine. One long-lived pod per workspace: Modulo creates it, streams commands into it through the Kubernetes exec subresource, reads its output, and destroys it when the workspace is done. Workspace pods carry the modulo.provider=kubernetes label, so you can watch them directly:

kubectl get pods -n modulo -l modulo.provider=kubernetes

Workspace pods are ephemeral. The workspace orphan sweep runs as a system cron job every five minutes and lists pods whose run has gone away. It reports orphans without destroying them until you set RUNNER_RECONCILER_DESTROY_ENABLED=true.

1. Enable the provider

Set MODULO_KUBERNETES_ENABLED on every Modulo pod that touches workspaces: the backend (profile testing), the queue workers (dispatch), and the system worker (the orphan sweep).

# values.yaml for helm
backend:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
saqRunner:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
saqSystem:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
1. Enable the provider
Variable Default Purpose
MODULO_KUBERNETES_ENABLED unset Registers the provider when set to any value except the explicit negatives 0, false, no, off. Unset leaves it unregistered, and a profile that asks for it fails with an error naming this variable.
MODULO_KUBERNETES_NAMESPACE modulo Namespace the provider creates workspace pods in.
MODULO_KUBERNETES_SERVICE_ACCOUNT default ServiceAccount the workspace pods run under. This is where your workload identity lives.

Authentication picks itself: inside a cluster the provider uses the pod’s own mounted ServiceAccount token, and outside one it falls back to the standard kubeconfig chain (KUBECONFIG, then the default path). Client configuration loads on first use, so the provider registers at boot on a machine that has no kubeconfig.

The Kubernetes client library ships with the backend image. There is nothing extra to install.

Enable the provider and put the credential from the next step in place before you run anything: with the provider enabled and no usable credential, the profile test fails and the system worker reports a workspace-sweep error every five minutes.

2. Give the control plane a scoped credential

Modulo’s pods talk to the Kubernetes API, so they need a token and Role-Based Access Control (RBAC) grants. The chart defaults to serviceAccount.automountServiceAccountToken: false (when the Kubernetes tier is off, nothing in the stack calls the API, and an unmounted token only widens the blast radius of a compromised pod). Add the knob to the values file from the previous step:

# values.yaml for helm
serviceAccount:
  automountServiceAccountToken: true

Enabling it mounts a namespaced ServiceAccount token into the three workloads that touch workspaces (backend, saq-runner, saq-system), and those pods then authenticate against the Role granted below. The setting is declarative: it survives helm upgrade with nothing to re-run. The frontend is a static nginx single-page app (SPA) that never calls the Kubernetes API, so the knob deliberately does not apply to it and its token stays unmounted.

Then grant the least-privilege Role, scoped to the workspace namespace only:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: modulo-workspace-manager
  namespace: modulo
rules:
  # Create, read, list, and delete workspace pods
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["create", "get", "list", "delete"]
  # Stream commands into a workspace (the exec subresource)
  - apiGroups: [""]
    resources: ["pods/exec"]
    verbs: ["create"]
  # Read a workspace's container log (kubectl logs)
  - apiGroups: [""]
    resources: ["pods/log"]
    verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: modulo-workspace-manager
  namespace: modulo
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: modulo-workspace-manager
subjects:
  - kind: ServiceAccount
    name: modulo        # the chart's ServiceAccount; set serviceAccount.name if you override it
    namespace: modulo

Keep this a Role, not a ClusterRole: the provider only ever works in MODULO_KUBERNETES_NAMESPACE. If you run Modulo in one namespace and workspaces in another, put the Role and RoleBinding in the workspace namespace and list the control plane’s ServiceAccount as the subject.

3. Run workspace pods under your ServiceAccount

Set MODULO_KUBERNETES_SERVICE_ACCOUNT to a ServiceAccount of your own in the workspace namespace, and create it there:

kubectl create serviceaccount modulo-workspaces -n modulo

# values.yaml
backend:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
    MODULO_KUBERNETES_SERVICE_ACCOUNT: "modulo-workspaces"
saqRunner:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
    MODULO_KUBERNETES_SERVICE_ACCOUNT: "modulo-workspaces"
saqSystem:
  env:
    MODULO_KUBERNETES_ENABLED: "1"
    MODULO_KUBERNETES_SERVICE_ACCOUNT: "modulo-workspaces"

This ServiceAccount is what gives agent workloads their identity in your cloud account. Annotate it with your platform’s workload-identity mechanism:

EKS: IAM Roles for Service Accounts (IRSA)

apiVersion: v1
kind: ServiceAccount
metadata:
  name: modulo-workspaces
  namespace: modulo
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/modulo-workspaces

The AWS IAM role’s trust policy must match this namespace and ServiceAccount name.

AKS: Microsoft Entra workload identity

apiVersion: v1
kind: ServiceAccount
metadata:
  name: modulo-workspaces
  namespace: modulo
  annotations:
    azure.workload.identity/client-id: <app-registration-client-id>
  labels:
    azure.workload.identity/use: "true"

GKE: Workload Identity

apiVersion: v1
kind: ServiceAccount
metadata:
  name: modulo-workspaces
  namespace: modulo
  annotations:
    iam.gke.io/gcp-service-account: modulo-workspaces@PROJECT_ID.iam.gserviceaccount.com

Bind the Google service account to the Kubernetes one:

gcloud iam service-accounts add-iam-policy-binding \
  modulo-workspaces@PROJECT_ID.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:PROJECT_ID.svc.id.goog[modulo/modulo-workspaces]"

Two cautions, stated plainly:

  • The identity exchange is yours to verify. Modulo builds the workspace pod itself and deliberately disables service-account token automount on it, because a workspace never needs the Kubernetes API. Confirm your platform’s identity injection still works for these pods before you rely on it. The conformance suite authenticates with a plain Kubernetes token and does not exercise cloud workload identity.
  • Attach image pull secrets to this ServiceAccount if your workspace image is private. The provider’s pod spec does not set imagePullSecrets, so the ServiceAccount’s own pull secrets are what the kubelet uses.

4. Pod Security (restricted) namespace

The namespace that hosts workspace pods should enforce the Pod Security Admission (PSA) restricted standard:

kubectl label namespace modulo \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/audit=restricted \
  pod-security.kubernetes.io/warn=restricted --overwrite

The Helm chart applies these labels itself when it creates the namespace (podSecurity.enforce defaults to restricted), so this step is only needed if you created the namespace some other way.

Every pod the provider builds already targets that profile: non-root user (UID 1001), allowPrivilegeEscalation disabled, read-only root filesystem, all Linux capabilities dropped, and the RuntimeDefault seccomp profile. A workspace image must work when forced to run as UID 1001 and must provide a POSIX shell, because the dispatcher’s file operations run through the exec subresource. Your own admission policy can be stricter than restricted and is free to reject any pod spec Modulo builds: that is the point of running in your cluster.

5. Create the environment profile

Environment profiles are a Team-tier feature: on the Community plan the environment-profiles API answers HTTP 402 for them. The Kubernetes profile carries the image, resources, and network posture for every workspace:

curl -X POST https://<your-modulo-host>/api/v1/environment-profiles \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Kubernetes workspaces",
    "description": "Agent workspaces as pods in the modulo namespace",
    "provider_type": "kubernetes",
    "image_ref": "python:3.13-slim",
    "capabilities": ["git", "python>=3.12", "shell"],
    "config_json": { "memory_mb": 2048 },
    "network_policy": "outbound",
    "persistence_policy": "ephemeral"
  }'
5. Create the environment profile
Field Value and why
provider_type kubernetes. This is the value that routes workspaces to pods.
image_ref The workspace pod image. Defaults to python:3.13-slim; it must work when forced to run as UID 1001 and provide a POSIX shell.
config_json.memory_mb Pod memory request and limit (default 1 GiB). CPU is fixed at one core.
network_policy Keep outbound. See Bounded egress is your NetworkPolicy for why none and selected are refused on this tier.
persistence_policy Keep ephemeral. Every workspace pod is destroyed with its workspace, so retained and cache are not honored here.

The profile form at /environment-profiles/new currently offers only the Docker and E2B provider types, so create a Kubernetes profile through the API as above; it then appears in the UI like any other profile.

6. Verify with a real workspace

The profile test endpoint provisions a real workspace pod, runs a command in it, and destroys it, streaming each step:

curl -X POST https://<your-modulo-host>/api/v1/environment-profiles/<profile-id>/test \
  -H "Authorization: Bearer <token>"

While it runs, watch the pod appear and go away:

kubectl get pods -n modulo -l modulo.provider=kubernetes -w

A green test proves the whole chain: your control-plane credential can create the pod, your workload ServiceAccount and admission policy accept the spec, the exec subresource works, and teardown is clean.

Bounded egress is your NetworkPolicy

On the Kubernetes tier, egress control is deliberately not Modulo’s. Pod networking belongs to your cluster’s Container Network Interface (CNI), so Modulo does not pretend to own it:

  • The provider enforces no egress rules of its own. There is no bespoke egress gateway and no in-pod firewall.
  • A workspace that asks for deny-all or a selected-domain allowlist is refused with a typed error that names NetworkPolicy as the fix. It is never silently downgraded to open egress.
  • Your environment profile’s network policy therefore stays at outbound on this tier.

Bounding egress is opt-in and it is yours: write a NetworkPolicy on the workspace namespace, and it applies whether or not Modulo knows about it.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: modulo-workspace-egress
  namespace: modulo
spec:
  podSelector:
    matchLabels:
      modulo.provider: kubernetes
  policyTypes:
    - Egress
  egress:
    # DNS, scoped to your cluster's DNS service
    - to:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53
    # Your database
    - to:
        - ipBlock:
            cidr: 10.0.0.0/24
      ports:
        - protocol: TCP
          port: 5432

Two requirements, both on your side:

  1. NetworkPolicy is opt-in. Until you write one, nothing restricts pod egress.
  2. Your CNI must enforce NetworkPolicy. A policy applied on a CNI that does not implement NetworkPolicy is documentation, not enforcement. Check your cluster’s CNI before you rely on the policy; several managed clusters ship a CNI that enforces it, and some do not.

This is a design point, not a gap: your NetworkPolicy, your CNI, your rules, enforced by infrastructure you control.

What this page does not prove

Validation status, stated so you can weigh it:

  • Chart install on EKS: validated on a live two-node EKS cluster in September 2026 (all workloads Ready, readiness endpoint green). That is the basis for the install claims above.
  • Runtime provider conformance: the provider’s conformance suite runs against a throwaway kind cluster on pull requests and pushes to main (path-filtered to the runtime surfaces), and that leg is green. It proves the provider contract, workspace lifecycle, and that workspace pods are admitted under Pod Security restricted.
  • Managed-cluster conformance: a scheduled leg runs the same suite against a real EKS, AKS, or GKE control plane in weekly rotation. It has not yet run green, so we make no managed-cloud conformance claim for the provider. No green run means no claim.
  • Workload identity: not exercised by the suite at all. It authenticates with a plain Kubernetes credential, so your IRSA, Entra, or GKE identity exchange is verified only in your own cluster (step 3).
  • NetworkPolicy enforcement: not exercised by the suite either, because the kind cluster’s default CNI does not enforce NetworkPolicy. The suite proves only that Modulo refuses to claim egress enforcement it cannot do.

The full claim discipline, including what each leg can and cannot prove, is documented in the product repository.

Next: Bring Your Own Agent