The operator guide

The daemon is one process that serves executions. Deploying it means answering a handful of questions that always arrive in the same order, and this page takes them in that order, with what each answer costs. The threat model says which of those questions the executor answers for a deployment and which it leaves open. Manifests under deploy/ are one deployment that has already answered every question here.

Every setting is a command-line flag and an environment variable both, and codemode-executor --help prints all of them with their defaults. Some carry no default at all, and the section that owns each says what leaving it unset means. Settings appear again in the table at the end, with no default beside them.

Commands and the image

codemode-executor                   # serve, reading the environment
codemode-executor check             # answer whether this machine can run sandboxed code
codemode-executor status            # ask a running daemon what it is doing
codemode-executor maintenance on    # turn away new callers, keep serving the open ones

check is the first thing a new machine or a new image should run. It pushes one real job through the whole path, isolate included, and names what is missing when the path is broken. check --json answers the same question as a single object carrying ok, on the same exit codes, and a script reads that form.

Exit	What it means
0	The command did what it was asked.
1	A setting was wrong, or the daemon it was told to reach did not answer.
2	The address it was told to listen on was already taken.
5	The check ran and the sandbox did not answer. Only check answers this.

codemode-executor --help prints the same list at the end of its output. But exit 1 is the ambiguous one: a script polling status cannot tell a daemon that has stopped from a setting the command refused, because both answer 1. The message separates them: an unreachable daemon names the address the command tried. Exit 5 carries no such ambiguity. It belongs to check alone, and it is the one worth wiring into a deployment. By default the image serves. So a machine that cannot build an isolate starts cleanly, and only breaks when the first caller arrives. Running check first turns that into a startup failure with a reason in it: as an init container, or as the command of a one-off Job, before the serving container takes traffic.

codemode-executor is the daemon’s own binary. Every example here assumes it on the path. Inside the image it is the entry point, so a bare docker run … serves and docker run … check checks. That name is also on the path of the container, so docker exec … codemode-executor status reaches a daemon that is already running. maintenance off reverses maintenance on, and status says which of the two states a daemon is in. A deployment outside a container runs the built entry directly, node apps/daemon/dist/bin.js, with the same flags. The daemon is not published to npm.

The engine compiles against the Node that runs it, so the daemon needs whichever Node line the workspace declares in its engines field. Most deployments install no Node at all and hand the image to docker run instead:

docker run --rm -p 8080:8080 ghcr.io/maksimyaromin/codemode-executor:latest

That image is ghcr.io/maksimyaromin/codemode-executor, tagged with each release version and with latest on the newest, and built for linux/amd64 and linux/arm64. Its base is the Debian slim image of the same Node line, never Alpine. Its engine is compiled from source in the build stage, and the binary has to meet at runtime the C library it was compiled against. It runs as the unprivileged node user. One daemon holds one pool of child processes, one child holds one execution at a time by default, and one execution holds one V8 isolate that is disposed when the run settles.

Two listeners and their addresses

HOST and PORT are the address callers reach. SOCKET_PATH replaces both with a unix socket. With it unset there is no socket, the host and port are used, and the host default is every interface.

The daemon runs whatever code reaches it and authenticates nobody by itself. So an address reachable beyond the callers a deployment intends is arbitrary code execution offered to whoever finds it. A cluster is that address by default: Kubernetes allows every connection to a Pod until a NetworkPolicy selects that Pod and closes it. Closing it is the first of the controls the operator owns. The same-Pod manifest repeats that in its own header, because there the daemon has a port open on the Pod address with no Service in front of it.

That same listener answers /healthz, /readyz and /metrics, for GET and HEAD. Probes and the scrape annotation in the manifests point at those paths. So the scrape path sits on the port that faces the callers, and that is one more reason to close it.

ADMIN, ADMIN_HOST and ADMIN_PORT configure a second listener. Its purpose is different. It answers the process status, the settings in force and the maintenance switch. Loopback is the address it binds by default, and it authenticates nobody either, so it belongs where an operator can reach it and a caller cannot. ADMIN set to off leaves no admin listener at all, and then status and maintenance have nothing to talk to. The bundled status and maintenance commands are clients of it, and so is anything else that speaks HTTP:

Route	Answers
GET /__status__	service, version, runtime, pid, uptime, maintenance, the protocol address with its accepting flag, the admin address, and the pool’s counters
GET /__config__	Every setting in force, each with the value and where it came from
GET /__maintenance__	{ "maintenance": boolean }
PUT /__maintenance__	Turns maintenance on and answers the new state
DELETE /__maintenance__	Turns it off and answers the new state

Every name carries its dunders so that no path an ordinary reverse proxy forwards can collide with it. Anything else on this listener answers 404.

Sizing is a memory decision

Memory is the bottleneck here, not CPU. An execution is an isolate with a heap bound, and how many run at once is limited by how much memory the machine has for them. Three settings decide it.

MAX_WORKERS is how many child processes may run at once, and it defaults to one per CPU the machine reports. That default is a CPU answer to a memory question, and it is worth replacing with a number derived from the arithmetic below. EXECUTIONS_PER_WORKER is how many executions one child serves at a time, one by default. At one, a native crash costs exactly one caller. MEMORY_LIMIT_MB bounds one isolate’s heap.

A container limit has to cover every worker at once, each holding its own runtime and an isolate that may overshoot its bound before the engine stops it. isolated-vm calls that limit a guideline, not a strict limit, and says a determined attacker could use two to three times it before the script is terminated. So the manifests size the container over three times the product of the worker count, the executions per worker and the isolate bound. One check in the daemon’s own suite holds them to it.

But some of the pool’s own bounds are not settings a daemon exposes at all: the process memory a worker retires at, the delay before a hung child is terminated, and the idle bound on retained handles. A deployment takes the pool’s numbers for those, and the pool chapter lists them.

Retirement and retained handles take capacity too. RETIRE_AFTER_EXECUTIONS bounds what a worker accumulates across runs by replacing it, and the nightly soak measures that every worker retires on that count and not on the process memory bound behind it. A retained execution is one the caller asked to keep handles for, and it holds its worker’s slot and its isolate until the handles are closed or an idle bound expires. Neither that bound nor the slot is an operator setting. So a deployment that grants handles counts those executions in its worker budget.

Bounds a caller meets

DEFAULT_TIMEOUT_MS is the run bound when a request names none, and MAX_TIMEOUT_MS is the longest any caller may ask for. With that ceiling unset it becomes the largest delay a Node timer accepts, near enough twenty-five days, so one caller may hold one worker for as long as it likes. GRACE_MS is how long an execution has to stop after its time is up, before the layer behind the engine ends it. The run clock and what pauses it are specified in the executor protocol.

MAX_QUEUE is how many executions may wait for a free worker. Past it a caller meets a typed, retryable fault, not a backlog that grows until the machine gives out. Callers retry that fault, and an operator scales on it.

RESPONSE_LIMIT_BYTES caps what one execution may return. With it unset, the only cap left is the protocol’s own. That bound belongs to the protocol and not to the operator: everything a run sends is bounded where it is built. So a program cannot end the worker holding it by answering with too much.

Network the code reaches

A caller decides per execution whether the sandbox reaches the network, and the executor protocol specifies the three states it chooses from. OUTBOUND_MODE is the operator’s veto over one of them. native, the default, lets the executor perform a request itself, and null denies that however the caller asked. A caller that supplies its own function, or that disables outbound outright, is unaffected either way. But native means the executor applies no URL policy of its own: the code reaches whatever the daemon’s network reaches. A deployment that keeps native closes the network around the daemon instead.

Health, readiness and metrics

Liveness answers while the process is up. Readiness reports whether the daemon is accepting work, and a rolling update and a drain both need that answer. But it attempts no execution, so a pool that accepts work it cannot finish still reads ready. The queue depth below is the signal that catches that. Kubernetes takes an unready Pod out of its Service endpoints without restarting it, so a drain is expressed as readiness and never as health.

The daemon measures itself through OpenTelemetry and registers a Prometheus reader for its own process, so /metrics answers a scrape with no configuration. OTEL_SDK_DISABLED turns measurement off entirely, and it takes /metrics with it: the path answers 404 while liveness and readiness keep answering. Manifests under deploy/k8s carry scrape annotations unconditionally. So a deployment that disables the SDK strips those annotations, and does not leave Prometheus scraping a 404. Traces and metrics also leave through OTLP when OTEL_EXPORTER_OTLP_ENDPOINT names a collector. What the daemon publishes, what each metric records, and the Prometheus name each is scraped under are in Telemetry.

Logging

The daemon writes one JSON object per line on stdout, and a container platform collects that without help. LOG_LEVEL chooses how much: trace, debug, info, warn, error, fatal or silent, and it defaults to info. At info the daemon reports its version, runtime and pid as it starts, the address each listener took, and the signal that stopped it. But executions are not logged one by one at any level. A log line per execution is a cost that scales with traffic, and the same facts are already metrics. Callers a connection policy turned away are logged beside their remote address, and so is a policy that threw.

Scaling on the queue, not on the CPU

An executor spends much of its time inside waits it does not own, so CPU utilization can read low on a replica whose callers are queuing. executor.pool.queue.depth is the signal that says callers are waiting for a worker. Every execution publishes the depth it found, so a replica carries the series from its first execution and not from its first queued one. With a metrics adapter in the cluster, a horizontal autoscaler reads it directly:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: codemode-executor
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: codemode-executor
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Pods
      pods:
        metric:
          name: executor_pool_queue_depth
        target:
          type: AverageValue
          averageValue: '2'

Two replicas is a floor, not a recommendation: a single replica has no capacity while it restarts. The target adds a Pod once the queue holds more than about two executions per Pod. That number is a starting point, to be tuned against the run durations a deployment actually sees.

Draining and stopping

maintenance on stops the daemon taking new callers while the connections it already holds go on executing, and readiness answers false for as long as it is on. Maintenance is the drain that is not a shutdown: it takes a replica out of rotation to watch it, to let it empty before a restart, or to hold it back while something upstream is repaired.

A stop signal starts the other one. SHUTDOWN_DELAY_MS is how long the daemon keeps serving after the signal: the window a load balancer needs to stop sending it work. Readiness answers false for that whole window, so a rolling update takes the Pod out of rotation before it stops accepting. SHUTDOWN_GRACE_MS bounds the drain itself, after which the process exits whatever is still running. The daemon refuses a delay that is not smaller than the grace.

PING_INTERVAL_MS is how often a caller connection is asked whether it is still there. MAX_CONNECTION_AGE_MS closes a connection that has been open too long, and it closes none while unset. A client reconnects, so setting it is how callers end up spread across replicas after a scale-out. Without it they stay pinned to the ones they first reached.

Every setting

Setting	The decision it serves
HOST, PORT, SOCKET_PATH	Where callers reach the daemon
ADMIN, ADMIN_HOST, ADMIN_PORT	Where an operator reaches it
MAX_WORKERS, EXECUTIONS_PER_WORKER, MEMORY_LIMIT_MB	How much memory the deployment needs
RETIRE_AFTER_EXECUTIONS	How long a worker lives before it is replaced
DEFAULT_TIMEOUT_MS, MAX_TIMEOUT_MS, GRACE_MS	How long one execution may run
MAX_QUEUE	When a caller is told to retry instead of waiting
RESPONSE_LIMIT_BYTES	How much one execution may return
OUTBOUND_MODE	Whether the executor may reach the network for the sandbox
SHUTDOWN_DELAY_MS, SHUTDOWN_GRACE_MS	How a stop signal is served
PING_INTERVAL_MS, MAX_CONNECTION_AGE_MS	How long a caller connection lives
LOG_LEVEL	How much the daemon says

This table is compared against the daemon’s own declarations on every pnpm verify, in both directions, so a setting that appears or leaves turns the gate red instead of leaving a stale page behind. That check is why a list the code owns is safe to print here. The defaults behind the names are not printed, because no check would catch those going stale.

Deploying

deploy/compose.yaml runs the daemon on one machine beside whatever calls it, through Docker Compose, and its own comments say where its posture is weaker than the manifests below. deploy/k8s/standalone.yaml runs it as its own Deployment and Service. deploy/k8s/same-pod.yaml runs it as a second container beside the host that calls it. That shape keeps the traffic on the loopback address and needs no Service at all.

Both Kubernetes manifests run the daemon without root, without privilege escalation, with every Linux capability dropped, on a read-only root filesystem and under the runtime default seccomp profile. Operators who apply one inherit that posture whole. No manifest can decide what surrounds the process, and Controls the operator owns is that list in full.