a) Most agents spend most of their time waiting. They wait for the model to respond and they wait for a tool to finish and they wait for a person to reply. For example if a coding agent works for 30 seconds and then waits 10 minutes for the user to read the result and reply then the agent is busy for about 5 percent of the time. Then in this example a pod that holds the agent all day keeps 95 percent of its CPU and memory reserved for nothing.
b) Agent Substrate is an open source runtime from Google that moves agents on and off compute as needed. When an agent is idle Substrate takes a snapshot of its memory and its files and saves the snapshot to storage. When a new request comes in Substrate restores the snapshot onto a free worker pod and the agent continues exactly where it stopped.
c) The worker pods are started ahead of time and wait for work. Each worker pod can host many agents at the same time and each agent runs in its own sandbox.
d) When a request arrives for an agent that is suspended following things happen: The router asks the Substrate control plane to resume the agent. The control plane picks a worker pod with free capacity and prefers the node that still has the snapshot on its local disk. The node restores the snapshot and the router sends the request to the agent.
e) Substrate implements its own control plane & keeps Kubernetes API server out of agent suspend/resume path. Kubernetes still creates the worker pods and scales the nodes.
f) Every agent runs inside gVisor or a microVM. An agent has no network access by default. All outbound traffic goes through a gateway that checks a policy for each agent. The gateway also adds API keys to approved requests so the secret never enters the sandbox or the snapshot.
g) This is still in implementation so there are gaps. GPUs cannot be passed into an agent and gVisor cannot snapshot a running CUDA context. Open network connections are lost when an agent is suspended so the agent has to reconnect after it resumes.