Always On Agent Ops

OpenClaw Always-On Operations

An always-on agent does not die loudly. It goes quiet — expired auth, an unloaded plugin, a dispatcher pointing at an id that no longer exists. Everything below exists to make silence impossible.

Access, data, and network — read before running anything

Everything a cron reaches out to is a system you are responsible for: honour its terms of service, and for any personal data touched, apply GDPR — legal basis, minimisation, and a stated retention window — before you schedule the first run.

Setup

# ClawHub ships text, not file modes: the exec bit does NOT survive install. Without this, a
# cron `kind: "command"` dies with exit 126 and NO status line — the silence this skill prevents.
chmod +x scripts/preflight-queue.sh scripts/event-dispatch.sh \
         scripts/oauth-expiry-check.sh scripts/purge-retention.sh   # cron-message.sh is sourced: 644
# The scripts read nothing from the network until this file exists.
cat > scripts/secrets.env <<'EOF'
QUEUE_URL=https://api.acme-corp.example/queue
API_TOKEN=replace-me
EOF
chmod 600 scripts/secrets.env      # never commit it; it is not shipped with this skill

When to use

Host constraints — the daemon is a user-level service tied to a GUI login session

The loopback cascade — bind: "loopback" (§Config) answers only on 127.0.0.1

An anti-SSRF default, correct, and it decides your whole architecture:

OpenClaw does expose an inbound hook (POST 127.0.0.1:18789/hooks/agent, Bearer), reachable only from the host: useful for a local process to nudge the agent, useless from the internet. Exposing the control plane beyond loopback to "fix" this trades an SSRF-hardened default for a remote-code surface — don't. For local push, run a listener bound to 127.0.0.1 that authenticates a Bearer and accepts one event name from a closed list hardcoded in the listener, passing it as the dispatcher's only argument: no request body, no free-form arguments, no raw agent turn. scripts/event-dispatch.sh enforces that same closed list — an unrecognised event exits 2 without touching a job.

Config as contract

~/.openclaw/openclaw.json (JSON5) is strictly validated: one unknown key, one wrong type, and the gateway refuses to start. OpenClaw keeps a last-known-good copy and restores it when the file is broken — so a bad edit can leave you running yesterday's config while you read today's file. Confirm with a restart, never by reading.

{
  gateway: {
    port: 18789, mode: "local", bind: "loopback",  // "local" mandatory; anything else = damaged
    auth: { mode: "token", token: "${GATEWAY_TOKEN}" },
    nodes: { denyCommands: ["camera.snap", "screen.record", "sms.send",  // capture/publish/agenda
                            "contacts.add", "calendar.add", "reminders.add"] },
  },
  agents: {
    defaults: {
      model: { primary: "${MODEL_PRIMARY}", fallbacks: ["${MODEL_FALLBACK}"] },
      models: {                          // MANDATORY allowlist: primary+fallbacks must appear here
        "${MODEL_PRIMARY}":  { agentRuntime: { id: "${RUNTIME_ID}" } },
        "${MODEL_FALLBACK}": { agentRuntime: { id: "${RUNTIME_ID}" } } },
      workspace: "${OPENCLAW_HOME}/workspace" },
    list: [{ id: "main" }] },
  plugins: { entries: {                  // EXCLUSIVE allowlist — omitted plugin = not loaded, silently
    "${NOTIFY_PLUGIN}": { enabled: true },   // example — your chat/notification transport
    "${PROVIDER_PLUGIN}": { enabled: true }, // example — your model provider
    "${RUNTIME_PLUGIN}": { enabled: true },  // the runtime named by agentRuntime.id above; without
  } },                                       // it, agentRuntime never executes
  tools: { profile: "restricted" },      // narrowest profile that boots — jobs inherit THIS
  commands: { native: "auto", restart: true },
}

Auth and runtime are two orthogonal axes

Auth = which account pays (a provider profile: OAuth or an API key). Runtime = which executor runs the agentic turn (agentRuntime.id). Configuring one does not configure the other. All four must be present together — miss one and it breaks before the first response:

openclaw models list --provider ${PROVIDER}   # never copy model names from a runbook — enumerate
# Output: no results  <- legacy -/* namespace is dead; `doctor --fix` rewrites it

For an unattended host, use an API key. It is the supported path: pay-per-token, no expiry, no human in the loop. OAuth is subscription-priced but expires in days and needs a browser — on a 24/7 host you trade money against a recurring silent outage, and the outage wins. auth-profiles.json (in ~/.openclaw/agents//agent/) is a credential, not a config file — treat it like a private key: never in git, no SecretRef support. Copying it to another host is gray-area and not the recommended path: only acceptable if your provider's terms allow that account on that machine. Check the terms before you copy.

Failure #1: OAuth dies silently

A subscription OAuth token expires in days and does not refresh headless. The daemon keeps running; every turn just fails. Nobody is watching at 03:00. This is the single most common way an always-on agent dies.

OPENCLAW_HOME=~/.openclaw ./scripts/oauth-expiry-check.sh
# EARLIEST epoch expiry wins: a fresh profile must never mask a dead one, nor a 90-day refresh token
# a dead access token. It names the deciding `field` and NEVER exits non-zero. "Cannot determine" is
# NEVER `ok`: corrupt JSON, no python3, an unreadable (ISO-8601) expiry all emit `warn` — the mtime
# proxy is reached ONLY when the file parses and genuinely carries no expiry field at all.
# Output: {"status":"warn","source":"expires_at","field":"profiles.dead.expires_at","remaining_days":-1,"note":"...relogin..."}
# Output: {"status":"warn","source":"unresolved","reason":"corrupt_json","note":"Cannot verify token expiry..."}

Wire it to cr-23 (below), whose only instruction is: run the script, and if status is warn or missing, alert ${NOTIFY_CHANNEL} that a human relogin is needed. The cron must not attempt the login itself — that flow needs a real browser. Alert on warn; stay silent on ok.

Cron layer

The scheduler is in-process in the gateway; jobs persist to ~/.openclaw/cron/jobs.json. No system crontab.

{ id: "cr-10-worker", agentId: "main", enabled: true,
  schedule: { kind: "cron", expr: "0 9-18 * * 1-5", tz: "UTC" },  // IANA tz; or kind:"every"/"at"
  sessionTarget: "isolated", wakeMode: "now",   // isolated = inherits no context, no credentials-in-
  payload: { kind: "agentTurn", message: "…",   // conversation, no half-done state from another run
             model: "${MODEL_PRIMARY}", fallbacks: ["${MODEL_FALLBACK}"],
             tools: ["exec", "read"],           // ALWAYS explicit: a read-only audit and a job that
             thinking: "medium",                // pushes branches must not share an authority
             timeoutSeconds: 2400 },            // surface just because they share a host
  delivery: { mode: "announce", channel: "${CHANNEL}", to: "${NOTIFY_CHANNEL}" },
  failureAlert: { after: 2, to: "${NOTIFY_CHANNEL}", cooldownMs: 1800000 } }

The four housekeeping jobs the §Ops checklist watches — argv[0] is an absolute path in every case, a cron's PATH is not your shell's — then the four laws for anything that runs unattended:

• Idempotence first. A cron wakes with no memory. Before acting, read the state: your own index file and the remote's. In doubt, SKIP — a skipped run costs nothing, a duplicate action costs trust.
• Failure = no ack. If a check goes red, don't acknowledge. The item stays queued and a healthy later run takes it. Never ack to make an alert stop.
• Bounded backoff, not correction. The scheduler retries a failed run with bounded backoff (roughly ~30s → 1m → 5m → 15m, ~3 attempts, reset on success). Never depend on retry to fix a logic failure — that's what the un-acked queue is for.
• One normalized status line. Every job and script emits the same shape via scripts/cron-message.sh. Without it, ten jobs invent ten formats and the channel becomes unreadable within a week.

Output format — every job, every script

 - 
Result: 
Item: 
Next action: 
Details: 

dispatch - Started               <- emitted by the DISPATCHER when a preflight counts > 0;
Result: Job requested.              the preflight stays silent and execs it. Verified output.
Item: cr-10-worker
Next action: OpenClaw runs it now.
Details: event=work_ready count=2

Preflight layer

Stop paying an LLM turn to discover there is nothing to do. Split every heavy job in two: the preflight counts and, if count > 0, calls the dispatcher with a named event; the dispatcher runs the matching job(s) now. The heavy job keeps a widely-spaced every schedule as a safety net for the day a preflight silently breaks. Result: near-event-driven latency at poll-loop cost, with a floor under it.

{ id: "pf-10-queue", enabled: true, schedule: { kind: "every", ms: 600000 },   // 10 min
  payload: { kind: "command", argv: ["/abs/path/to/scripts/preflight-queue.sh"] } }  // cron PATH != yours

THE PREFLIGHT DECIDES NOTHING. IT COUNTS. It must never claim an item, mutate state, or reason about what to do — the moment a preflight starts judging, you have a second, cheaper, untested agent making production decisions in bash. Count, dispatch, exit.

every does not spread a fleet — it phase-locks it. The table above tells you to give each heavy job a widely-spaced every as its safety net. Do that for five jobs in one provisioning pass and all five fire together, forever. Interval schedules do not drift: equal intervals preserve whatever phase the jobs were created with, and a fleet provisioned in one pass was created seconds apart. Pick a round anchor and it is worse — five jobs at everyMs: 21600000 created on a timestamp that divides evenly by 6h land on the same 00/06/12/18 grid to the second. Five agent turns spawning worktrees, installs and a browser at the same instant on one host is a safety net that has become the outage.

Two things sharpen it. The fallback bypasses your only concurrency guard: the scheduler fires the job directly, so it never passes through event-dispatch.sh and neither the mkdir lock nor the runningAtMs check is in the loop — the one moment the fleet is most likely to collide is the one moment nothing is guarding it. And the daily restart re-anchors: cr-20 reloads every job at once, so a fleet you hand-staggered can silently re-converge after a reboot.

Verify rather than trust the shape: openclaw cron list, read the next-run times, and check that no two heavy jobs share a minute. If they do, you have not built a safety net — you have built a thundering herd with a 6-hour period.

./scripts/preflight-queue.sh
# Tolerant count: numeric `count`, else the SHALLOWEST array anywhere (nested {data:{items:[…]}} still
# counts; an API renaming `items` to `results` degrades to a count, not a crash loop). "Counted zero"
# is NOT "found nothing to count": an empty array is 0, but no-count-and-no-array — a 200 carrying
# {"error":"unauthorized"}, 502 HTML, a captive portal — is an Error. A green "Nothing to do" while the
# queue fills is worse than a crash. No secrets.env, dead host, junk body -> one status line, exit 0,
# never a bare shell error. Hence no `set -e`: a broken preflight must still produce a line.
# Output: pf-10-queue - Nothing to do / Result: Queue empty. / ... / Details: count=0
# Output: pf-10-queue - Error / Result: Payload carries no count and no array... / Details: reason=no_countable_field

scripts/event-dispatch.sh maps events → jobs, with three guards:

1. mkdir lock per job id — atomic, no race, released on RETURN, EXIT, INT and TERM, plus a TTL. trap ... RETURN alone is not enough: a kill -9, the 06:00 restart or a sleeping machine leaves the dir behind and blocks that event forever, while the channel keeps printing a healthy-looking Blocked. A lock older than LOCK_TTL_MIN (default 15) is reclaimed.
2. Job exists — openclaw cron get first. A non-zero exit, an empty answer, or a reply that is not JSON carrying that id is an Error line naming the fix. It classifies into exactly unknown | running | idle, so a release whose cron get exits 0 and prints prose ("job not found") cannot fall through to a run on a bogus id. Verify yours: openclaw cron get nope; echo $?.
3. runningAtMs guard — if the job is mid-run, report Blocked and return 0. Never stack turns on a running job.

One event may fan out to several jobs (source_changed → reviewer + drift-watch). The shipped mapping's cr-30-reviewer and cr-40-drift-watch are illustrative placeholders — this skill defines only cr-10-worker and cr-20…cr-23; until you point them at jobs you installed, those two events dispatch to nothing but an Error line (which is the designed failure, not a silent one). Keep the event vocabulary about what happened in the world (work_ready, review_needed, source_changed), never about which job you want to run — that's the mapping's job, and it's the only place that changes when you rename a cron.

Gotchas

Ops checklist

Scope

This skill ONLY: configures an OpenClaw host you control; polls endpoints you own or are authorized to call; schedules jobs with per-job tool restrictions; monitors auth expiry and alerts a human; deletes only its own logs past the stated window; emits normalized status lines.

This skill NEVER: exposes the control plane beyond loopback; bypasses an anti-abuse control, rate limit, or authentication check — when a defence triggers, the run stops and pings a human instead of adapting; performs an irreversible action unattended; conceals a failure behind an ack; writes secrets into the workspace, the notification channel, or git.

Alexandre Bloch — Bloch Agents · ClawHub @AlexBloch-IA