No Stripe objects, draft status flag set in SKILL.md frontmatter and index.json. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
125 lines
8.2 KiB
Markdown
125 lines
8.2 KiB
Markdown
---
|
|
name: loop-engineering
|
|
description: "Turn a one-off agent prompt into a self-supervising loop that runs unattended, safely. Five stages — manifest, unit, watchdog, attribution, re-grade — plus a human-approval gate on any outward-facing or irreversible step. Distilled from a production fleet that has run thousands of concurrent scheduled agent units. By Elite AI Empire (Our Tools)."
|
|
license: MIT
|
|
homepage: https://ourtools.eliteaiempire.com
|
|
tested_with: claude-code v2.x
|
|
status: draft
|
|
---
|
|
|
|
# Loop Engineering
|
|
|
|
**DRAFT — not for sale yet.** This is the packaged, sanitised version of a pattern we run in
|
|
production at real scale (thousands of concurrent scheduled agent units on our own fleet). No
|
|
internal hostnames, IP addresses, credentials, or private tooling paths are included below — only
|
|
the shape of the pattern, which is generic and reusable on any agent stack.
|
|
|
|
## The problem this solves
|
|
"Set up a recurring agent task" usually means a cron job that calls an LLM and hopes for the
|
|
best. That fails quietly in five specific ways: the job gets renamed/duplicated and nobody
|
|
notices; the process dies and nothing restarts it; a "successful" run silently did nothing (wrong
|
|
output path, stale credentials, disarmed permission); nobody can tell WHICH code version produced
|
|
a given result six weeks later; and old conclusions are trusted forever even after the logic
|
|
that produced them changed. A loop that has run unattended for months without any of these
|
|
failures needs five separate mechanisms, not one cron line.
|
|
|
|
## The five stages
|
|
|
|
### 1. Manifest
|
|
A single declarative registry of every unit the loop is allowed to run — not tribal knowledge
|
|
scattered across scripts. Each entry states: what the unit does, its priority tier (can it be
|
|
delayed under load, or must it always run first), its restart policy, and its resource budget.
|
|
Classify units by an explicit, ordered rule set (first-match-wins) rather than trusting free-text
|
|
descriptions — match on the unit's *actual* name and command line, never on a human-readable
|
|
description field, which can drift from what the unit really does.
|
|
**Why it matters:** without a manifest, "which jobs exist" is itself an unanswered question, and
|
|
a job that silently stopped being scheduled looks identical to a job that was never needed.
|
|
|
|
### 2. Unit
|
|
The thing that actually executes — a scheduled process, not a suggestion. Two properties matter
|
|
more than people expect:
|
|
- **Content AND executable-mode both have to be verified**, not just content. A file can be
|
|
byte-identical to its last-known-good version and still be silently broken if its execute bit
|
|
was stripped by an unrelated deploy — that failure mode is invisible to a hash check alone.
|
|
- **The running process, not just the file on disk, is the ground truth.** A script can be
|
|
patched correctly while the process still running in memory is executing the OLD bytes (started
|
|
before the patch landed) — check the process's start time against the file's modification time,
|
|
not just whether the file "looks right."
|
|
Pin anything load-bearing (golden copy + hash manifest) so an unrelated automated change can't
|
|
silently revert it without a deliberate, logged re-bless step.
|
|
|
|
### 3. Watchdog
|
|
A supervisor that checks the units, not a human remembering to look. Two properties, learned the
|
|
hard way:
|
|
- **Admission control, not just a batch runner.** If your loop's supervisor starts units in an
|
|
arbitrary order (e.g. alphabetical) under resource pressure, low-priority research jobs can
|
|
starve out the units that actually matter, because they happen to sort first. Always start
|
|
highest-priority tier first, unconditionally, with no resource veto on that tier; let only the
|
|
lower, deferrable tiers compete for what's left.
|
|
- **A monitor that goes silent reads as "all clear," which is the worst possible failure mode.**
|
|
Treat "is my watchdog itself still running" as a first-class thing to watch, at the same
|
|
priority as the units it supervises — never lower.
|
|
|
|
### 4. Attribution
|
|
Every run's output should be traceable to exactly the code, inputs, and time window that
|
|
produced it — a labeled claim (e.g. a status field written by the same code being audited) is not
|
|
independent evidence of anything; it's a claim the code makes about itself. Prefer external,
|
|
structurally-verifiable evidence (a fresh, uniquely-tagged output compared against an
|
|
independently-counted source of truth) over trusting a self-reported label. State plainly, next to
|
|
every result, what evidence backs it and what's still unverified — "unknown" is a valid, honest
|
|
answer; a made-up number is not.
|
|
|
|
### 5. Re-grade
|
|
Conclusions decay. The logic that graded a result last month may have had a bug that's since been
|
|
fixed, or the population it was measured against may have grown. A loop that never revisits old
|
|
verdicts under current logic will happily keep citing a wrong number forever. Schedule a periodic
|
|
re-grade pass that reapplies today's logic to historical results and flags any verdict that
|
|
flips — and report both halves of a verdict (did enough independent evidence accumulate, AND does
|
|
the effect clear the real-world cost of acting on it), never just one.
|
|
|
|
## The approval gate (the part that makes "unattended" safe)
|
|
Everything above runs without a human in the loop — **except** the step that does something
|
|
irreversible, outward-facing, or that spends real money. That single step always routes through an
|
|
explicit approval gate before it executes: a signed, scoped, time-limited permission slip that
|
|
names exactly what surface (e.g. "publish content," "push to a production repo," "place a real
|
|
order") and exactly what scope (which target) it authorises, expires on its own, and is logged
|
|
whether it succeeds or fails. The rest of the loop can run at 3am with nobody watching; the
|
|
approval-gated step cannot execute without that slip existing and matching.
|
|
|
|
**Design rule:** put the approval gate as far downstream as possible — research, verification,
|
|
and even drafting the outward-facing action should all run freely; only the actual publish/push/
|
|
spend call blocks on approval. Gating too early turns "self-supervising" back into "someone has to
|
|
babysit every step," which defeats the point.
|
|
|
|
## Templates
|
|
Three ready-to-import Orchestrator flow templates are included in `templates/`, each a `Workflow`
|
|
(ordered `steps: [{tool_slug, params}]`, a `min_tier`, and one step in each flow flagged
|
|
`hil: true` — human-in-the-loop / approval-gate required before that step runs):
|
|
|
|
| Template | Stages | Approval-gated step |
|
|
|---|---|---|
|
|
| `research-loop.json` | research → verify → propose → **approve** → act(paper/draft) → learn → monitor | the `act` step — anything that would leave the sandbox |
|
|
| `content-loop.json` | draft → review → **approve** → publish → measure → re-grade | the `publish` step — anything customer-facing |
|
|
| `infra-loop.json` | diagnose → patch → verify-on-canary → **approve** → ship → watchdog → re-grade | the `ship` step — anything pushed to a production system |
|
|
|
|
Import a template, point its steps at your own tools, and the approval-gated step will simply not
|
|
execute until whatever approval mechanism you wire in (a Slack approval, an email link, a signed
|
|
token — your infra's own equivalent of the gate above) has granted it.
|
|
|
|
## When to use this skill
|
|
- You're about to turn a one-off "run this prompt" into something that runs on a schedule.
|
|
- Your current cron-based automation has ever silently stopped working and nobody noticed for
|
|
days.
|
|
- You want "self-improving" agent loops (research → verify → execute → learn → monitor) without
|
|
giving up a human veto on the one step that can't be undone.
|
|
|
|
## What this skill does NOT claim
|
|
This is a pattern, not a product that "prints results while you sleep." Every stage above still
|
|
requires the underlying task (the research method, the verification statistic, the execution
|
|
logic) to be sound; a well-supervised loop around a null result is still a null result — it just
|
|
fails safely and tells you so, instead of failing silently.
|
|
|
|
## Upgrade
|
|
The hosted version of this pattern — pre-built manifest/watchdog/attribution tooling wired to your
|
|
own infra, plus the approval-gate integration — is on the **Our Tools** roadmap; this skill is the
|
|
portable, sanitised logic. No hosted product exists yet for this skill (draft).
|