diff options
Diffstat (limited to 'docs/manual.md')
| -rw-r--r-- | docs/manual.md | 239 |
1 files changed, 239 insertions, 0 deletions
diff --git a/docs/manual.md b/docs/manual.md new file mode 100644 index 0000000..2b4594b --- /dev/null +++ b/docs/manual.md @@ -0,0 +1,239 @@ +# wf — owner's manual + +How to set up and run `wf` day to day. The README tells you what it is and why it exists. The +reference docs give the details: [design.md](design.md) (format, config, every command), +[orchestrator.md](orchestrator.md), [lanes.md](lanes.md), [multi-session.md](multi-session.md), +[resource-ledger.md](resource-ledger.md). Every command below also has `wf <command> -h`. + +This is still a personal setup. The manual describes how it runs on the author's machine. It is +not a supported product. + +## 1. Setup + +### Requirements +- Linux with systemd. `wf res` needs systemd user units and cgroup v2. Everything else only needs a shell. +- Python ≥ 3.11 (standard library only, nothing to `pip install`). +- git, and [Claude Code](https://claude.com/claude-code) (`claude` on `PATH`). +- Optional: the `superpowers` Claude Code plugin. The shared rules refer to its brainstorming, + debugging and plan-writing skills. Without it, agents just follow the plain rules. + +### One root folder +All projects live under one folder. The shipped rules, skills and agent file use the absolute +path `/projects` (for example `python3 /projects/public/workflow/wf.py`). Keep that path: create it once +and own it, or point it at a folder in your home directory. + +```sh +sudo mkdir /projects && sudo chown "$USER": /projects # or: sudo ln -s ~/projects /projects +git clone <this repo> /projects/public/workflow +ln -s workflow/shared/CLAUDE.md /projects/CLAUDE.md # every agent under /projects loads it +echo "alias wf='python3 /projects/public/workflow/wf.py'" >> ~/.bashrc +``` + +Claude Code reads `CLAUDE.md` from the working directory and from every parent folder, so every +session in `/projects/<anything>` gets the shared rules. If the symlink is not picked up, use a +one-line `/projects/CLAUDE.md` with the content `@workflow/shared/CLAUDE.md` instead. +`wf projects` scans the parent folder of the tool repo, or its grandparent when the parent holds no project (tool under e.g. `public/`; override: `WF_ROOT`). Workflow reports +go to `/projects/public/workflow/inbox.md` (override: `WF_INBOX`). + +### Agent and skills +The orchestrator spawns the worker subagent. The skills are the session modes in section 2. +Install them as symlinks, so that `git pull` updates them: + +```sh +mkdir -p ~/.claude/agents ~/.claude/skills +ln -s /projects/public/workflow/shared/agents/wf-worker.md ~/.claude/agents/wf-worker.md +for s in wf-orchestrate wf-pilot wf-design; do ln -s /projects/public/workflow/shared/skills/$s ~/.claude/skills/; done +``` + +Agents and skills appear only in sessions started after the install, so restart any running session. + +### A project +```sh +cd /projects/games/mygame # any depth ≤ 3 under /projects; a git repo is recommended +wf init # writes workflow.toml, TASKS.md, tasks/archive.md, CLAUDE.md (keeps existing files) +wf check # 0 errors +``` +If the project already has a numbered task list, `wf migrate` shows the conversion (dry run) and +`wf migrate --write` applies it. + +Edit the generated `CLAUDE.md`: layout, verify command, gotchas and one `### <area>` block per +code area (code map with grep anchors, test recipe, paths). Agents read these area notes before +they read code. + +`workflow.toml` (the template lists every key with a comment; unknown keys are errors): + +| Key | What it does | +|---|---| +| `tasks`, `archive` | file names (defaults `TASKS.md`, `tasks/archive.md`) | +| `docs` | files/folders whose `[[id]]` links `wf check` validates | +| `verify` | commands `wf done` reminds of (it never runs them) and `wf ctx` prints as Verify | +| `done` | extra checklist lines `wf done` prints | +| `quick_gate` | fast regression check; `wf gate` and `wf finish` run it and refuse when it is red | +| `worktree_setup` | idempotent commands `wf setup` runs in a fresh lane worktree (link git-ignored data, venv…; env `WF_MAIN` = main tree) | +| `slice_above`, `[lanes.*]` | task sizes per lane and when a task must be sliced (default lanes: fast `<1h`, slow `1h`); see lanes.md | +| `areas`, `code_root`, `area_stale_commits`, `area_ignore` | where the area notes live and when they count as stale | +| `ledgers`, `ctx_hint`, `[anchors]` | in-flight plan ledgers, `/clear` hint, required Ref anchors | +| `cloud`, `cloud_include`, `cloud_note` | cloud lane opt-in (below) | + +Add `out/` to the project's `.gitignore`: logs, batch summaries and scratch go there. + +### Resource ledger (`wf res`) +Optional, but needed once several agents build or test at the same time. + +```sh +wf res shell-init # prints an alias that starts claude inside agents.slice: add it to ~/.bashrc +wf res timer on # 1-minute timer: starts queued jobs, ends game mode, cleans scratch, adopts sessions +wf res status # budget, reservations, warnings +``` +Reserves and limits go in `~/.config/wf/resources.toml` (all keys optional: `user_reserve_gb`, +`user_reserve_cpus`, `game_reserve_gb`, `game_reserve_cpus`, `game_hours`, `small_headroom_gb`, +`scratch_hours`; see resource-ledger.md §3.3). Optional hook that hides the display from agent +commands while game mode is on, in `~/.claude/settings.json`: + +```json +"hooks": {"PreToolUse": [{"matcher": "Bash", "hooks": [{"type": "command", + "command": "python3 /projects/public/workflow/wf_res.py hook"}]}]} +``` + +`wf batch` (and `/wf-pilot`) need one allow rule in `~/.claude/settings.json`, because auto mode +does not let an agent start another unattended agent: `Bash(python3 /projects/public/workflow/wf.py batch:*)`. + +### Cloud lane (optional) +Tasks can also run as `claude --cloud` sessions, which are billed to your cloud budget, not your +machine. You need a Claude Code version with `--cloud` / `--teleport` and a claude.ai login. +Per project: `cloud = true` in `workflow.toml`, plus `cloud_include` for git-ignored inputs the +task needs. Set the spending cap once with `wf cloud ledger --budget 100` and check it with +`wf cloud ledger`. Only opus tasks marked `Cloud: yes` (or passing the fit rules) are sent. See +orchestrator.md "Cloud lane". + +## 2. Daily use + +There are four kinds of session. Start each one in the project folder with `claude` (the +`agents.slice` alias if you set it up). + +### Worker session: "continue" +Type `continue` (or "next task"). The agent runs `wf next --as <its model>`. It shows open +questions, items that need you, plans in flight and the next task, then works on that task: +test red → implement → green → `wf done` → one commit. It does not ask before it starts. +Several sessions in one project: give each a lane ("continue in the slow lane"). `wf next` then +switches them to worktree mode (one branch per task, `wf finish` merges it). One session per lane. + +### Orchestrator: `/wf-orchestrate` +After `/clear`, type `/wf-orchestrate` (`go` = start the proposed run without asking, +`batch N` = prepare an unattended batch). The orchestrator holds no lane and never reads code. +It spawns one fresh `wf-worker` per task (`wf orch pick` / `wf orch post`), one per lane in +parallel, on the task's model, and stops a lane on the first awaiting, handback or red +post-check. Use it when the backlog is runner-ready (section 3) and you want to steer. + +### Overnight: `/wf-pilot` +`/wf-pilot [--batch 4] [--lanes a,b] [--for 8h]` in an idle session, which can be one you opened +from the phone. The pilot starts headless `wf batch` runs one after another until the deadline. +The session stays small, and you get push notifications on new questions, on stops and at the +end. Steering: `wf batch --stop` (graceful, running workers finish), `wf move <id> deferred` +(skip a task), `wf batch --status` (newest batch and its summary). A one-off short run without a +pilot is `wf batch N [--lanes …]`. `wf batch N --prep` makes tasks without a Done line +runner-ready instead. + +### Design: `/wf-design` +`/wf-design [topic | id]` is for brainstorming, specs and rulings with you. Flow: brainstorm → +spec (committed) → your approval → `wf add` slices ≤ 1h with Done, Model and Ref → the +orchestrator picks them up. It answers open questions with you and records each ruling where it +binds. + +### Your side +- **Awaiting** (`wf list -s awaiting`): questions agents could not decide. Answer them in a + design or orchestrator session. The session records the ruling and runs `wf done <a-id>`, + which unblocks the waiting tasks. +- **Needs human** (`wf list -s human`): things only you can do (play-test, hardware, accounts). + Check boxes with `wf tick <id> <n>`. Done → tell any session ("t-x is done") or run `wf done`. + Tasks marked `Sessions: owner` are picked only when you say you are present (`wf next --owner`). +- **Stop file**: `wf batch --stop` writes `out/wf-batch.stop`. Batches, pilots and + `wf lanes --wait` stop before the next spawn. +- **Game mode**: `wf res game on [--for 4h]` before you play. Agents keep running but get less + CPU and IO, and big jobs that do not fit wait. `wf res game off` when you are done. +- **Overview**: `wf projects` (all projects), `wf lanes`, `wf log -n 10`, `wf res status`. +- **Workflow inbox**: agents file problems with `wf report`. Triage `/projects/public/workflow/inbox.md` + now and then: fix the tool, or turn entries into tasks. + +## 3. Writing good tasks + +```sh +wf add "Cave seams. Close the slit beside the lintel." -p 1 -e '<1h' --model sonnet \ + --done "tests/test_cave.py::test_seam passes; render of fixture shows no gap" --ref DESIGN.md#terrain +``` + +- **Title. Goal.** One line: the title, then the outcome in one sentence. +- **Done line** (`--done`): a check a worker can run without you, such as a file, a test result + or an artifact. Never "owner confirms". Only tasks with a Done line are runner-ready: the + orchestrator, batch and pilot skip the others. (`wf batch --prep` drafts them.) +- **Model line** (`--model`): the cheapest model that fits. haiku = mechanical, exact steps. + sonnet = exact Done, an existing pattern to copy, and a real-data pass/fail. opus (the default + when the line is missing) = design, debugging, reverse engineering, anything unclear. A worker + that finds the task too hard raises it itself and hands it back. +- **Effort** (`-e`): `<1h`, `1h`, `5h`, `10h`, `100h`. Anything above `slice_above` (1h) becomes a + slice job: an agent splits it into `--parent` slices, each with its own Done/Model, and never + implements it whole. Slice big work yourself when you already know the steps. +- **Code anchors**: when you know them, add body lines like + `Code: src/terrain.py build_cave; test to copy: tests/test_cave.py::test_floor`. Write anchors + (function or string names), not line numbers. They save the worker most of its exploring. +- **Refs and dependencies**: `--ref doc.md#heading` makes `wf ctx` print that section; + `--after t-other` keeps it unpicked until `t-other` is done. +- **Areas**: keep each area's code map and test recipe in the project `CLAUDE.md`. `wf ctx` + prints the areas a task names. When code drifts, `wf done` adds a `t-map-…` refresh task + automatically. +- **Cloud** (cloud projects): `--cloud yes` only when the Done line can be checked from the repo + alone (no GUI, live server, LAN, local installs). When unsure, `no`. +- **Sessions**: `--sessions solo` for repo-wide changes (nothing else runs meanwhile), + `--sessions owner` when you must be there. + +## 4. Best practices and pitfalls + +- **One lane per session**, at most one worker per lane worktree. Parallel lanes are safe + (project lock, branch per task). Two sessions in one lane pick against each other. +- **Fresh context per task.** Cost is mostly cache reads, which grow with context. Short-lived + workers on the cheapest fitting model are cheaper than one long session. `/clear` the + orchestrator after a big design talk, but only when no worker is running (a `/clear` kills its + running command). +- **Big jobs through `wf res`**: anything over 2 GB, 4 cores or 10 minutes goes through + `wf res run --mem … --for … --title … -- cmd`. Exit 3 = busy: do other work and retry later, + or use `--queue`. `wf res hist` suggests sizes from past runs. +- **Never hand-edit TASKS.md** while agents run. Use `wf add/set/note/move/done`. If you must + edit by hand, run `wf check` afterwards. +- **Commit ≠ merge ≠ push.** Agents commit their own task. Workers ff-merge only their own task + branch. Pushing goes only to a remote named `home` (`git push home --all`), and nothing else is + pushed unless you ask. +- **Close the loop on problems.** Agents work around a workflow problem and `wf report` it. + Triaging the inbox is how the rules get better. Put project-only needs in the project's + `CLAUDE.md` or `workflow.toml`, not in the shared rules. +- **Watch cost**: `wf usage` (this session per agent), `wf usage --report` (per lane, model and + effort across every project's `out/wf-cost.log`). Expensive lanes usually mean vague Done lines + or a model that is too large. +- **Model choice**: run the orchestrator and design sessions on opus. Tasks get the cheapest + model that fits. A sonnet task without an exact Done line comes back as a handback. +- **Updating the tool**: `git -C /projects/public/workflow pull`. Read the top of `CHANGES.md`: each line + ends with "Projects: …", which says what a project must do (usually nothing). +- **Publishing a public copy**: keep the tool repo private (its history names your projects) and + share a fresh-history snapshot instead. Put your private words (project names, home paths), one + per line, in `publish-denylist.local` in the tool repo (git-ignored), then run + `python3 scripts/publish_snapshot.py [DEST]` (default `~/src/wf-public`). It copies master's + tree without `inbox.md`, `.worktrees/`, `__pycache__/`, `out/`, refuses if any denylist word is + in a path or file, and commits once per run (first: "initial public snapshot"; later: the new + `CHANGES.md` lines). It never pushes and never adds a remote: review DEST, then add the public + remote and `git push` there yourself. + +## 5. Troubleshooting + +| Symptom | Cause / fix | +|---|---| +| `busy: …` and exit 3 from `wf res run/note` | Not enough memory or CPUs beyond your reserve. Retry after the printed time, add `--queue`, or check `wf res status` for stale entries (`wf res release r-N`). If memory really is free (unused reservations), `--force` fits against real free memory. | +| `wf orch post` says post-check red | The worker said done, but the archive line is missing, the branch is still there, the worktree is dirty, or `wf check` has errors. The message names the problem. Fix it in the worktree (`wf merge`, commit or clean), then post again. | +| Task stuck `in progress` though no one works on it | A session or worker died. `wf list --stale` shows them, and `wf status --clear-stale` clears every claim without a live session (or `wf status <id> clear`). Re-run the worker with `wf orch pick <lane> --id <id> --recovery "<why>"` to keep its WIP. | +| `wf start` refuses: worktree has uncommitted changes | A dead worker left WIP. `--recovery` keeps it and shows it. Otherwise commit or reset it in that worktree. | +| `branch <lane>/<id> exists (local WIP?)` (cloud pull) | A local branch for that task already exists. Merge or delete it, then `wf cloud pull <id>` again. The cloud session is kept. | +| `wf cloud pull` exit 4 | The session is still running. Pull again later; after 24 h without a result it counts as lost. | +| `wf cloud pull` handback / bad patch | The task goes back to its local lane with a `Recovery: cloud attempt …` note. The patch is kept in `out/cloud/<id>.patch`. | +| `wf merge`: rebase conflict | The branch is untouched. `git rebase master`, resolve, run verify, `wf merge` again. | +| `wf finish` refused | The quick gate is red, files outside the given paths are uncommitted, or a path is outside the repo, missing or unchanged. Fix it and rerun. If it fails after `done:`, a rerun resumes. | +| A worker or batch does nothing | `wf list --runner` is empty: tasks lack a Done line (`wf batch --prep`), are blocked on an awaiting item, or wait on `After:`. `wf lanes` shows counts per lane. | +| `wf check` errors after a hand edit | Read the message (duplicate id, broken `[[link]]`, bad section) and fix the item. Ids are never reused. | +| `project format N is newer than this wf` | Update the tool: `git -C /projects/public/workflow pull`. | |