From 9195c4be7f68df979e769f8bf09e8f277d49cfc1 Mon Sep 17 00:00:00 2001 From: Lauren ten Hoor Date: Sun, 8 Feb 2026 16:12:23 +0800 Subject: [PATCH] Add ONBOARDING.md and ARCHITECTURE.md docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ONBOARDING: Step-by-step setup guide — prerequisites, plugin install, agent config, GitLab labels, project registration, first test. ARCHITECTURE: Full component interaction — system overview diagram, complete ticket lifecycle with sequence diagrams for every phase (heartbeat → pickup → dev work → complete → QA → pass/fail/refine), data flow map, scope boundaries, error recovery, file locations. Co-Authored-By: Claude Opus 4.6 --- docs/ARCHITECTURE.md | 393 +++++++++++++++++++++++++++++++++++++++++++ docs/ONBOARDING.md | 156 +++++++++++++++++ 2 files changed, 549 insertions(+) create mode 100644 docs/ARCHITECTURE.md create mode 100644 docs/ONBOARDING.md diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..d8d0577 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,393 @@ +# DevClaw — Architecture & Component Interaction + +## System overview + +```mermaid +graph TB + subgraph "External" + USER[Human / User] + GL[GitLab] + TG[Telegram] + end + + subgraph "OpenClaw Runtime" + AGENT[Orchestrator Agent / PM] + DEV[DEV sub-agent session] + QA[QA sub-agent session] + end + + subgraph "DevClaw Plugin" + TP[task_pickup] + TC[task_complete] + QS[queue_status] + SH[session_health] + MS[Model Selector] + PJ[projects.json] + AL[audit.log] + end + + subgraph "Git" + REPO[Project Repository] + end + + USER -->|creates issues| GL + USER -->|sends messages| TG + TG -->|delivers messages| AGENT + + AGENT -->|calls| TP + AGENT -->|calls| TC + AGENT -->|calls| QS + AGENT -->|calls| SH + + TP -->|selects model| MS + TP -->|transitions labels| GL + TP -->|reads/writes| PJ + TP -->|appends| AL + + TC -->|transitions labels| GL + TC -->|closes/reopens| GL + TC -->|reads/writes| PJ + TC -->|git pull| REPO + TC -->|appends| AL + + QS -->|lists issues by label| GL + QS -->|reads| PJ + QS -->|appends| AL + + SH -->|reads/writes| PJ + SH -->|reverts labels| GL + SH -->|appends| AL + + AGENT -->|sessions_spawn| DEV + AGENT -->|sessions_spawn| QA + AGENT -->|sessions_send| DEV + AGENT -->|sessions_send| QA + + DEV -->|writes code, creates MRs| REPO + QA -->|reviews code, tests| REPO + + AGENT -->|announces to group| TG +``` + +## Complete ticket lifecycle + +This traces a single issue from creation to completion, showing every component interaction, data write, and message. + +### Phase 1: Issue created + +``` +Human → GitLab: creates issue #42 with label "To Do" +``` + +**State:** GitLab has issue #42 labeled "To Do". Nothing in DevClaw yet. + +### Phase 2: Heartbeat detects work + +``` +Heartbeat triggers → Agent calls queue_status() +``` + +```mermaid +sequenceDiagram + participant A as PM Agent + participant QS as queue_status + participant GL as GitLab + participant PJ as projects.json + participant AL as audit.log + + A->>QS: queue_status({ projectGroupId: "-123" }) + QS->>PJ: readProjects() + PJ-->>QS: { dev: idle, qa: idle } + QS->>GL: glab issue list --label "To Do" + GL-->>QS: [{ id: 42, title: "Add login page" }] + QS->>GL: glab issue list --label "To Test" + GL-->>QS: [] + QS->>GL: glab issue list --label "To Improve" + GL-->>QS: [] + QS->>AL: append { event: "queue_status", ... } + QS-->>A: { dev: idle, queue: { toDo: [#42] } } +``` + +**Agent decides:** DEV is idle, issue #42 is in To Do → pick it up. + +### Phase 3: DEV pickup + +```mermaid +sequenceDiagram + participant A as PM Agent + participant TP as task_pickup + participant GL as GitLab + participant MS as Model Selector + participant PJ as projects.json + participant AL as audit.log + participant TG as Telegram + + A->>TP: task_pickup({ issueId: 42, role: "dev", projectGroupId: "-123" }) + TP->>PJ: readProjects() + PJ-->>TP: { dev: { active: false, sessionId: "existing-session" } } + TP->>GL: glab issue view 42 --output json + GL-->>TP: { title: "Add login page", labels: ["To Do"] } + TP->>TP: Verify label is "To Do" ✓ + TP->>MS: selectModel("Add login page", description, "dev") + MS-->>TP: { alias: "sonnet", reason: "Standard dev task" } + TP->>TP: Existing session found → sessionAction: "send" + TP->>GL: glab issue update 42 --unlabel "To Do" --label "Doing" + TP->>PJ: activateWorker(-123, dev, { issueId: "42", model: "sonnet" }) + TP->>AL: append { event: "task_pickup", ... } + TP->>AL: append { event: "model_selection", ... } + TP-->>A: { sessionAction: "send", sessionId: "existing-session", announcement: "🔧 Sending DEV (sonnet) for #42: Add login page" } + A->>TG: "🔧 Sending DEV (sonnet) for #42: Add login page" + A->>A: sessions_send(sessionId, task description) +``` + +**Writes:** +- `GitLab`: label "To Do" → "Doing" +- `projects.json`: dev.active=true, dev.issueId="42", dev.model="sonnet" +- `audit.log`: 2 entries (task_pickup, model_selection) +- `Telegram`: announcement message + +### Phase 4: DEV works + +``` +DEV sub-agent → reads codebase, writes code, creates MR +DEV sub-agent → reports back to PM: "done, MR merged" +``` + +This happens inside the OpenClaw session. DevClaw is not involved — the DEV sub-agent works autonomously with the codebase. + +### Phase 5: DEV complete + +```mermaid +sequenceDiagram + participant A as PM Agent + participant TC as task_complete + participant GL as GitLab + participant PJ as projects.json + participant AL as audit.log + participant REPO as Git Repo + participant TG as Telegram + + A->>TC: task_complete({ role: "dev", result: "done", projectGroupId: "-123", summary: "Login page with OAuth" }) + TC->>PJ: readProjects() + PJ-->>TC: { dev: { active: true, issueId: "42", sessionId: "existing-session" } } + TC->>REPO: git pull + TC->>PJ: deactivateWorker(-123, dev) + Note over PJ: active→false, issueId→null
sessionId PRESERVED
model PRESERVED + TC->>GL: glab issue update 42 --unlabel "Doing" --label "To Test" + TC->>AL: append { event: "task_complete", role: "dev", result: "done" } + TC-->>A: { announcement: "✅ DEV done #42 — Login page with OAuth. Moved to QA queue." } + A->>TG: "✅ DEV done #42 — Login page with OAuth. Moved to QA queue." +``` + +**Writes:** +- `Git repo`: pulled latest (has DEV's merged code) +- `projects.json`: dev.active=false, dev.issueId=null (sessionId + model preserved for reuse) +- `GitLab`: label "Doing" → "To Test" +- `audit.log`: 1 entry (task_complete) +- `Telegram`: announcement + +### Phase 6: QA pickup + +Same as Phase 3, but with `role: "qa"`. Label transitions "To Test" → "Testing". Model defaults to Grok for QA. + +### Phase 7: QA result (3 possible outcomes) + +#### 7a. QA Pass + +```mermaid +sequenceDiagram + participant A as PM Agent + participant TC as task_complete + participant GL as GitLab + participant PJ as projects.json + participant AL as audit.log + participant TG as Telegram + + A->>TC: task_complete({ role: "qa", result: "pass", projectGroupId: "-123" }) + TC->>PJ: deactivateWorker(-123, qa) + TC->>GL: glab issue update 42 --unlabel "Testing" --label "Done" + TC->>GL: glab issue close 42 + TC->>AL: append { event: "task_complete", role: "qa", result: "pass" } + TC-->>A: { announcement: "🎉 QA PASS #42. Issue closed." } + A->>TG: "🎉 QA PASS #42. Issue closed." +``` + +**Ticket complete.** Issue closed, label "Done". + +#### 7b. QA Fail + +```mermaid +sequenceDiagram + participant A as PM Agent + participant TC as task_complete + participant GL as GitLab + participant MS as Model Selector + participant PJ as projects.json + participant AL as audit.log + participant TG as Telegram + + A->>TC: task_complete({ role: "qa", result: "fail", projectGroupId: "-123", summary: "OAuth redirect broken" }) + TC->>PJ: deactivateWorker(-123, qa) + TC->>GL: glab issue update 42 --unlabel "Testing" --label "To Improve" + TC->>GL: glab issue reopen 42 + TC->>GL: glab issue view 42 --output json + TC->>MS: selectModel(title, description, "dev") + MS-->>TC: { alias: "sonnet" } + TC->>AL: append { event: "task_complete", role: "qa", result: "fail" } + TC-->>A: { announcement: "❌ QA FAIL #42 — OAuth redirect broken. Sent back to DEV.", devFixInstructions: "Send QA feedback to existing DEV session..." } + A->>TG: "❌ QA FAIL #42 — OAuth redirect broken. Sent back to DEV." +``` + +**Cycle restarts:** Issue goes to "To Improve". Next heartbeat, DEV picks it up again (Phase 3, but from "To Improve" instead of "To Do"). + +#### 7c. QA Refine + +``` +Label: "Testing" → "Refining" +``` + +Issue needs human decision. Pipeline pauses until human moves it to "To Do" or closes it. + +### Phase 8: Heartbeat (continuous) + +The heartbeat runs periodically (triggered by the agent or a scheduled message). It combines health check + queue scan: + +```mermaid +sequenceDiagram + participant A as PM Agent + participant SH as session_health + participant QS as queue_status + participant TP as task_pickup + participant SL as sessions_list + + Note over A: Heartbeat triggered + + A->>SL: sessions_list + SL-->>A: [alive_session_1, alive_session_2] + + A->>SH: session_health({ activeSessions: [...], autoFix: true }) + SH-->>A: { healthy: false, issues: [{ type: "zombie_session", fixed: true }] } + + A->>QS: queue_status() + QS-->>A: { projects: [{ dev: idle, queue: { toDo: [#43], toTest: [#44] } }] } + + Note over A: DEV idle + To Do #43 → pick up + A->>TP: task_pickup({ issueId: 43, role: "dev", ... }) + + Note over A: QA idle + To Test #44 → pick up + A->>TP: task_pickup({ issueId: 44, role: "qa", ... }) +``` + +## Data flow map + +Every piece of data and where it lives: + +``` +┌─────────────────────────────────────────────────────────────────┐ +│ GitLab (source of truth for tasks) │ +│ │ +│ Issue #42: "Add login page" │ +│ Labels: [To Do | Doing | To Test | Testing | Done | ...] │ +│ State: open / closed │ +│ MRs: linked merge requests │ +└─────────────────────────────────────────────────────────────────┘ + ↕ glab CLI (read/write) +┌─────────────────────────────────────────────────────────────────┐ +│ DevClaw Plugin (orchestration logic) │ +│ │ +│ task_pickup → label transition + state update + model select│ +│ task_complete → label transition + state update + git pull │ +│ queue_status → read labels + read state │ +│ session_health → read state + fix zombies │ +└─────────────────────────────────────────────────────────────────┘ + ↕ atomic file I/O +┌─────────────────────────────────────────────────────────────────┐ +│ memory/projects.json (worker state) │ +│ │ +│ Per project (keyed by Telegram group ID): │ +│ dev: { active, sessionId, issueId, model, startTime } │ +│ qa: { active, sessionId, issueId, model, startTime } │ +│ │ +│ Preserved across tasks: sessionId, model, startTime │ +│ Cleared on complete: active → false, issueId → null │ +└─────────────────────────────────────────────────────────────────┘ + ↕ append-only +┌─────────────────────────────────────────────────────────────────┐ +│ memory/audit.log (observability) │ +│ │ +│ NDJSON, one line per event: │ +│ task_pickup, task_complete, model_selection, │ +│ queue_status, health_check │ +│ │ +│ Query with: cat audit.log | jq 'select(.event=="task_pickup")' │ +└─────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────┐ +│ Telegram (user-facing messages) │ +│ │ +│ Per group chat: │ +│ "🔧 Spawning DEV (sonnet) for #42: Add login page" │ +│ "✅ DEV done #42 — Login page with OAuth. Moved to QA queue."│ +│ "🎉 QA PASS #42. Issue closed." │ +│ "❌ QA FAIL #42 — OAuth redirect broken. Sent back to DEV." │ +└─────────────────────────────────────────────────────────────────┘ + +┌─────────────────────────────────────────────────────────────────┐ +│ Git Repository (codebase) │ +│ │ +│ DEV sub-agent: reads code, writes code, creates MRs │ +│ QA sub-agent: reads code, runs tests, reviews MRs │ +│ task_complete (DEV done): git pull to sync latest │ +└─────────────────────────────────────────────────────────────────┘ +``` + +## Scope boundaries + +What DevClaw controls vs. what it delegates: + +```mermaid +graph LR + subgraph "DevClaw controls" + L[Label transitions] + S[Worker state] + M[Model selection] + A[Audit logging] + Z[Zombie cleanup] + end + + subgraph "Agent handles (with DevClaw instructions)" + SP[Session spawn/send] + MSG[Telegram announcements] + HB[Heartbeat scheduling] + end + + subgraph "External (not DevClaw)" + IC[Issue creation] + CR[Code writing] + MR[MR creation/review] + DEPLOY[Deployment] + HR[Human decisions] + end +``` + +## Error recovery + +| Failure | Detection | Recovery | +|---|---|---| +| Session dies mid-task | `session_health` detects zombie (active=true but session not in sessions_list) | `autoFix`: reverts label, clears active state. Next heartbeat picks up task again. | +| glab command fails | Tool throws error, returns to agent | Agent retries or reports to Telegram group | +| projects.json corrupted | Tool can't parse JSON | Manual fix needed. Atomic writes (temp+rename) prevent partial writes. | +| Label out of sync | `task_pickup` verifies label before transitioning | Throws error if label doesn't match expected state. Agent reports mismatch. | +| Worker already active | `task_pickup` checks `active` flag | Throws error: "DEV worker already active on project". Must complete current task first. | +| Stale worker (>2h) | `session_health` flags as warning | Agent can investigate or `autoFix` can clear. | + +## File locations + +| File | Location | Purpose | +|---|---|---| +| Plugin source | `~/.openclaw/extensions/devclaw/` | Plugin code | +| Plugin manifest | `~/.openclaw/extensions/devclaw/openclaw.plugin.json` | Plugin registration | +| Agent config | `~/.openclaw/openclaw.json` | Agent definition + tool permissions | +| Worker state | `~/.openclaw/workspace-/memory/projects.json` | Per-project DEV/QA state | +| Audit log | `~/.openclaw/workspace-/memory/audit.log` | NDJSON event log | +| Git repos | `~/git//` | Project source code | diff --git a/docs/ONBOARDING.md b/docs/ONBOARDING.md new file mode 100644 index 0000000..8cd336d --- /dev/null +++ b/docs/ONBOARDING.md @@ -0,0 +1,156 @@ +# DevClaw — Onboarding Guide + +## What you need before starting + +| Requirement | Why | How to check | +|---|---|---| +| [OpenClaw](https://openclaw.ai) installed | DevClaw is an OpenClaw plugin | `openclaw --version` | +| Node.js >= 20 | Runtime for plugin | `node --version` | +| [`glab`](https://gitlab.com/gitlab-org/cli) CLI | GitLab issue/label management | `glab --version` | +| glab authenticated | Plugin calls glab for every label transition | `glab auth status` | +| A GitLab repo with issues | The task backlog lives in GitLab | `glab issue list` from your repo | +| An OpenClaw agent with Telegram | The PM agent that will orchestrate | Agent defined in `openclaw.json` | + +## Setup steps + +### 1. Install the plugin + +```bash +# Copy to extensions directory (auto-discovered on next restart) +cp -r devclaw ~/.openclaw/extensions/ +``` + +Verify: +```bash +openclaw plugins list +# Should show: DevClaw | devclaw | loaded +``` + +### 2. Configure your orchestrator agent + +In `openclaw.json`, your orchestrator agent needs access to the DevClaw tools: + +```json +{ + "agents": { + "list": [{ + "id": "my-orchestrator", + "name": "Dev PM", + "model": "anthropic/claude-sonnet-4-5", + "tools": { + "allow": [ + "task_pickup", + "task_complete", + "queue_status", + "session_health", + "sessions_spawn", + "sessions_send", + "sessions_list" + ] + } + }] + } +} +``` + +The agent also needs the OpenClaw session tools (`sessions_spawn`, `sessions_send`, `sessions_list`) — DevClaw handles the orchestration logic, but the agent executes the actual session operations. + +### 3. Create GitLab labels + +DevClaw uses these labels as a state machine. Create them once per GitLab project: + +```bash +cd ~/git/your-project +glab label create "Planning" --color "#6699cc" +glab label create "To Do" --color "#428bca" +glab label create "Doing" --color "#f0ad4e" +glab label create "To Test" --color "#5bc0de" +glab label create "Testing" --color "#9b59b6" +glab label create "Done" --color "#5cb85c" +glab label create "To Improve" --color "#d9534f" +glab label create "Refining" --color "#f39c12" +``` + +### 4. Register a project + +Add your project to `memory/projects.json` in the orchestrator's workspace: + +```json +{ + "projects": { + "": { + "name": "my-project", + "repo": "~/git/my-project", + "groupName": "Dev - My Project", + "deployUrl": "https://my-project.example.com", + "baseBranch": "development", + "deployBranch": "development", + "dev": { + "active": false, + "sessionId": null, + "issueId": null, + "startTime": null, + "model": null + }, + "qa": { + "active": false, + "sessionId": null, + "issueId": null, + "startTime": null, + "model": null + } + } + } +} +``` + +**Finding the Telegram group ID:** The group ID is the numeric ID of your Telegram supergroup (a negative number like `-1234567890`). You can find it via the Telegram bot API or from message metadata in OpenClaw logs. + +### 5. Add the agent to the Telegram group + +Add your orchestrator bot to the Telegram group for the project. The agent will now receive messages from this group and can operate on the linked project. + +### 6. Create your first issue + +```bash +cd ~/git/my-project +glab issue create --title "My first task" --label "To Do" +``` + +### 7. Test the pipeline + +Ask the agent in the Telegram group: + +> "Check the queue status" + +The agent should call `queue_status` and report the "To Do" issue. Then: + +> "Pick up issue #1 for DEV" + +The agent calls `task_pickup`, which selects a model, transitions the label to "Doing", and returns instructions to spawn or reuse a DEV session. + +## Adding more projects + +Repeat steps 3-5 for each new project: +1. Create labels in the GitLab repo +2. Add an entry to `projects.json` with the new Telegram group ID +3. Add the bot to the new Telegram group + +Each project is fully isolated — separate queue, separate workers, separate state. + +## What the plugin handles vs. what you handle + +| Responsibility | Who | Details | +|---|---|---| +| GitLab label setup | You (once per project) | 8 labels, created via `glab label create` | +| Project registration | You (once per project) | Entry in `projects.json` | +| Agent definition | You (once) | Agent in `openclaw.json` with tool permissions | +| Telegram group setup | You (once per project) | Add bot to group | +| Task creation | You or external | Create GitLab issues with labels | +| Label transitions | Plugin | Atomic `--unlabel` + `--label` via glab | +| Model selection | Plugin | Keyword-based heuristic per task | +| State management | Plugin | Atomic read/write to `projects.json` | +| Session reuse | Plugin | Detects existing sessions, returns spawn vs send | +| Audit logging | Plugin | Automatic NDJSON append per tool call | +| Zombie detection | Plugin | `session_health` checks active vs alive | +| Queue scanning | Plugin | `queue_status` queries GitLab per project |