[codex] feat(watchdog): add task watchdog control plane (#8339)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The task lifecycle and recovery subsystems decide when agent work is
still productive, stalled, or ready for review.
> - Existing recovery paths can observe stopped or incomplete work, but
there was no first-class per-task watchdog model with scoped review
permissions.
> - Watchdog follow-ups also need strict boundaries so
recovery/status-only runs cannot mutate approvals or perform deliverable
work.
> - This pull request adds the task watchdog data model, API/service
layer, scheduler/review flow, adapter wake context, UI configuration
surfaces, and docs.
> - The branch has been rebased onto current `paperclipai/paperclip`
`master`; the watchdog migration is now ordered after master's latest
migrations as `0104_issue_watchdogs`.
> - The benefit is a more explicit task-review loop that preserves
Paperclip's single-assignee and governance invariants while making
stalled work easier to route.

## Linked Issues or Issue Description

No linked GitHub issue. Paperclip task:
[PAP-11275](/PAP/issues/PAP-11275).

## Problem or motivation

Task recovery needs a first-class watchdog path that can inspect stopped
work and create scoped follow-ups without bypassing normal task
ownership. Board/UI users need a way to configure watchdogs on tasks and
see watchdog-related live work. Recovery/status-only runs must remain
limited to status reporting and must not create approvals, link
approvals, or submit approval comments.

## Proposed solution

Add a task-watchdog data model, scheduler/classifier, scoped mutation
guard, adapter wake context, API/UI configuration surfaces, and
documentation so watchdog agents can review stopped task subtrees under
explicit boundaries.

## Alternatives considered

Reuse the existing recovery-action flow only. That would keep
stopped-work detection implicit, make per-task watchdog assignment
harder to expose in the UI, and would not provide a durable
scoped-review issue for stalled task trees.

## Roadmap alignment

This is Paperclip control-plane lifecycle infrastructure for task
execution and recovery. I checked `ROADMAP.md`; this PR does not
duplicate an existing planned core item.

## What Changed

- Added issue watchdog schema, migration, shared contracts, validators,
CRUD API, and service support.
- Added task watchdog scheduler/classifier behavior, scoped mutation
enforcement, adapter wake context, and default watchdog mandate
guidance.
- Added UI surfaces for configuring watchdogs on new/existing tasks,
viewing watchdog activity, and exposing the experimental setting.
- Added docs for the user-facing task watchdog workflow and
implementation semantics.
- Gated new-task watchdog setup behind `enableTaskWatchdogs` and blocked
cheap status-only recovery runs from approval mutations.
- Rebased onto current `master` and renumbered the idempotent watchdog
migration from the branch-local `0102_issue_watchdogs` slot to
`0104_issue_watchdogs`.
- Addressed Greptile feedback by loading watchdog classifier input with
a recursive subtree query and centralizing the watchdog origin-kind
constant.
- Added and updated focused server/UI tests for watchdog routes,
scheduler/classifier behavior, scope boundaries, live task visibility,
settings, and new issue dialog behavior.

## Verification

- `pnpm vitest run server/src/__tests__/task-watchdogs-scheduler.test.ts
server/src/__tests__/task-watchdogs-classifier.test.ts`
- `pnpm vitest run
server/src/__tests__/approval-routes-idempotency.test.ts
server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts`
- `pnpm vitest run ui/src/components/NewIssueDialog.test.tsx`
- `pnpm --filter @paperclipai/server typecheck`
- `git diff --check`
- Verified the PR diff does not include `pnpm-lock.yaml` or
`.github/workflows`.

## Risks

- Medium risk: this introduces a new task lifecycle surface touching DB
schema, server routes/services, adapter wake context, and UI task
configuration.
- Watchdog scheduling behavior depends on the new experimental setting
and runtime context checks behaving consistently across local and
production agents.
- The watchdog migration is idempotent (`IF NOT EXISTS` /
duplicate-object guards) so users who tried the previous branch-local
migration number should not get duplicate-object failures.
- CI and the second Greptile pass are pending after the latest
review-fix push.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-class coding agent in the Paperclip workspace. Exact
runtime model id and context window were not exposed to the agent; tool
use and local command execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots — N/A per Paperclip task instruction: do not add
screenshots/images to this PR unless they are specifically part of the
work.
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dotta
2026-06-19 15:38:52 -05:00
committed by GitHub
parent 8f4b491d9a
commit a71c4b6782
56 changed files with 7231 additions and 68 deletions
+84 -4
View File
@@ -488,7 +488,87 @@ On startup and on the periodic recovery loop, Paperclip now does five things in
The stranded-work pass closes the gap where issue state survives a crash but the wake/run path does not. The silent-run scan covers the separate case where a live process exists but has stopped producing observable output. The productivity-review pass is later and separate; it reviews unusual progression patterns on assigned source issues, not stale run handles after a source issue already has a valid disposition.
## 11. Silent Active-Run Watchdog
## 11. Task Watchdog for Issue Trees
A task watchdog watches a configured issue subtree after that subtree has stopped moving. It is a product-level verification and recovery mechanism for selected work, not a process monitor.
Keep the three watchdog/recovery concepts separate:
- task watchdog: watches a configured source issue plus non-watchdog descendants and asks whether the stopped subtree is legitimate
- silent active-run watchdog: watches a still-running process that has stopped producing output
- liveness recovery: repairs stranded control-plane paths when a non-terminal issue has no live, waiting, or recovery path
### Configuration and scan scope
A source issue may have at most one active task watchdog configuration. The configuration names a same-company, invokable watchdog agent and optional custom instructions.
The scan scope is:
- the source issue
- descendants reached through `parentId`
- excluding every issue whose `originKind` is `task_watchdog`
- excluding every descendant below an excluded task-watchdog issue
The reusable watchdog issue is a child of the watched source issue for audit and navigation, but it is excluded from the watched work subtree. This prevents recursive watchdog loops.
### Stopped-subtree evaluation
Task watchdog evaluation is conservative. If any included issue has a live run, queued wake, or scheduled retry that should fire without intervention, the subtree is live and the task watchdog does not run.
If no included issue has a live path, Paperclip computes a stop fingerprint from durable subtree state, including at least:
- included leaf issue ids, statuses, assignees, and latest durable update timestamps
- first-class blockers and unresolved blocker leaf summaries
- pending interactions and approvals that define waiting paths
- active monitors and scheduled retries
- terminal or cancelled leaf evidence
- the watchdog configuration revision, including watchdog agent and instructions changes
If the fingerprint equals the watchdog's last reviewed fingerprint, Paperclip suppresses another watchdog wake. If the fingerprint is new, Paperclip creates or reopens the reusable watchdog issue and wakes the configured watchdog agent with the source issue, watchdog config, stop fingerprint, leaf summary, default mandate, custom instructions, and server-derived capability metadata that names the allowed operations, denied operations, reusable watchdog issue, and non-watchdog target scope.
Changing the watchdog agent or custom instructions invalidates the reviewed fingerprint and forces a fresh evaluation even if the subtree state did not otherwise change.
### Live path created by watchdog work
An active watchdog issue or queued watchdog wake can be the visible recovery path for a stopped watched subtree, but it is not proof that the original deliverable work is complete. It means the next action is watchdog verification.
When the source issue is non-terminal and has no other live path, the product should expose the watchdog issue or source-scoped recovery action as the reason the subtree is covered. When correctness requires the source issue to wait on watchdog review, the source issue should be blocked on the reusable watchdog issue or an equivalent explicit recovery action. Do not rely on parent/child structure alone.
### Watchdog authority during execution
The watchdog agent acts in a scoped capacity, not as the original deliverable worker and not as the board. The server must enforce the authority contract in `doc/SPEC-implementation.md` from persisted watchdog context. Prompt text and custom instructions may guide the watchdog's judgment, but they cannot grant authority outside the watched subtree or beyond the allowed mutation and interaction list.
Watchdogs must not create visible probe issues, comments, or throwaway tasks to discover capability boundaries. They should rely on the wake capability metadata and explicit API denials, then record any denied operation as evidence in the reusable watchdog issue.
The watchdog should verify stopped leaves against comments, documents, work products, tests, screenshots, blockers, review state, and run context. It should not accept "I could not" or "waiting for approval" as sufficient by itself.
When work should continue, the watchdog restores a live path inside the watched subtree: reopen or reassign stuck work, create follow-up issues, repair blockers, set a monitor, or resolve an eligible plan confirmation. When the stopped state is legitimate, the watchdog records why and leaves the subtree with a valid terminal, waiting, blocked, review, or explicit recovery path.
### Eligible interaction decisions
A task watchdog may resolve only eligible `request_confirmation` plan confirmations. Eligibility is defined in `doc/SPEC-implementation.md` and must be checked by the server at decision time. The critical constraints are:
- the interaction is pending, targeted at the current `plan` document revision for an included subtree issue, and explicitly marked as a plan-approval confirmation
- accepting it authorizes only decomposition or task-level continuation inside the watched subtree
- the plan is not asking for board-only governance, spend, hiring, security, deployment, secret, destructive data, legal/compliance, cross-company, or other sensitive approval
- no newer durable source activity or policy reserves the decision for a human, CTO, Security, or the board
The watchdog cannot resolve `request_checkbox_confirmation`, `ask_user_questions`, `suggest_tasks`, linked approvals, execution-policy decisions unless it is the typed participant outside watchdog capacity, or document comments written as freeform approval.
### Completion and fingerprint updates
The watchdog's reviewed fingerprint should update only after the watchdog issue reaches a valid disposition:
- `done` with evidence that the stopped state is acceptable
- `in_review` with a real reviewer, approval, interaction, user owner, monitor, or recovery path
- `blocked` with first-class blockers or a named external owner/action
- a watchdog mutation that restores live work, where the subsequent source-subtree mutation naturally changes the stop fingerprint
If the watchdog moved work forward, Paperclip should not mark the old fingerprint as permanently acceptable just because the watchdog issue completed. The next scan should observe the changed subtree state and either suppress because work is live or compute a new stopped fingerprint later.
Task watchdogs must not silently mark source work done from prose comments, must not duplicate child trees for the same accepted plan revision, and must not create another task-watchdog issue for the same source issue.
## 12. Silent Active-Run Watchdog
An active run can still be unhealthy even when its process is `running`. Paperclip treats prolonged output silence as a watchdog signal, not as proof that the run is failed.
@@ -540,7 +620,7 @@ This is distinct from productivity review. Productivity review asks whether an a
Detached process cleanup is operational hygiene, not source issue liveness. Cleanup should be best-effort and auditable. If cleanup fails but the source issue is already terminal with same-run durable evidence, Paperclip should preserve the cleanup failure on the run/watchdog audit trail and route only the cleanup concern to bounded recovery when a real owner/action remains.
## 12. Auto-Recover vs Explicit Recovery vs Human Escalation
## 13. Auto-Recover vs Explicit Recovery vs Human Escalation
Paperclip uses three different recovery outcomes, depending on how much it can safely infer.
@@ -584,7 +664,7 @@ Examples:
In these cases Paperclip should leave a visible issue/comment trail instead of silently retrying.
## 13. What This Does Not Mean
## 14. What This Does Not Mean
These semantics do not change V1 into an auto-reassignment system.
@@ -601,7 +681,7 @@ The recovery model is intentionally conservative:
- open an explicit recovery action when the system can identify a bounded recovery owner/action
- escalate visibly when the system cannot safely keep going
## 14. Practical Interpretation
## 15. Practical Interpretation
For a board operator, the intended meaning is: