[codex] feat(watchdog): add task watchdog control plane (#8339)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work.
> - The task lifecycle and recovery subsystems decide when agent work is
still productive, stalled, or ready for review.
> - Existing recovery paths can observe stopped or incomplete work, but
there was no first-class per-task watchdog model with scoped review
permissions.
> - Watchdog follow-ups also need strict boundaries so
recovery/status-only runs cannot mutate approvals or perform deliverable
work.
> - This pull request adds the task watchdog data model, API/service
layer, scheduler/review flow, adapter wake context, UI configuration
surfaces, and docs.
> - The branch has been rebased onto current `paperclipai/paperclip`
`master`; the watchdog migration is now ordered after master's latest
migrations as `0104_issue_watchdogs`.
> - The benefit is a more explicit task-review loop that preserves
Paperclip's single-assignee and governance invariants while making
stalled work easier to route.

## Linked Issues or Issue Description

No linked GitHub issue. Paperclip task:
[PAP-11275](/PAP/issues/PAP-11275).

## Problem or motivation

Task recovery needs a first-class watchdog path that can inspect stopped
work and create scoped follow-ups without bypassing normal task
ownership. Board/UI users need a way to configure watchdogs on tasks and
see watchdog-related live work. Recovery/status-only runs must remain
limited to status reporting and must not create approvals, link
approvals, or submit approval comments.

## Proposed solution

Add a task-watchdog data model, scheduler/classifier, scoped mutation
guard, adapter wake context, API/UI configuration surfaces, and
documentation so watchdog agents can review stopped task subtrees under
explicit boundaries.

## Alternatives considered

Reuse the existing recovery-action flow only. That would keep
stopped-work detection implicit, make per-task watchdog assignment
harder to expose in the UI, and would not provide a durable
scoped-review issue for stalled task trees.

## Roadmap alignment

This is Paperclip control-plane lifecycle infrastructure for task
execution and recovery. I checked `ROADMAP.md`; this PR does not
duplicate an existing planned core item.

## What Changed

- Added issue watchdog schema, migration, shared contracts, validators,
CRUD API, and service support.
- Added task watchdog scheduler/classifier behavior, scoped mutation
enforcement, adapter wake context, and default watchdog mandate
guidance.
- Added UI surfaces for configuring watchdogs on new/existing tasks,
viewing watchdog activity, and exposing the experimental setting.
- Added docs for the user-facing task watchdog workflow and
implementation semantics.
- Gated new-task watchdog setup behind `enableTaskWatchdogs` and blocked
cheap status-only recovery runs from approval mutations.
- Rebased onto current `master` and renumbered the idempotent watchdog
migration from the branch-local `0102_issue_watchdogs` slot to
`0104_issue_watchdogs`.
- Addressed Greptile feedback by loading watchdog classifier input with
a recursive subtree query and centralizing the watchdog origin-kind
constant.
- Added and updated focused server/UI tests for watchdog routes,
scheduler/classifier behavior, scope boundaries, live task visibility,
settings, and new issue dialog behavior.

## Verification

- `pnpm vitest run server/src/__tests__/task-watchdogs-scheduler.test.ts
server/src/__tests__/task-watchdogs-classifier.test.ts`
- `pnpm vitest run
server/src/__tests__/approval-routes-idempotency.test.ts
server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts`
- `pnpm vitest run ui/src/components/NewIssueDialog.test.tsx`
- `pnpm --filter @paperclipai/server typecheck`
- `git diff --check`
- Verified the PR diff does not include `pnpm-lock.yaml` or
`.github/workflows`.

## Risks

- Medium risk: this introduces a new task lifecycle surface touching DB
schema, server routes/services, adapter wake context, and UI task
configuration.
- Watchdog scheduling behavior depends on the new experimental setting
and runtime context checks behaving consistently across local and
production agents.
- The watchdog migration is idempotent (`IF NOT EXISTS` /
duplicate-object guards) so users who tried the previous branch-local
migration number should not get duplicate-object failures.
- CI and the second Greptile pass are pending after the latest
review-fix push.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

OpenAI Codex, GPT-5-class coding agent in the Paperclip workspace. Exact
runtime model id and context window were not exposed to the agent; tool
use and local command execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots — N/A per Paperclip task instruction: do not add
screenshots/images to this PR unless they are specifically part of the
work.
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dotta
2026-06-19 15:38:52 -05:00
committed by GitHub
parent 8f4b491d9a
commit a71c4b6782
56 changed files with 7231 additions and 68 deletions
+9
View File
@@ -158,6 +158,7 @@ import {
} from "./recovery/model-profile-hint.js";
import { recoveryService } from "./recovery/service.js";
import { productivityReviewService } from "./productivity-review.js";
import { taskWatchdogService } from "./task-watchdogs.js";
import { withAgentStartLock } from "./agent-start-lock.js";
import {
evaluateAgentInvokability,
@@ -2792,6 +2793,7 @@ export async function buildPaperclipWakePayload(input: {
? input.contextSnapshot.unresolvedBlockerSummaries
: [],
executionStage: Object.keys(executionStage).length > 0 ? executionStage : null,
taskWatchdog: (input.contextSnapshot.taskWatchdog ?? null) as unknown,
continuationSummary: safeContinuationSummary
? {
key: safeContinuationSummary.key,
@@ -3219,6 +3221,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
const budgets = budgetService(db, budgetHooks);
const recovery = recoveryService(db, { enqueueWakeup });
const productivityReviews = productivityReviewService(db, { enqueueWakeup });
const taskWatchdogs = taskWatchdogService(db, { enqueueWakeup });
let unsafeTextProjectionPromise: Promise<boolean> | null = null;
async function releaseEnvironmentLeasesForRun(input: {
@@ -7744,6 +7747,10 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
return productivityReviews.reconcileProductivityReviews(opts);
}
async function reconcileTaskWatchdogs(opts?: { companyId?: string | null; runId?: string | null }) {
return taskWatchdogs.reconcileTaskWatchdogs(opts);
}
async function buildRunOutputSilence(
run: Pick<
typeof heartbeatRuns.$inferSelect,
@@ -11738,6 +11745,8 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
reconcileProductivityReviews,
reconcileTaskWatchdogs,
buildRunOutputSilence,
tickTimers: async (now = new Date()) => {
+6
View File
@@ -27,6 +27,12 @@ export { issueTreeControlService } from "./issue-tree-control.js";
export { issueApprovalService } from "./issue-approvals.js";
export { issueReferenceService } from "./issue-references.js";
export { issueRecoveryActionService } from "./issue-recovery-actions.js";
export { taskWatchdogService } from "./task-watchdogs.js";
export {
issueIsInTaskWatchdogSubtree,
resolveTaskWatchdogMutationScope,
taskWatchdogScopeAllowsIssueMutation,
} from "./task-watchdog-scope.js";
export { goalService } from "./goals.js";
export { activityService, type ActivityFilters } from "./activity.js";
export { approvalService } from "./approvals.js";
+2
View File
@@ -49,6 +49,7 @@ export function normalizeExperimentalSettings(raw: unknown): InstanceExperimenta
enableConferenceRoomChat: parsed.data.enableConferenceRoomChat ?? false,
enableIssuePlanDecompositions: parsed.data.enableIssuePlanDecompositions ?? false,
enableExperimentalFileViewer: parsed.data.enableExperimentalFileViewer ?? false,
enableTaskWatchdogs: parsed.data.enableTaskWatchdogs ?? false,
enableCloudSync: parsed.data.enableCloudSync ?? false,
autoRestartDevServerWhenIdle: parsed.data.autoRestartDevServerWhenIdle ?? false,
enableIssueGraphLivenessAutoRecovery: parsed.data.enableIssueGraphLivenessAutoRecovery ?? false,
@@ -64,6 +65,7 @@ export function normalizeExperimentalSettings(raw: unknown): InstanceExperimenta
enableConferenceRoomChat: false,
enableIssuePlanDecompositions: false,
enableExperimentalFileViewer: false,
enableTaskWatchdogs: false,
enableCloudSync: false,
autoRestartDevServerWhenIdle: false,
enableIssueGraphLivenessAutoRecovery: false,
+59 -2
View File
@@ -19,6 +19,7 @@ import {
issueAttachments,
issueInboxArchives,
issueLabels,
issueWatchdogs,
issuePlanDecompositions,
issueRecoveryActions,
issueRelations,
@@ -44,6 +45,7 @@ import type {
IssueProductivityReview,
IssueProductivityReviewTrigger,
IssueRelationIssueSummary,
IssueWatchdogSummary,
LowTrustBoundary,
SuccessfulRunHandoffState,
} from "@paperclipai/shared";
@@ -76,6 +78,10 @@ import { resolveIssueGoalId, resolveNextIssueGoalId } from "./issue-goal-fallbac
import { getRunLogStore } from "./run-log-store.js";
import { getDefaultCompanyGoal } from "./goals.js";
import { assertAssignableAgent } from "./agent-assignability.js";
import {
summarizeIssueWatchdog,
upsertIssueWatchdogForIssue,
} from "./task-watchdogs.js";
import {
isVerifiedIssueTreeControlInteractionWake,
issueTreeControlService,
@@ -302,7 +308,11 @@ type IssueScheduledRetryRow = {
error?: string | null;
errorCode?: string | null;
};
type IssueWithLabels = IssueRow & { labels: IssueLabelRow[]; labelIds: string[] };
type IssueWithLabels = IssueRow & {
labels: IssueLabelRow[];
labelIds: string[];
watchdog?: IssueWatchdogSummary | null;
};
type IssueWithLabelsAndRun = IssueWithLabels & { activeRun: IssueActiveRunRow | null };
type IssueUserCommentStats = {
issueId: string;
@@ -354,6 +364,8 @@ type IssueCreateInput = Omit<typeof issues.$inferInsert, "companyId"> & {
labelIds?: string[];
blockedByIssueIds?: string[];
inheritExecutionWorkspaceFromIssueId?: string | null;
watchdog?: { agentId: string; instructions?: string | null } | null;
watchdogActorRunId?: string | null;
};
type IssueChildCreateInput = IssueCreateInput & {
acceptanceCriteria?: string[];
@@ -1143,17 +1155,49 @@ async function labelMapForIssues(dbOrTx: any, issueIds: string[]): Promise<Map<s
async function withIssueLabels(dbOrTx: any, rows: IssueRow[]): Promise<IssueWithLabels[]> {
if (rows.length === 0) return [];
const labelsByIssueId = await labelMapForIssues(dbOrTx, rows.map((row) => row.id));
const issueIds = rows.map((row) => row.id);
const [labelsByIssueId, watchdogByIssueId] = await Promise.all([
labelMapForIssues(dbOrTx, issueIds),
watchdogMapForIssues(dbOrTx, rows),
]);
return rows.map((row) => {
const issueLabels = labelsByIssueId.get(row.id) ?? [];
return {
...row,
labels: issueLabels,
labelIds: issueLabels.map((label) => label.id),
watchdog: watchdogByIssueId.get(row.id) ?? null,
};
});
}
async function watchdogMapForIssues(dbOrTx: any, rows: IssueRow[]): Promise<Map<string, IssueWatchdogSummary>> {
const map = new Map<string, IssueWatchdogSummary>();
if (rows.length === 0) return map;
const byCompany = new Map<string, string[]>();
for (const row of rows) {
const ids = byCompany.get(row.companyId) ?? [];
ids.push(row.id);
byCompany.set(row.companyId, ids);
}
for (const [companyId, issueIds] of byCompany.entries()) {
for (const issueIdChunk of chunkList([...new Set(issueIds)], ISSUE_LIST_RELATED_QUERY_CHUNK_SIZE)) {
const watchdogRows = await dbOrTx
.select()
.from(issueWatchdogs)
.where(and(
eq(issueWatchdogs.companyId, companyId),
inArray(issueWatchdogs.issueId, issueIdChunk),
eq(issueWatchdogs.status, "active"),
));
for (const row of watchdogRows) {
map.set(row.issueId, summarizeIssueWatchdog(row));
}
}
}
return map;
}
const ACTIVE_RUN_STATUSES = ["queued", "running"];
const BLOCKER_ATTENTION_ACTIVE_RUN_STATUSES = ["queued", "running"];
const BLOCKER_ATTENTION_ACTIVE_WAKE_STATUSES = ["queued", "deferred_issue_execution"];
@@ -4857,6 +4901,8 @@ export function issueService(db: Db) {
labelIds: inputLabelIds,
blockedByIssueIds,
inheritExecutionWorkspaceFromIssueId,
watchdog,
watchdogActorRunId,
...issueData
} = data;
const isolatedWorkspacesEnabled = (await instanceSettings.getExperimental()).enableIsolatedWorkspaces;
@@ -5073,6 +5119,17 @@ export function issueService(db: Db) {
);
const [issue] = await tx.insert(issues).values(values).returning();
if (watchdog) {
await upsertIssueWatchdogForIssue(tx, companyId, issue.id, {
agentId: watchdog.agentId,
instructions: watchdog.instructions,
actor: {
agentId: issueData.createdByAgentId ?? null,
userId: issueData.createdByUserId ?? null,
runId: watchdogActorRunId ?? null,
},
});
}
if (inputLabelIds) {
await syncIssueLabels(issue.id, companyId, inputLabelIds, tx);
}
+174
View File
@@ -0,0 +1,174 @@
import { and, eq } from "drizzle-orm";
import type { Db } from "@paperclipai/db";
import { heartbeatRuns, issues, issueWatchdogs } from "@paperclipai/db";
const MAX_WATCHDOG_SCOPE_ANCESTRY_DEPTH = 100;
export const TASK_WATCHDOG_ORIGIN_KIND = "task_watchdog";
type AgentRunActor = {
type: string;
agentId?: string | null;
companyId?: string | null;
runId?: string | null;
};
type IssueScopeTarget = {
id: string;
companyId: string;
parentId?: string | null;
};
export type TaskWatchdogMutationScope =
| { kind: "none" }
| { kind: "invalid"; detail: string }
| {
kind: "watchdog";
watchdogId: string;
companyId: string;
watchedIssueId: string;
watchdogIssueId: string | null;
stopFingerprint: string | null;
};
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return typeof value === "object" && value !== null && !Array.isArray(value);
}
function readString(value: unknown): string | null {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
}
function readTaskWatchdogContext(contextSnapshot: unknown) {
const context = isPlainRecord(contextSnapshot) ? contextSnapshot : null;
const taskWatchdog = isPlainRecord(context?.taskWatchdog) ? context.taskWatchdog : null;
if (!taskWatchdog && context?.taskWatchdog !== true) return null;
return {
watchedIssueId: readString(taskWatchdog?.watchedIssueId) ?? readString(context?.watchedIssueId),
stopFingerprint: readString(taskWatchdog?.stopFingerprint) ?? readString(context?.stopFingerprint),
};
}
export async function resolveTaskWatchdogMutationScope(
db: Db,
actor: AgentRunActor,
): Promise<TaskWatchdogMutationScope> {
if (actor.type !== "agent") return { kind: "none" };
const agentId = readString(actor.agentId);
const runId = readString(actor.runId);
const actorCompanyId = readString(actor.companyId);
if (!agentId || !runId) return { kind: "none" };
const run = await db
.select({
id: heartbeatRuns.id,
companyId: heartbeatRuns.companyId,
agentId: heartbeatRuns.agentId,
contextSnapshot: heartbeatRuns.contextSnapshot,
})
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId))
.then((rows) => rows[0] ?? null);
if (!run) return { kind: "none" };
const taskWatchdog = readTaskWatchdogContext(run.contextSnapshot);
if (!taskWatchdog) return { kind: "none" };
if (run.agentId !== agentId || (actorCompanyId && run.companyId !== actorCompanyId)) {
return {
kind: "invalid",
detail: "Task-watchdog run context does not belong to this agent.",
};
}
if (!taskWatchdog.watchedIssueId) {
return {
kind: "invalid",
detail: "Task-watchdog run context is missing a persisted watched issue id.",
};
}
const watchdog = await db
.select({
id: issueWatchdogs.id,
companyId: issueWatchdogs.companyId,
issueId: issueWatchdogs.issueId,
watchdogAgentId: issueWatchdogs.watchdogAgentId,
watchdogIssueId: issueWatchdogs.watchdogIssueId,
status: issueWatchdogs.status,
})
.from(issueWatchdogs)
.where(and(
eq(issueWatchdogs.companyId, run.companyId),
eq(issueWatchdogs.issueId, taskWatchdog.watchedIssueId),
eq(issueWatchdogs.watchdogAgentId, agentId),
eq(issueWatchdogs.status, "active"),
))
.then((rows) => rows[0] ?? null);
if (!watchdog) {
return {
kind: "invalid",
detail: "Task-watchdog run context is not backed by an active persisted watchdog.",
};
}
return {
kind: "watchdog",
watchdogId: watchdog.id,
companyId: watchdog.companyId,
watchedIssueId: watchdog.issueId,
watchdogIssueId: watchdog.watchdogIssueId ?? null,
stopFingerprint: taskWatchdog.stopFingerprint,
};
}
export async function issueIsInTaskWatchdogSubtree(
db: Db,
companyId: string,
issueId: string,
watchedIssueId: string,
) {
let currentId: string | null = issueId;
const seen = new Set<string>();
for (let depth = 0; currentId && depth < MAX_WATCHDOG_SCOPE_ANCESTRY_DEPTH; depth += 1) {
if (seen.has(currentId)) return false;
seen.add(currentId);
const parent: { id: string; companyId: string; parentId: string | null; originKind: string | null } | null = await db
.select({ id: issues.id, companyId: issues.companyId, parentId: issues.parentId, originKind: issues.originKind })
.from(issues)
.where(and(eq(issues.id, currentId), eq(issues.companyId, companyId)))
.then((rows) => rows[0] ?? null);
if (!parent) return false;
if (parent.originKind === TASK_WATCHDOG_ORIGIN_KIND) return false;
if (currentId === watchedIssueId) return true;
currentId = parent.parentId ?? null;
}
return false;
}
export async function taskWatchdogScopeAllowsIssueMutation(
db: Db,
scope: TaskWatchdogMutationScope,
issue: IssueScopeTarget,
opts: { allowWatchdogIssue?: boolean } = {},
) {
if (scope.kind !== "watchdog") return scope;
if (issue.companyId !== scope.companyId) {
return {
kind: "invalid" as const,
detail: "Task-watchdog mutation target is outside the watchdog company.",
};
}
if (opts.allowWatchdogIssue !== false && scope.watchdogIssueId && issue.id === scope.watchdogIssueId) {
return scope;
}
if (await issueIsInTaskWatchdogSubtree(db, scope.companyId, issue.id, scope.watchedIssueId)) {
return scope;
}
return {
kind: "invalid" as const,
detail: "Task-watchdog runs can only mutate the watched issue subtree.",
};
}
File diff suppressed because it is too large Load Diff