71a8464fee
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The control plane owns agent lifecycle, issue assignment, routine dispatch, heartbeat wakeups, and recovery paths > - Terminated, paused, pending-approval, or otherwise invalid agents should not receive new work or new execution attempts > - The old behavior left eligibility checks spread across routes and services, so assignment and run paths could drift apart > - This pull request centralizes agent lifecycle eligibility and applies it consistently to assignment, invocation, routines, recovery, and UI affordances > - The benefit is safer autonomy: terminated agents stay paused, invalid org-chain agents are surfaced, and active agents keep receiving valid work ## Linked Issues or Issue Description Refs #5103 Related: #1864 Bug fix context: - What happened: agent assignment and heartbeat/run paths did not share one eligibility contract, so invalid lifecycle states could still be considered in some paths. - Expected behavior: terminated agents must never receive new assignments or heartbeat runs, and paused or otherwise invalid agents should be treated as non-invokable consistently. - Steps to reproduce: create or select an agent in an invalid lifecycle state, then attempt assignment, routine dispatch, or heartbeat/recovery wake paths. - Paperclip version/commit: fixed on top of `paperclipai/paperclip` `master` at the PR base. - Deployment mode: applies to the server control plane in local and authenticated deployments. ## What Changed - Added shared agent lifecycle eligibility helpers and exported the related shared types. - Centralized server-side assignability and invokability checks for issue assignment, agent routes, heartbeat dispatch, routines, recovery, and liveness logic. - Hardened issue assignment so invalid assignees are rejected instead of queued for work. - Hardened heartbeat/routine/recovery paths so terminated and otherwise invalid agents are not woken for new runs. - Updated board UI affordances to disable invalid agent actions and surface org-chain warnings where relevant. - Added targeted shared, server, and UI tests for the new eligibility behavior. ## Verification - `pnpm exec vitest run packages/shared/src/agent-eligibility.test.ts server/src/__tests__/agent-invokability.test.ts server/src/__tests__/heartbeat-archived-company-guard.test.ts server/src/__tests__/issue-liveness.test.ts server/src/__tests__/issues-service.test.ts server/src/__tests__/routines-service.test.ts ui/src/lib/company-members.test.ts ui/src/pages/Agents.test.tsx` — 8 files, 144 tests passed. - `pnpm --filter @paperclipai/shared typecheck && pnpm --filter @paperclipai/server typecheck && pnpm --filter @paperclipai/ui typecheck` — passed. - Checked the PR diff does not include `pnpm-lock.yaml` or `.github/workflows` changes. - Checked `ROADMAP.md`; this is a targeted control-plane safety fix and does not duplicate a planned core feature. - Searched GitHub for duplicate or related PRs/issues; closest related items are linked above. - CI and Greptile verification are pending on the opened PR and will be followed up before requesting merge. ## Risks Low to moderate risk. The intended behavioral shift is that invalid agents are refused earlier and more consistently, which could expose existing data with paused, pending, terminated, or broken org-chain assignees. The added tests cover the critical assignment, heartbeat, routine, recovery, shared helper, and UI paths. No database migrations are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used OpenAI GPT-5 Codex via the Paperclip `codex_local` adapter, with shell/git/GitHub CLI tool use. Reasoning mode and context window are managed by the adapter runtime and not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (not applicable: no design screenshots requested; UI behavior is covered by tests) - [x] I have updated relevant documentation to reflect my changes (not applicable: no user-facing command or schema docs changed) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending CI) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending Greptile) - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
383 lines
11 KiB
TypeScript
383 lines
11 KiB
TypeScript
import { randomUUID } from "node:crypto";
|
|
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
|
|
import {
|
|
agents,
|
|
agentWakeupRequests,
|
|
companies,
|
|
createDb,
|
|
heartbeatRunEvents,
|
|
heartbeatRuns,
|
|
issues,
|
|
} from "@paperclipai/db";
|
|
import {
|
|
getEmbeddedPostgresTestSupport,
|
|
startEmbeddedPostgresTestDatabase,
|
|
} from "./helpers/embedded-postgres.js";
|
|
import { heartbeatService } from "../services/heartbeat.ts";
|
|
|
|
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
|
|
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
|
|
|
|
if (!embeddedPostgresSupport.supported) {
|
|
console.warn(
|
|
`Skipping embedded Postgres archived-company heartbeat guard tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
|
|
);
|
|
}
|
|
|
|
describeEmbeddedPostgres("heartbeat archived-company guard", () => {
|
|
let db!: ReturnType<typeof createDb>;
|
|
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
|
|
|
|
beforeAll(async () => {
|
|
tempDb = await startEmbeddedPostgresTestDatabase("heartbeat-archived-company-guard-");
|
|
db = createDb(tempDb.connectionString);
|
|
}, 20_000);
|
|
|
|
afterEach(async () => {
|
|
await db.delete(heartbeatRunEvents);
|
|
await db.delete(heartbeatRuns);
|
|
await db.delete(agentWakeupRequests);
|
|
await db.delete(issues);
|
|
await db.delete(agents);
|
|
await db.delete(companies);
|
|
});
|
|
|
|
afterAll(async () => {
|
|
await tempDb?.cleanup();
|
|
});
|
|
|
|
async function insertArchivedAgent() {
|
|
const companyId = randomUUID();
|
|
const agentId = randomUUID();
|
|
|
|
await db.insert(companies).values({
|
|
id: companyId,
|
|
name: "Archived Co",
|
|
status: "archived",
|
|
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
|
requireBoardApprovalForNewAgents: false,
|
|
});
|
|
|
|
await db.insert(agents).values({
|
|
id: agentId,
|
|
companyId,
|
|
name: "Archived Agent",
|
|
role: "engineer",
|
|
status: "idle",
|
|
adapterType: "codex_local",
|
|
adapterConfig: {},
|
|
runtimeConfig: {
|
|
heartbeat: {
|
|
enabled: true,
|
|
intervalSec: 60,
|
|
wakeOnDemand: true,
|
|
},
|
|
},
|
|
permissions: {},
|
|
});
|
|
|
|
return { companyId, agentId };
|
|
}
|
|
|
|
async function insertInvalidOrgChainAgent() {
|
|
const companyId = randomUUID();
|
|
const managerId = randomUUID();
|
|
const childId = randomUUID();
|
|
|
|
await db.insert(companies).values({
|
|
id: companyId,
|
|
name: "Invalid Org Co",
|
|
status: "active",
|
|
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
|
|
requireBoardApprovalForNewAgents: false,
|
|
});
|
|
|
|
await db.insert(agents).values([
|
|
{
|
|
id: managerId,
|
|
companyId,
|
|
name: "Terminated Manager",
|
|
role: "cto",
|
|
status: "terminated",
|
|
adapterType: "codex_local",
|
|
adapterConfig: {},
|
|
runtimeConfig: {
|
|
heartbeat: {
|
|
enabled: true,
|
|
intervalSec: 60,
|
|
wakeOnDemand: true,
|
|
},
|
|
},
|
|
permissions: {},
|
|
},
|
|
{
|
|
id: childId,
|
|
companyId,
|
|
name: "Invalid Chain Child",
|
|
role: "engineer",
|
|
reportsTo: managerId,
|
|
status: "idle",
|
|
adapterType: "codex_local",
|
|
adapterConfig: {},
|
|
runtimeConfig: {
|
|
heartbeat: {
|
|
enabled: true,
|
|
intervalSec: 60,
|
|
wakeOnDemand: true,
|
|
},
|
|
},
|
|
permissions: {},
|
|
},
|
|
]);
|
|
|
|
return { companyId, managerId, childId };
|
|
}
|
|
|
|
it("does not iterate archived-company agents in tickTimers", async () => {
|
|
const { agentId } = await insertArchivedAgent();
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
const result = await heartbeat.tickTimers(new Date("2026-06-04T00:10:00Z"));
|
|
|
|
expect(result).toMatchObject({
|
|
checked: 0,
|
|
enqueued: 0,
|
|
skipped: 0,
|
|
});
|
|
|
|
const runCount = await db
|
|
.select()
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows.filter((row) => row.agentId === agentId).length);
|
|
expect(runCount).toBe(0);
|
|
});
|
|
|
|
it("skips background wakeups for non-active companies with a company.inactive reason", async () => {
|
|
const { agentId } = await insertArchivedAgent();
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
const run = await heartbeat.wakeup(agentId, {
|
|
source: "automation",
|
|
triggerDetail: "system",
|
|
reason: "issue_commented",
|
|
payload: { issueId: randomUUID(), commentId: randomUUID() },
|
|
requestedByActorType: "system",
|
|
requestedByActorId: "comment_wake",
|
|
});
|
|
|
|
expect(run).toBeNull();
|
|
|
|
const wakeup = await db
|
|
.select({
|
|
agentId: agentWakeupRequests.agentId,
|
|
status: agentWakeupRequests.status,
|
|
reason: agentWakeupRequests.reason,
|
|
error: agentWakeupRequests.error,
|
|
})
|
|
.from(agentWakeupRequests)
|
|
.then((rows) => rows.find((row) => row.agentId === agentId) ?? null);
|
|
|
|
expect(wakeup).toMatchObject({
|
|
status: "skipped",
|
|
reason: "company.inactive",
|
|
error: "Wake suppressed because company status is archived",
|
|
});
|
|
});
|
|
|
|
it("does not advance issue monitors for archived companies", async () => {
|
|
const { companyId, agentId } = await insertArchivedAgent();
|
|
const issueId = randomUUID();
|
|
const monitorScheduledAt = new Date("2026-06-04T00:00:00Z");
|
|
|
|
await db.insert(issues).values({
|
|
id: issueId,
|
|
companyId,
|
|
title: "Archived-company monitor issue",
|
|
status: "in_progress",
|
|
assigneeAgentId: agentId,
|
|
monitorNextCheckAt: monitorScheduledAt,
|
|
monitorAttemptCount: 0,
|
|
});
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
await heartbeat.tickTimers(new Date("2026-06-04T00:10:00Z"));
|
|
|
|
const row = await db
|
|
.select({
|
|
monitorNextCheckAt: issues.monitorNextCheckAt,
|
|
monitorWakeRequestedAt: issues.monitorWakeRequestedAt,
|
|
monitorLastTriggeredAt: issues.monitorLastTriggeredAt,
|
|
monitorAttemptCount: issues.monitorAttemptCount,
|
|
})
|
|
.from(issues)
|
|
.then((rows) => rows[0] ?? null);
|
|
|
|
expect(row?.monitorWakeRequestedAt).toBeNull();
|
|
expect(row?.monitorLastTriggeredAt).toBeNull();
|
|
expect(row?.monitorAttemptCount).toBe(0);
|
|
expect(row?.monitorNextCheckAt?.getTime()).toBe(monitorScheduledAt.getTime());
|
|
});
|
|
|
|
it("does not resume queued runs for archived companies", async () => {
|
|
const { companyId, agentId } = await insertArchivedAgent();
|
|
const runId = randomUUID();
|
|
|
|
await db.insert(heartbeatRuns).values({
|
|
id: runId,
|
|
companyId,
|
|
agentId,
|
|
invocationSource: "timer",
|
|
status: "queued",
|
|
});
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
await heartbeat.resumeQueuedRuns();
|
|
|
|
const status = await db
|
|
.select({ status: heartbeatRuns.status })
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows[0]?.status ?? null);
|
|
expect(status).toBe("queued");
|
|
});
|
|
|
|
it("rejects explicit user invokes for non-active companies", async () => {
|
|
const { agentId } = await insertArchivedAgent();
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
|
|
await expect(heartbeat.wakeup(agentId, {
|
|
source: "on_demand",
|
|
triggerDetail: "manual",
|
|
requestedByActorType: "user",
|
|
requestedByActorId: "board-user",
|
|
})).rejects.toMatchObject({
|
|
status: 409,
|
|
details: { status: "archived" },
|
|
});
|
|
|
|
const runCount = await db
|
|
.select()
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows.filter((row) => row.agentId === agentId).length);
|
|
expect(runCount).toBe(0);
|
|
});
|
|
|
|
it("rejects explicit user invokes for invalid-org-chain agents", async () => {
|
|
const { childId } = await insertInvalidOrgChainAgent();
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
|
|
await expect(heartbeat.wakeup(childId, {
|
|
source: "on_demand",
|
|
triggerDetail: "manual",
|
|
requestedByActorType: "user",
|
|
requestedByActorId: "board-user",
|
|
})).rejects.toMatchObject({
|
|
status: 409,
|
|
details: {
|
|
reason: "manager_terminated",
|
|
invalidOrgChain: true,
|
|
},
|
|
});
|
|
|
|
const runCount = await db
|
|
.select()
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows.filter((row) => row.agentId === childId).length);
|
|
expect(runCount).toBe(0);
|
|
});
|
|
|
|
it("cancels existing queued runs for invalid-org-chain agents instead of starting them", async () => {
|
|
const { companyId, childId } = await insertInvalidOrgChainAgent();
|
|
const wakeupRequestId = randomUUID();
|
|
const runId = randomUUID();
|
|
|
|
await db.insert(agentWakeupRequests).values({
|
|
id: wakeupRequestId,
|
|
companyId,
|
|
agentId: childId,
|
|
source: "assignment",
|
|
status: "queued",
|
|
});
|
|
await db.insert(heartbeatRuns).values({
|
|
id: runId,
|
|
companyId,
|
|
agentId: childId,
|
|
invocationSource: "assignment",
|
|
status: "queued",
|
|
wakeupRequestId,
|
|
});
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
await heartbeat.resumeQueuedRuns();
|
|
|
|
const run = await db
|
|
.select({
|
|
status: heartbeatRuns.status,
|
|
error: heartbeatRuns.error,
|
|
})
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows.find((row) => row.status === "cancelled") ?? null);
|
|
expect(run).toMatchObject({
|
|
status: "cancelled",
|
|
error: "Cancelled because the agent is not invokable: manager_terminated",
|
|
});
|
|
|
|
const wakeup = await db
|
|
.select({
|
|
status: agentWakeupRequests.status,
|
|
error: agentWakeupRequests.error,
|
|
})
|
|
.from(agentWakeupRequests)
|
|
.then((rows) => rows[0] ?? null);
|
|
expect(wakeup).toMatchObject({
|
|
status: "cancelled",
|
|
error: "Cancelled because the agent is not invokable: manager_terminated",
|
|
});
|
|
});
|
|
|
|
it("suppresses due scheduled retries for invalid-org-chain agents", async () => {
|
|
const { companyId, childId } = await insertInvalidOrgChainAgent();
|
|
const wakeupRequestId = randomUUID();
|
|
const runId = randomUUID();
|
|
const now = new Date("2026-06-04T00:10:00Z");
|
|
|
|
await db.insert(agentWakeupRequests).values({
|
|
id: wakeupRequestId,
|
|
companyId,
|
|
agentId: childId,
|
|
source: "automation",
|
|
status: "queued",
|
|
});
|
|
await db.insert(heartbeatRuns).values({
|
|
id: runId,
|
|
companyId,
|
|
agentId: childId,
|
|
invocationSource: "automation",
|
|
status: "scheduled_retry",
|
|
wakeupRequestId,
|
|
scheduledRetryAt: new Date("2026-06-04T00:00:00Z"),
|
|
scheduledRetryReason: "transient_failure",
|
|
scheduledRetryAttempt: 1,
|
|
});
|
|
|
|
const heartbeat = heartbeatService(db);
|
|
const promoted = await heartbeat.promoteDueScheduledRetries(now);
|
|
|
|
expect(promoted).toEqual({ promoted: 0, runIds: [] });
|
|
const run = await db
|
|
.select({
|
|
status: heartbeatRuns.status,
|
|
errorCode: heartbeatRuns.errorCode,
|
|
error: heartbeatRuns.error,
|
|
})
|
|
.from(heartbeatRuns)
|
|
.then((rows) => rows[0] ?? null);
|
|
expect(run).toMatchObject({
|
|
status: "cancelled",
|
|
errorCode: "agent_not_invokable",
|
|
error: "Scheduled retry suppressed because the agent is not invokable",
|
|
});
|
|
});
|
|
});
|