Arnike/paperclip
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(server): enforce agent secret binding sync across lifecycle flows (#8307)

Browse Source 
## Thinking Path

> - Paperclip is the control plane people use to create, configure, and
run AI agents for work.
> - This change sits in the server-side agent lifecycle and
secret-binding subsystem, where adapter config `env` entries can
reference company secrets.
> - An incident (while trying to configure a Novita sandbox) showed that
an agent can reach a broken runtime state if `adapterConfig.env`
contains `secret_ref` entries but the matching `company_secret_bindings`
rows are missing.
> - The immediate run-path guard and error-surfacing work made the
failure diagnosable, but they did not fully prevent new broken agents
from being created.
> - The risk came from create and approval flows being responsible for
remembering to sync bindings at each call site, which is easy to miss as
new flows are added.
> - This pull request moves the invariant into `agentService`
create/update/activate paths, keeps the existing hire-flow fix, and adds
regression coverage for create, update, and legacy pending-approval
recovery.
> - The benefit is that agent secret binding integrity is enforced
closer to the data mutation point, so future callers inherit the
protection automatically.

## Linked Issues or Issue Description

Refs #8309

### What happened?
A Paperclip agent could persist `adapterConfig.env` `secret_ref` entries
without matching agent-scoped `company_secret_bindings` rows. When that
happened, the config UI could still look configured, but the real run
path failed pre-dispatch because the secret was not actually bound to
that agent.

### Expected behavior
Every normal agent create, config-update, and pending-approval
activation flow should leave the agent with secret bindings that match
its persisted secret-ref env config.

### Steps to reproduce
1. Create or activate an agent through a flow that persists
`adapterConfig.env` secret refs without synchronizing
`company_secret_bindings`.
2. Observe that the config state can still appear populated.
3. Start a run for that agent.
4. Observe that pre-dispatch binding validation fails because the secret
reference exists but the agent binding does not.

### Deployment mode
Local dev (`pnpm dev`)

### Installation method
Built from source (`pnpm dev` / `pnpm build`)

### Agent adapter(s) involved
- Claude Code
- Not adapter-specific (core bug)

### Database mode
Embedded PGlite / embedded local dev database flow

### Access context
Board (human operator) created or approved the agent; agent runtime
later consumed the config.

### Additional context
This PR focuses on preventing new broken states from normal service
flows and on backfilling the covered legacy pending-approval activation
path.

## What Changed

- Kept the existing branch-local hire-flow fix that synchronized
bindings for route and approval paths.
- Moved the binding integrity invariant into `agentService.create()`,
`agentService.update()` when `adapterConfig` changes, and
`agentService.activatePendingApproval()`.
- Added `server/src/__tests__/agents-service-secret-bindings.test.ts`
covering create-time sync, update-time resync, and backfill for legacy
pending-approval agents.
- Removed now-redundant route-layer and approval-layer binding sync
calls once the service layer became authoritative.
- Simplified the affected unit tests so route/approval tests no longer
assert service-owned binding writes directly.

## Verification

- `pnpm --filter @paperclipai/server typecheck`
- `pnpm exec vitest run
server/src/__tests__/agents-service-secret-bindings.test.ts
server/src/__tests__/approvals-service.test.ts
server/src/__tests__/agent-skills-routes.test.ts`

## Risks

- Low to medium risk.
- This changes where secret-binding synchronization is enforced, so any
unexpected caller that relied on upper-layer manual sync behavior could
behave differently.
- Agent create/update/activation flows now perform binding
synchronization consistently, which adds binding-table writes at those
mutation points.
- This PR does not retroactively scan and heal every already-broken
historical agent row; it prevents and backfills through the covered
service flows.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex / GPT-5 Codex class model via `codex_local`
- Session model family: GPT-5 Codex
- Tool-assisted coding with shell, git, HTTP, and local test execution
- Reasoning mode: medium interactive tool-use workflow

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
2026-06-18 21:26:36 -07:00
committed by GitHub
parent 67c98323b0
commit fc95699fde
23 changed files with 1003 additions and 104 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/src/__tests__/agent-skills-routes.test.ts
+10
View File
@@ -51,6 +51,7 @@ const mockCompanySkillService = vi.hoisted(() => ({
const mockSecretService = vi.hoisted(() => ({
resolveAdapterConfigForRuntime: vi.fn(),
normalizeAdapterConfigForPersistence: vi.fn(async (_companyId: string, config: Record<string, unknown>) => config),
syncEnvBindingsForTarget: vi.fn(),
}));
const mockLogActivity = vi.hoisted(() => vi.fn());
@@ -96,6 +97,10 @@ vi.mock("../services/index.js", () => ({
workspaceOperationService: () => mockWorkspaceOperationService,
}));
vi.mock("../services/secrets.js", () => ({
secretService: () => mockSecretService,
}));
vi.mock("../adapters/index.js", () => ({
findServerAdapter: vi.fn(() => mockAdapter),
findActiveServerAdapter: vi.fn(() => mockAdapter),
@@ -129,6 +134,10 @@ function registerModuleMocks() {
workspaceOperationService: () => mockWorkspaceOperationService,
}));
vi.doMock("../services/secrets.js", () => ({
secretService: () => mockSecretService,
}));
vi.doMock("../adapters/index.js", () => ({
findServerAdapter: vi.fn(() => mockAdapter),
findActiveServerAdapter: vi.fn(() => mockAdapter),
@@ -249,6 +258,7 @@ describe.sequential("agent skill routes", () => {
agent: makeAgent("claude_local"),
});
mockSecretService.resolveAdapterConfigForRuntime.mockResolvedValue({ config: { env: {} } });
mockSecretService.syncEnvBindingsForTarget.mockResolvedValue(undefined);
mockCompanySkillService.listRuntimeSkillEntries.mockResolvedValue([
{
key: "paperclipai/paperclip/paperclip",
server/src/__tests__/agents-service-clear-error.test.ts
+2
View File
@@ -66,6 +66,7 @@ describeEmbeddedPostgres("agent service clearError", () => {
status: "error",
pauseReason: "system",
pausedAt: new Date("2026-06-07T00:00:00.000Z"),
errorReason: "Secret is not bound to agent at env.ANTHROPIC_API_KEY",
adapterType: "codex_local",
adapterConfig: {},
runtimeConfig: {},
@@ -116,6 +117,7 @@ describeEmbeddedPostgres("agent service clearError", () => {
status: "idle",
pauseReason: null,
pausedAt: null,
errorReason: null,
});
const [run] = await db.select().from(heartbeatRuns).where(eq(heartbeatRuns.id, runId));
server/src/__tests__/agents-service-secret-bindings.test.ts
+331
View File
@@ -0,0 +1,331 @@
import { randomUUID } from "node:crypto";
import { mkdirSync, rmSync } from "node:fs";
import os from "node:os";
import path from "node:path";
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
import { and, eq } from "drizzle-orm";
import {
agents,
companies,
companySecretBindings,
companySecretProviderConfigs,
companySecretVersions,
companySecrets,
createDb,
} from "@paperclipai/db";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
import { agentService } from "../services/agents.ts";
import { secretService } from "../services/secrets.js";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
if (!embeddedPostgresSupport.supported) {
console.warn(
`Skipping embedded Postgres agent secret binding tests on this host: ${embeddedPostgresSupport.reason ?? "unsupported environment"}`,
);
}
describeEmbeddedPostgres("agent service secret binding sync", () => {
let stopDb: (() => Promise<void>) | null = null;
let db!: ReturnType<typeof createDb>;
const previousKeyFile = process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE;
const secretsTmpDir = path.join(os.tmpdir(), `paperclip-agent-secret-bindings-${randomUUID()}`);
beforeAll(async () => {
mkdirSync(secretsTmpDir, { recursive: true });
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = path.join(secretsTmpDir, "master.key");
const started = await startEmbeddedPostgresTestDatabase("agent-secret-bindings");
stopDb = started.cleanup;
db = createDb(started.connectionString);
}, 20_000);
afterEach(async () => {
await db.delete(companySecretBindings);
await db.delete(companySecretVersions);
await db.delete(companySecrets);
await db.delete(companySecretProviderConfigs);
await db.delete(agents);
await db.delete(companies);
});
afterAll(async () => {
await stopDb?.();
if (previousKeyFile === undefined) {
delete process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE;
} else {
process.env.PAPERCLIP_SECRETS_MASTER_KEY_FILE = previousKeyFile;
}
rmSync(secretsTmpDir, { recursive: true, force: true });
});
async function seedCompany() {
const companyId = randomUUID();
await db.insert(companies).values({
id: companyId,
name: "Paperclip",
issuePrefix: `T${companyId.replace(/-/g, "").slice(0, 6).toUpperCase()}`,
requireBoardApprovalForNewAgents: false,
});
return companyId;
}
it("creates agent secret bindings when a new agent persists secret_ref env", async () => {
const companyId = await seedCompany();
const secrets = secretService(db);
const secret = await secrets.create(companyId, {
name: `anthropic-${randomUUID()}`,
provider: "local_encrypted",
value: "sk-ant-123",
});
const created = await agentService(db).create(companyId, {
name: "Claude Novita",
role: "engineer",
status: "pending_approval",
adapterType: "claude_local",
adapterConfig: {
env: {
ANTHROPIC_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
},
},
runtimeConfig: {},
spentMonthlyCents: 0,
lastHeartbeatAt: null,
});
const bindings = await db
.select()
.from(companySecretBindings)
.where(and(
eq(companySecretBindings.companyId, companyId),
eq(companySecretBindings.targetType, "agent"),
eq(companySecretBindings.targetId, created.id),
));
expect(bindings).toHaveLength(1);
expect(bindings[0]).toMatchObject({
secretId: secret.id,
configPath: "env.ANTHROPIC_API_KEY",
versionSelector: "latest",
required: true,
});
});
it("replaces agent secret bindings when adapterConfig env changes", async () => {
const companyId = await seedCompany();
const secrets = secretService(db);
const oldSecret = await secrets.create(companyId, {
name: `old-${randomUUID()}`,
provider: "local_encrypted",
value: "old-value",
});
const nextSecret = await secrets.create(companyId, {
name: `next-${randomUUID()}`,
provider: "local_encrypted",
value: "next-value",
});
const created = await agentService(db).create(companyId, {
name: "Binding Swapper",
role: "engineer",
adapterType: "codex_local",
adapterConfig: {
env: {
OLD_KEY: { type: "secret_ref", secretId: oldSecret.id, version: "latest" },
},
},
runtimeConfig: {},
spentMonthlyCents: 0,
lastHeartbeatAt: null,
});
await agentService(db).update(created.id, {
adapterConfig: {
env: {
NEW_KEY: { type: "secret_ref", secretId: nextSecret.id, version: "latest" },
},
},
});
const bindings = await db
.select()
.from(companySecretBindings)
.where(and(
eq(companySecretBindings.companyId, companyId),
eq(companySecretBindings.targetType, "agent"),
eq(companySecretBindings.targetId, created.id),
));
expect(bindings).toHaveLength(1);
expect(bindings[0]).toMatchObject({
secretId: nextSecret.id,
configPath: "env.NEW_KEY",
});
});
it("backfills missing secret bindings when a legacy pending agent is approved", async () => {
const companyId = await seedCompany();
const secrets = secretService(db);
const secret = await secrets.create(companyId, {
name: `legacy-${randomUUID()}`,
provider: "local_encrypted",
value: "legacy-value",
});
const agentId = randomUUID();
await db.insert(agents).values({
id: agentId,
companyId,
name: "Legacy Pending Agent",
role: "engineer",
status: "pending_approval",
adapterType: "claude_local",
adapterConfig: {
env: {
ANTHROPIC_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
},
},
runtimeConfig: {},
permissions: {},
});
const beforeBindings = await db
.select()
.from(companySecretBindings)
.where(eq(companySecretBindings.targetId, agentId));
expect(beforeBindings).toHaveLength(0);
const approved = await agentService(db).activatePendingApproval(agentId);
expect(approved).toMatchObject({
activated: true,
agent: {
id: agentId,
status: "idle",
},
});
const afterBindings = await db
.select()
.from(companySecretBindings)
.where(and(
eq(companySecretBindings.companyId, companyId),
eq(companySecretBindings.targetType, "agent"),
eq(companySecretBindings.targetId, agentId),
));
expect(afterBindings).toHaveLength(1);
expect(afterBindings[0]).toMatchObject({
secretId: secret.id,
configPath: "env.ANTHROPIC_API_KEY",
});
});
it("rolls back create when binding sync fails", async () => {
const companyId = await seedCompany();
const missingSecretId = randomUUID();
await expect(
agentService(db).create(companyId, {
name: "Broken Create",
role: "engineer",
adapterType: "claude_local",
adapterConfig: {
env: {
ANTHROPIC_API_KEY: { type: "secret_ref", secretId: missingSecretId, version: "latest" },
},
},
runtimeConfig: {},
spentMonthlyCents: 0,
lastHeartbeatAt: null,
}),
).rejects.toBeTruthy();
const persistedAgents = await db
.select()
.from(agents)
.where(eq(agents.companyId, companyId));
expect(persistedAgents).toHaveLength(0);
});
it("rolls back adapterConfig updates when binding sync fails", async () => {
const companyId = await seedCompany();
const secrets = secretService(db);
const validSecret = await secrets.create(companyId, {
name: `valid-${randomUUID()}`,
provider: "local_encrypted",
value: "valid-value",
});
const created = await agentService(db).create(companyId, {
name: "Transactional Update",
role: "engineer",
adapterType: "codex_local",
adapterConfig: {
env: {
API_KEY: { type: "secret_ref", secretId: validSecret.id, version: "latest" },
},
},
runtimeConfig: {},
spentMonthlyCents: 0,
lastHeartbeatAt: null,
});
await expect(
agentService(db).update(created.id, {
adapterConfig: {
env: {
API_KEY: { type: "secret_ref", secretId: randomUUID(), version: "latest" },
},
},
}),
).rejects.toBeTruthy();
const reloaded = await agentService(db).getById(created.id);
expect(reloaded?.adapterConfig).toMatchObject({
env: {
API_KEY: { type: "secret_ref", secretId: validSecret.id, version: "latest" },
},
});
const bindings = await db
.select()
.from(companySecretBindings)
.where(and(
eq(companySecretBindings.companyId, companyId),
eq(companySecretBindings.targetType, "agent"),
eq(companySecretBindings.targetId, created.id),
));
expect(bindings).toHaveLength(1);
expect(bindings[0]?.secretId).toBe(validSecret.id);
});
it("keeps pending approval status when activation binding sync fails", async () => {
const companyId = await seedCompany();
const agentId = randomUUID();
await db.insert(agents).values({
id: agentId,
companyId,
name: "Broken Pending Agent",
role: "engineer",
status: "pending_approval",
adapterType: "claude_local",
adapterConfig: {
env: {
ANTHROPIC_API_KEY: { type: "secret_ref", secretId: randomUUID(), version: "latest" },
},
},
runtimeConfig: {},
permissions: {},
});
await expect(agentService(db).activatePendingApproval(agentId)).rejects.toBeTruthy();
const reloaded = await agentService(db).getById(agentId);
expect(reloaded?.status).toBe("pending_approval");
});
});
server/src/__tests__/approvals-service.test.ts
+31 -1
View File
@@ -58,7 +58,7 @@ function createDbStub(selectResults: ApprovalRecord[][], updateResults: Approval
describe("approvalService resolution idempotency", () => {
beforeEach(() => {
vi.clearAllMocks();
mockAgentService.activatePendingApproval.mockResolvedValue(undefined);
mockAgentService.activatePendingApproval.mockResolvedValue({ agent: { id: "agent-1" }, activated: true });
mockAgentService.create.mockResolvedValue({ id: "agent-1" });
mockAgentService.terminate.mockResolvedValue(undefined);
mockNotifyHireApproved.mockResolvedValue(undefined);
@@ -104,4 +104,34 @@ describe("approvalService resolution idempotency", () => {
expect(mockAgentService.activatePendingApproval).toHaveBeenCalledWith("agent-1");
expect(mockNotifyHireApproved).toHaveBeenCalledTimes(1);
});
it("creates the agent from payload when approval does not reference a pending agent", async () => {
const approved = {
...createApproval("approved"),
payload: {
name: "New Agent",
adapterConfig: {
env: {
API_KEY: {
type: "secret_ref",
secretId: "secret-1",
version: "latest",
},
},
},
},
};
const dbStub = createDbStub([[{ ...createApproval("pending"), payload: approved.payload }]], [approved]);
const svc = approvalService(dbStub.db as any);
const result = await svc.approve("approval-1", "board", "ship it");
expect(result.applied).toBe(true);
expect(mockAgentService.create).toHaveBeenCalledWith(
"company-1",
expect.objectContaining({
adapterConfig: approved.payload.adapterConfig,
}),
);
});
});
server/src/__tests__/heartbeat-process-recovery.test.ts
+89
View File
@@ -8,6 +8,8 @@ import {
agentRuntimeState,
agentWakeupRequests,
budgetPolicies,
companySecretBindings,
companySecrets,
companySkills,
companies,
costEvents,
@@ -96,6 +98,7 @@ import {
heartbeatService,
redactDetectedSuccessfulRunProgressSummaryForBoard,
} from "../services/heartbeat.ts";
import { secretService } from "../services/secrets.ts";
import {
SUCCESSFUL_RUN_HANDOFF_EXHAUSTED_NOTICE_BODY,
SUCCESSFUL_RUN_HANDOFF_REQUIRED_NOTICE_BODY,
@@ -411,6 +414,8 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
await db.delete(issueDocuments);
await db.delete(documentRevisions);
await db.delete(documents);
await db.delete(companySecretBindings);
await db.delete(companySecrets);
try {
await db.delete(companies);
break;
@@ -1432,6 +1437,90 @@ describeEmbeddedPostgres("heartbeat orphaned process recovery", () => {
expect(validationComment).toBeTruthy();
});
it("blocks before dispatch when a declared secret ref has no binding instead of emitting an opaque setup failure", async () => {
const { companyId, agentId, runId, issueId } = await seedQueuedIssueRunFixture();
const svc = secretService(db);
const secretName = `unbound-runtime-${randomUUID()}`;
const secret = await svc.create(companyId, {
name: secretName,
provider: "local_encrypted",
value: "never-resolved",
});
// Declare the secret ref on the agent env WITHOUT creating a binding so the
// pre-dispatch gate short-circuits to a configuration-incomplete blocker.
await db
.update(agents)
.set({
adapterConfig: {
env: {
UNBOUND_API_KEY: { type: "secret_ref", secretId: secret.id, version: "latest" },
},
},
})
.where(eq(agents.id, agentId));
const heartbeat = heartbeatService(db);
await heartbeat.resumeQueuedRuns();
await waitForRunToSettle(heartbeat, runId, 5_000);
expect(mockAdapterExecute).not.toHaveBeenCalled();
const failedRun = await db
.select()
.from(heartbeatRuns)
.where(eq(heartbeatRuns.id, runId))
.then((rows) => rows[0] ?? null);
expect(failedRun).toMatchObject({
status: "failed",
errorCode: "configuration_incomplete",
});
expect(failedRun?.error).toContain("configuration incomplete");
expect(failedRun?.error).toContain(secretName);
expect(failedRun?.error).toContain("env.UNBOUND_API_KEY");
expect(failedRun?.resultJson).toMatchObject({
configurationIncomplete: {
reason: "secret_binding_missing",
missingBindings: [
{
consumerType: "agent",
consumerId: agentId,
configPath: "env.UNBOUND_API_KEY",
envKey: "UNBOUND_API_KEY",
secretId: secret.id,
secretName,
},
],
},
});
// Value-free gate: no secret access events were recorded.
expect(await svc.listAccessEvents(companyId, secret.id)).toHaveLength(0);
const issue = await waitForValue(async () =>
db.select().from(issues).where(eq(issues.id, issueId)).then((rows) => {
const row = rows[0] ?? null;
return row?.status === "blocked" ? row : null;
}),
);
expect(issue?.executionRunId).toBeNull();
const recoveryAction = await db
.select()
.from(issueRecoveryActions)
.where(and(eq(issueRecoveryActions.companyId, companyId), eq(issueRecoveryActions.sourceIssueId, issueId)))
.then((rows) => rows[0] ?? null);
expect(recoveryAction).toMatchObject({
kind: "configuration_validation",
cause: "configuration_incomplete",
status: "active",
ownerAgentId: agentId,
recoveryIssueId: null,
});
expect(recoveryAction?.nextAction).toContain("Bind the missing secret");
const comments = await db.select().from(issueComments).where(eq(issueComments.issueId, issueId));
expect(comments.some((comment) => comment.body.includes("secret/env bindings are missing"))).toBe(true);
});
it("queues one finish-handoff wake when a successful run leaves in-progress work without a next action", async () => {
const { companyId, agentId, runId, issueId } = await seedQueuedIssueRunFixture();
mockAdapterExecute.mockImplementationOnce(async (ctx: { runId: string }) => {
server/src/__tests__/secrets-service.test.ts
+40
View File
@@ -223,6 +223,46 @@ describeEmbeddedPostgres("secretService", () => {
expect(JSON.stringify(events)).not.toContain("runtime-secret");
});
it("collects declared secret refs that have no binding without resolving values", async () => {
const companyId = await seedCompany();
const svc = secretService(db);
const secretName = `unbound-${randomUUID()}`;
const secret = await svc.create(companyId, {
name: secretName,
provider: "local_encrypted",
value: "runtime-secret",
});
const env = {
API_KEY: { type: "secret_ref" as const, secretId: secret.id, version: "latest" as const },
PLAIN_VALUE: "not-a-secret",
};
const missing = await svc.collectMissingRuntimeBindings(companyId, env, {
consumerType: "agent",
consumerId: "agent-1",
});
expect(missing).toHaveLength(1);
expect(missing[0]).toMatchObject({
consumerType: "agent",
consumerId: "agent-1",
configPath: "env.API_KEY",
envKey: "API_KEY",
secretId: secret.id,
secretName,
});
// Value-free validation: no access events recorded.
expect(await svc.listAccessEvents(companyId, secret.id)).toHaveLength(0);
await svc.syncEnvBindingsForTarget(companyId, { targetType: "agent", targetId: "agent-1" }, env);
const afterBinding = await svc.collectMissingRuntimeBindings(companyId, env, {
consumerType: "agent",
consumerId: "agent-1",
});
expect(afterBinding).toEqual([]);
});
it("denies runtime secret resolution outside the low-trust binding allowlist", async () => {
const companyId = await seedCompany();
const svc = secretService(db);
server/src/routes/agents.ts
-16
View File
@@ -2462,14 +2462,6 @@ export function agentRoutes(
lastHeartbeatAt: null,
});
const agent = await materializeDefaultInstructionsBundleForNewAgent(createdAgent, instructionsBundle);
const agentEnv = asRecord(agent.adapterConfig)?.env;
if (agentEnv) {
await secretsSvc.syncEnvBindingsForTarget?.(
companyId,
{ targetType: "agent", targetId: agent.id },
agentEnv,
);
}
const actor = getActorInfo(req);
await logActivity(db, {
@@ -2952,14 +2944,6 @@ export function agentRoutes(
res.status(404).json({ error: "Agent not found" });
return;
}
if (touchesAdapterConfiguration) {
const agentEnv = asRecord(agent.adapterConfig)?.env;
await secretsSvc.syncEnvBindingsForTarget?.(
agent.companyId,
{ targetType: "agent", targetId: agent.id },
agentEnv,
);
}
await logActivity(db, {
companyId: agent.companyId,
server/src/services/agent-secret-bindings.ts
+26
View File
@@ -0,0 +1,26 @@
interface AgentSecretBindingSyncService {
syncEnvBindingsForTarget?: (
companyId: string,
target: { targetType: "agent"; targetId: string; pathPrefix?: string },
envValue: unknown,
) => Promise<unknown>;
}
function asRecord(value: unknown): Record<string, unknown> | null {
if (typeof value !== "object" || value === null || Array.isArray(value)) return null;
return value as Record<string, unknown>;
}
export async function syncAgentAdapterEnvBindings(input: {
secretsSvc: AgentSecretBindingSyncService;
companyId: string;
agentId: string;
adapterConfig: unknown;
}) {
const envValue = asRecord(asRecord(input.adapterConfig)?.env);
await input.secretsSvc.syncEnvBindingsForTarget?.(
input.companyId,
{ targetType: "agent", targetId: input.agentId },
envValue,
);
}
server/src/services/agents.ts
+89 -39
View File
@@ -24,8 +24,10 @@ import {
type AgentEligibilityAgent,
} from "@paperclipai/shared";
import { conflict, notFound, unprocessable } from "../errors.js";
import { syncAgentAdapterEnvBindings } from "./agent-secret-bindings.js";
import { normalizeAgentPermissions } from "./agent-permissions.js";
import { REDACTED_EVENT_VALUE, sanitizeRecord } from "../redaction.js";
import { secretService } from "./secrets.js";
function hashToken(token: string) {
return createHash("sha256").update(token).digest("hex");
@@ -218,6 +220,8 @@ export function deduplicateAgentName(
}
export function agentService(db: Db) {
const secretsSvc = secretService(db);
function currentUtcMonthWindow(now = new Date()) {
const year = now.getUTCFullYear();
const month = now.getUTCMonth();
@@ -371,6 +375,19 @@ export function agentService(db: Db) {
}
}
async function syncAgentSecretBindings(
agent: { id: string; companyId: string; adapterConfig: unknown },
dbClient: Db = db,
) {
const scopedSecretsSvc = dbClient === db ? secretsSvc : secretService(dbClient);
await syncAgentAdapterEnvBindings({
secretsSvc: scopedSecretsSvc,
companyId: agent.companyId,
agentId: agent.id,
adapterConfig: agent.adapterConfig,
});
}
async function updateAgent(
id: string,
data: Partial<typeof agents.$inferInsert>,
@@ -415,33 +432,45 @@ export function agentService(db: Db) {
const shouldRecordRevision = Boolean(options?.recordRevision) && hasConfigPatchFields(normalizedPatch);
const beforeConfig = shouldRecordRevision ? buildConfigSnapshot(existing) : null;
const updated = await db
.update(agents)
.set({ ...normalizedPatch, updatedAt: new Date() })
.where(eq(agents.id, id))
.returning()
.then((rows) => rows[0] ?? null);
const normalizedUpdated = updated ? await getById(updated.id) : null;
return db.transaction(async (tx) => {
const txDb = tx as unknown as Db;
const updated = await tx
.update(agents)
.set({ ...normalizedPatch, updatedAt: new Date() })
.where(eq(agents.id, id))
.returning()
.then((rows) => rows[0] ?? null);
if (!updated) return null;
if (normalizedUpdated && shouldRecordRevision && beforeConfig) {
const afterConfig = buildConfigSnapshot(normalizedUpdated);
const changedKeys = diffConfigSnapshot(beforeConfig, afterConfig);
if (changedKeys.length > 0) {
await db.insert(agentConfigRevisions).values({
companyId: normalizedUpdated.companyId,
agentId: normalizedUpdated.id,
createdByAgentId: options?.recordRevision?.createdByAgentId ?? null,
createdByUserId: options?.recordRevision?.createdByUserId ?? null,
source: options?.recordRevision?.source ?? "patch",
rolledBackFromRevisionId: options?.recordRevision?.rolledBackFromRevisionId ?? null,
changedKeys,
beforeConfig: beforeConfig as unknown as Record<string, unknown>,
afterConfig: afterConfig as unknown as Record<string, unknown>,
});
if (Object.prototype.hasOwnProperty.call(normalizedPatch, "adapterConfig")) {
await syncAgentSecretBindings(updated, txDb);
}
}
return normalizedUpdated;
const normalizedUpdated = await agentService(txDb).getById(updated.id);
if (!normalizedUpdated) {
throw notFound("Agent not found");
}
if (shouldRecordRevision && beforeConfig) {
const afterConfig = buildConfigSnapshot(normalizedUpdated);
const changedKeys = diffConfigSnapshot(beforeConfig, afterConfig);
if (changedKeys.length > 0) {
await tx.insert(agentConfigRevisions).values({
companyId: normalizedUpdated.companyId,
agentId: normalizedUpdated.id,
createdByAgentId: options?.recordRevision?.createdByAgentId ?? null,
createdByUserId: options?.recordRevision?.createdByUserId ?? null,
source: options?.recordRevision?.source ?? "patch",
rolledBackFromRevisionId: options?.recordRevision?.rolledBackFromRevisionId ?? null,
changedKeys,
beforeConfig: beforeConfig as unknown as Record<string, unknown>,
afterConfig: afterConfig as unknown as Record<string, unknown>,
});
}
}
return normalizedUpdated;
});
}
return {
@@ -474,13 +503,20 @@ export function agentService(db: Db) {
const role = data.role ?? "general";
const normalizedPermissions = normalizeAgentPermissions(data.permissions, role);
const runtimeConfig = normalizeRuntimeConfigForNewAgent(data.runtimeConfig);
const created = await db
.insert(agents)
.values({ ...data, name: uniqueName, companyId, role, permissions: normalizedPermissions, runtimeConfig })
.returning()
.then((rows) => rows[0]);
return requireGetById(created.id);
return db.transaction(async (tx) => {
const txDb = tx as unknown as Db;
const created = await tx
.insert(agents)
.values({ ...data, name: uniqueName, companyId, role, permissions: normalizedPermissions, runtimeConfig })
.returning()
.then((rows) => rows[0]);
await syncAgentSecretBindings(created, txDb);
const normalizedCreated = await agentService(txDb).getById(created.id);
if (!normalizedCreated) {
throw notFound("Agent not found");
}
return normalizedCreated;
});
},
update: updateAgent,
@@ -496,6 +532,7 @@ export function agentService(db: Db) {
status: "paused",
pauseReason: reason,
pausedAt: new Date(),
errorReason: null,
updatedAt: new Date(),
})
.where(eq(agents.id, id))
@@ -518,6 +555,7 @@ export function agentService(db: Db) {
status: "idle",
pauseReason: null,
pausedAt: null,
errorReason: null,
updatedAt: new Date(),
})
.where(eq(agents.id, id))
@@ -543,6 +581,7 @@ export function agentService(db: Db) {
status: "idle",
pauseReason: null,
pausedAt: null,
errorReason: null,
updatedAt: new Date(),
})
.where(and(eq(agents.id, id), eq(agents.status, "error")))
@@ -565,6 +604,7 @@ export function agentService(db: Db) {
status: "terminated",
pauseReason: null,
pausedAt: null,
errorReason: null,
updatedAt: new Date(),
})
.where(eq(agents.id, id));
@@ -611,15 +651,25 @@ export function agentService(db: Db) {
},
activatePendingApproval: async (id: string) => {
const updated = await db
.update(agents)
.set({ status: "idle", updatedAt: new Date() })
.where(and(eq(agents.id, id), eq(agents.status, "pending_approval")))
.returning()
.then((rows) => rows[0] ?? null);
const activatedAgent = await db.transaction(async (tx) => {
const txDb = tx as unknown as Db;
const updated = await tx
.update(agents)
.set({ status: "idle", updatedAt: new Date() })
.where(and(eq(agents.id, id), eq(agents.status, "pending_approval")))
.returning()
.then((rows) => rows[0] ?? null);
if (!updated) return null;
await syncAgentSecretBindings(updated, txDb);
const agent = await agentService(txDb).getById(updated.id);
if (!agent) {
throw notFound("Agent not found");
}
return agent;
});
if (updated) {
return { agent: await requireGetById(updated.id), activated: true };
if (activatedAgent) {
return { agent: activatedAgent, activated: true };
}
const existing = await getById(id);
server/src/services/heartbeat.ts
+156 -22
View File
@@ -71,7 +71,7 @@ import { trackAgentFirstHeartbeat } from "@paperclipai/shared/telemetry";
import { getTelemetryClient } from "../telemetry.js";
import { companySkillService } from "./company-skills.js";
import { budgetService, type BudgetEnforcementScope } from "./budgets.js";
import { secretService } from "./secrets.js";
import { secretService, type MissingRuntimeBinding } from "./secrets.js";
import { resolveDefaultAgentWorkspaceDir, resolveManagedProjectWorkspaceDir } from "../home-paths.js";
import {
buildHeartbeatRunIssueComment,
@@ -254,6 +254,8 @@ const BOUNDED_TRANSIENT_HEARTBEAT_RETRY_WAKE_REASON = "transient_failure_retry";
const BOUNDED_TRANSIENT_HEARTBEAT_RETRY_MAX_ATTEMPTS = BOUNDED_TRANSIENT_HEARTBEAT_RETRY_DELAYS_MS.length;
const WORKSPACE_VALIDATION_FAILURE_CODE = "workspace_validation_failed";
const WORKSPACE_VALIDATION_RECOVERY_CAUSE = "workspace_validation_failed";
const CONFIGURATION_INCOMPLETE_FAILURE_CODE = "configuration_incomplete";
const CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE = "configuration_incomplete";
// Keep this in sync with local adapters that require a git workspace before launch.
const GIT_SENSITIVE_LOCAL_ADAPTER_TYPES = new Set([
"acpx_local",
@@ -296,6 +298,20 @@ export class WorkspaceValidationFailure extends Error {
}
}
// Pre-dispatch gate outcome: required secret/env bindings are missing, so the
// run must not be dispatched. Surfaced as a configuration-incomplete blocker
// routed to a human owner instead of N opaque dispatched-then-failed runs.
export class ConfigurationIncompleteFailure extends Error {
code = CONFIGURATION_INCOMPLETE_FAILURE_CODE;
resultJson: Record<string, unknown>;
constructor(message: string, resultJson: Record<string, unknown>) {
super(message);
this.name = "ConfigurationIncompleteFailure";
this.resultJson = resultJson;
}
}
function resolveCodexTransientFallbackMode(attempt: number): CodexTransientFallbackMode {
if (attempt <= 1) return "same_session";
if (attempt === 2) return "safer_invocation";
@@ -381,9 +397,16 @@ const INLINE_BASE64_IMAGE_DATA_RE = /("type":"image","source":\{"type":"base64",
type RuntimeConfigSecretResolver = Pick<
ReturnType<typeof secretService>,
"resolveAdapterConfigForRuntime" | "resolveEnvBindings"
"resolveAdapterConfigForRuntime" | "resolveEnvBindings" | "collectMissingRuntimeBindings"
>;
function formatMissingBindingForOperator(missing: MissingRuntimeBinding): string {
const secretLabel = missing.secretName
? `"${missing.secretName}"`
: missing.secretId;
return `secret ${secretLabel} not bound at ${missing.consumerType} ${missing.configPath}`;
}
const LOW_TRUST_SENSITIVE_ENV_KEY_RE =
/(api[-_]?key|access[-_]?token|auth(?:_?token)?|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring)/i;
@@ -449,6 +472,54 @@ export async function resolveExecutionRunAdapterConfig(input: {
assertLowTrustEnvConfigAllowed(projectEnv, "project.env");
assertLowTrustEnvConfigAllowed(routineEnv, "routine.env");
}
// Pre-dispatch binding-validation gate: detect declared secret refs that have
// no binding before resolving any secret value. Missing bindings short-circuit
// to a configuration-incomplete blocker routed to a human owner instead of a
// dispatched-then-failed run (which previously surfaced as opaque setup_failed).
if (typeof input.secretsSvc.collectMissingRuntimeBindings === "function") {
const missingBindings: MissingRuntimeBinding[] = [];
if (input.agentId) {
missingBindings.push(
...(await input.secretsSvc.collectMissingRuntimeBindings(
input.companyId,
parseObject(executionRunConfig.env),
{ consumerType: "agent", consumerId: input.agentId },
)),
);
}
if (projectEnv && input.projectId) {
missingBindings.push(
...(await input.secretsSvc.collectMissingRuntimeBindings(
input.companyId,
projectEnv,
{ consumerType: "project", consumerId: input.projectId },
)),
);
}
if (routineEnv && input.routineId) {
missingBindings.push(
...(await input.secretsSvc.collectMissingRuntimeBindings(
input.companyId,
routineEnv,
{ consumerType: "routine", consumerId: input.routineId },
)),
);
}
if (missingBindings.length > 0) {
const detail = missingBindings.map(formatMissingBindingForOperator).join("; ");
throw new ConfigurationIncompleteFailure(`configuration incomplete: ${detail}`, {
configurationIncomplete: {
reason: "secret_binding_missing",
companyId: input.companyId,
agentId: input.agentId ?? null,
issueId: input.issueId ?? null,
projectId: input.projectId ?? null,
routineId: input.routineId ?? null,
missingBindings,
},
});
}
}
const { config: resolvedConfig, secretKeys, manifest } = await input.secretsSvc.resolveAdapterConfigForRuntime(
input.companyId,
executionRunConfig,
@@ -939,6 +1010,16 @@ function isWorkspaceValidationFailedRun(
return run?.errorCode === WORKSPACE_VALIDATION_FAILURE_CODE;
}
function isConfigurationIncompleteFailure(error: unknown): error is ConfigurationIncompleteFailure {
return error instanceof ConfigurationIncompleteFailure;
}
function isConfigurationIncompleteFailedRun(
run: Pick<typeof heartbeatRuns.$inferSelect, "errorCode"> | null | undefined,
) {
return run?.errorCode === CONFIGURATION_INCOMPLETE_FAILURE_CODE;
}
async function hasGitMetadata(cwd: string | null | undefined) {
const normalized = readNonEmptyString(cwd);
if (!normalized) return false;
@@ -7199,9 +7280,17 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
return cancelled;
}
function truncateAgentErrorReason(reason: string | null | undefined): string | null {
if (!reason) return null;
const trimmed = reason.trim();
if (!trimmed) return null;
return trimmed.length > 500 ? `${trimmed.slice(0, 499)}…` : trimmed;
}
async function finalizeAgentStatus(
agentId: string,
outcome: "succeeded" | "failed" | "cancelled" | "timed_out",
failureReason?: string | null,
) {
const existing = await getAgent(agentId);
if (!existing) return;
@@ -7224,6 +7313,10 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
.update(agents)
.set({
status: nextStatus,
// Persist a human-readable reason on the agent record when it enters
// error so operators see it on the agent page without digging into run
// events; clear it whenever the agent leaves error.
errorReason: nextStatus === "error" ? truncateAgentErrorReason(failureReason) : null,
lastHeartbeatAt: new Date(),
updatedAt: new Date(),
})
@@ -7586,7 +7679,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
},
});
await finalizeAgentStatus(run.agentId, "failed");
await finalizeAgentStatus(run.agentId, "failed", baseMessage);
await startNextQueuedRunForAgent(run.agentId);
runningProcesses.delete(run.id);
reaped.push(run.id);
@@ -9439,14 +9532,20 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
}
}
}
await finalizeAgentStatus(agent.id, outcome);
await finalizeAgentStatus(
agent.id,
outcome,
outcome === "succeeded" ? null : (adapterResult.errorMessage ?? null),
);
} catch (err) {
const message = redactCurrentUserText(
err instanceof Error ? err.message : "Unknown adapter failure",
await getCurrentUserRedactionOptions(),
);
const workspaceValidationFailure = isWorkspaceValidationFailure(err) ? err : null;
const failureErrorCode = workspaceValidationFailure?.code ?? "adapter_failed";
const configurationIncompleteFailure = isConfigurationIncompleteFailure(err) ? err : null;
const failureErrorCode =
workspaceValidationFailure?.code ?? configurationIncompleteFailure?.code ?? "adapter_failed";
logger.error({ err, runId }, "heartbeat execution failed");
let logSummary: { bytes: number; sha256?: string; compressed: boolean } | null = null;
@@ -9472,7 +9571,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
resultJson: mergeRunStopMetadataForAgent(agent, "failed", {
errorCode: failureErrorCode,
errorMessage: message,
resultJson: workspaceValidationFailure?.resultJson ?? null,
resultJson: workspaceValidationFailure?.resultJson ?? configurationIncompleteFailure?.resultJson ?? null,
}),
stdoutExcerpt,
stderrExcerpt,
@@ -9507,7 +9606,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
});
const livenessRun = await classifyAndPersistRunLiveness(failedRun) ?? failedRun;
await refreshContinuationSummaryForRun(livenessRun, agent);
if (!isWorkspaceValidationFailedRun(livenessRun)) {
if (!isWorkspaceValidationFailedRun(livenessRun) && !isConfigurationIncompleteFailedRun(livenessRun)) {
await finalizeIssueCommentPolicy(livenessRun, agent);
}
await releaseIssueExecutionAndPromote(livenessRun);
@@ -9535,22 +9634,31 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
}
}
await finalizeAgentStatus(agent.id, "failed");
await finalizeAgentStatus(agent.id, "failed", message);
}
} catch (outerErr) {
// Setup code before adapter.execute threw (e.g. ensureRuntimeState, resolveWorkspaceForRun).
// The inner catch did not fire, so we must record the failure here.
const message = outerErr instanceof Error ? outerErr.message : "Unknown setup failure";
const message = redactCurrentUserText(
outerErr instanceof Error ? outerErr.message : "Unknown setup failure",
await getCurrentUserRedactionOptions(),
);
// A missing secret/env binding is a known pre-dispatch configuration gap,
// not an opaque setup crash. Surface it with its own errorCode so the
// recovery path routes it to a human owner instead of looping retries.
const configurationIncompleteSetupFailure = isConfigurationIncompleteFailure(outerErr) ? outerErr : null;
const setupFailureErrorCode = configurationIncompleteSetupFailure?.code ?? "setup_failed";
logger.error({ err: outerErr, runId }, "heartbeat execution setup failed");
const setupFailureAgent = await getAgent(run.agentId).catch(() => null);
const setupFailureWrite = await setRunStatusIfRunning(runId, "failed", {
error: message,
errorCode: "setup_failed",
errorCode: setupFailureErrorCode,
finishedAt: new Date(),
...(setupFailureAgent ? {
resultJson: mergeRunStopMetadataForAgent(setupFailureAgent, "failed", {
errorCode: "setup_failed",
errorCode: setupFailureErrorCode,
errorMessage: message,
resultJson: configurationIncompleteSetupFailure?.resultJson ?? null,
}),
} : {}),
}).catch(() => ({ run: null, updated: false as const }));
@@ -9583,7 +9691,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
const failedAgent = setupFailureAgent ?? await getAgent(run.agentId).catch(() => null);
if (failedAgent) {
await refreshContinuationSummaryForRun(livenessRun, failedAgent).catch(() => undefined);
if (!isWorkspaceValidationFailedRun(livenessRun)) {
if (!isWorkspaceValidationFailedRun(livenessRun) && !isConfigurationIncompleteFailedRun(livenessRun)) {
await finalizeIssueCommentPolicy(livenessRun, failedAgent).catch(() => undefined);
}
}
@@ -9593,7 +9701,7 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
// path owned the terminal transition. If another path already finalized
// the run, keep that terminal outcome authoritative.
if (setupFailureWrite.updated) {
await finalizeAgentStatus(run.agentId, "failed").catch(() => undefined);
await finalizeAgentStatus(run.agentId, "failed", message).catch(() => undefined);
}
} finally {
const latestRun = await getRun(run.id).catch(() => null);
@@ -9641,6 +9749,17 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
);
}
function buildConfigurationIncompleteRecoveryComment(input: {
latestRun: Pick<typeof heartbeatRuns.$inferSelect, "error" | "errorCode"> | null | undefined;
}) {
const failureSummary = summarizeRunFailureForIssueComment(input.latestRun);
return (
"Paperclip stopped before dispatching the adapter because required secret/env bindings are missing. " +
`Resolving them as a runtime failure would only produce repeated opaque setup failures.${failureSummary ?? ""} ` +
"Moving it to `blocked` with a source-scoped recovery action so an operator can bind the missing secret(s) before resuming."
);
}
async function releaseIssueExecutionAndPromote(run: typeof heartbeatRuns.$inferSelect) {
const runContext = parseObject(run.contextSnapshot);
const contextIssueId = readNonEmptyString(runContext.issueId);
@@ -9761,17 +9880,22 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
// Sibling lock cleanup is already done above; only the primary issue carries
// the recovery surface because the comment is attached to a single issue.
if (
isWorkspaceValidationFailedRun(run) &&
(isWorkspaceValidationFailedRun(run) || isConfigurationIncompleteFailedRun(run)) &&
(issue.status === "todo" || issue.status === "in_progress") &&
!issue.assigneeUserId &&
issue.assigneeAgentId === run.agentId
) {
const configurationIncomplete = isConfigurationIncompleteFailedRun(run);
return {
kind: "blocked" as const,
issue,
previousStatus: issue.status,
comment: buildWorkspaceValidationRecoveryComment({ latestRun: run }),
recoveryCause: WORKSPACE_VALIDATION_RECOVERY_CAUSE,
comment: configurationIncomplete
? buildConfigurationIncompleteRecoveryComment({ latestRun: run })
: buildWorkspaceValidationRecoveryComment({ latestRun: run }),
recoveryCause: configurationIncomplete
? CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE
: WORKSPACE_VALIDATION_RECOVERY_CAUSE,
};
}
@@ -10057,21 +10181,29 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
!recoveryAgentInvokable ||
!recoveryAgent ||
isWorkspaceValidationFailedRun(run) ||
isConfigurationIncompleteFailedRun(run) ||
didAutomaticRecoveryFail(run, issue.status === "todo" ? "assignment_recovery" : "issue_continuation_needed");
if (shouldBlockImmediately) {
const workspaceValidationFailure = isWorkspaceValidationFailedRun(run);
const configurationIncompleteFailure = isConfigurationIncompleteFailedRun(run);
const comment = workspaceValidationFailure
? buildWorkspaceValidationRecoveryComment({ latestRun: run })
: buildImmediateExecutionPathRecoveryComment({
status: issue.status as "todo" | "in_progress",
latestRun: run,
});
: configurationIncompleteFailure
? buildConfigurationIncompleteRecoveryComment({ latestRun: run })
: buildImmediateExecutionPathRecoveryComment({
status: issue.status as "todo" | "in_progress",
latestRun: run,
});
return {
kind: "blocked" as const,
issue,
previousStatus: issue.status,
comment,
recoveryCause: workspaceValidationFailure ? WORKSPACE_VALIDATION_RECOVERY_CAUSE : undefined,
recoveryCause: workspaceValidationFailure
? WORKSPACE_VALIDATION_RECOVERY_CAUSE
: configurationIncompleteFailure
? CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE
: undefined,
};
}
@@ -10157,7 +10289,9 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
recoveryCause:
promotionResult.recoveryCause === WORKSPACE_VALIDATION_RECOVERY_CAUSE
? WORKSPACE_VALIDATION_RECOVERY_CAUSE
: undefined,
: promotionResult.recoveryCause === CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE
? CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE
: undefined,
});
return;
}
server/src/services/recovery/service.ts
+12 -3
View File
@@ -119,6 +119,7 @@ type SuccessfulLatestIssueRun = NonNullable<LatestIssueRun> & { status: "succeed
type StrandedRecoveryCause =
| "stranded_assigned_issue"
| "workspace_validation_failed"
| "configuration_incomplete"
| typeof SUCCESSFUL_RUN_MISSING_STATE_REASON;
type SuccessfulRunHandoffRecoveryEvidence = {
@@ -2231,6 +2232,8 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
? "missing_disposition" as const
: cause === "workspace_validation_failed"
? "workspace_validation" as const
: cause === "configuration_incomplete"
? "configuration_validation" as const
: "stranded_assigned_issue" as const;
}
@@ -2306,11 +2309,13 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
? "Choose and record a valid issue disposition without copying transcript content."
: recoveryCause === "workspace_validation_failed"
? "Repair the source issue workspace link, project workspace cwd, or git checkout before resuming adapter execution."
: recoveryCause === "configuration_incomplete"
? "Bind the missing secret(s) named in the run failure to the agent/project/routine env before resuming adapter execution."
: "Restore a live execution path, fix the runtime/adapter failure, or record an intentional manual resolution.",
wakePolicy: recoveryCause === "workspace_validation_failed"
wakePolicy: recoveryCause === "workspace_validation_failed" || recoveryCause === "configuration_incomplete"
? {
type: "manual_repair_required",
reason: "workspace_validation_failed",
reason: recoveryCause,
ownerAgentId,
}
: ownerAgentId
@@ -2338,6 +2343,7 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
recoveryCause: StrandedRecoveryCause;
}) {
if (input.recoveryCause === "workspace_validation_failed") return;
if (input.recoveryCause === "configuration_incomplete") return;
if (!input.action.ownerAgentId) return;
await deps.enqueueWakeup(input.action.ownerAgentId, {
source: "assignment",
@@ -2543,7 +2549,8 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
const shouldPostEscalationComment =
recoveryAction.attemptCount === 1 ||
input.recoveryCause === "workspace_validation_failed";
input.recoveryCause === "workspace_validation_failed" ||
input.recoveryCause === "configuration_incomplete";
if (shouldPostEscalationComment) {
const escalationCommentMarker = `Recovery action: \`${recoveryAction.id}\``;
@@ -2597,6 +2604,8 @@ export function recoveryService(db: Db, deps: { enqueueWakeup: RecoveryWakeup })
? "recovery.reconcile_successful_run_handoff_missing_state"
: input.recoveryCause === "workspace_validation_failed"
? "recovery.reconcile_workspace_validation_failed"
: input.recoveryCause === "configuration_incomplete"
? "recovery.reconcile_configuration_incomplete"
: "recovery.reconcile_stranded_assigned_issue",
recoveryCause: input.recoveryCause ?? "stranded_assigned_issue",
latestRunId: input.latestRun?.id ?? null,
server/src/services/secrets.ts
+63
View File
@@ -203,6 +203,15 @@ export type RuntimeSecretManifestEntry = {
errorCode?: string | null;
};
export type MissingRuntimeBinding = {
consumerType: SecretBindingTargetType;
consumerId: string;
configPath: string;
envKey: string;
secretId: string;
secretName: string | null;
};
type RuntimeSecretResolution = {
value: string;
manifestEntry: RuntimeSecretManifestEntry;
@@ -2352,6 +2361,60 @@ export function secretService(db: Db) {
return { env: resolved, secretKeys, manifest };
},
// Pre-dispatch validation: list declared secret refs in an env-like config
// that have no binding for the given consumer, WITHOUT resolving any secret
// values. Callers use this to surface a configuration-incomplete blocker
// before a run is dispatched instead of letting resolution throw mid-setup.
collectMissingRuntimeBindings: async (
companyId: string,
envValue: unknown,
context: Omit<SecretConsumerContext, "configPath">,
): Promise<MissingRuntimeBinding[]> => {
const record = asRecord(envValue);
if (!record) return [];
const secretRefs = Object.entries(record).flatMap(([key, rawBinding]) => {
if (!ENV_KEY_RE.test(key)) return [];
const parsed = envBindingSchema.safeParse(rawBinding);
if (!parsed.success) return [];
const binding = canonicalizeBinding(parsed.data as EnvBinding);
if (binding.type !== "secret_ref") return [];
return [{ key, configPath: `env.${key}`, secretId: binding.secretId }];
});
if (secretRefs.length === 0) return [];
const bindingChecks = await Promise.all(secretRefs.map(async (entry) => ({
entry,
found: await getBinding({
companyId,
secretId: entry.secretId,
consumerType: context.consumerType,
consumerId: context.consumerId,
configPath: entry.configPath,
}),
})));
const missingEntries = bindingChecks
.filter((check) => !check.found)
.map((check) => check.entry);
if (missingEntries.length === 0) return [];
const secretRows = await Promise.all(
[...new Set(missingEntries.map((entry) => entry.secretId))].map(async (secretId) => [
secretId,
await getById(secretId).catch(() => null),
] as const),
);
const secretsById = new Map(secretRows);
return missingEntries.map((entry) => ({
consumerType: context.consumerType,
consumerId: context.consumerId,
configPath: entry.configPath,
envKey: entry.key,
secretId: entry.secretId,
secretName: secretsById.get(entry.secretId)?.name ?? null,
}));
},
resolveAdapterConfigForRuntime: async (
companyId: string,
adapterConfig: Record<string, unknown>,