Improve CLI API parity coverage (#6626)

## Thinking Path

> - Paperclip is a control plane for AI-agent companies, with the CLI
acting as a scriptable operator and agent interface to that control
plane.
> - The REST API surface has grown across companies, agents, issues,
routines, plugins, auth, workspaces, secrets, and operational inspection
commands.
> - The CLI had drifted from that API surface: some commands were
missing, some command shapes differed from docs/reference material, and
several edge cases only failed during end-to-end local-source testing.
> - The local development runbook requires these tests to be disposable
and isolated from a real `~/.paperclip`, `~/.codex`, or `~/.claude`
installation.
> - This pull request adds broad CLI/API parity coverage, fixes the
actionable bugs found during that pass, and records the reproducible
test log under `doc/logs`.
> - The benefit is a more complete, scriptable CLI surface with
regression coverage for the command families exercised by the parity
run.

## What Changed

- Added or expanded CLI command coverage for access/auth, companies,
agents, projects, goals, issues and subresources, routines, plugins,
workspaces, activity/run/cost/dashboard inspection, assets, skills,
secrets, tokens, prompt/wake flows, and local setup helpers.
- Fixed CLI/API parity bugs found during the run, including context
profile patching, issue interaction optional payloads, malformed
tree-hold errors, environment duplicate handling, configure
invalid-section exit codes, worktree pnpm invocation, token agent ID
resolution, plugin tool worker lookup, and routine webhook secret
cleanup.
- Added missing CLI wrappers and route coverage for health/access,
invite resolution URL forwarding, join status normalization, secret
lifecycle commands, LLM docs routes, available-skill isolation, positive
board-claim coverage, and interactive `connect` prompt-flow tests.
- Added a schema-backed `/api/openapi.json` route sufficient for CLI
parity and `paperclipai openapi --json` smoke coverage.
- Added `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` with the
detailed living test/bug log and renamed the log directory from
`doc/bugs` to `doc/logs`.
- Added `doc/plans/2026-05-23-cli-api-parity.md` and the OpenAPI parity
reference used during the pass.

OpenAPI note: this PR intentionally does not try to subsume
`feature/openapi-spec`. The OpenAPI implementation here is schema-backed
and better than the earlier route-inventory stub, but
`feature/openapi-spec` is the fuller/better OpenAPI branch because it
includes exact mounted-route coverage tests and additional current route
coverage. That branch should stay as its own PR and can supersede this
OpenAPI route implementation.

## Verification

Targeted automated checks run:

- `pnpm exec vitest run server/src/__tests__/openapi-routes.test.ts`
- `pnpm exec vitest run server/src/__tests__/board-claim.test.ts`
- `pnpm exec vitest run cli/src/__tests__/connect.test.ts`
- `pnpm exec vitest run cli/src/__tests__/agent-lifecycle.test.ts`
- `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts`
- `pnpm exec vitest run server/src/__tests__/routines-service.test.ts`
- `pnpm --dir cli typecheck`
- `pnpm --dir server typecheck`

Manual/local E2E verification:

- Ran the full disposable local-source CLI/API parity pass with isolated
`PAPERCLIP_HOME`, `PAPERCLIP_CONFIG`, `PAPERCLIP_CONTEXT`,
`PAPERCLIP_AUTH_STORE`, `CODEX_HOME`, and `CLAUDE_HOME` under
`tmp/cli-api-parity`.
- Verified `DATABASE_URL` and `DATABASE_MIGRATION_URL` stayed unset for
the scratch server.
- Verified live health and schema-backed OpenAPI responses on
non-default port `3197`.
- Revoked created board/agent tokens and cleaned up temporary plugins,
secrets, non-default environments, and project workspaces.
- See `doc/logs/2026-05-24-cli-api-parity-e2e-log.md` for the full
command-by-command reproduction log.

Not run:

- Full `pnpm test`, `pnpm test:run`, or `pnpm build` were not run after
the entire branch because the branch is broad and the parity pass used
focused test/typecheck verification plus live isolated CLI reruns.

## Risks

- This is a broad PR and touches many CLI command modules, so review
surface is high. The changes are grouped around one theme, but a split
may be easier if maintainers prefer narrower PRs.
- The OpenAPI route in this PR is not the final/best OpenAPI
implementation. `feature/openapi-spec` has stronger exact-route coverage
and should remain the source for the dedicated OpenAPI PR.
- The living log is intentionally detailed and large. It is useful for
reproducibility but adds documentation weight.
- No UI changes are intended; screenshots are not applicable.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex, GPT-5-based coding agent in Codex desktop. Exact served
model/context-window identifier was not exposed in the local app. Work
used shell/Git/GitHub CLI tooling, local source inspection, targeted
test execution, and live isolated Paperclip CLI/API smoke testing.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Devin Foley <devin@devinfoley.com>
This commit is contained in:
Aron Prins
2026-06-03 02:13:29 +02:00
committed by GitHub
parent 68401f82f3
commit 70b1a9109d
74 changed files with 18175 additions and 111 deletions
+128
View File
@@ -0,0 +1,128 @@
import { randomUUID } from "node:crypto";
import { and, eq } from "drizzle-orm";
import { afterAll, afterEach, beforeAll, describe, expect, it } from "vitest";
import {
authUsers,
companies,
companyMemberships,
createDb,
instanceUserRoles,
principalPermissionGrants,
} from "@paperclipai/db";
import {
claimBoardOwnership,
getBoardClaimWarningUrl,
initializeBoardClaimChallenge,
inspectBoardClaimChallenge,
} from "../board-claim.js";
import {
getEmbeddedPostgresTestSupport,
startEmbeddedPostgresTestDatabase,
} from "./helpers/embedded-postgres.js";
const embeddedPostgresSupport = await getEmbeddedPostgresTestSupport();
const describeEmbeddedPostgres = embeddedPostgresSupport.supported ? describe : describe.skip;
describeEmbeddedPostgres("board claim", () => {
let db!: ReturnType<typeof createDb>;
let tempDb: Awaited<ReturnType<typeof startEmbeddedPostgresTestDatabase>> | null = null;
beforeAll(async () => {
tempDb = await startEmbeddedPostgresTestDatabase("paperclip-board-claim-");
db = createDb(tempDb.connectionString);
}, 20_000);
afterEach(async () => {
await initializeBoardClaimChallenge(db, { deploymentMode: "local_trusted" });
await db.delete(principalPermissionGrants);
await db.delete(companyMemberships);
await db.delete(companies);
await db.delete(instanceUserRoles);
await db.delete(authUsers);
});
afterAll(async () => {
await tempDb?.cleanup();
});
it("lets a signed-in user claim a local-board-only authenticated instance", async () => {
const now = new Date();
const userId = `claim-user-${randomUUID()}`;
const company = await db
.insert(companies)
.values({
name: "Board Claim Co",
issuePrefix: `BC${randomUUID().slice(0, 6).toUpperCase()}`,
})
.returning()
.then((rows) => rows[0]!);
await db.insert(authUsers).values({
id: userId,
name: "Board Claim User",
email: "board-claim@example.test",
emailVerified: true,
createdAt: now,
updatedAt: now,
});
await db.insert(instanceUserRoles).values({
userId: "local-board",
role: "instance_admin",
});
await initializeBoardClaimChallenge(db, { deploymentMode: "authenticated" });
const warningUrl = getBoardClaimWarningUrl("127.0.0.1", 3197);
expect(warningUrl).toBeTruthy();
const parsed = new URL(warningUrl!);
const token = parsed.pathname.split("/").pop()!;
const code = parsed.searchParams.get("code")!;
expect(inspectBoardClaimChallenge(token, code)).toMatchObject({
status: "available",
requiresSignIn: true,
claimedByUserId: null,
});
await expect(
claimBoardOwnership(db, { token, code, userId }),
).resolves.toEqual({
status: "claimed",
claimedByUserId: userId,
});
await expect(
db
.select()
.from(instanceUserRoles)
.where(and(eq(instanceUserRoles.userId, "local-board"), eq(instanceUserRoles.role, "instance_admin"))),
).resolves.toHaveLength(0);
await expect(
db
.select()
.from(instanceUserRoles)
.where(and(eq(instanceUserRoles.userId, userId), eq(instanceUserRoles.role, "instance_admin"))),
).resolves.toHaveLength(1);
await expect(
db
.select()
.from(companyMemberships)
.where(
and(
eq(companyMemberships.companyId, company.id),
eq(companyMemberships.principalType, "user"),
eq(companyMemberships.principalId, userId),
),
),
).resolves.toMatchObject([
{
status: "active",
membershipRole: "owner",
},
]);
expect(inspectBoardClaimChallenge(token, code)).toMatchObject({
status: "claimed",
claimedByUserId: userId,
});
});
});
@@ -21,6 +21,9 @@ const mockBoardAuthService = vi.hoisted(() => ({
resolveBoardActivityCompanyIds: vi.fn(),
assertCurrentBoardKey: vi.fn(),
revokeBoardApiKey: vi.fn(),
listBoardApiKeys: vi.fn(),
createNamedBoardApiKey: vi.fn(),
getBoardApiKeyForUser: vi.fn(),
}));
const mockLogActivity = vi.hoisted(() => vi.fn());
@@ -302,4 +305,121 @@ describe.sequential("cli auth routes", () => {
}),
);
});
it.sequential("creates a named board API key and logs audit activity", async () => {
mockBoardAuthService.createNamedBoardApiKey.mockResolvedValue({
id: "board-key-4",
name: "external-admin",
token: "pcp_board_plaintext",
createdAt: new Date("2026-05-23T12:00:00.000Z"),
lastUsedAt: null,
revokedAt: null,
expiresAt: new Date("2026-06-23T12:00:00.000Z"),
});
mockBoardAuthService.resolveBoardActivityCompanyIds.mockResolvedValue(["11111111-1111-4111-8111-111111111111"]);
const app = await createApp({
type: "board",
userId: "user-1",
source: "board_key",
isInstanceAdmin: false,
companyIds: ["11111111-1111-4111-8111-111111111111"],
});
const res = await request(app)
.post("/api/board-api-keys")
.send({
name: "external-admin",
requestedCompanyId: "11111111-1111-4111-8111-111111111111",
expiresAt: "2026-06-23T12:00:00.000Z",
});
expect(res.status, res.text || JSON.stringify(res.body)).toBe(201);
expect(res.body).toMatchObject({
id: "board-key-4",
name: "external-admin",
token: "pcp_board_plaintext",
expiresAt: "2026-06-23T12:00:00.000Z",
});
expect(mockBoardAuthService.createNamedBoardApiKey).toHaveBeenCalledWith({
userId: "user-1",
name: "external-admin",
expiresAt: new Date("2026-06-23T12:00:00.000Z"),
});
expect(mockLogActivity).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
companyId: "11111111-1111-4111-8111-111111111111",
action: "board_api_key.created",
details: expect.objectContaining({ name: "external-admin" }),
}),
);
});
it.sequential("lists and revokes named board API keys for the current board user", async () => {
const keyId = "55555555-5555-4555-8555-555555555555";
mockBoardAuthService.listBoardApiKeys.mockResolvedValue([
{
id: keyId,
name: "external-admin",
createdAt: new Date("2026-05-23T12:00:00.000Z"),
lastUsedAt: null,
revokedAt: null,
expiresAt: null,
},
]);
mockBoardAuthService.getBoardApiKeyForUser.mockResolvedValue({
id: keyId,
userId: "user-1",
name: "external-admin",
});
mockBoardAuthService.revokeBoardApiKey.mockResolvedValue({
id: keyId,
userId: "user-1",
name: "external-admin",
});
mockBoardAuthService.resolveBoardActivityCompanyIds.mockResolvedValue(["company-1"]);
const app = await createApp({
type: "board",
userId: "user-1",
source: "board_key",
isInstanceAdmin: false,
companyIds: ["company-1"],
});
const listRes = await request(app).get("/api/board-api-keys");
expect(listRes.status).toBe(200);
expect(listRes.body[0]).toMatchObject({ id: keyId, name: "external-admin" });
expect(mockBoardAuthService.listBoardApiKeys).toHaveBeenCalledWith(
"user-1",
{ includeInactive: false },
);
const revokeRes = await request(app).delete(`/api/board-api-keys/${keyId}`);
expect(revokeRes.status).toBe(200);
expect(revokeRes.body).toEqual({ ok: true, keyId });
expect(mockLogActivity).toHaveBeenCalledWith(
expect.anything(),
expect.objectContaining({
companyId: "company-1",
action: "board_api_key.revoked",
}),
);
});
it.sequential("rejects malformed board API key IDs before database lookup", async () => {
const app = await createApp({
type: "board",
userId: "user-1",
source: "board_key",
isInstanceAdmin: false,
companyIds: ["company-1"],
});
const res = await request(app).delete("/api/board-api-keys/not-a-uuid");
expect(res.status).toBe(400);
expect(mockBoardAuthService.getBoardApiKeyForUser).not.toHaveBeenCalled();
expect(mockBoardAuthService.revokeBoardApiKey).not.toHaveBeenCalled();
});
});
@@ -146,6 +146,7 @@ describe("environment routes", () => {
mockIssueService.getById.mockReset();
mockProjectService.getById.mockReset();
mockEnvironmentService.list.mockReset();
mockEnvironmentService.list.mockResolvedValue([]);
mockEnvironmentService.getById.mockReset();
mockEnvironmentService.create.mockReset();
mockEnvironmentService.update.mockReset();
@@ -420,6 +421,27 @@ describe("environment routes", () => {
);
});
it("returns conflict when creating a second local environment", async () => {
mockEnvironmentService.list.mockResolvedValue([createEnvironment()]);
const app = createApp({
type: "board",
userId: "user-1",
source: "local_implicit",
});
const res = await request(app)
.post("/api/companies/company-1/environments")
.send({
name: "Another Local",
driver: "local",
config: {},
});
expect(res.status).toBe(409);
expect(res.body.error).toBe("A local environment already exists for this company.");
expect(mockEnvironmentService.create).not.toHaveBeenCalled();
});
it("allows non-admin board users with environments:manage to create environments", async () => {
const environment = createEnvironment();
mockAccessService.canUser.mockResolvedValue(true);
@@ -100,6 +100,27 @@ describe("issue tree control routes", () => {
expect(mockTreeControlService.createHold).not.toHaveBeenCalled();
});
it("rejects malformed tree hold IDs before querying the hold service", async () => {
const app = await createApp({
type: "board",
userId: "user-1",
companyIds: ["company-2"],
source: "session",
isInstanceAdmin: false,
});
const getRes = await request(app)
.get("/api/issues/11111111-1111-4111-8111-111111111111/tree-holds/null");
const releaseRes = await request(app)
.post("/api/issues/11111111-1111-4111-8111-111111111111/tree-holds/null/release")
.send({ reason: "bad hold id" });
expect(getRes.status).toBe(400);
expect(releaseRes.status).toBe(400);
expect(mockTreeControlService.getHold).not.toHaveBeenCalled();
expect(mockTreeControlService.releaseHold).not.toHaveBeenCalled();
});
it("cancels active descendant runs when creating a pause hold", async () => {
const app = await createApp({
type: "board",
@@ -0,0 +1,45 @@
import express from "express";
import request from "supertest";
import { describe, expect, it } from "vitest";
import { errorHandler } from "../middleware/index.js";
import { openApiRoutes } from "../routes/openapi.js";
function createApp() {
const app = express();
app.use("/api", openApiRoutes());
app.use(errorHandler);
return app;
}
describe("openapi routes", () => {
it("serves the generated OpenAPI document", async () => {
const res = await request(createApp()).get("/api/openapi.json");
expect(res.status).toBe(200);
expect(res.body.openapi).toBe("3.0.0");
expect(res.body.info.title).toBe("Paperclip API");
expect(res.body.paths["/api/openapi.json"].get.summary).toBe("Get the generated OpenAPI document");
expect(res.body.paths["/api/companies/{companyId}/agents"].get.summary).toBe("List agents in a company");
expect(res.body.paths["/api/agents/{id}/keys"].post.summary).toBe("Create an agent API key");
expect(res.body.components.securitySchemes).toMatchObject({
BoardSessionAuth: { type: "apiKey", in: "cookie" },
BoardApiKeyAuth: { type: "http", scheme: "bearer" },
AgentBearerAuth: { type: "http", scheme: "bearer" },
});
expect(res.body.paths["/api/health"].get.security).toEqual([]);
expect(res.body.paths["/api/companies"].post.responses["201"]).toBeDefined();
expect(res.body.paths["/api/companies"].post.requestBody.content["application/json"].schema).toMatchObject({
type: "object",
properties: {
name: { type: "string", minLength: 1 },
},
required: ["name"],
});
expect(res.body.paths["/api/agents/{id}/keys"].post.requestBody.content["application/json"].schema).toMatchObject({
type: "object",
properties: {
name: { type: "string" },
},
});
});
});
+20 -3
View File
@@ -520,10 +520,19 @@ describeEmbeddedPostgres("plugin database namespaces", () => {
const staleManifest = manifest("paperclip.refresh");
const refreshedManifest: PaperclipPluginManifestV1 = {
...staleManifest,
capabilities: [...staleManifest.capabilities, "agent.tools.register"],
database: {
...staleManifest.database!,
coreReadTables: ["companies"],
},
tools: [
{
name: "db-smoke",
displayName: "DB Smoke",
description: "Exercises plugin tool registration worker lookup.",
parametersSchema: { type: "object", properties: {} },
},
],
};
const namespace = derivePluginDatabaseNamespace(refreshedManifest.id);
const packageRoot = await createInstallablePluginPackage(
@@ -548,6 +557,9 @@ describeEmbeddedPostgres("plugin database namespaces", () => {
startWorker: vi.fn().mockResolvedValue(undefined),
stopAll: vi.fn().mockResolvedValue(undefined),
};
const toolDispatcher = {
registerPluginTools: vi.fn(),
};
const loader = pluginLoader(db, {
enableLocalFilesystem: false,
enableNpmDiscovery: false,
@@ -564,9 +576,7 @@ describeEmbeddedPostgres("plugin database namespaces", () => {
jobStore: {
syncJobDeclarations: vi.fn().mockResolvedValue(undefined),
},
toolDispatcher: {
registerPluginTools: vi.fn(),
},
toolDispatcher,
lifecycleManager: {
markError: vi.fn().mockResolvedValue(undefined),
},
@@ -595,6 +605,13 @@ describeEmbeddedPostgres("plugin database namespaces", () => {
}),
}),
);
expect(toolDispatcher.registerPluginTools).toHaveBeenCalledWith(
refreshedManifest.id,
expect.objectContaining({
tools: refreshedManifest.tools,
}),
pluginId,
);
const [plugin] = await db
.select()
.from(plugins)
@@ -477,6 +477,8 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
replayWindowSec: 300,
}, {});
await svc.deleteTrigger(created.trigger.id, {});
await expect(db.select().from(companySecrets).where(eq(companySecrets.id, created.trigger.secretId!))).resolves.toHaveLength(0);
await expect(db.select().from(companySecretBindings).where(eq(companySecretBindings.secretId, created.trigger.secretId!))).resolves.toHaveLength(0);
const restored = await svc.restoreRevision(routine.id, created.revision.id, {});
@@ -563,6 +565,8 @@ describeEmbeddedPostgres("routine service live-execution coalescing", () => {
const deleted = await svc.deleteTrigger(created.trigger.id, {});
expect(deleted.revision?.revisionNumber).toBe(5);
await expect(db.select().from(companySecrets).where(eq(companySecrets.id, created.trigger.secretId!))).resolves.toHaveLength(0);
await expect(db.select().from(companySecretBindings).where(eq(companySecretBindings.secretId, created.trigger.secretId!))).resolves.toHaveLength(0);
const revisions = await svc.listRevisions(routine.id);
const serialized = JSON.stringify(revisions.map((revision) => revision.snapshot));