Expand plugin host surface (#5205)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - The plugin system is the extension boundary for optional product capabilities > - Rich plugins need more than a worker entrypoint: they need scoped database storage, local project folders, managed agents/routines, host navigation, and reusable UI components > - The LLM Wiki work exposed those missing host surfaces while keeping plugin code outside the core control plane > - This pull request expands the core plugin host, SDK, server APIs, and UI bridge so plugins can declare and use those surfaces > - The benefit is that future plugins can integrate with Paperclip through documented, validated contracts instead of bespoke server or UI imports ## What Changed - Added plugin-managed database namespaces and migration tracking, including Drizzle schema/migration files and SQL validation for namespace isolation. - Added server support for plugin local folders, managed agents, managed routines, scoped plugin APIs, and plugin operation visibility. - Expanded shared plugin manifest/types/validators and SDK host/testing/UI exports for richer plugin surfaces. - Added reusable UI pieces for file trees, managed routines, resizable sidebars, route sidebars, and plugin bridge initialization. - Updated plugin docs and example plugins to use the expanded host and SDK surface. ## Verification - `pnpm install --frozen-lockfile` - `pnpm run preflight:workspace-links && pnpm exec vitest run packages/shared/src/validators/plugin.test.ts server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-local-folders.test.ts server/src/__tests__/plugin-managed-agents.test.ts server/src/__tests__/plugin-managed-routines.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts ui/src/api/plugins.test.ts ui/src/components/FileTree.test.tsx ui/src/components/ResizableSidebarPane.test.tsx ui/src/pages/PluginPage.test.tsx ui/src/plugins/bridge.test.ts` passed: 11 files, 67 tests. - Confirmed this PR changes 89 files and does not include `pnpm-lock.yaml` or `.github/workflows/*`. ## Risks - Medium: this expands plugin host contracts across db/shared/server/ui and includes a new core migration (`0076_useful_elektra.sql`). - The plugin database namespace validator is intentionally restrictive; plugin authors may need follow-up affordances for SQL patterns that remain blocked. - Merge this before the LLM Wiki plugin PR so the plugin can resolve the new SDK and host APIs. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled shell/git/GitHub workflow. Context window size was not exposed by the runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Dotta
+268
-46
@@ -1,6 +1,14 @@
|
||||
import { and, asc, desc, eq, inArray } from "drizzle-orm";
|
||||
import type { Db } from "@paperclipai/db";
|
||||
import { projects, projectGoals, goals, projectWorkspaces, workspaceRuntimeServices } from "@paperclipai/db";
|
||||
import {
|
||||
projects,
|
||||
projectGoals,
|
||||
goals,
|
||||
pluginManagedResources,
|
||||
plugins,
|
||||
projectWorkspaces,
|
||||
workspaceRuntimeServices,
|
||||
} from "@paperclipai/db";
|
||||
import {
|
||||
PROJECT_COLORS,
|
||||
deriveProjectUrlKey,
|
||||
@@ -10,9 +18,12 @@ import {
|
||||
type ProjectCodebase,
|
||||
type ProjectExecutionWorkspacePolicy,
|
||||
type ProjectGoalRef,
|
||||
type ProjectManagedByPlugin,
|
||||
type ProjectWorkspaceRuntimeConfig,
|
||||
type ProjectWorkspace,
|
||||
type WorkspaceRuntimeService,
|
||||
type PluginManagedProjectDeclaration,
|
||||
type PluginManagedProjectResolution,
|
||||
} from "@paperclipai/shared";
|
||||
import { listCurrentRuntimeServicesForProjectWorkspaces } from "./workspace-runtime-read-model.js";
|
||||
import { parseProjectExecutionWorkspacePolicy } from "./execution-workspace-policy.js";
|
||||
@@ -50,6 +61,7 @@ interface ProjectWithGoals extends Omit<ProjectRow, "executionWorkspacePolicy">
|
||||
codebase: ProjectCodebase;
|
||||
workspaces: ProjectWorkspace[];
|
||||
primaryWorkspace: ProjectWorkspace | null;
|
||||
managedByPlugin: ProjectManagedByPlugin | null;
|
||||
}
|
||||
|
||||
interface ProjectShortnameRow {
|
||||
@@ -245,6 +257,40 @@ async function attachWorkspaces(db: Db, rows: ProjectWithGoals[]): Promise<Proje
|
||||
arr.push(row);
|
||||
}
|
||||
|
||||
const managedRows = await db
|
||||
.select({
|
||||
id: pluginManagedResources.id,
|
||||
pluginId: pluginManagedResources.pluginId,
|
||||
pluginKey: pluginManagedResources.pluginKey,
|
||||
manifestJson: plugins.manifestJson,
|
||||
resourceKind: pluginManagedResources.resourceKind,
|
||||
resourceKey: pluginManagedResources.resourceKey,
|
||||
resourceId: pluginManagedResources.resourceId,
|
||||
defaultsJson: pluginManagedResources.defaultsJson,
|
||||
createdAt: pluginManagedResources.createdAt,
|
||||
updatedAt: pluginManagedResources.updatedAt,
|
||||
})
|
||||
.from(pluginManagedResources)
|
||||
.innerJoin(plugins, eq(pluginManagedResources.pluginId, plugins.id))
|
||||
.where(and(
|
||||
eq(pluginManagedResources.resourceKind, "project"),
|
||||
inArray(pluginManagedResources.resourceId, projectIds),
|
||||
));
|
||||
const managedByProjectId = new Map<string, ProjectManagedByPlugin>();
|
||||
for (const row of managedRows) {
|
||||
managedByProjectId.set(row.resourceId, {
|
||||
id: row.id,
|
||||
pluginId: row.pluginId,
|
||||
pluginKey: row.pluginKey,
|
||||
pluginDisplayName: row.manifestJson.displayName ?? row.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: row.resourceKey,
|
||||
defaultsJson: row.defaultsJson,
|
||||
createdAt: row.createdAt,
|
||||
updatedAt: row.updatedAt,
|
||||
});
|
||||
}
|
||||
|
||||
return rows.map((row) => {
|
||||
const projectWorkspaceRows = map.get(row.id) ?? [];
|
||||
const workspaces = projectWorkspaceRows.map((workspace) =>
|
||||
@@ -264,6 +310,7 @@ async function attachWorkspaces(db: Db, rows: ProjectWithGoals[]): Promise<Proje
|
||||
}),
|
||||
workspaces,
|
||||
primaryWorkspace,
|
||||
managedByPlugin: managedByProjectId.get(row.id) ?? null,
|
||||
};
|
||||
});
|
||||
}
|
||||
@@ -337,6 +384,17 @@ function deriveWorkspaceName(input: {
|
||||
return "Workspace";
|
||||
}
|
||||
|
||||
function buildManagedProjectDefaults(declaration: PluginManagedProjectDeclaration) {
|
||||
return {
|
||||
projectKey: declaration.projectKey,
|
||||
displayName: declaration.displayName,
|
||||
description: declaration.description ?? null,
|
||||
status: declaration.status ?? "in_progress",
|
||||
color: declaration.color ?? null,
|
||||
settings: declaration.settings ?? {},
|
||||
};
|
||||
}
|
||||
|
||||
export function resolveProjectNameForUniqueShortname(
|
||||
requestedName: string,
|
||||
existingProjects: ProjectShortnameRow[],
|
||||
@@ -398,6 +456,58 @@ async function ensureSinglePrimaryWorkspace(
|
||||
}
|
||||
|
||||
export function projectService(db: Db) {
|
||||
const createProject = async (
|
||||
companyId: string,
|
||||
data: Omit<typeof projects.$inferInsert, "companyId"> & { goalIds?: string[] },
|
||||
): Promise<ProjectWithGoals> => {
|
||||
const { goalIds: inputGoalIds, ...projectData } = data;
|
||||
const ids = resolveGoalIds({ goalIds: inputGoalIds, goalId: projectData.goalId });
|
||||
|
||||
// Auto-assign a color from the palette if none provided
|
||||
if (!projectData.color) {
|
||||
const existing = await db.select({ color: projects.color }).from(projects).where(eq(projects.companyId, companyId));
|
||||
const usedColors = new Set(existing.map((r) => r.color).filter(Boolean));
|
||||
const nextColor = PROJECT_COLORS.find((c) => !usedColors.has(c)) ?? PROJECT_COLORS[existing.length % PROJECT_COLORS.length];
|
||||
projectData.color = nextColor;
|
||||
}
|
||||
|
||||
const existingProjects = await db
|
||||
.select({ id: projects.id, name: projects.name })
|
||||
.from(projects)
|
||||
.where(eq(projects.companyId, companyId));
|
||||
projectData.name = resolveProjectNameForUniqueShortname(projectData.name, existingProjects);
|
||||
|
||||
// Also write goalId to the legacy column (first goal or null)
|
||||
const legacyGoalId = ids && ids.length > 0 ? ids[0] : projectData.goalId ?? null;
|
||||
|
||||
const row = await db
|
||||
.insert(projects)
|
||||
.values({ ...projectData, goalId: legacyGoalId, companyId })
|
||||
.returning()
|
||||
.then((rows) => rows[0]);
|
||||
|
||||
if (ids && ids.length > 0) {
|
||||
await syncGoalLinks(db, row.id, companyId, ids);
|
||||
}
|
||||
|
||||
const [withGoals] = await attachGoals(db, [row]);
|
||||
const [enriched] = withGoals ? await attachWorkspaces(db, [withGoals]) : [];
|
||||
return enriched!;
|
||||
};
|
||||
|
||||
const getProjectById = async (id: string): Promise<ProjectWithGoals | null> => {
|
||||
const row = await db
|
||||
.select()
|
||||
.from(projects)
|
||||
.where(eq(projects.id, id))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!row) return null;
|
||||
const [withGoals] = await attachGoals(db, [row]);
|
||||
if (!withGoals) return null;
|
||||
const [enriched] = await attachWorkspaces(db, [withGoals]);
|
||||
return enriched ?? null;
|
||||
};
|
||||
|
||||
return {
|
||||
list: async (companyId: string): Promise<ProjectWithGoals[]> => {
|
||||
const rows = await db.select().from(projects).where(eq(projects.companyId, companyId));
|
||||
@@ -418,58 +528,170 @@ export function projectService(db: Db) {
|
||||
return dedupedIds.map((id) => byId.get(id)).filter((project): project is ProjectWithGoals => Boolean(project));
|
||||
},
|
||||
|
||||
getById: async (id: string): Promise<ProjectWithGoals | null> => {
|
||||
const row = await db
|
||||
.select()
|
||||
.from(projects)
|
||||
.where(eq(projects.id, id))
|
||||
getById: getProjectById,
|
||||
|
||||
resolveManagedProject: async (input: {
|
||||
companyId: string;
|
||||
pluginId: string;
|
||||
pluginKey: string;
|
||||
projectKey: string;
|
||||
reset?: boolean;
|
||||
createIfMissing?: boolean;
|
||||
}): Promise<PluginManagedProjectResolution> => {
|
||||
const plugin = await db
|
||||
.select({ id: plugins.id, pluginKey: plugins.pluginKey, manifestJson: plugins.manifestJson })
|
||||
.from(plugins)
|
||||
.where(eq(plugins.id, input.pluginId))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (!row) return null;
|
||||
const [withGoals] = await attachGoals(db, [row]);
|
||||
if (!withGoals) return null;
|
||||
const [enriched] = await attachWorkspaces(db, [withGoals]);
|
||||
return enriched ?? null;
|
||||
},
|
||||
|
||||
create: async (
|
||||
companyId: string,
|
||||
data: Omit<typeof projects.$inferInsert, "companyId"> & { goalIds?: string[] },
|
||||
): Promise<ProjectWithGoals> => {
|
||||
const { goalIds: inputGoalIds, ...projectData } = data;
|
||||
const ids = resolveGoalIds({ goalIds: inputGoalIds, goalId: projectData.goalId });
|
||||
|
||||
// Auto-assign a color from the palette if none provided
|
||||
if (!projectData.color) {
|
||||
const existing = await db.select({ color: projects.color }).from(projects).where(eq(projects.companyId, companyId));
|
||||
const usedColors = new Set(existing.map((r) => r.color).filter(Boolean));
|
||||
const nextColor = PROJECT_COLORS.find((c) => !usedColors.has(c)) ?? PROJECT_COLORS[existing.length % PROJECT_COLORS.length];
|
||||
projectData.color = nextColor;
|
||||
if (!plugin || plugin.pluginKey !== input.pluginKey) {
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: null,
|
||||
project: null,
|
||||
status: "missing",
|
||||
};
|
||||
}
|
||||
|
||||
const existingProjects = await db
|
||||
.select({ id: projects.id, name: projects.name })
|
||||
.from(projects)
|
||||
.where(eq(projects.companyId, companyId));
|
||||
projectData.name = resolveProjectNameForUniqueShortname(projectData.name, existingProjects);
|
||||
|
||||
// Also write goalId to the legacy column (first goal or null)
|
||||
const legacyGoalId = ids && ids.length > 0 ? ids[0] : projectData.goalId ?? null;
|
||||
|
||||
const row = await db
|
||||
.insert(projects)
|
||||
.values({ ...projectData, goalId: legacyGoalId, companyId })
|
||||
.returning()
|
||||
.then((rows) => rows[0]);
|
||||
|
||||
if (ids && ids.length > 0) {
|
||||
await syncGoalLinks(db, row.id, companyId, ids);
|
||||
const declaration = plugin.manifestJson.projects?.find((project) => project.projectKey === input.projectKey);
|
||||
if (!declaration) {
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: null,
|
||||
project: null,
|
||||
status: "missing",
|
||||
};
|
||||
}
|
||||
|
||||
const [withGoals] = await attachGoals(db, [row]);
|
||||
const [enriched] = withGoals ? await attachWorkspaces(db, [withGoals]) : [];
|
||||
return enriched!;
|
||||
const defaults = buildManagedProjectDefaults(declaration);
|
||||
const existingBinding = await db
|
||||
.select()
|
||||
.from(pluginManagedResources)
|
||||
.where(and(
|
||||
eq(pluginManagedResources.companyId, input.companyId),
|
||||
eq(pluginManagedResources.pluginId, input.pluginId),
|
||||
eq(pluginManagedResources.resourceKind, "project"),
|
||||
eq(pluginManagedResources.resourceKey, input.projectKey),
|
||||
))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
|
||||
if (existingBinding) {
|
||||
const existingProject = await db
|
||||
.select({ id: projects.id })
|
||||
.from(projects)
|
||||
.where(and(eq(projects.companyId, input.companyId), eq(projects.id, existingBinding.resourceId)))
|
||||
.then((rows) => rows[0] ?? null);
|
||||
if (existingProject) {
|
||||
if (input.reset) {
|
||||
await db
|
||||
.update(projects)
|
||||
.set({
|
||||
name: declaration.displayName,
|
||||
description: declaration.description ?? null,
|
||||
status: declaration.status ?? "in_progress",
|
||||
color: declaration.color ?? null,
|
||||
updatedAt: new Date(),
|
||||
})
|
||||
.where(and(eq(projects.companyId, input.companyId), eq(projects.id, existingBinding.resourceId)));
|
||||
}
|
||||
if (input.createIfMissing !== false) {
|
||||
await db
|
||||
.update(pluginManagedResources)
|
||||
.set({ defaultsJson: defaults, updatedAt: new Date() })
|
||||
.where(eq(pluginManagedResources.id, existingBinding.id));
|
||||
}
|
||||
const project = await getProjectById(existingBinding.resourceId);
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: project?.id ?? existingBinding.resourceId,
|
||||
project: project as import("@paperclipai/shared").Project | null,
|
||||
status: input.reset ? "reset" : "resolved",
|
||||
};
|
||||
}
|
||||
|
||||
if (input.createIfMissing === false) {
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: null,
|
||||
project: null,
|
||||
status: "missing",
|
||||
};
|
||||
}
|
||||
|
||||
const project = await createProject(input.companyId, {
|
||||
name: declaration.displayName,
|
||||
description: declaration.description ?? null,
|
||||
status: declaration.status ?? "in_progress",
|
||||
color: declaration.color ?? undefined,
|
||||
});
|
||||
await db
|
||||
.update(pluginManagedResources)
|
||||
.set({ resourceId: project.id, defaultsJson: defaults, updatedAt: new Date() })
|
||||
.where(eq(pluginManagedResources.id, existingBinding.id));
|
||||
const hydrated = await getProjectById(project.id);
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: hydrated?.id ?? project.id,
|
||||
project: hydrated as import("@paperclipai/shared").Project | null,
|
||||
status: "relinked",
|
||||
};
|
||||
}
|
||||
|
||||
if (input.createIfMissing === false) {
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: null,
|
||||
project: null,
|
||||
status: "missing",
|
||||
};
|
||||
}
|
||||
|
||||
const project = await createProject(input.companyId, {
|
||||
name: declaration.displayName,
|
||||
description: declaration.description ?? null,
|
||||
status: declaration.status ?? "in_progress",
|
||||
color: declaration.color ?? undefined,
|
||||
});
|
||||
await db.insert(pluginManagedResources).values({
|
||||
companyId: input.companyId,
|
||||
pluginId: input.pluginId,
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
resourceId: project.id,
|
||||
defaultsJson: defaults,
|
||||
});
|
||||
const hydrated = await getProjectById(project.id);
|
||||
return {
|
||||
pluginKey: input.pluginKey,
|
||||
resourceKind: "project",
|
||||
resourceKey: input.projectKey,
|
||||
companyId: input.companyId,
|
||||
projectId: hydrated?.id ?? project.id,
|
||||
project: hydrated as import("@paperclipai/shared").Project | null,
|
||||
status: "created",
|
||||
};
|
||||
},
|
||||
|
||||
create: createProject,
|
||||
|
||||
update: async (
|
||||
id: string,
|
||||
data: Partial<typeof projects.$inferInsert> & { goalIds?: string[] },
|
||||
|
||||
Reference in New Issue
Block a user