Auto-build bundled plugins on install (#8254)

## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Plugins extend the server with worker/UI surfaces, and bundled local plugins under `packages/plugins/**` ship as TS source — their compiled `dist/` is not checked in > - On a fresh checkout, installing a bundled local plugin via the in-app **Install** button failed because `paperclipPlugin.manifest` points at `./dist/manifest.js`, which does not exist until the package is built > - The error surfaces as `Package … does not appear to be a Paperclip plugin (no manifest found)`, which is misleading — the manifest is real, the dist is just missing — and forces every contributor to run `pnpm --filter … build` by hand before the bundled-plugin installer works at all > - This pull request teaches the install path to detect that case and run the package's build (plus standalone runtime bootstrap for plugins outside the root workspace) before manifest resolution, gated by a kill switch and a bounded timeout > - The benefit is bundled plugins like `@paperclipai/plugin-workspace-diff` install in one click on a fresh checkout, with a clear error message and manual fallback when the autobuild itself fails ## Linked Issues or Issue Description No existing GitHub issue. Underlying bug, following the bug-report template: **What happened?** Installing a bundled local plugin from a fresh checkout fails with `Package @paperclipai/plugin-workspace-diff at packages/plugins/plugin-workspace-diff does not appear to be a Paperclip plugin (no manifest found)`. The manifest is declared in `package.json` (`paperclipPlugin.manifest = ./dist/manifest.js`) but `dist/` is not built/committed, so the loader cannot find it. **Expected behavior** Clicking **Install** on a bundled plugin builds it if needed and registers it, without a manual build step. **Steps to reproduce** 1. Fresh checkout of `master` 2. Start the server, open Plugin Manager 3. Click **Install** next to `@paperclipai/plugin-workspace-diff` 4. Observe the "no manifest found" failure **Scope** Same failure mode affects every bundled plugin without a checked-in `dist/` (`plugin-llm-wiki`, examples, sandbox-provider plugins, etc.). ## What Changed - `server/src/services/plugin-loader.ts`: added `ensureLocalPluginBuilt(packageRoot, pkgJson)` — when the package lives under `packages/plugins/**` and its declared paperclipPlugin entrypoints (`manifest`, `worker`, `ui`) are missing, run `pnpm --filter <name> build` (and a standalone runtime-deps bootstrap for plugins outside the root pnpm workspace) before manifest resolution - `server/src/routes/plugins.ts`: invoke the autobuild from the local-path install path; surface a `hasBuiltEntrypoints` boolean on the `AvailableBundledPlugin` listing; invalidate the bundled-plugins cache after a successful install so a freshly built plugin no longer reports `hasBuiltEntrypoints: false` - `ui/src/api/plugins.ts` + `ui/src/pages/PluginManager.tsx`: type and consume `hasBuiltEntrypoints` so the installer can show that an autobuild will run on install - `server/src/__tests__/plugin-install-autobuild.test.ts`: new suite — 9 tests covering success, kill-switch, build failure, timeout, manifest still missing after build, standalone variant, and the existing `plugin-routes-authz` listing assertion - `doc/plugins/LOCAL_PLUGIN_DEVELOPMENT.md`: documents the autobuild, the `PAPERCLIP_DISABLE_PLUGIN_AUTOBUILD=1` kill switch, and the manual fallback command - Detect the autobuild timeout via the child-process `killed` flag rather than string-matching the error message, so the "after timing out" context is actually emitted Knobs: - `PAPERCLIP_DISABLE_PLUGIN_AUTOBUILD=1` — skip autobuild entirely; restore prior behavior - Build timeout: 120s, with a clear error that points at the manual `pnpm --filter <name> build` recovery command ## Verification - `cd server && pnpm vitest run src/__tests__/plugin-install-autobuild.test.ts src/__tests__/plugin-routes-authz.test.ts` → 44/44 pass - End-to-end on a clean checkout: `rm -rf packages/plugins/plugin-workspace-diff/dist`, invoke `ensureLocalPluginBuilt()` against the real package, all declared entrypoints (`dist/manifest.js`, `dist/worker.js`, `dist/ui/index.js`) regenerated. The original `no manifest found` symptom no longer reproduces. ## Risks Low. The autobuild only fires when (a) the package sits under `packages/plugins/**`, (b) at least one declared entrypoint is missing, and (c) the kill switch is not set. In a packaged production server the `packages/plugins/**` path does not exist on disk, so the helper short-circuits and never shells out to `pnpm`. Failures from the spawned build are surfaced as an install error with the exact manual command to retry, so the worst-case is the same UX as before plus a clearer message. ## Model Used Claude Opus 4.7 (claude-opus-4-7), extended thinking enabled, tool use (filesystem + bash). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-17 22:39:55 -07:00
parent 61eb952e94
commit d47b4da655
7 changed files with 747 additions and 20 deletions
@@ -52,6 +52,11 @@ import { pluginDatabaseService } from "./plugin-database.js";

 const execFileAsync = promisify(execFile);
 const __dirname = path.dirname(fileURLToPath(import.meta.url));
+export const REPO_ROOT = path.resolve(__dirname, "../../..");
+export const BUNDLED_LOCAL_PLUGIN_ROOT = path.join(REPO_ROOT, "packages", "plugins");
+export const STANDALONE_BUNDLED_PLUGIN_ROOT = path.join(BUNDLED_LOCAL_PLUGIN_ROOT, "sandbox-providers");
+export const LOCAL_PLUGIN_AUTOBUILD_TIMEOUT_MS = 120_000;
+const STANDALONE_BUNDLED_PLUGIN_SDK_PACKAGE = "@paperclipai/plugin-sdk";

 // ---------------------------------------------------------------------------
 // Constants
@@ -184,6 +189,19 @@ export interface PluginDiscoveryResult {
  sources: PluginSource[];
 }

+type PluginEntrypointKey = "manifest" | "worker" | "ui";
+
+type PluginEntrypointPath = {
+  key: PluginEntrypointKey;
+  absolutePath: string;
+};
+
+type LocalPluginBuildCommand = {
+  file: string;
+  args: string[];
+  cwd: string;
+};
+
 function getDeclaredPageRoutePaths(manifest: PaperclipPluginManifestV1): string[] {
  return (manifest.ui?.slots ?? [])
    .filter((slot): slot is PluginUiSlotDeclaration => slot.type === "page" && typeof slot.routePath === "string" && slot.routePath.length > 0)
@@ -595,6 +613,263 @@ async function readPackageJson(
  }
 }

+function buildLocalPluginBuildCommand(pkgJson: Record<string, unknown>): string | null {
+  const packageName = pkgJson["name"];
+  if (typeof packageName !== "string" || packageName.trim().length === 0) return null;
+  return `pnpm --filter ${packageName} build`;
+}
+
+function readPackageDependencyNames(
+  pkgJson: Record<string, unknown>,
+  field: "dependencies" | "optionalDependencies",
+): string[] {
+  const deps = pkgJson[field];
+  if (deps === null || typeof deps !== "object" || Array.isArray(deps)) return [];
+  return Object.keys(deps as Record<string, unknown>);
+}
+
+function resolvePackageInstallPath(packageRoot: string, packageName: string): string {
+  return path.join(packageRoot, "node_modules", ...packageName.split("/"));
+}
+
+function isPathWithin(root: string, target: string): boolean {
+  const relativePath = path.relative(root, target);
+  return relativePath === "" || (!relativePath.startsWith("..") && !path.isAbsolute(relativePath));
+}
+
+export function isRepoBundledPluginPath(
+  packageRoot: string,
+  options: { repoRoot?: string } = {},
+): boolean {
+  const repoPluginsRoot = path.join(options.repoRoot ?? REPO_ROOT, "packages", "plugins");
+  return isPathWithin(repoPluginsRoot, packageRoot);
+}
+
+export function isStandaloneBundledPluginPath(
+  packageRoot: string,
+  options: { repoRoot?: string } = {},
+): boolean {
+  const repoRoot = options.repoRoot ?? REPO_ROOT;
+  return isPathWithin(path.join(repoRoot, "packages", "plugins", "sandbox-providers"), packageRoot);
+}
+
+export function resolveDeclaredPluginEntrypoints(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+): PluginEntrypointPath[] {
+  const paperclipPlugin = pkgJson["paperclipPlugin"];
+  if (
+    paperclipPlugin === null
+    || typeof paperclipPlugin !== "object"
+    || Array.isArray(paperclipPlugin)
+  ) {
+    return [];
+  }
+
+  const entrypoints: PluginEntrypointPath[] = [];
+  for (const key of ["manifest", "worker", "ui"] as const) {
+    const relativePath = (paperclipPlugin as Record<string, unknown>)[key];
+    if (typeof relativePath === "string" && relativePath.length > 0) {
+      entrypoints.push({
+        key,
+        absolutePath: path.resolve(packageRoot, relativePath),
+      });
+    }
+  }
+
+  return entrypoints;
+}
+
+export function listMissingDeclaredPluginEntrypoints(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+): PluginEntrypointPath[] {
+  return resolveDeclaredPluginEntrypoints(packageRoot, pkgJson).filter(
+    (entrypoint) => !existsSync(entrypoint.absolutePath),
+  );
+}
+
+function listMissingStandaloneBundledPluginRuntimeDependencies(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+): string[] {
+  // optionalDependencies are intentionally allowed to be absent (e.g.
+  // platform-specific packages), so they must not be treated as required here —
+  // otherwise post-build verification fails even when install/build succeeded.
+  const dependencyNames = new Set<string>([
+    STANDALONE_BUNDLED_PLUGIN_SDK_PACKAGE,
+    ...readPackageDependencyNames(pkgJson, "dependencies"),
+  ]);
+
+  return [...dependencyNames].filter(
+    (packageName) => !existsSync(resolvePackageInstallPath(packageRoot, packageName)),
+  );
+}
+
+function formatLocalPluginManualBuildHint(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+  options: { processEnv?: NodeJS.ProcessEnv; repoRoot?: string } = {},
+): string {
+  if (!isRepoBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })) return "";
+
+  const manualBuildCommand = buildLocalPluginRecoveryCommand(packageRoot, pkgJson, { repoRoot: options.repoRoot });
+  if (!manualBuildCommand) return "";
+
+  const autoBuildDisabled = (options.processEnv ?? process.env)["PAPERCLIP_DISABLE_PLUGIN_AUTOBUILD"] === "1"
+    ? " Auto-build is disabled by PAPERCLIP_DISABLE_PLUGIN_AUTOBUILD=1."
+    : "";
+
+  return `${autoBuildDisabled} Run \`${manualBuildCommand}\` from the repo root and retry.`;
+}
+
+function buildStandaloneBundledPluginInstallArgs(
+  packageRoot: string,
+): string[] {
+  const packageLockfilePath = path.join(packageRoot, "pnpm-lock.yaml");
+  return existsSync(packageLockfilePath)
+    ? ["install", "--ignore-workspace", "--frozen-lockfile"]
+    : ["install", "--ignore-workspace", "--no-lockfile"];
+}
+
+function buildStandaloneBundledPluginInstallCommand(
+  packageRoot: string,
+): string {
+  return `pnpm ${buildStandaloneBundledPluginInstallArgs(packageRoot).join(" ")}`;
+}
+
+function buildLocalPluginRecoveryCommand(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+  options: { repoRoot?: string } = {},
+): string | null {
+  if (isStandaloneBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })) {
+    const repoRoot = options.repoRoot ?? REPO_ROOT;
+    const relativePath = path.relative(repoRoot, packageRoot) || ".";
+    const installCommand = buildStandaloneBundledPluginInstallCommand(packageRoot);
+    return `cd ${relativePath} && ${installCommand} && pnpm build`;
+  }
+
+  return buildLocalPluginBuildCommand(pkgJson);
+}
+
+function buildLocalPluginBuildCommands(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+  options: {
+    repoRoot?: string;
+    needsBuild?: boolean;
+    needsStandaloneRuntimeBootstrap?: boolean;
+  } = {},
+): LocalPluginBuildCommand[] {
+  if (isStandaloneBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })) {
+    const commands: LocalPluginBuildCommand[] = [];
+    const shouldInstallStandaloneRuntime =
+      options.needsStandaloneRuntimeBootstrap === true
+      || (!existsSync(path.join(packageRoot, "node_modules")) && options.needsBuild !== false);
+
+    if (shouldInstallStandaloneRuntime) {
+      commands.push({
+        file: "pnpm",
+        args: buildStandaloneBundledPluginInstallArgs(packageRoot),
+        cwd: packageRoot,
+      });
+    }
+
+    if (options.needsBuild !== false) {
+      commands.push({
+        file: "pnpm",
+        args: ["build"],
+        cwd: packageRoot,
+      });
+    }
+    return commands;
+  }
+
+  const packageName = pkgJson["name"];
+  if (typeof packageName !== "string" || packageName.trim().length === 0) return [];
+  return [{
+    file: "pnpm",
+    args: ["--filter", packageName, "build"],
+    cwd: options.repoRoot ?? REPO_ROOT,
+  }];
+}
+
+export async function ensureLocalPluginBuilt(
+  packageRoot: string,
+  pkgJson: Record<string, unknown>,
+  options: {
+    processEnv?: NodeJS.ProcessEnv;
+    repoRoot?: string;
+    execFileAsyncImpl?: (
+      file: string,
+      args: readonly string[],
+      options: { cwd: string; timeout: number },
+    ) => Promise<{ stdout: string; stderr: string }>;
+  } = {},
+): Promise<void> {
+  const processEnv = options.processEnv ?? process.env;
+  if (processEnv["PAPERCLIP_DISABLE_PLUGIN_AUTOBUILD"] === "1") return;
+  if (!isRepoBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })) return;
+
+  const missingEntrypoints = listMissingDeclaredPluginEntrypoints(packageRoot, pkgJson);
+  const missingStandaloneRuntimeDeps = isStandaloneBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })
+    ? listMissingStandaloneBundledPluginRuntimeDependencies(packageRoot, pkgJson)
+    : [];
+  if (missingEntrypoints.length === 0 && missingStandaloneRuntimeDeps.length === 0) return;
+
+  const packageName = pkgJson["name"];
+  const manualBuildCommand = buildLocalPluginRecoveryCommand(packageRoot, pkgJson, { repoRoot: options.repoRoot });
+  if (typeof packageName !== "string" || packageName.trim().length === 0 || !manualBuildCommand) return;
+
+  const runExecFileAsync = options.execFileAsyncImpl ?? execFileAsync;
+  const buildCommands = buildLocalPluginBuildCommands(packageRoot, pkgJson, {
+    repoRoot: options.repoRoot,
+    needsBuild: missingEntrypoints.length > 0,
+    needsStandaloneRuntimeBootstrap: missingStandaloneRuntimeDeps.length > 0,
+  });
+
+  try {
+    for (const command of buildCommands) {
+      await runExecFileAsync(
+        command.file,
+        command.args,
+        { cwd: command.cwd, timeout: LOCAL_PLUGIN_AUTOBUILD_TIMEOUT_MS },
+      );
+    }
+  } catch (error) {
+    const stderr = typeof (error as { stderr?: unknown }).stderr === "string"
+      ? (error as { stderr: string }).stderr.trim()
+      : "";
+    const timeoutMessage = (error as { killed?: unknown }).killed === true
+      ? ` after timing out at ${LOCAL_PLUGIN_AUTOBUILD_TIMEOUT_MS}ms`
+      : "";
+    const stderrMessage = stderr.length > 0 ? ` stderr: ${stderr}` : "";
+    throw new Error(
+      `Failed to auto-build bundled local plugin ${packageName}${timeoutMessage}. ` +
+        `Run \`${manualBuildCommand}\` from the repo root and retry.${stderrMessage}`,
+    );
+  }
+
+  const stillMissingEntrypoints = listMissingDeclaredPluginEntrypoints(packageRoot, pkgJson);
+  const stillMissingStandaloneRuntimeDeps = isStandaloneBundledPluginPath(packageRoot, { repoRoot: options.repoRoot })
+    ? listMissingStandaloneBundledPluginRuntimeDependencies(packageRoot, pkgJson)
+    : [];
+  if (stillMissingEntrypoints.length > 0 || stillMissingStandaloneRuntimeDeps.length > 0) {
+    const missingDetails: string[] = [];
+    if (stillMissingEntrypoints.length > 0) {
+      missingDetails.push(`built entrypoints: ${stillMissingEntrypoints.map((entrypoint) => entrypoint.key).join(", ")}`);
+    }
+    if (stillMissingStandaloneRuntimeDeps.length > 0) {
+      missingDetails.push(`runtime dependencies: ${stillMissingStandaloneRuntimeDeps.join(", ")}`);
+    }
+    throw new Error(
+      `Bundled local plugin ${packageName} is still missing ${missingDetails.join("; ")} after auto-build. ` +
+        `Run \`${manualBuildCommand}\` from the repo root and retry.`,
+    );
+  }
+}
+
 /**
 * Resolve the manifest entrypoint from a package.json and package root.
 *
@@ -928,10 +1203,17 @@ export function pluginLoader(
    const pkgJson = await readPackageJson(resolvedPackagePath);
    if (!pkgJson) throw new Error(`Missing package.json at ${resolvedPackagePath}`);

+    if (localPath) {
+      await ensureLocalPluginBuilt(resolvedPackagePath, pkgJson);
+    }
+
    const manifestPath = resolveManifestPath(resolvedPackagePath, pkgJson);
    if (!manifestPath || !existsSync(manifestPath)) {
+      const manualBuildHint = localPath
+        ? formatLocalPluginManualBuildHint(resolvedPackagePath, pkgJson)
+        : "";
      throw new Error(
-        `Package ${resolvedPackageName} at ${resolvedPackagePath} does not appear to be a Paperclip plugin (no manifest found).`,
+        `Package ${resolvedPackageName} at ${resolvedPackagePath} does not appear to be a Paperclip plugin (no manifest found).${manualBuildHint}`,
      );
    }