d5ceb82571eddb1e618177f976bb4d82bfadeed2
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Devin Foley
|
547463d3a2 |
refactor(environments): make execution environments instance-scoped (#8375)
## Thinking Path > - Paperclip is the control plane for AI-agent companies, so execution environment selection has to stay inspectable and predictable across companies, agents, and runs. > - The environment subsystem decides where an agent heartbeat actually runs and how remote sandbox state is realized and restored. > - That subsystem previously mixed company-scoped environment catalogs with issue-level environment stamping, so a reassigned issue could keep executing in the previous assignee's sandbox. > - That behavior breaks the control-plane contract: changing the assignee should change the executing agent/environment path unless there is an explicit current override. > - Fixing it cleanly required more than a narrow patch; the environment model had to move to instance scope with a single inherited default and per-agent override semantics. > - This pull request rewires the schema, server/API surface, runtime resolution, and UI around that model, then adds regression coverage for cross-company inheritance and per-agent isolation. > - The benefit is that environment choice now follows the approved instance/agent configuration path instead of stale issue state, while shared environments only need to be configured once per instance. ## Linked Issues or Issue Description - No directly matching public GitHub issue or PR was found while searching for this refactor. ### What happened? Reassigning work between agents with different execution environments could keep running in the previous sandbox because environment choice was stamped onto the issue and outranked the current assignee. The same subsystem also forced environment catalogs to be duplicated per company even though the underlying execution environments were instance-wide resources. ### Expected behavior Execution should resolve through the current instance and agent configuration path, with one instance-scoped environment catalog, one instance default, optional per-agent override, and no stale issue-level environment authority surviving reassignment. ### Steps to reproduce 1. Configure two agents to use different execution environments. 2. Assign an issue to the first agent so the issue records execution state in that environment. 3. Reassign the same issue to the second agent and run another heartbeat. 4. Observe that the pre-fix runtime can still sync or execute in the original sandbox instead of the second agent's environment. ### Paperclip version or commit Current `master` before this PR. ### Deployment mode Self-hosted server. ### Installation method Built from source (`pnpm dev` / `pnpm build`). ### Agent adapter(s) involved - Claude Code - Not adapter-specific (core bug in environment authority / resolution) ### Database mode External Postgres. ### Access context Both board reassignment and agent heartbeats were involved. ## What Changed - Moved environments and their default selection contract to instance scope in DB/shared types, including the migration that dedupes legacy per-company environments and seeds the instance local default. - Reworked environment CRUD/auth flows to use instance-scoped APIs and added route/service coverage for instance-level environment management. - Changed runtime resolution to prefer `agent default -> instance default -> built-in local`, removed issue-level environment stamping from the active execution path, and isolated sandbox/plugin leases by `(executionWorkspaceId, agentId)`. - Added environment env-var runtime precedence so environment-provided values act as the baseline for agent execution. - Moved the environment UI into instance settings and updated agent configuration surfaces to reflect inherit/override behavior. - Added regression coverage for instance-default inheritance across companies and for the new runtime resolution behavior. - Fixed a rebase-only duplicate `enableTaskWatchdogs` flag regression in instance settings types/validators/services so the branch typechecks cleanly on current `master`. - Updated stale server tests so CI matches the shipped instance-scoped environment contract. ## Verification - `git diff --check` - `pnpm --filter @paperclipai/shared typecheck` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run server/src/__tests__/environment-runtime-driver-contract.test.ts server/src/__tests__/agent-permissions-routes.test.ts server/src/__tests__/environment-routes.test.ts server/src/__tests__/environment-instance-routes.test.ts server/src/__tests__/execution-workspace-policy.test.ts server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/instance-settings-routes.test.ts` ## Risks - The migration changes environment scope and dedupes existing rows, so installs with unusual legacy environment combinations should be reviewed carefully during upgrade. - Remote execution behavior now depends on instance-default inheritance semantics instead of issue-level stamping, so any remaining code paths that still assume issue-scoped environment authority would surface as follow-up bugs. - This PR includes both server/runtime behavior and UI relocation, so reviewers should watch for authorization edge cases around instance settings and environment management. > I checked [`ROADMAP.md`](ROADMAP.md). This work fits the existing Cloud / Sandbox agents direction as a bug-fix/refactor to current behavior, not a new parallel product surface. ## Model Used - OpenAI Codex coding agent in this Paperclip/Codex session; GPT-5-class tool-using model with code execution and shell access. The exact backend model ID is not exposed to the session runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
93291df5c8 |
fix(plugins): move dev SDK linking out of plugin postinstall scripts (#8255)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Sandbox-provider plugins (cloudflare, daytona, e2b, exe-dev, kubernetes, modal, novita) and `plugin-workspace-diff` are published as standalone npm packages, but during local dev they need the in-repo `@paperclipai/plugin-sdk` symlinked in > - Each of these plugins shipped a `postinstall` lifecycle script that traversed *out* of its own package directory (`node ../../../../scripts/link-plugin-dev-sdk.mjs`) to do that linking > - The publishable manifest is built by a `prepack` whitelist that drops the `scripts` field, so npm consumers don't see the postinstall today — but that safety property depends entirely on `prepack` running on every publish. A publish that skips lifecycle scripts would ship a tarball whose postinstall escapes its package directory at consumer install time > - This pull request removes the escape-the-package-dir lifecycle script from every plugin source manifest and moves the dev linking to a single root-level postinstall that iterates the excluded plugin directories itself > - The benefit is that plugin tarballs can no longer carry an install-time script that reaches outside their own directory, regardless of whether `prepack` runs ## Linked Issues or Issue Description This is a follow-up hardening change flagged during review of the Novita sandbox provider PR (#7595). **Problem (security):** Excluded plugin packages each carried `"postinstall": "node ../../../../scripts/link-plugin-dev-sdk.mjs"`. The relative path traverses outside the package root. Today the published manifest is sanitized by a `prepack` whitelist that drops `scripts`, so consumers are unaffected in the normal publish path. The risk is that this is a defense-in-depth gap: if a publish ever skips lifecycle scripts (e.g. `npm publish --ignore-scripts` is *not* used, or a tool publishes the raw manifest), the tarball would ship a postinstall that runs out-of-tree code at the consumer's install time. ## What Changed - Added a single root `package.json` `postinstall`: `node scripts/link-plugin-dev-sdk.mjs`. - Rewrote `scripts/link-plugin-dev-sdk.mjs` to iterate the excluded plugin directories itself (`packages/plugins/sandbox-providers/*` + the orchestration smoke example) instead of relying on each plugin to invoke it from its own cwd. Preserves both prior behaviors: leave a real installed SDK dir alone, and skip when already correctly symlinked (idempotent). - Removed `scripts.postinstall` from all 7 sandbox-provider plugins (cloudflare, daytona, e2b, exe-dev, kubernetes, modal, novita). - Removed `scripts.postinstall` from `plugin-workspace-diff` (a pnpm workspace member — pnpm already links the SDK, so the script was a no-op there). ## Verification - `node scripts/link-plugin-dev-sdk.mjs` from repo root: links the SDK into the excluded plugins and reports skipped (already-linked) dirs; re-running is idempotent. - `grep -r "link-plugin-dev-sdk" packages/plugins/*/package.json packages/plugins/sandbox-providers/*/package.json` returns no matches — no plugin source manifest references the linker any longer. - All affected `package.json` files re-validated as parseable JSON. ## Risks Low risk. Dev-only tooling: the linker only runs at the repo root during local install and only touches `node_modules/@paperclipai/plugin-sdk` symlinks inside excluded plugin dirs. No change to published plugin behavior or runtime code. Worst case if the root postinstall failed to run, local dev of an excluded plugin would not find the SDK symlink — easily re-run manually. ## Model Used Claude Opus (claude-opus-4-8), extended reasoning, with tool use / code execution in an agentic coding harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable (added `scripts/link-plugin-dev-sdk.test.js`, wired into `test:release-registry`) - [ ] If this change affects the UI, I have included before/after screenshots (N/A — no UI change) - [ ] I have updated relevant documentation to reflect my changes (N/A) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
20aea356cc |
refactor(deps-dev): bump vitest from 3.2.4 to 4.1.8 (#7581)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Test infrastructure across server, ui, packages/* runs on Vitest > - Dependabot opened a narrow bump (3.2.4 → 3.2.6), but the wider workspace is on 3.2.4 and the major-version bridge to v4 needs a coordinated change set across configs and tests > - Staying on 3.x indefinitely leaves us behind on Vitest 4 (perf, pool, and config improvements) and forces repeated patch-only dependabot churn > - This pull request upgrades Vitest to 4.1.8 across the workspace, updates `server/vitest.config.ts` and `scripts/run-vitest-stable.mjs` for the new API, and adjusts two UI tests for the new assertion semantics > - The benefit is a single, coherent Vitest 4 upgrade that supersedes #7570 and gets us on the supported major line ## What Changed - Bump `vitest` from `3.2.4` to `4.1.8` across root, `server`, `ui`, and all `packages/*` (including plugin examples and sandbox providers) - Update `server/vitest.config.ts` for Vitest 4 config surface - Update `scripts/run-vitest-stable.mjs` to match the new runner behavior - Adjust `ui/src/components/CommentThread.test.tsx` and `ui/src/components/MarkdownEditor.test.tsx` for Vitest 4 matcher/timing semantics - Refresh `pnpm-lock.yaml` ## Verification - `pnpm install` resolves cleanly with the new lockfile - `pnpm -w -r test` (server, ui, packages) runs under Vitest 4.1.8 ## Risks - Major-version Vitest bump: behavioral changes in pools, fake timers, and matcher strictness can surface flake. Test config and the two UI tests were updated to match v4 semantics; broader test runs should be watched on CI before merge. - Supersedes dependabot PR #7570 (3.2.4 → 3.2.6); that PR should be closed. ## Model Used - Claude (Anthropic) — `claude-opus-4-7`, extended thinking, tool use enabled ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [ ] I have run tests locally and they pass - [ ] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge Closes #7570 |
||
|
Devin Foley
|
aea35fe695 |
exe.dev config UX: advanced-options disclosure, form-default fix, SSH key handling (PAPA-407) (#7025)
## Thinking Path
> - Paperclip orchestrates AI agents and provisions sandboxed execution
environments for them; one of those provisioners is the exe.dev plugin,
which runs each agent inside a long-lived VM reached over SSH.
> - The instance-config form for that plugin is rendered generically by
`JsonSchemaForm` from the plugin's `instanceConfigSchema`, so any UX
problem with the form is split between the shared form component and the
plugin's schema/runtime code.
> - Users coming in cold hit a 12-field flat config they couldn't reason
about (PAPA-407), a form that silently submitted `cpu: 0` for untouched
optional fields (PAPA-407 root cause), a `sshPrivateKey` textarea that
truncated RSA-4096 keys at 4096 chars (PAPA-449), a save flow that
accepted clearly-malformed keys and only blew up at lease time with raw
SSH stderr (PAPA-450, PAPA-451), and a manifest that didn't distinguish
"essential" from "advanced" knobs (PAPA-410 / PAPA-411 — duplicate
sub-issues with identical scope; PAPA-418 reconciliation kept PAPA-410
canonical).
> - These problems all point at the same surface (exe.dev sandbox
config) and are tightly coupled in code — PAPA-449/450/451 patch fields
that PAPA-410/411 introduce — so they get reviewed together.
> - This pull request lands the shared-form changes (advanced-options
disclosure, optional-scalar defaults) and the exe.dev-specific changes
(manifest restructure, longer `maxLength`, stderr translation, save-time
key validation) as five focused commits stacked on `master`.
> - The benefit is a config form that defaults to the two fields a new
user actually needs (API key + SSH private key) with a collapsible
disclosure for the rest, no silent truncation or zero-default
submissions, and SSH key problems surfaced at save time with actionable
messages instead of cryptic post-provision failures.
## What Changed
- **JsonSchemaForm advanced-options disclosure** (PAPA-410, PAPA-411 —
same scope, see note above): adds `x-paperclip-advanced` /
`x-paperclip-group` schema annotations and renders flagged fields behind
a collapsible "Advanced options" disclosure that auto-opens when a
hidden field has a validation error. Exe.dev manifest is restructured to
use the new annotations, so essentials (`apiKey`, `sshPrivateKey`) show
by default while the long tail of optional knobs is grouped under "SSH
access" / "VM resources" / "More options" headings.
- **Omit optional scalar defaults** (PAPA-407): `getDefaultForSchema` no
longer materialises `0` / `""` for optional
`number`/`integer`/`string`/`secret-ref` fields without an explicit
`default`. Object recursion drops properties whose default is
`undefined`. Fields that declare a `default` (e.g. `sshPort: 22`) still
round-trip. Adds a regression test against `getDefaultValues`.
- **Raise `sshPrivateKey` `maxLength`** (PAPA-449): bumps the exe.dev
manifest cap from 4096 to 8192 so RSA-4096 OpenSSH private keys (which
can exceed 4 KB with comments/metadata) aren't silently truncated at
submit.
- **Translate `invalid format` SSH stderr** (PAPA-450):
`formatSshFailure` now recognises `Load key … invalid format` in
combined stderr/stdout and returns a specific message naming the
key-format problem ("isn't an OpenSSH/PEM private key — confirm the
secret starts with `-----BEGIN … PRIVATE KEY-----` and isn't the `.pub`
or a PuTTY `.ppk` export") instead of dumping the raw stderr.
- **Save-time SSH key validation** (PAPA-451):
`onEnvironmentValidateConfig` inline-parses `sshPrivateKey` and rejects
common failure modes — pasted public keys, PuTTY `.ppk` format, missing
`-----END-----` footer, non-base64 body — so the form surfaces an inline
error before any VM is provisioned. Secret-ref bindings (UUIDs) are
still passed through unchanged.
## Verification
CI gates (`pnpm typecheck`, `pnpm test`, the targeted vitest suites
below) all pass.
Run locally:
```bash
# Shared form
pnpm --filter @paperclipai/ui exec vitest run src/components/JsonSchemaForm
# 9 tests pass — includes the new "omits optional scalar fields" regression
# and the three advanced-options-disclosure tests.
# exe.dev plugin
cd packages/plugins/sandbox-providers/exe-dev && pnpm test
# 32 tests pass — includes the new sshPrivateKey-validation cases
# and the new "invalid format" stderr-translation case.
```
Manual smoke (after reinstalling the plugin so the DB manifest
refreshes):
1. Open the exe.dev environment config page. **Default view shows API
Key + SSH Private Key only**, with an "Advanced options" disclosure for
everything else (PAPA-410 / PAPA-411).
2. Paste a `.pub` file's contents into SSH Private Key, click Save.
**Inline error** rejecting the wrong-format key (PAPA-451).
3. Re-paste a valid OpenSSH/PEM private key longer than 4096 bytes —
saves cleanly (PAPA-449).
4. Save the form with everything optional left blank — server no longer
rejects with `"cpu must be greater than 0 when provided"` (PAPA-407).
5. Force a bad key through via a stored secret-ref binding and lease a
VM — failure message names the key-format problem instead of dumping raw
SSH stderr (PAPA-450).
## Risks
- **PAPA-410 / PAPA-411 manifest restructure** is the largest surface
here. Schemas using `x-paperclip-*` extensions are forward-compatible
with stricter JSON Schema validators (extensions are ignored by
default), and the form gracefully renders a flat layout when no field
opts in.
- **PAPA-407** changes form-default behaviour: optional scalar fields
that previously round-tripped as `""` / `0` will now be `undefined` and
absent from the submitted payload. Downstream consumers that expected
the empty-string/zero shape need to treat the field as optional.
Spot-checked the existing exe.dev driver — it already uses
`parseOptionalString` / `parseOptionalInteger`, which treat missing
fields as `null` rather than `0`/`""`.
- **PAPA-451** adds a save-time check, so a
previously-saved-but-malformed `sshPrivateKey` raw value will now fail
to re-save. Bound secret-refs are unaffected, matching how the user
reaches the bad-key state today (via the secrets picker).
- **PAPA-449** simply raises a cap; no semantic risk.
- **PAPA-450** only kicks in on the "invalid format" code path; existing
onboarding-marker branch is untouched.
## Model Used
- Provider: Anthropic
- Model: Claude Opus 4.7 (`claude-opus-4-7`)
- Capabilities used: code reading, code editing, test execution, git/PR
mechanics, Paperclip API for issue coordination
## Checklist
- [x] PR body sections present (Thinking Path, What Changed,
Verification, Risks, Model Used, Checklist)
- [x] Unit tests added for the new behaviours (JsonSchemaForm
default-value omission + advanced disclosure; exe.dev plugin validation
+ stderr translation)
- [x] Existing tests still pass locally (`vitest run` on both packages)
- [x] No raw secrets, IP addresses, or machine-local config in commits
or PR body
- [x] Commits are atomic per linked issue (PAPA-410 / PAPA-411,
PAPA-407, PAPA-449, PAPA-450, PAPA-451)
- [x] Branch is up-to-date with `origin/master`
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
|
||
|
Devin Foley
|
5a64cf52a1 |
Add exe.dev sandbox provider plugin (#5688)
> _Stacked on top of #5685 → #5686 → #5687. Diff against master includes commits from earlier PRs in the stack — review focuses on the two new commits (`Add long-secret textarea variant to JsonSchemaForm SecretField` + `Add exe.dev sandbox provider plugin`)._ ## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Each agent runs in a sandbox environment, and operators choose the provider — today E2B, Daytona, and (in this stack) Cloudflare > - exe.dev offers per-VM sandboxes via a small CLI / HTTP API — useful for operators who want full Linux VMs (vs container/runtime-only sandboxes) > - The plugin shape mirrors the e2b plugin: lifecycle hooks (`new`, `ls`, `rm`) drive exe.dev's CLI; SSH plumbing handles direct VM access for adapters that need it > - exe.dev VMs come up bare — `node` is not preinstalled, so the Paperclip sandbox callback bridge (a Node script) needs Node 20 installed at VM init via `--setup-script`. The plugin defaults the setup script to a Nodesource install > - The auth field accepts long SSH private keys, which need a textarea variant of the existing `SecretField` in `JsonSchemaForm` — added behind a `maxLength > THRESHOLD` opt-in so other secret fields are unaffected > - The benefit is that operators get exe.dev as a fully working sandbox provider out of the box, with no manual VM provisioning required ## What Changed **Shared UI support (`Add long-secret textarea variant to JsonSchemaForm SecretField`):** - `ui/src/components/JsonSchemaForm.tsx` + new `JsonSchemaForm.test.tsx`: when a secret-formatted field declares `maxLength` larger than the existing single-line threshold, render a monospace textarea instead of the masked input. Short secrets (API keys, tokens) keep the existing masked-input + show/hide toggle behavior. **The exe.dev plugin (`Add exe.dev sandbox provider plugin`):** - `packages/plugins/sandbox-providers/exe-dev/`: plugin entry, manifest, plugin runtime, README, and 19-test Vitest suite. - Manifest fields: API token (with `secret-ref` + `/exec` permission notes — needs `new`, `ls`, `rm`), API URL override, optional SSH username, optional SSH private key (uses the new `JsonSchemaForm` textarea variant via `maxLength: 4096`), optional SSH identity-file path, optional setup script. - Default `--setup-script` is a Nodesource Node 20 install. exe.dev VMs come up bare and the Paperclip sandbox callback bridge is a Node script, so without Node preinstalled the bridge can't start. Operators can override by supplying their own setup script. - `runLifecycleCommand` redacts env values from the executed command before surfacing it in error messages, so secrets passed via `--env=KEY=VALUE` don't leak into operator-visible failures. - The plugin distinguishes exe.dev's SSH onboarding failures (`Please complete registration by running: ssh exe.dev`) from general SSH failures and surfaces a clear remediation message. - `scripts/release-package-manifest.json`: register the new plugin for CI publish alongside the existing daytona / e2b providers. ## Verification - `pnpm typecheck` - `pnpm exec vitest run --no-coverage ui/src/components/JsonSchemaForm.test.tsx` - `(cd packages/plugins/sandbox-providers/exe-dev && pnpm test)` — 19 passing For an operator-side smoke test: 1. Get an exe.dev API token with `/exec` permission for `new`, `ls`, `rm`. 2. Register the plugin in your Paperclip instance, configure an environment with the token. 3. Create a sandbox env whose provider is `exe-dev`, then run a Codex or Claude job against it. The default Node 20 setup script should bring the VM up automatically. ## Risks - Adds a new sandbox provider plugin that follows the existing daytona / e2b shape; behavior on existing providers is unchanged. - The `JsonSchemaForm` textarea variant only engages for fields that opt in via `maxLength` larger than the existing threshold. All existing secret fields (which don't declare a `maxLength`) keep their current rendering. Test coverage pins both paths. - The redaction in `runLifecycleCommand` is a defense-in-depth measure; the test suite exercises the redaction path. If the redaction misses a future env-arg shape, the worst case is restored behavior (secrets in error messages), which is what the existing daytona / e2b plugins also do today. - Default setup script downloads from `deb.nodesource.com` over HTTPS at VM init. Operators on air-gapped networks or with a different package strategy can override the setup script. ## Model Used - Provider: Anthropic - Model: Claude Opus 4.7 (1M context) - Capabilities used: extended reasoning, tool use (Read/Edit/Bash/Grep) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots — UI change is a textarea variant of an existing secret field; will attach screenshots before requesting merge - [x] I have updated relevant documentation to reflect my changes (plugin README, manifest descriptions) - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |