Compare commits

...

10 Commits

Author SHA1 Message Date
Paperclip cdca784e6d fix(opencode-local): pad prompts below the opencode 1.16.2 silent-exit floor (ORA-284)
PR / policy (pull_request) Successful in 51s
commitperclip PR Review / review (pull_request_target) Failing after 2m5s
PR / Typecheck + Release Registry (pull_request) Successful in 11m54s
PR / General tests (server (1/3)) (pull_request) Failing after 9m2s
PR / General tests (server (2/3)) (pull_request) Failing after 8m31s
PR / General tests (server (3/3)) (pull_request) Failing after 10m35s
PR / General tests (workspaces-a) (pull_request) Failing after 8m18s
PR / General tests (workspaces-b) (pull_request) Failing after 6m31s
PR / Build (pull_request) Successful in 13m27s
PR / Verify serialized server suites (1/4) (pull_request) Failing after 7m4s
PR / Verify serialized server suites (2/4) (pull_request) Failing after 8m12s
PR / Verify serialized server suites (3/4) (pull_request) Failing after 6m27s
PR / Verify serialized server suites (4/4) (pull_request) Failing after 6m15s
PR / Canary Dry Run (pull_request) Successful in 18m1s
PR / e2e (pull_request) Failing after 5m44s
PR / verify (pull_request) Failing after 6s
opencode 1.16.2 has a non-deterministic silent-exit mode that fires on
short stdin prompts (observed 0-byte / <200-byte run logs, ~32% stall
rate on FoundingEngineer wake-payload-only runs). The root cause lives
in the opencode binary, not the adapter. As a server-side workaround,
ensure every prompt is at least 256 bytes by appending a deterministic
Paperclip Runtime Context block when the joined sections fall short.

- New exported helper padShortPrompt(input) in execute.ts
- New exported constant OPENCODE_PROMPT_MIN_LENGTH_CHARS = 256
- New promptMetrics.promptPadded (0/1) so the server can observe how
  often the workaround fires
- Four new unit tests cover the long-prompt, short-prompt, empty-cwd,
  and trailing-newline paths

Refs: ORA-284, ORA-275, ORA-276

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-06-22 01:05:11 +00:00
Svetlana Zolotenkova 9ac24317e6 fix(issues): ignore cancelled child blockers in attention (#7577)
Docker / build-and-push (push) Failing after 3m33s
Refresh Lockfile / refresh (push) Successful in 10m42s
Release / verify_canary (push) Failing after 16m21s
Release / verify_stable (push) Has been skipped
Release / publish_canary (push) Has been skipped
Release / preview_stable (push) Has been skipped
Release / publish_stable (push) Has been skipped
Fixes #7578

## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies.
> - Blocked issue health is surfaced through blockerAttention so
operators can see whether blocked work has a live path.
> - The blockerAttention graph uses both explicit blockedBy edges and
direct child issue edges.
> - Done children were already ignored, but cancelled direct children
still appeared as unresolved blockers.
> - Explicit cancelled dependencies should remain visible as dependency
problems, but terminal direct children should not inflate a parent
blocker count.
> - This pull request narrows child-edge traversal to non-terminal
children and adds a regression test for the observed case.
> - The benefit is that cancelled child issues no longer make blocked
parents look like they have extra unresolved blocker attention.

## What Changed

- Added terminal child status filtering for blockerAttention
parent-child traversal so cancelled direct children are ignored with
done children.
- Added a server regression test where a blocked parent has active
explicit blockers plus a cancelled direct child; the cancelled child no
longer increases unresolved counts or becomes the sample blocker.

## Verification

- `perl -e 'alarm shift; exec @ARGV' 300 pnpm exec vitest run
server/src/__tests__/issue-blocker-attention.test.ts` -> 19 tests
passed.

## Risks

- Low risk: the change only affects implicit direct-child
blockerAttention edges.
- Explicit blockedBy edges to cancelled issues are intentionally
unchanged and remain represented as attention-required dependency
problems.

## Model Used

- OpenAI GPT-5, Codex coding agent in tool-enabled CLI environment;
reasoning and code execution used for repository inspection, patching,
git operations, and targeted test verification.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots (not applicable; no UI change)
- [x] I have updated relevant documentation to reflect my changes (not
applicable; bugfix covered by regression test)
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-06-21 00:29:44 -07:00
Devin Foley d1e6662ed8 fix(server): let codex_local agents inherit host Codex login by default (#8425)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The `codex_local` adapter spawns the Codex CLI for an agent, and
`server/src/routes/agents.ts` normalizes each agent's
`adapterConfig.env` on create/hire/update
> - PR #8272 added an isolation guard that, on every codex_local agent,
force-set a per-agent `CODEX_HOME` and injected `OPENAI_API_KEY = ""`,
and rejected any "shared" home
> - This broke the common case: operators who deleted `CODEX_HOME` /
`OPENAI_API_KEY` in the UI saw them silently re-appear on save, and
every agent was forced into an isolated home instead of sharing the
device's existing Codex login (`~/.codex` / `$CODEX_HOME`)
> - This pull request replaces the always-on isolation guard with
key-scoped isolation: a keyless agent gets no env overrides and inherits
the host Codex login at runtime; we only carve out an isolated per-agent
`CODEX_HOME` when the agent sets its own `OPENAI_API_KEY`
> - The benefit is that env-var deletion now persists, agents on one
device share the host login by default, and per-account isolation is
still available by setting a per-agent key

## Linked Issues or Issue Description

No public GitHub issue exists. Bug report (following `bug_report.yml`):

**What happened?** In a `codex_local` agent's configuration, removing
the `CODEX_HOME` and `OPENAI_API_KEY` env vars via the UI (clicking the
X) appears to work, but on save they instantly re-appear. The persisted
config never loses the slots.

**Root cause.** `applyCodexLocalIsolationGuard` in
`server/src/routes/agents.ts` (added in #8272) re-injected `CODEX_HOME`
(defaulting to a per-agent home) and `OPENAI_API_KEY = ""` on every
create/hire/update, and rejected shared homes outright. So a PATCH that
omitted those keys had them written back server-side.

**Expected behavior.** Deleting these env vars should persist. A
codex_local agent with no key should inherit whatever Codex login
already exists on the device.

**Steps to reproduce.**
1. Open a `codex_local` agent's configuration with `CODEX_HOME` and
`OPENAI_API_KEY` set.
2. Remove both env vars and save.
3. Re-open the config — both slots are back.

Related (different approach): Refs #8399 (keeps per-agent isolation,
fixes only the empty `OPENAI_API_KEY` slot), Refs #8272 (introduced the
guard), Refs #8403 (managed-auth seeding into isolated homes).

## What Changed

- `server/src/routes/agents.ts` — replaced
`applyCodexLocalIsolationGuard` (+ `assertCodexLocalHomeIsNotShared` /
`normalizeCodexLocalHomePath`) with `applyCodexLocalKeyIsolation`. A
codex_local agent now receives **no** env overrides unless it explicitly
sets `OPENAI_API_KEY`; only then is an isolated per-agent `CODEX_HOME`
injected (and only if the agent has not set its own `CODEX_HOME`).
Keyless agents fall back to the host Codex login at runtime. Dropped the
now-unused `node:os` import and the shared-home rejection.
- `server/src/__tests__/agent-adapter-validation-routes.test.ts` —
updated the validation-route tests to assert env-var deletion persists,
keyless agents get no overrides, and key-bearing agents still get an
isolated `CODEX_HOME`.

## Verification

```bash
cd server && npx vitest run src/__tests__/agent-adapter-validation-routes.test.ts
```

All 6 tests pass locally. Manually verified in the live UI: removing
`CODEX_HOME` and `OPENAI_API_KEY` from a codex_local agent and saving
now persists the deletion (slots no longer re-appear on reload).

## Risks

- **Host-key leak for keyless agents on a host with `OPENAI_API_KEY`
set.** Low/intended: the product direction here is that codex_local
agents inherit the host's Codex login by default; per-account isolation
is opt-in via a per-agent `OPENAI_API_KEY`, which then gets its own
`CODEX_HOME`.
- **Divergence from #8399.** That PR retains the per-agent isolation
default and only stops injecting the empty key, so it does not address
the `CODEX_HOME` re-injection half of this bug. This PR intentionally
changes the default to host-login inheritance. Reviewers should pick one
direction.
- **No migration.** Existing agents that already store these slots are
not auto-cleaned, but operators can now delete them and the deletion
sticks.

## Model Used

- Provider: Anthropic (Claude)
- Model ID: `claude-opus-4-8`
- Capabilities: tool use, code execution, extended reasoning, agentic
workflow

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [ ] I will address all Greptile and reviewer comments before
requesting merge
2026-06-20 22:51:56 -07:00
Devin Foley 2c98c8e1e5 Fix sandbox git publishing and large workspace uploads (#8422)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox and SSH runtimes need to preserve agent work across isolated
execution environments
> - Git-backed workspaces were being copied mostly as filesystem
archives, which breaks when `.git` points outside the mounted workspace
and makes sandbox agents unable to publish their own branches
> - Large ignored dependency trees could also be swept into the sandbox
overlay, causing multi-GB transfers and max-string failures in some
sandbox clients
> - This pull request makes sandbox runtime setup use a git-backed HEAD
sync plus a small dirty/untracked overlay, and bounds sandbox file
transfers so large archives do not need one huge string
> - The benefit is that sandbox agents can commit and push from a usable
git checkout without uploading dependency trees such as `node_modules`

## Linked Issues or Issue Description

Refs #8395

No public duplicate issue or PR was found after searches for `sandbox
git workspace`, `git push sandbox`, and `node_modules sandbox upload`.

Bug report:

- What happened: sandbox-backed agent workspaces could receive a `.git`
file that pointed at host-only git state, leaving the sandbox unable to
run normal git workflows. The sandbox overlay upload could also include
ignored dependency directories, creating very large transfers.
- Expected behavior: sandbox and remote runtimes should prepare a usable
git-backed workspace, copy only the necessary workspace overlay, and
restore git history plus file changes without depending on a host-only
`.git` path.
- Steps to reproduce:
1. Run an agent in a sandbox-backed workspace whose local git checkout
is a worktree.
2. Ask the agent to complete a GitHub workflow that requires commit/push
access.
3. Observe that git operations can fail inside the sandbox, and ignored
dependency trees can be uploaded as part of the workspace overlay.
- Paperclip version or commit: reproduced against `master` before this
PR, base `7aa212296eb1`.
- Deployment mode: local dev / sandbox-backed runtime.
- Installation method: built from source.
- Agent adapters involved: local adapters using shared adapter-utils
runtime preparation.
- Database mode: not database-related.
- Access context: agent runtime.
- Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS
arm64.
- Privacy checklist: all pasted output was reviewed for secrets, private
hostnames, local usernames, and internal instance links.

## What Changed

- Added a GitHub workflow push preflight so agent runs can detect
missing push credentials when a workflow explicitly needs GitHub
publishing.
- Added shared git workspace sync helpers for shallow HEAD import/export
and dirty/untracked overlay tracking.
- Updated sandbox managed runtime setup to use git history plus a
selected overlay instead of uploading the full local workspace for
git-backed workspaces.
- Bounded sandbox archive upload/download paths so large payloads stream
or chunk instead of materializing one oversized string.
- Excluded `.git` and ignored dependency trees from sandbox upload,
download, and restore baselines while preserving local ignored
directories during sync-back.
- Added focused tests for git workspace sync, sandbox overlay selection,
transfer chunking, and heartbeat push-preflight behavior.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/git-workspace-sync.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
packages/adapter-utils/src/command-managed-runtime.test.ts
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/heartbeat-workspace-session.test.ts`
- `pnpm --filter @paperclipai/adapter-utils typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm build`
- Public-hygiene scan of the PR diff and commit messages for internal
issue ids, local paths, private hostnames, and obvious token patterns.

## Risks

- Medium risk: this changes sandbox runtime synchronization semantics
for git-backed workspaces, especially around dirty tracked files,
untracked files, deleted paths, and ignored files.
- The main mitigation is focused test coverage for upload contents,
restore exclusions, and git round-trip behavior.
- The SSH runtime keeps the current bundle-based implementation from
`master`; this PR only aligns shared excludes and sandbox behavior with
that model.

## Model Used

OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub
workflow, with code execution and repository inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 22:03:55 -07:00
Devin Foley 33353ce62b feat(skills): remove bundled paperclip-dev skill and retire required skill attribute (#7029)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Local adapters (Claude, Codex, Cursor, Gemini, Grok, OpenCode, Pi,
ACPX) ship bundled "skills" — opinionated Markdown prompt bundles
materialized into the agent's runtime
> - One of those bundled skills, `paperclip-dev`, existed to let agents
develop Paperclip itself; it has now moved to its own external repo and
no longer belongs in the core tree
> - The adapter skill model also carried a `required` / `requiredReason`
attribute plus a `paperclip_required` `AdapterSkillOrigin` variant, all
of which only existed to mark bundled skills as non-optional in the UI
and adapter sync logic
> - With `paperclip-dev` gone, no bundled skill is "required" anymore,
and the type / runtime surface for `required` is dead weight — but it is
computed at request time and never persisted, so a clean removal is safe
(no compatibility shim needed)
> - This pull request deletes `skills/paperclip-dev/` and removes every
trace of the `required` / `requiredReason` field and the
`paperclip_required` origin across shared types, validators,
adapter-utils, all eight local adapters, server routes, the
company-skills service, the UI, the storybook fixtures, and the test
suite
> - The benefit is a smaller, simpler adapter-skill surface: one origin
(`company_managed`) for managed bundled skills,
`resolvePaperclipDesiredSkillNames` collapses to "just the configured
desired set", and the AgentDetail skills tab no longer renders a
"Required by Paperclip" section that no longer applies

## Linked Issues or Issue Description

<!-- No existing public GitHub issue; describing the underlying work
inline (feature_request template fields). -->

**Summary**

Remove the bundled `paperclip-dev` skill (now maintained in its own
external repo) and retire the `required` / `requiredReason` skill
attribute and the `paperclip_required` skill origin, which only existed
to support it.

**Problem or motivation**

`paperclip-dev` is the only bundled skill that was ever marked
"required". Now that it lives in a separate repository, shipping it
inside the core tree is wrong, and the entire `required` surface (a type
field, a validator field, a synthesized `paperclip_required` origin, UI
"Required by Paperclip" section, and required-skill merging in the
desired-skills calculation) becomes dead weight. The `required` value is
computed at request time and never persisted, so it can be removed
cleanly without a migration or compatibility shim.

**Proposed solution**

Delete `skills/paperclip-dev/`, drop the `required` / `requiredReason`
fields and `paperclip_required` origin everywhere they are produced or
consumed, collapse managed-skill origin to a single `company_managed`
value, and simplify `resolvePaperclipDesiredSkillNames` to return only
the configured desired set.

**Alternatives considered**

Keeping the `required` attribute as a no-op for forward compatibility —
rejected because it is request-time only (nothing persists it), so
leaving it in place is pure dead surface area with no callers.

**Roadmap alignment**

Internal cleanup / dead-code removal that simplifies the adapter-skill
surface; it does not introduce or duplicate any planned core feature in
ROADMAP.md.

## What Changed

- Deleted bundled `skills/paperclip-dev/` (moved to a separate repo).
- Dropped `required`, `requiredReason`, and the `paperclip_required`
origin from `packages/shared/src/types/adapter-skills.ts`,
`packages/shared/src/validators/adapter-skills.ts`, and
`packages/adapter-utils/src/types.ts`.
- In `packages/adapter-utils/src/server-utils.ts`: removed
`readSkillRequired()`; dropped `required`/`requiredReason` from
`listPaperclipSkillEntries()`,
`normalizeConfiguredPaperclipRuntimeSkills()`,
`buildPersistentSkillSnapshot()`, and `PaperclipSkillEntry`; collapsed
`buildManagedSkillOrigin()` to always return `company_managed`;
simplified `resolvePaperclipDesiredSkillNames()` to return only the
configured desired set (signature preserved so adapter call sites are
untouched).
- Walked all eight local adapters (`acpx-local`, `claude-local`,
`codex-local`, `cursor-local`, `gemini-local`, `grok-local`,
`opencode-local`, `pi-local`) and removed every remaining
`requiredReason` / `paperclip_required` reference.
- `server/src/services/company-skills.ts`: dropped the `required =
sourceKind === "paperclip_bundled"` synthesis when listing runtime skill
entries.
- `server/src/routes/agents.ts`: removed required-skill merging from the
desired-skills calculation in the persist-config path and the
unsupported-snapshot path (keeping the current version-aware
`desiredSkillEntries` structure).
- `ui/src/pages/AgentDetail.tsx`: dropped required-based filters, the
required tooltip, and the entire "Required by Paperclip" section from
the agent skills tab; storybook fixtures in
`ui/storybook/stories/acpx-local.stories.tsx` cleaned up to match.
- Tests: deleted the `required: false` case in
`paperclip-skill-utils.test.ts` and the "keeps required bundled skills
installed" case in every `*-local-skill-sync.test.ts`;
`acpx-local-execute.test.ts`, `cursor-local-execute.test.ts`,
`cursor-local-skill-sync.test.ts`, `agent-skills-routes.test.ts`, and
`packages/adapter-utils/src/server-utils.test.ts` were updated to drop
removed fields and map `origin: "paperclip_required"` →
`"company_managed"`.
- `server/src/adapters/registry.ts`: two `as unknown as
ServerAdapterModule["..."]` casts on `hermesListSkills` /
`hermesSyncSkills` (matching the existing `executeHermesLocal` pattern).
`hermes-paperclip-adapter@0.2.0` still depends on the published
`@paperclipai/adapter-utils` which keeps the retired
`paperclip_required` variant; the cast bridges the
workspace-vs-published type mismatch at the registry seam and can drop
once hermes upgrades.

## Verification

Run from the workspace root:

```sh
grep -rn "skills/paperclip-dev" .
grep -rn "paperclip_required" --include="*.ts" --include="*.tsx" .
grep -rn "requiredReason" --include="*.ts" --include="*.tsx" .

pnpm -w typecheck
pnpm --filter @paperclipai/server exec vitest run paperclip-skill-utils
pnpm --filter @paperclipai/server exec vitest run skill-sync
```

The first three greps return only the explanatory comment in
`server/src/adapters/registry.ts` (no live `paperclip_required` /
`requiredReason` usage) and zero `skills/paperclip-dev` source hits.

Locally:

- `pnpm -w typecheck` → all packages this PR touches pass
(adapter-utils, shared, server, ui, cli, and the
cursor/gemini/opencode/pi adapters).
- Affected vitest suites pass: `paperclip-skill-utils`, `server-utils`,
all eight `*-local-skill-sync`, `agent-skills-routes`, and the
`acpx`/`cursor`/`pi` execute suites.

## Risks

- Behavioral shift in the agent skills UI: the "Required by Paperclip"
section disappears. No bundled skill is required anymore, so this only
affects environments that previously surfaced `paperclip-dev` as a
forced-on row; those installs will see the skill move into the regular
"company-managed" list (and be uninstalled on next sync unless
explicitly listed as desired).
- Existing agents may still have the string `"paperclip-dev"` in their
persisted `desiredSkills`. That entry is inert (no source for it to
install from); a one-time DB cleanup is out of scope. Low risk.
- Hermes adapter type bridge: two casts in `registry.ts` paper over a
type-only divergence between the workspace `@paperclipai/adapter-utils`
and the published version still pinned by
`hermes-paperclip-adapter@0.2.0`. Runtime behavior is unaffected because
the retired `paperclip_required` value is no longer produced by anything
in this tree. The casts can be removed once hermes upgrades its
dependency.

## Model Used

- Provider: Anthropic
- Model: Claude Opus 4.7 (`claude-opus-4-7`)
- Capability: agent tool use via Paperclip's `claude_local` adapter

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 21:58:44 -07:00
Neeraj Kumar Singh B 7aa212296e codex_local adapter output inactivity monitor (#5017)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - The `codex_local` adapter runs `codex exec` as a child process and
streams JSONL events from its stdout
> - NEE-79 caught a real-world `codex_local` orphan: codex sat in
`read()` on stdin for 1h+, no JSON events emitted past startup, no LLM
call in flight. The adapter had no inactivity timer; the only safety net
was the platform-level 1h silent-run detector
> - This is precisely the failure shape NEE-80 scoped: an
adapter-detected fault that should be killed *by the adapter* and
surfaced as `failed`, not waited out for an hour
> - This pull request adds an output-inactivity watchdog inside the
codex-local adapter that resets on every parsed JSONL event from stdout,
kills the child via SIGTERM → 5s grace → SIGKILL when it fires, and
resolves the run with a structured watchdog failure
> - The benefit is that NEE-79-class hangs shorter than 1h stop reaching
the platform-level safety net — they fail fast and visibly at the
adapter layer, with diagnostic logs that don't require host shell access

## Linked Issues or Issue Description

No existing GitHub issue covers this; describing the underlying bug
in-PR per the bug report template
(`.github/ISSUE_TEMPLATE/bug_report.yml`):

- **What happened?** A `codex_local` run sat in `read()` on stdin for
over an hour: codex emitted its startup JSONL events, then nothing — no
further events, no LLM call in flight, no exit. The adapter kept the
child alive indefinitely; the run was only reaped by the platform-level
1h silent-run safety net, an hour after it had effectively died.
- **Expected behavior:** The adapter should detect that its child has
stopped producing output long before the platform-level safety net, kill
it, and surface the run as `failed` with a diagnostic that explains what
happened.
- **Steps to reproduce:** Run any `codex_local` issue where `codex exec`
hangs after startup (e.g. codex blocks reading stdin and never emits
another JSONL event). Observe the run stays alive until the 1h platform
safety net fires.
- **Paperclip version or commit:** reproduced on master prior to this
branch.
- **Agent adapter(s) involved:** codex_local.

Related PRs found while searching for duplicates (none implement an
event-aware inactivity watchdog inside `codex_local`):
- #4004 — generic idle + wall watchdogs for `runChildProcess` in
adapter-utils; complementary, operates below the JSONL parse layer and
is not codex-event-aware
- #4742 — fail-fast on codex-local *startup* hang; this PR covers the
post-startup hang class
- #6861 — zombie-run termination for codex-local; reaping after exit,
not inactivity detection
- #7811 — the analogous output-idle timeout for grok-local

## What Changed

- `packages/adapters/codex-local/src/server/watchdog.ts` *(new)* — pure
watchdog primitive with injectable timers/clock:
`resolveCodexInactivityTimeout`, `createCodexInactivityWatchdog`,
`formatWatchdogErrorMessage`. Default `7 * 60_000` ms; honors `null` as
the disabled escape hatch
- `packages/adapters/codex-local/src/server/execute.ts` — `runAttempt`
now wraps `onSpawn` to capture `pid`/`processGroupId`, feeds stdout
chunks through `noteStdoutChunk`, and on watchdog fire sends SIGTERM to
the process group, schedules SIGKILL after 5s, and returns an
`AdapterExecutionResult` with `exitCode: null`, `signal:
SIGTERM|SIGKILL`, `errorMessage: "watchdog: no codex output for {N}m
{S}s"`, `errorCode: "codex_output_inactivity_watchdog"`. With
`errorMessage` set and `timedOut: false`, `heartbeat.ts:5860` maps the
run to `outcome === "failed"` (not `cancelled`)
- `packages/adapters/codex-local/src/index.ts` — `agentConfigurationDoc`
documents `outputInactivityTimeoutMs` (number ms; `null` disables;
non-positive falls back to default with a warning log at spawn)
- `packages/adapters/codex-local/src/server/watchdog.test.ts` *(new)* —
13 tests covering acceptance criteria 2 and 3 plus supporting cases:
fires after silence, no-fire across 12× (threshold − 1s) cycles,
multi-event chunks, non-JSON ignoring, single-fire idempotency,
formatter shape, full resolution table, `null` → disabled
-
`packages/adapters/codex-local/src/server/watchdog.integration.test.ts`
*(new)* — real Node subprocess that prints one JSONL event then sleeps;
`runChildProcess` reaps it within `threshold + 6s`; signal is SIGTERM or
SIGKILL; `parsedEventCount === 1` (acceptance criteria 1 and 4)

## Verification

```
$ pnpm --filter @paperclipai/adapter-codex-local typecheck
> tsc --noEmit
# clean

$ pnpm --filter @paperclipai/adapter-codex-local exec vitest run
 ✓ src/server/quota-spawn-error.test.ts (1 test)
 ✓ src/server/codex-home.test.ts (3 tests)
 ✓ src/server/codex-args.test.ts (3 tests)
 ✓ src/server/watchdog.test.ts (13 tests)
 ✓ src/server/parse.test.ts (9 tests)
 ✓ src/server/execute.remote.test.ts (4 tests)
 ✓ src/ui/parse-stdout.test.ts (3 tests)
 ✓ src/ui/build-config.test.ts (1 test)
 ✓ src/server/watchdog.integration.test.ts (1 test)

 Test Files  9 passed (9)
      Tests  38 passed (38)
```

Acceptance criteria check:

1.  Simulated child emits one event then sleeps → killed at threshold;
result `errorMessage` matches `watchdog: no codex output for {N}m {S}s`
(`watchdog.integration.test.ts`)
2.  Child emits events every (threshold − 1s) → not killed
(`watchdog.test.ts: "does not fire when events arrive every (threshold -
1s)"`)
3.  `outputInactivityTimeoutMs: null` disables the watchdog
(`resolveCodexInactivityTimeout` returns `disabled`; `execute.ts` skips
watchdog construction and logs a startup warning)
4.  Real subprocess reaped well within `threshold + 6s` — 250 ms
threshold, 290 ms wall clock in CI
5.  Adapter-level fault → `outcome === "failed"` per
`heartbeat.ts:5860`. NEE-79-class hangs <1h get caught at the adapter,
not the platform-level safety net


Post-rebase verification (head `2a8703fab`, rebased onto master
`69a368ed5`):

```
$ pnpm --filter @paperclipai/adapter-codex-local typecheck   # clean
$ pnpm --filter @paperclipai/adapter-codex-local exec vitest run
 Test Files  11 passed (11)
      Tests  64 passed (64)
```

## Risks

- Low risk. Behind a default-on watchdog that only fires after 7m of
zero parsed JSON events. Operators can disable it with
`outputInactivityTimeoutMs: null` for known-slow tasks
- The kill path reuses the same `process.kill(-pgid, signal)` pattern
that `runChildProcess` already uses for its terminal-result cleanup, so
signal semantics match the existing code path
- `timedOut: false` is preserved on watchdog fire — the platform-level
timeout outcome is unchanged, only the `failed`-vs-success
classification flips. No behavioral shift for already-failing runs
- Sandbox/SSH execution targets: the watchdog fires and emits the
structured log, but the kill is best-effort because remote pids aren't
owned by this process. The platform-level 1h safety net still applies.
Out of scope for NEE-81 by design

## Model Used

- Provider: Anthropic Claude
- Model: claude-opus-4-7
- Mode: Claude Code (extended thinking, tool use)

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (N/A — no UI changes)
- [x] I have updated relevant documentation to reflect my changes
(`agentConfigurationDoc` for `outputInactivityTimeoutMs`)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (re-running on the rebased head)
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(P2 addressed; review threads resolved)
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Neeraj Kumar Singh <b.nirajkumarsingh@hotmail.com>
Co-authored-by: Devin Foley <devin@paperclip.ing>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 20:06:27 -07:00
Devin Foley 5ebbe67ebd fix(workspace-runtime): base fresh worktrees on origin/master and refresh unstarted reuses (#8412)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Agent work runs inside git worktrees created by the workspace
runtime, each branched from a configured base ref (typically the repo's
default branch)
> - When the local `master` is stale or ahead of `origin/master`
(committed but unpushed work, or local-only commits), a freshly created
worktree inherits that divergence — so an unrelated task branch silently
carries commits it never intended to touch
> - This surfaced as a docs-only task whose PR accidentally pulled in
unrelated changes from a diverged local master
> - The base for a fresh worktree should be resolved authoritatively to
the remote-tracking ref (`origin/<branch>`), and an idle/unstarted
reused worktree should be safely fast-forwarded — without ever
destroying in-progress work
> - This pull request makes both behaviors explicit in the workspace
runtime
> - The benefit is that task branches start from a clean, authoritative
base, eliminating accidental inclusion of unrelated local changes

## Linked Issues or Issue Description

This is a bug fix. No public GitHub issue exists, so describing it
inline following the Bug Report template:

**What happened**

A task intended to change only docs produced a PR that also contained
unrelated changes pulled in from `master`.

The root cause: when a new worktree is created from a configured local
branch (e.g. `master`), the worktree inherits whatever that local branch
points at. If the local `master` has committed divergence from
`origin/master` (unpushed or local-only commits), that divergence leaks
into the new task branch. The leak comes from committed
local-vs-`origin/master` ref drift, not uncommitted working-tree changes
(each worktree has its own working tree).

**Expected behavior**

A fresh worktree should be based on the authoritative `origin/master`
head so unrelated local commits never seed a task branch.

**Steps to reproduce**

1. Have a local `master` that is ahead of `origin/master` (committed but
unpushed work).
2. Create a new worktree/task branched from `master` via the workspace
runtime.
3. Open a PR from that branch — it carries the unrelated local commits.

**Deployment mode**

Self-hosted / local workspace runtime.

## What Changed

- Fresh worktrees now resolve their base ref authoritatively: a
configured local branch (e.g. `"master"`) is mapped to its
`origin/<branch>` remote-tracking counterpart so unpushed/ahead local
commits can never seed a task branch. Remote-tracking refs, SHAs, and
tags are used verbatim; an unset/`HEAD` base falls back to the detected
default branch. The resolved ref is recorded (`repoRef`) so downstream
drift checks stay accurate.
- If a configured local branch has no matching `origin/<branch>`, the
runtime warns and falls back to the local ref rather than failing.
- On reuse, a *provably unstarted* worktree (no commits past base +
clean tree including untracked files) is fast-forwarded to the latest
`origin/master`. Started or dirty worktrees keep the prior warn-only
behavior, so in-progress work is never reset. Only remote-tracking bases
are eligible for the refresh.

## Verification

- `cd server && npx vitest run src/__tests__/workspace-runtime.test.ts`
- 5 new tests cover: local-branch→`origin/<branch>` mapping, no-remote
fallback warning, unstarted-reuse fast-forward, and that started/dirty
worktrees are left untouched.
- Result: 65 passed. 1 pre-existing failure (`auto-detects the default
branch via symbolic-ref when origin/HEAD is set`) is unrelated to this
change and fails only due to the test host's git default-branch config
(test setup runs `git push -u origin main master` but the local default
branch is `main`); it also fails on `master`.

## Risks

- Low risk. The refresh path is intentionally conservative: it only
fast-forwards worktrees that are provably unstarted (zero commits past
base and a fully clean tree, including untracked files) and only when
the base is a remote-tracking ref. Started or dirty worktrees fall
through to the existing warn-only drift behavior, so no in-progress work
can be destroyed.
- Behavioral shift: fresh worktrees configured against a local branch
will now base on `origin/<branch>` instead of the local ref. This is the
intended fix; the only case it changes is when local and remote have
diverged.

## Model Used

Claude — `claude-opus-4-8` (extended thinking, tool use / code execution
via Claude Code).

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots (N/A — no UI changes)
- [x] I have updated relevant documentation to reflect my changes (N/A —
no doc changes needed)
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending CI run)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending review)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-06-20 18:51:28 -07:00
Devin Foley a937b89a47 fix(daytona): valid memory input in env config form + size presets (#8389)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Environments are configured through a JSON-schema-driven form
(`JsonSchemaForm`), and each sandbox provider (here, Daytona) supplies a
manifest describing its fields
> - In the Daytona environment config form, the "memory" field rendered
as a free-text input; on save the value round-tripped to `0`, producing
a server-side "number must be greater than zero" error even when the
user typed a valid number like `2`
> - Rather than patch the free-text input, memory is now a true dropdown
of the sandbox sizes Daytona actually supports, so an invalid value can
no longer be typed or coerced
> - The dropdown leads with a blank "None" row that is selected by
default (meaning "not configured — use Daytona's defaults"); `0` is
never offered because it is not a valid configuration
> - The benefit is that operators pick a valid memory size from a
constrained list, it saves correctly as an integer, and leaving it unset
cleanly omits the field instead of submitting `0`

## Linked Issues or Issue Description

No public GitHub issue exists; describing the bug inline per the bug
report template.

### What happened?

In the Daytona environment configuration form, the memory field is a
free-text input labelled "gigabytes of RAM". Entering a value such as
`2` and saving coerced the value to `0`, and the server rejected the
save with "the number must be greater than zero". There was no way to
enter a valid memory size through the form.

### Expected behavior

Selecting a valid memory size (e.g. `2`) keeps that value and saves
cleanly as an integer. Leaving memory unset is valid and submits no
value (Daytona defaults apply). `0` is never selectable.

### Steps to reproduce

1. Open the environment configuration form for a Daytona sandbox
provider.
2. In the memory field, type `2`.
3. Click Save.
4. Observe the value becomes `0` and the form errors with "the number
must be greater than zero".

## What Changed

- Daytona manifest: `memory` is now an `enum` of the supported sandbox
sizes `[1, 2, 4, 8]` (GiB). It stays optional, so "not configured"
remains valid. `0` is not in the list.
- `JsonSchemaForm` `EnumField`: optional enums now render a leading
blank **None** row that is selected by default when no value is set,
letting the user express "not configured" and clear a previous selection
(Radix `Select` forbids an empty-string item value, so the unset state
maps to a sentinel that translates back to `undefined`).
- `JsonSchemaForm` `EnumField`: when every enum option is numeric, the
selected value is coerced back to a number on change so the payload
keeps the schema's integer type (a stringified `"2"` would otherwise
fail server-side integer validation — this is what fixes the original
bug for the dropdown path).
- Tests: added `EnumField` coverage in `JsonSchemaForm.test.tsx` (blank
row present, no `0`, blank selected by default, numeric coercion, blank
→ unset) and Daytona manifest coverage in the plugin test (memory enum
is `[1,2,4,8]`, excludes `0`, stays optional).

## Verification

- `cd ui && npx vitest run src/components/JsonSchemaForm.test.tsx
src/pages/CompanyEnvironments.test.tsx` → 16/16 passing (14 + 2).
- `vitest run` in `packages/plugins/sandbox-providers/daytona` → 19/19
passing (incl. 3 new manifest tests).
- `cd ui && npx tsc --noEmit` → clean.
- Behavior: the memory field now renders as a dropdown showing **None /
1 / 2 / 4 / 8**, with **None** selected by default. Picking `2` stores
the integer `2`; picking **None** clears the field so it is omitted from
the payload (no `0`, no "must be greater than zero" error).

## Risks

- Low risk. The blank-row + numeric-coercion changes live in
`EnumField`. The blank row is only added for **optional** enums
(required enums are unaffected); numeric coercion only triggers when
every option is numeric, so existing string enums (`egressMode`,
`backend`, `sessionStrategy`) are unchanged. The manifest change is
scoped to the Daytona provider.

## Model Used

Claude (Anthropic), Opus-class model, used via the Paperclip agent
workflow (author + reviewer agents) with tool use and code execution.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above (none found)
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links
- [x] My branch name describes the change and contains no internal
Paperclip ticket id
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-06-20 17:58:47 -07:00
Devin Foley 254b01d2af docs: drop contributor screenshot requirement in favor of cutter.sh bot (#8409)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Contributors follow `CONTRIBUTING.md` and the PR template when
opening pull requests
> - Those docs still tell contributors to manually capture and attach
before/after screenshots for UI changes
> - That requirement is now redundant: the `cutter.sh` bot automatically
captures and posts before/after UI screenshots to every PR
> - Asking contributors to also do it by hand adds friction and produces
duplicate screenshots
> - This pull request removes the manual screenshot obligation from
`CONTRIBUTING.md` and the PR template, and documents that `cutter.sh`
handles it
> - The benefit is a clearer, lower-friction contribution flow that
matches how screenshots actually get posted today

## Linked Issues or Issue Description

No public GitHub issue. Underlying change:

Our contributor docs and PR template instructed contributors to attach
before/after screenshots for any UI change. We now run a bot
(`cutter.sh`) that automatically captures and posts before/after UI
screenshots to the PR, so the manual step is no longer needed. This
change removes the manual requirement and instead documents that the bot
handles screenshots, asking contributors to simply describe the visible
change.

## What Changed

- `CONTRIBUTING.md`: removed the "Before / After screenshots"
requirement from the Path 2 PR checklist and from the "Writing a Good PR
message" section; both now explain that the `cutter.sh` bot posts UI
screenshots automatically and ask contributors to describe the visible
change instead.
- `.github/PULL_REQUEST_TEMPLATE.md`: updated the Verification guidance
to note that `cutter.sh` posts before/after screenshots automatically,
and removed the "include before/after screenshots" item from the
checklist.
- Issue templates were intentionally left unchanged — they only offer
screenshots as an optional "if helpful" field, never a requirement.

## Verification

- Docs-only change; no code paths affected.
- `grep -rin "screenshot" CONTRIBUTING.md .github/` confirms no
screenshot *requirement* remains — only the new `cutter.sh` wording and
the optional "if helpful" fields in issue templates.
- Render `CONTRIBUTING.md` and the PR template to confirm the wording
reads cleanly.

## Risks

Low risk — documentation-only change. No build, runtime, or migration
impact.

## Model Used

Claude (Anthropic), model `claude-opus-4-8`, extended thinking with tool
use.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass (docs-only; no tests apply)
- [ ] I have added or updated tests where applicable (N/A — docs-only)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green (pending CI)
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
(pending review)
- [x] I will address all Greptile and reviewer comments before
requesting merge
2026-06-20 17:58:14 -07:00
Devin Foley 67ca8287e8 fix(codex-local): seed managed auth into isolated homes (#8403)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - The `codex_local` adapter isolates local Codex runs by assigning
managed `CODEX_HOME` state per company and per agent
> - PR #8272 tightened that isolation, but it left a gap: once
`CODEX_HOME` became explicit, the adapter treated it like a user-managed
override and skipped auth seeding
> - That meant newly isolated agents could launch with no usable
`auth.json`, hit OpenAI unauthenticated, and fail with `401 Missing
bearer`
> - Users who had already persisted one of those broken managed homes
could remain stranded even after config changes unless Paperclip
repaired the home itself
> - This pull request teaches Paperclip to seed managed homes correctly,
backfill already-stranded managed homes on startup, and reject
credential-less managed homes before they reach the provider
> - The benefit is that affected managed `codex_local` agents recover
automatically after upgrade and restart, without manual `CODEX_HOME`
surgery

## Linked Issues or Issue Description

- Fixes #497
- Refs #5028
- Related PR: #8272
- Related PR: #8399

## What Changed

- Distinguished Paperclip-managed `CODEX_HOME` paths from genuine
external overrides and always seeded auth into managed homes, even when
`CODEX_HOME` is explicit in config.
- Wrote API-key-backed `auth.json` files for managed homes when
`OPENAI_API_KEY` is configured, otherwise symlinked the shared Codex
auth for subscription/OAuth flows.
- Added a startup reconciliation pass that backfills already-isolated
managed homes created by the broken release so upgrade plus restart
repairs stranded agents automatically.
- Preserved previously resolved API-key auth when the stored
`OPENAI_API_KEY` binding is secret-backed and startup cannot resolve the
secret value directly.
- Hardened the managed-home preflight to require a credential-bearing
`auth.json`, not just file presence, and documented the recovery
behavior.
- Added regression tests covering managed-home seeding, fail-fast
behavior, and server-side startup reconciliation.

## Verification

```bash
pnpm exec vitest run packages/adapters/codex-local/src/server/codex-home.test.ts packages/adapters/codex-local/src/server/execute.auth.test.ts server/src/__tests__/codex-auth-reconciliation.test.ts server/src/__tests__/codex-local-execute.test.ts server/src/__tests__/server-startup-feedback-export.test.ts
pnpm --filter @paperclipai/adapter-codex-local typecheck
pnpm --filter @paperclipai/server typecheck
pnpm check:tokens
pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts
```

GitHub Actions `PR` workflow is green on latest head `e5b1e08d4`,
including policy, typecheck, test shards, build, e2e, serialized server
suites, and canary dry run. Greptile Review is green on latest head with
0 comments added. No UI changes.

## Risks

- Startup reconciliation now mutates persisted managed Codex homes at
boot. Risk is low because it only touches Paperclip-managed
company/agent home paths and no-ops when a home already has usable auth.
- Genuine external `CODEX_HOME` overrides remain intentionally
self-managed, so those users still own repair steps inside their custom
home.
- Hosts with neither shared Codex auth nor an explicit per-agent API key
now fail earlier with a clearer adapter error instead of surfacing a
downstream `401`, which changes timing but not capability.

## Model Used

- OpenAI Codex, GPT-5-based coding agent in a local Codex session; exact
served model ID/context window were not exposed to the session. Tool use
and code execution were enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change and contains no internal
Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-20 16:17:54 -07:00
72 changed files with 4105 additions and 1015 deletions
+1 -2
View File
@@ -49,7 +49,7 @@
<!--
How can a reviewer confirm this works? Include test commands, manual
steps, or both. For UI changes, include before/after screenshots.
steps, or both.
-->
-
@@ -91,7 +91,6 @@
- [ ] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details
- [ ] I have run tests locally and they pass
- [ ] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after screenshots
- [ ] I have updated relevant documentation to reflect my changes
- [ ] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
-3
View File
@@ -36,7 +36,6 @@ These almost always get merged quickly when they're clean.
→ Share rough ideas / approach
- Once there's rough agreement, build it
- In your PR include:
- Before / After screenshots (or short video if UI/behavior change)
- Clear description of what & why
- Proof it works (manual testing notes)
- All tests passing and CI green
@@ -180,8 +179,6 @@ Then have the rest of your normal PR message after the Thinking Path.
This should include details about what you did, why you did it, why it matters & the benefits, how we can verify it works, and any risks.
Please include screenshots if possible if you have a visible change. (use something like the [agent-browser skill](https://github.com/vercel-labs/agent-browser/blob/main/skills/agent-browser/SKILL.md) or similar to take screenshots). Ideally, you include before and after screenshots.
Questions? Just ask in #dev — we're happy to help.
Happy hacking!
+4 -5
View File
@@ -502,7 +502,7 @@ function registerAgentSkillCommands(skills: Command): void {
addCommonClientOptions(
agent
.command("sync")
.description("Replace an agent's non-required desired company skills and sync runtime state")
.description("Replace an agent's desired company skills and sync runtime state")
.argument("<agentRef>", "Agent ID or shortname/url-key")
.option("--skill <skillRef>", "Desired company skill ID, key, or slug; may be repeated", collectOptionValue, [] as string[])
.action(async (agentRef: string, opts: AgentSkillSyncOptions) => {
@@ -535,7 +535,7 @@ function registerAgentSkillCommands(skills: Command): void {
addCommonClientOptions(
agent
.command("clear")
.description("Clear an agent's non-required desired company skills and sync runtime state")
.description("Clear an agent's desired company skills and sync runtime state")
.argument("<agentRef>", "Agent ID or shortname/url-key")
.option("--yes", "Confirm clear without prompting", false)
.action(async (agentRef: string, opts: ConfirmedSkillOptions) => {
@@ -544,7 +544,7 @@ function registerAgentSkillCommands(skills: Command): void {
const agentRow = await resolveAgent(ctx, agentRef);
await confirmDangerousAction(
opts.yes,
`Clear non-required desired company skills for "${agentRow.name}" (${agentRow.id})?`,
`Clear desired company skills for "${agentRow.name}" (${agentRow.id})?`,
);
const snapshot = await ctx.api.post<AgentSkillSnapshot>(
`/api/agents/${encodeURIComponent(agentRow.id)}/skills/sync`,
@@ -555,7 +555,7 @@ function registerAgentSkillCommands(skills: Command): void {
return;
}
console.log(
`Desired company skills cleared for ${agentRow.name} (${agentRow.id}); required Paperclip skills remain server-enforced.`,
`Desired company skills cleared for ${agentRow.name} (${agentRow.id}).`,
);
printAgentSkillSnapshot(snapshot, agentRow);
} catch (err) {
@@ -911,7 +911,6 @@ function printAgentSkillSnapshot(snapshot: AgentSkillSnapshot | null, agent: Age
runtimeName: entry.runtimeName,
desired: entry.desired,
managed: entry.managed,
required: entry.required ?? false,
state: entry.state,
origin: entry.origin,
detail: entry.detail,
+10
View File
@@ -45,6 +45,16 @@ Paperclip currently applies that only when the selected model is `gpt-5.4`. On o
When Paperclip is running inside a managed worktree instance (`PAPERCLIP_IN_WORKTREE=true`), the adapter instead uses a worktree-isolated `CODEX_HOME` under the Paperclip instance so Codex skills, sessions, logs, and other runtime state do not leak across checkouts. It seeds that isolated home from the user's main Codex home for shared auth/config continuity.
### Per-agent isolation and auth seeding
For `codex_local` agents the server isolation guard pins each agent to a per-agent home (`<instance>/companies/<companyId>/agents/<agentId>/codex-home`) and sets `OPENAI_API_KEY=""` so an agent can never spend against the host API key or share another agent's Codex state.
A managed home is created empty, so the adapter must provision auth into it before launching Codex — otherwise the agent runs with zero credentials and the provider returns `401 Missing bearer`. The seeding contract:
- **Managed homes** (the default home and any configured `CODEX_HOME` under the company tree) are always seeded: the ChatGPT-subscription `auth.json` is symlinked from the host Codex home, or, when a per-agent `OPENAI_API_KEY` is configured, an API-key `auth.json` is written instead.
- **Genuine external overrides** (a `CODEX_HOME` outside the Paperclip-managed company tree) are treated as self-managed and are never seeded or overwritten.
- **Fail-fast guard:** if a managed home ends up with no usable `auth.json` and no configured API key, the run fails with an explicit `adapter_failed` ("no Codex credentials provisioned for managed home …") rather than emitting an unauthenticated request.
## Manual Local CLI
For manual local CLI usage outside heartbeat runs (for example running as `codexcoder` directly), use:
@@ -22,7 +22,10 @@ interface SpawnRunnerHandle {
// A runner that actually executes the shell scripts (piping stdin through a real
// pipe so multi-MB payloads work) and replays stdout through onLog in several
// chunks so the streaming readFile byte-counter is exercised.
function makeSpawnRunner(options: { supportsSingleStreamStdinProgress?: boolean } = {}): SpawnRunnerHandle {
function makeSpawnRunner(options: {
supportsSingleStreamStdinProgress?: boolean;
maxStdoutBytes?: number;
} = {}): SpawnRunnerHandle {
const calls: Array<{ command: string; args?: string[]; cwd?: string; stdin?: string }> = [];
const runner: CommandManagedRuntimeRunner = {
supportsSingleStreamStdinProgress: options.supportsSingleStreamStdinProgress,
@@ -48,6 +51,21 @@ function makeSpawnRunner(options: { supportsSingleStreamStdinProgress?: boolean
resolve({ exitCode: 127, signal: null, timedOut: false, stdout, stderr, pid: null, startedAt });
});
child.on("close", async (code) => {
if (
options.maxStdoutBytes != null &&
Buffer.byteLength(stdout, "utf8") > options.maxStdoutBytes
) {
resolve({
exitCode: 1,
signal: null,
timedOut: false,
stdout,
stderr: `stdout exceeded ${options.maxStdoutBytes} bytes`,
pid: child.pid ?? null,
startedAt,
});
return;
}
if (input.onLog && stdout.length > 0) {
const chunkSize = Math.max(1, Math.ceil(stdout.length / 4));
for (let offset = 0; offset < stdout.length; offset += chunkSize) {
@@ -75,6 +93,26 @@ function toArrayBuffer(buffer: Buffer): ArrayBuffer {
return Uint8Array.from(buffer).buffer;
}
async function withBase64StringByteLimit<T>(limitBytes: number, fn: () => Promise<T>): Promise<T> {
const originalToString = Buffer.prototype.toString;
Buffer.prototype.toString = function patchedToString(
this: Buffer,
encoding?: BufferEncoding,
start?: number,
end?: number,
) {
if (encoding === "base64" && this.byteLength > limitBytes) {
throw new Error(`test guard: attempted to base64-encode ${this.byteLength} bytes at once`);
}
return originalToString.call(this, encoding, start, end);
} as typeof Buffer.prototype.toString;
try {
return await fn();
} finally {
Buffer.prototype.toString = originalToString;
}
}
describe("command managed runtime", () => {
const cleanupDirs: string[] = [];
@@ -243,10 +281,12 @@ describe("command managed runtime", () => {
const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
const progress: Array<{ done: number; total: number | null }> = [];
await client.writeFile(remotePath, toArrayBuffer(payload), {
onProgress: (done, total) => {
progress.push({ done, total });
},
await withBase64StringByteLimit(4 * 1024 * 1024, async () => {
await client.writeFile(remotePath, toArrayBuffer(payload), {
onProgress: (done, total) => {
progress.push({ done, total });
},
});
});
// Exactly one upload process: O(1) round-trips regardless of payload size.
@@ -288,6 +328,9 @@ describe("command managed runtime", () => {
// Provider-backed sandbox runners cannot surface mid-flight progress for a
// single stdin RPC, so we intentionally use several large append commands.
expect(calls.length).toBeGreaterThan(2);
const stdinCalls = calls.filter((call) => call.stdin != null);
expect(stdinCalls.length).toBeGreaterThan(2);
expect(stdinCalls.every((call) => Buffer.byteLength(call.stdin ?? "", "utf8") <= 4.1 * 1024 * 1024)).toBe(true);
expect(progress.length).toBeGreaterThan(2);
for (let i = 1; i < progress.length; i++) {
expect(progress[i].done).toBeGreaterThanOrEqual(progress[i - 1].done);
@@ -295,16 +338,41 @@ describe("command managed runtime", () => {
expect(progress.at(-1)).toEqual({ done: payload.length, total: payload.length });
});
it("streams a download through onLog and reports monotonic byte progress to the total", async () => {
it("falls back to bounded chunks when the runner does not explicitly opt in", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-command-write-fallback-no-progress-"));
cleanupDirs.push(rootDir);
const remotePath = path.join(rootDir, "nested", "payload.bin");
const payload = Buffer.alloc(12 * 1024 * 1024);
for (let i = 0; i < payload.length; i++) payload[i] = i % 256;
const { runner, calls } = makeSpawnRunner();
const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
await withBase64StringByteLimit(4 * 1024 * 1024, async () => {
await client.writeFile(remotePath, toArrayBuffer(payload));
});
const written = await readFile(remotePath);
expect(written.equals(payload)).toBe(true);
// A runner that doesn't mark single-stream stdin support must avoid passing
// the whole base64 archive as one string, so we expect multiple append calls.
const stdinCalls = calls.filter((call) => call.stdin != null);
expect(stdinCalls.length).toBeGreaterThan(1);
expect(stdinCalls.every((call) => Buffer.byteLength(call.stdin ?? "", "utf8") <= 4.1 * 1024 * 1024)).toBe(true);
});
it("downloads in bounded stdout chunks and reports monotonic byte progress to the total", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-command-read-"));
cleanupDirs.push(rootDir);
const remotePath = path.join(rootDir, "download.bin");
const payload = Buffer.alloc(512 * 1024);
const payload = Buffer.alloc(7 * 1024 * 1024);
for (let i = 0; i < payload.length; i++) payload[i] = (i * 7) % 256;
await writeFile(remotePath, payload);
const { runner } = makeSpawnRunner();
const { runner, calls } = makeSpawnRunner({ maxStdoutBytes: 5 * 1024 * 1024 });
const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
const progress: Array<{ done: number; total: number | null }> = [];
@@ -316,7 +384,10 @@ describe("command managed runtime", () => {
expect(Buffer.from(bytes as ArrayBuffer).equals(payload)).toBe(true);
// The runner replays stdout in several chunks, so progress arrives in steps.
// The old single `base64 < file` path would exceed the runner's stdout cap.
// The bounded path reads with several small `dd | base64` commands instead.
expect(calls.some((call) => call.args?.join(" ").includes("base64 <"))).toBe(false);
expect(calls.filter((call) => call.args?.join(" ").includes("dd if=")).length).toBeGreaterThan(1);
expect(progress.length).toBeGreaterThan(1);
for (let i = 1; i < progress.length; i++) {
expect(progress[i].done).toBeGreaterThanOrEqual(progress[i - 1].done);
@@ -56,6 +56,12 @@ const REMOTE_WRITE_SINGLE_STREAM_MAX_BASE64_BYTES = 96 * 1024 * 1024;
// Fallback chunk size (base64 bytes). Kept a multiple of 4 so each chunk is a
// self-contained base64 unit that decodes cleanly on its own.
const REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE = 4 * 1024 * 1024;
const REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE = (REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE / 4) * 3;
const REMOTE_READ_CHUNK_BYTES = REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE;
function base64EncodedLength(byteLength: number): number {
return Math.ceil(byteLength / 3) * 4;
}
function toBuffer(bytes: Buffer | Uint8Array | ArrayBuffer): Buffer {
if (Buffer.isBuffer(bytes)) return bytes;
@@ -104,11 +110,10 @@ export function createCommandManagedRuntimeClient(input: {
writeFile: async (remotePath, bytes, options) => {
const buffer = toBuffer(bytes);
const total = buffer.byteLength;
const body = buffer.toString("base64");
const encodedLength = base64EncodedLength(total);
const remoteDir = path.posix.dirname(remotePath);
const remoteTempPath = `${remotePath}.paperclip-upload`;
const canUseSingleStreamProgressPath =
!options?.onProgress || input.runner.supportsSingleStreamStdinProgress === true;
const canUseSingleStreamProgressPath = input.runner.supportsSingleStreamStdinProgress === true;
// Primary path: a single round-trip. Stream the entire base64 body to one
// `base64 -d` process via stdin, decode straight into a temp file, then
@@ -116,9 +121,10 @@ export function createCommandManagedRuntimeClient(input: {
// one `printf >> tmpfile` shell round-trip per 32 KB — thousands of serial
// processes for a large workspace — with exactly one process.
if (
body.length <= REMOTE_WRITE_SINGLE_STREAM_MAX_BASE64_BYTES &&
encodedLength <= REMOTE_WRITE_SINGLE_STREAM_MAX_BASE64_BYTES &&
canUseSingleStreamProgressPath
) {
const body = buffer.toString("base64");
await options?.onProgress?.(0, total);
await runShell(
`mkdir -p ${shellQuote(remoteDir)} && ` +
@@ -139,62 +145,50 @@ export function createCommandManagedRuntimeClient(input: {
`mkdir -p ${shellQuote(remoteDir)} && ` +
`rm -f ${shellQuote(remoteTempPath)} && : > ${shellQuote(remoteTempPath)}`,
);
for (let offset = 0; offset < body.length; offset += REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE) {
const chunk = body.slice(offset, offset + REMOTE_WRITE_FALLBACK_BASE64_CHUNK_SIZE);
for (let offset = 0; offset < total; offset += REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE) {
const end = Math.min(total, offset + REMOTE_WRITE_FALLBACK_DECODED_CHUNK_SIZE);
const chunk = buffer.subarray(offset, end).toString("base64");
await runShell(`base64 -d >> ${shellQuote(remoteTempPath)}`, { stdin: chunk });
const decodedSoFar = Math.min(total, Math.floor(((offset + chunk.length) * 3) / 4));
await options?.onProgress?.(decodedSoFar, total);
await options?.onProgress?.(end, total);
}
await runShell(`mv -f ${shellQuote(remoteTempPath)} ${shellQuote(remotePath)}`);
await options?.onProgress?.(total, total);
},
readFile: async (remotePath, options) => {
// Decoded file size up front so download progress can be a percentage.
// Only paid when a progress hook is attached.
let totalBytes: number | null = null;
if (options?.onProgress) {
const sizeResult = await runShell(`wc -c < ${shellQuote(remotePath)}`);
const parsed = Number.parseInt(sizeResult.stdout.trim(), 10);
totalBytes = Number.isFinite(parsed) && parsed >= 0 ? parsed : null;
// Chunked reads intentionally query the remote size first, even without
// a progress sink, so each sandbox RPC stays bounded and truncation is
// detected without materializing the whole file as one stdout string.
const sizeResult = await runShell(`wc -c < ${shellQuote(remotePath)}`);
const totalBytes = Number.parseInt(sizeResult.stdout.trim(), 10);
if (!Number.isFinite(totalBytes) || totalBytes < 0) {
throw new Error(`Could not determine remote file size for ${remotePath}`);
}
// Stream the remote `base64` stdout through a byte counter, decoding each
// 4-char-aligned slice incrementally rather than buffering the whole
// response as one string. Falls back to the buffered result when the
// runner does not surface incremental stdout.
// Read in bounded remote chunks so the runner never has to materialize a
// single base64 stdout string for the whole archive. The client API still
// returns the decoded file as a Buffer, but every command result stays
// small enough for provider-backed sandbox RPCs.
const decodedChunks: Buffer[] = [];
let b64Remainder = "";
let decodedSoFar = 0;
let streamed = false;
const result = await runShell(`base64 < ${shellQuote(remotePath)}`, {
onLog: async (stream, chunk) => {
if (stream !== "stdout") return;
streamed = true;
const data = b64Remainder + chunk.replace(/\s+/g, "");
const alignedLen = data.length - (data.length % 4);
if (alignedLen > 0) {
const decoded = Buffer.from(data.slice(0, alignedLen), "base64");
decodedChunks.push(decoded);
decodedSoFar += decoded.byteLength;
}
b64Remainder = data.slice(alignedLen);
if (options?.onProgress) {
const done = totalBytes != null ? Math.min(totalBytes, decodedSoFar) : decodedSoFar;
await options.onProgress(done, totalBytes);
}
},
});
let out: Buffer;
if (streamed) {
if (b64Remainder.length > 0) {
decodedChunks.push(Buffer.from(b64Remainder, "base64"));
}
out = Buffer.concat(decodedChunks);
} else {
out = Buffer.from(result.stdout.replace(/\s+/g, ""), "base64");
if (totalBytes === 0) {
await options?.onProgress?.(0, 0);
return Buffer.alloc(0);
}
await options?.onProgress?.(out.byteLength, totalBytes ?? out.byteLength);
for (let chunkIndex = 0; decodedSoFar < totalBytes; chunkIndex++) {
const result = await runShell(
`dd if=${shellQuote(remotePath)} bs=${REMOTE_READ_CHUNK_BYTES} skip=${chunkIndex} count=1 2>/dev/null | base64`,
);
const chunk = Buffer.from(result.stdout.replace(/\s+/g, ""), "base64");
if (chunk.byteLength === 0) break;
decodedChunks.push(chunk);
decodedSoFar += chunk.byteLength;
await options?.onProgress?.(Math.min(decodedSoFar, totalBytes), totalBytes);
}
const out = Buffer.concat(decodedChunks);
if (out.byteLength !== totalBytes) {
throw new Error(`Remote file read was truncated for ${remotePath}: ${out.byteLength}/${totalBytes} bytes`);
}
await options?.onProgress?.(out.byteLength, totalBytes);
return out;
},
listFiles: async (remotePath) => {
@@ -0,0 +1,28 @@
export function isRelativePathOrDescendant(relative: string, candidate: string): boolean {
return relative === candidate || relative.startsWith(`${candidate}/`);
}
function pathContainsSegmentOrDescendant(relative: string, segment: string): boolean {
return relative === segment ||
relative.startsWith(`${segment}/`) ||
relative.endsWith(`/${segment}`) ||
relative.includes(`/${segment}/`);
}
export function excludePatternMatches(relative: string, pattern: string): boolean {
if (pattern.startsWith("*/") && pattern.endsWith("/*")) {
return pathContainsSegmentOrDescendant(relative, pattern.slice(2, -2));
}
if (pattern.startsWith("*/")) {
return pathContainsSegmentOrDescendant(relative, pattern.slice(2));
}
if (pattern.endsWith("/*")) {
const base = pattern.slice(0, -2);
return relative.startsWith(`${base}/`);
}
return isRelativePathOrDescendant(relative, pattern);
}
export function shouldExcludePath(relative: string, exclude: readonly string[]): boolean {
return exclude.some((entry) => excludePatternMatches(relative, entry));
}
@@ -0,0 +1,128 @@
import { execFile as execFileCallback } from "node:child_process";
import { lstat, mkdir, mkdtemp, readFile, rm, stat, writeFile } from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { afterEach, describe, expect, it } from "vitest";
import {
buildRemoteGitDeltaBundleScript,
createImportedGitRef,
createRemoteGitExportRef,
deleteLocalGitRef,
fetchGitBundleIntoLocalRef,
readGitWorkspaceSnapshot,
runLocalGit,
withShallowGitWorkspaceClone,
} from "./git-workspace-sync.js";
const execFile = promisify(execFileCallback);
async function git(cwd: string, args: string[]): Promise<string> {
return (await runLocalGit(cwd, args)).stdout.trim();
}
describe("git workspace sync", () => {
const cleanupDirs: string[] = [];
afterEach(async () => {
while (cleanupDirs.length > 0) {
const dir = cleanupDirs.pop();
if (!dir) continue;
await rm(dir, { recursive: true, force: true }).catch(() => undefined);
}
});
async function createRepo(rootDir: string): Promise<string> {
const repo = path.join(rootDir, "repo");
await mkdir(repo, { recursive: true });
await git(repo, ["init"]);
await git(repo, ["checkout", "-b", "main"]);
await git(repo, ["config", "user.name", "Paperclip Test"]);
await git(repo, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(repo, "tracked.txt"), "base\n", "utf8");
await git(repo, ["add", "tracked.txt"]);
await git(repo, ["commit", "-m", "base"]);
return repo;
}
it("creates a shallow standalone clone from the local HEAD snapshot", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-git-sync-"));
cleanupDirs.push(rootDir);
const repo = await createRepo(rootDir);
const baseHead = await git(repo, ["rev-parse", "HEAD"]);
await rm(path.join(repo, "tracked.txt"));
const snapshot = await readGitWorkspaceSnapshot(repo);
expect(snapshot).toMatchObject({
headCommit: baseHead,
branchName: "main",
deletedPaths: ["tracked.txt"],
});
await withShallowGitWorkspaceClone({
localDir: repo,
snapshot: snapshot!,
}, async (cloneDir) => {
expect((await lstat(path.join(cloneDir, ".git"))).isDirectory()).toBe(true);
await expect(readFile(path.join(cloneDir, ".git", "shallow"), "utf8")).resolves.toContain(baseHead);
expect(await git(cloneDir, ["rev-list", "--count", "HEAD"])).toBe("1");
expect(await git(cloneDir, ["branch", "--show-current"])).toBe("main");
await expect(readFile(path.join(cloneDir, "tracked.txt"), "utf8")).resolves.toBe("base\n");
});
});
it("builds thin git delta bundles relative to the imported base", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-git-delta-"));
cleanupDirs.push(rootDir);
const repo = await createRepo(rootDir);
const baseHead = await git(repo, ["rev-parse", "HEAD"]);
const snapshot = await readGitWorkspaceSnapshot(repo);
expect(snapshot).not.toBeNull();
await withShallowGitWorkspaceClone({
localDir: repo,
snapshot: snapshot!,
}, async (remoteDir) => {
const emptyBundle = path.join(rootDir, "empty.bundle");
await execFile("sh", ["-c", buildRemoteGitDeltaBundleScript({
remoteDir,
baseSha: baseHead,
exportRef: createRemoteGitExportRef("test"),
bundlePath: emptyBundle,
})]);
expect((await stat(emptyBundle)).size).toBe(0);
await git(remoteDir, ["config", "user.name", "Paperclip Remote"]);
await git(remoteDir, ["config", "user.email", "remote@paperclip.dev"]);
await writeFile(path.join(remoteDir, "tracked.txt"), "remote\n", "utf8");
await git(remoteDir, ["commit", "-am", "remote update"]);
const remoteHead = await git(remoteDir, ["rev-parse", "HEAD"]);
const deltaBundle = path.join(rootDir, "delta.bundle");
const importedRef = createImportedGitRef("test");
const exportRef = createRemoteGitExportRef("test");
try {
await execFile("sh", ["-c", buildRemoteGitDeltaBundleScript({
remoteDir,
baseSha: baseHead,
exportRef,
bundlePath: deltaBundle,
})]);
expect((await stat(deltaBundle)).size).toBeGreaterThan(0);
const importedHead = await fetchGitBundleIntoLocalRef({
localDir: repo,
bundlePath: deltaBundle,
exportRef,
importedRef,
baseSha: baseHead,
});
expect(importedHead).toBe(remoteHead);
expect(await git(repo, ["rev-list", "--count", importedRef, "--not", baseHead])).toBe("1");
} finally {
await deleteLocalGitRef({ localDir: repo, ref: importedRef });
}
});
});
});
@@ -0,0 +1,321 @@
import { randomUUID } from "node:crypto";
import { execFile } from "node:child_process";
import { promises as fs } from "node:fs";
import os from "node:os";
import path from "node:path";
export interface GitCommandResult {
stdout: string;
stderr: string;
}
export interface GitWorkspaceSnapshot {
headCommit: string;
branchName: string | null;
overlayPaths: string[];
deletedPaths: string[];
ignoredPaths: string[];
}
export const GIT_ARCHIVE_EXCLUDES = [".git", ".git/*"] as const;
function shellQuote(value: string) {
return `'${value.replace(/'/g, `'\"'\"'`)}'`;
}
export async function runLocalGit(
localDir: string,
args: string[],
options: {
timeout?: number;
maxBuffer?: number;
} = {},
): Promise<GitCommandResult> {
return await new Promise<GitCommandResult>((resolve, reject) => {
execFile(
"git",
["-C", localDir, ...args],
{
timeout: options.timeout ?? 15_000,
maxBuffer: options.maxBuffer ?? 1024 * 128,
},
(error, stdout, stderr) => {
if (error) {
reject(Object.assign(error, { stdout: stdout ?? "", stderr: stderr ?? "" }));
return;
}
resolve({
stdout: stdout ?? "",
stderr: stderr ?? "",
});
},
);
});
}
export async function readGitWorkspaceSnapshot(localDir: string): Promise<GitWorkspaceSnapshot | null> {
try {
const insideWorkTree = await runLocalGit(localDir, ["rev-parse", "--is-inside-work-tree"], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
if (insideWorkTree.stdout.trim() !== "true") {
return null;
}
const [headCommitResult, branchResult, overlayDiffResult, untrackedResult, deletedResult, ignoredResult] = await Promise.all([
runLocalGit(localDir, ["rev-parse", "HEAD"], {
timeout: 10_000,
maxBuffer: 16 * 1024,
}),
runLocalGit(localDir, ["rev-parse", "--abbrev-ref", "HEAD"], {
timeout: 10_000,
maxBuffer: 16 * 1024,
}),
runLocalGit(localDir, ["diff", "--name-only", "-z", "--diff-filter=ACMRTUXB", "HEAD", "--"], {
timeout: 10_000,
maxBuffer: 1024 * 1024,
}),
runLocalGit(localDir, ["ls-files", "--others", "--exclude-standard", "-z"], {
timeout: 10_000,
maxBuffer: 1024 * 1024,
}),
runLocalGit(localDir, ["diff", "--name-only", "-z", "--diff-filter=D", "HEAD", "--"], {
timeout: 10_000,
maxBuffer: 256 * 1024,
}),
runLocalGit(localDir, ["status", "--ignored", "--porcelain=v1", "-z", "--untracked-files=normal"], {
timeout: 10_000,
maxBuffer: 1024 * 1024,
}),
]);
const branchName = branchResult.stdout.trim();
const splitNul = (value: string) => value.split("\0").map((entry) => entry.trim()).filter(Boolean);
return {
headCommit: headCommitResult.stdout.trim(),
branchName: branchName && branchName !== "HEAD" ? branchName : null,
overlayPaths: [...new Set([...splitNul(overlayDiffResult.stdout), ...splitNul(untrackedResult.stdout)])]
.sort((left, right) => left.localeCompare(right)),
deletedPaths: [...new Set(splitNul(deletedResult.stdout))]
.sort((left, right) => left.localeCompare(right)),
ignoredPaths: splitNul(ignoredResult.stdout)
.filter((entry) => entry.startsWith("!! "))
.map((entry) => entry.slice(3).replace(/\/+$/, ""))
.filter(Boolean)
.sort((left, right) => left.localeCompare(right)),
};
} catch {
return null;
}
}
export async function withShallowGitWorkspaceClone<T>(
input: {
localDir: string;
snapshot: GitWorkspaceSnapshot;
},
fn: (cloneDir: string) => Promise<T>,
): Promise<T> {
const cloneDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-git-workspace-"));
const tempRef = `refs/paperclip/git-sync/import/${randomUUID()}`;
try {
await runLocalGit(input.localDir, ["update-ref", tempRef, input.snapshot.headCommit], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
await runLocalGit(cloneDir, ["init"], {
timeout: 10_000,
maxBuffer: 64 * 1024,
});
await runLocalGit(cloneDir, ["fetch", "--depth=1", input.localDir, tempRef], {
timeout: 60_000,
maxBuffer: 1024 * 1024,
});
await runLocalGit(
cloneDir,
input.snapshot.branchName
? ["checkout", "--force", "-B", input.snapshot.branchName, "FETCH_HEAD"]
: ["checkout", "--force", "--detach", "FETCH_HEAD"],
{
timeout: 60_000,
maxBuffer: 1024 * 1024,
},
);
await runLocalGit(cloneDir, ["reset", "--hard", input.snapshot.headCommit], {
timeout: 60_000,
maxBuffer: 1024 * 1024,
});
return await fn(cloneDir);
} finally {
await runLocalGit(input.localDir, ["update-ref", "-d", tempRef], {
timeout: 10_000,
maxBuffer: 16 * 1024,
}).catch(() => undefined);
await fs.rm(cloneDir, { recursive: true, force: true }).catch(() => undefined);
}
}
export function createImportedGitRef(scope = "remote"): string {
return `refs/paperclip/git-sync/imported/${scope}/${randomUUID()}`;
}
export function createRemoteGitExportRef(scope = "remote"): string {
return `refs/paperclip/git-sync/export/${scope}/${randomUUID()}`;
}
export async function deleteLocalGitRef(input: {
localDir: string;
ref: string;
}): Promise<void> {
await runLocalGit(input.localDir, ["update-ref", "-d", input.ref], {
timeout: 10_000,
maxBuffer: 16 * 1024,
}).catch(() => undefined);
}
export async function fetchGitBundleIntoLocalRef(input: {
localDir: string;
bundlePath: string;
exportRef: string;
importedRef: string;
baseSha: string;
}): Promise<string> {
const bundleSize = (await fs.stat(input.bundlePath).catch(() => null))?.size ?? 0;
if (bundleSize === 0) {
return input.baseSha;
}
await runLocalGit(input.localDir, ["fetch", "--force", input.bundlePath, `${input.exportRef}:${input.importedRef}`], {
timeout: 60_000,
maxBuffer: 1024 * 1024,
});
const importedHead = await runLocalGit(input.localDir, ["rev-parse", input.importedRef], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
return importedHead.stdout.trim();
}
export function buildRemoteGitDeltaBundleScript(input: {
remoteDir: string;
baseSha: string;
exportRef: string;
bundlePath: string;
catBundle?: boolean;
cleanupBundle?: boolean;
}): string {
const remoteDir = shellQuote(input.remoteDir);
const bundlePath = shellQuote(input.bundlePath);
const exportRef = shellQuote(input.exportRef);
const baseSha = shellQuote(input.baseSha);
const cleanupParts = [
`rm -f ${bundlePath}`,
`git -C ${remoteDir} update-ref -d ${exportRef} >/dev/null 2>&1 || true`,
];
return [
"set -e",
input.cleanupBundle ? `cleanup() { ${cleanupParts.join("; ")}; }` : "",
input.cleanupBundle ? "trap cleanup EXIT" : "",
`mkdir -p ${shellQuote(path.posix.dirname(input.bundlePath))}`,
`rm -f ${bundlePath}`,
`git -C ${remoteDir} cat-file -e ${baseSha}^{commit}`,
`commit_count=$(git -C ${remoteDir} rev-list --count HEAD --not ${baseSha})`,
'if [ "$commit_count" -gt 0 ]; then',
` git -C ${remoteDir} update-ref ${exportRef} HEAD`,
` git -C ${remoteDir} bundle create ${bundlePath} ${exportRef} --not ${baseSha} >/dev/null`,
"else",
` : > ${bundlePath}`,
"fi",
input.catBundle ? `cat ${bundlePath}` : "",
].filter(Boolean).join("\n");
}
export async function integrateImportedGitHead(input: {
localDir: string;
importedHead: string;
}): Promise<void> {
const isConcurrentRefUpdateError = (error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
return message.includes("cannot lock ref") && message.includes("expected");
};
for (let attempt = 0; attempt < 5; attempt += 1) {
const snapshot = await readGitWorkspaceSnapshot(input.localDir);
if (!snapshot) return;
const currentHead = snapshot.headCommit;
if (!currentHead || currentHead === input.importedHead) return;
const headRef = snapshot.branchName ? `refs/heads/${snapshot.branchName}` : "HEAD";
const mergeBase = await runLocalGit(input.localDir, ["merge-base", currentHead, input.importedHead], {
timeout: 10_000,
maxBuffer: 16 * 1024,
}).catch(() => null);
const mergeBaseHead = mergeBase?.stdout.trim() ?? "";
if (mergeBaseHead === input.importedHead) {
return;
}
if (mergeBaseHead === currentHead) {
try {
await runLocalGit(input.localDir, ["update-ref", headRef, input.importedHead, currentHead], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
return;
} catch (error) {
if (isConcurrentRefUpdateError(error) && attempt < 4) continue;
throw error;
}
}
let mergedTree;
try {
mergedTree = await runLocalGit(input.localDir, ["merge-tree", "--write-tree", currentHead, input.importedHead], {
timeout: 60_000,
maxBuffer: 256 * 1024,
});
} catch (error) {
const reason = error instanceof Error ? error.message : String(error);
throw new Error(
`Failed to merge concurrent remote git histories for ${currentHead.slice(0, 12)} and ${input.importedHead.slice(0, 12)}: ${reason}`,
);
}
const mergedTreeId = mergedTree.stdout.trim().split("\n")[0]?.trim() ?? "";
if (!mergedTreeId) {
throw new Error("Failed to compute a merged git tree for workspace restore.");
}
const mergeCommit = await runLocalGit(
input.localDir,
[
"commit-tree",
mergedTreeId,
"-p",
currentHead,
"-p",
input.importedHead,
"-m",
`Paperclip remote git sync merge ${input.importedHead.slice(0, 12)}`,
],
{
timeout: 60_000,
maxBuffer: 64 * 1024,
},
);
try {
await runLocalGit(input.localDir, ["update-ref", headRef, mergeCommit.stdout.trim(), currentHead], {
timeout: 10_000,
maxBuffer: 16 * 1024,
});
return;
} catch (error) {
if (isConcurrentRefUpdateError(error) && attempt < 4) continue;
throw error;
}
}
throw new Error(`Failed to integrate concurrent remote git history for ${input.importedHead.slice(0, 12)} after multiple retries.`);
}
@@ -1,4 +1,5 @@
import path from "node:path";
import { GIT_ARCHIVE_EXCLUDES } from "./git-workspace-sync.js";
import {
type SshRemoteExecutionSpec,
prepareWorkspaceForSshExecution,
@@ -90,7 +91,7 @@ export async function prepareRemoteManagedRuntime(input: {
remoteDir: workspaceRemoteDir,
onProgress: input.onProgress,
});
const restoreExclude = preparedWorkspace.gitBacked ? [".git", ".paperclip-runtime"] : [".paperclip-runtime"];
const restoreExclude = preparedWorkspace.gitBacked ? [...GIT_ARCHIVE_EXCLUDES, ".paperclip-runtime"] : [".paperclip-runtime"];
const baselineSnapshot = await captureDirectorySnapshot(input.workspaceLocalDir, {
exclude: restoreExclude,
});
@@ -13,6 +13,20 @@ import {
const execFile = promisify(execFileCallback);
async function git(cwd: string, args: string[]): Promise<string> {
const { stdout } = await execFile("git", ["-C", cwd, ...args], {
maxBuffer: 32 * 1024 * 1024,
});
return stdout.trim();
}
async function listTarMembers(rootDir: string, name: string, bytes: Buffer): Promise<string[]> {
const tarPath = path.join(rootDir, name);
await writeFile(tarPath, bytes);
const { stdout } = await execFile("tar", ["-tf", tarPath], { maxBuffer: 32 * 1024 * 1024 });
return stdout.split("\n").map((line) => line.trim()).filter(Boolean);
}
describe("sandbox managed runtime", () => {
const cleanupDirs: string[] = [];
@@ -131,6 +145,224 @@ describe("sandbox managed runtime", () => {
await expect(readFile(path.join(localWorkspaceDir, ".paperclip-runtime", "state.json"), "utf8")).resolves.toBe("{}\n");
});
it("syncs git-backed workspaces through a shallow standalone clone and keeps .git out of archives", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-git-"));
cleanupDirs.push(rootDir);
const sourceRepoDir = path.join(rootDir, "source-repo");
const localWorkspaceDir = path.join(rootDir, "local-worktree");
const remoteWorkspaceDir = path.join(rootDir, "remote-workspace");
await mkdir(sourceRepoDir, { recursive: true });
await git(sourceRepoDir, ["init"]);
await git(sourceRepoDir, ["checkout", "-b", "main"]);
await git(sourceRepoDir, ["config", "user.name", "Paperclip Test"]);
await git(sourceRepoDir, ["config", "user.email", "test@paperclip.dev"]);
await writeFile(path.join(sourceRepoDir, ".gitignore"), "node_modules/\n", "utf8");
await writeFile(path.join(sourceRepoDir, "tracked.txt"), "base\n", "utf8");
await writeFile(path.join(sourceRepoDir, "clean.txt"), "from git\n", "utf8");
await writeFile(path.join(sourceRepoDir, "deleted.txt"), "delete me\n", "utf8");
await git(sourceRepoDir, ["add", ".gitignore", "tracked.txt", "clean.txt", "deleted.txt"]);
await git(sourceRepoDir, ["commit", "-m", "base"]);
await git(sourceRepoDir, ["worktree", "add", "-b", "work", localWorkspaceDir, "HEAD"]);
expect((await lstat(path.join(localWorkspaceDir, ".git"))).isFile()).toBe(true);
await mkdir(path.join(localWorkspaceDir, "node_modules"), { recursive: true });
await writeFile(path.join(localWorkspaceDir, "tracked.txt"), "dirty local\n", "utf8");
await writeFile(path.join(localWorkspaceDir, "untracked.txt"), "from local\n", "utf8");
await writeFile(path.join(localWorkspaceDir, "node_modules", "cache.bin"), "do not upload\n", "utf8");
await rm(path.join(localWorkspaceDir, "deleted.txt"));
const uploadedTars: { remotePath: string; bytes: Buffer }[] = [];
const downloadedTars: { remotePath: string; bytes: Buffer }[] = [];
const client: SandboxManagedRuntimeClient = {
makeDir: async (remotePath) => {
await mkdir(remotePath, { recursive: true });
},
writeFile: async (remotePath, bytes) => {
await mkdir(path.dirname(remotePath), { recursive: true });
const buffer = Buffer.from(bytes);
if (remotePath.endsWith("-upload.tar")) uploadedTars.push({ remotePath, bytes: buffer });
await writeFile(remotePath, buffer);
},
readFile: async (remotePath) => {
const buffer = await readFile(remotePath);
if (remotePath.endsWith("workspace-download.tar")) downloadedTars.push({ remotePath, bytes: buffer });
return buffer;
},
listFiles: async () => [],
remove: async (remotePath) => {
await rm(remotePath, { recursive: true, force: true });
},
run: async (command) => {
await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 });
},
};
const prepared = await prepareSandboxManagedRuntime({
spec: {
transport: "sandbox",
provider: "test",
sandboxId: "sandbox-1",
remoteCwd: remoteWorkspaceDir,
timeoutMs: 30_000,
apiKey: null,
},
adapterKey: "test-adapter",
client,
workspaceLocalDir: localWorkspaceDir,
});
expect((await lstat(path.join(remoteWorkspaceDir, ".git"))).isDirectory()).toBe(true);
await expect(readFile(path.join(remoteWorkspaceDir, ".git", "shallow"), "utf8")).resolves.toContain(
await git(localWorkspaceDir, ["rev-parse", "HEAD"]),
);
expect(await git(remoteWorkspaceDir, ["rev-list", "--count", "HEAD"])).toBe("1");
expect(await git(remoteWorkspaceDir, ["status", "--short"])).toContain("M tracked.txt");
expect(await git(remoteWorkspaceDir, ["status", "--short"])).toContain("?? untracked.txt");
await expect(readFile(path.join(remoteWorkspaceDir, "deleted.txt"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
const gitUpload = uploadedTars.find((entry) => path.posix.basename(entry.remotePath) === "git-workspace-upload.tar");
const workspaceUpload = uploadedTars.find((entry) => path.posix.basename(entry.remotePath) === "workspace-upload.tar");
expect(gitUpload).toBeDefined();
expect(workspaceUpload).toBeDefined();
const gitMembers = await listTarMembers(rootDir, "git-upload-list.tar", gitUpload!.bytes);
const workspaceMembers = await listTarMembers(rootDir, "workspace-upload-list.tar", workspaceUpload!.bytes);
expect(gitMembers.some((entry) => entry === ".git" || entry.startsWith(".git/"))).toBe(true);
expect(workspaceMembers.some((entry) => entry === ".git" || entry.startsWith(".git/"))).toBe(false);
expect(workspaceMembers).toContain("tracked.txt");
expect(workspaceMembers).toContain("untracked.txt");
expect(workspaceMembers).not.toContain("clean.txt");
expect(workspaceMembers.some((entry) => entry === "node_modules" || entry.startsWith("node_modules/"))).toBe(false);
await git(remoteWorkspaceDir, ["config", "user.name", "Paperclip Sandbox"]);
await git(remoteWorkspaceDir, ["config", "user.email", "sandbox@paperclip.dev"]);
await git(remoteWorkspaceDir, ["add", "-A"]);
await git(remoteWorkspaceDir, ["commit", "-m", "sandbox update"]);
await writeFile(path.join(remoteWorkspaceDir, "tracked.txt"), "remote dirty\n", "utf8");
await writeFile(path.join(remoteWorkspaceDir, "remote-only.txt"), "from sandbox\n", "utf8");
await prepared.restoreWorkspace();
expect((await lstat(path.join(localWorkspaceDir, ".git"))).isFile()).toBe(true);
expect(await git(localWorkspaceDir, ["log", "-1", "--pretty=%s"])).toBe("sandbox update");
await expect(readFile(path.join(localWorkspaceDir, "tracked.txt"), "utf8")).resolves.toBe("remote dirty\n");
await expect(readFile(path.join(localWorkspaceDir, "remote-only.txt"), "utf8")).resolves.toBe("from sandbox\n");
await expect(readFile(path.join(localWorkspaceDir, "deleted.txt"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
await expect(readFile(path.join(localWorkspaceDir, "node_modules", "cache.bin"), "utf8")).resolves.toBe("do not upload\n");
expect(downloadedTars).toHaveLength(1);
const downloadMembers = await listTarMembers(rootDir, "workspace-download-list.tar", downloadedTars[0]!.bytes);
expect(downloadMembers.some((entry) => entry === ".git" || entry.startsWith(".git/"))).toBe(false);
expect(downloadMembers.some((entry) => entry === "node_modules" || entry.startsWith("node_modules/"))).toBe(false);
});
it("excludes unignored dependency trees from git-backed workspace overlay archives", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-unignored-deps-"));
cleanupDirs.push(rootDir);
const sourceRepoDir = path.join(rootDir, "source-repo");
const localWorkspaceDir = path.join(rootDir, "local-worktree");
const remoteWorkspaceDir = path.join(rootDir, "remote-workspace");
await mkdir(sourceRepoDir, { recursive: true });
await git(sourceRepoDir, ["init"]);
await git(sourceRepoDir, ["checkout", "-b", "main"]);
await git(sourceRepoDir, ["config", "user.name", "Paperclip Test"]);
await git(sourceRepoDir, ["config", "user.email", "test@paperclip.dev"]);
await mkdir(path.join(sourceRepoDir, "src"), { recursive: true });
await writeFile(path.join(sourceRepoDir, "src", "tracked.ts"), "export const tracked = true;\n", "utf8");
await git(sourceRepoDir, ["add", "src/tracked.ts"]);
await git(sourceRepoDir, ["commit", "-m", "base"]);
await git(sourceRepoDir, ["worktree", "add", "-b", "work", localWorkspaceDir, "HEAD"]);
await mkdir(path.join(localWorkspaceDir, "node_modules", "root-package"), { recursive: true });
await mkdir(path.join(localWorkspaceDir, "packages", "ui", "node_modules", "nested-package"), { recursive: true });
await writeFile(path.join(localWorkspaceDir, "node_modules", "root-package", "cache.bin"), "root dependency\n", "utf8");
await writeFile(
path.join(localWorkspaceDir, "packages", "ui", "node_modules", "nested-package", "cache.bin"),
"nested dependency\n",
"utf8",
);
await writeFile(path.join(localWorkspaceDir, "src", "local-only.ts"), "export const local = true;\n", "utf8");
const uploadedTars: { remotePath: string; bytes: Buffer }[] = [];
const downloadedTars: { remotePath: string; bytes: Buffer }[] = [];
const client: SandboxManagedRuntimeClient = {
makeDir: async (remotePath) => {
await mkdir(remotePath, { recursive: true });
},
writeFile: async (remotePath, bytes) => {
await mkdir(path.dirname(remotePath), { recursive: true });
const buffer = Buffer.from(bytes);
if (remotePath.endsWith("-upload.tar")) uploadedTars.push({ remotePath, bytes: buffer });
await writeFile(remotePath, buffer);
},
readFile: async (remotePath) => {
const buffer = await readFile(remotePath);
if (remotePath.endsWith("workspace-download.tar")) downloadedTars.push({ remotePath, bytes: buffer });
return buffer;
},
listFiles: async () => [],
remove: async (remotePath) => {
await rm(remotePath, { recursive: true, force: true });
},
run: async (command) => {
await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 });
},
};
const prepared = await prepareSandboxManagedRuntime({
spec: {
transport: "sandbox",
provider: "test",
sandboxId: "sandbox-1",
remoteCwd: remoteWorkspaceDir,
timeoutMs: 30_000,
apiKey: null,
},
adapterKey: "test-adapter",
client,
workspaceLocalDir: localWorkspaceDir,
});
const workspaceUpload = uploadedTars.find((entry) => path.posix.basename(entry.remotePath) === "workspace-upload.tar");
expect(workspaceUpload).toBeDefined();
const workspaceMembers = await listTarMembers(rootDir, "unignored-deps-workspace-upload.tar", workspaceUpload!.bytes);
expect(workspaceMembers).toContain("src/local-only.ts");
expect(workspaceMembers.some((entry) => entry === "node_modules" || entry.startsWith("node_modules/"))).toBe(false);
expect(workspaceMembers.some((entry) => entry.includes("/node_modules/") || entry.endsWith("/node_modules"))).toBe(false);
await expect(readFile(path.join(remoteWorkspaceDir, "src", "local-only.ts"), "utf8")).resolves.toBe("export const local = true;\n");
await expect(readFile(path.join(remoteWorkspaceDir, "node_modules", "root-package", "cache.bin"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
await expect(
readFile(path.join(remoteWorkspaceDir, "packages", "ui", "node_modules", "nested-package", "cache.bin"), "utf8"),
).rejects.toMatchObject({ code: "ENOENT" });
await mkdir(path.join(remoteWorkspaceDir, "node_modules", "sandbox-package"), { recursive: true });
await mkdir(path.join(remoteWorkspaceDir, "packages", "ui", "node_modules", "sandbox-package"), { recursive: true });
await writeFile(path.join(remoteWorkspaceDir, "node_modules", "sandbox-package", "cache.bin"), "sandbox root dependency\n", "utf8");
await writeFile(
path.join(remoteWorkspaceDir, "packages", "ui", "node_modules", "sandbox-package", "cache.bin"),
"sandbox nested dependency\n",
"utf8",
);
await writeFile(path.join(remoteWorkspaceDir, "src", "remote-only.ts"), "export const remote = true;\n", "utf8");
await prepared.restoreWorkspace();
await expect(readFile(path.join(localWorkspaceDir, "node_modules", "root-package", "cache.bin"), "utf8")).resolves.toBe("root dependency\n");
await expect(
readFile(path.join(localWorkspaceDir, "packages", "ui", "node_modules", "nested-package", "cache.bin"), "utf8"),
).resolves.toBe("nested dependency\n");
await expect(readFile(path.join(localWorkspaceDir, "src", "remote-only.ts"), "utf8")).resolves.toBe("export const remote = true;\n");
await expect(readFile(path.join(localWorkspaceDir, "node_modules", "sandbox-package", "cache.bin"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
expect(downloadedTars).toHaveLength(1);
const downloadMembers = await listTarMembers(rootDir, "unignored-deps-workspace-download.tar", downloadedTars[0]!.bytes);
expect(downloadMembers.some((entry) => entry === ".git" || entry.startsWith(".git/"))).toBe(false);
expect(downloadMembers.some((entry) => entry === "node_modules" || entry.startsWith("node_modules/"))).toBe(false);
expect(downloadMembers.some((entry) => entry.includes("/node_modules/") || entry.endsWith("/node_modules"))).toBe(false);
});
it("builds workspace/asset tarballs without a './' self-entry (so untar does not chmod/utime an unowned target dir)", async () => {
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-tarself-"));
cleanupDirs.push(rootDir);
@@ -3,10 +3,44 @@ import { constants as fsConstants, promises as fs } from "node:fs";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import {
buildRemoteGitDeltaBundleScript,
createImportedGitRef,
createRemoteGitExportRef,
deleteLocalGitRef,
fetchGitBundleIntoLocalRef,
GIT_ARCHIVE_EXCLUDES,
integrateImportedGitHead,
readGitWorkspaceSnapshot,
withShallowGitWorkspaceClone,
} from "./git-workspace-sync.js";
import { captureDirectorySnapshot, mergeDirectoryWithBaseline } from "./workspace-restore-merge.js";
import { createRuntimeProgressReporter, type RuntimeProgressSink } from "./runtime-progress.js";
import {
createRuntimeProgressReporter,
type RuntimeProgressDirection,
type RuntimeProgressPhase,
type RuntimeProgressSink,
} from "./runtime-progress.js";
import { isRelativePathOrDescendant, shouldExcludePath } from "./exclude-patterns.js";
const execFile = promisify(execFileCallback);
const SANDBOX_WORKSPACE_HEAVY_DIR_NAMES = [
"node_modules",
"vendor",
"dist",
"build",
"out",
"coverage",
".next",
".turbo",
".cache",
] as const;
const SANDBOX_WORKSPACE_HEAVY_DIR_EXCLUDES = SANDBOX_WORKSPACE_HEAVY_DIR_NAMES.flatMap((entry) => [
entry,
`${entry}/*`,
`*/${entry}`,
`*/${entry}/*`,
]);
export interface SandboxRemoteExecutionSpec {
transport: "sandbox";
@@ -208,8 +242,28 @@ async function walkDirectory(root: string, relative = ""): Promise<string[]> {
return out.sort((left, right) => right.length - left.length);
}
function isRelativePathOrDescendant(relative: string, candidate: string): boolean {
return relative === candidate || relative.startsWith(`${candidate}/`);
async function copyWorkspaceEntry(sourceRoot: string, targetRoot: string, relative: string): Promise<void> {
const sourcePath = path.join(sourceRoot, relative);
const targetPath = path.join(targetRoot, relative);
const stats = await fs.lstat(sourcePath);
if (stats.isDirectory()) {
await fs.mkdir(targetPath, { recursive: true });
return;
}
await fs.mkdir(path.dirname(targetPath), { recursive: true });
await fs.rm(targetPath, { recursive: true, force: true }).catch(() => undefined);
if (stats.isSymbolicLink()) {
const linkTarget = await fs.readlink(sourcePath);
await fs.symlink(linkTarget, targetPath);
return;
}
await fs.copyFile(sourcePath, targetPath, fsConstants.COPYFILE_FICLONE).catch(async () => {
await fs.copyFile(sourcePath, targetPath);
});
await fs.chmod(targetPath, stats.mode);
}
export async function mirrorDirectory(
@@ -231,33 +285,24 @@ export async function mirrorDirectory(
}
}
const copyEntry = async (relative: string) => {
const sourcePath = path.join(sourceDir, relative);
const targetPath = path.join(targetDir, relative);
const stats = await fs.lstat(sourcePath);
if (stats.isDirectory()) {
await fs.mkdir(targetPath, { recursive: true });
return;
}
await fs.mkdir(path.dirname(targetPath), { recursive: true });
await fs.rm(targetPath, { recursive: true, force: true }).catch(() => undefined);
if (stats.isSymbolicLink()) {
const linkTarget = await fs.readlink(sourcePath);
await fs.symlink(linkTarget, targetPath);
return;
}
await fs.copyFile(sourcePath, targetPath, fsConstants.COPYFILE_FICLONE).catch(async () => {
await fs.copyFile(sourcePath, targetPath);
});
await fs.chmod(targetPath, stats.mode);
};
const entries = (await walkDirectory(sourceDir)).sort((left, right) => left.localeCompare(right));
for (const relative of entries) {
await copyEntry(relative);
await copyWorkspaceEntry(sourceDir, targetDir, relative);
}
}
async function copySelectedWorkspaceEntries(input: {
sourceDir: string;
targetDir: string;
relativePaths: string[];
exclude: string[];
}): Promise<void> {
await fs.mkdir(input.targetDir, { recursive: true });
for (const relative of input.relativePaths) {
if (shouldExcludePath(relative, input.exclude)) continue;
const sourceStats = await fs.lstat(path.join(input.sourceDir, relative)).catch(() => null);
if (!sourceStats) continue;
await copyWorkspaceEntry(input.sourceDir, input.targetDir, relative);
}
}
@@ -275,15 +320,37 @@ function tarExcludeFlags(exclude: string[] | undefined): string {
return ["._*", ...(exclude ?? [])].map((entry) => `--exclude ${shellQuote(entry)}`).join(" ");
}
function mergeExcludes(...groups: Array<string[] | undefined>): string[] {
return [...new Set(groups.flatMap((group) => group ?? []))];
}
function preserveFindArgs(entries: string[]): string {
return entries.map((entry) => `! -name ${shellQuote(entry)}`).join(" ");
}
async function removeDeletedPathsInSandbox(input: {
client: SandboxManagedRuntimeClient;
spec: SandboxRemoteExecutionSpec;
remoteDir: string;
deletedPaths: string[];
}): Promise<void> {
if (input.deletedPaths.length === 0) return;
const quotedPaths = input.deletedPaths.map((entry) => shellQuote(entry)).join(" ");
await input.client.run(
`sh -c ${shellQuote(`cd ${shellQuote(input.remoteDir)} && rm -rf -- ${quotedPaths}`)}`,
{ timeoutMs: input.spec.timeoutMs },
);
}
// Bridge a single byte-level transfer to the throttled progress reporter. The
// transport reports decoded bytes via `options.onProgress`; the reporter turns
// them into a throttled, fully-formatted log line. `finish()` emits the terminal
// completion line (idempotent) once the transfer returns.
function makeTransferProgress(
sink: RuntimeProgressSink | undefined,
phase: "Syncing" | "Restoring",
direction: "to" | "from",
label: string,
phase: RuntimeProgressPhase,
direction: RuntimeProgressDirection,
label?: string,
): { options: SandboxTransferProgressOptions | undefined; finish: () => Promise<void> } {
if (!sink) return { options: undefined, finish: async () => {} };
const reporter = createRuntimeProgressReporter({
@@ -320,16 +387,75 @@ export async function prepareSandboxManagedRuntime(input: {
}): Promise<PreparedSandboxManagedRuntime> {
const workspaceRemoteDir = input.workspaceRemoteDir ?? input.spec.remoteCwd;
const runtimeRootDir = path.posix.join(workspaceRemoteDir, ".paperclip-runtime", input.adapterKey);
const gitSnapshot = await readGitWorkspaceSnapshot(input.workspaceLocalDir);
const gitIgnoredExcludes = gitSnapshot?.ignoredPaths;
const workspaceArchiveExclude = mergeExcludes(
SANDBOX_WORKSPACE_HEAVY_DIR_EXCLUDES,
[...GIT_ARCHIVE_EXCLUDES],
input.workspaceExclude,
gitIgnoredExcludes,
);
const restoreExclude = mergeExcludes(
SANDBOX_WORKSPACE_HEAVY_DIR_EXCLUDES,
[...GIT_ARCHIVE_EXCLUDES],
[".paperclip-runtime"],
input.preserveAbsentOnRestore,
input.workspaceExclude,
gitIgnoredExcludes,
);
const baselineSnapshot = await captureDirectorySnapshot(input.workspaceLocalDir, {
exclude: [...new Set([".paperclip-runtime", ...(input.preserveAbsentOnRestore ?? []), ...(input.workspaceExclude ?? [])])],
exclude: restoreExclude,
});
await withTempDir("paperclip-sandbox-sync-", async (tempDir) => {
const preservedNames = new Set([
".paperclip-runtime",
...(gitSnapshot ? [".git"] : []),
...(input.preserveAbsentOnRestore ?? []),
]);
if (gitSnapshot) {
await withShallowGitWorkspaceClone({
localDir: input.workspaceLocalDir,
snapshot: gitSnapshot,
}, async (cloneDir) => {
const gitTarPath = path.join(tempDir, "git-workspace.tar");
await createTarballFromDirectory({
localDir: cloneDir,
archivePath: gitTarPath,
exclude: [".paperclip-runtime"],
});
const gitTarBytes = await fs.readFile(gitTarPath);
const remoteGitTar = path.posix.join(runtimeRootDir, "git-workspace-upload.tar");
await input.client.makeDir(runtimeRootDir);
const gitUpload = makeTransferProgress(input.onProgress, "Syncing", "to", "git history");
await input.client.writeFile(remoteGitTar, toArrayBuffer(gitTarBytes), gitUpload.options);
await gitUpload.finish();
await input.client.run(
`sh -c ${shellQuote(
`mkdir -p ${shellQuote(workspaceRemoteDir)} && ` +
`find ${shellQuote(workspaceRemoteDir)} -mindepth 1 -maxdepth 1 ${preserveFindArgs([".paperclip-runtime"])} -exec rm -rf -- {} + && ` +
`tar -xf ${shellQuote(remoteGitTar)} -C ${shellQuote(workspaceRemoteDir)} && ` +
`rm -f ${shellQuote(remoteGitTar)}`,
)}`,
{ timeoutMs: input.spec.timeoutMs },
);
});
}
const workspaceTarPath = path.join(tempDir, "workspace.tar");
const workspaceArchiveDir = gitSnapshot ? path.join(tempDir, "workspace-overlay") : input.workspaceLocalDir;
if (gitSnapshot) {
await copySelectedWorkspaceEntries({
sourceDir: input.workspaceLocalDir,
targetDir: workspaceArchiveDir,
relativePaths: gitSnapshot.overlayPaths,
exclude: workspaceArchiveExclude,
});
}
await createTarballFromDirectory({
localDir: input.workspaceLocalDir,
localDir: workspaceArchiveDir,
archivePath: workspaceTarPath,
exclude: input.workspaceExclude,
exclude: gitSnapshot ? undefined : workspaceArchiveExclude,
});
const workspaceTarBytes = await fs.readFile(workspaceTarPath);
const remoteWorkspaceTar = path.posix.join(runtimeRootDir, "workspace-upload.tar");
@@ -341,17 +467,26 @@ export async function prepareSandboxManagedRuntime(input: {
workspaceUpload.options,
);
await workspaceUpload.finish();
const preservedNames = new Set([".paperclip-runtime", ...(input.preserveAbsentOnRestore ?? [])]);
const findPreserveArgs = [...preservedNames].map((entry) => `! -name ${shellQuote(entry)}`).join(" ");
const extractWorkspaceTarCommand = gitSnapshot
? `mkdir -p ${shellQuote(workspaceRemoteDir)} && ` +
`tar -xf ${shellQuote(remoteWorkspaceTar)} -C ${shellQuote(workspaceRemoteDir)} && ` +
`rm -f ${shellQuote(remoteWorkspaceTar)}`
: `mkdir -p ${shellQuote(workspaceRemoteDir)} && ` +
`find ${shellQuote(workspaceRemoteDir)} -mindepth 1 -maxdepth 1 ${preserveFindArgs([...preservedNames])} -exec rm -rf -- {} + && ` +
`tar -xf ${shellQuote(remoteWorkspaceTar)} -C ${shellQuote(workspaceRemoteDir)} && ` +
`rm -f ${shellQuote(remoteWorkspaceTar)}`;
await input.client.run(
`sh -c ${shellQuote(
`mkdir -p ${shellQuote(workspaceRemoteDir)} && ` +
`find ${shellQuote(workspaceRemoteDir)} -mindepth 1 -maxdepth 1 ${findPreserveArgs} -exec rm -rf -- {} + && ` +
`tar -xf ${shellQuote(remoteWorkspaceTar)} -C ${shellQuote(workspaceRemoteDir)} && ` +
`rm -f ${shellQuote(remoteWorkspaceTar)}`,
)}`,
`sh -c ${shellQuote(extractWorkspaceTarCommand)}`,
{ timeoutMs: input.spec.timeoutMs },
);
if (gitSnapshot) {
await removeDeletedPathsInSandbox({
client: input.client,
spec: input.spec,
remoteDir: workspaceRemoteDir,
deletedPaths: gitSnapshot.deletedPaths,
});
}
for (const asset of input.assets ?? []) {
const assetTarPath = path.join(tempDir, `${asset.key}.tar`);
@@ -392,31 +527,76 @@ export async function prepareSandboxManagedRuntime(input: {
restoreWorkspace: async (onProgress?: RuntimeProgressSink) => {
const restoreSink = onProgress ?? input.onProgress;
await withTempDir("paperclip-sandbox-restore-", async (tempDir) => {
const remoteWorkspaceTar = path.posix.join(runtimeRootDir, "workspace-download.tar");
await input.client.run(
`sh -c ${shellQuote(
`mkdir -p ${shellQuote(runtimeRootDir)} && ` +
`tar -cf ${shellQuote(remoteWorkspaceTar)} -C ${shellQuote(workspaceRemoteDir)} ` +
`${tarExcludeFlags(input.workspaceExclude)} .`,
)}`,
{ timeoutMs: input.spec.timeoutMs },
);
const workspaceRestore = makeTransferProgress(restoreSink, "Restoring", "from", "workspace");
const archiveBytes = await input.client.readFile(remoteWorkspaceTar, workspaceRestore.options);
await workspaceRestore.finish();
await input.client.remove(remoteWorkspaceTar).catch(() => undefined);
const localArchivePath = path.join(tempDir, "workspace.tar");
const extractedDir = path.join(tempDir, "workspace");
await fs.writeFile(localArchivePath, toBuffer(archiveBytes));
await extractTarballToDirectory({
archivePath: localArchivePath,
localDir: extractedDir,
});
await mergeDirectoryWithBaseline({
baseline: baselineSnapshot,
sourceDir: extractedDir,
targetDir: input.workspaceLocalDir,
});
let importedRef: string | null = null;
let importedHead: string | null = null;
try {
if (gitSnapshot) {
importedRef = createImportedGitRef("sandbox");
const remoteGitBundle = path.posix.join(runtimeRootDir, "git-delta.bundle");
const exportRef = createRemoteGitExportRef("sandbox");
await input.client.run(
`sh -c ${shellQuote(buildRemoteGitDeltaBundleScript({
remoteDir: workspaceRemoteDir,
baseSha: gitSnapshot.headCommit,
exportRef,
bundlePath: remoteGitBundle,
}))}`,
{ timeoutMs: input.spec.timeoutMs },
);
const gitExport = makeTransferProgress(restoreSink, "Exporting git history", "from");
const bundleBytes = await input.client.readFile(remoteGitBundle, gitExport.options);
await gitExport.finish();
await input.client.remove(remoteGitBundle).catch(() => undefined);
const bundlePath = path.join(tempDir, "git-delta.bundle");
await fs.writeFile(bundlePath, toBuffer(bundleBytes));
importedHead = await fetchGitBundleIntoLocalRef({
localDir: input.workspaceLocalDir,
bundlePath,
exportRef,
importedRef,
baseSha: gitSnapshot.headCommit,
});
}
const remoteWorkspaceTar = path.posix.join(runtimeRootDir, "workspace-download.tar");
await input.client.run(
`sh -c ${shellQuote(
`mkdir -p ${shellQuote(runtimeRootDir)} && ` +
`tar -cf ${shellQuote(remoteWorkspaceTar)} -C ${shellQuote(workspaceRemoteDir)} ` +
`${tarExcludeFlags(restoreExclude)} .`,
)}`,
{ timeoutMs: input.spec.timeoutMs },
);
const workspaceRestore = makeTransferProgress(restoreSink, "Restoring", "from", "workspace");
const archiveBytes = await input.client.readFile(remoteWorkspaceTar, workspaceRestore.options);
await workspaceRestore.finish();
await input.client.remove(remoteWorkspaceTar).catch(() => undefined);
const localArchivePath = path.join(tempDir, "workspace.tar");
const extractedDir = path.join(tempDir, "workspace");
await fs.writeFile(localArchivePath, toBuffer(archiveBytes));
await extractTarballToDirectory({
archivePath: localArchivePath,
localDir: extractedDir,
});
const gitHeadToIntegrate = importedHead;
await mergeDirectoryWithBaseline({
baseline: baselineSnapshot,
sourceDir: extractedDir,
targetDir: input.workspaceLocalDir,
beforeApply: gitHeadToIntegrate
? async () => {
await integrateImportedGitHead({
localDir: input.workspaceLocalDir,
importedHead: gitHeadToIntegrate,
});
}
: undefined,
});
} finally {
if (importedRef) {
await deleteLocalGitRef({ localDir: input.workspaceLocalDir, ref: importedRef });
}
}
});
},
};
@@ -213,8 +213,6 @@ describe("adapter skill snapshots", () => {
key: "paperclipai/paperclip/paperclip",
runtimeName: "paperclip",
source: "/runtime/paperclip",
required: true,
requiredReason: "Required for Paperclip heartbeats.",
};
const optionalEntry = {
key: "company/ascii-heart",
@@ -245,8 +243,7 @@ describe("adapter skill snapshots", () => {
expect.objectContaining({
key: requiredEntry.key,
state: "configured",
origin: "paperclip_required",
required: true,
origin: "company_managed",
detail: "Mounted on next run.",
}),
]);
@@ -345,7 +342,7 @@ describe("adapter skill snapshots", () => {
key: requiredEntry.key,
state: "installed",
managed: true,
origin: "paperclip_required",
origin: "company_managed",
}));
expect(snapshot.entries).toContainEqual(expect.objectContaining({
key: optionalEntry.key,
+12 -61
View File
@@ -195,8 +195,6 @@ export interface PaperclipSkillEntry {
currentVersionId?: string | null;
sourceStatus?: "available" | "missing";
missingDetail?: string | null;
required?: boolean;
requiredReason?: string | null;
}
export interface PaperclipDesiredSkillEntry {
@@ -258,17 +256,10 @@ function skillLocationLabel(value: string | null | undefined): string | null {
return trimmed.length > 0 ? trimmed : null;
}
function buildManagedSkillOrigin(entry: { required?: boolean }): Pick<
function buildManagedSkillOrigin(): Pick<
AdapterSkillEntry,
"origin" | "originLabel" | "readOnly"
> {
if (entry.required) {
return {
origin: "paperclip_required",
originLabel: "Required by Paperclip",
readOnly: false,
};
}
return {
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -1594,20 +1585,6 @@ export async function resolvePaperclipSkillsDir(
return null;
}
async function readSkillRequired(skillDir: string): Promise<boolean> {
try {
const content = await fs.readFile(path.join(skillDir, "SKILL.md"), "utf8");
const normalized = content.replace(/\r\n/g, "\n");
if (!normalized.startsWith("---\n")) return true;
const closing = normalized.indexOf("\n---\n", 4);
if (closing < 0) return true;
const frontmatter = normalized.slice(4, closing);
return !/^\s*required\s*:\s*false\s*$/m.test(frontmatter);
} catch {
return true;
}
}
export async function listPaperclipSkillEntries(
moduleDir: string,
additionalCandidates: string[] = [],
@@ -1618,18 +1595,10 @@ export async function listPaperclipSkillEntries(
try {
const entries = await fs.readdir(root, { withFileTypes: true });
const dirs = entries.filter((entry) => entry.isDirectory());
return Promise.all(dirs.map(async (entry) => {
const skillDir = path.join(root, entry.name);
const required = await readSkillRequired(skillDir);
return {
key: `paperclipai/paperclip/${entry.name}`,
runtimeName: entry.name,
source: skillDir,
required,
requiredReason: required
? "Bundled Paperclip skills are always available for local adapters."
: null,
};
return dirs.map((entry) => ({
key: `paperclipai/paperclip/${entry.name}`,
runtimeName: entry.name,
source: path.join(root, entry.name),
}));
} catch {
return [];
@@ -1682,9 +1651,7 @@ export function buildRuntimeMountedSkillSnapshot(
sourcePath: null,
targetPath: null,
detail: resolvePaperclipSkillMissingDetail(available, missingDetail),
required: Boolean(available.required),
requiredReason: available.requiredReason ?? null,
...buildManagedSkillOrigin(available),
...buildManagedSkillOrigin(),
});
continue;
}
@@ -1709,9 +1676,7 @@ export function buildRuntimeMountedSkillSnapshot(
available,
)
: null,
required: Boolean(available.required),
requiredReason: available.requiredReason ?? null,
...buildManagedSkillOrigin(available),
...buildManagedSkillOrigin(),
});
}
@@ -1807,9 +1772,7 @@ export function buildPersistentSkillSnapshot(
available,
missingDetail,
),
required: Boolean(available.required),
requiredReason: available.requiredReason ?? null,
...buildManagedSkillOrigin(available),
...buildManagedSkillOrigin(),
});
continue;
}
@@ -1841,9 +1804,7 @@ export function buildPersistentSkillSnapshot(
sourcePath: available.source,
targetPath: path.join(skillsHome, available.runtimeName),
detail,
required: Boolean(available.required),
requiredReason: available.requiredReason ?? null,
...buildManagedSkillOrigin(available),
...buildManagedSkillOrigin(),
});
}
@@ -1925,11 +1886,6 @@ function normalizeConfiguredPaperclipRuntimeSkills(value: unknown): PaperclipSki
typeof entry.missingDetail === "string" && entry.missingDetail.trim().length > 0
? entry.missingDetail.trim()
: null,
required: asBoolean(entry.required, false),
requiredReason:
typeof entry.requiredReason === "string" && entry.requiredReason.trim().length > 0
? entry.requiredReason.trim()
: null,
});
}
return out;
@@ -2029,19 +1985,14 @@ function canonicalizeDesiredPaperclipSkillReference(
export function resolvePaperclipDesiredSkillNames(
config: Record<string, unknown>,
availableEntries: Array<{ key: string; runtimeName?: string | null; required?: boolean }>,
availableEntries: Array<{ key: string; runtimeName?: string | null }>,
): string[] {
const preference = readPaperclipSkillSyncPreference(config);
const requiredSkills = availableEntries
.filter((entry) => entry.required)
.map((entry) => entry.key);
if (!preference.explicit) {
return Array.from(new Set(requiredSkills));
}
if (!preference.explicit) return [];
const desiredSkills = preference.desiredSkills
.map((reference) => canonicalizeDesiredPaperclipSkillReference(reference, availableEntries))
.filter(Boolean);
return Array.from(new Set([...requiredSkills, ...desiredSkills]));
return Array.from(new Set(desiredSkills));
}
export function writePaperclipSkillSyncPreference(
-3
View File
@@ -186,7 +186,6 @@ export type AdapterSkillState =
export type AdapterSkillOrigin =
| "company_managed"
| "paperclip_required"
| "user_installed"
| "external_unknown";
@@ -197,8 +196,6 @@ export interface AdapterSkillEntry {
currentVersionId?: string | null;
desired: boolean;
managed: boolean;
required?: boolean;
requiredReason?: string | null;
state: AdapterSkillState;
origin?: AdapterSkillOrigin;
originLabel?: string | null;
@@ -2,6 +2,7 @@ import { createHash } from "node:crypto";
import { createReadStream } from "node:fs";
import { constants as fsConstants, promises as fs } from "node:fs";
import path from "node:path";
import { shouldExcludePath } from "./exclude-patterns.js";
type SnapshotEntry =
| { kind: "dir" }
@@ -13,14 +14,6 @@ export interface DirectorySnapshot {
entries: Map<string, SnapshotEntry>;
}
function isRelativePathOrDescendant(relative: string, candidate: string): boolean {
return relative === candidate || relative.startsWith(`${candidate}/`);
}
function shouldExclude(relative: string, exclude: readonly string[]): boolean {
return exclude.some((candidate) => isRelativePathOrDescendant(relative, candidate));
}
async function hashFile(filePath: string): Promise<string> {
return await new Promise((resolve, reject) => {
const hash = createHash("sha256");
@@ -43,7 +36,7 @@ async function walkDirectory(
for (const entry of entries) {
const nextRelative = relative ? path.posix.join(relative, entry.name) : entry.name;
if (shouldExclude(nextRelative, exclude)) continue;
if (shouldExcludePath(nextRelative, exclude)) continue;
const fullPath = path.join(root, nextRelative);
const stats = await fs.lstat(fullPath);
@@ -87,6 +87,7 @@ Core fields:
Operational fields:
- timeoutSec (number, optional): run timeout in seconds
- graceSec (number, optional): SIGTERM grace period in seconds
- outputInactivityTimeoutMs (number | null, optional): inactivity monitor around the codex child. Resets on every parsed JSONL event from stdout. Defaults to 7 * 60_000 ms when unset or non-positive. Set to \`null\` to disable the monitor entirely (only do this for known-slow tasks; the platform-level 1h silent-run safety net still applies). On fire, the adapter sends SIGTERM to the process group, waits 5s, then SIGKILL, and surfaces the run as failed with errorMessage "monitor: no codex output for {N}m {S}s".
Notes:
- Prompts are piped via stdin (Codex receives "-" prompt argument).
@@ -2,7 +2,14 @@ import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { ensureSymlink, prepareManagedCodexHome } from "./codex-home.js";
import {
codexHomeHasUsableAuth,
ensureSymlink,
isManagedCodexHomePath,
prepareManagedCodexHome,
reconcileManagedCodexHome,
seedManagedCodexHome,
} from "./codex-home.js";
describe("codex managed home", () => {
afterEach(() => {
@@ -194,3 +201,304 @@ describe("codex managed home", () => {
});
});
describe("isManagedCodexHomePath", () => {
const env = {
PAPERCLIP_HOME: "/srv/paperclip",
PAPERCLIP_INSTANCE_ID: "default",
} satisfies NodeJS.ProcessEnv;
const companyRoot = path.resolve(
"/srv/paperclip/instances/default/companies/company-1",
);
it("treats the per-agent managed home as managed", () => {
expect(
isManagedCodexHomePath(
env,
"company-1",
path.join(companyRoot, "agents", "agent-7", "codex-home"),
),
).toBe(true);
});
it("treats the shared company home as managed", () => {
expect(
isManagedCodexHomePath(env, "company-1", path.join(companyRoot, "codex-home")),
).toBe(true);
});
it("treats a path outside the company tree as an external override", () => {
expect(isManagedCodexHomePath(env, "company-1", "/home/dev/.codex")).toBe(false);
expect(
isManagedCodexHomePath(
env,
"company-1",
path.resolve("/srv/paperclip/instances/default/companies/company-2/codex-home"),
),
).toBe(false);
});
it("returns false without a companyId", () => {
expect(isManagedCodexHomePath(env, undefined, path.join(companyRoot, "codex-home"))).toBe(
false,
);
});
});
describe("codexHomeHasUsableAuth", () => {
it("is true for credential-bearing auth.json and false when missing", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-auth-"));
try {
expect(await codexHomeHasUsableAuth(root)).toBe(false);
await fs.writeFile(path.join(root, "auth.json"), "{}", "utf8");
expect(await codexHomeHasUsableAuth(root)).toBe(false);
await fs.writeFile(path.join(root, "auth.json"), '{"foo":"bar"}', "utf8");
expect(await codexHomeHasUsableAuth(root)).toBe(false);
await fs.writeFile(path.join(root, "auth.json"), '{"token":"shared"}', "utf8");
expect(await codexHomeHasUsableAuth(root)).toBe(true);
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it("is false for a dangling auth.json symlink", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-auth-dangling-"));
try {
await fs.symlink(path.join(root, "missing-source.json"), path.join(root, "auth.json"));
expect(await codexHomeHasUsableAuth(root)).toBe(false);
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
});
describe("seedManagedCodexHome", () => {
it("symlinks auth.json from the shared source into an explicit per-agent home", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-seed-"));
try {
const sharedCodexHome = path.join(root, "shared-codex-home");
const agentHome = path.join(
root,
"instances",
"default",
"companies",
"company-1",
"agents",
"agent-7",
"codex-home",
);
const sharedAuth = path.join(sharedCodexHome, "auth.json");
const agentAuth = path.join(agentHome, "auth.json");
await fs.mkdir(sharedCodexHome, { recursive: true });
await fs.writeFile(sharedAuth, '{"token":"shared"}', "utf8");
await seedManagedCodexHome(agentHome, { CODEX_HOME: sharedCodexHome }, async () => {});
expect((await fs.lstat(agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(agentAuth)).toBe(await fs.realpath(sharedAuth));
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
it("writes an API-key auth.json into the home when an apiKey is supplied", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-seed-apikey-"));
try {
const agentHome = path.join(root, "agent-home");
const emptyShared = path.join(root, "empty-shared");
await fs.mkdir(emptyShared, { recursive: true });
await seedManagedCodexHome(agentHome, { CODEX_HOME: emptyShared }, async () => {}, {
apiKey: "sk-test-123",
});
const written = JSON.parse(await fs.readFile(path.join(agentHome, "auth.json"), "utf8"));
expect(written).toEqual({ OPENAI_API_KEY: "sk-test-123" });
} finally {
await fs.rm(root, { recursive: true, force: true });
}
});
});
// Startup backfill for already-isolated managed homes.
describe("reconcileManagedCodexHome", () => {
async function makeFixture() {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-reconcile-"));
const sharedCodexHome = path.join(root, "shared-codex-home");
const paperclipHome = path.join(root, "paperclip-home");
const agentHome = path.join(
paperclipHome,
"instances",
"default",
"companies",
"company-1",
"agents",
"agent-7",
"codex-home",
);
const sharedAuth = path.join(sharedCodexHome, "auth.json");
const agentAuth = path.join(agentHome, "auth.json");
await fs.mkdir(sharedCodexHome, { recursive: true });
await fs.writeFile(sharedAuth, '{"token":"shared"}', "utf8");
const env = {
CODEX_HOME: sharedCodexHome,
PAPERCLIP_HOME: paperclipHome,
PAPERCLIP_INSTANCE_ID: "default",
} satisfies NodeJS.ProcessEnv;
return { root, sharedCodexHome, sharedAuth, agentHome, agentAuth, env };
}
it("seeds a previously-stranded managed home and is a no-op on re-run", async () => {
const fx = await makeFixture();
try {
// The isolation guard created the per-agent home with no auth.json.
expect(await codexHomeHasUsableAuth(fx.agentHome)).toBe(false);
const first = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
env: fx.env,
});
expect(first.status).toBe("seeded");
expect(first.home).toBe(fx.agentHome);
expect((await fs.lstat(fx.agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(fx.agentAuth)).toBe(await fs.realpath(fx.sharedAuth));
const second = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
env: fx.env,
});
expect(second.status).toBe("already_seeded");
expect((await fs.lstat(fx.agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(fx.agentAuth)).toBe(await fs.realpath(fx.sharedAuth));
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("reports source_auth_missing when shared auth is unavailable", async () => {
const fx = await makeFixture();
try {
await fs.rm(fx.sharedAuth, { force: true });
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
env: fx.env,
});
expect(result.status).toBe("source_auth_missing");
await expect(fs.lstat(fx.agentAuth)).rejects.toThrow();
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("leaves a genuine external override untouched", async () => {
const fx = await makeFixture();
try {
const external = path.join(fx.root, "user-codex");
await fs.mkdir(external, { recursive: true });
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: external,
env: fx.env,
});
expect(result.status).toBe("external_override");
expect(await codexHomeHasUsableAuth(external)).toBe(false);
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("reports no_managed_home when no CODEX_HOME is configured", async () => {
const fx = await makeFixture();
try {
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: null,
env: fx.env,
});
expect(result).toEqual({ status: "no_managed_home", home: null });
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("preserves an existing API-key auth.json when the key is secret-bound", async () => {
const fx = await makeFixture();
try {
// A prior execute-time run resolved the secret and wrote a regular-file
// auth.json containing the key.
await fs.mkdir(fx.agentHome, { recursive: true });
await fs.writeFile(
fx.agentAuth,
JSON.stringify({ OPENAI_API_KEY: "sk-secret-resolved" }),
{ mode: 0o600 },
);
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
apiKeySecretBound: true,
env: fx.env,
});
expect(result.status).toBe("already_seeded");
expect((await fs.lstat(fx.agentAuth)).isSymbolicLink()).toBe(false);
expect(JSON.parse(await fs.readFile(fx.agentAuth, "utf8"))).toEqual({
OPENAI_API_KEY: "sk-secret-resolved",
});
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("seeds the shared symlink for a secret-bound key when no auth exists yet", async () => {
const fx = await makeFixture();
try {
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
apiKeySecretBound: true,
env: fx.env,
});
expect(result.status).toBe("seeded");
expect((await fs.lstat(fx.agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(fx.agentAuth)).toBe(await fs.realpath(fx.sharedAuth));
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
it("writes an API-key auth.json into a managed home when an apiKey is supplied", async () => {
const fx = await makeFixture();
try {
const result = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
apiKey: "sk-reconcile-1",
env: fx.env,
});
expect(result.status).toBe("seeded");
const written = JSON.parse(await fs.readFile(fx.agentAuth, "utf8"));
expect(written).toEqual({ OPENAI_API_KEY: "sk-reconcile-1" });
const second = await reconcileManagedCodexHome({
companyId: "company-1",
configuredCodexHome: fx.agentHome,
apiKey: "sk-reconcile-1",
env: fx.env,
});
expect(second.status).toBe("already_seeded");
expect(JSON.parse(await fs.readFile(fx.agentAuth, "utf8"))).toEqual({
OPENAI_API_KEY: "sk-reconcile-1",
});
} finally {
await fs.rm(fx.root, { recursive: true, force: true });
}
});
});
@@ -7,6 +7,7 @@ import { resolvePaperclipInstanceRootForAdapter } from "@paperclipai/adapter-uti
const TRUTHY_ENV_RE = /^(1|true|yes|on)$/i;
const COPIED_SHARED_FILES = ["config.json", "config.toml", "instructions.md"] as const;
const SYMLINKED_SHARED_FILES = ["auth.json"] as const;
const AUTH_CREDENTIAL_KEYS = /(?:openai[_-]?key|api[_-]?key|access[_-]?token|refresh[_-]?token|token|secret|session|auth)/i;
function nonEmpty(value: string | undefined): string | null {
return typeof value === "string" && value.trim().length > 0 ? value.trim() : null;
@@ -16,6 +17,28 @@ export async function pathExists(candidate: string): Promise<boolean> {
return fs.access(candidate).then(() => true).catch(() => false);
}
function hasUsableAuthPayload(authPayload: unknown): boolean {
if (authPayload === null || typeof authPayload !== "object" || Array.isArray(authPayload)) {
return false;
}
for (const [key, value] of Object.entries(authPayload as Record<string, unknown>)) {
if (!AUTH_CREDENTIAL_KEYS.test(key)) continue;
if (key.toLowerCase() === "token_type") continue;
if (typeof value === "string" && value.trim().length > 0) return true;
}
return false;
}
function readApiKeyFromAuthPayload(authPayload: unknown): string | null {
if (authPayload === null || typeof authPayload !== "object" || Array.isArray(authPayload)) {
return null;
}
const raw = (authPayload as Record<string, unknown>).OPENAI_API_KEY;
return typeof raw === "string" && raw.trim().length > 0 ? raw.trim() : null;
}
export function resolveSharedCodexHomeDir(
env: NodeJS.ProcessEnv = process.env,
): string {
@@ -41,6 +64,59 @@ export function resolveManagedCodexHomeDir(
: path.resolve(instanceRoot, "codex-home");
}
/**
* True when `homePath` lives under the Paperclip-managed company tree
* (`<instanceRoot>/companies/<companyId>/...`). This covers both the shared
* company `codex-home` and the per-agent `agents/<agentId>/codex-home` set by
* the server-side isolation guard. A path outside that tree is a genuine
* external/user-supplied override that Paperclip must not seed or overwrite.
*/
export function isManagedCodexHomePath(
env: NodeJS.ProcessEnv,
companyId: string | undefined,
homePath: string,
): boolean {
if (!companyId) return false;
const instanceRoot = resolvePaperclipInstanceRootForAdapter({
homeDir: nonEmpty(env.PAPERCLIP_HOME) ?? undefined,
instanceId: nonEmpty(env.PAPERCLIP_INSTANCE_ID) ?? undefined,
env,
});
const companyRoot = path.resolve(instanceRoot, "companies", companyId);
const resolved = path.resolve(homePath);
return resolved === companyRoot || resolved.startsWith(companyRoot + path.sep);
}
/**
* True when the Codex home has a usable `auth.json`. Uses `fs.access` (follows
* symlinks), so a dangling auth symlink whose source has been removed counts as
* no usable credentials.
*/
export async function codexHomeHasUsableAuth(home: string): Promise<boolean> {
const authPath = path.join(home, "auth.json");
if (!(await pathExists(authPath))) return false;
try {
const raw = await fs.readFile(authPath, "utf8");
const parsed = JSON.parse(raw);
return hasUsableAuthPayload(parsed);
} catch {
return false;
}
}
async function codexHomeHasMatchingApiKeyAuth(home: string, apiKey: string): Promise<boolean> {
const authPath = path.join(home, "auth.json");
const existing = await fs.lstat(authPath).catch(() => null);
if (!existing || existing.isSymbolicLink()) return false;
try {
const raw = await fs.readFile(authPath, "utf8");
const parsed = JSON.parse(raw);
return readApiKeyFromAuthPayload(parsed) === apiKey.trim();
} catch {
return false;
}
}
async function ensureParentDir(target: string): Promise<void> {
await fs.mkdir(path.dirname(target), { recursive: true });
}
@@ -116,13 +192,20 @@ export async function writeApiKeyAuthJson(home: string, apiKey: string): Promise
await fs.writeFile(target, JSON.stringify({ OPENAI_API_KEY: apiKey }), { mode: 0o600 });
}
export async function prepareManagedCodexHome(
/**
* Seeds auth/config into an explicit Paperclip-managed `targetHome`. Symlinks
* `auth.json` from the shared source home (so ChatGPT-subscription credentials
* stay live and single-use refresh tokens are not copied), copies the static
* shared config files, and when an API key is supplied writes an API-key
* `auth.json` instead. Used both for the default company home and for the
* per-agent home set by the server isolation guard.
*/
export async function seedManagedCodexHome(
targetHome: string,
env: NodeJS.ProcessEnv,
onLog: AdapterExecutionContext["onLog"],
companyId?: string,
options: { apiKey?: string | null } = {},
): Promise<string> {
const targetHome = resolveManagedCodexHomeDir(env, companyId);
): Promise<void> {
const apiKey = nonEmpty(options.apiKey ?? undefined);
const sourceHome = resolveSharedCodexHomeDir(env);
@@ -168,6 +251,98 @@ export async function prepareManagedCodexHome(
`[paperclip] Wrote API-key auth.json into Codex home "${targetHome}" from configured OPENAI_API_KEY.\n`,
);
}
}
export async function prepareManagedCodexHome(
env: NodeJS.ProcessEnv,
onLog: AdapterExecutionContext["onLog"],
companyId?: string,
options: { apiKey?: string | null } = {},
): Promise<string> {
const targetHome = resolveManagedCodexHomeDir(env, companyId);
await seedManagedCodexHome(targetHome, env, onLog, options);
return targetHome;
}
export type ReconcileManagedCodexHomeStatus =
| "no_managed_home"
| "external_override"
| "already_seeded"
| "source_auth_missing"
| "seeded";
export interface ReconcileManagedCodexHomeInput {
companyId: string | undefined;
configuredCodexHome: string | null | undefined;
apiKey?: string | null;
/**
* Set when the agent's persisted `OPENAI_API_KEY` is a secret binding that
* could not be resolved in this context (e.g. startup reconciliation, which
* never resolves secrets). When true and the home already has usable auth,
* reconciliation preserves that auth instead of downgrading it to the shared
* subscription symlink.
*/
apiKeySecretBound?: boolean;
env?: NodeJS.ProcessEnv;
onLog?: AdapterExecutionContext["onLog"];
}
export interface ReconcileManagedCodexHomeResult {
status: ReconcileManagedCodexHomeStatus;
home: string | null;
}
const noopOnLog: AdapterExecutionContext["onLog"] = async () => {};
/**
* Idempotently reconciles a persisted `codex_local` agent home. Phase 1 seeds
* managed homes at execute time; this is the backfill for agents that already
* carry a persisted (but unseeded) per-agent `CODEX_HOME` and have not run
* since the seeding fix landed. Shares the managed-home detection
* (`isManagedCodexHomePath`) and seeding (`seedManagedCodexHome`) logic so a
* genuine external/user override is never touched. Safe to re-run: when a valid
* `auth.json` is already present (and no API-key rewrite is requested) it is a
* no-op and reports `already_seeded`.
*/
export async function reconcileManagedCodexHome(
input: ReconcileManagedCodexHomeInput,
): Promise<ReconcileManagedCodexHomeResult> {
const env = input.env ?? process.env;
const configured = nonEmpty(input.configuredCodexHome ?? undefined);
if (!configured) return { status: "no_managed_home", home: null };
const resolved = path.resolve(configured);
if (!isManagedCodexHomePath(env, input.companyId, resolved)) {
return { status: "external_override", home: resolved };
}
const apiKey = nonEmpty(input.apiKey ?? undefined);
const hadUsableAuth = await codexHomeHasUsableAuth(resolved);
// A secret-bound OPENAI_API_KEY cannot be resolved here, so we cannot rewrite
// it into auth.json. If the home already has usable auth — typically an
// API-key auth.json written at execute time when the secret WAS resolved —
// preserve it. Re-seeding without the key would delete that file and restore
// the shared subscription symlink, silently changing the agent's credentials
// on every boot while the persisted config still says "use the secret key".
if (input.apiKeySecretBound && hadUsableAuth) {
return { status: "already_seeded", home: resolved };
}
if (apiKey && await codexHomeHasMatchingApiKeyAuth(resolved, apiKey)) {
return { status: "already_seeded", home: resolved };
}
await seedManagedCodexHome(resolved, env, input.onLog ?? noopOnLog, { apiKey });
if (!apiKey && !(await codexHomeHasUsableAuth(resolved))) {
return { status: "source_auth_missing", home: resolved };
}
// Without an API key, seeding only changes disk state when auth was missing.
// With an API key, the matching-file short-circuit above filters out the
// already-seeded case before this write path.
const status: ReconcileManagedCodexHomeStatus =
!apiKey && hadUsableAuth ? "already_seeded" : "seeded";
return { status, home: resolved };
}
@@ -0,0 +1,77 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import { execute } from "./execute.js";
describe("codex managed-home auth fail-fast", () => {
const cleanupDirs: string[] = [];
afterEach(async () => {
vi.unstubAllEnvs();
while (cleanupDirs.length > 0) {
const dir = cleanupDirs.pop();
if (!dir) continue;
await fs.rm(dir, { recursive: true, force: true }).catch(() => undefined);
}
});
it("fails fast when a managed CODEX_HOME has no auth.json and OPENAI_API_KEY is empty", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-failfast-"));
cleanupDirs.push(root);
const paperclipHome = path.join(root, "paperclip-home");
const emptySharedHome = path.join(root, "shared-codex-home");
const workspaceDir = path.join(root, "workspace");
// A managed per-agent home with no credentials seeded into it.
const managedAgentHome = path.join(
paperclipHome,
"instances",
"default",
"companies",
"company-1",
"agents",
"agent-1",
"codex-home",
);
await fs.mkdir(emptySharedHome, { recursive: true });
await fs.mkdir(workspaceDir, { recursive: true });
// Source home has no auth.json, so nothing is symlinked into the managed home.
vi.stubEnv("PAPERCLIP_HOME", paperclipHome);
vi.stubEnv("PAPERCLIP_INSTANCE_ID", "default");
vi.stubEnv("CODEX_HOME", emptySharedHome);
await expect(
execute({
runId: "run-failfast",
agent: {
id: "agent-1",
companyId: "company-1",
name: "CodexCoder",
adapterType: "codex_local",
adapterConfig: {},
},
runtime: {
sessionId: null,
sessionParams: null,
sessionDisplayId: null,
taskKey: null,
},
config: {
command: "codex",
cwd: workspaceDir,
env: {
CODEX_HOME: managedAgentHome,
OPENAI_API_KEY: "",
},
},
context: {},
onLog: async () => {},
}),
).rejects.toThrow(/no Codex credentials provisioned for managed home/);
// The managed home must not have been left with a usable auth.json.
await expect(fs.access(path.join(managedAgentHome, "auth.json"))).rejects.toBeTruthy();
});
});
@@ -44,11 +44,25 @@ import {
isCodexTransientUpstreamError,
isCodexUnknownSessionError,
} from "./parse.js";
import { pathExists, prepareManagedCodexHome, resolveManagedCodexHomeDir, resolveSharedCodexHomeDir } from "./codex-home.js";
import {
codexHomeHasUsableAuth,
isManagedCodexHomePath,
pathExists,
prepareManagedCodexHome,
resolveManagedCodexHomeDir,
resolveSharedCodexHomeDir,
seedManagedCodexHome,
} from "./codex-home.js";
import { prepareCodexRuntimeConfig } from "./runtime-config.js";
import { resolveCodexDesiredSkillNames } from "./skills.js";
import { buildCodexExecArgs } from "./codex-args.js";
import { SANDBOX_INSTALL_COMMAND } from "../index.js";
import {
CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS,
createCodexOutputInactivityMonitor,
formatOutputInactivityMonitorErrorMessage,
resolveCodexInactivityTimeout,
} from "./output-inactivity-monitor.js";
const __moduleDir = path.dirname(fileURLToPath(import.meta.url));
const CODEX_ROLLOUT_NOISE_RE =
@@ -78,6 +92,29 @@ function firstNonEmptyLine(text: string): string {
);
}
function signalCodexChild(
target: { pid: number | null; processGroupId: number | null },
signal: NodeJS.Signals,
): boolean {
if (process.platform !== "win32" && target.processGroupId && target.processGroupId > 0) {
try {
process.kill(-target.processGroupId, signal);
return true;
} catch {
// Fall back to direct child signal if group signaling fails (e.g. group already gone).
}
}
if (target.pid && target.pid > 0) {
try {
process.kill(target.pid, signal);
return true;
} catch {
return false;
}
}
return false;
}
function hasNonEmptyEnvValue(env: Record<string, string>, key: string): boolean {
const raw = env[key];
return typeof raw === "string" && raw.trim().length > 0;
@@ -340,15 +377,44 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
typeof envConfig.OPENAI_API_KEY === "string" && envConfig.OPENAI_API_KEY.trim().length > 0
? envConfig.OPENAI_API_KEY.trim()
: null;
const preparedManagedCodexHome =
configuredCodexHome
? null
: await prepareManagedCodexHome(process.env, onLog, agent.companyId, {
apiKey: configuredOpenAiApiKey,
});
// A configured CODEX_HOME that lives under the Paperclip-managed company tree
// (the per-agent home set by the server isolation guard) still needs auth
// seeded — it ships with no credentials and OPENAI_API_KEY="" by default.
// Only a genuine external/user-supplied override is treated as self-managed
// and left untouched.
const configuredHomeIsManaged =
configuredCodexHome != null &&
isManagedCodexHomePath(process.env, agent.companyId, configuredCodexHome);
if (configuredCodexHome == null) {
await prepareManagedCodexHome(process.env, onLog, agent.companyId, {
apiKey: configuredOpenAiApiKey,
});
} else if (configuredHomeIsManaged) {
await seedManagedCodexHome(configuredCodexHome, process.env, onLog, {
apiKey: configuredOpenAiApiKey,
});
}
const defaultCodexHome = resolveManagedCodexHomeDir(process.env, agent.companyId);
const effectiveCodexHome = configuredCodexHome ?? preparedManagedCodexHome ?? defaultCodexHome;
const effectiveCodexHome = configuredCodexHome ?? defaultCodexHome;
await fs.mkdir(effectiveCodexHome, { recursive: true });
// Never launch a managed CODEX_HOME with no credentials. Without auth.json and
// with OPENAI_API_KEY="" the provider rejects every request with
// "401 Missing bearer"; fail fast with a clear adapter error instead of
// emitting unauthenticated calls. External overrides manage their own auth.
const effectiveHomeIsManaged = configuredCodexHome == null || configuredHomeIsManaged;
if (
effectiveHomeIsManaged &&
!configuredOpenAiApiKey &&
!(await codexHomeHasUsableAuth(effectiveCodexHome))
) {
throw new Error(
`no Codex credentials provisioned for managed home "${effectiveCodexHome}" ` +
`(no usable auth.json and OPENAI_API_KEY is empty). ` +
`Sign in to Codex on the host with a ChatGPT subscription, or configure a per-agent ` +
`OPENAI_API_KEY.`,
);
}
// Merge custom model providers (PAPERCLIP_CODEX_PROVIDERS) into the managed
// CODEX_HOME's config.toml BEFORE the home is shipped to a remote execution
// target, so both local and sandboxed Codex processes pick up the routing.
@@ -548,6 +614,18 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
resolvedCommand,
});
const monitorResolution = resolveCodexInactivityTimeout(config.outputInactivityTimeoutMs);
if (monitorResolution.mode === "disabled") {
await onLog(
"stdout",
`[paperclip] Codex output inactivity monitor is DISABLED via adapterConfig.outputInactivityTimeoutMs=null. Hung codex runs will only be detected by the platform-level silent-run safety net.\n`,
);
} else if (monitorResolution.mode === "default" && "reason" in monitorResolution) {
await onLog(
"stdout",
`[paperclip] Ignoring non-positive adapterConfig.outputInactivityTimeoutMs; falling back to default ${monitorResolution.timeoutMs}ms.\n`,
);
}
const runtimeSessionParams = parseObject(runtime.sessionParams);
const runtimeSessionId = asString(runtimeSessionParams.sessionId, runtime.sessionId ?? "");
const runtimeSessionCwd = asString(runtimeSessionParams.cwd, "");
@@ -729,39 +807,153 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
});
}
const proc = await runAdapterExecutionTargetProcess(runId, runtimeExecutionTarget, command, args, {
cwd,
env,
stdin: prompt,
timeoutSec,
graceSec,
onSpawn,
onLog: async (stream, chunk) => {
if (stream !== "stderr") {
await onLog(stream, chunk);
return;
}
const cleaned = stripCodexRolloutNoise(chunk);
if (!cleaned.trim()) return;
await onLog(stream, cleaned);
},
});
const cleanedStderr = stripCodexRolloutNoise(proc.stderr);
return {
proc: {
...proc,
stderr: cleanedStderr,
},
rawStderr: proc.stderr,
parsed: parseCodexJsonl(proc.stdout),
let monitorFired = false;
let monitorTerminationSignal: NodeJS.Signals | null = null;
let monitorElapsedMs = 0;
let monitorTimeoutMs = 0;
let killTarget: { pid: number | null; processGroupId: number | null } | null = null;
let sigkillTimer: ReturnType<typeof setTimeout> | null = null;
let monitorLogPromise: Promise<unknown> | null = null;
const monitor =
monitorResolution.mode === "disabled"
? null
: createCodexOutputInactivityMonitor({
timeoutMs: monitorResolution.timeoutMs,
onFire: (state) => {
monitorFired = true;
monitorElapsedMs = (state.firedAt ?? Date.now()) - state.lastEventAt;
monitorTimeoutMs = monitorResolution.timeoutMs;
const message = formatOutputInactivityMonitorErrorMessage(monitorElapsedMs);
const elapsedSec = Math.round(monitorElapsedMs / 1000);
const timeoutSecLabel = Math.round(monitorResolution.timeoutMs / 1000);
const logLine =
`[paperclip] adapter.invoke ${message}; ` +
`timeoutMs=${monitorResolution.timeoutMs} elapsedSinceLastEventMs=${monitorElapsedMs} ` +
`parsedEvents=${state.parsedEventCount} (timeout=${timeoutSecLabel}s elapsed=${elapsedSec}s); ` +
`terminating codex child via SIGTERM (5s grace, then SIGKILL).\n`;
// Issue the log without awaiting on the kill hot path, but capture
// the promise so the surrounding try/finally can await flush before
// the run resolves. Without this the diagnostic that explains the
// kill could be dropped if the child exits faster than onLog flushes.
monitorLogPromise = Promise.resolve(onLog("stderr", logLine)).catch(() => {});
const target = killTarget;
if (!target || (target.pid == null && target.processGroupId == null)) {
return;
}
const sentSig = signalCodexChild(target, "SIGTERM");
if (sentSig) monitorTerminationSignal = "SIGTERM";
sigkillTimer = setTimeout(() => {
sigkillTimer = null;
const stillSent = signalCodexChild(target, "SIGKILL");
if (stillSent) monitorTerminationSignal = "SIGKILL";
}, CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS);
if (typeof (sigkillTimer as { unref?: () => void }).unref === "function") {
(sigkillTimer as { unref: () => void }).unref();
}
},
});
const wrappedOnSpawn = async (meta: { pid: number; processGroupId: number | null; startedAt: string }) => {
killTarget = { pid: meta.pid ?? null, processGroupId: meta.processGroupId };
if (onSpawn) {
await onSpawn(meta);
}
};
try {
const proc = await runAdapterExecutionTargetProcess(runId, runtimeExecutionTarget, command, args, {
cwd,
env,
stdin: prompt,
timeoutSec,
graceSec,
onSpawn: wrappedOnSpawn,
onLog: async (stream, chunk) => {
if (stream === "stdout") {
monitor?.noteStdoutChunk(chunk);
await onLog(stream, chunk);
return;
}
const cleaned = stripCodexRolloutNoise(chunk);
if (!cleaned.trim()) return;
await onLog(stream, cleaned);
},
});
const cleanedStderr = stripCodexRolloutNoise(proc.stderr);
return {
proc: {
...proc,
stderr: cleanedStderr,
},
rawStderr: proc.stderr,
parsed: parseCodexJsonl(proc.stdout),
monitor: monitorFired
? {
fired: true as const,
terminationSignal: monitorTerminationSignal,
elapsedMsSinceLastEvent: monitorElapsedMs,
timeoutMs: monitorTimeoutMs,
}
: { fired: false as const },
};
} finally {
monitor?.stop();
if (sigkillTimer) {
clearTimeout(sigkillTimer);
sigkillTimer = null;
}
if (monitorLogPromise) {
await monitorLogPromise;
monitorLogPromise = null;
}
}
};
const toResult = (
attempt: { proc: { exitCode: number | null; signal: string | null; timedOut: boolean; stdout: string; stderr: string }; rawStderr: string; parsed: ReturnType<typeof parseCodexJsonl> },
attempt: {
proc: { exitCode: number | null; signal: string | null; timedOut: boolean; stdout: string; stderr: string };
rawStderr: string;
parsed: ReturnType<typeof parseCodexJsonl>;
monitor?:
| { fired: false }
| { fired: true; terminationSignal: NodeJS.Signals | null; elapsedMsSinceLastEvent: number; timeoutMs: number };
},
clearSessionOnMissingSession = false,
isRetry = false,
): AdapterExecutionResult => {
if (attempt.monitor?.fired) {
const errorMessage = formatOutputInactivityMonitorErrorMessage(attempt.monitor.elapsedMsSinceLastEvent);
return {
exitCode: null,
signal: attempt.monitor.terminationSignal ?? attempt.proc.signal,
timedOut: false,
errorMessage,
errorCode: "codex_output_inactivity_monitor",
errorFamily: null,
usage: attempt.parsed.usage,
sessionId: null,
sessionParams: null,
sessionDisplayId: null,
provider: "openai",
biller: resolveCodexBiller(effectiveEnv, billingType),
model,
billingType,
costUsd: null,
resultJson: {
stdout: attempt.proc.stdout,
stderr: attempt.proc.stderr,
outputInactivityMonitor: {
kind: "output_inactivity",
timeoutMs: attempt.monitor.timeoutMs,
elapsedMsSinceLastEvent: attempt.monitor.elapsedMsSinceLastEvent,
terminationSignal: attempt.monitor.terminationSignal,
},
},
summary: attempt.parsed.summary,
clearSession: clearSessionOnMissingSession,
};
}
if (attempt.proc.timedOut) {
return {
exitCode: attempt.proc.exitCode,
@@ -1,4 +1,11 @@
export { execute, ensureCodexSkillsInjected } from "./execute.js";
export {
reconcileManagedCodexHome,
isManagedCodexHomePath,
type ReconcileManagedCodexHomeInput,
type ReconcileManagedCodexHomeResult,
type ReconcileManagedCodexHomeStatus,
} from "./codex-home.js";
export { listCodexSkills, syncCodexSkills } from "./skills.js";
export { testEnvironment } from "./test.js";
export { parseCodexJsonl, isCodexTransientUpstreamError, isCodexUnknownSessionError } from "./parse.js";
@@ -0,0 +1,100 @@
import { describe, expect, it } from "vitest";
import { runChildProcess } from "@paperclipai/adapter-utils/server-utils";
import {
CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS,
createCodexOutputInactivityMonitor,
formatOutputInactivityMonitorErrorMessage,
} from "./output-inactivity-monitor.js";
const FAKE_CODEX_SCRIPT = `
process.stdout.write(JSON.stringify({ type: "thread.started", thread_id: "abc" }) + "\\n");
// Simulate a wedged codex: read stdin forever, never write again.
process.stdin.resume();
process.stdin.on("data", () => {});
setInterval(() => {}, 60_000);
`;
describe("codex inactivity monitor (integration: real subprocess)", () => {
it(
"kills a codex child that goes silent after one event and surfaces a monitor failure",
async () => {
const runId = `monitor-integration-${Date.now()}`;
const timeoutMs = 250;
const logs: Array<{ stream: string; chunk: string }> = [];
let killTarget: { pid: number | null; processGroupId: number | null } | null = null;
let monitorFired = false;
let terminationSignal: NodeJS.Signals | null = null;
let sigkillTimer: ReturnType<typeof setTimeout> | null = null;
let elapsedMs = 0;
const kill = (signal: NodeJS.Signals) => {
const target = killTarget;
if (!target) return false;
if (target.processGroupId && target.processGroupId > 0) {
try {
process.kill(-target.processGroupId, signal);
return true;
} catch {
/* fall through */
}
}
if (target.pid && target.pid > 0) {
try {
process.kill(target.pid, signal);
return true;
} catch {
return false;
}
}
return false;
};
const monitor = createCodexOutputInactivityMonitor({
timeoutMs,
onFire: (state) => {
monitorFired = true;
elapsedMs = (state.firedAt ?? Date.now()) - state.lastEventAt;
if (kill("SIGTERM")) terminationSignal = "SIGTERM";
sigkillTimer = setTimeout(() => {
if (kill("SIGKILL")) terminationSignal = "SIGKILL";
}, CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS);
},
});
try {
const proc = await runChildProcess(runId, process.execPath, ["-e", FAKE_CODEX_SCRIPT], {
cwd: process.cwd(),
env: process.env as Record<string, string>,
timeoutSec: 30,
graceSec: 1,
onSpawn: async (meta) => {
killTarget = { pid: meta.pid, processGroupId: meta.processGroupId };
},
onLog: async (stream, chunk) => {
logs.push({ stream, chunk });
if (stream === "stdout") {
monitor.noteStdoutChunk(chunk);
}
},
});
expect(monitorFired, "monitor should fire when codex goes silent").toBe(true);
// Process was killed by our signal, not by hitting timeoutSec.
expect(proc.timedOut).toBe(false);
expect(["SIGTERM", "SIGKILL"]).toContain(proc.signal);
expect(["SIGTERM", "SIGKILL"]).toContain(terminationSignal);
// The errorMessage shape mirrors the AdapterExecutionResult that
// execute.ts will produce for this case.
expect(formatOutputInactivityMonitorErrorMessage(elapsedMs)).toMatch(
/^monitor: no codex output for \d+m \d+s$/,
);
// We should have observed exactly one parsed JSONL event before silence.
expect(monitor.state().parsedEventCount).toBe(1);
} finally {
monitor.stop();
if (sigkillTimer) clearTimeout(sigkillTimer);
}
},
15_000,
);
});
@@ -0,0 +1,260 @@
import { describe, expect, it } from "vitest";
import {
CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS,
DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS,
createCodexOutputInactivityMonitor,
formatOutputInactivityMonitorErrorMessage,
resolveCodexInactivityTimeout,
} from "./output-inactivity-monitor.js";
class FakeClock {
private nowMs = 0;
private nextHandle = 1;
private timers = new Map<number, { fireAt: number; cb: () => void }>();
now(): number {
return this.nowMs;
}
setTimer(cb: () => void, ms: number): number {
const handle = this.nextHandle++;
this.timers.set(handle, { fireAt: this.nowMs + ms, cb });
return handle;
}
clearTimer(handle: unknown): void {
if (typeof handle === "number") this.timers.delete(handle);
}
advance(ms: number): void {
const targetMs = this.nowMs + ms;
while (true) {
let nextHandle: number | null = null;
let nextTimer: { fireAt: number; cb: () => void } | null = null;
for (const [h, timer] of this.timers) {
if (timer.fireAt <= targetMs && (!nextTimer || timer.fireAt < nextTimer.fireAt)) {
nextHandle = h;
nextTimer = timer;
}
}
if (!nextTimer || nextHandle == null) break;
this.timers.delete(nextHandle);
this.nowMs = nextTimer.fireAt;
nextTimer.cb();
}
this.nowMs = targetMs;
}
pendingTimerCount(): number {
return this.timers.size;
}
}
describe("resolveCodexInactivityTimeout", () => {
it("uses default when value is unset", () => {
expect(resolveCodexInactivityTimeout(undefined)).toEqual({
mode: "default",
timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS,
});
});
it("treats explicit null as disabled", () => {
expect(resolveCodexInactivityTimeout(null)).toEqual({
mode: "disabled",
reason: "explicit_null",
});
});
it("returns configured value for positive numbers", () => {
expect(resolveCodexInactivityTimeout(12_000)).toEqual({
mode: "configured",
timeoutMs: 12_000,
});
});
it("falls back to default for non-positive numbers", () => {
expect(resolveCodexInactivityTimeout(0)).toEqual({
mode: "default",
timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS,
reason: "non_positive",
});
expect(resolveCodexInactivityTimeout(-100)).toEqual({
mode: "default",
timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS,
reason: "non_positive",
});
});
it("falls back to default for non-number, non-null values", () => {
expect(resolveCodexInactivityTimeout("420000")).toEqual({
mode: "default",
timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS,
});
});
});
describe("formatOutputInactivityMonitorErrorMessage", () => {
it("formats minutes and seconds", () => {
expect(formatOutputInactivityMonitorErrorMessage(0)).toBe("monitor: no codex output for 0m 0s");
expect(formatOutputInactivityMonitorErrorMessage(7 * 60 * 1000)).toBe("monitor: no codex output for 7m 0s");
expect(formatOutputInactivityMonitorErrorMessage(7 * 60 * 1000 + 12_000)).toBe("monitor: no codex output for 7m 12s");
expect(formatOutputInactivityMonitorErrorMessage(45_000)).toBe("monitor: no codex output for 0m 45s");
});
});
describe("createCodexOutputInactivityMonitor (acceptance criteria 1: fires)", () => {
it("fires after timeoutMs when child emits one event then goes silent", () => {
const clock = new FakeClock();
const fires: Array<{ elapsed: number; parsedEventCount: number }> = [];
const monitor = createCodexOutputInactivityMonitor({
timeoutMs: 7 * 60 * 1000,
now: () => clock.now(),
setTimer: (cb, ms) => clock.setTimer(cb, ms),
clearTimer: (handle) => clock.clearTimer(handle),
onFire: (state) => {
fires.push({
elapsed: (state.firedAt ?? 0) - state.lastEventAt,
parsedEventCount: state.parsedEventCount,
});
},
});
// One event right after spawn.
clock.advance(50);
monitor.noteStdoutChunk('{"type":"thread.started","thread_id":"abc"}\n');
expect(fires).toHaveLength(0);
expect(monitor.state().parsedEventCount).toBe(1);
// Now go silent for 7 minutes; monitor should fire exactly at threshold.
clock.advance(7 * 60 * 1000 - 1);
expect(fires).toHaveLength(0);
clock.advance(1);
expect(fires).toHaveLength(1);
expect(fires[0].elapsed).toBe(7 * 60 * 1000);
expect(fires[0].parsedEventCount).toBe(1);
// Stopping after fire is a no-op for the timer but returns final state.
const finalState = monitor.stop();
expect(finalState.fired).toBe(true);
});
it("only fires once even if more silence elapses after firing", () => {
const clock = new FakeClock();
let fireCount = 0;
const monitor = createCodexOutputInactivityMonitor({
timeoutMs: 1_000,
now: () => clock.now(),
setTimer: (cb, ms) => clock.setTimer(cb, ms),
clearTimer: (handle) => clock.clearTimer(handle),
onFire: () => {
fireCount += 1;
},
});
clock.advance(2_000);
expect(fireCount).toBe(1);
clock.advance(10_000);
expect(fireCount).toBe(1);
monitor.stop();
});
it("ignores non-JSON lines when resetting the timer", () => {
const clock = new FakeClock();
let fireCount = 0;
const monitor = createCodexOutputInactivityMonitor({
timeoutMs: 1_000,
now: () => clock.now(),
setTimer: (cb, ms) => clock.setTimer(cb, ms),
clearTimer: (handle) => clock.clearTimer(handle),
onFire: () => {
fireCount += 1;
},
});
// Plain stderr-ish text should NOT reset the monitor.
clock.advance(500);
monitor.noteStdoutChunk("loading model...\n");
expect(monitor.state().parsedEventCount).toBe(0);
clock.advance(600);
expect(fireCount).toBe(1);
monitor.stop();
});
});
describe("createCodexOutputInactivityMonitor (acceptance criteria 2: does not fire)", () => {
it("does not fire when events arrive every (threshold - 1s)", () => {
const clock = new FakeClock();
let fireCount = 0;
const timeoutMs = 7 * 60 * 1000;
const monitor = createCodexOutputInactivityMonitor({
timeoutMs,
now: () => clock.now(),
setTimer: (cb, ms) => clock.setTimer(cb, ms),
clearTimer: (handle) => clock.clearTimer(handle),
onFire: () => {
fireCount += 1;
},
});
// Pump events at threshold-1s intervals for 12 cycles (~84 minutes).
for (let i = 0; i < 12; i += 1) {
clock.advance(timeoutMs - 1_000);
monitor.noteStdoutChunk(`{"type":"item.completed","item":{"type":"agent_message","text":"tick ${i}"}}\n`);
expect(fireCount).toBe(0);
}
// Final event lets us "complete" — total state shows 12 parsed events.
expect(monitor.state().parsedEventCount).toBe(12);
expect(fireCount).toBe(0);
// Stop cleanly before the timer would have fired.
monitor.stop();
expect(fireCount).toBe(0);
});
it("multiple events in one chunk all reset the timer", () => {
const clock = new FakeClock();
let fireCount = 0;
const monitor = createCodexOutputInactivityMonitor({
timeoutMs: 1_000,
now: () => clock.now(),
setTimer: (cb, ms) => clock.setTimer(cb, ms),
clearTimer: (handle) => clock.clearTimer(handle),
onFire: () => {
fireCount += 1;
},
});
clock.advance(500);
monitor.noteStdoutChunk(
'{"type":"thread.started","thread_id":"a"}\n{"type":"item.completed","item":{"type":"agent_message","text":"hi"}}\n',
);
expect(monitor.state().parsedEventCount).toBe(2);
// Now wait 999ms — should still not fire.
clock.advance(999);
expect(fireCount).toBe(0);
// Wait one more ms — fires now.
clock.advance(1);
expect(fireCount).toBe(1);
monitor.stop();
});
});
describe("createCodexOutputInactivityMonitor (acceptance criteria 3: disabled)", () => {
it("resolveCodexInactivityTimeout returns disabled for null and the adapter creates no monitor", () => {
const resolution = resolveCodexInactivityTimeout(null);
expect(resolution.mode).toBe("disabled");
// Sanity: when a caller (execute.ts) honors `disabled`, it must not
// construct a monitor at all. Verify the constructor would otherwise
// require a positive timeoutMs.
expect(() =>
createCodexOutputInactivityMonitor({
timeoutMs: 0,
onFire: () => {},
}),
).toThrow(/timeoutMs > 0/);
});
});
describe("CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS", () => {
it("matches the 5-second grace window required by NEE-81", () => {
expect(CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS).toBe(5_000);
});
});
@@ -0,0 +1,142 @@
import { parseJson } from "@paperclipai/adapter-utils/server-utils";
export const DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS = 7 * 60 * 1000;
export const CODEX_OUTPUT_INACTIVITY_MONITOR_SIGTERM_GRACE_MS = 5_000;
export type CodexOutputInactivityMonitorResolution =
| { mode: "default"; timeoutMs: number }
| { mode: "configured"; timeoutMs: number }
| { mode: "disabled"; reason: "explicit_null" }
| { mode: "default"; timeoutMs: number; reason: "non_positive" };
/**
* Resolve the inactivity monitor timeout from raw adapter config.
*
* - `null` disabled (explicit escape hatch).
* - missing/`undefined` default 7m.
* - number > 0 configured value.
* - number 0 default 7m (and a `non_positive` note for logging).
*/
export function resolveCodexInactivityTimeout(rawValue: unknown): CodexOutputInactivityMonitorResolution {
if (rawValue === null) return { mode: "disabled", reason: "explicit_null" };
if (typeof rawValue === "number" && Number.isFinite(rawValue)) {
if (rawValue > 0) return { mode: "configured", timeoutMs: rawValue };
return { mode: "default", timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS, reason: "non_positive" };
}
return { mode: "default", timeoutMs: DEFAULT_CODEX_OUTPUT_INACTIVITY_TIMEOUT_MS };
}
export interface CodexOutputInactivityMonitorState {
fired: boolean;
spawnedAt: number;
lastEventAt: number;
firedAt: number | null;
parsedEventCount: number;
}
export interface CodexOutputInactivityMonitorOptions {
timeoutMs: number;
onFire: (state: CodexOutputInactivityMonitorState) => void;
now?: () => number;
setTimer?: (cb: () => void, ms: number) => unknown;
clearTimer?: (handle: unknown) => void;
/**
* Per-line predicate. When omitted, any line that successfully parses as
* JSON via the codex JSONL parser counts as a heartbeat event.
*/
isHeartbeatLine?: (line: string) => boolean;
}
export interface CodexOutputInactivityMonitorHandle {
noteStdoutChunk(chunk: string): void;
/** Returns the current state without stopping the timer. */
state(): CodexOutputInactivityMonitorState;
/** Cancels any pending timer and returns the final state. */
stop(): CodexOutputInactivityMonitorState;
}
function defaultIsHeartbeatLine(line: string): boolean {
const trimmed = line.trim();
if (!trimmed) return false;
return parseJson(trimmed) !== null;
}
export function createCodexOutputInactivityMonitor(
options: CodexOutputInactivityMonitorOptions,
): CodexOutputInactivityMonitorHandle {
const now = options.now ?? (() => Date.now());
const setTimer = options.setTimer ?? ((cb, ms) => setTimeout(cb, ms));
const clearTimer = options.clearTimer ?? ((h) => clearTimeout(h as ReturnType<typeof setTimeout>));
const isHeartbeatLine = options.isHeartbeatLine ?? defaultIsHeartbeatLine;
const timeoutMs = options.timeoutMs;
if (!(timeoutMs > 0)) {
throw new Error(`createCodexOutputInactivityMonitor requires timeoutMs > 0 (got ${timeoutMs})`);
}
const spawnedAt = now();
const state: CodexOutputInactivityMonitorState = {
fired: false,
spawnedAt,
lastEventAt: spawnedAt,
firedAt: null,
parsedEventCount: 0,
};
let timerHandle: unknown = null;
let stopped = false;
const fire = () => {
if (state.fired || stopped) return;
state.fired = true;
state.firedAt = now();
timerHandle = null;
options.onFire({ ...state });
};
const arm = () => {
if (stopped || state.fired) return;
if (timerHandle != null) clearTimer(timerHandle);
timerHandle = setTimer(fire, timeoutMs);
};
arm();
return {
noteStdoutChunk(chunk: string) {
if (stopped || state.fired) return;
let sawHeartbeat = false;
for (const rawLine of chunk.split(/\r?\n/)) {
if (isHeartbeatLine(rawLine)) {
sawHeartbeat = true;
state.parsedEventCount += 1;
}
}
if (sawHeartbeat) {
state.lastEventAt = now();
arm();
}
},
state() {
return { ...state };
},
stop() {
stopped = true;
if (timerHandle != null) {
clearTimer(timerHandle);
timerHandle = null;
}
return { ...state };
},
};
}
/**
* Format the inactivity monitor error message in the canonical
* `monitor: no codex output for {N}m {S}s` shape consumed by NEE-81.
*/
export function formatOutputInactivityMonitorErrorMessage(elapsedMs: number): string {
const total = Math.max(0, Math.round(elapsedMs / 1000));
const minutes = Math.floor(total / 60);
const seconds = total - minutes * 60;
return `monitor: no codex output for ${minutes}m ${seconds}s`;
}
@@ -57,10 +57,7 @@ export async function syncCursorSkills(
desiredSkills: string[],
): Promise<AdapterSkillSnapshot> {
const availableEntries = await readPaperclipRuntimeSkillEntries(ctx.config, __moduleDir);
const desiredSet = new Set([
...desiredSkills,
...availableEntries.filter((entry) => entry.required).map((entry) => entry.key),
]);
const desiredSet = new Set(desiredSkills);
const skillsHome = resolveCursorSkillsHome(ctx.config);
await fs.mkdir(skillsHome, { recursive: true });
const installed = await readInstalledSkillTargets(skillsHome);
@@ -85,7 +82,7 @@ export async function syncCursorSkills(
export function resolveCursorDesiredSkillNames(
config: Record<string, unknown>,
availableEntries: Array<{ key: string; required?: boolean }>,
availableEntries: Array<{ key: string }>,
) {
return resolvePaperclipDesiredSkillNames(config, availableEntries);
}
@@ -57,10 +57,7 @@ export async function syncGeminiSkills(
desiredSkills: string[],
): Promise<AdapterSkillSnapshot> {
const availableEntries = await readPaperclipRuntimeSkillEntries(ctx.config, __moduleDir);
const desiredSet = new Set([
...desiredSkills,
...availableEntries.filter((entry) => entry.required).map((entry) => entry.key),
]);
const desiredSet = new Set(desiredSkills);
const skillsHome = resolveGeminiSkillsHome(ctx.config);
await fs.mkdir(skillsHome, { recursive: true });
const installed = await readInstalledSkillTargets(skillsHome);
@@ -85,7 +82,7 @@ export async function syncGeminiSkills(
export function resolveGeminiDesiredSkillNames(
config: Record<string, unknown>,
availableEntries: Array<{ key: string; required?: boolean }>,
availableEntries: Array<{ key: string }>,
) {
return resolvePaperclipDesiredSkillNames(config, availableEntries);
}
@@ -1,6 +1,10 @@
import { afterEach, describe, expect, it } from "vitest";
import { ensureRemoteOpenCodeModelConfiguredAndAvailable } from "./execute.js";
import {
OPENCODE_PROMPT_MIN_LENGTH_CHARS,
padShortPrompt,
} from "./execute.js";
describe("ensureRemoteOpenCodeModelConfiguredAndAvailable", () => {
afterEach(() => {
@@ -60,3 +64,56 @@ describe("ensureRemoteOpenCodeModelConfiguredAndAvailable", () => {
).rejects.toThrow();
});
});
describe("padShortPrompt (ORA-284)", () => {
// opencode 1.16.2 has a non-deterministic silent-exit mode that fires on
// short stdin prompts. The pad forces every prompt above the threshold so
// the binary boots into a normal session instead of exiting cleanly with
// 0 bytes of output.
it("returns the input untouched when it is already at or above the floor", () => {
const longPrompt = "x".repeat(OPENCODE_PROMPT_MIN_LENGTH_CHARS);
const out = padShortPrompt({
prompt: longPrompt,
agentId: "agent-1",
runId: "run-1",
executionCwd: "/tmp",
});
expect(out).toBe(longPrompt);
});
it("pads prompts that are below the floor and embeds run / agent context", () => {
const out = padShortPrompt({
prompt: "wake payload only",
agentId: "agent-1",
runId: "run-1",
executionCwd: "/tmp/oracle",
});
expect(out.length).toBeGreaterThanOrEqual(OPENCODE_PROMPT_MIN_LENGTH_CHARS);
expect(out).toContain("run-1");
expect(out).toContain("agent-1");
expect(out).toContain("/tmp/oracle");
expect(out).toContain("ORA-284");
});
it("uses a default cwd marker when executionCwd is empty", () => {
const out = padShortPrompt({
prompt: "tiny",
agentId: "agent-2",
runId: "run-2",
executionCwd: "",
});
expect(out).toContain("(default)");
});
it("does not insert a duplicate blank line when the prompt already ends with newline", () => {
const out = padShortPrompt({
prompt: "tiny\n",
agentId: "agent-3",
runId: "run-3",
executionCwd: "/tmp",
});
// Pad block already begins with a leading blank line, so we tolerate one
// extra newline rather than concatenating two of them.
expect(out.startsWith("tiny\n\n## Paperclip Runtime Context")).toBe(true);
});
});
@@ -66,6 +66,39 @@ function firstNonEmptyLine(text: string): string {
);
}
// ORA-284: opencode 1.16.2 has a non-deterministic silent-exit mode that fires
// on short stdin prompts (observed 0-byte / <200-byte run logs in production,
// ~32% stall rate on FoundingEngineer wake-payload-only runs). The root cause
// lives in the opencode binary, not in the adapter. As a server-side workaround
// we pad any prompt shorter than this floor with a deterministic, content-
// meaningful block drawn from the run context so the prompt is well above the
// observed failure threshold (~50 bytes, with a 4x safety margin).
export const OPENCODE_PROMPT_MIN_LENGTH_CHARS = 256;
export function padShortPrompt(input: {
prompt: string;
agentId: string;
runId: string;
executionCwd: string;
}): string {
const { prompt, agentId, runId, executionCwd } = input;
if (prompt.length >= OPENCODE_PROMPT_MIN_LENGTH_CHARS) return prompt;
const padBlock = [
"",
"## Paperclip Runtime Context",
"",
`Run: \`${runId}\``,
`Agent: \`${agentId}\``,
`Cwd: \`${executionCwd || "(default)"}\``,
"",
"You are running as a Paperclip-managed agent. The previous prompt was",
`shorter than the ${OPENCODE_PROMPT_MIN_LENGTH_CHARS}-byte opencode 1.16.2`,
"stability floor (ORA-284), so this metadata block was appended to avoid the",
"binary's silent-exit mode. Continue with the task described above.",
].join("\n");
return prompt.endsWith("\n") ? prompt + padBlock : prompt + "\n" + padBlock;
}
function parseModelProvider(model: string | null): string | null {
if (!model) return null;
const trimmed = model.trim();
@@ -546,20 +579,29 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
const shouldUseResumeDeltaPrompt = Boolean(sessionId) && wakePrompt.length > 0;
const renderedPrompt = shouldUseResumeDeltaPrompt ? "" : renderTemplate(promptTemplate, templateData);
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
const prompt = joinPromptSections([
const joinedSections = joinPromptSections([
instructionsPrefix,
renderedBootstrapPrompt,
wakePrompt,
sessionHandoffNote,
renderedPrompt,
]);
const promptMetrics = {
const promptWasPadded = joinedSections.length < OPENCODE_PROMPT_MIN_LENGTH_CHARS;
const prompt = padShortPrompt({
prompt: joinedSections,
agentId: agent.id,
runId,
executionCwd: effectiveExecutionCwd,
});
const promptMetrics: Record<string, number> = {
promptChars: prompt.length,
instructionsChars: instructionsPrefix.length,
bootstrapPromptChars: renderedBootstrapPrompt.length,
wakePromptChars: wakePrompt.length,
sessionHandoffChars: sessionHandoffNote.length,
heartbeatPromptChars: renderedPrompt.length,
// ORA-284: 1 when the short-prompt pad block was appended, 0 otherwise.
promptPadded: promptWasPadded ? 1 : 0,
};
// Optional diagnostic: surface OpenCode's own logs on stderr (captured into the
@@ -61,10 +61,7 @@ export async function syncOpenCodeSkills(
desiredSkills: string[],
): Promise<AdapterSkillSnapshot> {
const availableEntries = await readPaperclipRuntimeSkillEntries(ctx.config, __moduleDir);
const desiredSet = new Set([
...desiredSkills,
...availableEntries.filter((entry) => entry.required).map((entry) => entry.key),
]);
const desiredSet = new Set(desiredSkills);
const skillsHome = resolveOpenCodeSkillsHome(ctx.config);
await fs.mkdir(skillsHome, { recursive: true });
const installed = await readInstalledSkillTargets(skillsHome);
@@ -89,7 +86,7 @@ export async function syncOpenCodeSkills(
export function resolveOpenCodeDesiredSkillNames(
config: Record<string, unknown>,
availableEntries: Array<{ key: string; required?: boolean }>,
availableEntries: Array<{ key: string }>,
) {
return resolvePaperclipDesiredSkillNames(config, availableEntries);
}
@@ -57,10 +57,7 @@ export async function syncPiSkills(
desiredSkills: string[],
): Promise<AdapterSkillSnapshot> {
const availableEntries = await readPaperclipRuntimeSkillEntries(ctx.config, __moduleDir);
const desiredSet = new Set([
...desiredSkills,
...availableEntries.filter((entry) => entry.required).map((entry) => entry.key),
]);
const desiredSet = new Set(desiredSkills);
const skillsHome = resolvePiSkillsHome(ctx.config);
await fs.mkdir(skillsHome, { recursive: true });
const installed = await readInstalledSkillTargets(skillsHome);
@@ -85,7 +82,7 @@ export async function syncPiSkills(
export function resolvePiDesiredSkillNames(
config: Record<string, unknown>,
availableEntries: Array<{ key: string; required?: boolean }>,
availableEntries: Array<{ key: string }>,
) {
return resolvePaperclipDesiredSkillNames(config, availableEntries);
}
@@ -56,20 +56,25 @@ const manifest: PaperclipPluginManifestV1 = {
"Optional Daytona language hint for direct code execution. If omitted, Daytona uses its default runtime.",
},
cpu: {
type: "number",
type: "integer",
description: "Optional CPU allocation in cores.",
minimum: 1,
},
memory: {
type: "number",
description: "Optional memory allocation in GiB.",
type: "integer",
description:
"Optional memory allocation in GiB. Leave unset to use Daytona defaults; supported sandbox sizes are 1, 2, 4, and 8 GiB.",
enum: [1, 2, 4, 8],
},
disk: {
type: "number",
type: "integer",
description: "Optional disk allocation in GiB.",
minimum: 1,
},
gpu: {
type: "number",
type: "integer",
description: "Optional GPU allocation in units.",
minimum: 1,
},
timeoutMs: {
type: "number",
@@ -19,6 +19,7 @@ vi.mock("@daytonaio/sdk", () => ({
}));
import plugin from "./plugin.js";
import manifest from "./manifest.js";
function createMockSandbox(overrides: {
id?: string;
@@ -497,3 +498,24 @@ describe("Daytona sandbox provider plugin", () => {
});
});
});
describe("daytona manifest memory config", () => {
const memorySchema = (
manifest.environmentDrivers?.[0]?.configSchema as {
properties?: Record<string, { type?: string; enum?: unknown[] }>;
required?: string[];
}
);
it("offers memory as a fixed dropdown of supported sandbox sizes", () => {
expect(memorySchema.properties?.memory?.enum).toEqual([1, 2, 4, 8]);
});
it("excludes 0 — an invalid Daytona memory configuration", () => {
expect(memorySchema.properties?.memory?.enum).not.toContain(0);
});
it("keeps memory optional so the blank/default selection stays valid", () => {
expect(memorySchema.required ?? []).not.toContain("memory");
});
});
@@ -10,7 +10,6 @@ export type AgentSkillState =
export type AgentSkillOrigin =
| "company_managed"
| "paperclip_required"
| "user_installed"
| "external_unknown";
@@ -26,8 +25,6 @@ export interface AgentSkillEntry {
currentVersionId?: string | null;
desired: boolean;
managed: boolean;
required?: boolean;
requiredReason?: string | null;
state: AgentSkillState;
origin?: AgentSkillOrigin;
originLabel?: string | null;
@@ -11,7 +11,6 @@ export const agentSkillStateSchema = z.enum([
export const agentSkillOriginSchema = z.enum([
"company_managed",
"paperclip_required",
"user_installed",
"external_unknown",
]);
@@ -39,8 +38,6 @@ export const agentSkillEntrySchema = z.object({
currentVersionId: z.string().uuid().nullable().optional(),
desired: z.boolean(),
managed: z.boolean(),
required: z.boolean().optional(),
requiredReason: z.string().nullable().optional(),
state: agentSkillStateSchema,
origin: agentSkillOriginSchema.optional(),
originLabel: z.string().nullable().optional(),
@@ -128,12 +128,11 @@ async function createRuntimeSkill(root: string, input: {
const key = input.key ?? `company/${runtimeName}`;
const source = path.join(root, "skills", runtimeName);
await fs.mkdir(source, { recursive: true });
await fs.writeFile(path.join(source, "SKILL.md"), input.body ?? "---\nrequired: false\n---\nUse the test skill.\n", "utf8");
await fs.writeFile(path.join(source, "SKILL.md"), input.body ?? "---\n---\nUse the test skill.\n", "utf8");
return {
key,
runtimeName,
source,
required: false,
};
}
@@ -634,7 +633,7 @@ describe("acpx_local execute", () => {
it("includes skill content in the ACPX Claude session fingerprint", async () => {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-acpx-claude-fingerprint-"));
try {
const skill = await createRuntimeSkill(root, { body: "---\nrequired: false\n---\nFirst version.\n" });
const skill = await createRuntimeSkill(root, { body: "---\n---\nFirst version.\n" });
const runtimes: FakeRuntime[] = [];
const execute = createAcpxLocalExecutor({
createRuntime: (options) => {
@@ -657,7 +656,7 @@ describe("acpx_local execute", () => {
});
const first = await execute(context);
await fs.writeFile(path.join(skill.source, "SKILL.md"), "---\nrequired: false\n---\nSecond version.\n", "utf8");
await fs.writeFile(path.join(skill.source, "SKILL.md"), "---\n---\nSecond version.\n", "utf8");
const second = await execute({
...context,
runtime: {
@@ -6,7 +6,6 @@ import {
describe("acpx local skill sync", () => {
const paperclipKey = "paperclipai/paperclip/paperclip";
const createAgentKey = "paperclipai/paperclip/paperclip-create-agent";
it("reports ACPX Claude skills as supported runtime-mounted state", async () => {
const snapshot = await listAcpxSkills({
@@ -25,7 +24,6 @@ describe("acpx local skill sync", () => {
expect(snapshot.supported).toBe(true);
expect(snapshot.mode).toBe("ephemeral");
expect(snapshot.desiredSkills).toContain(paperclipKey);
expect(snapshot.desiredSkills).toContain(createAgentKey);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("configured");
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.detail).toContain("ACPX Claude session");
expect(snapshot.warnings).toEqual([]);
@@ -297,7 +297,7 @@ describe("agent routes adapter validation", () => {
expect(res.body.adapterType).toBe("external_test");
});
it("adds isolated CODEX_HOME and empty OPENAI_API_KEY override when creating codex_local agents", async () => {
it("does not inject CODEX_HOME or OPENAI_API_KEY when creating a keyless codex_local agent", async () => {
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
@@ -311,16 +311,13 @@ describe("agent routes adapter validation", () => {
expect(res.status, JSON.stringify(res.body)).toBe(201);
const createInput = mockAgentService.create.mock.calls.at(-1)?.[1] as Record<string, unknown>;
const agentId = String(createInput.id);
expect(agentId).toMatch(/^[0-9a-f-]{36}$/i);
const adapterConfig = createInput.adapterConfig as Record<string, unknown>;
const env = adapterConfig.env as Record<string, unknown>;
expect(env.OPENAI_API_KEY).toBe("");
expect(env.CODEX_HOME).toContain(`/companies/company-1/agents/${agentId}/codex-home`);
expect(String(env.CODEX_HOME)).not.toContain("/companies/company-1/codex-home");
const env = (adapterConfig.env as Record<string, unknown> | undefined) ?? {};
expect(env.OPENAI_API_KEY).toBeUndefined();
expect(env.CODEX_HOME).toBeUndefined();
});
it("adds isolated CODEX_HOME and empty OPENAI_API_KEY override when updating codex_local agents", async () => {
it("does not re-inject CODEX_HOME or OPENAI_API_KEY when updating a keyless codex_local agent", async () => {
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
@@ -333,16 +330,37 @@ describe("agent routes adapter validation", () => {
expect(res.status, JSON.stringify(res.body)).toBe(200);
const patch = mockAgentService.update.mock.calls.at(-1)?.[1] as Record<string, unknown>;
const adapterConfig = patch.adapterConfig as Record<string, unknown>;
const env = adapterConfig.env as Record<string, unknown>;
expect(env.OPENAI_API_KEY).toBe("");
expect(env.CODEX_HOME).toContain(
"/companies/company-1/agents/11111111-1111-4111-8111-111111111111/codex-home",
);
expect(String(env.CODEX_HOME)).not.toContain("/companies/company-1/codex-home");
const env = (adapterConfig.env as Record<string, unknown> | undefined) ?? {};
expect(env.OPENAI_API_KEY).toBeUndefined();
expect(env.CODEX_HOME).toBeUndefined();
});
it("rejects codex_local agents configured with the shared host Codex home", async () => {
it("isolates CODEX_HOME when updating a codex_local agent to set its own OPENAI_API_KEY", async () => {
const agentId = "11111111-1111-4111-8111-111111111111";
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.patch(`/api/agents/${agentId}`)
.send({
adapterConfig: {
env: {
OPENAI_API_KEY: "sk-test-key",
},
},
}),
);
expect(res.status, JSON.stringify(res.body)).toBe(200);
const patch = mockAgentService.update.mock.calls.at(-1)?.[1] as Record<string, unknown>;
const adapterConfig = patch.adapterConfig as Record<string, unknown>;
const env = adapterConfig.env as Record<string, unknown>;
expect(env.OPENAI_API_KEY).toBe("sk-test-key");
expect(String(env.CODEX_HOME)).toContain(`/companies/company-1/agents/${agentId}/codex-home`);
});
it("allows codex_local agents to share the host Codex home", async () => {
const app = await createApp();
const sharedHome = path.join(os.homedir(), ".codex");
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/agents")
@@ -351,36 +369,42 @@ describe("agent routes adapter validation", () => {
adapterType: "codex_local",
adapterConfig: {
env: {
CODEX_HOME: path.join(os.homedir(), ".codex"),
CODEX_HOME: sharedHome,
},
},
}),
);
expect(res.status, JSON.stringify(res.body)).toBe(422);
expect(String(res.body.error)).toContain("codex_local agents must use an isolated adapterConfig.env.CODEX_HOME");
expect(mockAgentService.create).not.toHaveBeenCalled();
expect(res.status, JSON.stringify(res.body)).toBe(201);
const createInput = mockAgentService.create.mock.calls.at(-1)?.[1] as Record<string, unknown>;
const adapterConfig = createInput.adapterConfig as Record<string, unknown>;
const env = adapterConfig.env as Record<string, unknown>;
expect(env.CODEX_HOME).toBe(sharedHome);
});
it("rejects codex_local agents configured with the ~/.codex alias", async () => {
it("isolates CODEX_HOME when a codex_local agent sets its own OPENAI_API_KEY", async () => {
const app = await createApp();
const res = await requestApp(app, (baseUrl) =>
request(baseUrl)
.post("/api/companies/company-1/agents")
.send({
name: "Shared Codex Alias",
name: "Keyed Codex",
adapterType: "codex_local",
adapterConfig: {
env: {
CODEX_HOME: "~/.codex",
OPENAI_API_KEY: "sk-test-key",
},
},
}),
);
expect(res.status, JSON.stringify(res.body)).toBe(422);
expect(String(res.body.error)).toContain("codex_local agents must use an isolated adapterConfig.env.CODEX_HOME");
expect(mockAgentService.create).not.toHaveBeenCalled();
expect(res.status, JSON.stringify(res.body)).toBe(201);
const createInput = mockAgentService.create.mock.calls.at(-1)?.[1] as Record<string, unknown>;
const agentId = String(createInput.id);
const adapterConfig = createInput.adapterConfig as Record<string, unknown>;
const env = adapterConfig.env as Record<string, unknown>;
expect(env.OPENAI_API_KEY).toBe("sk-test-key");
expect(String(env.CODEX_HOME)).toContain(`/companies/company-1/agents/${agentId}/codex-home`);
});
it("rejects unknown adapter types even when schema accepts arbitrary strings", async () => {
@@ -264,8 +264,6 @@ describe.sequential("agent skill routes", () => {
key: "paperclipai/paperclip/paperclip",
runtimeName: "paperclip",
source: "/tmp/paperclip",
required: true,
requiredReason: "required",
},
]);
mockCompanySkillService.resolveRequestedSkillKeys.mockImplementation(
@@ -1046,6 +1046,9 @@ describe("claude execute", () => {
PAPERCLIP_TEST_CAPTURE_PATH: capturePath1,
},
promptTemplate: "Follow the paperclip heartbeat.",
paperclipSkillSync: {
desiredSkills: ["paperclip"],
},
},
context: {},
authToken: "run-jwt-token",
@@ -1083,6 +1086,9 @@ describe("claude execute", () => {
PAPERCLIP_TEST_CAPTURE_PATH: capturePath2,
},
promptTemplate: "Follow the paperclip heartbeat.",
paperclipSkillSync: {
desiredSkills: ["paperclip"],
},
},
context: {
issueId: "issue-1",
@@ -28,7 +28,7 @@ describe("claude local skill sync", () => {
cleanupDirs.clear();
});
it("defaults to mounting all built-in Paperclip skills when no explicit selection exists", async () => {
it("reports built-in Paperclip skills as available when no explicit selection exists", async () => {
const snapshot = await listClaudeSkills({
agentId: "agent-1",
companyId: "company-1",
@@ -38,9 +38,8 @@ describe("claude local skill sync", () => {
expect(snapshot.mode).toBe("ephemeral");
expect(snapshot.supported).toBe(true);
expect(snapshot.desiredSkills).toContain(paperclipKey);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("configured");
expect(snapshot.desiredSkills).toEqual([]);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("available");
});
it("respects an explicit desired skill list without mutating a persistent home", async () => {
@@ -57,7 +56,7 @@ describe("claude local skill sync", () => {
expect(snapshot.desiredSkills).toContain(paperclipKey);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("configured");
expect(snapshot.entries.find((entry) => entry.key === createAgentKey)?.state).toBe("configured");
expect(snapshot.entries.find((entry) => entry.key === createAgentKey)?.state).toBe("available");
});
it("normalizes legacy flat Paperclip skill refs to canonical keys", async () => {
@@ -0,0 +1,206 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { afterEach, beforeEach, describe, expect, it } from "vitest";
import type { Db } from "@paperclipai/db";
import { reconcileCodexLocalManagedHomesOnStartup } from "../services/codex-auth-reconciliation.js";
type AgentRow = {
id: string;
companyId: string;
adapterConfig: Record<string, unknown>;
};
function makeDb(rows: AgentRow[]): Db {
return {
select: () => ({
from: () => ({
where: async () => rows,
}),
}),
} as unknown as Db;
}
describe("reconcileCodexLocalManagedHomesOnStartup", () => {
let root: string;
let paperclipHome: string;
let sharedCodexHome: string;
const savedEnv: Record<string, string | undefined> = {};
function managedAgentHome(companyId: string, agentId: string): string {
return path.join(
paperclipHome,
"instances",
"default",
"companies",
companyId,
"agents",
agentId,
"codex-home",
);
}
beforeEach(async () => {
root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-startup-"));
paperclipHome = path.join(root, "paperclip-home");
sharedCodexHome = path.join(root, "shared-codex-home");
await fs.mkdir(sharedCodexHome, { recursive: true });
await fs.writeFile(path.join(sharedCodexHome, "auth.json"), '{"token":"shared"}', "utf8");
for (const key of ["PAPERCLIP_HOME", "PAPERCLIP_INSTANCE_ID", "CODEX_HOME"]) {
savedEnv[key] = process.env[key];
}
process.env.PAPERCLIP_HOME = paperclipHome;
process.env.PAPERCLIP_INSTANCE_ID = "default";
process.env.CODEX_HOME = sharedCodexHome;
});
afterEach(async () => {
for (const [key, value] of Object.entries(savedEnv)) {
if (value === undefined) delete process.env[key];
else process.env[key] = value;
}
await fs.rm(root, { recursive: true, force: true });
});
it("seeds a stranded managed home and is a no-op on the next boot", async () => {
const agentHome = managedAgentHome("company-1", "agent-7");
const rows: AgentRow[] = [
{ id: "agent-7", companyId: "company-1", adapterConfig: { env: { CODEX_HOME: agentHome, OPENAI_API_KEY: "" } } },
];
const first = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(first).toMatchObject({ scanned: 1, seeded: 1, alreadySeeded: 0, failed: 0 });
expect(first.seededAgentIds).toEqual(["agent-7"]);
const agentAuth = path.join(agentHome, "auth.json");
expect((await fs.lstat(agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(agentAuth)).toBe(
await fs.realpath(path.join(sharedCodexHome, "auth.json")),
);
const second = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(second).toMatchObject({ scanned: 1, seeded: 0, alreadySeeded: 1, failed: 0 });
});
it("does not report a backfill when shared Codex auth is missing", async () => {
await fs.rm(path.join(sharedCodexHome, "auth.json"), { force: true });
const agentHome = managedAgentHome("company-1", "agent-missing-source");
const rows: AgentRow[] = [
{ id: "agent-missing-source", companyId: "company-1", adapterConfig: { env: { CODEX_HOME: agentHome } } },
];
const summary = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(summary).toMatchObject({
scanned: 1,
seeded: 0,
sourceAuthMissing: 1,
failed: 0,
});
await expect(fs.lstat(path.join(agentHome, "auth.json"))).rejects.toThrow();
});
it("classifies external overrides and unconfigured homes without seeding", async () => {
const external = path.join(root, "user-codex");
await fs.mkdir(external, { recursive: true });
const rows: AgentRow[] = [
{ id: "ext", companyId: "company-1", adapterConfig: { env: { CODEX_HOME: external } } },
{ id: "none", companyId: "company-1", adapterConfig: { env: {} } },
];
const summary = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(summary).toMatchObject({
scanned: 2,
seeded: 0,
externalOverride: 1,
noManagedHome: 1,
failed: 0,
});
expect(await fs.access(path.join(external, "auth.json")).then(() => true).catch(() => false)).toBe(
false,
);
});
it("does not write a secret-bound OPENAI_API_KEY placeholder as the API key", async () => {
const agentHome = managedAgentHome("company-2", "agent-9");
// A secret binding must never be written verbatim into auth.json; the home
// should fall back to the seeded subscription symlink instead.
const rows: AgentRow[] = [
{
id: "agent-9",
companyId: "company-2",
adapterConfig: {
env: {
CODEX_HOME: agentHome,
OPENAI_API_KEY: { type: "secret", secretId: "sec-123" },
},
},
},
];
const summary = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(summary).toMatchObject({ scanned: 1, seeded: 1, failed: 0 });
const agentAuth = path.join(agentHome, "auth.json");
expect((await fs.lstat(agentAuth)).isSymbolicLink()).toBe(true);
expect(await fs.realpath(agentAuth)).toBe(
await fs.realpath(path.join(sharedCodexHome, "auth.json")),
);
});
it("preserves a preexisting API-key auth.json when the key is secret-bound", async () => {
const agentHome = managedAgentHome("company-4", "agent-secret");
// Simulate an agent that previously ran with a RESOLVED secret API key:
// execute-time seeding wrote a regular-file auth.json containing the key.
await fs.mkdir(agentHome, { recursive: true });
await fs.writeFile(
path.join(agentHome, "auth.json"),
JSON.stringify({ OPENAI_API_KEY: "sk-secret-resolved" }),
{ mode: 0o600 },
);
const rows: AgentRow[] = [
{
id: "agent-secret",
companyId: "company-4",
adapterConfig: {
env: {
CODEX_HOME: agentHome,
// Canonical secret binding shape; unresolvable at startup.
OPENAI_API_KEY: { type: "secret_ref", secretId: "11111111-1111-1111-1111-111111111111" },
},
},
},
];
const summary = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(summary).toMatchObject({ scanned: 1, seeded: 0, alreadySeeded: 1, failed: 0 });
const agentAuth = path.join(agentHome, "auth.json");
// The resolved-key file must remain a regular file, not be downgraded to the
// shared subscription symlink.
expect((await fs.lstat(agentAuth)).isSymbolicLink()).toBe(false);
expect(JSON.parse(await fs.readFile(agentAuth, "utf8"))).toEqual({
OPENAI_API_KEY: "sk-secret-resolved",
});
});
it("writes a plain OPENAI_API_KEY into the managed home", async () => {
const agentHome = managedAgentHome("company-3", "agent-5");
const rows: AgentRow[] = [
{
id: "agent-5",
companyId: "company-3",
adapterConfig: { env: { CODEX_HOME: agentHome, OPENAI_API_KEY: "sk-plain-1" } },
},
];
const summary = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(summary).toMatchObject({ scanned: 1, seeded: 1, failed: 0 });
const written = JSON.parse(await fs.readFile(path.join(agentHome, "auth.json"), "utf8"));
expect(written).toEqual({ OPENAI_API_KEY: "sk-plain-1" });
const second = await reconcileCodexLocalManagedHomesOnStartup(makeDb(rows));
expect(second).toMatchObject({ scanned: 1, seeded: 0, alreadySeeded: 1, failed: 0 });
expect(JSON.parse(await fs.readFile(path.join(agentHome, "auth.json"), "utf8"))).toEqual({
OPENAI_API_KEY: "sk-plain-1",
});
});
});
@@ -58,6 +58,12 @@ type LogEntry = {
chunk: string;
};
async function seedSharedCodexAuth(homeRoot: string): Promise<void> {
const sharedCodexHome = path.join(homeRoot, ".codex");
await fs.mkdir(sharedCodexHome, { recursive: true });
await fs.writeFile(path.join(sharedCodexHome, "auth.json"), '{"token":"shared"}\n', "utf8");
}
function createLocalSandboxRunner() {
let counter = 0;
return {
@@ -201,6 +207,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
let commandNotes: string[] = [];
try {
@@ -261,6 +268,7 @@ describe("codex execute", () => {
const previousPath = process.env.PATH;
process.env.HOME = root;
process.env.PATH = `${binDir}${path.delimiter}${process.env.PATH ?? ""}`;
await seedSharedCodexAuth(root);
let loggedCommand: string | null = null;
let loggedEnv: Record<string, string> = {};
@@ -328,6 +336,7 @@ describe("codex execute", () => {
process.env.HOME = root;
process.env.PATH = `${binDir}${path.delimiter}${process.env.PATH ?? ""}`;
await seedSharedCodexAuth(root);
try {
const result = await execute({
@@ -395,6 +404,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
try {
const result = await execute({
@@ -505,6 +515,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
try {
const result = await execute({
@@ -555,6 +566,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
vi.useFakeTimers();
vi.setSystemTime(new Date(2026, 3, 22, 22, 29, 0));
@@ -615,6 +627,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
let commandNotes: string[] = [];
try {
@@ -691,6 +704,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
try {
const result = await execute({
@@ -845,6 +859,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
try {
const result = await execute({
@@ -943,6 +958,7 @@ describe("codex execute", () => {
const previousHome = process.env.HOME;
process.env.HOME = root;
await seedSharedCodexAuth(root);
let invocationPrompt = "";
let invocationNotes: string[] = [];
@@ -1098,6 +1114,9 @@ describe("codex execute", () => {
PAPERCLIP_TEST_CAPTURE_PATH: capturePath,
},
promptTemplate: "Follow the paperclip heartbeat.",
paperclipSkillSync: {
desiredSkills: ["paperclip"],
},
},
context: {},
authToken: "run-jwt-token",
@@ -1206,6 +1225,9 @@ describe("codex execute", () => {
CODEX_HOME: explicitCodexHome,
},
promptTemplate: "Follow the paperclip heartbeat.",
paperclipSkillSync: {
desiredSkills: ["paperclip"],
},
},
context: {},
authToken: "run-jwt-token",
@@ -13,7 +13,6 @@ async function makeTempDir(prefix: string): Promise<string> {
describe("codex local skill sync", () => {
const paperclipKey = "paperclipai/paperclip/paperclip";
const createAgentKey = "paperclipai/paperclip/paperclip-create-agent";
const cleanupDirs = new Set<string>();
afterEach(async () => {
@@ -42,11 +41,7 @@ describe("codex local skill sync", () => {
const before = await listCodexSkills(ctx);
expect(before.mode).toBe("ephemeral");
expect(before.desiredSkills).toContain(paperclipKey);
expect(before.desiredSkills).toContain(createAgentKey);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("configured");
expect(before.entries.find((entry) => entry.key === createAgentKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === createAgentKey)?.state).toBe("configured");
expect(before.entries.find((entry) => entry.key === paperclipKey)?.detail).toContain("CODEX_HOME/skills/");
});
@@ -76,31 +71,6 @@ describe("codex local skill sync", () => {
});
});
it("keeps required bundled Paperclip skills configured even when the desired set is emptied", async () => {
const codexHome = await makeTempDir("paperclip-codex-skill-required-");
cleanupDirs.add(codexHome);
const configuredCtx = {
agentId: "agent-2",
companyId: "company-1",
adapterType: "codex_local",
config: {
env: {
CODEX_HOME: codexHome,
},
paperclipSkillSync: {
desiredSkills: [],
},
},
} as const;
const after = await syncCodexSkills(configuredCtx, []);
expect(after.desiredSkills).toContain(paperclipKey);
expect(after.desiredSkills).toContain(createAgentKey);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("configured");
expect(after.entries.find((entry) => entry.key === createAgentKey)?.state).toBe("configured");
});
it("normalizes legacy flat Paperclip skill refs before reporting configured state", async () => {
const codexHome = await makeTempDir("paperclip-codex-legacy-skill-sync-");
cleanupDirs.add(codexHome);
@@ -288,8 +288,6 @@ describe("cursor execute", () => {
{
name: "paperclip",
source: paperclipDir,
required: true,
requiredReason: "Bundled Paperclip skills are always available for local adapters.",
},
{
name: "ascii-heart",
@@ -48,7 +48,6 @@ describe("cursor local skill sync", () => {
const before = await listCursorSkills(ctx);
expect(before.mode).toBe("persistent");
expect(before.desiredSkills).toContain(paperclipKey);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("missing");
const after = await syncCursorSkills(ctx, [paperclipKey]);
@@ -78,8 +77,6 @@ describe("cursor local skill sync", () => {
key: "paperclip",
runtimeName: "paperclip",
source: paperclipDir,
required: true,
requiredReason: "Bundled Paperclip skills are always available for local adapters.",
},
{
key: "ascii-heart",
@@ -95,7 +92,7 @@ describe("cursor local skill sync", () => {
const before = await listCursorSkills(ctx);
expect(before.warnings).toEqual([]);
expect(before.desiredSkills).toEqual(["paperclip", "ascii-heart"]);
expect(before.desiredSkills).toEqual(["ascii-heart"]);
expect(before.entries.find((entry) => entry.key === "ascii-heart")?.state).toBe("missing");
const after = await syncCursorSkills(ctx, ["ascii-heart"]);
@@ -104,41 +101,4 @@ describe("cursor local skill sync", () => {
expect((await fs.lstat(path.join(home, ".cursor", "skills", "ascii-heart"))).isSymbolicLink()).toBe(true);
});
it("keeps required bundled Paperclip skills installed even when the desired set is emptied", async () => {
const home = await makeTempDir("paperclip-cursor-skill-prune-");
cleanupDirs.add(home);
const configuredCtx = {
agentId: "agent-2",
companyId: "company-1",
adapterType: "cursor",
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [paperclipKey],
},
},
} as const;
await syncCursorSkills(configuredCtx, [paperclipKey]);
const clearedCtx = {
...configuredCtx,
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [],
},
},
} as const;
const after = await syncCursorSkills(clearedCtx, []);
expect(after.desiredSkills).toContain(paperclipKey);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".cursor", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
});
@@ -41,49 +41,10 @@ describe("gemini local skill sync", () => {
const before = await listGeminiSkills(ctx);
expect(before.mode).toBe("persistent");
expect(before.desiredSkills).toContain(paperclipKey);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("missing");
const after = await syncGeminiSkills(ctx, [paperclipKey]);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".gemini", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
it("keeps required bundled Paperclip skills installed even when the desired set is emptied", async () => {
const home = await makeTempDir("paperclip-gemini-skill-prune-");
cleanupDirs.add(home);
const configuredCtx = {
agentId: "agent-2",
companyId: "company-1",
adapterType: "gemini_local",
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [paperclipKey],
},
},
} as const;
await syncGeminiSkills(configuredCtx, [paperclipKey]);
const clearedCtx = {
...configuredCtx,
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [],
},
},
} as const;
const after = await syncGeminiSkills(clearedCtx, []);
expect(after.desiredSkills).toContain(paperclipKey);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".gemini", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
});
@@ -6,7 +6,6 @@ import {
describe("grok local skill sync", () => {
const paperclipKey = "paperclipai/paperclip/paperclip";
const createAgentKey = "paperclipai/paperclip/paperclip-create-agent";
it("reports Grok skills as ephemeral workspace-mounted state", async () => {
const snapshot = await listGrokSkills({
@@ -24,9 +23,7 @@ describe("grok local skill sync", () => {
expect(snapshot.supported).toBe(true);
expect(snapshot.mode).toBe("ephemeral");
expect(snapshot.desiredSkills).toContain(paperclipKey);
expect(snapshot.desiredSkills).toContain(createAgentKey);
expect(snapshot.entries.find((entry) => entry.key === paperclipKey)).toMatchObject({
required: true,
state: "configured",
detail: "Will be copied into `.claude/skills` in the execution workspace on the next run.",
});
@@ -313,6 +313,82 @@ describe("resolveExecutionRunAdapterConfig", () => {
details: { code: "low_trust_inline_sensitive_env_denied" },
});
});
it("fails push-capability preflight when no GitHub write credential is bound at agent or project scope", async () => {
await expect(resolveExecutionRunAdapterConfig({
companyId: "company-1",
agentId: "agent-1",
issueId: "issue-1",
executionRunConfig: { env: { AGENT_ONLY: "agent-only" } },
projectEnv: { PROJECT_ONLY: "project-only" },
requiredScopedEnvBinding: {
keys: ["GH_TOKEN", "GITHUB_TOKEN"],
consumerScopes: ["agent", "project"],
reason: "push_write_credential_missing",
remediation: "GitHub PR workflow requires GH_TOKEN or GITHUB_TOKEN bound at project or agent scope.",
},
secretsSvc: {
resolveAdapterConfigForRuntime: vi.fn(),
resolveEnvBindings: vi.fn(),
} as any,
})).rejects.toMatchObject({
code: "configuration_incomplete",
message: expect.stringContaining("GitHub PR workflow requires GH_TOKEN or GITHUB_TOKEN"),
resultJson: {
configurationIncomplete: {
reason: "push_write_credential_missing",
requiredEnvKeys: ["GH_TOKEN", "GITHUB_TOKEN"],
requiredScopes: ["agent", "project"],
missingBindings: [],
},
},
});
});
it("passes push-capability preflight when a project-scoped GitHub credential is configured", async () => {
const resolveAdapterConfigForRuntime = vi.fn().mockResolvedValue({
config: { env: { AGENT_ONLY: "agent-only" } },
secretKeys: new Set<string>(),
manifest: [],
});
const resolveEnvBindings = vi.fn().mockResolvedValue({
env: { GH_TOKEN: "github-token" },
secretKeys: new Set(["GH_TOKEN"]),
manifest: [],
});
const collectMissingRuntimeBindings = vi.fn().mockResolvedValue([]);
const result = await resolveExecutionRunAdapterConfig({
companyId: "company-1",
agentId: "agent-1",
issueId: "issue-1",
projectId: "project-1",
executionRunConfig: { env: { AGENT_ONLY: "agent-only" } },
projectEnv: { GH_TOKEN: { type: "plain", value: "github-token" } },
requiredScopedEnvBinding: {
keys: ["GH_TOKEN", "GITHUB_TOKEN"],
consumerScopes: ["agent", "project"],
reason: "push_write_credential_missing",
remediation: "GitHub PR workflow requires GH_TOKEN or GITHUB_TOKEN bound at project or agent scope.",
},
secretsSvc: {
resolveAdapterConfigForRuntime,
resolveEnvBindings,
collectMissingRuntimeBindings,
} as any,
});
expect(result.resolvedConfig.env).toEqual({
AGENT_ONLY: "agent-only",
GH_TOKEN: "github-token",
});
expect(resolveEnvBindings).toHaveBeenCalledOnce();
expect(collectMissingRuntimeBindings).toHaveBeenCalledTimes(2);
expect(collectMissingRuntimeBindings.mock.calls[1]?.[2]).toMatchObject({
consumerType: "project",
consumerId: "project-1",
});
});
});
describe("extractMentionedSkillIdsFromSources", () => {
@@ -1,3 +1,8 @@
import { execFile as execFileCallback } from "node:child_process";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { promisify } from "node:util";
import { describe, expect, it } from "vitest";
import type { agents } from "@paperclipai/db";
import { sessionCodec as codexSessionCodec } from "@paperclipai/adapter-codex-local/server";
@@ -5,6 +10,7 @@ import { resolveDefaultAgentWorkspaceDir } from "../home-paths.js";
import {
applyPersistedExecutionWorkspaceConfig,
assertGitSensitiveAdapterWorkspaceValid,
assertPushCapabilityCheckoutValid,
buildRealizedExecutionWorkspaceFromPersisted,
buildExplicitResumeSessionOverride,
deriveTaskKeyWithHeartbeatFallback,
@@ -16,6 +22,7 @@ import {
prioritizeProjectWorkspaceCandidatesForRun,
parseSessionCompactionPolicy,
resolveNextSessionState,
requiresPushCapabilityPreflight,
resolveWorkspaceAfterLowTrustPreflight,
resolveRuntimeSessionParamsForWorkspace,
shouldDeferFollowupWakeForSameIssue,
@@ -29,6 +36,8 @@ import {
} from "../services/heartbeat.ts";
import type { TrustPresetResolution } from "../services/trust-preset-resolver.ts";
const execFile = promisify(execFileCallback);
function buildResolvedWorkspace(overrides: Partial<ResolvedWorkspaceForRun> = {}): ResolvedWorkspaceForRun {
return {
cwd: "/tmp/project",
@@ -105,6 +114,19 @@ function buildWorkspaceValidationInput(
};
}
async function runGit(cwd: string, args: string[]) {
await execFile("git", args, { cwd });
}
async function createGitCheckout(options: { withRemote: boolean }) {
const root = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-push-preflight-"));
await runGit(root, ["init"]);
if (options.withRemote) {
await runGit(root, ["remote", "add", "origin", "https://github.com/example/repo.git"]);
}
return root;
}
async function expectWorkspaceValidationFailure(
input: WorkspaceValidationInput,
reason: string,
@@ -377,6 +399,72 @@ describe("assertGitSensitiveAdapterWorkspaceValid", () => {
});
});
describe("assertPushCapabilityCheckoutValid", () => {
it("rejects a GitHub PR workflow checkout without a configured push remote", async () => {
const cwd = await createGitCheckout({ withRemote: false });
try {
await expect(assertPushCapabilityCheckoutValid({
enabled: true,
issue: {
id: "issue-1",
identifier: "PAP-1",
},
cwd,
})).rejects.toMatchObject({
code: "workspace_validation_failed",
message: expect.stringContaining("has no configured push remote"),
resultJson: {
workspaceValidation: expect.objectContaining({
reason: "missing_git_push_remote",
issueId: "issue-1",
executionWorkspaceCwd: cwd,
}),
},
});
} finally {
await fs.rm(cwd, { recursive: true, force: true });
}
});
it("allows a GitHub PR workflow checkout when a push remote is configured", async () => {
const cwd = await createGitCheckout({ withRemote: true });
try {
await expect(assertPushCapabilityCheckoutValid({
enabled: true,
issue: {
id: "issue-1",
identifier: "PAP-1",
},
cwd,
})).resolves.toBeUndefined();
} finally {
await fs.rm(cwd, { recursive: true, force: true });
}
});
});
describe("requiresPushCapabilityPreflight", () => {
it("only enables the guard when the issue explicitly mentions the GitHub PR workflow skill", () => {
expect(requiresPushCapabilityPreflight({
adapterType: "codex_local",
issueId: "issue-1",
explicitRunScopedSkillKeys: ["paperclipai/bundled/software-development/github-pr-workflow"],
})).toBe(true);
expect(requiresPushCapabilityPreflight({
adapterType: "codex_local",
issueId: "issue-1",
explicitRunScopedSkillKeys: [],
})).toBe(false);
expect(requiresPushCapabilityPreflight({
adapterType: "cursor-cloud",
issueId: "issue-1",
explicitRunScopedSkillKeys: ["paperclipai/bundled/software-development/github-pr-workflow"],
})).toBe(false);
});
});
describe("stripHostWorkspaceProvisionForLowTrustSandbox", () => {
it("removes only the host-side provision command for sandbox-backed low-trust runs", () => {
const config = {
@@ -255,6 +255,49 @@ describeEmbeddedPostgres("issue blocker attention", () => {
});
});
it("ignores cancelled direct children when counting unresolved blocker attention", async () => {
const { companyId, agentId } = await createCompany("PBD");
const parentId = await insertIssue({ companyId, identifier: "PBD-1", title: "Parent", status: "blocked" });
const activeBlockerOneId = await insertIssue({
companyId,
identifier: "PBD-2",
title: "Running dependency one",
status: "todo",
assigneeAgentId: agentId,
});
const activeBlockerTwoId = await insertIssue({
companyId,
identifier: "PBD-3",
title: "Running dependency two",
status: "todo",
assigneeAgentId: agentId,
});
await insertIssue({
companyId,
identifier: "PBD-4",
title: "Cancelled child",
status: "cancelled",
parentId,
assigneeAgentId: agentId,
});
await block({ companyId, blockerIssueId: activeBlockerOneId, blockedIssueId: parentId });
await block({ companyId, blockerIssueId: activeBlockerTwoId, blockedIssueId: parentId });
await activeRun({ companyId, agentId, issueId: activeBlockerOneId });
await activeRun({ companyId, agentId, issueId: activeBlockerTwoId });
const parent = (await svc.list(companyId, { status: "blocked" })).find((issue) => issue.id === parentId);
expect(parent?.blockerAttention).toMatchObject({
state: "covered",
reason: "active_dependency",
unresolvedBlockerCount: 2,
coveredBlockerCount: 2,
stalledBlockerCount: 0,
attentionBlockerCount: 0,
});
expect(parent?.blockerAttention?.sampleBlockerIdentifier).not.toBe("PBD-4");
});
it("covers recursive blocker chains when the downstream leaf has active work", async () => {
const { companyId, agentId } = await createCompany("PBR");
const parentId = await insertIssue({ companyId, identifier: "PBR-1", title: "Parent", status: "blocked" });
@@ -61,12 +61,13 @@ async function waitFor(condition: () => boolean | Promise<boolean>, timeoutMs =
throw new Error("Timed out waiting for condition");
}
async function deleteHeartbeatRunsAfterActivityLogDrains(db: Db) {
async function deleteHeartbeatRunsAndWakeupsAfterActivityLogDrains(db: Db) {
let lastError: unknown = null;
for (let attempt = 0; attempt < 10; attempt += 1) {
await db.delete(activityLog);
try {
await db.delete(heartbeatRuns);
await db.delete(agentWakeupRequests);
return;
} catch (error) {
lastError = error;
@@ -527,8 +528,7 @@ describeEmbeddedPostgres("low-trust red-team HTTP route regression suite", () =>
await db.delete(issueRelations);
await db.delete(activityLog);
await db.delete(heartbeatRunEvents);
await deleteHeartbeatRunsAfterActivityLogDrains(db);
await db.delete(agentWakeupRequests);
await deleteHeartbeatRunsAndWakeupsAfterActivityLogDrains(db);
await db.delete(issues);
await db.delete(agentRuntimeState);
await db.delete(agents);
@@ -42,49 +42,10 @@ describe("opencode local skill sync", () => {
expect(before.mode).toBe("persistent");
expect(before.warnings).toContain("OpenCode currently uses the shared Claude skills home (~/.claude/skills).");
expect(before.desiredSkills).toContain(paperclipKey);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("missing");
const after = await syncOpenCodeSkills(ctx, [paperclipKey]);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".claude", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
it("keeps required bundled Paperclip skills installed even when the desired set is emptied", async () => {
const home = await makeTempDir("paperclip-opencode-skill-prune-");
cleanupDirs.add(home);
const configuredCtx = {
agentId: "agent-2",
companyId: "company-1",
adapterType: "opencode_local",
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [paperclipKey],
},
},
} as const;
await syncOpenCodeSkills(configuredCtx, [paperclipKey]);
const clearedCtx = {
...configuredCtx,
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [],
},
},
} as const;
const after = await syncOpenCodeSkills(clearedCtx, []);
expect(after.desiredSkills).toContain(paperclipKey);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".claude", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
});
@@ -79,34 +79,6 @@ describe("paperclip skill utils", () => {
await expect(fs.access(path.resolve("skills/create-issue-interaction-ui/SKILL.md"))).rejects.toThrow();
});
it("marks skills with required: false in SKILL.md frontmatter as optional", async () => {
const root = await makeTempDir("paperclip-skill-optional-");
cleanupDirs.add(root);
const moduleDir = path.join(root, "a", "b", "c", "d", "e");
await fs.mkdir(moduleDir, { recursive: true });
// Required skill (no frontmatter flag)
const requiredDir = path.join(root, "skills", "paperclip");
await fs.mkdir(requiredDir, { recursive: true });
await fs.writeFile(path.join(requiredDir, "SKILL.md"), "---\nname: paperclip\n---\n\n# Paperclip\n");
// Optional skill (required: false)
const optionalDir = path.join(root, "skills", "paperclip-dev");
await fs.mkdir(optionalDir, { recursive: true });
await fs.writeFile(path.join(optionalDir, "SKILL.md"), "---\nname: paperclip-dev\nrequired: false\n---\n\n# Dev\n");
const entries = await listPaperclipSkillEntries(moduleDir);
entries.sort((a, b) => a.runtimeName.localeCompare(b.runtimeName));
expect(entries).toHaveLength(2);
expect(entries[0]?.runtimeName).toBe("paperclip");
expect(entries[0]?.required).toBe(true);
expect(entries[1]?.runtimeName).toBe("paperclip-dev");
expect(entries[1]?.required).toBe(false);
expect(entries[1]?.requiredReason).toBeNull();
});
it("removes stale maintainer-only symlinks from a shared skills home", async () => {
const root = await makeTempDir("paperclip-skill-cleanup-");
cleanupDirs.add(root);
@@ -130,8 +130,11 @@ describe("pi_local execute", () => {
model: "google/gemini-3-flash-preview",
promptTemplate: "Keep working.",
paperclipRuntimeSkills: [
{ key: "demo-skill", runtimeName: "demo-skill", source: skillDir, required: true },
{ key: "demo-skill", runtimeName: "demo-skill", source: skillDir },
],
paperclipSkillSync: {
desiredSkills: ["demo-skill"],
},
},
context: {},
authToken: "run-jwt-token",
@@ -185,10 +188,10 @@ describe("pi_local execute", () => {
cwd: workspace,
model: "google/gemini-3-flash-preview",
promptTemplate: "Keep working.",
// required:false with no explicit paperclipSkillSync preference →
// No explicit paperclipSkillSync preference →
// resolvePaperclipDesiredSkillNames returns [] → skill is not injected.
paperclipRuntimeSkills: [
{ key: "not-injected", runtimeName: "not-injected", source: nonInjectedSkillDir, required: false },
{ key: "not-injected", runtimeName: "not-injected", source: nonInjectedSkillDir },
],
},
context: {},
@@ -41,49 +41,10 @@ describe("pi local skill sync", () => {
const before = await listPiSkills(ctx);
expect(before.mode).toBe("persistent");
expect(before.desiredSkills).toContain(paperclipKey);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.required).toBe(true);
expect(before.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("missing");
const after = await syncPiSkills(ctx, [paperclipKey]);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".pi", "agent", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
it("keeps required bundled Paperclip skills installed even when the desired set is emptied", async () => {
const home = await makeTempDir("paperclip-pi-skill-prune-");
cleanupDirs.add(home);
const configuredCtx = {
agentId: "agent-2",
companyId: "company-1",
adapterType: "pi_local",
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [paperclipKey],
},
},
} as const;
await syncPiSkills(configuredCtx, [paperclipKey]);
const clearedCtx = {
...configuredCtx,
config: {
env: {
HOME: home,
},
paperclipSkillSync: {
desiredSkills: [],
},
},
} as const;
const after = await syncPiSkills(clearedCtx, []);
expect(after.desiredSkills).toContain(paperclipKey);
expect(after.entries.find((entry) => entry.key === paperclipKey)?.state).toBe("installed");
expect((await fs.lstat(path.join(home, ".pi", "agent", "skills", "paperclip"))).isSymbolicLink()).toBe(true);
});
});
@@ -168,6 +168,16 @@ vi.mock("../services/index.js", () => ({
})),
})),
reconcileCloudUpstreamRunsOnStartup: vi.fn(async () => ({ reconciled: 0 })),
reconcileCodexLocalManagedHomesOnStartup: vi.fn(async () => ({
scanned: 0,
seeded: 0,
alreadySeeded: 0,
externalOverride: 0,
noManagedHome: 0,
sourceAuthMissing: 0,
failed: 0,
seededAgentIds: [],
})),
reconcilePersistedRuntimeServicesOnStartup: vi.fn(async () => ({ reconciled: 0 })),
routineService: vi.fn(() => ({
tickScheduledTriggers: vi.fn(async () => ({ triggered: 0 })),
@@ -94,6 +94,57 @@ async function createTempRepo(defaultBranch = "main") {
return repoRoot;
}
async function createClonedRepoWithRemote() {
const sourceRepo = await createTempRepo("master");
const remoteDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-worktree-remote-"));
const remotePath = path.join(remoteDir, "paperclip.git");
await execFileAsync("git", ["clone", "--bare", sourceRepo, remotePath]);
const cloneRoot = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-worktree-clone-"));
const repoRoot = path.join(cloneRoot, "paperclip");
await execFileAsync("git", ["clone", remotePath, repoRoot]);
await runGit(repoRoot, ["config", "user.email", "paperclip@example.com"]);
await runGit(repoRoot, ["config", "user.name", "Paperclip Test"]);
return { sourceRepo, remotePath, repoRoot };
}
async function advanceRemoteMaster(sourceRepo: string, remotePath: string, fileName: string) {
await fs.writeFile(path.join(sourceRepo, fileName), `${fileName}\n`, "utf8");
await runGit(sourceRepo, ["add", fileName]);
await runGit(sourceRepo, ["commit", "-m", `Add ${fileName}`]);
await runGit(sourceRepo, ["push", remotePath, "master"]);
return readGit(sourceRepo, ["rev-parse", "master"]);
}
function realizeWorktreeForTest(repoRoot: string, repoRef: string | null) {
return realizeExecutionWorkspace({
base: {
baseCwd: repoRoot,
source: "project_primary",
projectId: "project-1",
workspaceId: "workspace-1",
repoUrl: null,
repoRef,
},
config: {
workspaceStrategy: {
type: "git_worktree",
branchTemplate: "{{issue.identifier}}-{{slug}}",
},
},
issue: {
id: "issue-1",
identifier: "PAP-447",
title: "Add Worktree Support",
},
agent: {
id: "agent-1",
name: "Codex Coder",
companyId: "company-1",
},
});
}
function buildWorkspace(cwd: string): RealizedExecutionWorkspace {
return {
baseCwd: cwd,
@@ -508,6 +559,123 @@ describe("realizeExecutionWorkspace", () => {
]);
});
it("bases a fresh worktree on origin/master even when local master has unpushed commits", async () => {
const { repoRoot } = await createClonedRepoWithRemote();
const originHead = await readGit(repoRoot, ["rev-parse", "origin/master"]);
await fs.writeFile(path.join(repoRoot, "unpushed.txt"), "local only\n", "utf8");
await runGit(repoRoot, ["add", "unpushed.txt"]);
await runGit(repoRoot, ["commit", "-m", "Unpushed local work"]);
const localHead = await readGit(repoRoot, ["rev-parse", "master"]);
expect(localHead).not.toBe(originHead);
const workspace = await realizeWorktreeForTest(repoRoot, null);
expect(workspace.baseRefSha).toBe(originHead);
expect(await readGit(workspace.cwd, ["rev-parse", "HEAD"])).toBe(originHead);
});
it("maps a configured local branch base ref to origin/<branch> for fresh worktrees", async () => {
const { repoRoot } = await createClonedRepoWithRemote();
const originHead = await readGit(repoRoot, ["rev-parse", "origin/master"]);
await fs.writeFile(path.join(repoRoot, "unpushed.txt"), "local only\n", "utf8");
await runGit(repoRoot, ["add", "unpushed.txt"]);
await runGit(repoRoot, ["commit", "-m", "Unpushed local work"]);
const localHead = await readGit(repoRoot, ["rev-parse", "master"]);
expect(localHead).not.toBe(originHead);
const workspace = await realizeWorktreeForTest(repoRoot, "master");
expect(workspace.repoRef).toBe("origin/master");
expect(workspace.baseRefSha).toBe(originHead);
expect(await readGit(workspace.cwd, ["rev-parse", "HEAD"])).toBe(originHead);
});
it("fast-forwards an unstarted reused worktree to the advanced origin/master", async () => {
const { sourceRepo, remotePath, repoRoot } = await createClonedRepoWithRemote();
const initial = await realizeWorktreeForTest(repoRoot, null);
const initialHead = await readGit(initial.cwd, ["rev-parse", "HEAD"]);
const advancedHead = await advanceRemoteMaster(sourceRepo, remotePath, "auth-fix.txt");
expect(advancedHead).not.toBe(initialHead);
const reused = await realizeWorktreeForTest(repoRoot, null);
expect(reused.created).toBe(false);
expect(reused.cwd).toBe(initial.cwd);
expect(await readGit(reused.cwd, ["rev-parse", "HEAD"])).toBe(advancedHead);
expect(reused.baseRefSha).toBe(advancedHead);
expect(reused.warnings).toEqual([]);
});
it("does not reset a reused worktree that already has task commits", async () => {
const { sourceRepo, remotePath, repoRoot } = await createClonedRepoWithRemote();
const initial = await realizeWorktreeForTest(repoRoot, null);
await fs.writeFile(path.join(initial.cwd, "task-work.txt"), "in progress\n", "utf8");
await runGit(initial.cwd, ["add", "task-work.txt"]);
await runGit(initial.cwd, ["commit", "-m", "Task work in progress"]);
const taskHead = await readGit(initial.cwd, ["rev-parse", "HEAD"]);
await advanceRemoteMaster(sourceRepo, remotePath, "auth-fix.txt");
const reused = await realizeWorktreeForTest(repoRoot, null);
expect(reused.created).toBe(false);
expect(await readGit(reused.cwd, ["rev-parse", "HEAD"])).toBe(taskHead);
expect(reused.warnings).toEqual([
expect.stringContaining("is behind origin/master by 1 commit"),
]);
});
it("does not reset a reused worktree with untracked changes", async () => {
const { sourceRepo, remotePath, repoRoot } = await createClonedRepoWithRemote();
const initial = await realizeWorktreeForTest(repoRoot, null);
const initialHead = await readGit(initial.cwd, ["rev-parse", "HEAD"]);
await fs.writeFile(path.join(initial.cwd, "scratch.txt"), "uncommitted scratch\n", "utf8");
await advanceRemoteMaster(sourceRepo, remotePath, "auth-fix.txt");
const reused = await realizeWorktreeForTest(repoRoot, null);
expect(reused.created).toBe(false);
expect(await readGit(reused.cwd, ["rev-parse", "HEAD"])).toBe(initialHead);
await expect(fs.readFile(path.join(reused.cwd, "scratch.txt"), "utf8")).resolves.toBe(
"uncommitted scratch\n",
);
expect(reused.warnings).toEqual([
expect.stringContaining("is behind origin/master by 1 commit"),
]);
});
it("does not reset a reused worktree with untracked changes when status.showUntrackedFiles=no", async () => {
const { sourceRepo, remotePath, repoRoot } = await createClonedRepoWithRemote();
const initial = await realizeWorktreeForTest(repoRoot, null);
const initialHead = await readGit(initial.cwd, ["rev-parse", "HEAD"]);
// Without `--untracked-files=all`, this config hides untracked files from
// `git status --porcelain`, which would let the clean-tree guard pass and a
// `reset --hard` destroy the scratch file below.
await readGit(initial.cwd, ["config", "status.showUntrackedFiles", "no"]);
await fs.writeFile(path.join(initial.cwd, "scratch.txt"), "uncommitted scratch\n", "utf8");
await advanceRemoteMaster(sourceRepo, remotePath, "auth-fix.txt");
const reused = await realizeWorktreeForTest(repoRoot, null);
expect(reused.created).toBe(false);
expect(await readGit(reused.cwd, ["rev-parse", "HEAD"])).toBe(initialHead);
await expect(fs.readFile(path.join(reused.cwd, "scratch.txt"), "utf8")).resolves.toBe(
"uncommitted scratch\n",
);
expect(reused.warnings).toEqual([
expect.stringContaining("is behind origin/master by 1 commit"),
]);
});
it("rejects reusing an empty directory that only looks like a worktree because it sits inside the repo", async () => {
const repoRoot = await createTempRepo();
const branchName = "PAP-447-add-worktree-support";
+6 -2
View File
@@ -456,6 +456,10 @@ const piLocalAdapter: ServerAdapterModule = {
// hermes-paperclip-adapter v0.2.0 predates the authToken field; cast is
// intentional until hermes ships a matching AdapterExecutionContext type.
const executeHermesLocal = hermesExecute as unknown as ServerAdapterModule["execute"];
// hermes-paperclip-adapter v0.2.0 still depends on the published @paperclipai/adapter-utils
// that ships the "paperclip_required" origin; casts bridge until hermes upgrades.
const listHermesSkills = hermesListSkills as unknown as ServerAdapterModule["listSkills"];
const syncHermesSkills = hermesSyncSkills as unknown as ServerAdapterModule["syncSkills"];
const hermesLocalAdapter: ServerAdapterModule = {
type: "hermes_local",
@@ -510,8 +514,8 @@ const hermesLocalAdapter: ServerAdapterModule = {
},
testEnvironment: (ctx) => hermesTestEnvironment(normalizeHermesConfig(ctx) as never),
sessionCodec: hermesSessionCodec,
listSkills: hermesListSkills,
syncSkills: hermesSyncSkills,
listSkills: listHermesSkills,
syncSkills: syncHermesSkills,
models: hermesModels,
supportsLocalAgentJwt: true,
supportsInstructionsBundle: false,
+24 -1
View File
@@ -41,6 +41,7 @@ import {
heartbeatService,
instanceSettingsService,
reconcileCloudUpstreamRunsOnStartup,
reconcileCodexLocalManagedHomesOnStartup,
reconcilePersistedRuntimeServicesOnStartup,
routineService,
} from "./services/index.js";
@@ -728,7 +729,29 @@ export async function startServer(): Promise<StartedServer> {
.catch((err) => {
logger.error({ err }, "startup reconciliation of cloud upstream runs failed");
});
// Backfill auth.json into any already-isolated codex_local managed home that
// was created by the #8272 isolation guard before the Phase 1 seeding fix.
// Idempotent; the Phase 1 execute-time seeding covers new strandings.
void reconcileCodexLocalManagedHomesOnStartup(db)
.then((result) => {
if (result.seeded > 0 || result.failed > 0) {
logger.warn(
{ seeded: result.seeded, failed: result.failed, scanned: result.scanned },
"reconciled codex_local managed homes (backfilled missing auth)",
);
}
if (result.sourceAuthMissing > 0) {
logger.warn(
{ sourceAuthMissing: result.sourceAuthMissing, scanned: result.scanned },
"could not backfill codex_local managed homes because shared Codex auth is missing",
);
}
})
.catch((err) => {
logger.error({ err }, "startup reconciliation of codex_local managed homes failed");
});
// Force the instance onto the Kubernetes sandbox provider when configured via
// env (PAPERCLIP_EXECUTION_MODE=kubernetes). Runs BEFORE the heartbeat resumes
// queued runs so the policy + managed k8s environments are in place. A bad
+28 -60
View File
@@ -1,6 +1,5 @@
import { Router, type Request, type Response } from "express";
import { generateKeyPairSync, randomUUID } from "node:crypto";
import os from "node:os";
import path from "node:path";
import type { Db } from "@paperclipai/db";
import { agents as agentsTable, companies, heartbeatRuns, issues as issuesTable, projects as projectsTable } from "@paperclipai/db";
@@ -1152,53 +1151,33 @@ export function agentRoutes(
return path.resolve(instanceRoot, "companies", companyId, "agents", agentId, "codex-home");
}
function normalizeCodexLocalHomePath(rawHome: string): string {
if (rawHome === "~") return path.resolve(os.homedir());
if (rawHome.startsWith("~/") || rawHome.startsWith("~\\")) {
return path.resolve(path.join(os.homedir(), rawHome.slice(2)));
}
return path.resolve(rawHome);
function codexLocalEnvKeyConfigured(value: unknown): boolean {
if (asEnvBindingString(value)) return true;
const record = asRecord(value);
return record?.type === "secret_ref" && typeof record.secretId === "string";
}
function assertCodexLocalHomeIsNotShared(companyId: string, configuredHome: string) {
const instanceRoot = resolvePaperclipInstanceRootForAdapter({
homeDir: asNonEmptyString(process.env.PAPERCLIP_HOME) ?? undefined,
instanceId: asNonEmptyString(process.env.PAPERCLIP_INSTANCE_ID) ?? undefined,
env: process.env,
});
const normalizedHome = normalizeCodexLocalHomePath(configuredHome);
const sharedHomes = [
path.resolve(instanceRoot, "companies", companyId, "codex-home"),
path.resolve(path.join(os.homedir(), ".codex")),
];
const hostCodexHome = asNonEmptyString(process.env.CODEX_HOME);
if (hostCodexHome) {
sharedHomes.push(normalizeCodexLocalHomePath(hostCodexHome));
}
if (!sharedHomes.some((sharedHome) => sharedHome === normalizedHome)) return;
throw unprocessable(
"codex_local agents must use an isolated adapterConfig.env.CODEX_HOME; shared company codex-home or host Codex auth home is not allowed",
);
}
function applyCodexLocalIsolationGuard(
// codex_local agents inherit whatever Codex login is already on the device
// (the host's ~/.codex or $CODEX_HOME) by default, so a fresh agent needs no
// env overrides at all. We only carve out an isolated per-agent CODEX_HOME
// when the agent sets its own OPENAI_API_KEY, so that key's api-key auth.json
// does not collide with the shared company home other agents use for the host
// login. Agents without a key share the host credentials.
function applyCodexLocalKeyIsolation(
companyId: string,
agentId: string,
adapterType: string | null | undefined,
adapterConfig: Record<string, unknown>,
): Record<string, unknown> {
if (adapterType !== "codex_local") return adapterConfig;
const env = asRecord(adapterConfig.env) ? { ...(adapterConfig.env as Record<string, unknown>) } : {};
const configuredHome = asEnvBindingString(env.CODEX_HOME);
if (configuredHome) {
assertCodexLocalHomeIsNotShared(companyId, configuredHome);
} else {
env.CODEX_HOME = codexLocalAgentHome(companyId, agentId);
}
if (!Object.prototype.hasOwnProperty.call(env, "OPENAI_API_KEY")) {
env.OPENAI_API_KEY = "";
}
return { ...adapterConfig, env };
const existingEnv = asRecord(adapterConfig.env);
if (!existingEnv) return adapterConfig;
if (!codexLocalEnvKeyConfigured(existingEnv.OPENAI_API_KEY)) return adapterConfig;
if (codexLocalEnvKeyConfigured(existingEnv.CODEX_HOME)) return adapterConfig;
return {
...adapterConfig,
env: { ...existingEnv, CODEX_HOME: codexLocalAgentHome(companyId, agentId) },
};
}
function applyCreateDefaultsByAdapterType(
@@ -1480,18 +1459,13 @@ export function agentRoutes(
companyId,
requestedDesiredSkills,
);
const resolvedRequestedSkills = resolvedRequestedSkillEntries.map((entry) => entry.key);
const runtimeSkillEntries = await companySkills.listRuntimeSkillEntries(companyId, {
materializeMissing: shouldMaterializeRuntimeSkillsForAdapter(adapterType),
versionSelections: skillVersionSelectionMap(resolvedRequestedSkillEntries),
});
const requiredSkills = runtimeSkillEntries
.filter((entry) => entry.required)
.map((entry) => entry.key);
const desiredSkillEntries = [
...requiredSkills.map((key) => ({ key, versionId: null })),
...resolvedRequestedSkillEntries,
].filter((entry, index, entries) => entries.findIndex((candidate) => candidate.key === entry.key) === index);
const desiredSkillEntries = resolvedRequestedSkillEntries.filter(
(entry, index, entries) => entries.findIndex((candidate) => candidate.key === entry.key) === index,
);
const desiredSkills = desiredSkillEntries.map((entry) => entry.key);
return {
@@ -1712,15 +1686,9 @@ export function agentRoutes(
const preference = readPaperclipSkillSyncPreference(
agent.adapterConfig as Record<string, unknown>,
);
const runtimeSkillEntries = await companySkills.listRuntimeSkillEntries(agent.companyId, {
materializeMissing: false,
versionSelections: skillVersionSelectionMap(preference.desiredSkillEntries),
});
const requiredSkills = runtimeSkillEntries.filter((entry) => entry.required).map((entry) => entry.key);
const desiredSkillEntries = [
...requiredSkills.map((key) => ({ key, versionId: null })),
...preference.desiredSkillEntries,
].filter((entry, index, entries) => entries.findIndex((candidate) => candidate.key === entry.key) === index);
const desiredSkillEntries = preference.desiredSkillEntries.filter(
(entry, index, entries) => entries.findIndex((candidate) => candidate.key === entry.key) === index,
);
res.json(buildUnsupportedSkillSnapshot(agent.adapterType, desiredSkillEntries));
return;
}
@@ -2227,7 +2195,7 @@ export function agentRoutes(
assertNoAgentAdapterConfigMutation(req, rawHireAdapterConfig);
assertNoAgentRuntimeConfigAdapterConfigMutation(req, hireInput.runtimeConfig);
const hiredAgentId = randomUUID();
const requestedAdapterConfig = applyCodexLocalIsolationGuard(
const requestedAdapterConfig = applyCodexLocalKeyIsolation(
companyId,
hiredAgentId,
hireInput.adapterType,
@@ -2420,7 +2388,7 @@ export function agentRoutes(
assertNoAgentAdapterConfigMutation(req, rawCreateAdapterConfig);
assertNoAgentRuntimeConfigAdapterConfigMutation(req, createInput.runtimeConfig);
const agentId = randomUUID();
const requestedAdapterConfig = applyCodexLocalIsolationGuard(
const requestedAdapterConfig = applyCodexLocalKeyIsolation(
companyId,
agentId,
createInput.adapterType,
@@ -2894,7 +2862,7 @@ export function agentRoutes(
rawEffectiveAdapterConfig,
);
}
const effectiveAdapterConfig = applyCodexLocalIsolationGuard(
const effectiveAdapterConfig = applyCodexLocalKeyIsolation(
existing.companyId,
existing.id,
requestedAdapterType,
@@ -0,0 +1,147 @@
import type { Db } from "@paperclipai/db";
import { agents } from "@paperclipai/db";
import { reconcileManagedCodexHome } from "@paperclipai/adapter-codex-local/server";
import { eq } from "drizzle-orm";
import { logger } from "../middleware/logger.js";
export interface CodexAuthReconciliationSummary {
scanned: number;
seeded: number;
alreadySeeded: number;
externalOverride: number;
noManagedHome: number;
sourceAuthMissing: number;
failed: number;
seededAgentIds: string[];
}
function asRecord(value: unknown): Record<string, unknown> | null {
return typeof value === "object" && value !== null && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
/**
* Extracts a literal (non-secret) env value. Adapter env bindings persist either
* as a bare string or as a `{ type: "plain", value }` object; secret bindings
* are intentionally NOT resolved here (we never write an unresolved secret
* placeholder into auth.json). Mirrors the server-side env-binding extraction in
* routes/agents.ts.
*/
function readPlainEnvValue(value: unknown): string | null {
if (typeof value === "string") {
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : null;
}
const record = asRecord(value);
if (record?.type !== "plain") return null;
return readPlainEnvValue(record.value);
}
type ApiKeyBinding =
| { kind: "plain"; value: string }
| { kind: "secret" }
| { kind: "none" };
/**
* Classifies an `OPENAI_API_KEY` env binding so reconciliation can tell the
* difference between three states it must treat differently:
* - `plain`: a literal value we can write into auth.json.
* - `secret`: a secret binding (e.g. `{ type: "secret_ref", ... }`) we cannot
* resolve at startup; the resolved value may already exist on disk from a
* prior execute-time run, so reconciliation must not clobber it.
* - `none`: no key configured (chatgpt-subscription mode); seed the shared
* auth symlink.
*/
function classifyApiKeyBinding(value: unknown): ApiKeyBinding {
const plain = readPlainEnvValue(value);
if (plain) return { kind: "plain", value: plain };
const record = asRecord(value);
if (record && typeof record.type === "string" && record.type !== "plain") {
return { kind: "secret" };
}
return { kind: "none" };
}
/**
* Startup backfill: seed `auth.json` into any already-
* isolated `codex_local` managed home that was created (by the #8272 isolation
* guard) before the Phase 1 seeding fix landed. Phase 1 seeds at execute time;
* this repairs persisted homes proactively so a stranded agent recovers without
* waiting to run and without a manual symlink. Idempotent and safe to re-run on
* every boot: a home that already has valid auth is a no-op.
*/
export async function reconcileCodexLocalManagedHomesOnStartup(
db: Db,
): Promise<CodexAuthReconciliationSummary> {
const summary: CodexAuthReconciliationSummary = {
scanned: 0,
seeded: 0,
alreadySeeded: 0,
externalOverride: 0,
noManagedHome: 0,
sourceAuthMissing: 0,
failed: 0,
seededAgentIds: [],
};
const rows = await db
.select({
id: agents.id,
companyId: agents.companyId,
adapterConfig: agents.adapterConfig,
})
.from(agents)
.where(eq(agents.adapterType, "codex_local"));
for (const row of rows) {
summary.scanned += 1;
const env = asRecord(asRecord(row.adapterConfig)?.env);
const configuredCodexHome = env ? readPlainEnvValue(env.CODEX_HOME) : null;
const apiKeyBinding = classifyApiKeyBinding(env?.OPENAI_API_KEY);
try {
const result = await reconcileManagedCodexHome({
companyId: row.companyId,
configuredCodexHome,
apiKey: apiKeyBinding.kind === "plain" ? apiKeyBinding.value : null,
apiKeySecretBound: apiKeyBinding.kind === "secret",
});
switch (result.status) {
case "seeded":
summary.seeded += 1;
summary.seededAgentIds.push(row.id);
logger.info(
{ agentId: row.id, companyId: row.companyId, home: result.home },
"seeded auth into already-isolated codex_local managed home",
);
break;
case "already_seeded":
summary.alreadySeeded += 1;
break;
case "external_override":
summary.externalOverride += 1;
break;
case "no_managed_home":
summary.noManagedHome += 1;
break;
case "source_auth_missing":
summary.sourceAuthMissing += 1;
break;
}
} catch (err) {
summary.failed += 1;
logger.warn(
{
agentId: row.id,
companyId: row.companyId,
home: configuredCodexHome,
err: err instanceof Error ? err.message : String(err),
},
"failed to reconcile codex_local managed home on startup",
);
}
}
return summary;
}
-6
View File
@@ -4117,11 +4117,9 @@ export function companySkillService(db: Db) {
const out: PaperclipSkillEntry[] = [];
for (const skill of skills) {
const sourceKind = asString(getSkillMeta(skill).sourceKind);
const sourceResolution = await resolveRuntimeSkillSource(companyId, skill, options);
if (!sourceResolution) continue;
const required = sourceKind === "paperclip_bundled";
out.push({
key: skill.key,
runtimeName: buildSkillRuntimeName(skill.key, skill.slug),
@@ -4130,10 +4128,6 @@ export function companySkillService(db: Db) {
currentVersionId: skill.currentVersionId,
sourceStatus: sourceResolution.status,
missingDetail: sourceResolution.status === "missing" ? sourceResolution.detail : null,
required,
requiredReason: required
? "Bundled Paperclip skills are always available for local adapters."
: null,
});
}
+166 -5
View File
@@ -260,6 +260,9 @@ const WORKSPACE_VALIDATION_FAILURE_CODE = "workspace_validation_failed";
const WORKSPACE_VALIDATION_RECOVERY_CAUSE = "workspace_validation_failed";
const CONFIGURATION_INCOMPLETE_FAILURE_CODE = "configuration_incomplete";
const CONFIGURATION_INCOMPLETE_RECOVERY_CAUSE = "configuration_incomplete";
const GITHUB_PR_WORKFLOW_SKILL_KEY = "paperclipai/bundled/software-development/github-pr-workflow";
const GITHUB_PR_WORKFLOW_SKILL_SLUG = "github-pr-workflow";
const PUSH_CAPABILITY_ENV_KEYS = ["GH_TOKEN", "GITHUB_TOKEN"] as const;
// Keep this in sync with local adapters that require a git workspace before launch.
const GIT_SENSITIVE_LOCAL_ADAPTER_TYPES = new Set([
"acpx_local",
@@ -411,6 +414,34 @@ function formatMissingBindingForOperator(missing: MissingRuntimeBinding): string
return `secret ${secretLabel} not bound at ${missing.consumerType} ${missing.configPath}`;
}
function isConfiguredEnvBindingValue(binding: unknown) {
const parsed = envBindingSchema.safeParse(binding);
if (!parsed.success) return false;
const value = parsed.data;
if (typeof value === "string") return value.trim().length > 0;
if (value.type === "plain") return value.value.trim().length > 0;
return true;
}
function hasGithubPrWorkflowSkill(desiredSkills: string[]) {
return desiredSkills.some((skill) => {
const normalized = skill.trim();
return normalized === GITHUB_PR_WORKFLOW_SKILL_KEY
|| normalized === GITHUB_PR_WORKFLOW_SKILL_SLUG
|| normalized.endsWith(`/${GITHUB_PR_WORKFLOW_SKILL_SLUG}`);
});
}
export function requiresPushCapabilityPreflight(input: {
adapterType: string;
issueId: string | null | undefined;
explicitRunScopedSkillKeys: string[];
}) {
return Boolean(input.issueId)
&& GIT_SENSITIVE_LOCAL_ADAPTER_TYPES.has(input.adapterType)
&& hasGithubPrWorkflowSkill(input.explicitRunScopedSkillKeys);
}
const LOW_TRUST_SENSITIVE_ENV_KEY_RE =
/(api[-_]?key|access[-_]?token|auth(?:_?token)?|authorization|bearer|secret|passwd|password|credential|jwt|private[-_]?key|cookie|connectionstring)/i;
@@ -466,11 +497,18 @@ export async function resolveExecutionRunAdapterConfig(input: {
routineEnv?: unknown;
secretsSvc: RuntimeConfigSecretResolver;
trustPreset?: TrustPresetResolution;
requiredScopedEnvBinding?: {
keys: string[];
consumerScopes: Array<"agent" | "project">;
reason: string;
remediation: string;
};
}) {
const executionRunConfig = stripPaperclipRuntimeEnvFromAdapterConfig(input.executionRunConfig);
const environmentEnv = stripPaperclipRuntimeEnvBindings(input.environmentEnv);
const projectEnv = stripPaperclipRuntimeEnvBindings(input.projectEnv);
const routineEnv = stripPaperclipRuntimeEnvBindings(input.routineEnv);
const agentEnv = parseObject(executionRunConfig.env);
const lowTrustAllowedBindingIds = input.trustPreset?.kind === "low_trust_review"
? input.trustPreset.boundary.allowedSecretBindingIds ?? []
: undefined;
@@ -480,6 +518,31 @@ export async function resolveExecutionRunAdapterConfig(input: {
assertLowTrustEnvConfigAllowed(projectEnv, "project.env");
assertLowTrustEnvConfigAllowed(routineEnv, "routine.env");
}
const requiredScopedEnvBinding = input.requiredScopedEnvBinding ?? null;
const requiredScopedBindingsConfigured = requiredScopedEnvBinding
? requiredScopedEnvBinding.keys.some((key) => (
requiredScopedEnvBinding.consumerScopes.includes("agent")
&& isConfiguredEnvBindingValue(agentEnv[key])
) || (
requiredScopedEnvBinding.consumerScopes.includes("project")
&& isConfiguredEnvBindingValue(projectEnv?.[key])
))
: false;
if (requiredScopedEnvBinding && !requiredScopedBindingsConfigured) {
throw new ConfigurationIncompleteFailure(`configuration incomplete: ${requiredScopedEnvBinding.remediation}`, {
configurationIncomplete: {
reason: requiredScopedEnvBinding.reason,
companyId: input.companyId,
agentId: input.agentId ?? null,
issueId: input.issueId ?? null,
projectId: input.projectId ?? null,
routineId: input.routineId ?? null,
requiredEnvKeys: requiredScopedEnvBinding.keys,
requiredScopes: requiredScopedEnvBinding.consumerScopes,
missingBindings: [],
},
});
}
// Pre-dispatch binding-validation gate: detect declared secret refs that have
// no binding before resolving any secret value. Missing bindings short-circuit
// to a configuration-incomplete blocker routed to a human owner instead of a
@@ -522,6 +585,33 @@ export async function resolveExecutionRunAdapterConfig(input: {
)),
);
}
if (requiredScopedEnvBinding) {
const requiredEnvKeys = new Set(requiredScopedEnvBinding.keys);
const requiredScopes = new Set(requiredScopedEnvBinding.consumerScopes);
const requiredMissingBindings = missingBindings.filter((binding) =>
requiredScopes.has(binding.consumerType as "agent" | "project")
&& requiredEnvKeys.has(binding.envKey),
);
if (requiredMissingBindings.length > 0) {
const detail = requiredMissingBindings.map(formatMissingBindingForOperator).join("; ");
throw new ConfigurationIncompleteFailure(
`configuration incomplete: ${requiredScopedEnvBinding.remediation}; ${detail}`,
{
configurationIncomplete: {
reason: requiredScopedEnvBinding.reason,
companyId: input.companyId,
agentId: input.agentId ?? null,
issueId: input.issueId ?? null,
projectId: input.projectId ?? null,
routineId: input.routineId ?? null,
requiredEnvKeys: requiredScopedEnvBinding.keys,
requiredScopes: requiredScopedEnvBinding.consumerScopes,
missingBindings: requiredMissingBindings,
},
},
);
}
}
if (missingBindings.length > 0) {
const detail = missingBindings.map(formatMissingBindingForOperator).join("; ");
throw new ConfigurationIncompleteFailure(`configuration incomplete: ${detail}`, {
@@ -1080,6 +1170,53 @@ function sameResolvedPath(left: string | null | undefined, right: string | null
return path.resolve(leftPath) === path.resolve(rightPath);
}
async function hasGitPushRemote(cwd: string | null | undefined) {
const normalized = readNonEmptyString(cwd);
if (!normalized) return false;
const remoteNames = await execFile("git", ["remote"], { cwd: normalized })
.then((result) =>
result.stdout
.split(/\r?\n/)
.map((value) => value.trim())
.filter((value) => value.length > 0),
)
.catch(() => []);
for (const remoteName of remoteNames) {
const pushUrl = await execFile("git", ["remote", "get-url", "--push", remoteName], { cwd: normalized })
.then((result) => readNonEmptyString(result.stdout))
.catch(() => null);
if (pushUrl) return true;
}
return false;
}
export async function assertPushCapabilityCheckoutValid(input: {
enabled: boolean;
issue: {
id: string;
identifier: string | null;
} | null;
cwd: string | null | undefined;
}) {
if (!input.enabled || !input.issue) return;
const cwd = readNonEmptyString(input.cwd);
if (!cwd) return;
if (await hasGitPushRemote(cwd)) return;
throw new WorkspaceValidationFailure(
`Issue ${input.issue.identifier ?? input.issue.id} requested the GitHub PR workflow, but checkout "${cwd}" has no configured push remote. Bind the run to a writable repo checkout before dispatching the agent.`,
{
workspaceValidation: {
reason: "missing_git_push_remote",
issueId: input.issue.id,
issueIdentifier: input.issue.identifier,
executionWorkspaceCwd: cwd,
requiredEnvKeys: [...PUSH_CAPABILITY_ENV_KEYS],
},
},
);
}
export async function assertGitSensitiveAdapterWorkspaceValid(input: {
adapterType: string;
agentId: string;
@@ -8631,6 +8768,16 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
: selectedEnvironmentId
? await environmentsSvc.getById(selectedEnvironmentId)
: null;
const runScopedMentionedSkillKeys = await resolveRunScopedMentionedSkillKeys({
db,
companyId: agent.companyId,
issueId,
});
const pushCapabilityPreflightRequired = requiresPushCapabilityPreflight({
adapterType: agent.adapterType,
issueId,
explicitRunScopedSkillKeys: runScopedMentionedSkillKeys,
});
const { resolvedConfig, secretKeys, secretManifest } = await resolveExecutionRunAdapterConfig({
companyId: agent.companyId,
agentId: agent.id,
@@ -8645,6 +8792,15 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
routineEnv: routineEnvContext.env,
secretsSvc,
trustPreset,
requiredScopedEnvBinding: pushCapabilityPreflightRequired
? {
keys: [...PUSH_CAPABILITY_ENV_KEYS],
consumerScopes: ["agent", "project"],
reason: "push_write_credential_missing",
remediation:
"GitHub PR workflow requires GH_TOKEN or GITHUB_TOKEN bound at project or agent scope.",
}
: undefined,
});
if (secretManifest.length > 0) {
context.paperclipSecrets = {
@@ -8653,11 +8809,6 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
} else {
delete context.paperclipSecrets;
}
const runScopedMentionedSkillKeys = await resolveRunScopedMentionedSkillKeys({
db,
companyId: agent.companyId,
issueId,
});
const effectiveResolvedConfig = applyRunScopedMentionedSkillKeys(
resolvedConfig,
runScopedMentionedSkillKeys,
@@ -9271,6 +9422,16 @@ export function heartbeatService(db: Db, options: HeartbeatServiceOptions = {})
environmentDriver: selectedEnvironment.driver,
leaseMetadata: activeEnvironmentLease.lease.metadata,
});
await assertPushCapabilityCheckoutValid({
enabled: pushCapabilityPreflightRequired && executionTarget?.kind === "local",
issue: issueRef
? {
id: issueRef.id,
identifier: issueRef.identifier,
}
: null,
cwd: executionWorkspace.cwd,
});
const adapterEnv = Object.fromEntries(
Object.entries(parseObject(resolvedConfig.env)).filter(
(entry): entry is [string, string] => typeof entry[0] === "string" && typeof entry[1] === "string",
+4
View File
@@ -80,5 +80,9 @@ export { workProductService } from "./work-products.js";
export { logActivity, type LogActivityInput } from "./activity-log.js";
export { notifyHireApproved, type NotifyHireApprovedInput } from "./hire-hook.js";
export { publishLiveEvent, subscribeCompanyLiveEvents } from "./live-events.js";
export {
reconcileCodexLocalManagedHomesOnStartup,
type CodexAuthReconciliationSummary,
} from "./codex-auth-reconciliation.js";
export { reconcilePersistedRuntimeServicesOnStartup, restartDesiredRuntimeServicesOnStartup } from "./workspace-runtime.js";
export { createStorageServiceFromConfig, getStorageService } from "../storage/index.js";
+2 -1
View File
@@ -1204,6 +1204,7 @@ const BLOCKER_ATTENTION_ACTIVE_WAKE_STATUSES = ["queued", "deferred_issue_execut
const BLOCKER_ATTENTION_PENDING_INTERACTION_STATUSES = ["pending"];
const BLOCKER_ATTENTION_PENDING_APPROVAL_STATUSES = ["pending", "revision_requested"];
const BLOCKER_ATTENTION_OPEN_RECOVERY_ORIGIN_KIND = "harness_liveness_escalation";
const BLOCKER_ATTENTION_CHILD_TERMINAL_STATUSES = ["done", "cancelled"];
const PRODUCTIVITY_REVIEW_ORIGIN_KIND = "issue_productivity_review";
const PRODUCTIVITY_REVIEW_TERMINAL_STATUSES = ["done", "cancelled"];
const PRODUCTIVITY_REVIEW_ACTIVITY_ACTIONS = [
@@ -1636,7 +1637,7 @@ async function listIssueBlockerAttentionMap(
and(
eq(issues.companyId, companyId),
inArray(issues.parentId, chunk),
ne(issues.status, "done"),
notInArray(issues.status, BLOCKER_ATTENTION_CHILD_TERMINAL_STATUSES),
),
);
const [explicitBlockerRows, childRows] = await Promise.all([
+160 -9
View File
@@ -628,6 +628,123 @@ export async function inspectExecutionWorkspaceBaseDrift(input: {
return { warnings, currentBaseRefSha, branchBaseRefSha };
}
async function localBranchExists(repoRoot: string, branch: string): Promise<boolean> {
return runGit(["show-ref", "--verify", "--quiet", `refs/heads/${branch}`], repoRoot)
.then(() => true)
.catch(() => false);
}
async function remoteExists(repoRoot: string, remote: string): Promise<boolean> {
return runGit(["remote", "get-url", remote], repoRoot)
.then(() => true)
.catch(() => false);
}
// Resolve the authoritative base ref for a fresh worktree. A configured local
// branch is mapped to its `origin/<branch>` counterpart so unpushed local
// divergence never leaks into the task branch; remote-tracking refs, SHAs, and
// tags are used verbatim, and an unset/`HEAD` base falls back to the detected
// default branch (which already prefers `origin/master`).
async function resolveAuthoritativeBaseRef(
repoRoot: string,
configuredBaseRef: string | null,
): Promise<{ baseRef: string; warnings: string[]; refreshed: boolean }> {
const warnings: string[] = [];
const detectOrHead = async () => (await detectDefaultBranch(repoRoot)) ?? "HEAD";
const configured = configuredBaseRef?.trim();
if (!configured || configured === "HEAD") {
return { baseRef: await detectOrHead(), warnings, refreshed: false };
}
if (parseRemoteTrackingRef(configured)) {
return { baseRef: configured, warnings, refreshed: false };
}
if (await localBranchExists(repoRoot, configured)) {
const remoteCandidate = `origin/${configured}`;
// Refresh here and keep the warnings; the caller skips its own refresh of
// the returned ref (see `refreshed`) so we never fetch the same ref twice.
warnings.push(...await refreshRemoteTrackingBaseRef(repoRoot, remoteCandidate));
if (await resolveBaseRefSha(repoRoot, remoteCandidate)) {
return { baseRef: remoteCandidate, warnings, refreshed: true };
}
if (await remoteExists(repoRoot, "origin")) {
warnings.push(
`Configured base ref "${configured}" is a local branch with no matching origin/${configured}; basing the execution workspace on the local ref, which may include unpushed commits.`,
);
}
return { baseRef: configured, warnings, refreshed: false };
}
return { baseRef: configured, warnings, refreshed: false };
}
// Auto-refresh a reused worktree to the latest base only when it is provably
// unstarted: no task commits past the base and a clean tree (including untracked
// files). This pulls an idle worktree forward to the freshest `origin/master`
// after a long planning phase without ever destroying in-progress work. Only
// remote-tracking bases are eligible; local-only bases keep warn-only drift.
async function refreshUnstartedWorktreeToBase(input: {
repoRoot: string;
worktreePath: string;
branchName: string | null;
baseRef: string;
currentBaseRefSha: string;
recorder?: WorkspaceOperationRecorder | null;
}): Promise<{ refreshed: boolean; baseRefSha: string | null }> {
if (!parseRemoteTrackingRef(input.baseRef)) {
return { refreshed: false, baseRefSha: null };
}
const headSha = await runGit(["rev-parse", "HEAD"], input.worktreePath).catch(() => null);
if (!headSha) {
return { refreshed: false, baseRefSha: null };
}
if (headSha === input.currentBaseRefSha) {
return { refreshed: false, baseRefSha: input.currentBaseRefSha };
}
const commitsPastBaseRaw = await runGit(
["rev-list", "--count", `${input.currentBaseRefSha}..HEAD`],
input.worktreePath,
).catch(() => null);
const commitsPastBase = commitsPastBaseRaw === null ? null : Number.parseInt(commitsPastBaseRaw, 10);
if (commitsPastBase === null || !Number.isFinite(commitsPastBase) || commitsPastBase > 0) {
return { refreshed: false, baseRefSha: null };
}
// Force `--untracked-files=all` so untracked files are counted regardless of a
// local `status.showUntrackedFiles=no`; otherwise the clean-tree guard could
// pass and the `reset --hard` below would destroy untracked work.
const status = await runGit(
["status", "--porcelain", "--untracked-files=all"],
input.worktreePath,
).catch(() => null);
if (status === null || status.trim().length > 0) {
return { refreshed: false, baseRefSha: null };
}
await recordGitOperation(input.recorder, {
phase: "worktree_prepare",
args: ["reset", "--hard", input.currentBaseRefSha],
cwd: input.worktreePath,
metadata: {
repoRoot: input.repoRoot,
worktreePath: input.worktreePath,
branchName: input.branchName,
baseRef: input.baseRef,
previousHeadSha: headSha,
baseRefSha: input.currentBaseRefSha,
refreshedUnstartedWorktree: true,
},
successMessage: `Refreshed unstarted git worktree at ${input.worktreePath} to ${input.baseRef} (${formatShortSha(input.currentBaseRefSha)})\n`,
failureLabel: `git reset --hard ${input.currentBaseRefSha}`,
});
return { refreshed: true, baseRefSha: input.currentBaseRefSha };
}
type GitWorktreeListEntry = {
worktree: string;
@@ -1133,15 +1250,30 @@ export async function realizeExecutionWorkspace(input: {
const configuredBaseRef = typeof rawStrategy.baseRef === "string" && rawStrategy.baseRef.length > 0
? rawStrategy.baseRef
: input.base.repoRef ?? null;
const baseRef = configuredBaseRef
?? await detectDefaultBranch(repoRoot)
?? "HEAD";
const baseRefreshWarnings = await refreshRemoteTrackingBaseRef(repoRoot, baseRef);
const {
baseRef,
warnings: baseRefResolutionWarnings,
refreshed: baseRefAlreadyRefreshed,
} = await resolveAuthoritativeBaseRef(repoRoot, configuredBaseRef);
const baseRefreshWarnings = [
...baseRefResolutionWarnings,
...(baseRefAlreadyRefreshed ? [] : await refreshRemoteTrackingBaseRef(repoRoot, baseRef)),
];
const currentBaseRefSha = await resolveBaseRefSha(repoRoot, baseRef);
await fs.mkdir(worktreeParentDir, { recursive: true });
async function reuseExistingWorktree(reusablePath: string) {
const refresh = currentBaseRefSha
? await refreshUnstartedWorktreeToBase({
repoRoot,
worktreePath: reusablePath,
branchName,
baseRef,
currentBaseRefSha,
recorder: input.recorder ?? null,
})
: { refreshed: false, baseRefSha: null };
const baseDrift = await inspectExecutionWorkspaceBaseDrift({
repoRoot,
worktreePath: reusablePath,
@@ -1184,13 +1316,14 @@ export async function realizeExecutionWorkspace(input: {
});
return {
...input.base,
repoRef: baseRef,
strategy: "git_worktree" as const,
cwd: reusablePath,
branchName,
worktreePath: reusablePath,
warnings: [...baseRefreshWarnings, ...baseDrift.warnings],
created: false,
baseRefSha: baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha,
baseRefSha: refresh.baseRefSha ?? baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha,
};
}
@@ -1284,6 +1417,7 @@ export async function realizeExecutionWorkspace(input: {
return {
...input.base,
repoRef: baseRef,
strategy: "git_worktree",
cwd: worktreePath,
branchName,
@@ -1342,15 +1476,32 @@ export async function ensurePersistedExecutionWorkspaceAvailable(input: {
const repoRoot = await runGit(["rev-parse", "--show-toplevel"], input.base.baseCwd);
const recordedBaseRefSha = readRecordedBaseRefSha(input.workspace.metadata);
if (await directoryExists(cwd)) {
const reuseBaseRef = input.workspace.baseRef ?? input.base.repoRef ?? null;
const reuseWorktreePath = realized.worktreePath ?? cwd;
const baseRefreshWarnings = reuseBaseRef
? await refreshRemoteTrackingBaseRef(repoRoot, reuseBaseRef)
: [];
const currentBaseRefSha = reuseBaseRef ? await resolveBaseRefSha(repoRoot, reuseBaseRef) : null;
const refresh = reuseBaseRef && currentBaseRefSha
? await refreshUnstartedWorktreeToBase({
repoRoot,
worktreePath: reuseWorktreePath,
branchName: realized.branchName,
baseRef: reuseBaseRef,
currentBaseRefSha,
recorder: input.recorder ?? null,
})
: { refreshed: false, baseRefSha: null };
const baseDrift = await inspectExecutionWorkspaceBaseDrift({
repoRoot,
worktreePath: realized.worktreePath ?? cwd,
worktreePath: reuseWorktreePath,
branchName: realized.branchName,
baseRef: input.workspace.baseRef ?? input.base.repoRef ?? null,
baseRef: reuseBaseRef,
recordedBaseRefSha,
skipRefresh: true,
});
realized.warnings = baseDrift.warnings;
realized.baseRefSha = recordedBaseRefSha ?? baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha;
realized.warnings = [...baseRefreshWarnings, ...baseDrift.warnings];
realized.baseRefSha = refresh.baseRefSha ?? recordedBaseRefSha ?? baseDrift.branchBaseRefSha ?? baseDrift.currentBaseRefSha;
if (provisionCommand) {
await provisionExecutionWorktree({
strategy: {
-267
View File
@@ -1,267 +0,0 @@
---
name: paperclip-dev
required: false
description: >
Develop and operate a local Paperclip instance — start and stop servers,
pull updates from master, run builds and tests, manage worktrees, back up
databases, and diagnose problems. Use whenever you need to work on the
Paperclip codebase itself or keep a running instance healthy.
---
# Paperclip Dev
This skill covers the day-to-day workflows for developing and operating a local Paperclip instance. It assumes you are working inside the Paperclip repo checkout with `origin` pointing to `git@github.com:paperclipai/paperclip.git`.
> **OPEN SOURCE HYGIENE:** This repository is public-facing. Treat anything you push to `origin` as publishable. Never commit or push secrets, API keys, tokens, private logs, PII, customer data, or machine-local configuration that should stay private. Keep git history tidy as well: avoid pushing throwaway branches, noisy checkpoint commits, or speculative work that does not need to be shared upstream.
> **MANDATORY:** Before running any CLI command, building, testing, or managing worktrees, you MUST read `doc/DEVELOPING.md` in the Paperclip repo. It is the canonical reference for all `paperclipai` CLI commands, their options, build/test workflows, database operations, worktree management, and diagnostics. Do NOT guess at flags or options — read the doc first.
## Quick Command Reference
These are the most common commands. For full option tables and details, see `doc/DEVELOPING.md`.
| Task | Command |
|------|---------|
| Start server (first time or normal) | `npx paperclipai run` |
| Dev mode with hot reload | `pnpm dev` |
| Stop dev server | `pnpm dev:stop` |
| Build | `pnpm build` |
| Type-check | `pnpm typecheck` |
| Run tests | `pnpm test` |
| Run migrations | `pnpm db:migrate` |
| Regenerate Drizzle client | `pnpm db:generate` |
| Back up database | `npx paperclipai db:backup` |
| Health check | `npx paperclipai doctor --repair` |
| Print env vars | `npx paperclipai env` |
| Trigger agent heartbeat | `npx paperclipai heartbeat run --agent-id <id>` |
| Install agent skills locally | `npx paperclipai agent local-cli <agent> --company-id <id>` |
## Pulling from Master
```bash
git fetch origin && git pull origin master
pnpm install && pnpm build
```
If schema changes landed, also run `pnpm db:generate && pnpm db:migrate`.
## Worktrees
Paperclip worktrees combine git worktrees with isolated Paperclip instances — each gets its own database, server port, and environment seeded from the primary instance.
> **MANDATORY:** Before creating or managing worktrees, you MUST read the "Worktree-local Instances" and "Worktree CLI Reference" sections in `doc/DEVELOPING.md`. That is the canonical reference for all worktree commands, their options, seed modes, and environment variables.
### When to Use Worktrees
- Starting a feature branch that needs its own Paperclip environment
- Running parallel agent work without cross-contaminating the primary instance
- Testing Paperclip changes in isolation before merging
### Command Overview
The CLI has two tiers (see `doc/DEVELOPING.md` for full option tables):
| Command | Purpose |
|---------|---------|
| `worktree:make <name>` | Create worktree + isolated instance in one step |
| `worktree:list` | List worktrees and their Paperclip status |
| `worktree:merge-history` | Preview/import issue history between worktrees |
| `worktree:cleanup <name>` | Remove worktree, branch, and instance data |
| `worktree init` | Bootstrap instance inside existing worktree |
| `worktree env` | Print shell exports for worktree instance |
| `worktree reseed` | Refresh worktree DB from another instance |
| `worktree repair` | Fix broken/missing worktree instance metadata |
### Typical Workflow
```bash
# 1. Create a worktree for a feature
npx paperclipai worktree:make my-feature --start-point origin/main
# 2. Move into the worktree (path printed by worktree:make) and source the environment
cd <worktree-path>
eval "$(npx paperclipai worktree env)"
# 3. Start the isolated Paperclip server
npx paperclipai run
# 4. Do your work
# 5. When done, merge history back if needed
npx paperclipai worktree:merge-history --from paperclip-my-feature --to current --apply
# 6. Clean up
npx paperclipai worktree:cleanup my-feature
```
## Forks — Prefer Pushing to a User Fork
If the user has a personal fork of `paperclipai/paperclip` configured as a git remote, push your feature branches to **that fork** instead of creating branches on the main repo. This keeps the upstream branch list clean and matches the standard open-source contribution flow.
### Detect a fork remote
Before pushing or creating a PR, list remotes and check for one that points at a non-`paperclipai` GitHub fork:
```bash
git remote -v
```
Treat any remote whose URL points to `github.com:<user>/paperclip` (or `github.com/<user>/paperclip.git`) as the user's fork. Common names are `fork`, `<username>`, or `myfork`. The remote named `origin` or `upstream` that points at `paperclipai/paperclip` is the canonical upstream — do not push feature branches there if a fork exists.
### Pushing to the fork
```bash
# Push the current branch to the user's fork and set upstream
git push -u <fork-remote> HEAD
```
Then create the PR from the fork branch:
```bash
gh pr create --repo paperclipai/paperclip --head <fork-owner>:<branch-name> ...
```
`gh pr create` usually figures out the head ref automatically when run from a branch tracking the fork; the explicit `--head <owner>:<branch>` form is the reliable fallback when it does not.
### When no fork exists
If `git remote -v` shows only `paperclipai/paperclip` remotes (no user fork), fall back to pushing branches to `origin` as before. Do NOT create a fork on the user's behalf — ask first.
### Keeping the fork up to date
The canonical remote that points at `paperclipai/paperclip` may be named `origin` **or** `upstream` depending on how the user set up the repo. Detect it the same way as in the "Detect a fork remote" step, then fetch and push from/with that remote so the sync works under either convention:
```bash
UPSTREAM_REMOTE=$(git remote -v | awk '/paperclipai\/paperclip.*\(fetch\)/{print $1; exit}')
git fetch "$UPSTREAM_REMOTE"
git push <fork-remote> "${UPSTREAM_REMOTE}/master:master"
```
## Pull Requests
> **MANDATORY PRE-FLIGHT:** Before creating ANY pull request, you MUST read the canonical source files listed below. Do NOT run `gh pr create` until you have read these files and verified your PR body matches every required section.
### Step 1 — Read the canonical files
You MUST read all three of these files before creating a PR:
1. **`.github/PULL_REQUEST_TEMPLATE.md`** — the required PR body structure
2. **`CONTRIBUTING.md`** — contribution conventions, PR requirements, and thinking-path examples
3. **`.github/workflows/pr.yml`** — CI checks that gate merge
### Step 2 — Validate your PR body against this checklist
After reading the template, verify your `--body` includes every one of these sections (names must match exactly):
- [ ] `## Thinking Path` — blockquote style, 5-8 reasoning steps
- [ ] `## What Changed` — bullet list of concrete changes
- [ ] `## Verification` — how a reviewer confirms this works
- [ ] `## Risks` — what could go wrong
- [ ] `## Model Used` — provider, model ID, version, capabilities
- [ ] `## Checklist` — copied from the template, items checked off
If any section is missing or empty, do NOT submit the PR. Go back and fill it in.
### Step 3 — Create the PR
Only after completing Steps 1 and 2, run `gh pr create`. Use the template contents as the structure for `--body` — do not write a freeform summary.
## Hard Rules — Do NOT Bypass
These rules exist because agents have caused real damage by improvising around CLI failures. Follow them exactly.
1. **CLI is the only interface to worktrees and databases.** All worktree and database operations MUST go through `npx paperclipai` / `pnpm paperclipai` commands. You MUST NOT:
- Run `pg_dump`, `pg_restore`, `psql`, `createdb`, `dropdb`, or any raw postgres commands
- Manually set `DATABASE_URL` to point a worktree server at another instance's database
- Run `rm -rf` on any `.paperclip/`, `.paperclip-worktrees/`, or `db/` directory
- Directly manipulate embedded postgres data directories
- Kill postgres processes by PID
2. **If a CLI command fails, stop and report.** Do NOT attempt workarounds. If `worktree:make`, `worktree reseed`, `worktree init`, `worktree:cleanup`, or any other `paperclipai` command fails:
- Report the exact error message in your task comment
- Set the task to `blocked`
- Suggest running `npx paperclipai doctor --repair` or recreating the worktree from scratch
- Do NOT try to manually replicate what the CLI does
3. **Never share databases between instances.** Each worktree instance gets its own isolated database. Never override `DATABASE_URL` to point one instance at another's database. This destroys isolation and can corrupt production data.
4. **Starting a dev server in a worktree requires setup first.** The correct sequence is:
```bash
# If the worktree already exists but has no running instance:
cd <worktree-path>
eval "$(npx paperclipai worktree env)"
pnpm install && pnpm build
npx paperclipai run # or pnpm dev
# If the worktree needs a fresh database:
npx paperclipai worktree reseed --seed-mode full
# If the worktree is broken beyond repair:
npx paperclipai worktree:cleanup <name>
npx paperclipai worktree:make <name> --seed-mode full
```
If any step fails, follow rule 2 — stop and report.
5. **Seeding is a CLI operation.** When asked to seed a worktree database from the main instance, use `worktree reseed` or recreate with `worktree:make --seed-mode full`. Read `doc/DEVELOPING.md` for the full option tables. Never attempt manual database copying.
## Persistent Dev Servers (for Manual Testing)
When an agent needs to start a dev server that outlives the current heartbeat — for example, so a human or QA agent can manually test against it — the server process **must** be launched in a detached session. A process started directly from a heartbeat shell is killed when the heartbeat exits.
### Use `tmux` for persistent servers
```bash
# 1. cd into the worktree (or main repo) and source the environment
cd <worktree-path>
eval "$(npx paperclipai worktree env)" # skip if using the primary instance
# 2. Start the dev server in a named, detached tmux session
tmux new-session -d -s <session-name> 'pnpm dev'
# Example with a descriptive name:
tmux new-session -d -s auth-fix-3102 'pnpm dev'
```
### Managing the session
| Task | Command |
|------|---------|
| Check if the session is alive | `tmux has-session -t <session-name> 2>/dev/null && echo running` |
| View server output | `tmux capture-pane -t <session-name> -p` |
| Kill the session | `tmux kill-session -t <session-name>` |
| List all tmux sessions | `tmux list-sessions` |
### Verifying the server is reachable
After launching, confirm the port is listening before reporting success:
```bash
# Wait briefly for startup, then verify
sleep 3
curl -sf http://127.0.0.1:<port>/api/health && echo "Server is up"
lsof -nP -iTCP:<port> -sTCP:LISTEN
```
### Key rules
1. **Always use `tmux` (or equivalent)** when a dev server needs to stay running after the heartbeat ends. A server started directly from the agent shell will die when the heartbeat exits, even if it appeared healthy moments before.
2. **Name the session descriptively** — include the worktree name and port (e.g., `auth-fix-3102`).
3. **Verify the server is listening** before reporting the URL to anyone.
4. **Do not use `nohup` or `&` alone** — these are unreliable for agent shells that may have their entire process group killed.
5. **Clean up when done** — kill the tmux session when the testing is complete.
## Common Mistakes
| Mistake | Fix |
|---------|-----|
| Server won't start | Run `npx paperclipai doctor --repair` to diagnose and auto-fix |
| Forgetting to source worktree env | Run `eval "$(npx paperclipai worktree env)"` after cd-ing into the worktree |
| Stale dependencies after pull | Run `pnpm install && pnpm build` after pulling |
| Schema out of date after pull | Run `pnpm db:generate && pnpm db:migrate` |
| Reseeding while target DB is running | Stop the target server first, or use `--allow-live-target` |
| Cleaning up with unmerged commits | Merge or push first, or use `--force` if intentionally discarding |
| Running agents against wrong instance | Verify `PAPERCLIP_API_URL` points to the correct port |
| CLI command fails | Do NOT work around it — report the error and block (see Hard Rules above) |
| Agent tries manual postgres operations | NEVER do this — all DB ops go through the CLI (see Hard Rules above) |
| Dev server dies between heartbeats | Launch in a detached `tmux` session — see "Persistent Dev Servers" above |
| Pushed feature branch to `paperclipai/paperclip` when a fork exists | Push to the user's fork remote instead — see "Forks" above |
+194
View File
@@ -8,6 +8,24 @@ import { JsonSchemaForm, getDefaultValues } from "./JsonSchemaForm";
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).IS_REACT_ACT_ENVIRONMENT = true;
// Radix Select relies on PointerEvent, pointer capture, and ResizeObserver,
// none of which jsdom implements. Stub them so the dropdown can open in tests.
if (!globalThis.PointerEvent) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).PointerEvent = MouseEvent;
}
if (typeof Element !== "undefined" && !Element.prototype.hasPointerCapture) {
Element.prototype.hasPointerCapture = () => false;
Element.prototype.releasePointerCapture = () => {};
}
class ResizeObserverStub {
observe() {}
unobserve() {}
disconnect() {}
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(globalThis as any).ResizeObserver = (globalThis as any).ResizeObserver ?? ResizeObserverStub;
// SecretBindingPicker pulls in CompanyContext + react-query. Stub it so we can
// exercise SecretField in isolation. The stub renders a select with the same
// onChange contract as the real picker.
@@ -358,6 +376,7 @@ describe("JsonSchemaForm secret-ref rendering", () => {
sshPort: { type: "number", default: 22 },
cpu: { type: "number" },
memory: { type: "string" },
size: { type: "string", enum: ["small", "large"] },
reuseLease: { type: "boolean", default: false },
tags: { type: "array", items: { type: "string" } },
},
@@ -373,6 +392,42 @@ describe("JsonSchemaForm secret-ref rendering", () => {
expect("apiKey" in defaults).toBe(false);
expect("cpu" in defaults).toBe(false);
expect("memory" in defaults).toBe(false);
expect("size" in defaults).toBe(false);
});
it("renders datalist suggestions for numeric fields when examples are present", async () => {
const root = createRoot(container);
await act(async () => {
root.render(
<JsonSchemaForm
schema={{
type: "object",
properties: {
memory: {
type: "integer",
examples: [1, 2, 4, 8],
},
},
}}
values={{}}
onChange={() => {}}
/>,
);
});
const input = container.querySelector<HTMLInputElement>('input[type="number"]');
// The "/" in the field path is sanitized so the id is a valid CSS/HTML identifier.
expect(input?.getAttribute("list")).toBe("-memory-suggestions");
expect(container.querySelector("datalist")?.getAttribute("id")).toBe("-memory-suggestions");
const options = Array.from(container.querySelectorAll("datalist option")).map((option) =>
option.getAttribute("value"),
);
expect(options).toEqual(["1", "2", "4", "8"]);
await act(async () => {
root.unmount();
});
});
it("keeps the password fallback for short raw values", async () => {
@@ -407,3 +462,142 @@ describe("JsonSchemaForm secret-ref rendering", () => {
});
});
});
describe("JsonSchemaForm enum rendering", () => {
let container: HTMLDivElement;
const numericEnumSchema = {
type: "object" as const,
properties: {
memory: {
type: "integer" as const,
enum: [1, 2, 4, 8],
},
},
};
beforeEach(() => {
container = document.createElement("div");
document.body.appendChild(container);
});
afterEach(() => {
container.remove();
document.body.innerHTML = "";
vi.clearAllMocks();
});
async function openSelect() {
const trigger = container.querySelector<HTMLElement>('[role="combobox"]');
expect(trigger).not.toBeNull();
await act(async () => {
trigger!.dispatchEvent(
new PointerEvent("pointerdown", { bubbles: true, button: 0 }),
);
trigger!.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
}
function optionByLabel(label: string): Element | undefined {
return Array.from(document.querySelectorAll('[role="option"]')).find(
(option) => option.textContent?.trim() === label,
);
}
it("renders an optional numeric enum as a dropdown with a blank row and no 0", async () => {
const root = createRoot(container);
await act(async () => {
root.render(
<JsonSchemaForm schema={numericEnumSchema} values={{}} onChange={() => {}} />,
);
});
await openSelect();
const labels = Array.from(document.querySelectorAll('[role="option"]')).map(
(option) => option.textContent?.trim(),
);
// A blank "None" row is offered so the user can express "not configured".
expect(labels).toContain("None");
expect(labels).toEqual(expect.arrayContaining(["1", "2", "4", "8"]));
// 0 is not a valid Daytona memory size and must never appear.
expect(labels).not.toContain("0");
await act(async () => {
root.unmount();
});
});
it("selects the blank row by default when no value is configured", async () => {
const root = createRoot(container);
await act(async () => {
root.render(
<JsonSchemaForm schema={numericEnumSchema} values={{}} onChange={() => {}} />,
);
});
await openSelect();
const noneOption = optionByLabel("None");
expect(noneOption).toBeTruthy();
// Radix marks the active selection with aria-selected / data-state checked.
expect(noneOption?.getAttribute("aria-selected")).toBe("true");
await act(async () => {
root.unmount();
});
});
it("coerces the selected numeric enum value back to a number", async () => {
const onChange = vi.fn();
const root = createRoot(container);
await act(async () => {
root.render(
<JsonSchemaForm schema={numericEnumSchema} values={{}} onChange={onChange} />,
);
});
await openSelect();
await act(async () => {
optionByLabel("2")!.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
// Number, not the string "2", so server-side integer validation passes.
expect(onChange).toHaveBeenCalledWith({ memory: 2 });
await act(async () => {
root.unmount();
});
});
it("maps the blank row back to an unset (undefined) value", async () => {
const onChange = vi.fn();
const root = createRoot(container);
await act(async () => {
root.render(
<JsonSchemaForm
schema={numericEnumSchema}
values={{ memory: 2 }}
onChange={onChange}
/>,
);
});
await openSelect();
await act(async () => {
optionByLabel("None")!.dispatchEvent(new MouseEvent("click", { bubbles: true }));
});
expect(onChange).toHaveBeenCalledWith({ memory: undefined });
await act(async () => {
root.unmount();
});
});
});
+113 -46
View File
@@ -47,6 +47,7 @@ export interface JsonSchemaNode {
description?: string;
default?: unknown;
enum?: unknown[];
examples?: unknown[];
const?: unknown;
format?: string;
@@ -155,7 +156,7 @@ export function getDefaultForSchema(schema: JsonSchemaNode): unknown {
case "boolean":
return false;
case "enum":
return schema.enum?.[0] ?? "";
return undefined;
case "array":
return [];
case "object": {
@@ -438,6 +439,13 @@ const BooleanField = React.memo(({
BooleanField.displayName = "BooleanField";
/**
* Sentinel value for the "not configured" row of an optional enum select.
* Radix `Select` forbids an empty-string item value, so we map the unset state
* onto this sentinel and translate it back to `undefined` on change.
*/
const ENUM_UNSET_VALUE = "__paperclip_unset__";
/**
* Specialized field for enum (select) values.
*/
@@ -459,32 +467,63 @@ const EnumField = React.memo(({
description?: string;
error?: string;
options: unknown[];
}) => (
<FieldWrapper
label={label}
description={description}
required={isRequired}
error={error}
disabled={disabled}
>
<Select
value={String(value ?? "")}
onValueChange={onChange}
}) => {
// Optional enums get a leading blank row so the user can express "not
// configured"; it is also the selected row when no value is set.
const showUnsetOption = !isRequired;
// When every option is numeric, coerce the selected string back to a number
// so the payload keeps the schema's integer/number type — a stringified "2"
// would otherwise fail server-side integer validation.
const numericOptions =
options.length > 0 && options.every((option) => typeof option === "number");
const isUnset = value === undefined || value === null || value === "";
const selectValue = isUnset
? showUnsetOption
? ENUM_UNSET_VALUE
: ""
: String(value);
const handleChange = (next: string) => {
if (next === ENUM_UNSET_VALUE) {
onChange(undefined);
return;
}
onChange(numericOptions ? Number(next) : next);
};
return (
<FieldWrapper
label={label}
description={description}
required={isRequired}
error={error}
disabled={disabled}
>
<SelectTrigger className="w-full">
<SelectValue placeholder="Select an option" />
</SelectTrigger>
<SelectContent>
{options.map((option) => (
<SelectItem key={String(option)} value={String(option)}>
{String(option)}
</SelectItem>
))}
</SelectContent>
</Select>
</FieldWrapper>
));
<Select
value={selectValue}
onValueChange={handleChange}
disabled={disabled}
>
<SelectTrigger className="w-full">
<SelectValue placeholder="Select an option" />
</SelectTrigger>
<SelectContent>
{showUnsetOption && (
<SelectItem value={ENUM_UNSET_VALUE} textValue="None">
<span className="text-muted-foreground">None</span>
</SelectItem>
)}
{options.map((option) => (
<SelectItem key={String(option)} value={String(option)}>
{String(option)}
</SelectItem>
))}
</SelectContent>
</Select>
</FieldWrapper>
);
});
EnumField.displayName = "EnumField";
@@ -689,6 +728,7 @@ SecretField.displayName = "SecretField";
* Specialized field for numeric (number/integer) values.
*/
const NumberField = React.memo(({
id,
value,
onChange,
disabled,
@@ -698,7 +738,11 @@ const NumberField = React.memo(({
error,
defaultValue,
type,
minimum,
maximum,
suggestions,
}: {
id: string;
value: unknown;
onChange: (val: unknown) => void;
disabled: boolean;
@@ -708,28 +752,47 @@ const NumberField = React.memo(({
error?: string;
defaultValue?: unknown;
type: "number" | "integer";
}) => (
<FieldWrapper
label={label}
description={description}
required={isRequired}
error={error}
disabled={disabled}
>
<Input
type="number"
step={type === "integer" ? "1" : "any"}
value={value !== undefined ? String(value) : ""}
onChange={(e) => {
const val = e.target.value;
onChange(val === "" ? undefined : Number(val));
}}
placeholder={String(defaultValue ?? "")}
minimum?: number;
maximum?: number;
suggestions?: unknown[];
}) => {
const hasSuggestions = Array.isArray(suggestions) && suggestions.length > 0;
// Sanitize the path-based id so it is a valid CSS/HTML identifier (paths can contain "/").
const listId = hasSuggestions ? `${id.replace(/[^a-zA-Z0-9_-]/g, "-")}-suggestions` : undefined;
return (
<FieldWrapper
label={label}
description={description}
required={isRequired}
error={error}
disabled={disabled}
aria-invalid={!!error}
/>
</FieldWrapper>
));
>
<Input
type="number"
step={type === "integer" ? "1" : "any"}
min={minimum}
max={maximum}
list={listId}
value={value !== undefined ? String(value) : ""}
onChange={(e) => {
const val = e.target.value;
const trimmed = val.trim();
onChange(trimmed === "" ? undefined : Number(trimmed));
}}
placeholder={String(defaultValue ?? "")}
disabled={disabled}
aria-invalid={!!error}
/>
{listId ? (
<datalist id={listId}>
{suggestions!.map((suggestion) => (
<option key={String(suggestion)} value={String(suggestion)} />
))}
</datalist>
) : null}
</FieldWrapper>
);
});
NumberField.displayName = "NumberField";
@@ -1044,6 +1107,7 @@ const FormField = React.memo(({
case "integer":
return (
<NumberField
id={path}
value={value}
onChange={onChange}
disabled={isReadOnly}
@@ -1053,6 +1117,9 @@ const FormField = React.memo(({
error={error}
defaultValue={propSchema.default}
type={type as "number" | "integer"}
minimum={typeof propSchema.minimum === "number" ? propSchema.minimum : undefined}
maximum={typeof propSchema.maximum === "number" ? propSchema.maximum : undefined}
suggestions={Array.isArray(propSchema.examples) ? propSchema.examples : undefined}
/>
);
+16 -49
View File
@@ -2643,43 +2643,20 @@ export function AgentSkillsTab({
);
const optionalSkillRows = useMemo<SkillRow[]>(
() =>
(companySkills ?? [])
.filter((skill) => !adapterEntryByKey.get(skill.key)?.required)
.map((skill) => ({
id: skill.id,
key: skill.key,
name: skill.name,
description: skill.description,
detail: adapterEntryByKey.get(skill.key)?.detail ?? null,
locationLabel: adapterEntryByKey.get(skill.key)?.locationLabel ?? null,
originLabel: adapterEntryByKey.get(skill.key)?.originLabel ?? null,
linkTo: `/skills/${skill.id}`,
readOnly: false,
adapterEntry: adapterEntryByKey.get(skill.key) ?? null,
})),
(companySkills ?? []).map((skill) => ({
id: skill.id,
key: skill.key,
name: skill.name,
description: skill.description,
detail: adapterEntryByKey.get(skill.key)?.detail ?? null,
locationLabel: adapterEntryByKey.get(skill.key)?.locationLabel ?? null,
originLabel: adapterEntryByKey.get(skill.key)?.originLabel ?? null,
linkTo: `/skills/${skill.id}`,
readOnly: false,
adapterEntry: adapterEntryByKey.get(skill.key) ?? null,
})),
[adapterEntryByKey, companySkills],
);
const requiredSkillRows = useMemo<SkillRow[]>(
() =>
(skillSnapshot?.entries ?? [])
.filter((entry) => entry.required)
.map((entry) => {
const companySkill = companySkillByKey.get(entry.key);
return {
id: companySkill?.id ?? `required:${entry.key}`,
key: entry.key,
name: companySkill?.name ?? entry.key,
description: companySkill?.description ?? null,
detail: entry.detail ?? null,
locationLabel: entry.locationLabel ?? null,
originLabel: entry.originLabel ?? null,
linkTo: companySkill ? `/skills/${companySkill.id}` : null,
readOnly: false,
adapterEntry: entry,
};
}),
[companySkillByKey, skillSnapshot],
);
const unmanagedSkillRows = useMemo<SkillRow[]>(
() =>
(skillSnapshot?.entries ?? [])
@@ -2780,8 +2757,6 @@ export function AgentSkillsTab({
<>
{(() => {
const renderSkillRow = (skill: SkillRow) => {
const adapterEntry = skill.adapterEntry ?? adapterEntryByKey.get(skill.key);
const required = Boolean(adapterEntry?.required);
const summaryText = resolveSkillSummaryText(skill, { fallbackKey: true });
const rowClassName = cn(
"flex items-start gap-3 border-b border-border px-3 py-3 text-sm last:border-b-0",
@@ -2828,8 +2803,8 @@ export function AgentSkillsTab({
);
}
const checked = required || skillDraft.includes(skill.key);
const disabled = required || skillSnapshot?.mode === "unsupported";
const checked = skillDraft.includes(skill.key);
const disabled = skillSnapshot?.mode === "unsupported";
const checkbox = (
<input
type="checkbox"
@@ -2847,14 +2822,7 @@ export function AgentSkillsTab({
return (
<label key={skill.id} className={rowClassName}>
{required && adapterEntry?.requiredReason ? (
<Tooltip>
<TooltipTrigger asChild>
<span>{checkbox}</span>
</TooltipTrigger>
<TooltipContent side="top">{adapterEntry.requiredReason}</TooltipContent>
</Tooltip>
) : skillSnapshot?.mode === "unsupported" ? (
{skillSnapshot?.mode === "unsupported" ? (
<Tooltip>
<TooltipTrigger asChild>
<span>{checkbox}</span>
@@ -2895,7 +2863,7 @@ export function AgentSkillsTab({
);
};
if (optionalSkillRows.length === 0 && requiredSkillRows.length === 0 && unmanagedSkillRows.length === 0) {
if (optionalSkillRows.length === 0 && unmanagedSkillRows.length === 0) {
return (
<section className="border-y border-border">
<div className="px-3 py-6 text-sm text-muted-foreground">
@@ -2917,7 +2885,6 @@ export function AgentSkillsTab({
{renderSkillSection("Other skills", otherSkillRows)}
{renderSkillSection("Required by Paperclip", requiredSkillRows)}
{unmanagedSkillRows.length > 0 && (
<section className="border-y border-border">
+6 -18
View File
@@ -583,11 +583,9 @@ function buildAcpxClaudeSnapshot(): AgentSkillSnapshot {
runtimeName: "paperclip",
desired: true,
managed: true,
required: true,
requiredReason: "Paperclip coordination skill is mandatory for control-plane agents.",
state: "configured",
origin: "paperclip_required",
originLabel: "Required by Paperclip",
origin: "company_managed",
originLabel: "Managed by Paperclip",
readOnly: false,
sourcePath: "skills/paperclip",
targetPath: null,
@@ -598,7 +596,6 @@ function buildAcpxClaudeSnapshot(): AgentSkillSnapshot {
runtimeName: "design-guide",
desired: true,
managed: true,
required: false,
state: "configured",
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -612,7 +609,6 @@ function buildAcpxClaudeSnapshot(): AgentSkillSnapshot {
runtimeName: "mobile-app-qa",
desired: false,
managed: true,
required: false,
state: "available",
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -638,11 +634,9 @@ function buildAcpxCodexSnapshot(): AgentSkillSnapshot {
runtimeName: "paperclip",
desired: true,
managed: true,
required: true,
requiredReason: "Paperclip coordination skill is mandatory for control-plane agents.",
state: "configured",
origin: "paperclip_required",
originLabel: "Required by Paperclip",
origin: "company_managed",
originLabel: "Managed by Paperclip",
readOnly: false,
sourcePath: "skills/paperclip",
targetPath: null,
@@ -653,7 +647,6 @@ function buildAcpxCodexSnapshot(): AgentSkillSnapshot {
runtimeName: "design-guide",
desired: false,
managed: true,
required: false,
state: "available",
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -667,7 +660,6 @@ function buildAcpxCodexSnapshot(): AgentSkillSnapshot {
runtimeName: "mobile-app-qa",
desired: false,
managed: true,
required: false,
state: "available",
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -695,11 +687,9 @@ function buildAcpxCustomSnapshot(): AgentSkillSnapshot {
runtimeName: "paperclip",
desired: false,
managed: true,
required: true,
requiredReason: "Paperclip coordination skill is mandatory for control-plane agents.",
state: "available",
origin: "paperclip_required",
originLabel: "Required by Paperclip",
origin: "company_managed",
originLabel: "Managed by Paperclip",
readOnly: false,
sourcePath: "skills/paperclip",
targetPath: null,
@@ -710,7 +700,6 @@ function buildAcpxCustomSnapshot(): AgentSkillSnapshot {
runtimeName: "design-guide",
desired: true,
managed: true,
required: false,
state: "configured",
origin: "company_managed",
originLabel: "Managed by Paperclip",
@@ -725,7 +714,6 @@ function buildAcpxCustomSnapshot(): AgentSkillSnapshot {
runtimeName: "mobile-app-qa",
desired: false,
managed: true,
required: false,
state: "available",
origin: "company_managed",
originLabel: "Managed by Paperclip",