83a293b16155efed5aa002071c4f4b7007add400
80 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Jannes Stubbemann
|
362c30ccdc |
feat(server): opt-in OpenTelemetry auto-instrumentation (#3735)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP > - Today there is no OpenTelemetry bootstrap in the server, so operators who want traces have to patch their fork or run a sidecar that captures only HTTP-level info > - An opt-in bootstrap that costs nothing when disabled is the minimum-viable surface for this audience > - The OpenTelemetry packages are heavyweight enough that we don't want them in the default dependency graph — they should load only when the operator configures an OTLP endpoint > - This pull request adds a self-contained `server/src/instrumentation.ts` that dynamically imports the OTel SDK and starts it when `OTEL_EXPORTER_OTLP_ENDPOINT` is set, and is a complete no-op otherwise ## Linked Issues or Issue Description No existing issue covers this directly — feature-gap description following the feature-request template: **Problem or motivation** Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP — but the server has no OpenTelemetry bootstrap. Operators who want traces today must patch their fork or run a sidecar that captures only HTTP-level information. **Proposed solution** An opt-in OTel bootstrap gated on `OTEL_EXPORTER_OTLP_ENDPOINT`, loaded via dynamic `import()` only when configured, so the heavyweight OTel packages stay out of the default dependency graph. **Alternatives considered** Related open PRs found during the duplicate-PR search approach observability differently: #4894 adds OTLP instrumentation to Paperclip core unconditionally, and #3752 proposes an observability plugin. Not duplicates — different layering: this PR keeps the default install dependency-free via opt-in dynamic import. ## What Changed - New `server/src/instrumentation.ts` — opt-in OpenTelemetry auto-instrumentation. Gated on `OTEL_EXPORTER_OTLP_ENDPOINT`. Respects the standard OTel env vars (`OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`). Skips the fs/dns/net auto-instrumentations (too chatty). `sdk.start()` is wrapped in try/catch so a bad endpoint or missing native bindings doesn't crash the server. `process.once("SIGTERM" / "SIGINT", …)` for clean shutdown on the first signal only. OTel packages are loaded via dynamic `import()` so they are true optional runtime dependencies — no entries in `package.json`, no lockfile churn. - `server/src/index.ts` — import `./instrumentation.js` as the very first statement so auto-instrumentation can patch `http` / `express` / `pg` before they are evaluated by downstream modules. ## Verification - `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 pnpm start` after `pnpm install @opentelemetry/{sdk-node,auto-instrumentations-node,exporter-trace-otlp-grpc,resources,semantic-conventions}` in `server/` — traces show up in the configured collector; HTTP, Express, and Postgres spans are populated. - `OTEL_EXPORTER_OTLP_ENDPOINT` unset — server starts with no OTel-shaped output in logs, no behavior change. - `OTEL_EXPORTER_OTLP_ENDPOINT=…` set but packages not installed — single `console.warn` at startup telling the operator which packages to install. ## Risks Low. No behavior change unless the env var is set. The bootstrap never throws into the caller; every failure path ends in `console.warn` / `console.error` and falls through to non-traced operation. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dale Carman
|
d782c4cd53 |
fix(heartbeat): prevent zombie run coalescing and ensure startup reap completes before timer ticks (#1731)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents run in heartbeats — short execution windows triggered by the heartbeat service > - The heartbeat service coalesces overlapping wakeups: if a run for an agent is already active, a new wakeup merges into it rather than creating a duplicate > - But when the server restarts, in-progress runs are left in `"running"` status in the database — their child processes are gone, but the DB rows persist as orphans > - The startup `reapOrphanedRuns()` was fired as a `void` promise — the timer interval started immediately in parallel, so the first timer tick could coalesce a new wakeup into an orphaned "running" row before the reap had a chance to remove it > - Once coalesced, the orphan's `updatedAt` refreshed, making the reaper skip it as "not old enough" — a zombie run that prevents the agent from ever waking again > - This PR fixes both the coalescing guard (do not coalesce into a zombie) and the startup ordering (await reap before starting the timer), eliminating the death spiral ## What Changed - **`server/src/index.ts`** — `startServer` now `await`s `reapOrphanedRuns()` (with one retry) before calling `setInterval`. Timer ticks cannot start until orphaned runs are cleaned up. - **`server/src/services/heartbeat.ts`** — Added two exported pure functions: - `isZombieRun(run, tracked)` — returns `true` if a run is `"running"` in the DB but has no live entry in the in-memory `runningProcesses` Map - `filterZombieCoalesceTarget(target, tracked)` — returns `null` if the coalesce candidate is a zombie, letting the wakeup fall through to create a new queued run instead - Both coalescing call sites now use `filterZombieCoalesceTarget` before deciding to coalesce - **`server/src/__tests__/heartbeat-zombie-guard.test.ts`** — 8 new behavioral tests covering `isZombieRun` and `filterZombieCoalesceTarget`, including the critical zombie scenario, legitimate live runs, queued runs (must never be filtered), and null pass-through ## Verification ```bash # Run the new tests pnpm test:run ``` Manual reproduction (before fix): 1. Start an agent on a timer heartbeat 2. Kill the server mid-run (child process dies, DB row stays `"running"`) 3. Restart the server 4. Observe: agent never wakes again — subsequent wakeups coalesce into the dead run, refreshing `updatedAt`, keeping it alive forever After fix: startup reap clears the orphan before the timer starts; subsequent wakeups create fresh queued runs. The one pre-existing test failure (`worktree helpers > copies shared git hooks`) is unrelated — it fails on `upstream/master` as well due to a `pnpm install` failure in the test environment. ## Risks - **Startup latency**: `await reapOrphanedRuns()` adds a small delay before the timer starts. In practice this is a fast DB query. The retry adds at most one extra attempt on transient failure. - **Behavior change**: Wakeups that previously coalesced into zombie runs will now create new queued runs instead. This is the correct behavior — the zombie was preventing any forward progress. - **Queued runs unaffected**: `isZombieRun` only flags `"running"` status. Queued runs pass through `filterZombieCoalesceTarget` unchanged (covered by tests). ## Model Used - OpenAI GPT-5 via a Codex-style terminal coding agent with tool use and git/gh access. Exact hosted alias is not exposed in this environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge ## Cross-references and status (maintainer) Refs #3168 Refs #4174 Refs #4697 Refs #6399 Related PRs checked: #4075, #4705, #5232, #6952 - [x] I have searched GitHub for duplicate or related PRs and linked them above --------- Co-authored-by: Devin Foley <devin@paperclip.ing> |
||
|
Jannes Stubbemann
|
4ad94d0bde |
feat(server): kubernetes execution integration for sandbox-provider plugins (stage 2/3) (#7938)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The execution subsystem runs those agents in environments (local, ssh, sandbox), and sandbox-provider plugins let an environment materialize per-run sandboxes > - Stage 1 (#5790) contributed a first-party Kubernetes sandbox-provider plugin, but the server core has no way to adopt it operationally: no per-run adapter selection, no way to force an instance onto sandboxed execution, no declarative adapter/model configuration, and the plugin must be installed by hand > - Without this, a multi-tenant or security-conscious deployment cannot guarantee that agent runs never execute on the host, and a single environment cannot serve agents with different harnesses > - This pull request adds the server + SDK integration: per-run adapterType on the lease protocol, an env-gated forced-Kubernetes execution policy with provisioning and a per-run allowlist guard, a declarative adapter registry and model list, in-cluster env passthrough for sandbox plugin workers, fail-safe auto-install of the bundled plugin, and the matching UI affordance > - The benefit is that sandbox-provider plugins become fully usable for Kubernetes execution: operators configure everything via environment variables and GitOps, while self-hosters who set none of the variables see exactly the behavior they have today ## Linked Issues or Issue Description Refs #5790 (stage 1 of 3: the Kubernetes sandbox-provider plugin package). No existing issue. Feature description: the server core lacks the integration seams to operate a sandbox-provider plugin as the mandatory execution path of an instance. This PR is stage 2 of 3 of the staged Kubernetes contribution; stage 3 will contribute the agent runtime images and their build pipeline. ## What Changed One line per piece: - `packages/plugins/sdk/protocol.ts`: optional `adapterType` on `PluginEnvironmentAcquireLeaseParams` so a provider can select the runtime image per run; existing providers simply ignore it - `server/services/environment-runtime.ts` + `environment-run-orchestrator.ts`: thread the agent's adapter type into both lease-acquiring drivers, including the heartbeat path (the two call sites have historically drifted, hence the pinned test) - `server/services/environments.ts`: `ensureKubernetesEnvironment` / `findKubernetesEnvironment`, an idempotent managed Kubernetes environment per company, identified by a metadata marker and refreshed (not recreated) on config change; `timeoutMs` rides on the config for slow cold-start leases - `server/services/execution-allowlist.ts`: pure (driver, provider, policy) -> allow/deny guard; `executionMode=kubernetes` only allows the kubernetes sandbox provider - `server/services/execution-policy-bootstrap.ts` + startup hook in `server/index.ts`: parse `PAPERCLIP_EXECUTION_MODE` / `PAPERCLIP_K8S_*`, persist `executionMode` into instance general settings, and provision the managed environment for every company; fails loud on misconfiguration - `server/services/heartbeat.ts`: when the policy forces Kubernetes, pin run selection to the managed environment (also overriding any persisted workspace environment id), refuse to fall back to local, and re-check the actually acquired environment against the allowlist as defense in depth - `server/services/adapter-registry-bootstrap.ts` + shared `AdapterRegistryEntry` type/validator: declarative `PAPERCLIP_ADAPTERS` registry (inline JSON or file) that reconciles adapter availability at startup and rides on the Kubernetes environment config - `server/services/adapter-models-env.ts` + `adapters/registry.ts`: `PAPERCLIP_ADAPTER_MODELS` lets an operator declare picker model lists the server cannot CLI-discover - `server/services/plugin-loader.ts`: pass `KUBERNETES_SERVICE_HOST/PORT(_HTTPS)` through to plugin workers that register environment drivers, so in-cluster API clients can be constructed; all other host env stays stripped - `server/app.ts`: fail-safe auto-install of the bundled kubernetes plugin at boot; no-ops when the bundle is absent and never blocks startup on error - `packages/shared` types/validators: `InstanceExecutionMode` on general settings (optional, strict schema) - `ui/lib/forced-kubernetes-environment.ts` + `AgentConfigForm`: when the policy is active, show a read-only Kubernetes environment instead of the environment picker and default new agents onto the managed environment - Tests for every new module plus the adapterType pin in `heartbeat-plugin-environment` and the managed-environment lifecycle in `environment-service` Everything is gated: with `PAPERCLIP_EXECUTION_MODE`, `PAPERCLIP_ADAPTERS`, and `PAPERCLIP_ADAPTER_MODELS` unset (and no bundled plugin present), every code path reduces to current behavior. The per-run `adapterType` is an optional SDK parameter that existing providers ignore. ## Verification - `cd server && npx tsc --noEmit`: clean (0 errors); `ui` typecheck also clean - Targeted suites all green (11 files, 90 tests): `npx vitest run server/src/__tests__/heartbeat-plugin-environment.test.ts server/src/__tests__/environment-service.test.ts server/src/__tests__/environment-runtime.test.ts server/src/__tests__/environment-run-orchestrator.test.ts server/src/__tests__/plugin-database.test.ts server/src/services/execution-policy-bootstrap.test.ts server/src/services/execution-allowlist.test.ts server/src/services/adapter-registry-bootstrap.test.ts server/src/services/adapter-registry-bootstrap.reconcile.test.ts server/src/services/adapter-models-env.test.ts packages/shared/src/validators/adapter-registry.test.ts` - `npx vitest run ui/src/components/AgentConfigForm.test.ts`: green (6 tests) - Full `npx vitest run server/src/__tests__`: 2323 passed, 1 skipped; the only failures (heartbeat-process-recovery pid-retry, workspace-runtime symbolic-ref/git tests) reproduce identically on pristine `master` in the same environment, so they are machine-environment issues unrelated to this change; `server-startup-feedback-export` needed its `services/index.js` mock extended with the new export and is green - This integration has been running in production on a hosted multi-tenant deployment, where it executes agent runs across five different harnesses through the stage 1 plugin ## Risks - Low for existing deployments: every behavior is env-gated and the defaults preserve current semantics; the auto-install block is wrapped fail-safe and skips silently when the plugin bundle is absent - `executionMode` is a new optional field on a strict zod schema; absent input normalizes exactly as before - The forced policy intentionally fails runs loudly (rather than falling back to local) when no managed Kubernetes environment exists; this only affects instances that explicitly set `PAPERCLIP_EXECUTION_MODE=kubernetes` ## Model Used Claude Opus 4.8 (claude-opus-4-8, 1M context), extended thinking, agentic tool use via Claude Code. ## UI screenshots The UI change is a new read-only "Execution" section in `AgentConfigForm`, shown only when the instance execution policy forces Kubernetes (`executionMode=kubernetes`); there is no "before" state for it (the section did not exist, and instances without the forced policy render the existing picker unchanged). Captured from the new Storybook stories added in this PR (`Product/Agent Management`): Managed Kubernetes environment present (read-only display, no local/SSH picker):  No managed environment available yet (warning notice, no silent local fallback):  ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Nicolás Rodrigues
|
f3db7b88ea |
Clear stale checkoutRunId on run finalization and add backstop sweeper (#6008)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The issue subsystem holds per-row lock columns (`checkoutRunId`, `executionRunId`, `executionAgentNameKey`, `executionLockedAt`) that gate checkout, ownership, and release > - When a heartbeat run terminates, `releaseIssueExecutionAndPromote` clears the execution-lock columns but stale checkout locks could remain attached to dead runs in edge paths > - The original fix closed the finalization, checkout, release, and sweeper paths, but PR CI exposed one more process-loss retry path where a queued retry advanced `executionRunId` while leaving `checkoutRunId` pinned to the failed run > - This pull request closes the asymmetry: terminal-run cleanup and process-loss retry recovery release dead checkout locks while preserving live execution ownership > - The benefit is permanent, automatic self-heal of stale lock columns and fewer false checkout 409s requiring board intervention > - Related upstream issue: #6007 ## Linked Issues or Issue Description Refs #6007. Duplicate/related PR search performed on 2026-06-10 with query `checkoutRunId process loss retry stale checkout lock repo:paperclipai/paperclip`. Related PRs found and reviewed for overlap: - #7727 `fix(heartbeat): atomically advance checkoutRunId on process-loss retry` - #7707 `test: cover same-agent stale checkout adoption` - #3068 `fix: clear checkoutRunId when releasing issue execution lock` ## What Changed - `server/src/services/heartbeat.ts` `releaseIssueExecutionAndPromote`: extend the per-issue update to also null `checkoutRunId` when it matches the terminating run id. WHERE clause scoped to `executionRunId = run.id OR checkoutRunId = run.id` for idempotence. - `server/src/services/heartbeat.ts` process-loss retry: when queuing the retry run, move `executionRunId` to the retry and clear the failed run's `checkoutRunId` so the dead run no longer owns checkout. - `server/src/services/issues.ts`: add `clearCheckoutRunIfTerminal` helper, symmetric to `clearExecutionRunIfTerminal`. No assignee/status precondition. Wired into `checkout`, `assertCheckoutOwner`, and `release`. Exported on the issue service. - `server/src/services/recovery/service.ts`: add `sweepStaleIssueLocks`. Scans `issues` where `checkoutRunId IS NOT NULL OR executionRunId IS NOT NULL`, joins each referenced run, and clears all lock columns on issues whose referenced runs are all terminal or missing. Emits one `issue.stale_lock_cleared` activity log row per cleared issue. - `server/src/services/heartbeat.ts`: re-export the sweeper on the heartbeat facade. - `server/src/index.ts`: invoke `sweepStaleIssueLocks` in both the startup recovery sequence and the periodic heartbeat timer chain. - Tests: route-level coverage of the new self-heal path on the next checkout attempt, service-level sweeper coverage, and heartbeat recovery assertions that terminal process-loss cleanup releases `checkoutRunId`. ## Verification ```bash pnpm --filter @paperclipai/server typecheck pnpm --filter @paperclipai/server exec vitest run \ src/__tests__/recovery-stale-issue-lock-sweep.test.ts \ src/__tests__/issue-stale-execution-lock-routes.test.ts NODE_ENV=test pnpm exec vitest run src/__tests__/heartbeat-process-recovery.test.ts -t "queues exactly one retry when the recorded local pid is dead|does not block paused-tree work when immediate continuation recovery is suppressed by the hold" NODE_ENV=test pnpm exec vitest run src/__tests__/heartbeat-process-recovery.test.ts ``` All listed local checks pass. The new and updated tests cover: - Run termination clears `checkoutRunId` when it points at the terminating run. - Process-loss retry clears the failed run's `checkoutRunId` while assigning `executionRunId` to the queued retry. - A different agent calling `POST /api/issues/:id/checkout` on an issue whose prior owner died self-heals via `clearCheckoutRunIfTerminal` and succeeds. - Sweeper clears stale lock columns for issues whose run row is terminal. - Sweeper leaves issues alone while the referenced run is still running. - Sweeper leaves issues alone when `executionRunId` is still running even if `checkoutRunId` is terminal. - Sweeper is idempotent; second pass clears nothing. Manual reproduction of the original bug shape: 1. Create an issue assigned to agent A, set `status='in_progress'`, `checkoutRunId=R1`, `executionRunId=null`, where `heartbeat_runs.status = 'failed'` for `R1`. 2. Reassign to agent B and move to `status='todo'`. 3. Before this PR: agent B `POST /checkout` returns `409 Issue checkout conflict` indefinitely. After this PR: succeeds, lock columns rewritten to agent B's current run id. ## Risks - Low. All clears are scoped by run id, so they only fire when the lock column unambiguously points at the terminating or terminal run. No schema change. No migration. No API surface change. - Behavioral shift: an issue that previously stayed `in_progress` with a dead `checkoutRunId` after run termination now self-heals. Downstream code that reads stale `checkoutRunId` as a proxy for recent run history should already be reading `executionRunId` or the `heartbeat_runs` table. - Sweeper cost: one indexed scan per recovery tick over rows where `checkoutRunId IS NOT NULL OR executionRunId IS NOT NULL` plus a single batched `heartbeatRuns` lookup per candidate. Negligible at expected cardinality; further bounded by the existing recovery cadence. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. This is a bug fix, not a feature. No roadmap overlap. ## Model Used - Claude (Anthropic), model ID `claude-opus-4-7`, extended-thinking off, tool use enabled. - OpenAI Codex, GPT-5-based coding agent, tool use enabled, used for the follow-up process-loss retry fix and PR body update. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> Co-authored-by: Dotta <bippadotta@protonmail.com> |
||
|
Dotta
|
ad6effa65c |
[codex] Improve runtime and import reliability (#6549)
## Thinking Path > - Paperclip coordinates autonomous company work through local and hosted runtime surfaces. > - Local embedded Postgres and tenant import/export paths are foundational reliability pieces. > - A runtime failure in either path can stop agents or imports before useful work begins. > - The branch included remaining fixes for embedded native library bootstrap and async tenant import handling. > - This pull request groups those runtime/import reliability changes into one standalone PR. > - The benefit is a more robust local runtime and safer cloud tenant import behavior. ## What Changed - Prepared embedded Postgres native runtime before startup in CLI/server/test entrypoints. - Added embedded Postgres native bootstrap coverage. - Added async tenant import job handling and deferred validation coverage. - Kept the runtime/import changes based directly on current `origin/master` after related upstream PRs had already merged. ## Verification - `pnpm --filter @paperclipai/plugin-sdk build` - `NODE_ENV=test pnpm exec vitest run packages/db/src/embedded-postgres-native.test.ts server/src/__tests__/company-portability-routes.test.ts` ## Risks - Medium-low: this touches startup/import paths, but the branch is small and covered by targeted tests. - The embedded Postgres change depends on platform-specific native-library behavior, so CI and follow-up checks should still verify supported runners. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 Codex via `codex_local`, tool-enabled coding session; exact context window not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
e43b392a79 |
[codex] Add local Cloud Upstream sync (#6548)
## Thinking Path > - Paperclip is the control plane for AI-agent companies. > - Operators need a path to move local company state toward Paperclip Cloud without losing local-first control. > - The Cloud Upstream flow needs API, persistence, CLI, and board UI surfaces that agree on the same manifest/run model. > - The existing branch had the feature work plus UX and error-handling follow-ups. > - This pull request packages the remaining Cloud Upstream sync work into one standalone branch. > - The benefit is an inspectable local-to-cloud sync workflow with preview, conflicts, activation, and captured UX review states. ## What Changed - Added Cloud Upstream shared types, server routes/services, and persisted run schema/migration. - Added Paperclip Cloud CLI sync helpers and local connection storage. - Added the Cloud Upstream board UI, settings entry points, query keys, and UX lab page. - Added preview/activation checklist behavior, redirect handling, manifest-only preview support, friendly errors, in-flight hints, and entity count summaries. ## Verification - `pnpm --filter @paperclipai/plugin-sdk build` - `NODE_ENV=test pnpm exec vitest run cli/src/__tests__/cloud.test.ts server/src/__tests__/instance-settings-routes.test.ts server/src/__tests__/instance-settings-service.test.ts ui/src/pages/CloudUpstream.test.tsx ui/src/components/CompanySettingsSidebar.test.tsx` - `NODE_ENV=test pnpm exec vitest run server/src/__tests__/cloud-upstreams.test.ts` Worktree setup note: the isolated worktree install skipped native sqlite build scripts, so I copied the already-built local sqlite binding from the main checkout before running `server/src/__tests__/cloud-upstreams.test.ts`. The test then passed. ## Risks - Medium: this adds a database migration and a broad feature path across CLI/server/UI. - Merge order: this is the only PR in this split with a DB migration; merge it before any future Cloud Upstream migration follow-up. - Mitigation: the PR is based directly on current `origin/master`, has targeted route/service/UI tests, and keeps the feature behind existing experimental Cloud Sync settings. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI GPT-5 Codex via `codex_local`, tool-enabled coding session; exact context window not exposed by this runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, screenshot artifacts are intentionally omitted per reviewer request - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
38c185fb8b |
[codex] Add agent permissions and controls plan (#6386)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies by keeping task ownership, approvals, and operator control inside one control plane. > - Agent permissions and plugin-hosted company settings sit on the boundary between autonomy and governance. > - V1 needs scoped task assignment rules, plugin extension points, and clearer company access surfaces without weakening company boundaries. > - The branch builds the core authorization service, plugin SDK/host APIs, and UI simplifications needed to support those controls. > - Paperclip EE plugin surfaces were intentionally moved out of this core PR per review direction, so this PR now carries only the public core/plugin infrastructure work. > - The latest updates preserve the PAP-9937 branch changes that belong in this PR, remove the `design/` artifacts, and exclude the experimental `plugin-briefs` package. > - Greptile feedback was applied through the authorization/audit paths and the final cleanup commit was re-reviewed at 5/5 with no unresolved Greptile threads. > - The benefit is safer assignment control with extension hooks for richer permission products while preserving simple defaults for normal operators. ## What Changed - Added scoped task-assignment authorization decisions and routed issue/agent assignment mutations through the authorization service. - Added plugin SDK and host APIs for company settings slots, authorization policy/grant management, assignment previews, and bridge invocation scope propagation. - Simplified core company access UI and moved advanced controls behind plugin-provided settings surfaces. - Added retry-now affordances for blocked issue next-step notices. - Added protected-assignment enforcement for persisted agent/project/issue policies, including explicit-grant fallback behavior. - Added incremental principal-access compatibility backfill for active agent memberships and role-default human permission grants. - Added the Markdown code block wrap action fix from the latest branch changes. - Removed `design/` artifacts from the PR and removed `packages/plugins/plugin-briefs` from the final diff. - Addressed Greptile feedback for plugin actor sanitization, legacy membership handling, audit pagination, unknown grant-scope metadata, and startup test mocks. ## Verification - `pnpm exec vitest run server/src/__tests__/access-service.test.ts server/src/__tests__/company-portability.test.ts` -> 2 files passed, 54 tests passed. - `pnpm exec vitest run server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/access-service.test.ts server/src/__tests__/company-portability.test.ts` -> 3 files passed, 62 tests passed. - `pnpm exec vitest run server/src/__tests__/authorization-service.test.ts server/src/__tests__/plugin-access-authorization-host-services.test.ts server/src/__tests__/server-startup-feedback-export.test.ts` -> 3 files passed, 28 tests passed. - `pnpm --filter @paperclipai/server typecheck` -> passed. - `git diff --check` -> passed. - `node ./scripts/check-docker-deps-stage.mjs` -> passed. - `CI=true pnpm install --frozen-lockfile --ignore-scripts` -> passed with no lockfile update. - `pnpm exec vitest run ui/src/components/MarkdownBody.interaction.test.tsx` -> 1 test passed. - `git ls-files design packages/plugins/plugin-briefs | wc -l` -> 0. - GitHub CI on `40cd83b53` -> all checks passed, merge state `CLEAN`. - Greptile on `40cd83b53` -> 5/5, 102 files reviewed, 0 comments/annotations added, 0 unresolved review threads. - Confirmed the PR diff contains no `design/`, `packages/plugins/plugin-briefs`, `pnpm-lock.yaml`, or `.github/workflows` changes. ## Risks - Medium: task assignment authorization paths are behaviorally stricter for protected/private policy data, so existing plugin-authored policies may block assignment until explicit grants or approval flows are configured. - Medium: plugin-host authorization APIs expand the surface area available to trusted plugins and need careful review for company scoping. - Low: startup now performs a principal-access compatibility backfill, but the migration and runtime backfill use conflict-tolerant inserts. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled workflow with shell, git, and GitHub CLI access. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
c91a062326 |
[codex] Runtime control-plane fixes (#6380)
## Thinking Path > - Paperclip orchestrates AI agents through a server-side control plane > - That control plane depends on reliable issue state transitions, plugin lifecycle behavior, import limits, and startup/shutdown handling > - Several small runtime fixes had accumulated on the working branch and were mixed with larger feature work > - Keeping them separate makes the correctness fixes reviewable and mergeable without waiting for cloud-sync UI work > - This pull request groups the server/runtime control-plane fixes into one standalone branch > - The benefit is a tighter, safer runtime baseline for retries, imports, plugin migrations, feedback flushing, and trusted cloud import handling ## What Changed - Fixed updated issue list pagination sorting and scheduled retry comment handling. - Re-applied pending plugin migrations during hot reload and fixed plugin-schema worktree seed restore. - Hardened public tenant DB startup, portable import body limits, trusted cloud import errors, and trusted cloud tenant import mutation access. - Expired stale request confirmations after user comments. - Added feedback export shutdown hardening so database-unavailable flush loops stop cleanly. - Guarded plugin worker `error` event emission when no listener is registered. ## Verification - `pnpm install --frozen-lockfile --ignore-scripts` - `pnpm --filter @paperclipai/plugin-sdk build` - `npm run install --prefix node_modules/.pnpm/sqlite3@5.1.7/node_modules/sqlite3` - `pnpm exec vitest run server/src/__tests__/issues-service.test.ts server/src/__tests__/plugin-lifecycle-restart.test.ts server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/issue-comment-reopen-routes.test.ts server/src/__tests__/issue-thread-interactions-service.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts server/src/__tests__/body-limits.test.ts server/src/__tests__/feedback-flush-controller.test.ts server/src/__tests__/error-handler.test.ts server/src/__tests__/board-mutation-guard.test.ts packages/db/src/backup-lib.test.ts` initially exposed local setup issues and two 5s test timeouts. - Rerun after local prereq build: `pnpm exec vitest run --testTimeout 15000 server/src/__tests__/issue-comment-reopen-routes.test.ts server/src/__tests__/issue-thread-interaction-routes.test.ts server/src/__tests__/feedback-flush-controller.test.ts server/src/__tests__/server-startup-feedback-export.test.ts` passed. - Some embedded Postgres-backed tests skipped on this host because local Postgres init was unavailable. ## Risks - Runtime-touching branch: startup/shutdown and issue interaction behavior should be reviewed carefully. - The feedback export change disables repeated flush attempts only for database connection-refused failures; other upload failures still log normally. - The plugin worker error guard avoids process crashes from unhandled EventEmitter errors but may hide errors from code paths that expected an emitted listener. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent with local shell/git/tool use. Exact hosted model ID and context-window size are not exposed by the local Paperclip adapter runtime. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
454edfe81e |
Add recovery handoff system notices (#5289)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - Agent runs can end productively while the source issue still lacks a durable final disposition. > - That leaves the control plane unsure whether to resume, escalate, or close the work. > - Issue comments also need a presentation contract so system-authored recovery notices can render as first-class thread messages without overloading normal comments. > - This pull request adds successful-run handoff recovery, comment presentation metadata, and system notice rendering. > - The benefit is stricter task liveness with clearer operator-facing recovery state. ## What Changed - Added successful-run handoff decisions, wake payloads, escalation behavior, and recovery tests. - Added issue comment presentation metadata with migration `0078_white_darwin.sql` and shared/server/company portability support. - Rendered recovery/system notices in issue chat with dedicated UI components, fixtures, tests, and storybook/lab coverage. - Included the current recovery model-profile hint patch so automatic recovery follow-ups use the cheap profile. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run server/src/services/recovery/successful-run-handoff.test.ts ui/src/components/SystemNotice.test.tsx ui/src/lib/system-notice-comment.test.ts ui/src/components/IssueChatThreadSystemNotice.test.tsx` ## Risks - Migration-bearing PR: merge this before any other branch that might later add a migration. - The branch touches both recovery services and issue-thread rendering, so review should pay attention to recovery wake idempotency and comment metadata compatibility. ## Model Used - OpenAI GPT-5 Codex via Paperclip `codex_local` adapter, with shell/git/GitHub CLI tool use. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
367d4cab72 |
Fix SSH callback URL selection for LAN and private networks (#4799)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Agents can run on remote hosts via SSH environments > - When a remote agent needs to call back to the Paperclip API, it needs a reachable URL > - But the runtime API URL candidate builder did not account for private network topologies where the server is only reachable via LAN or VPN addresses > - Agents on SSH hosts were failing to connect because the callback URL pointed to localhost or an unreachable address > - This PR fixes callback URL selection to honor `PAPERCLIP_API_URL`, prefer LAN-reachable candidates, filter unreachable link-local addresses, and include interface hosts in onboarding invite URLs > - The benefit is SSH-based agents can reliably reach the Paperclip API on private networks without manual URL configuration ## What Changed - `runtime-api.ts`: Added `PAPERCLIP_API_URL` as a first-priority candidate in `buildRuntimeApiCandidateUrls`; extracted `collectReachableInterfaceHosts` to enumerate non-loopback, non-link-local network interface IPs with IPv4 preference - `server/src/index.ts`: Export `PAPERCLIP_API_URL` from the server environment so it is available to callback candidate resolution - `server/src/routes/access.ts`: Include LAN interface hosts in onboarding invite connection candidates - `server/src/config.ts`: Attempted auto-allowing LAN interface hosts, then reverted to the per-instance allowlist approach (both commits included for history clarity) ## Verification - `pnpm test` — all existing and new tests pass, including new tests for LAN candidate ordering and link-local filtering - `pnpm typecheck` — clean - Manual: start a Paperclip server on a machine with a LAN IP, create an SSH environment pointing to another host on the same LAN, verify the agent's callback URL uses the LAN IP rather than localhost ## Risks - Low-medium. The candidate list now includes more addresses (all non-loopback LAN interfaces). These are candidates for the agent to try, not an allowlist — the server's allowed hostnames still gate which origins are accepted. Ordering change (LAN preferred over loopback) could affect existing setups where localhost was intentionally preferred. ## Model Used Codex GPT 5.4 high via Paperclip. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
1991ec9d6f |
[codex] Split backend control-plane QoL slice (#4700)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, so backend task ownership, recovery, review visibility, and company-scoped limits need to stay enforceable without UI-only coupling. > - Closed PR #4692 bundled those backend changes with UI workflow, docs, skills, workflow, and lockfile churn. > - PAP-2694 asks for a clean backend/control-plane slice from that closed branch. > - This branch starts from current `master` and mines only the `cli`, `packages/db`, `packages/shared`, and `server` contracts/tests needed for the backend behavior. > - It explicitly excludes UI workflow/performance work, `.github/workflows/pr.yml`, `pnpm-lock.yaml`, docs, skills, package-script, adapter UI build-config, and perf fixture script changes; the only UI files are fixture/test updates required by the tightened shared `Company` contract. > - The benefit is a smaller reviewable PR that preserves the control-plane fixes while staying under Greptile s 100-file review limit. ## What Changed - Added company-scoped attachment-size limits through DB schema/migrations, shared company portability contracts, CLI import/export coverage, and server attachment upload enforcement. - Added productivity review service/API behavior for no-comment streak, long-active, and high-churn review issues, including request-depth clamping and issue summary exposure. - Hardened issue ownership and recovery/control-plane paths: peer-agent mutation denial, issue tree pause/resume behavior, stranded recovery origins, and related activity/test coverage. - Preserved related backend contract updates for routine timestamp variables and managed agent instruction bundles because they live in shared/server contracts from the source branch. - Addressed Greptile feedback by making `Company.attachmentMaxBytes` non-optional, simplifying review request-depth clamping, fixing the migration final newline, and enforcing the process-level attachment cap as the final ceiling for uploads. - Added minimal company fixtures needed for repo-wide typecheck/build and kept the PR to 66 changed files with forbidden/non-slice paths excluded. ## Verification - `pnpm install --frozen-lockfile` - `git diff --check origin/master..HEAD` - `git diff --name-only origin/master..HEAD | wc -l` -> 66 files - `git diff --name-only origin/master..HEAD -- .github/workflows/pr.yml pnpm-lock.yaml package.json doc skills .agents scripts packages/adapters` -> no output - `pnpm exec vitest run --config vitest.config.ts packages/shared/src/validators/issue.test.ts packages/shared/src/routine-variables.test.ts packages/shared/src/adapter-types.test.ts cli/src/__tests__/company-import-export-e2e.test.ts cli/src/__tests__/company.test.ts server/src/__tests__/productivity-review-service.test.ts server/src/__tests__/issue-tree-control-service.test.ts server/src/__tests__/issue-tree-control-routes.test.ts server/src/__tests__/issue-agent-mutation-ownership-routes.test.ts server/src/__tests__/issue-attachment-routes.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/issues-service.test.ts` -> 12 files, 147 tests passed - `pnpm exec vitest run --config vitest.config.ts cli/src/__tests__/company-delete.test.ts cli/src/__tests__/company-import-export-e2e.test.ts server/src/__tests__/productivity-review-service.test.ts` -> 3 files, 18 tests passed - `pnpm exec vitest run --config vitest.config.ts server/src/__tests__/issue-attachment-routes.test.ts` -> 1 file, 6 tests passed - `pnpm --filter @paperclipai/db typecheck && pnpm --filter @paperclipai/shared typecheck && pnpm --filter @paperclipai/server typecheck && pnpm --filter paperclipai typecheck` - `pnpm --filter @paperclipai/server typecheck` - `pnpm --filter @paperclipai/ui typecheck && pnpm --filter @paperclipai/ui build` ## Risks - Includes migrations `0073_shiny_salo.sql` and `0074_striped_genesis.sql`; merge ordering matters if another PR adds migrations first. - This is intentionally backend-only apart from fixture/test updates forced by shared type correctness; UI affordances from PR #4692 are not present here and should land in separate UI slices. - The worktree install emitted plugin SDK bin-link warnings for unbuilt plugin packages, but the targeted tests and package typechecks completed successfully. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected; check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5 coding agent, tool-enabled terminal/GitHub workflow. Exact runtime context window was not exposed by the harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
68c37660f0 |
Dispatch assigned todo work during recovery sweeps (#4614)
## Thinking Path > - Paperclip orchestrates AI agents for autonomous companies. > - Agent assignments must reliably turn into heartbeat work without board operators manually nudging stuck tasks. > - The stranded-assignment recovery sweep already handles failed or lost runs. > - But assigned `todo` issues with no prior run could sit idle because there was nothing to retry or recover. > - This pull request dispatches those never-started assigned todos as normal assignment wakes. > - The benefit is that recovery fixes missed initial dispatches without creating unnecessary recovery issues. ## What Changed - Added an initial assigned-todo dispatch path to the recovery service when an assigned `todo` issue has no heartbeat run yet. - Reused invocation budget hard-stop checks before dispatching or requeueing recovery work. - Counted `assignmentDispatched` in startup/scheduled recovery logs. - Added heartbeat recovery regressions for first dispatch, duplicate queued wake prevention, budget-blocked skips, and paused-agent skips. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-process-recovery.test.ts` ## Risks - Low to medium risk: this changes liveness recovery behavior for assigned `todo` issues, but it stays on the existing assignment wake path and skips paused or budget-blocked agents. - No migrations. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, tool-enabled local repository and shell access, Paperclip heartbeat context. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
b2496c8067 |
fix(auth): trust allowed hostname port variants on detected listen port (#4554)
## Thinking Path > - Paperclip is the control plane for autonomous AI companies, so authenticated board access has to be predictable across local and worktree deployments. > - This change sits in the authenticated-mode server startup and Better Auth origin-trust wiring. > - The original auth branch fixed one real gap by adding port-qualified trusted origins for allowed hostnames on non-default ports. > - Review of that branch found a second-order bug: trusted origins were still derived from the configured port before startup detected the actual listen port. > - In isolated worktrees, that meant a common `3100 -> 3101` port shift could still leave Better Auth trusting the stale origin. > - This pull request keeps the original allowed-hostname port-variant fix, then moves trust derivation onto the resolved listen port and adds regression coverage around startup wiring. > - The benefit is that authenticated sessions keep working on allowed private hostnames even when Paperclip has to auto-shift to a different local port. ## What Changed - Added `:port` trusted-origin variants for authenticated-mode `allowedHostnames` when Paperclip runs on non-default ports. - Changed authenticated startup so `listenPort` is detected before Better Auth initialization, and explicit auth base URLs are rewritten before auth startup. - Updated `deriveAuthTrustedOrigins()` to accept the resolved listen port so Better Auth trusts the actual browser origin instead of the stale configured port. - Added focused regression coverage in `server/src/__tests__/better-auth.test.ts` and `server/src/__tests__/server-startup-feedback-export.test.ts`. ## Verification - `pnpm exec vitest run server/src/__tests__/better-auth.test.ts server/src/__tests__/server-startup-feedback-export.test.ts` - Reviewer re-check: reviewed commits `380f5b9f` and `092bb34c` after the follow-up fix landed and found no remaining issues. ## Risks - Low risk: this only affects authenticated-mode origin derivation and startup ordering around detected listen ports. - Main behavioral shift: startup no longer mutates `config.port` to the selected port; it now carries `requestedListenPort` separately and uses `listenPort` where runtime behavior needs the resolved value. - If another path was implicitly relying on `config.port` being overwritten during startup, that path would need follow-up, though the current startup/test coverage did not reveal one. > I checked `ROADMAP.md` and did not find an overlapping planned core work item for this auth trusted-origin port handling fix. ## Model Used - OpenAI Codex via Paperclip `codex_local` agents for implementation and review. Exact backend model ID/context window were not surfaced in this run context; work was performed through the Codex local adapter with tool use, code execution, and review passes. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Devin Foley
|
08af830430 |
Tighten publicBaseUrl port rewriting (#4553)
## Thinking Path > - Paperclip is a control plane for autonomous agent companies, so its local and authenticated deployment behavior has to stay predictable under port rebinding and worktree isolation. > - This change sits in the server/worktree configuration path that derives runtime URLs and auth origins from `auth.publicBaseUrl`. > - The original hostname-port rewrite change fixed one real gap for private/tailnet host:port worktree setups, but it widened the rewrite rule too far. > - Rewriting every explicit `auth.publicBaseUrl` can corrupt public or reverse-proxy URLs by turning a stable origin like `https://paperclip.example` into a local listen-port URL. > - Paperclip's auth and trusted-origin handling depend on that URL staying semantically correct, so this had to be narrowed before merge. > - This pull request tightens the rewrite rule to explicit-port URLs only and adds regression coverage across the CLI helper, worktree config persistence, and server startup path. > - The benefit is that private host:port worktree flows still work, while public/default-port URLs remain stable and safe. ## What Changed - Tightened `rewriteLocalUrlPort` in `cli/src/commands/worktree-lib.ts`, `server/src/worktree-config.ts`, and `server/src/index.ts` so it only rewrites URLs that already include an explicit port. - Removed the old loopback-only hostname gate from the CLI/worktree helpers and replaced it with the more precise `parsed.port` guard. - Updated CLI helper coverage to assert that explicit-port non-loopback URLs still rewrite while no-port public URLs stay unchanged. - Expanded `server/src/__tests__/worktree-config.test.ts` to cover explicit-port rewrite and no-port stability for both persisted worktree config and in-memory runtime port selection. - Added startup-path coverage in `server/src/__tests__/server-startup-feedback-export.test.ts` for `detect-port` rebinding with both explicit-port and no-port `auth.publicBaseUrl` values. ## Verification - `pnpm --filter @paperclipai/plugin-sdk build` - `npx vitest run server/src/__tests__/server-startup-feedback-export.test.ts` - `npx vitest run cli/src/__tests__/worktree.test.ts server/src/__tests__/worktree-config.test.ts` - All of the above were run locally in this issue worktree and passed. ## Risks - Low risk. The behavior change is deliberately narrower than the reviewed broad-host rewrite and is guarded by regression coverage for both the explicit-port and no-port cases. - The main remaining risk is behavioral only if another code path starts depending on port rewriting for URLs that never declared a port, which would be a separate bug. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex local agent using `gpt-5.4` with high reasoning effort, tool use, shell execution, and file editing. - Anthropic Claude local agent using `claude-opus-4-6` for follow-up code review approval on the implementation issue. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
5a0c1979cf | [codex] Add runtime lifecycle recovery and live issue visibility (#4419) | ||
|
Devin Foley
|
70679a3321 |
Add sandbox environment support (#4415)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The environment/runtime layer decides where agent work executes and how the control plane reaches those runtimes. > - Today Paperclip can run locally and over SSH, but sandboxed execution needs a first-class environment model instead of one-off adapter behavior. > - We also want sandbox providers to be pluggable so the core does not hardcode every provider implementation. > - This branch adds the Sandbox environment path, the provider contract, and a deterministic fake provider plugin. > - That required synchronized changes across shared contracts, plugin SDK surfaces, server runtime orchestration, and the UI environment/workspace flows. > - The result is that sandbox execution becomes a core control-plane capability while keeping provider implementations extensible and testable. ## What Changed - Added sandbox runtime support to the environment execution path, including runtime URL discovery, sandbox execution targeting, orchestration, and heartbeat integration. - Added plugin-provider support for sandbox environments so providers can be supplied via plugins instead of hardcoded server logic. - Added the fake sandbox provider plugin with deterministic behavior suitable for local and automated testing. - Updated shared types, validators, plugin protocol definitions, and SDK helpers to carry sandbox provider and workspace-runtime contracts across package boundaries. - Updated server routes and services so companies can create sandbox environments, select them for work, and execute work through the sandbox runtime path. - Updated the UI environment and workspace surfaces to expose sandbox environment configuration and selection. - Added test coverage for sandbox runtime behavior, provider seams, environment route guards, orchestration, and the fake provider plugin. ## Verification - Ran locally before the final fixture-only scrub: - `pnpm -r typecheck` - `pnpm test:run` - `pnpm build` - Ran locally after the final scrub amend: - `pnpm vitest run server/src/__tests__/runtime-api.test.ts` - Reviewer spot checks: - create a sandbox environment backed by the fake provider plugin - run work through that environment - confirm sandbox provider execution does not inherit host secrets implicitly ## Risks - This touches shared contracts, plugin SDK plumbing, server runtime orchestration, and UI environment/workspace flows, so regressions would likely show up as cross-layer mismatches rather than isolated type errors. - Runtime URL discovery and sandbox callback selection are sensitive to host/bind configuration; if that logic is wrong, sandbox-backed callbacks may fail even when execution succeeds. - The fake provider plugin is intentionally deterministic and test-oriented; future providers may expose capability gaps that this branch does not yet cover. ## Model Used - OpenAI Codex coding agent on a GPT-5-class backend in the Paperclip/Codex harness. Exact backend model ID is not exposed in-session. Tool-assisted workflow with shell execution, file editing, git history inspection, and local test execution. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
09d0678840 |
[codex] Harden heartbeat scheduling and runtime controls (#4223)
## Thinking Path > - Paperclip orchestrates AI agents through issue checkout, heartbeat runs, routines, and auditable control-plane state > - The runtime path has to recover from lost local processes, transient adapter failures, blocked dependencies, and routine coalescing without stranding work > - The existing branch carried several reliability fixes across heartbeat scheduling, issue runtime controls, routine dispatch, and operator-facing run state > - These changes belong together because they share backend contracts, migrations, and runtime status semantics > - This pull request groups the control-plane/runtime slice so it can merge independently from board UI polish and adapter sandbox work > - The benefit is safer heartbeat recovery, clearer runtime controls, and more predictable recurring execution behavior ## What Changed - Adds bounded heartbeat retry scheduling, scheduled retry state, and Codex transient failure recovery handling. - Tightens heartbeat process recovery, blocker wake behavior, issue comment wake handling, routine dispatch coalescing, and activity/dashboard bounds. - Adds runtime-control MCP tools and Paperclip skill docs for issue workspace runtime management. - Adds migrations `0061_lively_thor_girl.sql` and `0062_routine_run_dispatch_fingerprint.sql`. - Surfaces retry state in run ledger/agent UI and keeps related shared types synchronized. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-retry-scheduling.test.ts server/src/__tests__/heartbeat-process-recovery.test.ts server/src/__tests__/routines-service.test.ts` - `pnpm exec vitest run src/tools.test.ts` from `packages/mcp-server` ## Risks - Medium risk: this touches heartbeat recovery and routine dispatch, which are central execution paths. - Migration order matters if split branches land out of order: merge this PR before branches that assume the new runtime/routine fields. - Runtime retry behavior should be watched in CI and in local operator smoke tests because it changes how transient failures are resumed. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent runtime, shell/git tool use enabled. Exact hosted model build and context window are not exposed in this Paperclip heartbeat environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
1954eb3048 |
[codex] Detect issue graph liveness deadlocks (#4209)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The heartbeat harness is responsible for waking agents, reconciling issue state, and keeping execution moving. > - Some dependency graphs can become live-locks when a blocked issue depends on an unassigned, cancelled, or otherwise uninvokable issue. > - Review and approval stages can also stall when the recorded participant can no longer be resolved. > - This pull request adds issue graph liveness classification plus heartbeat reconciliation that creates durable escalation work for those cases. > - The benefit is that harness-level deadlocks become visible, assigned, logged, and recoverable instead of silently leaving task sequences blocked. ## What Changed - Added an issue graph liveness classifier for blocked dependency and invalid review participant states. - Added heartbeat reconciliation that creates one stable escalation issue per liveness incident, links it as a blocker, comments on the affected issue, wakes the recommended owner, and logs activity. - Wired startup and periodic server reconciliation for issue graph liveness incidents. - Added focused tests for classifier behavior, heartbeat escalation creation/deduplication, and queued dependency wake promotion. - Fixed queued issue wakes so a coalesced wake re-runs queue selection, allowing dependency-unblocked work to start immediately. ## Verification - `pnpm exec vitest run server/src/__tests__/heartbeat-dependency-scheduling.test.ts server/src/__tests__/issue-liveness.test.ts server/src/__tests__/heartbeat-issue-liveness-escalation.test.ts` - Passed locally: `server/src/__tests__/issue-liveness.test.ts` (5 tests) - Skipped locally: embedded Postgres suites because optional package `@embedded-postgres/darwin-x64` is not installed on this host - `pnpm --filter @paperclipai/server typecheck` - `git diff --check` - Greptile review loop: ran 3 times as requested; the final Greptile-reviewed head `0a864eab` had 0 comments and all Greptile threads were resolved. Later commits are CI/test-stability fixes after the requested max Greptile pass count. - GitHub PR checks on head `87493ed4`: `policy`, `verify`, `e2e`, and `security/snyk (cryppadotta)` all passed. ## Risks - Moderate operational risk: the reconciler creates escalation issues automatically, so incorrect classification could create noise. Stable incident keys and deduplication limit repeated escalation. - Low schema risk: this uses existing issue, relation, comment, wake, and activity log tables with no migration. - No UI screenshots included because this change is server-side harness behavior only. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5-based coding agent. Exact runtime model ID and context window were not exposed in this session. Used tool execution for git, tests, typecheck, Greptile review handling, and GitHub CLI operations. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge |
||
|
Dotta
|
9c6f551595 |
[codex] Add plugin orchestration host APIs (#4114)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies. > - The plugin system is the extension path for optional capabilities that should not require core product changes for every integration. > - Plugins need scoped host APIs for issue orchestration, documents, wakeups, summaries, activity attribution, and isolated database state. > - Without those host APIs, richer plugins either cannot coordinate Paperclip work safely or need privileged core-side special cases. > - This pull request adds the plugin orchestration host surface, scoped route dispatch, a database namespace layer, and a smoke plugin that exercises the contract. > - The benefit is a broader plugin API that remains company-scoped, auditable, and covered by tests. ## What Changed - Added plugin orchestration host APIs for issue creation, document access, wakeups, summaries, plugin-origin activity, and scoped API route dispatch. - Added plugin database namespace tables, schema exports, migration checks, and idempotent replay coverage under migration `0059_plugin_database_namespaces`. - Added shared plugin route/API types and validators used by server and SDK boundaries. - Expanded plugin SDK types, protocol helpers, worker RPC host behavior, and testing utilities for orchestration flows. - Added the `plugin-orchestration-smoke-example` package to exercise scoped routes, restricted database namespaces, issue orchestration, documents, wakeups, summaries, and UI status surfaces. - Kept the new orchestration smoke fixture out of the root pnpm workspace importer so this PR preserves the repository policy of not committing `pnpm-lock.yaml`. - Updated plugin docs and database docs for the new orchestration and database namespace surfaces. - Rebased the branch onto `public-gh/master`, resolved conflicts, and removed `pnpm-lock.yaml` from the final PR diff. ## Verification - `pnpm install --frozen-lockfile` - `pnpm --filter @paperclipai/db typecheck` - `pnpm exec vitest run packages/db/src/client.test.ts` - `pnpm exec vitest run server/src/__tests__/plugin-database.test.ts server/src/__tests__/plugin-orchestration-apis.test.ts server/src/__tests__/plugin-routes-authz.test.ts server/src/__tests__/plugin-scoped-api-routes.test.ts server/src/__tests__/plugin-sdk-orchestration-contract.test.ts` - From `packages/plugins/examples/plugin-orchestration-smoke-example`: `pnpm exec vitest run --config ./vitest.config.ts` - `pnpm --dir packages/plugins/examples/plugin-orchestration-smoke-example run typecheck` - `pnpm --filter @paperclipai/server typecheck` - PR CI on latest head `293fc67c`: `policy`, `verify`, `e2e`, and `security/snyk` all passed. ## Risks - Medium risk: this expands plugin host authority, so route auth, company scoping, and plugin-origin activity attribution need careful review. - Medium risk: database namespace migration behavior must remain idempotent for environments that may have seen earlier branch versions. - Medium risk: the orchestration smoke fixture is intentionally excluded from the root workspace importer to avoid a `pnpm-lock.yaml` PR diff; direct fixture verification remains listed above. - Low operational risk from the PR setup itself: the branch is rebased onto current `master`, the migration is ordered after upstream `0057`/`0058`, and `pnpm-lock.yaml` is not in the final diff. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. Roadmap checked: this work aligns with the completed Plugin system milestone and extends the plugin surface rather than duplicating an unrelated planned core feature. ## Model Used - OpenAI Codex, GPT-5-based coding agent in a tool-enabled CLI environment. Exact hosted model build and context-window size are not exposed by the runtime; reasoning/tool use were enabled for repository inspection, editing, testing, git operations, and PR creation. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots (N/A: no core UI screen change; example plugin UI contract is covered by tests) - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
e89d3f7e11 |
[codex] Add backup endpoint and dev runtime hardening (#4087)
## Thinking Path > - Paperclip is a local-first control plane for AI-agent companies. > - Operators need predictable local dev behavior, recoverable instance data, and scripts that do not churn the running app. > - Several accumulated changes improve backup streaming, dev-server health, static UI caching/logging, diagnostic-file ignores, and instance isolation. > - These are operational improvements that can land independently from product UI work. > - This pull request groups the dev-infra and backup changes from the split branch into one standalone branch. > - The benefit is safer local operation, easier manual backups, less noisy dev output, and less cross-instance auth leakage. ## What Changed - Added a manual instance database backup endpoint and route tests. - Streamed backup/restore handling to avoid materializing large payloads at once. - Reduced dev static UI log/cache churn and ignored Node diagnostic report captures. - Added guarded dev auto-restart health polling coverage. - Preserved worktree config during provisioning and scoped auth cookies by instance. - Added a Discord daily digest helper script and environment documentation. - Hardened adapter-route and startup feedback export tests around the changed infrastructure. ## Verification - `pnpm install --frozen-lockfile` - `pnpm exec vitest run packages/db/src/backup-lib.test.ts server/src/__tests__/instance-database-backups-routes.test.ts server/src/__tests__/server-startup-feedback-export.test.ts server/src/__tests__/adapter-routes.test.ts server/src/__tests__/dev-runner-paths.test.ts server/src/__tests__/health-dev-server-token.test.ts server/src/__tests__/http-log-policy.test.ts server/src/__tests__/vite-html-renderer.test.ts server/src/__tests__/workspace-runtime.test.ts server/src/__tests__/better-auth.test.ts` - Split integration check: merged after the runtime/governance branch and before UI branches with no merge conflicts. - Confirmed this branch does not include `pnpm-lock.yaml`. ## Risks - Medium risk: touches server startup, backup streaming, auth cookie naming, dev health checks, and worktree provisioning. - Backup endpoint behavior depends on existing board/admin access controls and database backup helpers. - No database migrations are included. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex, GPT-5.4 tool-enabled coding model, agentic code-editing/runtime with local shell and GitHub CLI access; exact context window and reasoning mode are not exposed by the Paperclip harness. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Chris Farhood
|
b816809a1e |
fix(server): respect externally set PAPERCLIP_API_URL env var (#3472)
## Thinking Path > - Paperclip server starts up and sets internal `PAPERCLIP_API_URL` for downstream services and adapters > - The server startup code was unconditionally overwriting `PAPERCLIP_API_URL` with `http://localhost:3100` (or equivalent based on `config.host`) > - In Kubernetes deployments, `PAPERCLIP_API_URL` is set via a ConfigMap to the externally accessible load balancer URL (e.g. `https://paperclip.example.com`) > - Because the env var was unconditionally set after loading the ConfigMap value, the ConfigMap-provided URL was ignored and replaced with the internal localhost address > - This caused downstream services (adapter env building) to use the wrong URL, breaking external access > - This pull request makes the assignment conditional — only set if not already provided by the environment > - External deployments can now supply `PAPERCLIP_API_URL` and it will be respected; local development continues to work without setting it ## What Changed - `server/src/index.ts`: Wrapped `PAPERCLIP_API_URL` assignment in `if (!process.env.PAPERCLIP_API_URL)` guard so externally provided values are preserved - `server/src/__tests__/server-startup-feedback-export.test.ts`: Added tests verifying external `PAPERCLIP_API_URL` is respected and fallback behavior is correct - `docs/deploy/environment-variables.md`: Updated `PAPERCLIP_API_URL` description to clarify it can be externally provided and the load balancer/reverse proxy use case ## Verification - Run the existing test suite: `pnpm test:run server/src/__tests__/server-startup-feedback-export.test.ts` — all 3 tests pass - Manual verification: Set `PAPERCLIP_API_URL` to a custom value before starting the server and confirm it is not overwritten ## Risks - Low risk — purely additive conditional check; existing behavior for unset env var is unchanged ## Model Used MiniMax M2.7 — reasoning-assisted for tracing the root cause through the startup chain (`buildPaperclipEnv` → `startServer` → `config.host` → `HOST` env var) ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Pawla Abdul (Bot) <pawla@groombook.dev> Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
7f893ac4ec |
[codex] Harden execution reliability and heartbeat tooling (#3679)
## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Reliable execution depends on heartbeat routing, issue lifecycle semantics, telemetry, and a fast enough local verification loop to keep regressions visible > - The remaining commits on this branch were mostly server/runtime correctness fixes plus test and documentation follow-ups in that area > - Those changes are logically separate from the UI-focused issue-detail and workspace/navigation branches even when they touch overlapping issue APIs > - This pull request groups the execution reliability, heartbeat, telemetry, and tooling changes into one standalone branch > - The benefit is a focused review of the control-plane correctness work, including the follow-up fix that restored the implicit comment-reopen helpers after branch splitting ## What Changed - Hardened issue/heartbeat execution behavior, including self-review stage skipping, deferred mention wakes during active execution, stranded execution recovery, active-run scoping, assignee resolution, and blocked-to-todo wake resumption - Reduced noisy polling/logging overhead by trimming issue run payloads, compacting persisted run logs, silencing high-volume request logs, and capping heartbeat-run queries in dashboard/inbox surfaces - Expanded telemetry and status semantics with adapter/model fields on task completion plus clearer status guidance in docs/onboarding material - Updated test infrastructure and verification defaults with faster route-test module isolation, cheaper default `pnpm test`, e2e isolation from local state, and repo verification follow-ups - Included docs/release housekeeping from the branch and added a small follow-up commit restoring the implicit comment-reopen helpers that were dropped during branch reconstruction ## Verification - `pnpm vitest run server/src/__tests__/issue-comment-reopen-routes.test.ts server/src/__tests__/issue-telemetry-routes.test.ts` - `pnpm vitest run server/src/__tests__/http-log-policy.test.ts server/src/__tests__/heartbeat-run-log.test.ts server/src/__tests__/health.test.ts` - `server/src/__tests__/activity-service.test.ts`, `server/src/__tests__/heartbeat-comment-wake-batching.test.ts`, and `server/src/__tests__/heartbeat-process-recovery.test.ts` were attempted on this host but the embedded Postgres harness reported init-script/data-dir problems and skipped or failed to start, so they are noted as environment-limited ## Risks - Medium: this branch changes core issue/heartbeat routing and reopen/wakeup behavior, so regressions would affect agent execution flow rather than isolated UI polish - Because it also updates verification infrastructure, reviewers should pay attention to whether the new tests are asserting the right failure modes and not just reshaping harness behavior ## Model Used - OpenAI Codex coding agent (GPT-5-class runtime in Codex CLI; exact deployed model ID is not exposed in this environment), reasoning enabled, tool use and local code execution enabled ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [ ] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing> |
||
|
dotta
|
2a84e53c1b |
Introduce bind presets for deployment setup
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
b00d52c5b6 |
Merge pull request #3015 from aronprins/feature/backups-configuration
feat(backups): gzip compression and tiered retention with UI controls |
||
|
Daniel Luca
|
b7a7dacfa3 | fix: remove hardcoded JWT secret fallback from createBetterAuthInstance | ||
|
Aron Prins
|
b1e457365b |
fix: clean up orphaned .sql on compression failure and fix stale startup log
- backup-lib: delete uncompressed .sql file in catch block when gzip compression fails, preventing silent disk usage accumulation - server: replace stale retentionDays scalar with retentionSource in startup log since retention is now read from DB on each backup tick |
||
|
Aron Prins
|
fcbae62baf |
feat(backups): tiered daily/weekly/monthly retention with UI controls
Replace single retentionDays with a three-tier BackupRetentionPolicy: - Daily: keep all backups (presets: 3, 7, 14 days; default 7) - Weekly: keep one per calendar week (presets: 1, 2, 4 weeks; default 4) - Monthly: keep one per calendar month (presets: 1, 3, 6 months; default 1) Pruning sorts backups newest-first and applies each tier's cutoff, keeping only the newest entry per ISO week/month bucket. The Instance Settings General page now shows three preset selectors (no icon, matches existing page design). Remove Database icon import. |
||
|
Aron Prins
|
cc44d309c0 |
feat(backups): gzip compress backups and add retention config to Instance Settings
Compress database backups with gzip (.sql.gz), reducing file size ~83%. Add backup retention configuration to Instance Settings UI with preset options (7 days, 2 weeks, 1 month). The backup scheduler now reads retention from the database on each tick so changes take effect without restart. Default retention changed from 30 to 7 days. |
||
|
Khairul
|
e2962e6528 |
fix: increase Node keepAliveTimeout behind reverse proxies to prevent 502s
- Set server.keepAliveTimeout to 185s to safely outlive default Traefik/AWS ALB idle timeouts (typically 60-180s) - Resolves random "Failed to fetch" edge cases caused by Node.js's notoriously short 5s default timeout Closes #3008 |
||
|
HenkDz
|
80b81459a7 | Merge remote-tracking branch 'upstream/master' into feat/external-adapter-phase1 | ||
|
dotta
|
00898e8194 | Restore feedback trace export fixes | ||
|
HenkDz
|
14d59da316 |
feat(adapters): external adapter plugin system with dynamic UI parser
- Plugin loader: install/reload/remove/reinstall external adapters from npm packages or local directories - Plugin store persisted at ~/.paperclip/adapter-plugins.json - Self-healing UI parser resolution with version caching - UI: Adapter Manager page, dynamic loader, display registry with humanized names for unknown adapter types - Dev watch: exclude adapter-plugins dir from tsx watcher to prevent mid-request server restarts during reinstall - All consumer fallbacks use getAdapterLabel() for consistent display - AdapterTypeDropdown uses controlled open state for proper close behavior - Remove hermes-local from built-in UI (externalized to plugin) - Add docs for external adapters and UI parser contract |
||
|
dotta
|
f16de6026d |
fix: add periodic flush and graceful shutdown for server-side telemetry
The TelemetryClient only flushed at 50 events, so the server silently lost all queued telemetry on restart. Add startPeriodicFlush/stop methods to TelemetryClient, wire up 60s periodic flush in server initTelemetry, and flush on SIGTERM/SIGINT before exit. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dotta
|
34044cdfce |
feat: implement app-side telemetry sender
Add the shared telemetry sender, wire the CLI/server emit points, and cover the config and completion behavior with tests. Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dotta
|
c0d0d03bce |
Add feedback voting and thumbs capture flow
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dotta
|
76a692c260 |
Improve embedded Postgres bootstrap errors
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
dotta
|
ab82e3f022 |
Fix worktree runtime isolation recovery
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
ff8b839f42 |
Merge pull request #1658 from paperclipai/fix/migration-auto-apply-precedence
fix(server): check MIGRATION_AUTO_APPLY before MIGRATION_PROMPT |
||
|
dotta
|
5602576ae1 |
Fix embedded Postgres initdb failure in Docker slim containers
The embedded-postgres library hardcodes --lc-messages=en_US.UTF-8 and strips the parent process environment when spawning initdb/postgres. In slim Docker images (e.g. node:20-bookworm-slim), the en_US.UTF-8 locale isn't installed, causing initdb to exit with code 1. Two fixes applied: 1. Add --lc-messages=C to all initdbFlags arrays (overrides the library's hardcoded locale since our flags come after in the spread) 2. pnpm patch on embedded-postgres to preserve process.env in spawn calls, preventing loss of PATH, LD_LIBRARY_PATH, and other vars Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Devin Foley
|
dd1d9bed80 |
fix(server): check MIGRATION_AUTO_APPLY before MIGRATION_PROMPT
PAPERCLIP_MIGRATION_PROMPT=never was checked before PAPERCLIP_MIGRATION_AUTO_APPLY=true, causing auto-apply to never trigger when both env vars are set (as in dev:watch). Swap the check order so AUTO_APPLY takes precedence. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
dotta
|
8f5196f7d6 |
Add routines automation workflows
Co-Authored-By: Paperclip <noreply@paperclip.ing> |
||
|
Dotta
|
88bf1b23a3 | Harden embedded postgres adoption on startup | ||
|
Dotta
|
bb7d1b2c71 |
Merge remote-tracking branch 'public-gh/master' into paperclip-subissues
* public-gh/master: Fix budget incident resolution edge cases Fix agent budget tab routing Fix budget auth and monthly spend rollups Harden budget enforcement and migration startup Add budget tabs and sidebar budget indicators feat(costs): add billing, quota, and budget control plane refactor(quota): move provider quota logic into adapter layer, add unit tests fix(costs): replace non-null map assertions with nullish coalescing, clarify weekData guard fix(costs): guard byProject against duplicate null keys, memoize ProviderQuotaCard row aggregations fix(costs): align byAgent run filter to startedAt, tighten providerTabItems memo deps, stabilize byProject row keys feat(costs): add agent model breakdown, harden date validation, sync CostByProject type, fix quota threshold and tab-gated queries fix(costs): harden company auth check, fix frozen date memo, hide empty quota rows fix(costs): guard routes, fix DST ranges, sync provider state, wire live updates feat(costs): consolidate /usage into /costs with Spend + Providers tabs feat(usage): add subscription quota windows per provider on /usage page address greptile review: per-provider deficit notch, startedAt filter, weekRange refresh, deduplicate providerDisplayName feat(ui): add resource and usage dashboard (/usage route) # Conflicts: # packages/db/src/migration-runtime.ts # packages/db/src/migrations/meta/0031_snapshot.json # packages/db/src/migrations/meta/_journal.json |
||
|
Dotta
|
5f2c2ee0e2 | Harden budget enforcement and migration startup | ||
|
Dotta
|
7a06a577ce | Fix dev startup with embedded postgres reuse | ||
|
Dotta
|
fe764cac75 |
fix: resolve type errors in process-lost-reaper PR
- Fix malformed try/catch/finally blocks in heartbeat executeRun - Declare activeRunExecutions Set to track in-flight runs - Add resumeQueuedRuns function and export from heartbeat service - Add initdbFlags to EmbeddedPostgresCtor type Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Dotta
|
f81d37fbf7 |
fix(heartbeat): prevent false process_lost failures on queued and non-child-process runs
- reapOrphanedRuns() now only scans running runs; queued runs are legitimately absent from runningProcesses (waiting on concurrency limits or issue locks) so including them caused false process_lost failures (closes #90) - Add module-level activeRunExecutions set so non-child-process adapters (http, openclaw) are protected from the reaper during execution - Add resumeQueuedRuns() to restart persisted queued runs after a server restart, called at startup and each periodic tick - Add outer catch in executeRun() so setup failures (ensureRuntimeState, resolveWorkspaceForRun, etc.) are recorded as failed runs instead of leaving them stuck in running state - Guard resumeQueuedRuns() against paused/terminated/pending_approval agents - Increase opencode models discovery timeout from 20s to 45s |
||
|
Volodymyr Kartavyi
|
057e3a494c |
fix: ensure embedded PostgreSQL databases use UTF-8 encoding
On macOS, `initdb` defaults to SQL_ASCII encoding because it infers locale from the system environment. When `ensurePostgresDatabase()` creates a database without specifying encoding, the new database inherits SQL_ASCII from the cluster. This causes string functions like `left()` to operate on bytes instead of characters, producing invalid UTF-8 when multi-byte characters are truncated. Two-part fix: 1. Pass `--encoding=UTF8 --locale=C` via `initdbFlags` to all EmbeddedPostgres constructors so the cluster defaults to UTF-8. 2. Explicitly set `encoding 'UTF8'` in the CREATE DATABASE statement with `template template0` (required because template1 may already have a different encoding) and `C` locale for portability. Existing databases created with SQL_ASCII are NOT automatically fixed; users must delete their local `data/db` directory and restart to re-initialize the cluster. Relates to #636 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Dotta
|
3120c72372 | Add worktree-aware workspace runtime support | ||
|
Dotta
|
83738b45cd | Fix worktree minimal clone startup |
