Fix sandbox git publishing and large workspace uploads (#8422)

## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Sandbox and SSH runtimes need to preserve agent work across isolated
execution environments
> - Git-backed workspaces were being copied mostly as filesystem
archives, which breaks when `.git` points outside the mounted workspace
and makes sandbox agents unable to publish their own branches
> - Large ignored dependency trees could also be swept into the sandbox
overlay, causing multi-GB transfers and max-string failures in some
sandbox clients
> - This pull request makes sandbox runtime setup use a git-backed HEAD
sync plus a small dirty/untracked overlay, and bounds sandbox file
transfers so large archives do not need one huge string
> - The benefit is that sandbox agents can commit and push from a usable
git checkout without uploading dependency trees such as `node_modules`

## Linked Issues or Issue Description

Refs #8395

No public duplicate issue or PR was found after searches for `sandbox
git workspace`, `git push sandbox`, and `node_modules sandbox upload`.

Bug report:

- What happened: sandbox-backed agent workspaces could receive a `.git`
file that pointed at host-only git state, leaving the sandbox unable to
run normal git workflows. The sandbox overlay upload could also include
ignored dependency directories, creating very large transfers.
- Expected behavior: sandbox and remote runtimes should prepare a usable
git-backed workspace, copy only the necessary workspace overlay, and
restore git history plus file changes without depending on a host-only
`.git` path.
- Steps to reproduce:
1. Run an agent in a sandbox-backed workspace whose local git checkout
is a worktree.
2. Ask the agent to complete a GitHub workflow that requires commit/push
access.
3. Observe that git operations can fail inside the sandbox, and ignored
dependency trees can be uploaded as part of the workspace overlay.
- Paperclip version or commit: reproduced against `master` before this
PR, base `7aa212296eb1`.
- Deployment mode: local dev / sandbox-backed runtime.
- Installation method: built from source.
- Agent adapters involved: local adapters using shared adapter-utils
runtime preparation.
- Database mode: not database-related.
- Access context: agent runtime.
- Local verification environment: Node.js v25.6.1, pnpm 9.15.4, macOS
arm64.
- Privacy checklist: all pasted output was reviewed for secrets, private
hostnames, local usernames, and internal instance links.

## What Changed

- Added a GitHub workflow push preflight so agent runs can detect
missing push credentials when a workflow explicitly needs GitHub
publishing.
- Added shared git workspace sync helpers for shallow HEAD import/export
and dirty/untracked overlay tracking.
- Updated sandbox managed runtime setup to use git history plus a
selected overlay instead of uploading the full local workspace for
git-backed workspaces.
- Bounded sandbox archive upload/download paths so large payloads stream
or chunk instead of materializing one oversized string.
- Excluded `.git` and ignored dependency trees from sandbox upload,
download, and restore baselines while preserving local ignored
directories during sync-back.
- Added focused tests for git workspace sync, sandbox overlay selection,
transfer chunking, and heartbeat push-preflight behavior.

## Verification

- `pnpm exec vitest run
packages/adapter-utils/src/git-workspace-sync.test.ts
packages/adapter-utils/src/sandbox-managed-runtime.test.ts
packages/adapter-utils/src/command-managed-runtime.test.ts
server/src/__tests__/heartbeat-project-env.test.ts
server/src/__tests__/heartbeat-workspace-session.test.ts`
- `pnpm --filter @paperclipai/adapter-utils typecheck`
- `pnpm --filter @paperclipai/server typecheck`
- `pnpm build`
- Public-hygiene scan of the PR diff and commit messages for internal
issue ids, local paths, private hostnames, and obvious token patterns.

## Risks

- Medium risk: this changes sandbox runtime synchronization semantics
for git-backed workspaces, especially around dirty tracked files,
untracked files, deleted paths, and ignored files.
- The main mitigation is focused test coverage for upload contents,
restore exclusions, and git round-trip behavior.
- The SSH runtime keeps the current bundle-based implementation from
`master`; this PR only aligns shared excludes and sandbox behavior with
that model.

## Model Used

OpenAI Codex, GPT-5-based coding agent, tool-enabled shell/git/GitHub
workflow, with code execution and repository inspection.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>

packages/adapter-utils/src/command-managed-runtime.test.ts

@@ -22,7 +22,10 @@ interface SpawnRunnerHandle {
 // A runner that actually executes the shell scripts (piping stdin through a real
 // pipe so multi-MB payloads work) and replays stdout through onLog in several
 // chunks so the streaming readFile byte-counter is exercised.
-function makeSpawnRunner(options: { supportsSingleStreamStdinProgress?: boolean } = {}): SpawnRunnerHandle {
+function makeSpawnRunner(options: {
+  supportsSingleStreamStdinProgress?: boolean;
+  maxStdoutBytes?: number;
+} = {}): SpawnRunnerHandle {
   const calls: Array<{ command: string; args?: string[]; cwd?: string; stdin?: string }> = [];
   const runner: CommandManagedRuntimeRunner = {
     supportsSingleStreamStdinProgress: options.supportsSingleStreamStdinProgress,
@@ -48,6 +51,21 @@ function makeSpawnRunner(options: { supportsSingleStreamStdinProgress?: boolean
           resolve({ exitCode: 127, signal: null, timedOut: false, stdout, stderr, pid: null, startedAt });
         });
         child.on("close", async (code) => {
+          if (
+            options.maxStdoutBytes != null &&
+            Buffer.byteLength(stdout, "utf8") > options.maxStdoutBytes
+          ) {
+            resolve({
+              exitCode: 1,
+              signal: null,
+              timedOut: false,
+              stdout,
+              stderr: `stdout exceeded ${options.maxStdoutBytes} bytes`,
+              pid: child.pid ?? null,
+              startedAt,
+            });
+            return;
+          }
           if (input.onLog && stdout.length > 0) {
             const chunkSize = Math.max(1, Math.ceil(stdout.length / 4));
             for (let offset = 0; offset < stdout.length; offset += chunkSize) {
@@ -75,6 +93,26 @@ function toArrayBuffer(buffer: Buffer): ArrayBuffer {
   return Uint8Array.from(buffer).buffer;
 }
 
+async function withBase64StringByteLimit<T>(limitBytes: number, fn: () => Promise<T>): Promise<T> {
+  const originalToString = Buffer.prototype.toString;
+  Buffer.prototype.toString = function patchedToString(
+    this: Buffer,
+    encoding?: BufferEncoding,
+    start?: number,
+    end?: number,
+  ) {
+    if (encoding === "base64" && this.byteLength > limitBytes) {
+      throw new Error(`test guard: attempted to base64-encode ${this.byteLength} bytes at once`);
+    }
+    return originalToString.call(this, encoding, start, end);
+  } as typeof Buffer.prototype.toString;
+  try {
+    return await fn();
+  } finally {
+    Buffer.prototype.toString = originalToString;
+  }
+}
+
 describe("command managed runtime", () => {
   const cleanupDirs: string[] = [];
 
@@ -243,10 +281,12 @@ describe("command managed runtime", () => {
     const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
 
     const progress: Array<{ done: number; total: number | null }> = [];
-    await client.writeFile(remotePath, toArrayBuffer(payload), {
-      onProgress: (done, total) => {
-        progress.push({ done, total });
-      },
+    await withBase64StringByteLimit(4 * 1024 * 1024, async () => {
+      await client.writeFile(remotePath, toArrayBuffer(payload), {
+        onProgress: (done, total) => {
+          progress.push({ done, total });
+        },
+      });
     });
 
     // Exactly one upload process: O(1) round-trips regardless of payload size.
@@ -288,6 +328,9 @@ describe("command managed runtime", () => {
     // Provider-backed sandbox runners cannot surface mid-flight progress for a
     // single stdin RPC, so we intentionally use several large append commands.
     expect(calls.length).toBeGreaterThan(2);
+    const stdinCalls = calls.filter((call) => call.stdin != null);
+    expect(stdinCalls.length).toBeGreaterThan(2);
+    expect(stdinCalls.every((call) => Buffer.byteLength(call.stdin ?? "", "utf8") <= 4.1 * 1024 * 1024)).toBe(true);
     expect(progress.length).toBeGreaterThan(2);
     for (let i = 1; i < progress.length; i++) {
       expect(progress[i].done).toBeGreaterThanOrEqual(progress[i - 1].done);
@@ -295,16 +338,41 @@ describe("command managed runtime", () => {
     expect(progress.at(-1)).toEqual({ done: payload.length, total: payload.length });
   });
 
-  it("streams a download through onLog and reports monotonic byte progress to the total", async () => {
+  it("falls back to bounded chunks when the runner does not explicitly opt in", async () => {
+    const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-command-write-fallback-no-progress-"));
+    cleanupDirs.push(rootDir);
+    const remotePath = path.join(rootDir, "nested", "payload.bin");
+
+    const payload = Buffer.alloc(12 * 1024 * 1024);
+    for (let i = 0; i < payload.length; i++) payload[i] = i % 256;
+
+    const { runner, calls } = makeSpawnRunner();
+    const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
+
+    await withBase64StringByteLimit(4 * 1024 * 1024, async () => {
+      await client.writeFile(remotePath, toArrayBuffer(payload));
+    });
+
+    const written = await readFile(remotePath);
+    expect(written.equals(payload)).toBe(true);
+
+    // A runner that doesn't mark single-stream stdin support must avoid passing
+    // the whole base64 archive as one string, so we expect multiple append calls.
+    const stdinCalls = calls.filter((call) => call.stdin != null);
+    expect(stdinCalls.length).toBeGreaterThan(1);
+    expect(stdinCalls.every((call) => Buffer.byteLength(call.stdin ?? "", "utf8") <= 4.1 * 1024 * 1024)).toBe(true);
+  });
+
+  it("downloads in bounded stdout chunks and reports monotonic byte progress to the total", async () => {
     const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-command-read-"));
     cleanupDirs.push(rootDir);
     const remotePath = path.join(rootDir, "download.bin");
 
-    const payload = Buffer.alloc(512 * 1024);
+    const payload = Buffer.alloc(7 * 1024 * 1024);
     for (let i = 0; i < payload.length; i++) payload[i] = (i * 7) % 256;
     await writeFile(remotePath, payload);
 
-    const { runner } = makeSpawnRunner();
+    const { runner, calls } = makeSpawnRunner({ maxStdoutBytes: 5 * 1024 * 1024 });
     const client = createCommandManagedRuntimeClient({ runner, commandCwd: "/", timeoutMs: 30_000 });
 
     const progress: Array<{ done: number; total: number | null }> = [];
@@ -316,7 +384,10 @@ describe("command managed runtime", () => {
 
     expect(Buffer.from(bytes as ArrayBuffer).equals(payload)).toBe(true);
 
-    // The runner replays stdout in several chunks, so progress arrives in steps.
+    // The old single `base64 < file` path would exceed the runner's stdout cap.
+    // The bounded path reads with several small `dd | base64` commands instead.
+    expect(calls.some((call) => call.args?.join(" ").includes("base64 <"))).toBe(false);
+    expect(calls.filter((call) => call.args?.join(" ").includes("dd if=")).length).toBeGreaterThan(1);
     expect(progress.length).toBeGreaterThan(1);
     for (let i = 1; i < progress.length; i++) {
       expect(progress[i].done).toBeGreaterThanOrEqual(progress[i - 1].done);