93cdc5c1ce
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Agents can run in remote/sandboxed environments via the shared sandbox managed-runtime in `@paperclipai/adapter-utils` (used by SSH/E2B/Daytona and other sandbox providers), which syncs the workspace into the sandbox by tarring it up and extracting it inside the pod/host > - When the sandbox runs the harness as a non-root user whose home/workspace dir it does not own (for example a hardened, non-root, gVisor pod with an `emptyDir`-mounted workspace), the workspace upload aborts before the agent can start > - Root cause: `createTarballFromDirectory` archives `.`, embedding a `./` self-entry whose mode/mtime tar then tries to restore onto the **extraction target directory**; `chmod`/`utime` of `.` fails with `Operation not permitted` for a non-owner > - This is not specific to any one deployment: the `.` self-entry EPERM can bite every sandbox provider built on the shared managed runtime as soon as the extracting user does not own the target directory, which is the norm for hardened non-root sandboxes > - This pull request archives the directory's top-level entries by name instead of `.`, so there is no `./` self-entry and extraction never touches the target dir's metadata > - The benefit is that workspace sync works in any sandbox where the target dir is non-root or not owned by the extracting user, without GNU-only tar flags ## Linked Issues or Issue Description No existing issue; describing in-PR (bug). - **What happens:** managed sandbox runs that sync the workspace fail at upload with `tar: .: Cannot utime: Operation not permitted` / `tar: .: Cannot change mode to ... : Operation not permitted`, aborting the run before the harness starts. - **Where:** `packages/adapter-utils/src/sandbox-managed-runtime.ts`, in `createTarballFromDirectory` (archives `.`). - **When:** the extraction target directory is not owned by the (non-root) user extracting the tar inside the sandbox. - Closely related (different root cause): #6560 (E2B workspace upload + lease idle failures). ## What Changed - `createTarballFromDirectory` enumerates the directory's top-level entries with `fs.readdir` and passes them by name after `--` (guards flag-like filenames) instead of archiving `.`, eliminating the `./` self-entry that triggers the EPERM. - Empty workspaces (legitimate for blank-workspace runs) write a valid 1024-byte all-zero EOF tar instead of invoking `tar` with no paths. - `--exclude` patterns continue to apply (to nested matches and any named entry). ## Verification - `pnpm --filter @paperclipai/adapter-utils build` (tsc clean) - `pnpm exec vitest run packages/adapter-utils/src/sandbox-managed-runtime.test.ts` runs green - New tests: uploaded workspace/asset tarballs contain no `.`/`./` member yet still extract correctly; empty workspace produces a valid (no-op) tarball. Existing managed-runtime sync test unchanged. - Manually verified in a hardened (non-root, gVisor) sandbox pod: with the fix, the workspace upload that previously aborted with the EPERM now succeeds. That deployment is the reproduction and verification environment; the fix itself is provider-agnostic. ## Risks Low. Behavior is unchanged for owned/root targets; the archive contents are the same minus the `./` self-entry (which tar recreates implicitly on extract). Portable across GNU/BSD/busybox tar (no GNU-only `--no-overwrite-dir`). No API/migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work (bug fix in shared sandbox utils, not core feature work) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#6560) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (n/a, internal behavior, no docs reference this) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (the only finding was the description-template P2, resolved by this description; the latest review covers the current head with no code findings and all CI gates are green) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
242 lines
11 KiB
TypeScript
242 lines
11 KiB
TypeScript
import { lstat, mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from "node:fs/promises";
|
|
import os from "node:os";
|
|
import path from "node:path";
|
|
import { execFile as execFileCallback } from "node:child_process";
|
|
import { promisify } from "node:util";
|
|
import { afterEach, describe, expect, it } from "vitest";
|
|
|
|
import {
|
|
mirrorDirectory,
|
|
prepareSandboxManagedRuntime,
|
|
type SandboxManagedRuntimeClient,
|
|
} from "./sandbox-managed-runtime.js";
|
|
|
|
const execFile = promisify(execFileCallback);
|
|
|
|
describe("sandbox managed runtime", () => {
|
|
const cleanupDirs: string[] = [];
|
|
|
|
afterEach(async () => {
|
|
while (cleanupDirs.length > 0) {
|
|
const dir = cleanupDirs.pop();
|
|
if (!dir) continue;
|
|
await rm(dir, { recursive: true, force: true }).catch(() => undefined);
|
|
}
|
|
});
|
|
|
|
it("preserves excluded local workspace artifacts during restore mirroring", async () => {
|
|
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-restore-"));
|
|
cleanupDirs.push(rootDir);
|
|
const sourceDir = path.join(rootDir, "source");
|
|
const targetDir = path.join(rootDir, "target");
|
|
await mkdir(path.join(sourceDir, "src"), { recursive: true });
|
|
await mkdir(path.join(targetDir, ".claude"), { recursive: true });
|
|
await mkdir(path.join(targetDir, ".paperclip-runtime"), { recursive: true });
|
|
await writeFile(path.join(sourceDir, "src", "app.ts"), "export const value = 2;\n", "utf8");
|
|
await writeFile(path.join(targetDir, "stale.txt"), "remove me\n", "utf8");
|
|
await writeFile(path.join(targetDir, ".claude", "settings.json"), "{\"keep\":true}\n", "utf8");
|
|
await writeFile(path.join(targetDir, ".claude.json"), "{\"keep\":true}\n", "utf8");
|
|
await writeFile(path.join(targetDir, ".paperclip-runtime", "state.json"), "{}\n", "utf8");
|
|
|
|
await mirrorDirectory(sourceDir, targetDir, {
|
|
preserveAbsent: [".paperclip-runtime", ".claude", ".claude.json"],
|
|
});
|
|
|
|
await expect(readFile(path.join(targetDir, "src", "app.ts"), "utf8")).resolves.toBe("export const value = 2;\n");
|
|
await expect(readFile(path.join(targetDir, ".claude", "settings.json"), "utf8")).resolves.toBe("{\"keep\":true}\n");
|
|
await expect(readFile(path.join(targetDir, ".claude.json"), "utf8")).resolves.toBe("{\"keep\":true}\n");
|
|
await expect(readFile(path.join(targetDir, ".paperclip-runtime", "state.json"), "utf8")).resolves.toBe("{}\n");
|
|
await expect(readFile(path.join(targetDir, "stale.txt"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
|
|
});
|
|
|
|
it("syncs workspace and assets through a provider-neutral sandbox client", async () => {
|
|
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-managed-"));
|
|
cleanupDirs.push(rootDir);
|
|
const localWorkspaceDir = path.join(rootDir, "local-workspace");
|
|
const remoteWorkspaceDir = path.join(rootDir, "remote-workspace");
|
|
const localAssetsDir = path.join(rootDir, "local-assets");
|
|
const linkedAssetPath = path.join(rootDir, "linked-skill.md");
|
|
await mkdir(path.join(localWorkspaceDir, ".claude"), { recursive: true });
|
|
await mkdir(localAssetsDir, { recursive: true });
|
|
await writeFile(path.join(localWorkspaceDir, "README.md"), "local workspace\n", "utf8");
|
|
await writeFile(path.join(localWorkspaceDir, "._README.md"), "appledouble\n", "utf8");
|
|
await writeFile(path.join(localWorkspaceDir, ".claude", "settings.json"), "{\"local\":true}\n", "utf8");
|
|
await writeFile(linkedAssetPath, "skill body\n", "utf8");
|
|
await symlink(linkedAssetPath, path.join(localAssetsDir, "skill.md"));
|
|
|
|
const client: SandboxManagedRuntimeClient = {
|
|
makeDir: async (remotePath) => {
|
|
await mkdir(remotePath, { recursive: true });
|
|
},
|
|
writeFile: async (remotePath, bytes) => {
|
|
await mkdir(path.dirname(remotePath), { recursive: true });
|
|
await writeFile(remotePath, Buffer.from(bytes));
|
|
},
|
|
readFile: async (remotePath) => await readFile(remotePath),
|
|
listFiles: async (remotePath) => {
|
|
const entries = await readdir(remotePath, { withFileTypes: true }).catch(() => []);
|
|
return entries
|
|
.filter((entry) => entry.isFile())
|
|
.map((entry) => entry.name)
|
|
.sort((left, right) => left.localeCompare(right));
|
|
},
|
|
remove: async (remotePath) => {
|
|
await rm(remotePath, { recursive: true, force: true });
|
|
},
|
|
run: async (command) => {
|
|
await execFile("sh", ["-c", command], {
|
|
maxBuffer: 32 * 1024 * 1024,
|
|
});
|
|
},
|
|
};
|
|
|
|
const prepared = await prepareSandboxManagedRuntime({
|
|
spec: {
|
|
transport: "sandbox",
|
|
provider: "test",
|
|
sandboxId: "sandbox-1",
|
|
remoteCwd: remoteWorkspaceDir,
|
|
timeoutMs: 30_000,
|
|
apiKey: null,
|
|
},
|
|
adapterKey: "test-adapter",
|
|
client,
|
|
workspaceLocalDir: localWorkspaceDir,
|
|
workspaceExclude: [".claude"],
|
|
preserveAbsentOnRestore: [".claude"],
|
|
assets: [{
|
|
key: "skills",
|
|
localDir: localAssetsDir,
|
|
followSymlinks: true,
|
|
}],
|
|
});
|
|
|
|
await expect(readFile(path.join(remoteWorkspaceDir, "README.md"), "utf8")).resolves.toBe("local workspace\n");
|
|
await expect(readFile(path.join(remoteWorkspaceDir, "._README.md"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
|
|
await expect(readFile(path.join(remoteWorkspaceDir, ".claude", "settings.json"), "utf8")).rejects.toMatchObject({ code: "ENOENT" });
|
|
await expect(readFile(path.join(prepared.assetDirs.skills, "skill.md"), "utf8")).resolves.toBe("skill body\n");
|
|
expect((await lstat(path.join(prepared.assetDirs.skills, "skill.md"))).isFile()).toBe(true);
|
|
|
|
await writeFile(path.join(remoteWorkspaceDir, "README.md"), "remote workspace\n", "utf8");
|
|
await writeFile(path.join(remoteWorkspaceDir, "remote-only.txt"), "sync back\n", "utf8");
|
|
await mkdir(path.join(localWorkspaceDir, ".paperclip-runtime"), { recursive: true });
|
|
await writeFile(path.join(localWorkspaceDir, ".paperclip-runtime", "state.json"), "{}\n", "utf8");
|
|
await writeFile(path.join(localWorkspaceDir, "local-stale.txt"), "remove\n", "utf8");
|
|
await prepared.restoreWorkspace();
|
|
|
|
await expect(readFile(path.join(localWorkspaceDir, "README.md"), "utf8")).resolves.toBe("remote workspace\n");
|
|
await expect(readFile(path.join(localWorkspaceDir, "remote-only.txt"), "utf8")).resolves.toBe("sync back\n");
|
|
await expect(readFile(path.join(localWorkspaceDir, "local-stale.txt"), "utf8")).resolves.toBe("remove\n");
|
|
await expect(readFile(path.join(localWorkspaceDir, ".claude", "settings.json"), "utf8")).resolves.toBe("{\"local\":true}\n");
|
|
await expect(readFile(path.join(localWorkspaceDir, ".paperclip-runtime", "state.json"), "utf8")).resolves.toBe("{}\n");
|
|
});
|
|
|
|
it("builds workspace/asset tarballs without a './' self-entry (so untar does not chmod/utime an unowned target dir)", async () => {
|
|
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-tarself-"));
|
|
cleanupDirs.push(rootDir);
|
|
const localWorkspaceDir = path.join(rootDir, "local-workspace");
|
|
const remoteWorkspaceDir = path.join(rootDir, "remote-workspace");
|
|
const localAssetsDir = path.join(rootDir, "local-assets");
|
|
await mkdir(path.join(localWorkspaceDir, "src"), { recursive: true });
|
|
await mkdir(localAssetsDir, { recursive: true });
|
|
await writeFile(path.join(localWorkspaceDir, "README.md"), "ws\n", "utf8");
|
|
await writeFile(path.join(localWorkspaceDir, "src", "main.ts"), "x\n", "utf8");
|
|
await writeFile(path.join(localAssetsDir, "asset.txt"), "a\n", "utf8");
|
|
|
|
// Capture every tar uploaded to the sandbox so we can inspect its members.
|
|
const uploadedTars: { remotePath: string; bytes: Buffer }[] = [];
|
|
const client: SandboxManagedRuntimeClient = {
|
|
makeDir: async (remotePath) => {
|
|
await mkdir(remotePath, { recursive: true });
|
|
},
|
|
writeFile: async (remotePath, bytes) => {
|
|
await mkdir(path.dirname(remotePath), { recursive: true });
|
|
const buffer = Buffer.from(bytes);
|
|
if (remotePath.endsWith("-upload.tar")) uploadedTars.push({ remotePath, bytes: buffer });
|
|
await writeFile(remotePath, buffer);
|
|
},
|
|
readFile: async (remotePath) => await readFile(remotePath),
|
|
listFiles: async () => [],
|
|
remove: async (remotePath) => {
|
|
await rm(remotePath, { recursive: true, force: true });
|
|
},
|
|
run: async (command) => {
|
|
await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 });
|
|
},
|
|
};
|
|
|
|
await prepareSandboxManagedRuntime({
|
|
spec: {
|
|
transport: "sandbox",
|
|
provider: "test",
|
|
sandboxId: "sandbox-1",
|
|
remoteCwd: remoteWorkspaceDir,
|
|
timeoutMs: 30_000,
|
|
apiKey: null,
|
|
},
|
|
adapterKey: "test-adapter",
|
|
client,
|
|
workspaceLocalDir: localWorkspaceDir,
|
|
assets: [{ key: "skills", localDir: localAssetsDir }],
|
|
});
|
|
|
|
expect(uploadedTars.length).toBeGreaterThanOrEqual(2);
|
|
for (const { remotePath, bytes } of uploadedTars) {
|
|
const listPath = path.join(rootDir, `list-${path.basename(remotePath)}`);
|
|
await writeFile(listPath, bytes);
|
|
const { stdout } = await execFile("tar", ["-tf", listPath], { maxBuffer: 32 * 1024 * 1024 });
|
|
const members = stdout.split("\n").map((line) => line.trim()).filter(Boolean);
|
|
// The archive must NOT contain a self-entry for the root directory; that is
|
|
// what makes tar try to mutate the (possibly unowned) extraction target.
|
|
expect(members).not.toContain(".");
|
|
expect(members).not.toContain("./");
|
|
}
|
|
|
|
// And the workspace still extracts correctly into an existing target dir.
|
|
await expect(readFile(path.join(remoteWorkspaceDir, "README.md"), "utf8")).resolves.toBe("ws\n");
|
|
await expect(readFile(path.join(remoteWorkspaceDir, "src", "main.ts"), "utf8")).resolves.toBe("x\n");
|
|
});
|
|
|
|
it("creates a valid empty workspace tarball when the local workspace is empty", async () => {
|
|
const rootDir = await mkdtemp(path.join(os.tmpdir(), "paperclip-sandbox-empty-"));
|
|
cleanupDirs.push(rootDir);
|
|
const localWorkspaceDir = path.join(rootDir, "local-workspace");
|
|
const remoteWorkspaceDir = path.join(rootDir, "remote-workspace");
|
|
await mkdir(localWorkspaceDir, { recursive: true });
|
|
|
|
const client: SandboxManagedRuntimeClient = {
|
|
makeDir: async (remotePath) => {
|
|
await mkdir(remotePath, { recursive: true });
|
|
},
|
|
writeFile: async (remotePath, bytes) => {
|
|
await mkdir(path.dirname(remotePath), { recursive: true });
|
|
await writeFile(remotePath, Buffer.from(bytes));
|
|
},
|
|
readFile: async (remotePath) => await readFile(remotePath),
|
|
listFiles: async () => [],
|
|
remove: async (remotePath) => {
|
|
await rm(remotePath, { recursive: true, force: true });
|
|
},
|
|
run: async (command) => {
|
|
await execFile("sh", ["-c", command], { maxBuffer: 32 * 1024 * 1024 });
|
|
},
|
|
};
|
|
|
|
await expect(
|
|
prepareSandboxManagedRuntime({
|
|
spec: {
|
|
transport: "sandbox",
|
|
provider: "test",
|
|
sandboxId: "sandbox-1",
|
|
remoteCwd: remoteWorkspaceDir,
|
|
timeoutMs: 30_000,
|
|
apiKey: null,
|
|
},
|
|
adapterKey: "test-adapter",
|
|
client,
|
|
workspaceLocalDir: localWorkspaceDir,
|
|
}),
|
|
).resolves.toBeDefined();
|
|
});
|
|
});
|