feat(adapter-utils): add observable sandbox sync progress (#8395)
## Thinking Path > - Paperclip is the control plane for running AI-agent companies, so long-running remote work needs to stay observable to human operators. > - Cloud / sandbox agents are an active roadmap area, and their workspace sync path is part of the runtime substrate every remote coding run depends on. > - In the sandbox and SSH execution-target flows, Paperclip logged that sync had started, then often went silent for the full transfer window. > - That made large remote syncs feel stalled and also hid a real performance problem in the command-managed sandbox upload path. > - The first part of this pull request threads a throttled progress-reporting surface through the adapter execution-target stack so sync and restore work can emit meaningful updates. > - The second part fixes the command-managed sandbox transport itself: it removes the old serial 32KB append bottleneck, but also falls back away from the single-stream path when a provider-backed sandbox runner cannot surface mid-flight stdin progress. > - The result is that sandbox and SSH transfers are both faster and more observable, including the live Daytona-style sandbox case that previously only emitted `0%` and `100%`. ## Linked Issues or Issue Description No public GitHub issue exists for this bug, so it is described inline below following the bug report template. ### What happened - Remote sandbox and SSH workspace syncs could spend a long time transferring data while only logging a start line (`Syncing workspace and runtime assets to sandbox environment`) and, at best, a terminal line. - In the command-managed sandbox path, the original upload implementation also paid a large performance cost by appending base64 data in many small sequential remote writes (thousands of serial 32KB round-trips on a large workspace). - After the initial transport rewrite, live provider-backed sandbox runs still only emitted `0%` and `100%` because the single-stream stdin RPC buffered progress until completion. ### Expected behavior - Long-running sandbox and SSH syncs should periodically report how much of the transfer is complete (a percentage and/or MB transferred) so an operator can tell the run is healthy and making progress rather than stuck. - The main sandbox upload path should not be artificially slow. - A transfer that fails partway should leave an explicit failure marker in the log rather than a dangling intermediate percentage. ### Steps to reproduce 1. Run an agent against a sandbox (command-managed) or SSH (remote-managed) execution target with a non-trivial workspace. 2. Watch the run log during the workspace/runtime asset sync phase. 3. Observe that the log shows the sync start line and then stays silent for the full transfer (live provider-backed sandbox runs only show `0%` then `100%`). ### Paperclip version or commit - Branch `PAPA-825-provide-status-updates-when-syncing-sandboxes` off `master`. ### Deployment mode - Self-hosted / local instance using sandbox (command-managed) and SSH (remote-managed) execution targets, including provider-backed sandbox runners. ## What Changed - Added shared throttled runtime progress reporting and threaded `onProgress` through the adapter execution-target surface and adapter `execute.ts` entrypoints. - Added sync and restore progress reporting for the command-managed sandbox path and the SSH/remote-managed path, including git import/export progress where totals are known. - Reworked command-managed sandbox transfer behavior so uploads use the faster single-stream path when appropriate, but fall back to chunked progress-emitting writes when the runner cannot expose mid-stream stdin progress. - Marked provider-backed environment sandbox runners as not supporting single-stream stdin progress so live sandbox runs emit meaningful intermediate updates instead of only `0%` and `100%`. - Emit an explicit terminal failure marker (`failed at NN% (x/y MB)`) when an SSH/tar transfer rejects, so a failed sync no longer leaves a dangling intermediate percentage in the log. - Run the SSH sync/restore size estimate (local directory walk / remote `du` probe) concurrently with the transfer instead of awaiting it before opening the pipe, so progress instrumentation no longer adds startup latency proportional to workspace file count. - Added and extended focused regression coverage for runtime progress throttling and the new failure marker, command-managed sandbox transfers, sandbox orchestration, SSH transfer progress, and environment execution-target wiring. ## Verification - `pnpm exec vitest run packages/adapter-utils/src/runtime-progress.test.ts packages/adapter-utils/src/ssh-fixture.test.ts packages/adapter-utils/src/command-managed-runtime.test.ts packages/adapter-utils/src/sandbox-managed-runtime.test.ts` - `pnpm exec vitest run packages/adapter-utils/src/command-managed-runtime.test.ts server/src/__tests__/environment-execution-target.test.ts` - `npx tsc --noEmit` for `packages/adapter-utils` ## Risks - The provider-backed sandbox fallback now prefers chunked command-managed writes when progress hooks are active, so small-to-medium uploads may trade some raw throughput for observable intermediate progress on runtimes that cannot surface true mid-stream stdin progress. - Progress percentages on tar-based transfers still depend on estimates in some cases, so operators may briefly see MB-only lines before the estimate resolves, then near-final clamping before the terminal `100%` line. - This PR changes shared execution-target behavior used by multiple adapters, so regressions would most likely appear in remote runtime setup/teardown flows rather than in a single adapter. ## Model Used - Initial implementation: OpenAI GPT-5.4 via Codex local agent (`codex_local`), high reasoning mode. - Observability follow-ups (failure marker, concurrent size estimate, added tests): Claude Opus 4.8 via Claude Code (`claude_local`). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots - [ ] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [ ] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
This commit is contained in:
Devin Foley
@@ -4,10 +4,17 @@ import { constants as fsConstants, createReadStream, createWriteStream, promises
|
||||
import net from "node:net";
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import { Transform } from "node:stream";
|
||||
import type { CommandManagedRuntimeRunner } from "./command-managed-runtime.js";
|
||||
import type { RunProcessResult } from "./server-utils.js";
|
||||
import type { DirectorySnapshot } from "./workspace-restore-merge.js";
|
||||
import { mergeDirectoryWithBaseline } from "./workspace-restore-merge.js";
|
||||
import {
|
||||
createRuntimeProgressReporter,
|
||||
type RuntimeProgressDirection,
|
||||
type RuntimeProgressPhase,
|
||||
type RuntimeProgressSink,
|
||||
} from "./runtime-progress.js";
|
||||
|
||||
export interface SshConnectionConfig {
|
||||
host: string;
|
||||
@@ -409,6 +416,152 @@ function tarSpawnEnv(): NodeJS.ProcessEnv {
|
||||
};
|
||||
}
|
||||
|
||||
// Converts a tar `--exclude` pattern into a regexp for the local-size estimate.
|
||||
// We only need approximate fidelity here (the estimate feeds a clamped percent),
|
||||
// so we support the literal names and `*`/`?` globs used in practice.
|
||||
function tarPatternToRegExp(pattern: string): RegExp {
|
||||
const escaped = pattern
|
||||
.replace(/[.+^${}()|[\]\\]/g, "\\$&")
|
||||
.replace(/\*/g, "[^/]*")
|
||||
.replace(/\?/g, "[^/]");
|
||||
return new RegExp(`^${escaped}$`);
|
||||
}
|
||||
|
||||
// Walks `localDir` summing regular-file sizes, mirroring tar's `--exclude`
|
||||
// handling (plus the implicit `._*`) and `followSymlinks` so the to-ssh upload
|
||||
// can report an estimated total before tar finishes producing the stream.
|
||||
async function estimateLocalDirSize(input: {
|
||||
localDir: string;
|
||||
exclude?: string[];
|
||||
followSymlinks?: boolean;
|
||||
}): Promise<number> {
|
||||
const regexes = ["._*", ...(input.exclude ?? [])].map(tarPatternToRegExp);
|
||||
const isExcluded = (relPath: string, base: string) =>
|
||||
regexes.some((regex) => regex.test(relPath) || regex.test(base));
|
||||
|
||||
let total = 0;
|
||||
const walk = async (dir: string, relative: string): Promise<void> => {
|
||||
const entries = await fs.readdir(dir, { withFileTypes: true }).catch(() => []);
|
||||
for (const entry of entries) {
|
||||
const entryRelative = relative ? `${relative}/${entry.name}` : entry.name;
|
||||
if (isExcluded(entryRelative, entry.name)) continue;
|
||||
const full = path.join(dir, entry.name);
|
||||
const stats = await (input.followSymlinks ? fs.stat(full) : fs.lstat(full)).catch(() => null);
|
||||
if (!stats) continue;
|
||||
if (stats.isDirectory()) {
|
||||
await walk(full, entryRelative);
|
||||
} else if (stats.isFile()) {
|
||||
total += stats.size;
|
||||
}
|
||||
}
|
||||
};
|
||||
await walk(input.localDir, "");
|
||||
return total;
|
||||
}
|
||||
|
||||
// Best-effort remote size probe for the from-ssh restore. `du -sk` is POSIX and
|
||||
// available on the BSD/Linux remotes we target; it over-counts (block-rounded,
|
||||
// includes excluded dirs) which keeps the reported percent safely below 100
|
||||
// until the stream actually closes. Returns null when unavailable so the caller
|
||||
// falls back to MB-received mode.
|
||||
async function probeRemoteDirSize(input: {
|
||||
spec: SshConnectionConfig;
|
||||
remoteDir: string;
|
||||
}): Promise<number | null> {
|
||||
try {
|
||||
const result = await runSshScript(
|
||||
input.spec,
|
||||
`du -sk ${shellQuote(input.remoteDir)} 2>/dev/null | cut -f1`,
|
||||
{ timeoutMs: 15_000, maxBuffer: 16 * 1024 },
|
||||
);
|
||||
const kilobytes = Number.parseInt(result.stdout.trim(), 10);
|
||||
return Number.isFinite(kilobytes) && kilobytes > 0 ? kilobytes * 1024 : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
interface TransferProgress {
|
||||
// Backpressure-respecting counter to splice into a transport pipe.
|
||||
counter: Transform;
|
||||
// Last cumulative byte count observed by the counter.
|
||||
transferred: () => number;
|
||||
// Emit the terminal completion line. Idempotent.
|
||||
finish: () => Promise<void>;
|
||||
// Emit a terminal failure marker instead of a completion line. Idempotent.
|
||||
fail: () => Promise<void>;
|
||||
}
|
||||
|
||||
// Wraps a throttled progress reporter behind a counting Transform so transports
|
||||
// can `source.pipe(progress.counter).pipe(dest)`. When `totalBytes` is a known
|
||||
// exact size (e.g. a git bundle) the reporter emits an exact percentage. When it
|
||||
// is an estimate (tar upload / remote probe) we clamp the reported bytes to 99%
|
||||
// of the estimate so an inaccurate total never shows a premature 100%; `finish`
|
||||
// then emits the terminal 100% (or, in MB-only mode, the final MB) line.
|
||||
//
|
||||
// `totalBytes` may be a promise so an expensive size estimate (a local dir walk
|
||||
// or a remote `du` probe) runs concurrently with the transfer instead of
|
||||
// blocking the pipe from opening. Until it resolves the counter reports bytes in
|
||||
// MB-only mode, then adopts the percentage once the total is known; `finish`
|
||||
// awaits the estimate so the terminal 100% line is still guaranteed.
|
||||
function createTransferProgress(input: {
|
||||
onProgress: RuntimeProgressSink;
|
||||
phase: RuntimeProgressPhase;
|
||||
direction: RuntimeProgressDirection;
|
||||
label?: string;
|
||||
totalBytes: number | null | Promise<number | null>;
|
||||
estimated: boolean;
|
||||
}): TransferProgress {
|
||||
const reporter = createRuntimeProgressReporter({
|
||||
sink: input.onProgress,
|
||||
phase: input.phase,
|
||||
direction: input.direction,
|
||||
label: input.label,
|
||||
target: "ssh",
|
||||
});
|
||||
|
||||
let total: number | null = null;
|
||||
let cap: number | null = null;
|
||||
const applyTotal = (value: number | null) => {
|
||||
total = value != null && value > 0 ? value : null;
|
||||
cap = total != null && input.estimated ? Math.floor(total * 0.99) : null;
|
||||
};
|
||||
const totalReady: Promise<void> =
|
||||
input.totalBytes != null && typeof (input.totalBytes as Promise<number | null>).then === "function"
|
||||
? (input.totalBytes as Promise<number | null>).then(applyTotal, () => applyTotal(null))
|
||||
: (applyTotal(input.totalBytes as number | null), Promise.resolve());
|
||||
|
||||
let transferred = 0;
|
||||
let chain: Promise<void> = Promise.resolve();
|
||||
const enqueue = (work: () => Promise<void>) => {
|
||||
chain = chain.then(work).catch(() => undefined);
|
||||
};
|
||||
|
||||
const counter = new Transform({
|
||||
transform(chunk: Buffer, _encoding, callback) {
|
||||
transferred += chunk.length;
|
||||
const reported = cap != null ? Math.min(transferred, cap) : transferred;
|
||||
const totalSnapshot = total;
|
||||
enqueue(() => reporter.report(reported, totalSnapshot));
|
||||
callback(null, chunk);
|
||||
},
|
||||
});
|
||||
|
||||
return {
|
||||
counter,
|
||||
transferred: () => transferred,
|
||||
finish: async () => {
|
||||
await chain.catch(() => undefined);
|
||||
await totalReady.catch(() => undefined);
|
||||
await reporter.complete(total != null ? total : transferred, total).catch(() => undefined);
|
||||
},
|
||||
fail: async () => {
|
||||
await chain.catch(() => undefined);
|
||||
await reporter.fail(transferred, total).catch(() => undefined);
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
async function runSshScript(
|
||||
config: SshConnectionConfig,
|
||||
script: string,
|
||||
@@ -493,6 +646,7 @@ async function streamLocalFileToSsh(input: {
|
||||
spec: SshConnectionConfig;
|
||||
localFile: string;
|
||||
remoteScript: string;
|
||||
progress?: TransferProgress;
|
||||
}): Promise<void> {
|
||||
const auth = await createSshAuthArgs(input.spec);
|
||||
const sshArgs = [
|
||||
@@ -525,7 +679,12 @@ async function streamLocalFileToSsh(input: {
|
||||
});
|
||||
source.on("error", fail);
|
||||
ssh.on("error", fail);
|
||||
source.pipe(ssh.stdin ?? null);
|
||||
if (input.progress) {
|
||||
input.progress.counter.on("error", fail);
|
||||
source.pipe(input.progress.counter).pipe(ssh.stdin ?? null);
|
||||
} else {
|
||||
source.pipe(ssh.stdin ?? null);
|
||||
}
|
||||
ssh.on("close", (code) => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
@@ -542,6 +701,7 @@ async function streamSshToLocalFile(input: {
|
||||
spec: SshConnectionConfig;
|
||||
remoteScript: string;
|
||||
localFile: string;
|
||||
progress?: TransferProgress;
|
||||
}): Promise<void> {
|
||||
const auth = await createSshAuthArgs(input.spec);
|
||||
const sshArgs = [
|
||||
@@ -569,7 +729,12 @@ async function streamSshToLocalFile(input: {
|
||||
reject(error);
|
||||
};
|
||||
|
||||
ssh.stdout?.pipe(sink);
|
||||
if (input.progress) {
|
||||
input.progress.counter.on("error", fail);
|
||||
ssh.stdout?.pipe(input.progress.counter).pipe(sink);
|
||||
} else {
|
||||
ssh.stdout?.pipe(sink);
|
||||
}
|
||||
ssh.stderr?.on("data", (chunk) => {
|
||||
sshStderr += String(chunk);
|
||||
});
|
||||
@@ -594,6 +759,7 @@ async function importGitWorkspaceToSsh(input: {
|
||||
localDir: string;
|
||||
remoteDir: string;
|
||||
snapshot: LocalGitWorkspaceSnapshot;
|
||||
onProgress?: RuntimeProgressSink;
|
||||
}): Promise<void> {
|
||||
const bundleDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-bundle-"));
|
||||
const bundlePath = path.join(bundleDir, "workspace.bundle");
|
||||
@@ -628,11 +794,31 @@ async function importGitWorkspaceToSsh(input: {
|
||||
`git -C ${shellQuote(input.remoteDir)} update-ref -d ${shellQuote(tempRef)} >/dev/null 2>&1 || true`,
|
||||
].join("\n");
|
||||
|
||||
await streamLocalFileToSsh({
|
||||
spec: input.spec,
|
||||
localFile: bundlePath,
|
||||
remoteScript: remoteSetupScript,
|
||||
});
|
||||
// The git bundle is a real local file of known size, so report an exact
|
||||
// percentage. No `workspace` label: the "Importing git history" phase is
|
||||
// already self-describing in the log line.
|
||||
const progress = input.onProgress
|
||||
? createTransferProgress({
|
||||
onProgress: input.onProgress,
|
||||
phase: "Importing git history",
|
||||
direction: "to",
|
||||
totalBytes: (await fs.stat(bundlePath)).size,
|
||||
estimated: false,
|
||||
})
|
||||
: null;
|
||||
|
||||
try {
|
||||
await streamLocalFileToSsh({
|
||||
spec: input.spec,
|
||||
localFile: bundlePath,
|
||||
remoteScript: remoteSetupScript,
|
||||
progress: progress ?? undefined,
|
||||
});
|
||||
await progress?.finish();
|
||||
} catch (error) {
|
||||
await progress?.fail();
|
||||
throw error;
|
||||
}
|
||||
} finally {
|
||||
await runLocalGit(input.localDir, ["update-ref", "-d", tempRef], {
|
||||
timeout: 10_000,
|
||||
@@ -648,6 +834,7 @@ async function exportGitWorkspaceFromSsh(input: {
|
||||
localDir: string;
|
||||
importedRef?: string;
|
||||
resetLocalWorkspace?: boolean;
|
||||
onProgress?: RuntimeProgressSink;
|
||||
}): Promise<string> {
|
||||
const bundleDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-bundle-"));
|
||||
const bundlePath = path.join(bundleDir, "workspace.bundle");
|
||||
@@ -665,11 +852,30 @@ async function exportGitWorkspaceFromSsh(input: {
|
||||
'cat "$tmp_bundle"',
|
||||
].join("\n");
|
||||
|
||||
await streamSshToLocalFile({
|
||||
spec: input.spec,
|
||||
remoteScript: exportScript,
|
||||
localFile: bundlePath,
|
||||
});
|
||||
// The remote bundle size isn't known before streaming, so report bytes
|
||||
// received (MB mode) with a terminal completion line.
|
||||
const progress = input.onProgress
|
||||
? createTransferProgress({
|
||||
onProgress: input.onProgress,
|
||||
phase: "Exporting git history",
|
||||
direction: "from",
|
||||
totalBytes: null,
|
||||
estimated: false,
|
||||
})
|
||||
: null;
|
||||
|
||||
try {
|
||||
await streamSshToLocalFile({
|
||||
spec: input.spec,
|
||||
remoteScript: exportScript,
|
||||
localFile: bundlePath,
|
||||
progress: progress ?? undefined,
|
||||
});
|
||||
await progress?.finish();
|
||||
} catch (error) {
|
||||
await progress?.fail();
|
||||
throw error;
|
||||
}
|
||||
|
||||
await runLocalGit(input.localDir, ["fetch", "--force", bundlePath, `refs/paperclip/ssh-sync/export:${importedRef}`], {
|
||||
timeout: 60_000,
|
||||
@@ -1049,6 +1255,8 @@ export async function syncDirectoryToSsh(input: {
|
||||
remoteDir: string;
|
||||
exclude?: string[];
|
||||
followSymlinks?: boolean;
|
||||
onProgress?: RuntimeProgressSink;
|
||||
progressLabel?: string;
|
||||
}): Promise<void> {
|
||||
const auth = await createSshAuthArgs(input.spec);
|
||||
const sshArgs = [
|
||||
@@ -1059,7 +1267,27 @@ export async function syncDirectoryToSsh(input: {
|
||||
`sh -c ${shellQuote(`mkdir -p ${shellQuote(input.remoteDir)} && tar -xf - -C ${shellQuote(input.remoteDir)}`)}`,
|
||||
];
|
||||
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
// tar's archive size isn't known until tar finishes, so estimate it from the
|
||||
// local file sizes and clamp the reported percent to 99% until the pipe closes.
|
||||
// The estimate walk runs concurrently with the transfer so it never delays the
|
||||
// pipe from opening on large workspaces.
|
||||
const progress = input.onProgress
|
||||
? createTransferProgress({
|
||||
onProgress: input.onProgress,
|
||||
phase: "Syncing",
|
||||
direction: "to",
|
||||
label: input.progressLabel,
|
||||
totalBytes: estimateLocalDirSize({
|
||||
localDir: input.localDir,
|
||||
exclude: input.exclude,
|
||||
followSymlinks: input.followSymlinks,
|
||||
}),
|
||||
estimated: true,
|
||||
})
|
||||
: null;
|
||||
|
||||
try {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const tarArgs = [
|
||||
...(input.followSymlinks ? ["-h"] : []),
|
||||
"-C",
|
||||
@@ -1111,7 +1339,12 @@ export async function syncDirectoryToSsh(input: {
|
||||
reject(error);
|
||||
};
|
||||
|
||||
tar.stdout?.pipe(ssh.stdin ?? null);
|
||||
if (progress) {
|
||||
progress.counter.on("error", fail);
|
||||
tar.stdout?.pipe(progress.counter).pipe(ssh.stdin ?? null);
|
||||
} else {
|
||||
tar.stdout?.pipe(ssh.stdin ?? null);
|
||||
}
|
||||
tar.stderr?.on("data", (chunk) => {
|
||||
tarStderr += String(chunk);
|
||||
});
|
||||
@@ -1131,7 +1364,12 @@ export async function syncDirectoryToSsh(input: {
|
||||
sshExitCode = code;
|
||||
maybeFinish();
|
||||
});
|
||||
}).finally(auth.cleanup);
|
||||
}).finally(auth.cleanup);
|
||||
await progress?.finish();
|
||||
} catch (error) {
|
||||
await progress?.fail();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function syncDirectoryFromSsh(input: {
|
||||
@@ -1140,6 +1378,8 @@ export async function syncDirectoryFromSsh(input: {
|
||||
localDir: string;
|
||||
exclude?: string[];
|
||||
preserveLocalEntries?: string[];
|
||||
onProgress?: RuntimeProgressSink;
|
||||
progressLabel?: string;
|
||||
}): Promise<void> {
|
||||
const auth = await createSshAuthArgs(input.spec);
|
||||
const stagingDir = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-ssh-sync-back-"));
|
||||
@@ -1155,6 +1395,21 @@ export async function syncDirectoryFromSsh(input: {
|
||||
`sh -c ${shellQuote(remoteTarScript)}`,
|
||||
];
|
||||
|
||||
// The remote tar size isn't known locally, so probe the remote directory for
|
||||
// an estimate (clamped to 99%). The probe runs concurrently with the transfer
|
||||
// so its round-trip never delays the restore; when it is unavailable we report
|
||||
// bytes received in MB mode with a terminal completion line.
|
||||
const progress = input.onProgress
|
||||
? createTransferProgress({
|
||||
onProgress: input.onProgress,
|
||||
phase: "Restoring",
|
||||
direction: "from",
|
||||
label: input.progressLabel,
|
||||
totalBytes: probeRemoteDirSize({ spec: input.spec, remoteDir: input.remoteDir }),
|
||||
estimated: true,
|
||||
})
|
||||
: null;
|
||||
|
||||
try {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const ssh = spawn("ssh", sshArgs, {
|
||||
@@ -1195,7 +1450,12 @@ export async function syncDirectoryFromSsh(input: {
|
||||
reject(error);
|
||||
};
|
||||
|
||||
ssh.stdout?.pipe(tar.stdin ?? null);
|
||||
if (progress) {
|
||||
progress.counter.on("error", fail);
|
||||
ssh.stdout?.pipe(progress.counter).pipe(tar.stdin ?? null);
|
||||
} else {
|
||||
ssh.stdout?.pipe(tar.stdin ?? null);
|
||||
}
|
||||
ssh.stderr?.on("data", (chunk) => {
|
||||
sshStderr += String(chunk);
|
||||
});
|
||||
@@ -1216,9 +1476,13 @@ export async function syncDirectoryFromSsh(input: {
|
||||
maybeFinish();
|
||||
});
|
||||
});
|
||||
await progress?.finish();
|
||||
|
||||
await clearLocalDirectory(input.localDir, input.preserveLocalEntries);
|
||||
await copyDirectoryContents(stagingDir, input.localDir);
|
||||
} catch (error) {
|
||||
await progress?.fail();
|
||||
throw error;
|
||||
} finally {
|
||||
await fs.rm(stagingDir, { recursive: true, force: true }).catch(() => undefined);
|
||||
await auth.cleanup();
|
||||
@@ -1229,6 +1493,7 @@ export async function prepareWorkspaceForSshExecution(input: {
|
||||
spec: SshRemoteExecutionSpec;
|
||||
localDir: string;
|
||||
remoteDir?: string;
|
||||
onProgress?: RuntimeProgressSink;
|
||||
}): Promise<{ gitBacked: boolean }> {
|
||||
const remoteDir = input.remoteDir ?? input.spec.remoteCwd;
|
||||
const gitSnapshot = await readLocalGitWorkspaceSnapshot(input.localDir);
|
||||
@@ -1239,12 +1504,15 @@ export async function prepareWorkspaceForSshExecution(input: {
|
||||
localDir: input.localDir,
|
||||
remoteDir,
|
||||
snapshot: gitSnapshot,
|
||||
onProgress: input.onProgress,
|
||||
});
|
||||
await syncDirectoryToSsh({
|
||||
spec: input.spec,
|
||||
localDir: input.localDir,
|
||||
remoteDir,
|
||||
exclude: [".git", ".paperclip-runtime"],
|
||||
onProgress: input.onProgress,
|
||||
progressLabel: "workspace",
|
||||
});
|
||||
await removeDeletedPathsOnSsh({
|
||||
spec: input.spec,
|
||||
@@ -1264,6 +1532,8 @@ export async function prepareWorkspaceForSshExecution(input: {
|
||||
localDir: input.localDir,
|
||||
remoteDir,
|
||||
exclude: [".paperclip-runtime"],
|
||||
onProgress: input.onProgress,
|
||||
progressLabel: "workspace",
|
||||
});
|
||||
return { gitBacked: false };
|
||||
}
|
||||
@@ -1274,6 +1544,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
remoteDir?: string;
|
||||
baselineSnapshot?: DirectorySnapshot;
|
||||
restoreGitHistory?: boolean;
|
||||
onProgress?: RuntimeProgressSink;
|
||||
}): Promise<void> {
|
||||
const remoteDir = input.remoteDir ?? input.spec.remoteCwd;
|
||||
if (input.baselineSnapshot) {
|
||||
@@ -1289,6 +1560,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
localDir: input.localDir,
|
||||
importedRef: importedRef ?? undefined,
|
||||
resetLocalWorkspace: false,
|
||||
onProgress: input.onProgress,
|
||||
})
|
||||
: null;
|
||||
await syncDirectoryFromSsh({
|
||||
@@ -1296,6 +1568,8 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
remoteDir,
|
||||
localDir: stagingDir,
|
||||
exclude: input.baselineSnapshot.exclude,
|
||||
onProgress: input.onProgress,
|
||||
progressLabel: "workspace",
|
||||
});
|
||||
await mergeDirectoryWithBaseline({
|
||||
baseline: input.baselineSnapshot,
|
||||
@@ -1330,6 +1604,7 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
spec: input.spec,
|
||||
remoteDir,
|
||||
localDir: input.localDir,
|
||||
onProgress: input.onProgress,
|
||||
});
|
||||
await syncDirectoryFromSsh({
|
||||
spec: input.spec,
|
||||
@@ -1337,6 +1612,8 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
localDir: input.localDir,
|
||||
exclude: [".git", ".paperclip-runtime"],
|
||||
preserveLocalEntries: [".git"],
|
||||
onProgress: input.onProgress,
|
||||
progressLabel: "workspace",
|
||||
});
|
||||
return;
|
||||
}
|
||||
@@ -1346,6 +1623,8 @@ export async function restoreWorkspaceFromSshExecution(input: {
|
||||
remoteDir,
|
||||
localDir: input.localDir,
|
||||
exclude: [".paperclip-runtime"],
|
||||
onProgress: input.onProgress,
|
||||
progressLabel: "workspace",
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user