feat(codex-local): env-driven gateway routing via PAPERCLIP_CODEX_PROVIDERS config.toml (#7919)
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The `codex-local` adapter runs the OpenAI Codex CLI; Paperclip already maintains a managed `CODEX_HOME` per company and ships it to remote/sandboxed execution targets > - Deployments increasingly put an OpenAI-compatible LLM gateway between the harness and the model for cost, governance, or data-residency reasons: LiteLLM, OpenRouter, Portkey, Kong, a corporate proxy, self-hosted models (vLLM/Ollama), or region-pinned/sovereign endpoints. But Codex has no CLI flag or env var for a custom endpoint: its only mechanism is `[model_providers.<id>]` tables (with `base_url`, `env_key`, `wire_api`) in `$CODEX_HOME/config.toml`, selected by a root-level `model_provider` key > - Today there is no supported way to get such provider config into the managed `CODEX_HOME`, so gateway routing requires hand-editing files the adapter owns and regenerates > - This pull request adds the codex analogue of #7837's opencode mechanism: a `PAPERCLIP_CODEX_PROVIDERS` JSON env var whose shape maps 1:1 onto codex's TOML schema, merged into the managed `config.toml` so the existing asset-shipping + `env.CODEX_HOME` mechanics deliver it to local and sandboxed runs alike; nothing here is specific to one hosting setup > - The benefit is Codex works behind any OpenAI-compatible gateway with config only; with no env set, behavior is unchanged ## Linked Issues or Issue Description No existing issue; describing in-PR (feature / adapter enhancement). - **Gap:** there is no supported way to register a custom/gateway `[model_providers.*]` endpoint for `codex-local`. Codex's only custom-endpoint mechanism is `config.toml` (`base_url` + `env_key` + `wire_api`, selected via the root `model_provider` key), and the adapter owns/regenerates the managed `CODEX_HOME`, so operators cannot durably hand-edit it. - Related: #7837 (the opencode-local analogue of this change, same env-driven gateway-routing pattern). Searched for duplicate/related PRs: no existing codex-local gateway/provider-routing PR found. > Note on ROADMAP: this is adapter-level, opt-in config (defaults unchanged) that *enables* gateway routing for one harness; it is not the core "Cloud / Sandbox agents" platform work itself. ## What Changed - New `prepareCodexRuntimeConfig()` (`packages/adapters/codex-local/src/server/runtime-config.ts`): reads `PAPERCLIP_CODEX_PROVIDERS` (run env first, then `process.env`), shaped as `{"providers": {"<id>": {base_url, env_key, wire_api, ...}}, "model_provider": "<id>"}`, and merges it into the managed `CODEX_HOME`'s `config.toml`. No-op when unset or empty. - A malformed value (invalid JSON, not a JSON object, no `providers` object, no usable provider entries, or individual entries with empty names or non-object values, which are skipped by name) is never silently dropped: each case surfaces a distinct, user-visible note (via the prepare notes, which flow into command notes + `onLog`) and unusable input leaves `config.toml` untouched. - Merge is marker-delimited and TOML-correct: existing `config.toml` content is preserved between two managed blocks. Root keys (e.g. `model_provider`) are prepended **before the first table header** (TOML root-region rule), `[model_providers.*]` tables are appended. Pre-existing same-name provider sections and root `model_provider` keys are excised so the managed definitions win without duplicate-table parse errors. - `{env:VAR}` placeholders are expanded server-side for literal-credential fields; `env_key` indirection remains the preferred path. - Crash-safe restore: prepare writes a pre-run backup (`config.toml.paperclip-backup`) before the merged file; `cleanup()` restores the original in the execute `finally` and removes the backup. If a run never reaches `cleanup()` (a throw during the setup between prepare and execution, or SIGKILL), the next prepare restores the original from the backup with full fidelity, including user `[model_providers.*]` sections the merge excised (review feedback, P2); plain block-stripping remains the fallback for pre-backup state. - An explicit adapter-config `env.CODEX_HOME` override is treated as user-managed: no merge, surfaced as a command note. - Dependency-free hand-emitted TOML (strings/numbers/booleans, arrays of scalars, plain objects as inline tables); basic strings escape U+0000-U+001F and U+007F per TOML 1.0 (review feedback, P2). Merged output was additionally validated locally with python tomllib during development; the committed tests assert the structural invariants. - `execute.ts` wiring: `prepareCodexRuntimeConfig` runs after `prepareManagedCodexHome` (before the home ships to the remote target), notes surface via `onLog` + command notes, and the `finally` calls `cleanup()`. **Note for reviewers:** current codex removed `wire_api = "chat"` (openai/codex#10157, Feb 2026), so gateway provider configs must use `wire_api = "responses"`, i.e. the gateway must speak `/v1/responses`. The adapter passes the value through verbatim; this is a codex-side constraint worth knowing when configuring it. ## Verification - `pnpm --filter @paperclipai/adapter-codex-local build` and `typecheck`: tsc clean against current `master` - `pnpm exec vitest run packages/adapters/codex-local`: 45 passing (incl. 17 `runtime-config` tests: fresh-merge + cleanup restore, root-region placement, same-name provider override, inline tables/arrays, DEL escaping, `{env:}` expansion from run env + `process.env`, per-case malformed-input notes with `config.toml` untouched, skipped-entry notes alongside a successful merge, silent no-op when unset/empty, explicit-`CODEX_HOME` skip note, backup restore of excised user sections after an interrupted run, backup removal on cleanup, stale-block self-heal, re-run replacement) - Verified end-to-end: a codex agent in a hardened Kubernetes (gVisor) sandbox completed a real task routed through an OpenAI-compatible gateway's `/v1/responses`, with a billed usage row recorded on the gateway. That deployment supplies the verification evidence; the mechanism is gateway-agnostic. ## Risks Low. Entirely env-driven and opt-in; with `PAPERCLIP_CODEX_PROVIDERS` unset the adapter never touches `config.toml` and behavior is byte-identical to before. The merge preserves user content, restores the original file on cleanup, and survives interrupted runs via the pre-run backup; malformed input surfaces a visible note and is ignored without touching `config.toml`. No migration/UI impact. ## Model Used Claude Opus 4.8 (`claude-opus-4-8`, 1M context), extended thinking + tool use, via Claude Code. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md (adapter-level opt-in config enabling gateway routing; not the core sandbox-platform work, noted above) - [x] I have searched GitHub for duplicate or related PRs and linked them above (#7837 is the opencode analogue; no codex-local duplicate found) - [x] I have either (a) linked existing issues OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (n/a, no UI) - [ ] I have updated relevant documentation to reflect my changes (env var documented inline; no central doc references the adapter env yet) - [x] I have considered and documented any risks above - [x] All Paperclip CI gates are green (green on the previous head; re-running on the final note-copy polish commit) - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (both review P2s are fixed at head: the interrupted-run restore via the pre-run backup and the U+007F escaping; a re-review is requested for the note-copy polish) - [x] I will address all Greptile and reviewer comments before requesting merge 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jannes Stubbemann
committed by
GitHub
GitHub
parent
6e4aca9c67
commit
9e750d3e92
@@ -45,6 +45,7 @@ import {
|
|||||||
isCodexUnknownSessionError,
|
isCodexUnknownSessionError,
|
||||||
} from "./parse.js";
|
} from "./parse.js";
|
||||||
import { pathExists, prepareManagedCodexHome, resolveManagedCodexHomeDir, resolveSharedCodexHomeDir } from "./codex-home.js";
|
import { pathExists, prepareManagedCodexHome, resolveManagedCodexHomeDir, resolveSharedCodexHomeDir } from "./codex-home.js";
|
||||||
|
import { prepareCodexRuntimeConfig } from "./runtime-config.js";
|
||||||
import { resolveCodexDesiredSkillNames } from "./skills.js";
|
import { resolveCodexDesiredSkillNames } from "./skills.js";
|
||||||
import { buildCodexExecArgs } from "./codex-args.js";
|
import { buildCodexExecArgs } from "./codex-args.js";
|
||||||
import { SANDBOX_INSTALL_COMMAND } from "../index.js";
|
import { SANDBOX_INSTALL_COMMAND } from "../index.js";
|
||||||
@@ -348,276 +349,306 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
|||||||
const defaultCodexHome = resolveManagedCodexHomeDir(process.env, agent.companyId);
|
const defaultCodexHome = resolveManagedCodexHomeDir(process.env, agent.companyId);
|
||||||
const effectiveCodexHome = configuredCodexHome ?? preparedManagedCodexHome ?? defaultCodexHome;
|
const effectiveCodexHome = configuredCodexHome ?? preparedManagedCodexHome ?? defaultCodexHome;
|
||||||
await fs.mkdir(effectiveCodexHome, { recursive: true });
|
await fs.mkdir(effectiveCodexHome, { recursive: true });
|
||||||
// Inject skills into the same CODEX_HOME that Codex will actually run with
|
// Merge custom model providers (PAPERCLIP_CODEX_PROVIDERS) into the managed
|
||||||
// (managed home in the default case, or an explicit override from adapter config).
|
// CODEX_HOME's config.toml BEFORE the home is shipped to a remote execution
|
||||||
const codexSkillsDir = resolveCodexSkillsDir(effectiveCodexHome);
|
// target, so both local and sandboxed Codex processes pick up the routing.
|
||||||
await ensureCodexSkillsInjected(
|
// An explicit env.CODEX_HOME override is treated as user-managed and skipped.
|
||||||
onLog,
|
const envConfigStrings = Object.fromEntries(
|
||||||
{
|
Object.entries(envConfig).filter(
|
||||||
skillsHome: codexSkillsDir,
|
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||||
skillsEntries: codexSkillEntries,
|
),
|
||||||
desiredSkillNames,
|
|
||||||
},
|
|
||||||
);
|
);
|
||||||
const timeoutSec = resolveAdapterExecutionTargetTimeoutSec(
|
const preparedRuntimeConfig = await prepareCodexRuntimeConfig({
|
||||||
executionTarget,
|
env: envConfigStrings,
|
||||||
asNumber(config.timeoutSec, 0),
|
codexHome: configuredCodexHome ? null : effectiveCodexHome,
|
||||||
);
|
|
||||||
const graceSec = asNumber(config.graceSec, 20);
|
|
||||||
let effectiveExecutionCwd = adapterExecutionTargetRemoteCwd(executionTarget, cwd);
|
|
||||||
const preparedExecutionTargetRuntime = executionTargetIsRemote
|
|
||||||
? await (async () => {
|
|
||||||
await onLog(
|
|
||||||
"stdout",
|
|
||||||
`[paperclip] Syncing workspace and CODEX_HOME to ${describeAdapterExecutionTarget(executionTarget)}.\n`,
|
|
||||||
);
|
|
||||||
return await prepareAdapterExecutionTargetRuntime({
|
|
||||||
runId,
|
|
||||||
target: executionTarget,
|
|
||||||
adapterKey: "codex",
|
|
||||||
timeoutSec,
|
|
||||||
workspaceLocalDir: cwd,
|
|
||||||
installCommand: SANDBOX_INSTALL_COMMAND,
|
|
||||||
detectCommand: command,
|
|
||||||
assets: [
|
|
||||||
{
|
|
||||||
key: "home",
|
|
||||||
localDir: effectiveCodexHome,
|
|
||||||
followSymlinks: true,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
});
|
|
||||||
})()
|
|
||||||
: null;
|
|
||||||
if (preparedExecutionTargetRuntime?.workspaceRemoteDir) {
|
|
||||||
effectiveExecutionCwd = preparedExecutionTargetRuntime.workspaceRemoteDir;
|
|
||||||
}
|
|
||||||
const runtimeExecutionTarget = overrideAdapterExecutionTargetRemoteCwd(executionTarget, effectiveExecutionCwd);
|
|
||||||
const executionTargetIsSandbox =
|
|
||||||
runtimeExecutionTarget?.kind === "remote" && runtimeExecutionTarget.transport === "sandbox";
|
|
||||||
const restoreRemoteWorkspace = preparedExecutionTargetRuntime
|
|
||||||
? () => preparedExecutionTargetRuntime.restoreWorkspace()
|
|
||||||
: null;
|
|
||||||
let paperclipBridge: Awaited<ReturnType<typeof startAdapterExecutionTargetPaperclipBridge>> = null;
|
|
||||||
const remoteCodexHome = executionTargetIsRemote
|
|
||||||
? preparedExecutionTargetRuntime?.assetDirs.home ??
|
|
||||||
path.posix.join(effectiveExecutionCwd, ".paperclip-runtime", "codex", "home")
|
|
||||||
: null;
|
|
||||||
const hasExplicitApiKey =
|
|
||||||
typeof envConfig.PAPERCLIP_API_KEY === "string" && envConfig.PAPERCLIP_API_KEY.trim().length > 0;
|
|
||||||
const env: Record<string, string> = { ...buildPaperclipEnv(agent) };
|
|
||||||
env.PAPERCLIP_RUN_ID = runId;
|
|
||||||
const wakeTaskId =
|
|
||||||
(typeof context.taskId === "string" && context.taskId.trim().length > 0 && context.taskId.trim()) ||
|
|
||||||
(typeof context.issueId === "string" && context.issueId.trim().length > 0 && context.issueId.trim()) ||
|
|
||||||
null;
|
|
||||||
const wakeReason =
|
|
||||||
typeof context.wakeReason === "string" && context.wakeReason.trim().length > 0
|
|
||||||
? context.wakeReason.trim()
|
|
||||||
: null;
|
|
||||||
const wakeCommentId =
|
|
||||||
(typeof context.wakeCommentId === "string" && context.wakeCommentId.trim().length > 0 && context.wakeCommentId.trim()) ||
|
|
||||||
(typeof context.commentId === "string" && context.commentId.trim().length > 0 && context.commentId.trim()) ||
|
|
||||||
null;
|
|
||||||
const approvalId =
|
|
||||||
typeof context.approvalId === "string" && context.approvalId.trim().length > 0
|
|
||||||
? context.approvalId.trim()
|
|
||||||
: null;
|
|
||||||
const approvalStatus =
|
|
||||||
typeof context.approvalStatus === "string" && context.approvalStatus.trim().length > 0
|
|
||||||
? context.approvalStatus.trim()
|
|
||||||
: null;
|
|
||||||
const linkedIssueIds = Array.isArray(context.issueIds)
|
|
||||||
? context.issueIds.filter((value): value is string => typeof value === "string" && value.trim().length > 0)
|
|
||||||
: [];
|
|
||||||
const wakePayloadJson = stringifyPaperclipWakePayload(context.paperclipWake);
|
|
||||||
const issueWorkMode = readPaperclipIssueWorkModeFromContext(context);
|
|
||||||
if (wakeTaskId) {
|
|
||||||
env.PAPERCLIP_TASK_ID = wakeTaskId;
|
|
||||||
}
|
|
||||||
if (issueWorkMode) {
|
|
||||||
env.PAPERCLIP_ISSUE_WORK_MODE = issueWorkMode;
|
|
||||||
}
|
|
||||||
if (wakeReason) {
|
|
||||||
env.PAPERCLIP_WAKE_REASON = wakeReason;
|
|
||||||
}
|
|
||||||
if (wakeCommentId) {
|
|
||||||
env.PAPERCLIP_WAKE_COMMENT_ID = wakeCommentId;
|
|
||||||
}
|
|
||||||
if (approvalId) {
|
|
||||||
env.PAPERCLIP_APPROVAL_ID = approvalId;
|
|
||||||
}
|
|
||||||
if (approvalStatus) {
|
|
||||||
env.PAPERCLIP_APPROVAL_STATUS = approvalStatus;
|
|
||||||
}
|
|
||||||
if (linkedIssueIds.length > 0) {
|
|
||||||
env.PAPERCLIP_LINKED_ISSUE_IDS = linkedIssueIds.join(",");
|
|
||||||
}
|
|
||||||
if (wakePayloadJson) {
|
|
||||||
env.PAPERCLIP_WAKE_PAYLOAD_JSON = wakePayloadJson;
|
|
||||||
}
|
|
||||||
refreshPaperclipWorkspaceEnvForExecution({
|
|
||||||
env,
|
|
||||||
envConfig,
|
|
||||||
workspaceCwd: effectiveWorkspaceCwd,
|
|
||||||
workspaceSource,
|
|
||||||
workspaceStrategy,
|
|
||||||
workspaceId,
|
|
||||||
workspaceRepoUrl,
|
|
||||||
workspaceRepoRef,
|
|
||||||
workspaceBranch,
|
|
||||||
workspaceWorktreePath,
|
|
||||||
workspaceHints,
|
|
||||||
agentHome,
|
|
||||||
executionTargetIsRemote,
|
|
||||||
executionCwd: effectiveExecutionCwd,
|
|
||||||
});
|
});
|
||||||
if (runtimeServiceIntents.length > 0) {
|
try {
|
||||||
env.PAPERCLIP_RUNTIME_SERVICE_INTENTS_JSON = JSON.stringify(runtimeServiceIntents);
|
for (const note of preparedRuntimeConfig.notes) {
|
||||||
}
|
await onLog("stdout", `[paperclip] ${note}\n`);
|
||||||
if (runtimeServices.length > 0) {
|
}
|
||||||
env.PAPERCLIP_RUNTIME_SERVICES_JSON = JSON.stringify(runtimeServices);
|
// Inject skills into the same CODEX_HOME that Codex will actually run with
|
||||||
}
|
// (managed home in the default case, or an explicit override from adapter config).
|
||||||
if (runtimePrimaryUrl) {
|
const codexSkillsDir = resolveCodexSkillsDir(effectiveCodexHome);
|
||||||
env.PAPERCLIP_RUNTIME_PRIMARY_URL = runtimePrimaryUrl;
|
await ensureCodexSkillsInjected(
|
||||||
}
|
onLog,
|
||||||
env.CODEX_HOME = remoteCodexHome ?? effectiveCodexHome;
|
{
|
||||||
if (!hasExplicitApiKey && authToken) {
|
skillsHome: codexSkillsDir,
|
||||||
env.PAPERCLIP_API_KEY = authToken;
|
skillsEntries: codexSkillEntries,
|
||||||
}
|
desiredSkillNames,
|
||||||
if (executionTargetIsRemote && adapterExecutionTargetUsesPaperclipBridge(runtimeExecutionTarget)) {
|
},
|
||||||
paperclipBridge = await startAdapterExecutionTargetPaperclipBridge({
|
);
|
||||||
|
const timeoutSec = resolveAdapterExecutionTargetTimeoutSec(
|
||||||
|
executionTarget,
|
||||||
|
asNumber(config.timeoutSec, 0),
|
||||||
|
);
|
||||||
|
const graceSec = asNumber(config.graceSec, 20);
|
||||||
|
let effectiveExecutionCwd = adapterExecutionTargetRemoteCwd(executionTarget, cwd);
|
||||||
|
const preparedExecutionTargetRuntime = executionTargetIsRemote
|
||||||
|
? await (async () => {
|
||||||
|
await onLog(
|
||||||
|
"stdout",
|
||||||
|
`[paperclip] Syncing workspace and CODEX_HOME to ${describeAdapterExecutionTarget(executionTarget)}.\n`,
|
||||||
|
);
|
||||||
|
return await prepareAdapterExecutionTargetRuntime({
|
||||||
|
runId,
|
||||||
|
target: executionTarget,
|
||||||
|
adapterKey: "codex",
|
||||||
|
timeoutSec,
|
||||||
|
workspaceLocalDir: cwd,
|
||||||
|
installCommand: SANDBOX_INSTALL_COMMAND,
|
||||||
|
detectCommand: command,
|
||||||
|
assets: [
|
||||||
|
{
|
||||||
|
key: "home",
|
||||||
|
localDir: effectiveCodexHome,
|
||||||
|
followSymlinks: true,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
});
|
||||||
|
})()
|
||||||
|
: null;
|
||||||
|
if (preparedExecutionTargetRuntime?.workspaceRemoteDir) {
|
||||||
|
effectiveExecutionCwd = preparedExecutionTargetRuntime.workspaceRemoteDir;
|
||||||
|
}
|
||||||
|
const runtimeExecutionTarget = overrideAdapterExecutionTargetRemoteCwd(executionTarget, effectiveExecutionCwd);
|
||||||
|
const executionTargetIsSandbox =
|
||||||
|
runtimeExecutionTarget?.kind === "remote" && runtimeExecutionTarget.transport === "sandbox";
|
||||||
|
const restoreRemoteWorkspace = preparedExecutionTargetRuntime
|
||||||
|
? () => preparedExecutionTargetRuntime.restoreWorkspace()
|
||||||
|
: null;
|
||||||
|
let paperclipBridge: Awaited<ReturnType<typeof startAdapterExecutionTargetPaperclipBridge>> = null;
|
||||||
|
const remoteCodexHome = executionTargetIsRemote
|
||||||
|
? preparedExecutionTargetRuntime?.assetDirs.home ??
|
||||||
|
path.posix.join(effectiveExecutionCwd, ".paperclip-runtime", "codex", "home")
|
||||||
|
: null;
|
||||||
|
const hasExplicitApiKey =
|
||||||
|
typeof envConfig.PAPERCLIP_API_KEY === "string" && envConfig.PAPERCLIP_API_KEY.trim().length > 0;
|
||||||
|
const env: Record<string, string> = { ...buildPaperclipEnv(agent) };
|
||||||
|
env.PAPERCLIP_RUN_ID = runId;
|
||||||
|
const wakeTaskId =
|
||||||
|
(typeof context.taskId === "string" && context.taskId.trim().length > 0 && context.taskId.trim()) ||
|
||||||
|
(typeof context.issueId === "string" && context.issueId.trim().length > 0 && context.issueId.trim()) ||
|
||||||
|
null;
|
||||||
|
const wakeReason =
|
||||||
|
typeof context.wakeReason === "string" && context.wakeReason.trim().length > 0
|
||||||
|
? context.wakeReason.trim()
|
||||||
|
: null;
|
||||||
|
const wakeCommentId =
|
||||||
|
(typeof context.wakeCommentId === "string" && context.wakeCommentId.trim().length > 0 && context.wakeCommentId.trim()) ||
|
||||||
|
(typeof context.commentId === "string" && context.commentId.trim().length > 0 && context.commentId.trim()) ||
|
||||||
|
null;
|
||||||
|
const approvalId =
|
||||||
|
typeof context.approvalId === "string" && context.approvalId.trim().length > 0
|
||||||
|
? context.approvalId.trim()
|
||||||
|
: null;
|
||||||
|
const approvalStatus =
|
||||||
|
typeof context.approvalStatus === "string" && context.approvalStatus.trim().length > 0
|
||||||
|
? context.approvalStatus.trim()
|
||||||
|
: null;
|
||||||
|
const linkedIssueIds = Array.isArray(context.issueIds)
|
||||||
|
? context.issueIds.filter((value): value is string => typeof value === "string" && value.trim().length > 0)
|
||||||
|
: [];
|
||||||
|
const wakePayloadJson = stringifyPaperclipWakePayload(context.paperclipWake);
|
||||||
|
const issueWorkMode = readPaperclipIssueWorkModeFromContext(context);
|
||||||
|
if (wakeTaskId) {
|
||||||
|
env.PAPERCLIP_TASK_ID = wakeTaskId;
|
||||||
|
}
|
||||||
|
if (issueWorkMode) {
|
||||||
|
env.PAPERCLIP_ISSUE_WORK_MODE = issueWorkMode;
|
||||||
|
}
|
||||||
|
if (wakeReason) {
|
||||||
|
env.PAPERCLIP_WAKE_REASON = wakeReason;
|
||||||
|
}
|
||||||
|
if (wakeCommentId) {
|
||||||
|
env.PAPERCLIP_WAKE_COMMENT_ID = wakeCommentId;
|
||||||
|
}
|
||||||
|
if (approvalId) {
|
||||||
|
env.PAPERCLIP_APPROVAL_ID = approvalId;
|
||||||
|
}
|
||||||
|
if (approvalStatus) {
|
||||||
|
env.PAPERCLIP_APPROVAL_STATUS = approvalStatus;
|
||||||
|
}
|
||||||
|
if (linkedIssueIds.length > 0) {
|
||||||
|
env.PAPERCLIP_LINKED_ISSUE_IDS = linkedIssueIds.join(",");
|
||||||
|
}
|
||||||
|
if (wakePayloadJson) {
|
||||||
|
env.PAPERCLIP_WAKE_PAYLOAD_JSON = wakePayloadJson;
|
||||||
|
}
|
||||||
|
refreshPaperclipWorkspaceEnvForExecution({
|
||||||
|
env,
|
||||||
|
envConfig,
|
||||||
|
workspaceCwd: effectiveWorkspaceCwd,
|
||||||
|
workspaceSource,
|
||||||
|
workspaceStrategy,
|
||||||
|
workspaceId,
|
||||||
|
workspaceRepoUrl,
|
||||||
|
workspaceRepoRef,
|
||||||
|
workspaceBranch,
|
||||||
|
workspaceWorktreePath,
|
||||||
|
workspaceHints,
|
||||||
|
agentHome,
|
||||||
|
executionTargetIsRemote,
|
||||||
|
executionCwd: effectiveExecutionCwd,
|
||||||
|
});
|
||||||
|
if (runtimeServiceIntents.length > 0) {
|
||||||
|
env.PAPERCLIP_RUNTIME_SERVICE_INTENTS_JSON = JSON.stringify(runtimeServiceIntents);
|
||||||
|
}
|
||||||
|
if (runtimeServices.length > 0) {
|
||||||
|
env.PAPERCLIP_RUNTIME_SERVICES_JSON = JSON.stringify(runtimeServices);
|
||||||
|
}
|
||||||
|
if (runtimePrimaryUrl) {
|
||||||
|
env.PAPERCLIP_RUNTIME_PRIMARY_URL = runtimePrimaryUrl;
|
||||||
|
}
|
||||||
|
env.CODEX_HOME = remoteCodexHome ?? effectiveCodexHome;
|
||||||
|
if (!hasExplicitApiKey && authToken) {
|
||||||
|
env.PAPERCLIP_API_KEY = authToken;
|
||||||
|
}
|
||||||
|
if (executionTargetIsRemote && adapterExecutionTargetUsesPaperclipBridge(runtimeExecutionTarget)) {
|
||||||
|
paperclipBridge = await startAdapterExecutionTargetPaperclipBridge({
|
||||||
|
runId,
|
||||||
|
target: runtimeExecutionTarget,
|
||||||
|
runtimeRootDir: preparedExecutionTargetRuntime?.runtimeRootDir,
|
||||||
|
adapterKey: "codex",
|
||||||
|
timeoutSec,
|
||||||
|
hostApiToken: env.PAPERCLIP_API_KEY,
|
||||||
|
onLog,
|
||||||
|
});
|
||||||
|
if (paperclipBridge) {
|
||||||
|
Object.assign(env, paperclipBridge.env);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const effectiveEnv = Object.fromEntries(
|
||||||
|
Object.entries({ ...process.env, ...env }).filter(
|
||||||
|
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||||
|
),
|
||||||
|
);
|
||||||
|
const billingType = resolveCodexBillingType(effectiveEnv);
|
||||||
|
const runtimeEnv = Object.fromEntries(
|
||||||
|
Object.entries(ensurePathInEnv(effectiveEnv)).filter(
|
||||||
|
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||||
|
),
|
||||||
|
);
|
||||||
|
await ensureAdapterExecutionTargetRuntimeCommandInstalled({
|
||||||
runId,
|
runId,
|
||||||
target: runtimeExecutionTarget,
|
target: executionTarget,
|
||||||
runtimeRootDir: preparedExecutionTargetRuntime?.runtimeRootDir,
|
installCommand: ctx.runtimeCommandSpec?.installCommand,
|
||||||
adapterKey: "codex",
|
detectCommand: ctx.runtimeCommandSpec?.detectCommand,
|
||||||
|
cwd,
|
||||||
|
env: runtimeEnv,
|
||||||
timeoutSec,
|
timeoutSec,
|
||||||
hostApiToken: env.PAPERCLIP_API_KEY,
|
graceSec,
|
||||||
onLog,
|
onLog,
|
||||||
});
|
});
|
||||||
if (paperclipBridge) {
|
await ensureAdapterExecutionTargetCommandResolvable(command, executionTarget, cwd, runtimeEnv);
|
||||||
Object.assign(env, paperclipBridge.env);
|
const resolvedCommand = await resolveAdapterExecutionTargetCommandForLogs(command, executionTarget, cwd, runtimeEnv);
|
||||||
}
|
const loggedEnv = buildInvocationEnvForLogs(env, {
|
||||||
}
|
runtimeEnv,
|
||||||
const effectiveEnv = Object.fromEntries(
|
includeRuntimeKeys: ["HOME"],
|
||||||
Object.entries({ ...process.env, ...env }).filter(
|
resolvedCommand,
|
||||||
(entry): entry is [string, string] => typeof entry[1] === "string",
|
});
|
||||||
),
|
|
||||||
);
|
|
||||||
const billingType = resolveCodexBillingType(effectiveEnv);
|
|
||||||
const runtimeEnv = Object.fromEntries(
|
|
||||||
Object.entries(ensurePathInEnv(effectiveEnv)).filter(
|
|
||||||
(entry): entry is [string, string] => typeof entry[1] === "string",
|
|
||||||
),
|
|
||||||
);
|
|
||||||
await ensureAdapterExecutionTargetRuntimeCommandInstalled({
|
|
||||||
runId,
|
|
||||||
target: executionTarget,
|
|
||||||
installCommand: ctx.runtimeCommandSpec?.installCommand,
|
|
||||||
detectCommand: ctx.runtimeCommandSpec?.detectCommand,
|
|
||||||
cwd,
|
|
||||||
env: runtimeEnv,
|
|
||||||
timeoutSec,
|
|
||||||
graceSec,
|
|
||||||
onLog,
|
|
||||||
});
|
|
||||||
await ensureAdapterExecutionTargetCommandResolvable(command, executionTarget, cwd, runtimeEnv);
|
|
||||||
const resolvedCommand = await resolveAdapterExecutionTargetCommandForLogs(command, executionTarget, cwd, runtimeEnv);
|
|
||||||
const loggedEnv = buildInvocationEnvForLogs(env, {
|
|
||||||
runtimeEnv,
|
|
||||||
includeRuntimeKeys: ["HOME"],
|
|
||||||
resolvedCommand,
|
|
||||||
});
|
|
||||||
|
|
||||||
const runtimeSessionParams = parseObject(runtime.sessionParams);
|
const runtimeSessionParams = parseObject(runtime.sessionParams);
|
||||||
const runtimeSessionId = asString(runtimeSessionParams.sessionId, runtime.sessionId ?? "");
|
const runtimeSessionId = asString(runtimeSessionParams.sessionId, runtime.sessionId ?? "");
|
||||||
const runtimeSessionCwd = asString(runtimeSessionParams.cwd, "");
|
const runtimeSessionCwd = asString(runtimeSessionParams.cwd, "");
|
||||||
const runtimeRemoteExecution = parseObject(runtimeSessionParams.remoteExecution);
|
const runtimeRemoteExecution = parseObject(runtimeSessionParams.remoteExecution);
|
||||||
const canResumeSession =
|
const canResumeSession =
|
||||||
runtimeSessionId.length > 0 &&
|
runtimeSessionId.length > 0 &&
|
||||||
(runtimeSessionCwd.length === 0 || path.resolve(runtimeSessionCwd) === path.resolve(effectiveExecutionCwd)) &&
|
(runtimeSessionCwd.length === 0 || path.resolve(runtimeSessionCwd) === path.resolve(effectiveExecutionCwd)) &&
|
||||||
adapterExecutionTargetSessionMatches(runtimeRemoteExecution, runtimeExecutionTarget);
|
adapterExecutionTargetSessionMatches(runtimeRemoteExecution, runtimeExecutionTarget);
|
||||||
const codexTransientFallbackMode = readCodexTransientFallbackMode(context);
|
const codexTransientFallbackMode = readCodexTransientFallbackMode(context);
|
||||||
const forceSaferInvocation = fallbackModeUsesSaferInvocation(codexTransientFallbackMode);
|
const forceSaferInvocation = fallbackModeUsesSaferInvocation(codexTransientFallbackMode);
|
||||||
const forceFreshSession = fallbackModeUsesFreshSession(codexTransientFallbackMode);
|
const forceFreshSession = fallbackModeUsesFreshSession(codexTransientFallbackMode);
|
||||||
const sessionId = canResumeSession && !forceFreshSession ? runtimeSessionId : null;
|
const sessionId = canResumeSession && !forceFreshSession ? runtimeSessionId : null;
|
||||||
if (executionTargetIsRemote && runtimeSessionId && !canResumeSession) {
|
if (executionTargetIsRemote && runtimeSessionId && !canResumeSession) {
|
||||||
await onLog(
|
|
||||||
"stdout",
|
|
||||||
`[paperclip] Codex session "${runtimeSessionId}" does not match the current remote execution identity and will not be resumed in "${effectiveExecutionCwd}". Starting a fresh remote session.\n`,
|
|
||||||
);
|
|
||||||
} else if (runtimeSessionId && !canResumeSession) {
|
|
||||||
await onLog(
|
|
||||||
"stdout",
|
|
||||||
`[paperclip] Codex session "${runtimeSessionId}" was saved for cwd "${runtimeSessionCwd}" and will not be resumed in "${effectiveExecutionCwd}".\n`,
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const instructionsFilePath = asString(config.instructionsFilePath, "").trim();
|
|
||||||
const instructionsDir = instructionsFilePath ? `${path.dirname(instructionsFilePath)}/` : "";
|
|
||||||
let instructionsPrefix = "";
|
|
||||||
let instructionsChars = 0;
|
|
||||||
if (instructionsFilePath) {
|
|
||||||
try {
|
|
||||||
const instructionsContents = await fs.readFile(instructionsFilePath, "utf8");
|
|
||||||
instructionsPrefix =
|
|
||||||
`${instructionsContents}\n\n` +
|
|
||||||
`The above agent instructions were loaded from ${instructionsFilePath}. ` +
|
|
||||||
`Resolve any relative file references from ${instructionsDir}.\n\n`;
|
|
||||||
instructionsChars = instructionsPrefix.length;
|
|
||||||
} catch (err) {
|
|
||||||
const reason = err instanceof Error ? err.message : String(err);
|
|
||||||
await onLog(
|
await onLog(
|
||||||
"stdout",
|
"stdout",
|
||||||
`[paperclip] Warning: could not read agent instructions file "${instructionsFilePath}": ${reason}\n`,
|
`[paperclip] Codex session "${runtimeSessionId}" does not match the current remote execution identity and will not be resumed in "${effectiveExecutionCwd}". Starting a fresh remote session.\n`,
|
||||||
|
);
|
||||||
|
} else if (runtimeSessionId && !canResumeSession) {
|
||||||
|
await onLog(
|
||||||
|
"stdout",
|
||||||
|
`[paperclip] Codex session "${runtimeSessionId}" was saved for cwd "${runtimeSessionCwd}" and will not be resumed in "${effectiveExecutionCwd}".\n`,
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
const instructionsFilePath = asString(config.instructionsFilePath, "").trim();
|
||||||
const repoAgentsNote =
|
const instructionsDir = instructionsFilePath ? `${path.dirname(instructionsFilePath)}/` : "";
|
||||||
"Codex exec automatically applies repo-scoped AGENTS.md instructions from the current workspace; Paperclip does not currently suppress that discovery.";
|
let instructionsPrefix = "";
|
||||||
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
let instructionsChars = 0;
|
||||||
const templateData = {
|
if (instructionsFilePath) {
|
||||||
agentId: agent.id,
|
try {
|
||||||
companyId: agent.companyId,
|
const instructionsContents = await fs.readFile(instructionsFilePath, "utf8");
|
||||||
runId,
|
instructionsPrefix =
|
||||||
company: { id: agent.companyId },
|
`${instructionsContents}\n\n` +
|
||||||
agent,
|
`The above agent instructions were loaded from ${instructionsFilePath}. ` +
|
||||||
run: { id: runId, source: "on_demand" },
|
`Resolve any relative file references from ${instructionsDir}.\n\n`;
|
||||||
context,
|
instructionsChars = instructionsPrefix.length;
|
||||||
};
|
} catch (err) {
|
||||||
const renderedBootstrapPrompt =
|
const reason = err instanceof Error ? err.message : String(err);
|
||||||
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
await onLog(
|
||||||
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
"stdout",
|
||||||
: "";
|
`[paperclip] Warning: could not read agent instructions file "${instructionsFilePath}": ${reason}\n`,
|
||||||
const wakePrompt = renderPaperclipWakePrompt(context.paperclipWake, { resumedSession: Boolean(sessionId) });
|
);
|
||||||
const shouldUseResumeDeltaPrompt = Boolean(sessionId) && wakePrompt.length > 0;
|
|
||||||
const promptInstructionsPrefix = shouldUseResumeDeltaPrompt ? "" : instructionsPrefix;
|
|
||||||
instructionsChars = promptInstructionsPrefix.length;
|
|
||||||
const continuationSummary = parseObject(context.paperclipContinuationSummary);
|
|
||||||
const continuationSummaryBody = asString(continuationSummary.body, "").trim() || null;
|
|
||||||
const codexFallbackHandoffNote =
|
|
||||||
forceFreshSession
|
|
||||||
? buildCodexTransientHandoffNote({
|
|
||||||
previousSessionId: runtimeSessionId || runtime.sessionId || null,
|
|
||||||
fallbackMode: codexTransientFallbackMode ?? "fresh_session",
|
|
||||||
continuationSummaryBody,
|
|
||||||
})
|
|
||||||
: "";
|
|
||||||
const commandNotes = (() => {
|
|
||||||
if (!instructionsFilePath) {
|
|
||||||
const notes = [repoAgentsNote];
|
|
||||||
if (forceSaferInvocation) {
|
|
||||||
notes.push("Codex transient fallback requested safer invocation settings for this retry.");
|
|
||||||
}
|
}
|
||||||
if (forceFreshSession) {
|
|
||||||
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
|
||||||
}
|
|
||||||
return notes;
|
|
||||||
}
|
}
|
||||||
if (instructionsPrefix.length > 0) {
|
const repoAgentsNote =
|
||||||
if (shouldUseResumeDeltaPrompt) {
|
"Codex exec automatically applies repo-scoped AGENTS.md instructions from the current workspace; Paperclip does not currently suppress that discovery.";
|
||||||
|
const bootstrapPromptTemplate = asString(config.bootstrapPromptTemplate, "");
|
||||||
|
const templateData = {
|
||||||
|
agentId: agent.id,
|
||||||
|
companyId: agent.companyId,
|
||||||
|
runId,
|
||||||
|
company: { id: agent.companyId },
|
||||||
|
agent,
|
||||||
|
run: { id: runId, source: "on_demand" },
|
||||||
|
context,
|
||||||
|
};
|
||||||
|
const renderedBootstrapPrompt =
|
||||||
|
!sessionId && bootstrapPromptTemplate.trim().length > 0
|
||||||
|
? renderTemplate(bootstrapPromptTemplate, templateData).trim()
|
||||||
|
: "";
|
||||||
|
const wakePrompt = renderPaperclipWakePrompt(context.paperclipWake, { resumedSession: Boolean(sessionId) });
|
||||||
|
const shouldUseResumeDeltaPrompt = Boolean(sessionId) && wakePrompt.length > 0;
|
||||||
|
const promptInstructionsPrefix = shouldUseResumeDeltaPrompt ? "" : instructionsPrefix;
|
||||||
|
instructionsChars = promptInstructionsPrefix.length;
|
||||||
|
const continuationSummary = parseObject(context.paperclipContinuationSummary);
|
||||||
|
const continuationSummaryBody = asString(continuationSummary.body, "").trim() || null;
|
||||||
|
const codexFallbackHandoffNote =
|
||||||
|
forceFreshSession
|
||||||
|
? buildCodexTransientHandoffNote({
|
||||||
|
previousSessionId: runtimeSessionId || runtime.sessionId || null,
|
||||||
|
fallbackMode: codexTransientFallbackMode ?? "fresh_session",
|
||||||
|
continuationSummaryBody,
|
||||||
|
})
|
||||||
|
: "";
|
||||||
|
const commandNotes = (() => {
|
||||||
|
if (!instructionsFilePath) {
|
||||||
|
const notes = [repoAgentsNote];
|
||||||
|
if (forceSaferInvocation) {
|
||||||
|
notes.push("Codex transient fallback requested safer invocation settings for this retry.");
|
||||||
|
}
|
||||||
|
if (forceFreshSession) {
|
||||||
|
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
||||||
|
}
|
||||||
|
return notes;
|
||||||
|
}
|
||||||
|
if (instructionsPrefix.length > 0) {
|
||||||
|
if (shouldUseResumeDeltaPrompt) {
|
||||||
|
const notes = [
|
||||||
|
`Loaded agent instructions from ${instructionsFilePath}`,
|
||||||
|
"Skipped stdin instruction reinjection because an existing Codex session is being resumed with a wake delta.",
|
||||||
|
repoAgentsNote,
|
||||||
|
];
|
||||||
|
if (forceSaferInvocation) {
|
||||||
|
notes.push("Codex transient fallback requested safer invocation settings for this retry.");
|
||||||
|
}
|
||||||
|
if (forceFreshSession) {
|
||||||
|
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
||||||
|
}
|
||||||
|
return notes;
|
||||||
|
}
|
||||||
const notes = [
|
const notes = [
|
||||||
`Loaded agent instructions from ${instructionsFilePath}`,
|
`Loaded agent instructions from ${instructionsFilePath}`,
|
||||||
"Skipped stdin instruction reinjection because an existing Codex session is being resumed with a wake delta.",
|
`Prepended instructions + path directive to stdin prompt (relative references from ${instructionsDir}).`,
|
||||||
repoAgentsNote,
|
repoAgentsNote,
|
||||||
];
|
];
|
||||||
if (forceSaferInvocation) {
|
if (forceSaferInvocation) {
|
||||||
@@ -629,8 +660,7 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
|||||||
return notes;
|
return notes;
|
||||||
}
|
}
|
||||||
const notes = [
|
const notes = [
|
||||||
`Loaded agent instructions from ${instructionsFilePath}`,
|
`Configured instructionsFilePath ${instructionsFilePath}, but file could not be read; continuing without injected instructions.`,
|
||||||
`Prepended instructions + path directive to stdin prompt (relative references from ${instructionsDir}).`,
|
|
||||||
repoAgentsNote,
|
repoAgentsNote,
|
||||||
];
|
];
|
||||||
if (forceSaferInvocation) {
|
if (forceSaferInvocation) {
|
||||||
@@ -640,218 +670,220 @@ export async function execute(ctx: AdapterExecutionContext): Promise<AdapterExec
|
|||||||
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
||||||
}
|
}
|
||||||
return notes;
|
return notes;
|
||||||
|
})();
|
||||||
|
if (executionTargetIsSandbox) {
|
||||||
|
commandNotes.push(
|
||||||
|
"Added --skip-git-repo-check for sandbox execution because Codex requires an explicit trust bypass in headless remote workspaces.",
|
||||||
|
);
|
||||||
}
|
}
|
||||||
const notes = [
|
if (preparedRuntimeConfig.notes.length > 0) {
|
||||||
`Configured instructionsFilePath ${instructionsFilePath}, but file could not be read; continuing without injected instructions.`,
|
commandNotes.unshift(...preparedRuntimeConfig.notes);
|
||||||
repoAgentsNote,
|
|
||||||
];
|
|
||||||
if (forceSaferInvocation) {
|
|
||||||
notes.push("Codex transient fallback requested safer invocation settings for this retry.");
|
|
||||||
}
|
}
|
||||||
if (forceFreshSession) {
|
const renderedPrompt = shouldUseResumeDeltaPrompt ? "" : renderTemplate(promptTemplate, templateData);
|
||||||
notes.push("Codex transient fallback forced a fresh session with a continuation handoff.");
|
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
||||||
}
|
const prompt = joinPromptSections([
|
||||||
return notes;
|
promptInstructionsPrefix,
|
||||||
})();
|
renderedBootstrapPrompt,
|
||||||
if (executionTargetIsSandbox) {
|
wakePrompt,
|
||||||
commandNotes.push(
|
codexFallbackHandoffNote,
|
||||||
"Added --skip-git-repo-check for sandbox execution because Codex requires an explicit trust bypass in headless remote workspaces.",
|
sessionHandoffNote,
|
||||||
);
|
renderedPrompt,
|
||||||
}
|
]);
|
||||||
const renderedPrompt = shouldUseResumeDeltaPrompt ? "" : renderTemplate(promptTemplate, templateData);
|
const promptMetrics = {
|
||||||
const sessionHandoffNote = asString(context.paperclipSessionHandoffMarkdown, "").trim();
|
promptChars: prompt.length,
|
||||||
const prompt = joinPromptSections([
|
instructionsChars,
|
||||||
promptInstructionsPrefix,
|
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
||||||
renderedBootstrapPrompt,
|
wakePromptChars: wakePrompt.length,
|
||||||
wakePrompt,
|
sessionHandoffChars: sessionHandoffNote.length,
|
||||||
codexFallbackHandoffNote,
|
heartbeatPromptChars: renderedPrompt.length,
|
||||||
sessionHandoffNote,
|
|
||||||
renderedPrompt,
|
|
||||||
]);
|
|
||||||
const promptMetrics = {
|
|
||||||
promptChars: prompt.length,
|
|
||||||
instructionsChars,
|
|
||||||
bootstrapPromptChars: renderedBootstrapPrompt.length,
|
|
||||||
wakePromptChars: wakePrompt.length,
|
|
||||||
sessionHandoffChars: sessionHandoffNote.length,
|
|
||||||
heartbeatPromptChars: renderedPrompt.length,
|
|
||||||
};
|
|
||||||
|
|
||||||
const runAttempt = async (resumeSessionId: string | null) => {
|
|
||||||
const execArgs = buildCodexExecArgs(
|
|
||||||
forceSaferInvocation ? { ...config, fastMode: false } : config,
|
|
||||||
{
|
|
||||||
resumeSessionId,
|
|
||||||
skipGitRepoCheck: executionTargetIsSandbox,
|
|
||||||
},
|
|
||||||
);
|
|
||||||
const args = execArgs.args;
|
|
||||||
const commandNotesWithFastMode =
|
|
||||||
execArgs.fastModeIgnoredReason == null
|
|
||||||
? commandNotes
|
|
||||||
: [...commandNotes, execArgs.fastModeIgnoredReason];
|
|
||||||
if (onMeta) {
|
|
||||||
await onMeta({
|
|
||||||
adapterType: "codex_local",
|
|
||||||
command: resolvedCommand,
|
|
||||||
cwd: effectiveExecutionCwd,
|
|
||||||
commandNotes: commandNotesWithFastMode,
|
|
||||||
commandArgs: args.map((value, idx) => {
|
|
||||||
if (idx === args.length - 1 && value !== "-") return `<prompt ${prompt.length} chars>`;
|
|
||||||
return value;
|
|
||||||
}),
|
|
||||||
env: loggedEnv,
|
|
||||||
prompt,
|
|
||||||
promptMetrics,
|
|
||||||
context,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const proc = await runAdapterExecutionTargetProcess(runId, runtimeExecutionTarget, command, args, {
|
|
||||||
cwd,
|
|
||||||
env,
|
|
||||||
stdin: prompt,
|
|
||||||
timeoutSec,
|
|
||||||
graceSec,
|
|
||||||
onSpawn,
|
|
||||||
onLog: async (stream, chunk) => {
|
|
||||||
if (stream !== "stderr") {
|
|
||||||
await onLog(stream, chunk);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
const cleaned = stripCodexRolloutNoise(chunk);
|
|
||||||
if (!cleaned.trim()) return;
|
|
||||||
await onLog(stream, cleaned);
|
|
||||||
},
|
|
||||||
});
|
|
||||||
const cleanedStderr = stripCodexRolloutNoise(proc.stderr);
|
|
||||||
return {
|
|
||||||
proc: {
|
|
||||||
...proc,
|
|
||||||
stderr: cleanedStderr,
|
|
||||||
},
|
|
||||||
rawStderr: proc.stderr,
|
|
||||||
parsed: parseCodexJsonl(proc.stdout),
|
|
||||||
};
|
};
|
||||||
};
|
|
||||||
|
|
||||||
const toResult = (
|
const runAttempt = async (resumeSessionId: string | null) => {
|
||||||
attempt: { proc: { exitCode: number | null; signal: string | null; timedOut: boolean; stdout: string; stderr: string }; rawStderr: string; parsed: ReturnType<typeof parseCodexJsonl> },
|
const execArgs = buildCodexExecArgs(
|
||||||
clearSessionOnMissingSession = false,
|
forceSaferInvocation ? { ...config, fastMode: false } : config,
|
||||||
isRetry = false,
|
{
|
||||||
): AdapterExecutionResult => {
|
resumeSessionId,
|
||||||
if (attempt.proc.timedOut) {
|
skipGitRepoCheck: executionTargetIsSandbox,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const args = execArgs.args;
|
||||||
|
const commandNotesWithFastMode =
|
||||||
|
execArgs.fastModeIgnoredReason == null
|
||||||
|
? commandNotes
|
||||||
|
: [...commandNotes, execArgs.fastModeIgnoredReason];
|
||||||
|
if (onMeta) {
|
||||||
|
await onMeta({
|
||||||
|
adapterType: "codex_local",
|
||||||
|
command: resolvedCommand,
|
||||||
|
cwd: effectiveExecutionCwd,
|
||||||
|
commandNotes: commandNotesWithFastMode,
|
||||||
|
commandArgs: args.map((value, idx) => {
|
||||||
|
if (idx === args.length - 1 && value !== "-") return `<prompt ${prompt.length} chars>`;
|
||||||
|
return value;
|
||||||
|
}),
|
||||||
|
env: loggedEnv,
|
||||||
|
prompt,
|
||||||
|
promptMetrics,
|
||||||
|
context,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const proc = await runAdapterExecutionTargetProcess(runId, runtimeExecutionTarget, command, args, {
|
||||||
|
cwd,
|
||||||
|
env,
|
||||||
|
stdin: prompt,
|
||||||
|
timeoutSec,
|
||||||
|
graceSec,
|
||||||
|
onSpawn,
|
||||||
|
onLog: async (stream, chunk) => {
|
||||||
|
if (stream !== "stderr") {
|
||||||
|
await onLog(stream, chunk);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const cleaned = stripCodexRolloutNoise(chunk);
|
||||||
|
if (!cleaned.trim()) return;
|
||||||
|
await onLog(stream, cleaned);
|
||||||
|
},
|
||||||
|
});
|
||||||
|
const cleanedStderr = stripCodexRolloutNoise(proc.stderr);
|
||||||
|
return {
|
||||||
|
proc: {
|
||||||
|
...proc,
|
||||||
|
stderr: cleanedStderr,
|
||||||
|
},
|
||||||
|
rawStderr: proc.stderr,
|
||||||
|
parsed: parseCodexJsonl(proc.stdout),
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
const toResult = (
|
||||||
|
attempt: { proc: { exitCode: number | null; signal: string | null; timedOut: boolean; stdout: string; stderr: string }; rawStderr: string; parsed: ReturnType<typeof parseCodexJsonl> },
|
||||||
|
clearSessionOnMissingSession = false,
|
||||||
|
isRetry = false,
|
||||||
|
): AdapterExecutionResult => {
|
||||||
|
if (attempt.proc.timedOut) {
|
||||||
|
return {
|
||||||
|
exitCode: attempt.proc.exitCode,
|
||||||
|
signal: attempt.proc.signal,
|
||||||
|
timedOut: true,
|
||||||
|
errorMessage: `Timed out after ${timeoutSec}s`,
|
||||||
|
clearSession: clearSessionOnMissingSession,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const canFallbackToRuntimeSession = !isRetry && !forceFreshSession;
|
||||||
|
const resolvedSessionId =
|
||||||
|
attempt.parsed.sessionId ??
|
||||||
|
(canFallbackToRuntimeSession ? (runtimeSessionId ?? runtime.sessionId ?? null) : null);
|
||||||
|
const resolvedSessionParams = resolvedSessionId
|
||||||
|
? ({
|
||||||
|
sessionId: resolvedSessionId,
|
||||||
|
cwd: effectiveExecutionCwd,
|
||||||
|
...(executionTargetIsRemote
|
||||||
|
? {
|
||||||
|
remoteExecution: adapterExecutionTargetSessionIdentity(runtimeExecutionTarget),
|
||||||
|
}
|
||||||
|
: {}),
|
||||||
|
...(workspaceId ? { workspaceId } : {}),
|
||||||
|
...(workspaceRepoUrl ? { repoUrl: workspaceRepoUrl } : {}),
|
||||||
|
...(workspaceRepoRef ? { repoRef: workspaceRepoRef } : {}),
|
||||||
|
} as Record<string, unknown>)
|
||||||
|
: null;
|
||||||
|
const parsedError = typeof attempt.parsed.errorMessage === "string" ? attempt.parsed.errorMessage.trim() : "";
|
||||||
|
const stderrLine = firstNonEmptyLine(attempt.proc.stderr);
|
||||||
|
const fallbackErrorMessage =
|
||||||
|
parsedError ||
|
||||||
|
stderrLine ||
|
||||||
|
`Codex exited with code ${attempt.proc.exitCode ?? -1}`;
|
||||||
|
const transientRetryNotBefore =
|
||||||
|
(attempt.proc.exitCode ?? 0) !== 0
|
||||||
|
? extractCodexRetryNotBefore({
|
||||||
|
stdout: attempt.proc.stdout,
|
||||||
|
stderr: attempt.proc.stderr,
|
||||||
|
errorMessage: fallbackErrorMessage,
|
||||||
|
})
|
||||||
|
: null;
|
||||||
|
const transientUpstream =
|
||||||
|
(attempt.proc.exitCode ?? 0) !== 0 &&
|
||||||
|
isCodexTransientUpstreamError({
|
||||||
|
stdout: attempt.proc.stdout,
|
||||||
|
stderr: attempt.proc.stderr,
|
||||||
|
errorMessage: fallbackErrorMessage,
|
||||||
|
});
|
||||||
|
|
||||||
return {
|
return {
|
||||||
exitCode: attempt.proc.exitCode,
|
exitCode: attempt.proc.exitCode,
|
||||||
signal: attempt.proc.signal,
|
signal: attempt.proc.signal,
|
||||||
timedOut: true,
|
timedOut: false,
|
||||||
errorMessage: `Timed out after ${timeoutSec}s`,
|
errorMessage:
|
||||||
clearSession: clearSessionOnMissingSession,
|
(attempt.proc.exitCode ?? 0) === 0
|
||||||
};
|
? null
|
||||||
}
|
: fallbackErrorMessage,
|
||||||
|
errorCode:
|
||||||
const canFallbackToRuntimeSession = !isRetry && !forceFreshSession;
|
transientUpstream
|
||||||
const resolvedSessionId =
|
? "codex_transient_upstream"
|
||||||
attempt.parsed.sessionId ??
|
: null,
|
||||||
(canFallbackToRuntimeSession ? (runtimeSessionId ?? runtime.sessionId ?? null) : null);
|
errorFamily: transientUpstream ? "transient_upstream" : null,
|
||||||
const resolvedSessionParams = resolvedSessionId
|
retryNotBefore: transientRetryNotBefore ? transientRetryNotBefore.toISOString() : null,
|
||||||
? ({
|
usage: attempt.parsed.usage,
|
||||||
sessionId: resolvedSessionId,
|
sessionId: resolvedSessionId,
|
||||||
cwd: effectiveExecutionCwd,
|
sessionParams: resolvedSessionParams,
|
||||||
...(executionTargetIsRemote
|
sessionDisplayId: resolvedSessionId,
|
||||||
? {
|
provider: "openai",
|
||||||
remoteExecution: adapterExecutionTargetSessionIdentity(runtimeExecutionTarget),
|
biller: resolveCodexBiller(effectiveEnv, billingType),
|
||||||
}
|
model,
|
||||||
: {}),
|
billingType,
|
||||||
...(workspaceId ? { workspaceId } : {}),
|
costUsd: null,
|
||||||
...(workspaceRepoUrl ? { repoUrl: workspaceRepoUrl } : {}),
|
resultJson: {
|
||||||
...(workspaceRepoRef ? { repoRef: workspaceRepoRef } : {}),
|
stdout: attempt.proc.stdout,
|
||||||
} as Record<string, unknown>)
|
stderr: attempt.proc.stderr,
|
||||||
: null;
|
...(transientUpstream ? { errorFamily: "transient_upstream" } : {}),
|
||||||
const parsedError = typeof attempt.parsed.errorMessage === "string" ? attempt.parsed.errorMessage.trim() : "";
|
...(transientRetryNotBefore ? { retryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
||||||
const stderrLine = firstNonEmptyLine(attempt.proc.stderr);
|
...(transientRetryNotBefore ? { transientRetryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
||||||
const fallbackErrorMessage =
|
},
|
||||||
parsedError ||
|
summary: attempt.parsed.summary,
|
||||||
stderrLine ||
|
clearSession: Boolean((clearSessionOnMissingSession || forceFreshSession) && !resolvedSessionId),
|
||||||
`Codex exited with code ${attempt.proc.exitCode ?? -1}`;
|
};
|
||||||
const transientRetryNotBefore =
|
|
||||||
(attempt.proc.exitCode ?? 0) !== 0
|
|
||||||
? extractCodexRetryNotBefore({
|
|
||||||
stdout: attempt.proc.stdout,
|
|
||||||
stderr: attempt.proc.stderr,
|
|
||||||
errorMessage: fallbackErrorMessage,
|
|
||||||
})
|
|
||||||
: null;
|
|
||||||
const transientUpstream =
|
|
||||||
(attempt.proc.exitCode ?? 0) !== 0 &&
|
|
||||||
isCodexTransientUpstreamError({
|
|
||||||
stdout: attempt.proc.stdout,
|
|
||||||
stderr: attempt.proc.stderr,
|
|
||||||
errorMessage: fallbackErrorMessage,
|
|
||||||
});
|
|
||||||
|
|
||||||
return {
|
|
||||||
exitCode: attempt.proc.exitCode,
|
|
||||||
signal: attempt.proc.signal,
|
|
||||||
timedOut: false,
|
|
||||||
errorMessage:
|
|
||||||
(attempt.proc.exitCode ?? 0) === 0
|
|
||||||
? null
|
|
||||||
: fallbackErrorMessage,
|
|
||||||
errorCode:
|
|
||||||
transientUpstream
|
|
||||||
? "codex_transient_upstream"
|
|
||||||
: null,
|
|
||||||
errorFamily: transientUpstream ? "transient_upstream" : null,
|
|
||||||
retryNotBefore: transientRetryNotBefore ? transientRetryNotBefore.toISOString() : null,
|
|
||||||
usage: attempt.parsed.usage,
|
|
||||||
sessionId: resolvedSessionId,
|
|
||||||
sessionParams: resolvedSessionParams,
|
|
||||||
sessionDisplayId: resolvedSessionId,
|
|
||||||
provider: "openai",
|
|
||||||
biller: resolveCodexBiller(effectiveEnv, billingType),
|
|
||||||
model,
|
|
||||||
billingType,
|
|
||||||
costUsd: null,
|
|
||||||
resultJson: {
|
|
||||||
stdout: attempt.proc.stdout,
|
|
||||||
stderr: attempt.proc.stderr,
|
|
||||||
...(transientUpstream ? { errorFamily: "transient_upstream" } : {}),
|
|
||||||
...(transientRetryNotBefore ? { retryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
|
||||||
...(transientRetryNotBefore ? { transientRetryNotBefore: transientRetryNotBefore.toISOString() } : {}),
|
|
||||||
},
|
|
||||||
summary: attempt.parsed.summary,
|
|
||||||
clearSession: Boolean((clearSessionOnMissingSession || forceFreshSession) && !resolvedSessionId),
|
|
||||||
};
|
};
|
||||||
};
|
|
||||||
|
|
||||||
try {
|
try {
|
||||||
const initial = await runAttempt(sessionId);
|
const initial = await runAttempt(sessionId);
|
||||||
if (
|
if (
|
||||||
sessionId &&
|
sessionId &&
|
||||||
!initial.proc.timedOut &&
|
!initial.proc.timedOut &&
|
||||||
(initial.proc.exitCode ?? 0) !== 0 &&
|
(initial.proc.exitCode ?? 0) !== 0 &&
|
||||||
isCodexUnknownSessionError(initial.proc.stdout, initial.rawStderr)
|
isCodexUnknownSessionError(initial.proc.stdout, initial.rawStderr)
|
||||||
) {
|
) {
|
||||||
await onLog(
|
await onLog(
|
||||||
"stdout",
|
"stdout",
|
||||||
`[paperclip] Codex resume session "${sessionId}" is unavailable; retrying with a fresh session.\n`,
|
`[paperclip] Codex resume session "${sessionId}" is unavailable; retrying with a fresh session.\n`,
|
||||||
);
|
);
|
||||||
const retry = await runAttempt(null);
|
const retry = await runAttempt(null);
|
||||||
return toResult(retry, true, true);
|
return toResult(retry, true, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
return toResult(initial, false, false);
|
||||||
|
} finally {
|
||||||
|
if (paperclipBridge) {
|
||||||
|
await paperclipBridge.stop();
|
||||||
|
}
|
||||||
|
if (restoreRemoteWorkspace) {
|
||||||
|
await onLog(
|
||||||
|
"stdout",
|
||||||
|
`[paperclip] Restoring workspace changes from ${describeAdapterExecutionTarget(executionTarget)}.\n`,
|
||||||
|
);
|
||||||
|
await restoreRemoteWorkspace();
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return toResult(initial, false, false);
|
|
||||||
} finally {
|
} finally {
|
||||||
if (paperclipBridge) {
|
// Restore the managed config.toml so PAPERCLIP_CODEX_PROVIDERS changes
|
||||||
await paperclipBridge.stop();
|
// (or removal) between runs never leave stale provider routing behind. This
|
||||||
}
|
// finally starts the moment prepareCodexRuntimeConfig returns, so a throw
|
||||||
if (restoreRemoteWorkspace) {
|
// anywhere in the remaining setup (skill injection, remote runtime
|
||||||
await onLog(
|
// preparation, command building) restores the original config.toml too.
|
||||||
"stdout",
|
// If the process dies before reaching this, the next
|
||||||
`[paperclip] Restoring workspace changes from ${describeAdapterExecutionTarget(executionTarget)}.\n`,
|
// prepareCodexRuntimeConfig restores the original from the pre-run backup
|
||||||
);
|
// written at prepare time.
|
||||||
await restoreRemoteWorkspace();
|
await preparedRuntimeConfig.cleanup();
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,416 @@
|
|||||||
|
import fs from "node:fs/promises";
|
||||||
|
import os from "node:os";
|
||||||
|
import path from "node:path";
|
||||||
|
import { afterEach, describe, expect, it } from "vitest";
|
||||||
|
import { prepareCodexRuntimeConfig } from "./runtime-config.js";
|
||||||
|
|
||||||
|
const cleanupPaths = new Set<string>();
|
||||||
|
|
||||||
|
afterEach(async () => {
|
||||||
|
await Promise.all(
|
||||||
|
[...cleanupPaths].map(async (filepath) => {
|
||||||
|
await fs.rm(filepath, { recursive: true, force: true });
|
||||||
|
cleanupPaths.delete(filepath);
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
async function makeCodexHome(configToml?: string): Promise<string> {
|
||||||
|
const home = await fs.mkdtemp(path.join(os.tmpdir(), "paperclip-codex-home-"));
|
||||||
|
cleanupPaths.add(home);
|
||||||
|
if (configToml !== undefined) {
|
||||||
|
await fs.writeFile(path.join(home, "config.toml"), configToml, "utf8");
|
||||||
|
}
|
||||||
|
return home;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readConfigToml(home: string): Promise<string> {
|
||||||
|
return fs.readFile(path.join(home, "config.toml"), "utf8");
|
||||||
|
}
|
||||||
|
|
||||||
|
const BIFROST_PROVIDERS = {
|
||||||
|
providers: {
|
||||||
|
bifrost: {
|
||||||
|
name: "bifrost",
|
||||||
|
base_url: "http://gateway.example.svc.cluster.local:8080/v1",
|
||||||
|
env_key: "OPENAI_API_KEY",
|
||||||
|
wire_api: "responses",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
model_provider: "bifrost",
|
||||||
|
};
|
||||||
|
|
||||||
|
describe("prepareCodexRuntimeConfig", () => {
|
||||||
|
it("is a no-op when PAPERCLIP_CODEX_PROVIDERS is unset", async () => {
|
||||||
|
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env: { FOO: "bar" }, codexHome: home });
|
||||||
|
|
||||||
|
expect(prepared.notes).toEqual([]);
|
||||||
|
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("is a no-op when the home has no config.toml and the env is unset", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
|
||||||
|
|
||||||
|
expect(prepared.notes).toEqual([]);
|
||||||
|
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("merges providers + model_provider into a fresh config.toml and cleans it up", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain('model_provider = "bifrost"');
|
||||||
|
expect(content).toContain("[model_providers.bifrost]");
|
||||||
|
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
|
||||||
|
expect(content).toContain('env_key = "OPENAI_API_KEY"');
|
||||||
|
expect(content).toContain('wire_api = "responses"');
|
||||||
|
expect(prepared.notes.some((n) => n.includes("bifrost"))).toBe(true);
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
await expect(fs.access(path.join(home, "config.toml"))).rejects.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("preserves existing config.toml content, keeps model_provider in the root region, and restores on cleanup", async () => {
|
||||||
|
const original = [
|
||||||
|
'model = "gpt-5.1-codex"',
|
||||||
|
"",
|
||||||
|
"[profiles.dev]",
|
||||||
|
'model = "gpt-5.1-codex-mini"',
|
||||||
|
"",
|
||||||
|
].join("\n");
|
||||||
|
const home = await makeCodexHome(original);
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
// Existing content survives.
|
||||||
|
expect(content).toContain('model = "gpt-5.1-codex"');
|
||||||
|
expect(content).toContain("[profiles.dev]");
|
||||||
|
// model_provider must precede the first table header (TOML root region),
|
||||||
|
// and the provider tables must come after the user's tables.
|
||||||
|
expect(content.indexOf('model_provider = "bifrost"')).toBeLessThan(
|
||||||
|
content.indexOf("[profiles.dev]"),
|
||||||
|
);
|
||||||
|
expect(content.indexOf("[model_providers.bifrost]")).toBeGreaterThan(
|
||||||
|
content.indexOf("[profiles.dev]"),
|
||||||
|
);
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
expect(await readConfigToml(home)).toBe(original);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("wins over a pre-existing same-name [model_providers.*] section and root model_provider key", async () => {
|
||||||
|
const original = [
|
||||||
|
'model_provider = "stale"',
|
||||||
|
"",
|
||||||
|
"[model_providers.bifrost]",
|
||||||
|
'base_url = "http://old.example/v1"',
|
||||||
|
'env_key = "OLD_KEY"',
|
||||||
|
"",
|
||||||
|
"[model_providers.bifrost.http_headers]",
|
||||||
|
'"X-Old" = "1"',
|
||||||
|
"",
|
||||||
|
"[model_providers.other]",
|
||||||
|
'base_url = "http://other.example/v1"',
|
||||||
|
"",
|
||||||
|
].join("\n");
|
||||||
|
const home = await makeCodexHome(original);
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).not.toContain('model_provider = "stale"');
|
||||||
|
expect(content).not.toContain("http://old.example/v1");
|
||||||
|
expect(content).not.toContain("X-Old");
|
||||||
|
// Unrelated provider sections survive.
|
||||||
|
expect(content).toContain("[model_providers.other]");
|
||||||
|
expect(content).toContain("http://other.example/v1");
|
||||||
|
// Exactly one bifrost section: ours.
|
||||||
|
expect(content.split("[model_providers.bifrost]").length).toBe(2);
|
||||||
|
expect(content).toContain('base_url = "http://gateway.example.svc.cluster.local:8080/v1"');
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
expect(await readConfigToml(home)).toBe(original);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("emits plain-object fields as inline tables and arrays as TOML arrays", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: {
|
||||||
|
gw: {
|
||||||
|
base_url: "http://gw.example/v1",
|
||||||
|
query_params: { "api-version": "2026-01-01" },
|
||||||
|
http_headers: { "X Team": "agents" },
|
||||||
|
request_max_retries: 4,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain("[model_providers.gw]");
|
||||||
|
// Bare keys (incl. hyphens) stay bare; keys with other characters get quoted.
|
||||||
|
expect(content).toContain('query_params = { api-version = "2026-01-01" }');
|
||||||
|
expect(content).toContain('http_headers = { "X Team" = "agents" }');
|
||||||
|
expect(content).toContain("request_max_retries = 4");
|
||||||
|
// No model_provider was requested, so none is emitted.
|
||||||
|
expect(content).not.toContain("model_provider =");
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("expands {env:VAR} placeholders from the run env and process.env, leaving unresolvable ones intact", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY = "from-process-env";
|
||||||
|
try {
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: {
|
||||||
|
gw: {
|
||||||
|
base_url: "http://gw.example/v1",
|
||||||
|
http_headers: {
|
||||||
|
"X-Run": "{env:PAPERCLIP_CODEX_TEST_RUN_KEY}",
|
||||||
|
"X-Process": "{env:PAPERCLIP_CODEX_TEST_PROCESS_KEY}",
|
||||||
|
"X-Missing": "{env:DEFINITELY_UNSET_VAR_XYZ}",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
model_provider: "gw",
|
||||||
|
}),
|
||||||
|
PAPERCLIP_CODEX_TEST_RUN_KEY: "from-run-env",
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain('X-Run = "from-run-env"');
|
||||||
|
expect(content).toContain('X-Process = "from-process-env"');
|
||||||
|
expect(content).toContain('X-Missing = "{env:DEFINITELY_UNSET_VAR_XYZ}"');
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
} finally {
|
||||||
|
delete process.env.PAPERCLIP_CODEX_TEST_PROCESS_KEY;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("reads PAPERCLIP_CODEX_PROVIDERS from process.env when absent from the run env", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
process.env.PAPERCLIP_CODEX_PROVIDERS = JSON.stringify(BIFROST_PROVIDERS);
|
||||||
|
try {
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain("[model_providers.bifrost]");
|
||||||
|
await prepared.cleanup();
|
||||||
|
} finally {
|
||||||
|
delete process.env.PAPERCLIP_CODEX_PROVIDERS;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("surfaces a note for malformed PAPERCLIP_CODEX_PROVIDERS without touching config.toml", async () => {
|
||||||
|
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
|
||||||
|
const cases: Array<[raw: string, noteFragment: string]> = [
|
||||||
|
["not json", "contains invalid JSON"],
|
||||||
|
[JSON.stringify(["providers"]), "is not a JSON object"],
|
||||||
|
[JSON.stringify({ no_providers: true }), 'has no "providers" object'],
|
||||||
|
[JSON.stringify({ providers: {} }), "contains no usable entries"],
|
||||||
|
[JSON.stringify({ providers: { gw: "nope" } }), "skipped provider(s) with empty names or non-object values: gw"],
|
||||||
|
];
|
||||||
|
for (const [raw, noteFragment] of cases) {
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: raw },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
expect(prepared.notes).toHaveLength(1);
|
||||||
|
expect(prepared.notes[0]).toContain(noteFragment);
|
||||||
|
expect(prepared.notes[0]).toContain("custom providers ignored");
|
||||||
|
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
|
||||||
|
await prepared.cleanup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("stays silent when PAPERCLIP_CODEX_PROVIDERS is unset or empty", async () => {
|
||||||
|
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
|
||||||
|
const envs: Array<Record<string, string>> = [
|
||||||
|
{},
|
||||||
|
{ PAPERCLIP_CODEX_PROVIDERS: "" },
|
||||||
|
{ PAPERCLIP_CODEX_PROVIDERS: " " },
|
||||||
|
];
|
||||||
|
for (const env of envs) {
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env, codexHome: home });
|
||||||
|
expect(prepared.notes).toEqual([]);
|
||||||
|
expect(await readConfigToml(home)).toBe("model = \"gpt-5.1-codex\"\n");
|
||||||
|
await prepared.cleanup();
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it("surfaces skipped non-object provider entries while merging the usable ones", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: {
|
||||||
|
bad: "http://gateway.example/v1",
|
||||||
|
good: { base_url: "http://gateway.example/v1", env_key: "OPENAI_API_KEY" },
|
||||||
|
},
|
||||||
|
model_provider: "good",
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain("[model_providers.good]");
|
||||||
|
expect(content).not.toContain("[model_providers.bad]");
|
||||||
|
expect(
|
||||||
|
prepared.notes.some((n) => n.includes("skipped provider(s) with empty names or non-object values: bad")),
|
||||||
|
).toBe(true);
|
||||||
|
expect(prepared.notes.some((n) => n.includes("Merged 1 custom Codex model provider(s)"))).toBe(true);
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("escapes DEL (U+007F) in emitted TOML strings", async () => {
|
||||||
|
const del = String.fromCharCode(0x7f);
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: { gw: { base_url: "http://gw.example/v1", name: `del${del}name` } },
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).not.toContain(del);
|
||||||
|
expect(content).toContain("u007fname");
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("restores user provider sections from the pre-run backup after an interrupted run", async () => {
|
||||||
|
const userConfig = [
|
||||||
|
'model = "gpt-5.1-codex"',
|
||||||
|
"",
|
||||||
|
"[model_providers.bifrost]",
|
||||||
|
'base_url = "http://user.example/v1"',
|
||||||
|
"",
|
||||||
|
].join("\n");
|
||||||
|
const home = await makeCodexHome(userConfig);
|
||||||
|
// Simulate a crash: the merge excises the user's same-name provider
|
||||||
|
// section (the managed definition must win), and cleanup() never runs.
|
||||||
|
await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
expect(await readConfigToml(home)).toContain('env_key = "OPENAI_API_KEY"');
|
||||||
|
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
|
||||||
|
expect(prepared.notes.some((n) => n.includes("backup"))).toBe(true);
|
||||||
|
expect(await readConfigToml(home)).toBe(userConfig);
|
||||||
|
await expect(fs.access(path.join(home, "config.toml.paperclip-backup"))).rejects.toThrow();
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("removes the pre-run backup on cleanup", async () => {
|
||||||
|
const home = await makeCodexHome('approval_policy = "never"\n');
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
const backupPath = path.join(home, "config.toml.paperclip-backup");
|
||||||
|
expect(await fs.readFile(backupPath, "utf8")).toBe('approval_policy = "never"\n');
|
||||||
|
|
||||||
|
await prepared.cleanup();
|
||||||
|
expect(await readConfigToml(home)).toBe('approval_policy = "never"\n');
|
||||||
|
await expect(fs.access(backupPath)).rejects.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("skips the merge and surfaces a note when CODEX_HOME is explicitly configured", async () => {
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: null,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(prepared.notes).toHaveLength(1);
|
||||||
|
expect(prepared.notes[0]).toContain("CODEX_HOME");
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("self-heals stale managed blocks when the env is no longer set", async () => {
|
||||||
|
const home = await makeCodexHome();
|
||||||
|
const crashed = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
// Simulate a crash: cleanup never runs, the managed blocks persist.
|
||||||
|
void crashed;
|
||||||
|
expect(await readConfigToml(home)).toContain("[model_providers.bifrost]");
|
||||||
|
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({ env: {}, codexHome: home });
|
||||||
|
expect(prepared.notes.some((n) => n.includes("stale"))).toBe(true);
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).not.toContain("model_provider");
|
||||||
|
expect(content).not.toContain("bifrost");
|
||||||
|
await prepared.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("re-running with changed providers replaces the previous managed blocks", async () => {
|
||||||
|
const home = await makeCodexHome("approval_policy = \"never\"\n");
|
||||||
|
const first = await prepareCodexRuntimeConfig({
|
||||||
|
env: { PAPERCLIP_CODEX_PROVIDERS: JSON.stringify(BIFROST_PROVIDERS) },
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
void first; // simulate crash: no cleanup
|
||||||
|
const second = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: { gw2: { base_url: "http://gw2.example/v1", env_key: "OPENAI_API_KEY" } },
|
||||||
|
model_provider: "gw2",
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toContain('approval_policy = "never"');
|
||||||
|
expect(content).toContain("[model_providers.gw2]");
|
||||||
|
expect(content).toContain('model_provider = "gw2"');
|
||||||
|
expect(content).not.toContain("bifrost");
|
||||||
|
expect(content.split("model_provider =").length).toBe(2);
|
||||||
|
await second.cleanup();
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects the block when model_provider names a filtered or missing provider", async () => {
|
||||||
|
const home = await makeCodexHome("model = \"gpt-5.1-codex\"\n");
|
||||||
|
const prepared = await prepareCodexRuntimeConfig({
|
||||||
|
env: {
|
||||||
|
PAPERCLIP_CODEX_PROVIDERS: JSON.stringify({
|
||||||
|
providers: { good: { base_url: "https://gateway.example/v1" }, bad: "not-an-object" },
|
||||||
|
model_provider: "bad",
|
||||||
|
}),
|
||||||
|
},
|
||||||
|
codexHome: home,
|
||||||
|
});
|
||||||
|
expect(prepared.notes).toContain(
|
||||||
|
'PAPERCLIP_CODEX_PROVIDERS: model_provider "bad" does not match any usable provider entry; custom providers ignored.',
|
||||||
|
);
|
||||||
|
const content = await readConfigToml(home);
|
||||||
|
expect(content).toBe("model = \"gpt-5.1-codex\"\n");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,430 @@
|
|||||||
|
import fs from "node:fs/promises";
|
||||||
|
import path from "node:path";
|
||||||
|
|
||||||
|
type PreparedCodexRuntimeConfig = {
|
||||||
|
notes: string[];
|
||||||
|
cleanup: () => Promise<void>;
|
||||||
|
};
|
||||||
|
|
||||||
|
type ParsedCodexProvidersConfig = {
|
||||||
|
providers: Record<string, Record<string, unknown>>;
|
||||||
|
modelProvider: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
// Marker comments delimiting the Paperclip-managed regions of config.toml.
|
||||||
|
// TOML requires root-level keys (model_provider) to appear before the first
|
||||||
|
// table header, while [model_providers.*] tables must not swallow the user's
|
||||||
|
// root keys, so the managed content is split into a root block prepended to
|
||||||
|
// the file and a tables block appended to it.
|
||||||
|
const MANAGED_ROOT_BEGIN = "# >>> paperclip codex providers (root) -- managed, do not edit >>>";
|
||||||
|
const MANAGED_ROOT_END = "# <<< paperclip codex providers (root) <<<";
|
||||||
|
const MANAGED_TABLES_BEGIN = "# >>> paperclip codex providers (tables) -- managed, do not edit >>>";
|
||||||
|
const MANAGED_TABLES_END = "# <<< paperclip codex providers (tables) <<<";
|
||||||
|
|
||||||
|
function isPlainObject(value: unknown): value is Record<string, unknown> {
|
||||||
|
return typeof value === "object" && value !== null && !Array.isArray(value);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Recursively replace {env:VAR} placeholders with the resolved value. Used to bake
|
||||||
|
// gateway provider secrets into config.toml SERVER-SIDE, where the value is
|
||||||
|
// reliably present. Prefer codex's own `env_key` indirection (codex reads the
|
||||||
|
// named env var at request time); placeholder expansion exists for fields that
|
||||||
|
// must carry a literal value (e.g. http_headers). Unresolvable placeholders are
|
||||||
|
// left intact.
|
||||||
|
function expandEnvPlaceholders<T>(value: T, resolve: (name: string) => string | undefined): T {
|
||||||
|
if (typeof value === "string") {
|
||||||
|
return value.replace(/\{env:([A-Za-z_][A-Za-z0-9_]*)\}/g, (match, name: string) => {
|
||||||
|
const resolved = resolve(name);
|
||||||
|
return resolved !== undefined && resolved.length > 0 ? resolved : match;
|
||||||
|
}) as unknown as T;
|
||||||
|
}
|
||||||
|
if (Array.isArray(value)) {
|
||||||
|
return value.map((entry) => expandEnvPlaceholders(entry, resolve)) as unknown as T;
|
||||||
|
}
|
||||||
|
if (isPlainObject(value)) {
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const [key, entry] of Object.entries(value)) {
|
||||||
|
out[key] = expandEnvPlaceholders(entry, resolve);
|
||||||
|
}
|
||||||
|
return out as unknown as T;
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
// PAPERCLIP_CODEX_PROVIDERS is a JSON object that maps 1:1 onto codex's
|
||||||
|
// config.toml schema:
|
||||||
|
//
|
||||||
|
// {
|
||||||
|
// "providers": {
|
||||||
|
// "<id>": { // -> [model_providers.<id>]
|
||||||
|
// "name": "My gateway", // optional display name
|
||||||
|
// "base_url": "http://...", // OpenAI-compatible endpoint
|
||||||
|
// "env_key": "OPENAI_API_KEY", // env var codex reads the bearer key from
|
||||||
|
// "wire_api": "responses", // protocol codex speaks to the provider
|
||||||
|
// ... // any other field codex supports
|
||||||
|
// // (query_params, http_headers,
|
||||||
|
// // env_http_headers, request_max_retries, ...)
|
||||||
|
// }
|
||||||
|
// },
|
||||||
|
// "model_provider": "<id>" // optional: top-level provider selection
|
||||||
|
// }
|
||||||
|
//
|
||||||
|
// Scalar fields are emitted verbatim as TOML key = value pairs; plain-object
|
||||||
|
// fields (query_params, http_headers, ...) are emitted as inline tables and
|
||||||
|
// arrays of scalars as TOML arrays. String values may use {env:VAR}
|
||||||
|
// placeholders, expanded server-side against the run env and process.env.
|
||||||
|
function parseCodexProvidersConfig(
|
||||||
|
raw: unknown,
|
||||||
|
resolveEnv: (name: string) => string | undefined,
|
||||||
|
notes: string[],
|
||||||
|
): ParsedCodexProvidersConfig | null {
|
||||||
|
if (typeof raw !== "string" || raw.trim().length === 0) return null;
|
||||||
|
let parsed: unknown;
|
||||||
|
try {
|
||||||
|
parsed = JSON.parse(raw);
|
||||||
|
} catch {
|
||||||
|
// Surface the misconfiguration instead of silently dropping the provider
|
||||||
|
// config; an unparseable value would otherwise be undiagnosable.
|
||||||
|
notes.push("PAPERCLIP_CODEX_PROVIDERS contains invalid JSON; custom providers ignored.");
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (!isPlainObject(parsed)) {
|
||||||
|
notes.push("PAPERCLIP_CODEX_PROVIDERS is set but is not a JSON object; custom providers ignored.");
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
const rawProviders = parsed.providers;
|
||||||
|
if (!isPlainObject(rawProviders)) {
|
||||||
|
notes.push(
|
||||||
|
'PAPERCLIP_CODEX_PROVIDERS has no "providers" object; custom providers ignored.',
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
// Only keep provider entries with non-empty names and object values; surface
|
||||||
|
// the ones we drop so a malformed entry is just as diagnosable as malformed JSON.
|
||||||
|
const providers: Record<string, Record<string, unknown>> = {};
|
||||||
|
const skipped: string[] = [];
|
||||||
|
for (const [key, value] of Object.entries(rawProviders)) {
|
||||||
|
if (key.trim().length === 0 || !isPlainObject(value)) {
|
||||||
|
skipped.push(key.trim().length === 0 ? "(empty name)" : key);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
providers[key] = expandEnvPlaceholders(value, resolveEnv);
|
||||||
|
}
|
||||||
|
if (Object.keys(providers).length === 0) {
|
||||||
|
notes.push(
|
||||||
|
`PAPERCLIP_CODEX_PROVIDERS "providers" contains no usable entries${
|
||||||
|
skipped.length > 0
|
||||||
|
? ` (skipped provider(s) with empty names or non-object values: ${skipped.join(", ")})`
|
||||||
|
: ""
|
||||||
|
}; custom providers ignored.`,
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
if (skipped.length > 0) {
|
||||||
|
notes.push(
|
||||||
|
`PAPERCLIP_CODEX_PROVIDERS: skipped provider(s) with empty names or non-object values: ${skipped.join(", ")}.`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const modelProvider =
|
||||||
|
typeof parsed.model_provider === "string" && parsed.model_provider.trim().length > 0
|
||||||
|
? parsed.model_provider.trim()
|
||||||
|
: null;
|
||||||
|
// A selector pointing at a provider that did not survive filtering (or was
|
||||||
|
// never defined) would emit model_provider = "x" with no [model_providers.x]
|
||||||
|
// table, which codex rejects at runtime with an error that points nowhere
|
||||||
|
// near the env var. Treat it as the same class of misconfiguration as
|
||||||
|
// malformed JSON: reject the whole block with a visible note.
|
||||||
|
if (modelProvider !== null && !(modelProvider in providers)) {
|
||||||
|
notes.push(
|
||||||
|
`PAPERCLIP_CODEX_PROVIDERS: model_provider "${modelProvider}" does not match any usable provider entry; custom providers ignored.`,
|
||||||
|
);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return { providers, modelProvider };
|
||||||
|
}
|
||||||
|
|
||||||
|
function escapeTomlString(value: string): string {
|
||||||
|
// TOML 1.0 basic strings require escaping U+0000-U+001F and U+007F (DEL).
|
||||||
|
return value.replace(/[\\"\u0000-\u001f\u007f]/g, (char) => {
|
||||||
|
switch (char) {
|
||||||
|
case "\\":
|
||||||
|
return "\\\\";
|
||||||
|
case '"':
|
||||||
|
return '\\"';
|
||||||
|
case "\n":
|
||||||
|
return "\\n";
|
||||||
|
case "\r":
|
||||||
|
return "\\r";
|
||||||
|
case "\t":
|
||||||
|
return "\\t";
|
||||||
|
default:
|
||||||
|
return `\\u${char.charCodeAt(0).toString(16).padStart(4, "0")}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
const BARE_TOML_KEY_RE = /^[A-Za-z0-9_-]+$/;
|
||||||
|
|
||||||
|
function tomlKey(key: string): string {
|
||||||
|
return BARE_TOML_KEY_RE.test(key) ? key : `"${escapeTomlString(key)}"`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Hand-emitted TOML for a constrained value space (strings, numbers, booleans,
|
||||||
|
// arrays of scalars, plain objects as inline tables). Returns null for values
|
||||||
|
// that cannot be represented, which are then skipped.
|
||||||
|
function tomlValue(value: unknown): string | null {
|
||||||
|
if (typeof value === "string") return `"${escapeTomlString(value)}"`;
|
||||||
|
if (typeof value === "boolean") return value ? "true" : "false";
|
||||||
|
if (typeof value === "number") return Number.isFinite(value) ? String(value) : null;
|
||||||
|
if (Array.isArray(value)) {
|
||||||
|
const entries = value.map((entry) => tomlValue(entry));
|
||||||
|
if (entries.some((entry) => entry === null)) return null;
|
||||||
|
return `[${entries.join(", ")}]`;
|
||||||
|
}
|
||||||
|
if (isPlainObject(value)) {
|
||||||
|
const pairs: string[] = [];
|
||||||
|
for (const [key, entry] of Object.entries(value)) {
|
||||||
|
const emitted = tomlValue(entry);
|
||||||
|
if (emitted === null) continue;
|
||||||
|
pairs.push(`${tomlKey(key)} = ${emitted}`);
|
||||||
|
}
|
||||||
|
return `{ ${pairs.join(", ")} }`;
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function emitProviderTable(name: string, fields: Record<string, unknown>): string[] {
|
||||||
|
const lines = [`[model_providers.${tomlKey(name)}]`];
|
||||||
|
for (const [key, value] of Object.entries(fields)) {
|
||||||
|
const emitted = tomlValue(value);
|
||||||
|
if (emitted === null) continue;
|
||||||
|
lines.push(`${tomlKey(key)} = ${emitted}`);
|
||||||
|
}
|
||||||
|
return lines;
|
||||||
|
}
|
||||||
|
|
||||||
|
function stripManagedBlock(lines: string[], begin: string, end: string): string[] {
|
||||||
|
const out: string[] = [];
|
||||||
|
let inBlock = false;
|
||||||
|
for (const line of lines) {
|
||||||
|
const trimmed = line.trim();
|
||||||
|
if (!inBlock && trimmed === begin) {
|
||||||
|
inBlock = true;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (inBlock) {
|
||||||
|
if (trimmed === end) inBlock = false;
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out.push(line);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function stripManagedCodexProviderBlocks(content: string): string {
|
||||||
|
let lines = content.split("\n");
|
||||||
|
lines = stripManagedBlock(lines, MANAGED_ROOT_BEGIN, MANAGED_ROOT_END);
|
||||||
|
lines = stripManagedBlock(lines, MANAGED_TABLES_BEGIN, MANAGED_TABLES_END);
|
||||||
|
return lines.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
const TABLE_HEADER_RE = /^\s*\[\s*([^\]]*?)\s*\]\s*(?:#.*)?$/;
|
||||||
|
|
||||||
|
// Best-effort parse of a TOML table header into its dotted path segments,
|
||||||
|
// stripping surrounding quotes per segment. Dotted quoted segment names are
|
||||||
|
// out of scope for this merge (codex provider ids are simple identifiers).
|
||||||
|
function parseTableHeaderPath(line: string): string[] | null {
|
||||||
|
const match = TABLE_HEADER_RE.exec(line);
|
||||||
|
if (!match) return null;
|
||||||
|
return match[1]
|
||||||
|
.split(".")
|
||||||
|
.map((segment) => segment.trim())
|
||||||
|
.map((segment) => segment.replace(/^"(.*)"$/, "$1").replace(/^'(.*)'$/, "$1"));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Remove pre-existing definitions that would conflict with (or override) the
|
||||||
|
// managed content: [model_providers.<name>] tables (and their subtables) for
|
||||||
|
// names we are about to define, and the root-level `model_provider` key when
|
||||||
|
// we set one. Duplicate TOML tables/keys are parse errors in codex, so the
|
||||||
|
// managed definitions must win by excising the originals.
|
||||||
|
function stripConflictingDefinitions(
|
||||||
|
content: string,
|
||||||
|
providerNames: string[],
|
||||||
|
removeRootModelProvider: boolean,
|
||||||
|
): string {
|
||||||
|
const names = new Set(providerNames);
|
||||||
|
const lines = content.split("\n");
|
||||||
|
const out: string[] = [];
|
||||||
|
let inRootRegion = true;
|
||||||
|
let skippingSection = false;
|
||||||
|
for (const line of lines) {
|
||||||
|
const headerPath = parseTableHeaderPath(line);
|
||||||
|
if (headerPath) {
|
||||||
|
inRootRegion = false;
|
||||||
|
skippingSection =
|
||||||
|
headerPath.length >= 2 &&
|
||||||
|
headerPath[0] === "model_providers" &&
|
||||||
|
names.has(headerPath[1]);
|
||||||
|
if (skippingSection) continue;
|
||||||
|
} else if (skippingSection) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (inRootRegion && removeRootModelProvider && /^\s*model_provider\s*=/.test(line)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
out.push(line);
|
||||||
|
}
|
||||||
|
return out.join("\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildMergedConfigToml(base: string, parsed: ParsedCodexProvidersConfig): string {
|
||||||
|
const sections: string[] = [];
|
||||||
|
if (parsed.modelProvider) {
|
||||||
|
sections.push(
|
||||||
|
[
|
||||||
|
MANAGED_ROOT_BEGIN,
|
||||||
|
`model_provider = "${escapeTomlString(parsed.modelProvider)}"`,
|
||||||
|
MANAGED_ROOT_END,
|
||||||
|
].join("\n"),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const trimmedBase = base.replace(/^\n+/, "").replace(/\n+$/, "");
|
||||||
|
if (trimmedBase.length > 0) sections.push(trimmedBase);
|
||||||
|
const tableLines: string[] = [MANAGED_TABLES_BEGIN];
|
||||||
|
for (const [name, fields] of Object.entries(parsed.providers)) {
|
||||||
|
tableLines.push(...emitProviderTable(name, fields), "");
|
||||||
|
}
|
||||||
|
while (tableLines[tableLines.length - 1] === "") tableLines.pop();
|
||||||
|
tableLines.push(MANAGED_TABLES_END);
|
||||||
|
sections.push(tableLines.join("\n"));
|
||||||
|
return `${sections.join("\n\n")}\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readFileOrNull(filePath: string): Promise<string | null> {
|
||||||
|
return fs.readFile(filePath, "utf8").catch(() => null);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pre-run backup of the original config.toml, written before the merged file.
|
||||||
|
// If a run dies without reaching cleanup() (a setup throw between prepare and
|
||||||
|
// execution, SIGKILL, ...), the next prepare restores the original from this
|
||||||
|
// backup with full fidelity -- including user [model_providers.*] sections the
|
||||||
|
// merge excised, which block-stripping alone cannot bring back.
|
||||||
|
function configTomlBackupPath(configTomlPath: string): string {
|
||||||
|
return `${configTomlPath}.paperclip-backup`;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Merge custom Codex model providers supplied via PAPERCLIP_CODEX_PROVIDERS
|
||||||
|
// into the managed CODEX_HOME's config.toml.
|
||||||
|
//
|
||||||
|
// Codex has no CLI flag or env var for pointing at a custom OpenAI-compatible
|
||||||
|
// endpoint: custom endpoints are `[model_providers.<id>]` tables in
|
||||||
|
// $CODEX_HOME/config.toml, selected by a top-level `model_provider = "<id>"`
|
||||||
|
// key (the `--model` CLI flag picks the model WITHIN the selected provider).
|
||||||
|
// We accept the providers as config (not hard-coded) so the gateway URL, key
|
||||||
|
// indirection, and wire protocol stay declarative.
|
||||||
|
//
|
||||||
|
// The merge preserves any existing config.toml content (seeded from the shared
|
||||||
|
// ~/.codex by prepareManagedCodexHome): managed content lives between marker
|
||||||
|
// comments and conflicting pre-existing definitions are excised so the managed
|
||||||
|
// definitions win. cleanup() restores the original file; if a run dies before
|
||||||
|
// cleanup, the next prepare restores the original from the pre-run backup file
|
||||||
|
// written alongside config.toml (including when PAPERCLIP_CODEX_PROVIDERS is
|
||||||
|
// no longer set), falling back to stripping the stale managed blocks.
|
||||||
|
//
|
||||||
|
// When the adapter config explicitly sets env.CODEX_HOME (a user-managed home),
|
||||||
|
// pass codexHome: null -- the file is left untouched and a note is surfaced.
|
||||||
|
export async function prepareCodexRuntimeConfig(input: {
|
||||||
|
env: Record<string, string>;
|
||||||
|
codexHome: string | null;
|
||||||
|
}): Promise<PreparedCodexRuntimeConfig> {
|
||||||
|
const resolveEnv = (name: string): string | undefined => input.env[name] ?? process.env[name];
|
||||||
|
const notes: string[] = [];
|
||||||
|
const parsed = parseCodexProvidersConfig(
|
||||||
|
input.env.PAPERCLIP_CODEX_PROVIDERS ?? process.env.PAPERCLIP_CODEX_PROVIDERS,
|
||||||
|
resolveEnv,
|
||||||
|
notes,
|
||||||
|
);
|
||||||
|
|
||||||
|
if (!parsed) {
|
||||||
|
// Self-heal state left behind by a crashed run (cleanup() never ran).
|
||||||
|
if (input.codexHome) {
|
||||||
|
const configTomlPath = path.join(input.codexHome, "config.toml");
|
||||||
|
const reason = notes.length === 0 ? " (PAPERCLIP_CODEX_PROVIDERS is no longer set)" : "";
|
||||||
|
const backupPath = configTomlBackupPath(configTomlPath);
|
||||||
|
const backup = await readFileOrNull(backupPath);
|
||||||
|
if (backup !== null) {
|
||||||
|
// Full-fidelity restore: the backup is the pre-run original, including
|
||||||
|
// any user provider sections the crashed run's merge excised.
|
||||||
|
await fs.writeFile(configTomlPath, backup, "utf8");
|
||||||
|
await fs.rm(backupPath, { force: true });
|
||||||
|
return {
|
||||||
|
notes: [
|
||||||
|
...notes,
|
||||||
|
`Restored "${configTomlPath}" from its pre-run backup, removing stale Paperclip-managed model providers left by an interrupted run${reason}.`,
|
||||||
|
],
|
||||||
|
cleanup: async () => {},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
// Fallback for pre-backup stale state: strip the managed blocks.
|
||||||
|
const existing = await readFileOrNull(configTomlPath);
|
||||||
|
if (existing !== null) {
|
||||||
|
const stripped = stripManagedCodexProviderBlocks(existing);
|
||||||
|
if (stripped !== existing) {
|
||||||
|
await fs.writeFile(configTomlPath, stripped, "utf8");
|
||||||
|
return {
|
||||||
|
notes: [
|
||||||
|
...notes,
|
||||||
|
`Removed stale Paperclip-managed model provider blocks from "${configTomlPath}"${reason}.`,
|
||||||
|
],
|
||||||
|
cleanup: async () => {},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return { notes, cleanup: async () => {} };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!input.codexHome) {
|
||||||
|
return {
|
||||||
|
notes: [
|
||||||
|
...notes,
|
||||||
|
"PAPERCLIP_CODEX_PROVIDERS is set but the adapter config explicitly sets env.CODEX_HOME; leaving the user-managed Codex home untouched (no model provider merge).",
|
||||||
|
],
|
||||||
|
cleanup: async () => {},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
const configTomlPath = path.join(input.codexHome, "config.toml");
|
||||||
|
const backupPath = configTomlBackupPath(configTomlPath);
|
||||||
|
// A surviving backup from an interrupted run is the true pre-run content;
|
||||||
|
// the current config.toml would still carry that run's managed blocks.
|
||||||
|
const original = (await readFileOrNull(backupPath)) ?? (await readFileOrNull(configTomlPath));
|
||||||
|
const providerNames = Object.keys(parsed.providers);
|
||||||
|
const base = stripConflictingDefinitions(
|
||||||
|
stripManagedCodexProviderBlocks(original ?? ""),
|
||||||
|
providerNames,
|
||||||
|
parsed.modelProvider !== null,
|
||||||
|
);
|
||||||
|
await fs.mkdir(input.codexHome, { recursive: true });
|
||||||
|
// Persist the original BEFORE writing the merged file so a run that never
|
||||||
|
// reaches cleanup() can be restored by the next prepare.
|
||||||
|
await fs.writeFile(backupPath, original ?? "", "utf8");
|
||||||
|
await fs.writeFile(configTomlPath, buildMergedConfigToml(base, parsed), "utf8");
|
||||||
|
|
||||||
|
return {
|
||||||
|
notes: [
|
||||||
|
...notes,
|
||||||
|
`Merged ${providerNames.length} custom Codex model provider(s) from PAPERCLIP_CODEX_PROVIDERS into "${configTomlPath}": ${providerNames.join(", ")}${
|
||||||
|
parsed.modelProvider ? `; selected model_provider "${parsed.modelProvider}"` : ""
|
||||||
|
}.`,
|
||||||
|
],
|
||||||
|
cleanup: async () => {
|
||||||
|
if (original === null) {
|
||||||
|
await fs.rm(configTomlPath, { force: true });
|
||||||
|
} else {
|
||||||
|
await fs.writeFile(configTomlPath, original, "utf8");
|
||||||
|
}
|
||||||
|
await fs.rm(backupPath, { force: true });
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user