From ce7b49e4f1d18be34e68eaa6f7cc25ea309fe92c Mon Sep 17 00:00:00 2001 From: Dotta <34892728+cryppadotta@users.noreply.github.com> Date: Tue, 9 Jun 2026 15:35:07 -0500 Subject: [PATCH] [codex] Recover duplicate npm provenance canary publishes (#7839) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - The release workflow publishes canary npm packages on every push to `master` > - The failing canary job built successfully and published several packages before npm failed on `@paperclipai/mcp-server` > - The concrete failure was npm trusted-publishing provenance returning `TLOG_CREATE_ENTRY_ERROR` because an equivalent Sigstore transparency-log entry already existed > - The package version was not visible on npm afterward, so the release script could not safely treat that error as success by itself > - This pull request adds a narrow recovery path for that npm provenance failure and keeps the existing registry verification as the final source of truth > - The benefit is that transient duplicate transparency-log failures do not break canary publication when a package can be republished without provenance or is already visible on npm ## Linked Issues or Issue Description Bug fix, no public GitHub issue found in duplicate search. - What happened: the Release workflow canary publish failed in `publish_canary` after npm returned `TLOG_CREATE_ENTRY_ERROR` while publishing `@paperclipai/mcp-server@2026.609.0-canary.2`. - Expected behavior: canary publishing should either recover from npm's duplicate transparency-log failure when the package can still be published, or fail later in registry verification if the package never appears. - Steps to reproduce: inspect https://github.com/paperclipai/paperclip/actions/runs/27230012891/job/80411422155 from push `05cb18cf28074a6d1074c7575c5a44133146e368`. - Deployment mode: GitHub Actions Release workflow, npm trusted publishing. - Duplicate search: no open PRs or issues found for `canary publish TLOG provenance release` or the failing run/job IDs. ## What Changed - Added `publish_package_to_npm` in `scripts/release-lib.sh` to wrap canary/stable package publishing. - Detects npm's duplicate Sigstore transparency-log error and checks whether the package version is already visible on npm. - Retries that exact package once with `--provenance=false` when npm hit the duplicate tlog error but the version is not visible yet. - Keeps unrelated publish failures as hard failures. - Added shell-helper tests with fake `pnpm` and `npm` commands, and included them in `pnpm test:release-registry`. ## Verification - `node --test scripts/release-lib.test.mjs` - `pnpm test:release-registry` - Confirmed `pnpm publish --dry-run --no-git-checks --tag canary --access public --provenance=false` is accepted by pnpm 9.15.4. ## Risks - Low risk: the recovery only triggers when npm output contains both `TLOG_CREATE_ENTRY_ERROR` and the duplicate transparency-log message. - Publishing without provenance is a fallback for canary continuity; if npm still does not expose the package, the existing registry verification step still fails the release. - The same helper is used by stable publishing too, but only for this exact npm provenance failure path. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. This is a release reliability bug fix. I checked `ROADMAP.md`; it does not duplicate planned core product work. ## Model Used OpenAI Codex coding agent, GPT-5-class model, tool-enabled local shell and GitHub CLI workflow, medium reasoning mode. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [x] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip --- package.json | 2 +- scripts/release-lib.sh | 49 +++++++++++ scripts/release-lib.test.mjs | 163 +++++++++++++++++++++++++++++++++++ scripts/release.sh | 2 +- 4 files changed, 214 insertions(+), 2 deletions(-) create mode 100644 scripts/release-lib.test.mjs diff --git a/package.json b/package.json index af587171..28e27bc9 100644 --- a/package.json +++ b/package.json @@ -42,7 +42,7 @@ "smoke:openclaw-docker-ui": "./scripts/smoke/openclaw-docker-ui.sh", "smoke:openclaw-sse-standalone": "./scripts/smoke/openclaw-sse-standalone.sh", "smoke:terminal-bench-loop-skill": "node scripts/smoke/terminal-bench-loop-skill-smoke.mjs", - "test:release-registry": "node --test scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs", + "test:release-registry": "node --test scripts/verify-release-registry-state.test.mjs scripts/release-package-map.test.mjs scripts/check-release-package-bootstrap.test.mjs scripts/check-no-git-push.test.mjs scripts/release-lib.test.mjs", "test:e2e": "npx playwright test --config tests/e2e/playwright.config.ts", "test:e2e:headed": "npx playwright test --config tests/e2e/playwright.config.ts --headed", "test:e2e:multiuser-authenticated": "npx playwright test --config tests/e2e/playwright-multiuser-authenticated.config.ts", diff --git a/scripts/release-lib.sh b/scripts/release-lib.sh index 1befae9e..240725e3 100644 --- a/scripts/release-lib.sh +++ b/scripts/release-lib.sh @@ -263,6 +263,55 @@ wait_for_npm_package_version() { return 1 } +is_npm_tlog_duplicate_error() { + local output="$1" + + grep -q "TLOG_CREATE_ENTRY_ERROR" <<< "$output" && + grep -q "equivalent entry already exists in the transparency log" <<< "$output" +} + +publish_package_to_npm() { + local dist_tag="$1" + local package_name="$2" + local package_version="$3" + local publish_log + + publish_log="$(mktemp "${TMPDIR:-/tmp}/paperclip-npm-publish.XXXXXX")" + + if (set -o pipefail; pnpm publish --no-git-checks --tag "$dist_tag" --access public 2>&1 | tee "$publish_log"); then + rm -f "$publish_log" + return 0 + fi + + if ! is_npm_tlog_duplicate_error "$(cat "$publish_log")"; then + rm -f "$publish_log" + return 1 + fi + + release_warn "npm publish hit a duplicate Sigstore transparency-log entry for ${package_name}@${package_version}." + + if npm_package_version_exists "$package_name" "$package_version"; then + release_warn "npm already exposes ${package_name}@${package_version}; continuing to registry verification." + rm -f "$publish_log" + return 0 + fi + + if [ "$dist_tag" != "canary" ]; then + release_warn "Not retrying ${package_name}@${package_version} without provenance for dist-tag ${dist_tag}." + rm -f "$publish_log" + return 1 + fi + + release_warn "Retrying ${package_name}@${package_version} once with npm provenance disabled." + if pnpm publish --no-git-checks --tag "$dist_tag" --access public --provenance=false; then + rm -f "$publish_log" + return 0 + fi + + rm -f "$publish_log" + return 1 +} + wait_for_release_registry_state() { local attempts="${1:-12}" local delay_seconds="${2:-5}" diff --git a/scripts/release-lib.test.mjs b/scripts/release-lib.test.mjs new file mode 100644 index 00000000..fcaf52e6 --- /dev/null +++ b/scripts/release-lib.test.mjs @@ -0,0 +1,163 @@ +import assert from "node:assert/strict"; +import { execFileSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import test from "node:test"; + +const repoRoot = new URL("..", import.meta.url).pathname.replace(/\/$/, ""); + +function writeExecutable(path, body) { + writeFileSync(path, body, { mode: 0o755 }); +} + +function runPublishHelper({ pnpmMode, npmVersionExists = false, distTag = "canary", callerPipefail = true }) { + const fixtureDir = mkdtempSync(join(tmpdir(), "paperclip-release-lib-")); + const binDir = join(fixtureDir, "bin"); + const stateDir = join(fixtureDir, "state"); + const callLog = join(fixtureDir, "calls.log"); + mkdirSync(binDir); + mkdirSync(stateDir); + + writeExecutable( + join(binDir, "pnpm"), + `#!/usr/bin/env bash +set -euo pipefail +printf 'pnpm %s\\n' "$*" >> "$FAKE_CALL_LOG" +case "$PNPM_MODE" in + success) + echo "published" + exit 0 + ;; + tlog-then-success) + if [ ! -f "$FAKE_STATE_DIR/pnpm-called" ]; then + touch "$FAKE_STATE_DIR/pnpm-called" + echo "npm error code TLOG_CREATE_ENTRY_ERROR" + echo "npm error error creating tlog entry - (409) an equivalent entry already exists in the transparency log with UUID abc" + exit 1 + fi + case " $* " in + *" --provenance=false "*) + echo "published without provenance" + exit 0 + ;; + *) + echo "retry did not disable provenance" + exit 1 + ;; + esac + ;; + tlog-always-fails) + echo "npm error code TLOG_CREATE_ENTRY_ERROR" + echo "npm error error creating tlog entry - (409) an equivalent entry already exists in the transparency log with UUID abc" + exit 1 + ;; + non-tlog-failure) + echo "npm error code E500" + exit 1 + ;; +esac +exit 1 +`, + ); + + writeExecutable( + join(binDir, "npm"), + `#!/usr/bin/env bash +set -euo pipefail +printf 'npm %s\\n' "$*" >> "$FAKE_CALL_LOG" +if [ "$1" = "view" ] && [ "$NPM_VERSION_EXISTS" = "true" ]; then + echo "1.2.3" + exit 0 +fi +exit 1 +`, + ); + + const shellOptions = callerPipefail ? "set -euo pipefail" : "set -eu"; + const script = ` +${shellOptions} +source "${repoRoot}/scripts/release-lib.sh" +publish_package_to_npm ${distTag} @paperclipai/example 1.2.3 +`; + + let status = 0; + let output = ""; + try { + output = execFileSync("bash", ["-lc", script], { + cwd: fixtureDir, + encoding: "utf8", + env: { + ...process.env, + PATH: `${binDir}:${process.env.PATH}`, + FAKE_CALL_LOG: callLog, + FAKE_STATE_DIR: stateDir, + NPM_VERSION_EXISTS: npmVersionExists ? "true" : "false", + PNPM_MODE: pnpmMode, + REPO_ROOT: fixtureDir, + }, + stdio: ["ignore", "pipe", "pipe"], + }); + } catch (error) { + status = error.status ?? 1; + output = `${error.stdout ?? ""}${error.stderr ?? ""}`; + } + + return { + calls: readFileSync(callLog, "utf8"), + output, + status, + }; +} + +test("publish_package_to_npm returns after a successful pnpm publish", () => { + const result = runPublishHelper({ pnpmMode: "success" }); + + assert.equal(result.status, 0); + assert.match(result.calls, /^pnpm publish --no-git-checks --tag canary --access public$/m); + assert.doesNotMatch(result.calls, /npm view/); + assert.doesNotMatch(result.calls, /--provenance=false/); +}); + +test("publish_package_to_npm retries duplicate tlog failures without provenance", () => { + const result = runPublishHelper({ pnpmMode: "tlog-then-success" }); + + assert.equal(result.status, 0); + assert.match(result.calls, /^npm view @paperclipai\/example@1\.2\.3 version$/m); + assert.match( + result.calls, + /^pnpm publish --no-git-checks --tag canary --access public --provenance=false$/m, + ); +}); + +test("publish_package_to_npm treats a duplicate tlog failure as complete when npm exposes the version", () => { + const result = runPublishHelper({ pnpmMode: "tlog-always-fails", npmVersionExists: true }); + + assert.equal(result.status, 0); + assert.match(result.calls, /^npm view @paperclipai\/example@1\.2\.3 version$/m); + assert.doesNotMatch(result.calls, /--provenance=false/); +}); + +test("publish_package_to_npm does not retry unrelated publish failures", () => { + const result = runPublishHelper({ pnpmMode: "non-tlog-failure" }); + + assert.notEqual(result.status, 0); + assert.doesNotMatch(result.calls, /npm view/); + assert.doesNotMatch(result.calls, /--provenance=false/); +}); + +test("publish_package_to_npm does not mask failures when caller has no pipefail", () => { + const result = runPublishHelper({ pnpmMode: "non-tlog-failure", callerPipefail: false }); + + assert.notEqual(result.status, 0); + assert.doesNotMatch(result.calls, /npm view/); + assert.doesNotMatch(result.calls, /--provenance=false/); +}); + +test("publish_package_to_npm does not retry stable publishes without provenance", () => { + const result = runPublishHelper({ pnpmMode: "tlog-then-success", distTag: "latest" }); + + assert.notEqual(result.status, 0); + assert.match(result.calls, /^npm view @paperclipai\/example@1\.2\.3 version$/m); + assert.doesNotMatch(result.calls, /--provenance=false/); +}); diff --git a/scripts/release.sh b/scripts/release.sh index 35da947d..aa394e91 100755 --- a/scripts/release.sh +++ b/scripts/release.sh @@ -258,7 +258,7 @@ else [ -z "$pkg_dir" ] && continue release_info " Publishing $pkg_name@$pkg_version" cd "$REPO_ROOT/$pkg_dir" - pnpm publish --no-git-checks --tag "$DIST_TAG" --access public + publish_package_to_npm "$DIST_TAG" "$pkg_name" "$pkg_version" done <<< "$VERSIONED_PACKAGE_INFO" release_info " ✓ Published all packages under dist-tag $DIST_TAG" fi