2853a9ae69
## Thinking Path > - Paperclip is the open source app people use to manage AI agents for work > - Every PR runs the `PR` GitHub Actions workflow, whose `verify` gate fans out into parallel test lanes (general tests, serialized server route suites, build, typecheck) > - The `General tests (server)` lane had grown into the run's critical path: it executed all ~213 non-route server suites serially in a single job (~7.2m of test time), more than 2x any other job > - It runs serially because `server/vitest.config.ts` pins `maxWorkers: 1`, so server suites cannot parallelize within a single runner — the only lever is spreading them across runners > - This pull request shards that lane into 3 even partitions that run on separate runners, mirroring the 4-way sharding already used for the serialized route suites > - The benefit is the lane drops from ~7.7m to ~2.4m/shard, cutting overall PR wall time roughly in half (~8.5m → ~4.2m) ## Linked Issues or Issue Description No public GitHub issue exists for this work, so the underlying issue is described inline following the feature-request template. ### Problem or motivation PR CI wall time had crept back up to ~8.5m. On a recent fully-green run, the `General tests (server)` job took 7.72m — more than double any other job and the clear critical path. Of that, 7.23m was pure test execution (dependency install was a cached 0.27m). The job ran all server suites that are not route/authz tests (213 files) one after another, because the server vitest project pins `maxWorkers: 1`, making these suites inherently serial within a single runner. ### Proposed solution Shard the general-server lane across 3 parallel runners — the same technique the route/authz suites already use — so the suite set is split into even, deterministic partitions that run concurrently. Add a regression test that proves the shards always cover the full suite set with no gaps or overlap. ### Alternatives considered - **Raise `maxWorkers` for the server project** to parallelize within one runner — rejected: the server suites share process-level state (DB/port), which is exactly why `maxWorkers: 1` is pinned. - **Two shards instead of three** — would leave the lane at ~3.6m, still above the next bottleneck (Canary Dry Run, ~4.1m wouldn't be the gate). Three lands the lane comfortably below it. - **Do nothing / accept the slow lane** — rejected: it gates every PR. ### Roadmap alignment Developer-experience / CI tooling. Not core product roadmap work; does not overlap with planned features in `ROADMAP.md`. ## What Changed - `scripts/run-vitest-stable.mjs`: the `general-server` general-test group now accepts `--shard-index` / `--shard-count`. It enumerates the full server test set (the whole `server/src` tree, minus the route/authz suites that already run in their own serialized shards) and splits it deterministically by modulo. The non-sharded local invocation (`pnpm test:run:general --group general-server`) is unchanged. - `.github/workflows/pr.yml`: the `general_tests` matrix runs `general-server` as 3 parallel shards (1/3, 2/3, 3/3). Workspace groups are unchanged. The `verify` gate already aggregates the whole matrix result, so the required check name is unaffected. - `scripts/__tests__/run-vitest-stable-shard.test.mjs`: a `node:test` suite asserting the 3 shards form a complete, non-overlapping partition of the general-server set, that no route/authz suite leaks into it, and that shard flags are rejected for the parallel workspace groups. Wired into the `policy` job. ## Verification - New partition test passes locally: `node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs` (3/3). - Confirmed the 3 shards form a complete, non-overlapping partition of all 213 files (71/71/71). - Ran a live thin shard (3 real server suites, including one outside `__tests__`) — 23 tests passed, confirming positional-include execution works end to end. - This PR's own CI is the authoritative check: all three `General tests (server (n/3))` jobs went green on the prior run, collectively covering every suite the old single job ran. ## Risks - Low risk. No product code changes — only test orchestration and CI matrix. Shard partitioning is deterministic and is now covered by an automated test that fails if the partition ever develops a gap or overlap. Modulo-on-sorted-filenames balances duration reasonably, matching the approach already proven by the serialized route shards. ## Model Used - Claude (Anthropic), `claude-opus-4-8`, extended thinking + tool use (agentic coding via Paperclip). ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have not referenced internal/instance-local Paperclip issues or links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip` URLs) - [x] My branch name describes the change (e.g. `docs/...`, `fix/...`) and contains no internal Paperclip ticket id or instance-derived details - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [ ] If this change affects the UI, I have included before/after screenshots (N/A — no UI change) - [x] I have updated relevant documentation to reflect my changes (inline comments explain the sharding rationale) - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green (pending this PR's run) - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups (pending review) - [x] I will address all Greptile and reviewer comments before requesting merge Co-authored-by: Paperclip <noreply@paperclip.ing>
419 lines
13 KiB
YAML
419 lines
13 KiB
YAML
name: PR
|
|
|
|
on:
|
|
pull_request:
|
|
branches:
|
|
- master
|
|
|
|
concurrency:
|
|
group: pr-${{ github.event.pull_request.number }}
|
|
cancel-in-progress: true
|
|
|
|
jobs:
|
|
policy:
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
outputs:
|
|
lockfile_regenerated: ${{ steps.regen_lockfile.outputs.regenerated }}
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
with:
|
|
fetch-depth: 0
|
|
|
|
- name: Block manual lockfile edits
|
|
if: >-
|
|
github.head_ref != 'chore/refresh-lockfile' &&
|
|
github.event.pull_request.user.login != 'dependabot[bot]'
|
|
run: |
|
|
# Diff the PR branch against its merge base so recent base-branch commits
|
|
# do not masquerade as changes made by the PR itself.
|
|
changed="$(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")"
|
|
if printf '%s\n' "$changed" | grep -qx 'pnpm-lock.yaml'; then
|
|
echo "Do not commit pnpm-lock.yaml in pull requests. CI owns lockfile updates."
|
|
exit 1
|
|
fi
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
run_install: false
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
|
|
- name: Validate Dockerfile deps stage
|
|
run: node ./scripts/check-docker-deps-stage.mjs
|
|
|
|
- name: Reject git push in adapter/runtime code
|
|
run: node ./scripts/check-no-git-push.mjs
|
|
|
|
- name: Test no-git-push check
|
|
run: node --test ./scripts/check-no-git-push.test.mjs
|
|
|
|
- name: Test general-server shard partition
|
|
run: node --test ./scripts/__tests__/run-vitest-stable-shard.test.mjs
|
|
|
|
- name: Validate release package manifest
|
|
run: node ./scripts/release-package-map.mjs check
|
|
|
|
- name: Verify release package bootstrap for changed manifests
|
|
run: |
|
|
mapfile -t changed_paths < <(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")
|
|
PAPERCLIP_RELEASE_BOOTSTRAP_BASE_SHA="${{ github.event.pull_request.base.sha }}" \
|
|
node ./scripts/check-release-package-bootstrap.mjs "${changed_paths[@]}"
|
|
|
|
- name: Validate dependency resolution when manifests change
|
|
id: regen_lockfile
|
|
run: |
|
|
changed="$(git diff --name-only "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}")"
|
|
manifest_pattern='(^|/)package\.json$|^pnpm-workspace\.yaml$|^\.npmrc$|^pnpmfile\.(cjs|js|mjs)$'
|
|
if printf '%s\n' "$changed" | grep -Eq "$manifest_pattern"; then
|
|
pnpm install --lockfile-only --ignore-scripts --no-frozen-lockfile
|
|
echo "regenerated=1" >> "$GITHUB_OUTPUT"
|
|
else
|
|
echo "regenerated=0" >> "$GITHUB_OUTPUT"
|
|
fi
|
|
|
|
# Manifest-only PRs (where pnpm-lock.yaml stays at base because the policy
|
|
# job above blocks committing it) need the regenerated lockfile for the
|
|
# downstream `pnpm install --frozen-lockfile` steps. Upload it here so
|
|
# every job consumes the same hash without recomputing.
|
|
- name: Upload regenerated lockfile for downstream jobs
|
|
if: steps.regen_lockfile.outputs.regenerated == '1'
|
|
uses: actions/upload-artifact@v7
|
|
with:
|
|
name: pr-lockfile
|
|
path: pnpm-lock.yaml
|
|
retention-days: 1
|
|
if-no-files-found: error
|
|
|
|
typecheck_release_registry:
|
|
name: Typecheck + Release Registry
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Typecheck workspaces whose build scripts skip TypeScript
|
|
run: pnpm run typecheck:build-gaps
|
|
|
|
- name: Verify release registry test coverage
|
|
run: pnpm run test:release-registry
|
|
|
|
general_tests:
|
|
name: General tests (${{ matrix.group_label }})
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
# The server suite is pinned to maxWorkers=1 (server/vitest.config.ts),
|
|
# so it can only be parallelized across runners. Shard it to keep this
|
|
# lane off the PR critical path.
|
|
- group: general-server
|
|
group_label: server (1/3)
|
|
shard_index: 0
|
|
shard_count: 3
|
|
- group: general-server
|
|
group_label: server (2/3)
|
|
shard_index: 1
|
|
shard_count: 3
|
|
- group: general-server
|
|
group_label: server (3/3)
|
|
shard_index: 2
|
|
shard_count: 3
|
|
- group: general-workspaces-a
|
|
group_label: workspaces-a
|
|
- group: general-workspaces-b
|
|
group_label: workspaces-b
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Run grouped general test suites
|
|
run: |
|
|
if [ -n "${{ matrix.shard_count }}" ]; then
|
|
pnpm test:run:general -- --group '${{ matrix.group }}' \
|
|
--shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}
|
|
else
|
|
pnpm test:run:general -- --group '${{ matrix.group }}'
|
|
fi
|
|
|
|
verify:
|
|
# Preserve the legacy required-check name while the underlying work runs in parallel.
|
|
name: verify
|
|
if: ${{ always() }}
|
|
needs: [typecheck_release_registry, general_tests, build]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 5
|
|
|
|
steps:
|
|
- name: Fail if any split verify lane failed
|
|
env:
|
|
TYPECHECK_RELEASE_REGISTRY_RESULT: ${{ needs.typecheck_release_registry.result }}
|
|
GENERAL_TESTS_RESULT: ${{ needs.general_tests.result }}
|
|
BUILD_RESULT: ${{ needs.build.result }}
|
|
run: |
|
|
test "$TYPECHECK_RELEASE_REGISTRY_RESULT" = "success"
|
|
test "$GENERAL_TESTS_RESULT" = "success"
|
|
test "$BUILD_RESULT" = "success"
|
|
|
|
build:
|
|
name: Build
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Build
|
|
run: pnpm build
|
|
|
|
verify_serialized_server:
|
|
name: Verify serialized server suites (${{ matrix.shard_label }})
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- shard_index: 0
|
|
shard_count: 4
|
|
shard_label: 1/4
|
|
- shard_index: 1
|
|
shard_count: 4
|
|
shard_label: 2/4
|
|
- shard_index: 2
|
|
shard_count: 4
|
|
shard_label: 3/4
|
|
- shard_index: 3
|
|
shard_count: 4
|
|
shard_label: 4/4
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Run serialized server test shard
|
|
run: pnpm test:run:serialized -- --shard-index ${{ matrix.shard_index }} --shard-count ${{ matrix.shard_count }}
|
|
|
|
canary_dry_run:
|
|
name: Canary Dry Run
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 20
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
# `release.sh` always executes its Step 2/7 workspace build, even when
|
|
# `--skip-verify` bypasses the initial verification gate. release.sh
|
|
# also requires a clean working tree, so any in-place lockfile churn
|
|
# from `pnpm install --frozen-lockfile` must be reverted first — unless
|
|
# the policy job uploaded a regenerated lockfile (manifest-changing
|
|
# PRs), in which case we stage the artifact-restored copy into an
|
|
# ephemeral local commit so release.sh sees a clean tree and its
|
|
# workspace build sees a lockfile that matches the manifest.
|
|
- name: Release canary dry run via release.sh internal build
|
|
env:
|
|
USED_ARTIFACT_LOCKFILE: ${{ needs.policy.outputs.lockfile_regenerated || '0' }}
|
|
run: |
|
|
git checkout -B master HEAD
|
|
if [ "$USED_ARTIFACT_LOCKFILE" = "1" ]; then
|
|
git add pnpm-lock.yaml
|
|
if ! git diff --cached --quiet; then
|
|
git -c user.email=ci@paperclip.local -c user.name=CI \
|
|
commit --no-verify -m "ci(canary): stage regenerated lockfile"
|
|
fi
|
|
else
|
|
git checkout -- pnpm-lock.yaml
|
|
fi
|
|
./scripts/release.sh canary --skip-verify --dry-run
|
|
|
|
e2e:
|
|
needs: [policy]
|
|
runs-on: ubuntu-latest
|
|
timeout-minutes: 30
|
|
|
|
steps:
|
|
- name: Checkout repository
|
|
uses: actions/checkout@v6
|
|
|
|
- name: Setup pnpm
|
|
uses: pnpm/action-setup@v6
|
|
with:
|
|
version: 9.15.4
|
|
|
|
- name: Restore regenerated PR lockfile (if policy uploaded one)
|
|
uses: actions/download-artifact@v8
|
|
continue-on-error: true
|
|
with:
|
|
name: pr-lockfile
|
|
path: .
|
|
|
|
- name: Setup Node.js
|
|
uses: actions/setup-node@v6
|
|
with:
|
|
node-version: 24
|
|
cache: pnpm
|
|
|
|
- name: Install dependencies
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Verify runner Chrome
|
|
# GitHub's Ubuntu runner image already ships Google Chrome, so use that
|
|
# directly for the headless e2e lane instead of downloading Playwright
|
|
# browser bundles inside the 30 minute job budget.
|
|
run: google-chrome --version
|
|
|
|
- name: Generate Paperclip config
|
|
run: |
|
|
mkdir -p ~/.paperclip/instances/default
|
|
cat > ~/.paperclip/instances/default/config.json << 'CONF'
|
|
{
|
|
"$meta": { "version": 1, "updatedAt": "2026-01-01T00:00:00.000Z", "source": "onboard" },
|
|
"database": { "mode": "embedded-postgres" },
|
|
"logging": { "mode": "file" },
|
|
"server": { "deploymentMode": "local_trusted", "host": "127.0.0.1", "port": 3100 },
|
|
"auth": { "baseUrlMode": "auto" },
|
|
"storage": { "provider": "local_disk" },
|
|
"secrets": { "provider": "local_encrypted", "strictMode": false }
|
|
}
|
|
CONF
|
|
|
|
- name: Run e2e tests
|
|
env:
|
|
PAPERCLIP_E2E_SKIP_LLM: "true"
|
|
PAPERCLIP_PLAYWRIGHT_CHANNEL: "chrome"
|
|
run: pnpm run test:e2e
|
|
|
|
- name: Upload Playwright report
|
|
uses: actions/upload-artifact@v7
|
|
if: always()
|
|
with:
|
|
name: playwright-report
|
|
path: |
|
|
tests/e2e/playwright-report/
|
|
tests/e2e/test-results/
|
|
retention-days: 14
|