Commit Graph

4 Commits

Author SHA1 Message Date
Devin Foley d5889919a9 feat(commitperclip): widen linked-issue gate, add dedup-search check (#7632)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - Contributor onboarding leans on a small set of CI gates
("commitperclip") that read PR metadata and enforce the rules in
CONTRIBUTING.md
> - We recently landed two new contributor rules: every PR must either
link an existing issue or inline an issue-template-shaped description,
and the author must affirm they searched for duplicate PRs first
> - The existing `check-pr-linked-issue` gate only accepted
`Fixes/Closes/Resolves #N`, and there was no gate at all for the
dedup-search affirmation, so the new rules were unenforced
> - This pull request widens the linked-issue gate (accept `Refs #N`,
accept inline template-shaped descriptions) and adds a new dedup-search
gate wired into `run-quality-gates`
> - The benefit is that the rules we ask contributors to follow are now
mechanically enforced, lowering review noise without raising contributor
friction

## Linked Issues or Issue Description

Refs #4260 — the PR cited in the original rule discussion, whose body is
the canonical example of the inline-issue-description shape this gate
now accepts.

**Problem or motivation**

The repo recently adopted two contributor rules: (1) link the issue your
PR fixes or inline a description in the issue-template shape, and (2)
confirm you searched for duplicate PRs before opening one. Neither rule
was enforced — the existing linked-issue check only matched
`Fixes/Closes/Resolves #N`, and nothing looked for the dedup-search
affirmation. Reviewers had to remember and re-state the rules by hand on
every PR.

**Proposed solution**

Widen `check-pr-linked-issue.mjs` and add a new
`check-pr-dedup-search.mjs` so commitperclip enforces the rules already
documented in CONTRIBUTING.md and the PR template:

- Accept `Refs #N` as a valid link verb alongside `Fixes`, `Closes`,
`Resolves`.
- Accept a PR body with an inline issue-template-shaped description (≥3
fields matched against the bug, feature, or adapter templates) as a
valid alternative to a linked issue.
- New `check-pr-dedup-search.mjs` looks for a checked checkbox affirming
the author searched for similar/duplicate/prior PRs, wired into
`run-quality-gates.mjs` with the same skip/override semantics as the
other gates.

**Alternatives considered**

- A single regex over the whole PR body looking for issue-template field
names — rejected, too brittle and gave no useful error message when it
failed. The per-field counter lets us tell the author exactly how many
template fields we matched and which template they're closest to.
- Making the dedup-search check live inside `check-pr-linked-issue` —
rejected, the two rules are orthogonal and a separate gate gives a
cleaner error message and respects `[skip-quality-gates]` independently.

**Roadmap alignment**

This is contributor-workflow plumbing for rules that already landed on
`master`. It is not a roadmap feature and does not overlap with planned
core work.

## What Changed

- `check-pr-linked-issue.mjs`: accept `Refs #N`; add
`hasInlineIssueDescription` (per-template field counter, ≥3 fields) so a
fully inlined description satisfies the gate.
- `check-pr-dedup-search.mjs`: new gate, looks for a checked
dedup-search checkbox in the PR body, with clear failure guidance.
- `run-quality-gates.mjs`: wire the new gate in alongside the existing
checks.
- `CONTRIBUTING.md`: "Before You Start: Search First" callout pointing
at the PR-template checkbox the gate enforces.
- Tests: unit tests for `Refs #N`, the inline-description threshold
(bolded labels, plain labels, headings), and the dedup-search gate
across checked/unchecked/missing/skip-prefix paths.

## Verification

- `node --test .github/scripts/tests/check-pr-linked-issue.test.mjs` →
all pass
- `node --test .github/scripts/tests/check-pr-dedup-search.test.mjs` →
all pass
- Full repo test matrix: 116/116 pass locally

For the gate itself, this PR exercises both new code paths: the body
inlines a feature-template-shaped description with ≥3 matched fields,
and the dedup-search checkbox below is checked.

## Risks

Low risk. The change is contributor-CI plumbing — no runtime or
migration impact. The widened linked-issue gate is strictly more
permissive (it can only flip prior FAILs to PASS), and the new
dedup-search gate honors the existing `[skip-quality-gates]` prefix, so
an author can always bypass it the same way as the other gates if
needed.

## Model Used

Claude Opus 4.7 (`claude-opus-4-7`), extended-thinking enabled, tool use
for filesystem + shell.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-05 22:10:52 -07:00
Devin Foley 03e1e3abd2 Revert "Remove linked-issue gate from commitperclip" (#7426)
Reverts paperclipai/paperclip#7423

Decided to keep this in place so we can automate issue reproduction in
the future. We all make mistakes. Even me, if you can believe it.
2026-06-03 08:54:52 -07:00
Devin Foley 68401f82f3 Remove linked-issue gate from commitperclip (#7423)
## Thinking Path

> - Paperclip orchestrates AI agents for zero-human companies
> - The `commitperclip` PR quality gates enforce hygiene on every PR
before merge
> - One of those gates required PRs to link to a tracking issue, which
adds friction for small/internal changes that don't need a tracker entry
> - The repository owner decided the linked-issue requirement is no
longer the right default
> - This pull request removes the linked-issue gate (the script, its
tests, and the orchestrator wiring)
> - The benefit is fewer false-failing PR checks and one less mandatory
authoring step

## What Changed

- Deleted `.github/scripts/check-pr-linked-issue.mjs`
- Deleted `.github/scripts/tests/check-pr-linked-issue.test.mjs`
- Removed the `checkLinkedIssue` import, the
`Promise.resolve(checkLinkedIssue(...))` entry in the `Promise.all`
block, the `issueResult` destructured binding, and the
`...issueResult.failures` spread from
`.github/scripts/run-quality-gates.mjs`

## Verification

- `node --test .github/scripts/tests/*.test.mjs` — 72/72 tests pass
across the remaining 4 gate suites
- `git grep -n
'check-pr-linked-issue\|checkLinkedIssue\|check-pr-linked'` — no matches
- Inspected `run-quality-gates.mjs` — no orphaned `issueResult`
references

## Risks

- Low risk. Pure removal of one optional gate; the
`.github/workflows/commitperclip-review.yml` workflow only invokes the
orchestrator and needs no changes. PR template and `CONTRIBUTING.md` do
not mention linked issues, so no docs change is required.

## Model Used

- Claude (Anthropic), `claude-opus-4-7`, extended-thinking mode, tool
use enabled

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable (existing
gate-suite tests still pass; removed gate's tests deleted with it)
- [x] If this change affects the UI, I have included before/after
screenshots (n/a — CI script change)
- [x] I have updated relevant documentation to reflect my changes (no
docs reference the removed gate)
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-02 17:12:41 -07:00
brandonburr 96feaa331a feat(commitperclip): add automated PR quality and security gates (#6469)
Fixes #6470

## Thinking Path

> - Paperclip is an open-source AI agent platform receiving a high
volume of community PRs — currently 2,398 open
> - The contributor experience is broken: PRs sit for months with no
feedback, contributors don't know why they're stuck, and maintainers
spend review time on PRs that are missing basics
> - Common problems: no linked issue, no test coverage, incomplete PR
template, manually-edited lockfile — all catchable before human review
> - At the same time, accepting untrusted PRs from unknown contributors
is a real attack surface: malicious packages, secret injection,
tampering with CI scripts, and code touching the sensitive paths from
the April security advisories
> - This PR adds automated gates that run on every PR: quality failures
get a clear comment telling contributors exactly what to fix, security
concerns are silently flagged as draft advisories and block merge via a
pending check run
> - The benefit is a dramatically faster feedback loop for good-faith
contributors and a meaningful security layer for the maintainers
reviewing them

## What Changed

- **`.github/workflows/commitperclip-review.yml`** — new workflow using
`pull_request_target` (runs in base branch context, has secrets, never
executes PR code). Runs quality gates + security gates on every PR
open/update.
- **`.github/dependabot.yml`** — weekly automated dependency
vulnerability PRs for npm and GitHub Actions.
- **`.github/scripts/get-bot-token.mjs`** — generates a short-lived
commitperclip installation token from `COMMITPERCLIP_KEY` secret.
- **`.github/scripts/run-quality-gates.mjs`** — orchestrates 5 quality
gates, posts/updates a single consolidated comment on the PR.
- **`.github/scripts/check-pr-template.mjs`** — validates all 5 required
template sections, Thinking Path depth (≥3 sentences), Model Used not
placeholder.
- **`.github/scripts/check-pr-linked-issue.mjs`** — requires `Fixes
#NNN` or issue URL in PR body.
- **`.github/scripts/check-pr-test-coverage.mjs`** — requires at least
one test file in the diff.
- **`.github/scripts/check-pr-lockfile.mjs`** — blocks manual
`pnpm-lock.yaml` edits (only the refresh bot may change it).
- **`.github/scripts/check-pr-dependencies.mjs`** — informational
comment when new npm packages are added.
- **`.github/scripts/check-pr-security.mjs`** — 6 silent security
checks: secret patterns, CI workflow tampering, build script changes,
supply chain (new packages in lockfile), suspicious test patterns
(outbound network/shell exec/env var reads), and changes to the 9
sensitive path prefixes from the April advisories. When any fire:
creates a draft security advisory + sets `security-review` check to
`in_progress` (blocks merge). When clean: sets `security-review` to
`success`.
- **`actions/dependency-review-action@v4`** — per-PR dependency
vulnerability check (fails if new dep has known CVE).
- **44 unit tests** across all gate modules (`node:test`, no external
deps).

## Verification

Run all unit tests locally:
```bash
node --test .github/scripts/tests/*.test.mjs
```
Expected: 44 pass, 0 fail.

End-to-end: open a PR missing the template, linked issue, and test files
→ commitperclip posts a consolidated comment listing all failures. Open
a PR with all gates satisfied → ` All checks passing` comment posted,
all check runs green.

## Risks

**`pull_request_target` security model:** This workflow runs in base
branch context and has access to secrets. It explicitly checks out `ref:
master` (never PR code) and reads the PR diff via GitHub API only — no
PR code is ever executed. This is the correct pattern for running
secret-bearing checks on fork PRs; deviating from it (e.g. checking out
the PR branch) would be a security vulnerability.

**False positives on security gates:** The sensitive-path gate flags any
PR touching the 9 path prefixes from the April advisories. Legitimate
fixes to those paths will trigger draft advisories. This is intentional
— those paths warrant a human look regardless. The `security-review`
check can be manually resolved by a maintainer once reviewed.

**commitperclip not yet installed:** Until the app is installed on this
repo and the `COMMITPERCLIP_KEY` secret is added, the workflow will fail
on the token generation step. The quality gate comment won't post, but
Dependency Review will still run independently.

## Model Used

Claude Sonnet 4.5, 200k context window, extended thinking enabled, tool
use: read/edit files, bash execution, GitHub API calls

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have run tests locally and they pass (44/44)
- [x] I have added or updated tests where applicable (44 unit tests
across all gate modules)
- [ ] If this change affects the UI, I have included before/after
screenshots (N/A — CI only)
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [x] I will address all Greptile and reviewer comments before
requesting merge

---

## One-time setup needed from you, Dotta

1. **Install commitperclip app** on this repo:
https://github.com/apps/commitperclip/installations/new
2. **Add `COMMITPERCLIP_KEY`** as a repository secret (Actions →
Secrets) — ask @brandonburr for the key
3. **Add `security_advisories: write` and `checks: write`** to the
commitperclip app permissions (commit-capital org → Settings → Apps →
commitperclip → Permissions)
4. **Install Socket.dev** from GitHub Marketplace for supply chain
scanning
5. **Branch protection** (optional but recommended): require
`commitperclip-review` and `security-review` checks to pass before merge

## Dashboard integration note

The `commitperclip-review` check run result maps cleanly to your PR
triage dashboard. A single filter on your Worker:

```javascript
const gatesCheck = checkRuns.find(r => r.name === 'commitperclip-review');
if (gatesCheck?.conclusion === 'failure') return null; // filter from queue
```

For security flags: `GET
/repos/paperclipai/paperclip/security-advisories?state=draft` — advisory
titles include `PR #NNN` for cross-referencing. PRs with a matching
draft advisory have `security-review` in `in_progress` state (grey
spinner, can't merge via branch protection).

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Devin Foley <devin@devinfoley.com>
Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-01 09:52:53 -07:00