Arnike/paperclip
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: parse YAML block scalar skill descriptions (#5046)

Browse Source 
## Thinking Path

> - Paperclip is the open source control plane teams use to manage AI
agents for work.
> - Company skills are imported from `SKILL.md` files and rely on YAML
frontmatter to describe what each skill does.
> - Multi-line descriptions commonly use YAML block scalars (`>` and
`|`), but the broken parser path behind #4989 reduced those descriptions
to a literal `>` or `|`.
> - The earliest contributor fix for that bug was PR #5046, so this
branch keeps that PR as the canonical merge target instead of replacing
it.
> - Follow-up work from #5071 and #8258 was then transplanted onto this
earlier branch so the final PR preserves contributor credit while still
shipping the strongest complete fix.
> - The resulting change fixes block-scalar parsing in the shared
frontmatter path, aligns server company-skill imports with that shared
parser, and prevents already-stale stored markers from rendering as junk
in the UI.

## Linked Issues or Issue Description

- Fixes #4989
- Refs #2863
- Refs #788
- Related superseded PRs: #5071, #8258

## What Changed

- Kept the original PR #5046 server-side company-skill fix and
regression coverage as the base branch history.
- Added the missing YAML chomping and indicator hardening explored
further in #5071.
- Moved frontmatter parsing to the shared parser path so
`packages/shared`, `packages/skills-catalog`, and server company-skill
imports stay aligned.
- Added UI summary sanitization and fallback handling so stale stored
`>` / `|` values no longer render as visible junk in company-skill
cards.
- Added regression coverage for shared frontmatter parsing,
skills-catalog parsing, company-skill imports, and stale-summary
fallback behavior.

## Verification

- Passed locally: `pnpm exec vitest run
packages/shared/src/frontmatter.test.ts
packages/skills-catalog/src/frontmatter.test.ts
server/src/__tests__/company-skills.test.ts
ui/src/lib/company-skill-summary.test.ts`
- Passed locally: `pnpm --filter @paperclipai/shared typecheck`
- Passed locally: `pnpm --filter @paperclipai/skills-catalog typecheck`
- Not fully runnable in this worktree: `pnpm --filter
@paperclipai/server typecheck` currently fails in `packages/plugins/sdk`
before reaching server code because local workspace `node_modules` type
deps are missing (`TS2688` for `node` / `react`).
- GitHub Actions / PR checks are rerunning on PR #5046 head
`005290b7557725abf748d00f36dd24ea0d919aba`.

## Risks

- Medium-low risk: the fix now touches shared parser code, server
company-skill imports, and UI fallback display rather than only the
server import path.
- The parser is still intentionally narrower than a full YAML
implementation; this change focuses on block-scalar correctness and the
stale-description rendering path relevant to #4989 / #2863.
- This branch intentionally supersedes narrower overlapping work from
#5071 and duplicate work from #8258 once the survivor PR is green.

> For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and
discuss it in `#dev` before opening the PR. Feature PRs that overlap
with planned core work may need to be redirected — check the roadmap
first. See `CONTRIBUTING.md`.

## Model Used

- OpenAI Codex / GPT-5-based coding agent with local shell and
code-editing tools enabled.

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [ ] If this change affects the UI, I have included before/after
screenshots
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Devin Foley <devin@paperclip.ing>
This commit is contained in:
MrBob
2026-06-18 16:01:11 -03:00
committed by GitHub
parent 5320a44088
commit 5f16efb3d0
13 changed files with 496 additions and 316 deletions
Download Patch File Download Diff File Expand all files Collapse all files
server/src/__tests__/company-skills.test.ts
+120
View File
@@ -185,6 +185,126 @@ describe("project workspace skill discovery", () => {
],
});
});
it("parses folded and literal block scalar descriptions in skill frontmatter", async () => {
const foldedWorkspace = await makeTempDir("paperclip-folded-skill-yaml-");
await fs.mkdir(foldedWorkspace, { recursive: true });
await fs.writeFile(
path.join(foldedWorkspace, "SKILL.md"),
[
"---",
"name: Folded Metadata Skill",
"description: >",
" Use when you need website engagement data - sessions,",
" pageviews, and conversions.",
"",
"---",
"",
"# Folded Metadata Skill",
"",
].join("\n"),
"utf8",
);
const literalWorkspace = await makeTempDir("paperclip-literal-skill-yaml-");
await fs.mkdir(literalWorkspace, { recursive: true });
await fs.writeFile(
path.join(literalWorkspace, "SKILL.md"),
[
"---",
"name: Literal Metadata Skill",
"description: |",
" First line.",
" Second line.",
"",
"metadata:",
" clipNote: |",
" Keep this line.",
" And this one.",
"",
" stripNote: |-",
" Strip this line.",
" And this one.",
"---",
"",
"# Literal Metadata Skill",
"",
].join("\n"),
"utf8",
);
const stripWorkspace = await makeTempDir("paperclip-strip-skill-yaml-");
await fs.mkdir(stripWorkspace, { recursive: true });
await fs.writeFile(
path.join(stripWorkspace, "SKILL.md"),
[
"---",
"name: Strip Metadata Skill",
"description: >-",
" Strip this folded description",
" without a trailing newline.",
"---",
"",
"# Strip Metadata Skill",
"",
].join("\n"),
"utf8",
);
const folded = await readLocalSkillImportFromDirectory(
"33333333-3333-4333-8333-333333333333",
foldedWorkspace,
{ inventoryMode: "full" },
);
const literal = await readLocalSkillImportFromDirectory(
"33333333-3333-4333-8333-333333333333",
literalWorkspace,
{ inventoryMode: "full" },
);
const strip = await readLocalSkillImportFromDirectory(
"33333333-3333-4333-8333-333333333333",
stripWorkspace,
{ inventoryMode: "full" },
);
expect(folded.description).toBe(
"Use when you need website engagement data - sessions, pageviews, and conversions.",
);
expect(literal.description).toBe("First line.\nSecond line.");
expect(literal.metadata).toMatchObject({
clipNote: "Keep this line.\nAnd this one.\n",
stripNote: "Strip this line.\nAnd this one.",
});
expect(strip.description).toBe("Strip this folded description without a trailing newline.");
});
it("parses YAML block-scalar chomping variants from SKILL.md frontmatter", async () => {
const workspace = await makeTempDir("paperclip-block-scalar-chomp-skill-");
await fs.mkdir(workspace, { recursive: true });
await fs.writeFile(
path.join(workspace, "SKILL.md"),
[
"---",
"name: Block Scalar Chomp Skill",
"description: >-",
" First line",
" second line",
"---",
"",
"# Block Scalar Chomp Skill",
"",
].join("\n"),
"utf8",
);
const imported = await readLocalSkillImportFromDirectory(
"33333333-3333-4333-8333-333333333333",
workspace,
{ inventoryMode: "full" },
);
expect(imported.description).toBe("First line second line");
});
});
describe("missing local skill reconciliation", () => {
server/src/services/company-skills.ts
+1 -134
View File
@@ -45,7 +45,7 @@ import type {
CompanySkillVersionCreateRequest,
CompanySkillVersionFileInventoryEntry,
} from "@paperclipai/shared";
import { normalizeAgentUrlKey } from "@paperclipai/shared";
import { normalizeAgentUrlKey, parseFrontmatterMarkdown } from "@paperclipai/shared";
import { resolvePaperclipInstanceRoot } from "../home-paths.js";
import { conflict, notFound, unprocessable } from "../errors.js";
import { ghFetch, gitHubApiBase, resolveRawGitHubUrl } from "./github-fetch.js";
@@ -565,139 +565,6 @@ function deriveTrustLevel(fileInventory: CompanySkillFileInventoryEntry[]): Comp
return "markdown_only";
}
function prepareYamlLines(raw: string) {
return raw
.split("\n")
.map((line) => ({
indent: line.match(/^ */)?.[0].length ?? 0,
content: line.trim(),
}))
.filter((line) => line.content.length > 0 && !line.content.startsWith("#"));
}
function parseYamlScalar(rawValue: string): unknown {
const trimmed = rawValue.trim();
if (trimmed === "") return "";
if (trimmed === "null" || trimmed === "~") return null;
if (trimmed === "true") return true;
if (trimmed === "false") return false;
if (trimmed === "[]") return [];
if (trimmed === "{}") return {};
if (/^-?\d+(\.\d+)?$/.test(trimmed)) return Number(trimmed);
if (trimmed.startsWith("\"") || trimmed.startsWith("[") || trimmed.startsWith("{")) {
try {
return JSON.parse(trimmed);
} catch {
return trimmed;
}
}
return trimmed;
}
function parseYamlBlock(
lines: Array<{ indent: number; content: string }>,
startIndex: number,
indentLevel: number,
): { value: unknown; nextIndex: number } {
let index = startIndex;
while (index < lines.length && lines[index]!.content.length === 0) index += 1;
if (index >= lines.length || lines[index]!.indent < indentLevel) {
return { value: {}, nextIndex: index };
}
const isArray = lines[index]!.indent === indentLevel && lines[index]!.content.startsWith("-");
if (isArray) {
const values: unknown[] = [];
while (index < lines.length) {
const line = lines[index]!;
if (line.indent < indentLevel) break;
if (line.indent !== indentLevel || !line.content.startsWith("-")) break;
const remainder = line.content.slice(1).trim();
index += 1;
if (!remainder) {
const nested = parseYamlBlock(lines, index, indentLevel + 2);
values.push(nested.value);
index = nested.nextIndex;
continue;
}
const inlineObjectSeparator = remainder.indexOf(":");
if (
inlineObjectSeparator > 0 &&
!remainder.startsWith("\"") &&
!remainder.startsWith("{") &&
!remainder.startsWith("[")
) {
const key = remainder.slice(0, inlineObjectSeparator).trim();
const rawValue = remainder.slice(inlineObjectSeparator + 1).trim();
const nextObject: Record<string, unknown> = {
[key]: parseYamlScalar(rawValue),
};
if (index < lines.length && lines[index]!.indent > indentLevel) {
const nested = parseYamlBlock(lines, index, indentLevel + 2);
if (isPlainRecord(nested.value)) {
Object.assign(nextObject, nested.value);
}
index = nested.nextIndex;
}
values.push(nextObject);
continue;
}
values.push(parseYamlScalar(remainder));
}
return { value: values, nextIndex: index };
}
const record: Record<string, unknown> = {};
while (index < lines.length) {
const line = lines[index]!;
if (line.indent < indentLevel) break;
if (line.indent !== indentLevel) {
index += 1;
continue;
}
const separatorIndex = line.content.indexOf(":");
if (separatorIndex <= 0) {
index += 1;
continue;
}
const key = line.content.slice(0, separatorIndex).trim();
const remainder = line.content.slice(separatorIndex + 1).trim();
index += 1;
if (!remainder) {
const nested = parseYamlBlock(lines, index, indentLevel + 2);
record[key] = nested.value;
index = nested.nextIndex;
continue;
}
record[key] = parseYamlScalar(remainder);
}
return { value: record, nextIndex: index };
}
function parseYamlFrontmatter(raw: string): Record<string, unknown> {
const prepared = prepareYamlLines(raw);
if (prepared.length === 0) return {};
const parsed = parseYamlBlock(prepared, 0, prepared[0]!.indent);
return isPlainRecord(parsed.value) ? parsed.value : {};
}
function parseFrontmatterMarkdown(raw: string): { frontmatter: Record<string, unknown>; body: string } {
const normalized = raw.replace(/\r\n/g, "\n");
if (!normalized.startsWith("---\n")) {
return { frontmatter: {}, body: normalized.trim() };
}
const closing = normalized.indexOf("\n---\n", 4);
if (closing < 0) {
return { frontmatter: {}, body: normalized.trim() };
}
const frontmatterRaw = normalized.slice(4, closing).trim();
const body = normalized.slice(closing + 5).trim();
return {
frontmatter: parseYamlFrontmatter(frontmatterRaw),
body,
};
}
async function fetchText(url: string) {
const response = await ghFetch(url);
if (!response.ok) {