[codex] Render video artifact thumbnails (#7667)

## Thinking Path > - Paperclip is the open source control plane people use to manage AI agents for work. > - The artifacts surface is where operators inspect concrete outputs from agent work. > - Video artifacts currently render as HTML video previews, but browsers often paint a blank first frame until a playable frame is decoded. > - That makes generated video work products harder to recognize from the artifacts grid. > - This pull request seeks a tiny thumbnail frame after metadata loads and fades the video preview in once a frame is ready. > - The benefit is that video artifacts are easier to scan without changing the artifact data model or download flow. ## Linked Issues or Issue Description Internal source work: PAP-10477, split from PAP-10495. ### What happened? Video artifact cards can show a blank or black preview in the artifacts grid even when the underlying video artifact is valid and playable. ### Expected behavior Video artifact cards should paint a representative frame so operators can visually scan generated videos from the artifact grid. ### Steps to reproduce 1. Generate or upload a video artifact whose first decoded frame is not immediately painted by the browser. 2. Open the artifacts grid or an issue surface that renders artifact cards. 3. Observe that the video preview can appear blank until playback or decoding catches up. ### Paperclip version or commit `3c65f784b640a0012ce4672c1295331d4acd8a0c` before this PR. ### Deployment mode Board UI component behavior in local dev and hosted deployments; no database or API behavior changes. ## What Changed - Video artifact previews now seek to a small timestamp after metadata loads so a real frame can be painted. - The preview remains hidden until a frame is ready, while preserving the existing play overlay and error fallback. - Added a timeout safety valve so unusual media that never reports seek completion still becomes visible. - Added jsdom component tests covering metadata load, seek, frame-ready behavior, and the silent seek fallback. ## Screenshots | Before frame ready | After frame ready | | --- | --- | | ![Video artifact card before frame ready](https://gist.githubusercontent.com/cryppadotta/1de42c1fa916520703771bd9ad20399a/raw/01af0d1aefe9fa3227f9b92232ff7cdce73c6c3f/pap-10499-before.svg) | ![Video artifact card after frame ready](https://gist.githubusercontent.com/cryppadotta/1de42c1fa916520703771bd9ad20399a/raw/0ce810ca33e2401176fbe1e18718e6644ad8c65f/pap-10499-after.svg) | ## Verification - `pnpm exec vitest run ui/src/components/artifacts/ArtifactCard.test.tsx` passed in `.paperclip/worktrees/PAP-10495-artifact-video-thumbnails`. - `pnpm --filter @paperclipai/ui build` passed in `.paperclip/worktrees/PAP-10495-artifact-video-thumbnails`. - `pnpm build` passed in `.paperclip/worktrees/PAP-10495-artifact-video-thumbnails` before the Greptile fallback patch; the post-patch UI build covers the changed TypeScript/Vite surface. ## Risks - Low risk. The change is limited to UI preview rendering for video artifacts. - Some browsers may reject or silently ignore programmatic seeking for unusual media; the code now falls back to revealing the preview after a short timeout rather than leaving the video invisible. > For core feature work, check [`ROADMAP.md`](ROADMAP.md) first and discuss it in `#dev` before opening the PR. Feature PRs that overlap with planned core work may need to be redirected — check the roadmap first. See `CONTRIBUTING.md`. ## Model Used - OpenAI Codex coding agent based on GPT-5, with local repository inspection, shell execution, git, and GitHub CLI tool use. Runtime context window was not exposed by the environment. ## Checklist - [x] I have included a thinking path that traces from project context to this change - [x] I have specified the model used (with version and capability details) - [x] I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] I have either (a) linked existing issues with `Fixes: #` / `Closes #` / `Refs #` OR (b) described the issue in-PR following the relevant issue template - [x] I have run tests locally and they pass - [x] I have added or updated tests where applicable - [x] If this change affects the UI, I have included before/after screenshots - [x] I have updated relevant documentation to reflect my changes - [x] I have considered and documented any risks above - [ ] All Paperclip CI gates are green - [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups - [x] I will address all Greptile and reviewer comments before requesting merge --------- Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-06-06 16:18:25 -05:00
parent 71a8464fee
commit 86253f52a7
2 changed files with 133 additions and 2 deletions
@@ -1,4 +1,8 @@
+// @vitest-environment jsdom
+
 import type { ReactNode } from "react";
+import { flushSync } from "react-dom";
+import { createRoot } from "react-dom/client";
 import { renderToStaticMarkup } from "react-dom/server";
 import { describe, expect, it, vi } from "vitest";

@@ -88,6 +92,82 @@ describe("ArtifactCard", () => {
    expect(markup).toContain('preload="metadata"');
  });

+  it("seeks video previews after metadata loads so a thumbnail frame is painted", () => {
+    const container = document.createElement("div");
+    document.body.appendChild(container);
+    const root = createRoot(container);
+
+    flushSync(() => {
+      root.render(
+        <ArtifactCard
+          artifact={makeArtifact({ mediaKind: "video", contentType: "video/mp4", contentPath: "/files/clip.mp4" })}
+        />,
+      );
+    });
+
+    const video = container.querySelector("video") as HTMLVideoElement;
+    expect(video).not.toBeNull();
+    expect(video.dataset.frameReady).toBe("false");
+
+    Object.defineProperty(video, "readyState", {
+      configurable: true,
+      value: HTMLMediaElement.HAVE_METADATA,
+    });
+
+    flushSync(() => {
+      video.dispatchEvent(new Event("loadedmetadata", { bubbles: true }));
+    });
+
+    expect(video.currentTime).toBe(0.05);
+
+    flushSync(() => {
+      video.dispatchEvent(new Event("seeked", { bubbles: true }));
+    });
+
+    expect(video.dataset.frameReady).toBe("true");
+
+    flushSync(() => root.unmount());
+    container.remove();
+  });
+
+  it("reveals video previews if the browser does not report seek completion", () => {
+    vi.useFakeTimers();
+    const container = document.createElement("div");
+    document.body.appendChild(container);
+    const root = createRoot(container);
+
+    try {
+      flushSync(() => {
+        root.render(
+          <ArtifactCard
+            artifact={makeArtifact({ mediaKind: "video", contentType: "video/mp4", contentPath: "/files/clip.mp4" })}
+          />,
+        );
+      });
+
+      const video = container.querySelector("video") as HTMLVideoElement;
+      expect(video).not.toBeNull();
+      expect(video.dataset.frameReady).toBe("false");
+
+      flushSync(() => {
+        video.dispatchEvent(new Event("loadedmetadata", { bubbles: true }));
+      });
+
+      expect(video.currentTime).toBe(0.05);
+      expect(video.dataset.frameReady).toBe("false");
+
+      flushSync(() => {
+        vi.advanceTimersByTime(3000);
+      });
+
+      expect(video.dataset.frameReady).toBe("true");
+    } finally {
+      flushSync(() => root.unmount());
+      container.remove();
+      vi.useRealTimers();
+    }
+  });
+
  it("renders a falling-back video placeholder when no content path exists", () => {
    const markup = renderToStaticMarkup(
      <ArtifactCard artifact={makeArtifact({ mediaKind: "video", contentPath: null })} />,
@@ -1,4 +1,4 @@
-import { useState } from "react";
+import { type SyntheticEvent, useEffect, useRef, useState } from "react";
 import { Download, ExternalLink, Paperclip, Play } from "lucide-react";
 import type { CompanyArtifact } from "@/api/artifacts";
 import { Link } from "@/lib/router";
@@ -52,6 +52,16 @@ function ImagePreview({ artifact }: { artifact: CompanyArtifact }) {

 function VideoPreview({ artifact }: { artifact: CompanyArtifact }) {
  const [errored, setErrored] = useState(false);
+  const [frameReady, setFrameReady] = useState(false);
+  const thumbnailSeekRequested = useRef(false);
+  const frameReadyFallbackTimer = useRef<number | null>(null);
+  useEffect(() => {
+    return () => {
+      if (frameReadyFallbackTimer.current !== null) {
+        window.clearTimeout(frameReadyFallbackTimer.current);
+      }
+    };
+  }, []);
  if (errored || !artifact.contentPath) {
    return (
      <PreviewFrame className="flex items-center justify-center bg-black/80">
@@ -61,6 +71,43 @@ function VideoPreview({ artifact }: { artifact: CompanyArtifact }) {
      </PreviewFrame>
    );
  }
+
+  const markFrameReady = () => {
+    if (frameReadyFallbackTimer.current !== null) {
+      window.clearTimeout(frameReadyFallbackTimer.current);
+      frameReadyFallbackTimer.current = null;
+    }
+    setFrameReady(true);
+  };
+  const scheduleFrameReadyFallback = () => {
+    if (frameReadyFallbackTimer.current !== null) {
+      window.clearTimeout(frameReadyFallbackTimer.current);
+    }
+    frameReadyFallbackTimer.current = window.setTimeout(markFrameReady, 3000);
+  };
+  const loadThumbnailFrame = (event: SyntheticEvent<HTMLVideoElement>) => {
+    if (thumbnailSeekRequested.current) return;
+    thumbnailSeekRequested.current = true;
+    const video = event.currentTarget;
+    const duration = Number.isFinite(video.duration) && video.duration > 0 ? video.duration : 0;
+    const seekTarget = duration > 0 ? Math.min(0.12, duration / 2) : 0.05;
+    try {
+      if (Math.abs(video.currentTime - seekTarget) > 0.001) {
+        video.currentTime = seekTarget;
+        scheduleFrameReadyFallback();
+      } else {
+        markFrameReady();
+      }
+    } catch {
+      markFrameReady();
+    }
+  };
+  const handleLoadedData = (event: SyntheticEvent<HTMLVideoElement>) => {
+    if (thumbnailSeekRequested.current || event.currentTarget.currentTime > 0) {
+      markFrameReady();
+    }
+  };
+
  return (
    <PreviewFrame className="bg-black">
      <video
@@ -68,7 +115,11 @@ function VideoPreview({ artifact }: { artifact: CompanyArtifact }) {
        preload="metadata"
        muted
        playsInline
-        className="h-full w-full object-contain"
+        data-frame-ready={frameReady ? "true" : "false"}
+        className={cn("h-full w-full object-contain transition-opacity", frameReady ? "opacity-100" : "opacity-0")}
+        onLoadedMetadata={loadThumbnailFrame}
+        onLoadedData={handleLoadedData}
+        onSeeked={markFrameReady}
        onError={() => setErrored(true)}
      />
      <div className="pointer-events-none absolute inset-0 flex items-center justify-center">