feat(server): opt-in OpenTelemetry auto-instrumentation (#3735)

## Thinking Path > - Paperclip orchestrates AI agents for zero-human companies > - Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP > - Today there is no OpenTelemetry bootstrap in the server, so operators who want traces have to patch their fork or run a sidecar that captures only HTTP-level info > - An opt-in bootstrap that costs nothing when disabled is the minimum-viable surface for this audience > - The OpenTelemetry packages are heavyweight enough that we don't want them in the default dependency graph — they should load only when the operator configures an OTLP endpoint > - This pull request adds a self-contained `server/src/instrumentation.ts` that dynamically imports the OTel SDK and starts it when `OTEL_EXPORTER_OTLP_ENDPOINT` is set, and is a complete no-op otherwise ## Linked Issues or Issue Description No existing issue covers this directly — feature-gap description following the feature-request template: **Problem or motivation** Production self-hosters increasingly expect telemetry out of the box — Jaeger, Tempo, Honeycomb, Datadog, Grafana Cloud, Dynatrace all speak OTLP — but the server has no OpenTelemetry bootstrap. Operators who want traces today must patch their fork or run a sidecar that captures only HTTP-level information. **Proposed solution** An opt-in OTel bootstrap gated on `OTEL_EXPORTER_OTLP_ENDPOINT`, loaded via dynamic `import()` only when configured, so the heavyweight OTel packages stay out of the default dependency graph. **Alternatives considered** Related open PRs found during the duplicate-PR search approach observability differently: #4894 adds OTLP instrumentation to Paperclip core unconditionally, and #3752 proposes an observability plugin. Not duplicates — different layering: this PR keeps the default install dependency-free via opt-in dynamic import. ## What Changed - New `server/src/instrumentation.ts` — opt-in OpenTelemetry auto-instrumentation. Gated on `OTEL_EXPORTER_OTLP_ENDPOINT`. Respects the standard OTel env vars (`OTEL_SERVICE_NAME`, `OTEL_SERVICE_VERSION`, `OTEL_EXPORTER_OTLP_ENDPOINT`). Skips the fs/dns/net auto-instrumentations (too chatty). `sdk.start()` is wrapped in try/catch so a bad endpoint or missing native bindings doesn't crash the server. `process.once("SIGTERM" / "SIGINT", …)` for clean shutdown on the first signal only. OTel packages are loaded via dynamic `import()` so they are true optional runtime dependencies — no entries in `package.json`, no lockfile churn. - `server/src/index.ts` — import `./instrumentation.js` as the very first statement so auto-instrumentation can patch `http` / `express` / `pg` before they are evaluated by downstream modules. ## Verification - `OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 pnpm start` after `pnpm install @opentelemetry/{sdk-node,auto-instrumentations-node,exporter-trace-otlp-grpc,resources,semantic-conventions}` in `server/` — traces show up in the configured collector; HTTP, Express, and Postgres spans are populated. - `OTEL_EXPORTER_OTLP_ENDPOINT` unset — server starts with no OTel-shaped output in logs, no behavior change. - `OTEL_EXPORTER_OTLP_ENDPOINT=…` set but packages not installed — single `console.warn` at startup telling the operator which packages to install. ## Risks Low. No behavior change unless the env var is set. The bootstrap never throws into the caller; every failure path ends in `console.warn` / `console.error` and falls through to non-traced operation. ## Model Used Claude Opus 4.6 (1M context), extended thinking mode. ## Checklist - [x] I have searched GitHub for duplicate or related PRs and linked them above - [x] Thinking path traces from project context to this change - [x] Model used specified - [x] Tests run locally and pass - [x] CI green - [x] Greptile review addressed --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-12 19:44:22 +02:00
parent 937fe62d10
commit 362c30ccdc
5 changed files with 410 additions and 0 deletions
@@ -0,0 +1,110 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
+
+/**
+ * Tests for the opt-in OpenTelemetry bootstrap. The @opentelemetry/* packages
+ * are optional peer dependencies and are NOT installed in CI, which is itself
+ * part of the contract under test: with the endpoint set but packages absent,
+ * the module must warn and settle instead of crashing the server.
+ *
+ * The module reads OTEL_* env vars at import time, so each test resets the
+ * module registry and imports a fresh copy.
+ */
+
+const ENDPOINT_ENV = "OTEL_EXPORTER_OTLP_ENDPOINT";
+const PROTOCOL_ENV = "OTEL_EXPORTER_OTLP_PROTOCOL";
+
+const originalEndpoint = process.env[ENDPOINT_ENV];
+const originalProtocol = process.env[PROTOCOL_ENV];
+
+async function importFreshInstrumentation() {
+  vi.resetModules();
+  return await import("../instrumentation.js");
+}
+
+beforeEach(() => {
+  delete process.env[ENDPOINT_ENV];
+  delete process.env[PROTOCOL_ENV];
+});
+
+afterEach(() => {
+  if (originalEndpoint === undefined) delete process.env[ENDPOINT_ENV];
+  else process.env[ENDPOINT_ENV] = originalEndpoint;
+  if (originalProtocol === undefined) delete process.env[PROTOCOL_ENV];
+  else process.env[PROTOCOL_ENV] = originalProtocol;
+  vi.restoreAllMocks();
+});
+
+describe("resolveProtocol", () => {
+  it.each([
+    [undefined, "grpc", "@opentelemetry/exporter-trace-otlp-grpc"],
+    ["", "grpc", "@opentelemetry/exporter-trace-otlp-grpc"],
+    ["grpc", "grpc", "@opentelemetry/exporter-trace-otlp-grpc"],
+    ["http/protobuf", "http/protobuf", "@opentelemetry/exporter-trace-otlp-proto"],
+    ["http/json", "http/json", "@opentelemetry/exporter-trace-otlp-http"],
+    ["HTTP/JSON", "http/json", "@opentelemetry/exporter-trace-otlp-http"],
+  ])("maps OTEL_EXPORTER_OTLP_PROTOCOL=%s to %s", async (raw, protocol, packageName) => {
+    if (raw === undefined) delete process.env[PROTOCOL_ENV];
+    else process.env[PROTOCOL_ENV] = raw;
+
+    const { resolveProtocol } = await importFreshInstrumentation();
+
+    expect(resolveProtocol()).toEqual({ protocol, packageName });
+  });
+
+  it("warns and falls back to grpc on an unrecognized protocol", async () => {
+    process.env[PROTOCOL_ENV] = "carrier-pigeon";
+    // Spy before the import so the assertion holds even if a future change
+    // makes the warning fire at module load time instead of on the call.
+    const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
+
+    const { resolveProtocol } = await importFreshInstrumentation();
+
+    expect(resolveProtocol().protocol).toBe("grpc");
+    expect(warn).toHaveBeenCalledWith(expect.stringContaining("carrier-pigeon"));
+  });
+});
+
+describe("instrumentationReady", () => {
+  it("resolves immediately when OTEL_EXPORTER_OTLP_ENDPOINT is unset", async () => {
+    const { instrumentationReady } = await importFreshInstrumentation();
+
+    await expect(instrumentationReady).resolves.toBeUndefined();
+  });
+
+  it("settles with a diagnostic instead of throwing when the endpoint is set but packages are missing", async () => {
+    process.env[ENDPOINT_ENV] = "http://collector:4318";
+    const warn = vi.spyOn(console, "warn").mockImplementation(() => {});
+
+    const { instrumentationReady } = await importFreshInstrumentation();
+
+    // Bootstrap must absorb the failed dynamic imports — the server keeps
+    // booting without tracing rather than crashing on an opt-in feature.
+    await expect(instrumentationReady).resolves.toBeUndefined();
+    expect(warn).toHaveBeenCalledWith(
+      expect.stringContaining("@opentelemetry/* packages are not installed"),
+      expect.anything(),
+    );
+  });
+});
+
+describe("shutdownInstrumentation", () => {
+  it("is a no-op when tracing is off and idempotent across calls", async () => {
+    const { shutdownInstrumentation } = await importFreshInstrumentation();
+
+    const first = shutdownInstrumentation();
+    const second = shutdownInstrumentation();
+
+    // Memoized: concurrent callers share one shutdown promise.
+    expect(first).toBe(second);
+    await expect(first).resolves.toBeUndefined();
+  });
+
+  it("resolves after a failed bootstrap instead of hanging", async () => {
+    process.env[ENDPOINT_ENV] = "http://collector:4318";
+    vi.spyOn(console, "warn").mockImplementation(() => {});
+
+    const { shutdownInstrumentation } = await importFreshInstrumentation();
+
+    await expect(shutdownInstrumentation()).resolves.toBeUndefined();
+  });
+});