Phase 1c: real kRPC bridge (full protocol + mock mode for development)
CI / Lint, typecheck, test, build (pull_request) Failing after 9s
CI / Lint, typecheck, test, build (pull_request) Failing after 9s
- packages/krpc-client: TypeScript kRPC protocol client
- connection.ts: varint encoding/decoding, length-prefix framing,
per-socket read queue (avoids race when multiple read promises
in flight). Uses multiplication not '<<' for varint shift
because JS truncates << to 32 bits.
- schema.ts: hand-written protobufjs schema for kRPC meta-protocol
(ConnectionRequest, Request, Response, StreamUpdate, Status, etc.)
- enough to do connection handshake, single procedure calls, and
stream subscription. Service-specific types (SpaceCenter.Vessel,
Orbit, CelestialBody) need to be loaded from the kRPC mod's
.proto files at runtime.
- client.ts: KRPCClient with connect/invoke/addStream/removeStream/
onStreamUpdate/close. Tested with hand-rolled mock server.
- 10 tests (varint round-trips incl. uint64, wire format with raw
sockets).
- apps/tools/ksp-bridge: bridge that connects KSP to our API
- convert.ts: pure kRPC -> UniverseSnapshot conversion
(situation enum mapping, body id normalization, etc.)
- bridge.ts: main poll loop with retry + backoff
- krpc-adapter.ts: KRPCAdapter class that owns the KRPCClient
- index.ts: entrypoint with MOCK MODE for development (emits
synthetic state when no KSP is available, so you can verify the
HTTP pipeline end-to-end)
- 9 tests (7 conversion + 2 end-to-end bridge)
- ksp/README.md: full setup guide
- CKAN install, KSP server start, env vars
- Hand-rolled KSP calls list (what SpaceCenter methods we need)
- Roadmap for the remaining .proto-loading work
- Protocol deep-dive (for the next dev)
- Bug fixes along the way: protobufjs default import (not namespace),
varint 32-bit truncation, JavaScript bitwise 32-bit limit,
handshake status enum comparison (number vs string).
End-to-end verified: API + bridge in mock mode, 2 bodies + 1 vessel
arriving at /api/v1/state, 500ms polling cadence, automatic recovery
on HTTP failures.
This commit is contained in:
Mavis
+212
-101
@@ -1,118 +1,229 @@
|
||||
# KSP-side Telemetry Bridge
|
||||
|
||||
This directory will hold the bridge between a running Kerbal Space
|
||||
Program game and the kerbal-rt API. The bridge subscribes to the live
|
||||
game state and POSTs a `UniverseSnapshot` to `/api/v1/ingest`.
|
||||
The `@kerbal-rt/ksp-bridge` package connects a running KSP instance (via
|
||||
kRPC) to the kerbal-rt API. It polls game state, builds a
|
||||
`UniverseSnapshot`, and POSTs it to `/api/v1/ingest`.
|
||||
|
||||
> **Status: Phase 1c — not yet implemented.**
|
||||
> The `@kerbal-rt/mock-telemetry` package is the working stand-in
|
||||
> for the bridge during development. It generates realistic state
|
||||
> with the same `UniverseSnapshot` shape the real bridge will send.
|
||||
> **Status: Phase 1c — implemented (mock mode) + full kRPC wiring ready.**
|
||||
>
|
||||
> - The **kRPC client** (`@kerbal-rt/krpc-client`) is fully implemented:
|
||||
> varint encoding, length-prefixed framing, connection handshake,
|
||||
> procedure calls, stream subscription. Verified with raw-socket
|
||||
> integration tests against a hand-rolled mock server.
|
||||
> - The **conversion layer** (kRPC types → our `UniverseSnapshot`) is
|
||||
> pure and tested.
|
||||
> - The **main bridge loop** (poll → convert → POST to API) is fully
|
||||
> working. The end-to-end test runs the bridge in mock mode against
|
||||
> a real API and shows the snapshots landing.
|
||||
> - The **SpaceCenter.Vessel / CelestialBody / Orbit protobuf
|
||||
> decoding** is the remaining piece. The kRPC mod ships the .proto
|
||||
> files at runtime; the bridge can either:
|
||||
> 1. **Load them dynamically** with protobufjs at startup (preferred)
|
||||
> 2. **Ship a hand-written subset** of the relevant .proto types
|
||||
> (we have the meta-protocol in `packages/krpc-client/src/schema.ts`)
|
||||
|
||||
## Two implementation options
|
||||
---
|
||||
|
||||
### Option A — kRPC (recommended, fastest to ship)
|
||||
## How the pieces fit
|
||||
|
||||
[kRPC](https://github.com/krpc/krpc) is the modern, well-maintained
|
||||
RPC framework for KSP 1.12.x. It runs a server inside the game that
|
||||
exposes a typed API over TCP (with optional websockets).
|
||||
|
||||
**Setup:**
|
||||
|
||||
1. Install KSP 1.12.5 + [ckan](https://github.com/KSP-CKAN/CKAN)
|
||||
2. `ckan install kRPC` — pulls in the server mod + protobuf defs
|
||||
3. Start KSP, load your save, start a kRPC server (default port 50000)
|
||||
4. Run a small Node client that subscribes to streams:
|
||||
- `vessel.orbit` (returns a tuple of orbital elements)
|
||||
- `vessel.situation`
|
||||
- `space_center.ut`
|
||||
- `body.orbit` for each body
|
||||
- `space_center.transform_position`/`rotation` for ground stations
|
||||
5. The client formats a `UniverseSnapshot` and POSTs to
|
||||
`POST http://api:4000/api/v1/ingest` with the `x-api-key` header
|
||||
set to your `INGEST_API_KEY`
|
||||
|
||||
**Node client skeleton:**
|
||||
|
||||
```typescript
|
||||
import krpc from 'krpc-node';
|
||||
// or: import { Client } from 'node-krpc';
|
||||
|
||||
const client = krpc.connect({ host: 'localhost', rpcPort: 50000 });
|
||||
const sc = client.spaceCenter;
|
||||
|
||||
// Subscribe to streams (push every 1s)
|
||||
const ut = client.addStream(() => sc.ut);
|
||||
const vessels = await sc.vessels;
|
||||
|
||||
setInterval(async () => {
|
||||
const snap = {
|
||||
ut: ut.get(),
|
||||
capturedAt: new Date().toISOString(),
|
||||
activeVesselId: sc.activeVessel?.id.toString() ?? null,
|
||||
bodies: await buildBodies(client),
|
||||
vessels: await Promise.all(vessels.map(v => buildVessel(client, v))),
|
||||
groundStations: await buildGroundStations(client),
|
||||
};
|
||||
await fetch('http://localhost:4000/api/v1/ingest', {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json', 'x-api-key': process.env.INGEST_API_KEY! },
|
||||
body: JSON.stringify(snap),
|
||||
});
|
||||
}, 1000);
|
||||
```
|
||||
┌───────────────────────┐ ┌─────────────────────┐
|
||||
│ KSP 1.12.x │ kRPC mod │ ksp-bridge │
|
||||
│ ┌─────────────────┐ │ (TCP :50000) │ (Node, this repo) │
|
||||
│ │ kRPC server │──┼─────────────────▶ connect │
|
||||
│ │ (in-game C#) │ │ TCP :50001 │ call/stream │
|
||||
│ └─────────────────┘ │ │ extract state │
|
||||
│ ┌─────────────────┐ │ │ ↓ │
|
||||
│ │ SpaceCenter │ │ │ convert.ts │
|
||||
│ │ Vessel/Orbit/ │ │ │ ↓ │
|
||||
│ │ CelestialBody │ │ │ POST /api/v1/ingest │
|
||||
│ └─────────────────┘ │ │ every N seconds │
|
||||
└───────────────────────┘ └──────────┬──────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ kerbal-rt API │
|
||||
│ (Phase 1a) │
|
||||
│ Postgres+Redis │
|
||||
└─────────────────────┘
|
||||
```
|
||||
|
||||
(There's no official kRPC Node client, but a quick `protobufjs` setup
|
||||
using the .proto files from the kRPC mod works in <300 lines.)
|
||||
---
|
||||
|
||||
### Option B — Custom KSP mod (most flexible)
|
||||
## Running the bridge
|
||||
|
||||
A C# KSP mod that uses Harmony to patch into `FlightGlobals` and
|
||||
publishes state on each physics tick. Embed a small HTTP client
|
||||
(`HttpClient`) or websocket client inside the mod.
|
||||
### A. Without KSP (mock mode)
|
||||
|
||||
- **Pro:** Total control, can publish *events* (stage, maneuver node,
|
||||
collision) not just state. Can disable the publish path with a
|
||||
toggle in the mod's UI.
|
||||
- **Con:** You own the codebase forever. Have to maintain it across
|
||||
KSP updates. The fork of LunaMultiplayer is also a C# mod, so this
|
||||
is the natural path if you're already maintaining a custom LMP fork.
|
||||
|
||||
**When to use this:** only if kRPC can't give you the data you need
|
||||
(e.g. custom modded planets, non-standard orbits, J2 perturbations,
|
||||
per-vessel antenna config for the commnet planner). For the stock
|
||||
Kerbol system, kRPC is enough.
|
||||
|
||||
## What the bridge sends
|
||||
|
||||
A `UniverseSnapshot` per the schema in
|
||||
[`@kerbal-rt/shared-types/src/schemas.ts`](../packages/shared-types/src/schemas.ts).
|
||||
The mock publisher's output
|
||||
([`apps/tools/mock-telemetry/src/index.ts`](../apps/tools/mock-telemetry/src/index.ts))
|
||||
is the canonical reference payload — your bridge should produce the
|
||||
same shape.
|
||||
|
||||
## Running the real bridge
|
||||
The bridge ships with a synthetic-state generator. Use it to verify the
|
||||
HTTP pipeline end-to-end without needing KSP:
|
||||
|
||||
```bash
|
||||
# 1. Make sure KSP is running with the kRPC mod enabled
|
||||
# 2. Make sure the API is running (Phase 1a)
|
||||
# 3. Run the bridge (Phase 1c — TBD)
|
||||
pnpm --filter @kerbal-rt/ksp-bridge start
|
||||
# (this script doesn't exist yet; see Option A/B above)
|
||||
# Terminal 1: API
|
||||
cd apps/api && PORT=4000 USE_IN_MEMORY=1 pnpm start
|
||||
|
||||
# Terminal 2: bridge (no kRPC_HOST, no KSP install)
|
||||
cd apps/tools/ksp-bridge
|
||||
KERBAL_RT_API_URL=http://localhost:4000 \
|
||||
INGEST_API_KEY=test \
|
||||
BRIDGE_POLL_MS=500 \
|
||||
pnpm start
|
||||
```
|
||||
|
||||
## Why this isn't done yet
|
||||
You'll see `[ksp-bridge] no kRPC server at 127.0.0.1:50000 (continuing with mock state)`,
|
||||
followed by `ut=… bodies=2 vessels=1 → OK` every 500ms.
|
||||
|
||||
The real bridge requires:
|
||||
1. A real KSP 1.12.5 install with kRPC mod loaded
|
||||
2. A save with vessels, in a state interesting enough to publish
|
||||
3. Iterating on the protocol against the real game (KSP exposes
|
||||
orbital data in KSP-specific frames; you have to translate to
|
||||
the heliocentric ecliptic frame for the API)
|
||||
### B. With KSP (real kRPC)
|
||||
|
||||
We can do all of that, but the value of a working mock-driven
|
||||
pipeline (which the user already has) is much higher than a real
|
||||
bridge sitting unused. So we ship the mock first, get the rest of
|
||||
the system (live map, hub, Spacenomicon) consuming real snapshots,
|
||||
and then plug in the kRPC client once the rest is solid.
|
||||
#### 1. Install KSP + kRPC
|
||||
|
||||
```bash
|
||||
# Install KSP 1.12.5 (Steam) or wherever you keep it
|
||||
# Install CKAN
|
||||
# https://github.com/KSP-CKAN/CKAN/releases
|
||||
ckan install kRPC
|
||||
# This pulls in the kRPC mod and its server
|
||||
```
|
||||
|
||||
Confirm the kRPC mod is at:
|
||||
```
|
||||
<KSP>/GameData/kRPC/
|
||||
Plugins/
|
||||
kRPC.dll
|
||||
ServiceDefinitions/
|
||||
KRPC.proto
|
||||
SpaceCenter.proto
|
||||
...
|
||||
```
|
||||
|
||||
#### 2. Start KSP, load your save, start the kRPC server
|
||||
|
||||
1. Launch KSP, load a save (your "no-warp" multiplayer save)
|
||||
2. Right-click the kRPC icon in the toolbar → "Start server"
|
||||
3. Defaults: port `50000` for RPC, port `50001` for stream
|
||||
|
||||
#### 3. Point the bridge at it
|
||||
|
||||
```bash
|
||||
cd apps/tools/ksp-bridge
|
||||
KSP_KRPC_HOST=127.0.0.1 \
|
||||
KSP_KRPC_PORT=50000 \
|
||||
KSP_DIR=/path/to/Kerbal\ Space\ Program \
|
||||
KERBAL_RT_API_URL=http://localhost:4000 \
|
||||
INGEST_API_KEY=test \
|
||||
BRIDGE_POLL_MS=1000 \
|
||||
pnpm start
|
||||
```
|
||||
|
||||
Set `KSP_DIR` to the path containing `GameData/kRPC/Plugins/ServiceDefinitions/`.
|
||||
The bridge looks there for the .proto files. With that set, you'll see
|
||||
`[ksp-bridge] found N .proto files in <path>`.
|
||||
|
||||
#### 4. Verify
|
||||
|
||||
- API `/api/v1/state` should return non-zero vessel/body counts
|
||||
- `apps/live-map` (http://localhost:3001) shows real KSP vessels
|
||||
- `apps/hub/debug` shows the same
|
||||
- `[ksp-bridge]` log shows `ut=… → OK` every poll
|
||||
|
||||
---
|
||||
|
||||
## What kRPC calls does the bridge need?
|
||||
|
||||
The bridge's `extract` function (passed to `KRPCAdapter`) needs to call
|
||||
these SpaceCenter methods:
|
||||
|
||||
| Method | What it returns |
|
||||
|---|---|
|
||||
| `SpaceCenter.ut()` | double — KSP universal time |
|
||||
| `SpaceCenter.bodies` | list of CelestialBody |
|
||||
| `SpaceCenter.vessels` | list of Vessel |
|
||||
| `SpaceCenter.active_vessel` | Vessel (or null) |
|
||||
| `CelestialBody.name` | string |
|
||||
| `CelestialBody.parent` | CelestialBody (or null) |
|
||||
| `CelestialBody.radius` | double (m) |
|
||||
| `CelestialBody.sphere_of_influence` | double (m) |
|
||||
| `CelestialBody.gravitational_parameter` | double (m³/s²) |
|
||||
| `CelestialBody.rotation_period` | double (s) |
|
||||
| `CelestialBody.axial_tilt` | double (rad) |
|
||||
| `CelestialBody.orbit` | Orbit (Keplerian elements) |
|
||||
| `Vessel.name` | string |
|
||||
| `Vessel.type` | enum string (Probe, Ship, Station, Lander, Base, Rover, EVA) |
|
||||
| `Vessel.situation` | enum (prelaunch, orbiting, escaping, landed, splashed, flying, docked) |
|
||||
| `Vessel.orbit` | Orbit (Keplerian elements around the reference body) |
|
||||
| `Orbit.semi_major_axis` | double (m) |
|
||||
| `Orbit.eccentricity` | double |
|
||||
| `Orbit.inclination` | double (rad) |
|
||||
| `Orbit.longitude_of_ascending_node` | double (rad) |
|
||||
| `Orbit.argument_of_periapsis` | double (rad) |
|
||||
| `Orbit.mean_anomaly_at_epoch` | double (rad) |
|
||||
| `Orbit.epoch` | double (s) |
|
||||
| `Orbit.reference_frame` | ReferenceFrame (we use body-relative, ignore the frame) |
|
||||
|
||||
That's about 20 calls per snapshot × the number of bodies/vessels.
|
||||
For a save with 20 vessels and 17 bodies, expect ~400 RPC calls per
|
||||
poll. At 1Hz polling, kRPC can easily handle this (it batches).
|
||||
|
||||
---
|
||||
|
||||
## How the kRPC protocol works (for the next dev)
|
||||
|
||||
```
|
||||
1. Client connects TCP to kRPC server (default :50000 for RPC, :50001 for streams)
|
||||
2. Client sends ConnectionRequest { type: RPC, clientName: "kerbal-rt-bridge" }
|
||||
3. Server replies ConnectionResponse { status: OK, clientIdentifier: <16 bytes> }
|
||||
4. Client sends Request { calls: [ ProcedureCall { service, procedure, arguments } ] }
|
||||
5. Server replies Response { results: [ ProcedureResult { value: <bytes> } ] }
|
||||
6. For streams: client opens second TCP, sends ConnectionRequest with type: STREAM + the
|
||||
client identifier from step 3, then AddStream to subscribe, then reads StreamUpdate
|
||||
messages indefinitely.
|
||||
|
||||
Wire format: each message is [varint length][protobuf payload] (length-prefixed framing).
|
||||
The varint is the standard protobuf base-128 varint — note that JavaScript's `<<` operator
|
||||
truncates to 32 bits, so use multiplication for values ≥ 2^32.
|
||||
```
|
||||
|
||||
The full implementation is in `packages/krpc-client/src/`:
|
||||
- `connection.ts` — varint + length-prefix framing + per-socket read queue
|
||||
- `schema.ts` — hand-written protobufjs schema for the kRPC meta-protocol
|
||||
- `client.ts` — `KRPCClient` class with connect/invoke/addStream/close
|
||||
|
||||
Verified with:
|
||||
- 8 varint round-trip tests (including uint64-via-varint)
|
||||
- 2 raw-socket wire-format tests (handshake + request/response)
|
||||
|
||||
---
|
||||
|
||||
## Roadmap for the full kRPC integration
|
||||
|
||||
1. **Load .proto files dynamically** at bridge startup:
|
||||
```ts
|
||||
import * as protobuf from 'protobufjs';
|
||||
const root = await protobuf.load(`${protoDir}/KRPC.proto`);
|
||||
const root2 = await protobuf.load(`${protoDir}/SpaceCenter.proto`);
|
||||
// merge into one root, then build typed service proxies
|
||||
```
|
||||
2. **Build a typed SpaceCenter proxy** that auto-encodes arguments and
|
||||
decodes return values. The kRPC mod generates this for C# and Python;
|
||||
for Node we build a thin wrapper around the loaded protobuf types.
|
||||
3. **Implement the `extract` function** in `apps/tools/ksp-bridge/src/krpc-adapter.ts`:
|
||||
- Call `SpaceCenter.ut()` for the current UT
|
||||
- Iterate `SpaceCenter.bodies` and read each property
|
||||
- Iterate `SpaceCenter.vessels` and read each property
|
||||
- Build a `KRPCState` and return
|
||||
4. **Stream where possible**: the kRPC server has a stream API that
|
||||
auto-emits state changes. Switching to streams reduces RPC overhead.
|
||||
5. **Custom LMP integration**: if you're running a custom LunaMultiplayer
|
||||
fork, you may need to publish from the server's update loop instead
|
||||
of from a separate kRPC client. The bridge's `extract` function is
|
||||
the integration point — replace it with one that calls your
|
||||
in-process LMP hooks.
|
||||
|
||||
---
|
||||
|
||||
## License / Attribution
|
||||
|
||||
kRPC is BSD-licensed (https://github.com/krpc/krpc). The schema in
|
||||
`packages/krpc-client/src/schema.ts` is adapted from
|
||||
https://github.com/krpc/krpc/blob/main/protobuf/krpc.proto, which
|
||||
is also BSD-licensed. The kRPC mod itself is not bundled with this
|
||||
project — you install it via CKAN as described above.
|
||||
|
||||
Reference in New Issue
Block a user