mirror of
https://github.com/ruvnet/RuView
synced 2026-07-25 17:51:48 +00:00
docs(adr): deep review of the RuView npm surface — ADR-263/264/265 optimization strategies (#1229)
* docs(adr): deep review of the RuView npm surface — ADR-263/264/265 optimization strategies
ADR-263 — @ruvnet/ruview@0.1.0 harness review (O1–O9):
- HIGH: claim-check CLI fails open on empty input (no --text/--file -> PASS exit 0)
- HIGH: MCP stdio server head-of-line blocking (spawnSync verify/calibrate up to 600s)
- MEASURED: optionalDependencies triple the cold npx install (4 pkgs/620kB/71 files
vs 1 pkg/172kB/22 files with --omit=optional) for a path that never imports them
- maxBuffer truncation, python -c port interpolation, version drift, duplicate skills,
guardrail METRIC_TERMS substring false positives ('map'/'F1' — found by dogfooding
claim-check on these very ADRs), zero CI
ADR-264 — @ruvnet/rvagent@0.1.0 + @ruv/ruview-cli review (O1–O9), verified against
the published registry tarball:
- HIGH: exports.require -> dist/index.cjs which is never built nor published
- MEASURED: 44 dead source-map files = 62,698B of the 188kB unpacked payload
- stdio-only server described as dual-transport; mixed dot/underscore tool names;
double Zod validation + hand-duplicated advertised schemas; 2-fd leak per training
job; unbounded body in the unwired HTTP scaffold; dead detectCogBinary candidates;
ruview bin-name collision
ADR-265 — cross-cutting npm distribution strategy: npm-packages.yml CI matrix
(test + pack-content/size gate + tarball-install smoke test), publish-from-CI-only
with npm provenance, version single-sourcing from package.json, bin/namespace
ownership (ruview bin belongs to @ruvnet/ruview), claim-check on package READMEs.
Docs only — no runtime code changed. Index/CHANGELOG/CLAUDE.md/README counts updated.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz
* fix(npm): implement ADR-263/264/265 — harness fail-closed + async MCP, rvagent packaging/transport/naming, npm CI+provenance gate
ADR-263 (@ruvnet/ruview 0.2.0), O1-O9:
- claim-check fails closed on empty input (CLI exit 2, empty_text tool error)
- MCP stdio server dispatches tools/call asynchronously (promise-based spawn);
ping answers while a 3s fake verify runs — pinned by new e2e test
- optionalDependencies dropped: cold npx installs exactly 1 package
(MEASURED: was 4 pkgs/620kB/71 files via npm i in a clean prefix)
- bounded rolling output tails replace spawnSync 1MiB maxBuffer
- node_monitor port passed via sys.argv, never spliced into python -c source
- serverInfo.version read from package.json; resources/prompts stubs
- skills single-sourced: prepack sync script generates .claude/skills/ copies
- which() = memoized dep-free PATH scan
- tools underscore-canonical (ruview_claim_check, ...) + dotted aliases
- guardrail precision: word-boundary map/f1/auc/iou, code-span + F1/O2 label
scrubbing, quantitative-claims-only; packaging reproducer hints
- 30/30 tests (was 17), incl. concurrency e2e + fail-open regression pins
ADR-264 (@ruvnet/rvagent 0.2.0), O1-O9:
- exports fixed: types-first, phantom dist/index.cjs require target removed
- tarball map-free: 127,704B unpacked / 46 files / 0 maps (MEASURED,
npm pack --dry-run; was 188kB incl. 44 maps referencing unshipped src)
- Streamable HTTP actually wired behind RVAGENT_HTTP_PORT: one transport +
one MCP server per session (mcp-session-id routing), 1MiB body cap (413),
port-aware localhost origin gate; dual-transport description now true
- tools renamed underscore-canonical with dotted router-only aliases
- single Zod validation gate; advertised inputSchema generated from the same
Zod source (zod-to-json-schema)
- train_count: parent log fds closed (was leaking 2/job); job records
persisted to <jobsDir>/<id>.json (job_status survives restarts); bounded
log-tail reads
- detectCogBinary probes its candidates instead of dead-coding them
- version from package.json; @types/express dropped; @types/jest -> 29
- README rewritten to match reality (no phantom subcommands/policy layer)
- 99/99 jest tests (incl. new session/body-cap suite + previously-broken
manifest suite); stdio handshake + HTTP session flow smoke-tested live
ADR-265 D1-D4:
- .github/workflows/npm-packages.yml: 3-package x Node 20/22 gate — tests,
version-literal grep (D3), pack-content/size gate, tarball-install smoke
test (catches the ADR-264 F1 class), README claim-check (D4)
- .github/workflows/ruview-npm-release.yml: publish from CI only with
npm publish --provenance
- @ruv/ruview-cli bin renamed ruview-cli (ruview bin belongs to
@ruvnet/ruview); version single-sourced
- ci.yml NODE_VERSION 18 -> 20
ADR statuses updated to Accepted/implemented; harness manifest re-pinned;
ADR-263/264/265 + both package READMEs pass claim-check.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz
* perf(rvagent): lazy-load HTTP transport + memoize generated tool schemas
stdio time-to-first-response ~242ms -> ~189ms (-22%; MEASURED, median of
repeated initialize round-trips against dist/index.js in this container).
- ./http-transport.js now imported lazily inside the RVAGENT_HTTP_PORT
branch: it chain-loads the MCP SDK streamableHttp module (~48ms MEASURED
via per-module import() timing) which the default stdio path never uses
- toolInputJsonSchema memoized per tool: schemas are static for the process
lifetime; under the session-per-server HTTP model every session calls
tools/list, so stop re-walking the Zod tree each time
No behavior change: 99/99 jest tests; HTTP session flow re-smoke-tested
through the lazy import path (initialize -> 200 + mcp-session-id).
Profiled @ruvnet/ruview too and left it alone: 50ms CLI startup vs ~29ms
bare 'node -e ""' floor on the same box (MEASURED) — already near the
interpreter floor with zero dependencies.
Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz
* ci(ruview-cli): pass jest --passWithNoTests so the private no-test package doesn't fail the npm-packages matrix
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(npm): address 10 verified review findings in harness + rvagent before 0.2.0 publish
harness/ruview (@ruvnet/ruview):
- guardrails: digit gate now sees numbers inside code spans; F1-style
metric tokens followed by ':' or a nearby number are no longer scrubbed
(fail-open regressions in the honesty gate)
- mcp-server: tools/call requests serialize through a FIFO promise chain
(hardware/mutating tools never overlap) while ping/tools/list stay
immediate; stdin close drains in-flight responses before exit
- tools: which() no longer memoizes negative lookups
tools/ruview-mcp (@ruvnet/rvagent):
- index: realpath invoked-directly guard — library import no longer
connects a stdio transport to the consumer's process
- http-transport: explicit allowedOrigins is exact-match only (localhost
any-port convenience applies only with no configured allowlist);
session map gains maxSessions=64 + 5min idle TTL sweep
- train-count: job records persist the child pid and reconcile stale
'running' status after a server restart (exit-code marker or dead pid)
- config: cog binary candidates ordered by process.arch
.github/workflows/ruview-npm-release.yml: port the full ADR-265 D1 gate
(version-literal check, unpacked-size budget, tarball-install smoke test)
from npm-packages.yml so the publish path enforces what the header claims.
Tests: harness 30→36, rvagent 99→112, all passing.
Co-Authored-By: claude-flow <ruv@ruv.net>
---------
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
rUv
@@ -6,24 +6,24 @@ description: Run the ADR-151 per-room calibration pipeline — baseline → enro
|
||||
# calibrate-room
|
||||
|
||||
Turn a provisioned node + sensing-server into a working room model. Pure-Rust,
|
||||
edge-deployable (ADR-151). Use the `ruview.calibrate` tool (installed
|
||||
edge-deployable (ADR-151). Use the `ruview_calibrate` tool (installed
|
||||
`wifi-densepose` binary, else `cargo run -p wifi-densepose-cli`).
|
||||
|
||||
## Sequence
|
||||
|
||||
1. **baseline** — capture the empty room (Welford amplitude + von Mises phase). Leave
|
||||
the room empty.
|
||||
`ruview.calibrate {step: "baseline"}`
|
||||
`ruview_calibrate {step: "baseline"}`
|
||||
2. **enroll** — record the occupant(s) doing the target activities.
|
||||
`ruview.calibrate {step: "enroll"}`
|
||||
`ruview_calibrate {step: "enroll"}`
|
||||
3. **train-room** — train the bank of small specialists from baseline + enrollment.
|
||||
`ruview.calibrate {step: "train-room"}`
|
||||
`ruview_calibrate {step: "train-room"}`
|
||||
4. **room-watch** — live presence/posture/breathing from the trained room.
|
||||
`ruview.calibrate {step: "room-watch"}` (or the `room-watch` skill)
|
||||
`ruview_calibrate {step: "room-watch"}` (or the `room-watch` skill)
|
||||
|
||||
## Honesty
|
||||
|
||||
The specialists are calibrated to *this* room; cross-room transfer is a separate
|
||||
problem (LoRA recalibration, ADR-079 P9). Report which room a number came from, and
|
||||
tag presence/vitals accuracy MEASURED only with a held-out check — run
|
||||
`ruview.claim_check` on the writeup.
|
||||
`ruview_claim_check` on the writeup.
|
||||
|
||||
@@ -8,12 +8,12 @@ description: Zero-to-sensing path picker for RuView (WiFi-DensePose) — pick do
|
||||
Get a newcomer from nothing to a working RuView setup. **First fact to set:** WiFi
|
||||
sensing infers *coarse* pose/presence/breathing from Channel State Information — it
|
||||
is **not a camera**, and any accuracy number must be MEASURED against a baseline
|
||||
(use the `verify` skill / `ruview.claim_check` tool). Never present WiFi output as
|
||||
(use the `verify` skill / `ruview_claim_check` tool). Never present WiFi output as
|
||||
camera-grade.
|
||||
|
||||
## Pick a path
|
||||
|
||||
Run `ruview.onboard {path}` or decide from:
|
||||
Run `ruview_onboard {path}` or decide from:
|
||||
|
||||
1. **docker-demo** — fastest, no hardware. Replays sample CSI into the dashboard.
|
||||
`docker run -p 8000:8000 ruvnet/wifi-densepose` → open `http://localhost:8000`.
|
||||
@@ -26,5 +26,5 @@ Run `ruview.onboard {path}` or decide from:
|
||||
## Then
|
||||
|
||||
- Live sensing → go to **provision-node**, then **calibrate-room**.
|
||||
- Evaluating a model/claim → go to **verify** and run `ruview.claim_check` on any
|
||||
- Evaluating a model/claim → go to **verify** and run `ruview_claim_check` on any
|
||||
report before you quote a number.
|
||||
|
||||
@@ -28,7 +28,7 @@ esptool --chip esp32s3 -p <PORT> -b 460800 write_flash \
|
||||
0xf000 ota_data_initial.bin 0x20000 esp32-csi-node-s3-8mb.bin
|
||||
```
|
||||
|
||||
(`ruview.node_flash` returns the exact pinned command rather than running an
|
||||
(`ruview_node_flash` returns the exact pinned command rather than running an
|
||||
unattended flash.)
|
||||
|
||||
## 3. Provision
|
||||
@@ -44,6 +44,6 @@ Never echo or commit the WiFi password.
|
||||
|
||||
## 4. Confirm CSI is flowing
|
||||
|
||||
`ruview.node_monitor {port}` — PASS criteria: serial shows `CSI cb #...` callbacks and
|
||||
`ruview_node_monitor {port}` — PASS criteria: serial shows `CSI cb #...` callbacks and
|
||||
(on a bare board) `CSI filter upgraded to MGMT+DATA`. No callbacks → the node isn't
|
||||
capturing; do not proceed to calibration.
|
||||
|
||||
@@ -29,5 +29,5 @@ or temporal leakage. Example honest result (ADR-181):
|
||||
1. Run the mean-pose baseline on the same split.
|
||||
2. Report `(model − baseline)` in pp, with the split definition (chronological /
|
||||
blocked-gap / grouped-bucket; no leakage).
|
||||
3. `ruview.claim_check` the writeup — it flags any untagged or 100%/perfect claim.
|
||||
3. `ruview_claim_check` the writeup — it flags any untagged or 100%/perfect claim.
|
||||
4. If it's a benchmark vs SOTA, tag MEASURED-EQUIVALENT only with the reproducer.
|
||||
|
||||
@@ -9,7 +9,7 @@ The "prove everything" skill. Nothing ships as validated without this.
|
||||
|
||||
## Deterministic proof (Trust Kill Switch)
|
||||
|
||||
`ruview.verify` runs `archive/v1/data/proof/verify.py`: it feeds a reference signal
|
||||
`ruview_verify` runs `archive/v1/data/proof/verify.py`: it feeds a reference signal
|
||||
through the production pipeline and hashes the output against
|
||||
`expected_features.sha256`. Must print **VERDICT: PASS**. If numpy/scipy changed the
|
||||
hash, regenerate with `verify.py --generate-hash` then re-verify.
|
||||
@@ -28,7 +28,7 @@ crate versions — a recipient can re-verify with one command.
|
||||
|
||||
## Claim honesty
|
||||
|
||||
Run `ruview.claim_check {text}` on any report, README section, PR body, or model card
|
||||
Run `ruview_claim_check {text}` on any report, README section, PR body, or model card
|
||||
before quoting accuracy. It flags:
|
||||
- untagged accuracy numbers (must be MEASURED / CLAIMED / SYNTHETIC),
|
||||
- MEASURED claims with no reproducer cited,
|
||||
|
||||
@@ -13,27 +13,27 @@
|
||||
],
|
||||
"files": {
|
||||
".claude/settings.json": "b0ea971383716f18b89db73010b8f0ea0f1b16bdec4cd1068245772ba1c27bdd",
|
||||
".claude/skills/calibrate-room/SKILL.md": "6a6c8211a7109feb76620c618963c10ad9a9f633ffce7676e631a80a1181986d",
|
||||
".claude/skills/onboard/SKILL.md": "22323732fe746b38b77a7c8c052e952dff2fe87ae939ba125379125827385f21",
|
||||
".claude/skills/provision-node/SKILL.md": "5ffe5a75873e873b80758d9c81005774d4191317227f2e9aa4345cbce3f29751",
|
||||
".claude/skills/train-pose/SKILL.md": "b3ee95bfb0b678eb3d101138b9ea0e7cab3db3a9906d19c4059f9cca0598e87b",
|
||||
".claude/skills/verify/SKILL.md": "c0314d5ead465d9089b6a4917fd125051a5be20dc07ba92d5b601fcaada32e19",
|
||||
"CLAUDE.md": "7ecdb2b9d9abcf4aa22dd3ce553b60216a135e147893a59fa944fc1a8c81f5ef",
|
||||
".claude/skills/calibrate-room/SKILL.md": "4b29c7c331f47acad3c0f51b3d3d8f5b5573e316e081bae71dbe21a47fa95240",
|
||||
".claude/skills/onboard/SKILL.md": "97ee71f0aa985cfc03bb8e764789bb55c4f9fd5dae10a116c1071eab85b5893f",
|
||||
".claude/skills/provision-node/SKILL.md": "5f73823794ed5f0b25c102aa8b1bf2dd534a1ec468173d8330c2af0ca24f239c",
|
||||
".claude/skills/train-pose/SKILL.md": "92aebd4423470eb10eabaee642ec3493284d98b7ae9785e0f34378c709746e65",
|
||||
".claude/skills/verify/SKILL.md": "2d38d240e9810a7827e2ebd3717dc0f85c646cc92e46c3812fe77c5b9eb40b76",
|
||||
"CLAUDE.md": "1d7af0c310dd8093b4ae6c9c94a1c0cc9ff02ac9c8d5b45caba5363c3af99475",
|
||||
"LICENSE": "631f94984f626818d42ecf717aa6e8e0afd4f9f355ca706bd2effafbd1416d06",
|
||||
"README.md": "b77d30428de8efb6758f2ca3eb22e84849013b2c0e6c601d488d2ea5a6f0da44",
|
||||
"bin/cli.js": "b0d74690cff4329dfe342271fc475eaa140b767bdb66b37cf4992ad209012fe8",
|
||||
"package.json": "2af49561ef0d59cafc4b99885816e580635b2d2ad329dfe17c69b9df6f8afceb",
|
||||
"skills/calibrate-room.md": "6a6c8211a7109feb76620c618963c10ad9a9f633ffce7676e631a80a1181986d",
|
||||
"skills/onboard.md": "22323732fe746b38b77a7c8c052e952dff2fe87ae939ba125379125827385f21",
|
||||
"skills/provision-node.md": "5ffe5a75873e873b80758d9c81005774d4191317227f2e9aa4345cbce3f29751",
|
||||
"skills/train-pose.md": "b3ee95bfb0b678eb3d101138b9ea0e7cab3db3a9906d19c4059f9cca0598e87b",
|
||||
"skills/verify.md": "c0314d5ead465d9089b6a4917fd125051a5be20dc07ba92d5b601fcaada32e19",
|
||||
"src/guardrails.js": "1631cea02c4354fe6126c576300faf5f8b68ae2f5e2e3a658c99eb25a7403e55",
|
||||
"src/mcp-server.js": "e51379f5ebb0b7b4670c7412714e559931ef1be8df20551f8f7309b53f0fb7af",
|
||||
"src/tools.js": "b558f61bb202abf5a967ce3a6ccaea351f2d186238cf49c7fc151d1de028eee8"
|
||||
"README.md": "ac35157d66243a5f9eba262bdf2d593e978d935b3dde6e455b7acf650768eac6",
|
||||
"bin/cli.js": "85d8394375edb1e967418451452e68bdbe26e69fc6877ed4936894f6101e1a12",
|
||||
"package.json": "4509b68bb4211217f1e9f3f95f3134b326ee23a2322aef8d19b99a4b1d415b08",
|
||||
"skills/calibrate-room.md": "4b29c7c331f47acad3c0f51b3d3d8f5b5573e316e081bae71dbe21a47fa95240",
|
||||
"skills/onboard.md": "97ee71f0aa985cfc03bb8e764789bb55c4f9fd5dae10a116c1071eab85b5893f",
|
||||
"skills/provision-node.md": "5f73823794ed5f0b25c102aa8b1bf2dd534a1ec468173d8330c2af0ca24f239c",
|
||||
"skills/train-pose.md": "92aebd4423470eb10eabaee642ec3493284d98b7ae9785e0f34378c709746e65",
|
||||
"skills/verify.md": "2d38d240e9810a7827e2ebd3717dc0f85c646cc92e46c3812fe77c5b9eb40b76",
|
||||
"src/guardrails.js": "66407b00d31c4f7939b75ee3e29598855c36a4154ccf1436655a4e52b0d7c034",
|
||||
"src/mcp-server.js": "ad0f21be65a37237b9c2aad69e6e75166e5f101d902cb986377043545a7a80fb",
|
||||
"src/tools.js": "1d72377ae53ad2b0c6dc03eb66f584422d8a60e442cb0d4f08355590f3edf031"
|
||||
},
|
||||
"meta": {
|
||||
"surface": "cli+mcp",
|
||||
"adr": "ADR-182"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1 +1 @@
|
||||
6c6c1431c37472494c9b309c8b5d761dd4fc41e30313baead6320831fb982e57 manifest.json
|
||||
380d4bf928fd7c5fa753d11a30c1e24e2ea471caca57b439f765a9d864cef472 manifest.json
|
||||
|
||||
@@ -10,15 +10,15 @@ accuracy number:
|
||||
1. It must be tagged **MEASURED** (with a reproducer named), **CLAIMED**, or **SYNTHETIC**.
|
||||
2. Pose PCK is quoted only as a **delta over the mean-pose baseline** on a leakage-free
|
||||
held-out split. (A mean-pose predictor already scores ~50% PCK.)
|
||||
3. Run `ruview.claim_check` on any report/PR/model-card. It flags untagged numbers and
|
||||
3. Run `ruview_claim_check` on any report/PR/model-card. It flags untagged numbers and
|
||||
the retracted "100%/perfect accuracy" framing.
|
||||
4. Firmware is "hardware-validated" only with a captured **boot log on real silicon** —
|
||||
never on a build-passes signal.
|
||||
|
||||
## Tools
|
||||
|
||||
`ruview.onboard`, `ruview.claim_check`, `ruview.verify`, `ruview.node_monitor`,
|
||||
`ruview.calibrate`, `ruview.node_flash`. All fail-closed. Mutating/hardware tools
|
||||
`ruview_onboard`, `ruview_claim_check`, `ruview_verify`, `ruview_node_monitor`,
|
||||
`ruview_calibrate`, `ruview_node_flash`. All fail-closed. Mutating/hardware tools
|
||||
(`node_flash`) require explicit confirmation and are Windows/ESP-IDF gated.
|
||||
|
||||
## Skills
|
||||
|
||||
+14
-12
@@ -7,34 +7,36 @@ crucially — **refuse to overstate accuracy**. Minted from the RuView monorepo
|
||||
|
||||
WiFi sensing infers *coarse* pose/presence/breathing from Channel State Information.
|
||||
It is **not a camera**. Every accuracy number this harness emits must be MEASURED
|
||||
against a baseline — that rule is enforced in code (`ruview.claim_check`).
|
||||
against a baseline — that rule is enforced in code (`ruview_claim_check`).
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
npx @ruvnet/ruview # onboard — pick a setup path
|
||||
npx @ruvnet/ruview claim-check --text "we hit 100% accuracy" # the honesty guardrail
|
||||
npx @ruvnet/ruview claim-check --file REPORT.md # the honesty guardrail (non-zero exit on untagged claims)
|
||||
npx @ruvnet/ruview verify # run the deterministic proof (VERDICT: PASS)
|
||||
npx @ruvnet/ruview doctor # self-check (tools + optional kernel/host)
|
||||
npx @ruvnet/ruview --help
|
||||
```
|
||||
|
||||
The operator tools are pure Node and run with **zero install weight**. The
|
||||
`@metaharness/kernel` + host adapter are `optionalDependencies` — only `doctor` /
|
||||
`install` use them, only if present.
|
||||
The operator tools are pure Node and run with **zero install weight** — the
|
||||
package has no dependencies at all (ADR-263 O3). `doctor` / `install` can
|
||||
additionally use `@metaharness/kernel` + a host adapter if you install them
|
||||
(`npm i @metaharness/kernel @metaharness/host-claude-code`); everything else
|
||||
runs without them.
|
||||
|
||||
## Tools (`ruview.*`)
|
||||
## Tools (`ruview_*`)
|
||||
|
||||
Exposed both as CLI verbs and as an MCP server (`npx @ruvnet/ruview mcp start`):
|
||||
|
||||
| Tool | What it does |
|
||||
|------|--------------|
|
||||
| `ruview.onboard` | Pick docker-demo / repo-build / live-esp32; print the next command |
|
||||
| `ruview.claim_check` | Lint text for untagged / overstated accuracy claims (guardrail) |
|
||||
| `ruview.verify` | Run `verify.py` deterministic proof → VERDICT |
|
||||
| `ruview.node_monitor` | Assert CSI is flowing on an ESP32 (read-only) |
|
||||
| `ruview.calibrate` | ADR-151 room pipeline (baseline→enroll→train-room→room-watch) |
|
||||
| `ruview.node_flash` | Build+flash firmware (Windows/ESP-IDF; mutating, guarded) |
|
||||
| `ruview_onboard` | Pick docker-demo / repo-build / live-esp32; print the next command |
|
||||
| `ruview_claim_check` | Lint text for untagged / overstated accuracy claims (guardrail) |
|
||||
| `ruview_verify` | Run `verify.py` deterministic proof → VERDICT |
|
||||
| `ruview_node_monitor` | Assert CSI is flowing on an ESP32 (read-only) |
|
||||
| `ruview_calibrate` | ADR-151 room pipeline (baseline→enroll→train-room→room-watch) |
|
||||
| `ruview_node_flash` | Build+flash firmware (Windows/ESP-IDF; mutating, guarded) |
|
||||
|
||||
Every tool is **fail-closed**: missing repo / python / binary / port → an honest
|
||||
negative, never a fabricated success.
|
||||
|
||||
@@ -18,14 +18,14 @@ const NAME = 'ruview';
|
||||
const ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const SKILLS_DIR = join(ROOT, 'skills');
|
||||
|
||||
// Map friendly CLI verbs → registry tool names.
|
||||
// Map friendly CLI verbs → registry tool names (underscore-canonical, ADR-263).
|
||||
const VERB_TO_TOOL = {
|
||||
onboard: 'ruview.onboard',
|
||||
verify: 'ruview.verify',
|
||||
'claim-check': 'ruview.claim_check',
|
||||
calibrate: 'ruview.calibrate',
|
||||
monitor: 'ruview.node_monitor',
|
||||
flash: 'ruview.node_flash',
|
||||
onboard: 'ruview_onboard',
|
||||
verify: 'ruview_verify',
|
||||
'claim-check': 'ruview_claim_check',
|
||||
calibrate: 'ruview_calibrate',
|
||||
monitor: 'ruview_node_monitor',
|
||||
flash: 'ruview_node_flash',
|
||||
};
|
||||
|
||||
function pjson(o) { console.log(JSON.stringify(o, null, 2)); }
|
||||
@@ -112,13 +112,18 @@ export async function run(args) {
|
||||
const toolArgs = { ...flags };
|
||||
if (cmd === 'claim-check') {
|
||||
if (flags.file) toolArgs.text = readFileSync(flags.file, 'utf8');
|
||||
const res = runTool('ruview.claim_check', toolArgs);
|
||||
// Fail closed (ADR-263 O1): an honesty gate must never PASS on no input.
|
||||
if (typeof toolArgs.text !== 'string' || toolArgs.text.trim().length === 0) {
|
||||
console.error('claim-check: no input — pass --text "..." or --file <path> (empty input is an error, not a PASS).');
|
||||
return 2;
|
||||
}
|
||||
const res = await runTool('ruview_claim_check', toolArgs);
|
||||
pjson(res);
|
||||
return res.ok ? 0 : 1;
|
||||
}
|
||||
if (cmd === 'monitor' && flags.seconds) toolArgs.seconds = Number(flags.seconds);
|
||||
if (cmd === 'calibrate' && typeof flags.args === 'string') toolArgs.args = flags.args.split(',');
|
||||
const res = runTool(VERB_TO_TOOL[cmd], toolArgs);
|
||||
const res = await runTool(VERB_TO_TOOL[cmd], toolArgs);
|
||||
pjson(res);
|
||||
return res.ok ? 0 : 1;
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@ruvnet/ruview",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"description": "RuView WiFi-sensing operator agent harness — onboard, calibrate, train, and verify camera-free WiFi-CSI sensing, with the project's MEASURED-vs-CLAIMED honesty guardrail enforced. Minted via metaharness (ADR-182).",
|
||||
"type": "module",
|
||||
"bin": {
|
||||
@@ -23,11 +23,10 @@
|
||||
"scripts": {
|
||||
"test": "node --test test/*.test.mjs",
|
||||
"doctor": "node ./bin/cli.js doctor",
|
||||
"mcp": "node ./bin/cli.js mcp start"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"@metaharness/kernel": "^0.1.0",
|
||||
"@metaharness/host-claude-code": "^0.1.0"
|
||||
"mcp": "node ./bin/cli.js mcp start",
|
||||
"sync-skills": "node ./scripts/sync-skills.mjs",
|
||||
"prepack": "node ./scripts/sync-skills.mjs",
|
||||
"prepublishOnly": "npm test"
|
||||
},
|
||||
"keywords": [
|
||||
"wifi-sensing",
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
#!/usr/bin/env node
|
||||
// SPDX-License-Identifier: MIT
|
||||
// ADR-263 O7: skills/*.md is the single source of truth; the host-projected
|
||||
// copies (.claude/skills/<name>/SKILL.md) are GENERATED here at pack time.
|
||||
// Run with --check to verify without writing (used by tests/CI).
|
||||
|
||||
import { readdirSync, readFileSync, writeFileSync, mkdirSync, existsSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const SRC = join(ROOT, 'skills');
|
||||
const DST = join(ROOT, '.claude', 'skills');
|
||||
const checkOnly = process.argv.includes('--check');
|
||||
|
||||
let drift = 0;
|
||||
for (const f of readdirSync(SRC).filter((f) => f.endsWith('.md'))) {
|
||||
const name = f.replace(/\.md$/, '');
|
||||
const src = readFileSync(join(SRC, f), 'utf8');
|
||||
const dstDir = join(DST, name);
|
||||
const dstFile = join(dstDir, 'SKILL.md');
|
||||
const current = existsSync(dstFile) ? readFileSync(dstFile, 'utf8') : null;
|
||||
if (current === src) continue;
|
||||
drift++;
|
||||
if (checkOnly) {
|
||||
console.error(`DRIFT: .claude/skills/${name}/SKILL.md != skills/${f}`);
|
||||
} else {
|
||||
mkdirSync(dstDir, { recursive: true });
|
||||
writeFileSync(dstFile, src);
|
||||
console.error(`synced .claude/skills/${name}/SKILL.md`);
|
||||
}
|
||||
}
|
||||
if (checkOnly && drift > 0) {
|
||||
console.error(`sync-skills --check: ${drift} file(s) out of sync — run \`npm run sync-skills\`.`);
|
||||
process.exit(1);
|
||||
}
|
||||
console.error(`sync-skills: ${drift === 0 ? 'all in sync' : `${drift} file(s) ${checkOnly ? 'OUT OF SYNC' : 'synced'}`}`);
|
||||
@@ -6,24 +6,24 @@ description: Run the ADR-151 per-room calibration pipeline — baseline → enro
|
||||
# calibrate-room
|
||||
|
||||
Turn a provisioned node + sensing-server into a working room model. Pure-Rust,
|
||||
edge-deployable (ADR-151). Use the `ruview.calibrate` tool (installed
|
||||
edge-deployable (ADR-151). Use the `ruview_calibrate` tool (installed
|
||||
`wifi-densepose` binary, else `cargo run -p wifi-densepose-cli`).
|
||||
|
||||
## Sequence
|
||||
|
||||
1. **baseline** — capture the empty room (Welford amplitude + von Mises phase). Leave
|
||||
the room empty.
|
||||
`ruview.calibrate {step: "baseline"}`
|
||||
`ruview_calibrate {step: "baseline"}`
|
||||
2. **enroll** — record the occupant(s) doing the target activities.
|
||||
`ruview.calibrate {step: "enroll"}`
|
||||
`ruview_calibrate {step: "enroll"}`
|
||||
3. **train-room** — train the bank of small specialists from baseline + enrollment.
|
||||
`ruview.calibrate {step: "train-room"}`
|
||||
`ruview_calibrate {step: "train-room"}`
|
||||
4. **room-watch** — live presence/posture/breathing from the trained room.
|
||||
`ruview.calibrate {step: "room-watch"}` (or the `room-watch` skill)
|
||||
`ruview_calibrate {step: "room-watch"}` (or the `room-watch` skill)
|
||||
|
||||
## Honesty
|
||||
|
||||
The specialists are calibrated to *this* room; cross-room transfer is a separate
|
||||
problem (LoRA recalibration, ADR-079 P9). Report which room a number came from, and
|
||||
tag presence/vitals accuracy MEASURED only with a held-out check — run
|
||||
`ruview.claim_check` on the writeup.
|
||||
`ruview_claim_check` on the writeup.
|
||||
|
||||
@@ -8,12 +8,12 @@ description: Zero-to-sensing path picker for RuView (WiFi-DensePose) — pick do
|
||||
Get a newcomer from nothing to a working RuView setup. **First fact to set:** WiFi
|
||||
sensing infers *coarse* pose/presence/breathing from Channel State Information — it
|
||||
is **not a camera**, and any accuracy number must be MEASURED against a baseline
|
||||
(use the `verify` skill / `ruview.claim_check` tool). Never present WiFi output as
|
||||
(use the `verify` skill / `ruview_claim_check` tool). Never present WiFi output as
|
||||
camera-grade.
|
||||
|
||||
## Pick a path
|
||||
|
||||
Run `ruview.onboard {path}` or decide from:
|
||||
Run `ruview_onboard {path}` or decide from:
|
||||
|
||||
1. **docker-demo** — fastest, no hardware. Replays sample CSI into the dashboard.
|
||||
`docker run -p 8000:8000 ruvnet/wifi-densepose` → open `http://localhost:8000`.
|
||||
@@ -26,5 +26,5 @@ Run `ruview.onboard {path}` or decide from:
|
||||
## Then
|
||||
|
||||
- Live sensing → go to **provision-node**, then **calibrate-room**.
|
||||
- Evaluating a model/claim → go to **verify** and run `ruview.claim_check` on any
|
||||
- Evaluating a model/claim → go to **verify** and run `ruview_claim_check` on any
|
||||
report before you quote a number.
|
||||
|
||||
@@ -28,7 +28,7 @@ esptool --chip esp32s3 -p <PORT> -b 460800 write_flash \
|
||||
0xf000 ota_data_initial.bin 0x20000 esp32-csi-node-s3-8mb.bin
|
||||
```
|
||||
|
||||
(`ruview.node_flash` returns the exact pinned command rather than running an
|
||||
(`ruview_node_flash` returns the exact pinned command rather than running an
|
||||
unattended flash.)
|
||||
|
||||
## 3. Provision
|
||||
@@ -44,6 +44,6 @@ Never echo or commit the WiFi password.
|
||||
|
||||
## 4. Confirm CSI is flowing
|
||||
|
||||
`ruview.node_monitor {port}` — PASS criteria: serial shows `CSI cb #...` callbacks and
|
||||
`ruview_node_monitor {port}` — PASS criteria: serial shows `CSI cb #...` callbacks and
|
||||
(on a bare board) `CSI filter upgraded to MGMT+DATA`. No callbacks → the node isn't
|
||||
capturing; do not proceed to calibration.
|
||||
|
||||
@@ -29,5 +29,5 @@ or temporal leakage. Example honest result (ADR-181):
|
||||
1. Run the mean-pose baseline on the same split.
|
||||
2. Report `(model − baseline)` in pp, with the split definition (chronological /
|
||||
blocked-gap / grouped-bucket; no leakage).
|
||||
3. `ruview.claim_check` the writeup — it flags any untagged or 100%/perfect claim.
|
||||
3. `ruview_claim_check` the writeup — it flags any untagged or 100%/perfect claim.
|
||||
4. If it's a benchmark vs SOTA, tag MEASURED-EQUIVALENT only with the reproducer.
|
||||
|
||||
@@ -9,7 +9,7 @@ The "prove everything" skill. Nothing ships as validated without this.
|
||||
|
||||
## Deterministic proof (Trust Kill Switch)
|
||||
|
||||
`ruview.verify` runs `archive/v1/data/proof/verify.py`: it feeds a reference signal
|
||||
`ruview_verify` runs `archive/v1/data/proof/verify.py`: it feeds a reference signal
|
||||
through the production pipeline and hashes the output against
|
||||
`expected_features.sha256`. Must print **VERDICT: PASS**. If numpy/scipy changed the
|
||||
hash, regenerate with `verify.py --generate-hash` then re-verify.
|
||||
@@ -28,7 +28,7 @@ crate versions — a recipient can re-verify with one command.
|
||||
|
||||
## Claim honesty
|
||||
|
||||
Run `ruview.claim_check {text}` on any report, README section, PR body, or model card
|
||||
Run `ruview_claim_check {text}` on any report, README section, PR body, or model card
|
||||
before quoting accuracy. It flags:
|
||||
- untagged accuracy numbers (must be MEASURED / CLAIMED / SYNTHETIC),
|
||||
- MEASURED claims with no reproducer cited,
|
||||
|
||||
@@ -4,15 +4,44 @@
|
||||
// The project was accused of AI-slop; the cultural fix is that every accuracy
|
||||
// number must be tagged MEASURED (with a reproducer) or CLAIMED/SYNTHETIC, and
|
||||
// the retracted "100% accuracy" framing must never reappear untagged. This module
|
||||
// is the static enforcement of that, shared by the `ruview.claim_check` MCP tool,
|
||||
// is the static enforcement of that, shared by the `ruview_claim_check` MCP tool,
|
||||
// the `npx ruview claim-check` CLI, and the claude-code pre-output hook.
|
||||
|
||||
/** Phrases that signal a quantitative accuracy claim. */
|
||||
/** Phrases that signal a quantitative accuracy claim (safe as substrings). */
|
||||
const METRIC_TERMS = [
|
||||
'accuracy', 'pck', 'pck@', 'f1', 'precision', 'recall', 'map', 'auc',
|
||||
'iou', 'mpjpe', 'error rate', 'detection rate', 'true positive',
|
||||
'accuracy', 'pck', 'precision', 'recall',
|
||||
'mpjpe', 'error rate', 'detection rate', 'true positive',
|
||||
];
|
||||
|
||||
// Short/ambiguous metric tokens (ADR-263 F11): 'map' is usually the English
|
||||
// word or a file extension, 'f1'/'o1' collide with finding/option labels.
|
||||
// They only count as metric mentions when word-bounded, not a `.map` file
|
||||
// reference, and the line (after scrubbing) carries a number — "mAP 62.3" is
|
||||
// a claim, "F-numbers map to findings" is not.
|
||||
// 'map' additionally must not be a `.map` file suffix or a hyphenated
|
||||
// compound ("map-free", "map-reduce") — mAP the metric never appears as either.
|
||||
const METRIC_TERMS_SHORT = [/(?<![.\w])map\b(?!-)/, /\bf1\b/, /\bauc\b/, /\biou\b/];
|
||||
// Finding/option labels (F1, O2, …) count as labels unless the token sits in a
|
||||
// metric context: an immediately following score/=/%/digit or colon ("F1: 0.91"),
|
||||
// or a number later in the same clause ("F1 reaches 0.91" — an F1-score claim).
|
||||
// Bare option refs ("F7 fixes", "O1–O9", "ADR-263 O2") carry no clause number of
|
||||
// their own and stay labels. (A surviving 'f1' still only fires as a metric when
|
||||
// its scrubbed line actually carries a number — see mentionsMetricTerm.)
|
||||
const LABEL_TOKEN_RE = /\b[fo]\d+\b(?!\s*(?:score|=|\d|%|:))(?![^\n.;]*\d)/g;
|
||||
const CODE_SPAN_RE = /`[^`]*`/g; // backticked identifiers are code, not claims
|
||||
const HAS_NUMBER_RE = /\d/;
|
||||
|
||||
/** Line with code spans and finding/option labels removed. */
|
||||
function scrubLine(lower) {
|
||||
return lower.replace(CODE_SPAN_RE, ' ').replace(LABEL_TOKEN_RE, ' ');
|
||||
}
|
||||
|
||||
function mentionsMetricTerm(lower, scrubbed) {
|
||||
if (METRIC_TERMS.some((t) => lower.includes(t))) return true;
|
||||
if (!HAS_NUMBER_RE.test(scrubbed)) return false;
|
||||
return METRIC_TERMS_SHORT.some((re) => re.test(scrubbed));
|
||||
}
|
||||
|
||||
/** Tags that make a claim honest (case-insensitive). */
|
||||
const HONEST_TAGS = ['measured', 'claimed', 'synthetic', 'unvalidated', 'baseline'];
|
||||
|
||||
@@ -20,6 +49,8 @@ const HONEST_TAGS = ['measured', 'claimed', 'synthetic', 'unvalidated', 'baselin
|
||||
const REPRODUCER_HINTS = [
|
||||
'verify.py', 'witness', 'mean-pose', 'mean pose', 'held-out', 'held out',
|
||||
'baseline', 'reproduce', 'sha256', 'boot log', 'pck@20 vs', 'expected_features',
|
||||
// Packaging-claim reproducers (ADR-263/264 npm reviews): the tarball itself.
|
||||
'npm pack', 'npm view', 'npm i ', 'npm install', 'tarball', 'cargo test',
|
||||
];
|
||||
|
||||
const PERCENT_RE = /\b(\d{1,3}(?:\.\d+)?)\s?%/g;
|
||||
@@ -49,7 +80,8 @@ export function claimCheck(text) {
|
||||
|
||||
const hasPercent = PERCENT_RE.test(line);
|
||||
PERCENT_RE.lastIndex = 0; // reset stateful global regex
|
||||
const mentionsMetric = METRIC_TERMS.some((t) => lower.includes(t));
|
||||
const scrubbed = scrubLine(lower);
|
||||
const mentionsMetric = mentionsMetricTerm(lower, scrubbed);
|
||||
if (!hasPercent && !mentionsMetric) return;
|
||||
|
||||
const tagged = HONEST_TAGS.some((t) => lower.includes(t));
|
||||
@@ -67,6 +99,15 @@ export function claimCheck(text) {
|
||||
return;
|
||||
}
|
||||
|
||||
// A quantitative claim needs a number. Digits hidden in a code span still
|
||||
// count — "accuracy reached `0.95`" is a claim — so test the line with only
|
||||
// finding/option labels stripped, NOT the code-span-scrubbed copy: scrubbing
|
||||
// dropped `0.95` and wrongly short-circuited both the untagged and the
|
||||
// MEASURED-without-reproducer checks below. A bare metric word in prose
|
||||
// ("precision matters here", "every accuracy number must be MEASURED") has no
|
||||
// number and is not a taggable claim (ADR-263 F11).
|
||||
if (!hasPercent && !HAS_NUMBER_RE.test(lower.replace(LABEL_TOKEN_RE, ' '))) return;
|
||||
|
||||
// A metric/percent with no honesty tag at all.
|
||||
if (!tagged) {
|
||||
findings.push({
|
||||
@@ -79,7 +120,8 @@ export function claimCheck(text) {
|
||||
return;
|
||||
}
|
||||
|
||||
// Tagged MEASURED but cites no reproducer — still a gap.
|
||||
// Tagged MEASURED but cites no reproducer — still a gap (reached now even
|
||||
// when the only number is inside a code span, e.g. "accuracy `0.97` (MEASURED)").
|
||||
if (lower.includes('measured') && !hasReproducer) {
|
||||
findings.push({
|
||||
severity: 'medium',
|
||||
|
||||
@@ -3,14 +3,23 @@
|
||||
//
|
||||
// Dependency-free on purpose: a published `npx ruview` must `mcp start` without
|
||||
// pulling the full MCP SDK. Implements the subset hosts use: `initialize`,
|
||||
// `tools/list`, `tools/call`, and the `notifications/initialized` ack. Logs go to
|
||||
// stderr ONLY — stdout is the JSON-RPC channel and must stay clean.
|
||||
// `tools/list`, `tools/call`, `ping`, empty `resources/list`/`prompts/list`
|
||||
// stubs, and the `notifications/initialized` ack. Logs go to stderr ONLY —
|
||||
// stdout is the JSON-RPC channel and must stay clean.
|
||||
//
|
||||
// ADR-263 O2: `tools/call` is dispatched asynchronously — a long-running
|
||||
// verify/calibrate no longer blocks ping/tools/list, so hosts that health-check
|
||||
// mid-run see a live server. Responses may therefore arrive out of request
|
||||
// order, which JSON-RPC permits (ids correlate them).
|
||||
|
||||
import { createInterface } from 'node:readline';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { listTools, runTool } from './tools.js';
|
||||
|
||||
const PROTOCOL_VERSION = '2024-11-05';
|
||||
const SERVER_INFO = { name: 'ruview', version: '0.1.0' };
|
||||
// Single-source the version from package.json (ADR-263 O6).
|
||||
const PKG = JSON.parse(readFileSync(new URL('../package.json', import.meta.url), 'utf8'));
|
||||
const SERVER_INFO = { name: 'ruview', version: PKG.version };
|
||||
|
||||
function send(msg) {
|
||||
process.stdout.write(JSON.stringify(msg) + '\n');
|
||||
@@ -19,7 +28,7 @@ function result(id, res) { send({ jsonrpc: '2.0', id, result: res }); }
|
||||
function error(id, code, message) { send({ jsonrpc: '2.0', id, error: { code, message } }); }
|
||||
function log(...a) { process.stderr.write('[ruview-mcp] ' + a.join(' ') + '\n'); }
|
||||
|
||||
function handle(msg) {
|
||||
async function handle(msg) {
|
||||
const { id, method, params } = msg;
|
||||
switch (method) {
|
||||
case 'initialize':
|
||||
@@ -27,19 +36,24 @@ function handle(msg) {
|
||||
protocolVersion: PROTOCOL_VERSION,
|
||||
capabilities: { tools: { listChanged: false } },
|
||||
serverInfo: SERVER_INFO,
|
||||
instructions: 'RuView WiFi-sensing operator tools. All results are fail-closed; accuracy claims must pass ruview.claim_check.',
|
||||
instructions: 'RuView WiFi-sensing operator tools. All results are fail-closed; accuracy claims must pass ruview_claim_check.',
|
||||
});
|
||||
case 'notifications/initialized':
|
||||
case 'initialized':
|
||||
return; // notification — no response
|
||||
case 'notifications/cancelled':
|
||||
return; // notifications — no response
|
||||
case 'ping':
|
||||
return result(id, {});
|
||||
case 'tools/list':
|
||||
return result(id, { tools: listTools() });
|
||||
case 'resources/list':
|
||||
return result(id, { resources: [] });
|
||||
case 'prompts/list':
|
||||
return result(id, { prompts: [] });
|
||||
case 'tools/call': {
|
||||
const name = params?.name;
|
||||
const args = params?.arguments || {};
|
||||
const out = runTool(name, args);
|
||||
const out = await runTool(name, args);
|
||||
// MCP content envelope: text block with the JSON, isError reflects ok=false.
|
||||
return result(id, {
|
||||
content: [{ type: 'text', text: JSON.stringify(out, null, 2) }],
|
||||
@@ -52,17 +66,42 @@ function handle(msg) {
|
||||
}
|
||||
|
||||
export function startMcpServer() {
|
||||
log(`starting (protocol ${PROTOCOL_VERSION}, ${listTools().length} tools)`);
|
||||
log(`starting v${SERVER_INFO.version} (protocol ${PROTOCOL_VERSION}, ${listTools().length} tools)`);
|
||||
const rl = createInterface({ input: process.stdin, crlfDelay: Infinity });
|
||||
|
||||
// tools/call runs are serialized through a FIFO promise chain: hardware/mutating
|
||||
// tools (calibrate, serial monitor, flash) must never overlap. ping/tools/list/
|
||||
// initialize/resources/prompts stay immediate (ADR-263 O2 — a health check must
|
||||
// answer during a long tool run). `toolChain` also lets stdin-close drain the
|
||||
// in-flight call so its response is flushed instead of dropped by process.exit.
|
||||
let toolChain = Promise.resolve();
|
||||
|
||||
const dispatch = (msg) => handle(msg).catch((err) => {
|
||||
if (msg && msg.id !== undefined) error(msg.id, -32603, String(err && err.message || err));
|
||||
log('handler error:', String(err));
|
||||
});
|
||||
|
||||
rl.on('line', (line) => {
|
||||
const s = line.trim();
|
||||
if (!s) return;
|
||||
let msg;
|
||||
try { msg = JSON.parse(s); } catch { return log('bad JSON line dropped'); }
|
||||
try { handle(msg); } catch (err) {
|
||||
if (msg && msg.id !== undefined) error(msg.id, -32603, String(err && err.message || err));
|
||||
log('handler error:', String(err));
|
||||
if (msg && msg.method === 'tools/call') {
|
||||
toolChain = toolChain.then(() => dispatch(msg)); // one tool at a time
|
||||
} else {
|
||||
dispatch(msg); // health/list/handshake answer immediately, even mid tool run
|
||||
}
|
||||
});
|
||||
rl.on('close', () => { log('stdin closed — exiting'); process.exit(0); });
|
||||
|
||||
rl.on('close', () => {
|
||||
// Wait for any queued/in-flight tool call to settle (its response written)
|
||||
// before exiting — fire-and-forget used to race this and drop the response.
|
||||
toolChain.then(() => {
|
||||
log('stdin closed — exiting');
|
||||
const done = () => process.exit(0);
|
||||
// Pipe writes are async; flush buffered stdout before exit.
|
||||
if (process.stdout.writableLength) process.stdout.once('drain', done);
|
||||
else done();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
+140
-59
@@ -7,10 +7,15 @@
|
||||
// `wifi-densepose` binary, an ESP32 on a port) is absent, it returns an honest
|
||||
// negative — never a fabricated success. This mirrors the project's "prove
|
||||
// everything" rule and the RuField fail-closed posture (ADR-262 §3.3).
|
||||
//
|
||||
// ADR-263: handlers are async (promise-based spawn, never spawnSync) so the MCP
|
||||
// server keeps answering ping/tools/list while a long verify/calibrate runs.
|
||||
// Canonical tool names use underscores (host tool-name regexes commonly enforce
|
||||
// ^[a-zA-Z0-9_-]{1,64}$); the historical dotted names are accepted as aliases.
|
||||
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { existsSync, readFileSync } from 'node:fs';
|
||||
import { join, dirname, resolve } from 'node:path';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { existsSync, accessSync, constants } from 'node:fs';
|
||||
import { join, dirname, resolve, delimiter } from 'node:path';
|
||||
import { claimCheck, summarize } from './guardrails.js';
|
||||
|
||||
/** Walk up from `start` to find the RuView monorepo root (or null). */
|
||||
@@ -27,22 +32,75 @@ export function findRepoRoot(start = process.cwd()) {
|
||||
return null;
|
||||
}
|
||||
|
||||
function which(cmd) {
|
||||
const probe = process.platform === 'win32'
|
||||
? spawnSync('where', [cmd], { encoding: 'utf8' })
|
||||
: spawnSync('command', ['-v', cmd], { encoding: 'utf8', shell: true });
|
||||
return probe.status === 0 ? (probe.stdout || '').trim().split(/\r?\n/)[0] : null;
|
||||
// Dep-free PATH scan (ADR-263 O8) — no shell subprocess per lookup. Only hits
|
||||
// are memoized: a miss can resolve later in a long-lived MCP session (the
|
||||
// operator installs python/the CLI mid-run), so misses are re-probed each call.
|
||||
const whichCache = new Map();
|
||||
export function which(cmd) {
|
||||
if (whichCache.has(cmd)) return whichCache.get(cmd);
|
||||
const isWin = process.platform === 'win32';
|
||||
const exts = isWin
|
||||
? (process.env.PATHEXT || '.COM;.EXE;.BAT;.CMD').split(';').filter(Boolean)
|
||||
: [''];
|
||||
let found = null;
|
||||
outer:
|
||||
for (const dir of (process.env.PATH || '').split(delimiter)) {
|
||||
if (!dir) continue;
|
||||
for (const ext of isWin ? ['', ...exts] : exts) {
|
||||
const p = join(dir, cmd + ext);
|
||||
try {
|
||||
accessSync(p, isWin ? constants.F_OK : constants.X_OK);
|
||||
found = p;
|
||||
break outer;
|
||||
} catch { /* keep scanning */ }
|
||||
}
|
||||
}
|
||||
if (found !== null) whichCache.set(cmd, found);
|
||||
return found;
|
||||
}
|
||||
|
||||
function run(cmd, args, opts = {}) {
|
||||
const r = spawnSync(cmd, args, { encoding: 'utf8', timeout: opts.timeout ?? 120000, ...opts });
|
||||
return {
|
||||
status: r.status,
|
||||
ok: r.status === 0,
|
||||
stdout: (r.stdout || '').slice(-8000),
|
||||
stderr: (r.stderr || '').slice(-4000),
|
||||
error: r.error ? r.error.message : null,
|
||||
};
|
||||
// Bounded output tails (ADR-263 O4): spawnSync's default 1 MiB maxBuffer killed
|
||||
// chatty children with ENOBUFS; handlers only ever surface the last few kB, so
|
||||
// keep rolling tails instead of the full stream.
|
||||
const STDOUT_TAIL = 65536;
|
||||
const STDERR_TAIL = 16384;
|
||||
|
||||
/** Promise-based spawn with timeout + rolling output tails. */
|
||||
export function run(cmd, args, opts = {}) {
|
||||
const timeout = opts.timeout ?? 120000;
|
||||
return new Promise((resolvePromise) => {
|
||||
let stdout = '';
|
||||
let stderr = '';
|
||||
let child;
|
||||
try {
|
||||
child = spawn(cmd, args, { cwd: opts.cwd, stdio: ['ignore', 'pipe', 'pipe'] });
|
||||
} catch (e) {
|
||||
resolvePromise({ status: null, ok: false, stdout: '', stderr: '', error: e.message });
|
||||
return;
|
||||
}
|
||||
let timedOut = false;
|
||||
const timer = setTimeout(() => { timedOut = true; child.kill('SIGKILL'); }, timeout);
|
||||
child.stdout.on('data', (d) => {
|
||||
stdout = (stdout + d).slice(-STDOUT_TAIL);
|
||||
});
|
||||
child.stderr.on('data', (d) => {
|
||||
stderr = (stderr + d).slice(-STDERR_TAIL);
|
||||
});
|
||||
child.on('error', (e) => {
|
||||
clearTimeout(timer);
|
||||
resolvePromise({ status: null, ok: false, stdout, stderr, error: e.message });
|
||||
});
|
||||
child.on('close', (status) => {
|
||||
clearTimeout(timer);
|
||||
resolvePromise({
|
||||
status,
|
||||
ok: status === 0,
|
||||
stdout,
|
||||
stderr,
|
||||
error: timedOut ? `timed out after ${timeout} ms` : null,
|
||||
});
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
const ONBOARD_PATHS = {
|
||||
@@ -51,12 +109,36 @@ const ONBOARD_PATHS = {
|
||||
'live-esp32': 'Real sensing. Flash an ESP32-S3 (see `provision-node` skill), point it at the sensing-server, then `calibrate → enroll → train-room → room-watch` (see `calibrate-room`). Good for an actual install.',
|
||||
};
|
||||
|
||||
// Read-only serial monitor script; the port arrives via sys.argv (ADR-263 O5 —
|
||||
// never spliced into interpreter source).
|
||||
const MONITOR_SCRIPT = [
|
||||
'import sys,time',
|
||||
'try:',
|
||||
' import serial',
|
||||
'except Exception as e:',
|
||||
" print('NO_PYSERIAL'); sys.exit(3)",
|
||||
'port=sys.argv[1]',
|
||||
'dur=float(sys.argv[2])',
|
||||
'ser=serial.Serial(port,115200,timeout=1)',
|
||||
'csi=0; n=0; t=time.time()',
|
||||
'while time.time()-t<dur:',
|
||||
' ln=ser.readline()',
|
||||
' if not ln: continue',
|
||||
" s=ln.decode('utf-8','replace')",
|
||||
' n+=1',
|
||||
" if 'CSI cb' in s or 'csi_collector' in s: csi+=1",
|
||||
" if 'MGMT+DATA' in s: print('UPGRADE_MGMT_DATA')",
|
||||
'ser.close()',
|
||||
"print(f'LINES={n} CSI={csi}')",
|
||||
].join('\n');
|
||||
|
||||
/**
|
||||
* The tool registry. Each entry: { title, description, inputSchema, handler }.
|
||||
* inputSchema is JSON-Schema (object). handler(args) → JSON-serializable result.
|
||||
* inputSchema is JSON-Schema (object). handler(args) → JSON-serializable result
|
||||
* (sync or promise). Canonical names are underscore-form.
|
||||
*/
|
||||
export const TOOLS = {
|
||||
'ruview.onboard': {
|
||||
ruview_onboard: {
|
||||
title: 'Onboard',
|
||||
description: 'Pick a RuView setup path (docker-demo | repo-build | live-esp32) and print the next concrete command.',
|
||||
inputSchema: {
|
||||
@@ -74,46 +156,50 @@ export const TOOLS = {
|
||||
repo_root: repo,
|
||||
paths: ONBOARD_PATHS,
|
||||
recommend: repo ? 'repo-build' : 'docker-demo',
|
||||
note: 'WiFi sensing infers coarse pose/presence from CSI — it is not a camera. Accuracy claims must be MEASURED vs a baseline (run `ruview.claim_check`).',
|
||||
note: 'WiFi sensing infers coarse pose/presence from CSI — it is not a camera. Accuracy claims must be MEASURED vs a baseline (run `ruview_claim_check`).',
|
||||
};
|
||||
},
|
||||
},
|
||||
|
||||
'ruview.claim_check': {
|
||||
ruview_claim_check: {
|
||||
title: 'Claim check',
|
||||
description: 'Static lint: scan text for untagged or overstated accuracy claims (the "prove everything" guardrail). Returns findings.',
|
||||
description: 'Static lint: scan text for untagged or overstated accuracy claims (the "prove everything" guardrail). Returns findings. Fail-closed: empty input is an error, not a pass.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
required: ['text'],
|
||||
properties: { text: { type: 'string', description: 'The text to lint (a report, README section, PR body, model card).' } },
|
||||
},
|
||||
handler(args = {}) {
|
||||
const result = claimCheck(String(args.text ?? ''));
|
||||
const text = typeof args.text === 'string' ? args.text : '';
|
||||
if (text.trim().length === 0) {
|
||||
return { ok: false, reason: 'empty_text', hint: 'Pass the text to lint — an empty input must not pass an honesty gate.' };
|
||||
}
|
||||
const result = claimCheck(text);
|
||||
return { ...result, summary: summarize(result) };
|
||||
},
|
||||
},
|
||||
|
||||
'ruview.verify': {
|
||||
ruview_verify: {
|
||||
title: 'Verify (witness)',
|
||||
description: 'Run the deterministic proof (archive/v1/data/proof/verify.py) and report VERDICT. Fail-closed if not in a RuView repo or python is missing.',
|
||||
inputSchema: {
|
||||
type: 'object',
|
||||
properties: { repo: { type: 'string', description: 'RuView repo root. Default: auto-detect from cwd.' } },
|
||||
},
|
||||
handler(args = {}) {
|
||||
async handler(args = {}) {
|
||||
const repo = args.repo ? resolve(args.repo) : findRepoRoot();
|
||||
if (!repo) return { ok: false, reason: 'not_in_ruview_repo', hint: 'Run inside the RuView monorepo or pass {repo}.' };
|
||||
const proof = join(repo, 'archive', 'v1', 'data', 'proof', 'verify.py');
|
||||
if (!existsSync(proof)) return { ok: false, reason: 'proof_missing', path: proof };
|
||||
const py = which('python') || which('python3');
|
||||
if (!py) return { ok: false, reason: 'python_missing', hint: 'Install python to run the deterministic proof.' };
|
||||
const r = run(py, [proof], { cwd: repo, timeout: 180000 });
|
||||
const r = await run(py, [proof], { cwd: repo, timeout: 180000 });
|
||||
const verdict = /VERDICT:\s*PASS/i.test(r.stdout) ? 'PASS' : (/VERDICT:\s*FAIL/i.test(r.stdout) ? 'FAIL' : 'UNKNOWN');
|
||||
return { ok: r.ok && verdict === 'PASS', verdict, exit: r.status, tail: r.stdout.slice(-1200), stderr: r.stderr.slice(-400) };
|
||||
},
|
||||
},
|
||||
|
||||
'ruview.node_monitor': {
|
||||
ruview_node_monitor: {
|
||||
title: 'Node monitor',
|
||||
description: 'Open an ESP32 serial port and assert CSI is flowing (MGMT+DATA). Fail-closed if python+pyserial or the port is absent. Read-only.',
|
||||
inputSchema: {
|
||||
@@ -123,31 +209,13 @@ export const TOOLS = {
|
||||
seconds: { type: 'number', description: 'Capture window (default 12).' },
|
||||
},
|
||||
},
|
||||
handler(args = {}) {
|
||||
async handler(args = {}) {
|
||||
const port = args.port;
|
||||
if (!port) return { ok: false, reason: 'no_port', hint: 'Pass {port} (e.g. COM8).' };
|
||||
if (!port || typeof port !== 'string') return { ok: false, reason: 'no_port', hint: 'Pass {port} (e.g. COM8).' };
|
||||
const py = which('python') || which('python3');
|
||||
if (!py) return { ok: false, reason: 'python_missing' };
|
||||
const dur = Number(args.seconds) > 0 ? Number(args.seconds) : 12;
|
||||
const script = [
|
||||
'import sys,time',
|
||||
'try:',
|
||||
' import serial',
|
||||
'except Exception as e:',
|
||||
" print('NO_PYSERIAL'); sys.exit(3)",
|
||||
`ser=serial.Serial(${JSON.stringify(port)},115200,timeout=1)`,
|
||||
'csi=0; n=0; t=time.time()',
|
||||
`while time.time()-t<${dur}:`,
|
||||
' ln=ser.readline()',
|
||||
' if not ln: continue',
|
||||
" s=ln.decode('utf-8','replace')",
|
||||
' n+=1',
|
||||
" if 'CSI cb' in s or 'csi_collector' in s: csi+=1",
|
||||
" if 'MGMT+DATA' in s: print('UPGRADE_MGMT_DATA')",
|
||||
'ser.close()',
|
||||
"print(f'LINES={n} CSI={csi}')",
|
||||
].join('\n');
|
||||
const r = run(py, ['-c', script], { timeout: (dur + 10) * 1000 });
|
||||
const r = await run(py, ['-c', MONITOR_SCRIPT, port, String(dur)], { timeout: (dur + 10) * 1000 });
|
||||
if (r.stdout.includes('NO_PYSERIAL')) return { ok: false, reason: 'pyserial_missing', hint: 'pip install pyserial' };
|
||||
if (!r.ok) return { ok: false, reason: 'port_error', stderr: r.stderr, error: r.error };
|
||||
const csi = Number((r.stdout.match(/CSI=(\d+)/) || [])[1] || 0);
|
||||
@@ -156,7 +224,7 @@ export const TOOLS = {
|
||||
},
|
||||
},
|
||||
|
||||
'ruview.calibrate': {
|
||||
ruview_calibrate: {
|
||||
title: 'Calibrate room',
|
||||
description: 'Run the ADR-151 room pipeline via the wifi-densepose CLI (baseline→enroll→train-room). Fail-closed if the binary is absent.',
|
||||
inputSchema: {
|
||||
@@ -166,7 +234,7 @@ export const TOOLS = {
|
||||
args: { type: 'array', items: { type: 'string' }, description: 'Extra CLI args passed through.' },
|
||||
},
|
||||
},
|
||||
handler(args = {}) {
|
||||
async handler(args = {}) {
|
||||
const step = args.step || 'baseline';
|
||||
const bin = which('wifi-densepose');
|
||||
const repo = findRepoRoot();
|
||||
@@ -174,13 +242,13 @@ export const TOOLS = {
|
||||
const passthru = Array.isArray(args.args) ? args.args.map(String) : [];
|
||||
// Prefer the installed binary; otherwise cargo-run from the repo.
|
||||
const r = bin
|
||||
? run(bin, [step, ...passthru], { timeout: 300000 })
|
||||
: run('cargo', ['run', '-q', '-p', 'wifi-densepose-cli', '--', step, ...passthru], { cwd: repo, timeout: 600000 });
|
||||
? await run(bin, [step, ...passthru], { timeout: 300000 })
|
||||
: await run('cargo', ['run', '-q', '-p', 'wifi-densepose-cli', '--', step, ...passthru], { cwd: repo, timeout: 600000 });
|
||||
return { ok: r.ok, step, via: bin ? 'binary' : 'cargo', exit: r.status, tail: r.stdout.slice(-1500), stderr: r.stderr.slice(-500) };
|
||||
},
|
||||
},
|
||||
|
||||
'ruview.node_flash': {
|
||||
ruview_node_flash: {
|
||||
title: 'Node flash',
|
||||
description: 'Build+flash an ESP32 firmware variant. MUTATING + hardware. Fail-closed off-Windows or without ESP-IDF. Never claims hardware validation without a boot log.',
|
||||
inputSchema: {
|
||||
@@ -203,14 +271,27 @@ export const TOOLS = {
|
||||
},
|
||||
};
|
||||
|
||||
/** Run one tool by name; returns the structured result (or an error envelope). */
|
||||
export function runTool(name, args) {
|
||||
const tool = TOOLS[name];
|
||||
if (!tool) return { ok: false, reason: 'unknown_tool', name, available: Object.keys(TOOLS) };
|
||||
// Historical dotted names (pre-ADR-263) accepted as call-time aliases; the
|
||||
// underscore form is what tools/list advertises.
|
||||
export const TOOL_ALIASES = Object.fromEntries(
|
||||
Object.keys(TOOLS).map((name) => [name.replace(/_/, '.'), name])
|
||||
);
|
||||
|
||||
/** Resolve a canonical or aliased tool name (or null). */
|
||||
export function resolveToolName(name) {
|
||||
if (TOOLS[name]) return name;
|
||||
if (TOOL_ALIASES[name]) return TOOL_ALIASES[name];
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Run one tool by name (canonical or dotted alias); always resolves to the structured result. */
|
||||
export async function runTool(name, args) {
|
||||
const canonical = resolveToolName(name);
|
||||
if (!canonical) return { ok: false, reason: 'unknown_tool', name, available: Object.keys(TOOLS) };
|
||||
try {
|
||||
return tool.handler(args || {});
|
||||
return await TOOLS[canonical].handler(args || {});
|
||||
} catch (err) {
|
||||
return { ok: false, reason: 'tool_threw', name, error: String(err && err.message || err) };
|
||||
return { ok: false, reason: 'tool_threw', name: canonical, error: String(err && err.message || err) };
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,148 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
// MCP stdio server e2e — spawns `bin/cli.js mcp start` and speaks JSON-RPC.
|
||||
// Pins ADR-263 O2 (ping answered while a long tools/call runs), O6 (version
|
||||
// from package.json), and O8 (underscore names advertised, dotted accepted,
|
||||
// resources/prompts stubs).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs';
|
||||
import { join, dirname } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { which } from '../src/tools.js';
|
||||
|
||||
const PKG_ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
const CLI = join(PKG_ROOT, 'bin', 'cli.js');
|
||||
|
||||
/** Start the MCP server; returns {send, next, close} where next(id) resolves the response with that id. */
|
||||
function startServer() {
|
||||
const child = spawn(process.execPath, [CLI, 'mcp', 'start'], { stdio: ['pipe', 'pipe', 'pipe'] });
|
||||
const waiters = new Map();
|
||||
let buf = '';
|
||||
child.stdout.on('data', (d) => {
|
||||
buf += d;
|
||||
let nl;
|
||||
while ((nl = buf.indexOf('\n')) !== -1) {
|
||||
const line = buf.slice(0, nl).trim();
|
||||
buf = buf.slice(nl + 1);
|
||||
if (!line) continue;
|
||||
const msg = JSON.parse(line);
|
||||
const w = waiters.get(msg.id);
|
||||
if (w) { waiters.delete(msg.id); w(msg); }
|
||||
}
|
||||
});
|
||||
return {
|
||||
send(msg) { child.stdin.write(JSON.stringify(msg) + '\n'); },
|
||||
next(id) { return new Promise((res) => waiters.set(id, res)); },
|
||||
close() { child.stdin.end(); child.kill(); },
|
||||
};
|
||||
}
|
||||
|
||||
test('MCP handshake: initialize reports the package.json version; list endpoints respond', async () => {
|
||||
const pkg = JSON.parse(readFileSync(join(PKG_ROOT, 'package.json'), 'utf8'));
|
||||
const s = startServer();
|
||||
try {
|
||||
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
|
||||
const init = await s.next(1);
|
||||
assert.equal(init.result.serverInfo.version, pkg.version, 'ADR-263 O6: version must match package.json');
|
||||
|
||||
s.send({ jsonrpc: '2.0', id: 2, method: 'tools/list' });
|
||||
const tools = (await s.next(2)).result.tools;
|
||||
assert.equal(tools.length, 6);
|
||||
for (const t of tools) assert.match(t.name, /^[a-zA-Z0-9_-]{1,64}$/, `advertised name not host-safe: ${t.name}`);
|
||||
|
||||
s.send({ jsonrpc: '2.0', id: 3, method: 'resources/list' });
|
||||
assert.deepEqual((await s.next(3)).result, { resources: [] });
|
||||
s.send({ jsonrpc: '2.0', id: 4, method: 'prompts/list' });
|
||||
assert.deepEqual((await s.next(4)).result, { prompts: [] });
|
||||
|
||||
// Dotted legacy name still callable (alias).
|
||||
s.send({ jsonrpc: '2.0', id: 5, method: 'tools/call', params: { name: 'ruview.onboard', arguments: {} } });
|
||||
const call = await s.next(5);
|
||||
assert.equal(call.result.isError, false);
|
||||
} finally {
|
||||
s.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('MCP server answers ping while a long tools/call is in flight (ADR-263 O2)', { skip: !which('python') && !which('python3') ? 'python not on PATH' : false }, async () => {
|
||||
// Fake RuView repo whose verify.py sleeps 3 s then passes.
|
||||
const repo = mkdtempSync(join(tmpdir(), 'ruview-mcp-e2e-'));
|
||||
const proofDir = join(repo, 'archive', 'v1', 'data', 'proof');
|
||||
mkdirSync(proofDir, { recursive: true });
|
||||
writeFileSync(join(proofDir, 'verify.py'), 'import time\ntime.sleep(3)\nprint("VERDICT: PASS")\n');
|
||||
|
||||
const s = startServer();
|
||||
try {
|
||||
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
|
||||
await s.next(1);
|
||||
|
||||
const verifyDone = s.next(10);
|
||||
s.send({ jsonrpc: '2.0', id: 10, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
|
||||
|
||||
// Give the server a beat to start the child, then ping.
|
||||
await new Promise((r) => setTimeout(r, 300));
|
||||
const t0 = Date.now();
|
||||
const pinged = s.next(11);
|
||||
s.send({ jsonrpc: '2.0', id: 11, method: 'ping' });
|
||||
await pinged;
|
||||
const pingMs = Date.now() - t0;
|
||||
assert.ok(pingMs < 1000, `ping took ${pingMs} ms while verify was in flight — server is blocking`);
|
||||
|
||||
const verify = await verifyDone;
|
||||
const payload = JSON.parse(verify.result.content[0].text);
|
||||
assert.equal(payload.verdict, 'PASS');
|
||||
} finally {
|
||||
s.close();
|
||||
rmSync(repo, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('tools/call executions are serialized — two slow calls run sequentially', { skip: !which('python') && !which('python3') ? 'python not on PATH' : false }, async () => {
|
||||
// Two verify.py that each sleep 0.8 s. Serialized ⇒ ~1.6 s+; concurrent ⇒ ~0.8 s.
|
||||
const repo = mkdtempSync(join(tmpdir(), 'ruview-mcp-serial-'));
|
||||
const proofDir = join(repo, 'archive', 'v1', 'data', 'proof');
|
||||
mkdirSync(proofDir, { recursive: true });
|
||||
writeFileSync(join(proofDir, 'verify.py'), 'import time\ntime.sleep(0.8)\nprint("VERDICT: PASS")\n');
|
||||
|
||||
const s = startServer();
|
||||
try {
|
||||
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
|
||||
await s.next(1);
|
||||
|
||||
const t0 = Date.now();
|
||||
const a = s.next(20);
|
||||
const b = s.next(21);
|
||||
s.send({ jsonrpc: '2.0', id: 20, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
|
||||
s.send({ jsonrpc: '2.0', id: 21, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
|
||||
const [ra, rb] = await Promise.all([a, b]);
|
||||
const elapsed = Date.now() - t0;
|
||||
|
||||
assert.equal(JSON.parse(ra.result.content[0].text).verdict, 'PASS');
|
||||
assert.equal(JSON.parse(rb.result.content[0].text).verdict, 'PASS');
|
||||
assert.ok(elapsed > 1400, `two 0.8 s tool calls finished in ${elapsed} ms — they overlapped instead of serializing`);
|
||||
} finally {
|
||||
s.close();
|
||||
rmSync(repo, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('stdin close flushes an in-flight tools/call response before exit', async () => {
|
||||
const child = spawn(process.execPath, [CLI, 'mcp', 'start'], { stdio: ['pipe', 'pipe', 'pipe'] });
|
||||
let out = '';
|
||||
child.stdout.on('data', (d) => { out += d; });
|
||||
const exited = new Promise((res) => child.on('exit', res));
|
||||
|
||||
// Write a tools/call then immediately close stdin. The old fire-and-forget
|
||||
// dispatch raced rl 'close' → process.exit and could drop this response.
|
||||
child.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 42, method: 'tools/call', params: { name: 'ruview_onboard', arguments: {} } }) + '\n');
|
||||
child.stdin.end();
|
||||
|
||||
await exited;
|
||||
const msgs = out.trim().split('\n').filter(Boolean).map((l) => JSON.parse(l));
|
||||
const resp = msgs.find((m) => m.id === 42);
|
||||
assert.ok(resp, 'the in-flight tools/call response must be flushed to stdout before exit');
|
||||
assert.equal(resp.result.isError, false);
|
||||
});
|
||||
@@ -1,12 +1,18 @@
|
||||
// SPDX-License-Identifier: MIT
|
||||
// RuView harness tests — Node's built-in test runner (no devDeps to install).
|
||||
// Run: `node --test test/` (or `npm test`).
|
||||
// Run: `node --test test/*.test.mjs` (or `npm test`).
|
||||
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readdirSync, readFileSync, mkdtempSync, writeFileSync, rmSync } from 'node:fs';
|
||||
import { join, dirname, delimiter } from 'node:path';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { claimCheck, summarize } from '../src/guardrails.js';
|
||||
import { TOOLS, runTool, listTools, findRepoRoot } from '../src/tools.js';
|
||||
import { run } from '../bin/cli.js';
|
||||
import { TOOLS, TOOL_ALIASES, runTool, listTools, findRepoRoot, run, which } from '../src/tools.js';
|
||||
import { run as cliRun } from '../bin/cli.js';
|
||||
|
||||
const PKG_ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
|
||||
|
||||
test('guardrail flags the retracted 100% framing as high severity', () => {
|
||||
const r = claimCheck('Our model reaches 100% accuracy on every pose.');
|
||||
@@ -37,71 +43,190 @@ test('guardrail ignores non-metric prose', () => {
|
||||
assert.equal(claimCheck('').ok, true);
|
||||
});
|
||||
|
||||
// ADR-263 F11/O9: precision pins — short metric tokens must not fire on prose.
|
||||
test('guardrail does not false-positive on "map"/"F1" prose (ADR-263 F11)', () => {
|
||||
assert.equal(claimCheck('F-numbers map to findings.').ok, true);
|
||||
assert.equal(claimCheck('### F1 (HIGH, broken export): `require` points at a missing file').ok, true);
|
||||
assert.equal(claimCheck('The 0.1.0 tarball ships 44 `.map` files = 62,698 B of dead weight.').ok, true);
|
||||
assert.equal(claimCheck('the source maps can never resolve').ok, true);
|
||||
assert.equal(claimCheck('- **O1 (F1):** fix `exports` (see F2 for the 33% map weight — MEASURED, tarball listing)').ok, true);
|
||||
assert.equal(claimCheck('ADR-264: exports fix, map-free tarball, session-per-transport').ok, true);
|
||||
});
|
||||
|
||||
test('guardrail still catches real short-token metric claims', () => {
|
||||
assert.equal(claimCheck('We reach mAP 62.3 on COCO.').ok, false);
|
||||
assert.equal(claimCheck('F1 score of 0.91 on the held set.').ok, false, 'f1 with a real score must still fire');
|
||||
assert.equal(claimCheck('IoU 0.75 across rooms.').ok, false);
|
||||
});
|
||||
|
||||
// Digits hidden in a code span still make a claim — scrubbing must not blind the
|
||||
// number gate to `0.95` (regression: code-span number bypassed the gate).
|
||||
test('guardrail flags an accuracy number stated inside a code span', () => {
|
||||
const r = claimCheck('Count accuracy reached `0.95` in our tests.');
|
||||
assert.equal(r.ok, false, JSON.stringify(r.findings));
|
||||
assert.ok(r.findings.some((f) => /not tagged/i.test(f.reason)));
|
||||
});
|
||||
|
||||
// A MEASURED claim whose only number hides in a code span must still reach the
|
||||
// missing-reproducer check (regression: the scrubbed gate short-circuited it).
|
||||
// Bare metric prose with no number at all (e.g. the README rule text) stays a pass.
|
||||
test('guardrail flags a MEASURED code-span number with no reproducer', () => {
|
||||
const r = claimCheck('Detection accuracy `0.97` on the set (MEASURED).');
|
||||
assert.equal(r.ok, false, JSON.stringify(r.findings));
|
||||
assert.ok(r.findings.some((f) => /no reproducer/i.test(f.reason)));
|
||||
assert.equal(claimCheck('Every accuracy number must be MEASURED against a baseline.').ok, true);
|
||||
});
|
||||
|
||||
// F1-score phrasings ("F1: 0.91", "F1 reaches 0.91") were scrubbed as option
|
||||
// labels and slipped through; option refs alone must still not false-positive.
|
||||
test('guardrail catches F1-score claims but not bare option refs (ADR-263 F11)', () => {
|
||||
assert.equal(claimCheck('F1: 0.91 on the held-out set.').ok, false, 'F1: value is a metric claim');
|
||||
assert.equal(claimCheck('F1 reaches 0.91 on the held-out set.').ok, false, 'F1 with a nearby number is a claim');
|
||||
assert.equal(claimCheck('Options O1–O9 are tracked in ADR-263 O2.').ok, true, 'option labels are not metrics');
|
||||
assert.equal(claimCheck('ADR-263 O2 lands the exports fix.').ok, true);
|
||||
});
|
||||
|
||||
test('summarize gives PASS/finding text', () => {
|
||||
assert.match(summarize(claimCheck('nothing here')), /PASS/);
|
||||
assert.match(summarize(claimCheck('100% accuracy')), /finding/);
|
||||
});
|
||||
|
||||
test('registry exposes the documented tools with schemas', () => {
|
||||
test('registry exposes the documented tools with schemas (underscore-canonical)', () => {
|
||||
const names = Object.keys(TOOLS);
|
||||
for (const n of ['ruview.onboard', 'ruview.claim_check', 'ruview.verify', 'ruview.node_monitor', 'ruview.calibrate', 'ruview.node_flash']) {
|
||||
for (const n of ['ruview_onboard', 'ruview_claim_check', 'ruview_verify', 'ruview_node_monitor', 'ruview_calibrate', 'ruview_node_flash']) {
|
||||
assert.ok(names.includes(n), `missing ${n}`);
|
||||
assert.equal(TOOLS[n].inputSchema.type, 'object');
|
||||
assert.match(n, /^[a-zA-Z0-9_-]{1,64}$/, 'canonical names must satisfy host tool-name regexes');
|
||||
}
|
||||
assert.equal(listTools().length, names.length);
|
||||
});
|
||||
|
||||
test('ruview.onboard returns paths and a recommendation', () => {
|
||||
const r = runTool('ruview.onboard', {});
|
||||
test('dotted legacy names resolve via aliases (ADR-263 O8)', async () => {
|
||||
assert.equal(TOOL_ALIASES['ruview.claim_check'], 'ruview_claim_check');
|
||||
assert.equal(TOOL_ALIASES['ruview.node_monitor'], 'ruview_node_monitor');
|
||||
const r = await runTool('ruview.onboard', {});
|
||||
assert.equal(r.ok, true);
|
||||
});
|
||||
|
||||
test('ruview_onboard returns paths and a recommendation', async () => {
|
||||
const r = await runTool('ruview_onboard', {});
|
||||
assert.equal(r.ok, true);
|
||||
assert.ok(r.paths['live-esp32']);
|
||||
assert.ok(['repo-build', 'docker-demo'].includes(r.recommend));
|
||||
});
|
||||
|
||||
test('ruview.claim_check tool wraps the guardrail', () => {
|
||||
const r = runTool('ruview.claim_check', { text: '100% accuracy' });
|
||||
test('ruview_claim_check tool wraps the guardrail', async () => {
|
||||
const r = await runTool('ruview_claim_check', { text: '100% accuracy' });
|
||||
assert.equal(r.ok, false);
|
||||
assert.match(r.summary, /honesty|tag|MEASURED|finding/i);
|
||||
});
|
||||
|
||||
test('unknown tool fails closed', () => {
|
||||
const r = runTool('ruview.does_not_exist', {});
|
||||
// ADR-263 F1/O1: the honesty gate must fail closed on empty input.
|
||||
test('ruview_claim_check fails closed on empty/missing text', async () => {
|
||||
const empty = await runTool('ruview_claim_check', { text: '' });
|
||||
assert.equal(empty.ok, false);
|
||||
assert.equal(empty.reason, 'empty_text');
|
||||
const missing = await runTool('ruview_claim_check', {});
|
||||
assert.equal(missing.ok, false);
|
||||
assert.equal(missing.reason, 'empty_text');
|
||||
});
|
||||
|
||||
test('unknown tool fails closed', async () => {
|
||||
const r = await runTool('ruview_does_not_exist', {});
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'unknown_tool');
|
||||
});
|
||||
|
||||
test('node_monitor fails closed without a port', () => {
|
||||
const r = runTool('ruview.node_monitor', {});
|
||||
test('node_monitor fails closed without a port', async () => {
|
||||
const r = await runTool('ruview_node_monitor', {});
|
||||
assert.equal(r.ok, false);
|
||||
assert.equal(r.reason, 'no_port');
|
||||
});
|
||||
|
||||
test('node_flash refuses without confirm (mutating guard)', () => {
|
||||
const r = runTool('ruview.node_flash', { port: 'COM8', variant: 's3-8mb' });
|
||||
test('node_flash refuses without confirm (mutating guard)', async () => {
|
||||
const r = await runTool('ruview_node_flash', { port: 'COM8', variant: 's3-8mb' });
|
||||
assert.equal(r.ok, false);
|
||||
// either not-confirmed (win32) or unsupported_platform (posix) — both fail-closed
|
||||
assert.ok(['not_confirmed', 'unsupported_platform'].includes(r.reason));
|
||||
});
|
||||
|
||||
test('verify fails closed when not in a RuView repo', () => {
|
||||
test('verify fails closed when not in a RuView repo', async () => {
|
||||
// point at a tmp dir with no repo markers
|
||||
const r = runTool('ruview.verify', { repo: process.platform === 'win32' ? 'C:/Windows/Temp' : '/tmp' });
|
||||
const r = await runTool('ruview_verify', { repo: process.platform === 'win32' ? 'C:/Windows/Temp' : '/tmp' });
|
||||
assert.equal(r.ok, false);
|
||||
assert.ok(['proof_missing', 'python_missing'].includes(r.reason), r.reason);
|
||||
});
|
||||
|
||||
// ADR-263 F2/O2: registry-level concurrency — a slow child must not block
|
||||
// other tool calls (run() is promise-based, never spawnSync).
|
||||
test('run() is non-blocking: a fast tool completes while a slow child runs', async () => {
|
||||
const slow = run('node', ['-e', 'setTimeout(() => {}, 2000)'], { timeout: 5000 });
|
||||
const t0 = Date.now();
|
||||
const fast = await runTool('ruview_onboard', {});
|
||||
const elapsed = Date.now() - t0;
|
||||
assert.equal(fast.ok, true);
|
||||
assert.ok(elapsed < 1000, `onboard took ${elapsed} ms while a 2 s child was running`);
|
||||
const r = await slow;
|
||||
assert.equal(r.ok, true);
|
||||
});
|
||||
|
||||
test('run() reports a timeout as a failure, not a hang', async () => {
|
||||
const r = await run('node', ['-e', 'setTimeout(() => {}, 10000)'], { timeout: 300 });
|
||||
assert.equal(r.ok, false);
|
||||
assert.match(String(r.error), /timed out/);
|
||||
});
|
||||
|
||||
test('run() bounds captured output instead of dying on big streams (ADR-263 O4)', async () => {
|
||||
// 4 MiB of stdout would have hit spawnSync's 1 MiB default maxBuffer (ENOBUFS).
|
||||
const r = await run('node', ['-e', "process.stdout.write('x'.repeat(4 * 1024 * 1024)); console.log('TAIL_MARKER')"], { timeout: 30000 });
|
||||
assert.equal(r.ok, true);
|
||||
assert.ok(r.stdout.length <= 65536, `tail not bounded: ${r.stdout.length}`);
|
||||
assert.ok(r.stdout.includes('TAIL_MARKER'), 'tail must keep the end of the stream');
|
||||
});
|
||||
|
||||
test('which() finds node and re-probes misses (hits are cached)', () => {
|
||||
assert.ok(which('node'), 'node must be on PATH in the test env');
|
||||
assert.equal(which('definitely-not-a-binary-xyz'), null);
|
||||
assert.equal(which('definitely-not-a-binary-xyz'), null); // re-probed, still absent
|
||||
});
|
||||
|
||||
// ADR-263 O8: a miss must not be cached — an operator who installs a tool
|
||||
// mid-session (e.g. python after a python_missing failure) must be found next call.
|
||||
test('which() re-probes after a miss so a newly-installed tool is found', () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'ruview-which-'));
|
||||
const name = 'ruview-probe-xyz';
|
||||
const isWin = process.platform === 'win32';
|
||||
const bin = join(dir, isWin ? `${name}.cmd` : name);
|
||||
const prevPath = process.env.PATH;
|
||||
try {
|
||||
assert.equal(which(name), null, 'not on PATH yet → miss');
|
||||
writeFileSync(bin, isWin ? '@echo off\n' : '#!/bin/sh\n', { mode: 0o755 });
|
||||
process.env.PATH = dir + delimiter + prevPath;
|
||||
assert.ok(which(name), 'installed mid-session → the miss must not have been cached');
|
||||
} finally {
|
||||
process.env.PATH = prevPath;
|
||||
rmSync(dir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('CLI run(): claim-check exits non-zero on a bad claim', async () => {
|
||||
const code = await run(['claim-check', '--text', '100% accuracy']);
|
||||
const code = await cliRun(['claim-check', '--text', '100% accuracy']);
|
||||
assert.notEqual(code, 0);
|
||||
});
|
||||
|
||||
// ADR-263 F1/O1: the CLI must not PASS silently with no input.
|
||||
test('CLI run(): claim-check with no input exits 2 (fail-closed)', async () => {
|
||||
assert.equal(await cliRun(['claim-check']), 2);
|
||||
assert.equal(await cliRun(['claim-check', '--text', ' ']), 2);
|
||||
});
|
||||
|
||||
test('CLI run(): doctor exits 0 (tools-only path)', async () => {
|
||||
const code = await run(['doctor']);
|
||||
const code = await cliRun(['doctor']);
|
||||
assert.equal(code, 0);
|
||||
});
|
||||
|
||||
test('CLI run(): unknown command exits non-zero', async () => {
|
||||
assert.notEqual(await run(['definitely-not-a-command']), 0);
|
||||
assert.notEqual(await cliRun(['definitely-not-a-command']), 0);
|
||||
});
|
||||
|
||||
test('findRepoRoot locates this monorepo from cwd', () => {
|
||||
@@ -109,3 +234,23 @@ test('findRepoRoot locates this monorepo from cwd', () => {
|
||||
const root = findRepoRoot();
|
||||
assert.ok(root === null || typeof root === 'string');
|
||||
});
|
||||
|
||||
// ADR-263 F7/O7: skills ship from one source; the projected copies must match.
|
||||
test('.claude/skills/*/SKILL.md are byte-identical to skills/*.md', () => {
|
||||
const srcDir = join(PKG_ROOT, 'skills');
|
||||
for (const f of readdirSync(srcDir).filter((f) => f.endsWith('.md'))) {
|
||||
const name = f.replace(/\.md$/, '');
|
||||
const src = readFileSync(join(srcDir, f), 'utf8');
|
||||
const projected = readFileSync(join(PKG_ROOT, '.claude', 'skills', name, 'SKILL.md'), 'utf8');
|
||||
assert.equal(projected, src, `skill drift: ${name} — run \`npm run sync-skills\``);
|
||||
}
|
||||
});
|
||||
|
||||
// ADR-263 F6/O6 + F3/O3: package hygiene pins.
|
||||
test('package.json has no optionalDependencies and no hardcoded server version drift', () => {
|
||||
const pkg = JSON.parse(readFileSync(join(PKG_ROOT, 'package.json'), 'utf8'));
|
||||
assert.equal(pkg.optionalDependencies, undefined, 'ADR-263 O3: optional deps tripled the cold npx install');
|
||||
assert.equal(pkg.dependencies, undefined, 'the harness is dependency-free by design');
|
||||
const mcpSrc = readFileSync(join(PKG_ROOT, 'src', 'mcp-server.js'), 'utf8');
|
||||
assert.ok(!/version:\s*'\d+\.\d+\.\d+'/.test(mcpSrc), 'ADR-263 O6: server version must come from package.json');
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user