Files
ruvnet--RuView/harness/ruview/test/mcp.test.mjs
T
rUv e6f26e9ac9 docs(adr): deep review of the RuView npm surface — ADR-263/264/265 optimization strategies (#1229)
* docs(adr): deep review of the RuView npm surface — ADR-263/264/265 optimization strategies

ADR-263 — @ruvnet/ruview@0.1.0 harness review (O1–O9):
- HIGH: claim-check CLI fails open on empty input (no --text/--file -> PASS exit 0)
- HIGH: MCP stdio server head-of-line blocking (spawnSync verify/calibrate up to 600s)
- MEASURED: optionalDependencies triple the cold npx install (4 pkgs/620kB/71 files
  vs 1 pkg/172kB/22 files with --omit=optional) for a path that never imports them
- maxBuffer truncation, python -c port interpolation, version drift, duplicate skills,
  guardrail METRIC_TERMS substring false positives ('map'/'F1' — found by dogfooding
  claim-check on these very ADRs), zero CI

ADR-264 — @ruvnet/rvagent@0.1.0 + @ruv/ruview-cli review (O1–O9), verified against
the published registry tarball:
- HIGH: exports.require -> dist/index.cjs which is never built nor published
- MEASURED: 44 dead source-map files = 62,698B of the 188kB unpacked payload
- stdio-only server described as dual-transport; mixed dot/underscore tool names;
  double Zod validation + hand-duplicated advertised schemas; 2-fd leak per training
  job; unbounded body in the unwired HTTP scaffold; dead detectCogBinary candidates;
  ruview bin-name collision

ADR-265 — cross-cutting npm distribution strategy: npm-packages.yml CI matrix
(test + pack-content/size gate + tarball-install smoke test), publish-from-CI-only
with npm provenance, version single-sourcing from package.json, bin/namespace
ownership (ruview bin belongs to @ruvnet/ruview), claim-check on package READMEs.

Docs only — no runtime code changed. Index/CHANGELOG/CLAUDE.md/README counts updated.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz

* fix(npm): implement ADR-263/264/265 — harness fail-closed + async MCP, rvagent packaging/transport/naming, npm CI+provenance gate

ADR-263 (@ruvnet/ruview 0.2.0), O1-O9:
- claim-check fails closed on empty input (CLI exit 2, empty_text tool error)
- MCP stdio server dispatches tools/call asynchronously (promise-based spawn);
  ping answers while a 3s fake verify runs — pinned by new e2e test
- optionalDependencies dropped: cold npx installs exactly 1 package
  (MEASURED: was 4 pkgs/620kB/71 files via npm i in a clean prefix)
- bounded rolling output tails replace spawnSync 1MiB maxBuffer
- node_monitor port passed via sys.argv, never spliced into python -c source
- serverInfo.version read from package.json; resources/prompts stubs
- skills single-sourced: prepack sync script generates .claude/skills/ copies
- which() = memoized dep-free PATH scan
- tools underscore-canonical (ruview_claim_check, ...) + dotted aliases
- guardrail precision: word-boundary map/f1/auc/iou, code-span + F1/O2 label
  scrubbing, quantitative-claims-only; packaging reproducer hints
- 30/30 tests (was 17), incl. concurrency e2e + fail-open regression pins

ADR-264 (@ruvnet/rvagent 0.2.0), O1-O9:
- exports fixed: types-first, phantom dist/index.cjs require target removed
- tarball map-free: 127,704B unpacked / 46 files / 0 maps (MEASURED,
  npm pack --dry-run; was 188kB incl. 44 maps referencing unshipped src)
- Streamable HTTP actually wired behind RVAGENT_HTTP_PORT: one transport +
  one MCP server per session (mcp-session-id routing), 1MiB body cap (413),
  port-aware localhost origin gate; dual-transport description now true
- tools renamed underscore-canonical with dotted router-only aliases
- single Zod validation gate; advertised inputSchema generated from the same
  Zod source (zod-to-json-schema)
- train_count: parent log fds closed (was leaking 2/job); job records
  persisted to <jobsDir>/<id>.json (job_status survives restarts); bounded
  log-tail reads
- detectCogBinary probes its candidates instead of dead-coding them
- version from package.json; @types/express dropped; @types/jest -> 29
- README rewritten to match reality (no phantom subcommands/policy layer)
- 99/99 jest tests (incl. new session/body-cap suite + previously-broken
  manifest suite); stdio handshake + HTTP session flow smoke-tested live

ADR-265 D1-D4:
- .github/workflows/npm-packages.yml: 3-package x Node 20/22 gate — tests,
  version-literal grep (D3), pack-content/size gate, tarball-install smoke
  test (catches the ADR-264 F1 class), README claim-check (D4)
- .github/workflows/ruview-npm-release.yml: publish from CI only with
  npm publish --provenance
- @ruv/ruview-cli bin renamed ruview-cli (ruview bin belongs to
  @ruvnet/ruview); version single-sourced
- ci.yml NODE_VERSION 18 -> 20

ADR statuses updated to Accepted/implemented; harness manifest re-pinned;
ADR-263/264/265 + both package READMEs pass claim-check.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz

* perf(rvagent): lazy-load HTTP transport + memoize generated tool schemas

stdio time-to-first-response ~242ms -> ~189ms (-22%; MEASURED, median of
repeated initialize round-trips against dist/index.js in this container).

- ./http-transport.js now imported lazily inside the RVAGENT_HTTP_PORT
  branch: it chain-loads the MCP SDK streamableHttp module (~48ms MEASURED
  via per-module import() timing) which the default stdio path never uses
- toolInputJsonSchema memoized per tool: schemas are static for the process
  lifetime; under the session-per-server HTTP model every session calls
  tools/list, so stop re-walking the Zod tree each time

No behavior change: 99/99 jest tests; HTTP session flow re-smoke-tested
through the lazy import path (initialize -> 200 + mcp-session-id).

Profiled @ruvnet/ruview too and left it alone: 50ms CLI startup vs ~29ms
bare 'node -e ""' floor on the same box (MEASURED) — already near the
interpreter floor with zero dependencies.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01WrGfTGKv1oWZ6iwXZACULz

* ci(ruview-cli): pass jest --passWithNoTests so the private no-test package doesn't fail the npm-packages matrix

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(npm): address 10 verified review findings in harness + rvagent before 0.2.0 publish

harness/ruview (@ruvnet/ruview):
- guardrails: digit gate now sees numbers inside code spans; F1-style
  metric tokens followed by ':' or a nearby number are no longer scrubbed
  (fail-open regressions in the honesty gate)
- mcp-server: tools/call requests serialize through a FIFO promise chain
  (hardware/mutating tools never overlap) while ping/tools/list stay
  immediate; stdin close drains in-flight responses before exit
- tools: which() no longer memoizes negative lookups

tools/ruview-mcp (@ruvnet/rvagent):
- index: realpath invoked-directly guard — library import no longer
  connects a stdio transport to the consumer's process
- http-transport: explicit allowedOrigins is exact-match only (localhost
  any-port convenience applies only with no configured allowlist);
  session map gains maxSessions=64 + 5min idle TTL sweep
- train-count: job records persist the child pid and reconcile stale
  'running' status after a server restart (exit-code marker or dead pid)
- config: cog binary candidates ordered by process.arch

.github/workflows/ruview-npm-release.yml: port the full ADR-265 D1 gate
(version-literal check, unpacked-size budget, tarball-install smoke test)
from npm-packages.yml so the publish path enforces what the header claims.

Tests: harness 30→36, rvagent 99→112, all passing.

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-07-02 13:11:15 -04:00

149 lines
6.5 KiB
JavaScript

// SPDX-License-Identifier: MIT
// MCP stdio server e2e — spawns `bin/cli.js mcp start` and speaks JSON-RPC.
// Pins ADR-263 O2 (ping answered while a long tools/call runs), O6 (version
// from package.json), and O8 (underscore names advertised, dotted accepted,
// resources/prompts stubs).
import { test } from 'node:test';
import assert from 'node:assert/strict';
import { spawn } from 'node:child_process';
import { mkdtempSync, mkdirSync, writeFileSync, readFileSync, rmSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { tmpdir } from 'node:os';
import { fileURLToPath } from 'node:url';
import { which } from '../src/tools.js';
const PKG_ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
const CLI = join(PKG_ROOT, 'bin', 'cli.js');
/** Start the MCP server; returns {send, next, close} where next(id) resolves the response with that id. */
function startServer() {
const child = spawn(process.execPath, [CLI, 'mcp', 'start'], { stdio: ['pipe', 'pipe', 'pipe'] });
const waiters = new Map();
let buf = '';
child.stdout.on('data', (d) => {
buf += d;
let nl;
while ((nl = buf.indexOf('\n')) !== -1) {
const line = buf.slice(0, nl).trim();
buf = buf.slice(nl + 1);
if (!line) continue;
const msg = JSON.parse(line);
const w = waiters.get(msg.id);
if (w) { waiters.delete(msg.id); w(msg); }
}
});
return {
send(msg) { child.stdin.write(JSON.stringify(msg) + '\n'); },
next(id) { return new Promise((res) => waiters.set(id, res)); },
close() { child.stdin.end(); child.kill(); },
};
}
test('MCP handshake: initialize reports the package.json version; list endpoints respond', async () => {
const pkg = JSON.parse(readFileSync(join(PKG_ROOT, 'package.json'), 'utf8'));
const s = startServer();
try {
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
const init = await s.next(1);
assert.equal(init.result.serverInfo.version, pkg.version, 'ADR-263 O6: version must match package.json');
s.send({ jsonrpc: '2.0', id: 2, method: 'tools/list' });
const tools = (await s.next(2)).result.tools;
assert.equal(tools.length, 6);
for (const t of tools) assert.match(t.name, /^[a-zA-Z0-9_-]{1,64}$/, `advertised name not host-safe: ${t.name}`);
s.send({ jsonrpc: '2.0', id: 3, method: 'resources/list' });
assert.deepEqual((await s.next(3)).result, { resources: [] });
s.send({ jsonrpc: '2.0', id: 4, method: 'prompts/list' });
assert.deepEqual((await s.next(4)).result, { prompts: [] });
// Dotted legacy name still callable (alias).
s.send({ jsonrpc: '2.0', id: 5, method: 'tools/call', params: { name: 'ruview.onboard', arguments: {} } });
const call = await s.next(5);
assert.equal(call.result.isError, false);
} finally {
s.close();
}
});
test('MCP server answers ping while a long tools/call is in flight (ADR-263 O2)', { skip: !which('python') && !which('python3') ? 'python not on PATH' : false }, async () => {
// Fake RuView repo whose verify.py sleeps 3 s then passes.
const repo = mkdtempSync(join(tmpdir(), 'ruview-mcp-e2e-'));
const proofDir = join(repo, 'archive', 'v1', 'data', 'proof');
mkdirSync(proofDir, { recursive: true });
writeFileSync(join(proofDir, 'verify.py'), 'import time\ntime.sleep(3)\nprint("VERDICT: PASS")\n');
const s = startServer();
try {
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
await s.next(1);
const verifyDone = s.next(10);
s.send({ jsonrpc: '2.0', id: 10, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
// Give the server a beat to start the child, then ping.
await new Promise((r) => setTimeout(r, 300));
const t0 = Date.now();
const pinged = s.next(11);
s.send({ jsonrpc: '2.0', id: 11, method: 'ping' });
await pinged;
const pingMs = Date.now() - t0;
assert.ok(pingMs < 1000, `ping took ${pingMs} ms while verify was in flight — server is blocking`);
const verify = await verifyDone;
const payload = JSON.parse(verify.result.content[0].text);
assert.equal(payload.verdict, 'PASS');
} finally {
s.close();
rmSync(repo, { recursive: true, force: true });
}
});
test('tools/call executions are serialized — two slow calls run sequentially', { skip: !which('python') && !which('python3') ? 'python not on PATH' : false }, async () => {
// Two verify.py that each sleep 0.8 s. Serialized ⇒ ~1.6 s+; concurrent ⇒ ~0.8 s.
const repo = mkdtempSync(join(tmpdir(), 'ruview-mcp-serial-'));
const proofDir = join(repo, 'archive', 'v1', 'data', 'proof');
mkdirSync(proofDir, { recursive: true });
writeFileSync(join(proofDir, 'verify.py'), 'import time\ntime.sleep(0.8)\nprint("VERDICT: PASS")\n');
const s = startServer();
try {
s.send({ jsonrpc: '2.0', id: 1, method: 'initialize', params: {} });
await s.next(1);
const t0 = Date.now();
const a = s.next(20);
const b = s.next(21);
s.send({ jsonrpc: '2.0', id: 20, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
s.send({ jsonrpc: '2.0', id: 21, method: 'tools/call', params: { name: 'ruview_verify', arguments: { repo } } });
const [ra, rb] = await Promise.all([a, b]);
const elapsed = Date.now() - t0;
assert.equal(JSON.parse(ra.result.content[0].text).verdict, 'PASS');
assert.equal(JSON.parse(rb.result.content[0].text).verdict, 'PASS');
assert.ok(elapsed > 1400, `two 0.8 s tool calls finished in ${elapsed} ms — they overlapped instead of serializing`);
} finally {
s.close();
rmSync(repo, { recursive: true, force: true });
}
});
test('stdin close flushes an in-flight tools/call response before exit', async () => {
const child = spawn(process.execPath, [CLI, 'mcp', 'start'], { stdio: ['pipe', 'pipe', 'pipe'] });
let out = '';
child.stdout.on('data', (d) => { out += d; });
const exited = new Promise((res) => child.on('exit', res));
// Write a tools/call then immediately close stdin. The old fire-and-forget
// dispatch raced rl 'close' → process.exit and could drop this response.
child.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 42, method: 'tools/call', params: { name: 'ruview_onboard', arguments: {} } }) + '\n');
child.stdin.end();
await exited;
const msgs = out.trim().split('\n').filter(Boolean).map((l) => JSON.parse(l));
const resp = msgs.find((m) => m.id === 42);
assert.ok(resp, 'the in-flight tools/call response must be flushed to stdout before exit');
assert.equal(resp.result.isError, false);
});