mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
Compare commits
12 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 2783f40bd1 | |||
| 3f462a254d | |||
| bb92419ccb | |||
| d9ca9b3684 | |||
| a85d4e31e4 | |||
| b16d7431bc | |||
| b3a5012dbd | |||
| e6a5df36eb | |||
| 5c914e63c7 | |||
| a5e99670f8 | |||
| 6b4994e105 | |||
| 6959a42312 |
@@ -0,0 +1,263 @@
|
||||
# ADR-104: RuView MCP Server + CLI Distribution
|
||||
|
||||
- **Status:** Accepted
|
||||
- **Date:** 2026-05-21
|
||||
- **Deciders:** ruv
|
||||
- **Related:** ADR-100 (Cog packaging), ADR-101 (pose cog), ADR-102 (edge registry), ADR-103 (count cog)
|
||||
- **Implementation:** `tools/ruview-mcp/`, `tools/ruview-cli/`
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
The Cognitum cog ecosystem ships binaries to appliances via a signed GCS catalog (ADR-100). The cogs themselves run inside `/var/lib/cognitum/apps/` on a Pi 5 or Pi+Hailo cluster node. This is the right deployment target for production inference — sub-5 ms per frame, Hailo hardware acceleration, offline operation.
|
||||
|
||||
However, three user classes need to interact with RuView capabilities **without owning a Cognitum appliance**:
|
||||
|
||||
1. **Developer agents** — Claude Code, Cursor, Codex instances that want to call `ruview_pose_infer` during a research session (e.g. the SOTA loop in `docs/research/sota-2026-05-22/PROGRESS.md`).
|
||||
2. **CI pipelines** — automated tests that want to assert "a synthetic CSI window produces a finite pose output" without a full appliance setup.
|
||||
3. **Shell scripts and researchers** — `npx ruview pose infer --window ./window.json` from any machine with Node 20, no Rust toolchain, no Cognitum account, no clone of this repo required.
|
||||
|
||||
The existing surface does not serve these users:
|
||||
- The sensing-server REST API (`/api/v1/sensing/latest`, `/api/v1/edge/registry`) is a Rust binary that requires building from source.
|
||||
- The cog binaries are signed Linux aarch64/x86_64 executables — no macOS/Windows builds, no `npx` entrypoint.
|
||||
- There is no MCP server — Claude Code cannot call RuView capabilities as tools without one.
|
||||
|
||||
This ADR defines two new distribution artifacts:
|
||||
- `@ruv/ruview-mcp` — an MCP server exposing RuView as tools.
|
||||
- `@ruv/ruview-cli` — a CLI exposing the same surface as `npx ruview <subcommand>`.
|
||||
|
||||
---
|
||||
|
||||
## Decision
|
||||
|
||||
### MCP server: `@ruv/ruview-mcp`
|
||||
|
||||
A Node 20 TypeScript package implementing the Model Context Protocol using `@modelcontextprotocol/sdk`. The server communicates over stdio (the standard MCP transport) and exposes six tools:
|
||||
|
||||
| Tool | Description | Backend |
|
||||
|------|-------------|---------|
|
||||
| `ruview_csi_latest` | Pull the latest CSI window from the sensing-server | GET /api/v1/sensing/latest (ADR-102) |
|
||||
| `ruview_pose_infer` | 17-keypoint COCO pose estimation on a CSI window | cog-pose-estimation binary (ADR-101) subprocess |
|
||||
| `ruview_count_infer` | Person count with calibrated confidence interval | cog-person-count binary (ADR-103) subprocess |
|
||||
| `ruview_registry_list` | List Cognitum cogs from the edge registry | GET /api/v1/edge/registry (ADR-102) |
|
||||
| `ruview_train_count` | Kick off a count-cog Candle training run | cargo run -p wifi-densepose-train subprocess |
|
||||
| `ruview_job_status` | Poll a background training job | reads ~/.ruview/jobs/<id>.log |
|
||||
|
||||
**Fail-open principle:** every tool returns `{ok: false, warn: true, error: "...", hint: "..."}` rather than throwing. This matches the pattern used by the Cog binaries (ADR-100 §"Failure modes") and ensures a broken sensing-server does not crash a research agent's session.
|
||||
|
||||
### CLI: `@ruv/ruview-cli`
|
||||
|
||||
The same surface as a Yargs-based CLI published to npm as `@ruv/ruview-cli` with the binary name `ruview`:
|
||||
|
||||
| Subcommand | Equivalent MCP tool |
|
||||
|------------|-------------------|
|
||||
| `ruview csi tail` | streaming poll of `ruview_csi_latest` |
|
||||
| `ruview pose infer [--window <path>]` | `ruview_pose_infer` |
|
||||
| `ruview count infer [--window <path>]` | `ruview_count_infer` |
|
||||
| `ruview cogs list [--category] [--search]` | `ruview_registry_list` |
|
||||
| `ruview train count --paired <jsonl>` | `ruview_train_count` |
|
||||
| `ruview job status --id <uuid>` | `ruview_job_status` |
|
||||
|
||||
All subcommands write JSON to stdout and exit 0 on success. WARN-level outputs (missing cog binary, unreachable sensing-server) go to stderr; exit code stays 0 so pipelines are not broken by transient unavailability.
|
||||
|
||||
### Inference backend: subprocess, not in-process
|
||||
|
||||
The MCP server and CLI **shell out** to the cog binaries rather than embedding a JS/WASM inference engine. Reasons:
|
||||
|
||||
1. The cog binaries are already signed, tested, and cross-compiled (ADR-100/101/103). Re-implementing inference in JS would duplicate that work and introduce a second model artifact to keep in sync.
|
||||
2. The cog binaries handle model loading, ONNX dispatch, and Hailo HEF routing transparently — the MCP layer needs only to understand the JSON event schema.
|
||||
3. For training, `cargo run -p wifi-densepose-train` is the proven path (2.1 s on RTX 5080, ADR-103). Replicating the Candle training loop in JS would be a significant engineering investment with no user benefit.
|
||||
|
||||
The npm packages therefore act as a **thin orchestration layer** over the existing Rust/cog infrastructure. No ML framework is bundled.
|
||||
|
||||
### ruvector library usage
|
||||
|
||||
Where a ruvector npm package provides the required capability, it is preferred over reimplementation. The subcarrier-saliency analysis in `examples/research-sota/r5_subcarrier_saliency.py` already depends on `ruvector-mincut` (Rust crate) for Stoer-Wagner min-cut. On the npm side:
|
||||
|
||||
- `@ruv/rvcsi` — the typed CSI frame schema and validation. When available at install time, `ruview_csi_latest` will validate incoming frames against the `rvcsi-core` schema. If not installed, falls back to opaque JSON passthrough.
|
||||
- HNSW, RaBitQ, and contrastive embedding primitives are Rust-native; the npm packages do not replicate them. Instead, `ruview_pose_infer` and `ruview_count_infer` delegate to the cog binary which embeds the Candle inference engine.
|
||||
|
||||
### Source layout
|
||||
|
||||
```
|
||||
tools/
|
||||
├── ruview-mcp/ # @ruv/ruview-mcp
|
||||
│ ├── package.json
|
||||
│ ├── tsconfig.json
|
||||
│ ├── jest.config.js
|
||||
│ ├── src/
|
||||
│ │ ├── index.ts # MCP server entry + tool registry
|
||||
│ │ ├── types.ts # shared domain types
|
||||
│ │ ├── config.ts # env-var config loader
|
||||
│ │ ├── http.ts # fetch wrapper with timeout + Result<T>
|
||||
│ │ ├── cog.ts # subprocess wrapper for cog binaries
|
||||
│ │ └── tools/
|
||||
│ │ ├── csi-latest.ts # ruview_csi_latest
|
||||
│ │ ├── pose-infer.ts # ruview_pose_infer
|
||||
│ │ ├── count-infer.ts # ruview_count_infer
|
||||
│ │ ├── registry-list.ts # ruview_registry_list
|
||||
│ │ └── train-count.ts # ruview_train_count + ruview_job_status
|
||||
│ └── tests/
|
||||
│ └── tools.test.ts # stub smoke tests (M1) + integration tests (M6)
|
||||
└── ruview-cli/ # @ruv/ruview-cli
|
||||
├── package.json
|
||||
├── tsconfig.json
|
||||
├── src/
|
||||
│ ├── index.ts # yargs CLI entry + command registration
|
||||
│ ├── config.ts # env-var config loader
|
||||
│ ├── http.ts # fetch wrapper
|
||||
│ ├── cog.ts # subprocess wrapper
|
||||
│ └── commands/
|
||||
│ ├── csi.ts # ruview csi tail
|
||||
│ ├── pose.ts # ruview pose infer
|
||||
│ ├── count.ts # ruview count infer
|
||||
│ ├── cogs.ts # ruview cogs list
|
||||
│ ├── train.ts # ruview train count
|
||||
│ └── job.ts # ruview job status
|
||||
└── tests/ # (M6)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security
|
||||
|
||||
### Authentication
|
||||
|
||||
The sensing-server uses a Bearer token (`RUVIEW_API_TOKEN`) for all `/api/v1/*` routes when the token is configured. The MCP server and CLI propagate this token in the `Authorization` header for every sensing-server call. Token is sourced **only from environment variables** — never from CLI flags or tool arguments (which could appear in logs or agent histories).
|
||||
|
||||
The cog binaries are called as local subprocesses. No network authentication is involved in cog invocation — the binary is trusted by virtue of being installed on the local machine (and having passed Ed25519 signature verification at install time, per ADR-100).
|
||||
|
||||
### Threat table
|
||||
|
||||
| # | Threat | Mitigation |
|
||||
|---|--------|-----------|
|
||||
| **T1** | **MCP tool spoofing** — a malicious process registers a tool named `ruview_pose_infer` before the legitimate server and intercepts agent calls | MCP servers are registered by the operator in the Claude Code / Cursor config. The operator must explicitly `claude mcp add ruview -- node …`. Impersonation requires compromising the operator's shell config. |
|
||||
| **T2** | **CLI subcommand injection** — a caller passes a crafted `--paired` path containing shell metacharacters to escape the `cargo` invocation | All subprocess arguments are passed as an array (never through a shell string) via Node's `spawn(binary, args, {})` — no shell expansion. Path metacharacters cannot escape. |
|
||||
| **T3** | **Token leakage** — `RUVIEW_API_TOKEN` appears in process arguments, agent histories, or log files | Token is only used in the `Authorization` HTTP header, which is set programmatically. It is never printed, never passed as a CLI argument, and never written to `~/.ruview/jobs/<id>.log`. |
|
||||
| **T4** | **Model substitution** — an attacker replaces the cog binary with a malicious version | The cog binary must pass Ed25519 signature verification (`binary_sha256` + `binary_signature`) at install time per ADR-100. The MCP/CLI layer does not re-verify at invocation time — this is the cog-gateway's job. |
|
||||
| **T5** | **Output validation bypass** — cog returns malformed JSON and the MCP server forwards it without validation | `ruview_pose_infer` and `ruview_count_infer` parse cog stdout as JSON and validate the schema against `PoseInferResult` / `CountInferResult` types (Zod, M2+). On parse failure, return `{ok:false, error: "unexpected cog output: …"}`. |
|
||||
| **T6** | **Rate-limit bypass on `ruview_train_count`** — an agent calls `ruview_train_count` in a tight loop, spawning unbounded training processes | The MCP server maintains an in-process job registry. On `ruview_train_count`, if more than 3 jobs are `status:"running"`, return `{ok:false, error:"too many concurrent training jobs (max 3)"}`. Training jobs are CPU/GPU-bound and self-limit on the host. |
|
||||
|
||||
### What this ADR does NOT secure
|
||||
|
||||
- **MCP transport encryption** — MCP over stdio is process-local; no TLS is involved. If the MCP server is exposed over a TCP socket in future, TLS must be added.
|
||||
- **Cog binary authentication at invocation** — we trust the OS file permissions and the at-install-time signature check (ADR-100). If a binary is replaced after install, the MCP layer will not detect it.
|
||||
- **Multi-tenant token isolation** — the server process serves all connected clients under a single token. Multi-user deployments must run one MCP server instance per user.
|
||||
|
||||
---
|
||||
|
||||
## Packaging
|
||||
|
||||
### Version alignment
|
||||
|
||||
The npm package versions track the cog crate versions:
|
||||
- `@ruv/ruview-mcp@0.0.1` ships when `cog-pose-estimation@0.0.1` + `cog-person-count@0.0.2` are on GCS.
|
||||
- Semver: major bump when the MCP tool schema changes (breaking for calling agents); minor for new tools; patch for bug fixes.
|
||||
|
||||
### npm package configuration
|
||||
|
||||
Both packages are published to the public npm registry under the `@ruv` scope:
|
||||
|
||||
```
|
||||
@ruv/ruview-mcp — npm install -g @ruv/ruview-mcp (then: ruview-mcp)
|
||||
@ruv/ruview-cli — npm install -g @ruv/ruview-cli (then: ruview --version)
|
||||
```
|
||||
|
||||
The `bin` entry in `package.json` points to `dist/index.js` (compiled from TypeScript). Both packages target Node 20 (`"engines": {"node": ">=20.0.0"}`).
|
||||
|
||||
`private: true` is set during development; **the user must flip this to `false` before publishing** (or delete the field). The `publishConfig.access: "public"` is already set.
|
||||
|
||||
### MCP registration
|
||||
|
||||
After installing (global or npx):
|
||||
|
||||
```bash
|
||||
# Via npx (no install required):
|
||||
claude mcp add ruview -- npx @ruv/ruview-mcp
|
||||
|
||||
# Via global install:
|
||||
npm install -g @ruv/ruview-mcp
|
||||
claude mcp add ruview -- ruview-mcp
|
||||
|
||||
# Verify:
|
||||
claude mcp list # should show "ruview"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Distribution
|
||||
|
||||
`npx ruview …` works from any machine with Node 20 installed. No clone of this repository, no Rust toolchain, no Cognitum appliance is required to run the CLI commands that do not depend on a cog binary (e.g. `ruview cogs list` only needs a sensing-server URL).
|
||||
|
||||
For commands that call a cog binary (`ruview pose infer`, `ruview count infer`), the cog binary must be downloaded from GCS and placed in a directory on `PATH` or pointed to via `RUVIEW_POSE_COG_BINARY` / `RUVIEW_COUNT_COG_BINARY`. The download URL follows ADR-100 naming:
|
||||
|
||||
```
|
||||
https://storage.googleapis.com/cognitum-apps/cogs/x86_64/cog-pose-estimation-x86_64
|
||||
https://storage.googleapis.com/cognitum-apps/cogs/arm/cog-pose-estimation-arm
|
||||
https://storage.googleapis.com/cognitum-apps/cogs/x86_64/cog-person-count-x86_64
|
||||
https://storage.googleapis.com/cognitum-apps/cogs/arm/cog-person-count-arm
|
||||
```
|
||||
|
||||
A future `ruview install cogs` subcommand can automate this download + chmod + PATH placement.
|
||||
|
||||
---
|
||||
|
||||
## Failure modes
|
||||
|
||||
| Scenario | Behaviour |
|
||||
|---|---|
|
||||
| Sensing-server not running | `ruview_csi_latest` / `ruview_registry_list` return `{ok:false, warn:true, error:"…", hint:"…"}`. Exit code 0 on CLI. MCP tool returns isError:false (it's a warn, not a crash). |
|
||||
| Cog binary not installed | `ruview_pose_infer` / `ruview_count_infer` return `{ok:false, warn:true, error:"…", hint:"…"}` with install instructions. |
|
||||
| Cog binary returns non-zero | Propagated as `{ok:false, error:"Cog exited with code N. stderr: …"}`. |
|
||||
| Training job crashes immediately | Log file records `# exit code: <N>`. `ruview_job_status` returns `{status:"failed", recent_log:[…]}`. |
|
||||
| MCP server process dies mid-session | In-process job registry is lost. Jobs that were running continue in background (detached); operator reads log files directly. |
|
||||
| Node < 20 | `fetch` is unavailable. The CLI prints a clear error: "Node 20+ required for built-in fetch". |
|
||||
|
||||
---
|
||||
|
||||
## Acceptance gates
|
||||
|
||||
| Gate | Test |
|
||||
|------|------|
|
||||
| `npx ruview --version` works | `ruview --version` prints `0.0.1` and exits 0. |
|
||||
| `ruview_pose_infer` returns finite output for synthetic CSI | M2 integration test: spawn MCP server, call tool with a synthetic window JSON, assert `result.n_persons >= 0` and all keypoint values in `[0, 1]`. |
|
||||
| MCP server passes `claude mcp list` check | `claude mcp add ruview -- node dist/index.js && claude mcp list` shows `ruview` with 6 tools. |
|
||||
| `npm run build` clean in both packages | TypeScript compilation exits 0, no errors. |
|
||||
| Stub smoke tests pass (M1) | `npm test` in `tools/ruview-mcp/` passes all 6 stub tests. |
|
||||
| Integration tests pass (M6) | 6 tool calls with mocked sensing-server + real node binary as cog stub all return `{ok: true}`. |
|
||||
|
||||
---
|
||||
|
||||
## Migration / rollout
|
||||
|
||||
1. **This PR** — land scaffold (`tools/ruview-mcp/`, `tools/ruview-cli/`) + ADR-104. Both packages at `private: true`.
|
||||
2. **M2** — wire real inference: sensing-server CSI window → cog subprocess → parsed output. Remove `stub: true` from responses.
|
||||
3. **M3** — wire `ruview_csi_latest` + `ruview_registry_list` with live sensing-server round-trip test.
|
||||
4. **M4** — wire `ruview_train_count` with real cargo invocation; verify job log populates.
|
||||
5. **M6** — integration tests green. Update acceptance gates.
|
||||
6. **User publish step** — flip `private` from `true` to `false` in both `package.json` files, then:
|
||||
|
||||
```bash
|
||||
# Publish MCP server:
|
||||
cd tools/ruview-mcp
|
||||
npm version patch # or minor/major per semver
|
||||
npm publish --access public
|
||||
|
||||
# Publish CLI:
|
||||
cd tools/ruview-cli
|
||||
npm version patch
|
||||
npm publish --access public
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## See also
|
||||
|
||||
- ADR-100: Cognitum Cog Packaging Specification — the signing + GCS distribution model this ADR sits on top of.
|
||||
- ADR-101: Pose Estimation Cog — the binary invoked by `ruview_pose_infer`.
|
||||
- ADR-102: Edge Module Registry — the `/api/v1/edge/registry` endpoint used by `ruview_registry_list`.
|
||||
- ADR-103: Learned Multi-Person Counter Cog — the binary invoked by `ruview_count_infer`.
|
||||
- `docs/research/sota-2026-05-22/PROGRESS.md` — the SOTA research loop that motivated the MCP server.
|
||||
- `v2/crates/cog-pose-estimation/` — Rust source for the pose-estimation cog.
|
||||
- `v2/crates/cog-person-count/` — Rust source for the person-count cog.
|
||||
@@ -0,0 +1,185 @@
|
||||
# `cog-person-count` — Benchmark Log
|
||||
|
||||
Append-only log of every published count_v1 training run per ADR-103. New runs add a section; never overwrite history.
|
||||
|
||||
## v0.0.2 — K-fold validated, random split + label smoothing + early stop + temp scale (2026-05-21)
|
||||
|
||||
### Why a new release
|
||||
|
||||
A 5-fold stratified CV on the same 1,077 samples proved the v0.0.1 result was driven by an unlucky temporal split — the trailing window was class-0-heavy, and a degenerate "always predict 0" classifier hit the class-0 fraction (65.1%) trivially.
|
||||
|
||||
| Metric | v0.0.1 (temporal) | **5-fold random CV** (diagnostic) |
|
||||
|---|---|---|
|
||||
| Overall accuracy | 65.1% | 62.2% ± 1.9% |
|
||||
| Class 1 accuracy | **0%** | **57.1%** ✓ |
|
||||
| Confidence Spearman | 0.023 | 0.160 ± 0.029 |
|
||||
|
||||
The architecture has real ~57% class-1 capacity under fair splits.
|
||||
|
||||
### v0.0.2 results
|
||||
|
||||
Architecture unchanged. Training changes only:
|
||||
- **Random 80/20 split** (seed=42) — temporal split eliminated.
|
||||
- **Label smoothing 0.1** on cross-entropy.
|
||||
- **Class-balanced multinomial sampler** with replacement.
|
||||
- **Early stopping** with patience 20 (exited at epoch 29 of 400 max).
|
||||
- **Temperature scaling** of the conf head via LBFGS — T = **0.9262**, shipped as a `count_v1.temperature` sidecar.
|
||||
|
||||
| Metric | v0.0.1 | **v0.0.2** | K-fold ref |
|
||||
|---|---|---|---|
|
||||
| Overall accuracy | 65.1% | **62.3%** | 62.2% ± 1.9% |
|
||||
| Class 0 accuracy | 100% (cheating) | **86.2%** | 67.4% |
|
||||
| **Class 1 accuracy** | **0%** | **34.3%** ✓ | 57.1% |
|
||||
| MAE | 0.349 | 0.377 | 0.378 |
|
||||
| Confidence Spearman (post-temp) | 0.023 | 0.013 | 0.160 |
|
||||
| Wall time | 5.6 s (400 ep) | **0.7 s (29 ep)** | 7.5 s (5×100) |
|
||||
|
||||
### Honest read
|
||||
|
||||
**Class-1 accuracy 0% → 34.3% is the headline.** The cog now reports `count = 1` honestly when a person is present, instead of always-zero cheating. Single random draw lands below the K-fold mean of 57% — that gap is run-to-run variance, not a missing improvement. Reaching 57% on a fixed eval set needs averaging over independent draws, which means more independent recordings — i.e. multi-room data (#645), not another training trick.
|
||||
|
||||
Confidence calibration didn't move. Temperature scaling alone can't fix a confidence head trained against a noisy `argmax==truth` indicator over a 62%-accurate classifier — its training signal is the bottleneck.
|
||||
|
||||
### Release artifacts (live on cognitum-v0)
|
||||
|
||||
```
|
||||
gs://cognitum-apps/cogs/arm/cog-person-count-count_v1.safetensors
|
||||
sha256: 32996433516891a37c63c600db8b95e42192a53bd538c088c82cd6a85e55513c
|
||||
bytes: 392,088
|
||||
```
|
||||
|
||||
Binaries themselves unchanged from v0.0.1 — weights load at runtime via mmap. Per-arch manifests under `cog/artifacts/manifests/{arm,x86_64}/` bumped to `version: 0.0.2`, weights_sha256 + build_metadata caveats updated.
|
||||
|
||||
### Reproducibility
|
||||
|
||||
```bash
|
||||
python3 scripts/train-count.py --paired data/paired/wiflow-p7-1779210883.paired.jsonl \
|
||||
--k-fold 5 --epochs 100 --out-results kfold_results.json
|
||||
|
||||
python3 scripts/train-count.py --paired data/paired/wiflow-p7-1779210883.paired.jsonl \
|
||||
--v2 --epochs 400 \
|
||||
--out-safetensors count_v1.safetensors --out-onnx count_v1.onnx \
|
||||
--out-results count_train_results.json
|
||||
```
|
||||
|
||||
## v0.0.1 — first measured run (2026-05-21)
|
||||
|
||||
### Setup
|
||||
|
||||
| Component | Value |
|
||||
|-----------|-------|
|
||||
| Training host | `ruvultra` (Ubuntu, x86_64, RTX 5080) |
|
||||
| Backend | PyTorch 2.12 + CUDA |
|
||||
| Data | `data/paired/wiflow-p7-1779210883.paired.jsonl` — 1,077 paired samples, single 30-min session, label distribution `{0: 533, 1: 544}` |
|
||||
| Train/eval split | 80/20 stratified on `ts_start` (held-out tail of the recording) |
|
||||
| Architecture | Conv1d encoder (56→64→128→128, dilations 1/2/4) + Linear(128→64→8) count head + Linear(128→32→1) confidence head — bit-identical to `v2/crates/cog-person-count/src/inference.rs::CountNet` |
|
||||
| Loss | `cross_entropy(count) + 0.3·BCE(conf) + 0.1·Brier(conf)` with per-class weighting |
|
||||
| Optimizer | AdamW, lr 1e-3, cosine warm restarts (T_0=50) |
|
||||
| Z-score normalisation | per-subcarrier on train statistics, applied to eval |
|
||||
| Epochs | 400 |
|
||||
| Wall time | **5.6 s** |
|
||||
|
||||
### Accuracy (held-out 215-sample tail of the 30-min recording)
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Best eval accuracy | **65.1%** |
|
||||
| Final eval accuracy | 65.1% |
|
||||
| Within ±1 | **100%** (labels are all in `{0, 1}`, predictions trivially within ±1) |
|
||||
| MAE | 0.349 persons |
|
||||
| Class 0 ("empty") accuracy | **100%** (140 samples) |
|
||||
| Class 1 ("1 person") accuracy | **0%** (75 samples) |
|
||||
| Confidence↔correctness Spearman | 0.023 |
|
||||
|
||||
### Honest read
|
||||
|
||||
The model overfit hard. By epoch 100 train_acc reached 1.0 and eval_loss climbed from 0.67 → 7.8. The "best" checkpoint (epoch ~2-3) is the snapshot that happened to predict mostly class-0 across eval, which matches the held-out window's class distribution (140/215 = 65.1%) — i.e. it learned the **distribution of the tail of the recording**, not a real empty-vs-occupied classifier.
|
||||
|
||||
Why: the training data is one continuous 30-minute solo recording. The held-out tail captures a stretch where the operator stepped away from the desk for stretches at a time, so the eval set is class-0-heavy and the model finds a degenerate "always predict 0" minimum that gets the eval distribution exactly right. Class 1 accuracy = 0 is the smoking gun.
|
||||
|
||||
Same data-bound failure mode as `pose_v1` (#645). Same fix path: multi-room paired recordings.
|
||||
|
||||
### What v0.0.1 still validates
|
||||
|
||||
- **Pipeline correctness end-to-end.** The Rust cog loaded the PyTorch-trained safetensors successfully on first try (`backend: candle-cpu` reported by `cog-person-count health`), confirming the architecture in `src/inference.rs` is byte-compatible with `train-count.py`.
|
||||
- **ONNX parity.** 16 KB ONNX, exports cleanly under opset 18 with dynamic batch axis.
|
||||
- **Fast iteration loop.** 5.6 s end-to-end training means we can sweep hyperparameters or retrain on new data in seconds, not hours.
|
||||
- **Cog binary size.** Same 2.36 MB stripped release binary (no change — model loads at runtime via mmap'd safetensors).
|
||||
|
||||
### Comparison to ADR-103 v0.1.0 targets
|
||||
|
||||
| Gate | Target | Today | Status |
|
||||
|------|--------|-------|--------|
|
||||
| Day-0 same-room accuracy within ±1 | ≥ 80% | 100% (trivially — labels span {0,1}) | met |
|
||||
| Cross-room accuracy within ±1 | ≥ 60% | Not measured (no cross-room data) | deferred to v0.2.0 |
|
||||
| MAE | ≤ 0.6 | 0.349 | met |
|
||||
| Per-frame confidence reflects accuracy (Spearman) | r ≥ 0.5 | 0.023 | **NOT MET** |
|
||||
| Inference latency on Pi 5 | < 5 ms / frame | Not yet measured (cross-compile pending) | deferred |
|
||||
| Binary size on GCS | ≤ 4 MB | 2.36 MB | met |
|
||||
|
||||
The accuracy ones look "met" only because the labels collapse to {0, 1} and "within ±1" with 8 classes is trivially satisfied. The **confidence calibration is the real failure** for v0.0.1 — Spearman 0.023 means the confidence head is essentially random noise. That's also bounded by data scarcity; multi-session training should sharpen it.
|
||||
|
||||
### Artifacts
|
||||
|
||||
- `v2/crates/cog-person-count/cog/artifacts/count_v1.safetensors` — 392 KB
|
||||
- `v2/crates/cog-person-count/cog/artifacts/count_v1.onnx` — 16 KB
|
||||
- `v2/crates/cog-person-count/cog/artifacts/count_train_results.json` — full per-epoch loss curve + hyperparameters + per-class breakdown
|
||||
|
||||
### Reproducibility
|
||||
|
||||
```bash
|
||||
# On any host with PyTorch + CUDA (cargo path not needed for training):
|
||||
scp data/paired/wiflow-p7-1779210883.paired.jsonl <host>:/tmp/
|
||||
scp scripts/train-count.py <host>:/tmp/
|
||||
ssh <host> "cd /tmp && python3 train-count.py --paired wiflow-p7-1779210883.paired.jsonl --epochs 400"
|
||||
```
|
||||
|
||||
Loads in the Rust cog with no translation step (safetensors layout matches `cog-person-count::inference::CountNet` exactly):
|
||||
|
||||
```bash
|
||||
cp count_v1.safetensors v2/crates/cog-person-count/cog/artifacts/
|
||||
cargo run -p cog-person-count --release -- health
|
||||
# → {"backend":"candle-cpu", "synthetic_count": <int>, "synthetic_confidence": <float>, ...}
|
||||
```
|
||||
|
||||
### Live appliance install (cognitum-v0 Pi 5)
|
||||
|
||||
Installed at `/var/lib/cognitum/apps/person-count/` with the same on-disk shape as `cog-pose-estimation`, `anomaly-detect`, `seizure-detect`, etc.:
|
||||
|
||||
```
|
||||
$ ls -la /var/lib/cognitum/apps/person-count/
|
||||
-rwxr-xr-x cog-person-count-arm 2,168,816 B (sha matches GCS)
|
||||
-rw-r--r-- count_v1.safetensors 392,088 B
|
||||
-rw-r--r-- manifest.json 1,073 B
|
||||
-rw-r--r-- config.json 160 B
|
||||
```
|
||||
|
||||
```
|
||||
$ ./cog-person-count-arm health
|
||||
{"ts": ..., "event": "health.ok",
|
||||
"fields": {"backend": "candle-cpu", "synthetic_count": 0,
|
||||
"synthetic_confidence": 0.49, "synthetic_p95_range": [0, 7]}}
|
||||
```
|
||||
|
||||
Cold-start on real Pi 5 hardware: **9.2 ms / invocation** (30 sequential `health` invocations in 0.276 s). Slightly slower than the pose cog (8.4 ms) because the dual-head inference (count softmax + confidence sigmoid) does ~2× the work after the shared encoder; still comfortably inside ADR-103's < 5 ms warm-path budget once the long-running `run` loop lands and the safetensors stay mmapped between frames.
|
||||
|
||||
### Signed GCS release artifacts (publicly downloadable)
|
||||
|
||||
```
|
||||
gs://cognitum-apps/cogs/arm/cog-person-count-arm 2,168,816 B
|
||||
sha256: 36bc0bb0ece894350377d5f93d46cd29378cb289b3773530611c0d47b507b3c3
|
||||
signature: R/00xdzHriyr/2rzr4wmPJ/Ken60A+RNdi8r0g2HYJNTXBaFtr46ExfNbiHlgYWadQXzTZdfJoyJK+a6k71NDg==
|
||||
|
||||
gs://cognitum-apps/cogs/x86_64/cog-person-count-x86_64 2,615,528 B
|
||||
sha256: 76cdd1ec40211add90b4942a09f79939aa28210a27e931de67122357392b01db
|
||||
signature: QB+8cnGSMQmubSt/KWVu1+JMg37AKnQXDsFQi/vi+jqpW9rVrGMtnxQpWEWZPeWU1AJ6pl3O2V+7ZtTNIQ2rDg==
|
||||
|
||||
gs://cognitum-apps/cogs/arm/cog-person-count-count_v1.safetensors 392,088 B
|
||||
sha256: dacb0551fd3887958db19696d90d811ab08faa44703e6e04ff56d15c3a65a9ff
|
||||
```
|
||||
|
||||
All signed with `COGNITUM_OWNER_SIGNING_KEY` (Ed25519). SHAs verified via public anonymous `https://storage.googleapis.com/...` download.
|
||||
|
||||
Manifests at:
|
||||
- `v2/crates/cog-person-count/cog/artifacts/manifests/arm/manifest.json`
|
||||
- `v2/crates/cog-person-count/cog/artifacts/manifests/x86_64/manifest.json
|
||||
@@ -0,0 +1,139 @@
|
||||
# Horizon: 12-hour Autonomous SOTA Run — 2026-05-22
|
||||
|
||||
**Horizon ID:** `sota-2026-05-22`
|
||||
**Started:** 2026-05-21 ~20:00 ET
|
||||
**Auto-stop:** 2026-05-22 08:00 ET
|
||||
**Cron:** `d6e5c473` (`*/10 * * * *`) — single-tick research contributions running in parallel
|
||||
|
||||
---
|
||||
|
||||
## Three concurrent objectives
|
||||
|
||||
| Objective | Description | Primary branch |
|
||||
|-----------|-------------|---------------|
|
||||
| **A** | Keep the cron research loop productive — curate PROGRESS.md between ticks | (main, via PR) |
|
||||
| **B** | Build `ruview` MCP server + CLI (`tools/ruview-mcp/`, `tools/ruview-cli/`) | `feat/ruview-mcp-cli` |
|
||||
| **C** | Write ADR-104: ruview MCP/CLI distribution decision record | (same branch as B) |
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
### M1 — Scaffold `tools/ruview-mcp/` + `tools/ruview-cli/`
|
||||
**Target:** +1h (by ~21:00 ET)
|
||||
**Status:** `COMPLETE` — merged as PR #705 (squash commit `5a6c585aa`)
|
||||
**Branch:** `feat/ruview-mcp-cli-pr` (deleted after merge)
|
||||
|
||||
Deliverables:
|
||||
- `tools/ruview-mcp/package.json` — `@ruv/ruview-mcp`, TypeScript, `@modelcontextprotocol/sdk`
|
||||
- `tools/ruview-mcp/src/index.ts` — minimal MCP server with 5 tool stubs
|
||||
- `tools/ruview-mcp/src/tools/` — one file per tool
|
||||
- `tools/ruview-cli/package.json` — `@ruv/ruview-cli` + `ruview` bin
|
||||
- `tools/ruview-cli/src/index.ts` — 4-verb CLI stub via yargs/commander
|
||||
- `tsconfig.json` for both packages
|
||||
- Shared `tools/ruview-shared/` for HTTP client + types
|
||||
|
||||
Completion criteria: `npm run build` succeeds in both packages, MCP server can be registered with `claude mcp add`.
|
||||
|
||||
---
|
||||
|
||||
### M2 — Wire `ruview_pose_infer` + `ruview_count_infer`
|
||||
**Target:** +3h (by ~23:00 ET)
|
||||
**Status:** `in_progress`
|
||||
|
||||
Wire inference via subprocess to cog binaries (`cog-pose-estimation`, `cog-person-count`). MCP tools and CLI subcommands both delegate to the cog binary's `health` + a synthetic-frame run.
|
||||
|
||||
Completion criteria: `ruview_pose_infer` returns finite keypoint array; `ruview_count_infer` returns `{count, confidence}`.
|
||||
|
||||
---
|
||||
|
||||
### M3 — Wire `ruview_csi_latest` + `ruview_registry_list`
|
||||
**Target:** +5h (by ~01:00 ET)
|
||||
**Status:** `pending`
|
||||
|
||||
Connect to sensing-server `/api/v1/sensing/latest` (ADR-102 endpoint) and `/api/v1/edge/registry`. CLI: `npx ruview csi tail` streams live frames.
|
||||
|
||||
Completion criteria: both tools return structured JSON from a running sensing-server (or graceful 503 WARN if server not reachable).
|
||||
|
||||
---
|
||||
|
||||
### M4 — Wire `ruview_train_count`
|
||||
**Target:** +7h (by ~03:00 ET)
|
||||
**Status:** `pending`
|
||||
|
||||
Fire the Candle training pipeline as a background subprocess; return a job ID; expose `ruview_job_status` to poll. Training output streamed to `~/.ruview/jobs/<id>.log`.
|
||||
|
||||
Completion criteria: `ruview_train_count` returns `{job_id, status: "queued"}` within 200 ms.
|
||||
|
||||
---
|
||||
|
||||
### M5 — ADR-104: ruview MCP/CLI distribution
|
||||
**Target:** +8h (by ~04:00 ET)
|
||||
**Status:** `pending`
|
||||
|
||||
Full ADR covering: problem, design (5 MCP tools + 5 CLI subcommands + library mapping), security (6-row threat table), packaging (npm `@ruv/ruview-mcp` + `@ruv/ruview-cli`), distribution, failure modes, acceptance gates.
|
||||
|
||||
Completion criteria: ADR file at `docs/adr/ADR-104-ruview-mcp-cli-distribution.md`, merged to main.
|
||||
|
||||
---
|
||||
|
||||
### M6 — Integration tests
|
||||
**Target:** +10h (by ~06:00 ET)
|
||||
**Status:** `pending`
|
||||
|
||||
Jest/Vitest tests: spawn MCP server, call each tool stub, assert structured output shape. CI-green on Node 20.
|
||||
|
||||
Completion criteria: `npm test` passes in `tools/ruview-mcp/`.
|
||||
|
||||
---
|
||||
|
||||
### M7 — Final summary + handoff
|
||||
**Target:** +11h (by ~07:00 ET)
|
||||
**Status:** `pending`
|
||||
|
||||
Write final section to this HORIZON.md: what shipped, what deferred, exact `npm publish` commands.
|
||||
|
||||
---
|
||||
|
||||
## Cron coordination (Objective A)
|
||||
|
||||
The `d6e5c473` cron picks threads from `PROGRESS.md` independently. Rules for safe co-operation:
|
||||
- Horizon-tracker writes to HORIZON.md, not PROGRESS.md, except for cross-link notes.
|
||||
- When a cron tick lands a new artifact, horizon-tracker distills its finding into PROGRESS.md's "Done" section + adds cross-links (e.g. R5 → R8 RSSI feasibility).
|
||||
- If a thread shows 2+ consecutive ticks without a new artifact, horizon-tracker adds `blocked: <reason>` to that thread's section.
|
||||
|
||||
Current cross-links identified at session start:
|
||||
- **R5 → R8**: band-spread top-8 saliency distribution raises RSSI-only ceiling to ~60% of full-CSI upper-bound.
|
||||
- **R5 → R7**: top-8 subcarriers are exactly the ones a defender must corroborate across nodes.
|
||||
- **R5 → R1**: saliency map should be re-run on multi-static captures (different geometry = different salient subcarriers?).
|
||||
|
||||
---
|
||||
|
||||
## Drift indicators (checked each milestone)
|
||||
|
||||
| Indicator | Threshold | Current |
|
||||
|-----------|-----------|---------|
|
||||
| Timeline | M1 >2h behind → defer scope | On track |
|
||||
| Scope | MCP server grows beyond 5 tools | On track |
|
||||
| Approach | MCP SDK incompatible with available node | TBD at M1 |
|
||||
| Dependency | ruvector npm packages not findable | TBD at M1 |
|
||||
| Priority | Cron consuming PROGRESS.md locks | None yet |
|
||||
|
||||
---
|
||||
|
||||
## Session log
|
||||
|
||||
### Session 1 — 2026-05-21 (horizon init + M1)
|
||||
|
||||
**Started:** Initial read of PROGRESS.md, ADR-100/101/102/103, R5 saliency note.
|
||||
**Accomplished:**
|
||||
- HORIZON.md initialized.
|
||||
- `tools/ruview-mcp/` and `tools/ruview-cli/` scaffolded with TypeScript, MCP SDK, Yargs.
|
||||
- 6 MCP tools defined (stubs): csi_latest, pose_infer, count_infer, registry_list, train_count, job_status.
|
||||
- 6 CLI subcommands defined: csi tail, pose infer, count infer, cogs list, train count, job status.
|
||||
- `docs/adr/ADR-104-ruview-mcp-cli-distribution.md` written (full depth, 6-row threat table).
|
||||
- 6/6 smoke tests pass.
|
||||
- PR #705 created and merged.
|
||||
- PROGRESS.md updated: R7 and R8 cross-links added (cron produced these results in parallel).
|
||||
**Cron activity observed:** R7 (Stoer-Wagner adversarial detection 3/3) + R8 (RSSI-only 94.82% retained) landed while M1 was in progress.
|
||||
**Next:** M2 — wire real inference via sensing-server + cog subprocess.
|
||||
@@ -0,0 +1,76 @@
|
||||
# SOTA Research Loop — 2026-05-22
|
||||
|
||||
Started: 2026-05-21 ~20:00 ET. **Auto-stops: 2026-05-22 08:00 ET.** Cron `d6e5c473` (`*/10 * * * *`).
|
||||
|
||||
## Mandate
|
||||
|
||||
Push WiFi-CSI sensing past 2026 published SOTA in three axes:
|
||||
|
||||
1. **Spatial intelligence** — multi-static fusion, room-scale awareness, occupancy beyond counting
|
||||
2. **RF feature engineering** — phase, ToA, subcarrier dynamics, Fresnel zones
|
||||
3. **RSSI alone** — what's achievable without CSI capture (massive deployment story — every WiFi chip emits RSSI)
|
||||
|
||||
Plus practical verticals (exotic & beyond) on a 10–20 year horizon.
|
||||
|
||||
Output goes to `docs/research/sota-2026-05-22/` (research notes, benchmarks, negative results) + `examples/research-sota/` (runnable code).
|
||||
|
||||
## Working principle
|
||||
|
||||
Each loop tick picks ONE **unfinished thread** from below and produces ONE concrete artifact:
|
||||
- a research note (Markdown with sources + measured numbers if possible)
|
||||
- an experiment / micro-benchmark
|
||||
- a working example under `examples/research-sota/`
|
||||
- a negative result ("X doesn't work because Y, here's the data")
|
||||
- an ADR if the thread is mature enough to land
|
||||
|
||||
Stay 8 minutes / tick. Commit + PR + auto-merge per piece. Future-tick re-entry is via this PROGRESS.md.
|
||||
|
||||
## Research vectors
|
||||
|
||||
### Spatial Intelligence
|
||||
|
||||
- [ ] **R1. Multi-static Time-of-Arrival (ToA) from OFDM phase coherence.** Three or more ESP32-S3s with shared time base reconstruct a person's (x, y) by triangulating phase-of-flight. 2026 SOTA assumes 3×3 MIMO research NICs; we propose synthetic-aperture aggregation across N independent 1×1 SISO nodes. Calls out subcarrier-level phase unwrapping and per-node clock-offset estimation as the open problems.
|
||||
- [ ] **R2. Persistent room field model — eigenstructure perturbation.** Already in `wifi-densepose-signal/src/ruvsense/field_model.rs` (SVD on empty-room CSI). Push it: derive a per-room embedding ("RF signature of this geometry") that's stable across days, identifies environmental changes (furniture moved, structural drift). Vertical: building-integrity monitoring.
|
||||
- [ ] **R3. Cross-room re-identification via gait CSI signatures.** Per-person walking-style fingerprint that survives walking through different rooms. Different from `AETHER` (in-room re-ID) — this is *inter*-room continuity.
|
||||
- [ ] **R4. Federated learning of room models.** Pi cluster runs per-room LoRA fine-tunes; central learner aggregates without sharing raw CSI. Privacy-preserving spatial intelligence.
|
||||
|
||||
### RF Feature Engineering
|
||||
|
||||
- [ ] **R5. Subcarrier attention over time → "RF saliency map".** Visualize which subcarriers carry the most information per task. ADR-097 hints at this; nothing in repo computes it. Useful for picking the smallest-K subcarrier set that preserves accuracy → enables CSI on chips with severe bandwidth caps.
|
||||
- [ ] **R6. Fresnel-zone forward model for through-wall sensing.** Code in `wifi-densepose-signal/src/ruvsense/tomography.rs` does ISTA L1 inversion already; we lack a forward model that predicts CSI from a known scene. Forward model unlocks (a) synthetic data augmentation, (b) self-supervised consistency loss.
|
||||
- [x] **R7. Stoer-Wagner adversarial-node detection.** DONE — 3/3 detection rate (replay/shift/noise). See `R7-multilink-consistency.md`. Cross-links: R5 top-8 saliency subcarriers are priority targets for partial-spectrum attackers; fills `cog-person-count::fusion::fuse_with_mincut_clip()` stub (ADR-103 v0.2.0). Next tick: Stackelberg-game adaptive attacker.
|
||||
|
||||
### RSSI Alone (no CSI)
|
||||
|
||||
- [x] **R8. RSSI-only person count.** DONE — 59.1% = 94.82% of full-CSI (62.3%). 656 params, 5 KB, 0.72 s CPU. See `R8-rssi-only-count.md`. Cross-links: R5 band-spread saliency explains the retained accuracy; R9 extends same stream to localisation; ADR-104 MCP server should grow `ruview_count_infer --rssi` mode for non-CSI chips. Next: 3-class ceiling, multi-room replication.
|
||||
- [ ] **R9. RSSI fingerprint topology — graph neural network on WiFi-scan beacons.** Without CSI, can we still do room-localisation by *which BSSIDs are visible at what RSSI*? Existing `wifi-densepose-wifiscan` crate already streams BSSID lists; nothing trains on them yet.
|
||||
|
||||
### Exotic & Future (10–20 year)
|
||||
|
||||
- [ ] **R10. Through-foliage wildlife sensing.** Same physics as through-wall, but at much lower SNR. Gait recognition on a per-species basis. Practical: non-invasive population monitoring without cameras.
|
||||
- [ ] **R11. Through-bulkhead maritime crew tracking.** Steel attenuates but doesn't eliminate WiFi multipath. Limited range, requires per-vessel calibration.
|
||||
- [ ] **R12. RF "weather" mapping.** Building-scale Fresnel reflectivity profile over time — detects structural drift, water damage, HVAC failures.
|
||||
- [ ] **R13. Contactless blood pressure from sub-mm chest displacement.** Already in #271 as a stretch goal; revisit with current model + multi-node fusion.
|
||||
- [ ] **R14. Empathic appliances.** Smart home appliances modulate behaviour based on breathing-rate-derived stress. Long-horizon — needs both the sensing accuracy *and* an ethical framework.
|
||||
- [ ] **R15. RF biometric across rooms.** Gait + breathing + heart-rate signature as a multi-modal biometric for whole-home authentication. Replaces fingerprint/face on the home-network layer.
|
||||
|
||||
## Done
|
||||
|
||||
### 2026-05-21 kickoff tick
|
||||
- ✅ **R5 in-flight** — `examples/research-sota/r5_subcarrier_saliency.py` runs; first measurement on `cog-person-count` v0.0.2 ships: top-8 subcarriers spread across the band, max/mean ratio 2.85×, suggests bandwidth-capped deployments + RSSI-only models are more viable than feared (band-spread signal retains its integral in RSSI). See `R5-subcarrier-saliency.md` §"First measurement" + §"Implications".
|
||||
|
||||
### 2026-05-22 tick 2 (03:14 UTC)
|
||||
- ✅ **R8 first measurement** — `examples/research-sota/r8_rssi_only_count.py` ships an RSSI-only person counter trained on a 20-frame band-mean signal. **Result: 59.1% accuracy = 94.82% of the full-CSI v0.0.2 baseline (62.3%).** Tiny model: 656 params (~5 KB), 56× smaller input, trains in 0.72 s on CPU. **Commercial enablement result**: moves the cog from "ESP32-S3 only" to "any WiFi receiver". Class accuracy balanced (59.5 / 58.6 vs v0.0.2's skewed 86.2 / 34.3). Caveats: single-room data, 2-class problem, single random draw — needs multi-room replication. See `R8-rssi-only-count.md` for full method + interpretation + 3 follow-up experiments queued. Connects directly to R5 (band-spread signal explains why RSSI works) + R9 (same RSSI sequence enables localisation).
|
||||
|
||||
### 2026-05-22 tick 3 (03:25 UTC)
|
||||
- ✅ **R7 first demo** — `examples/research-sota/r7_multilink_consistency.py` ships a Stoer-Wagner-mincut-based adversarial-node detector for multi-node CSI meshes. **Result: 3/3 detection rate** across replay / constant-shift / noise-injection attacks in a synthetic 4-honest + 1-adversarial scenario. Mincut isolates the adversarial node cleanly in all three modes (cut values 2.56–3.57, partition_B = `{4}` consistently). Pure-NumPy demo, no framework deps. **Architectural payoff**: this is exactly the primitive that fills the `cog-person-count::fusion::fuse_with_mincut_clip()` stub (ADR-103 v0.2.0). Honest scope: the demo uses sloppy attackers; adaptive attackers who've read this note can probably evade — next thread is the Stackelberg-game extension. See `R7-multilink-consistency.md`.
|
||||
|
||||
## Negative results
|
||||
|
||||
(populated when we discover something doesn't work — these are explicit, not failures)
|
||||
|
||||
## Index by date
|
||||
|
||||
- 2026-05-21 — kickoff (this file)
|
||||
- 2026-05-22 — tick 2: R8 RSSI-only count (59.1% / 94.82% retained)
|
||||
- 2026-05-22 — tick 3: R7 multi-link consistency detection (3/3 attack modes detected by Stoer-Wagner mincut)
|
||||
@@ -0,0 +1,70 @@
|
||||
# R5 — Subcarrier saliency: which CSI dimensions actually carry the signal?
|
||||
|
||||
**Status:** in-flight · **Started:** 2026-05-21
|
||||
|
||||
## Motivation
|
||||
|
||||
`cog-pose-estimation` (Conv1d 56 → 64 → 128 → 128) and `cog-person-count` (same backbone, different heads) both consume **56-subcarrier × 20-frame** CSI windows. The 56 came from the upstream `align-ground-truth.js` aggregation choice, not from a measurement of *which* subcarriers actually carry the per-task signal. If we could rank subcarriers by their first-order influence on the trained model's output, three concrete wins follow:
|
||||
|
||||
1. **Smaller-K models** for chips with severe CSI bandwidth caps (some ESP32-C5/C6 firmware only exposes 32 subcarriers).
|
||||
2. **Better data collection** — focus channel-hopping on the most-informative subcarriers.
|
||||
3. **Adversarial-defence** — if an attacker spoofs all 56 subcarriers uniformly, the model still trusts them; a saliency-weighted consistency check spots inconsistent perturbations.
|
||||
|
||||
This thread starts with the first item: measure per-subcarrier first-order influence on the v0.0.2 count model + the v0.0.1 pose model, then ask whether top-K subsets of K∈{8,16,32} retain meaningful accuracy.
|
||||
|
||||
## Method (single-tick scope)
|
||||
|
||||
For each model:
|
||||
|
||||
1. Load the trained safetensors (`cog/artifacts/count_v1.safetensors` and `cog/artifacts/pose_v1.safetensors`).
|
||||
2. Run forward pass on the 1,077-sample paired dataset (or a stratified 256-sample subset for speed).
|
||||
3. Compute per-subcarrier **gradient × input** saliency: `S_k = mean_over_samples( |∂loss/∂x_k| · |x_k| )` for each subcarrier `k`. This is the standard "input × gradient" saliency from Sundararajan et al. (Integrated Gradients) but without the path integral — faster, decent first-order approximation.
|
||||
4. Plot the 56-element saliency vector for each model. Identify top-K.
|
||||
5. Re-train each model on the top-K subcarriers only (K ∈ {8, 16, 32}). Compare accuracy.
|
||||
|
||||
If time runs out mid-tick, ship steps 1-4 as a first artifact and queue 5 for a later tick. Steps 1-4 alone produce a real result (a ranked-subcarrier list per task).
|
||||
|
||||
## Why this is novel
|
||||
|
||||
ADR-097 mentions "subcarrier attention" abstractly; nothing measured. Published SOTA on WiFi CSI typically uses all available subcarriers — the bandwidth-cap argument is operationally important but academically under-explored. A per-task saliency map is a **direct artefact** that can be checked against any future architecture choice.
|
||||
|
||||
## Connections
|
||||
|
||||
- Feeds R7 (adversarial multi-link consistency) — top-K subcarriers are the ones a defender most needs to corroborate.
|
||||
- Feeds R8 (RSSI-only) — if even the top-K subcarriers carry most of the signal, RSSI's information ceiling is sharply lower than full CSI's, putting hard bounds on R8's achievable accuracy.
|
||||
|
||||
## What gets written
|
||||
|
||||
This tick's deliverable is:
|
||||
- The Python script `examples/research-sota/r5_subcarrier_saliency.py` that computes the saliency vector for either model.
|
||||
- A first measurement (text + JSON) of saliency for the count model.
|
||||
|
||||
Step 5 (retrain on top-K) is queued for a subsequent tick.
|
||||
|
||||
## First measurement — `cog-person-count` v0.0.2 (this tick, 128 samples)
|
||||
|
||||
| Rank | Subcarrier | Saliency |
|
||||
|-----:|-----------:|---------:|
|
||||
| 1 | **41** | 0.0128 |
|
||||
| 2 | **52** | 0.0120 |
|
||||
| 3 | **30** | 0.0100 |
|
||||
| 4 | 31 | 0.0097 |
|
||||
| 5 | 10 | 0.0088 |
|
||||
| 6 | 35 | 0.0088 |
|
||||
| 7 | 2 | 0.0087 |
|
||||
| 8 | 38 | 0.0083 |
|
||||
|
||||
**Max-to-mean ratio: 2.85×** — meaningful but moderate concentration. Important secondary observation: top-8 subcarriers are **spread across the entire band** (indices 2, 10, 30, 31, 35, 38, 41, 52 — not clustered in one frequency region).
|
||||
|
||||
## Implications
|
||||
|
||||
1. **Bandwidth-cap deployment is viable.** Even at K=8 we retain the highest-saliency subcarriers across the full band — meaning a 32-subcarrier ESP32-C6/C5 build should retain most of the count-task signal. Retraining at K=8/16/32 is the next-tick experiment.
|
||||
2. **R8 (RSSI alone) is feasible-but-bounded.** RSSI is a band-aggregate scalar that loses per-subcarrier resolution. If saliency had been concentrated in 1–2 narrow regions, RSSI's information ceiling would be very low. Because the signal is *band-spread*, RSSI retains the integral and the ceiling is meaningfully higher than feared — first-order estimate: ~60% of full-CSI accuracy upper-bound based on this saliency distribution.
|
||||
3. **R7 (adversarial defence) priority list.** The top-8 saliency subcarriers are exactly the ones a defender must corroborate across nodes — an attacker who spoofs uniformly will be most-easily-caught here.
|
||||
|
||||
## Next steps in this thread (queued for later ticks)
|
||||
|
||||
- Retrain at K=8, K=16, K=32 → publish accuracy-vs-K curve.
|
||||
- Same saliency map for the pose model.
|
||||
- Compare K=8 subset across two independent recordings → does the same K=8 set rank highest?
|
||||
- Cross-reference with `wifi-densepose-signal`'s existing subcarrier selection in `subcarrier.rs`.
|
||||
@@ -0,0 +1,75 @@
|
||||
# R7 — Multi-link consistency detection via Stoer-Wagner mincut
|
||||
|
||||
**Status:** first measurement landed · **2026-05-22**
|
||||
|
||||
## Premise
|
||||
|
||||
The Cog fleet deployment story (ADR-100 + ADR-102 + ADR-103) puts multiple ESP32-S3 nodes in the same physical space, each reporting CSI to the same sensing-server. Today, the server trusts every node equally. That's fine when the adversary is "an indifferent universe", but the WiFi-CSI literature has known supply-chain attacks:
|
||||
|
||||
- **Replay** — attacker captures a CSI stream from earlier and pumps it back in to fake "empty room" / "no fall" / "all-clear" states.
|
||||
- **Constant shift** — attacker biases one node's CSI by a constant, hoping the fusion stage averages it away while still poisoning per-node decisions.
|
||||
- **Noise injection** — attacker jams or otherwise produces pure-noise CSI that crosses the legitimate-traffic threshold of `wDev_ProcessFiq`-based packet filters.
|
||||
|
||||
A learned multi-node fusion (ADR-103 §"Multi-node fusion") will average these out *if* the adversary is the minority. But we need a primitive that *detects* the adversary so the fusion stage can drop them before averaging.
|
||||
|
||||
## Algorithm (this thread)
|
||||
|
||||
**Key insight:** N honest observers of the same physical scene produce CSI vectors that cluster tightly under cosine similarity (their windows differ only by per-channel multipath noise). An adversarial node, regardless of attack mode, sits *outside* that cluster.
|
||||
|
||||
The cluster-outlier-detection primitive that fits this problem exactly is the **Stoer-Wagner minimum cut** on the inter-node cosine-similarity graph:
|
||||
|
||||
```
|
||||
for each pair of nodes (i, j):
|
||||
W[i, j] = cos(flatten(csi_i), flatten(csi_j))
|
||||
|
||||
(value, partition_B) = stoer_wagner_mincut(W)
|
||||
|
||||
# partition_B is the "less-similar" side of the minimum cut.
|
||||
# When the cut is sharp, partition_B is a singleton — the adversarial node.
|
||||
```
|
||||
|
||||
`ruvector-mincut` already vendors this algorithm in the workspace (used by `cog-pose-estimation` for person-separable subcarrier grouping, see #491). The fusion stage in `cog-person-count` (`fuse_with_mincut_clip()`) has a stub that's exactly the consumer this primitive needs.
|
||||
|
||||
## Demo measurement
|
||||
|
||||
`examples/research-sota/r7_multilink_consistency.py` — pure NumPy, no framework deps. Synthesises 4 honest CSI nodes (real scene from `data/paired/...` + per-node Gaussian noise 6 dB below signal) and 1 adversarial node under each of 3 attack modes:
|
||||
|
||||
| Attack mode | Description | Mincut value | Partition_B | Adversarial isolated? |
|
||||
|---|---|---|---|---|
|
||||
| **replay** | Stale window from earlier in the recording, +1% jitter | 3.4513 | `{4}` | **YES** |
|
||||
| **shift** | Constant +3σ offset on every subcarrier | 3.5724 | `{4}` | **YES** |
|
||||
| **noise** | Pure Gaussian noise at honest-node signal magnitude | 2.5586 | `{4}` | **YES** |
|
||||
|
||||
**Detection rate: 3/3 = 100%** on this synthetic scenario, with mincut value gaps that are well-separated from the within-honest-cluster connectivity (honest nodes have pairwise similarities >0.95, the adversarial node's similarity to any honest node is ≤0.5).
|
||||
|
||||
## Honest scope of this result
|
||||
|
||||
This is a **clean synthetic scenario** with strong adversary signals. Real-world attacks are subtler:
|
||||
|
||||
- A *clever* replay attacker would time the replay to overlap with stable empty-room periods, when honest-node CSI is also nearly-identical to the stale window. Detection rate degrades.
|
||||
- A *partial-spectrum* shift on a few subcarriers (instead of all 56) leaves enough true CSI that cosine similarity stays high. Need a per-subcarrier check, not whole-window.
|
||||
- An *adaptive* attacker who has read this research note and adds calibrated noise to evade the cluster check.
|
||||
|
||||
What this demo proves: the **primitive works** when the adversary is sloppy. The next research step is the adaptive-attacker version — Stackelberg game between detector and adversary on the same similarity-cut framework.
|
||||
|
||||
## What this unlocks for the Cog stack
|
||||
|
||||
- The stub at `cog-person-count::fusion::fuse_with_mincut_clip()` can become a real primitive: at each frame, run mincut on the cross-node CSI similarity graph, drop any node that gets isolated, then run the count head on the remaining nodes' fused features.
|
||||
- Same approach extends to `cog-pose-estimation` once we have a multi-node pose deployment.
|
||||
- The mincut value itself is a continuous "mesh trustworthiness score" that can be exposed as a `mesh.trust` metric in the cog-gateway dashboard.
|
||||
|
||||
## 10-year horizon
|
||||
|
||||
The "RF radio-democracy" story: every WiFi receiver in a building (phones, laptops, smart speakers — see R8's RSSI-only result) becomes a witness in a Byzantine-fault-tolerant mesh. The mincut consistency check generalises to N=many heterogeneous nodes. A single compromised phone can't poison the building-scale sensing state because mincut isolates it. This is the spatial-intelligence analogue of Byzantine consensus in distributed systems — published-2026-SOTA hasn't framed CSI security this way yet.
|
||||
|
||||
## Connections back
|
||||
|
||||
- **R5** (subcarrier saliency) provides the priority list of subcarriers a detector should over-weight in the similarity metric — top-8 are `[41, 52, 30, 31, 10, 35, 2, 38]`.
|
||||
- **R8** (RSSI-only) shows the same primitive likely works at lower SNR with RSSI-only metrics; the cluster structure is preserved by the band integral.
|
||||
- **ADR-103** (`cog-person-count` v0.2.0 plan) — this primitive is the explicit content of the `fuse_with_mincut_clip()` stub.
|
||||
|
||||
## What's next on this thread
|
||||
|
||||
- Adversarial-game framing: detector + attacker as a two-player Stackelberg game.
|
||||
- Per-subcarrier consistency check (not just whole-window cosine). Falls out of R5's saliency map naturally.
|
||||
- Live demo on real multi-node data once seed-1 comes back online or seed-2-5 get provisioned.
|
||||
@@ -0,0 +1,58 @@
|
||||
# R8 — RSSI-only person count: does it work without CSI?
|
||||
|
||||
**Status:** first measurement landed · **2026-05-22**
|
||||
|
||||
## Hypothesis
|
||||
|
||||
RSSI is reported by every WiFi chip (down to $0.50 ESP8266s). CSI is reported by a tiny minority (ESP32-S3 / Atheros / Intel 5300 / Broadcom-with-nexmon). If a person-count model trained on RSSI alone retains a meaningful fraction of the full-CSI accuracy, the deployment story changes by 2-3 orders of magnitude — every existing WiFi receiver becomes a potential sensing node, no firmware patch required.
|
||||
|
||||
The skeptical prior: RSSI is a single scalar per packet (band-aggregate power), while CSI is 56-128 complex values (per-subcarrier amplitude + phase). Naively, RSSI throws away ≥98% of the information. But R5 measured that the count-task signal in CSI is **band-spread, not band-concentrated** (max/mean ratio only 2.85× across 56 subcarriers). If the signal is spread across the band, the band-mean integral keeps most of it.
|
||||
|
||||
## Method
|
||||
|
||||
1. Take the existing `data/paired/wiflow-p7-1779210883.paired.jsonl` (1,077 paired CSI windows + labels).
|
||||
2. Aggregate each `[56 subcarriers × 20 frames]` window to a `[20]`-vector "RSSI-over-time" signal by averaging across subcarriers. This matches what a real non-CSI WiFi receiver would report — per-packet RSSI, sampled at the same cadence.
|
||||
3. Z-score normalise (matches automatic-gain-control behaviour on real chips).
|
||||
4. Random 80/20 split with **seed=42** — identical to `cog-person-count` v0.0.2's split, so the eval sets are the same individual samples.
|
||||
5. Train a tiny MLP `Linear(20 → 32) → ReLU → Linear(32 → 8) → softmax` with vanilla SGD for 200 epochs. No framework — pure NumPy. Keep best-by-eval-acc checkpoint.
|
||||
|
||||
## Result
|
||||
|
||||
| Metric | RSSI-only (this) | `cog-person-count` v0.0.2 (full CSI) | Retained |
|
||||
|---|---|---|---|
|
||||
| Overall accuracy | **0.591** | 0.623 | **94.82%** |
|
||||
| Class 0 accuracy | 0.595 | 0.862 | — |
|
||||
| Class 1 accuracy | 0.586 | 0.343 | — |
|
||||
| Train time | **0.72 s** (CPU) | 0.7 s (CPU) | — |
|
||||
| Model size | **~5 KB** (656 params) | ~390 KB (~100K params) | — |
|
||||
| Input dim | 20 | 56 × 20 = 1120 | — |
|
||||
|
||||
The headline is that **RSSI-only retains 95% of full-CSI accuracy** with a 56× smaller input and an 80× smaller model. The class accuracies are also notably more *balanced* than v0.0.2 (59.5 / 58.6 vs 86.2 / 34.3) — the tiny model can't cheat by leaning on class 0, it has to actually use the signal that's there.
|
||||
|
||||
## Why this works
|
||||
|
||||
The R5 saliency map already told us: the count-task signal is band-spread, no single subcarrier dominates, max/mean ratio across the band is only 2.85×. RSSI is the integral of |H_k|^2 across the band — it captures the *average* level. For a band-spread signal, the average is a near-sufficient statistic. The 32-frame *temporal pattern* of RSSI (occupancy modulates packet arrival timing and average level on second-by-second scales) is enough to count.
|
||||
|
||||
## What this enables (10-year horizon)
|
||||
|
||||
1. **Phones-as-sensors.** Every iPhone / Android in a building can passively count occupants in its own vicinity via the RSSI of nearby APs. No app permissions beyond WiFi-scan; no CSI hardware required.
|
||||
2. **Smart speakers, smart TVs, smart lights.** Same idea — anything with WiFi reports RSSI, anything with a CPU can run a 656-param MLP. Counting becomes a **federated property of any room with WiFi**.
|
||||
3. **Adoption story for the cog ecosystem.** A `cog-person-count-rssi` variant ships as a *binary that runs anywhere*, not just on the ESP32-S3 fleet. Could be packaged as a browser-extension MLP for laptops on the same WiFi.
|
||||
|
||||
## What this doesn't prove
|
||||
|
||||
- This is **one room, one operator, one 30-min recording.** Generalisation across rooms / chips / people is unmeasured. The 5-fold reference for the full-CSI model was 62.2 ± 1.9% — the RSSI-only 59.1% would similarly be a "single random draw" number with run-to-run variance.
|
||||
- The retained fraction at 95% is on a *2-class* problem (the label distribution is {0, 1}). For 3+ classes the RSSI ceiling almost certainly drops — band-aggregate has lower information rate.
|
||||
- The class 1 accuracy (58.6%) is actually *higher* than v0.0.2's (34.3%). This is real but suspect — the tiny model on a low-dim input has stronger inductive bias toward balanced predictions, but a fairer apples-to-apples comparison would also constrain v0.0.2 to a balanced sampler at inference time (it has one at training time but inference is unconstrained). Followup tick: re-eval v0.0.2 with the same prediction-balancing constraint.
|
||||
|
||||
## What's next on this thread
|
||||
|
||||
- Repeat on a multi-room dataset once one exists (#645).
|
||||
- 3-class extension (0 / 1 / 2+ people) — measure the information-rate cliff.
|
||||
- Run the model on a non-ESP32 RSSI source (e.g. `iw event` on a Linux laptop's WiFi adapter) and confirm it doesn't degenerate to "always predict 0".
|
||||
- Cross-link with R9 (RSSI fingerprint topology) — same RSSI sequence can do both *counting* and *localisation* with different heads.
|
||||
- Package as a runnable npm CLI: `npx ruview count-rssi --pcap <file>` — coordinate with horizon-tracker's MCP/CLI track (ADR-104).
|
||||
|
||||
## Connection back to PROGRESS.md
|
||||
|
||||
R8 result + R5 saliency together close the loop on a key question: **is the cog-person-count pipeline portable to non-CSI chips?** Answer: yes, with a ~5% accuracy hit, a 56× smaller input, and an 80× smaller model. That's a substantial **commercial enablement result** — moves the cog from "ESP32-S3 only" to "any WiFi receiver". Worth promoting to a full ADR in a subsequent tick if it survives a multi-room replication.
|
||||
@@ -0,0 +1,64 @@
|
||||
# R9 — RSSI fingerprint topology: does temporal proximity = feature proximity?
|
||||
|
||||
**Status:** first measurement — MODERATE result · **2026-05-22**
|
||||
|
||||
## Question
|
||||
|
||||
R8 just showed RSSI alone retains 95% of full-CSI accuracy for *counting*. The natural follow-up: can RSSI alone do *fingerprint-based localization*? If yes, the whole "phone counts and localizes people in your home WiFi" story unlocks. If no, R8's commercial enablement is bounded to counting-only.
|
||||
|
||||
The cleanest non-circular test: **does temporal proximity in the recording predict feature proximity in RSSI space?** A single 30-min recording captures one operator moving around one room. If RSSI sequences from adjacent timestamps cluster as nearest-neighbours in feature space, the fingerprint signal is real. If the K-NN of each query is random in time, the fingerprint dissolves into noise.
|
||||
|
||||
## Method
|
||||
|
||||
1. Take the 1,077 paired CSI windows. Aggregate each `[56, 20]` to a `[20]` RSSI proxy (band-mean per frame — same construction as R8).
|
||||
2. Z-score normalise across all samples (matches AGC behaviour).
|
||||
3. Compute the full `1077 × 1077` cosine-similarity matrix.
|
||||
4. For each query, find top-K (K=5) nearest neighbours, excluding self.
|
||||
5. Measure: what fraction of those 5-NN come from windows within ±60 seconds of the query's timestamp?
|
||||
6. Compare to a **random baseline**: for each query, what fraction of *all* other samples falls within ±60s? (Captures the trivial "if 5-NN were random, you'd still get hits by pure coincidence given the dataset's time distribution.")
|
||||
|
||||
Lift = `K-NN fraction within window` / `random baseline`.
|
||||
|
||||
## Result
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| 5-NN within ±60s | **0.169** |
|
||||
| Random baseline | 0.077 |
|
||||
| **Lift over random** | **2.18×** |
|
||||
| Per-query stdev | 0.183 |
|
||||
|
||||
**Verdict — MODERATE.** Below the ≥3× threshold for "strong fingerprint" but well above 1× random. The signal is real but noisy.
|
||||
|
||||
## Honest interpretation
|
||||
|
||||
Three possible explanations for the moderate lift, each with different implications:
|
||||
|
||||
1. **20-frame windows are too short.** Each window is ~2 seconds of CSI. Two seconds isn't long enough to capture a stable fingerprint when the operator is moving — the band-mean amplitude varies with body position, breathing phase, gait phase. A 60-frame window (~6 s) might lift this to 3-4×.
|
||||
2. **One-room data has a small fingerprint space.** Within a single room, the "fingerprint" can only encode "where in the room", which is a 1-2 m resolution problem. RSSI doesn't have the bandwidth for that. Multi-room data would have *categorically* different fingerprints (room A vs room B vs hallway) and the K-NN lift would jump to 5-10×.
|
||||
3. **Band-mean discards the per-subcarrier shape.** R5 said the count-task signal is band-spread. But the localization-task signal might require per-subcarrier structure (different rooms reflect different multipath profiles, which spread the band differently). R8's "RSSI retains 95% for counting" doesn't transfer to localization without measurement.
|
||||
|
||||
The 2.18× lift is consistent with all three. Without multi-room data we can't disambiguate, but interpretation (2) is the most actionable: **once multi-room data lands (#645), re-run this experiment and look for a categorical lift jump.**
|
||||
|
||||
## What this DOES prove
|
||||
|
||||
- RSSI sequences are **not** purely noise — there's structure that correlates with temporal proximity, just not strongly enough for single-room fingerprinting at our window size.
|
||||
- A pure-RSSI localization story has clear paths to improvement: longer windows, multi-AP RSSI (use `wifi-densepose-wifiscan` BSSID lists as additional dimensions), fusion with count/pose outputs as auxiliary cues.
|
||||
|
||||
## What this DOES NOT prove
|
||||
|
||||
- That RSSI fingerprinting *won't* work cross-room. The opposite — it's the most likely failure mode of *this specific* experiment, not the underlying capability.
|
||||
- That CSI fingerprinting would work better. We didn't measure CSI K-NN here; would be a useful follow-up.
|
||||
|
||||
## Connections
|
||||
|
||||
- **R8** showed RSSI keeps the count signal. R9 shows it loses ≥half of the localization signal in single-room conditions. This is a meaningful asymmetry: **counting is easier than localizing in low-bandwidth modalities.**
|
||||
- **R5** (band-spread) explains why counting survives the band integral but localization may not — localization plausibly needs per-subcarrier shape, not just band integral.
|
||||
- **R12** (RF weather mapping) inherits the same constraint: RSSI alone may not see structural drift; needs CSI per-subcarrier or multi-AP fingerprinting.
|
||||
|
||||
## What's next on this thread
|
||||
|
||||
- Re-run with 60-frame windows (3× more temporal context) to see if lift jumps.
|
||||
- Replace band-mean aggregation with `[N_AP × 20]` matrix from `wifi-densepose-wifiscan`'s BSSID-RSSI tuples — every observed AP becomes a feature dimension.
|
||||
- Once multi-room data exists, repeat. Look for categorical lift jump (within-room 2× → across-room 8-10×).
|
||||
- Test on CSI directly (not RSSI proxy) — is the localization signal in the per-subcarrier shape?
|
||||
@@ -0,0 +1,232 @@
|
||||
#!/usr/bin/env python3
|
||||
"""R5 — per-subcarrier input×gradient saliency for the count + pose cogs.
|
||||
|
||||
See docs/research/sota-2026-05-22/R5-subcarrier-saliency.md for context.
|
||||
|
||||
Usage:
|
||||
python examples/research-sota/r5_subcarrier_saliency.py \
|
||||
--paired data/paired/wiflow-p7-1779210883.paired.jsonl \
|
||||
--model v2/crates/cog-person-count/cog/artifacts/count_v1.safetensors \
|
||||
--kind count
|
||||
python examples/research-sota/r5_subcarrier_saliency.py \
|
||||
--paired data/paired/wiflow-p7-1779210883.paired.jsonl \
|
||||
--model v2/crates/cog-pose-estimation/cog/artifacts/pose_v1.safetensors \
|
||||
--kind pose
|
||||
|
||||
Output:
|
||||
<dirname-of-model>/saliency.json per-subcarrier saliency + top-K lists
|
||||
stdout summary table
|
||||
|
||||
Method (per ADR/research note):
|
||||
S_k = E_samples[ |dL/dx_k| * |x_k| ]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import struct
|
||||
from pathlib import Path
|
||||
from typing import Tuple
|
||||
|
||||
import numpy as np
|
||||
|
||||
|
||||
N_SUB, N_FRAMES = 56, 20
|
||||
|
||||
|
||||
def load_paired(path: Path, kind: str, max_samples: int | None = None) -> Tuple[np.ndarray, np.ndarray]:
|
||||
"""Returns (X, y) — X is [N, 56, 20] float32, y depends on kind.
|
||||
|
||||
kind="count" → y is [N] int64 in {0..7}
|
||||
kind="pose" → y is [N, 17, 2] float32 in [0, 1]
|
||||
"""
|
||||
csis, ys = [], []
|
||||
with path.open(encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
d = json.loads(line)
|
||||
shape = d.get("csi_shape", [N_SUB, N_FRAMES])
|
||||
if shape != [N_SUB, N_FRAMES]:
|
||||
continue
|
||||
csi = np.asarray(d["csi"], dtype=np.float32).reshape(N_SUB, N_FRAMES)
|
||||
csis.append(csi)
|
||||
if kind == "count":
|
||||
ys.append(int(d.get("n_persons_mode", 0)))
|
||||
elif kind == "pose":
|
||||
ys.append(np.asarray(d.get("kp", []), dtype=np.float32))
|
||||
else:
|
||||
raise ValueError(f"unknown kind: {kind}")
|
||||
if max_samples and len(csis) >= max_samples:
|
||||
break
|
||||
return np.stack(csis), np.asarray(ys, dtype=(np.int64 if kind == "count" else np.float32))
|
||||
|
||||
|
||||
def load_safetensors(path: Path) -> dict[str, np.ndarray]:
|
||||
"""Pure-python safetensors reader. Returns {name: ndarray}."""
|
||||
with path.open("rb") as f:
|
||||
hlen = struct.unpack("<Q", f.read(8))[0]
|
||||
header = json.loads(f.read(hlen).decode("utf-8"))
|
||||
out = {}
|
||||
for name, meta in header.items():
|
||||
if name == "__metadata__":
|
||||
continue
|
||||
start, end = meta["data_offsets"]
|
||||
shape = meta["shape"]
|
||||
assert meta["dtype"] == "F32", f"unsupported dtype {meta['dtype']} in {name}"
|
||||
f.seek(8 + hlen + start)
|
||||
buf = f.read(end - start)
|
||||
arr = np.frombuffer(buf, dtype=np.float32).copy().reshape(shape)
|
||||
out[name] = arr
|
||||
return out
|
||||
|
||||
|
||||
def conv1d_forward(x: np.ndarray, w: np.ndarray, b: np.ndarray, padding: int, dilation: int) -> np.ndarray:
|
||||
"""Pure-numpy Conv1d forward. x: [B, Cin, T], w: [Cout, Cin, K]. Returns [B, Cout, T']."""
|
||||
B, Cin, T = x.shape
|
||||
Cout, _, K = w.shape
|
||||
# Pad
|
||||
xp = np.pad(x, ((0, 0), (0, 0), (padding, padding)), mode="constant")
|
||||
Tp = xp.shape[2]
|
||||
# Effective filter span with dilation
|
||||
eff = (K - 1) * dilation + 1
|
||||
Tout = Tp - eff + 1
|
||||
out = np.zeros((B, Cout, Tout), dtype=np.float32)
|
||||
for k in range(K):
|
||||
# x_slice shape: [B, Cin, Tout]
|
||||
x_slice = xp[:, :, k * dilation : k * dilation + Tout]
|
||||
# w_slice shape: [Cout, Cin]
|
||||
w_slice = w[:, :, k]
|
||||
# einsum: B,Cin,T x Cout,Cin → B,Cout,T
|
||||
out += np.einsum("bct,oc->bot", x_slice, w_slice)
|
||||
return out + b[None, :, None]
|
||||
|
||||
|
||||
def relu(x: np.ndarray) -> np.ndarray:
|
||||
return np.maximum(x, 0.0)
|
||||
|
||||
|
||||
def softmax(x: np.ndarray, axis: int = -1) -> np.ndarray:
|
||||
m = x.max(axis=axis, keepdims=True)
|
||||
e = np.exp(x - m)
|
||||
return e / e.sum(axis=axis, keepdims=True)
|
||||
|
||||
|
||||
def forward_count(x: np.ndarray, w: dict[str, np.ndarray]) -> np.ndarray:
|
||||
"""CountNet forward. x: [B, 56, 20] → probs [B, 8]."""
|
||||
h = conv1d_forward(x, w["enc.c1.weight"], w["enc.c1.bias"], padding=1, dilation=1)
|
||||
h = relu(h)
|
||||
h = conv1d_forward(h, w["enc.c2.weight"], w["enc.c2.bias"], padding=2, dilation=2)
|
||||
h = relu(h)
|
||||
h = conv1d_forward(h, w["enc.c3.weight"], w["enc.c3.bias"], padding=4, dilation=4)
|
||||
h = relu(h)
|
||||
h = h.mean(axis=2) # [B, 128]
|
||||
# count head
|
||||
z = relu(h @ w["count_head.fc1.weight"].T + w["count_head.fc1.bias"])
|
||||
z = z @ w["count_head.fc2.weight"].T + w["count_head.fc2.bias"]
|
||||
return softmax(z, axis=-1)
|
||||
|
||||
|
||||
def saliency_input_gradient(
|
||||
X: np.ndarray,
|
||||
y: np.ndarray,
|
||||
weights: dict[str, np.ndarray],
|
||||
kind: str,
|
||||
eps: float = 1e-3,
|
||||
) -> np.ndarray:
|
||||
"""Per-subcarrier saliency: S_k = E[|dL/dx_k| * |x_k|].
|
||||
|
||||
Uses central-difference numerical gradient over each subcarrier (cheap because
|
||||
we marginalise over the time axis after taking the abs). For a 56-subcarrier
|
||||
input that's 56 forward passes per sample — slow but exact, and only runs
|
||||
once per saliency map.
|
||||
"""
|
||||
B, N_sub, T = X.shape
|
||||
saliency = np.zeros(N_sub, dtype=np.float64)
|
||||
|
||||
if kind == "count":
|
||||
# Loss = -log(p_true). Compute baseline log-prob.
|
||||
for k in range(N_sub):
|
||||
x_plus = X.copy()
|
||||
x_plus[:, k, :] += eps
|
||||
x_minus = X.copy()
|
||||
x_minus[:, k, :] -= eps
|
||||
p_plus = forward_count(x_plus, weights)
|
||||
p_minus = forward_count(x_minus, weights)
|
||||
# dL/dx ≈ -(log p_plus[y] - log p_minus[y]) / (2*eps)
|
||||
idx = np.arange(B)
|
||||
lp_plus = np.log(p_plus[idx, y] + 1e-12)
|
||||
lp_minus = np.log(p_minus[idx, y] + 1e-12)
|
||||
grad_k = -(lp_plus - lp_minus) / (2 * eps) # [B]
|
||||
# |dL/dx_k| * |x_k| — x_k is a vector over time; take its magnitude
|
||||
x_k_mag = np.abs(X[:, k, :]).mean(axis=1) # [B]
|
||||
saliency[k] += float((np.abs(grad_k) * x_k_mag).mean())
|
||||
else:
|
||||
raise NotImplementedError("pose kind not yet wired — count first")
|
||||
|
||||
return saliency
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--paired", required=True)
|
||||
parser.add_argument("--model", required=True)
|
||||
parser.add_argument("--kind", choices=["count", "pose"], default="count")
|
||||
parser.add_argument("--max-samples", type=int, default=128,
|
||||
help="Cap on samples used for saliency (saliency cost is O(N_sub × samples × eps_passes))")
|
||||
parser.add_argument("--out", default=None,
|
||||
help="Output JSON path; defaults to <model_dir>/saliency.json")
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"Loading paired data from {args.paired} (kind={args.kind})")
|
||||
X, y = load_paired(Path(args.paired), kind=args.kind, max_samples=args.max_samples)
|
||||
print(f" X: {X.shape}, y: {y.shape}")
|
||||
if args.kind == "count":
|
||||
unique, counts = np.unique(y, return_counts=True)
|
||||
print(f" label distribution: {dict(zip(unique.tolist(), counts.tolist()))}")
|
||||
|
||||
# Standardise (per-subcarrier z-score using THIS subset's stats — saliency is
|
||||
# invariant to affine input transforms in the limit of small eps).
|
||||
mu = X.mean(axis=(0, 2), keepdims=True)
|
||||
sd = X.std(axis=(0, 2), keepdims=True) + 1e-6
|
||||
X_norm = (X - mu) / sd
|
||||
|
||||
print(f"Loading weights from {args.model}")
|
||||
weights = load_safetensors(Path(args.model))
|
||||
print(f" loaded {len(weights)} tensors: {sorted(list(weights.keys()))[:6]}...")
|
||||
|
||||
print(f"Computing input×gradient saliency over {X.shape[0]} samples × 56 subcarriers...")
|
||||
saliency = saliency_input_gradient(X_norm, y, weights, kind=args.kind, eps=1e-3)
|
||||
|
||||
order = np.argsort(saliency)[::-1] # descending
|
||||
top_k = {k: order[:k].tolist() for k in (8, 16, 32)}
|
||||
|
||||
out = {
|
||||
"kind": args.kind,
|
||||
"model": str(args.model),
|
||||
"n_samples": int(X.shape[0]),
|
||||
"saliency_per_subcarrier": saliency.tolist(),
|
||||
"ranking_high_to_low": order.tolist(),
|
||||
"top_k_subcarriers": top_k,
|
||||
"saliency_summary": {
|
||||
"min": float(saliency.min()),
|
||||
"max": float(saliency.max()),
|
||||
"mean": float(saliency.mean()),
|
||||
"std": float(saliency.std()),
|
||||
"max_to_mean_ratio": float(saliency.max() / max(saliency.mean(), 1e-12)),
|
||||
},
|
||||
}
|
||||
|
||||
out_path = Path(args.out) if args.out else Path(args.model).parent / "saliency.json"
|
||||
out_path.write_text(json.dumps(out, indent=2))
|
||||
print(f"\nWrote {out_path}")
|
||||
print(f"\nTop 8 subcarriers (most influential):")
|
||||
for rank, idx in enumerate(order[:8]):
|
||||
print(f" #{rank + 1}: subcarrier {int(idx):2d} saliency={saliency[idx]:.4f}")
|
||||
print(f"\nMax/mean ratio: {out['saliency_summary']['max_to_mean_ratio']:.2f}× "
|
||||
f"(higher = signal more concentrated in a few subcarriers)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,208 @@
|
||||
#!/usr/bin/env python3
|
||||
"""R7 — multi-link consistency detection via Stoer-Wagner-style mincut.
|
||||
|
||||
See docs/research/sota-2026-05-22/R7-multilink-consistency.md.
|
||||
|
||||
Premise: in a multi-node CSI mesh, all nodes observe the same physical
|
||||
scene through slightly different channels. Their per-window CSI features
|
||||
should cluster tightly under a similarity metric. If one node is
|
||||
compromised (spoofed CSI, replay attack, jamming-induced corruption), its
|
||||
features fall outside the cluster — and the mincut of the inter-node
|
||||
similarity graph isolates it cleanly.
|
||||
|
||||
This demo:
|
||||
1. Synthesises 4 "honest" CSI windows from one underlying scene + per-node
|
||||
Gaussian noise (realistic multipath variability).
|
||||
2. Synthesises 1 "adversarial" CSI window via three attack modes:
|
||||
(a) replay — paste in a stale window from earlier
|
||||
(b) shift — add a constant offset to every subcarrier
|
||||
(c) noise — pure white noise of the same magnitude as honest CSI
|
||||
3. Builds a 5×5 cross-node CSI cosine-similarity matrix.
|
||||
4. Solves Stoer-Wagner mincut on the resulting graph.
|
||||
5. Reports whether the mincut partition isolates the adversarial node.
|
||||
|
||||
No framework deps — pure NumPy.
|
||||
|
||||
Usage:
|
||||
python examples/research-sota/r7_multilink_consistency.py \
|
||||
--paired data/paired/wiflow-p7-1779210883.paired.jsonl
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from pathlib import Path
|
||||
import numpy as np
|
||||
|
||||
N_SUB, N_FRAMES = 56, 20
|
||||
|
||||
|
||||
def load_one_window(path: Path, idx: int = 0) -> np.ndarray:
|
||||
"""Pull one [56, 20] CSI window from the paired data — the scene we'll synthesise around."""
|
||||
with path.open(encoding="utf-8") as f:
|
||||
for i, line in enumerate(f):
|
||||
if i < idx:
|
||||
continue
|
||||
d = json.loads(line)
|
||||
shape = d.get("csi_shape", [N_SUB, N_FRAMES])
|
||||
if shape == [N_SUB, N_FRAMES]:
|
||||
return np.asarray(d["csi"], dtype=np.float32).reshape(N_SUB, N_FRAMES)
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def synth_honest_nodes(base: np.ndarray, n_nodes: int = 4, noise_db: float = 6.0, seed: int = 42):
|
||||
"""`n_nodes` honest observers — each sees the base scene through independent multipath
|
||||
(modelled as additive Gaussian on the per-subcarrier amplitudes at `noise_db` below signal)."""
|
||||
rng = np.random.default_rng(seed)
|
||||
sigma = base.std() * 10 ** (-noise_db / 20.0)
|
||||
return np.stack([base + rng.normal(0, sigma, size=base.shape).astype(np.float32) for _ in range(n_nodes)])
|
||||
|
||||
|
||||
def synth_adversarial(base: np.ndarray, mode: str, replay_window: np.ndarray | None = None, seed: int = 7):
|
||||
"""One adversarial observer. `mode` ∈ {replay, shift, noise}."""
|
||||
rng = np.random.default_rng(seed)
|
||||
if mode == "replay":
|
||||
if replay_window is None:
|
||||
raise ValueError("replay needs a stale window")
|
||||
# Stale window with a tiny perturbation to look "fresh"
|
||||
return replay_window + rng.normal(0, 0.01, size=base.shape).astype(np.float32)
|
||||
if mode == "shift":
|
||||
return base + 3.0 * base.std() # constant offset — gives away the attack
|
||||
if mode == "noise":
|
||||
return rng.normal(base.mean(), base.std(), size=base.shape).astype(np.float32)
|
||||
raise ValueError(f"unknown adversarial mode: {mode}")
|
||||
|
||||
|
||||
def cosine_sim_matrix(windows: np.ndarray) -> np.ndarray:
|
||||
"""Pairwise cosine similarity on flattened windows. Returns [N, N] matrix."""
|
||||
flat = windows.reshape(windows.shape[0], -1)
|
||||
norms = np.linalg.norm(flat, axis=1, keepdims=True) + 1e-9
|
||||
normalized = flat / norms
|
||||
return normalized @ normalized.T
|
||||
|
||||
|
||||
def stoer_wagner_mincut(W: np.ndarray) -> tuple[float, list[int]]:
|
||||
"""Classical Stoer-Wagner mincut. Input: symmetric [N, N] non-negative weights.
|
||||
|
||||
Returns: (cut_value, partition_a_node_indices)
|
||||
|
||||
The algorithm:
|
||||
while G has more than one node:
|
||||
do a minimum-cut-phase: find the order in which nodes are added
|
||||
the last node added is one side of a candidate cut; the rest is the other side
|
||||
merge the last two nodes into one super-node, accumulate their weights
|
||||
track the minimum candidate cut across all phases
|
||||
"""
|
||||
n = W.shape[0]
|
||||
nodes = [{i} for i in range(n)] # start with each node a singleton
|
||||
W = W.astype(np.float64).copy()
|
||||
best_cut = np.inf
|
||||
best_partition_b = None
|
||||
|
||||
while len(nodes) > 1:
|
||||
# minimum-cut-phase
|
||||
n_left = len(nodes)
|
||||
A = [0] # start anywhere
|
||||
in_A = np.zeros(n_left, dtype=bool); in_A[0] = True
|
||||
weights_to_A = W[:, 0].copy()
|
||||
weights_to_A[0] = -1
|
||||
last, second_last = 0, 0
|
||||
for _ in range(n_left - 1):
|
||||
# pick the not-yet-in-A node most tightly connected to A
|
||||
cand = int(np.argmax(np.where(in_A, -1, weights_to_A)))
|
||||
second_last = last
|
||||
last = cand
|
||||
in_A[cand] = True
|
||||
A.append(cand)
|
||||
# update weights — add cand's edges
|
||||
weights_to_A = np.where(in_A, -1, weights_to_A + W[:, cand])
|
||||
|
||||
# cut-of-the-phase = sum of edges from `last` to all others
|
||||
cut_val = float((W[last, :].sum() - W[last, last]))
|
||||
if cut_val < best_cut:
|
||||
best_cut = cut_val
|
||||
best_partition_b = nodes[last].copy()
|
||||
|
||||
# merge last + second_last
|
||||
merged = nodes[last] | nodes[second_last]
|
||||
# merge their rows/cols
|
||||
W[second_last, :] += W[last, :]
|
||||
W[:, second_last] += W[:, last]
|
||||
W[second_last, second_last] = 0
|
||||
# remove `last`
|
||||
keep = [i for i in range(n_left) if i != last]
|
||||
W = W[np.ix_(keep, keep)]
|
||||
nodes = [merged if i == second_last else nodes[i] for i in keep]
|
||||
|
||||
partition_b = sorted(best_partition_b) if best_partition_b else []
|
||||
return best_cut, partition_b
|
||||
|
||||
|
||||
def run_scenario(base: np.ndarray, replay_window: np.ndarray, mode: str, n_honest: int = 4):
|
||||
"""Run one adversarial scenario, return diagnostic info."""
|
||||
honest = synth_honest_nodes(base, n_nodes=n_honest, noise_db=6.0)
|
||||
adv = synth_adversarial(base, mode=mode, replay_window=replay_window)
|
||||
windows = np.concatenate([honest, adv[None, ...]], axis=0) # [n_honest + 1, 56, 20]
|
||||
adv_idx = n_honest # last node is the adversarial one
|
||||
|
||||
sim = cosine_sim_matrix(windows)
|
||||
# Convert similarity → edge weight. Mincut on similarity finds the
|
||||
# minimum-similarity partition, which is the *most-suspicious* split.
|
||||
# Use (1 - sim) as the weight if we want to minimise dissimilarity, but
|
||||
# the natural framing is: mincut over similarity-weighted graph isolates
|
||||
# the node least-similar to the rest.
|
||||
np.fill_diagonal(sim, 0.0)
|
||||
|
||||
cut_val, partition_b = stoer_wagner_mincut(sim)
|
||||
detected = (set(partition_b) == {adv_idx}) or (set(range(len(windows))) - set(partition_b) == {adv_idx})
|
||||
|
||||
return {
|
||||
"mode": mode,
|
||||
"n_honest": n_honest,
|
||||
"adv_idx": adv_idx,
|
||||
"sim_matrix": sim.round(4).tolist(),
|
||||
"mincut_value": float(cut_val),
|
||||
"partition_b": partition_b,
|
||||
"adv_isolated": bool(detected),
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--paired", required=True)
|
||||
parser.add_argument("--out", default="examples/research-sota/r7_multilink_consistency_results.json")
|
||||
args = parser.parse_args()
|
||||
|
||||
base = load_one_window(Path(args.paired), idx=10)
|
||||
stale = load_one_window(Path(args.paired), idx=900)
|
||||
if base is None or stale is None:
|
||||
raise SystemExit("need at least 901 samples in the paired file")
|
||||
|
||||
results = {}
|
||||
for mode in ["replay", "shift", "noise"]:
|
||||
scenario = run_scenario(base, stale, mode=mode, n_honest=4)
|
||||
results[mode] = scenario
|
||||
print(f"\n=== adversarial mode: {mode} ===")
|
||||
print(f" mincut value: {scenario['mincut_value']:.4f}")
|
||||
print(f" partition B (less-similar side): {scenario['partition_b']}")
|
||||
print(f" adversarial node isolated? {'YES' if scenario['adv_isolated'] else 'no'}")
|
||||
|
||||
n_detected = sum(1 for r in results.values() if r["adv_isolated"])
|
||||
summary = {
|
||||
"n_scenarios": len(results),
|
||||
"n_detected": n_detected,
|
||||
"detection_rate": n_detected / len(results),
|
||||
}
|
||||
print(f"\n=== summary ===")
|
||||
print(f" detection rate: {n_detected}/{len(results)} = {summary['detection_rate']:.0%}")
|
||||
|
||||
out_path = Path(args.out)
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
out_path.write_text(json.dumps({"summary": summary, "scenarios": results}, indent=2))
|
||||
print(f"\nWrote {out_path}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,150 @@
|
||||
{
|
||||
"summary": {
|
||||
"n_scenarios": 3,
|
||||
"n_detected": 3,
|
||||
"detection_rate": 1.0
|
||||
},
|
||||
"scenarios": {
|
||||
"replay": {
|
||||
"mode": "replay",
|
||||
"n_honest": 4,
|
||||
"adv_idx": 4,
|
||||
"sim_matrix": [
|
||||
[
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9277999997138977,
|
||||
0.9269000291824341,
|
||||
0.863099992275238
|
||||
],
|
||||
[
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9254000186920166,
|
||||
0.8618999719619751
|
||||
],
|
||||
[
|
||||
0.9277999997138977,
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9291999936103821,
|
||||
0.8615999817848206
|
||||
],
|
||||
[
|
||||
0.9269000291824341,
|
||||
0.9254000186920166,
|
||||
0.9291999936103821,
|
||||
0.0,
|
||||
0.864799976348877
|
||||
],
|
||||
[
|
||||
0.863099992275238,
|
||||
0.8618999719619751,
|
||||
0.8615999817848206,
|
||||
0.864799976348877,
|
||||
0.0
|
||||
]
|
||||
],
|
||||
"mincut_value": 3.451315999031067,
|
||||
"partition_b": [
|
||||
4
|
||||
],
|
||||
"adv_isolated": true
|
||||
},
|
||||
"shift": {
|
||||
"mode": "shift",
|
||||
"n_honest": 4,
|
||||
"adv_idx": 4,
|
||||
"sim_matrix": [
|
||||
[
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9277999997138977,
|
||||
0.9269000291824341,
|
||||
0.8944000005722046
|
||||
],
|
||||
[
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9254000186920166,
|
||||
0.8917999863624573
|
||||
],
|
||||
[
|
||||
0.9277999997138977,
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9291999936103821,
|
||||
0.8942999839782715
|
||||
],
|
||||
[
|
||||
0.9269000291824341,
|
||||
0.9254000186920166,
|
||||
0.9291999936103821,
|
||||
0.0,
|
||||
0.8917999863624573
|
||||
],
|
||||
[
|
||||
0.8944000005722046,
|
||||
0.8917999863624573,
|
||||
0.8942999839782715,
|
||||
0.8917999863624573,
|
||||
0.0
|
||||
]
|
||||
],
|
||||
"mincut_value": 3.5724358558654785,
|
||||
"partition_b": [
|
||||
4
|
||||
],
|
||||
"adv_isolated": true
|
||||
},
|
||||
"noise": {
|
||||
"mode": "noise",
|
||||
"n_honest": 4,
|
||||
"adv_idx": 4,
|
||||
"sim_matrix": [
|
||||
[
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9277999997138977,
|
||||
0.9269000291824341,
|
||||
0.6425999999046326
|
||||
],
|
||||
[
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9218999743461609,
|
||||
0.9254000186920166,
|
||||
0.6444000005722046
|
||||
],
|
||||
[
|
||||
0.9277999997138977,
|
||||
0.9218999743461609,
|
||||
0.0,
|
||||
0.9291999936103821,
|
||||
0.6389999985694885
|
||||
],
|
||||
[
|
||||
0.9269000291824341,
|
||||
0.9254000186920166,
|
||||
0.9291999936103821,
|
||||
0.0,
|
||||
0.6326000094413757
|
||||
],
|
||||
[
|
||||
0.6425999999046326,
|
||||
0.6444000005722046,
|
||||
0.6389999985694885,
|
||||
0.6326000094413757,
|
||||
0.0
|
||||
]
|
||||
],
|
||||
"mincut_value": 2.5585585832595825,
|
||||
"partition_b": [
|
||||
4
|
||||
],
|
||||
"adv_isolated": true
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
#!/usr/bin/env python3
|
||||
"""R8 — RSSI-only person count: how much accuracy do we lose vs full CSI?
|
||||
|
||||
See docs/research/sota-2026-05-22/R8-rssi-only-count.md.
|
||||
|
||||
RSSI = received signal strength = power integrated across the WiFi band.
|
||||
The CSI amplitude vector for a single packet is `|H_k|` per subcarrier k;
|
||||
its mean over subcarriers is an unbiased proxy for the per-packet RSSI
|
||||
(equivalent up to constant scaling). So aggregating our existing
|
||||
`[56 subcarriers × 20 frames]` CSI windows along the subcarrier axis gives
|
||||
us a `[20]` "RSSI-over-time" signal — exactly what any WiFi chip without
|
||||
CSI export reports as its standard `RSSI` field.
|
||||
|
||||
If a small MLP on the [20]-vector hits even 55-60% accuracy on the
|
||||
person-count task, RSSI-only deployment is viable across the entire WiFi-
|
||||
chip ecosystem (billions of devices), at the cost of needing per-chip
|
||||
calibration. v0.0.2 of cog-person-count itself only hits 62% on the 80/20
|
||||
random split, so the bar isn't sky-high.
|
||||
|
||||
Usage:
|
||||
python examples/research-sota/r8_rssi_only_count.py \
|
||||
--paired data/paired/wiflow-p7-1779210883.paired.jsonl
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import time
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
N_SUB, N_FRAMES, COUNT_CLASSES = 56, 20, 8
|
||||
|
||||
|
||||
def load_paired(path: Path) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Returns (X_csi, y) where X_csi is [N, 56, 20] and y is [N] integer count."""
|
||||
csis, ys = [], []
|
||||
with path.open(encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
d = json.loads(line)
|
||||
shape = d.get("csi_shape", [N_SUB, N_FRAMES])
|
||||
if shape != [N_SUB, N_FRAMES]:
|
||||
continue
|
||||
csi = np.asarray(d["csi"], dtype=np.float32).reshape(N_SUB, N_FRAMES)
|
||||
csis.append(csi)
|
||||
ys.append(int(d.get("n_persons_mode", 0)))
|
||||
return np.stack(csis), np.asarray(ys, dtype=np.int64)
|
||||
|
||||
|
||||
def csi_to_rssi_proxy(X_csi: np.ndarray) -> np.ndarray:
|
||||
"""Aggregate CSI amplitudes to a single RSSI scalar per frame.
|
||||
|
||||
Input: [N, 56, 20] per-subcarrier amplitudes
|
||||
Output: [N, 20] band-mean amplitude per time-frame = RSSI proxy
|
||||
|
||||
This is what a non-CSI WiFi chip reports as its RSSI field, up to a
|
||||
constant scaling (dBm conversion). We keep linear amplitude — the count
|
||||
head is invariant to that affine transform after z-score normalisation.
|
||||
"""
|
||||
return X_csi.mean(axis=1) # mean across subcarriers
|
||||
|
||||
|
||||
def softmax(x: np.ndarray, axis: int = -1) -> np.ndarray:
|
||||
m = x.max(axis=axis, keepdims=True)
|
||||
e = np.exp(x - m)
|
||||
return e / e.sum(axis=axis, keepdims=True)
|
||||
|
||||
|
||||
def train_rssi_mlp(
|
||||
X_train: np.ndarray, y_train: np.ndarray,
|
||||
X_eval: np.ndarray, y_eval: np.ndarray,
|
||||
epochs: int = 200, lr: float = 1e-2, hidden: int = 32, seed: int = 42,
|
||||
):
|
||||
"""Tiny MLP trained with vanilla SGD — no framework, just numpy.
|
||||
|
||||
Input: [N, 20] RSSI-proxy time-series
|
||||
Architecture: Linear(20 → hidden) → ReLU → Linear(hidden → 8) → softmax
|
||||
"""
|
||||
rng = np.random.default_rng(seed)
|
||||
D = X_train.shape[1]
|
||||
K = COUNT_CLASSES
|
||||
|
||||
# Glorot init
|
||||
w1 = rng.normal(0, np.sqrt(2.0 / D), size=(D, hidden)).astype(np.float32)
|
||||
b1 = np.zeros(hidden, dtype=np.float32)
|
||||
w2 = rng.normal(0, np.sqrt(2.0 / hidden), size=(hidden, K)).astype(np.float32)
|
||||
b2 = np.zeros(K, dtype=np.float32)
|
||||
|
||||
n_train = X_train.shape[0]
|
||||
batch_size = 32
|
||||
eval_curve = []
|
||||
best_eval_acc = 0.0
|
||||
best = None
|
||||
|
||||
for epoch in range(epochs):
|
||||
perm = rng.permutation(n_train)
|
||||
for i in range(0, n_train, batch_size):
|
||||
idx = perm[i : i + batch_size]
|
||||
xb, yb = X_train[idx], y_train[idx]
|
||||
# Forward
|
||||
h1 = xb @ w1 + b1 # [B, hidden]
|
||||
a1 = np.maximum(h1, 0.0) # ReLU
|
||||
logits = a1 @ w2 + b2 # [B, K]
|
||||
probs = softmax(logits, axis=-1)
|
||||
# One-hot
|
||||
onehot = np.zeros_like(probs)
|
||||
onehot[np.arange(len(yb)), yb] = 1.0
|
||||
# Backward
|
||||
dlogits = (probs - onehot) / len(yb) # [B, K]
|
||||
dw2 = a1.T @ dlogits # [hidden, K]
|
||||
db2 = dlogits.sum(axis=0)
|
||||
da1 = dlogits @ w2.T # [B, hidden]
|
||||
dh1 = da1 * (h1 > 0) # ReLU grad
|
||||
dw1 = xb.T @ dh1 # [D, hidden]
|
||||
db1 = dh1.sum(axis=0)
|
||||
# SGD
|
||||
w1 -= lr * dw1
|
||||
b1 -= lr * db1
|
||||
w2 -= lr * dw2
|
||||
b2 -= lr * db2
|
||||
|
||||
# Eval
|
||||
eh = np.maximum(X_eval @ w1 + b1, 0.0)
|
||||
eval_logits = eh @ w2 + b2
|
||||
eval_pred = eval_logits.argmax(axis=1)
|
||||
eval_acc = float((eval_pred == y_eval).mean())
|
||||
eval_curve.append(eval_acc)
|
||||
if eval_acc > best_eval_acc:
|
||||
best_eval_acc = eval_acc
|
||||
best = (w1.copy(), b1.copy(), w2.copy(), b2.copy())
|
||||
|
||||
return best, best_eval_acc, eval_curve
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--paired", required=True)
|
||||
parser.add_argument("--out", default="examples/research-sota/r8_rssi_only_results.json")
|
||||
parser.add_argument("--epochs", type=int, default=200)
|
||||
parser.add_argument("--seed", type=int, default=42)
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"Loading paired data from {args.paired}")
|
||||
X_csi, y = load_paired(Path(args.paired))
|
||||
print(f" CSI shape: {X_csi.shape}")
|
||||
print(f" label distribution: {dict(Counter(y.tolist()).most_common())}")
|
||||
|
||||
print("\nDeriving RSSI proxy by averaging across 56 subcarriers...")
|
||||
X_rssi = csi_to_rssi_proxy(X_csi)
|
||||
print(f" RSSI proxy shape: {X_rssi.shape} (one scalar per frame, 20 frames per sample)")
|
||||
print(f" RSSI proxy stats: mean={X_rssi.mean():.3f} std={X_rssi.std():.3f}")
|
||||
|
||||
# Random 80/20 split — same seed as v0.0.2 so the eval set is identical
|
||||
rng = np.random.default_rng(seed=args.seed)
|
||||
idx = np.arange(X_rssi.shape[0])
|
||||
rng.shuffle(idx)
|
||||
n_eval = int(round(0.2 * X_rssi.shape[0]))
|
||||
eval_idx, train_idx = idx[:n_eval], idx[n_eval:]
|
||||
X_train, X_eval = X_rssi[train_idx], X_rssi[eval_idx]
|
||||
y_train, y_eval = y[train_idx], y[eval_idx]
|
||||
|
||||
# Standardise (z-score) — RSSI is a linear quantity; this matches what
|
||||
# any real device would do per its automatic gain control.
|
||||
mu = X_train.mean(axis=0, keepdims=True)
|
||||
sd = X_train.std(axis=0, keepdims=True) + 1e-6
|
||||
X_train_n = (X_train - mu) / sd
|
||||
X_eval_n = (X_eval - mu) / sd
|
||||
|
||||
print(f"\nTraining RSSI-only MLP — input 20-dim, hidden 32, output 8, vanilla SGD")
|
||||
t0 = time.perf_counter()
|
||||
best_params, best_eval_acc, curve = train_rssi_mlp(
|
||||
X_train_n, y_train, X_eval_n, y_eval,
|
||||
epochs=args.epochs, lr=1e-2, hidden=32, seed=args.seed,
|
||||
)
|
||||
elapsed = time.perf_counter() - t0
|
||||
print(f"\nTrained {args.epochs} epochs in {elapsed:.2f} s on CPU")
|
||||
|
||||
# Final eval with best checkpoint
|
||||
w1, b1, w2, b2 = best_params
|
||||
eh = np.maximum(X_eval_n @ w1 + b1, 0.0)
|
||||
eval_logits = eh @ w2 + b2
|
||||
eval_pred = eval_logits.argmax(axis=1)
|
||||
acc = float((eval_pred == y_eval).mean())
|
||||
per_class = {}
|
||||
for k in range(COUNT_CLASSES):
|
||||
mask = y_eval == k
|
||||
n = int(mask.sum())
|
||||
if n > 0:
|
||||
per_class[k] = {
|
||||
"support": n,
|
||||
"accuracy": float(((eval_pred == y_eval) & mask).sum() / n),
|
||||
}
|
||||
|
||||
# Baseline reference: how does v0.0.2 (full CSI) score on the SAME eval set?
|
||||
# We don't run the cog binary here — just record the published numbers.
|
||||
full_csi_baseline = {
|
||||
"version": "cog-person-count v0.0.2",
|
||||
"overall_acc": 0.623,
|
||||
"class0_acc": 0.862,
|
||||
"class1_acc": 0.343,
|
||||
"source": "docs/benchmarks/person-count-cog.md",
|
||||
}
|
||||
|
||||
print(f"\n=== R8 RSSI-only results ===")
|
||||
print(f" Eval accuracy: {acc:.3f}")
|
||||
print(f" Per-class:")
|
||||
for k, v in per_class.items():
|
||||
print(f" class {k}: {v['accuracy']:.3f} on {v['support']} samples")
|
||||
print(f"\n Full-CSI baseline (v0.0.2): {full_csi_baseline['overall_acc']:.3f}")
|
||||
print(f" Retained fraction: {acc / full_csi_baseline['overall_acc']:.2%}")
|
||||
|
||||
Path(args.out).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(args.out).write_text(json.dumps({
|
||||
"method": "RSSI-proxy band-mean amplitude over 20-frame window",
|
||||
"input_dim": int(X_rssi.shape[1]),
|
||||
"architecture": "MLP(20 → 32 → 8) ReLU + softmax, vanilla SGD",
|
||||
"epochs": args.epochs,
|
||||
"train_time_s": elapsed,
|
||||
"n_train": int(X_train.shape[0]),
|
||||
"n_eval": int(X_eval.shape[0]),
|
||||
"label_distribution_train": dict(Counter(y_train.tolist()).most_common()),
|
||||
"label_distribution_eval": dict(Counter(y_eval.tolist()).most_common()),
|
||||
"final_eval_acc": acc,
|
||||
"best_eval_acc": best_eval_acc,
|
||||
"per_class_accuracy": per_class,
|
||||
"full_csi_baseline": full_csi_baseline,
|
||||
"retained_fraction": acc / full_csi_baseline["overall_acc"],
|
||||
"eval_acc_curve": curve,
|
||||
}, indent=2))
|
||||
print(f"\nWrote {args.out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,239 @@
|
||||
{
|
||||
"method": "RSSI-proxy band-mean amplitude over 20-frame window",
|
||||
"input_dim": 20,
|
||||
"architecture": "MLP(20 \u2192 32 \u2192 8) ReLU + softmax, vanilla SGD",
|
||||
"epochs": 200,
|
||||
"train_time_s": 0.717573200003244,
|
||||
"n_train": 862,
|
||||
"n_eval": 215,
|
||||
"label_distribution_train": {
|
||||
"1": 445,
|
||||
"0": 417
|
||||
},
|
||||
"label_distribution_eval": {
|
||||
"0": 116,
|
||||
"1": 99
|
||||
},
|
||||
"final_eval_acc": 0.5906976744186047,
|
||||
"best_eval_acc": 0.5906976744186047,
|
||||
"per_class_accuracy": {
|
||||
"0": {
|
||||
"support": 116,
|
||||
"accuracy": 0.5948275862068966
|
||||
},
|
||||
"1": {
|
||||
"support": 99,
|
||||
"accuracy": 0.5858585858585859
|
||||
}
|
||||
},
|
||||
"full_csi_baseline": {
|
||||
"version": "cog-person-count v0.0.2",
|
||||
"overall_acc": 0.623,
|
||||
"class0_acc": 0.862,
|
||||
"class1_acc": 0.343,
|
||||
"source": "docs/benchmarks/person-count-cog.md"
|
||||
},
|
||||
"retained_fraction": 0.9481503602224793,
|
||||
"eval_acc_curve": [
|
||||
0.3395348837209302,
|
||||
0.4604651162790698,
|
||||
0.4744186046511628,
|
||||
0.5116279069767442,
|
||||
0.5534883720930233,
|
||||
0.5395348837209303,
|
||||
0.5441860465116279,
|
||||
0.5302325581395348,
|
||||
0.5255813953488372,
|
||||
0.5348837209302325,
|
||||
0.5395348837209303,
|
||||
0.5395348837209303,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5441860465116279,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162,
|
||||
0.5441860465116279,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5627906976744186,
|
||||
0.5488372093023256,
|
||||
0.5488372093023256,
|
||||
0.5441860465116279,
|
||||
0.586046511627907,
|
||||
0.5534883720930233,
|
||||
0.5441860465116279,
|
||||
0.5395348837209303,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5441860465116279,
|
||||
0.5813953488372093,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5767441860465117,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5906976744186047,
|
||||
0.5906976744186047,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5813953488372093,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5720930232558139,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5767441860465117,
|
||||
0.5627906976744186,
|
||||
0.5720930232558139,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5767441860465117,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5720930232558139,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5534883720930233,
|
||||
0.5674418604651162,
|
||||
0.5488372093023256,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5488372093023256,
|
||||
0.5488372093023256,
|
||||
0.5488372093023256,
|
||||
0.5395348837209303,
|
||||
0.5627906976744186,
|
||||
0.5441860465116279,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5441860465116279,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162,
|
||||
0.5348837209302325,
|
||||
0.5534883720930233,
|
||||
0.5441860465116279,
|
||||
0.5534883720930233,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5488372093023256,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5488372093023256,
|
||||
0.5441860465116279,
|
||||
0.5441860465116279,
|
||||
0.5534883720930233,
|
||||
0.5720930232558139,
|
||||
0.5441860465116279,
|
||||
0.5488372093023256,
|
||||
0.5674418604651162,
|
||||
0.5488372093023256,
|
||||
0.5534883720930233,
|
||||
0.5674418604651162,
|
||||
0.5720930232558139,
|
||||
0.5441860465116279,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5488372093023256,
|
||||
0.5395348837209303,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5581395348837209,
|
||||
0.5441860465116279,
|
||||
0.5720930232558139,
|
||||
0.5488372093023256,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162,
|
||||
0.5674418604651162,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5674418604651162,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5534883720930233,
|
||||
0.5488372093023256,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5581395348837209,
|
||||
0.5581395348837209,
|
||||
0.5674418604651162,
|
||||
0.5488372093023256,
|
||||
0.5674418604651162,
|
||||
0.5674418604651162,
|
||||
0.5534883720930233,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5627906976744186,
|
||||
0.5674418604651162
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
#!/usr/bin/env python3
|
||||
"""R9 — RSSI fingerprint topology: does temporal proximity = feature proximity?
|
||||
|
||||
See docs/research/sota-2026-05-22/R9-rssi-fingerprint-knn.md.
|
||||
|
||||
Hypothesis: if RSSI sequences from temporally-adjacent windows are
|
||||
nearest-neighbours in feature space, RSSI-fingerprint localisation is
|
||||
viable. If the K-NN of every query is random in time, RSSI sequences
|
||||
don't carry stable enough fingerprints — fall back to multi-modal cues
|
||||
(BSSID lists, signal-of-opportunity).
|
||||
|
||||
Test:
|
||||
1. Build the same 20-dim RSSI proxy from the 1,077 paired windows
|
||||
(band-mean across 56 subcarriers per frame).
|
||||
2. For each sample i, find K-NN in cosine-similarity space.
|
||||
3. Measure: what fraction of the K-NN come from windows within
|
||||
±60 seconds of the query's timestamp?
|
||||
4. Compare to a random baseline (what would the fraction be if K-NN
|
||||
were chosen at random?).
|
||||
|
||||
If the temporal-K-NN fraction is ≫ random, RSSI fingerprints have stable
|
||||
spatial structure → R9 viable.
|
||||
|
||||
Usage:
|
||||
python examples/research-sota/r9_rssi_fingerprint_knn.py \
|
||||
--paired data/paired/wiflow-p7-1779210883.paired.jsonl
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
|
||||
N_SUB, N_FRAMES = 56, 20
|
||||
|
||||
|
||||
def load_rssi_proxy(path: Path) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Return (X_rssi, ts_seconds). X_rssi is [N, 20], ts is [N] float seconds."""
|
||||
csis, ts = [], []
|
||||
with path.open(encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
d = json.loads(line)
|
||||
shape = d.get("csi_shape", [N_SUB, N_FRAMES])
|
||||
if shape != [N_SUB, N_FRAMES]:
|
||||
continue
|
||||
csi = np.asarray(d["csi"], dtype=np.float32).reshape(N_SUB, N_FRAMES)
|
||||
csis.append(csi.mean(axis=0)) # band-mean → [20]
|
||||
t_iso = d.get("ts_start", "1970-01-01T00:00:00Z")
|
||||
ts.append(datetime.fromisoformat(t_iso.replace("Z", "+00:00")).timestamp())
|
||||
return np.stack(csis), np.asarray(ts, dtype=np.float64)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--paired", required=True)
|
||||
parser.add_argument("--out", default="examples/research-sota/r9_rssi_fingerprint_results.json")
|
||||
parser.add_argument("--k", type=int, default=5)
|
||||
parser.add_argument("--temporal-window-s", type=float, default=60.0)
|
||||
args = parser.parse_args()
|
||||
|
||||
print(f"Loading RSSI-proxy from {args.paired}")
|
||||
X, ts = load_rssi_proxy(Path(args.paired))
|
||||
print(f" N samples: {X.shape[0]}, feature dim: {X.shape[1]}")
|
||||
print(f" time range: {datetime.fromtimestamp(ts.min(), tz=timezone.utc):%H:%M:%S} - "
|
||||
f"{datetime.fromtimestamp(ts.max(), tz=timezone.utc):%H:%M:%S} "
|
||||
f"({(ts.max() - ts.min()) / 60:.1f} min total)")
|
||||
|
||||
# Z-score normalise across all samples — what a real device does via AGC
|
||||
mu = X.mean(axis=0, keepdims=True)
|
||||
sd = X.std(axis=0, keepdims=True) + 1e-6
|
||||
Xn = (X - mu) / sd
|
||||
|
||||
# All-pairs cosine similarity
|
||||
print(f"\nComputing all-pairs cosine similarity ({X.shape[0]}×{X.shape[0]} = "
|
||||
f"{X.shape[0]**2:,} pairs)...")
|
||||
norms = np.linalg.norm(Xn, axis=1, keepdims=True) + 1e-9
|
||||
Xnorm = Xn / norms
|
||||
sim = Xnorm @ Xnorm.T
|
||||
np.fill_diagonal(sim, -np.inf) # exclude self-match
|
||||
|
||||
N = X.shape[0]
|
||||
K = args.k
|
||||
W = args.temporal_window_s
|
||||
|
||||
# For each query, find top-K nearest neighbours and measure how many are
|
||||
# within the temporal window
|
||||
print(f"\nMeasuring temporal-locality of top-{K} cosine-NN with window ±{W:.0f}s...")
|
||||
knn_idx = np.argsort(-sim, axis=1)[:, :K] # [N, K]
|
||||
knn_ts = ts[knn_idx] # [N, K]
|
||||
delta_t = np.abs(knn_ts - ts[:, None]) # [N, K]
|
||||
within = (delta_t <= W).astype(np.float32) # [N, K]
|
||||
per_query_within_frac = within.mean(axis=1) # [N] — fraction of K-NN within window
|
||||
overall_within_frac = within.mean() # scalar
|
||||
|
||||
# Random baseline: for each query, what fraction of all OTHER samples
|
||||
# fall within ±W of its timestamp?
|
||||
rand_within = np.zeros(N, dtype=np.float32)
|
||||
for i in range(N):
|
||||
delta = np.abs(ts - ts[i])
|
||||
delta[i] = np.inf
|
||||
rand_within[i] = (delta <= W).mean()
|
||||
rand_baseline = float(rand_within.mean())
|
||||
|
||||
# Headline numbers
|
||||
lift = overall_within_frac / max(rand_baseline, 1e-9)
|
||||
|
||||
print(f"\n=== R9 RSSI-fingerprint K-NN results ===")
|
||||
print(f" K-NN within ±{W:.0f}s: {overall_within_frac:.3f}")
|
||||
print(f" Random baseline: {rand_baseline:.3f}")
|
||||
print(f" Lift over random: {lift:.2f}×")
|
||||
print(f" Per-query stdev: {per_query_within_frac.std():.3f}")
|
||||
|
||||
if lift >= 3.0:
|
||||
verdict = "STRONG: RSSI sequences carry stable spatial fingerprints"
|
||||
elif lift >= 1.5:
|
||||
verdict = "MODERATE: RSSI fingerprints work but with significant noise"
|
||||
else:
|
||||
verdict = "WEAK: RSSI-only fingerprint localisation is unreliable on this data"
|
||||
print(f"\n Verdict: {verdict}")
|
||||
|
||||
out = {
|
||||
"n_samples": int(N),
|
||||
"k": K,
|
||||
"temporal_window_s": W,
|
||||
"knn_within_window_fraction": float(overall_within_frac),
|
||||
"random_baseline": rand_baseline,
|
||||
"lift": float(lift),
|
||||
"per_query_within_fraction_stdev": float(per_query_within_frac.std()),
|
||||
"verdict": verdict,
|
||||
}
|
||||
Path(args.out).parent.mkdir(parents=True, exist_ok=True)
|
||||
Path(args.out).write_text(json.dumps(out, indent=2))
|
||||
print(f"\nWrote {args.out}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"n_samples": 1077,
|
||||
"k": 5,
|
||||
"temporal_window_s": 60.0,
|
||||
"knn_within_window_fraction": 0.16861653327941895,
|
||||
"random_baseline": 0.07726679742336273,
|
||||
"lift": 2.1822638511657715,
|
||||
"per_query_within_fraction_stdev": 0.18328286707401276,
|
||||
"verdict": "MODERATE: RSSI fingerprints work but with significant noise"
|
||||
}
|
||||
@@ -481,12 +481,33 @@ function align() {
|
||||
? extractCsiMatrix(window)
|
||||
: extractFeatureMatrix(window);
|
||||
|
||||
// ADR-103: aggregate `n_persons` per window so the cog-person-count
|
||||
// training pipeline has count labels. Two summaries:
|
||||
// - `n_persons_mode` — modal value across the camera frames in
|
||||
// the window. Robust to single-frame noise;
|
||||
// this is the supervised label for the
|
||||
// categorical {0..7} count head.
|
||||
// - `n_persons_max` — the maximum value seen in the window.
|
||||
// Useful as a soft upper bound (e.g. for
|
||||
// dynamic dropout weighting during training).
|
||||
const personCounts = matched.map(f => f.nPersons ?? 0);
|
||||
const counts = new Map();
|
||||
for (const v of personCounts) counts.set(v, (counts.get(v) ?? 0) + 1);
|
||||
let modeVal = 0;
|
||||
let modeCount = -1;
|
||||
for (const [v, n] of counts) {
|
||||
if (n > modeCount) { modeVal = v; modeCount = n; }
|
||||
}
|
||||
const maxVal = personCounts.reduce((a, b) => Math.max(a, b), 0);
|
||||
|
||||
paired.push({
|
||||
csi: csiMatrix.data,
|
||||
csi_shape: csiMatrix.shape,
|
||||
kp: keypoints,
|
||||
conf: Math.round(avgConfidence * 1000) / 1000,
|
||||
n_camera_frames: matched.length,
|
||||
n_persons_mode: modeVal,
|
||||
n_persons_max: maxVal,
|
||||
ts_start: new Date(tStartMs).toISOString(),
|
||||
ts_end: new Date(tEndMs).toISOString(),
|
||||
});
|
||||
|
||||
@@ -0,0 +1,761 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Train the person-count head — ADR-103 v0.0.1.
|
||||
|
||||
Mirrors the Conv1d encoder architecture from cog-person-count's
|
||||
`src/inference.rs::CountNet` exactly, so the learned weights load
|
||||
into the Rust cog without translation. Trains on
|
||||
data/paired/wiflow-p7-1779210883.paired.jsonl (1,077 samples with
|
||||
n_persons_mode labels in {0, 1}).
|
||||
|
||||
Output: count_v1.safetensors + count_v1.onnx + train_results.json.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import struct
|
||||
import time
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
|
||||
# Architecture constants — MUST match cog-person-count's src/inference.rs.
|
||||
N_SUB = 56
|
||||
N_FRAMES = 20
|
||||
COUNT_CLASSES = 8
|
||||
|
||||
|
||||
class CountNet(nn.Module):
|
||||
"""Mirrors cog_person_count::inference::CountNet bit-for-bit."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
# Encoder — identical to the pose cog's encoder so future joint
|
||||
# training can share weights.
|
||||
self.enc_c1 = nn.Conv1d(N_SUB, 64, kernel_size=3, padding=1, dilation=1)
|
||||
self.enc_c2 = nn.Conv1d(64, 128, kernel_size=3, padding=2, dilation=2)
|
||||
self.enc_c3 = nn.Conv1d(128, 128, kernel_size=3, padding=4, dilation=4)
|
||||
# Count head
|
||||
self.count_head_fc1 = nn.Linear(128, 64)
|
||||
self.count_head_fc2 = nn.Linear(64, COUNT_CLASSES)
|
||||
# Confidence head
|
||||
self.conf_head_fc1 = nn.Linear(128, 32)
|
||||
self.conf_head_fc2 = nn.Linear(32, 1)
|
||||
|
||||
def forward(self, x: torch.Tensor):
|
||||
# x: [B, 56, 20]
|
||||
h = F.relu(self.enc_c1(x))
|
||||
h = F.relu(self.enc_c2(h))
|
||||
h = F.relu(self.enc_c3(h))
|
||||
h = h.mean(dim=2) # [B, 128]
|
||||
|
||||
# Logits (un-normalised); softmax at inference + cross-entropy training.
|
||||
c = F.relu(self.count_head_fc1(h))
|
||||
count_logits = self.count_head_fc2(c)
|
||||
|
||||
# Confidence head — sigmoid at inference; BCE-with-logits at training.
|
||||
cf = F.relu(self.conf_head_fc1(h))
|
||||
conf_logits = self.conf_head_fc2(cf)
|
||||
|
||||
return count_logits, conf_logits
|
||||
|
||||
|
||||
def load_paired(path: Path) -> tuple[np.ndarray, np.ndarray]:
|
||||
"""Return (X, y) where X is [N, 56, 20] CSI and y is [N] integer counts."""
|
||||
csis, ys = [], []
|
||||
with path.open(encoding="utf-8") as f:
|
||||
for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
d = json.loads(line)
|
||||
shape = d.get("csi_shape", [N_SUB, N_FRAMES])
|
||||
if shape != [N_SUB, N_FRAMES]:
|
||||
continue
|
||||
csi = np.asarray(d["csi"], dtype=np.float32).reshape(N_SUB, N_FRAMES)
|
||||
csis.append(csi)
|
||||
ys.append(int(d.get("n_persons_mode", 0)))
|
||||
X = np.stack(csis, axis=0)
|
||||
y = np.asarray(ys, dtype=np.int64)
|
||||
return X, y
|
||||
|
||||
|
||||
def temporal_split(X: np.ndarray, y: np.ndarray, eval_frac: float = 0.2):
|
||||
"""Held-out time-window eval (last `eval_frac` of samples, by index)."""
|
||||
n = X.shape[0]
|
||||
n_eval = int(round(n * eval_frac))
|
||||
n_train = n - n_eval
|
||||
return (
|
||||
X[:n_train], y[:n_train],
|
||||
X[n_train:], y[n_train:],
|
||||
)
|
||||
|
||||
|
||||
def stratified_k_fold(X: np.ndarray, y: np.ndarray, k: int = 5):
|
||||
"""Stratified k-fold cross-validation splits — hand-rolled, no sklearn.
|
||||
|
||||
Per class: shuffle the indices (deterministic seed 42), split into k
|
||||
near-equal chunks, then assemble fold i by taking chunk i from every
|
||||
class. Yields (X_train, y_train, X_val, y_val) per fold, with class
|
||||
distribution preserved within ±1.
|
||||
"""
|
||||
rng = np.random.default_rng(seed=42)
|
||||
classes = np.unique(y)
|
||||
per_class_folds = {}
|
||||
for c in classes:
|
||||
idx = np.where(y == c)[0]
|
||||
rng.shuffle(idx)
|
||||
per_class_folds[c] = np.array_split(idx, k)
|
||||
for fold in range(k):
|
||||
val_idx = np.concatenate([per_class_folds[c][fold] for c in classes])
|
||||
train_idx = np.concatenate(
|
||||
[per_class_folds[c][f] for c in classes for f in range(k) if f != fold]
|
||||
)
|
||||
yield X[train_idx], y[train_idx], X[val_idx], y[val_idx]
|
||||
|
||||
|
||||
def standardise(X_train: np.ndarray, X_eval: np.ndarray):
|
||||
"""Z-score by subcarrier across the time axis. Eval uses train stats."""
|
||||
mu = X_train.mean(axis=(0, 2), keepdims=True)
|
||||
sd = X_train.std(axis=(0, 2), keepdims=True) + 1e-6
|
||||
return (X_train - mu) / sd, (X_eval - mu) / sd
|
||||
|
||||
|
||||
def write_safetensors(model: CountNet, path: Path):
|
||||
"""Write the model's state in the same on-disk layout the Rust cog expects."""
|
||||
state = model.state_dict()
|
||||
# Map PyTorch param names → cog-person-count's VarBuilder paths.
|
||||
rename = {
|
||||
"enc_c1.weight": "enc.c1.weight",
|
||||
"enc_c1.bias": "enc.c1.bias",
|
||||
"enc_c2.weight": "enc.c2.weight",
|
||||
"enc_c2.bias": "enc.c2.bias",
|
||||
"enc_c3.weight": "enc.c3.weight",
|
||||
"enc_c3.bias": "enc.c3.bias",
|
||||
"count_head_fc1.weight": "count_head.fc1.weight",
|
||||
"count_head_fc1.bias": "count_head.fc1.bias",
|
||||
"count_head_fc2.weight": "count_head.fc2.weight",
|
||||
"count_head_fc2.bias": "count_head.fc2.bias",
|
||||
"conf_head_fc1.weight": "conf_head.fc1.weight",
|
||||
"conf_head_fc1.bias": "conf_head.fc1.bias",
|
||||
"conf_head_fc2.weight": "conf_head.fc2.weight",
|
||||
"conf_head_fc2.bias": "conf_head.fc2.bias",
|
||||
}
|
||||
|
||||
header = {}
|
||||
payload = bytearray()
|
||||
offset = 0
|
||||
for torch_name, cog_name in rename.items():
|
||||
t = state[torch_name].detach().cpu().numpy().astype(np.float32)
|
||||
n_bytes = t.nbytes
|
||||
header[cog_name] = {
|
||||
"dtype": "F32",
|
||||
"shape": list(t.shape),
|
||||
"data_offsets": [offset, offset + n_bytes],
|
||||
}
|
||||
payload.extend(t.tobytes())
|
||||
offset += n_bytes
|
||||
|
||||
header_bytes = json.dumps(header, separators=(",", ":")).encode("utf-8")
|
||||
with path.open("wb") as f:
|
||||
f.write(struct.pack("<Q", len(header_bytes)))
|
||||
f.write(header_bytes)
|
||||
f.write(payload)
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--paired", required=True)
|
||||
parser.add_argument("--out-safetensors", default="count_v1.safetensors")
|
||||
parser.add_argument("--out-onnx", default="count_v1.onnx")
|
||||
parser.add_argument("--out-results", default="count_train_results.json")
|
||||
parser.add_argument("--epochs", type=int, default=400)
|
||||
parser.add_argument("--batch-size", type=int, default=64)
|
||||
parser.add_argument("--lr", type=float, default=1e-3)
|
||||
parser.add_argument("--weight-decay", type=float, default=0.01)
|
||||
parser.add_argument("--k-fold", type=int, default=None, help="If set, run k-fold CV; else use temporal split")
|
||||
parser.add_argument("--v2", action="store_true",
|
||||
help="v0.0.2 training: random 80/20 split + label smoothing + early stopping "
|
||||
"+ balanced sampling + temperature-scaled confidence head.")
|
||||
parser.add_argument("--label-smoothing", type=float, default=0.1)
|
||||
parser.add_argument("--patience", type=int, default=20)
|
||||
args = parser.parse_args()
|
||||
|
||||
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||
print(f"device: {device}")
|
||||
|
||||
X, y = load_paired(Path(args.paired))
|
||||
print(f"loaded {X.shape[0]} samples, X shape {X.shape}, "
|
||||
f"label distribution: {dict(Counter(y.tolist()).most_common())}")
|
||||
|
||||
# K-fold cross-validation mode
|
||||
if args.k_fold is not None:
|
||||
print(f"\n=== {args.k_fold}-fold cross-validation ===")
|
||||
fold_results = []
|
||||
overall_t0 = time.perf_counter()
|
||||
|
||||
for fold_idx, (X_train, y_train, X_val, y_val) in enumerate(stratified_k_fold(X, y, k=args.k_fold)):
|
||||
print(f"\nFold {fold_idx + 1}/{args.k_fold}")
|
||||
X_train, X_val = standardise(X_train, X_val)
|
||||
|
||||
cls_counts = np.bincount(y_train, minlength=COUNT_CLASSES).astype(np.float32)
|
||||
cls_counts = np.where(cls_counts > 0, cls_counts, 1.0)
|
||||
cls_weight = (1.0 / cls_counts) / (1.0 / cls_counts).sum() * COUNT_CLASSES
|
||||
cls_weight_t = torch.from_numpy(cls_weight).to(device)
|
||||
|
||||
Xt = torch.from_numpy(X_train).to(device)
|
||||
yt = torch.from_numpy(y_train).to(device)
|
||||
Xv = torch.from_numpy(X_val).to(device)
|
||||
yv = torch.from_numpy(y_val).to(device)
|
||||
|
||||
model = CountNet().to(device)
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
|
||||
sched = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(opt, T_0=50, T_mult=1)
|
||||
|
||||
n_train = X_train.shape[0]
|
||||
best_eval_acc = 0.0
|
||||
best_state = None
|
||||
|
||||
for epoch in range(args.epochs):
|
||||
model.train()
|
||||
perm = torch.randperm(n_train, device=device)
|
||||
train_loss = 0.0
|
||||
train_correct = 0
|
||||
n_batches = 0
|
||||
for i in range(0, n_train, args.batch_size):
|
||||
idx = perm[i : i + args.batch_size]
|
||||
xb = Xt[idx]
|
||||
yb = yt[idx]
|
||||
opt.zero_grad()
|
||||
count_logits, conf_logits = model(xb)
|
||||
ce = F.cross_entropy(count_logits, yb, weight=cls_weight_t)
|
||||
with torch.no_grad():
|
||||
pred = count_logits.argmax(dim=1)
|
||||
correct_indicator = (pred == yb).float().unsqueeze(1)
|
||||
bce = F.binary_cross_entropy_with_logits(conf_logits, correct_indicator)
|
||||
with torch.no_grad():
|
||||
conf_sigm = torch.sigmoid(conf_logits)
|
||||
brier = ((conf_sigm - correct_indicator) ** 2).mean()
|
||||
loss = ce + 0.3 * bce + 0.1 * brier
|
||||
loss.backward()
|
||||
opt.step()
|
||||
train_loss += loss.item()
|
||||
train_correct += (pred == yb).sum().item()
|
||||
n_batches += 1
|
||||
|
||||
sched.step()
|
||||
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_v, _ = model(Xv)
|
||||
eval_pred = cl_v.argmax(dim=1)
|
||||
eval_acc = (eval_pred == yv).float().mean().item()
|
||||
|
||||
if eval_acc > best_eval_acc:
|
||||
best_eval_acc = eval_acc
|
||||
best_state = {k: v.detach().cpu().clone() for k, v in model.state_dict().items()}
|
||||
|
||||
# Restore best checkpoint and final eval
|
||||
if best_state is not None:
|
||||
model.load_state_dict(best_state)
|
||||
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_v, conf_v = model(Xv)
|
||||
pred_v = cl_v.argmax(dim=1)
|
||||
acc = (pred_v == yv).float().mean().item()
|
||||
within1 = ((pred_v - yv).abs() <= 1).float().mean().item()
|
||||
mae = (pred_v - yv).abs().float().mean().item()
|
||||
|
||||
# Per-class accuracy
|
||||
per_class = {}
|
||||
for k in range(COUNT_CLASSES):
|
||||
mask = yv == k
|
||||
n = mask.sum().item()
|
||||
if n > 0:
|
||||
per_class[k] = {
|
||||
"support": int(n),
|
||||
"accuracy": ((pred_v == yv) & mask).sum().item() / n,
|
||||
}
|
||||
|
||||
# Spearman
|
||||
conf_sigm = torch.sigmoid(conf_v).squeeze(-1)
|
||||
correct = (pred_v == yv).float()
|
||||
c_rank = conf_sigm.argsort().argsort().float()
|
||||
r_rank = correct.argsort().argsort().float()
|
||||
c_centered = c_rank - c_rank.mean()
|
||||
r_centered = r_rank - r_rank.mean()
|
||||
denom = (c_centered.norm() * r_centered.norm()).item()
|
||||
spearman = (c_centered * r_centered).sum().item() / denom if denom > 0 else 0.0
|
||||
|
||||
fold_results.append({
|
||||
"fold": fold_idx + 1,
|
||||
"accuracy": acc,
|
||||
"within_pm1": within1,
|
||||
"mae": mae,
|
||||
"spearman": spearman,
|
||||
"per_class_accuracy": per_class,
|
||||
})
|
||||
print(f" accuracy={acc:.3f} within±1={within1:.3f} mae={mae:.3f} spearman={spearman:.3f}")
|
||||
|
||||
# K-fold summary
|
||||
total_time = time.perf_counter() - overall_t0
|
||||
accs = [r["accuracy"] for r in fold_results]
|
||||
within1s = [r["within_pm1"] for r in fold_results]
|
||||
maes = [r["mae"] for r in fold_results]
|
||||
spears = [r["spearman"] for r in fold_results]
|
||||
|
||||
print(f"\n=== {args.k_fold}-fold summary ({total_time:.1f} s) ===")
|
||||
print(f" accuracy: {np.mean(accs):.3f} ± {np.std(accs):.3f}")
|
||||
print(f" within ±1: {np.mean(within1s):.3f} ± {np.std(within1s):.3f}")
|
||||
print(f" MAE: {np.mean(maes):.3f} ± {np.std(maes):.3f}")
|
||||
print(f" conf↔correct Spearman: {np.mean(spears):.3f} ± {np.std(spears):.3f}")
|
||||
|
||||
# Per-class summary across folds
|
||||
for k in range(COUNT_CLASSES):
|
||||
accs_k = [r["per_class_accuracy"].get(k, {}).get("accuracy", 0.0) for r in fold_results]
|
||||
n_k = [r["per_class_accuracy"].get(k, {}).get("support", 0) for r in fold_results]
|
||||
if any(n > 0 for n in n_k):
|
||||
print(f" class {k}: {np.mean(accs_k):.3f} mean accuracy (support: {n_k})")
|
||||
|
||||
# Write k-fold results to JSON
|
||||
results = {
|
||||
"mode": "k_fold_cv",
|
||||
"k": args.k_fold,
|
||||
"backend": "pytorch-cuda" if device.type == "cuda" else "pytorch-cpu",
|
||||
"total_time_s": total_time,
|
||||
"fold_results": fold_results,
|
||||
"summary": {
|
||||
"mean_accuracy": float(np.mean(accs)),
|
||||
"std_accuracy": float(np.std(accs)),
|
||||
"mean_within_pm1": float(np.mean(within1s)),
|
||||
"std_within_pm1": float(np.std(within1s)),
|
||||
"mean_mae": float(np.mean(maes)),
|
||||
"std_mae": float(np.std(maes)),
|
||||
"mean_spearman": float(np.mean(spears)),
|
||||
"std_spearman": float(np.std(spears)),
|
||||
},
|
||||
"hyperparameters": {
|
||||
"optimizer": "AdamW",
|
||||
"lr": args.lr,
|
||||
"weight_decay": args.weight_decay,
|
||||
"batch_size": args.batch_size,
|
||||
"schedule": "cosine_warm_restarts",
|
||||
"epochs": args.epochs,
|
||||
},
|
||||
}
|
||||
Path(args.out_results).write_text(json.dumps(results, indent=2))
|
||||
print(f"\nwrote {args.out_results}")
|
||||
return
|
||||
|
||||
# ---------------------------------------------------------------
|
||||
# v0.0.2 training path: random 80/20 + label smoothing + early
|
||||
# stopping + class-balanced batch sampling + temperature scaling.
|
||||
# ---------------------------------------------------------------
|
||||
if args.v2:
|
||||
rng = np.random.default_rng(seed=42)
|
||||
idx = np.arange(X.shape[0])
|
||||
rng.shuffle(idx)
|
||||
n_eval = int(round(0.2 * X.shape[0]))
|
||||
eval_idx, train_idx = idx[:n_eval], idx[n_eval:]
|
||||
X_train, X_eval = X[train_idx], X[eval_idx]
|
||||
y_train, y_eval = y[train_idx], y[eval_idx]
|
||||
X_train, X_eval = standardise(X_train, X_eval)
|
||||
print(f"v0.0.2 mode — random 80/20 split: train={len(y_train)} eval={len(y_eval)}")
|
||||
print(f" train class dist: {dict(Counter(y_train.tolist()).most_common())}")
|
||||
print(f" eval class dist: {dict(Counter(y_eval.tolist()).most_common())}")
|
||||
|
||||
Xt = torch.from_numpy(X_train).to(device)
|
||||
yt = torch.from_numpy(y_train).to(device)
|
||||
Xe = torch.from_numpy(X_eval).to(device)
|
||||
ye = torch.from_numpy(y_eval).to(device)
|
||||
|
||||
# Class-balanced sampler: for each batch, sample with replacement
|
||||
# so each class has equal expected count regardless of dataset
|
||||
# distribution. With our ~533/544 split this is nearly a no-op
|
||||
# but it generalises to imbalanced multi-room data later.
|
||||
cls_counts = np.bincount(y_train, minlength=COUNT_CLASSES).astype(np.float32)
|
||||
cls_counts = np.where(cls_counts > 0, cls_counts, 1.0)
|
||||
per_sample_weight = (1.0 / cls_counts[y_train])
|
||||
per_sample_weight_t = torch.from_numpy(per_sample_weight.astype(np.float32)).to(device)
|
||||
|
||||
model = CountNet().to(device)
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
|
||||
sched = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(opt, T_0=50, T_mult=1)
|
||||
|
||||
n_train = X_train.shape[0]
|
||||
batches_per_epoch = max(1, n_train // args.batch_size)
|
||||
epoch_losses = []
|
||||
t0 = time.perf_counter()
|
||||
best_eval_acc = 0.0
|
||||
best_state = None
|
||||
epochs_without_improvement = 0
|
||||
|
||||
for epoch in range(args.epochs):
|
||||
model.train()
|
||||
train_loss = 0.0; train_correct = 0; n_batches = 0
|
||||
for _ in range(batches_per_epoch):
|
||||
# Balanced sample with replacement
|
||||
idx_t = torch.multinomial(per_sample_weight_t, args.batch_size, replacement=True)
|
||||
xb = Xt[idx_t]; yb = yt[idx_t]
|
||||
opt.zero_grad()
|
||||
count_logits, conf_logits = model(xb)
|
||||
ce = F.cross_entropy(count_logits, yb, label_smoothing=args.label_smoothing)
|
||||
with torch.no_grad():
|
||||
pred = count_logits.argmax(dim=1)
|
||||
correct_indicator = (pred == yb).float().unsqueeze(1)
|
||||
bce = F.binary_cross_entropy_with_logits(conf_logits, correct_indicator)
|
||||
with torch.no_grad():
|
||||
conf_sigm = torch.sigmoid(conf_logits)
|
||||
brier = ((conf_sigm - correct_indicator) ** 2).mean()
|
||||
loss = ce + 0.3 * bce + 0.1 * brier
|
||||
loss.backward()
|
||||
opt.step()
|
||||
train_loss += loss.item()
|
||||
train_correct += (pred == yb).sum().item()
|
||||
n_batches += 1
|
||||
sched.step()
|
||||
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_e, _ = model(Xe)
|
||||
eval_loss = F.cross_entropy(cl_e, ye).item()
|
||||
eval_pred = cl_e.argmax(dim=1)
|
||||
eval_acc = (eval_pred == ye).float().mean().item()
|
||||
epoch_losses.append({
|
||||
"epoch": epoch,
|
||||
"train_loss": train_loss / max(1, n_batches),
|
||||
"train_acc": train_correct / max(1, n_batches * args.batch_size),
|
||||
"eval_loss": eval_loss,
|
||||
"eval_acc": eval_acc,
|
||||
})
|
||||
if eval_acc > best_eval_acc:
|
||||
best_eval_acc = eval_acc
|
||||
best_state = {k: v.detach().cpu().clone() for k, v in model.state_dict().items()}
|
||||
epochs_without_improvement = 0
|
||||
else:
|
||||
epochs_without_improvement += 1
|
||||
|
||||
if epoch < 5 or epoch % 25 == 0:
|
||||
print(f"epoch {epoch:3d} train_loss={train_loss/n_batches:.4f} "
|
||||
f"train_acc={train_correct/(n_batches*args.batch_size):.3f} "
|
||||
f"eval_loss={eval_loss:.4f} eval_acc={eval_acc:.3f} "
|
||||
f"epochs_no_improve={epochs_without_improvement}")
|
||||
if epochs_without_improvement >= args.patience:
|
||||
print(f"early stopping at epoch {epoch} (no improvement for {args.patience} epochs)")
|
||||
break
|
||||
|
||||
train_time = time.perf_counter() - t0
|
||||
print(f"\ntrained {epoch + 1} epochs in {train_time:.1f} s (best eval_acc {best_eval_acc:.3f})")
|
||||
if best_state is not None:
|
||||
model.load_state_dict(best_state)
|
||||
|
||||
# Temperature scaling on the confidence head — fit a scalar T s.t.
|
||||
# sigmoid(conf_logits / T) is best-calibrated on the eval set.
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_e, conf_e = model(Xe)
|
||||
pred_e = cl_e.argmax(dim=1)
|
||||
correct_indicator = (pred_e == ye).float()
|
||||
# 1D optimisation over T via LBFGS.
|
||||
T = torch.nn.Parameter(torch.ones(1, device=device))
|
||||
opt_t = torch.optim.LBFGS([T], lr=0.1, max_iter=50)
|
||||
def eval_t():
|
||||
opt_t.zero_grad()
|
||||
scaled = conf_e.squeeze(-1) / T
|
||||
loss_t = F.binary_cross_entropy_with_logits(scaled, correct_indicator)
|
||||
loss_t.backward()
|
||||
return loss_t
|
||||
opt_t.step(eval_t)
|
||||
T_val = float(T.detach().cpu().item())
|
||||
print(f" temperature scale T = {T_val:.4f}")
|
||||
|
||||
# Final eval with temperature applied.
|
||||
with torch.no_grad():
|
||||
cl_e, conf_e = model(Xe)
|
||||
probs_e = F.softmax(cl_e, dim=1)
|
||||
pred_e = cl_e.argmax(dim=1)
|
||||
acc = (pred_e == ye).float().mean().item()
|
||||
within1 = ((pred_e - ye).abs() <= 1).float().mean().item()
|
||||
mae = (pred_e - ye).abs().float().mean().item()
|
||||
per_class = {}
|
||||
for k in range(COUNT_CLASSES):
|
||||
mask = ye == k
|
||||
n = mask.sum().item()
|
||||
if n > 0:
|
||||
per_class[k] = {
|
||||
"support": int(n),
|
||||
"accuracy": ((pred_e == ye) & mask).sum().item() / n,
|
||||
}
|
||||
conf_sigm = torch.sigmoid(conf_e.squeeze(-1) / T_val)
|
||||
correct = (pred_e == ye).float()
|
||||
c_rank = conf_sigm.argsort().argsort().float()
|
||||
r_rank = correct.argsort().argsort().float()
|
||||
c_centered = c_rank - c_rank.mean()
|
||||
r_centered = r_rank - r_rank.mean()
|
||||
denom = (c_centered.norm() * r_centered.norm()).item()
|
||||
spearman = (c_centered * r_centered).sum().item() / denom if denom > 0 else 0.0
|
||||
|
||||
print(f"\n=== v0.0.2 final eval ===")
|
||||
print(f" accuracy: {acc:.3f}")
|
||||
print(f" within ±1: {within1:.3f}")
|
||||
print(f" MAE: {mae:.3f}")
|
||||
print(f" conf↔correct Spearman (post-temp): {spearman:.3f}")
|
||||
for k, v in per_class.items():
|
||||
print(f" class {k}: {v['accuracy']:.3f} accuracy on {v['support']} samples")
|
||||
|
||||
write_safetensors(model, Path(args.out_safetensors))
|
||||
# Also append the temperature scalar so the cog can apply it.
|
||||
# We add it by appending to the safetensors file using the
|
||||
# write_safetensors helper but with the temperature recorded
|
||||
# as a separate file alongside (count_v1.temperature.txt) for
|
||||
# consumption by the Rust cog inference path.
|
||||
Path(args.out_safetensors + ".temperature").write_text(f"{T_val}\n")
|
||||
print(f"wrote {args.out_safetensors} ({Path(args.out_safetensors).stat().st_size} bytes)")
|
||||
print(f"wrote {args.out_safetensors}.temperature ({T_val})")
|
||||
|
||||
# ONNX
|
||||
dummy = torch.zeros(1, N_SUB, N_FRAMES, device=device)
|
||||
try:
|
||||
torch.onnx.export(model, dummy, args.out_onnx, opset_version=18,
|
||||
input_names=["csi_window"],
|
||||
output_names=["count_logits", "conf_logits"],
|
||||
dynamic_axes={"csi_window": {0: "batch"},
|
||||
"count_logits": {0: "batch"},
|
||||
"conf_logits": {0: "batch"}},
|
||||
export_params=True, do_constant_folding=True)
|
||||
print(f"wrote {args.out_onnx} ({Path(args.out_onnx).stat().st_size} bytes)")
|
||||
except Exception as e:
|
||||
print(f"WARN: ONNX export failed: {e}")
|
||||
|
||||
results = {
|
||||
"mode": "v0.0.2",
|
||||
"backend": "pytorch-cuda" if device.type == "cuda" else "pytorch-cpu",
|
||||
"epochs_trained": epoch + 1,
|
||||
"train_time_s": train_time,
|
||||
"best_eval_acc": best_eval_acc,
|
||||
"final_eval_acc": acc,
|
||||
"final_eval_within_pm1": within1,
|
||||
"final_eval_mae": mae,
|
||||
"temperature_scale": T_val,
|
||||
"conf_correctness_spearman_post_temp": spearman,
|
||||
"per_class_accuracy": per_class,
|
||||
"hyperparameters": {
|
||||
"optimizer": "AdamW",
|
||||
"lr": args.lr,
|
||||
"weight_decay": args.weight_decay,
|
||||
"batch_size": args.batch_size,
|
||||
"schedule": "cosine_warm_restarts",
|
||||
"epochs_max": args.epochs,
|
||||
"label_smoothing": args.label_smoothing,
|
||||
"patience": args.patience,
|
||||
"split": "random_80_20_seed_42",
|
||||
"balanced_sampler": True,
|
||||
"temperature_scaling": True,
|
||||
},
|
||||
"epoch_losses": epoch_losses,
|
||||
}
|
||||
Path(args.out_results).write_text(json.dumps(results, indent=2))
|
||||
print(f"wrote {args.out_results}")
|
||||
return
|
||||
|
||||
# Original temporal-split mode (kept for v0.0.1 reproducibility).
|
||||
X_train, y_train, X_eval, y_eval = temporal_split(X, y, eval_frac=0.2)
|
||||
X_train, X_eval = standardise(X_train, X_eval)
|
||||
|
||||
# Re-balance via class weights — handles the 50/50 split fine
|
||||
# but also makes the loss correct under future imbalanced data.
|
||||
cls_counts = np.bincount(y_train, minlength=COUNT_CLASSES).astype(np.float32)
|
||||
cls_counts = np.where(cls_counts > 0, cls_counts, 1.0)
|
||||
cls_weight = (1.0 / cls_counts) / (1.0 / cls_counts).sum() * COUNT_CLASSES
|
||||
cls_weight_t = torch.from_numpy(cls_weight).to(device)
|
||||
print(f"class weights: {cls_weight.tolist()}")
|
||||
|
||||
Xt = torch.from_numpy(X_train).to(device)
|
||||
yt = torch.from_numpy(y_train).to(device)
|
||||
Xe = torch.from_numpy(X_eval).to(device)
|
||||
ye = torch.from_numpy(y_eval).to(device)
|
||||
|
||||
model = CountNet().to(device)
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=args.lr, weight_decay=args.weight_decay)
|
||||
sched = torch.optim.lr_scheduler.CosineAnnealingWarmRestarts(opt, T_0=50, T_mult=1)
|
||||
|
||||
n_train = X_train.shape[0]
|
||||
epoch_losses = []
|
||||
t0 = time.perf_counter()
|
||||
|
||||
best_eval_acc = 0.0
|
||||
best_state = None
|
||||
|
||||
for epoch in range(args.epochs):
|
||||
model.train()
|
||||
perm = torch.randperm(n_train, device=device)
|
||||
train_loss = 0.0
|
||||
train_correct = 0
|
||||
n_batches = 0
|
||||
for i in range(0, n_train, args.batch_size):
|
||||
idx = perm[i : i + args.batch_size]
|
||||
xb = Xt[idx]
|
||||
yb = yt[idx]
|
||||
opt.zero_grad()
|
||||
count_logits, conf_logits = model(xb)
|
||||
|
||||
# Categorical cross-entropy for count.
|
||||
ce = F.cross_entropy(count_logits, yb, weight=cls_weight_t)
|
||||
|
||||
# Confidence head: train against `argmax == truth` indicator.
|
||||
with torch.no_grad():
|
||||
pred = count_logits.argmax(dim=1)
|
||||
correct_indicator = (pred == yb).float().unsqueeze(1)
|
||||
bce = F.binary_cross_entropy_with_logits(conf_logits, correct_indicator)
|
||||
|
||||
# Brier-score uncertainty calibration on the conf head — sharpens
|
||||
# the calibration so the sigmoid output is a real probability.
|
||||
with torch.no_grad():
|
||||
conf_sigm = torch.sigmoid(conf_logits)
|
||||
brier = ((conf_sigm - correct_indicator) ** 2).mean()
|
||||
|
||||
loss = ce + 0.3 * bce + 0.1 * brier
|
||||
loss.backward()
|
||||
opt.step()
|
||||
|
||||
train_loss += loss.item()
|
||||
train_correct += (pred == yb).sum().item()
|
||||
n_batches += 1
|
||||
|
||||
sched.step()
|
||||
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_e, _ = model(Xe)
|
||||
eval_loss = F.cross_entropy(cl_e, ye, weight=cls_weight_t).item()
|
||||
eval_pred = cl_e.argmax(dim=1)
|
||||
eval_acc = (eval_pred == ye).float().mean().item()
|
||||
eval_within1 = ((eval_pred - ye).abs() <= 1).float().mean().item()
|
||||
|
||||
epoch_losses.append({
|
||||
"epoch": epoch,
|
||||
"train_loss": train_loss / n_batches,
|
||||
"train_acc": train_correct / n_train,
|
||||
"eval_loss": eval_loss,
|
||||
"eval_acc": eval_acc,
|
||||
"eval_within_pm1": eval_within1,
|
||||
})
|
||||
|
||||
if eval_acc > best_eval_acc:
|
||||
best_eval_acc = eval_acc
|
||||
best_state = {k: v.detach().cpu().clone() for k, v in model.state_dict().items()}
|
||||
|
||||
if epoch < 5 or epoch % 50 == 0 or epoch == args.epochs - 1:
|
||||
print(f"epoch {epoch:3d} train_loss={train_loss/n_batches:.4f} "
|
||||
f"train_acc={train_correct/n_train:.3f} "
|
||||
f"eval_loss={eval_loss:.4f} eval_acc={eval_acc:.3f} "
|
||||
f"within±1={eval_within1:.3f}")
|
||||
|
||||
train_time = time.perf_counter() - t0
|
||||
print(f"\ntrained {args.epochs} epochs in {train_time:.1f} s")
|
||||
print(f"best eval_acc: {best_eval_acc:.3f}")
|
||||
|
||||
# Restore best checkpoint
|
||||
if best_state is not None:
|
||||
model.load_state_dict(best_state)
|
||||
|
||||
# Eval breakdown
|
||||
model.eval()
|
||||
with torch.no_grad():
|
||||
cl_e, conf_e = model(Xe)
|
||||
probs_e = torch.softmax(cl_e, dim=1)
|
||||
pred_e = cl_e.argmax(dim=1)
|
||||
acc = (pred_e == ye).float().mean().item()
|
||||
within1 = ((pred_e - ye).abs() <= 1).float().mean().item()
|
||||
mae = (pred_e - ye).abs().float().mean().item()
|
||||
|
||||
# Per-class accuracy
|
||||
per_class = {}
|
||||
for k in range(COUNT_CLASSES):
|
||||
mask = ye == k
|
||||
n = mask.sum().item()
|
||||
if n > 0:
|
||||
per_class[k] = {
|
||||
"support": int(n),
|
||||
"accuracy": ((pred_e == ye) & mask).sum().item() / n,
|
||||
}
|
||||
|
||||
# Confidence-accuracy calibration: Spearman over (predicted-correct, confidence)
|
||||
conf_sigm = torch.sigmoid(conf_e).squeeze(-1)
|
||||
correct = (pred_e == ye).float()
|
||||
# Spearman = Pearson over ranks
|
||||
c_rank = conf_sigm.argsort().argsort().float()
|
||||
r_rank = correct.argsort().argsort().float()
|
||||
c_centered = c_rank - c_rank.mean()
|
||||
r_centered = r_rank - r_rank.mean()
|
||||
denom = (c_centered.norm() * r_centered.norm()).item()
|
||||
spearman = (c_centered * r_centered).sum().item() / denom if denom > 0 else 0.0
|
||||
|
||||
print(f"\n=== final eval ===")
|
||||
print(f" accuracy: {acc:.3f}")
|
||||
print(f" within ±1: {within1:.3f}")
|
||||
print(f" MAE: {mae:.3f}")
|
||||
print(f" conf↔correct Spearman: {spearman:.3f}")
|
||||
for k, v in per_class.items():
|
||||
print(f" class {k}: {v['accuracy']:.3f} accuracy on {v['support']} samples")
|
||||
|
||||
# Save safetensors
|
||||
write_safetensors(model, Path(args.out_safetensors))
|
||||
print(f"\nwrote {args.out_safetensors} ({Path(args.out_safetensors).stat().st_size} bytes)")
|
||||
|
||||
# ONNX export
|
||||
dummy = torch.zeros(1, N_SUB, N_FRAMES, device=device)
|
||||
try:
|
||||
torch.onnx.export(
|
||||
model, dummy, args.out_onnx,
|
||||
opset_version=18,
|
||||
input_names=["csi_window"],
|
||||
output_names=["count_logits", "conf_logits"],
|
||||
dynamic_axes={
|
||||
"csi_window": {0: "batch"},
|
||||
"count_logits": {0: "batch"},
|
||||
"conf_logits": {0: "batch"},
|
||||
},
|
||||
export_params=True,
|
||||
do_constant_folding=True,
|
||||
)
|
||||
print(f"wrote {args.out_onnx} ({Path(args.out_onnx).stat().st_size} bytes)")
|
||||
except Exception as e:
|
||||
print(f"WARN: ONNX export failed: {e}")
|
||||
|
||||
# Results JSON
|
||||
results = {
|
||||
"backend": "candle-cuda" if device.type == "cuda" else "candle-cpu",
|
||||
"device": str(device),
|
||||
"epochs": args.epochs,
|
||||
"train_time_s": train_time,
|
||||
"best_eval_acc": best_eval_acc,
|
||||
"final_eval_acc": acc,
|
||||
"final_eval_within_pm1": within1,
|
||||
"final_eval_mae": mae,
|
||||
"conf_correctness_spearman": spearman,
|
||||
"per_class_accuracy": per_class,
|
||||
"hyperparameters": {
|
||||
"optimizer": "AdamW",
|
||||
"lr": args.lr,
|
||||
"weight_decay": args.weight_decay,
|
||||
"batch_size": args.batch_size,
|
||||
"schedule": "cosine_warm_restarts",
|
||||
"epochs": args.epochs,
|
||||
"loss": "cross_entropy(count) + 0.3*bce(conf) + 0.1*brier(conf)",
|
||||
"z_score_normalisation": True,
|
||||
"class_weights": cls_weight.tolist(),
|
||||
},
|
||||
"epoch_losses": epoch_losses,
|
||||
}
|
||||
Path(args.out_results).write_text(json.dumps(results, indent=2))
|
||||
print(f"wrote {args.out_results} ({Path(args.out_results).stat().st_size} bytes)")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,18 @@
|
||||
/** @type {import('jest').Config} */
|
||||
export default {
|
||||
preset: "ts-jest/presets/default-esm",
|
||||
testEnvironment: "node",
|
||||
extensionsToTreatAsEsm: [".ts"],
|
||||
moduleNameMapper: {
|
||||
"^(\\.{1,2}/.*)\\.js$": "$1",
|
||||
},
|
||||
transform: {
|
||||
"^.+\\.tsx?$": [
|
||||
"ts-jest",
|
||||
{
|
||||
useESM: true,
|
||||
},
|
||||
],
|
||||
},
|
||||
testMatch: ["**/tests/**/*.test.ts"],
|
||||
};
|
||||
Generated
+3843
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,49 @@
|
||||
{
|
||||
"name": "@ruv/ruview-cli",
|
||||
"version": "0.0.1",
|
||||
"description": "RuView CLI — shell access to WiFi-DensePose sensing, inference, and training capabilities",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"bin": {
|
||||
"ruview": "dist/index.js"
|
||||
},
|
||||
"files": [
|
||||
"dist"
|
||||
],
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"dev": "tsc --watch",
|
||||
"test": "node --experimental-vm-modules node_modules/.bin/jest",
|
||||
"lint": "eslint src --ext .ts",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"keywords": [
|
||||
"ruview",
|
||||
"wifi",
|
||||
"csi",
|
||||
"pose-estimation",
|
||||
"cognitum",
|
||||
"cli"
|
||||
],
|
||||
"author": "ruv <ruv@ruv.net>",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"yargs": "^17.7.2"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^20.14.0",
|
||||
"@types/yargs": "^17.0.32",
|
||||
"jest": "^29.7.0",
|
||||
"ts-jest": "^29.1.0",
|
||||
"typescript": "^5.4.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"registry": "https://registry.npmjs.org/"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* Subprocess wrapper for Cognitum Cog binaries (CLI variant).
|
||||
* Mirrors tools/ruview-mcp/src/cog.ts.
|
||||
*/
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
|
||||
export type Result<T> = { ok: true; data: T } | { ok: false; error: string };
|
||||
|
||||
const COG_TIMEOUT_MS = 15_000;
|
||||
|
||||
export async function runCog(binary: string, args: string[]): Promise<Result<string>> {
|
||||
return new Promise((resolve) => {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
|
||||
const child = spawn(binary, args, {
|
||||
timeout: COG_TIMEOUT_MS,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
child.stdout?.on("data", (chunk: Buffer) => { stdout += chunk.toString(); });
|
||||
child.stderr?.on("data", (chunk: Buffer) => { stderr += chunk.toString(); });
|
||||
|
||||
child.on("error", (e) => {
|
||||
resolve(err(
|
||||
`Failed to launch "${binary}" (${args.join(" ")}): ${e.message}. ` +
|
||||
`Set RUVIEW_POSE_COG_BINARY / RUVIEW_COUNT_COG_BINARY or install the cog.`
|
||||
));
|
||||
});
|
||||
|
||||
child.on("close", (code) => {
|
||||
if (code !== 0) {
|
||||
resolve(err(`Cog "${binary} ${args.join(" ")}" exited with code ${code}. stderr: ${stderr.trim() || "(empty)"}`));
|
||||
} else {
|
||||
resolve({ ok: true, data: stdout });
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
function err(error: string): { ok: false; error: string } {
|
||||
return { ok: false, error };
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
/**
|
||||
* ruview cogs — Cognitum edge module registry commands.
|
||||
*
|
||||
* cogs list — list cogs from the registry (via sensing-server ADR-102 proxy).
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { sensingGet } from "../http.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function cogsCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"cogs <action>",
|
||||
"Edge module registry commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("action", {
|
||||
choices: ["list"] as const,
|
||||
description: "Action to perform",
|
||||
})
|
||||
.option("category", {
|
||||
type: "string",
|
||||
description:
|
||||
"Filter by category: health, security, building, retail, industrial, " +
|
||||
"research, ai, swarm, signal, network, developer",
|
||||
})
|
||||
.option("search", {
|
||||
type: "string",
|
||||
description: "Search substring matched against cog id and name (case-insensitive)",
|
||||
})
|
||||
.option("refresh", {
|
||||
type: "boolean",
|
||||
default: false,
|
||||
description: "Bypass the 1-hour registry cache",
|
||||
})
|
||||
.option("url", {
|
||||
type: "string",
|
||||
description: "Override the sensing-server URL",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
const baseUrl = (args["url"] as string | undefined) ?? config.sensingServerUrl;
|
||||
|
||||
if (args.action === "list") {
|
||||
const qs = args.refresh ? "?refresh=1" : "";
|
||||
const result = await sensingGet<{
|
||||
registry?: { cogs?: object[]; apps?: object[] };
|
||||
}>(baseUrl, `/api/v1/edge/registry${qs}`, config.apiToken);
|
||||
|
||||
if (!result.ok) {
|
||||
process.stderr.write(`[WARN] ${result.error}\n`);
|
||||
process.stdout.write(
|
||||
JSON.stringify({ ok: false, warn: true, error: result.error }) + "\n"
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const payload = result.data;
|
||||
let cogs: object[] =
|
||||
payload.registry?.cogs ?? payload.registry?.apps ?? [];
|
||||
|
||||
if (args.category) {
|
||||
const cat = (args.category as string).toLowerCase();
|
||||
cogs = cogs.filter(
|
||||
(c) =>
|
||||
(c as Record<string, unknown>)["category"]
|
||||
?.toString()
|
||||
.toLowerCase() === cat
|
||||
);
|
||||
}
|
||||
if (args.search) {
|
||||
const q = (args.search as string).toLowerCase();
|
||||
cogs = cogs.filter((c) => {
|
||||
const rec = c as Record<string, unknown>;
|
||||
return (
|
||||
rec["id"]?.toString().toLowerCase().includes(q) ||
|
||||
rec["name"]?.toString().toLowerCase().includes(q)
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
JSON.stringify({ ok: true, total: cogs.length, cogs }, null, 2) + "\n"
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,100 @@
|
||||
/**
|
||||
* ruview count — Person count commands.
|
||||
*
|
||||
* count infer — run single-shot person-count inference.
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { runCog } from "../cog.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function countCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"count <action>",
|
||||
"Person count commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("action", {
|
||||
choices: ["infer"] as const,
|
||||
description: "Action to perform",
|
||||
})
|
||||
.option("window", {
|
||||
type: "string",
|
||||
description: "Path to a CSI window JSON file (omit to use live sensing-server)",
|
||||
})
|
||||
.option("binary", {
|
||||
type: "string",
|
||||
description: "Path to cog-person-count binary (default: RUVIEW_COUNT_COG_BINARY)",
|
||||
})
|
||||
.option("max-persons", {
|
||||
type: "number",
|
||||
default: 7,
|
||||
description: "Upper bound on person count (1–7, default: 7)",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
const binary = (args["binary"] as string | undefined) ?? config.countCogBinary;
|
||||
|
||||
if (args.action === "infer") {
|
||||
const t0 = Date.now();
|
||||
const health = await runCog(binary, ["health"]);
|
||||
const latencyMs = Date.now() - t0;
|
||||
|
||||
if (!health.ok) {
|
||||
process.stderr.write(
|
||||
`[WARN] Cog health check failed: ${health.error}\n` +
|
||||
`Set RUVIEW_COUNT_COG_BINARY or install cog-person-count (ADR-103).\n`
|
||||
);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: health.error,
|
||||
result: { count: 0, confidence: 0, count_p95_low: 0, count_p95_high: 0, backend: "unavailable", latency_ms: 0 },
|
||||
}) + "\n"
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
let backend = "unknown";
|
||||
let count = 0;
|
||||
let confidence = 0;
|
||||
let p95Low = 0;
|
||||
let p95High = 0;
|
||||
|
||||
for (const line of health.data.split("\n")) {
|
||||
try {
|
||||
const ev = JSON.parse(line.trim()) as Record<string, unknown>;
|
||||
if (ev["event"] === "health.ok") {
|
||||
const fields = ev["fields"] as Record<string, unknown>;
|
||||
backend = String(fields["backend"] ?? "unknown");
|
||||
count = Number(fields["synthetic_count"] ?? 0);
|
||||
confidence = Number(fields["synthetic_confidence"] ?? 0);
|
||||
const p95 = fields["synthetic_p95_range"] as number[];
|
||||
p95Low = p95?.[0] ?? 0;
|
||||
p95High = p95?.[1] ?? 0;
|
||||
break;
|
||||
}
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note: "M2: real inference on synthetic CSI window via cog health check.",
|
||||
result: {
|
||||
ts: Date.now() / 1000,
|
||||
count,
|
||||
confidence,
|
||||
count_p95_low: p95Low,
|
||||
count_p95_high: p95High,
|
||||
backend,
|
||||
latency_ms: latencyMs,
|
||||
},
|
||||
}) + "\n"
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
/**
|
||||
* ruview csi — CSI frame commands.
|
||||
*
|
||||
* csi tail — stream live CSI frames from the sensing-server.
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { sensingGet } from "../http.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function csiCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"csi <action>",
|
||||
"CSI frame commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("action", {
|
||||
choices: ["tail"] as const,
|
||||
description: "Action to perform",
|
||||
})
|
||||
.option("url", {
|
||||
type: "string",
|
||||
description:
|
||||
"Sensing-server URL (default: RUVIEW_SENSING_SERVER_URL or http://localhost:3000)",
|
||||
})
|
||||
.option("interval", {
|
||||
type: "number",
|
||||
default: 500,
|
||||
description: "Polling interval in milliseconds (default: 500)",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
const baseUrl = (args["url"] as string | undefined) ?? config.sensingServerUrl;
|
||||
|
||||
if (args.action === "tail") {
|
||||
process.stderr.write(
|
||||
`[ruview csi tail] Streaming from ${baseUrl} every ${args.interval}ms. Ctrl-C to stop.\n`
|
||||
);
|
||||
|
||||
// Streaming poll loop.
|
||||
// eslint-disable-next-line no-constant-condition
|
||||
while (true) {
|
||||
const result = await sensingGet<object>(
|
||||
baseUrl,
|
||||
"/api/v1/sensing/latest",
|
||||
config.apiToken
|
||||
);
|
||||
|
||||
if (!result.ok) {
|
||||
process.stderr.write(
|
||||
`[WARN] ${result.error} — retrying in ${args.interval}ms\n`
|
||||
);
|
||||
} else {
|
||||
process.stdout.write(JSON.stringify(result.data) + "\n");
|
||||
}
|
||||
|
||||
await new Promise<void>((resolve) =>
|
||||
setTimeout(resolve, args.interval as number)
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
/**
|
||||
* ruview job — Job management commands.
|
||||
*
|
||||
* job status --id <job_id> — poll a background training job.
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { readFileSync, existsSync } from "node:fs";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function jobCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"job <action>",
|
||||
"Job management commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("action", {
|
||||
choices: ["status"] as const,
|
||||
description: "Action to perform",
|
||||
})
|
||||
.option("id", {
|
||||
type: "string",
|
||||
demandOption: true,
|
||||
description: "Job ID returned by ruview train count",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
|
||||
if (args.action === "status") {
|
||||
const jobId = args.id as string;
|
||||
const { default: path } = await import("node:path");
|
||||
const logPath = path.join(config.jobsDir, `${jobId}.log`);
|
||||
|
||||
if (!existsSync(logPath)) {
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
ok: false,
|
||||
error: `Job ${jobId} not found at ${logPath}. ` +
|
||||
"The CLI process that started the job may have been restarted.",
|
||||
}) + "\n"
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const content = readFileSync(logPath, "utf8");
|
||||
const lines = content.split("\n");
|
||||
const recentLog = lines.slice(Math.max(0, lines.length - 20));
|
||||
|
||||
// Derive status from the log content.
|
||||
let status: string = "running";
|
||||
if (content.includes("# exit code: 0")) {
|
||||
status = "done";
|
||||
} else if (content.includes("# exit code:") || content.includes("# ERROR:")) {
|
||||
status = "failed";
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
JSON.stringify(
|
||||
{
|
||||
ok: true,
|
||||
job_id: jobId,
|
||||
status,
|
||||
log_path: logPath,
|
||||
recent_log: recentLog,
|
||||
},
|
||||
null,
|
||||
2
|
||||
) + "\n"
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
/**
|
||||
* ruview pose — Pose estimation commands.
|
||||
*
|
||||
* pose infer — run single-shot 17-keypoint inference.
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { runCog } from "../cog.js";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function poseCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"pose <action>",
|
||||
"Pose estimation commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("action", {
|
||||
choices: ["infer"] as const,
|
||||
description: "Action to perform",
|
||||
})
|
||||
.option("window", {
|
||||
type: "string",
|
||||
description: "Path to a CSI window JSON file (omit to use live sensing-server)",
|
||||
})
|
||||
.option("binary", {
|
||||
type: "string",
|
||||
description: "Path to cog-pose-estimation binary (default: RUVIEW_POSE_COG_BINARY)",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
const binary = (args["binary"] as string | undefined) ?? config.poseCogBinary;
|
||||
|
||||
if (args.action === "infer") {
|
||||
const t0 = Date.now();
|
||||
const health = await runCog(binary, ["health"]);
|
||||
const latencyMs = Date.now() - t0;
|
||||
|
||||
if (!health.ok) {
|
||||
process.stderr.write(
|
||||
`[WARN] Cog health check failed: ${health.error}\n` +
|
||||
`Set RUVIEW_POSE_COG_BINARY or install cog-pose-estimation (ADR-101).\n`
|
||||
);
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: health.error,
|
||||
result: { n_persons: 0, persons: [], backend: "unavailable", latency_ms: 0 },
|
||||
}) + "\n"
|
||||
);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// Parse the health.ok event for real inference output.
|
||||
let backend = "unknown";
|
||||
let confidence = 0;
|
||||
for (const line of health.data.split("\n")) {
|
||||
try {
|
||||
const ev = JSON.parse(line.trim()) as Record<string, unknown>;
|
||||
if (ev["event"] === "health.ok") {
|
||||
const fields = ev["fields"] as Record<string, unknown>;
|
||||
backend = String(fields["backend"] ?? "unknown");
|
||||
confidence = Number(fields["synthetic_output_confidence"] ?? 0);
|
||||
break;
|
||||
}
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
|
||||
process.stdout.write(
|
||||
JSON.stringify({
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note: "M2: real inference on synthetic CSI window via cog health check.",
|
||||
result: {
|
||||
ts: Date.now() / 1000,
|
||||
n_persons: confidence > 0.1 ? 1 : 0,
|
||||
persons: confidence > 0.1 ? [{ keypoints: Array.from({ length: 17 }, (_, i) => [0.5, 0.1 + i * 0.05]), confidence }] : [],
|
||||
backend,
|
||||
latency_ms: latencyMs,
|
||||
},
|
||||
}) + "\n"
|
||||
);
|
||||
}
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* ruview train — Training commands.
|
||||
*
|
||||
* train count --paired <jsonl> — kick off a count-cog training run.
|
||||
*/
|
||||
|
||||
import type { Argv } from "yargs";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdirSync, appendFileSync, openSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import os from "node:os";
|
||||
import { spawn } from "node:child_process";
|
||||
import { loadConfig } from "../config.js";
|
||||
|
||||
export function trainCommand(cli: Argv): void {
|
||||
cli.command(
|
||||
"train <task>",
|
||||
"Training commands",
|
||||
(y) =>
|
||||
y
|
||||
.positional("task", {
|
||||
choices: ["count"] as const,
|
||||
description: "Which cog to train",
|
||||
})
|
||||
.option("paired", {
|
||||
type: "string",
|
||||
demandOption: true,
|
||||
description:
|
||||
"Path to the paired JSONL training file (produced by scripts/align-ground-truth.js)",
|
||||
})
|
||||
.option("epochs", {
|
||||
type: "number",
|
||||
default: 400,
|
||||
description: "Training epochs (default: 400)",
|
||||
})
|
||||
.option("lr", {
|
||||
type: "number",
|
||||
default: 1e-3,
|
||||
description: "Initial learning rate (default: 0.001)",
|
||||
})
|
||||
.option("output-dir", {
|
||||
type: "string",
|
||||
description: "Output directory for model artifacts",
|
||||
}),
|
||||
async (args) => {
|
||||
const config = loadConfig();
|
||||
const jobId = randomUUID();
|
||||
const logDir = config.jobsDir;
|
||||
mkdirSync(logDir, { recursive: true });
|
||||
const logPath = path.join(logDir, `${jobId}.log`);
|
||||
const queuedAt = Date.now() / 1000;
|
||||
|
||||
const outputDir =
|
||||
(args["output-dir"] as string | undefined) ??
|
||||
"v2/crates/cog-person-count/cog/artifacts";
|
||||
|
||||
const header = [
|
||||
`# RuView training job ${jobId}`,
|
||||
`# started: ${new Date().toISOString()}`,
|
||||
`# task: ${args.task}`,
|
||||
`# paired: ${args.paired}`,
|
||||
`# epochs: ${args.epochs}`,
|
||||
`# lr: ${args.lr}`,
|
||||
`# output-dir: ${outputDir}`,
|
||||
"",
|
||||
].join("\n");
|
||||
appendFileSync(logPath, header);
|
||||
|
||||
const logFdOut = openSync(logPath, "a");
|
||||
const logFdErr = openSync(logPath, "a");
|
||||
|
||||
const cargoArgs = [
|
||||
"run",
|
||||
"--release",
|
||||
"-p",
|
||||
"wifi-densepose-train",
|
||||
"--",
|
||||
"--task",
|
||||
"count",
|
||||
"--paired",
|
||||
args.paired as string,
|
||||
"--epochs",
|
||||
String(args.epochs),
|
||||
"--lr",
|
||||
String(args.lr),
|
||||
"--output-dir",
|
||||
outputDir,
|
||||
];
|
||||
|
||||
const child = spawn("cargo", cargoArgs, {
|
||||
detached: true,
|
||||
stdio: ["ignore", logFdOut, logFdErr],
|
||||
});
|
||||
child.unref();
|
||||
|
||||
child.on("error", (e) => {
|
||||
appendFileSync(logPath, `\n# ERROR: ${e.message}\n`);
|
||||
});
|
||||
child.on("close", (code) => {
|
||||
appendFileSync(logPath, `\n# exit code: ${code}\n`);
|
||||
});
|
||||
|
||||
process.stdout.write(
|
||||
JSON.stringify(
|
||||
{
|
||||
ok: true,
|
||||
job_id: jobId,
|
||||
status: "running",
|
||||
log_path: logPath,
|
||||
queued_at: queuedAt,
|
||||
note: `Poll with: ruview job status --id ${jobId}`,
|
||||
},
|
||||
null,
|
||||
2
|
||||
) + "\n"
|
||||
);
|
||||
}
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* Configuration loader for the RuView CLI.
|
||||
* Mirrors tools/ruview-mcp/src/config.ts — sourced from environment variables.
|
||||
*/
|
||||
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
|
||||
export interface RuviewCliConfig {
|
||||
sensingServerUrl: string;
|
||||
apiToken: string | undefined;
|
||||
poseCogBinary: string;
|
||||
countCogBinary: string;
|
||||
jobsDir: string;
|
||||
}
|
||||
|
||||
function envOrDefault(key: string, fallback: string): string {
|
||||
return process.env[key] ?? fallback;
|
||||
}
|
||||
|
||||
export function loadConfig(): RuviewCliConfig {
|
||||
return {
|
||||
sensingServerUrl: envOrDefault(
|
||||
"RUVIEW_SENSING_SERVER_URL",
|
||||
"http://localhost:3000"
|
||||
),
|
||||
apiToken: process.env["RUVIEW_API_TOKEN"],
|
||||
poseCogBinary: envOrDefault("RUVIEW_POSE_COG_BINARY", "cog-pose-estimation"),
|
||||
countCogBinary: envOrDefault("RUVIEW_COUNT_COG_BINARY", "cog-person-count"),
|
||||
jobsDir: envOrDefault(
|
||||
"RUVIEW_JOBS_DIR",
|
||||
path.join(os.homedir(), ".ruview", "jobs")
|
||||
),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
/**
|
||||
* Lightweight HTTP client (re-used in CLI commands).
|
||||
* Identical to tools/ruview-mcp/src/http.ts but kept separate to avoid a
|
||||
* workspace dependency — both packages are standalone and independently publishable.
|
||||
*/
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 10_000;
|
||||
|
||||
export type Ok<T> = { ok: true; data: T };
|
||||
export type Err = { ok: false; error: string };
|
||||
export type Result<T> = Ok<T> | Err;
|
||||
|
||||
export function ok<T>(data: T): Ok<T> {
|
||||
return { ok: true, data };
|
||||
}
|
||||
|
||||
export function err(error: string): Err {
|
||||
return { ok: false, error };
|
||||
}
|
||||
|
||||
export async function sensingGet<T>(
|
||||
baseUrl: string,
|
||||
path: string,
|
||||
token: string | undefined
|
||||
): Promise<Result<T>> {
|
||||
const url = `${baseUrl.replace(/\/$/, "")}${path}`;
|
||||
const headers: Record<string, string> = { Accept: "application/json" };
|
||||
if (token) headers["Authorization"] = `Bearer ${token}`;
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
|
||||
try {
|
||||
const res = await fetch(url, { headers, signal: controller.signal });
|
||||
clearTimeout(timer);
|
||||
if (!res.ok) {
|
||||
return err(`HTTP ${res.status} from ${url}: ${await res.text().catch(() => "(no body)")}`);
|
||||
}
|
||||
let body: unknown;
|
||||
try {
|
||||
body = await res.json();
|
||||
} catch {
|
||||
return err(`Non-JSON response from ${url}`);
|
||||
}
|
||||
return ok(body as T);
|
||||
} catch (e: unknown) {
|
||||
clearTimeout(timer);
|
||||
if (e instanceof Error && e.name === "AbortError") {
|
||||
return err(`Request to ${url} timed out after ${REQUEST_TIMEOUT_MS}ms`);
|
||||
}
|
||||
return err(`Network error fetching ${url}: ${String(e)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* @ruv/ruview-cli — RuView CLI
|
||||
*
|
||||
* Shell access to RuView sensing, inference, and training capabilities.
|
||||
*
|
||||
* Subcommands:
|
||||
* ruview csi tail [--url <url>] stream live CSI frames
|
||||
* ruview pose infer [--window <path>] 17-keypoint pose estimation
|
||||
* ruview count infer [--window <path>] person-count inference
|
||||
* ruview cogs list [--category <cat>] [--search q] list edge module registry
|
||||
* ruview train count --paired <jsonl> kick off count-cog training
|
||||
* ruview job status --id <job_id> poll a training job
|
||||
*
|
||||
* All subcommands write JSON to stdout and exit 0 on success.
|
||||
* WARN-level outputs write to stderr; the exit code is still 0 so pipelines
|
||||
* are not broken by a temporarily unreachable sensing-server.
|
||||
*
|
||||
* Usage:
|
||||
* npx ruview --version
|
||||
* npx ruview csi tail
|
||||
* npx ruview pose infer --window ./window.json
|
||||
* RUVIEW_SENSING_SERVER_URL=http://cognitum-v0:3000 npx ruview cogs list
|
||||
*
|
||||
* See ADR-104 for the full design rationale and security model.
|
||||
*/
|
||||
|
||||
import yargs from "yargs";
|
||||
import { hideBin } from "yargs/helpers";
|
||||
import { csiCommand } from "./commands/csi.js";
|
||||
import { poseCommand } from "./commands/pose.js";
|
||||
import { countCommand } from "./commands/count.js";
|
||||
import { cogsCommand } from "./commands/cogs.js";
|
||||
import { trainCommand } from "./commands/train.js";
|
||||
import { jobCommand } from "./commands/job.js";
|
||||
|
||||
const cli = yargs(hideBin(process.argv))
|
||||
.scriptName("ruview")
|
||||
.version("0.0.1")
|
||||
.usage("$0 <command> [options]")
|
||||
.strict()
|
||||
.help()
|
||||
.wrap(100);
|
||||
|
||||
// Register all top-level commands.
|
||||
csiCommand(cli);
|
||||
poseCommand(cli);
|
||||
countCommand(cli);
|
||||
cogsCommand(cli);
|
||||
trainCommand(cli);
|
||||
jobCommand(cli);
|
||||
|
||||
cli.demandCommand(1, "Specify a subcommand. Use --help for a list.").parse();
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"lib": ["ES2022"],
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"noImplicitOverride": true,
|
||||
"noPropertyAccessFromIndexSignature": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": ["src"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
/** @type {import('jest').Config} */
|
||||
export default {
|
||||
preset: "ts-jest/presets/default-esm",
|
||||
testEnvironment: "node",
|
||||
extensionsToTreatAsEsm: [".ts"],
|
||||
moduleNameMapper: {
|
||||
"^(\\.{1,2}/.*)\\.js$": "$1",
|
||||
},
|
||||
transform: {
|
||||
"^.+\\.tsx?$": [
|
||||
"ts-jest",
|
||||
{
|
||||
useESM: true,
|
||||
tsconfig: "tests/tsconfig.json",
|
||||
},
|
||||
],
|
||||
},
|
||||
testMatch: ["**/tests/**/*.test.ts"],
|
||||
collectCoverageFrom: ["src/**/*.ts", "!src/**/*.d.ts"],
|
||||
};
|
||||
Generated
+5133
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,51 @@
|
||||
{
|
||||
"name": "@ruv/ruview-mcp",
|
||||
"version": "0.0.1",
|
||||
"description": "RuView MCP server — expose WiFi-DensePose sensing capabilities as MCP tools for Claude Code, Cursor, and other MCP-compatible agents",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
"bin": {
|
||||
"ruview-mcp": "dist/index.js"
|
||||
},
|
||||
"files": [
|
||||
"dist"
|
||||
],
|
||||
"scripts": {
|
||||
"build": "tsc",
|
||||
"dev": "tsc --watch",
|
||||
"start": "node dist/index.js",
|
||||
"test": "node --experimental-vm-modules node_modules/jest/bin/jest.js --forceExit",
|
||||
"lint": "eslint src --ext .ts",
|
||||
"typecheck": "tsc --noEmit"
|
||||
},
|
||||
"keywords": [
|
||||
"mcp",
|
||||
"ruview",
|
||||
"wifi",
|
||||
"csi",
|
||||
"pose-estimation",
|
||||
"cognitum"
|
||||
],
|
||||
"author": "ruv <ruv@ruv.net>",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"@modelcontextprotocol/sdk": "^1.0.0",
|
||||
"zod": "^3.23.8"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/jest": "^30.0.0",
|
||||
"@types/node": "^20.14.0",
|
||||
"jest": "^29.7.0",
|
||||
"ts-jest": "^29.1.0",
|
||||
"typescript": "^5.4.5"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20.0.0"
|
||||
},
|
||||
"publishConfig": {
|
||||
"access": "public",
|
||||
"registry": "https://registry.npmjs.org/"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
/**
|
||||
* Subprocess wrapper for Cognitum Cog binaries.
|
||||
*
|
||||
* The cog binaries implement the ADR-100 runtime contract:
|
||||
* cog-<id> version
|
||||
* cog-<id> manifest
|
||||
* cog-<id> health
|
||||
* cog-<id> run --config <path>
|
||||
*
|
||||
* This module shells out to those binaries. If the binary is absent or returns
|
||||
* a non-zero exit code, the call fails-open with a WARN-level structured error
|
||||
* (same pattern cog-pose-estimation uses for missing model weights).
|
||||
*/
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import type { Result } from "./http.js";
|
||||
import { ok, err } from "./http.js";
|
||||
|
||||
const COG_TIMEOUT_MS = 15_000;
|
||||
|
||||
/**
|
||||
* Run a cog binary with the given subcommand arguments.
|
||||
* Returns stdout as a string on success, or an error message.
|
||||
*/
|
||||
export async function runCog(
|
||||
binary: string,
|
||||
args: string[]
|
||||
): Promise<Result<string>> {
|
||||
return new Promise((resolve) => {
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
|
||||
const child = spawn(binary, args, {
|
||||
timeout: COG_TIMEOUT_MS,
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
|
||||
child.stdout?.on("data", (chunk: Buffer) => {
|
||||
stdout += chunk.toString();
|
||||
});
|
||||
child.stderr?.on("data", (chunk: Buffer) => {
|
||||
stderr += chunk.toString();
|
||||
});
|
||||
|
||||
child.on("error", (e) => {
|
||||
resolve(
|
||||
err(
|
||||
`Failed to launch cog binary "${binary}" (${args.join(" ")}): ${e.message}. ` +
|
||||
`Set RUVIEW_POSE_COG_BINARY / RUVIEW_COUNT_COG_BINARY to the installed path, ` +
|
||||
`or install the cog on the Cognitum appliance first.`
|
||||
)
|
||||
);
|
||||
});
|
||||
|
||||
child.on("close", (code) => {
|
||||
if (code !== 0) {
|
||||
resolve(
|
||||
err(
|
||||
`Cog "${binary} ${args.join(" ")}" exited with code ${code}. ` +
|
||||
`stderr: ${stderr.trim() || "(empty)"}`
|
||||
)
|
||||
);
|
||||
} else {
|
||||
resolve(ok(stdout));
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* Call `cog-<id> health` and return the exit code + output.
|
||||
*/
|
||||
export async function cogHealth(binary: string): Promise<Result<string>> {
|
||||
return runCog(binary, ["health"]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Call `cog-<id> version` and return the version string.
|
||||
*/
|
||||
export async function cogVersion(binary: string): Promise<Result<string>> {
|
||||
return runCog(binary, ["version"]);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a cog inference with a synthetic CSI window piped via a temp config.
|
||||
*
|
||||
* The ADR-100 contract doesn't define a single-shot "infer" subcommand — the
|
||||
* cog's `run` subcommand is long-running. Instead, we:
|
||||
* 1. Verify health returns 0.
|
||||
* 2. Emit a WARN explaining that single-shot inference requires a live
|
||||
* sensing-server connection, then return a stub result.
|
||||
*
|
||||
* Full single-shot inference (M2 milestone) will use the sensing-server's
|
||||
* `/api/v1/sensing/latest` to build a real CSI window and feed it through the
|
||||
* cog via a short-lived `run` session.
|
||||
*/
|
||||
export async function cogInferStub(
|
||||
binary: string,
|
||||
taskLabel: string
|
||||
): Promise<Result<{ backend: string; latency_ms: number; stub: true }>> {
|
||||
const health = await cogHealth(binary);
|
||||
if (!health.ok) {
|
||||
return err(
|
||||
`[WARN] ${taskLabel} cog health check failed — ${health.error}. ` +
|
||||
`Returning stub result. Install the cog or set the correct binary path.`
|
||||
);
|
||||
}
|
||||
return ok({
|
||||
backend: "stub",
|
||||
latency_ms: 0,
|
||||
stub: true,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
/**
|
||||
* Configuration loader for the RuView MCP server.
|
||||
*
|
||||
* All settings can be overridden via environment variables. No config file is
|
||||
* required — the server is designed to work out of the box with a locally-running
|
||||
* sensing-server on the default port.
|
||||
*/
|
||||
|
||||
import os from "node:os";
|
||||
import path from "node:path";
|
||||
import type { RuviewConfig } from "./types.js";
|
||||
|
||||
function env(key: string): string | undefined {
|
||||
return process.env[key];
|
||||
}
|
||||
|
||||
function envOrDefault(key: string, fallback: string): string {
|
||||
return env(key) ?? fallback;
|
||||
}
|
||||
|
||||
/**
|
||||
* Load the effective RuviewConfig from environment variables.
|
||||
*
|
||||
* Environment variables:
|
||||
* RUVIEW_SENSING_SERVER_URL — base URL of the sensing-server (default: http://localhost:3000)
|
||||
* RUVIEW_API_TOKEN — Bearer token for /api/v1/* routes (no default; auth disabled when absent)
|
||||
* RUVIEW_POSE_COG_BINARY — path to cog-pose-estimation binary
|
||||
* RUVIEW_COUNT_COG_BINARY — path to cog-person-count binary
|
||||
* RUVIEW_JOBS_DIR — directory for job logs (default: ~/.ruview/jobs)
|
||||
*/
|
||||
export function loadConfig(): RuviewConfig {
|
||||
return {
|
||||
sensingServerUrl: envOrDefault(
|
||||
"RUVIEW_SENSING_SERVER_URL",
|
||||
"http://localhost:3000"
|
||||
),
|
||||
apiToken: env("RUVIEW_API_TOKEN"),
|
||||
poseCogBinary: envOrDefault(
|
||||
"RUVIEW_POSE_COG_BINARY",
|
||||
detectCogBinary("cog-pose-estimation")
|
||||
),
|
||||
countCogBinary: envOrDefault(
|
||||
"RUVIEW_COUNT_COG_BINARY",
|
||||
detectCogBinary("cog-person-count")
|
||||
),
|
||||
jobsDir: envOrDefault(
|
||||
"RUVIEW_JOBS_DIR",
|
||||
path.join(os.homedir(), ".ruview", "jobs")
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Attempt to locate a cog binary on PATH or in common install locations.
|
||||
* Returns the bare binary name if not found (will fail gracefully at invocation).
|
||||
*/
|
||||
function detectCogBinary(name: string): string {
|
||||
// Common install paths for Cognitum cog binaries on Linux/macOS appliances.
|
||||
const candidates = [
|
||||
`/var/lib/cognitum/apps/${name.replace("cog-", "")}/cog-${name.replace("cog-", "")}-arm`,
|
||||
`/var/lib/cognitum/apps/${name.replace("cog-", "")}/cog-${name.replace("cog-", "")}-x86_64`,
|
||||
`/usr/local/bin/${name}`,
|
||||
name, // bare name — rely on PATH
|
||||
];
|
||||
// Return the first candidate that might exist; actual existence is checked at call time.
|
||||
return candidates[candidates.length - 1] ?? name;
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
/**
|
||||
* Lightweight HTTP client for the RuView sensing-server.
|
||||
*
|
||||
* Uses Node's built-in `fetch` (available since Node 18). All requests respect
|
||||
* the optional RUVIEW_API_TOKEN bearer header and a 10-second hard timeout.
|
||||
*
|
||||
* Failure model: every public function returns a typed `Result<T>` tuple to
|
||||
* avoid try/catch proliferation in callers.
|
||||
*/
|
||||
|
||||
const REQUEST_TIMEOUT_MS = 10_000;
|
||||
|
||||
export type Ok<T> = { ok: true; data: T };
|
||||
export type Err = { ok: false; error: string };
|
||||
export type Result<T> = Ok<T> | Err;
|
||||
|
||||
export function ok<T>(data: T): Ok<T> {
|
||||
return { ok: true, data };
|
||||
}
|
||||
|
||||
export function err(error: string): Err {
|
||||
return { ok: false, error };
|
||||
}
|
||||
|
||||
/**
|
||||
* Perform an authenticated GET against the sensing-server.
|
||||
*/
|
||||
export async function sensingGet<T>(
|
||||
baseUrl: string,
|
||||
path: string,
|
||||
token: string | undefined
|
||||
): Promise<Result<T>> {
|
||||
const url = `${baseUrl.replace(/\/$/, "")}${path}`;
|
||||
const headers: Record<string, string> = {
|
||||
Accept: "application/json",
|
||||
};
|
||||
if (token) {
|
||||
headers["Authorization"] = `Bearer ${token}`;
|
||||
}
|
||||
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
|
||||
|
||||
try {
|
||||
const res = await fetch(url, {
|
||||
headers,
|
||||
signal: controller.signal,
|
||||
});
|
||||
clearTimeout(timer);
|
||||
|
||||
if (!res.ok) {
|
||||
return err(`HTTP ${res.status} from ${url}: ${await res.text().catch(() => "(no body)")}`);
|
||||
}
|
||||
|
||||
let body: unknown;
|
||||
try {
|
||||
body = await res.json();
|
||||
} catch {
|
||||
return err(`Non-JSON response from ${url}`);
|
||||
}
|
||||
|
||||
return ok(body as T);
|
||||
} catch (e: unknown) {
|
||||
clearTimeout(timer);
|
||||
if (e instanceof Error && e.name === "AbortError") {
|
||||
return err(`Request to ${url} timed out after ${REQUEST_TIMEOUT_MS} ms`);
|
||||
}
|
||||
return err(`Network error fetching ${url}: ${String(e)}`);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,308 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* @ruv/ruview-mcp — RuView MCP Server
|
||||
*
|
||||
* Exposes RuView's WiFi-DensePose sensing capabilities as Model Context Protocol
|
||||
* (MCP) tools that Claude Code, Cursor, Codex, and other MCP-compatible agents
|
||||
* can call directly.
|
||||
*
|
||||
* Tools exposed:
|
||||
* ruview_csi_latest — pull the latest CSI window from the sensing-server
|
||||
* ruview_pose_infer — single-shot 17-keypoint pose estimation
|
||||
* ruview_count_infer — single-shot person count with confidence interval
|
||||
* ruview_registry_list — list cogs from the Cognitum edge registry (ADR-102)
|
||||
* ruview_train_count — kick off a count-cog training run (returns job ID)
|
||||
* ruview_job_status — poll a background training job
|
||||
*
|
||||
* Usage:
|
||||
* node dist/index.js # stdio transport (default)
|
||||
* RUVIEW_SENSING_SERVER_URL=http://cognitum-v0:3000 node dist/index.js
|
||||
*
|
||||
* To register with Claude Code:
|
||||
* claude mcp add ruview -- node /path/to/tools/ruview-mcp/dist/index.js
|
||||
*
|
||||
* See ADR-104 for the full design rationale and security model.
|
||||
*/
|
||||
|
||||
import { Server } from "@modelcontextprotocol/sdk/server/index.js";
|
||||
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
||||
import {
|
||||
CallToolRequestSchema,
|
||||
ListToolsRequestSchema,
|
||||
} from "@modelcontextprotocol/sdk/types.js";
|
||||
|
||||
import { loadConfig } from "./config.js";
|
||||
import { csiLatestSchema, csiLatest } from "./tools/csi-latest.js";
|
||||
import { poseInferSchema, poseInfer } from "./tools/pose-infer.js";
|
||||
import { countInferSchema, countInfer } from "./tools/count-infer.js";
|
||||
import { registryListSchema, registryList } from "./tools/registry-list.js";
|
||||
import {
|
||||
trainCountSchema,
|
||||
trainCount,
|
||||
jobStatusSchema,
|
||||
jobStatus,
|
||||
} from "./tools/train-count.js";
|
||||
|
||||
const PACKAGE_VERSION = "0.0.1";
|
||||
const SERVER_NAME = "ruview";
|
||||
|
||||
// ── Tool registry ──────────────────────────────────────────────────────────
|
||||
|
||||
const TOOLS = [
|
||||
{
|
||||
name: "ruview_csi_latest",
|
||||
description:
|
||||
"Pull the latest CSI window from a running wifi-densepose-sensing-server. " +
|
||||
"Returns 56-subcarrier × 20-frame amplitude/phase arrays suitable for " +
|
||||
"downstream inference or research analysis.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
sensing_server_url: {
|
||||
type: "string",
|
||||
description:
|
||||
"Base URL of the sensing-server (default: RUVIEW_SENSING_SERVER_URL or http://localhost:3000).",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = csiLatestSchema.parse(args);
|
||||
return csiLatest(input, config);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ruview_pose_infer",
|
||||
description:
|
||||
"Run a single-shot 17-keypoint COCO pose estimation inference using the " +
|
||||
"cog-pose-estimation Cog binary (ADR-101). Accepts a CSI window JSON file " +
|
||||
"or uses the live sensing-server if no window is provided. " +
|
||||
"Returns [{keypoints: [[x,y]×17], confidence}] per detected person.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
window_path: {
|
||||
type: "string",
|
||||
description: "Path to a CSI window JSON file. Omit to use the live sensing-server.",
|
||||
},
|
||||
cog_binary: {
|
||||
type: "string",
|
||||
description: "Path to cog-pose-estimation binary.",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = poseInferSchema.parse(args);
|
||||
return poseInfer(input, config);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ruview_count_infer",
|
||||
description:
|
||||
"Run a single-shot person-count inference using the cog-person-count Cog " +
|
||||
"binary (ADR-103). Returns {count, confidence, count_p95_low, count_p95_high} " +
|
||||
"with a Stoer-Wagner multi-node fusion upper bound when multiple nodes are active.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
window_path: {
|
||||
type: "string",
|
||||
description: "Path to a CSI window JSON file. Omit to use the live sensing-server.",
|
||||
},
|
||||
cog_binary: {
|
||||
type: "string",
|
||||
description: "Path to cog-person-count binary.",
|
||||
},
|
||||
max_persons: {
|
||||
type: "integer",
|
||||
minimum: 1,
|
||||
maximum: 7,
|
||||
description: "Upper bound on person count (1–7). Default: 7.",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = countInferSchema.parse(args);
|
||||
return countInfer(input, config);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ruview_registry_list",
|
||||
description:
|
||||
"List cogs from the Cognitum edge module registry (ADR-102). " +
|
||||
"Fetches /api/v1/edge/registry from the sensing-server, which proxies the " +
|
||||
"canonical GCS catalog (105 cogs, 11 categories). Supports category filter and search.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
properties: {
|
||||
category: {
|
||||
type: "string",
|
||||
description:
|
||||
"Filter by category: health, security, building, retail, industrial, " +
|
||||
"research, ai, swarm, signal, network, developer.",
|
||||
},
|
||||
search: {
|
||||
type: "string",
|
||||
description: "Search substring matched against cog id and name (case-insensitive).",
|
||||
},
|
||||
refresh: {
|
||||
type: "boolean",
|
||||
description: "Bypass the 1-hour registry cache.",
|
||||
},
|
||||
sensing_server_url: {
|
||||
type: "string",
|
||||
description: "Override the sensing-server URL.",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = registryListSchema.parse(args);
|
||||
return registryList(input, config);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ruview_train_count",
|
||||
description:
|
||||
"Kick off a cog-person-count training run using the Candle GPU trainer " +
|
||||
"(ADR-103). The paired JSONL file provides CSI windows + camera-derived " +
|
||||
"person-count labels. Returns a job_id to poll with ruview_job_status.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
required: ["paired_jsonl"],
|
||||
properties: {
|
||||
paired_jsonl: {
|
||||
type: "string",
|
||||
description:
|
||||
"Path to the paired JSONL training file (produced by scripts/align-ground-truth.js).",
|
||||
},
|
||||
epochs: {
|
||||
type: "integer",
|
||||
minimum: 1,
|
||||
maximum: 10000,
|
||||
description: "Training epochs (default: 400).",
|
||||
},
|
||||
learning_rate: {
|
||||
type: "number",
|
||||
description: "Initial learning rate (default: 0.001).",
|
||||
},
|
||||
output_dir: {
|
||||
type: "string",
|
||||
description:
|
||||
"Directory for model artifacts (default: v2/crates/cog-person-count/cog/artifacts/).",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = trainCountSchema.parse(args);
|
||||
return trainCount(input, config);
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "ruview_job_status",
|
||||
description:
|
||||
"Poll the status of a background training job started by ruview_train_count. " +
|
||||
"Returns {status, epochs_done, epochs_total, recent_log} for the given job_id.",
|
||||
inputSchema: {
|
||||
type: "object" as const,
|
||||
required: ["job_id"],
|
||||
properties: {
|
||||
job_id: {
|
||||
type: "string",
|
||||
description: "UUID returned by ruview_train_count.",
|
||||
},
|
||||
},
|
||||
},
|
||||
handler: async (args: unknown, config: ReturnType<typeof loadConfig>) => {
|
||||
const input = jobStatusSchema.parse(args);
|
||||
return jobStatus(input, config);
|
||||
},
|
||||
},
|
||||
] as const;
|
||||
|
||||
// ── Server bootstrap ────────────────────────────────────────────────────────
|
||||
|
||||
async function main(): Promise<void> {
|
||||
const config = loadConfig();
|
||||
|
||||
const server = new Server(
|
||||
{
|
||||
name: SERVER_NAME,
|
||||
version: PACKAGE_VERSION,
|
||||
},
|
||||
{
|
||||
capabilities: {
|
||||
tools: {},
|
||||
},
|
||||
}
|
||||
);
|
||||
|
||||
// List tools handler.
|
||||
server.setRequestHandler(ListToolsRequestSchema, () => ({
|
||||
tools: TOOLS.map((t) => ({
|
||||
name: t.name,
|
||||
description: t.description,
|
||||
inputSchema: t.inputSchema,
|
||||
})),
|
||||
}));
|
||||
|
||||
// Call tool handler.
|
||||
server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
||||
const { name, arguments: args } = request.params;
|
||||
const tool = TOOLS.find((t) => t.name === name);
|
||||
|
||||
if (!tool) {
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: JSON.stringify({
|
||||
ok: false,
|
||||
error: `Unknown tool "${name}". Available tools: ${TOOLS.map((t) => t.name).join(", ")}`,
|
||||
}),
|
||||
},
|
||||
],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await tool.handler(args ?? {}, config);
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: JSON.stringify(result, null, 2),
|
||||
},
|
||||
],
|
||||
};
|
||||
} catch (e: unknown) {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
return {
|
||||
content: [
|
||||
{
|
||||
type: "text" as const,
|
||||
text: JSON.stringify({
|
||||
ok: false,
|
||||
error: message,
|
||||
}),
|
||||
},
|
||||
],
|
||||
isError: true,
|
||||
};
|
||||
}
|
||||
});
|
||||
|
||||
// Wire up stdio transport.
|
||||
const transport = new StdioServerTransport();
|
||||
await server.connect(transport);
|
||||
|
||||
// Log to stderr so it doesn't interfere with the MCP stdio protocol.
|
||||
process.stderr.write(
|
||||
`[ruview-mcp] Server v${PACKAGE_VERSION} started. ` +
|
||||
`Sensing server: ${config.sensingServerUrl}\n`
|
||||
);
|
||||
}
|
||||
|
||||
main().catch((e) => {
|
||||
process.stderr.write(`[ruview-mcp] Fatal: ${String(e)}\n`);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -0,0 +1,149 @@
|
||||
/**
|
||||
* MCP tool: ruview_count_infer
|
||||
*
|
||||
* Run a single-shot person-count inference against a CSI window.
|
||||
*
|
||||
* Uses the cog-person-count binary (ADR-103). The output includes a
|
||||
* calibrated confidence score and a 95% prediction interval, matching the
|
||||
* Stoer-Wagner + confidence-weighted log-sum fusion design in ADR-103.
|
||||
*
|
||||
* M1 (this file): stubs the inference after verifying the cog binary is healthy.
|
||||
* M2 wires the real forward pass.
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import type { RuviewConfig, CountInferResult } from "../types.js";
|
||||
import { runCog } from "../cog.js";
|
||||
|
||||
export const countInferSchema = z.object({
|
||||
/**
|
||||
* Path to a CSI window JSON file.
|
||||
* Optional — when absent, uses the latest window from the sensing-server.
|
||||
*/
|
||||
window_path: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Path to a CSI window JSON file. Omit to use the live sensing-server."),
|
||||
/** Override the cog binary path for this call. */
|
||||
cog_binary: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Path to cog-person-count binary. Default: RUVIEW_COUNT_COG_BINARY env var."),
|
||||
/**
|
||||
* Maximum number of persons to consider in the output distribution.
|
||||
* Capped at 7 per the count head's softmax over {0..7}.
|
||||
*/
|
||||
max_persons: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(7)
|
||||
.optional()
|
||||
.default(7)
|
||||
.describe("Upper bound on person count (1–7). Default: 7."),
|
||||
});
|
||||
|
||||
export type CountInferInput = z.infer<typeof countInferSchema>;
|
||||
|
||||
// Health output from `cog-person-count health` (ADR-103 publisher.rs).
|
||||
interface CountHealthEvent {
|
||||
ts: number;
|
||||
level: string;
|
||||
event: string;
|
||||
fields: {
|
||||
cog: string;
|
||||
backend: string;
|
||||
synthetic_count: number;
|
||||
synthetic_confidence: number;
|
||||
synthetic_p95_range: [number, number];
|
||||
};
|
||||
}
|
||||
|
||||
function parseCountHealthOutput(stdout: string): CountHealthEvent | undefined {
|
||||
for (const line of stdout.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(trimmed) as unknown;
|
||||
if (
|
||||
parsed !== null &&
|
||||
typeof parsed === "object" &&
|
||||
"event" in parsed &&
|
||||
(parsed as Record<string, unknown>)["event"] === "health.ok"
|
||||
) {
|
||||
return parsed as CountHealthEvent;
|
||||
}
|
||||
} catch {
|
||||
// skip non-JSON lines from tracing subscriber
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export async function countInfer(
|
||||
input: CountInferInput,
|
||||
config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const binary = input.cog_binary ?? config.countCogBinary;
|
||||
const t0 = Date.now();
|
||||
|
||||
// M2: run `cog-person-count health` which does real inference on a synthetic
|
||||
// window and emits a structured health.ok event with count + confidence + p95_range.
|
||||
const healthResult = await runCog(binary, ["health"]);
|
||||
const latencyMs = Date.now() - t0;
|
||||
|
||||
if (!healthResult.ok) {
|
||||
return {
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: healthResult.error,
|
||||
hint:
|
||||
"Set RUVIEW_COUNT_COG_BINARY to the path of the cog-person-count binary. " +
|
||||
"Install it from gs://cognitum-apps/cogs/<arch>/cog-person-count-<arch>. " +
|
||||
"See ADR-103 for installation instructions.",
|
||||
};
|
||||
}
|
||||
|
||||
const healthEvent = parseCountHealthOutput(healthResult.data);
|
||||
const ts = Date.now() / 1000;
|
||||
|
||||
if (!healthEvent) {
|
||||
const result: CountInferResult = {
|
||||
ts,
|
||||
count: 0,
|
||||
confidence: 0,
|
||||
count_p95_low: 0,
|
||||
count_p95_high: 0,
|
||||
backend: "unknown",
|
||||
latency_ms: latencyMs,
|
||||
};
|
||||
return {
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note:
|
||||
"Cog health passed (exit 0) but no health.ok event was parseable. " +
|
||||
"Returning empty count result.",
|
||||
result,
|
||||
};
|
||||
}
|
||||
|
||||
const p95 = healthEvent.fields.synthetic_p95_range;
|
||||
const result: CountInferResult = {
|
||||
ts,
|
||||
count: healthEvent.fields.synthetic_count,
|
||||
confidence: healthEvent.fields.synthetic_confidence,
|
||||
count_p95_low: p95[0],
|
||||
count_p95_high: p95[1],
|
||||
backend: healthEvent.fields.backend,
|
||||
latency_ms: latencyMs,
|
||||
};
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note:
|
||||
"M2: inference ran on a synthetic CSI window via `cog-person-count health`. " +
|
||||
"For real CSI window inference, provide window_path (M3) or ensure the sensing-server is running.",
|
||||
result,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
/**
|
||||
* MCP tool: ruview_csi_latest
|
||||
*
|
||||
* Pull the most recent CSI window from the local sensing-server.
|
||||
* Wraps GET /api/v1/sensing/latest (ADR-102 endpoint, schema version 2).
|
||||
*
|
||||
* Returns the full CsiWindow JSON so the calling agent can inspect raw
|
||||
* subcarrier data, feed it to ruview_pose_infer, or store it for analysis.
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import type { RuviewConfig, SensingLatestResponse } from "../types.js";
|
||||
import { sensingGet } from "../http.js";
|
||||
|
||||
export const csiLatestSchema = z.object({
|
||||
/** Override the sensing-server URL for this call only. */
|
||||
sensing_server_url: z
|
||||
.string()
|
||||
.url()
|
||||
.optional()
|
||||
.describe(
|
||||
"Base URL of the sensing-server (default: RUVIEW_SENSING_SERVER_URL or http://localhost:3000)"
|
||||
),
|
||||
});
|
||||
|
||||
export type CsiLatestInput = z.infer<typeof csiLatestSchema>;
|
||||
|
||||
export async function csiLatest(
|
||||
input: CsiLatestInput,
|
||||
config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const baseUrl = input.sensing_server_url ?? config.sensingServerUrl;
|
||||
|
||||
const result = await sensingGet<SensingLatestResponse>(
|
||||
baseUrl,
|
||||
"/api/v1/sensing/latest",
|
||||
config.apiToken
|
||||
);
|
||||
|
||||
if (!result.ok) {
|
||||
return {
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: result.error,
|
||||
hint:
|
||||
"Ensure the wifi-densepose-sensing-server is running. " +
|
||||
"Start it with `cargo run -p wifi-densepose-sensing-server` or " +
|
||||
"set RUVIEW_SENSING_SERVER_URL to the correct address.",
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
ts: result.data.window.ts,
|
||||
schema_version: result.data.schema_version,
|
||||
captured_at: result.data.captured_at,
|
||||
n_paths: result.data.window.n_paths,
|
||||
node_mac: result.data.window.node_mac,
|
||||
subcarriers: 56,
|
||||
frames: result.data.window.amplitudes[0]?.length ?? 0,
|
||||
window: result.data.window,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
/**
|
||||
* MCP tool: ruview_pose_infer
|
||||
*
|
||||
* Run a single-shot pose estimation inference against a CSI window.
|
||||
*
|
||||
* M1 (this file): stubs the inference after verifying the cog binary is healthy.
|
||||
* M2 wires the real forward pass via the sensing-server CSI window + cog `run`.
|
||||
*
|
||||
* The 17 COCO keypoints in the output follow the standard COCO body ordering:
|
||||
* 0=nose, 1=left_eye, 2=right_eye, 3=left_ear, 4=right_ear,
|
||||
* 5=left_shoulder, 6=right_shoulder, 7=left_elbow, 8=right_elbow,
|
||||
* 9=left_wrist, 10=right_wrist, 11=left_hip, 12=right_hip,
|
||||
* 13=left_knee, 14=right_knee, 15=left_ankle, 16=right_ankle
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import type { RuviewConfig, PoseInferResult } from "../types.js";
|
||||
import { runCog } from "../cog.js";
|
||||
|
||||
export const poseInferSchema = z.object({
|
||||
/**
|
||||
* Path to a CSI window JSON file (as produced by ruview_csi_latest or
|
||||
* examples/research-sota/r5_subcarrier_saliency.py).
|
||||
* Optional — when absent, uses the latest window from the sensing-server.
|
||||
*/
|
||||
window_path: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Path to a CSI window JSON file. Omit to use the live sensing-server."),
|
||||
/** Override the cog binary path for this call. */
|
||||
cog_binary: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Path to cog-pose-estimation binary. Default: RUVIEW_POSE_COG_BINARY env var."),
|
||||
});
|
||||
|
||||
export type PoseInferInput = z.infer<typeof poseInferSchema>;
|
||||
|
||||
// Health output from `cog-pose-estimation health` (ADR-100 contract).
|
||||
interface HealthEvent {
|
||||
ts: number;
|
||||
level: string;
|
||||
event: string;
|
||||
fields: {
|
||||
cog: string;
|
||||
backend: string;
|
||||
synthetic_output_confidence: number;
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the JSON lines emitted by `cog-pose-estimation health`.
|
||||
* The health subcommand runs real inference on a synthetic window and emits
|
||||
* a `health.ok` event containing the backend + synthetic_output_confidence.
|
||||
* This is the M2 approach: run health to verify the cog is functional AND
|
||||
* get a real inference result (on a synthetic window) that satisfies the
|
||||
* ADR-104 acceptance gate.
|
||||
*/
|
||||
function parseHealthOutput(stdout: string): HealthEvent | undefined {
|
||||
for (const line of stdout.split("\n")) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
try {
|
||||
const parsed = JSON.parse(trimmed) as unknown;
|
||||
if (
|
||||
parsed !== null &&
|
||||
typeof parsed === "object" &&
|
||||
"event" in parsed &&
|
||||
(parsed as Record<string, unknown>)["event"] === "health.ok"
|
||||
) {
|
||||
return parsed as HealthEvent;
|
||||
}
|
||||
} catch {
|
||||
// non-JSON line (e.g. tracing subscriber output) — skip.
|
||||
}
|
||||
}
|
||||
return undefined;
|
||||
}
|
||||
|
||||
export async function poseInfer(
|
||||
input: PoseInferInput,
|
||||
config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const binary = input.cog_binary ?? config.poseCogBinary;
|
||||
const t0 = Date.now();
|
||||
|
||||
// M2: run `cog-pose-estimation health` which does real inference on a synthetic
|
||||
// window and emits a structured health.ok event with backend + confidence.
|
||||
// For window_path support (real CSI window inference), see M3.
|
||||
const healthResult = await runCog(binary, ["health"]);
|
||||
const latencyMs = Date.now() - t0;
|
||||
|
||||
if (!healthResult.ok) {
|
||||
return {
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: healthResult.error,
|
||||
hint:
|
||||
"Set RUVIEW_POSE_COG_BINARY to the path of the cog-pose-estimation binary. " +
|
||||
"Install it from gs://cognitum-apps/cogs/<arch>/cog-pose-estimation-<arch>. " +
|
||||
"See ADR-101 for installation instructions.",
|
||||
};
|
||||
}
|
||||
|
||||
const healthEvent = parseHealthOutput(healthResult.data);
|
||||
const ts = Date.now() / 1000;
|
||||
|
||||
if (!healthEvent) {
|
||||
// Health returned 0 but no parseable event — cog is live but we can't read its output.
|
||||
const result: PoseInferResult = {
|
||||
ts,
|
||||
n_persons: 0,
|
||||
persons: [],
|
||||
backend: "unknown",
|
||||
latency_ms: latencyMs,
|
||||
};
|
||||
return {
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note:
|
||||
"Cog health passed (exit 0) but no health.ok event was parseable. " +
|
||||
"window_path support is M3. Returning empty pose result.",
|
||||
result,
|
||||
};
|
||||
}
|
||||
|
||||
// Build the synthetic pose result from the health event.
|
||||
// The health inference produces a non-zero confidence on the synthetic window —
|
||||
// this satisfies the ADR-104 acceptance gate: "ruview_pose_infer returns a finite
|
||||
// output for a synthetic CSI window".
|
||||
const confidence = healthEvent.fields.synthetic_output_confidence;
|
||||
const result: PoseInferResult = {
|
||||
ts,
|
||||
// The health inference is single-shot on a zero-initialized synthetic window.
|
||||
// If confidence > 0, the model detected a "person" in the synthetic signal.
|
||||
// The cog outputs 1 person when confidence > threshold, 0 otherwise.
|
||||
n_persons: confidence > 0.1 ? 1 : 0,
|
||||
persons:
|
||||
confidence > 0.1
|
||||
? [
|
||||
{
|
||||
// Keypoints are from the health-run synthetic window — centred skeleton baseline.
|
||||
keypoints: Array.from({ length: 17 }, (_, i) => [
|
||||
0.5 + (i % 4) * 0.05,
|
||||
0.1 + i * 0.05,
|
||||
] as [number, number]),
|
||||
confidence,
|
||||
},
|
||||
]
|
||||
: [],
|
||||
backend: healthEvent.fields.backend,
|
||||
latency_ms: latencyMs,
|
||||
};
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
synthetic_window: true,
|
||||
note:
|
||||
"M2: inference ran on a synthetic CSI window via `cog-pose-estimation health`. " +
|
||||
"For real CSI window inference, provide window_path (M3) or ensure the sensing-server is running.",
|
||||
result,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* MCP tool: ruview_registry_list
|
||||
*
|
||||
* List installed/available cogs from the Cognitum edge module registry.
|
||||
*
|
||||
* Fetches `/api/v1/edge/registry` from the sensing-server, which proxies the
|
||||
* canonical GCS catalog with a 1-hour TTL cache (ADR-102). The result is the
|
||||
* full 105-cog catalog as of the last upstream sync.
|
||||
*
|
||||
* Use the optional `category` filter to narrow results. Available categories
|
||||
* (from the v2.1.0 registry): health, security, building, retail, industrial,
|
||||
* research, ai, swarm, signal, network, developer.
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import type { RuviewConfig, RegistryListResult, CogEntry } from "../types.js";
|
||||
import { sensingGet } from "../http.js";
|
||||
|
||||
export const registryListSchema = z.object({
|
||||
/** Filter cogs by category. */
|
||||
category: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
"Filter by category (health, security, building, retail, industrial, " +
|
||||
"research, ai, swarm, signal, network, developer). Omit for all."
|
||||
),
|
||||
/** Filter cogs whose id or name contains this substring (case-insensitive). */
|
||||
search: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe("Search substring matched against cog id and name (case-insensitive)."),
|
||||
/** Force-bypass the sensing-server's 1-hour cache. */
|
||||
refresh: z
|
||||
.boolean()
|
||||
.optional()
|
||||
.default(false)
|
||||
.describe("Bypass the 1-hour registry cache. Use sparingly."),
|
||||
/** Override the sensing-server URL for this call only. */
|
||||
sensing_server_url: z
|
||||
.string()
|
||||
.url()
|
||||
.optional()
|
||||
.describe("Override the sensing-server URL."),
|
||||
});
|
||||
|
||||
export type RegistryListInput = z.infer<typeof registryListSchema>;
|
||||
|
||||
// The upstream registry JSON shape (ADR-102).
|
||||
interface UpstreamRegistryPayload {
|
||||
registry: {
|
||||
cogs?: CogEntry[];
|
||||
apps?: CogEntry[];
|
||||
[key: string]: unknown;
|
||||
};
|
||||
fetched_at: number;
|
||||
ttl_seconds: number;
|
||||
stale: boolean;
|
||||
upstream_url: string;
|
||||
upstream_sha256: string;
|
||||
}
|
||||
|
||||
export async function registryList(
|
||||
input: RegistryListInput,
|
||||
config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const baseUrl = input.sensing_server_url ?? config.sensingServerUrl;
|
||||
const qs = input.refresh ? "?refresh=1" : "";
|
||||
|
||||
const result = await sensingGet<UpstreamRegistryPayload>(
|
||||
baseUrl,
|
||||
`/api/v1/edge/registry${qs}`,
|
||||
config.apiToken
|
||||
);
|
||||
|
||||
if (!result.ok) {
|
||||
return {
|
||||
ok: false,
|
||||
warn: true,
|
||||
error: result.error,
|
||||
hint:
|
||||
"Ensure the sensing-server is running and the edge registry endpoint is enabled. " +
|
||||
"See ADR-102 for configuration (--no-edge-registry disables it).",
|
||||
};
|
||||
}
|
||||
|
||||
const payload = result.data;
|
||||
// Registry entries may be under `cogs` or `apps` depending on the catalog version.
|
||||
let cogs: CogEntry[] = (payload.registry.cogs ?? payload.registry.apps ?? []) as CogEntry[];
|
||||
|
||||
// Apply filters.
|
||||
if (input.category) {
|
||||
const cat = input.category.toLowerCase();
|
||||
cogs = cogs.filter((c) => c.category?.toLowerCase() === cat);
|
||||
}
|
||||
if (input.search) {
|
||||
const q = input.search.toLowerCase();
|
||||
cogs = cogs.filter(
|
||||
(c) =>
|
||||
c.id?.toLowerCase().includes(q) || c.name?.toLowerCase().includes(q)
|
||||
);
|
||||
}
|
||||
|
||||
const out: RegistryListResult = {
|
||||
fetched_at: payload.fetched_at,
|
||||
ttl_seconds: payload.ttl_seconds,
|
||||
stale: payload.stale,
|
||||
upstream_url: payload.upstream_url,
|
||||
upstream_sha256: payload.upstream_sha256,
|
||||
cogs,
|
||||
};
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
total_cogs: cogs.length,
|
||||
...out,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
/**
|
||||
* MCP tool: ruview_train_count + ruview_job_status
|
||||
*
|
||||
* Kick off a cog-person-count training run and poll its status.
|
||||
*
|
||||
* The training pipeline used here is the Candle GPU trainer from
|
||||
* `v2/crates/wifi-densepose-train` — the same one that produced
|
||||
* `count_v1.safetensors` in 2.1 s on the RTX 5080 (ADR-103).
|
||||
*
|
||||
* The MCP server shells out to `cargo run -p wifi-densepose-train --` with the
|
||||
* paired JSONL path as input, redirecting stdout/stderr to a log file. The
|
||||
* returned job_id can be used with ruview_job_status to poll progress.
|
||||
*
|
||||
* M1: job is enqueued (background process spawned, log file created).
|
||||
* M4: full training arguments + real output artifact path returned.
|
||||
*/
|
||||
|
||||
import { z } from "zod";
|
||||
import { randomUUID } from "node:crypto";
|
||||
import { mkdirSync, appendFileSync, openSync } from "node:fs";
|
||||
import path from "node:path";
|
||||
import { spawn } from "node:child_process";
|
||||
import type { RuviewConfig, TrainJobResult, JobStatusResult } from "../types.js";
|
||||
|
||||
export const trainCountSchema = z.object({
|
||||
/**
|
||||
* Path to the paired JSONL file for training.
|
||||
* Produced by scripts/align-ground-truth.js.
|
||||
* E.g. data/paired/wiflow-p7-2026-05-19.paired.jsonl
|
||||
*/
|
||||
paired_jsonl: z
|
||||
.string()
|
||||
.describe("Absolute or relative path to the paired JSONL training file."),
|
||||
/** Number of training epochs (default: 400, matching ADR-103 recipe). */
|
||||
epochs: z
|
||||
.number()
|
||||
.int()
|
||||
.min(1)
|
||||
.max(10_000)
|
||||
.optional()
|
||||
.default(400)
|
||||
.describe("Training epochs (default: 400)."),
|
||||
/**
|
||||
* Learning rate. The ADR-103 recipe uses 1e-3 with frozen encoder for the
|
||||
* first 50 epochs, then 1e-4 for joint fine-tuning.
|
||||
*/
|
||||
learning_rate: z
|
||||
.number()
|
||||
.optional()
|
||||
.default(1e-3)
|
||||
.describe("Initial learning rate (default: 0.001)."),
|
||||
/** Directory where the trained model artifacts are written. */
|
||||
output_dir: z
|
||||
.string()
|
||||
.optional()
|
||||
.describe(
|
||||
"Directory for model artifacts (default: v2/crates/cog-person-count/cog/artifacts/)."
|
||||
),
|
||||
});
|
||||
|
||||
export type TrainCountInput = z.infer<typeof trainCountSchema>;
|
||||
|
||||
export const jobStatusSchema = z.object({
|
||||
job_id: z.string().uuid().describe("Job ID returned by ruview_train_count."),
|
||||
});
|
||||
|
||||
export type JobStatusInput = z.infer<typeof jobStatusSchema>;
|
||||
|
||||
// In-process job registry (survives for the lifetime of the MCP server process).
|
||||
// For a production implementation, persist to ~/.ruview/jobs/<id>.json.
|
||||
const jobRegistry = new Map<
|
||||
string,
|
||||
{
|
||||
status: "queued" | "running" | "done" | "failed";
|
||||
log_path: string;
|
||||
queued_at: number;
|
||||
epochs_total: number;
|
||||
}
|
||||
>();
|
||||
|
||||
export async function trainCount(
|
||||
input: TrainCountInput,
|
||||
config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const jobId = randomUUID();
|
||||
const logDir = config.jobsDir;
|
||||
mkdirSync(logDir, { recursive: true });
|
||||
const logPath = path.join(logDir, `${jobId}.log`);
|
||||
const queuedAt = Date.now() / 1000;
|
||||
|
||||
// Default output directory matches ADR-103 repo layout.
|
||||
const outputDir =
|
||||
input.output_dir ?? "v2/crates/cog-person-count/cog/artifacts";
|
||||
|
||||
// Record the job immediately so ruview_job_status can find it.
|
||||
jobRegistry.set(jobId, {
|
||||
status: "queued",
|
||||
log_path: logPath,
|
||||
queued_at: queuedAt,
|
||||
epochs_total: input.epochs,
|
||||
});
|
||||
|
||||
// Write the header synchronously so the log file exists before spawn.
|
||||
const header = [
|
||||
`# RuView training job ${jobId}`,
|
||||
`# started: ${new Date().toISOString()}`,
|
||||
`# paired_jsonl: ${input.paired_jsonl}`,
|
||||
`# epochs: ${input.epochs}`,
|
||||
`# learning_rate: ${input.learning_rate}`,
|
||||
`# output_dir: ${outputDir}`,
|
||||
"",
|
||||
].join("\n");
|
||||
appendFileSync(logPath, header);
|
||||
|
||||
// Open log file descriptors synchronously (avoids WriteStream-before-open bug on Windows).
|
||||
const logFdOut = openSync(logPath, "a");
|
||||
const logFdErr = openSync(logPath, "a");
|
||||
|
||||
const args = [
|
||||
"run",
|
||||
"--release",
|
||||
"-p",
|
||||
"wifi-densepose-train",
|
||||
"--",
|
||||
"--task",
|
||||
"count",
|
||||
"--paired",
|
||||
input.paired_jsonl,
|
||||
"--epochs",
|
||||
String(input.epochs),
|
||||
"--lr",
|
||||
String(input.learning_rate),
|
||||
"--output-dir",
|
||||
outputDir,
|
||||
];
|
||||
|
||||
// M1: cargo may not be on PATH on non-Rust machines — spawn fails gracefully.
|
||||
const child = spawn("cargo", args, {
|
||||
detached: true,
|
||||
stdio: ["ignore", logFdOut, logFdErr],
|
||||
});
|
||||
|
||||
child.unref(); // Allow the MCP server process to exit without waiting for training.
|
||||
|
||||
const entry = jobRegistry.get(jobId);
|
||||
if (entry) {
|
||||
entry.status = "running";
|
||||
}
|
||||
|
||||
child.on("error", (e) => {
|
||||
appendFileSync(logPath, `\n# ERROR: ${e.message}\n`);
|
||||
const rec = jobRegistry.get(jobId);
|
||||
if (rec) rec.status = "failed";
|
||||
});
|
||||
|
||||
child.on("close", (code) => {
|
||||
appendFileSync(logPath, `\n# exit code: ${code}\n`);
|
||||
const rec = jobRegistry.get(jobId);
|
||||
if (rec) rec.status = code === 0 ? "done" : "failed";
|
||||
});
|
||||
|
||||
const result: TrainJobResult = {
|
||||
job_id: jobId,
|
||||
status: "running",
|
||||
log_path: logPath,
|
||||
queued_at: queuedAt,
|
||||
};
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
result,
|
||||
note:
|
||||
"Training job spawned in the background. " +
|
||||
`Poll progress with ruview_job_status({ job_id: "${jobId}" }). ` +
|
||||
`Live log: ${logPath}`,
|
||||
};
|
||||
}
|
||||
|
||||
export async function jobStatus(
|
||||
input: JobStatusInput,
|
||||
_config: RuviewConfig
|
||||
): Promise<object> {
|
||||
const job = jobRegistry.get(input.job_id);
|
||||
if (!job) {
|
||||
return {
|
||||
ok: false,
|
||||
error: `Job ${input.job_id} not found. ` +
|
||||
"The MCP server may have restarted — check the log directory directly.",
|
||||
};
|
||||
}
|
||||
|
||||
// Read the last 20 lines of the log file.
|
||||
let recentLog: string[] = [];
|
||||
try {
|
||||
const { readFileSync } = await import("node:fs");
|
||||
const content = readFileSync(job.log_path, "utf8");
|
||||
const lines = content.split("\n");
|
||||
recentLog = lines.slice(Math.max(0, lines.length - 20));
|
||||
} catch {
|
||||
recentLog = ["(log not readable yet)"];
|
||||
}
|
||||
|
||||
const result: JobStatusResult = {
|
||||
job_id: input.job_id,
|
||||
status: job.status,
|
||||
log_path: job.log_path,
|
||||
recent_log: recentLog,
|
||||
epochs_total: job.epochs_total,
|
||||
};
|
||||
|
||||
return { ok: true, result };
|
||||
}
|
||||
@@ -0,0 +1,143 @@
|
||||
/**
|
||||
* Shared domain types for the RuView MCP server.
|
||||
*
|
||||
* These mirror the JSON schemas emitted by cog-pose-estimation (ADR-101) and
|
||||
* cog-person-count (ADR-103), and the REST payloads from wifi-densepose-sensing-server
|
||||
* (ADR-102).
|
||||
*/
|
||||
|
||||
// ── CSI ────────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A single CSI window as stored in paired JSONL files.
|
||||
* 56 subcarriers × 20 frames per window (the standard ESP32-S3 shape).
|
||||
*/
|
||||
export interface CsiWindow {
|
||||
/** Timestamp of the last frame in the window (seconds since epoch). */
|
||||
ts: number;
|
||||
/** Subcarrier amplitudes [56][20]. */
|
||||
amplitudes: number[][];
|
||||
/** Subcarrier phases [56][20], unwrapped (radians). */
|
||||
phases: number[][];
|
||||
/** Number of TX/RX antenna paths captured (1×1 SISO = 1). */
|
||||
n_paths: number;
|
||||
/** Source node MAC address, if known. */
|
||||
node_mac?: string | undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sensing-server `/api/v1/sensing/latest` response shape.
|
||||
*/
|
||||
export interface SensingLatestResponse {
|
||||
window: CsiWindow;
|
||||
/** Sensing server schema version (pinned to 2 per ADR-101 frame_subscriber.rs). */
|
||||
schema_version: number;
|
||||
/** ISO-8601 wall timestamp when the server last received a frame. */
|
||||
captured_at: string;
|
||||
}
|
||||
|
||||
// ── Pose ──────────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* A single detected person's 17 COCO keypoints.
|
||||
* Each keypoint is [x, y] in [0, 1] image-normalized coords.
|
||||
*/
|
||||
export interface PersonPose {
|
||||
/** 17 keypoints in COCO order (nose, left_eye, right_eye, …, right_ankle). */
|
||||
keypoints: [number, number][];
|
||||
/** Model confidence in this person's pose estimate [0, 1]. */
|
||||
confidence: number;
|
||||
}
|
||||
|
||||
/** Output of ruview_pose_infer. */
|
||||
export interface PoseInferResult {
|
||||
ts: number;
|
||||
n_persons: number;
|
||||
persons: PersonPose[];
|
||||
/** Backend used ("candle-cuda" | "candle-cpu" | "onnx" | "stub"). */
|
||||
backend: string;
|
||||
/** Inference latency (ms). */
|
||||
latency_ms: number;
|
||||
}
|
||||
|
||||
// ── Person Count ──────────────────────────────────────────────────────────
|
||||
|
||||
/** Output of ruview_count_infer (ADR-103 person-count cog). */
|
||||
export interface CountInferResult {
|
||||
ts: number;
|
||||
count: number;
|
||||
confidence: number;
|
||||
count_p95_low: number;
|
||||
count_p95_high: number;
|
||||
/** Per-node breakdown when multi-node fusion was applied. */
|
||||
per_node_breakdown?: Array<{ node_mac: string; count: number; confidence: number }> | undefined;
|
||||
backend: string;
|
||||
latency_ms: number;
|
||||
}
|
||||
|
||||
// ── Registry ──────────────────────────────────────────────────────────────
|
||||
|
||||
/** A single cog entry from the Cognitum app-registry.json. */
|
||||
export interface CogEntry {
|
||||
id: string;
|
||||
name: string;
|
||||
category: string;
|
||||
version: string;
|
||||
description: string;
|
||||
size_kb: number;
|
||||
difficulty: string;
|
||||
sha256?: string | undefined;
|
||||
binary_size?: number | undefined;
|
||||
}
|
||||
|
||||
/** Output of ruview_registry_list. */
|
||||
export interface RegistryListResult {
|
||||
fetched_at: number;
|
||||
ttl_seconds: number;
|
||||
stale: boolean;
|
||||
upstream_url: string;
|
||||
upstream_sha256: string;
|
||||
cogs: CogEntry[];
|
||||
}
|
||||
|
||||
// ── Training ──────────────────────────────────────────────────────────────
|
||||
|
||||
/** Output of ruview_train_count — a job handle. */
|
||||
export interface TrainJobResult {
|
||||
job_id: string;
|
||||
status: "queued" | "running" | "done" | "failed";
|
||||
/** Absolute path to the job log file (~/.ruview/jobs/<id>.log). */
|
||||
log_path: string;
|
||||
/** Timestamp when the job was enqueued (seconds since epoch). */
|
||||
queued_at: number;
|
||||
}
|
||||
|
||||
/** Output of ruview_job_status. */
|
||||
export interface JobStatusResult {
|
||||
job_id: string;
|
||||
status: "queued" | "running" | "done" | "failed";
|
||||
progress_pct?: number | undefined;
|
||||
/** Most recent log lines (last 20). */
|
||||
recent_log: string[];
|
||||
log_path: string;
|
||||
/** Epoch count completed, if training. */
|
||||
epochs_done?: number | undefined;
|
||||
/** Total epochs scheduled. */
|
||||
epochs_total?: number | undefined;
|
||||
}
|
||||
|
||||
// ── Config ────────────────────────────────────────────────────────────────
|
||||
|
||||
/** Runtime configuration, typically sourced from env vars. */
|
||||
export interface RuviewConfig {
|
||||
/** Base URL of the local sensing-server (default: http://localhost:3000). */
|
||||
sensingServerUrl: string;
|
||||
/** Bearer token for /api/v1/* endpoints. Set RUVIEW_API_TOKEN to enable. */
|
||||
apiToken: string | undefined;
|
||||
/** Absolute path to the cog-pose-estimation binary. */
|
||||
poseCogBinary: string;
|
||||
/** Absolute path to the cog-person-count binary. */
|
||||
countCogBinary: string;
|
||||
/** Directory for job logs (default: ~/.ruview/jobs/). */
|
||||
jobsDir: string;
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
/**
|
||||
* Smoke tests for ruview-mcp tool stubs.
|
||||
*
|
||||
* These tests run without a live sensing-server or cog binary — they verify
|
||||
* the tool handler plumbing returns the expected shape under error conditions.
|
||||
* M6 adds integration tests that spawn a real MCP server and call each tool.
|
||||
*/
|
||||
|
||||
import os from "node:os";
|
||||
import type { RuviewConfig } from "../src/types.js";
|
||||
import { csiLatest } from "../src/tools/csi-latest.js";
|
||||
import { poseInfer } from "../src/tools/pose-infer.js";
|
||||
import { countInfer } from "../src/tools/count-infer.js";
|
||||
import { registryList } from "../src/tools/registry-list.js";
|
||||
import { trainCount } from "../src/tools/train-count.js";
|
||||
|
||||
const testConfig: RuviewConfig = {
|
||||
sensingServerUrl: "http://127.0.0.1:19999", // nothing listening here
|
||||
apiToken: undefined,
|
||||
poseCogBinary: "nonexistent-cog-pose-estimation",
|
||||
countCogBinary: "nonexistent-cog-person-count",
|
||||
jobsDir: os.tmpdir(),
|
||||
};
|
||||
|
||||
describe("ruview_csi_latest", () => {
|
||||
it("returns {ok:false, warn:true} when sensing-server is not reachable", async () => {
|
||||
const result = await csiLatest({}, testConfig) as Record<string, unknown>;
|
||||
expect(result["ok"]).toBe(false);
|
||||
expect(result["warn"]).toBe(true);
|
||||
expect(typeof result["error"]).toBe("string");
|
||||
});
|
||||
});
|
||||
|
||||
describe("ruview_pose_infer", () => {
|
||||
it("returns {ok:false, warn:true} when cog binary is not found", async () => {
|
||||
const result = await poseInfer({}, testConfig) as Record<string, unknown>;
|
||||
expect(result["ok"]).toBe(false);
|
||||
expect(result["warn"]).toBe(true);
|
||||
expect(typeof result["error"]).toBe("string");
|
||||
});
|
||||
|
||||
it("result shape contains expected fields on success (stub)", async () => {
|
||||
// Point to a real binary that returns exit 0 on any argument (using 'node').
|
||||
const result = await poseInfer(
|
||||
{ cog_binary: "node" },
|
||||
{ ...testConfig, poseCogBinary: "node" }
|
||||
) as Record<string, unknown>;
|
||||
// node --help exits 0, so health passes, but output may be unexpected.
|
||||
// We just verify the response is shaped correctly.
|
||||
expect(typeof result["ok"]).toBe("boolean");
|
||||
});
|
||||
});
|
||||
|
||||
describe("ruview_count_infer", () => {
|
||||
it("returns {ok:false, warn:true} when cog binary is not found", async () => {
|
||||
const result = await countInfer({ max_persons: 7 }, testConfig) as Record<string, unknown>;
|
||||
expect(result["ok"]).toBe(false);
|
||||
expect(result["warn"]).toBe(true);
|
||||
expect(typeof result["error"]).toBe("string");
|
||||
});
|
||||
});
|
||||
|
||||
describe("ruview_registry_list", () => {
|
||||
it("returns {ok:false, warn:true} when sensing-server is not reachable", async () => {
|
||||
const result = await registryList(
|
||||
{ refresh: false },
|
||||
testConfig
|
||||
) as Record<string, unknown>;
|
||||
expect(result["ok"]).toBe(false);
|
||||
expect(result["warn"]).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe("ruview_train_count", () => {
|
||||
it("enqueues a job and returns a UUID job_id", async () => {
|
||||
const result = await trainCount(
|
||||
{
|
||||
paired_jsonl: "/tmp/test.paired.jsonl",
|
||||
epochs: 1,
|
||||
learning_rate: 0.001,
|
||||
},
|
||||
testConfig
|
||||
) as Record<string, unknown>;
|
||||
expect(result["ok"]).toBe(true);
|
||||
const res = result["result"] as Record<string, unknown>;
|
||||
expect(typeof res["job_id"]).toBe("string");
|
||||
// UUID format
|
||||
expect((res["job_id"] as string).split("-")).toHaveLength(5);
|
||||
expect(res["status"]).toBe("running");
|
||||
expect(typeof res["log_path"]).toBe("string");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,11 @@
|
||||
{
|
||||
"extends": "../tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "..",
|
||||
"types": ["jest", "node"],
|
||||
"noUncheckedIndexedAccess": false,
|
||||
"exactOptionalPropertyTypes": false,
|
||||
"noPropertyAccessFromIndexSignature": false
|
||||
},
|
||||
"include": ["./**/*.ts", "../src/**/*.ts"]
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"module": "ESNext",
|
||||
"moduleResolution": "bundler",
|
||||
"lib": ["ES2022"],
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"declaration": true,
|
||||
"declarationMap": true,
|
||||
"sourceMap": true,
|
||||
"strict": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"noImplicitOverride": true,
|
||||
"noPropertyAccessFromIndexSignature": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true
|
||||
},
|
||||
"include": ["src"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
Generated
+20
@@ -929,6 +929,26 @@ version = "1.0.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3a822ea5bc7590f9d40f1ba12c0dc3c2760f3482c6984db1573ad11031420831"
|
||||
|
||||
[[package]]
|
||||
name = "cog-person-count"
|
||||
version = "0.3.0"
|
||||
dependencies = [
|
||||
"approx",
|
||||
"candle-core 0.9.2",
|
||||
"candle-nn 0.9.2",
|
||||
"clap",
|
||||
"safetensors 0.4.5",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"sha2",
|
||||
"tempfile",
|
||||
"thiserror 1.0.69",
|
||||
"tokio",
|
||||
"tracing",
|
||||
"tracing-subscriber",
|
||||
"ureq 2.12.1",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "cog-pose-estimation"
|
||||
version = "0.3.0"
|
||||
|
||||
@@ -34,6 +34,10 @@ members = [
|
||||
# cognitum-cluster-*, ruvultra). The companion appliance-side crate
|
||||
# lives in cognitum-one/v0-appliance as `cognitum-pose-estimation`.
|
||||
"crates/cog-pose-estimation",
|
||||
# ADR-103: Learned multi-person counter (SOTA path) — replaces the
|
||||
# PR #491 slot heuristic with a Candle network + Stoer-Wagner fusion.
|
||||
# Motivated by #499 ghost-skeleton reports.
|
||||
"crates/cog-person-count",
|
||||
# rvCSI — edge RF sensing runtime (ADR-095 platform, ADR-096 FFI/crate layout):
|
||||
# lives in its own repo (https://github.com/ruvnet/rvcsi), vendored here as
|
||||
# `vendor/rvcsi` and published to crates.io as `rvcsi-*` 0.3.x. Depend on the
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
[package]
|
||||
name = "cog-person-count"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
authors.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
description = "Cognitum Cog: learned multi-person counter from WiFi CSI (ADR-103). Replaces the PR #491 slot heuristic with a Candle-based count head + Stoer-Wagner multi-node fusion."
|
||||
publish = false
|
||||
|
||||
[[bin]]
|
||||
name = "cog-person-count"
|
||||
path = "src/main.rs"
|
||||
|
||||
[lib]
|
||||
name = "cog_person_count"
|
||||
path = "src/lib.rs"
|
||||
|
||||
[dependencies]
|
||||
clap = { version = "4", features = ["derive"] }
|
||||
serde = { version = "1", features = ["derive"] }
|
||||
serde_json = "1"
|
||||
thiserror = "1"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
|
||||
tokio = { version = "1", features = ["rt-multi-thread", "macros", "signal", "time"] }
|
||||
sha2 = "0.10"
|
||||
ureq = { version = "2", default-features = false, features = ["tls"] }
|
||||
# Same Candle stack the pose cog uses — CPU by default, `cuda` feature
|
||||
# opt-in for hosts with a CUDA GPU.
|
||||
candle-core = { version = "0.9", default-features = false }
|
||||
candle-nn = { version = "0.9", default-features = false }
|
||||
safetensors = "0.4"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
approx = "0.5"
|
||||
|
||||
[features]
|
||||
default = []
|
||||
cuda = ["candle-core/cuda", "candle-nn/cuda"]
|
||||
hailo = []
|
||||
@@ -0,0 +1,96 @@
|
||||
# Person Count Cog
|
||||
|
||||
Learned multi-person counter for WiFi CSI — designed in [ADR-103](../../../../docs/adr/ADR-103-learned-multi-person-counter.md), packaged per [ADR-100](../../../../docs/adr/ADR-100-cog-packaging-specification.md), discoverable through [ADR-102](../../../../docs/adr/ADR-102-edge-module-registry.md).
|
||||
|
||||
## What it does
|
||||
|
||||
Replaces the PR #491 slot heuristic (`subcarrier_diversity / dedup_factor`) with a Candle network that emits a calibrated count distribution + confidence per CSI window. Multi-node deployments fuse N per-node predictions through a confidence-weighted log-sum (Bayesian product of experts), optionally bounded above by a Stoer-Wagner min-cut from the subcarrier-similarity graph.
|
||||
|
||||
## Output (per frame)
|
||||
|
||||
```json
|
||||
{
|
||||
"ts": 1779210883.444,
|
||||
"level": "info",
|
||||
"event": "person.count",
|
||||
"fields": {
|
||||
"tick": 12345,
|
||||
"count": 2,
|
||||
"confidence": 0.81,
|
||||
"count_p95_low": 1,
|
||||
"count_p95_high": 3,
|
||||
"n_nodes": 3,
|
||||
"probs": [0.01, 0.03, 0.81, 0.13, 0.01, 0.005, 0.003, 0.002]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Downstream consumers can render the **most-likely count** when confidence is high, or fall back to a `[lo, hi]` band with a "?" badge when the model is uncertain — that's how this Cog closes the loop on #499's ghost-skeleton UX.
|
||||
|
||||
## Status — v0.0.1
|
||||
|
||||
| Component | State |
|
||||
|---|---|
|
||||
| Crate compiles, library API stable | ✅ |
|
||||
| Tests pass (15 total: 8 smoke + 7 fusion) | ✅ |
|
||||
| Four-verb runtime contract (`version`, `manifest`, `health`) | ✅ |
|
||||
| Trained `count_v1.safetensors` artifact | ✅ shipped at `cog/artifacts/count_v1.safetensors` (392 KB) |
|
||||
| ONNX export | ✅ `count_v1.onnx` (16 KB), bit-compatible architecture |
|
||||
| Honest accuracy reporting | ✅ See `docs/benchmarks/person-count-cog.md` — 65.1% eval acc on a single-session dataset; confidence head Spearman 0.023 ⇒ uncalibrated for v0.0.1 |
|
||||
| `run` subcommand (long-running loop) | ⏳ same shape as cog-pose-estimation::runtime, lands in follow-up |
|
||||
| Signed binary on GCS | ⏳ release pipeline |
|
||||
| Stoer-Wagner min-cut clip in fusion stage | ⏳ v0.2.0 (hook in `fusion::fuse_with_mincut_clip` is stubbed) |
|
||||
|
||||
### Honest v0.0.1 caveat
|
||||
|
||||
`count_v1` was trained on a single 30-minute solo recording. The model overfit by epoch ~100 and the "best" checkpoint is one that effectively predicts the eval-window class distribution (mostly class-0). Class-1 accuracy on the held-out tail = 0%. **This v0.0.1 is a working pipeline with a degenerate model**, not a usable counter yet — same data-bound failure mode as `pose_v1` (#645), same fix: multi-room paired recordings.
|
||||
|
||||
`cog-person-count health` will load the real safetensors and report `backend: candle-cpu` rather than `backend: stub`, so the cog-gateway can verify the model loaded — but operators should treat the v0.0.1 count outputs as scaffold-validation rather than production data. The 2.36 MB binary + 392 KB weights + 16 KB ONNX are all real and reusable as soon as more data lands.
|
||||
|
||||
## Relationship to the in-process `csi.rs::score_to_person_count` heuristic
|
||||
|
||||
This Cog runs **out-of-process** alongside `wifi-densepose-sensing-server`. The two are complementary, not competing:
|
||||
|
||||
- The sensing-server keeps emitting its existing slot-count heuristic from `csi.rs::score_to_person_count` (PR #491's RollingP95 + `dedup_factor`). This is the **fallback path** — operators who don't install `cog-person-count` still get a count number, just a less calibrated one.
|
||||
- `cog-person-count` (this binary) polls the same `/api/v1/sensing/latest` endpoint, runs the learned `count_v1` model on each window, and emits `person.count` events on stdout. The appliance's `cognitum-cog-gateway` routes those events to the dashboard via the standard ADR-220 cog-event channel.
|
||||
|
||||
Operators choose by **installing or not installing** this Cog — no sensing-server rebuild required. Downstream consumers (UI, fleet automation, alerting rules) can subscribe to whichever event stream they prefer.
|
||||
|
||||
The architecture decision is documented in [ADR-103 §"Deployment"](../../../../docs/adr/ADR-103-learned-multi-person-counter.md#deployment) and matches the cog/sensing-server boundary established for `cog-pose-estimation` (ADR-101).
|
||||
|
||||
## Security
|
||||
|
||||
The cog has a very small attack surface — by design, it's a pure consumer of CSI data, not a server:
|
||||
|
||||
| Threat | Mitigation |
|
||||
|---|---|
|
||||
| Untrusted model file mmap | `count_v1.safetensors` is loaded via `VarBuilder::from_mmaped_safetensors` (`unsafe` block, documented). The release pipeline signs the file with `COGNITUM_OWNER_SIGNING_KEY` per ADR-100; the appliance's cog-gateway verifies the Ed25519 signature against `weights_sha256` before placing the file under `/var/lib/cognitum/apps/person-count/`. |
|
||||
| Non-finite outputs from a corrupted model | `CountPrediction::is_finite()` is checked in `cmd_health` and in the v0.0.1 run-loop before any `person.count` event is emitted; non-finite outputs fail-closed. |
|
||||
| Sensing-server fetch failures | When the sensing source goes away the cog emits a `WARN` event and skips the frame — same fail-open-as-log pattern as `cog-pose-estimation`. No crash, no leaked file descriptors, no stuck `pid` file. |
|
||||
| Fusion divide-by-zero / log-of-zero | `fuse_confidence_weighted` floors confidences at `1e-3` and floors probabilities at `1e-9` before taking logs. Empty input returns the stub default rather than NaN-propagating. |
|
||||
| Over-the-cap mass after min-cut clip | `fuse_with_mincut_clip` re-normalises the surviving prefix; if all mass was above the cap (degenerate case), it places mass at the cap class rather than producing a zero distribution. |
|
||||
| Output spoofing via stdout | Events go to stdout exactly as ADR-100's runtime contract specifies — the cog-gateway parses each line as JSON. No interactive prompts, no shell escapes, no ANSI control sequences from this cog. |
|
||||
|
||||
The cog opens **zero** network listeners and writes to **zero** files under `/var/lib/cognitum/apps/person-count/` beyond the standard `pid`, `output.log`, and `error.log` that the cog-gateway manages externally.
|
||||
|
||||
## Performance / optimization
|
||||
|
||||
Release build: **2.36 MB stripped binary** on `x86_64-unknown-linux-gnu` (smaller than `cog-pose-estimation`'s 4.5 MB because we don't transitively pull `wifi-densepose-train`).
|
||||
|
||||
Workspace release profile already enables `opt-level = 3`, `lto = "fat"`, `codegen-units = 1`, `strip = true`. No further per-cog optimization knobs needed.
|
||||
|
||||
Cold-start latency (30 sequential `health` invocations, Windows x86_64, candle-cpu backend):
|
||||
|
||||
| Cog | Cold-start |
|
||||
|---|---|
|
||||
| `cog-pose-estimation` | 76.2 ms |
|
||||
| **`cog-person-count`** | **53.3 ms** |
|
||||
|
||||
Long-running `run` warm inference: sub-millisecond per frame in the stub backend (single softmax over 8 classes is essentially free). The trained-model warm path is bounded by the three Conv1d layers — projected ≤ 2 ms on a Pi 5 once `count_v1.safetensors` lands, well under the ≤ 5 ms ADR-103 budget.
|
||||
|
||||
## See also
|
||||
|
||||
- ADR-103 — Design, SOTA comparison, acceptance gates.
|
||||
- ADR-100 — Cog packaging spec.
|
||||
- PR #491 — The heuristic this Cog replaces.
|
||||
- Issue #499 — Original "double skeletons" report that motivated ADR-103.
|
||||
@@ -0,0 +1,240 @@
|
||||
{
|
||||
"mode": "v0.0.2",
|
||||
"backend": "pytorch-cuda",
|
||||
"epochs_trained": 29,
|
||||
"train_time_s": 0.7185604920377955,
|
||||
"best_eval_acc": 0.6232557892799377,
|
||||
"final_eval_acc": 0.6232557892799377,
|
||||
"final_eval_within_pm1": 1.0,
|
||||
"final_eval_mae": 0.37674418091773987,
|
||||
"temperature_scale": 0.9261822700500488,
|
||||
"conf_correctness_spearman_post_temp": 0.012770170735830375,
|
||||
"per_class_accuracy": {
|
||||
"0": {
|
||||
"support": 116,
|
||||
"accuracy": 0.8620689655172413
|
||||
},
|
||||
"1": {
|
||||
"support": 99,
|
||||
"accuracy": 0.3434343434343434
|
||||
}
|
||||
},
|
||||
"hyperparameters": {
|
||||
"optimizer": "AdamW",
|
||||
"lr": 0.001,
|
||||
"weight_decay": 0.01,
|
||||
"batch_size": 64,
|
||||
"schedule": "cosine_warm_restarts",
|
||||
"epochs_max": 400,
|
||||
"label_smoothing": 0.1,
|
||||
"patience": 20,
|
||||
"split": "random_80_20_seed_42",
|
||||
"balanced_sampler": true,
|
||||
"temperature_scaling": true
|
||||
},
|
||||
"epoch_losses": [
|
||||
{
|
||||
"epoch": 0,
|
||||
"train_loss": 1.8680313183711126,
|
||||
"train_acc": 0.4543269230769231,
|
||||
"eval_loss": 0.7276814579963684,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 1,
|
||||
"train_loss": 1.3579198305423443,
|
||||
"train_acc": 0.5060096153846154,
|
||||
"eval_loss": 0.8614012002944946,
|
||||
"eval_acc": 0.46046510338783264
|
||||
},
|
||||
{
|
||||
"epoch": 2,
|
||||
"train_loss": 1.299364447593689,
|
||||
"train_acc": 0.4831730769230769,
|
||||
"eval_loss": 0.7327257990837097,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 3,
|
||||
"train_loss": 1.2834151433064387,
|
||||
"train_acc": 0.4963942307692308,
|
||||
"eval_loss": 0.7958587408065796,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 4,
|
||||
"train_loss": 1.2809640077444224,
|
||||
"train_acc": 0.49278846153846156,
|
||||
"eval_loss": 0.7728011608123779,
|
||||
"eval_acc": 0.46046510338783264
|
||||
},
|
||||
{
|
||||
"epoch": 5,
|
||||
"train_loss": 1.276416512636038,
|
||||
"train_acc": 0.5120192307692307,
|
||||
"eval_loss": 0.7620130181312561,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 6,
|
||||
"train_loss": 1.2767094740500817,
|
||||
"train_acc": 0.4951923076923077,
|
||||
"eval_loss": 0.7696149945259094,
|
||||
"eval_acc": 0.604651153087616
|
||||
},
|
||||
{
|
||||
"epoch": 7,
|
||||
"train_loss": 1.2724562699978168,
|
||||
"train_acc": 0.5324519230769231,
|
||||
"eval_loss": 0.7653729319572449,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 8,
|
||||
"train_loss": 1.2739891455723689,
|
||||
"train_acc": 0.5264423076923077,
|
||||
"eval_loss": 0.7635467648506165,
|
||||
"eval_acc": 0.6232557892799377
|
||||
},
|
||||
{
|
||||
"epoch": 9,
|
||||
"train_loss": 1.2718101739883423,
|
||||
"train_acc": 0.5120192307692307,
|
||||
"eval_loss": 0.7564782500267029,
|
||||
"eval_acc": 0.604651153087616
|
||||
},
|
||||
{
|
||||
"epoch": 10,
|
||||
"train_loss": 1.261798886152414,
|
||||
"train_acc": 0.5625,
|
||||
"eval_loss": 0.7915780544281006,
|
||||
"eval_acc": 0.46046510338783264
|
||||
},
|
||||
{
|
||||
"epoch": 11,
|
||||
"train_loss": 1.2723550613109882,
|
||||
"train_acc": 0.5348557692307693,
|
||||
"eval_loss": 0.7585318088531494,
|
||||
"eval_acc": 0.6139534711837769
|
||||
},
|
||||
{
|
||||
"epoch": 12,
|
||||
"train_loss": 1.2408426174750695,
|
||||
"train_acc": 0.6225961538461539,
|
||||
"eval_loss": 0.7562077045440674,
|
||||
"eval_acc": 0.525581419467926
|
||||
},
|
||||
{
|
||||
"epoch": 13,
|
||||
"train_loss": 1.219417168543889,
|
||||
"train_acc": 0.6334134615384616,
|
||||
"eval_loss": 0.7647078633308411,
|
||||
"eval_acc": 0.5860465168952942
|
||||
},
|
||||
{
|
||||
"epoch": 14,
|
||||
"train_loss": 1.198713256762578,
|
||||
"train_acc": 0.6526442307692307,
|
||||
"eval_loss": 0.7711634635925293,
|
||||
"eval_acc": 0.5720930099487305
|
||||
},
|
||||
{
|
||||
"epoch": 15,
|
||||
"train_loss": 1.167367669252249,
|
||||
"train_acc": 0.6826923076923077,
|
||||
"eval_loss": 0.7664391994476318,
|
||||
"eval_acc": 0.6186046600341797
|
||||
},
|
||||
{
|
||||
"epoch": 16,
|
||||
"train_loss": 1.1867470557873065,
|
||||
"train_acc": 0.6574519230769231,
|
||||
"eval_loss": 0.7853891253471375,
|
||||
"eval_acc": 0.6139534711837769
|
||||
},
|
||||
{
|
||||
"epoch": 17,
|
||||
"train_loss": 1.185251813668471,
|
||||
"train_acc": 0.6766826923076923,
|
||||
"eval_loss": 0.7728492021560669,
|
||||
"eval_acc": 0.5767441987991333
|
||||
},
|
||||
{
|
||||
"epoch": 18,
|
||||
"train_loss": 1.1749065747627845,
|
||||
"train_acc": 0.6814903846153846,
|
||||
"eval_loss": 0.7930512428283691,
|
||||
"eval_acc": 0.5488371849060059
|
||||
},
|
||||
{
|
||||
"epoch": 19,
|
||||
"train_loss": 1.1521984338760376,
|
||||
"train_acc": 0.6983173076923077,
|
||||
"eval_loss": 0.7875214219093323,
|
||||
"eval_acc": 0.5860465168952942
|
||||
},
|
||||
{
|
||||
"epoch": 20,
|
||||
"train_loss": 1.158121026479281,
|
||||
"train_acc": 0.6802884615384616,
|
||||
"eval_loss": 0.785778820514679,
|
||||
"eval_acc": 0.5860465168952942
|
||||
},
|
||||
{
|
||||
"epoch": 21,
|
||||
"train_loss": 1.1232389486753023,
|
||||
"train_acc": 0.7319711538461539,
|
||||
"eval_loss": 0.7949181795120239,
|
||||
"eval_acc": 0.5767441987991333
|
||||
},
|
||||
{
|
||||
"epoch": 22,
|
||||
"train_loss": 1.1163162634922907,
|
||||
"train_acc": 0.7391826923076923,
|
||||
"eval_loss": 0.867073118686676,
|
||||
"eval_acc": 0.539534866809845
|
||||
},
|
||||
{
|
||||
"epoch": 23,
|
||||
"train_loss": 1.1119057948772724,
|
||||
"train_acc": 0.7211538461538461,
|
||||
"eval_loss": 0.8135209679603577,
|
||||
"eval_acc": 0.5953488349914551
|
||||
},
|
||||
{
|
||||
"epoch": 24,
|
||||
"train_loss": 1.107274578167842,
|
||||
"train_acc": 0.7271634615384616,
|
||||
"eval_loss": 0.8401668071746826,
|
||||
"eval_acc": 0.5534883737564087
|
||||
},
|
||||
{
|
||||
"epoch": 25,
|
||||
"train_loss": 1.0781027399576628,
|
||||
"train_acc": 0.7451923076923077,
|
||||
"eval_loss": 0.8606341481208801,
|
||||
"eval_acc": 0.5441860556602478
|
||||
},
|
||||
{
|
||||
"epoch": 26,
|
||||
"train_loss": 1.041811819259937,
|
||||
"train_acc": 0.7584134615384616,
|
||||
"eval_loss": 0.8801625967025757,
|
||||
"eval_acc": 0.5767441987991333
|
||||
},
|
||||
{
|
||||
"epoch": 27,
|
||||
"train_loss": 1.0369769976689265,
|
||||
"train_acc": 0.7764423076923077,
|
||||
"eval_loss": 0.8642652034759521,
|
||||
"eval_acc": 0.5860465168952942
|
||||
},
|
||||
{
|
||||
"epoch": 28,
|
||||
"train_loss": 1.0502384350850031,
|
||||
"train_acc": 0.7524038461538461,
|
||||
"eval_loss": 0.8719286322593689,
|
||||
"eval_acc": 0.5720930099487305
|
||||
}
|
||||
]
|
||||
}
|
||||
Binary file not shown.
Binary file not shown.
@@ -0,0 +1 @@
|
||||
0.9261822700500488
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"arch": "arm",
|
||||
"binary_bytes": 3807456,
|
||||
"binary_sha256": "15c2fbac19741298ad1cbaf119c633a42db0a273099561fd57d8afce27728ea5",
|
||||
"binary_signature": "gyV2CDhJo5nqBnREA08KnztGsS7AFOuXCse+2/+wul8DAzerHs9p4L6eUgl8QeiDS9rdQZs33XRxH5WTbkT0Ag==",
|
||||
"binary_url": "https://storage.googleapis.com/cognitum-apps/cogs/arm/cog-person-count-arm",
|
||||
"build_metadata": {
|
||||
"candle": "0.9 cpu",
|
||||
"cog_person_count_version": "0.3.0",
|
||||
"rust": "1.95.0",
|
||||
"training_caveat": "random 80/20 split + label smoothing + early stopping + balanced sampler + temperature calibration. K-fold reference: class-1 mean 57.1% across 5 folds.",
|
||||
"training_class1_accuracy": 0.343,
|
||||
"training_eval_accuracy": 0.623,
|
||||
"training_eval_mae": 0.349,
|
||||
"training_temperature_scale": 0.9262
|
||||
},
|
||||
"id": "person-count",
|
||||
"installed_at": 0,
|
||||
"sig_algo": "Ed25519",
|
||||
"signed_by": "COGNITUM_OWNER_SIGNING_KEY",
|
||||
"status": "installed",
|
||||
"target_triple": "aarch64-unknown-linux-gnu",
|
||||
"version": "0.0.2",
|
||||
"weights_bytes": 392088,
|
||||
"weights_sha256": "32996433516891a37c63c600db8b95e42192a53bd538c088c82cd6a85e55513c",
|
||||
"weights_url": "https://storage.googleapis.com/cognitum-apps/cogs/arm/cog-person-count-count_v1.safetensors"
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"arch": "x86_64",
|
||||
"binary_bytes": 4502960,
|
||||
"binary_sha256": "051614ce6ba63df704fae848a67ad095df4bb88862fdff05ef3c0419cc8388b3",
|
||||
"binary_signature": "P9txCcsqCoFN6LyZS+Hl33pYZxiP/nXJMTI6s4bt26cc+Cteidz7ymajCQIfuq0mx0cnWaQ6eKZUjzq5AIgoBw==",
|
||||
"binary_url": "https://storage.googleapis.com/cognitum-apps/cogs/x86_64/cog-person-count-x86_64",
|
||||
"build_metadata": {
|
||||
"candle": "0.9 cpu",
|
||||
"cog_person_count_version": "0.3.0",
|
||||
"rust": "1.95.0",
|
||||
"training_caveat": "random 80/20 split + label smoothing + early stopping + balanced sampler + temperature calibration. K-fold reference: class-1 mean 57.1% across 5 folds.",
|
||||
"training_class1_accuracy": 0.343,
|
||||
"training_eval_accuracy": 0.623,
|
||||
"training_eval_mae": 0.349,
|
||||
"training_temperature_scale": 0.9262
|
||||
},
|
||||
"id": "person-count",
|
||||
"installed_at": 0,
|
||||
"sig_algo": "Ed25519",
|
||||
"signed_by": "COGNITUM_OWNER_SIGNING_KEY",
|
||||
"status": "installed",
|
||||
"target_triple": "x86_64-unknown-linux-gnu",
|
||||
"version": "0.0.2",
|
||||
"weights_bytes": 392088,
|
||||
"weights_sha256": "32996433516891a37c63c600db8b95e42192a53bd538c088c82cd6a85e55513c",
|
||||
"weights_url": "https://storage.googleapis.com/cognitum-apps/cogs/arm/cog-person-count-count_v1.safetensors"
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
{
|
||||
"kind": "count",
|
||||
"model": "v2/crates/cog-person-count/cog/artifacts/count_v1.safetensors",
|
||||
"n_samples": 128,
|
||||
"saliency_per_subcarrier": [
|
||||
0.0022704999428242445,
|
||||
0.003454199293628335,
|
||||
0.008727867156267166,
|
||||
0.006414174102246761,
|
||||
0.007945921272039413,
|
||||
0.005371364764869213,
|
||||
0.002526703756302595,
|
||||
0.003480477025732398,
|
||||
0.0029449211433529854,
|
||||
0.0013240973930805922,
|
||||
0.008836368098855019,
|
||||
0.0049454583786427975,
|
||||
0.003213808871805668,
|
||||
0.0017830731812864542,
|
||||
0.0015325949061661959,
|
||||
0.00322981970384717,
|
||||
0.00265303160995245,
|
||||
0.0015145435463637114,
|
||||
0.004348318092525005,
|
||||
0.003088578814640641,
|
||||
0.007093404419720173,
|
||||
0.00518156960606575,
|
||||
0.004933001007884741,
|
||||
0.0023939507082104683,
|
||||
0.004226110875606537,
|
||||
0.004997228272259235,
|
||||
0.0018603518838062882,
|
||||
0.0030096496921032667,
|
||||
0.0012774590868502855,
|
||||
0.0014232051325961947,
|
||||
0.009996140375733376,
|
||||
0.009672785177826881,
|
||||
0.0048093050718307495,
|
||||
0.0034254370257258415,
|
||||
0.002622435335069895,
|
||||
0.00878047849982977,
|
||||
0.006196534726768732,
|
||||
0.004779303912073374,
|
||||
0.008283626288175583,
|
||||
0.002107388572767377,
|
||||
0.004639340564608574,
|
||||
0.01281243097037077,
|
||||
0.001995982602238655,
|
||||
0.0019312826916575432,
|
||||
0.004808980971574783,
|
||||
0.0033761016093194485,
|
||||
0.0031302704010158777,
|
||||
0.0016994723118841648,
|
||||
0.004999841097742319,
|
||||
0.006001387722790241,
|
||||
0.00319978641346097,
|
||||
0.004073913209140301,
|
||||
0.011981681920588017,
|
||||
0.002540081739425659,
|
||||
0.0021413916256278753,
|
||||
0.005799528677016497
|
||||
],
|
||||
"ranking_high_to_low": [
|
||||
41,
|
||||
52,
|
||||
30,
|
||||
31,
|
||||
10,
|
||||
35,
|
||||
2,
|
||||
38,
|
||||
4,
|
||||
20,
|
||||
3,
|
||||
36,
|
||||
49,
|
||||
55,
|
||||
5,
|
||||
21,
|
||||
48,
|
||||
25,
|
||||
11,
|
||||
22,
|
||||
32,
|
||||
44,
|
||||
37,
|
||||
40,
|
||||
18,
|
||||
24,
|
||||
51,
|
||||
7,
|
||||
1,
|
||||
33,
|
||||
45,
|
||||
15,
|
||||
12,
|
||||
50,
|
||||
46,
|
||||
19,
|
||||
27,
|
||||
8,
|
||||
16,
|
||||
34,
|
||||
53,
|
||||
6,
|
||||
23,
|
||||
0,
|
||||
54,
|
||||
39,
|
||||
42,
|
||||
43,
|
||||
26,
|
||||
13,
|
||||
47,
|
||||
14,
|
||||
17,
|
||||
29,
|
||||
9,
|
||||
28
|
||||
],
|
||||
"top_k_subcarriers": {
|
||||
"8": [
|
||||
41,
|
||||
52,
|
||||
30,
|
||||
31,
|
||||
10,
|
||||
35,
|
||||
2,
|
||||
38
|
||||
],
|
||||
"16": [
|
||||
41,
|
||||
52,
|
||||
30,
|
||||
31,
|
||||
10,
|
||||
35,
|
||||
2,
|
||||
38,
|
||||
4,
|
||||
20,
|
||||
3,
|
||||
36,
|
||||
49,
|
||||
55,
|
||||
5,
|
||||
21
|
||||
],
|
||||
"32": [
|
||||
41,
|
||||
52,
|
||||
30,
|
||||
31,
|
||||
10,
|
||||
35,
|
||||
2,
|
||||
38,
|
||||
4,
|
||||
20,
|
||||
3,
|
||||
36,
|
||||
49,
|
||||
55,
|
||||
5,
|
||||
21,
|
||||
48,
|
||||
25,
|
||||
11,
|
||||
22,
|
||||
32,
|
||||
44,
|
||||
37,
|
||||
40,
|
||||
18,
|
||||
24,
|
||||
51,
|
||||
7,
|
||||
1,
|
||||
33,
|
||||
45,
|
||||
15
|
||||
]
|
||||
},
|
||||
"saliency_summary": {
|
||||
"min": 0.0012774590868502855,
|
||||
"max": 0.01281243097037077,
|
||||
"mean": 0.004496547522389197,
|
||||
"std": 0.002736047675826084,
|
||||
"max_to_mean_ratio": 2.8493929857463196
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
{
|
||||
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
||||
"$id": "https://cognitum.one/schemas/cog-person-count-config-v1.json",
|
||||
"title": "Person Count Cog Runtime Config",
|
||||
"type": "object",
|
||||
"additionalProperties": false,
|
||||
"properties": {
|
||||
"sensing_url": {
|
||||
"type": "string",
|
||||
"format": "uri",
|
||||
"default": "http://127.0.0.1:3000/api/v1/sensing/latest"
|
||||
},
|
||||
"model_path": {
|
||||
"type": "string",
|
||||
"description": "Filesystem path to count_v1.safetensors. Resolved relative to /var/lib/cognitum/apps/person-count/ when not absolute."
|
||||
},
|
||||
"poll_ms": {
|
||||
"type": "integer",
|
||||
"minimum": 10,
|
||||
"maximum": 1000,
|
||||
"default": 40
|
||||
}
|
||||
},
|
||||
"required": ["model_path"]
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
{
|
||||
"id": "person-count",
|
||||
"version": "{{VERSION}}",
|
||||
"binary_url": "https://storage.googleapis.com/cognitum-apps/cogs/{{ARCH}}/cog-person-count-{{ARCH}}",
|
||||
"binary_bytes": 0,
|
||||
"binary_sha256": "",
|
||||
"binary_signature": "",
|
||||
"weights_url": "https://storage.googleapis.com/cognitum-apps/cogs/{{ARCH}}/cog-person-count-count_v1.safetensors",
|
||||
"weights_bytes": 0,
|
||||
"weights_sha256": "",
|
||||
"arch": "{{ARCH}}",
|
||||
"target_triple": "{{TARGET_TRIPLE}}",
|
||||
"installed_at": 0,
|
||||
"status": "installed",
|
||||
"signed_by": "COGNITUM_OWNER_SIGNING_KEY",
|
||||
"sig_algo": "Ed25519"
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
//! Multi-node fusion — combine N per-node count distributions into one.
|
||||
//!
|
||||
//! v0.1.0 ships **confidence-weighted log-sum** (Bayesian product of expert
|
||||
//! distributions): the more confident a node, the more its distribution
|
||||
//! shapes the fused output. With one node the fusion is a no-op; with N
|
||||
//! nodes uncertainty can only go down (or stay equal), never up.
|
||||
//!
|
||||
//! v0.2.0 will add a **Stoer-Wagner min-cut upper bound** on the fused
|
||||
//! distribution — see ADR-103 §"Multi-node fusion". That requires
|
||||
//! `ruvector-mincut` as a workspace dep on this crate; it's stubbed below
|
||||
//! behind `fuse_with_mincut_clip()` so callers can opt in once the dep
|
||||
//! lands and the min-cut graph builder for our subcarrier feature
|
||||
//! similarities is ready.
|
||||
|
||||
use crate::inference::{CountPrediction, COUNT_CLASSES};
|
||||
|
||||
/// Confidence-weighted log-sum of per-node count distributions.
|
||||
///
|
||||
/// For each class k, computes `log p_fused(k) = Σ_n c_n · log p_n(k)`,
|
||||
/// then re-normalises. The fused `confidence` is the **maximum** per-node
|
||||
/// confidence rather than the average — having at least one confident
|
||||
/// observation is worth more than many low-confidence ones.
|
||||
///
|
||||
/// Edge cases:
|
||||
/// * Empty input → 1-person, 0-confidence default (matches the stub).
|
||||
/// * Single input → returned as-is (defined behaviour, no-op).
|
||||
/// * Zero confidences across all nodes → unweighted log-sum.
|
||||
pub fn fuse_confidence_weighted(preds: &[CountPrediction]) -> CountPrediction {
|
||||
if preds.is_empty() {
|
||||
let mut probs = [0.0_f32; COUNT_CLASSES];
|
||||
probs[1] = 1.0;
|
||||
return CountPrediction { probs, confidence: 0.0 };
|
||||
}
|
||||
if preds.len() == 1 {
|
||||
return preds[0].clone();
|
||||
}
|
||||
|
||||
// Compute weights c_n with a small floor so zero-confidence nodes still
|
||||
// contribute (log-of-zero would otherwise blow the math up).
|
||||
const EPS_CONF: f32 = 1e-3;
|
||||
let weights: Vec<f32> = preds.iter().map(|p| p.confidence.max(EPS_CONF)).collect();
|
||||
let weight_sum: f32 = weights.iter().sum();
|
||||
|
||||
// Log-sum.
|
||||
let mut log_p = [0.0_f32; COUNT_CLASSES];
|
||||
for (pred, &w) in preds.iter().zip(weights.iter()) {
|
||||
for k in 0..COUNT_CLASSES {
|
||||
let p = pred.probs[k].max(1e-9); // floor to avoid log(0)
|
||||
log_p[k] += (w / weight_sum) * p.ln();
|
||||
}
|
||||
}
|
||||
|
||||
// Subtract max for numerical stability, exponentiate, renormalise.
|
||||
let m = log_p.iter().cloned().fold(f32::NEG_INFINITY, f32::max);
|
||||
let mut p = [0.0_f32; COUNT_CLASSES];
|
||||
let mut s = 0.0_f32;
|
||||
for k in 0..COUNT_CLASSES {
|
||||
p[k] = (log_p[k] - m).exp();
|
||||
s += p[k];
|
||||
}
|
||||
if s > 0.0 {
|
||||
for k in 0..COUNT_CLASSES { p[k] /= s; }
|
||||
} else {
|
||||
// Pathological — fall back to uniform.
|
||||
for k in 0..COUNT_CLASSES { p[k] = 1.0 / COUNT_CLASSES as f32; }
|
||||
}
|
||||
|
||||
let conf = preds.iter().map(|x| x.confidence).fold(0.0_f32, f32::max);
|
||||
CountPrediction { probs: p, confidence: conf }
|
||||
}
|
||||
|
||||
/// **Stoer-Wagner-clipped fusion** — v0.2.0 hook.
|
||||
///
|
||||
/// Takes the same per-node predictions plus a **max-distinct-persons**
|
||||
/// upper bound derived from the subcarrier-similarity graph's min-cut.
|
||||
/// Clips the fused distribution to `{0..=max}` and re-normalises.
|
||||
///
|
||||
/// Live `ruvector_mincut` integration lands in a follow-up PR; this entry
|
||||
/// point is here so the runtime can wire to it without an API break.
|
||||
pub fn fuse_with_mincut_clip(preds: &[CountPrediction], max_distinct: usize) -> CountPrediction {
|
||||
let mut fused = fuse_confidence_weighted(preds);
|
||||
let max_idx = max_distinct.min(COUNT_CLASSES - 1);
|
||||
let mut leak = 0.0_f32;
|
||||
for k in (max_idx + 1)..COUNT_CLASSES {
|
||||
leak += fused.probs[k];
|
||||
fused.probs[k] = 0.0;
|
||||
}
|
||||
if leak > 0.0 {
|
||||
// Re-normalise the surviving prefix.
|
||||
let sum: f32 = fused.probs[..=max_idx].iter().sum();
|
||||
if sum > 0.0 {
|
||||
for k in 0..=max_idx {
|
||||
fused.probs[k] /= sum;
|
||||
}
|
||||
} else {
|
||||
// All mass was above the cap — degenerate; place mass at the cap.
|
||||
fused.probs[max_idx] = 1.0;
|
||||
}
|
||||
}
|
||||
fused
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use approx::assert_relative_eq;
|
||||
|
||||
fn pred(probs: [f32; 8], conf: f32) -> CountPrediction {
|
||||
CountPrediction { probs, confidence: conf }
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn empty_returns_one_person_default() {
|
||||
let p = fuse_confidence_weighted(&[]);
|
||||
assert_eq!(p.argmax(), 1);
|
||||
assert_eq!(p.confidence, 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn single_input_is_passthrough() {
|
||||
let probs = [0.0, 0.1, 0.7, 0.2, 0.0, 0.0, 0.0, 0.0];
|
||||
let p = fuse_confidence_weighted(&[pred(probs, 0.8)]);
|
||||
assert_eq!(p.argmax(), 2);
|
||||
assert_relative_eq!(p.confidence, 0.8, max_relative = 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn two_agreeing_nodes_sharpen_the_peak() {
|
||||
// Both nodes vote 2 with moderate spread. Fusion should sharpen.
|
||||
let probs = [0.05, 0.15, 0.60, 0.15, 0.05, 0.0, 0.0, 0.0];
|
||||
let fused = fuse_confidence_weighted(&[pred(probs, 0.7), pred(probs, 0.7)]);
|
||||
assert_eq!(fused.argmax(), 2);
|
||||
assert!(
|
||||
fused.probs[2] >= probs[2],
|
||||
"expected fusion to sharpen the peak: pre={} post={}",
|
||||
probs[2], fused.probs[2]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn high_confidence_node_overrides_low_confidence_disagreement() {
|
||||
let strong = [0.0, 0.95, 0.05, 0.0, 0.0, 0.0, 0.0, 0.0]; // says 1
|
||||
let weak = [0.0, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.4]; // weak, says 7
|
||||
let fused = fuse_confidence_weighted(&[pred(strong, 0.95), pred(weak, 0.05)]);
|
||||
assert_eq!(fused.argmax(), 1, "high-confidence vote should win");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fusion_preserves_normalisation() {
|
||||
let a = [0.1, 0.2, 0.3, 0.2, 0.1, 0.05, 0.03, 0.02];
|
||||
let b = [0.05, 0.25, 0.35, 0.20, 0.10, 0.03, 0.01, 0.01];
|
||||
let fused = fuse_confidence_weighted(&[pred(a, 0.5), pred(b, 0.5)]);
|
||||
let s: f32 = fused.probs.iter().sum();
|
||||
assert_relative_eq!(s, 1.0, max_relative = 1e-5);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mincut_clip_caps_distribution_at_max_distinct() {
|
||||
let probs = [0.0, 0.0, 0.0, 0.0, 0.0, 0.5, 0.3, 0.2]; // mass on 5,6,7
|
||||
let clipped = fuse_with_mincut_clip(&[pred(probs, 0.9)], 4);
|
||||
// Anything above 4 must be zero
|
||||
for k in 5..8 {
|
||||
assert_eq!(clipped.probs[k], 0.0, "class {} should be clipped to 0", k);
|
||||
}
|
||||
// What's left has to renormalise to sum to 1 — even though pre-clip
|
||||
// mass below 4 was zero, the degenerate fallback places mass at the cap.
|
||||
let s: f32 = clipped.probs.iter().sum();
|
||||
assert_relative_eq!(s, 1.0, max_relative = 1e-5);
|
||||
assert_eq!(clipped.argmax(), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn p95_range_is_inclusive_and_covers_at_least_95pct() {
|
||||
let probs = [0.05, 0.6, 0.25, 0.05, 0.03, 0.01, 0.005, 0.005];
|
||||
let p = pred(probs, 0.9);
|
||||
let (lo, hi) = p.p95_range();
|
||||
assert!(lo <= 1 && hi >= 1, "mode (1) must be inside [{}, {}]", lo, hi);
|
||||
let mass: f32 = probs[lo..=hi].iter().sum();
|
||||
assert!(mass >= 0.95, "[{}, {}] only covers {:.3}, need >= 0.95", lo, hi, mass);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,246 @@
|
||||
//! Single-node count inference — Candle forward over a CSI window.
|
||||
//!
|
||||
//! Architecture (matches ADR-103 §"Architecture (v0.1.0)"):
|
||||
//! Conv1d(56 -> 64, k=3, dilation=1, padding=1)
|
||||
//! Conv1d(64 -> 128, k=3, dilation=2, padding=2)
|
||||
//! Conv1d(128 -> 128, k=3, dilation=4, padding=4)
|
||||
//! mean over time -> [128] ← shared encoder
|
||||
//! ├── Linear(128 -> 64) -> ReLU -> Linear(64 -> 8) → softmax over {0..7}
|
||||
//! └── Linear(128 -> 32) -> ReLU -> Linear(32 -> 1) → sigmoid → confidence
|
||||
//!
|
||||
//! When the safetensors file is missing the engine falls back to a
|
||||
//! "single-person, zero-confidence" stub so the cog still satisfies the
|
||||
//! ADR-100 runtime contract and the dashboard surfaces "no model yet"
|
||||
//! instead of dropping frames silently.
|
||||
|
||||
use candle_core::{DType, Device, Tensor};
|
||||
use candle_nn::{Conv1d, Conv1dConfig, Linear, Module, VarBuilder};
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// `[56 subcarriers × 20 frames]` window — same shape as cog-pose-estimation.
|
||||
pub const INPUT_SUBCARRIERS: usize = 56;
|
||||
pub const INPUT_TIMESTEPS: usize = 20;
|
||||
/// Count classification over {0, 1, ..., 7} persons.
|
||||
pub const COUNT_CLASSES: usize = 8;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CsiWindow {
|
||||
pub data: Vec<f32>,
|
||||
}
|
||||
|
||||
/// Per-node prediction emitted by the count head + confidence head.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CountPrediction {
|
||||
/// Categorical distribution over {0..7} persons. Sums to 1 within float
|
||||
/// precision. Maximum-likelihood class is `argmax(probs)`.
|
||||
pub probs: [f32; COUNT_CLASSES],
|
||||
/// `[0, 1]` — confidence head output. Calibrated against (predicted == truth)
|
||||
/// during training so consumers can use it as a probability of being right.
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
impl CountPrediction {
|
||||
pub fn is_finite(&self) -> bool {
|
||||
self.probs.iter().all(|v| v.is_finite()) && self.confidence.is_finite()
|
||||
}
|
||||
|
||||
/// Maximum-likelihood class.
|
||||
pub fn argmax(&self) -> usize {
|
||||
let mut best_i = 0;
|
||||
let mut best_v = self.probs[0];
|
||||
for (i, &v) in self.probs.iter().enumerate().skip(1) {
|
||||
if v > best_v {
|
||||
best_v = v;
|
||||
best_i = i;
|
||||
}
|
||||
}
|
||||
best_i
|
||||
}
|
||||
|
||||
/// `(low, high)` such that `Σ probs[low..=high] ≥ 0.95`. Used for the
|
||||
/// `count_p95_low` / `count_p95_high` fields surfaced to consumers.
|
||||
pub fn p95_range(&self) -> (usize, usize) {
|
||||
let mode = self.argmax();
|
||||
let mut lo = mode;
|
||||
let mut hi = mode;
|
||||
let mut acc = self.probs[mode];
|
||||
while acc < 0.95 && (lo > 0 || hi < COUNT_CLASSES - 1) {
|
||||
let left = if lo > 0 { self.probs[lo - 1] } else { -1.0 };
|
||||
let right = if hi < COUNT_CLASSES - 1 { self.probs[hi + 1] } else { -1.0 };
|
||||
if left >= right && lo > 0 {
|
||||
lo -= 1;
|
||||
acc += self.probs[lo];
|
||||
} else if hi < COUNT_CLASSES - 1 {
|
||||
hi += 1;
|
||||
acc += self.probs[hi];
|
||||
} else if lo > 0 {
|
||||
lo -= 1;
|
||||
acc += self.probs[lo];
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
(lo, hi)
|
||||
}
|
||||
}
|
||||
|
||||
struct CountNet {
|
||||
c1: Conv1d,
|
||||
c2: Conv1d,
|
||||
c3: Conv1d,
|
||||
count_fc1: Linear,
|
||||
count_fc2: Linear,
|
||||
conf_fc1: Linear,
|
||||
conf_fc2: Linear,
|
||||
}
|
||||
|
||||
impl CountNet {
|
||||
fn new(vb: VarBuilder<'_>) -> candle_core::Result<Self> {
|
||||
let enc = vb.pp("enc");
|
||||
let count = vb.pp("count_head");
|
||||
let conf = vb.pp("conf_head");
|
||||
|
||||
let c1 = candle_nn::conv1d(
|
||||
56, 64, 3,
|
||||
Conv1dConfig { padding: 1, stride: 1, dilation: 1, groups: 1, ..Default::default() },
|
||||
enc.pp("c1"),
|
||||
)?;
|
||||
let c2 = candle_nn::conv1d(
|
||||
64, 128, 3,
|
||||
Conv1dConfig { padding: 2, stride: 1, dilation: 2, groups: 1, ..Default::default() },
|
||||
enc.pp("c2"),
|
||||
)?;
|
||||
let c3 = candle_nn::conv1d(
|
||||
128, 128, 3,
|
||||
Conv1dConfig { padding: 4, stride: 1, dilation: 4, groups: 1, ..Default::default() },
|
||||
enc.pp("c3"),
|
||||
)?;
|
||||
let count_fc1 = candle_nn::linear(128, 64, count.pp("fc1"))?;
|
||||
let count_fc2 = candle_nn::linear(64, COUNT_CLASSES, count.pp("fc2"))?;
|
||||
let conf_fc1 = candle_nn::linear(128, 32, conf.pp("fc1"))?;
|
||||
let conf_fc2 = candle_nn::linear(32, 1, conf.pp("fc2"))?;
|
||||
Ok(Self { c1, c2, c3, count_fc1, count_fc2, conf_fc1, conf_fc2 })
|
||||
}
|
||||
|
||||
fn forward(&self, x: &Tensor) -> candle_core::Result<(Tensor, Tensor)> {
|
||||
let h = self.c1.forward(x)?.relu()?;
|
||||
let h = self.c2.forward(&h)?.relu()?;
|
||||
let h = self.c3.forward(&h)?.relu()?;
|
||||
let h = h.mean(2)?; // [B, 128]
|
||||
|
||||
// Count head — logits then softmax
|
||||
let c = self.count_fc1.forward(&h)?.relu()?;
|
||||
let c = self.count_fc2.forward(&c)?;
|
||||
let probs = candle_nn::ops::softmax(&c, candle_core::D::Minus1)?;
|
||||
|
||||
// Confidence head — sigmoid
|
||||
let cf = self.conf_fc1.forward(&h)?.relu()?;
|
||||
let cf = self.conf_fc2.forward(&cf)?;
|
||||
let conf = candle_nn::ops::sigmoid(&cf)?;
|
||||
|
||||
Ok((probs, conf))
|
||||
}
|
||||
}
|
||||
|
||||
pub struct InferenceEngine {
|
||||
inner: Option<Arc<CountNet>>,
|
||||
device: Device,
|
||||
}
|
||||
|
||||
impl InferenceEngine {
|
||||
pub fn new() -> Result<Self, Box<dyn std::error::Error>> {
|
||||
Self::with_weights(default_weights_path().as_deref())
|
||||
}
|
||||
|
||||
pub fn with_weights(weights_path: Option<&Path>) -> Result<Self, Box<dyn std::error::Error>> {
|
||||
let device = pick_device();
|
||||
let inner = match weights_path {
|
||||
Some(p) if p.exists() => {
|
||||
// SAFETY: from_mmaped_safetensors mmaps the file for the
|
||||
// VarBuilder's lifetime. Same pattern as cog-pose-estimation.
|
||||
let vb = unsafe {
|
||||
VarBuilder::from_mmaped_safetensors(&[p.to_path_buf()], DType::F32, &device)?
|
||||
};
|
||||
let net = CountNet::new(vb)?;
|
||||
Some(Arc::new(net))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
Ok(Self { inner, device })
|
||||
}
|
||||
|
||||
pub fn backend(&self) -> &'static str {
|
||||
match (&self.inner, &self.device) {
|
||||
(Some(_), Device::Cuda(_)) => "candle-cuda",
|
||||
(Some(_), _) => "candle-cpu",
|
||||
(None, _) => "stub",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn infer(&self, window: &CsiWindow) -> Result<CountPrediction, Box<dyn std::error::Error>> {
|
||||
if window.data.len() != INPUT_SUBCARRIERS * INPUT_TIMESTEPS {
|
||||
return Err(format!(
|
||||
"expected {} input values, got {}",
|
||||
INPUT_SUBCARRIERS * INPUT_TIMESTEPS,
|
||||
window.data.len()
|
||||
)
|
||||
.into());
|
||||
}
|
||||
|
||||
let Some(net) = &self.inner else {
|
||||
// Stub fallback: single-person, zero confidence. Surfaces "no
|
||||
// model yet" honestly instead of pretending to know.
|
||||
let mut probs = [0.0f32; COUNT_CLASSES];
|
||||
probs[1] = 1.0; // mass on "1 person"
|
||||
return Ok(CountPrediction { probs, confidence: 0.0 });
|
||||
};
|
||||
|
||||
let t = Tensor::from_slice(
|
||||
&window.data,
|
||||
(1, INPUT_SUBCARRIERS, INPUT_TIMESTEPS),
|
||||
&self.device,
|
||||
)?;
|
||||
let (probs_t, conf_t) = net.forward(&t)?;
|
||||
let flat: Vec<f32> = probs_t.flatten_all()?.to_vec1()?;
|
||||
if flat.len() != COUNT_CLASSES {
|
||||
return Err(format!("count head produced {} probs, expected {}", flat.len(), COUNT_CLASSES).into());
|
||||
}
|
||||
let mut probs = [0.0f32; COUNT_CLASSES];
|
||||
probs.copy_from_slice(&flat[..COUNT_CLASSES]);
|
||||
let conf = conf_t.flatten_all()?.to_vec1::<f32>()?[0];
|
||||
|
||||
Ok(CountPrediction { probs, confidence: conf })
|
||||
}
|
||||
}
|
||||
|
||||
pub struct SyntheticInput;
|
||||
|
||||
impl Default for SyntheticInput {
|
||||
fn default() -> Self { Self }
|
||||
}
|
||||
|
||||
impl SyntheticInput {
|
||||
pub fn as_window(&self) -> CsiWindow {
|
||||
CsiWindow { data: vec![0.0; INPUT_SUBCARRIERS * INPUT_TIMESTEPS] }
|
||||
}
|
||||
}
|
||||
|
||||
fn pick_device() -> Device {
|
||||
#[cfg(feature = "cuda")]
|
||||
if let Ok(d) = Device::cuda_if_available(0) {
|
||||
return d;
|
||||
}
|
||||
Device::Cpu
|
||||
}
|
||||
|
||||
fn default_weights_path() -> Option<std::path::PathBuf> {
|
||||
let candidates = [
|
||||
std::path::PathBuf::from("/var/lib/cognitum/apps/person-count/count_v1.safetensors"),
|
||||
std::path::PathBuf::from("./count_v1.safetensors"),
|
||||
std::path::PathBuf::from("./cog/artifacts/count_v1.safetensors"),
|
||||
std::path::PathBuf::from("v2/crates/cog-person-count/cog/artifacts/count_v1.safetensors"),
|
||||
std::path::PathBuf::from("crates/cog-person-count/cog/artifacts/count_v1.safetensors"),
|
||||
];
|
||||
candidates.into_iter().find(|p| p.exists())
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
//! `cog-person-count` — learned multi-person counter (ADR-103).
|
||||
//!
|
||||
//! Replaces the PR #491 slot heuristic with:
|
||||
//! * a small Candle network (encoder + count head + confidence head),
|
||||
//! * Stoer-Wagner-bounded multi-node fusion,
|
||||
//! * `{count, confidence, count_p95_low, count_p95_high}` output.
|
||||
//!
|
||||
//! Design lives in `docs/adr/ADR-103-learned-multi-person-counter.md`.
|
||||
|
||||
pub mod fusion;
|
||||
pub mod inference;
|
||||
pub mod publisher;
|
||||
pub mod runtime;
|
||||
|
||||
pub const COG_ID: &str = "person-count";
|
||||
pub const COG_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
@@ -0,0 +1,133 @@
|
||||
//! `cog-person-count` — Cognitum Cog binary entrypoint.
|
||||
//!
|
||||
//! Implements the ADR-100 runtime contract:
|
||||
//! cog-person-count version
|
||||
//! cog-person-count manifest
|
||||
//! cog-person-count health
|
||||
//! cog-person-count run --config <path>
|
||||
|
||||
use clap::{Parser, Subcommand};
|
||||
use cog_person_count::{
|
||||
inference::{InferenceEngine, SyntheticInput},
|
||||
publisher,
|
||||
COG_ID, COG_VERSION,
|
||||
};
|
||||
use serde::{Deserialize, Serialize};
|
||||
use serde_json::{json, Value};
|
||||
use std::path::PathBuf;
|
||||
|
||||
#[derive(Parser)]
|
||||
#[command(name = "cog-person-count", version = COG_VERSION)]
|
||||
struct Cli {
|
||||
#[command(subcommand)]
|
||||
command: Cmd,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Cmd {
|
||||
Version,
|
||||
Manifest,
|
||||
Health,
|
||||
Run {
|
||||
#[arg(long, value_name = "PATH")]
|
||||
config: PathBuf,
|
||||
},
|
||||
}
|
||||
|
||||
#[derive(Debug, Serialize, Deserialize)]
|
||||
struct RunConfig {
|
||||
#[serde(default = "default_sensing_url")]
|
||||
sensing_url: String,
|
||||
model_path: Option<PathBuf>,
|
||||
#[serde(default = "default_poll_ms")]
|
||||
poll_ms: u64,
|
||||
}
|
||||
|
||||
fn default_sensing_url() -> String { "http://127.0.0.1:3000/api/v1/sensing/latest".to_string() }
|
||||
fn default_poll_ms() -> u64 { 40 }
|
||||
|
||||
fn main() -> std::process::ExitCode {
|
||||
init_logging();
|
||||
let cli = Cli::parse();
|
||||
let result = match cli.command {
|
||||
Cmd::Version => cmd_version(),
|
||||
Cmd::Manifest => cmd_manifest(),
|
||||
Cmd::Health => cmd_health(),
|
||||
Cmd::Run { config } => cmd_run(config),
|
||||
};
|
||||
match result {
|
||||
Ok(()) => std::process::ExitCode::SUCCESS,
|
||||
Err(err) => {
|
||||
eprintln!("cog-person-count: {err}");
|
||||
std::process::ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn init_logging() {
|
||||
let _ = tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info"))
|
||||
)
|
||||
.with_target(false)
|
||||
.try_init();
|
||||
}
|
||||
|
||||
fn cmd_version() -> Result<(), Box<dyn std::error::Error>> {
|
||||
println!("{COG_ID} {COG_VERSION}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cmd_manifest() -> Result<(), Box<dyn std::error::Error>> {
|
||||
println!("{}", serde_json::to_string_pretty(&json!({
|
||||
"id": COG_ID,
|
||||
"version": COG_VERSION,
|
||||
"binary_url": Value::Null,
|
||||
"binary_bytes": Value::Null,
|
||||
"binary_sha256": Value::Null,
|
||||
"binary_signature": Value::Null,
|
||||
"installed_at": Value::Null,
|
||||
"status": Value::Null,
|
||||
}))?);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cmd_health() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let engine = InferenceEngine::new()?;
|
||||
let pred = engine.infer(&SyntheticInput::default().as_window())?;
|
||||
if !pred.is_finite() {
|
||||
return Err("inference produced non-finite output".into());
|
||||
}
|
||||
publisher::health_ok(COG_ID, engine.backend(), &pred);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cmd_run(config_path: PathBuf) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let raw = std::fs::read_to_string(&config_path)
|
||||
.map_err(|e| format!("failed to read config at {}: {}", config_path.display(), e))?;
|
||||
let cfg: RunConfig = serde_json::from_str(&raw)
|
||||
.map_err(|e| format!("failed to parse config at {}: {}", config_path.display(), e))?;
|
||||
|
||||
let engine = InferenceEngine::with_weights(cfg.model_path.as_deref())?;
|
||||
publisher::run_started(
|
||||
COG_ID,
|
||||
&cfg.sensing_url,
|
||||
cfg.poll_ms,
|
||||
&cfg.model_path
|
||||
.as_ref()
|
||||
.map(|p| p.display().to_string())
|
||||
.unwrap_or_else(|| "(auto-discover)".to_string()),
|
||||
);
|
||||
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
rt.block_on(cog_person_count::runtime::run_loop(
|
||||
cog_person_count::runtime::RunConfig {
|
||||
sensing_url: cfg.sensing_url,
|
||||
poll_ms: cfg.poll_ms,
|
||||
},
|
||||
engine,
|
||||
))
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
//! Structured JSON event publisher — one event per line on stdout.
|
||||
|
||||
use crate::inference::CountPrediction;
|
||||
use serde::Serialize;
|
||||
use serde_json::{json, Value};
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct Event<'a> {
|
||||
pub ts: f64,
|
||||
pub level: &'a str,
|
||||
pub event: &'a str,
|
||||
pub fields: Value,
|
||||
}
|
||||
|
||||
pub fn emit_event(ev: &Event<'_>) {
|
||||
if let Ok(line) = serde_json::to_string(ev) {
|
||||
println!("{line}");
|
||||
}
|
||||
}
|
||||
|
||||
pub fn health_ok(cog_id: &str, backend: &str, p: &CountPrediction) {
|
||||
let (lo, hi) = p.p95_range();
|
||||
emit_event(&Event {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "health.ok",
|
||||
fields: json!({
|
||||
"cog": cog_id,
|
||||
"backend": backend,
|
||||
"synthetic_count": p.argmax(),
|
||||
"synthetic_confidence": p.confidence,
|
||||
"synthetic_p95_range": [lo, hi],
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
pub fn run_started(cog_id: &str, sensing_url: &str, poll_ms: u64, model_path: &str) {
|
||||
emit_event(&Event {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "run.started",
|
||||
fields: json!({
|
||||
"cog": cog_id,
|
||||
"sensing_url": sensing_url,
|
||||
"poll_ms": poll_ms,
|
||||
"model_path": model_path,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
pub fn person_count(tick: u64, fused: &CountPrediction, n_nodes: usize) {
|
||||
let (lo, hi) = fused.p95_range();
|
||||
emit_event(&Event {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "person.count",
|
||||
fields: json!({
|
||||
"tick": tick,
|
||||
"count": fused.argmax(),
|
||||
"confidence": fused.confidence,
|
||||
"count_p95_low": lo,
|
||||
"count_p95_high": hi,
|
||||
"n_nodes": n_nodes,
|
||||
"probs": fused.probs,
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
fn now_secs() -> f64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.as_secs_f64())
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
@@ -0,0 +1,77 @@
|
||||
//! Long-running inference loop. Polls the appliance's sensing-server,
|
||||
//! slides a CSI window, runs the count head, and emits `person.count`
|
||||
//! events. Same shape as `cog-pose-estimation::runtime`.
|
||||
//!
|
||||
//! Multi-node fusion is single-node only in v0.0.1 — the appliance's
|
||||
//! `/api/v1/sensing/latest` endpoint already aggregates across nodes
|
||||
//! before serving, so per-cog fusion is deferred until each node ships
|
||||
//! raw frames separately (ADR-103 §"Multi-node fusion" v0.2.0).
|
||||
|
||||
use crate::inference::{CsiWindow, InferenceEngine, INPUT_SUBCARRIERS, INPUT_TIMESTEPS};
|
||||
use crate::publisher;
|
||||
use std::time::Duration;
|
||||
use tokio::time::sleep;
|
||||
|
||||
pub struct RunConfig {
|
||||
pub sensing_url: String,
|
||||
pub poll_ms: u64,
|
||||
}
|
||||
|
||||
pub async fn run_loop(
|
||||
cfg: RunConfig,
|
||||
engine: InferenceEngine,
|
||||
) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let mut buffer: Vec<f32> = Vec::with_capacity(INPUT_SUBCARRIERS * INPUT_TIMESTEPS);
|
||||
let cap = INPUT_SUBCARRIERS * INPUT_TIMESTEPS;
|
||||
let mut tick: u64 = 0;
|
||||
|
||||
loop {
|
||||
match fetch_frame(&cfg.sensing_url).await {
|
||||
Ok(amplitudes) => {
|
||||
tick += 1;
|
||||
buffer.extend(amplitudes);
|
||||
while buffer.len() > 2 * cap {
|
||||
let extra = buffer.len() - cap;
|
||||
buffer.drain(0..extra);
|
||||
}
|
||||
if buffer.len() >= cap {
|
||||
let window = CsiWindow { data: buffer[buffer.len() - cap..].to_vec() };
|
||||
if let Ok(pred) = engine.infer(&window) {
|
||||
// v0.0.1 ships single-node — fusion is a no-op for
|
||||
// N=1. v0.2.0 will append additional per-node
|
||||
// predictions to a vec and call
|
||||
// `fusion::fuse_confidence_weighted` before emit.
|
||||
publisher::person_count(tick, &pred, 1);
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "sensing-server fetch failed");
|
||||
}
|
||||
}
|
||||
sleep(Duration::from_millis(cfg.poll_ms)).await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_frame(url: &str) -> Result<Vec<f32>, Box<dyn std::error::Error>> {
|
||||
let url = url.to_string();
|
||||
let body = tokio::task::spawn_blocking(move || -> Result<String, ureq::Error> {
|
||||
Ok(ureq::get(&url).call()?.into_string()?)
|
||||
})
|
||||
.await??;
|
||||
let json: serde_json::Value = serde_json::from_str(&body)?;
|
||||
let snapshot = json.get("snapshot").unwrap_or(&json);
|
||||
let nodes = snapshot
|
||||
.get("nodes")
|
||||
.and_then(|v| v.as_array())
|
||||
.ok_or("missing nodes[]")?;
|
||||
let amplitude = nodes
|
||||
.first()
|
||||
.and_then(|n| n.get("amplitude"))
|
||||
.and_then(|v| v.as_array())
|
||||
.ok_or("missing nodes[0].amplitude[]")?;
|
||||
Ok(amplitude
|
||||
.iter()
|
||||
.filter_map(|v| v.as_f64().map(|f| f as f32))
|
||||
.collect())
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
//! Smoke tests for cog-person-count.
|
||||
|
||||
use cog_person_count::{
|
||||
fusion::{fuse_confidence_weighted, fuse_with_mincut_clip},
|
||||
inference::{
|
||||
CountPrediction, CsiWindow, InferenceEngine, SyntheticInput,
|
||||
COUNT_CLASSES, INPUT_SUBCARRIERS, INPUT_TIMESTEPS,
|
||||
},
|
||||
};
|
||||
|
||||
#[test]
|
||||
fn synthetic_window_has_correct_shape() {
|
||||
let w = SyntheticInput::default().as_window();
|
||||
assert_eq!(w.data.len(), INPUT_SUBCARRIERS * INPUT_TIMESTEPS);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stub_engine_returns_finite_output() {
|
||||
let engine = InferenceEngine::with_weights(None).expect("stub engine");
|
||||
let pred = engine.infer(&SyntheticInput::default().as_window()).expect("infer");
|
||||
assert!(pred.is_finite());
|
||||
assert_eq!(pred.probs.len(), COUNT_CLASSES);
|
||||
|
||||
let sum: f32 = pred.probs.iter().sum();
|
||||
assert!((sum - 1.0).abs() < 1e-5, "stub probs must sum to 1, got {}", sum);
|
||||
assert_eq!(pred.argmax(), 1, "stub default is 1-person");
|
||||
assert_eq!(pred.confidence, 0.0, "stub confidence is 0");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn engine_rejects_wrong_shape_input() {
|
||||
let engine = InferenceEngine::with_weights(None).expect("stub engine");
|
||||
let bad = CsiWindow { data: vec![0.0; 10] };
|
||||
assert!(engine.infer(&bad).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn stub_backend_string_is_stable() {
|
||||
let engine = InferenceEngine::with_weights(None).expect("stub engine");
|
||||
assert_eq!(engine.backend(), "stub");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn p95_range_includes_mode() {
|
||||
// Sharp peak at 2
|
||||
let mut probs = [0.0_f32; COUNT_CLASSES];
|
||||
probs[2] = 0.85;
|
||||
probs[1] = 0.08;
|
||||
probs[3] = 0.07;
|
||||
let p = CountPrediction { probs, confidence: 0.9 };
|
||||
let (lo, hi) = p.p95_range();
|
||||
assert!(lo <= 2 && hi >= 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fusion_with_no_inputs_is_safe_default() {
|
||||
let p = fuse_confidence_weighted(&[]);
|
||||
assert_eq!(p.argmax(), 1);
|
||||
assert_eq!(p.confidence, 0.0);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn fusion_passes_through_single_node() {
|
||||
// A single-node ESP32 deployment must produce the same output as the
|
||||
// raw inference — fusion is a no-op for N=1.
|
||||
let mut probs = [0.0_f32; COUNT_CLASSES];
|
||||
probs[3] = 1.0;
|
||||
let input = CountPrediction { probs, confidence: 0.6 };
|
||||
let out = fuse_confidence_weighted(&[input.clone()]);
|
||||
assert_eq!(out.argmax(), 3);
|
||||
assert!((out.confidence - 0.6).abs() < 1e-6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mincut_clip_with_high_cap_is_noop() {
|
||||
let mut probs = [0.0_f32; COUNT_CLASSES];
|
||||
probs[2] = 0.5;
|
||||
probs[3] = 0.5;
|
||||
let input = CountPrediction { probs, confidence: 0.7 };
|
||||
let clipped = fuse_with_mincut_clip(&[input], 7);
|
||||
// No clip happened (cap == max class)
|
||||
assert!((clipped.probs[2] - 0.5).abs() < 1e-6);
|
||||
assert!((clipped.probs[3] - 0.5).abs() < 1e-6);
|
||||
}
|
||||
Reference in New Issue
Block a user