mirror of
https://github.com/ruvnet/RuView
synced 2026-07-20 17:03:24 +00:00
test(calibration): full-loop integration test — baseline→enroll→train→infer proven in-process (ADR-151 §7 gap, software half)
Closes the software half of PR #989's headline validation gap: the complete calibration loop had never run end-to-end anywhere, even in-process. tests/full_loop.rs (412 lines, deterministic xorshift32 room simulator, HT20/52-subcarrier/20Hz, same fingerprint family as the ADR-135 roundtrip test) now drives the CLI's exact stage order through the public API: 1. baseline — 600 static frames, zero motion flags post-warmup, calibration_uuid() exactly as the CLI derives it 2. enroll — all 8 AnchorLabel::SEQUENCE anchors through AnchorQualityGate::default(), session is_complete() 3. extract — AnchorFeature::from_series recovers injected 0.25Hz and 0.125Hz breathing within ±0.04Hz 4. train — SpecialistBank::train fits all 6 specialists; JSON round-trip and the runtime consumes the RELOADED bank 5. infer — positive: never-enrolled 0.30Hz subject reads present, 18±2 BPM; negative: empty window reads absent; degradation: foreign baseline_id flags STALE Seed-robust (5 seeds), passes with and without default features: 36 unit + 1 integration green. Validation docs updated (ADR-151 §7 + integration doc §7 matrix): what remains is strictly the on-target hardware session (real CSI, physically empty room, operator performing the guided anchors). Three behavioral findings from building the test are recorded for pre-session triage: z-band squeeze between baseline motion flagging (z>2.0) and the still- anchor gate (presence_z≥1.5) — likeliest on-hardware enroll failure; variance-only PresenceSpecialist missing motionless-person mean shift; ungated breathing_hz/heart_hz in noise-window embeddings. Co-Authored-By: RuFlo <ruv@ruv.net>
This commit is contained in:
@@ -247,7 +247,7 @@ wifi-densepose room-status --room living-room
|
||||
| **P5** | heartbeat + restlessness + anomaly specialists; `runtime.rs` mixture + veto + confidence gating | End-to-end RoomState on hardware; anomaly veto verified | ✅ Done (`runtime.rs`, CLI `room-watch`; breathing read live on COM8 ESP32) |
|
||||
| **P6** | Baseline-drift `STALE` invalidation; SONA online adaptation; optional ADR-105 federation; HF teacher–student distillation | Drift marks bank STALE; AetherArena entry | ◐ Partial (STALE done; SONA/federation/HF-backbone = follow-ups) |
|
||||
|
||||
**Current status (2026-06-09):** Stages 1–5 implemented with *statistical* specialists (threshold/prototype/autocorrelation). 54 unit tests (incl. multistatic), all passing under qemu-aarch64. **Validation scope is precise:** baseline capture + HTTP API + auth are proven on real CSI (Pi-5 nexmon, 6,813 frames; and an ESP32-S3). The one live runtime signal (breathing ~16–31 BPM via `room-watch`) used the *stateless* breathing head, **not** a trained bank; a clean empty-room `enroll → train-room → room-watch` loop has **not** yet run on-target — the headline remaining gap. Open refinements: `--source-format adr018v6` (drive from the Pi's own nexmon), phase-based breathing carrier, RVF/HNSW storage, and the ADR-150 frozen HF backbone the specialists would distill from.
|
||||
**Current status (2026-06-10):** Stages 1–5 implemented with *statistical* specialists (threshold/prototype/autocorrelation). 55 tests (35 unit incl. multistatic + 1 full-loop integration + 19 CLI), all passing under qemu-aarch64. **Validation scope is precise:** baseline capture + HTTP API + auth are proven on real CSI (Pi-5 nexmon, 6,813 frames; and an ESP32-S3). The complete `baseline → enroll → train-room → infer` loop is now **proven in-process** on deterministic synthetic CSI (`tests/full_loop.rs`: clean baseline with zero motion flags, 8/8 anchors through the quality gate, 6 specialists trained, JSON bank round-trip, trained-bank inference 18±2 BPM positive / absent negative / foreign-baseline STALE; seed-robust). The one live runtime signal (breathing ~16–31 BPM via `room-watch`) used the *stateless* breathing head, **not** a trained bank; the clean empty-room loop has **not** yet run on-target — the remaining gap is strictly the hardware session (empty room + operator anchors). See the integration doc §7 for three behavioral findings from the full-loop test (z-band squeeze, variance-only presence, ungated hz embedding) worth fixing before that session. SOTA-intake decisions affecting this system (geometry conditioning, checkerboard alignment) are recorded in ADR-152. Open refinements: `--source-format adr018v6` (drive from the Pi's own nexmon), phase-based breathing carrier, RVF/HNSW storage, and the ADR-150 frozen HF backbone the specialists would distill from.
|
||||
|
||||
Validation per CLAUDE.md: `cargo test --workspace --no-default-features` green; hardware verification on the ESP32-S3 (currently COM8) before any release; witness bundle regenerated if the proof surface changes.
|
||||
|
||||
|
||||
@@ -214,20 +214,21 @@ exchange bank/model deltas, never raw CSI. Hardening already in place:
|
||||
|
||||
## 7. Status & validation
|
||||
|
||||
- **Implemented:** all 5 stages + multistatic fusion; CLI + Stage-1 HTTP API (auth + path-traversal hardened). **54 tests** (35 calibration + 19 CLI), all passing under qemu-aarch64.
|
||||
- **Implemented:** all 5 stages + multistatic fusion; CLI + Stage-1 HTTP API (auth + path-traversal hardened). **55 tests** (35 calibration unit + 1 full-loop integration + 19 CLI), all passing under qemu-aarch64.
|
||||
|
||||
**Precise validation matrix (don't overstate this — no clean full calibration has run on-target yet):**
|
||||
|
||||
| Stage | Pi-5 (real nexmon→`0xC5110001`, 6,813 frames) | ESP32-S3 (COM8, `edge_tier=0`) | qemu / unit |
|
||||
| Stage | Pi-5 (real nexmon→`0xC5110001`, 6,813 frames) | ESP32-S3 (COM8, `edge_tier=0`) | qemu / unit / integration |
|
||||
|---|---|---|---|
|
||||
| baseline capture + HTTP API + **auth gate** | ✅ | ✅ (120-frame) | — |
|
||||
| **clean** empty-room baseline | ❌ `motion_flagged` (artifact) | ❌ (occupied) | — |
|
||||
| enroll → train-room | ❌ | ❌ (needs operator poses) | logic ✅ |
|
||||
| runtime infer | ❌ on-target | ◐ single-node breathing ~16–31 BPM via the **stateless** head (not a trained bank) + node-id fusion | veto/STALE/fusion ✅ |
|
||||
| baseline capture + HTTP API + **auth gate** | ✅ | ✅ (120-frame) | full-loop ✅ |
|
||||
| **clean** empty-room baseline | ❌ `motion_flagged` (artifact) | ❌ (occupied) | full-loop ✅ (synthetic, zero motion flags) |
|
||||
| enroll → train-room | ❌ | ❌ (needs operator poses) | full-loop ✅ (8/8 anchors, 6 specialists, JSON round-trip) |
|
||||
| runtime infer | ❌ on-target | ◐ single-node breathing ~16–31 BPM via the **stateless** head (not a trained bank) + node-id fusion | full-loop ✅ (trained bank: 18±2 BPM positive, absent negative, foreign-baseline STALE) |
|
||||
|
||||
So capture + API + auth are proven on real CSI (both boxes); a **clean-baseline `enroll → train-room → room-watch`** loop is **not yet proven on-target** — that's the headline gap to close on the appliance.
|
||||
The complete `baseline → enroll → train-room → infer` loop is now **proven in-process** on deterministic synthetic CSI (`wifi-densepose-calibration/tests/full_loop.rs` — drives the CLI's exact stage order through the public API, seed-robust across 5 seeds, runs with and without default features). Capture + API + auth are proven on real CSI (both boxes). What remains is strictly the **on-target** run: real CSI, a physically empty room for baseline, and an operator performing the 8 guided anchors — that hardware session is the last open item.
|
||||
|
||||
- **Known follow-ups (appliance backlog):** `--source-format adr018v6` to drive calibration from the Pi's own nexmon (no ESP32/transcoder); a clean empty-room enroll→train→infer on-target; phase-based (vs mean-amplitude) breathing carrier; RVF/HNSW persistence (currently JSON); enroll/train HTTP endpoints (live `/room/state` already added); ADR-150 Hailo backbone; true 2-node multistatic; ADR-105 federation.
|
||||
- **Known follow-ups (appliance backlog):** `--source-format adr018v6` to drive calibration from the Pi's own nexmon (no ESP32/transcoder); the on-target clean-room enroll→train→infer session (above); phase-based (vs mean-amplitude) breathing carrier; RVF/HNSW persistence (currently JSON); enroll/train HTTP endpoints (live `/room/state` already added); ADR-150 Hailo backbone; true 2-node multistatic; ADR-105 federation.
|
||||
- **Behavioral findings from the full-loop test (pre-hardware-session fixes worth considering):** (1) *z-band squeeze*: `BaselineCalibration::deviation` flags motion at `amplitude_z_median > 2.0` while the still-anchor gate needs `presence_z ≥ 1.5` — a strongly-reflecting still person can be rejected as "moving"; presence strength and motion are conflated. Most likely on-hardware enroll failure mode. (2) `PresenceSpecialist` is variance-only — a motionless person raises the scalar *mean* but not variance, so a quiet subject can read "absent" at runtime even though enroll accepted them; adding mean/`presence_z` to the presence decision would close it. (3) `Features::from_series` emits a best-in-band `breathing_hz`/`heart_hz` even at negligible score, injecting random in-band frequencies into the prototype embeddings for noise windows; gating the hz fields on score would tighten posture/anomaly classification.
|
||||
|
||||
**Reference:** ADR-151 (`docs/adr/ADR-151-room-calibration-specialist-training.md`), ADR-135 (baseline),
|
||||
ADR-029 (multistatic), ADR-150 (RF Foundation Encoder), ADR-105 (federation), ADR-147 (OccWorld/Hailo).
|
||||
|
||||
Reference in New Issue
Block a user