Compare commits

..

1 Commits

Author SHA1 Message Date
ruv c27d6cc98e fix(sensing-server): make multistatic guard interval operator-configurable (#1049)
Two ESP32-S3 nodes on WiFi/ESP-NOW sync drift 10-150 ms (~70 ms typ.), exceeding
the 60 ms default guard → permanent trust demotion to Restricted, all pose output
suppressed, 200k+ errors, no escape but a container restart.

Add a direct WDP_GUARD_INTERVAL_US override (+ optional WDP_SOFT_GUARD_US) to
multistatic_guard_config_from_env. Precedence (most-specific wins): direct
override > WDP_TDM_SLOTS+WDP_TDM_SLOT_US schedule-derived > 60ms/20ms default.
Soft band always clamped strictly below hard; malformed/zero ignored (falls back,
never breaks fusion). Effective guard logged at startup.

Pinned by 6 tests (multistatic_guard_config_tests). sensing-server bin tests
449 -> 455, 0 failed. Python proof PASS, hash unchanged (off signal path).

Closes #1049.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 13:41:43 -04:00
3 changed files with 129 additions and 288 deletions
+3
View File
@@ -7,6 +7,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Fixed
- **Multistatic fusion guard interval is now operator-configurable — fixes permanent trust demotion with WiFi-synced ESP32 nodes (#1049).** Two independently-clocked ESP32-S3 boards on ESP-NOW sync drift 10150 ms (typ. ~70 ms) — the 100 ms beacon + WiFi-MAC jitter cannot hold them within the published 60 ms default guard, so the governed-trust cycle permanently demoted to `Restricted`, suppressed all pose output, and spun the error counter to 200k+ with **no escape hatch but a container restart**. Added a **direct `WDP_GUARD_INTERVAL_US` override** (+ optional `WDP_SOFT_GUARD_US`) to `multistatic_guard_config_from_env`, so a deployment can lift the hard guard past its measured spread (e.g. `WDP_GUARD_INTERVAL_US=200000`) without having to know its exact TDM schedule. Precedence is most-specific-wins: a direct override beats the existing `WDP_TDM_SLOTS`+`WDP_TDM_SLOT_US` schedule-derived guard, which beats the 60 ms/20 ms default; the override is applied on top of whichever base is selected, the soft band is always clamped strictly below the hard guard, and a malformed/zero value is ignored (falls back to the base rather than breaking fusion). The effective guard is now logged at startup. Pinned by 6 new tests (`multistatic_guard_config_tests`): direct-override-wins / beats-TDM-derived / soft-clamped-below-hard / lowering-hard-pulls-soft-down / malformed-or-zero-falls-back / default-when-unset. `wifi-densepose-sensing-server` bin tests **449 → 455**, 0 failed; Python proof VERDICT PASS, hash unchanged (off the signal proof path).
### Security
- **`wifi-densepose-occworld-candle` — beyond-SOTA security + correctness review (Milestone #9, crate 4/4).** (1) **HIGH (MEASURED) — checkpoint-load crash on any int32 tensor** (`model.rs::safetensor_dtype_to_candle`). `safetensors::Dtype::I32` was mapped to `candle_core::DType::I64` and the raw int32 byte buffer (4 bytes/elem) was then handed to `Tensor::from_raw_buffer(.., I64, shape, ..)`. Candle derives `elem_count = data.len() / dtype.size_in_bytes()`, so the I64 path halved the element count while keeping the *original* shape — yielding a tensor whose declared shape claims twice as many elements as its backing storage holds. Reading it **panics** (`range end index 6 out of range for slice of length 3` — slice OOB inside candle-core) on any attacker-supplied or PyTorch-exported checkpoint containing an int32 tensor (common: index/buffer tensors). Fixed by mapping `I32 → DType::I32` (and `I16 → DType::I16`), both first-class candle dtypes. Reproduction recorded on old code; pinned by `tests/checkpoint_loading.rs::int32_tensor_loads_with_consistent_shape_and_values` (panics on old, passes on new) plus F32/I64/corrupt-file control cases. (2) **LOW (MEASURED) — `predict()` lacked frame/batch validation at the input boundary** (`inference.rs`). It validated H/W/D but not the externally-supplied frame count; an `f_in > num_frames*2` over-indexed the temporal positional embedding deep in the transformer and surfaced as a cryptic candle "gather" `InvalidIndex` (returned error, not a panic — candle bounds-checks), and a zero frame/batch dim fed a zero-element tensor into the pipeline. Now rejected at the boundary with a clear `ShapeMismatch`. Pinned by `predict_rejects_zero_frames` / `predict_rejects_too_many_frames` / `predict_accepts_frame_count_at_capacity`. (3) **LOW (MEASURED) — divide-by-zero panic on a degenerate input to the public `VQCodebook::encode`** (`vqvae.rs`): a rank-0 / empty-last-dim tensor made `last == 0` and panicked on `elem_count() / last`. Now fails closed with a clear error. Pinned by `encode_rejects_scalar_without_panicking`. **Dimensions confirmed CLEAN with evidence:** panic surface — zero `unwrap()`/`expect()`/`panic!`/`unreachable!` in production code paths (grep evidence; all error handling via `?`/`map_err`); NaN-state-poisoning — N/A (engine is stateless between `predict` calls, input is `u8` class indices so non-finite input is structurally impossible, no persistent world-model buffer to latch into); unbounded-alloc / shape-data mismatch from malformed weights — defended upstream by `safetensors::validate()` (overflow-checked `nelements*dtype.size()` vs declared byte range, rejected before reaching candle); secrets — none (grep clean, only `token_h`/`token_w` config fields match). `unsafe_code = forbid` in the crate manifest. **Build/validation status (MEASURED on Windows):** crate builds and tests under `cargo test -p wifi-densepose-occworld-candle --no-default-features`**29/29 pass** (20 unit + 4 checkpoint_loading + 3 predict_honesty + 2 doc) after fixes; `cargo test --workspace --no-default-features` = 0 failed across all crates (lone `wifi-densepose-desktop` `api_integration` failure was a Windows "Access is denied (os error 5)" file-lock flake — re-ran in isolation **21/21 pass**); Python proof VERDICT PASS, hash `f8e76f21…446f7a` unchanged. *Warrants ADR slot 179 (parent to author).*
- **`wifi-densepose-wasm-edge` beyond-SOTA closing review — boundary NaN-state-poisoning guard + clean-with-evidence attestation (ADR-040 edge crate, ~70 modules).** Closing pass of the security campaign over the last untouched sizeable crate. **One real finding fixed (LOW / source-analysis + reproduced):** the two WASM↔host frame boundaries (`lib.rs::on_frame`/`on_timer` and `bin/ghost_hunter.rs::on_frame`) read raw IEEE-754 `f32` from the `csi_get_phase`/`csi_get_amplitude`/`csi_get_variance`/`csi_get_motion_energy` host imports **without any finiteness check** — the entire crate had **zero** `is_finite`/`is_nan` guards, and the in-crate `clamp` helpers propagate NaN (`NaN < lo` and `NaN > hi` are both false). A single non-finite value (firmware DSP bug, uninitialised buffer, or hostile host) latches NaN into the long-lived per-module accumulators (EMA, Welford, phasor sums, anomaly baselines); once latched, every downstream comparison evaluates `false`, so detectors fail **degraded** (stuck gate state, silently-disabled anomaly checks) — silent corruption, not a crash (WASM `panic=abort` is *not* tripped: no indexing/`unwrap` on the poisoned value). Threat model is a **semi-trusted** boundary (the Tier-2 DSP firmware supplies the imports, not direct network/JS), hence LOW severity / defense-in-depth. **Fix:** added `sanitize_host_f32()` (maps non-finite→`0.0`, `core`-only so it holds in `no_std`) applied at every `host_get_*` float read — a single chokepoint covering all ~70 downstream modules, mirroring the existing M-01 negative-`n_subcarriers` boundary clamp. **Pinned by** `boundary_tests::{sanitize_passes_finite_values_through, sanitize_maps_non_finite_to_zero, coherence_monitor_nan_latches_without_sanitize_but_not_with}` — the last asserts on the *current* `CoherenceMonitor` that a raw NaN frame latches the smoothed score (documents the hazard) while the boundary-sanitized path stays finite. **Dimensions attested CLEAN with evidence (source-analysis):** (a) **panic-on-input** — every non-test `unwrap()`/`expect()` is either `#[cfg(test)]` or in the `std`-gated RVF *builder* host tool writing to an in-memory `Vec` (infallible); no `panic!`/`unreachable!`/`todo!`/`get_unchecked` in any hot path. (b) **shape/bounds** — all frame-buffer access is `min()`-clamped (`MAX_SC=32`, `DTW_MAX_LEN`, `LCS_WINDOW`, `PATTERN_LEN`), all index-by-cast sites (`feature_id as usize`, `conclusion_id`, `minute_counter`, `plan_step`) are either compile-time-const-bounded or `if idx <`/`%`-guarded; negative `n_subcarriers` already mapped to 0 (M-01). (c) **memory/leak** — no `move ||` closures, no `mem::forget`/`Box::leak`/`.leak()`; the only `Box::new` is in the `std`-gated `skill_registry` (one-time init, bounded). (d) **secrets** — none (grep clean). **MEASURED build/test evidence:** host `cargo test --features std,medical-experimental` = **672 passed / 0 failed** (was 669 pre-fix; +3 new tests); the real deployment artifacts all build clean on the actual target — `cargo build --target wasm32-unknown-unknown --release` (no_std/panic=abort default lib), `--bin ghost_hunter --no-default-features --features standalone-bin`, and `--features medical-experimental` (toolchain 1.89 per `rust-toolchain.toml`). No ADR slot needed — a single LOW defense-in-depth boundary fix; CHANGELOG attestation suffices.
@@ -1,272 +0,0 @@
# ADR-180: Through-Wall Camera↔CSI Hand-off Demo ("Behind the Wall")
| Field | Value |
|-------|-------|
| **Status** | Proposed |
| **Date** | 2026-06-15 |
| **Deciders** | ruv |
| **Codename** | **BEHIND-THE-WALL** |
| **Builds on** | ADR-079 (camera ground-truth training), ADR-031 (sensing-first RF mode), ADR-134 (CSI→CIR multipath), ADR-029/030 (RuvSense multistatic + persistent field), ADR-024 (AETHER re-ID), ADR-151 (per-room calibration), ADR-173 (metric-locked PCK), ADR-095/096 (rvcsi nexmon) |
## Context
### The demo we want
A single self-contained **HTML page** that tells one honest, visceral story:
1. You stand in front of the laptop. The camera tracks your **full skeletal pose**;
the WiFi-CSI model, trained on *your* movements moments earlier, infers the **same
skeleton** in parallel — a side-by-side "camera vs RF agree" view.
2. You **walk out the door and behind the wall**. The camera **goes blind** (you are
occluded — it honestly shows "no person in frame"). The CSI model **keeps inferring
your skeleton** from the WiFi signal alone — the 3D figure keeps walking, behind the
wall, smoothly. A badge flips from `CAMERA` to `RF-INFERRED (through-wall)`.
3. You **walk back into view**. The camera **re-acquires**; the badge flips back to
`CAMERA`, and the RF-inferred and camera skeletons reconverge.
This is the "WiFi sees through walls" demo — and the user explicitly wants the **inferred
skeleton through the wall**, not just a blob. The project's "prove everything / no AI-slop"
bar means we make that claim **only because we measure it**: a second camera on the far side
of the wall records ground-truth pose *behind* the wall, so the through-wall skeleton's
accuracy is a **reported, reproducible number** — never an unfalsifiable "trust me."
### Honest capability framing (the load-bearing section)
Through-wall **per-joint skeletal inference from WiFi CSI is not a generally-validated
capability** in open settings — WiFi-DensePose (CMU) is camera-*co-located*. What makes it
defensible *here* is the tightly-controlled regime and the measurement:
- **Controlled regime:** one room, one subject (you), one doorway, a model **camera-supervised
on your exact gait and your exact through-door transition** (ADR-079) minutes earlier. This
is in-distribution for *this* demo, not a universal claim.
- **Measured, not asserted:** a far-side camera (cognitum-v0 has 17 `/dev/video*` nodes — use
one, or a phone) records ground-truth pose behind the wall. The through-wall CSI skeleton is
scored against it with the metric-locked PCK harness (ADR-173). **We publish the number.**
- **Uncertainty is rendered, not hidden:** the through-wall skeleton is drawn **translucent**,
with a live **per-joint confidence** and an explicit `RF-INFERRED` badge. High-confidence
joints render solid; low-confidence joints fade. It never masquerades as the camera's
ground-truth pose.
| While… | Camera | WiFi CSI (S3 / Pi5 nexmon, fused) | 60 GHz mmWave (C6 + MR60BHA2) |
|--------|--------|-----------------------------------|-------------------------------|
| In frame | **Full 17-kpt pose** — ground truth | full skeleton (supervised model) — *agrees with camera* | presence + range + micro-motion |
| Behind a **drywall** | nothing (occluded) | **inferred full skeleton** (camera-supervised model + multistatic fusion), confidence-scored, **measured vs far-side camera** | presence + range + breathing — independent through-thin-wall confirm |
| Behind **brick/metal** | nothing | degrades to coarse motion/position only — report honestly | blocked |
**The claim — stated precisely:** *"A WiFi-CSI model, camera-supervised on this subject and
room, infers a continuous skeletal pose that tracks the subject through a drywall partition;
through-wall accuracy is measured at X% PCK@k against a far-side camera (declared, not
claimed)."* If X turns out low, that is the **honest result we report** — the skeleton is still
rendered (the user wants it) but flagged with its true confidence, and the headline number is
whatever we measured, good or bad.
### Why multistatic + supervision is the enabler
A single node behind a wall sees only "something moved." Three spatially-diverse vantage points
around the doorway (RuvSense multistatic + cross-viewpoint fusion, ADR-029/030) triangulate the
moving scatterer — drywall attenuates and diffracts 2.4/5 GHz but does not block it — giving the
model a rich enough multipath signature to regress a skeleton it was *trained* to associate with
your through-door motion. AETHER re-ID embeddings (ADR-024) keep it locked to **you** across the
camera→RF→camera hand-off.
### Available hardware (the user's actual rig)
| Role | Device | Where | Stream |
|------|--------|-------|--------|
| Near ground truth (visible) | Laptop / USB camera | front of workstation (ruvzen) | MediaPipe pose → keypoints |
| **Far ground truth (validation)** | cognitum-v0 camera (1 of 17 `/dev/video*`) or a phone | **behind the wall** | MediaPipe pose → keypoints (for MEASURING the through-wall skeleton) |
| CSI node A | ESP32-S3 (8 MB) | COM9 (ruvzen) | UDP CSI :5005 |
| CSI + mmWave node B | ESP32-C6 + Seeed MR60BHA2 | COM12 (ruvzen) | WiFi CSI + 60 GHz FMCW presence/range |
| CSI node C (through-wall vantage) | Pi 5, BCM43455c0 | cognitum-v0 (other room) | nexmon_csi `.pcap` → rvcsi → CsiFrame |
| Fusion + serving | sensing-server | ruvzen :3000/:8765 | `/ws/sensing`, `/ws/pose`, new `/ws/handoff` |
Place **node C (Pi 5) and the far camera on the far side of the wall** — the Pi 5 gives the
fuser a vantage the camera lacks, and the far camera turns the through-wall claim into a
measurement.
## Decision
Build a **camera↔CSI hand-off demo** as a thin, additive layer over existing components (no new
heavy crate). Five parts: a multi-source capture plane, a camera-supervised calibration walk
that **learns to infer the skeleton through the wall**, a **hand-off state machine**, a
**dead-reckoning smoother** so dropped CSI never makes the figure jump, and a single-file HTML
viewer that renders the inferred skeleton with honest confidence.
### 1. Capture plane (reuse, don't rebuild)
- **Near camera:** `scripts/collect-ground-truth.py` already does MediaPipe pose + ESP32 CSI
paired capture (ADR-079). Extend it to also subscribe to the Pi 5 nexmon stream (rvcsi), the
C6 mmWave presence, **and the far camera**, so every frame is
`(near_pose|null, far_pose|null, csi_S3, csi_C6, mmwave_C6, csi_Pi5, t)`.
- **CSI nodes:** S3 over UDP :5005, Pi 5 via `rvcsi` (vendor/rvcsi nexmon adapter → `CsiFrame`),
C6 WiFi CSI + the MR60BHA2 60 GHz presence/range/breathing.
- **Fusion:** all CSI sources into the existing `MultistaticFuser`
(`signal/src/ruvsense/multistatic.rs`); node positions around the doorway via
`--node-positions` (geometric-diversity index drives confidence). **#1049:** with 3
independently-clocked nodes set `WDP_GUARD_INTERVAL_US` to the real inter-node spread or
fusion demotes.
### 2. Calibration walk — "it learns my movements **and infers them through the wall**" (ADR-079)
A 35 minute guided routine. The HTML page scripts the walk: stand, step left/right, walk to the
door, **cross fully behind the wall and back**, repeat — covering the visible AND the occluded
zone, because **both cameras label ground truth**:
- **Visible-zone supervision:** near camera labels pose; synchronized CSI window is the input.
- **Through-wall supervision (the key part):** while you are behind the wall, the **far camera**
labels your pose. So the CSI→skeleton model is trained on *real behind-wall poses* paired with
the *behind-wall multistatic CSI* — the model genuinely learns to infer your skeleton through
the wall, supervised by ground truth, not extrapolated blindly.
- Train/fine-tune on `ruvultra` (RTX 5080) if available, else the local recipe. Persist as a
per-room calibration bank (ADR-151 `baseline → enroll → extract → train`). AETHER re-ID
embeddings (ADR-024) bind the track to you across the hand-off.
- **Held-out split:** reserve some behind-wall passes for evaluation so through-wall PCK is
measured on data the model never trained on (no leakage — the ADR-152 measurement discipline).
### 3. Hand-off state machine (`sensing-server/src/handoff.rs`, < 300 lines)
States: `CAMERA``HANDOFF_OUT``RF_INFERRED``HANDOFF_IN``CAMERA` (+ `LOST`).
- **`CAMERA`** — near camera has a confident pose → render it; RF-inferred skeleton ghosted
alongside for the "they agree" effect.
- **`HANDOFF_OUT`** — near-camera confidence drops at the doorway **while** CSI motion stays high
and the multistatic track heads into the door zone → cross-fade source camera→RF.
- **`RF_INFERRED`** — no camera pose; the CSI model emits a **full 17-kpt skeleton** + per-joint
confidence; AETHER confirms it is still you. Render the translucent skeleton + confidence,
badge `RF-INFERRED (through-wall)`. (When fusion confidence is too low for a credible skeleton,
degrade gracefully to a coarse marker rather than a flailing one — honest fallback.)
- **`HANDOFF_IN`** — near camera re-acquires a pose positionally consistent with the last RF
skeleton (continuity gate) → cross-fade RF→camera.
- **`LOST`** — neither source for N cycles → "no track," never invented.
Fail-closed: `RF_INFERRED` requires real multistatic motion energy + an AETHER identity match
above calibrated floors; absent that → `LOST`, never a phantom. Mirrors the governed-trust gate
(ADR-031 / ADR-141).
### 4. Dead reckoning & smoothing — fluid, never jumpy (the user's requirement)
CSI does **not** arrive cleanly: UDP frames drop, nexmon `.pcap` has gaps, the fuser skips
cycles when the #1049 guard rejects a spread, and the model's per-frame skeleton jitters. Render
only on real frames and the figure teleports and shakes — which also *reads as fake*. A
**predict/correct (dead-reckoning) layer** keeps the skeleton continuous and smooth between
measurements, with **bounded** extrapolation so we never invent motion that didn't happen:
- **Per-joint constant-velocity Kalman filter** — reuse `signal/src/ruvsense/pose_tracker.rs`
(the project's existing 17-keypoint Kalman tracker with AETHER re-ID). The renderer runs at a
**fixed ~30 Hz, decoupled from CSI arrival**:
- **Measurement this tick** → Kalman *update* (correct) each joint with the new inferred pose.
- **Dropped CSI this tick** → Kalman *predict* only: advance each joint by `x += v·dt`, so the
skeleton keeps moving along its trajectory instead of freezing then snapping. **This is the
dead reckoning** — the limbs keep their motion through a dropout.
- **Confidence decay (honesty governor):** every predict-only tick multiplies confidence and
widens covariance. Dead reckoning is trusted for a **bounded** horizon (default ≤ ~500 ms,
`WDP_DEADRECKON_MAX_MS`); past it, confidence hits the floor → state machine → `LOST`. **We
coast briefly to stay smooth; we never coast forever to fake a track.** Someone who actually
stopped behind the wall converges to a still pose then `LOST`, not perpetual phantom walking.
- **Re-acquire smoothing:** a returning measurement after a gap is blended in with a
critically-damped step (no overshoot) over 23 ticks, so the skeleton eases onto truth.
- **Client render smoothing (already present):** `ui/observatory/js/figure-pool.js`
`applyKeypoints` already `lerp`s joints with a small velocity overshoot for secondary motion;
the hand-off viewer reuses it. The camera↔RF cross-fade is an alpha-lerp over ~300 ms.
**Dead-reckoning honesty invariants (testable):**
1. Predicted-only frames carry `"dead_reckoned": true` + `"age_ms"`; the UI dims them —
extrapolation is never shown as a fresh measurement.
2. Confidence is **monotonically non-increasing** across consecutive predict-only ticks.
3. After `WDP_DEADRECKON_MAX_MS` of silence the state **must** become `LOST` (pinned test:
measurements then silence → assert transition within the horizon; no perpetual motion).
4. Dead reckoning extrapolates an **existing** track only — no measurement ever ⇒ no track ⇒
`LOST`, never a phantom from zero.
### 5. The HTML demo (single file, vanilla — mirrors the Observatory)
`ui/through-wall/index.html` (+ a small JS bundle, zero build step, like `ui/observatory/`):
- **Left:** near camera feed with the MediaPipe skeleton overlaid while visible; greys to
"CAMERA BLIND" when occluded. (Optional second tile: the far camera, shown only in a
"validation" view, not the hero view.)
- **Right:** a top-down 3D room (Three.js) with the **wall** drawn, the doorway, the three
sensor positions, and the figure: a **solid skeleton** in `CAMERA`, a **translucent skeleton
with per-joint confidence fade** in `RF_INFERRED`, eased by the dead-reckoning smoother.
- **Banner / `BannerState`** (strict, mirrors rufield-viewer): `CAMERA` / `RF-INFERRED — through
wall (conf X%, measured Y% PCK@k)` / `DEAD-RECKONED (age N ms)` / `LOST` — mutually exclusive,
with a one-line honesty caption. The measured through-wall PCK is shown, not invented.
- Consumes a new `GET /ws/handoff` WS/SSE topic of `HandoffFrame`s; `?demo=1` replays a recorded
session badged `REPLAY`.
### Output contract (`HandoffFrame`, JSON)
```jsonc
{
"t_ns": 1718400000000,
"state": "RF_INFERRED", // CAMERA | HANDOFF_OUT | RF_INFERRED | HANDOFF_IN | LOST
"source": "fused_csi", // camera | fused_csi | mmwave | dead_reckoned
"pose": [[x,y,z,conf], …×17], // inferred skeleton WITH per-joint confidence (present in CAMERA/HANDOFF/RF_INFERRED)
"pose_confidence": 0.58, // aggregate; the rendered translucency
"identity_match": 0.81, // AETHER re-ID — is it still you?
"coarse": { "cell":[x,y], "zone":"behind_wall", "heading_deg":95, "node_diversity":0.48 },
"dead_reckoned": false, // true on predict-only (extrapolated) ticks
"age_ms": 0, // ms since the last real measurement (0 = fresh)
"camera_blind": true,
"measured_pck": { "k": 20, "value": null }, // filled from the far-camera validation run; null until measured
"caption": "RF-inferred skeleton — model camera-supervised on this room; through-wall PCK measured separately"
}
```
## Phased plan (each phase independently demoable + falsifiable)
- **P1 — wiring (no claim):** 3-source CSI capture (S3+C6+Pi5) + near camera into the multistatic
fuser. Gate: `/ws/sensing` shows ≥3 active nodes + a fused position with the camera running.
- **P2 — supervised calibration + through-wall training:** the guided walk with **both cameras**;
fine-tune CSI→skeleton on visible AND far-camera-labeled behind-wall poses (ADR-079). Gate:
while-visible PCK declared (metric-locked, ADR-173) on a held-out segment.
- **P3 — MEASURE the through-wall skeleton:** score the RF-inferred skeleton against the far
camera on held-out behind-wall passes → **publish the through-wall PCK@k** (good or bad). Gate:
a committed eval script reproduces the number; honest negative if low.
- **P4 — hand-off + dead reckoning + HTML:** the camera→RF→camera transition renders end-to-end,
smooth through dropped CSI. Gate: a recorded live walk where the camera goes blind, the inferred
skeleton keeps walking fluidly behind the wall, dead-reckons through dropouts without jumps, and
re-acquisition is position-continuous. **This is the demo.**
- **P5 — multi-modal corroboration (optional):** overlay C6 60 GHz presence/range as an
independent through-thin-wall confirm (two physics, one conclusion).
## Consequences
### Positive
- A genuinely compelling demo that does what the user asked — **infers and renders the skeleton
through the wall** — while staying honest because the through-wall accuracy is **measured**
against a far-side camera, not claimed. Reuses the multistatic fuser, ADR-079 supervision, the
Kalman pose tracker, AETHER re-ID, the calibration crate, and the Observatory UI: the new code
is a hand-off module + dead-reckoning smoother + an HTML page.
### Negative / Risks
- **Through-wall skeletal accuracy may be modest or poor.** That is acceptable *iff* reported
honestly — the headline is the measured PCK, whatever it is; the skeleton renders with its true
per-joint confidence (low-confidence joints fade), never as fake certainty.
- **Material dependence:** drywall good; brick/metal degrades to coarse-only — shoot on drywall
and say so.
- **3-node clock sync** is the #1049 hazard — tune `WDP_GUARD_INTERVAL_US`.
- **Per-room, per-subject:** the model that "learned your movements" does not transfer without
re-calibration — stated on the page.
- **Over-claiming is the failure mode.** Mitigations baked in: translucent confidence-faded
skeleton, `dead_reckoned`/`age_ms` flags, the measured-PCK banner, bounded extrapolation→`LOST`.
### Neutral
- No new heavy crate; signal-path proof (`verify.py`) untouched — capture/fusion/UI orchestration
over hardened, already-reviewed components.
## Acceptance criteria (falsifiable — "prove the haters wrong")
On a recorded live session, all must hold:
1. A contiguous window where the **near camera reports no person** (verifiable from raw frames)
**and** the system renders an `RF_INFERRED` skeleton.
2. The inferred skeleton's **gross motion matches reality** — direction of travel and rough gait
phase — confirmed against the **far camera** (not eyeballed).
3. **Through-wall per-joint accuracy is MEASURED** against the far camera and **reported** as
PCK@k from a committed script. Low is fine *if* honestly published; fabricated is not.
4. The figure is **smooth through dropped CSI** — no teleports/jitter — and every predicted-only
frame is flagged `dead_reckoned`; after `WDP_DEADRECKON_MAX_MS` of silence it goes `LOST`.
5. Re-acquisition is **position-continuous** (camera re-detects within a cell of the last RF
position), and AETHER confirms identity across the hand-off.
6. Every number (visible PCK, through-wall PCK, confidences) is MEASURED and reproducible — no
hand-typed metrics.
A demo that cannot meet (1)(2) and (4)(5) on the available hardware is reported as a **negative
result** (honest), not dressed up; a poor (3) is published as the real number.
## Links
- ADR-079 — camera ground-truth training (supervision pipeline; extended here to a far camera)
- ADR-031 — sensing-first RF mode / coherence gate (fail-closed honesty pattern)
- ADR-134 — CSI→CIR multipath (through-wall multipath physics)
- ADR-029 / ADR-030 — RuvSense multistatic + persistent field (the localization engine)
- ADR-024 — AETHER contrastive re-ID (identity lock across the hand-off)
- ADR-151 — per-room calibration crate (bank persistence)
- ADR-152 / ADR-173 — measurement discipline + metric-locked PCK (the honest accuracy readout)
- ADR-095 / ADR-096 — rvcsi nexmon (Pi 5 BCM43455c0 capture)
- `signal/src/ruvsense/pose_tracker.rs` — 17-kpt Kalman tracker reused for dead reckoning
- `ui/observatory/` — the vanilla-JS 3D viewer pattern this demo mirrors
@@ -6391,32 +6391,71 @@ fn vitals_snapshots_from_sensing_json(
}
}
/// Build the multistatic guard config, optionally derived from the TDM schedule
/// declared in the environment (#1031).
/// Build the multistatic guard config from the environment (#1031, #1049).
///
/// When both `WDP_TDM_SLOTS` and `WDP_TDM_SLOT_US` parse as positive integers,
/// the guard is derived via [`MultistaticConfig::for_tdm_schedule`] so a
/// deployment can match its exact schedule. Otherwise the published default
/// (60 ms hard / 20 ms soft) is returned. `min_nodes` is *not* set here — the
/// caller overrides it for single-node passthrough.
/// Three precedence layers, most-specific wins:
/// 1. `WDP_GUARD_INTERVAL_US` (+ optional `WDP_SOFT_GUARD_US`) — a **direct**
/// hard-guard override. This is the #1049 escape hatch: WiFi/ESP-NOW-synced
/// ESP32 nodes drift 10150 ms (the 100 ms beacon + WiFi-MAC jitter cannot
/// hold two independently-clocked boards within the published default), so a
/// deployment can simply lift the guard past its measured spread (e.g.
/// `WDP_GUARD_INTERVAL_US=200000`) without knowing its exact TDM schedule.
/// 2. `WDP_TDM_SLOTS` + `WDP_TDM_SLOT_US` (both positive) — derive the guard
/// from the declared schedule via [`MultistaticConfig::for_tdm_schedule`].
/// 3. Otherwise the published default (60 ms hard / 20 ms soft).
///
/// The direct override (1) is applied **on top of** whichever base (2 or 3) is
/// selected, so `WDP_GUARD_INTERVAL_US` always wins for the hard guard while a
/// TDM-derived soft band is preserved unless it would exceed the new hard guard.
/// `min_nodes` is *not* set here — the caller overrides it for single-node
/// passthrough.
fn multistatic_guard_config_from_env() -> MultistaticConfig {
multistatic_guard_config_from(
std::env::var("WDP_TDM_SLOTS").ok().as_deref(),
std::env::var("WDP_TDM_SLOT_US").ok().as_deref(),
std::env::var("WDP_GUARD_INTERVAL_US").ok().as_deref(),
std::env::var("WDP_SOFT_GUARD_US").ok().as_deref(),
)
}
/// Pure core of [`multistatic_guard_config_from_env`] for testability.
fn multistatic_guard_config_from(slots: Option<&str>, slot_us: Option<&str>) -> MultistaticConfig {
match (
fn multistatic_guard_config_from(
slots: Option<&str>,
slot_us: Option<&str>,
guard_us: Option<&str>,
soft_us: Option<&str>,
) -> MultistaticConfig {
// Base: TDM-schedule-derived when both slot params are valid, else default.
let mut cfg = match (
slots.and_then(|s| s.trim().parse::<usize>().ok()),
slot_us.and_then(|s| s.trim().parse::<u64>().ok()),
) {
(Some(n), Some(us)) if n >= 1 && us >= 1 => {
MultistaticConfig::for_tdm_schedule(n, us)
}
(Some(n), Some(us)) if n >= 1 && us >= 1 => MultistaticConfig::for_tdm_schedule(n, us),
_ => MultistaticConfig::default(),
};
// Direct hard-guard override (#1049). Ignored when unset/zero/unparseable so
// a malformed env var falls back to the base rather than breaking fusion.
if let Some(g) = guard_us
.and_then(|s| s.trim().parse::<u64>().ok())
.filter(|&g| g >= 1)
{
cfg.guard_interval_us = g;
// Keep the soft band strictly below the (possibly lowered) hard guard.
if cfg.soft_guard_us >= g {
cfg.soft_guard_us = g.saturating_sub(1).max(1);
}
}
// Optional explicit soft-guard override, always clamped strictly below hard.
if let Some(s) = soft_us
.and_then(|s| s.trim().parse::<u64>().ok())
.filter(|&s| s >= 1)
{
cfg.soft_guard_us = s.min(cfg.guard_interval_us.saturating_sub(1).max(1));
}
cfg
}
/// Turn a `ProgressiveLoader::new` failure into an actionable diagnostic (#894).
@@ -7485,11 +7524,16 @@ async fn main() {
pose_tracker: PoseTracker::new(),
last_tracker_instant: None,
multistatic_fuser: {
// #1031: the default guard (60 ms hard / 20 ms soft) accommodates a
// real TDM slot offset. A deployment can override it to match its
// own schedule via WDP_TDM_SLOTS + WDP_TDM_SLOT_US (both set ⇒ derive
// from the schedule), else the published default is used.
// #1031/#1049: the default guard (60 ms hard / 20 ms soft)
// accommodates a real TDM slot offset. A deployment overrides it via
// WDP_GUARD_INTERVAL_US (direct, e.g. 200000 for WiFi/ESP-NOW sync —
// #1049) or WDP_TDM_SLOTS + WDP_TDM_SLOT_US (derive from schedule).
let cfg = multistatic_guard_config_from_env();
info!(
"Multistatic fusion guard: {} µs hard / {} µs soft (override via \
WDP_GUARD_INTERVAL_US / WDP_SOFT_GUARD_US, or WDP_TDM_SLOTS+WDP_TDM_SLOT_US)",
cfg.guard_interval_us, cfg.soft_guard_us
);
let mut fuser = MultistaticFuser::with_config(MultistaticConfig {
min_nodes: 1, // single-node passthrough
..cfg
@@ -7797,6 +7841,72 @@ async fn main() {
info!("Server shut down cleanly");
}
#[cfg(test)]
mod multistatic_guard_config_tests {
//! #1049 — the multistatic guard interval must be operator-configurable so a
//! WiFi/ESP-NOW deployment (10150 ms inter-node clock drift) can lift the
//! guard past its measured timestamp spread instead of being permanently
//! demoted to Restricted with no escape hatch.
use super::*;
#[test]
fn default_guard_when_nothing_set() {
let cfg = multistatic_guard_config_from(None, None, None, None);
assert_eq!(cfg.guard_interval_us, MultistaticConfig::default().guard_interval_us);
assert_eq!(cfg.soft_guard_us, MultistaticConfig::default().soft_guard_us);
}
#[test]
fn direct_guard_override_wins_and_unblocks_wifi_spread() {
// The #1049 reporter's measured ~70 ms spread exceeds the 60 ms default
// → permanent demotion. A direct 200 ms override accepts it.
let cfg = multistatic_guard_config_from(None, None, Some("200000"), None);
assert_eq!(cfg.guard_interval_us, 200_000);
assert!(cfg.soft_guard_us < cfg.guard_interval_us);
// 70 ms spread now sits inside the guard.
assert!(70_000 < cfg.guard_interval_us);
}
#[test]
fn direct_guard_override_beats_tdm_derived() {
// Both TDM params AND a direct override set → the direct hard guard wins,
// the TDM-derived soft band is preserved (still strictly below hard).
let cfg = multistatic_guard_config_from(Some("2"), Some("18000"), Some("200000"), None);
assert_eq!(cfg.guard_interval_us, 200_000);
assert!(cfg.soft_guard_us < cfg.guard_interval_us);
assert!(cfg.soft_guard_us >= 1);
}
#[test]
fn soft_override_is_clamped_strictly_below_hard() {
// A soft guard ≥ hard would be nonsensical → clamped below the hard guard.
let cfg = multistatic_guard_config_from(None, None, Some("50000"), Some("999999"));
assert_eq!(cfg.guard_interval_us, 50_000);
assert!(cfg.soft_guard_us < 50_000);
}
#[test]
fn lowering_hard_below_default_soft_pulls_soft_down() {
// Override hard to 10 ms (< default 20 ms soft) → soft drops below it.
let cfg = multistatic_guard_config_from(None, None, Some("10000"), None);
assert_eq!(cfg.guard_interval_us, 10_000);
assert!(cfg.soft_guard_us < 10_000);
}
#[test]
fn malformed_or_zero_override_falls_back_to_base() {
// Garbage / zero must not break fusion — fall back to the base config.
for bad in ["", "abc", "0", "-5", "12.5"] {
let cfg = multistatic_guard_config_from(None, None, Some(bad), None);
assert_eq!(
cfg.guard_interval_us,
MultistaticConfig::default().guard_interval_us,
"override {bad:?} should be ignored"
);
}
}
}
#[cfg(test)]
mod node_sync_snapshot_serialization_tests {
//! ADR-110 iter 24 — JSON public-API contract for the iter 23