Compare commits

..

7 Commits

Author SHA1 Message Date
ruv a4d5ea88f3 feat(examples): in-browser WiFlow trainer + camera-supervised pipeline + ADR-180/181/181A
Tonight's real WiFlow work, all honest:
- examples/through-wall/: live 2-node CSI demo (index.html), the WiFlow
  camera-supervised pipeline (wiflow_capture/train/infer.py — proven +9.4pp
  over mean-pose baseline on ruvultra), the live pose viewer (pose.html),
  and the COMPLETE in-browser trainer (wiflow_browser.html): 4-stage
  calibrate->capture->train->infer, TF.js WebGPU/WASM/WebGL, MediaPipe
  camera supervision, IndexedDB persistence, mean-pose-baseline honesty.
- ADR-180 (through-wall hand-off demo), ADR-181 (full browser WiFlow,
  WASM+WebGPU, calibration phase, mobile/secure-context matrix),
  ADR-181A (binary CSI framing protocol).

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 17:31:19 -04:00
ruv ebe217569b feat(examples): real live WiFi-CSI through-wall sensing demo
Self-contained Three.js r128 demo at examples/through-wall/ that renders
ONLY genuine data streamed from the running sensing-server over
ws://localhost:8765/ws/sensing. No simulation, no fabricated frames, no
fake skeleton.

Renders, driven by real /ws/sensing frames:
- 20x20 signal_field floor heatmap (real values)
- coarse RF-localization puck from persons[0].position (labeled coarse,
  NOT pose; peak signal_field cell as fallback)
- live motion/breathing/variance/rssi bars + motion sparkline
- presence/confidence/estimated_persons/active-node/tick/Hz meters
- 3D room with wall + doorway dividing office (node 9) / hallway (node 13)
- honest mutually-exclusive banner: LIVE (source=esp32) / SIMULATED /
  NO SERVER, showing the real source verbatim
- optional webcam tile (ground-truth-when-visible, separate from CSI)

Reuses scene/lights/bloom/CSS + webcam path from
examples/three.js/demos/05-skinned-realtime.html, the floor-heatmap idea
from ui/observatory/js/, and the threaded no-cache server from
examples/three.js/server/serve-demo.py (serve.py on :8080).

Verified against the live server: real frame source=esp32, nodes [9,13],
400 signal_field values, persons[0].position present. Python proof PASS.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 16:20:49 -04:00
rUv cafbeb1e81 fix(wasm-edge): sanitize non-finite host floats at the WASM↔host frame boundary (#1102)
Closing beyond-SOTA security review of wifi-densepose-wasm-edge (ADR-040,
~70 edge modules). The two WASM↔host boundaries (lib.rs::on_frame/on_timer
and bin/ghost_hunter.rs::on_frame) read raw IEEE-754 f32 from the csi_get_*
imports with no finiteness check — the crate had zero is_finite/is_nan
guards and its clamp helpers propagate NaN. A single non-finite host value
latches NaN into long-lived per-module accumulators (EMA / Welford / phasor
sums / anomaly baselines), after which detectors fail degraded (stuck gate
state, silently-disabled checks) — silent corruption, not a crash.

Add sanitize_host_f32() (non-finite -> 0.0, core-only for no_std) applied at
every host_get_* float read: one chokepoint covering all downstream modules,
mirroring the existing M-01 negative-n_subcarriers boundary clamp. LOW /
defense-in-depth (the Tier-2 DSP firmware supplies the imports, a semi-trusted
boundary).

Pinned by boundary_tests::{sanitize_passes_finite_values_through,
sanitize_maps_non_finite_to_zero,
coherence_monitor_nan_latches_without_sanitize_but_not_with} — the last
asserts on the current CoherenceMonitor that a raw NaN frame latches the
smoothed score while the sanitized path stays finite.

Other review dimensions attested clean with evidence (see CHANGELOG): no
hot-path panics (all unwrap/expect are test-only or std-gated RVF builder),
all bounds min()-clamped, all index-by-cast const-bounded or guarded, no
leaking closures (no move||/forget/leak), no secrets.

Verified: host `cargo test --features std,medical-experimental` 672 passed /
0 failed (+3 new tests); all three wasm32-unknown-unknown release artifacts
build clean (lib default no_std/panic=abort, ghost_hunter standalone-bin,
medical-experimental); Python proof VERDICT PASS, hash unchanged.
2026-06-15 13:06:46 -04:00
rUv c859f6f743 security(occworld-candle): int32-checkpoint crash + degenerate-input guards + ADR-179 (closes Milestone #9) (#1101)
* fix(occworld-candle): security review fixes — int32 checkpoint crash + predict input validation

Beyond-SOTA security + correctness review of wifi-densepose-occworld-candle
(Milestone #9, crate 4/4 — the last ungated crate).

Findings fixed:

1. HIGH (MEASURED) — checkpoint-load crash on any int32 tensor.
   model.rs mapped safetensors I32 -> candle DType::I64 and passed the raw
   int32 byte buffer (4 bytes/elem) to Tensor::from_raw_buffer(.., I64, ..).
   Candle derives elem_count = data.len() / dtype.size(), so the I64 path
   halved the count while keeping the original shape -> a tensor whose shape
   claims 2x its storage. Reading it PANICS (slice OOB: "range end index 6
   out of range for slice of length 3") on any checkpoint containing an int32
   tensor. Fixed: I32 -> DType::I32, I16 -> DType::I16 (both first-class
   candle dtypes). Reproduced on old code; pinned in tests/checkpoint_loading.rs.

2. LOW (MEASURED) — predict() lacked frame/batch validation at the input
   boundary. f_in > num_frames*2 over-indexed the temporal embedding (cryptic
   candle "gather" error); zero frame/batch fed a zero-element tensor in. Now
   rejected with a clear ShapeMismatch. Pinned in tests/input_validation.rs.

3. LOW (MEASURED) — divide-by-zero panic in the public VQCodebook::encode on a
   rank-0 / empty-last-dim tensor (last == 0). Now fails closed with a clear
   error. Pinned in vqvae.rs unit tests.

Dimensions confirmed clean with evidence: panic surface (no unwrap/expect/
panic in prod paths), NaN-state-poisoning (N/A — stateless engine, u8 input),
unbounded-alloc/shape-data mismatch (defended upstream by safetensors::
validate), secrets (none). unsafe_code = forbid.

Validation (MEASURED, Windows): crate 31/31 pass; workspace 0 failed (lone
desktop api_integration "Access is denied" file-lock flake passes 21/21 in
isolation); Python proof VERDICT PASS, hash f8e76f21…446f7a unchanged.

Warrants ADR slot 179 (parent to author).

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs(adr): ADR-179 — occworld-candle checkpoint-load hardening (closes Milestone #9)

Records the HIGH int32-checkpoint crash fix (I32→I64 dtype-widening → slice-OOB
panic on load = DoS) + 2 LOW degenerate-input fixes from 5e77f47e5. Stateless
engine (NaN-poisoning N/A), unsafe forbidden, safetensors validate() defends
malloc upstream. occworld 31/31. Final ungated crate — Milestone #9 complete.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 12:35:29 -04:00
rUv 10c813fde3 security(desktop): IPC serial-command-injection + over-broad shell capability + ADR-178 (#1100)
* fix(security): desktop IPC serial-command-injection + over-broad shell capability (ADR-178)

Beyond-SOTA security review of wifi-densepose-desktop (Tauri v2). Two real
findings, each MEASURED on Windows (crate builds + tests under
--no-default-features):

WDP-DESK-01 (MODERATE) — serial command injection via configure_esp32_wifi.
The #[tauri::command] handler concatenated webview-supplied ssid/password into
newline-terminated serial commands with no validation; a \r\n let a compromised
webview inject an arbitrary follow-up firmware command (reboot/erase). Added
validate_wifi_credentials() enforcing WPA2 length bounds and rejecting all
control characters, called fail-closed before any serial write. Pinned by 3
new tests (rejects \r\n / \n / NUL injection, rejects out-of-range, accepts
valid boundaries).

WDP-DESK-02 (MODERATE) — removed unused shell:allow-execute / shell:allow-open
from capabilities/default.json. The Rust backend spawns processes via
std::process::Command (bypassing the allowlist) and the UI only uses
dialog.open; the shell perms were unused privilege granting the webview
arbitrary host command execution on compromise. Regenerated capabilities.json
confirms only core:default + dialog perms remain.

lib tests 18 -> 21 (+3 pins), integration 21 -> 21, 0 failed. Python
deterministic proof unchanged (f8e76f21...46f7a; desktop off the signal path).

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs(adr): ADR-178 — desktop IPC injection fix + capability least-privilege

Records the 2 MEASURED MODERATE fixes in feddcde9d: WDP-DESK-01 (webview
ssid/password \r\n-injected arbitrary firmware serial commands → validated
fail-closed) and WDP-DESK-02 (unused shell:allow-execute/open capability
granted to the webview → removed). 30-command IPC surface + capability scope
audited; 6 dimensions clean-with-evidence. desktop 18→21.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 12:01:17 -04:00
rUv 20ad75f30c feat(ADR-131): HOMECORE-UI dashboard + BFF gateway — review-fixed (supersedes #1082) (#1099)
* feat(ADR-131): HOMECORE-UI operational dashboard + BFF gateway

Complete two-tier Cognitum operator dashboard (ADR-131), served by
homecore-server at /homecore, plus the single-origin BFF gateway that
wires it to real backends.

Front-end (zero-dep vanilla TS/JS + CSS, exact Cognitum design tokens):
- All 10 panels (§4.1-4.10): dashboard, SEED fleet + detail, fleet map,
  entities (live WS subscribe_events, never polls), rooms, COGs,
  calibration wizard, events + automation builder, witness/audit, settings.
- §6 UX invariants in code: first-class provenance, prominent stale/veto/
  fragility, null(not-trained) vs withheld vs error, --mono everywhere,
  Hailo vs CPU COG distinction.
- api.js calls the gateway routes in production; mock demoted to a
  dev-only ?demo=1 fixture (no mock in prod); typed error states.
- Tests under plain node: import-graph, boot, render-smoke (22),
  interaction (3), prod-errors (13) — 5 files green; bundle ~137 KB
  (~37x smaller than HA), <2 ms/cold-render.

BFF gateway (homecore-server/src/gateway.rs, compiled + tested on Rust 1.89):
- /api/cal/* reverse-proxy to the calibration API (ADR-151).
- GET /api/homecore/rooms with the RoomState adapter (breathing->breathing_bpm,
  heartbeat:null->heart_bpm:null, injected anomaly.threshold/room_id).
- GET /api/homecore/cogs supervisor over /var/lib/cognitum/apps/.
- GET /api/homecore/appliance from /proc + TCP service probes.
- SEED-device/appliance routes return typed 503 upstream_unavailable.
- cargo test -p homecore-server = 12/12; run live (curl-verified);
  fixed a real double-v1 proxy-URL bug found during live testing.

Honest scope: W1/W2/W4/W6-appliance functional; W3/W5/W6-Hailo/federation
return typed 503 (depend on services/hardware not in this repo).

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix(homecore-ui): resolve code-review findings — SSRF guard, CORS/trace coverage, §6 honesty, crash guards

Addresses the high-effort review of PR #1082:
- SECURITY: cal_proxy rejects path-traversal/confused-deputy SSRF (`.`/`..`
  segments, backslash, %2e%2e/%2f, absolute) on raw+decoded forms → 400,
  before attaching the server-side calibration bearer.
- CORRECTNESS: /api/homecore/* + /api/cal/* now covered by the shared CORS
  allowlist (build_cors_layer, exported from homecore-api) + TraceLayer —
  previously merged outside router()'s layers (no CORS, no tracing).
- §6 HONESTY (no fabricated data): dashboard renders '—' for null metrics
  (not "null%"/"null°C"); cogs Hailo pill reflects the REAL appliance probe
  (not hardcoded "connected"); room anomaly threshold passed through / null,
  not a fabricated 0.5.
- ROBUSTNESS: cogs asArray(hef) guards a non-array manifest field; calibration
  progress guards target<=0 (no NaN%/Infinity%); restart clears the poll timer.
- CLEANUP: mock.js is now a cached DYNAMIC import (demo-only) — never bundled
  in production (§2.2).
- New ui/tests/unit-fixes.mjs pins the above; ADR-131 + CHANGELOG updated.

Co-Authored-By: claude-flow <ruv@ruv.net>

---------

Co-authored-by: Nick Ruest <127058086+nicholas-ruest@users.noreply.github.com>
2026-06-15 11:11:19 -04:00
rUv 1df6d1e1ee security(nvsim): guard degenerate input — config panic + NaN silent-corruption + ADR-177 (#1098)
* fix(nvsim): guard degenerate input — config-induced panic + NaN-state poisoning

Beyond-SOTA security review of the ADR-089 NV-diamond simulator (milestone #9,
crate 2 of 4). Two real degenerate-input findings, each pinned fails-on-old:

NVSIM-DT-01 (config panic/DoS, pipeline.rs): an external f_s_hz == 0 made
dt == +Inf, dt_us saturated to u64::MAX, and `sample * dt_us` panicked with
"attempt to multiply with overflow" at sample >= 2 (debug/WASM panic=abort;
garbage t_us in release). Fix: sanitise dt (non-finite/non-positive -> 1 µs
fallback), cap the u64 cast, and saturating_mul the timestamp.

NVSIM-NAN-01 (NaN-state poisoning, digitiser.rs): a non-finite scene parameter
(NaN dipole position / Inf moment / NaN loop radius) bypasses the near-field
clamp (NaN < R_MIN_M is false) and yields a NaN field; at the ADC `NaN as i32`
== 0 silently emitted b_pt=[0,0,0] with ADC_SATURATED CLEAR — indistinguishable
from a legit zero-field reading. Fix at the funnel: adc_quantise treats any
non-finite input as out-of-range -> clamps to code 0 AND raises the saturation
flag, so the corruption is visible downstream.

Determinism integrity, panic-free MagFrame deserialisation, and RNG seeding
confirmed clean with evidence. The published cross-machine witness
(cc8de9b0…93b4) is unchanged — guards only affect degenerate inputs.

cargo test -p nvsim --no-default-features: 50 -> 53 passed, 0 failed.
Workspace green; Python deterministic proof unchanged (f8e76f21…46f7a,
nvsim off the signal proof path). Needs ADR slot 177.

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs(adr): ADR-177 — nvsim degenerate-input hardening

Records the 2 MEASURED MEDIUM fixes in 37764be55 (NVSIM-DT-01 config-induced
overflow panic / WASM-abort DoS; NVSIM-NAN-01 non-finite scene param →
silent fake zero-field reading with saturation flag clear) + 3 pins, and the
clean-with-evidence determinism/deser/div-by-zero verdict. Cross-machine
witness cc8de9b0…93b4 reproduces unchanged.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-06-15 10:55:04 -04:00
20 changed files with 2696 additions and 14 deletions
+4
View File
File diff suppressed because one or more lines are too long
@@ -0,0 +1,92 @@
# ADR-177: `nvsim` Degenerate-Input Hardening (NV-Diamond Simulator)
| Field | Value |
|-------|-------|
| **Status** | Accepted — 2 real MEDIUM bugs fixed + pinned; determinism preserved |
| **Date** | 2026-06-15 |
| **Deciders** | ruv |
| **Codename** | **NVSIM-FAILCLOSED** |
| **Reviews** | ADR-089 (`nvsim` NV-diamond magnetometer pipeline simulator) |
| **Milestone** | #9 (ungated-crate security sweep) — crate 2 of 4 |
## Context
`nvsim` (ADR-089) is a standalone, **WASM-ready** deterministic NV-diamond
magnetometer pipeline simulator — a forward-only leaf:
`scene → source → propagation → NV ensemble → digitiser → MagFrame + SHA-256
witness`. It has no network surface, so the real attack surface is **degenerate
physical-parameter input** crossing the external boundary — specifically the
WASM `config_json` / `scene_json` entry points.
Two properties matter for this crate that don't for others: it is billed
**deterministic** (a published cross-machine witness must reproduce bit-exactly),
and under `panic=abort` WASM any panic **aborts the whole module**. So a
config-induced panic is a denial-of-service, and a silent numeric corruption
defeats the simulator's entire purpose.
## Decision
Fix the two reachable degenerate-input bugs at their funnel points, each pinned
by a fails-on-old test, **without perturbing the deterministic happy path** (the
guards fire only on non-finite / degenerate input; the published witness is
unchanged).
### Findings fixed (both MEASURED-reproduced)
| # | Severity | Location | Issue | Fix |
|---|----------|----------|-------|-----|
| NVSIM-DT-01 | MEDIUM (DoS) | `pipeline.rs:58,95` | `dt = config.dt_s.unwrap_or(1.0 / f_s_hz)`; an external `f_s_hz == 0.0``dt = +Inf``(dt*1e6) as u64` saturates to `u64::MAX``(sample as u64) * dt_us` **panics `attempt to multiply with overflow`** at `sample ≥ 2` (debug/WASM-abort; garbage `t_us` in release). MEASURED: panic at `pipeline.rs:95:30`. | Sanitise `dt` (non-finite/non-positive → 1 µs fallback), cap the `u64` cast at `u64::MAX`, `saturating_mul` the timestamp — no config can overflow it. |
| NVSIM-NAN-01 | MEDIUM (silent corruption) | funnel `digitiser.rs::adc_quantise` (root: near-field clamp bypass in `source.rs`) | A non-finite scene param (NaN/Inf dipole position, Inf moment, NaN loop radius) **bypasses the near-field clamp** (`NaN < R_MIN_M == false` → the `1/r³` path runs → NaN field), and at the ADC `NaN as i32 == 0` (Rust saturating cast) emits a frame `b_pt=[0,0,0]` with **`ADC_SATURATED` CLEAR** — indistinguishable from a legitimate zero-field reading. MEASURED: `b=[NaN,NaN,NaN] sat=false``b_pt=[0,0,0] flags=0b0000`. | `adc_quantise`: any non-finite input → code `0` **with the saturation flag raised**; the pipeline's existing `adc_sat` OR-reduction propagates `ADC_SATURATED` onto the frame, making the corruption visible downstream. |
This is the same **NaN-fail-open / NaN-poisoning** family seen across
calibration/vitals/geo and ruview-swarm — non-finite input defeating a guard —
but bounded here to a single frame (no cross-timestep accumulator).
### Dimensions confirmed clean (with evidence)
1. **Determinism integrity — clean.** One RNG only: `ChaCha20Rng::seed_from_u64(seed)`,
fully caller-seeded (grep: one `seed_from_u64`, **zero** `thread_rng`/`getrandom`/
`SystemTime`/`Instant`/`HashMap`); `Cargo.toml` pins `rand`/`rand_chacha`
`default-features=false` (no OS entropy). BoxMuller draws
`gen_range(f64::EPSILON..=1.0)` (avoids `ln(0)=-Inf` by construction). Frame
bytes fixed LE; source summation order fixed by `Vec` order. **The published
cross-machine witness `cc8de9b0…93b4` (`proof_witness_publishes_a_known_value`)
passes UNCHANGED after both fixes** — the happy path is byte-identical; guards
touch only degenerate inputs. *Attested caveat (not a finding): libm
`cos`/`ln`/`sqrt` could differ x86↔wasm; the witness is documented as
x86_64-captured.*
2. **Panic-free deserialisation — clean.** `MagFrame::from_bytes` validates
len/magic/version, then per-field `buf[a..b].try_into().expect(...)` are over
fixed sub-ranges of an already-length-checked 60-byte buffer (provably
infallible). No `unsafe`, no `panic!`/`unreachable!` in production; every other
`unwrap`/`expect` is `#[cfg(test)]`.
3. **Div-by-zero / numerical landmines — clean.** `dipole_field`/`current_loop_field`
clamp `r_norm < R_MIN_M` before `1/r³`,`1/r²` (finite inputs); `shot_noise_floor`
guards `denom <= 0`; `vec3_normalise` guards `n < 1e-20`. The only hole was the
NaN *bypass* of the clamp — closed at the ADC funnel (NVSIM-NAN-01).
## Validation
- `cargo test -p nvsim --no-default-features`**50 → 53** passed, 0 failed (+3 pins:
`degenerate_zero_sample_rate_does_not_panic`,
`non_finite_scene_input_flags_frame_instead_of_silently_zeroing`,
`adc_quantise_flags_non_finite_as_saturated`).
- `cargo test --workspace --no-default-features`**exit 0**, 0 failed.
- `python archive/v1/data/proof/verify.py`**VERDICT: PASS**, hash
`f8e76f21…46f7a` unchanged (nvsim off the signal proof path).
- nvsim's own cross-machine witness `cc8de9b0…93b4` reproduces unchanged.
## Consequences
### Positive
- A config-induced WASM-abort DoS and a silent NaN→fake-zero-field corruption are
closed at their funnel points, each regression-pinned, with the deterministic
witness proven intact.
### Negative / Neutral
- None. Guards affect only degenerate inputs; happy-path output is byte-identical.
## Links
- ADR-089 — `nvsim` NV-diamond magnetometer simulator
- ADR-176 — `ruview-swarm` (sibling NaN-fail-open review)
- ADR-172 — core/cli (where the NaN-bug-class root was settled NO)
@@ -0,0 +1,87 @@
# ADR-178: `wifi-densepose-desktop` IPC Injection Fix + Capability Least-Privilege
| Field | Value |
|-------|-------|
| **Status** | Accepted — 2 real MODERATE bugs fixed + pinned (MEASURED on Windows) |
| **Date** | 2026-06-15 |
| **Deciders** | ruv |
| **Codename** | **DESK-LOCKDOWN** |
| **Reviews** | `wifi-densepose-desktop` (Tauri v2 desktop app) |
| **Milestone** | #9 (ungated-crate security sweep) — crate 3 of 4 |
## Context
`wifi-densepose-desktop` is the Tauri v2 desktop app (ESP32 discovery, firmware
flashing, OTA, provisioning, server control). The real attack surface is the
**Tauri IPC boundary**`#[tauri::command]` handlers that take arguments from the
webview/JS — and the **capability/allowlist scope**. The crate **builds and tests
on Windows** (Tauri 2.10.3, webview2 path, no GTK), so both findings are MEASURED,
not source-analysis-only.
## Decision
Fix the two real findings; attest the rest of the surface clean with evidence.
### Findings fixed (both MEASURED)
| # | Severity | Location | Issue | Fix |
|---|----------|----------|-------|-----|
| WDP-DESK-01 | MODERATE | `src/commands/discovery.rs:438` (`configure_esp32_wifi`) | Webview-supplied `ssid`/`password` are concatenated into newline-terminated serial commands (`wifi_config {} {}\r\n`, `set ssid {}\r\n`) with **no validation** → a `\r\n` in either field **injects an arbitrary follow-up firmware command** (`reboot`, `erase_nvs`) across the IPC trust boundary. | `validate_wifi_credentials()` — WPA2 length bounds (SSID 132, password 863) **+ reject all control chars** (`char::is_control()`), called fail-closed before any serial write. |
| WDP-DESK-02 | MODERATE | `capabilities/default.json:7-8` | `shell:allow-execute` + `shell:allow-open` granted to the webview but **unused** (Rust spawns via `std::process::Command`; the UI uses only `dialog.open`). A webview compromise (a UI-dependency XSS) → arbitrary **unscoped host command execution**. | Removed both `shell:` permissions (kept `core:default` + the two in-use `dialog:` perms); regenerated `gen/schemas/capabilities.json` now asserts `["core:default","dialog:allow-open","dialog:allow-save"]`. |
Both are MODERATE (not HIGH): each requires a webview compromise or a malicious
local caller to weaponize. The unifying lesson is **least privilege at the IPC
boundary** — validate every webview-supplied argument that reaches a serial/FS/
process sink, and grant only the capabilities actually exercised.
### Tauri-command + capability audit (every handler)
All 30+ command handlers were mapped. Only `configure_esp32_wifi` lacked input
validation on a string that reached a command sink (WDP-DESK-01). Every
subprocess uses `Command::new(prog).args([...])` (argv vector — no shell-string
interpolation), so `port`/`source`/`chip`/`baud` cannot inject a second command
even unvalidated. `tauri.conf.json` ships **no** `fs`/`http` plugin and **no**
`"all":true`/`"$HOME/**"` scope; after WDP-DESK-02 the allowlist is minimal.
### Dimensions confirmed clean (with evidence)
1. **Directory traversal / arbitrary file** — path args (`firmware_path`/`wasm_path`)
are blobs the local user selects via the native `dialog.open` picker; settings
I/O is a fixed filename under `app_data_dir`. No attacker-named path sink.
2. **Shell-string injection** — every subprocess is an argv vector; grep found no
shell-string interpolation anywhere.
3. **SSRF-to-secret**`node_ip`-built URLs target the local ESP32 mesh and return
only device status JSON; no credential returned to the webview.
4. **Panic-on-input** — handlers use `.map_err(|e| e.to_string())?`; the one
`expect` is guarded by an `is_none()` early-return; provision/discovery
deserializers bounds-check every slice index (NVS size capped ≤ 4096).
5. **Hardcoded secrets**`ota_psk` is a per-call `Option<String>`, never embedded;
grep for embedded keys/tokens over `src/` is empty.
6. **Shell plugin genuinely unused**`tauri_plugin_shell` is `init()`-ed but its
`Command`/`open` API is never invoked from Rust or the TS UI (which imports only
`@tauri-apps/plugin-dialog`) — confirming WDP-DESK-02 is safe to remove.
## Validation
- `cargo check -p wifi-densepose-desktop --no-default-features``Finished` (Windows, MEASURED).
- `cargo test -p wifi-densepose-desktop --no-default-features` → lib **18 → 21** (+3 validator pins:
`test_validate_wifi_credentials_rejects_injection` / `_rejects_out_of_range` / `_accepts_valid`),
integration 21/21, **0 failed**.
- Capability narrowing MEASURED: regenerated `capabilities.json` permission set verified.
- `python archive/v1/data/proof/verify.py`**VERDICT: PASS**, hash `f8e76f21…46f7a`
unchanged (desktop off the signal proof path).
## Consequences
### Positive
- An IPC serial-command-injection path and an over-broad shell capability are
closed in the desktop app, each pinned / verified, with the rest of the
30-command IPC surface attested clean.
### Negative / Neutral
- None. The removed shell capability was unused; the validator rejects only
malformed/hostile credentials.
## Links
- ADR-176 / ADR-177 — sibling Milestone-#9 reviews (ruview-swarm, nvsim)
- ADR-172 — core/cli review
@@ -0,0 +1,81 @@
# ADR-179: `wifi-densepose-occworld-candle` Checkpoint-Load Hardening
| Field | Value |
|-------|-------|
| **Status** | Accepted — 1 HIGH + 2 LOW bugs fixed + pinned (MEASURED on Windows) |
| **Date** | 2026-06-15 |
| **Deciders** | ruv |
| **Codename** | **OCCWORLD-DTYPE** |
| **Reviews** | `wifi-densepose-occworld-candle` (Candle occupancy-world model) |
| **Milestone** | #9 (ungated-crate security sweep) — crate 4 of 4 — **CLOSES the milestone** |
## Context
`wifi-densepose-occworld-candle` is a Candle-based occupancy-world model
(VQ-VAE + transformer over occupancy tokens). The real risk surface for an ML
crate is degenerate-input / malformed-weights handling: a `#[forbid(unsafe_code)]`
crate can still **panic** (a DoS, and under WASM an abort) when a tensor op hits an
inconsistent shape. The crate **builds and tests on Windows**, so all findings are
MEASURED.
## Decision
Fix the three reachable bugs, each pinned by a fails-on-old test; attest the rest
clean with evidence.
### Findings fixed (all MEASURED)
| # | Severity | Location | Issue | Fix |
|---|----------|----------|-------|-----|
| 1 | **HIGH** | `model.rs:95` (`Dtype::I32 => Some(DType::I64)`) | **Crash on any int32-tensor checkpoint.** An I32 byte buffer (4 B/elem) is handed to `from_raw_buffer(.., I64, shape, ..)`; candle derives `elem_count = data.len()/8`, **halving** the count while keeping the original shape → a tensor that claims 2× its storage. Reading it **panics** with a slice-OOB (`range end index 6 out of range for slice of length 3`) inside candle-core. A checkpoint with any int32 tensor (index/buffer tensors are common in PyTorch exports) → **DoS on load**. | Map `I32 → DType::I32`, `I16 → DType::I16` (both first-class candle dtypes). Pinned by `int32_tensor_loads_with_consistent_shape_and_values` (panics on old, passes on new). |
| 2 | LOW | `inference.rs::predict` | Frame/batch dims weren't validated (only H/W/D were): `f_in > num_frames*2` over-indexes the temporal embedding → a cryptic candle `InvalidIndex` *error* (not a panic — candle bounds-checks); zero frame/batch feeds a zero-element tensor. | Boundary guard rejects zero / over-capacity frame+batch with a clear `ShapeMismatch`. 5 pins. |
| 3 | LOW | `vqvae.rs:141` (`z.elem_count() / last`) | **Divide-by-zero panic** in public `VQCodebook::encode` on a rank-0 / empty-last-dim tensor (`last == 0`). | Fail-closed guard returns a clear error. Pinned by `encode_rejects_scalar_without_panicking`. |
The HIGH finding is the notable one: the crate's own dtype mapping **defeated**
the upstream `safetensors::validate()` byte-length guarantee by misdeclaring the
dtype — the one place malformed/widened weights could reach a panicking candle op.
### Dimensions confirmed clean (with evidence)
- **Panic surface** — grep for `unwrap()/expect()/panic!/unreachable!` across `src/`
**zero in production paths**; all ops use `?`/`map_err`; the `last().unwrap_or(&0)`
is now guarded. `as` casts operate only on config-bounded/internal values.
- **NaN-state-poisoning (the named class) — N/A.** The engine is **stateless between
`predict` calls** (no persistent world-model buffer to latch into), and input is
`u8` class indices (non-finite input structurally impossible). NaN weights flow to
`argmax` (deterministic, bounded to a valid class index) — no panic, no persistence.
- **Unbounded alloc / shape-data mismatch from malformed weights** — defended upstream
by `safetensors::validate()` (overflow-checked `nelements*dtype.size()` vs declared
byte range + contiguous-offset + buffer-length checks), rejected before reaching
candle. Finding #1 was the one place the crate defeated that guarantee.
- **Model/path loading** — `load`/`load_safetensors` check `path.exists()` → typed
`CheckpointNotFound`; corrupt bytes → `CheckpointParse` (pinned). No path-traversal
surface (caller-supplied path, opened read-only, never joined with untrusted segments).
- **Secrets** — grep clean (only `token_h`/`token_w` config fields match `token`).
- **Determinism** — the crate's central honesty claim, verified by the pre-existing
`tests/predict_honesty.rs` (3 tests, still pass).
- `unsafe_code = "forbid"` in the manifest.
## Validation
- `cargo test -p wifi-densepose-occworld-candle --no-default-features`**31/31**
(lib 17, checkpoint_loading 4, input_validation 5, predict_honesty 3, doctests 2),
0 failed.
- `cargo test --workspace --no-default-features` → 0 failed across every crate (a lone
`wifi-densepose-desktop --test api_integration` "Access is denied (os error 5)" was a
Windows file-lock/AV flake — re-ran isolated 21/21, unrelated).
- `python archive/v1/data/proof/verify.py`**VERDICT: PASS**, hash `f8e76f21…46f7a`
unchanged (occworld off the signal proof path).
## Consequences
### Positive
- A checkpoint-load DoS (the int32 dtype-widening panic) and two degenerate-input
panics are closed in the world-model crate, each pinned. **Milestone #9 (all 4
ungated crates) is complete.**
### Negative / Neutral
- None. Guards reject only malformed/degenerate inputs.
## Links
- ADR-176 / ADR-177 / ADR-178 — sibling Milestone-#9 reviews (ruview-swarm, nvsim, desktop)
+135
View File
@@ -0,0 +1,135 @@
# WiFlow Browser Trainer (`wiflow_browser.html`)
A **single self-contained HTML page** that does the entire camera-supervised
WiFi-pose loop **in your browser, in your laptop camera's coordinate frame**, as
a **4-stage gated flow** with a progress stepper (each stage unlocks the next):
0. **CALIBRATE** *(ADR-151 empty-room baseline)* — you step OUT of the space; the
page captures ~10 s of the quiescent CSI and computes a per-feature running
**mean + std (Welford)** over the 410-d vector. Every CSI vector afterwards is
expressed as **deviation from baseline**
(`x_norm = (x base_mean) / (base_std + ε)`), so a body's perturbation stands
out from the static channel. Persisted to IndexedDB. *Can't capture without it.*
1. **CAPTURE** — MediaPipe Pose runs on your laptop camera → 17 COCO keypoints
(the *label*), paired with the **baseline-normalized** 410-d ESP32 CSI vector
(the *input*). A **guided, balanced routine** cycles big on-screen prompts
(stand / turn / walk / arms / crouch / sit / reach) with a countdown, and a
**per-pose coverage meter** so you build a balanced dataset, not 2 000 frames
of standing.
2. **TRAIN** — a TensorFlow.js MLP learns `CSI → pose` in-browser. Honest
held-out PCK@0.10 / PCK@0.05 / MPJPE, plus a **mean-pose baseline** the model
must beat (the project's whole ethos — no baseline-beating signal, it says so).
*Can't train with <200 samples.*
3. **INFER** — the trained model drives a skeleton **from WiFi CSI only**
(baseline-normalized → standardized → model), drawn over the **same** camera
frame it trained in — so the inferred skeleton **aligns** with the camera
image. That alignment is the entire point of doing this in-browser instead of
with a separate Python camera. *Can't infer without a model.*
## Why in-browser
The Python pipeline (`wiflow_capture.py``wiflow_train.py``wiflow_infer.py`)
proved the signal is real (held-out PCK@0.10 ≈ 59.5% vs a 50% mean-pose baseline
= +9.4 pp). But it trained in a *different* camera's frame, so the inferred
skeleton never lined up with the laptop camera. Doing capture + train + infer all
in the browser with the **same** camera makes the training frame and the
inference frame identical → the skeleton aligns.
## Compute backends (WebGPU / WASM / WebGL)
Training and inference run on TensorFlow.js. The page selects the backend at
startup, preferring the fastest available:
- **WebGPU** (Chrome / Edge, secure context — `localhost` qualifies) — GPU compute.
- **WASM-SIMD** fallback (`tfjs-backend-wasm`, SIMD enabled, `.wasm` from the CDN).
- **WebGL** last-resort fallback (ships inside tfjs core).
The **active backend is shown as a badge in the header** (`compute: WebGPU` /
`WASM-SIMD` / `WebGL`) so it's honest about what's actually running. The model
code is backend-agnostic — tf.js abstracts the device.
## Honesty (baked in)
- The **CAPTURE** skeleton (blue) is the camera = ground truth, labeled as such.
- The **INFER** skeleton (green) is **CSI-only**, labeled, and **coarse** — the
real measured held-out PCK is shown, not a marketing number.
- The **mean-pose baseline** is always computed and shown in TRAIN; the verdict
states plainly whether the model **beats** it (real signal) or **does not**
(no usable signal). This guards against the project's retracted 92.9% that
failed exactly this check.
- Status banner is strict and mutually exclusive:
**LIVE** (real `source: "esp32"`) / **SIMULATED — not real** (any other source)
/ **NO-CSI-SERVER**. The page never invents frames.
## How to run
### 1. Start the real sensing-server (provides the CSI WebSocket on :8765)
```bash
cd v2
cargo build -p wifi-densepose-sensing-server
./target/debug/sensing-server.exe --ws-port 8765 --udp-port 5005
```
A real ESP32-S3 must be provisioned and streaming for `source` to read `esp32`
(see `CLAUDE.local.md` for the firmware build/provision steps). The page expects
the verified live endpoint **`ws://localhost:8765/ws/sensing`** with
`source:"esp32"`, nodes `[9, 13]`, `features.*`, `node_features[].features.*`,
and `signal_field.values` (400 floats).
### 2. Serve this page over localhost (camera + WebGPU need a localhost/secure origin)
Any static localhost server works. For example:
```bash
python -m http.server 8099
# then open: http://localhost:8099/examples/through-wall/wiflow_browser.html
```
(8099 is just the static file server — 8765 is a separate process, the CSI
WebSocket.) Allow camera access when the browser prompts.
Point at a CSI server on another host with `?ws=`:
```
http://localhost:8099/examples/through-wall/wiflow_browser.html?ws=ws://192.168.1.20:8765/ws/sensing
```
### 3. Use it
1. **CAPTURE** tab → *enable laptop camera**start recording*. Follow the guided
routine (stand / turn / walk / arms / crouch / sit). A pair is stored only when
a confident pose AND a fresh live `esp32` CSI frame coexist. Aim for a few
thousand samples. Samples persist in IndexedDB across refreshes.
2. **TRAIN** tab → *train model*. Watch the live loss curve, held-out PCK, and the
baseline verdict. The model saves to IndexedDB.
3. **INFER** tab → the green skeleton is now driven by WiFi CSI only, aligned over
your camera. Toggle *hide camera* to see the CSI-only skeleton on black.
## The 410-d CSI vector (matches the Python pipeline exactly)
```
[ mean_rssi, variance, motion_band_power, breathing_band_power ] # 4 (features.*)
+ for node 9 then node 13: [ mean_rssi, variance, motion_band_power ] # 6 (node_features[].features.*)
+ signal_field.values, padded / truncated to 400 # 400
= 410-d
```
Verified against a real live frame: the in-browser `csiVector()` produces the
identical 410 vector as `wiflow_capture.py`'s `csi_vector()` (node 9 first, then
node 13; field zero-padded).
## Libraries (CDN only, no bundler)
| Library | CDN |
|---|---|
| TensorFlow.js core | `@tensorflow/tfjs@4.22.0/dist/tf.min.js` |
| TF.js WebGPU backend | `@tensorflow/tfjs-backend-webgpu@4.22.0/dist/tf-backend-webgpu.min.js` |
| TF.js WASM backend | `@tensorflow/tfjs-backend-wasm@4.22.0/dist/tf-backend-wasm.min.js` |
| MediaPipe Pose 0.5 (legacy solutions) | `@mediapipe/pose@0.5/pose.js` |
## Scope / honesty caveats
Same person, same room, same session. **Not** validated cross-day, cross-room, or
through-wall. The inferred pose is coarse (PCK@0.05 is typically weak). If the
model does not beat the mean-pose baseline, the page says so — that is a feature.
+644
View File
@@ -0,0 +1,644 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>RuView · Through-Wall WiFi Sensing · LIVE CSI (no skeleton, no simulation)</title>
<!--
THROUGH-WALL WiFi-CSI SENSING DEMO — honest, real-data-only.
Renders ONLY what the running sensing-server actually streams over
ws://localhost:8765/ws/sensing :
- the 20x20 `signal_field` floor heatmap (real values)
- a coarse RF-localization puck from persons[0].position (NOT pose)
- live motion / presence / rssi / confidence meters
- the real `source` ("esp32" = LIVE) verbatim in the banner
It deliberately does NOT draw a skeleton. The server's
persons[].keypoints carry confidence:0.0 (image-pixel garbage, not
real 3D joints) so we never render them. WiFi CSI gives
motion/presence/coarse-position — that is the honest wow, and it
penetrates drywall. See README.md.
-->
<style>
:root {
--bg: #050507; --bg-panel: rgba(8,10,14,0.80);
--amber: #ffb840; --amber-hot: #ffe09f;
--cyan: #4cf; --magenta: #ff4cc8;
--text: #d8c69a; --text-mute: #6b6155;
--green: #4f4; --red: #f64;
--border: rgba(255,184,64,0.18);
}
* { box-sizing: border-box; }
body {
margin: 0; background: var(--bg); color: var(--text); overflow: hidden;
font-family: 'SF Mono', 'Cascadia Code', Consolas, monospace;
-webkit-font-smoothing: antialiased; font-size: 12px;
}
canvas { display: block; }
.overlay-frame {
position: fixed; inset: 0; pointer-events: none; z-index: 5;
background:
radial-gradient(ellipse at center, transparent 55%, rgba(0,0,0,0.55) 100%),
linear-gradient(180deg, rgba(0,0,0,0.32) 0%, transparent 18%, transparent 82%, rgba(0,0,0,0.38) 100%);
}
.scanlines {
position: fixed; inset: 0; pointer-events: none; z-index: 6;
background: repeating-linear-gradient(0deg, rgba(0,0,0,0.04) 0px, rgba(0,0,0,0.04) 1px, transparent 1px, transparent 3px);
mix-blend-mode: overlay; opacity: 0.5;
}
.panel {
position: absolute; background: var(--bg-panel); border: 1px solid var(--border);
border-radius: 4px; padding: 12px 14px; backdrop-filter: blur(8px);
box-shadow: 0 1px 0 rgba(255,184,64,0.04), 0 8px 32px rgba(0,0,0,0.55); z-index: 10;
}
.panel h2 {
margin: 0 0 8px 0; font-size: 10px; text-transform: uppercase; letter-spacing: 2px;
color: var(--amber); font-weight: 600; border-bottom: 1px solid var(--border); padding-bottom: 6px;
}
/* ---- Honest status banner (top-center, mutually exclusive states) ---- */
#banner {
position: fixed; top: 0; left: 0; right: 0; z-index: 30;
text-align: center; padding: 7px 12px; font-size: 12px; letter-spacing: 1px;
font-weight: 600; border-bottom: 1px solid rgba(0,0,0,0.4);
transition: background 0.3s, color 0.3s;
}
#banner.live { background: rgba(40,255,80,0.12); color: var(--green); border-bottom-color: rgba(80,255,120,0.4); }
#banner.sim { background: rgba(255,120,40,0.16); color: #ffae5a; border-bottom-color: rgba(255,140,60,0.5); }
#banner.noserver { background: rgba(255,80,80,0.16); color: var(--red); border-bottom-color: rgba(255,90,90,0.5); }
#banner .src { opacity: 0.8; font-weight: 400; }
#banner-caption {
position: fixed; top: 30px; left: 0; right: 0; z-index: 29;
text-align: center; font-size: 10px; color: var(--text-mute); letter-spacing: 0.5px;
pointer-events: none; padding-top: 2px;
}
#info { top: 64px; left: 20px; min-width: 270px; }
#info h1 { margin: 0 0 1px 0; font-size: 13px; letter-spacing: 1px; color: var(--amber-hot); font-weight: 600; }
#info .sub { font-size: 10px; color: var(--text-mute); letter-spacing: 0.5px; margin-bottom: 10px; padding-bottom: 8px; border-bottom: 1px solid var(--border); }
#info .row { display: flex; justify-content: space-between; gap: 12px; padding: 2px 0; }
#info .row .k { color: var(--text-mute); font-size: 11px; }
#info .row .v { color: var(--text); font-variant-numeric: tabular-nums; font-size: 11px; }
#info .row .v.amber { color: var(--amber); }
#info .row .v.cyan { color: var(--cyan); }
#info .row .v.green { color: var(--green); }
#info .row .v.red { color: var(--red); }
#info .row .v.mag { color: var(--magenta); }
#info .row .v.mute { color: var(--text-mute); }
#csi { top: 64px; right: 20px; min-width: 270px; }
#csi .bar-row { display: flex; align-items: center; gap: 8px; padding: 3px 0; font-size: 10px; }
#csi .bar-row .label { width: 86px; color: var(--text-mute); }
#csi .bar-row .bar-track { flex: 1; height: 6px; background: rgba(255,184,64,0.08); border-radius: 2px; overflow: hidden; }
#csi .bar-row .bar-fill {
height: 100%; background: linear-gradient(90deg, var(--amber-hot), var(--amber));
box-shadow: 0 0 6px var(--amber); transition: width 0.1s linear;
}
#csi .bar-row .val { width: 44px; text-align: right; color: var(--amber); font-variant-numeric: tabular-nums; }
#csi .spark { margin-top: 8px; }
#csi canvas { width: 100%; height: 38px; display: block; border: 1px solid var(--border); border-radius: 3px; background: rgba(0,0,0,0.3); }
#csi .legend { margin-top: 8px; padding-top: 8px; border-top: 1px solid var(--border); font-size: 10px; color: var(--text-mute); line-height: 1.5; }
/* ---- waiting / no-server overlay ---- */
#waiting {
position: fixed; inset: 0; z-index: 25; display: none;
flex-direction: column; align-items: center; justify-content: center;
background: rgba(5,5,7,0.94); color: var(--amber); text-align: center; padding: 24px;
}
#waiting.show { display: flex; }
#waiting .big { font-size: 22px; letter-spacing: 2px; color: var(--red); margin-bottom: 16px; text-transform: uppercase; }
#waiting code {
display: block; text-align: left; max-width: 640px; margin: 8px auto;
background: rgba(255,184,64,0.06); border: 1px solid var(--border); border-radius: 4px;
padding: 10px 14px; color: var(--amber-hot); font-size: 12px; white-space: pre-wrap;
}
#waiting .pulse { animation: pulse 1.4s ease-in-out infinite; }
@keyframes pulse { 0%,100% { opacity: 0.55; } 50% { opacity: 1; } }
/* ---- optional webcam ground-truth tile ---- */
#cam-tile {
position: absolute; bottom: 20px; right: 20px; width: 240px; z-index: 12;
background: var(--bg-panel); border: 1px solid var(--border); border-radius: 4px;
padding: 8px; backdrop-filter: blur(8px);
}
#cam-tile h2 { margin: 0 0 6px 0; font-size: 9px; text-transform: uppercase; letter-spacing: 1.5px;
color: var(--cyan); font-weight: 600; }
#cam-tile .gt-note { font-size: 9px; color: var(--text-mute); margin-top: 4px; line-height: 1.4; }
#cam-video { width: 100%; border-radius: 3px; display: none; background: #000; }
#cam-tile button {
width: 100%; margin-top: 6px; padding: 5px 8px; font-family: inherit; font-size: 11px;
background: transparent; color: var(--cyan); border: 1px solid var(--cyan); border-radius: 3px; cursor: pointer;
}
#cam-tile button:hover { background: rgba(68,204,255,0.12); }
#cam-tile button:disabled { opacity: 0.5; cursor: not-allowed; }
#legend-nodes {
position: absolute; bottom: 20px; left: 20px; min-width: 220px;
background: var(--bg-panel); border: 1px solid var(--border); border-radius: 4px;
padding: 12px 14px; backdrop-filter: blur(8px); z-index: 10;
}
#legend-nodes h2 { margin: 0 0 8px 0; font-size: 10px; text-transform: uppercase; letter-spacing: 2px;
color: var(--amber); font-weight: 600; border-bottom: 1px solid var(--border); padding-bottom: 6px; }
#legend-nodes .lr { display: flex; align-items: center; gap: 8px; padding: 2px 0; font-size: 11px; }
#legend-nodes .dot { width: 9px; height: 9px; border-radius: 50%; box-shadow: 0 0 6px currentColor; flex: 0 0 auto; }
#legend-nodes .muted { color: var(--text-mute); }
</style>
<!-- three.js r128 + addons (same CDN set as examples/three.js/demos/05) -->
<script src="https://cdnjs.cloudflare.com/ajax/libs/three.js/r128/three.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/controls/OrbitControls.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/postprocessing/EffectComposer.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/postprocessing/RenderPass.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/postprocessing/ShaderPass.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/postprocessing/UnrealBloomPass.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/shaders/CopyShader.js"></script>
<script src="https://cdn.jsdelivr.net/npm/three@0.128.0/examples/js/shaders/LuminosityHighPassShader.js"></script>
</head>
<body>
<div id="banner" class="noserver">NO SERVER — start the sensing-server <span class="src"></span></div>
<div id="banner-caption">Real WiFi CSI motion / presence / coarse-localization — penetrates drywall. Not skeletal pose.</div>
<div class="overlay-frame"></div>
<div class="scanlines"></div>
<div class="panel" id="info">
<h1>THROUGH-WALL WiFi SENSING</h1>
<div class="sub">Live CSI · ws://localhost:8765/ws/sensing</div>
<div class="row"><span class="k">source</span><span class="v amber" id="m-source"></span></div>
<div class="row"><span class="k">presence</span><span class="v" id="m-presence"></span></div>
<div class="row"><span class="k">motion level</span><span class="v" id="m-motion"></span></div>
<div class="row"><span class="k">confidence</span><span class="v cyan" id="m-conf"></span></div>
<div class="row"><span class="k">est. persons</span><span class="v amber" id="m-persons"></span></div>
<div class="row"><span class="k">active nodes</span><span class="v" id="m-nodes"></span></div>
<div class="row"><span class="k">tick</span><span class="v" id="m-tick"></span></div>
<div class="row"><span class="k">update rate</span><span class="v cyan" id="m-fps"></span></div>
</div>
<div class="panel" id="csi">
<h2>Live RF features</h2>
<div class="bar-row"><span class="label">motion</span><div class="bar-track"><div class="bar-fill" id="bar-motion"></div></div><span class="val" id="v-motion"></span></div>
<div class="bar-row"><span class="label">breathing</span><div class="bar-track"><div class="bar-fill" id="bar-breath"></div></div><span class="val" id="v-breath"></span></div>
<div class="bar-row"><span class="label">variance</span><div class="bar-track"><div class="bar-fill" id="bar-var"></div></div><span class="val" id="v-var"></span></div>
<div class="bar-row"><span class="label">mean rssi</span><div class="bar-track"><div class="bar-fill" id="bar-rssi"></div></div><span class="val" id="v-rssi"></span></div>
<div class="spark"><canvas id="spark" width="252" height="38"></canvas></div>
<div class="legend">motion sparkline (last ~6s of real motion_band_power)</div>
</div>
<div id="legend-nodes">
<h2>Sensor nodes</h2>
<div class="lr"><span class="dot" style="color:#4cf"></span><span>ESP32-S3 office <span class="muted">(node 9)</span></span></div>
<div class="lr"><span class="dot" style="color:#ff4cc8"></span><span>ESP32-S3 hallway <span class="muted">(node 13)</span></span></div>
<div class="lr" style="margin-top:6px"><span class="dot" style="color:#4f4"></span><span>RF localization <span class="muted">(coarse)</span></span></div>
<div class="lr"><span class="muted" style="font-size:10px;line-height:1.4">Office &amp; hallway split by a wall + doorway. WiFi motion still shows through drywall.</span></div>
</div>
<div id="cam-tile">
<h2>camera — ground truth when visible</h2>
<video id="cam-video" autoplay muted playsinline></video>
<button id="cam-btn">▶ enable webcam (optional)</button>
<div class="gt-note">Independent of the CSI sensing. The WiFi works in the dark and through walls; the camera does not.</div>
</div>
<div id="waiting" class="show">
<div class="big pulse">Waiting for live sensing-server</div>
<div>No connection to <b>ws://localhost:8765/ws/sensing</b>. Start the real server, then this page connects automatically.</div>
<code>cd v2
cargo build -p wifi-densepose-sensing-server
./target/debug/sensing-server.exe --ws-port 8765 --udp-port 5005</code>
<div style="margin-top:10px; color:var(--text-mute); font-size:11px;">This demo renders ONLY real data. It never invents frames.</div>
</div>
<script>
"use strict";
// =====================================================================
// Config + WS endpoint (allow ?ws= override)
// =====================================================================
const params = new URLSearchParams(location.search);
const WS_URL = params.get('ws') || 'ws://localhost:8765/ws/sensing';
const ROOM_HALF = 5; // half-extent of the floor plane in metres
const GRID_N = 20; // signal_field is 20 x 20
// Known node anchor positions (server sends node 9 @ [2,0,1.5]; node 13
// joins later from the hallway side once its firmware is flashed). These
// are anchors for the room model + labels, NOT fabricated sensing data.
const NODE_ANCHORS = {
9: { pos: [ 2.0, 0.0, 1.5], color: 0x44ccff, label: 'office (node 9)' },
13: { pos: [-2.0, 0.0, -3.0], color: 0xff4cc8, label: 'hallway (node 13)' },
};
// =====================================================================
// Three.js scene (reused pattern from demos/05-skinned-realtime.html)
// =====================================================================
const scene = new THREE.Scene();
scene.background = new THREE.Color(0x050507);
scene.fog = new THREE.FogExp2(0x050507, 0.045);
const camera = new THREE.PerspectiveCamera(50, window.innerWidth/window.innerHeight, 0.05, 100);
camera.position.set(4.5, 4.2, 6.0);
const renderer = new THREE.WebGLRenderer({ antialias: true, powerPreference: 'high-performance' });
renderer.setPixelRatio(Math.min(2, window.devicePixelRatio));
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.toneMapping = THREE.ACESFilmicToneMapping;
renderer.toneMappingExposure = 0.85;
renderer.outputEncoding = THREE.sRGBEncoding;
document.body.appendChild(renderer.domElement);
const controls = new THREE.OrbitControls(camera, renderer.domElement);
controls.target.set(0, 0.4, -0.5);
controls.enableDamping = true; controls.dampingFactor = 0.06;
controls.minDistance = 3; controls.maxDistance = 18;
controls.maxPolarAngle = Math.PI * 0.49;
scene.add(new THREE.HemisphereLight(0x553a18, 0x080606, 0.7));
const keyLight = new THREE.DirectionalLight(0xffc070, 0.9);
keyLight.position.set(3, 6, 4);
scene.add(keyLight);
// Post-processing — gentle bloom so the heatmap + puck glow.
const composer = new THREE.EffectComposer(renderer);
composer.addPass(new THREE.RenderPass(scene, camera));
const bloom = new THREE.UnrealBloomPass(
new THREE.Vector2(window.innerWidth, window.innerHeight), 0.55, 0.45, 0.82);
composer.addPass(bloom);
// =====================================================================
// Room: floor grid + wall + doorway dividing office / hallway
// =====================================================================
const gridHelper = new THREE.GridHelper(2*ROOM_HALF, GRID_N, 0x554a32, 0x2a2418);
gridHelper.position.y = 0.002;
scene.add(gridHelper);
// Dividing wall runs along world X near z = -1 (office z>-1, hallway z<-1),
// with a doorway gap. Two wall segments leave a gap in the middle.
const wallMat = new THREE.MeshStandardMaterial({
color: 0x1b2330, transparent: true, opacity: 0.55, roughness: 0.9,
side: THREE.DoubleSide,
});
const wallH = 1.4, wallZ = -1.0;
function addWallSeg(cx, w) {
const m = new THREE.Mesh(new THREE.BoxGeometry(w, wallH, 0.08), wallMat);
m.position.set(cx, wallH/2, wallZ);
scene.add(m);
// top edge highlight
const edge = new THREE.Mesh(new THREE.BoxGeometry(w, 0.02, 0.10),
new THREE.MeshBasicMaterial({ color: 0x4cf, transparent: true, opacity: 0.5 }));
edge.position.set(cx, wallH, wallZ);
scene.add(edge);
}
// left segment, doorway gap (-0.7..0.7), right segment
addWallSeg(-3.15, 3.7);
addWallSeg( 3.15, 3.7);
// Room labels (sprite text) for OFFICE / HALLWAY
function makeLabel(text, color) {
const c = document.createElement('canvas'); c.width = 256; c.height = 64;
const ctx = c.getContext('2d');
ctx.fillStyle = color; ctx.font = 'bold 30px Consolas, monospace';
ctx.textAlign = 'center'; ctx.textBaseline = 'middle';
ctx.fillText(text, 128, 34);
const tex = new THREE.CanvasTexture(c);
const spr = new THREE.Sprite(new THREE.SpriteMaterial({ map: tex, transparent: true, depthTest: false }));
spr.scale.set(2.0, 0.5, 1);
return spr;
}
const officeLbl = makeLabel('OFFICE', '#ffb840'); officeLbl.position.set(2.6, 0.06, 2.6); scene.add(officeLbl);
const hallLbl = makeLabel('HALLWAY', '#ff4cc8'); hallLbl.position.set(-2.6, 0.06, -3.2); scene.add(hallLbl);
// =====================================================================
// Node markers (office / hallway). The hallway node is dimmed until it
// actually appears in the live `nodes` list.
// =====================================================================
const nodeMeshes = {};
function buildNode(id) {
const a = NODE_ANCHORS[id];
const g = new THREE.Group();
const post = new THREE.Mesh(
new THREE.CylinderGeometry(0.05, 0.07, 0.9, 12),
new THREE.MeshStandardMaterial({ color: a.color, emissive: a.color, emissiveIntensity: 0.4, roughness: 0.4 }));
post.position.y = 0.45; g.add(post);
const orb = new THREE.Mesh(
new THREE.SphereGeometry(0.12, 20, 16),
new THREE.MeshBasicMaterial({ color: a.color }));
orb.position.y = 0.95; g.add(orb);
const ring = new THREE.Mesh(
new THREE.RingGeometry(0.18, 0.24, 32),
new THREE.MeshBasicMaterial({ color: a.color, transparent: true, opacity: 0.6, side: THREE.DoubleSide }));
ring.rotation.x = -Math.PI/2; ring.position.y = 0.01; g.add(ring);
const lbl = makeLabel('ESP32-S3 ' + a.label, '#' + a.color.toString(16).padStart(6,'0'));
lbl.scale.set(2.6, 0.65, 1); lbl.position.set(0, 1.25, 0); g.add(lbl);
g.position.set(a.pos[0], 0, a.pos[2]);
g.userData.parts = { post, orb, ring };
scene.add(g);
return g;
}
Object.keys(NODE_ANCHORS).forEach(id => { nodeMeshes[id] = buildNode(+id); });
function setNodeActive(id, active) {
const g = nodeMeshes[id]; if (!g) return;
const o = active ? 1.0 : 0.22;
const parts = g.userData.parts;
parts.orb.material.opacity = o; parts.orb.material.transparent = true;
parts.ring.material.opacity = 0.6 * o;
parts.post.material.emissiveIntensity = active ? 0.5 : 0.12;
}
setNodeActive(9, false); setNodeActive(13, false);
// =====================================================================
// signal_field 20x20 floor heatmap — instanced colored tiles.
// Driven ONLY by real `signal_field.values` (400 floats ~0..1).
// =====================================================================
const TILE = (2*ROOM_HALF) / GRID_N;
const heatGeo = new THREE.PlaneGeometry(TILE * 0.96, TILE * 0.96);
const heatMat = new THREE.MeshBasicMaterial({ vertexColors: true, transparent: true, opacity: 0.85, side: THREE.DoubleSide });
const heatMesh = new THREE.InstancedMesh(heatGeo, heatMat, GRID_N * GRID_N);
heatMesh.instanceMatrix.setUsage(THREE.DynamicDrawUsage);
const heatColor = new THREE.InstancedBufferAttribute(new Float32Array(GRID_N * GRID_N * 3), 3);
heatMesh.instanceColor = heatColor;
const _m = new THREE.Matrix4();
const _q = new THREE.Quaternion().setFromAxisAngle(new THREE.Vector3(1,0,0), -Math.PI/2);
const _s = new THREE.Vector3(1,1,1);
const _p = new THREE.Vector3();
// gridCell (gx,gz) -> world (x,z). gx,gz in [0,GRID_N).
function cellToWorld(gx, gz) {
return [ (gx + 0.5) * TILE - ROOM_HALF, (gz + 0.5) * TILE - ROOM_HALF ];
}
for (let gz = 0; gz < GRID_N; gz++) {
for (let gx = 0; gx < GRID_N; gx++) {
const i = gz * GRID_N + gx;
const [wx, wz] = cellToWorld(gx, gz);
_p.set(wx, 0.012, wz);
_m.compose(_p, _q, _s);
heatMesh.setMatrixAt(i, _m);
heatColor.setXYZ(i, 0.02, 0.02, 0.03);
}
}
heatMesh.instanceMatrix.needsUpdate = true;
scene.add(heatMesh);
// amber→white heat ramp for a value in [0,1]
function heatRamp(v, out) {
v = Math.max(0, Math.min(1, v));
// dark -> amber -> hot white
const r = Math.min(1, 0.05 + 1.6 * v);
const g = Math.min(1, 0.02 + 1.1 * v * v);
const b = Math.min(1, 0.04 + 0.9 * Math.pow(v, 3));
out.set(r, g, b);
return out;
}
const _c = new THREE.Color();
let lastFieldPeak = { gx: GRID_N/2|0, gz: GRID_N/2|0, v: 0 };
function updateHeatmap(field) {
if (!field || !Array.isArray(field.values)) return;
const vals = field.values;
// grid_size is [20,1,20]; values are row-major 400 floats.
let peakV = -1, peakGx = lastFieldPeak.gx, peakGz = lastFieldPeak.gz;
const n = Math.min(vals.length, GRID_N * GRID_N);
for (let i = 0; i < n; i++) {
const v = vals[i];
heatRamp(v, _c);
heatColor.setXYZ(i, _c.r, _c.g, _c.b);
if (v > peakV) { peakV = v; peakGx = i % GRID_N; peakGz = (i / GRID_N) | 0; }
}
heatColor.needsUpdate = true;
lastFieldPeak = { gx: peakGx, gz: peakGz, v: peakV };
}
// =====================================================================
// RF-localization puck — from persons[0].position (coarse, NOT pose).
// Falls back to the signal_field peak cell when no person is present.
// =====================================================================
const puck = new THREE.Group();
const puckCore = new THREE.Mesh(
new THREE.SphereGeometry(0.16, 24, 18),
new THREE.MeshBasicMaterial({ color: 0x66ff88 }));
puckCore.position.y = 0.16; puck.add(puckCore);
const puckRing = new THREE.Mesh(
new THREE.RingGeometry(0.28, 0.36, 40),
new THREE.MeshBasicMaterial({ color: 0x66ff88, transparent: true, opacity: 0.7, side: THREE.DoubleSide }));
puckRing.rotation.x = -Math.PI/2; puckRing.position.y = 0.02; puck.add(puckRing);
const puckBeam = new THREE.Mesh(
new THREE.CylinderGeometry(0.03, 0.03, 1.2, 8),
new THREE.MeshBasicMaterial({ color: 0x66ff88, transparent: true, opacity: 0.35 }));
puckBeam.position.y = 0.6; puck.add(puckBeam);
puck.visible = false;
scene.add(puck);
const puckTarget = new THREE.Vector3(0, 0, 0);
function updatePuck(frame) {
let wx = null, wz = null, present = false;
const persons = frame.persons || [];
if (persons.length && Array.isArray(persons[0].position)) {
// server position is [x, 0, z] in metres, origin at room centre
wx = persons[0].position[0];
wz = persons[0].position[2];
present = true;
}
// If no person but the field has clear energy, show the peak cell
// (coarse) so the puck honestly tracks "where the RF energy is".
if (!present && lastFieldPeak.v > 0.55) {
const peak = cellToWorld(lastFieldPeak.gx, lastFieldPeak.gz);
wx = peak[0]; wz = peak[1]; present = true;
}
if (present && wx !== null) {
// clamp into the room so it never flies off the floor
wx = Math.max(-ROOM_HALF+0.3, Math.min(ROOM_HALF-0.3, wx));
wz = Math.max(-ROOM_HALF+0.3, Math.min(ROOM_HALF-0.3, wz));
puckTarget.set(wx, 0, wz);
puck.visible = true;
} else {
puck.visible = false;
}
}
// =====================================================================
// HUD updates
// =====================================================================
const $ = id => document.getElementById(id);
function clamp01(x){ return Math.max(0, Math.min(1, x)); }
function setBar(barId, valId, frac, text) {
$(barId).style.width = (clamp01(frac) * 100).toFixed(0) + '%';
$(valId).textContent = text;
}
// motion sparkline ring buffer
const sparkCtx = $('spark').getContext('2d');
const SPARK_N = 120;
const sparkBuf = new Array(SPARK_N).fill(0);
function pushSpark(v) {
sparkBuf.push(v); if (sparkBuf.length > SPARK_N) sparkBuf.shift();
const w = sparkCtx.canvas.width, h = sparkCtx.canvas.height;
sparkCtx.clearRect(0,0,w,h);
let maxV = 40; for (const x of sparkBuf) if (x > maxV) maxV = x;
sparkCtx.strokeStyle = '#ffb840'; sparkCtx.lineWidth = 1.5; sparkCtx.beginPath();
for (let i = 0; i < sparkBuf.length; i++) {
const x = (i / (SPARK_N-1)) * w;
const y = h - (sparkBuf[i] / maxV) * (h - 3) - 1.5;
i === 0 ? sparkCtx.moveTo(x, y) : sparkCtx.lineTo(x, y);
}
sparkCtx.stroke();
}
// =====================================================================
// Honest status banner (strict, mutually exclusive)
// =====================================================================
const banner = $('banner');
function setBannerLive(source, nodeCount) {
if (source === 'esp32') {
banner.className = 'live';
banner.innerHTML = 'LIVE — real ESP32 CSI <span class="src">(source=' + source + ', ' + nodeCount + ' node' + (nodeCount === 1 ? '' : 's') + ')</span>';
} else {
// anything not esp32 = explicitly NOT real, badged
banner.className = 'sim';
banner.innerHTML = 'SIMULATED — not real <span class="src">(source=' + source + ' — start an ESP32 for live CSI)</span>';
}
}
function setBannerNoServer() {
banner.className = 'noserver';
banner.innerHTML = 'NO SERVER — start the sensing-server <span class="src">(ws://localhost:8765/ws/sensing unreachable)</span>';
}
// =====================================================================
// WebSocket — render ONLY real frames. Reconnect; never fabricate.
// =====================================================================
let ws = null, gotFrame = false;
let frameTimes = []; // for measured update rate (fps)
let lastFrame = null; // most recent real frame (render loop interpolates puck)
function connect() {
setBannerNoServer();
try { ws = new WebSocket(WS_URL); }
catch (e) { scheduleReconnect(); return; }
ws.onopen = () => { /* wait for first frame before claiming LIVE */ };
ws.onmessage = (ev) => {
let d; try { d = JSON.parse(ev.data); } catch (e) { return; }
if (!d || d.type !== 'sensing_update') return;
onFrame(d);
};
ws.onclose = () => { gotFrame = false; $('waiting').classList.add('show'); setBannerNoServer(); scheduleReconnect(); };
ws.onerror = () => { try { ws.close(); } catch (e) {} };
}
let reconnectT = null;
function scheduleReconnect() {
if (reconnectT) return;
reconnectT = setTimeout(() => { reconnectT = null; connect(); }, 1500);
}
function onFrame(d) {
gotFrame = true;
lastFrame = d;
$('waiting').classList.remove('show');
const source = d.source || 'unknown';
const nodes = Array.isArray(d.nodes) ? d.nodes : [];
setBannerLive(source, nodes.length);
// measured update rate
const now = performance.now();
frameTimes.push(now);
while (frameTimes.length && now - frameTimes[0] > 2000) frameTimes.shift();
const fps = frameTimes.length > 1 ? (frameTimes.length - 1) / ((frameTimes[frameTimes.length-1] - frameTimes[0]) / 1000) : 0;
const cls = d.classification || {};
const feat = d.features || {};
// info panel
$('m-source').textContent = source.toUpperCase();
$('m-source').className = 'v ' + (source === 'esp32' ? 'green' : 'red');
const presence = !!cls.presence;
$('m-presence').textContent = presence ? (cls.motion_level === 'present_moving' ? 'PRESENT · MOVING' : 'PRESENT') : 'CLEAR';
$('m-presence').className = 'v ' + (presence ? 'green' : 'mute');
$('m-motion').textContent = cls.motion_level || '—';
$('m-conf').textContent = (cls.confidence != null) ? cls.confidence.toFixed(2) : '—';
$('m-persons').textContent = (d.estimated_persons != null) ? d.estimated_persons : '—';
$('m-nodes').textContent = nodes.length + ' (' + nodes.map(n => n.node_id).join(', ') + ')';
$('m-tick').textContent = (d.tick != null) ? d.tick : '—';
$('m-fps').textContent = fps ? fps.toFixed(1) + ' Hz' : '—';
// feature bars (real values, scaled into 0..1 for the bar width only)
const motion = feat.motion_band_power || 0;
const breath = feat.breathing_band_power || 0;
const variance = feat.variance || 0;
const rssi = feat.mean_rssi != null ? feat.mean_rssi : -100;
setBar('bar-motion', 'v-motion', motion / 100, motion.toFixed(1));
setBar('bar-breath', 'v-breath', breath / 100, breath.toFixed(1));
setBar('bar-var', 'v-var', variance / 80, variance.toFixed(1));
// rssi: map -90..-30 dBm -> 0..1
setBar('bar-rssi', 'v-rssi', (rssi + 90) / 60, rssi.toFixed(0));
pushSpark(motion);
// node activity
const activeIds = new Set(nodes.map(n => n.node_id));
[9, 13].forEach(id => setNodeActive(id, activeIds.has(id)));
// heatmap + puck
updateHeatmap(d.signal_field);
updatePuck(d);
}
// =====================================================================
// Optional webcam ground-truth tile (reused from demos/05). Camera is
// separate from CSI sensing — labeled "ground truth when visible".
// =====================================================================
let camStream = null;
$('cam-btn').addEventListener('click', async () => {
const btn = $('cam-btn');
if (camStream) { // toggle off
camStream.getTracks().forEach(t => t.stop());
$('cam-video').style.display = 'none'; camStream = null;
btn.textContent = '▶ enable webcam (optional)';
return;
}
btn.disabled = true; btn.textContent = 'requesting camera…';
try {
camStream = await navigator.mediaDevices.getUserMedia({
video: { width: { ideal: 640 }, height: { ideal: 480 }, facingMode: 'user' }, audio: false,
});
const v = $('cam-video'); v.srcObject = camStream; v.style.display = 'block';
btn.textContent = '■ stop webcam'; btn.disabled = false;
} catch (e) {
btn.textContent = '✗ camera unavailable'; btn.disabled = false; console.error(e);
setTimeout(() => { if (!camStream) btn.textContent = '▶ enable webcam (optional)'; }, 2000);
}
});
// =====================================================================
// Render loop — smooth the puck toward its real target; pulse rings.
// =====================================================================
const clock = new THREE.Clock();
function animate() {
requestAnimationFrame(animate);
const t = clock.getElapsedTime();
controls.update();
if (puck.visible) {
puck.position.lerp(puckTarget, 0.18);
const pulse = 0.8 + 0.25 * Math.sin(t * 3.0);
puckRing.scale.set(pulse, pulse, pulse);
puckRing.material.opacity = 0.5 + 0.25 * Math.sin(t * 3.0);
}
// node rings breathe when active
[9,13].forEach(id => {
const g = nodeMeshes[id]; if (!g) return;
const r = g.userData.parts.ring;
const s = 1 + 0.08 * Math.sin(t * 2 + id);
r.scale.set(s, s, s);
});
composer.render();
}
animate();
window.addEventListener('resize', () => {
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
renderer.setSize(window.innerWidth, window.innerHeight);
composer.setSize(window.innerWidth, window.innerHeight);
});
// kick off
connect();
</script>
</body>
</html>
+65
View File
@@ -0,0 +1,65 @@
"""Tiny threaded static server for the through-wall WiFi-CSI sensing demo.
Adapted from examples/three.js/server/serve-demo.py. Serves the
`examples/through-wall/` page so a browser can fetch index.html, then the
page connects directly to the LIVE sensing-server WebSocket at
ws://localhost:8765/ws/sensing (NOT proxied through here).
Why a threaded server (not `python -m http.server`)?
The stdlib SimpleHTTPServer is single-threaded; a browser opens several
parallel connections (HTML + the three.js CDN tags fetch in parallel),
the first eats the worker, the rest can stall. ThreadingHTTPServer fixes it.
IMPORTANT: this serves on port 8080 port 8765 is taken by the
sensing-server's WebSocket. They are two different processes.
Usage:
# 1) start the REAL sensing-server (separate terminal):
# cd v2
# cargo build -p wifi-densepose-sensing-server
# ./target/debug/sensing-server.exe --ws-port 8765 --udp-port 5005
# 2) start this static server:
python examples/through-wall/serve.py
# 3) open:
# http://localhost:8080/examples/through-wall/index.html
Override the WS endpoint with a query param, e.g.:
http://localhost:8080/examples/through-wall/index.html?ws=ws://192.168.1.20:8765/ws/sensing
"""
from http.server import ThreadingHTTPServer, SimpleHTTPRequestHandler
import os
import sys
PORT = int(os.environ.get("PORT", 8080))
# Serve from the repo root regardless of where this script is launched.
# This file lives at examples/through-wall/serve.py — two levels deep.
os.chdir(os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")))
class NoCacheHandler(SimpleHTTPRequestHandler):
def end_headers(self):
# Aggressive no-cache so the browser ALWAYS fetches the latest
# index.html after edits, even on a soft refresh.
self.send_header("Cache-Control", "no-store, no-cache, must-revalidate, max-age=0")
self.send_header("Pragma", "no-cache")
self.send_header("Expires", "0")
super().end_headers()
def log_message(self, fmt, *args): # quieter logs
sys.stderr.write("[serve] " + (fmt % args) + "\n")
PAGE = "examples/through-wall/index.html"
with ThreadingHTTPServer(("127.0.0.1", PORT), NoCacheHandler) as srv:
print(f"serving {os.getcwd()} on http://127.0.0.1:{PORT}/")
print(f" open http://localhost:{PORT}/{PAGE}")
print("")
print(" The page connects to the LIVE sensing-server at")
print(" ws://localhost:8765/ws/sensing (start it first — see README.md).")
print(" Override with ?ws=ws://HOST:PORT/ws/sensing")
try:
srv.serve_forever()
except KeyboardInterrupt:
sys.exit(0)
+926
View File
@@ -0,0 +1,926 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>WiFlow Browser Trainer · calibrate → capture → train → infer, in your camera's frame</title>
<!--
WiFlow in-browser trainer (ADR-079 / ADR-180 + ADR-151 empty-room baseline).
A 4-STAGE GATED FLOW, all in the LAPTOP camera's coordinate frame:
0. CALIBRATE — empty-room baseline (Welford mean+std over the 410-d CSI vector).
Every CSI vector afterwards is expressed as deviation-from-baseline,
so a body's perturbation stands out from the static channel.
1. CAPTURE — MediaPipe Pose on the laptop camera = 17 COCO keypoints (the LABEL),
paired with the baseline-normalized live ESP32 CSI vector (the INPUT).
Guided, balanced routine with a per-pose coverage meter.
2. TRAIN — a TF.js MLP (WebGPU/WASM/WebGL) learns CSI -> pose in-browser. Honest
held-out PCK + a mean-pose baseline it must beat.
3. INFER — the trained model drives a skeleton FROM WiFi CSI ONLY, drawn over the
same camera frame, so it ALIGNS (the whole point of doing it in-browser).
Self-contained. CDN libs only. No bundler. Real data only — CSI source must read "esp32".
-->
<style>
:root{--bg:#0a0c10;--panel:#11151c;--panel2:#0d1117;--amber:#ffb840;--green:#46e08a;
--red:#ff5a5a;--blue:#5aa9ff;--mute:#7d8796;--line:#1d2430;--txt:#dfe6ee}
*{box-sizing:border-box}
body{margin:0;background:var(--bg);color:var(--txt);
font:14px/1.5 'JetBrains Mono',ui-monospace,Menlo,Consolas,monospace}
header{padding:14px 18px;border-bottom:1px solid var(--line);display:flex;align-items:center;gap:14px;flex-wrap:wrap}
h1{font-size:15px;margin:0;letter-spacing:1px;text-transform:uppercase;font-weight:600}
h1 span{color:var(--amber)}
#compute{padding:4px 10px;border-radius:5px;font-weight:600;font-size:11px;letter-spacing:.5px;
background:rgba(90,169,255,.12);color:var(--blue);border:1px solid var(--blue)}
#banner{margin-left:auto;padding:5px 12px;border-radius:5px;font-weight:600;font-size:12px;letter-spacing:.5px}
.live{background:rgba(70,224,138,.15);color:var(--green);border:1px solid var(--green)}
.sim{background:rgba(255,184,64,.15);color:var(--amber);border:1px solid var(--amber)}
.down{background:rgba(255,90,90,.15);color:var(--red);border:1px solid var(--red)}
/* progress stepper */
.steps{display:flex;gap:6px;padding:14px 18px 0;flex-wrap:wrap;align-items:center}
.step{display:flex;align-items:center;gap:8px;background:var(--panel);color:var(--mute);
border:1px solid var(--line);border-radius:8px;padding:8px 16px;cursor:pointer;font-weight:600;letter-spacing:.5px}
.step .num{display:inline-flex;width:20px;height:20px;border-radius:50%;background:var(--line);color:var(--txt);
align-items:center;justify-content:center;font-size:11px}
.step.on{color:var(--amber);border-color:var(--amber)}
.step.on .num{background:var(--amber);color:#0a0c10}
.step.done .num{background:var(--green);color:#0a0c10}
.step.locked{opacity:.45;cursor:not-allowed}
.arrow{color:var(--mute)}
main{padding:14px 18px 24px}
.panel{display:none;background:var(--panel2);border:1px solid var(--line);border-radius:10px;padding:18px}
.panel.on{display:block}
.cols{display:flex;gap:18px;flex-wrap:wrap}
.card{background:var(--panel);border:1px solid var(--line);border-radius:10px;padding:14px}
canvas{background:#070a0e;border-radius:8px;display:block}
.label{font-size:11px;text-transform:uppercase;letter-spacing:1.5px;color:var(--mute);margin-bottom:8px}
.stats{min-width:260px;flex:1}
.row{display:flex;justify-content:space-between;padding:3px 0;border-bottom:1px dashed var(--line)}
.row .k{color:var(--mute)} .row .v{color:var(--amber);font-variant-numeric:tabular-nums;text-align:right}
.v.green{color:var(--green)} .v.red{color:var(--red)} .v.blue{color:var(--blue)}
.note{margin-top:12px;font-size:11px;color:var(--mute);line-height:1.6}
.note b{color:var(--txt)}
button.btn{background:var(--amber);color:#0a0c10;border:0;border-radius:6px;padding:8px 16px;
font:inherit;font-weight:600;cursor:pointer}
button.btn:disabled{opacity:.4;cursor:not-allowed}
button.ghost{background:transparent;color:var(--txt);border:1px solid var(--line)}
select,input{background:var(--panel);color:var(--txt);border:1px solid var(--line);border-radius:6px;
padding:7px;font:inherit;max-width:260px}
.bar{height:8px;background:var(--line);border-radius:5px;overflow:hidden;margin-top:4px}
.bar>i{display:block;height:100%;background:var(--green);width:0%}
.verdict{padding:10px 14px;border-radius:8px;margin-top:12px;font-weight:600;font-size:13px}
.verdict.good{background:rgba(70,224,138,.12);color:var(--green);border:1px solid var(--green)}
.verdict.bad{background:rgba(255,90,90,.12);color:var(--red);border:1px solid var(--red)}
.verdict.idle{background:rgba(125,135,150,.1);color:var(--mute);border:1px solid var(--line)}
.pill{display:inline-block;padding:2px 8px;border-radius:10px;font-size:11px;font-weight:600;margin-left:6px}
.pill.gt{background:rgba(90,169,255,.15);color:var(--blue);border:1px solid var(--blue)}
.pill.csi{background:rgba(70,224,138,.15);color:var(--green);border:1px solid var(--green)}
code{background:#0a0c10;border:1px solid var(--line);border-radius:4px;padding:1px 5px;color:var(--amber)}
a{color:var(--blue)}
/* big guided prompt */
#prompt{font-size:30px;font-weight:700;color:var(--amber);letter-spacing:1px;text-align:center;margin:6px 0}
#countdown{font-size:13px;color:var(--mute);text-align:center}
/* coverage meter */
.cov{display:flex;flex-direction:column;gap:5px;margin-top:8px}
.covrow{display:flex;align-items:center;gap:8px;font-size:11px}
.covrow .nm{width:90px;color:var(--mute);text-transform:capitalize}
.covrow .bar{flex:1;margin:0}
.covrow .ct{width:42px;text-align:right;color:var(--txt);font-variant-numeric:tabular-nums}
</style>
</head>
<body>
<header>
<h1>WiFlow <span>Browser Trainer</span> — calibrate · capture · train · infer</h1>
<div id="compute">compute: …</div>
<div id="banner" class="down">CONNECTING…</div>
</header>
<!-- progress stepper, each gated on the previous -->
<div class="steps">
<div class="step on" data-stage="calibrate"><span class="num">0</span> CALIBRATE</div>
<span class="arrow"></span>
<div class="step locked" data-stage="capture"><span class="num">1</span> CAPTURE</div>
<span class="arrow"></span>
<div class="step locked" data-stage="train"><span class="num">2</span> TRAIN</div>
<span class="arrow"></span>
<div class="step locked" data-stage="infer"><span class="num">3</span> INFER</div>
</div>
<main>
<!-- ============================ STAGE 0 · CALIBRATE ============================ -->
<section id="stage-calibrate" class="panel on">
<div class="cols">
<div class="card">
<div class="label">empty-room baseline (ADR-151) — step OUT of the space</div>
<canvas id="calCv" width="420" height="300"></canvas>
<div style="margin-top:10px;display:flex;gap:8px;align-items:center;flex-wrap:wrap">
<button id="calBtn" class="btn">calibrate baseline (10 s)</button>
<button id="recalBtn" class="ghost btn">recalibrate</button>
</div>
</div>
<div class="card stats">
<div class="label">baseline</div>
<div class="row"><span class="k">CSI source</span><span class="v" id="calSrc"></span></div>
<div class="row"><span class="k">status</span><span class="v" id="calStatus">NOT CALIBRATED</span></div>
<div class="row"><span class="k">frames in baseline</span><span class="v" id="calN">0</span></div>
<div class="row"><span class="k">age</span><span class="v" id="calAge"></span></div>
<div style="margin-top:8px"><div class="bar"><i id="calBar"></i></div></div>
<div class="note">
The room's static WiFi channel is mostly constant. We capture ~10 s of the
<b>quiescent</b> field (you OUT of the space) and compute a per-feature running
<b>mean + std</b> (Welford) over the 410-d CSI vector. Afterwards every CSI vector
is expressed as <b>deviation from baseline</b>:
<code>x_norm = (x base_mean) / (base_std + ε)</code> — applied consistently in
capture, train, and infer. This makes a <b>body's perturbation</b> stand out from
the static channel. You must calibrate before capturing.
</div>
</div>
</div>
</section>
<!-- ============================ STAGE 1 · CAPTURE ============================ -->
<section id="stage-capture" class="panel">
<div class="cols">
<div class="card">
<div class="label">laptop camera <span class="pill gt">MediaPipe skeleton = GROUND TRUTH (the label)</span></div>
<canvas id="capCv" width="420" height="480"></canvas>
<div id="prompt">stand still</div>
<div id="countdown"></div>
<div style="margin-top:8px;display:flex;gap:8px;align-items:center;flex-wrap:wrap">
<button id="camBtn" class="btn">enable laptop camera</button>
<select id="camSel" style="display:none"></select>
</div>
<div id="camStatus" class="note" style="margin-top:6px">camera: off</div>
</div>
<div class="card stats">
<div class="label">guided capture</div>
<div class="row"><span class="k">CSI source</span><span class="v" id="capSrc"></span></div>
<div class="row"><span class="k">CSI nodes</span><span class="v" id="capNodes"></span></div>
<div class="row"><span class="k">pose visibility</span><span class="v" id="capVis"></span></div>
<div class="row"><span class="k">total samples</span><span class="v green" id="capN">0</span></div>
<div class="row"><span class="k">last skip reason</span><span class="v" id="capSkip"></span></div>
<div style="margin-top:12px;display:flex;gap:8px;flex-wrap:wrap">
<button id="recBtn" class="btn" disabled>● start guided recording</button>
<button id="clrBtn" class="ghost btn">clear dataset</button>
</div>
<div class="label" style="margin-top:16px">per-pose coverage (balance the dataset)</div>
<div id="cov" class="cov"></div>
<div class="note">
A pair is recorded <b>only</b> when BOTH (a) a confident MediaPipe pose
(mean visibility &gt; 0.5) AND (b) a fresh <b>live</b> CSI frame (<code>source==esp32</code>)
exist. We store the <b>baseline-normalized</b> CSI + the 17 keypoints, mirrored to
IndexedDB so a refresh keeps them. Follow the prompt so every pose bucket fills up —
a balanced set beats 2 000 frames of standing.
</div>
</div>
</div>
</section>
<!-- ============================ STAGE 2 · TRAIN ============================ -->
<section id="stage-train" class="panel">
<div class="cols">
<div class="card stats">
<div class="label">train (TensorFlow.js)</div>
<div class="row"><span class="k">total samples</span><span class="v" id="trN">0</span></div>
<div class="row"><span class="k">train / val split</span><span class="v" id="trSplit">— / — (chronological 80/20)</span></div>
<div class="row"><span class="k">epoch</span><span class="v" id="trEpoch">0</span></div>
<div class="row"><span class="k">train MSE</span><span class="v" id="trLoss"></span></div>
<div class="row"><span class="k">val MSE</span><span class="v" id="trVal"></span></div>
<div class="row"><span class="k">held-out PCK@0.10</span><span class="v green" id="trP10"></span></div>
<div class="row"><span class="k">held-out PCK@0.05</span><span class="v" id="trP05"></span></div>
<div class="row"><span class="k">held-out MPJPE</span><span class="v" id="trMpj"></span></div>
<div class="row"><span class="k">mean-pose baseline PCK@0.10</span><span class="v red" id="trBase"></span></div>
<div style="margin-top:8px"><div class="bar"><i id="trBar"></i></div></div>
<div style="margin-top:12px;display:flex;gap:8px;flex-wrap:wrap;align-items:center">
<label class="note" style="margin:0">epochs <input id="trEpochs" type="number" value="200" min="20" max="600" style="width:80px"></label>
<button id="trainBtn" class="btn" disabled>train model</button>
<button id="trStop" class="ghost btn" disabled>stop</button>
</div>
<div id="verdict" class="verdict idle">no model yet — calibrate, capture, then train.</div>
<div class="note">
<b>The bar to beat</b> is the mean-pose baseline (predict the train-mean pose for
everything). A model that doesn't clear it has learned <b>no usable CSI→pose signal</b>
this page says so plainly. Inputs are standardized on the <b>train split only</b>
(after baseline-normalization); the val split is the chronological last 20%, never trained on.
</div>
</div>
<div class="card">
<div class="label">loss curve — train (amber) vs val (blue)</div>
<canvas id="lossCv" width="460" height="300"></canvas>
<div class="note" id="trMsg">Idle.</div>
</div>
</div>
</section>
<!-- ============================ STAGE 3 · INFER ============================ -->
<section id="stage-infer" class="panel">
<div class="cols">
<div class="card">
<div class="label">WiFi-inferred pose <span class="pill csi">CSI ONLY — no camera in the loop</span></div>
<canvas id="infCv" width="420" height="560"></canvas>
<div style="margin-top:10px;display:flex;gap:8px;align-items:center;flex-wrap:wrap">
<label class="note" style="margin:0"><input type="checkbox" id="hideCam"> hide camera (skeleton on black)</label>
<span class="note" id="infModelState" style="margin:0">no model loaded</span>
</div>
</div>
<div class="card stats">
<div class="label">live inference</div>
<div class="row"><span class="k">CSI source</span><span class="v" id="infSrc"></span></div>
<div class="row"><span class="k">CSI nodes</span><span class="v" id="infNodes"></span></div>
<div class="row"><span class="k">presence</span><span class="v" id="infPres"></span></div>
<div class="row"><span class="k">infer fps</span><span class="v" id="infFps"></span></div>
<div class="row"><span class="k">measured held-out PCK@0.10</span><span class="v green" id="infPck"></span></div>
<div class="note">
This skeleton is inferred <b>from WiFi CSI only</b> (baseline-normalized, then through
the model). It is <b>coarse</b> — the held-out PCK above is the real number. It is drawn
over the <b>same</b> laptop-camera frame it trained in, so it <b>aligns</b> with the image.
Same person / room / session — not validated cross-day or through-wall.
</div>
</div>
</div>
</section>
</main>
<!-- TensorFlow.js core + WebGPU/WASM backends (WebGL ships inside core as the final fallback) -->
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@4.22.0/dist/tf.min.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-webgpu@4.22.0/dist/tf-backend-webgpu.min.js" crossorigin="anonymous"></script>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@4.22.0/dist/tf-backend-wasm.min.js" crossorigin="anonymous"></script>
<!-- MediaPipe Pose 0.5 (legacy solutions API — same CDN the 05-skinned-realtime demo uses) -->
<script src="https://cdn.jsdelivr.net/npm/@mediapipe/pose@0.5/pose.js" crossorigin="anonymous"></script>
<script>
"use strict";
// ============================================================================
// Constants & shared state
// ============================================================================
const CSI_WS = (new URLSearchParams(location.search)).get('ws')
|| `ws://${location.hostname || 'localhost'}:8765/ws/sensing`;
const NODE_IDS = [9, 13]; // per-node features in this fixed order (matches Python pipeline)
const FIELD_LEN = 400; // signal_field.values padded/truncated to 400
const CSI_DIM = 4 + NODE_IDS.length * 3 + FIELD_LEN; // 4 + 6 + 400 = 410
const N_KP = 17, OUT_DIM = N_KP * 2; // 17 COCO keypoints -> 34 coords
const BASELINE_SECONDS = 10; // empty-room calibration window
const EPS = 1e-6;
// MediaPipe BlazePose (33) -> 17 COCO keypoints (identical to wiflow_capture.py / ADR-079)
const COCO_FROM_MP = [0, 2, 5, 7, 8, 11, 12, 13, 14, 15, 16, 23, 24, 25, 26, 27, 28];
const EDGES = [[5,7],[7,9],[6,8],[8,10],[5,6],[11,12],[5,11],[6,12],
[11,13],[13,15],[12,14],[14,16],[0,1],[0,2],[1,3],[2,4],[0,5],[0,6]];
const $ = id => document.getElementById(id);
function banner(state, txt){ const b=$('banner'); b.className=state; b.textContent=txt; }
// In-memory dataset of {csi:Float32Array(410, baseline-normalized), kps:Float32Array(34), bucket:int}
let SAMPLES = [];
// Latest live CSI frame + RAW 410-vector (baseline-normalization applied at use sites)
let latestCSI = { t: 0, frame: null, vec: null, source: null, nodes: [] };
// Empty-room baseline: per-feature mean + std (ADR-151)
let baseline = null; // { mean:Float32Array(410), std:Float32Array(410), n:int, ts:number }
// ============================================================================
// TF.js backend selection — WebGPU primary, WASM-SIMD fallback, WebGL last.
// ============================================================================
const BACKEND_LABEL = { webgpu:'WebGPU', wasm:'WASM-SIMD', webgl:'WebGL', cpu:'CPU (slow)' };
let activeBackend = null;
async function selectBackend(){
try{ if (tf.wasm && tf.wasm.setWasmPaths)
tf.wasm.setWasmPaths('https://cdn.jsdelivr.net/npm/@tensorflow/tfjs-backend-wasm@4.22.0/dist/'); }catch(e){}
const tryBackend = async (name)=>{
try{ const ok = await tf.setBackend(name); if (!ok) return false; await tf.ready();
return tf.getBackend() === name; }
catch(e){ console.warn('backend '+name+' unavailable:', e.message); return false; }
};
if (await tryBackend('webgpu')) activeBackend = 'webgpu';
else if (await tryBackend('wasm')) activeBackend = 'wasm';
else if (await tryBackend('webgl')) activeBackend = 'webgl';
else { await tf.ready(); activeBackend = tf.getBackend(); }
const badge = $('compute');
badge.textContent = 'compute: ' + (BACKEND_LABEL[activeBackend] || activeBackend);
badge.title = 'TensorFlow.js backend actually running (WebGPU → WASM-SIMD → WebGL)';
return activeBackend;
}
// ============================================================================
// CSI vector construction — MUST match wiflow_capture.py csi_vector() exactly.
// [mean_rssi, variance, motion_band_power, breathing_band_power] (4 global)
// + for node 9 then node 13: [mean_rssi, variance, motion_band_power] (6 per-node)
// + signal_field.values padded/truncated to 400 (400 field)
// = 410-d (RAW — baseline-normalization applied separately, see baselineNorm)
// ============================================================================
function csiVector(frame){
const f = frame.features || {};
const out = new Float32Array(CSI_DIM);
let o = 0;
out[o++] = +f.mean_rssi || 0;
out[o++] = +f.variance || 0;
out[o++] = +f.motion_band_power || 0;
out[o++] = +f.breathing_band_power || 0;
const perNode = {};
for (const nf of (frame.node_features || [])) perNode[nf.node_id] = (nf.features || {});
for (const nid of NODE_IDS){
const nf = perNode[nid] || {};
out[o++] = +nf.mean_rssi || 0;
out[o++] = +nf.variance || 0;
out[o++] = +nf.motion_band_power || 0;
}
const field = ((frame.signal_field || {}).values) || [];
for (let i = 0; i < FIELD_LEN; i++) out[o++] = +field[i] || 0;
return out;
}
// ADR-151 baseline-deviation normalization: x_norm = (x - base_mean) / (base_std + eps).
// Applied BEFORE the model's own input standardization, consistently everywhere.
function baselineNorm(vecRaw){
if (!baseline) return null;
const out = new Float32Array(CSI_DIM);
for (let j = 0; j < CSI_DIM; j++)
out[j] = (vecRaw[j] - baseline.mean[j]) / (baseline.std[j] + EPS);
return out;
}
// ============================================================================
// CSI WebSocket
// ============================================================================
function connectCSI(){
banner('down','CONNECTING…');
let ws;
try { ws = new WebSocket(CSI_WS); }
catch(e){ banner('down','NO-CSI-SERVER — start sensing-server :8765'); setTimeout(connectCSI, 1500); return; }
ws.onopen = ()=> banner('sim','WAITING FOR CSI…');
ws.onmessage = ev => {
let d; try { d = JSON.parse(ev.data); } catch(e){ return; }
if (!d.features && !d.signal_field) return;
const src = d.source || 'unknown';
latestCSI = {
t: performance.now(),
frame: d,
vec: csiVector(d), // RAW
source: src,
nodes: (d.nodes || []).map(n => n.node_id).filter(x => x != null).sort((a,b)=>a-b)
};
if (src === 'esp32') banner('live','LIVE — real ESP32 CSI');
else banner('sim',`SIMULATED — not real (source=${src})`);
};
ws.onerror = ()=>{ try{ws.close();}catch(e){} };
ws.onclose = ()=>{ banner('down','NO-CSI-SERVER — start sensing-server :8765'); setTimeout(connectCSI, 1500); };
}
function freshLiveCSI(){
return latestCSI.frame && latestCSI.source === 'esp32' && (performance.now() - latestCSI.t) < 400;
}
// ============================================================================
// Camera + MediaPipe Pose
// ============================================================================
let camStream = null;
const camEl = document.createElement('video');
camEl.autoplay = true; camEl.muted = true; camEl.playsInline = true;
let mpPose = null, mpReady = false, mpBusy = false;
let latestKps = null, latestVis = 0;
function initPose(){
if (mpPose || typeof Pose === 'undefined') return;
mpPose = new Pose({ locateFile: f => `https://cdn.jsdelivr.net/npm/@mediapipe/pose@0.5/${f}` });
mpPose.setOptions({ modelComplexity:1, smoothLandmarks:true, enableSegmentation:false,
minDetectionConfidence:0.5, minTrackingConfidence:0.5 });
mpPose.onResults(onPoseResults);
mpReady = true;
}
function onPoseResults(res){
mpBusy = false;
if (!res.poseLandmarks){ latestKps = null; latestVis = 0; return; }
const lm = res.poseLandmarks;
const kps = new Float32Array(OUT_DIM);
let visSum = 0;
for (let i = 0; i < N_KP; i++){
const p = lm[COCO_FROM_MP[i]];
kps[i*2] = p.x; kps[i*2+1] = p.y;
visSum += (p.visibility != null ? p.visibility : 0);
}
latestKps = kps; latestVis = visSum / N_KP;
}
async function startCam(deviceId){
if (camStream) camStream.getTracks().forEach(t => t.stop());
const constraints = deviceId ? { video:{ deviceId:{ exact:deviceId } } } : { video:true };
const st = $('camStatus');
try{
st.textContent = 'camera: requesting…';
camStream = await navigator.mediaDevices.getUserMedia(constraints);
camEl.srcObject = camStream;
await camEl.play().catch(()=>{});
const tr = camStream.getVideoTracks()[0];
const tick = ()=>{ st.textContent =
`camera: "${tr.label}" ${camEl.videoWidth}x${camEl.videoHeight} ${tr.readyState} ${camEl.paused?'PAUSED':'playing'}`; };
tick(); setInterval(tick, 1000);
$('camBtn').textContent = 'switch camera ↻';
$('recBtn').disabled = !baseline; // still gated on a baseline
const devs = (await navigator.mediaDevices.enumerateDevices()).filter(d => d.kind === 'videoinput');
const sel = $('camSel'); sel.style.display = devs.length > 1 ? 'inline-block' : 'none';
sel.innerHTML = devs.map((d,i)=>`<option value="${d.deviceId}">${d.label || ('camera '+(i+1))}</option>`).join('');
const cur = tr.getSettings().deviceId; if (cur) sel.value = cur;
initPose();
}catch(e){
$('camBtn').textContent = 'camera error: ' + e.name +
(e.name === 'NotReadableError' ? ' (in use by Zoom/Teams?)' : '');
console.error('getUserMedia', e);
}
}
$('camBtn').addEventListener('click', ()=> startCam());
$('camSel').addEventListener('change', e => startCam(e.target.value));
// ============================================================================
// Drawing helpers
// ============================================================================
function drawCameraFrame(ctx, W, H, alpha){
if (camEl && camEl.videoWidth > 0){
ctx.save(); ctx.globalAlpha = alpha;
const vr = camEl.videoWidth / camEl.videoHeight, cr = W / H;
let dw=W, dh=H, dx=0, dy=0;
if (vr > cr){ dh=H; dw=H*vr; dx=(W-dw)/2; } else { dw=W; dh=W/vr; dy=(H-dh)/2; }
ctx.drawImage(camEl, dx, dy, dw, dh); ctx.restore();
return true;
}
ctx.fillStyle = '#070a0e'; ctx.fillRect(0,0,W,H);
return false;
}
function drawSkeleton(ctx, kps, W, H, color, glow){
const k = [];
for (let i = 0; i < N_KP; i++) k.push([kps[i*2]*W, kps[i*2+1]*H]);
ctx.lineWidth = 5; ctx.strokeStyle = color; ctx.lineCap = 'round';
ctx.shadowColor = glow; ctx.shadowBlur = 8;
for (const [a,b] of EDGES){ ctx.beginPath(); ctx.moveTo(k[a][0],k[a][1]); ctx.lineTo(k[b][0],k[b][1]); ctx.stroke(); }
ctx.shadowBlur = 0;
for (const [x,y] of k){ ctx.beginPath(); ctx.arc(x,y,5,0,7); ctx.fillStyle = color; ctx.fill(); }
}
// ============================================================================
// Stage navigation + gating
// ============================================================================
const STAGES = ['calibrate','capture','train','infer'];
let stageDone = { calibrate:false, capture:false, train:false };
function stageUnlocked(name){
if (name === 'calibrate') return true;
if (name === 'capture') return stageDone.calibrate;
if (name === 'train') return stageDone.calibrate && SAMPLES.length >= 200;
if (name === 'infer') return !!model;
return false;
}
function gotoStage(name){
if (!stageUnlocked(name)) return;
document.querySelectorAll('.step').forEach(s => s.classList.remove('on'));
document.querySelectorAll('.panel').forEach(p => p.classList.remove('on'));
document.querySelector(`.step[data-stage="${name}"]`).classList.add('on');
$('stage-' + name).classList.add('on');
}
function refreshGates(){
document.querySelectorAll('.step').forEach(s=>{
const name = s.dataset.stage;
s.classList.toggle('locked', !stageUnlocked(name));
s.classList.toggle('done', !!stageDone[name]);
});
$('recBtn').disabled = !(baseline && camStream);
refreshTrainAvail();
}
document.querySelectorAll('.step').forEach(s => s.addEventListener('click', ()=> gotoStage(s.dataset.stage)));
// ============================================================================
// STAGE 0 · CALIBRATE — Welford running mean+std over the 410-d CSI vector
// ============================================================================
const calCtx = $('calCv').getContext('2d');
let calibrating = false;
let cw = null; // welford accumulators { n, mean:Float64Array, m2:Float64Array, t0 }
function startCalibration(){
if (calibrating) return;
cw = { n:0, mean:new Float64Array(CSI_DIM), m2:new Float64Array(CSI_DIM), t0:performance.now() };
calibrating = true;
$('calStatus').textContent = 'CALIBRATING…'; $('calStatus').className = 'v';
$('calBtn').disabled = true;
}
function welfordUpdate(vec){
cw.n++;
for (let j = 0; j < CSI_DIM; j++){
const d = vec[j] - cw.mean[j];
cw.mean[j] += d / cw.n;
cw.m2[j] += d * (vec[j] - cw.mean[j]);
}
}
function finishCalibration(){
calibrating = false;
const mean = new Float32Array(CSI_DIM), std = new Float32Array(CSI_DIM);
for (let j = 0; j < CSI_DIM; j++){
mean[j] = cw.mean[j];
std[j] = cw.n > 1 ? Math.sqrt(cw.m2[j] / (cw.n - 1)) : 0;
}
baseline = { mean, std, n: cw.n, ts: Date.now() };
stageDone.calibrate = true;
$('calStatus').textContent = 'CALIBRATED'; $('calStatus').className = 'v green';
$('calN').textContent = cw.n; $('calBtn').disabled = false;
$('calBar').style.width = '100%';
saveBaseline();
refreshGates();
}
$('calBtn').addEventListener('click', startCalibration);
$('recalBtn').addEventListener('click', ()=>{ baseline = null; stageDone.calibrate = false;
$('calStatus').textContent = 'NOT CALIBRATED'; $('calStatus').className = 'v';
$('calBar').style.width = '0%'; $('calN').textContent = '0'; idbDel('baseline'); refreshGates(); startCalibration(); });
function calibrateLoop(){
const W = $('calCv').width, H = $('calCv').height;
calCtx.fillStyle = '#070a0e'; calCtx.fillRect(0,0,W,H);
// little live trace of motion_band_power to show the channel is quiescent
$('calSrc').textContent = latestCSI.source || '—';
$('calSrc').className = latestCSI.source === 'esp32' ? 'v green' : 'v';
if (baseline){
const ageS = Math.round((Date.now() - baseline.ts)/1000);
$('calAge').textContent = ageS < 60 ? ageS+' s ago' : Math.round(ageS/60)+' min ago';
}
if (calibrating){
const el = (performance.now() - cw.t0) / 1000;
$('calBar').style.width = Math.min(100, 100*el/BASELINE_SECONDS) + '%';
// accumulate only fresh live frames; ignore sim so the baseline is real
if (freshLiveCSI() && latestCSI.vec){ welfordUpdate(latestCSI.vec); $('calN').textContent = cw.n; }
// draw a centered "STEP OUT" reminder + countdown
calCtx.fillStyle = '#ffb840'; calCtx.font = 'bold 22px monospace'; calCtx.textAlign='center';
calCtx.fillText('STEP OUT OF THE SPACE', W/2, H/2-10);
calCtx.fillStyle = '#7d8796'; calCtx.font = '14px monospace';
calCtx.fillText('baseline: '+Math.max(0,Math.ceil(BASELINE_SECONDS-el))+' s · '+cw.n+' frames', W/2, H/2+18);
calCtx.textAlign='start';
if (el >= BASELINE_SECONDS && cw.n > 0) finishCalibration();
else if (el >= BASELINE_SECONDS*2){ // safety: timed out with no live frames
calibrating = false; $('calStatus').textContent = 'NO LIVE CSI — check esp32'; $('calStatus').className='v red';
$('calBtn').disabled = false;
}
} else {
calCtx.fillStyle = baseline ? '#46e08a' : '#7d8796'; calCtx.font = '14px monospace'; calCtx.textAlign='center';
calCtx.fillText(baseline ? 'baseline ready ('+baseline.n+' frames)' : 'click “calibrate baseline”', W/2, H/2);
calCtx.textAlign='start';
}
requestAnimationFrame(calibrateLoop);
}
// ============================================================================
// STAGE 1 · GUIDED CAPTURE — balanced buckets + coverage meter
// ============================================================================
const capCtx = $('capCv').getContext('2d');
let recording = false;
// pose buckets (the guided routine cycles through these)
const BUCKETS = ['stand still','turn left','turn right','walk left','walk right',
'arms up','arms down','crouch','sit','reach'];
const SECS_PER_BUCKET = 12;
let bucketIx = 0, bucketT0 = performance.now();
let covCounts = new Array(BUCKETS.length).fill(0);
function renderCoverage(){
const max = Math.max(1, ...covCounts);
$('cov').innerHTML = BUCKETS.map((b,i)=>
`<div class="covrow"><span class="nm">${b}</span>`+
`<span class="bar"><i style="width:${Math.round(100*covCounts[i]/max)}%"></i></span>`+
`<span class="ct">${covCounts[i]}</span></div>`).join('');
}
$('recBtn').addEventListener('click', ()=>{
if (!baseline || !camStream) return;
recording = !recording;
$('recBtn').textContent = recording ? '◼ stop recording' : '● start guided recording';
$('recBtn').classList.toggle('ghost', recording);
if (recording){ bucketT0 = performance.now(); }
});
$('clrBtn').addEventListener('click', async ()=>{
SAMPLES = []; covCounts = new Array(BUCKETS.length).fill(0);
await idbPut('samples', []);
$('capN').textContent = '0'; $('trN').textContent = '0'; renderCoverage(); refreshGates();
});
function captureLoop(){
const W = $('capCv').width, H = $('capCv').height;
drawCameraFrame(capCtx, W, H, 0.9);
if (mpReady && !mpBusy && camEl.videoWidth > 0){
mpBusy = true; mpPose.send({ image: camEl }).catch(()=>{ mpBusy = false; });
}
if (latestKps) drawSkeleton(capCtx, latestKps, W, H, 'rgba(90,169,255,.95)', 'rgba(90,169,255,.6)');
$('capSrc').textContent = latestCSI.source || '—';
$('capSrc').className = latestCSI.source === 'esp32' ? 'v green' : 'v';
$('capNodes').textContent = latestCSI.nodes.length ? latestCSI.nodes.join(', ') : '—';
$('capVis').textContent = latestKps ? latestVis.toFixed(2) : '—';
if (recording){
// advance the guided bucket
const el = (performance.now() - bucketT0)/1000;
if (el >= SECS_PER_BUCKET){ bucketIx = (bucketIx+1) % BUCKETS.length; bucketT0 = performance.now(); }
$('prompt').textContent = BUCKETS[bucketIx];
$('countdown').textContent = `${Math.max(0,Math.ceil(SECS_PER_BUCKET - el))} s · bucket ${bucketIx+1}/${BUCKETS.length}`;
let skip = null;
if (!latestKps || latestVis <= 0.5) skip = 'no confident pose';
else if (!freshLiveCSI()) skip = (latestCSI.source && latestCSI.source!=='esp32') ? 'CSI not esp32 (sim)' : 'no fresh CSI';
else if (!baseline) skip = 'no baseline';
if (skip){ $('capSkip').textContent = skip; }
else {
const norm = baselineNorm(latestCSI.vec); // baseline-deviation normalized
SAMPLES.push({ csi: norm, kps: latestKps.slice(), bucket: bucketIx });
covCounts[bucketIx]++;
$('capSkip').textContent = '—';
const n = SAMPLES.length; $('capN').textContent = n; $('trN').textContent = n;
if (n % 20 === 0){ renderCoverage(); idbSave(); refreshGates(); }
}
} else {
$('prompt').textContent = baseline ? 'ready — press start' : 'calibrate baseline first';
$('countdown').textContent = '—';
}
requestAnimationFrame(captureLoop);
}
// ============================================================================
// IndexedDB persistence
// ============================================================================
const IDB_NAME = 'wiflow-browser', IDB_STORE = 'kv';
function idbOpen(){
return new Promise((res, rej)=>{
const r = indexedDB.open(IDB_NAME, 1);
r.onupgradeneeded = ()=> r.result.createObjectStore(IDB_STORE);
r.onsuccess = ()=> res(r.result); r.onerror = ()=> rej(r.error);
});
}
async function idbPut(key, val){
const db = await idbOpen();
return new Promise((res, rej)=>{
const tx = db.transaction(IDB_STORE, 'readwrite');
tx.objectStore(IDB_STORE).put(val, key); tx.oncomplete = res; tx.onerror = ()=> rej(tx.error);
});
}
async function idbGet(key){
const db = await idbOpen();
return new Promise((res, rej)=>{
const tx = db.transaction(IDB_STORE, 'readonly');
const r = tx.objectStore(IDB_STORE).get(key);
r.onsuccess = ()=> res(r.result); r.onerror = ()=> rej(r.error);
});
}
async function idbDel(key){
const db = await idbOpen();
return new Promise((res, rej)=>{
const tx = db.transaction(IDB_STORE, 'readwrite');
tx.objectStore(IDB_STORE).delete(key); tx.oncomplete = res; tx.onerror = ()=> rej(tx.error);
});
}
async function idbSave(){
try{
const flat = SAMPLES.map(s => ({ csi: Array.from(s.csi), kps: Array.from(s.kps), bucket: s.bucket }));
await idbPut('samples', flat);
}catch(e){ console.warn('idbSave', e); }
}
async function idbLoad(){
try{
const flat = await idbGet('samples');
if (Array.isArray(flat) && flat.length){
SAMPLES = flat.map(s => ({ csi: Float32Array.from(s.csi), kps: Float32Array.from(s.kps), bucket: s.bucket||0 }));
covCounts = new Array(BUCKETS.length).fill(0);
for (const s of SAMPLES) if (s.bucket < BUCKETS.length) covCounts[s.bucket]++;
$('capN').textContent = SAMPLES.length; $('trN').textContent = SAMPLES.length;
}
}catch(e){ console.warn('idbLoad', e); }
}
async function saveBaseline(){
try{ await idbPut('baseline', { mean: Array.from(baseline.mean), std: Array.from(baseline.std), n: baseline.n, ts: baseline.ts }); }
catch(e){ console.warn('saveBaseline', e); }
}
async function loadBaseline(){
try{
const b = await idbGet('baseline');
if (b && b.mean){
baseline = { mean: Float32Array.from(b.mean), std: Float32Array.from(b.std), n: b.n, ts: b.ts };
stageDone.calibrate = true;
$('calStatus').textContent = 'CALIBRATED (restored)'; $('calStatus').className = 'v green';
$('calN').textContent = baseline.n; $('calBar').style.width = '100%';
}
}catch(e){ /* none yet */ }
}
// ============================================================================
// STAGE 2 · TRAIN (TensorFlow.js)
// ============================================================================
let model = null, normMu = null, normSd = null, trainedPck10 = null, trainStop = false;
const lossCtx = $('lossCv').getContext('2d');
let lossHist = [];
function refreshTrainAvail(){
const ok = !!baseline && SAMPLES.length >= 200;
$('trainBtn').disabled = !ok;
if (!baseline) $('trMsg').innerHTML = 'Calibrate a baseline first (stage 0).';
else $('trMsg').innerHTML = SAMPLES.length >= 200
? `Ready: ${SAMPLES.length} samples. Click <b>train model</b>.`
: `Need ≥200 samples to train (have ${SAMPLES.length}). Capture more in stage 1.`;
}
function buildMatrices(){
const n = SAMPLES.length;
const X = new Float32Array(n * CSI_DIM), Y = new Float32Array(n * OUT_DIM);
for (let i = 0; i < n; i++){ X.set(SAMPLES[i].csi, i*CSI_DIM); Y.set(SAMPLES[i].kps, i*OUT_DIM); }
return { X, Y, n };
}
function pckMpjpe(predArr, gtArr, m, thr){
let hit = 0, tot = 0, dsum = 0;
for (let i = 0; i < m; i++) for (let j = 0; j < N_KP; j++){
const dx = predArr[i*OUT_DIM+j*2]-gtArr[i*OUT_DIM+j*2];
const dy = predArr[i*OUT_DIM+j*2+1]-gtArr[i*OUT_DIM+j*2+1];
const d = Math.hypot(dx, dy);
if (d < thr) hit++; dsum += d; tot++;
}
return { pck: tot?hit/tot:0, mpjpe: tot?dsum/tot:NaN };
}
function drawLoss(){
const W = $('lossCv').width, H = $('lossCv').height;
lossCtx.fillStyle = '#070a0e'; lossCtx.fillRect(0,0,W,H);
if (lossHist.length < 2) return;
let mx = 0; for (const p of lossHist) mx = Math.max(mx, p.tr, p.va||0); mx = mx||1;
const X = i => 8 + (W-16)*i/(lossHist.length-1);
const Yv = v => H-8 - (H-16)*Math.min(v/mx,1);
const line = (key,color)=>{
lossCtx.strokeStyle=color; lossCtx.lineWidth=2; lossCtx.beginPath(); let st=false;
lossHist.forEach((p,i)=>{ const v=p[key]; if(v==null) return;
const x=X(i), y=Yv(v); st?lossCtx.lineTo(x,y):lossCtx.moveTo(x,y); st=true; });
lossCtx.stroke();
};
line('tr','#ffb840'); line('va','#5aa9ff');
}
async function trainModel(){
if (!baseline || SAMPLES.length < 200) return;
trainStop = false;
$('trainBtn').disabled = true; $('trStop').disabled = false; lossHist = [];
const epochs = Math.max(20, Math.min(600, parseInt($('trEpochs').value)||200));
const { X, Y, n } = buildMatrices(); // X is already baseline-normalized
const cut = Math.floor(n*0.8);
$('trSplit').textContent = `${cut} / ${n-cut} (chronological 80/20)`;
// input standardization on TRAIN split only (on top of baseline-normalization)
normMu = new Float32Array(CSI_DIM); normSd = new Float32Array(CSI_DIM);
for (let j = 0; j < CSI_DIM; j++){
let s=0; for (let i=0;i<cut;i++) s += X[i*CSI_DIM+j];
const mu=s/cut; normMu[j]=mu;
let v=0; for (let i=0;i<cut;i++){ const d=X[i*CSI_DIM+j]-mu; v+=d*d; }
normSd[j]=Math.sqrt(v/cut)+EPS;
}
const Xn = new Float32Array(n*CSI_DIM);
for (let i=0;i<n;i++) for (let j=0;j<CSI_DIM;j++) Xn[i*CSI_DIM+j]=(X[i*CSI_DIM+j]-normMu[j])/normSd[j];
// mean-pose baseline — the bar to beat
const meanPose = new Float32Array(OUT_DIM);
for (let i=0;i<cut;i++) for (let j=0;j<OUT_DIM;j++) meanPose[j]+=Y[i*OUT_DIM+j];
for (let j=0;j<OUT_DIM;j++) meanPose[j]/=cut;
const mVal = n-cut;
const basePred = new Float32Array(mVal*OUT_DIM);
for (let i=0;i<mVal;i++) basePred.set(meanPose, i*OUT_DIM);
const gtVal = Y.slice(cut*OUT_DIM);
const base = pckMpjpe(basePred, gtVal, mVal, 0.10);
$('trBase').textContent = (base.pck*100).toFixed(1)+'%';
const xtr = tf.tensor2d(Xn.slice(0,cut*CSI_DIM),[cut,CSI_DIM]);
const ytr = tf.tensor2d(Y.slice(0,cut*OUT_DIM),[cut,OUT_DIM]);
const xva = tf.tensor2d(Xn.slice(cut*CSI_DIM),[mVal,CSI_DIM]);
if (model){ model.dispose(); }
model = tf.sequential();
model.add(tf.layers.dense({ inputShape:[CSI_DIM], units:512, activation:'relu' }));
model.add(tf.layers.dropout({ rate:0.3 }));
model.add(tf.layers.dense({ units:256, activation:'relu' }));
model.add(tf.layers.dropout({ rate:0.3 }));
model.add(tf.layers.dense({ units:128, activation:'relu' }));
model.add(tf.layers.dense({ units:OUT_DIM, activation:'sigmoid' }));
model.compile({ optimizer: tf.train.adam(1e-3), loss:'meanSquaredError' });
let bestP10 = 0, bestVal = 1e9;
$('trMsg').innerHTML = 'Training… on <code>'+(BACKEND_LABEL[tf.getBackend()]||tf.getBackend())+'</code>';
await model.fit(xtr, ytr, {
epochs, batchSize:64, shuffle:true, verbose:0,
callbacks:{ onEpochEnd: async (ep, logs)=>{
let va=null,p10=null,p05=null,mpj=null;
if (ep % 5 === 0 || ep === epochs-1){
const pv = model.predict(xva); const pvArr = await pv.data(); pv.dispose();
let vsum=0; for (let i=0;i<pvArr.length;i++){ const d=pvArr[i]-gtVal[i]; vsum+=d*d; }
va = vsum/pvArr.length;
const r10=pckMpjpe(pvArr,gtVal,mVal,0.10), r05=pckMpjpe(pvArr,gtVal,mVal,0.05);
p10=r10.pck; p05=r05.pck; mpj=r10.mpjpe;
$('trP10').textContent=(p10*100).toFixed(1)+'%'; $('trP05').textContent=(p05*100).toFixed(1)+'%';
$('trMpj').textContent=mpj.toFixed(4); $('trVal').textContent=va.toFixed(4);
if (va<bestVal){ bestVal=va; bestP10=p10; }
}
$('trEpoch').textContent=(ep+1); $('trLoss').textContent=logs.loss.toFixed(4);
$('trBar').style.width=(100*(ep+1)/epochs)+'%';
lossHist.push({ ep, tr:logs.loss, va }); drawLoss();
if (trainStop) model.stopTraining = true;
await tf.nextFrame();
}}
});
const pvF = model.predict(xva); const pvFArr = await pvF.data(); pvF.dispose();
const fin10 = pckMpjpe(pvFArr,gtVal,mVal,0.10), fin05 = pckMpjpe(pvFArr,gtVal,mVal,0.05);
const finPck = Math.max(bestP10, fin10.pck); trainedPck10 = finPck;
$('trP10').textContent=(fin10.pck*100).toFixed(1)+'%'; $('trP05').textContent=(fin05.pck*100).toFixed(1)+'%';
$('trMpj').textContent=fin10.mpjpe.toFixed(4); $('infPck').textContent=(finPck*100).toFixed(1)+'%';
const delta = (finPck - base.pck)*100;
const v = $('verdict');
if (delta > 1){
v.className='verdict good';
v.innerHTML = `model <b>BEATS</b> mean-pose baseline by <b>+${delta.toFixed(1)} pp</b> → real CSI→pose signal.`;
} else {
v.className='verdict bad';
v.innerHTML = `model does <b>NOT</b> beat baseline (Δ ${delta.toFixed(1)} pp) → <b>no usable signal (honest)</b>. Capture more / more varied data.`;
}
stageDone.train = true;
$('infModelState').textContent = `model ready · held-out PCK@0.10 ${(finPck*100).toFixed(1)}%`;
$('trMsg').innerHTML = 'Done. Saving model to IndexedDB…';
xtr.dispose(); ytr.dispose(); xva.dispose();
await saveModel();
$('trMsg').innerHTML = 'Saved. Go to <b>3 · INFER</b> to see WiFi drive the skeleton.';
$('trainBtn').disabled = false; $('trStop').disabled = true;
refreshGates();
}
$('trainBtn').addEventListener('click', trainModel);
$('trStop').addEventListener('click', ()=>{ trainStop = true; });
async function saveModel(){
if (!model) return;
try{
await model.save('indexeddb://wiflow-model');
await idbPut('norm', { mu:Array.from(normMu), sd:Array.from(normSd), pck10:trainedPck10 });
}catch(e){ console.warn('saveModel', e); }
}
async function loadModel(){
try{
const m = await tf.loadLayersModel('indexeddb://wiflow-model');
const norm = await idbGet('norm');
if (m && norm){
model = m; normMu = Float32Array.from(norm.mu); normSd = Float32Array.from(norm.sd);
trainedPck10 = norm.pck10; stageDone.train = true;
$('infPck').textContent = trainedPck10!=null ? (trainedPck10*100).toFixed(1)+'%' : '—';
$('infModelState').textContent = `model loaded · held-out PCK@0.10 ${trainedPck10!=null?(trainedPck10*100).toFixed(1)+'%':'?'}`;
}
}catch(e){ /* none yet */ }
}
// ============================================================================
// STAGE 3 · INFER — live CSI → baseline-normalize → standardize → model
// ============================================================================
const infCtx = $('infCv').getContext('2d');
let infSm = null, infFrames = 0, infT0 = performance.now();
function inferSmooth(kps){
if (!infSm){ infSm = Float32Array.from(kps); return infSm; }
const a = 0.35; for (let i=0;i<kps.length;i++) infSm[i]+=a*(kps[i]-infSm[i]);
return infSm;
}
function inferLoop(){
const W = $('infCv').width, H = $('infCv').height;
const showCam = !$('hideCam').checked;
if (showCam) drawCameraFrame(infCtx, W, H, 0.85);
else { infCtx.fillStyle='#070a0e'; infCtx.fillRect(0,0,W,H); }
$('infSrc').textContent = latestCSI.source || '—';
$('infSrc').className = latestCSI.source === 'esp32' ? 'v green' : 'v';
$('infNodes').textContent = latestCSI.nodes.length ? latestCSI.nodes.join(', ') : '—';
const cls = (latestCSI.frame && latestCSI.frame.classification) || {};
$('infPres').textContent = cls.presence ? 'PRESENT' : '—';
if (model && normMu && baseline && latestCSI.vec){
const out = tf.tidy(()=>{
const xn = new Float32Array(CSI_DIM);
for (let j=0;j<CSI_DIM;j++){
const bn = (latestCSI.vec[j]-baseline.mean[j])/(baseline.std[j]+EPS); // baseline-normalize
xn[j] = (bn - normMu[j])/normSd[j]; // then standardize
}
return model.predict(tf.tensor2d(xn,[1,CSI_DIM]));
});
out.data().then(arr=>{
const sm = inferSmooth(arr); const present = !!cls.presence;
drawSkeleton(infCtx, sm, W, H,
present?'rgba(70,224,138,.95)':'rgba(125,135,150,.85)','rgba(70,224,138,.6)');
out.dispose();
}).catch(()=> out.dispose());
infFrames++;
} else {
infCtx.fillStyle='#7d8796'; infCtx.font='13px monospace';
infCtx.fillText(model?'waiting for CSI…':'train a model first (stage 2)', 20, 30);
}
const now = performance.now();
if (now-infT0 > 1000){ $('infFps').textContent = infFrames; infFrames = 0; infT0 = now; }
requestAnimationFrame(inferLoop);
}
// ============================================================================
// Boot
// ============================================================================
(async function boot(){
connectCSI();
await selectBackend();
await loadBaseline();
await idbLoad();
await loadModel();
renderCoverage();
refreshGates();
requestAnimationFrame(calibrateLoop);
requestAnimationFrame(captureLoop);
requestAnimationFrame(inferLoop);
})();
</script>
</body>
</html>
+30
View File
@@ -39,7 +39,20 @@ pub const DEFAULT_SAMPLE_RATE_HZ: f64 = 10_000.0;
pub const DEFAULT_F_MOD_HZ: f64 = 1_000.0;
/// Quantise one input sample (T) to a signed ADC code. Returns `(code, saturated)`.
///
/// A **non-finite** input (`NaN` / `±Inf`) is treated as an out-of-range
/// condition: it clamps to code `0` and raises the saturation flag. This is
/// the funnel point that stops the NaN-state-poisoning class — a non-finite
/// physical field (e.g. produced by a degenerate scene with a NaN dipole
/// position) would otherwise coerce silently to code `0` *with the saturation
/// flag clear*, yielding a frame indistinguishable from a legitimate
/// zero-field reading. Flagging it preserves the "every frame is honest about
/// its own validity" contract the proof bundle relies on.
pub fn adc_quantise(b_in_t: f64) -> (i32, bool) {
if !b_in_t.is_finite() {
// Non-finite => not representable on the ±FS scale; mark saturated.
return (0, true);
}
let code_f = (b_in_t / ADC_LSB_T).round();
let max_code = (1_i32 << (ADC_BITS - 1)) - 1; // 32_767 for 16-bit signed
let min_code = -max_code; // symmetric
@@ -153,6 +166,23 @@ mod tests {
}
}
#[test]
fn adc_quantise_flags_non_finite_as_saturated() {
// Security pinning (NaN-state-poisoning guard): a non-finite field
// value must clamp to code 0 AND raise the saturation flag, so the
// pipeline can flag the frame rather than emitting it as a silent,
// indistinguishable zero-field reading. Pre-fix this returned
// (0, false) for NaN — a silent corruption.
for bad in [f64::NAN, f64::INFINITY, f64::NEG_INFINITY] {
let (code, sat) = adc_quantise(bad);
assert_eq!(code, 0, "non-finite input {bad} must clamp to code 0");
assert!(sat, "non-finite input {bad} must raise the saturation flag");
}
// A finite in-range value is unaffected (no false positives).
let (_, sat) = adc_quantise(1.0e-7);
assert!(!sat, "a finite in-range value must NOT be flagged saturated");
}
#[test]
fn adc_saturates_above_full_scale() {
let (code_pos, sat_pos) = adc_quantise(20.0e-6);
+76 -3
View File
@@ -51,11 +51,28 @@ impl Pipeline {
/// (sensor × sample) — i.e. `n_samples · scene.sensors.len()` frames
/// in scene-major / sample-minor order.
pub fn run(&self, n_samples: usize) -> Vec<MagFrame> {
let dt = self
// `dt` is derived from caller-supplied config — an external boundary
// (e.g. the WASM `config_json`). A degenerate `f_s_hz == 0` makes
// `1.0 / f_s_hz == +Inf`; a non-finite or non-positive `dt_s` is
// equally hostile. Sanitise before any arithmetic that could panic.
let raw_dt = self
.config
.dt_s
.unwrap_or(1.0 / self.config.digitiser.f_s_hz);
let dt_us = (dt * 1.0e6) as u64;
// Fall back to a 1 µs step (the smallest physically meaningful
// sample interval here) when `dt` is non-finite or non-positive, so
// the run produces well-defined frames instead of garbage / a panic.
let dt = if raw_dt.is_finite() && raw_dt > 0.0 {
raw_dt
} else {
1.0e-6
};
// `dt` is now finite & positive, so `dt * 1e6` is finite. Cap the
// `u64` cast defensively (a huge but finite `dt` could still exceed
// `u64::MAX`) and use `saturating_mul` for the per-sample timestamp so
// a pathological config can never trigger a multiply-with-overflow
// panic (debug / WASM panic=abort) or wrap to a garbage timestamp.
let dt_us = (dt * 1.0e6).min(u64::MAX as f64) as u64;
let nv = NvSensor::new(self.config.sensor);
let mut out: Vec<MagFrame> =
@@ -92,7 +109,7 @@ impl Pipeline {
];
let mut frame = MagFrame::empty(sensor_idx as u16);
frame.t_us = (sample as u64) * dt_us;
frame.t_us = (sample as u64).saturating_mul(dt_us);
frame.b_pt = b_pt;
frame.sigma_pt = sigma_pt;
frame.noise_floor_pt_sqrt_hz = (reading.noise_floor_t_sqrt_hz * 1.0e12) as f32;
@@ -205,6 +222,62 @@ mod tests {
}
}
#[test]
fn degenerate_zero_sample_rate_does_not_panic() {
// Security pinning (panic / DoS guard): an externally-supplied
// `f_s_hz == 0` makes `1/f_s_hz == +Inf`; pre-fix that produced
// `dt_us == u64::MAX`, and `sample * dt_us` panicked with
// "attempt to multiply with overflow" (debug / WASM panic=abort) at
// sample >= 2, or wrapped to a garbage timestamp in release. The
// sanitised `dt` + `saturating_mul` must keep the run finite.
let scene = fixture_scene();
let cfg = PipelineConfig {
digitiser: crate::digitiser::DigitiserConfig {
f_s_hz: 0.0,
f_mod_hz: 1000.0,
},
..PipelineConfig::default()
};
let frames = Pipeline::new(scene, cfg, 42).run(8);
assert_eq!(frames.len(), 8);
for f in &frames {
// Timestamps are monotone-well-defined, not garbage.
assert!(f.t_us < u64::MAX);
}
}
#[test]
fn non_finite_scene_input_flags_frame_instead_of_silently_zeroing() {
// Security pinning (NaN-state-poisoning guard): a NaN dipole position
// makes `r_norm` NaN, which bypasses the near-field clamp
// (`NaN < R_MIN_M` is false) and yields a NaN field. Pre-fix the
// digitiser silently coerced that NaN to code 0 with the saturation
// flag CLEAR — a frame indistinguishable from a real zero-field
// reading. Post-fix the frame must carry ADC_SATURATED so the
// corruption is visible downstream.
let mut scene = Scene::new();
scene.add_dipole(DipoleSource::new([f64::NAN, 0.0, 0.5], [0.0, 0.0, 1.0e-3]));
scene.add_sensor([0.0, 0.0, 0.0]);
let cfg = PipelineConfig {
sensor: NvSensorConfig {
shot_noise_disabled: true,
..NvSensorConfig::default()
},
..PipelineConfig::default()
};
let frames = Pipeline::new(scene, cfg, 0).run(4);
for f in &frames {
assert!(
f.has_flag(flag::ADC_SATURATED),
"non-finite field must raise ADC_SATURATED, not emit a silent zero frame"
);
// And the emitted value is a defined number, not NaN.
for b in f.b_pt {
assert!(b.is_finite());
}
}
}
#[test]
fn adc_saturation_flag_fires_above_full_scale() {
// Place a dipole close enough to drive the field above ±10 µT FS.
@@ -4,8 +4,6 @@
"windows": ["main"],
"permissions": [
"core:default",
"shell:allow-execute",
"shell:allow-open",
"dialog:allow-open",
"dialog:allow-save"
]
@@ -1 +1 @@
{"default":{"identifier":"default","description":"RuView default capability set","local":true,"windows":["main"],"permissions":["core:default","shell:allow-execute","shell:allow-open","dialog:allow-open","dialog:allow-save"]}}
{"default":{"identifier":"default","description":"RuView default capability set","local":true,"windows":["main"],"permissions":["core:default","dialog:allow-open","dialog:allow-save"]}}
@@ -430,6 +430,35 @@ fn is_esp32_compatible(vid: u16, pid: u16) -> bool {
false
}
/// Validate WiFi credentials before they are interpolated into a
/// newline-delimited serial command protocol.
///
/// The ESP32 firmware accepts line-oriented commands such as
/// `wifi_config <ssid> <password>\r\n`. Because the SSID and password
/// arrive from the webview (untrusted) and are concatenated directly into
/// those command strings, a control character (`\r`, `\n`, or NUL) embedded
/// in either field would let a malicious caller terminate the current line
/// early and inject an arbitrary follow-up command (e.g. `reboot`, `erase`).
///
/// Enforce the IEEE 802.11 / WPA2 bounds and reject any control characters:
/// - SSID: 1-32 bytes, no control characters
/// - Password: 8-63 bytes (WPA2 PSK ASCII range), no control characters
fn validate_wifi_credentials(ssid: &str, password: &str) -> Result<(), String> {
if ssid.is_empty() || ssid.len() > 32 {
return Err("SSID must be 1-32 characters".into());
}
if password.len() < 8 || password.len() > 63 {
return Err("WiFi password must be 8-63 characters".into());
}
if ssid.chars().any(|c| c.is_control()) {
return Err("SSID must not contain control characters".into());
}
if password.chars().any(|c| c.is_control()) {
return Err("WiFi password must not contain control characters".into());
}
Ok(())
}
/// Configure WiFi credentials on an ESP32 via serial port.
///
/// Sends WiFi credentials to the ESP32 using a simple serial protocol.
@@ -443,6 +472,10 @@ pub async fn configure_esp32_wifi(
use std::io::{Read, Write};
use std::time::Duration;
// Reject control characters / out-of-range lengths before the credentials
// are spliced into the line-oriented serial command protocol below.
validate_wifi_credentials(&ssid, &password)?;
tracing::info!("Configuring WiFi on port: {}", port);
// Open serial port
@@ -549,6 +582,37 @@ mod tests {
assert_eq!(node.tdm_total, Some(4));
}
#[test]
fn test_validate_wifi_credentials_accepts_valid() {
assert!(validate_wifi_credentials("MyNetwork", "password123").is_ok());
// Boundary: 32-char SSID, 63-char password are allowed.
assert!(validate_wifi_credentials(&"A".repeat(32), &"B".repeat(63)).is_ok());
// Boundary: 8-char password (WPA2 minimum) is allowed.
assert!(validate_wifi_credentials("net", "12345678").is_ok());
}
#[test]
fn test_validate_wifi_credentials_rejects_injection() {
// Newline/CR in SSID would terminate the serial command line early and
// let the caller inject a follow-up firmware command. Must be rejected.
assert!(validate_wifi_credentials("net\r\nreboot", "password123").is_err());
assert!(validate_wifi_credentials("net\ninjected", "password123").is_err());
// Same vector via the password field.
assert!(validate_wifi_credentials("net", "pass\r\nerase_nvs").is_err());
// Embedded NUL.
assert!(validate_wifi_credentials("net", "pass\0word1").is_err());
}
#[test]
fn test_validate_wifi_credentials_rejects_out_of_range() {
// Empty / over-length SSID.
assert!(validate_wifi_credentials("", "password123").is_err());
assert!(validate_wifi_credentials(&"A".repeat(33), "password123").is_err());
// Too-short / too-long password (WPA2 PSK bounds).
assert!(validate_wifi_credentials("net", "short").is_err());
assert!(validate_wifi_credentials("net", &"B".repeat(64)).is_err());
}
#[test]
fn test_is_esp32_compatible() {
// CP2102
@@ -206,6 +206,27 @@ impl OccWorldCandle {
)));
}
// Validate the externally-supplied frame and batch counts at this
// system boundary. The temporal positional embedding has only
// `num_frames * 2` rows, so a larger `f_in` would over-index the
// embedding table deep inside the transformer and surface as a cryptic
// "gather" index error; a zero frame/batch count would feed a
// zero-element tensor into the reshape/conv pipeline. Reject both here
// with a clear, domain-level error instead.
if f_in == 0 || b == 0 {
return Err(OccWorldError::ShapeMismatch(format!(
"past_occupancy must have non-zero batch and frame dims, got \
batch={b}, frames={f_in}"
)));
}
if f_in > cfg.num_frames * 2 {
return Err(OccWorldError::ShapeMismatch(format!(
"past_occupancy frame count {f_in} exceeds the temporal embedding \
capacity ({} = num_frames*2)",
cfg.num_frames * 2
)));
}
// ── Step 1: VQVAE encode each past frame ──────────────────────────
// Flatten batch*frames: (B, F, H, W, D) → (B*F, H, W, D)
let occ_flat = past_occupancy
@@ -455,4 +476,8 @@ mod tests {
"expected CheckpointNotFound, got {result:?}"
);
}
// The `predict` input-validation boundary guards (zero/over-capacity frame
// counts) live in `tests/input_validation.rs` so they exercise only the
// public API and keep this file under the 500-line limit.
}
@@ -92,8 +92,21 @@ fn safetensor_dtype_to_candle(dt: safetensors::Dtype) -> Option<candle_core::DTy
Dtype::F64 => Some(DType::F64),
Dtype::F16 => Some(DType::F16),
Dtype::BF16 => Some(DType::BF16),
Dtype::I32 => Some(DType::I64), // widen for Candle compatibility
// I32 MUST map to DType::I32, not I64. `Tensor::from_raw_buffer`
// derives its element count from `data.len() / dtype.size_in_bytes()`;
// handing an int32 byte buffer (4 bytes/elem) to the I64 path
// (8 bytes/elem) halves the element count while keeping the original
// shape, producing a tensor whose declared shape claims twice as many
// elements as its storage holds. That silent shape/storage mismatch
// panics (slice OOB) the moment the tensor is read — a crash on any
// checkpoint containing an int32 tensor. See
// `tests/checkpoint_loading.rs::int32_tensor_loads_with_consistent_shape_and_values`.
Dtype::I32 => Some(DType::I32),
Dtype::I64 => Some(DType::I64),
// I16 is also a first-class Candle dtype (2 bytes/elem); map it
// directly rather than rejecting it, for the same byte-size-correctness
// reason as I32 above.
Dtype::I16 => Some(DType::I16),
Dtype::U8 => Some(DType::U8),
Dtype::U32 => Some(DType::U32),
_ => None,
@@ -137,6 +137,17 @@ impl VQCodebook {
let orig_shape = z.shape().clone();
let orig_dims = orig_shape.dims().to_vec();
let last = *orig_shape.dims().last().unwrap_or(&0);
// Guard the divide below: a scalar (rank-0) or empty-last-dim tensor
// would make `last == 0` and panic on the `elem_count() / last`
// division. `encode` is a `pub fn` on a `pub struct`, so this is a
// reachable public boundary — fail closed with a clear error instead.
if last == 0 {
return Err(candle_core::Error::Msg(format!(
"VQCodebook::encode expects a tensor with a non-zero last dim of \
size embed_dim={}, got shape {orig_dims:?}",
self.embed_dim
)));
}
// Flatten to (N, embed_dim)
let n = z.elem_count() / last;
let z_flat = z.reshape((n, last))?; // (N, D)
@@ -339,6 +350,21 @@ mod tests {
Ok(())
}
#[test]
fn encode_rejects_scalar_without_panicking() {
// A rank-0 (scalar) tensor has an empty dims list → `last == 0`.
// Before the guard this divided by zero and panicked; now it returns
// a clean error. `encode` is public, so this is a reachable boundary.
let device = Device::Cpu;
let codebook = VQCodebook::dummy(4, 8, &device).unwrap();
let scalar = Tensor::from_vec(vec![1.0f32], (), &device).unwrap();
let result = codebook.encode(&scalar);
assert!(
result.is_err(),
"scalar input must error, not panic; got {result:?}"
);
}
#[test]
fn test_fold_unfold_roundtrip() -> candle_core::Result<()> {
let device = Device::Cpu;
@@ -0,0 +1,185 @@
//! Checkpoint-loading robustness tests for `crate::model::load_safetensors`.
//!
//! Security review (Milestone #9, crate 4/4). These tests pin the behaviour of
//! the SafeTensors weight-loading path against malformed / degenerate
//! checkpoints — the only externally-controlled file-input surface in the crate.
//!
//! The headline regression is the **int32 dtype-widening byte-size bug**
//! (`security/occworld-candle` finding #1): `model.rs` mapped
//! `safetensors::Dtype::I32` → `candle_core::DType::I64` and then handed the
//! raw *int32* byte buffer (4 bytes/elem) to `Tensor::from_raw_buffer(.., I64,
//! shape, ..)`. Candle's `from_raw_buffer` computes `elem_count =
//! data.len() / 8`, producing a tensor whose declared shape claims twice as
//! many elements as the backing storage actually holds — a silent
//! shape/storage inconsistency on attacker-supplied checkpoints.
//!
//! `build_safetensors` hand-assembles the binary container
//! (`<u64 LE header_len><JSON header><raw data>`) so the test states exactly
//! what bytes reach the loader, independent of the `safetensors` writer API.
use candle_core::Device;
use wifi_densepose_occworld_candle::model::load_safetensors;
/// Hand-build a single-tensor SafeTensors buffer.
///
/// `dtype` is the safetensors dtype string (e.g. `"I32"`, `"F32"`).
/// `shape` is the declared shape. `data` is the raw little-endian tensor bytes
/// — the caller is responsible for making `data.len()` consistent with
/// `shape × dtype_size` (safetensors itself validates this, so an inconsistent
/// pair is rejected before reaching the candle conversion).
fn build_safetensors(name: &str, dtype: &str, shape: &[usize], data: &[u8]) -> Vec<u8> {
let shape_json: Vec<String> = shape.iter().map(|d| d.to_string()).collect();
let header = format!(
"{{\"{name}\":{{\"dtype\":\"{dtype}\",\"shape\":[{}],\"data_offsets\":[0,{}]}}}}",
shape_json.join(","),
data.len()
);
let header_bytes = header.into_bytes();
let mut buf = Vec::new();
buf.extend_from_slice(&(header_bytes.len() as u64).to_le_bytes());
buf.extend_from_slice(&header_bytes);
buf.extend_from_slice(data);
buf
}
fn write_temp(bytes: &[u8], stem: &str) -> std::path::PathBuf {
let mut p = std::env::temp_dir();
p.push(format!(
"occworld_ckpt_{stem}_{}_{}.safetensors",
std::process::id(),
// nanosecond-ish disambiguator so parallel tests never collide
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.map(|d| d.as_nanos())
.unwrap_or(0)
));
std::fs::write(&p, bytes).expect("write temp checkpoint");
p
}
/// REGRESSION (finding #1): an int32 tensor in a checkpoint must load into a
/// tensor whose element count matches its declared shape.
///
/// On the OLD code (`I32 -> DType::I64`) the 6-element int32 tensor below was
/// handed to `from_raw_buffer(.., I64, [2,3], ..)`, which derived
/// `elem_count = 24 bytes / 8 = 3` and built a 3-element storage carrying a
/// shape claiming 6 elements — reading it panicked with a slice-OOB
/// (`range end index 6 out of range for slice of length 3`). On the FIXED code
/// (`I32 -> DType::I32`) the tensor round-trips: dtype I32, 6 elements,
/// values `[1,2,3,4,5,6]`.
#[test]
fn int32_tensor_loads_with_consistent_shape_and_values() {
let device = Device::Cpu;
let shape = [2usize, 3];
let vals: [i32; 6] = [1, 2, 3, 4, 5, 6];
let mut data = Vec::with_capacity(24);
for v in vals {
data.extend_from_slice(&v.to_le_bytes());
}
let bytes = build_safetensors("quantize.embedding.weight", "I32", &shape, &data);
let path = write_temp(&bytes, "i32");
let map = load_safetensors(&path, &device).expect("int32 checkpoint must load");
let t = map
.get("quantize.embedding.weight")
.expect("mapped key present");
// The declared shape's element count MUST equal the storage's element
// count. On the old code these disagreed (6 vs 3).
assert_eq!(
t.dims(),
&[2, 3],
"int32 tensor must preserve its declared shape"
);
assert_eq!(
t.elem_count(),
6,
"element count must match shape — storage/shape consistency"
);
// The dtype must be I32 — the int32 byte buffer is interpreted as int32,
// not reinterpreted as half as many int64 lanes.
assert_eq!(
t.dtype(),
candle_core::DType::I32,
"int32 checkpoint tensor must load as DType::I32"
);
// And the values must be exactly recovered (no reinterpretation of two
// int32 lanes as one int64). This is the strongest proof the dtype is
// handled correctly end-to-end.
let flat = t.flatten_all().expect("flatten");
let got: Vec<i32> = flat.to_vec1::<i32>().expect("to_vec i32");
assert_eq!(
got,
vec![1i32, 2, 3, 4, 5, 6],
"int32 values must be recovered exactly"
);
let _ = std::fs::remove_file(&path);
}
/// A well-formed F32 tensor must round-trip unchanged (control case — proves
/// the fix does not regress the common float path).
#[test]
fn f32_tensor_round_trips() {
let device = Device::Cpu;
let shape = [4usize];
let vals: [f32; 4] = [0.5, -1.0, 2.25, 3.0];
let mut data = Vec::with_capacity(16);
for v in vals {
data.extend_from_slice(&v.to_le_bytes());
}
let bytes = build_safetensors("post_quant_conv.bias", "F32", &shape, &data);
let path = write_temp(&bytes, "f32");
let map = load_safetensors(&path, &device).expect("f32 checkpoint must load");
let t = map.get("post_quant_conv.bias").expect("key present");
assert_eq!(t.dims(), &[4]);
let got: Vec<f32> = t.to_vec1::<f32>().expect("to_vec f32");
assert_eq!(got, vec![0.5, -1.0, 2.25, 3.0]);
let _ = std::fs::remove_file(&path);
}
/// A truncated / corrupt header must produce a parse error, never a panic.
/// (Defense-in-depth: the loader is fed an untrusted file.)
#[test]
fn corrupt_checkpoint_errors_cleanly() {
let device = Device::Cpu;
// Garbage that is not a valid SafeTensors container.
let bytes = vec![0xFFu8; 32];
let path = write_temp(&bytes, "corrupt");
let result = load_safetensors(&path, &device);
assert!(
result.is_err(),
"corrupt checkpoint must error, got Ok: {result:?}"
);
let _ = std::fs::remove_file(&path);
}
/// An int64 tensor must still load correctly (proves the fix narrows only the
/// I32 mapping and leaves the genuine I64 path intact).
#[test]
fn int64_tensor_round_trips() {
let device = Device::Cpu;
let shape = [3usize];
let vals: [i64; 3] = [10, -20, 30];
let mut data = Vec::with_capacity(24);
for v in vals {
data.extend_from_slice(&v.to_le_bytes());
}
let bytes = build_safetensors("transformer.output_head.bias", "I64", &shape, &data);
let path = write_temp(&bytes, "i64");
let map = load_safetensors(&path, &device).expect("i64 checkpoint must load");
let t = map.get("transformer.output_head.bias").expect("key present");
assert_eq!(t.dims(), &[3]);
assert_eq!(t.elem_count(), 3);
let got: Vec<i64> = t.to_vec1::<i64>().expect("to_vec i64");
assert_eq!(got, vec![10, -20, 30]);
let _ = std::fs::remove_file(&path);
}
@@ -0,0 +1,139 @@
//! Input-validation boundary tests for `OccWorldCandle::predict`.
//!
//! Security review (Milestone #9, crate 4/4). `predict` takes an
//! externally-supplied occupancy tensor; per the project's "validate input at
//! system boundaries" rule it must reject degenerate / out-of-capacity shapes
//! with a clear domain error rather than surfacing a cryptic deep-pipeline
//! Candle error (over-capacity frame counts over-index the temporal positional
//! embedding) or processing a zero-element tensor.
//!
//! These exercise only the public API and live here (not inline in
//! `inference.rs`) to keep that module under the 500-line cap.
use candle_core::{DType, Device, Tensor};
use wifi_densepose_occworld_candle::config::OccWorldConfig;
use wifi_densepose_occworld_candle::inference::OccWorldCandle;
use wifi_densepose_occworld_candle::error::OccWorldError;
fn small_cfg() -> OccWorldConfig {
OccWorldConfig {
grid_h: 8,
grid_w: 8,
grid_d: 4,
num_classes: 4,
free_class: 3,
base_channels: 8,
z_channels: 8,
codebook_size: 4,
embed_dim: 8,
num_frames: 2,
token_h: 4,
token_w: 4,
num_heads: 2,
num_layers: 1,
ffn_hidden: 16,
}
}
/// Zero frames is a degenerate input that would otherwise feed a zero-element
/// tensor into the reshape/conv pipeline. Must be rejected at the boundary.
#[test]
fn predict_rejects_zero_frames() {
let device = Device::Cpu;
let cfg = small_cfg();
let engine = OccWorldCandle::dummy(cfg.clone(), device.clone()).unwrap();
let past = Tensor::zeros(
(1usize, 0usize, cfg.grid_h, cfg.grid_w, cfg.grid_d),
DType::U8,
&device,
)
.unwrap();
let result = engine.predict(&past);
assert!(
matches!(result, Err(OccWorldError::ShapeMismatch(_))),
"zero-frame input must be rejected with ShapeMismatch"
);
}
/// Zero batch must also be rejected (same zero-element-tensor hazard).
#[test]
fn predict_rejects_zero_batch() {
let device = Device::Cpu;
let cfg = small_cfg();
let engine = OccWorldCandle::dummy(cfg.clone(), device.clone()).unwrap();
let past = Tensor::zeros(
(0usize, cfg.num_frames, cfg.grid_h, cfg.grid_w, cfg.grid_d),
DType::U8,
&device,
)
.unwrap();
let result = engine.predict(&past);
assert!(
matches!(result, Err(OccWorldError::ShapeMismatch(_))),
"zero-batch input must be rejected with ShapeMismatch"
);
}
/// More frames than the temporal embedding can index (`> num_frames*2`).
///
/// On the old code this over-indexed the temporal positional embedding deep in
/// the transformer and surfaced as a cryptic Candle "gather" `InvalidIndex`
/// error. The boundary guard now rejects it cleanly with `ShapeMismatch`.
#[test]
fn predict_rejects_too_many_frames() {
let device = Device::Cpu;
let cfg = small_cfg(); // num_frames = 2 → temporal capacity = 4
let engine = OccWorldCandle::dummy(cfg.clone(), device.clone()).unwrap();
let too_many = cfg.num_frames * 2 + 1;
let past = Tensor::zeros(
(1usize, too_many, cfg.grid_h, cfg.grid_w, cfg.grid_d),
DType::U8,
&device,
)
.unwrap();
let result = engine.predict(&past);
assert!(
matches!(result, Err(OccWorldError::ShapeMismatch(_))),
"over-capacity frame count must be rejected with ShapeMismatch"
);
}
/// A frame count exactly at capacity (`num_frames*2`) must still succeed —
/// the guard rejects only *over*-capacity, not the boundary value.
#[test]
fn predict_accepts_frame_count_at_capacity() {
let device = Device::Cpu;
let cfg = small_cfg();
let engine = OccWorldCandle::dummy(cfg.clone(), device.clone()).unwrap();
let at_cap = cfg.num_frames * 2;
let past = Tensor::zeros(
(1usize, at_cap, cfg.grid_h, cfg.grid_w, cfg.grid_d),
DType::U8,
&device,
)
.unwrap();
let out = engine
.predict(&past)
.expect("at-capacity frame count must predict");
assert_eq!(out.sem_pred.dims()[1], at_cap, "frame dim preserved");
}
/// Wrong spatial geometry (H/W/D) is still rejected — pins the pre-existing
/// guard alongside the new frame/batch ones.
#[test]
fn predict_rejects_wrong_grid_dims() {
let device = Device::Cpu;
let cfg = small_cfg();
let engine = OccWorldCandle::dummy(cfg.clone(), device.clone()).unwrap();
let past = Tensor::zeros(
(1usize, cfg.num_frames, cfg.grid_h + 1, cfg.grid_w, cfg.grid_d),
DType::U8,
&device,
)
.unwrap();
let result = engine.predict(&past);
assert!(
matches!(result, Err(OccWorldError::ShapeMismatch(_))),
"wrong grid dims must be rejected with ShapeMismatch"
);
}
@@ -20,6 +20,7 @@ use wifi_densepose_wasm_edge::{
host_get_phase, host_get_amplitude, host_get_variance,
host_get_presence, host_get_motion_energy,
host_emit_event, host_log,
sanitize_host_f32,
exo_ghost_hunter::GhostHunterDetector,
};
@@ -64,14 +65,16 @@ pub extern "C" fn on_frame(n_subcarriers: i32) {
for i in 0..max_sc {
unsafe {
phases[i] = host_get_phase(i as i32);
amplitudes[i] = host_get_amplitude(i as i32);
variances[i] = host_get_variance(i as i32);
// Sanitize at the boundary: a non-finite host value would otherwise
// latch NaN into the detector's persistent anomaly-energy state.
phases[i] = sanitize_host_f32(host_get_phase(i as i32));
amplitudes[i] = sanitize_host_f32(host_get_amplitude(i as i32));
variances[i] = sanitize_host_f32(host_get_variance(i as i32));
}
}
let presence = unsafe { host_get_presence() };
let motion_energy = unsafe { host_get_motion_energy() };
let motion_energy = sanitize_host_f32(unsafe { host_get_motion_energy() });
let detector = unsafe { &mut *core::ptr::addr_of_mut!(DETECTOR) };
let events = detector.process_frame(
+95 -3
View File
@@ -572,6 +572,35 @@ pub mod event_types {
pub const HEALING_COMPLETE: i32 = 888;
}
/// Sanitize a raw `f32` read from the host CSI imports.
///
/// ## NaN-state-poisoning guard (ADR-040 boundary hardening)
///
/// The `csi_get_phase`/`csi_get_amplitude`/`csi_get_variance`/… host imports
/// return raw IEEE-754 `f32`. A single non-finite value (NaN / ±∞) — from a
/// firmware DSP bug, an uninitialised buffer, or a hostile host — propagates
/// silently into the long-lived per-module accumulators (EMA, Welford,
/// phasor sums, baseline means). Once latched, every downstream comparison
/// against the poisoned state evaluates `false`, so detectors fail *degraded*
/// (stuck gate state, suppressed anomaly checks) rather than recovering.
///
/// This is the single chokepoint: every one of the ~70 edge modules receives
/// its frame data from the `on_frame` boundaries below, so mapping non-finite
/// host floats to `0.0` here protects the entire surface without per-module
/// churn. Mirrors the M-01 negative-`n_subcarriers` clamp at the same site.
///
/// `0.0` is the neutral choice: a zero phase/amplitude/variance reads as a
/// quiet subcarrier, which the detectors already handle (it cannot, itself,
/// trip an anomaly the way a poisoned NaN can permanently disable one).
#[inline]
pub fn sanitize_host_f32(v: f32) -> f32 {
if v.is_finite() {
v
} else {
0.0
}
}
/// Log a message string to the ESP32 console (via host_log import).
#[cfg(target_arch = "wasm32")]
pub fn log_msg(msg: &str) {
@@ -650,8 +679,10 @@ pub extern "C" fn on_frame(n_subcarriers: i32) {
for i in 0..max_sc {
unsafe {
phases[i] = host_get_phase(i as i32);
amps[i] = host_get_amplitude(i as i32);
// Sanitize at the boundary: a non-finite host value would otherwise
// latch NaN into the gesture/coherence/anomaly persistent state.
phases[i] = sanitize_host_f32(host_get_phase(i as i32));
amps[i] = sanitize_host_f32(host_get_amplitude(i as i32));
}
}
@@ -677,10 +708,71 @@ pub extern "C" fn on_frame(n_subcarriers: i32) {
pub extern "C" fn on_timer() {
// Periodic summary.
let state = unsafe { &*core::ptr::addr_of!(STATE) };
let motion = unsafe { host_get_motion_energy() };
let motion = sanitize_host_f32(unsafe { host_get_motion_energy() });
emit(event_types::CUSTOM_METRIC, motion);
if state.frame_count % 100 == 0 {
log_msg("wasm-edge: heartbeat");
}
}
// ── Boundary-hardening tests (ADR-040) ───────────────────────────────────────
#[cfg(test)]
mod boundary_tests {
use super::*;
#[test]
fn sanitize_passes_finite_values_through() {
assert_eq!(sanitize_host_f32(0.0), 0.0);
assert_eq!(sanitize_host_f32(-3.5), -3.5);
assert_eq!(sanitize_host_f32(1234.5), 1234.5);
assert_eq!(sanitize_host_f32(f32::MIN), f32::MIN);
assert_eq!(sanitize_host_f32(f32::MAX), f32::MAX);
}
#[test]
fn sanitize_maps_non_finite_to_zero() {
// NaN / ±∞ from a buggy or hostile host must not reach module state.
assert_eq!(sanitize_host_f32(f32::NAN), 0.0);
assert_eq!(sanitize_host_f32(f32::INFINITY), 0.0);
assert_eq!(sanitize_host_f32(f32::NEG_INFINITY), 0.0);
// A subnormal-resulting NaN (0.0 * inf) is also caught.
assert_eq!(sanitize_host_f32(0.0f32 * f32::INFINITY), 0.0);
}
/// Demonstrates the downstream hazard the boundary guard prevents:
/// feeding a raw NaN phase into a persistent module permanently latches
/// its smoothed state, whereas a boundary-sanitized 0.0 keeps it healthy.
#[test]
fn coherence_monitor_nan_latches_without_sanitize_but_not_with() {
use crate::coherence::CoherenceMonitor;
// Without sanitize: a single NaN frame poisons the EMA forever.
let mut poisoned = CoherenceMonitor::new();
poisoned.process_frame(&[0.1, 0.2, 0.3]); // init
let _ = poisoned.process_frame(&[f32::NAN, 0.2, 0.3]); // raw host NaN
// Subsequent *clean* frames can never restore a finite score.
for _ in 0..50 {
poisoned.process_frame(&[0.1, 0.2, 0.3]);
}
assert!(
poisoned.coherence_score().is_nan(),
"raw NaN should latch the smoothed coherence (documents the hazard)"
);
// With the boundary guard applied (what on_frame now does), the NaN is
// mapped to a finite value before it ever reaches the module.
let mut guarded = CoherenceMonitor::new();
let f = |x: f32| sanitize_host_f32(x);
guarded.process_frame(&[f(0.1), f(0.2), f(0.3)]); // init
let _ = guarded.process_frame(&[f(f32::NAN), f(0.2), f(0.3)]);
for _ in 0..50 {
guarded.process_frame(&[f(0.1), f(0.2), f(0.3)]);
}
assert!(
guarded.coherence_score().is_finite(),
"boundary-sanitized input keeps the module state finite"
);
}
}