docs: ADR-288/289/290 — benchmark harness, wideband CSI ingest, vitals ground-truth rig

Three ADRs closing the highest-impact gaps from the 2026 SOTA research
sweep: public-benchmark comparability (Widar3.0 ingest + leakage-guarded
split protocols), wideband 802.11ax CSI ingest (FeitCSI/AX210), and a
vitals ground-truth rig (reference ingest, alignment, Bland-Altman
agreement, evidence grading). Implementations follow in this branch.

Co-Authored-By: claude-flow <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_015TcKegTS7QqhWPC2L2SzaS
This commit is contained in:
Claude
2026-08-10 23:18:54 +00:00
parent 5780c239e4
commit de88e37de5
4 changed files with 293 additions and 0 deletions
@@ -0,0 +1,106 @@
# ADR-288: Public-benchmark evaluation harness — Widar3.0 ingest, standard split protocols, leakage guards
- **Status**: Accepted — initial implementation (this PR)
- **Date**: 2026-08-10
- **Deciders**: ruv
- **Tags**: training, evaluation, benchmarks, widar, mm-fi, leakage, honesty
## Context
RuView implements the field's key techniques (CSI ratio, BVP features, MAE
pretraining, rapid adaptation) but reports results only on self-collected data
with self-defined metrics (e.g. the README's held-out temporal-triplet
accuracy). A 2026 deep-research sweep of the WiFi-sensing literature found:
1. Cross-domain generalization is the field's central unsolved problem; the
only widely reproduced cross-domain result is Widar3.0's BVP benchmark.
2. MM-Fi (NeurIPS 2023) is the standard WiFi-pose benchmark, with defined
cross-subject and cross-environment protocols.
3. The field had a documented leakage reckoning in 20242025: window-level
random splits on continuous recordings inflate accuracy (one dataset's F1
collapsed from ~90% to ~22% under subject-disjoint splits — Sensors
24(10):3159; Signals 6(4):59).
`wifi-densepose-train` already has an `MmFiDataset` NPY loader and a
deterministic `SyntheticCsiDataset`, but no Widar3.0 ingest, no standard split
protocols, and no structural leakage guard. CLAUDE.md already requires
mean-pose baselines and leakage-free held-out splits for pose PCK; nothing in
the code enforces this.
Without leaderboard-comparable numbers, RuView's claims cannot be ranked
against published systems, which blocks both scientific credibility and
commercial (OEM licensing) conversations.
## Options considered
1. **Do nothing; keep self-collected metrics.** Rejected: perpetuates the
comparability gap.
2. **Port a Python eval stack (SenseFi) alongside the Rust pipeline.**
Rejected: violates the v2 Rust-workspace direction and adds an unreviewed
dependency surface.
3. **Extend `wifi-densepose-train` with native loaders + protocol machinery.**
Chosen.
## Decision
Extend `v2/crates/wifi-densepose-train` with three additions:
### 1. Widar3.0 ingest (`dataset::widar`)
- A parser for the Intel 5300 `.dat` CSI log format ("bfee" records) used by
the Widar3.0 raw distribution: framed records with a 3-byte header
(2-byte little-endian length + 1-byte code 0xBB), a 20-byte bfee header
(timestamp_low, bfee_count, Nrx, Ntx, RSSI a/b/c, noise, agc, antenna_sel,
len, rate), and a packed 10-bit-per-component complex CSI payload of
30 subcarrier groups. Invalid records are skipped with a warning, not a
panic — untrusted file input is validated at the boundary per CLAUDE.md.
- A `WidarDataset` implementing the existing `CsiDataset` trait, mapping
Widar's `Nrx × Ntx × 30` CSI into windowed `CsiSample`s via the existing
subcarrier interpolation, with domain metadata (user, room, orientation,
gesture) parsed from Widar's documented directory/file naming convention.
- No network access: the loader reads a local dataset root. Dataset download
remains a documented manual step.
### 2. Split protocols (`protocols`)
- A `SplitProtocol` type expressing the standard evaluations: cross-subject
(MM-Fi style), cross-environment/room, cross-orientation (Widar style), and
random-baseline (explicitly labelled as leakage-prone, for comparison only).
- Split assignment is a pure function of sample metadata + a seed — fully
deterministic, no RNG state.
### 3. Leakage guards (`protocols::leakage`)
- A structural `LeakageAudit` that, given a proposed train/test split,
verifies: (a) subject-disjointness, (b) environment-disjointness where the
protocol claims it, (c) no two windows from the same continuous recording
span both sides of the split. A failed audit is an `Err`, not a warning.
- PCK/accuracy reporting requires a `MeanPoseBaseline` computed from the
training split only, and reports model-vs-baseline together, enforcing the
CLAUDE.md rule in the type system rather than by convention.
- Evaluation output is an evidence-tagged report (`MEASURED` requires a
reproducer command line embedded in the report; anything else is emitted as
`SYNTHETIC` or `CLAIMED`).
## Consequences
- RuView results become comparable to published numbers (Widar3.0 cross-domain
gesture; MM-Fi cross-subject pose) for the first time.
- The leakage audit will make some existing internal numbers look worse. That
is the point.
- Parsing a legacy binary format adds maintenance surface; mitigated by
fixture-based tests with synthetic, deterministically generated `.dat`
bytes (no dataset redistribution).
- Widar's raw distribution is Intel 5300-specific; ESP32-captured data
continues through existing loaders. The protocols/leakage machinery is
loader-agnostic.
## Validation
- `cargo test -p wifi-densepose-train` — unit tests for the bfee parser
(truncated, corrupt, and valid synthetic fixtures), split determinism,
leakage-audit rejection cases, and mean-pose baseline math.
- `cargo bench -p wifi-densepose-train` — criterion benchmark for parser
throughput and split assignment on synthetic corpora.
- No accuracy numbers are claimed by this ADR; it delivers the machinery to
produce MEASURED ones.
@@ -0,0 +1,91 @@
# ADR-289: Wideband 802.11ax CSI ingest — FeitCSI/AX210 adapter and subcarrier-agnostic plumbing
- **Status**: Accepted — initial implementation (this PR)
- **Date**: 2026-08-10
- **Deciders**: ruv
- **Tags**: hardware, csi, 80211ax, ax210, feitcsi, ingest, mat
## Context
RuView's CSI ingest (`wifi-densepose-mat/src/integration/hardware_adapter.rs`)
supports ESP32 serial streams, the legacy Intel 5300 tool, and Atheros/Nexmon
paths. All of these are 802.11n-class: ≤40 MHz bandwidth, ≤114 subcarriers,
2.4/5 GHz.
The 2026 research sweep found the field's center of gravity has moved to
Intel AX200/AX210 NICs via PicoScenes (closed-source core) and FeitCSI
(open-source, GPL): 802.11ax CSI at up to 160 MHz / 1992 subcarriers,
including the 6 GHz band. This is both the research-grade tier today and the
shape of the data 802.11bf silicon will deliver from ~2026 onward. RuView's
`wifi-densepose-hardware` crate already models 802.11bf session types, but no
ingest path can carry wideband CSI into the pipeline.
Without a wideband path, RuView cannot develop against the best available
signal, cannot compare ESP32-grade results to wideband upper bounds, and will
meet 802.11bf silicon with no tested plumbing for >114-subcarrier frames.
## Options considered
1. **PicoScenes `.csi` ingest.** Rejected for now: the format is produced by a
closed-source core and is versioned/complex; parsing it without a
maintained spec invites silent corruption.
2. **Raw pcap + radiotap parsing.** Rejected: duplicates what FeitCSI already
does on-device, and pulls a packet-capture dependency into the pipeline.
3. **FeitCSI file/stream ingest.** Chosen: FeitCSI is open-source (its header
layout is auditable against the source), targets AX200/AX210, covers
20160 MHz including 6 GHz, and emits a compact binary record per frame.
## Decision
Extend `v2/crates/wifi-densepose-mat/src/integration` with:
### 1. `feitcsi` record parser
- A validated parser for FeitCSI's binary CSI record layout (header with
CSI buffer length, rate/bandwidth/channel metadata, antenna counts, RSSI,
timestamp, followed by interleaved complex CSI). The parser is written
against the documented layout, is version-checked, and rejects
records whose declared dimensions disagree with the buffer length —
untrusted file/stream input is validated at the boundary.
- Bounded allocation: a hard cap on subcarrier count (4096) and antenna
count (8) so a corrupt length field cannot cause unbounded allocation.
### 2. `DeviceType::FeitCsi` in the hardware adapter
- File-replay mode (read a recorded FeitCSI capture deterministically) and a
streaming mode fed by an external process writing to a path/pipe. No
privileged operations inside the crate: RuView does not configure the NIC;
FeitCSI's own tooling owns that, per least-authority.
### 3. Subcarrier-agnostic plumbing
- Ingest carries native subcarrier dimensionality end-to-end and converts to
pipeline width explicitly via the existing interpolation/decimation stage,
recording the native → pipeline mapping in frame metadata so downstream
consumers know the true spectral resolution. Bandwidth (20160 MHz) and
band (2.4/5/6 GHz) become first-class frame metadata.
## Consequences
- RuView gains a research-grade wideband development path and a tested
ingest shape for future 802.11bf reporting (truncated CIR is a natural
extension of the same plumbing).
- GPL FeitCSI is used as an external tool, never linked: only its output
format is parsed. No licensing contamination of the MIT workspace.
- The parser tracks an external project's format; version checks fail loudly
on mismatch rather than misparse.
- ESP32 remains the deployed sensor tier; wideband is a development/
validation tier. Accuracy claims from wideband captures must be tagged with
the capture hardware.
## Validation
- `cargo test -p wifi-densepose-mat` — parser tests over synthetic fixtures:
valid records at 20/80/160 MHz shapes, truncated buffer, dimension
mismatch, version mismatch, allocation-cap enforcement; adapter replay
determinism.
- `cargo bench -p wifi-densepose-mat` — criterion benchmark for record parse
throughput at 1992-subcarrier frames.
- Hardware validation on real AX210 silicon is explicitly out of scope for
this PR and remains required (per CLAUDE.md) before any capture-path
hardware claim; the file-replay path is testable without silicon.
@@ -0,0 +1,93 @@
# ADR-290: Vitals ground-truth rig — reference ingest, time alignment, and agreement metrics
- **Status**: Accepted — initial implementation (this PR)
- **Date**: 2026-08-10
- **Deciders**: ruv
- **Tags**: vitals, validation, ground-truth, bland-altman, evidence, honesty
## Context
`wifi-densepose-vitals` (ADR-021) extracts breathing (0.10.5 Hz) and heart
rate (0.82.0 Hz) from CSI. The 2026 research sweep found that every credible
vitals result in the literature ships with reference-sensor ground truth
(chest strap, pulse oximeter, ECG, or PSG), and that WiFi heart-rate numbers
without stated scope (single person, static, line-of-sight, short range) are
systematically misleading. RuView currently has no way to produce a MEASURED
vitals number: there is no reference-signal ingest, no time alignment between
CSI-derived estimates and a reference device, and no agreement statistics.
CLAUDE.md requires accuracy statements to be tagged MEASURED (with a
reproducer), CLAIMED, or SYNTHETIC. For vitals, MEASURED is currently
unreachable.
## Options considered
1. **Live BLE/ANT+ integration with reference devices.** Rejected for now:
drivers and pairing are a hardware/product concern; the blocking gap is
the evaluation math, not the radio link.
2. **File-based reference ingest + offline agreement analysis.** Chosen:
every consumer reference device (Polar, Garmin, oximeters) exports
timestamped series; a file boundary keeps the crate dependency-free and
the pipeline deterministic.
## Decision
Add a `groundtruth` module to `v2/crates/wifi-densepose-vitals`:
### 1. Reference series ingest
- `ReferenceSeries`: timestamped samples (unix millis + value) for one
measurand (`HeartRateBpm` or `BreathingRateBrpm`), with device metadata
(make/model, measurement principle). Parsed from CSV (`timestamp_ms,value`
with a header line); malformed rows are rejected with row-numbered errors —
untrusted file input validated at the boundary. Non-monotonic timestamps
are an error, not silently sorted.
### 2. Time alignment
- Constant-offset estimation by maximizing normalized cross-correlation of
the estimate series against the reference over a bounded lag window
(default ±30 s), on a common resampled grid (nearest-sample, no
interpolation of physiological values across gaps larger than a
configurable limit).
- Optional linear clock-drift fit (offset + rate) for long sessions.
Alignment parameters are reported, never silently applied.
### 3. Agreement metrics
- `AgreementReport`: n paired samples, coverage fraction (time where both
series had valid samples), MAE, RMSE, mean error (bias), BlandAltman
95% limits of agreement, and percentage-within-tolerance (configurable,
default ±2 bpm HR / ±1 brpm breathing).
- Session scope is mandatory metadata: subject count, motion state
(static/moving), line-of-sight (LOS/NLOS/through-wall), distance band.
A report without scope cannot be constructed.
### 4. Evidence tagging
- `EvidenceGrade::Measured` is only constructible when the report carries a
reference device, non-zero paired samples, minimum coverage, and a
reproducer command string; otherwise the report grades as `Claimed` (real
data, no reference) or `Synthetic` (generated input). This mirrors
ADR-288's enforcement-in-types approach and the CLAUDE.md tagging rule.
## Consequences
- RuView can convert vitals claims from CLAIMED to MEASURED with a
reproducible offline analysis, session by session, scope by scope.
- Honest reporting will likely show heart-rate performance below marketing
intuition, especially NLOS/moving — that is the purpose.
- CSV ingest means a manual export step per session; acceptable at current
scale, and the format is the de-facto export of consumer reference gear.
- No clinical claim is implied: agreement statistics against consumer
reference devices are engineering evidence, not medical validation.
## Validation
- `cargo test -p wifi-densepose-vitals` — CSV rejection cases, alignment
recovery of known synthetic offsets/drifts, agreement metrics against
hand-computed fixtures, evidence-grade constructibility rules.
- `cargo bench -p wifi-densepose-vitals` — criterion benchmark for alignment
over hour-scale synthetic sessions.
- Real-session validation (ESP32 capture + chest strap) remains a follow-up
requiring hardware evidence per CLAUDE.md.
+3
View File
@@ -145,6 +145,9 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
| [ADR-287](ADR-287-coherent-wideband-rf-tomography-crate.md) | `wifi-densepose-sar` — coherent wideband RF tomography research crate | Accepted (implemented, published) | | [ADR-287](ADR-287-coherent-wideband-rf-tomography-crate.md) | `wifi-densepose-sar` — coherent wideband RF tomography research crate | Accepted (implemented, published) |
| [ADR-285](ADR-285-homecore-wasm-first-metaharness.md) | WASM-first Homecore developer metaharness via `npx homecore` | Accepted (implemented and validated) | | [ADR-285](ADR-285-homecore-wasm-first-metaharness.md) | WASM-first Homecore developer metaharness via `npx homecore` | Accepted (implemented and validated) |
| [ADR-286](ADR-286-wifi-densepose-sar-harness-via-metaharness.md) | `wifi-densepose-sar-harness` — MetaHarness with darwin/router/flywheel | Accepted (implemented, published) | | [ADR-286](ADR-286-wifi-densepose-sar-harness-via-metaharness.md) | `wifi-densepose-sar-harness` — MetaHarness with darwin/router/flywheel | Accepted (implemented, published) |
| [ADR-288](ADR-288-public-benchmark-evaluation-harness.md) | Public-benchmark evaluation harness — Widar3.0 ingest, split protocols, leakage guards | Accepted (initial implementation) |
| [ADR-289](ADR-289-wideband-80211ax-csi-ingest.md) | Wideband 802.11ax CSI ingest — FeitCSI/AX210 adapter, subcarrier-agnostic plumbing | Accepted (initial implementation) |
| [ADR-290](ADR-290-vitals-ground-truth-rig.md) | Vitals ground-truth rig — reference ingest, alignment, agreement metrics | Accepted (initial implementation) |
--- ---