Files
ruvnet--RuView/rust-port/wifi-densepose-rs/crates/wifi-densepose-pointcloud/src/parser.rs
T
rUv 0943a32248 feat: Real-time dense point cloud from camera + WiFi CSI (#405)
* Add wifi-densepose-pointcloud: real-time dense point cloud from camera + WiFi CSI

New crate with 5 modules:
- depth: monocular depth estimation + 3D backprojection (ONNX-ready, synthetic fallback)
- pointcloud: Point3D/ColorPoint types, PLY export, Gaussian splat conversion
- fusion: WiFi occupancy volume → point cloud + multi-modal voxel fusion
- stream: HTTP + Three.js viewer server (Axum, port 9880)
- main: CLI with serve/capture/demo subcommands

Demo output: 271 WiFi points + 19,200 depth points → 4,886 fused → 1,718 Gaussian splats.
Serves interactive 3D viewer at http://localhost:9880 with Three.js orbit controls.

ADR-SYS-0021 documents the architecture for camera + WiFi CSI dense point cloud pipeline.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Optimize pointcloud: larger splat voxels, smaller responses, faster fusion

- Gaussian splat voxel size: 0.10 → 0.15 (42% fewer splats: 1718 → 994)
- Splat response: 399 KB → 225 KB (44% smaller)
- Pipeline: 22.2ms mean (100 runs, σ=0.3ms)
- Cloud API: 1.11ms avg, 905 req/s
- Splats API: 1.39ms avg, 719 req/s
- Binary: 1.0 MB arm64 (Mac Mini), tested

Co-Authored-By: claude-flow <ruv@ruv.net>

* Complete implementation: camera capture, WiFi CSI receiver, training pipeline

Three new modules added to wifi-densepose-pointcloud:

1. camera.rs — Cross-platform camera capture
   - macOS: AVFoundation via Swift, ffmpeg avfoundation
   - Linux: V4L2, ffmpeg v4l2
   - Camera detection, listing, frame capture to RGB
   - Graceful fallback to synthetic data when no camera

2. csi.rs — WiFi CSI receiver for ESP32 nodes
   - UDP listener for CSI JSON frames from ESP32
   - Per-link attenuation tracking with EMA smoothing
   - Simplified RF tomography (backprojection to occupancy grid)
   - Test frame sender for development without hardware
   - Ready for real ESP32 CSI data from ruvzen

3. training.rs — Calibration and training pipeline
   - Depth calibration: grid search over scale/offset/gamma
   - Occupancy training: threshold optimization for presence detection
   - Ground truth reference points for depth RMSE measurement
   - Preference pair export (JSONL) for DPO training on ruOS brain
   - Brain integration: submit observations as memories
   - Persistent calibration files (JSON)

New CLI commands:
   ruview-pointcloud cameras         # list available cameras
   ruview-pointcloud train           # run calibration + training
   ruview-pointcloud csi-test        # send test CSI frames
   ruview-pointcloud serve --csi     # serve with live CSI input

All tested: demo, training (10 samples, 4 reference points, 3 pairs),
CSI receiver (50 test frames), server API.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Fix viewer: replace WebSocket with fetch polling

Co-Authored-By: claude-flow <ruv@ruv.net>

* Wire live camera into server — real-time updating point cloud

- Server captures from /dev/video0 at 2fps via ffmpeg
- Background tokio task refreshes cloud + splats every 500ms
- Viewer polls /api/splats every 500ms, only updates on new frame
- Shows 🟢 LIVE / 🔴 DEMO indicator
- Camera position set for first-person view (looking forward into scene)
- Downsample 4x for performance (19,200 points per frame)
- Graceful fallback to demo data if camera capture fails

Co-Authored-By: claude-flow <ruv@ruv.net>

* Add MiDaS GPU depth, serial CSI reader, full sensor fusion

- MiDaS depth server: PyTorch on CUDA, real monocular depth estimation
- Rust server calls MiDaS via HTTP for neural depth (falls back to luminance)
- Serial CSI reader for ESP32 with motion detection + presence estimation
- CSI disabled by default (RUVIEW_CSI=1 to enable) — serial reader needs baud config
- Edge-enhanced depth for better object boundaries
- All sensors wired: camera, ESP32 CSI, mmWave (CSI gated until serial fixed)

Co-Authored-By: claude-flow <ruv@ruv.net>

* Complete 7-component sensor fusion pipeline (all working)

1. ADR-018 binary parser — decodes ESP32 CSI UDP frames, extracts I/Q subcarriers
2. WiFlow pose — 17 COCO keypoints from CSI (186K param model loaded)
3. Camera depth — MiDaS on CUDA + luminance fallback
4. Sensor fusion — camera depth + CSI occupancy grid + skeleton overlay
5. RF tomography — ISTA-inspired backprojection from per-node RSSI
6. Vital signs — breathing rate from CSI phase analysis
7. Motion-adaptive — skip expensive depth when CSI shows no motion

Live results: 510 CSI frames/session, 17 keypoints, 26% motion, 40 BPM breathing.
Both ESP32 nodes provisioned to send CSI to 192.168.1.123:3333.
Magic number fix: supports both 0xC5110001 (v1) and 0xC5110006 (v6) frames.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Add brain bridge — sparse spatial observation sync every 60s

Stores room scan summaries, motion events, and vital signs
in the ruOS brain as memories. Only syncs every 120 frames
(~60 seconds) to keep the brain sparse and optimized.

Categories: spatial-observation, spatial-motion, spatial-vitals.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Update README + user guide with dense point cloud features

Added pointcloud section to README (quick start, CLI, performance).
Added comprehensive user guide section: setup, sensors, commands,
pipeline components, API endpoints, training, output formats,
deep room scan, ESP32 provisioning.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Add ruview-geo: geospatial satellite integration (11 modules, 8/8 tests)

New crate with free satellite imagery, terrain, OSM, weather, and brain integration.

Modules: types, coord, locate, cache, tiles, terrain, osm, register, fuse, brain, temporal
Tests: 8 passed (haversine, ENU roundtrip, tiles, HGT parse, registration)
Validation: real data — 43.49N 79.71W, 4 Sentinel-2 tiles, 2°C weather, brain stored

Data sources (all free, no API keys):
- EOX Sentinel-2 cloudless (10m satellite tiles)
- SRTM GL1 (30m elevation)
- Overpass API (OSM buildings/roads)
- ip-api.com (geolocation)
- Open Meteo (weather)

ADR-044 documents architecture decisions.
README.md in crate subdirectory.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Update ADR-044: add Common Crawl WET, NASA FIRMS, OpenAQ, Overture Maps sources

Extended geospatial data sources leveraging ruvector's existing web_ingest
and Common Crawl support for hyperlocal context.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Fix OSM/SRTM queries, add change detection + night mode

- OSM: use inclusive building filter with relation query and 25s timeout
- SRTM: switch to NASA public mirror with viewfinderpanoramas fallback
- Add detect_tile_changes() for pixel-diff satellite change detection
- Add is_night() solar-declination model for CSI-only night mode
- 6 new unit tests (night mode + tile change detection)

Co-Authored-By: claude-flow <ruv@ruv.net>

* Enhance viewer: skeleton overlay, weather, buildings, better camera

Add COCO skeleton rendering with yellow keypoint spheres and white bone
lines, info panel sections for weather/buildings/CSI rate/confidence,
overhead camera at (0,2,-4), and denser point size with sizeAttenuation.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Add CSI fingerprint DB + night mode detection

Co-Authored-By: claude-flow <ruv@ruv.net>

* Fix ADR-044 numbering conflict, update geo README

Renumbered provisioning tool ADR from 044 to 050 to avoid conflict
with geospatial satellite integration ADR-044.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Clean up warnings: suppress dead_code for conditional pipeline modules

Removes unused imports/variables via cargo fix and adds #[allow(dead_code)]
for modules used conditionally at runtime (CSI, depth, fusion, serial).
Pointcloud: 28 → 0 warnings. Geo: 2 → 0 warnings. 8/8 tests pass.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Fix PR #405 blockers: async runtime panic, crate rename, path traversal, brain URL config

- brain_bridge.rs: replace `Handle::current().block_on(...)` inside async fn
  with `.await` (was a guaranteed "runtime within runtime" panic). Brain URL
  now read from RUVIEW_BRAIN_URL env var (default http://127.0.0.1:9876),
  logged once via OnceLock.
- wifi-densepose-geo: rename Cargo package from `ruview-geo` to
  `wifi-densepose-geo` to match directory and workspace conventions. Update
  all use sites (tests/examples/README). Same env-var pattern for brain URL
  in brain.rs + temporal.rs.
- training.rs: add sanitize_data_path() rejecting `..` components and
  safe_join() that canonicalises + enforces base-dir containment on every
  write (calibration.json, samples.json, preference_pairs.jsonl,
  occupancy_calibration.json). Defence-in-depth check also in main.rs
  before TrainingSession::new.
- osm.rs: clamp Overpass radius to MAX_RADIUS_M=5000m; return Err beyond
  that. Add parse_overpass_json() that rejects malformed payloads
  (missing top-level `elements` array).

Co-Authored-By: claude-flow <ruv@ruv.net>

* csi_pipeline: rename WiFlow stub to heuristic_pose_from_amplitude, decouple UDP

Blocker 3 (PR #405 review): The "WiFlow inference" path was a stub that
built a model from empty weight vectors and synthesised keypoints from
amplitude energy. Presenting this as "WiFlow inference" was misleading.

- Rename WiFlowModel to PoseModelMetadata (empty tag struct; we only care
  if the on-disk file exists)
- Rename load_wiflow_model() -> detect_pose_model_metadata() and log
  "amplitude-energy heuristic enabled/disabled" (no "WiFlow" claim)
- Rename estimate_pose() -> heuristic_pose_from_amplitude() with
  prominent `STUB:` doc comment saying this is NOT a trained model

Blocker 4 (PR #405 review): The UDP receiver held the shared Arc<Mutex>
across a synchronous process_frame() call, starving HTTP handlers.

- Introduce a std::sync::mpsc channel between the UDP thread (which only
  parses + pushes) and a dedicated processor thread (which locks only
  briefly around a single process_frame). HTTP snapshots via
  get_pipeline_output no longer contend with the socket read loop.

Also:
- Move ADR-018 parser to parser.rs (see next commit); csi_pipeline re-exports
- send_test_frames now uses parser::build_test_frame for synthetic frames
- Log a one-line node stats summary every 500 frames (reads every public
  CsiFrame field on the runtime path)

Co-Authored-By: claude-flow <ruv@ruv.net>

* Extract ADR-018 parser into parser.rs + wire Fingerprint CLI

File-split (strong concern #9 in PR #405 review): csi_pipeline.rs was 602
LOC; extract the pure-function ADR-018 parser + synthetic frame builder
into src/parser.rs. Inline unit tests in parser.rs cover:

- 0xC5110001 (raw CSI, v1) roundtrip
- 0xC5110006 (feature state, v6) roundtrip
- wrong magic is rejected
- truncated header is rejected
- truncated payload is rejected

main.rs: expose `fingerprint NAME [--seconds N]` subcommand wiring
record_fingerprint() (this was the only caller needed to make the public
API non-dead on the runtime path). Also:

- Replace `--host/--port` + external `--csi` with a single `--bind`
  defaulting to loopback (`127.0.0.1:9880`) — addresses strong concern
  #7 about exposing camera/CSI/vitals by default.
- Update synthetic `csi-test` to target UDP 3333 (matching the ADR-018
  listener) and use the shared parser::build_test_frame.
- Defence-in-depth: call training::sanitize_data_path on the expanded
  --data-dir before TrainingSession::new does the same.

Co-Authored-By: claude-flow <ruv@ruv.net>

* stream: extract viewer HTML to viewer.html, default bind to loopback

Strong concern #7 (PR #405): default HTTP bind leaked camera/CSI/vitals
to the LAN. The `serve` fn now takes a single `bind` arg and prints a
loud WARNING when bound outside loopback.

Strong concern #10 (PR #405): embedded HTML+JS was ~220 LOC of the 418
LOC stream.rs. Moved the markup verbatim into viewer.html and inlined
via `include_str!("viewer.html")`. Also:

- Drop the #![allow(dead_code)] crate-level silencing (reviewer point
  #11). Remove the now-unused AppState.csi_pipeline field.
- capture_camera_cloud_with_luminance returns the mean luminance of the
  captured frame; the background loop feeds that to
  CsiPipelineState::set_light_level so the night-mode flag actually
  toggles at runtime (previously it could only be set from tests).

Net effect on file size: stream.rs 418 → 232 LOC.

Co-Authored-By: claude-flow <ruv@ruv.net>

* Dead-code cleanup + tests for fusion/depth/OSM/training/fingerprinting

Reviewer point #11 (PR #405): remove the `#![allow(dead_code)]`
silencing added in 8eb808d and fix the underlying issues.

- Delete csi.rs: duplicate of csi_pipeline.rs with incompatible wire
  format (JSON vs ADR-018 binary). csi_pipeline is the real path.
- Delete serial_csi.rs: never referenced by any module.
- Drop Frame.timestamp_ms (unread), AppState.csi_pipeline (unread),
  brain_bridge::brain_available (caller-less), fusion::fetch_wifi_occupancy
  (caller-less) — these had no runtime users.
- Drop crate-level #![allow(dead_code)] from camera.rs, depth.rs,
  fusion.rs, pointcloud.rs.

Tests (target: 8-12, actual: 15 unit + 9 geo unit + 8 geo integration
= 32 total, all pass):

- parser.rs: 5 tests (v1/v6 magic roundtrip, wrong magic, truncated
  header, truncated payload).
- fusion.rs: 2 tests (non-overlapping merge, voxel dedup).
- depth.rs: 2 tests (2x2 backproject → 4 points at z=1, NaN rejected).
- training.rs: 4 tests (rejects `..`, accepts relative child, refuses
  TrainingSession::new("../etc/passwd"), accepts a clean tmpdir).
- csi_pipeline.rs: 2 tests (set_light_level toggles is_dark,
  record_fingerprint stores and self-identifies).
- osm.rs: 3 tests (parse_overpass_json minimal fixture, rejects
  malformed payload, fetch_buildings rejects > MAX_RADIUS_M).

Co-Authored-By: claude-flow <ruv@ruv.net>

* Update README + user-guide for PR #405 review-fix additions

- serve now uses --bind 127.0.0.1:9880 (loopback default) instead of --port
- Add fingerprint subcommand to CLI tables
- Document RUVIEW_BRAIN_URL env var + --brain flag
- Flag pose path as amplitude-energy heuristic stub (not trained WiFlow)
- Security note on exposing server outside loopback
- Add wifi-densepose-pointcloud + wifi-densepose-geo rows to crate table

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-04-20 12:48:54 -04:00

164 lines
6.3 KiB
Rust

//! ADR-018 binary CSI frame parser.
//!
//! Two header magics are accepted: `0xC5110001` (raw CSI, v1) and
//! `0xC5110006` (feature state, v6). The header is 20 bytes; everything
//! after is interleaved I/Q bytes per subcarrier per antenna.
//!
//! Returns `None` when the buffer is truncated or the magic is wrong —
//! this is a hot path (one call per UDP packet) so we prefer Option over
//! a full `anyhow::Error` that would allocate.
const CSI_MAGIC_V6: u32 = 0xC511_0006;
const CSI_MAGIC_V1: u32 = 0xC511_0001;
pub(crate) const CSI_HEADER_SIZE: usize = 20;
/// Accept both header magics — `0xC5110001` (raw CSI) and
/// `0xC5110006` (feature state). Exposed for tests.
#[allow(dead_code)]
pub(crate) const MAGIC_V1: u32 = CSI_MAGIC_V1;
#[allow(dead_code)]
pub(crate) const MAGIC_V6: u32 = CSI_MAGIC_V6;
#[derive(Clone, Debug)]
pub struct CsiFrame {
pub node_id: u8,
pub n_antennas: u8,
pub n_subcarriers: u16,
pub channel: u8,
pub rssi: i8,
pub noise_floor: i8,
pub timestamp_us: u32,
/// Raw I/Q data: [I0, Q0, I1, Q1, ...] for each subcarrier
pub iq_data: Vec<i8>,
/// Computed amplitude per subcarrier: sqrt(I^2 + Q^2)
pub amplitudes: Vec<f32>,
/// Computed phase per subcarrier: atan2(Q, I)
pub phases: Vec<f32>,
}
/// Parse an ADR-018 binary CSI frame from a UDP packet.
///
/// Returns `None` if:
/// - the buffer is shorter than the 20-byte header
/// - the magic does not match either accepted value
/// - the declared I/Q payload is truncated
pub fn parse_adr018(data: &[u8]) -> Option<CsiFrame> {
if data.len() < CSI_HEADER_SIZE { return None; }
let magic = u32::from_le_bytes([data[0], data[1], data[2], data[3]]);
if magic != CSI_MAGIC_V6 && magic != CSI_MAGIC_V1 { return None; }
let node_id = data[4];
let n_antennas = data[5].max(1);
let n_subcarriers = u16::from_le_bytes([data[6], data[7]]);
let channel = data[8];
let rssi = data[9] as i8;
let noise_floor = data[10] as i8;
let timestamp_us = u32::from_le_bytes([data[16], data[17], data[18], data[19]]);
let iq_len = (n_subcarriers as usize) * 2 * (n_antennas as usize);
if data.len() < CSI_HEADER_SIZE + iq_len { return None; }
let iq_data: Vec<i8> = data[CSI_HEADER_SIZE..CSI_HEADER_SIZE + iq_len]
.iter().map(|&b| b as i8).collect();
// Compute amplitude and phase per subcarrier (first antenna).
let mut amplitudes = Vec::with_capacity(n_subcarriers as usize);
let mut phases = Vec::with_capacity(n_subcarriers as usize);
for i in 0..n_subcarriers as usize {
let idx = i * 2;
if idx + 1 < iq_data.len() {
let ii = iq_data[idx] as f32;
let qq = iq_data[idx + 1] as f32;
amplitudes.push((ii * ii + qq * qq).sqrt());
phases.push(qq.atan2(ii));
}
}
Some(CsiFrame {
node_id, n_antennas, n_subcarriers, channel, rssi, noise_floor,
timestamp_us, iq_data, amplitudes, phases,
})
}
/// Build a synthetic ADR-018 binary frame. Used by the `csi-test` CLI
/// subcommand and by the unit tests in this module.
pub fn build_test_frame(magic: u32, node_id: u8, n_subcarriers: u16, i: usize) -> Vec<u8> {
let mut buf = Vec::with_capacity(CSI_HEADER_SIZE + (n_subcarriers as usize) * 2);
buf.extend_from_slice(&magic.to_le_bytes()); // magic (0..4)
buf.push(node_id); // node_id (4)
buf.push(1u8); // n_antennas (5)
buf.extend_from_slice(&n_subcarriers.to_le_bytes()); // n_subcarriers (6..8)
buf.push(6u8); // channel (8)
buf.push((-40i8 - (i % 30) as i8) as u8); // rssi (9)
buf.push((-90i8) as u8); // noise_floor (10)
buf.extend_from_slice(&[0u8; 5]); // reserved (11..16)
buf.extend_from_slice(&(i as u32).to_le_bytes()); // timestamp_us (16..20)
for j in 0..(n_subcarriers as usize) {
buf.push(((i + j) as i8).wrapping_mul(3) as u8);
buf.push(((i + j) as i8).wrapping_mul(5) as u8);
}
buf
}
// ─── Tests ──────────────────────────────────────────────────────────────────
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parse_magic_v1_roundtrips() {
let frame_bytes = build_test_frame(MAGIC_V1, 0x42, 56, 7);
let frame = parse_adr018(&frame_bytes).expect("v1 frame should parse");
assert_eq!(frame.node_id, 0x42);
assert_eq!(frame.n_antennas, 1);
assert_eq!(frame.n_subcarriers, 56);
assert_eq!(frame.channel, 6);
assert_eq!(frame.timestamp_us, 7);
assert_eq!(frame.iq_data.len(), 56 * 2);
assert_eq!(frame.amplitudes.len(), 56);
assert_eq!(frame.phases.len(), 56);
}
#[test]
fn parse_magic_v6_roundtrips() {
let frame_bytes = build_test_frame(MAGIC_V6, 0x09, 114, 0);
let frame = parse_adr018(&frame_bytes).expect("v6 frame should parse");
assert_eq!(frame.node_id, 0x09);
assert_eq!(frame.n_antennas, 1);
assert_eq!(frame.n_subcarriers, 114);
assert_eq!(frame.channel, 6);
// With i=0, noise_floor=-90 per build_test_frame.
assert_eq!(frame.noise_floor, -90);
// With i=0, timestamp_us=0.
assert_eq!(frame.timestamp_us, 0);
assert_eq!(frame.iq_data.len(), 114 * 2);
}
#[test]
fn parse_rejects_wrong_magic() {
let mut bad = build_test_frame(MAGIC_V1, 0, 8, 0);
// Flip magic to something unrelated.
bad[0] = 0xFF;
bad[1] = 0xFF;
bad[2] = 0xFF;
bad[3] = 0xFF;
assert!(parse_adr018(&bad).is_none(), "bad magic should not parse");
}
#[test]
fn parse_rejects_truncated_header() {
let short = vec![0u8; CSI_HEADER_SIZE - 1];
assert!(parse_adr018(&short).is_none(), "truncated header must not parse");
}
#[test]
fn parse_rejects_truncated_payload() {
let mut frame = build_test_frame(MAGIC_V1, 0, 32, 0);
// Drop half the declared payload.
frame.truncate(CSI_HEADER_SIZE + 20);
assert!(parse_adr018(&frame).is_none(), "truncated payload must not parse");
}
}