mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
rUv
7c1351fd5d
* feat: dual-modal WASM browser pose estimation demo (ADR-058) Live webcam video + WiFi CSI fusion for real-time pose estimation. Two parallel CNN pipelines (ruvector-cnn-wasm) with attention-weighted fusion and dynamic confidence gating. Three modes: Dual, Video-only, CSI-only. Includes pre-built WASM package (~52KB) for browser deployment. - ADR-058: Dual-modal architecture design - ui/pose-fusion.html: Main demo page with dark theme UI - 7 JS modules: video-capture, csi-simulator, cnn-embedder, fusion-engine, pose-decoder, canvas-renderer, main orchestrator - Pre-built ruvector-cnn-wasm WASM package for browser - CSI heatmap, embedding space visualization, latency metrics - WebSocket support for live ESP32 CSI data - Navigation link added to main dashboard Co-Authored-By: claude-flow <ruv@ruv.net> * fix: motion-responsive skeleton + through-wall CSI tracking - Pose decoder now uses per-cell motion grid to track actual arm/head positions — raising arms moves the skeleton's arms, head follows lateral movement - Motion grid (10x8 cells) tracks intensity per body zone: head, left/right arm upper/mid, legs - Through-wall mode: when person exits frame, CSI maintains presence with slow decay (~10s) and skeleton drifts in exit direction - CSI simulator persists sensing after video loss, ghost pose renders with decreasing confidence - Reduced temporal smoothing (0.45) for faster response to movement Co-Authored-By: claude-flow <ruv@ruv.net> * fix: video fills available space + correct WASM path resolution - Remove fixed aspect-ratio and max-height from video panel so it fills the available viewport space without scrolling - Grid uses 1fr row for content area, overflow:hidden on main grid - Fix WASM path: resolve relative to JS module file using import.meta.url instead of hardcoded ./pkg/ which resolved incorrectly on gh-pages - Responsive: mobile still gets aspect-ratio constraint Co-Authored-By: claude-flow <ruv@ruv.net> * feat: live ESP32 CSI pipeline + auto-connect WebSocket - Add auto-connect to local sensing server WebSocket (ws://localhost:8765) - Demo shows "Live ESP32" when connected to real CSI data - Add build_firmware.ps1 for native Windows ESP-IDF builds (no Docker) - Add read_serial.ps1 for ESP32 serial monitor Pipeline: ESP32 → UDP:5005 → sensing-server → WS:8765 → browser demo Co-Authored-By: claude-flow <ruv@ruv.net> * docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links - ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline - README: Add dual-modal pose fusion demo link, update ADR count to 49 - References issue #245 Co-Authored-By: claude-flow <ruv@ruv.net> * feat: RSSI visualization, RuVector attention WASM, cache-bust fixes - Add animated RSSI Signal Strength panel with sparkline history - Fix RuVector WasmMultiHeadAttention retptr calling convention - Wire up RuVector Multi-Head + Flash Attention in CNN embedder - Add ambient temporal drift to CSI simulator for visible heatmap animation - Fix embedding space projection (sparse projection replaces cancelling sum) - Add auto-scaling to embedding space renderer - Add cache busters (?v=4) to all ES module imports to prevent stale caches - Add diagnostic logging for module version verification - Add RSSI tracking with quality labels and color-coded dBm display - Includes ruvector-attention-wasm v2.0.5 browser ESM wrapper Co-Authored-By: claude-flow <ruv@ruv.net> * feat: 26-keypoint dexterous pose + full RuVector attention pipeline Pose Decoder (17 → 26 keypoints): - Add finger approximations: thumb, index, pinky per hand (6 new) - Add toe tips: left/right foot index (2 new) - Add neck keypoint (1 new) - Hand openness driven by arm motion intensity - Finger positions computed from wrist-elbow axis angles CNN Embedder (full RuVector WASM pipeline): - Stage 1: Multi-Head Attention (global spatial reasoning) - Stage 2: Hyperbolic Attention (hierarchical body-part tree) - Stage 3: MoE Attention (3 experts: upper/lower/extremities, top-2) - Blended 40/30/30 weighting → final embedding projection Canvas Renderer: - Magenta finger joints with distinct glow - Cyan toe tips - White neck keypoint - Thinner limb lines for hand/foot connections - Joint count shown in overlay label CSI Simulator: - Skip synthetic person state when live ESP32 connected - Only simulate CSI data in demo mode (was already correct) Embedding Space: - Fixed projection: sparse 8-dim projection replaces cancelling sum - Auto-scaling normalizes point spread to fill canvas Cache busters bumped to v=5 on all imports. Co-Authored-By: claude-flow <ruv@ruv.net> * fix: centroid-based pose tracking for responsive limb movement Rewrites pose decoder from intensity-based to position-based tracking: - Arms now track toward motion centroid in each body zone - Elbow/wrist positions computed along shoulder→centroid vector - Legs track toward lower-body zone centroids - Smoothing reduced from 0.45 to 0.25 for responsiveness - Zone centroids blend 30% old / 70% new each frame 6 body zones with overlapping coverage: - Head (top 20%, center cols) - Left/Right Arm (rows 10-60%, outer cols) - Torso (rows 15-55%, center cols) - Left/Right Leg (rows 50-100%, half cols each) Hand openness now driven by arm spread distance + raise amount. Cache busters v=6. Co-Authored-By: claude-flow <ruv@ruv.net> * fix: remove duplicate lAnkleX/rAnkleX declarations in pose-decoder Stale code block from old intensity-based tracking was left behind, re-declaring variables already defined by centroid-based tracking. Co-Authored-By: claude-flow <ruv@ruv.net> * feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion - Add WasmLinearAttention and WasmLocalGlobalAttention to browser ESM wrapper - Add 6 WASM utility functions (batch_normalize, pairwise_distances, etc.) - Extend CnnEmbedder to 6-stage pipeline: Flash → MHA → Hyperbolic → Linear → MoE → L+G - Use log-energy softmax blending across all 6 stages - Wire WASM cosine_similarity and normalize into FusionEngine - Add RuVector pipeline stats panel to UI (energy, refinement, pose impact) - Compute embedding-to-joint mapping stats without modifying joint positions - Center camera prompt with flexbox layout - Add cache busters v=12 Co-Authored-By: claude-flow <ruv@ruv.net>
Architecture Decision Records
This folder contains 44 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project.
Why ADRs?
Building a system that turns WiFi signals into human pose estimation involves hundreds of non-obvious decisions: which signal processing algorithms to use, how to bridge ESP32 firmware to a Rust pipeline, whether to run inference on-device or on a server, how to handle multi-person separation with limited subcarriers.
ADRs capture the context, options considered, decision made, and consequences for each of these choices. They serve three purposes:
-
Institutional memory — Six months from now, anyone (human or AI) can read why we chose IIR bandpass filters over FIR for vital sign extraction, not just see the code.
-
AI-assisted development — When an AI agent works on this codebase, ADRs give it the constraints and rationale it needs to make changes that align with the existing architecture. Without them, AI-generated code tends to drift — reinventing patterns that already exist, contradicting earlier decisions, or optimizing for the wrong tradeoffs.
-
Review checkpoints — Each ADR is a reviewable artifact. When a proposed change touches the architecture, the ADR forces the author to articulate tradeoffs before writing code, not after.
ADRs and Domain-Driven Design
The project uses Domain-Driven Design (DDD) to organize code into bounded contexts — each with its own language, types, and responsibilities. ADRs and DDD work together:
- ADRs define boundaries: ADR-029 (RuvSense) established multistatic sensing as a separate bounded context from single-node CSI. ADR-042 (CHCI) defined a new aggregate root for coherent channel imaging.
- DDD models define the language: The RuvSense domain model defines terms like "coherence gate", "dwell time", and "TDM slot" that ADRs reference precisely.
- Together they prevent drift: An AI agent reading ADR-039 knows that edge processing tiers are configured via NVS keys, not compile-time flags — because the ADR says so. The DDD model tells it which aggregate owns that configuration.
How ADRs are structured
Each ADR follows a consistent format:
- Context — What problem or gap prompted this decision
- Decision — What we chose to do and how
- Consequences — What improved, what got harder, and what risks remain
- References — Related ADRs, papers, and code paths
Statuses: Proposed (under discussion), Accepted (approved and/or implemented), Superseded (replaced by a later ADR).
ADR Index
Hardware and firmware
| ADR | Title | Status |
|---|---|---|
| ADR-012 | ESP32 CSI Sensor Mesh for Distributed Sensing | Accepted (partial) |
| ADR-018 | ESP32 Development Implementation Path | Proposed |
| ADR-028 | ESP32 Capability Audit and Witness Record | Accepted |
| ADR-029 | RuvSense Multistatic Sensing Mode (TDM, channel hopping) | Proposed |
| ADR-032 | Multistatic Mesh Security Hardening | Accepted |
| ADR-039 | ESP32-S3 Edge Intelligence Pipeline (on-device vitals) | Accepted (hardware-validated) |
| ADR-040 | WASM Programmable Sensing (Tier 3) | Accepted |
| ADR-041 | WASM Module Collection (65 edge modules) | Accepted (hardware-validated) |
| ADR-044 | Provisioning Tool Enhancements | Proposed |
Signal processing and sensing
| ADR | Title | Status |
|---|---|---|
| ADR-013 | Feature-Level Sensing on Commodity Gear | Accepted |
| ADR-014 | SOTA Signal Processing Algorithms | Accepted |
| ADR-021 | Vital Sign Detection (breathing, heart rate) | Partial |
| ADR-030 | Persistent Field Model and Drift Detection | Proposed |
| ADR-033 | CRV Signal Line Sensing Integration | Proposed |
| ADR-037 | Multi-Person Pose Detection from Single ESP32 | Proposed |
| ADR-042 | Coherent Human Channel Imaging (beyond CSI) | Proposed |
Machine learning and training
| ADR | Title | Status |
|---|---|---|
| ADR-005 | SONA Self-Learning for Pose Estimation | Partial |
| ADR-006 | GNN-Enhanced CSI Pattern Recognition | Partial |
| ADR-015 | Public Dataset Strategy (MM-Fi, Wi-Pose) | Accepted |
| ADR-016 | RuVector Training Pipeline Integration | Accepted |
| ADR-017 | RuVector Signal + MAT Integration | Proposed |
| ADR-020 | Migrate AI Inference to Rust (ONNX Runtime) | Accepted |
| ADR-023 | Trained DensePose Model with RuVector Pipeline | Proposed |
| ADR-024 | Project AETHER: Contrastive CSI Embeddings | Required |
| ADR-027 | Project MERIDIAN: Cross-Environment Generalization | Proposed |
Platform and UI
| ADR | Title | Status |
|---|---|---|
| ADR-019 | Sensing-Only UI with Gaussian Splats | Accepted |
| ADR-022 | Windows WiFi Enhanced Fidelity (multi-BSSID) | Partial |
| ADR-025 | macOS CoreWLAN WiFi Sensing | Proposed |
| ADR-031 | RuView Sensing-First RF Mode | Proposed |
| ADR-034 | Expo React Native Mobile App | Accepted |
| ADR-035 | Live Sensing UI Accuracy and Data Transparency | Accepted |
| ADR-036 | Training Pipeline UI Integration | Proposed |
| ADR-043 | Sensing Server UI API Completion (14 endpoints) | Accepted |
Architecture and infrastructure
| ADR | Title | Status |
|---|---|---|
| ADR-001 | WiFi-Mat Disaster Detection Architecture | Accepted |
| ADR-002 | RuVector RVF Integration Strategy | Superseded |
| ADR-003 | RVF Cognitive Containers for CSI | Proposed |
| ADR-004 | HNSW Vector Search for Fingerprinting | Partial |
| ADR-007 | Post-Quantum Cryptography for Sensing | Proposed |
| ADR-008 | Distributed Consensus for Multi-AP | Proposed |
| ADR-009 | RVF WASM Runtime for Edge Deployment | Proposed |
| ADR-010 | Witness Chains for Audit Trail Integrity | Proposed |
| ADR-011 | Proof-of-Reality and Mock Elimination | Proposed |
| ADR-026 | Survivor Track Lifecycle (MAT crate) | Accepted |
| ADR-038 | Sublinear GOAP for Roadmap Optimization | Proposed |
Related
- DDD Domain Models — Bounded context definitions, aggregate roots, and ubiquitous language
- User Guide — Setup, API reference, and hardware instructions
- Build Guide — Building from source