frank/ruvnet--RuView
1
0
Fork 0
mirror of https://github.com/ruvnet/RuView synced 2026-06-09 10:13:17 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
7c1351fd5d56e4657b638df65f29cc48b4467182
ruvnet--RuView/ui/pose-fusion.html
T
rUv 7c1351fd5d feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion 
* feat: dual-modal WASM browser pose estimation demo (ADR-058)

Live webcam video + WiFi CSI fusion for real-time pose estimation.
Two parallel CNN pipelines (ruvector-cnn-wasm) with attention-weighted
fusion and dynamic confidence gating. Three modes: Dual, Video-only,
CSI-only. Includes pre-built WASM package (~52KB) for browser deployment.

- ADR-058: Dual-modal architecture design
- ui/pose-fusion.html: Main demo page with dark theme UI
- 7 JS modules: video-capture, csi-simulator, cnn-embedder, fusion-engine,
  pose-decoder, canvas-renderer, main orchestrator
- Pre-built ruvector-cnn-wasm WASM package for browser
- CSI heatmap, embedding space visualization, latency metrics
- WebSocket support for live ESP32 CSI data
- Navigation link added to main dashboard

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: motion-responsive skeleton + through-wall CSI tracking

- Pose decoder now uses per-cell motion grid to track actual arm/head
  positions — raising arms moves the skeleton's arms, head follows
  lateral movement
- Motion grid (10x8 cells) tracks intensity per body zone: head,
  left/right arm upper/mid, legs
- Through-wall mode: when person exits frame, CSI maintains presence
  with slow decay (~10s) and skeleton drifts in exit direction
- CSI simulator persists sensing after video loss, ghost pose renders
  with decreasing confidence
- Reduced temporal smoothing (0.45) for faster response to movement

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: video fills available space + correct WASM path resolution

- Remove fixed aspect-ratio and max-height from video panel so it
  fills the available viewport space without scrolling
- Grid uses 1fr row for content area, overflow:hidden on main grid
- Fix WASM path: resolve relative to JS module file using import.meta.url
  instead of hardcoded ./pkg/ which resolved incorrectly on gh-pages
- Responsive: mobile still gets aspect-ratio constraint

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: live ESP32 CSI pipeline + auto-connect WebSocket

- Add auto-connect to local sensing server WebSocket (ws://localhost:8765)
- Demo shows "Live ESP32" when connected to real CSI data
- Add build_firmware.ps1 for native Windows ESP-IDF builds (no Docker)
- Add read_serial.ps1 for ESP32 serial monitor

Pipeline: ESP32 → UDP:5005 → sensing-server → WS:8765 → browser demo

Co-Authored-By: claude-flow <ruv@ruv.net>

* docs: add ADR-059 live ESP32 CSI pipeline + update README with demo links

- ADR-059: Documents end-to-end ESP32 → sensing server → browser pipeline
- README: Add dual-modal pose fusion demo link, update ADR count to 49
- References issue #245

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: RSSI visualization, RuVector attention WASM, cache-bust fixes

- Add animated RSSI Signal Strength panel with sparkline history
- Fix RuVector WasmMultiHeadAttention retptr calling convention
- Wire up RuVector Multi-Head + Flash Attention in CNN embedder
- Add ambient temporal drift to CSI simulator for visible heatmap animation
- Fix embedding space projection (sparse projection replaces cancelling sum)
- Add auto-scaling to embedding space renderer
- Add cache busters (?v=4) to all ES module imports to prevent stale caches
- Add diagnostic logging for module version verification
- Add RSSI tracking with quality labels and color-coded dBm display
- Includes ruvector-attention-wasm v2.0.5 browser ESM wrapper

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat: 26-keypoint dexterous pose + full RuVector attention pipeline

Pose Decoder (17 → 26 keypoints):
- Add finger approximations: thumb, index, pinky per hand (6 new)
- Add toe tips: left/right foot index (2 new)
- Add neck keypoint (1 new)
- Hand openness driven by arm motion intensity
- Finger positions computed from wrist-elbow axis angles

CNN Embedder (full RuVector WASM pipeline):
- Stage 1: Multi-Head Attention (global spatial reasoning)
- Stage 2: Hyperbolic Attention (hierarchical body-part tree)
- Stage 3: MoE Attention (3 experts: upper/lower/extremities, top-2)
- Blended 40/30/30 weighting → final embedding projection

Canvas Renderer:
- Magenta finger joints with distinct glow
- Cyan toe tips
- White neck keypoint
- Thinner limb lines for hand/foot connections
- Joint count shown in overlay label

CSI Simulator:
- Skip synthetic person state when live ESP32 connected
- Only simulate CSI data in demo mode (was already correct)

Embedding Space:
- Fixed projection: sparse 8-dim projection replaces cancelling sum
- Auto-scaling normalizes point spread to fill canvas

Cache busters bumped to v=5 on all imports.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: centroid-based pose tracking for responsive limb movement

Rewrites pose decoder from intensity-based to position-based tracking:
- Arms now track toward motion centroid in each body zone
- Elbow/wrist positions computed along shoulder→centroid vector
- Legs track toward lower-body zone centroids
- Smoothing reduced from 0.45 to 0.25 for responsiveness
- Zone centroids blend 30% old / 70% new each frame

6 body zones with overlapping coverage:
- Head (top 20%, center cols)
- Left/Right Arm (rows 10-60%, outer cols)
- Torso (rows 15-55%, center cols)
- Left/Right Leg (rows 50-100%, half cols each)

Hand openness now driven by arm spread distance + raise amount.
Cache busters v=6.

Co-Authored-By: claude-flow <ruv@ruv.net>

* fix: remove duplicate lAnkleX/rAnkleX declarations in pose-decoder

Stale code block from old intensity-based tracking was left behind,
re-declaring variables already defined by centroid-based tracking.

Co-Authored-By: claude-flow <ruv@ruv.net>

* feat(demo): wire all 6 RuVector WASM attention mechanisms into pose fusion

- Add WasmLinearAttention and WasmLocalGlobalAttention to browser ESM wrapper
- Add 6 WASM utility functions (batch_normalize, pairwise_distances, etc.)
- Extend CnnEmbedder to 6-stage pipeline: Flash → MHA → Hyperbolic → Linear → MoE → L+G
- Use log-energy softmax blending across all 6 stages
- Wire WASM cosine_similarity and normalize into FusionEngine
- Add RuVector pipeline stats panel to UI (energy, refinement, pose impact)
- Compute embedding-to-joint mapping stats without modifying joint positions
- Center camera prompt with flexbox layout
- Add cache busters v=12

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-12 20:59:57 -04:00

202 lines
7.8 KiB
HTML
Raw Blame History

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>WiFi-DensePose — Dual-Modal Pose Estimation</title>
<link rel="stylesheet" href="pose-fusion/css/style.css?v=12">
</head>
<body>
<!-- Header -->
<header class="header">
<div class="header-left">
<div class="logo"><span class="pi">&pi;</span> DensePose</div>
<div class="header-title">Dual-Modal Pose Estimation — Live Video + WiFi CSI Fusion</div>
</div>
<div class="header-right">
<select id="mode-select" class="mode-select">
<option value="dual">Dual Mode (Video + CSI)</option>
<option value="video">Video Only</option>
<option value="csi">CSI Only (WiFi)</option>
</select>
<div class="status-badge">
<span id="status-dot" class="status-dot offline"></span>
<span id="status-label">READY</span>
</div>
<span id="fps-display" class="fps-badge">-- FPS</span>
<a href="index.html" class="back-link">&larr; Dashboard</a>
<a href="observatory.html" class="back-link">Observatory &rarr;</a>
</div>
</header>
<!-- Main Grid -->
<div class="main-grid">
<!-- Video + Skeleton Panel -->
<div class="video-panel">
<video id="webcam" autoplay playsinline muted></video>
<canvas id="skeleton-canvas"></canvas>
<div class="video-overlay-label" id="mode-label">DUAL FUSION</div>
<div id="camera-prompt" class="camera-prompt">
<div class="camera-prompt-label" id="prompt-mode-label">DUAL FUSION</div>
<p>Enable your webcam for live video pose estimation.<br>
Or switch to <strong>CSI Only</strong> mode for WiFi-based sensing.</p>
<button id="start-camera-btn">Enable Camera</button>
</div>
</div>
<!-- Side Panels -->
<div class="side-panels">
<!-- Fusion Confidence -->
<div class="panel">
<div class="panel-title">&#9670; Fusion Confidence</div>
<div class="fusion-bars">
<div class="bar-row">
<span class="bar-label">Video</span>
<div class="bar-track"><div class="bar-fill video" id="video-bar" style="width:0%"></div></div>
<span class="bar-value" id="video-bar-val">0%</span>
</div>
<div class="bar-row">
<span class="bar-label">CSI</span>
<div class="bar-track"><div class="bar-fill csi" id="csi-bar" style="width:0%"></div></div>
<span class="bar-value" id="csi-bar-val">0%</span>
</div>
<div class="bar-row">
<span class="bar-label">Fused</span>
<div class="bar-track"><div class="bar-fill fused" id="fused-bar" style="width:0%"></div></div>
<span class="bar-value" id="fused-bar-val">0%</span>
</div>
</div>
<div style="margin-top:8px; font-size:10px; color:var(--text-label)">
Cross-modal: <span id="cross-modal-sim" style="color:var(--green-glow)">0.000</span>
</div>
</div>
<!-- CSI Heatmap -->
<div class="panel">
<div class="panel-title">&#9670; CSI Amplitude Heatmap</div>
<div class="csi-canvas-wrapper">
<canvas id="csi-canvas" width="320" height="100"></canvas>
</div>
</div>
<!-- RSSI Signal Strength -->
<div class="panel">
<div class="panel-title">&#9670; RSSI Signal Strength</div>
<div class="rssi-row">
<div class="rssi-gauge">
<div class="rssi-bar-track">
<div class="rssi-bar-fill" id="rssi-bar" style="width:0%"></div>
</div>
<div class="rssi-values">
<span class="rssi-dbm" id="rssi-value">-- dBm</span>
<span class="rssi-quality" id="rssi-quality">--</span>
</div>
</div>
<canvas id="rssi-sparkline" width="160" height="32"></canvas>
</div>
</div>
<!-- Embedding Space -->
<div class="panel">
<div class="panel-title">&#9670; Embedding Space (2D Projection)</div>
<div class="embedding-canvas-wrapper">
<canvas id="embedding-canvas" width="320" height="100"></canvas>
</div>
</div>
<!-- RuVector Attention Pipeline -->
<div class="panel">
<div class="panel-title">&#9670; RuVector WASM Attention Pipeline</div>
<div class="rv-pipeline">
<div class="rv-stage" id="rv-flash">Flash</div>
<div class="rv-arrow">&rarr;</div>
<div class="rv-stage" id="rv-mha">MHA</div>
<div class="rv-arrow">&rarr;</div>
<div class="rv-stage" id="rv-hyp">Hyper</div>
<div class="rv-arrow">&rarr;</div>
<div class="rv-stage" id="rv-lin">Linear</div>
<div class="rv-arrow">&rarr;</div>
<div class="rv-stage" id="rv-moe">MoE</div>
<div class="rv-arrow">&rarr;</div>
<div class="rv-stage" id="rv-lg">L+G</div>
</div>
<div class="rv-stats">
<span>Energy: <span id="rv-energy" style="color:var(--green-glow)">--</span></span>
<span>Refinement: <span id="rv-refine" style="color:var(--cyan)">--</span></span>
<span>Pose Impact: <span id="rv-impact" style="color:var(--amber)">--</span></span>
</div>
</div>
<!-- Latency -->
<div class="panel">
<div class="panel-title">&#9670; Pipeline Latency</div>
<div class="latency-grid">
<div class="latency-item">
<div class="latency-value" id="lat-video">--</div>
<div class="latency-label">Video CNN</div>
</div>
<div class="latency-item">
<div class="latency-value" id="lat-csi">--</div>
<div class="latency-label">CSI CNN</div>
</div>
<div class="latency-item">
<div class="latency-value" id="lat-fusion">--</div>
<div class="latency-label">Fusion</div>
</div>
<div class="latency-item">
<div class="latency-value" id="lat-total">--</div>
<div class="latency-label">Total</div>
</div>
</div>
</div>
<!-- Controls -->
<div class="panel">
<div class="panel-title">&#9670; Controls</div>
<div class="controls-row">
<button class="btn" id="pause-btn">⏸ Pause</button>
</div>
<div class="slider-row">
<label>Confidence</label>
<input type="range" id="confidence-slider" min="0" max="1" step="0.05" value="0.3">
<span class="slider-val" id="confidence-value">0.30</span>
</div>
<div style="margin-top:10px">
<div class="panel-title" style="margin-bottom:6px">&#9670; Live CSI Source</div>
<div style="display:flex;gap:6px">
<input type="text" id="ws-url" placeholder="ws://localhost:3030/ws/csi"
style="flex:1;background:rgba(255,255,255,0.05);border:1px solid var(--bg-panel-border);
color:var(--text-primary);padding:5px 8px;border-radius:4px;font-size:11px;
font-family:'JetBrains Mono',monospace">
<button class="btn" id="connect-ws-btn">Connect</button>
</div>
</div>
</div>
</div><!-- /side-panels -->
<!-- Bottom Bar -->
<div class="bottom-bar">
<div>
WiFi-DensePose &middot; Dual-Modal Pose Estimation &middot;
Architecture: Conv2D &rarr; RuVector 6-Stage Attention (Flash+MHA+Hyperbolic+Linear+MoE+L/G) &rarr; Fusion &rarr; 26-Keypoint Pose
</div>
<div>
<a href="https://github.com/ruvnet/wifi-densepose">GitHub</a> &middot;
CNN: <span id="cnn-backend">ruvector-cnn (loading…)</span> &middot;
<a href="observatory.html">Observatory</a>
</div>
</div>
</div><!-- /main-grid -->
<script type="module" src="pose-fusion/js/main.js?v=12"></script>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink