mirror of
https://github.com/ruvnet/RuView
synced 2026-07-28 18:21:42 +00:00
docs: update README with ADR-045–048, Observatory, adaptive classifier, AMOLED display
- Update ADR count from 44 to 48 - Add adaptive classifier (ADR-048) to Intelligence features - Add Observatory visualization (ADR-047) and AMOLED display (ADR-045) to Deployment features - Update screenshot to v2-screen.png - Add ADR-045 (AMOLED), ADR-046 (Android TV), ADR-047 (Observatory), DDD deployment model - Add AMOLED display firmware (display_hal, display_task, display_ui, LVGL config) - Add Observatory UI (13 Three.js modules, CSS, HTML entry point) - Add trained adaptive model JSON - Update .gitignore for managed_components, recordings, .swarm Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
@@ -0,0 +1,110 @@
|
||||
# ADR-045: AMOLED Display Support for ESP32-S3 CSI Node
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
|
||||
The ESP32-S3 board (LilyGO T-Display-S3 AMOLED) has an integrated RM67162 QSPI AMOLED display (536x240) and 8MB octal PSRAM that were unused by the CSI firmware. Users want real-time on-device visualization of CSI statistics, vital signs, and system health without relying on an external server.
|
||||
|
||||
### Constraints
|
||||
|
||||
- Binary was 947 KB in a 1 MB partition — needed 8MB flash + custom partition table
|
||||
- SPIRAM was disabled in sdkconfig despite hardware having 8MB PSRAM
|
||||
- Core 1 is pinned to DSP (edge processing) — display must use Core 0
|
||||
- Existing CSI pipeline must not be affected
|
||||
|
||||
### Available APIs
|
||||
|
||||
Thread-safe edge APIs already exist (`edge_get_vitals()`, `edge_get_multi_person()`) — the display task only reads from these, no new synchronization needed.
|
||||
|
||||
## Decision
|
||||
|
||||
Add optional AMOLED display support with the following architecture:
|
||||
|
||||
### Hardware Abstraction Layer
|
||||
|
||||
- `display_hal.c/h`: RM67162 QSPI panel driver + CST816S capacitive touch via I2C
|
||||
- Auto-detect at boot: probe RM67162 and check SPIRAM; log warning and skip if absent
|
||||
|
||||
### UI Layer
|
||||
|
||||
- `display_ui.c/h`: LVGL 8.3 with 4 swipeable views via tileview widget
|
||||
- Dark theme (#0a0a0f) with cyan (#00d4ff) accent for three.js-like aesthetic
|
||||
- Views: Dashboard (CSI amplitude chart + stats), Vitals (breathing + HR line graphs), Presence (4x4 occupancy grid), System (CPU, heap, PSRAM, WiFi, uptime, FPS)
|
||||
|
||||
### Task Layer
|
||||
|
||||
- `display_task.c/h`: FreeRTOS task on Core 0, priority 1 (lowest)
|
||||
- LVGL pump loop at configurable FPS (default 30)
|
||||
- Double-buffered draw buffers allocated in SPIRAM
|
||||
|
||||
### Compile-Time Control
|
||||
|
||||
- `CONFIG_DISPLAY_ENABLE=y` (default): compiles display code, auto-detects hardware at boot
|
||||
- `CONFIG_DISPLAY_ENABLE=n`: zero-cost — no display code compiled
|
||||
- `CONFIG_SPIRAM_IGNORE_NOTFOUND=y`: boots fine on boards without PSRAM
|
||||
|
||||
### Flash Layout
|
||||
|
||||
8MB partition table (`partitions_display.csv`):
|
||||
- Dual OTA partitions: 2 x 2MB (supports larger binaries with LVGL)
|
||||
- SPIFFS: 1.9MB (for future font/asset storage)
|
||||
- NVS + otadata + phy: standard sizes
|
||||
|
||||
### Core/Task Layout
|
||||
|
||||
| Task | Core | Priority | Impact |
|
||||
|------|------|----------|--------|
|
||||
| WiFi/LwIP | 0 | 18-23 | unchanged |
|
||||
| OTA httpd | 0 | 5 | unchanged |
|
||||
| **display_task** | **0** | **1** | **NEW — lowest priority** |
|
||||
| edge_task (DSP) | 1 | 5 | unchanged |
|
||||
|
||||
### Dependencies
|
||||
|
||||
- LVGL ~8.3 (via ESP-IDF managed components)
|
||||
- espressif/esp_lcd_touch_cst816s ^1.0
|
||||
- espressif/esp_lcd_touch ^1.0
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Real-time on-device stats without network dependency
|
||||
- Zero impact on CSI pipeline (display reads thread-safe APIs, runs at lowest priority)
|
||||
- Graceful degradation: works on boards without display or PSRAM
|
||||
- SPIRAM enabled for all boards (benefits WASM runtime too)
|
||||
- 8MB flash + dual OTA 2MB partitions give headroom for future features
|
||||
|
||||
### Negative
|
||||
|
||||
- Binary size increase (~200-300 KB with LVGL)
|
||||
- SPIRAM + 8MB flash config is specific to T-Display-S3 AMOLED boards
|
||||
- Boards with only 4MB flash need `CONFIG_DISPLAY_ENABLE=n` and the old partition table
|
||||
|
||||
### Risks
|
||||
|
||||
- RM67162 init sequence is board-specific; other AMOLED panels may need different commands
|
||||
- QSPI bus conflicts if other peripherals use SPI2_HOST (currently unused)
|
||||
|
||||
## New Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `main/display_hal.c/h` | RM67162 QSPI + CST816S touch HAL |
|
||||
| `main/display_ui.c/h` | LVGL 4-view UI |
|
||||
| `main/display_task.c/h` | FreeRTOS task, LVGL pump |
|
||||
| `main/lv_conf.h` | LVGL compile config |
|
||||
| `partitions_display.csv` | 8MB partition table |
|
||||
| `idf_component.yml` | Managed component deps |
|
||||
|
||||
## Modified Files
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `sdkconfig.defaults` | 8MB flash, SPIRAM, custom partitions |
|
||||
| `main/CMakeLists.txt` | Conditional display sources + deps |
|
||||
| `main/main.c` | +1 include, +5 lines guarded init |
|
||||
| `main/Kconfig.projbuild` | "AMOLED Display" menu |
|
||||
@@ -0,0 +1,263 @@
|
||||
# ADR-046: Android TV Box / Armbian Deployment Target
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
|
||||
## Context
|
||||
|
||||
Issue [#138](https://github.com/ruvnet/wifi-densepose/issues/138) requests ESP8266 and mobile device support. The ESP8266 lacks CSI capability and sufficient resources, but the discussion revealed a compelling deployment target: **Android TV boxes** (Amlogic/Allwinner/Rockchip SoCs) running **Armbian** (Debian for ARM).
|
||||
|
||||
These devices cost $15–35, are always-on mains-powered, include 802.11ac WiFi, 2–4 GB RAM, quad-core ARM Cortex-A53/A55 CPUs, and HDMI output. They are widely available as consumer "IPTV boxes" (T95, H96 Max, X96, MXQ Pro, etc.) and can boot Armbian from SD card without modifying the factory Android installation.
|
||||
|
||||
### Current deployment model
|
||||
|
||||
```
|
||||
[ESP32-S3 nodes] --UDP CSI--> [Laptop/PC running sensing-server] --browser--> [UI]
|
||||
```
|
||||
|
||||
This requires a general-purpose computer ($300+) to run the Rust sensing server, NN inference, and web dashboard. For permanent installations (elder care, smart home, security), dedicating a laptop is impractical.
|
||||
|
||||
### Proposed deployment model
|
||||
|
||||
```
|
||||
[ESP32-S3 nodes] --UDP CSI--> [TV Box running Armbian + sensing-server] --HDMI--> [Display]
|
||||
$25, always-on, fanless
|
||||
```
|
||||
|
||||
### Future: custom WiFi firmware for standalone operation
|
||||
|
||||
Many TV box WiFi chipsets (Realtek RTL8822CS, MediaTek MT7661, Broadcom BCM43455) can potentially be patched for CSI extraction when running under Linux with custom drivers. This would eliminate the ESP32 dependency entirely for basic sensing:
|
||||
|
||||
```
|
||||
[TV Box with patched WiFi driver] --CSI extraction--> [sensing-server on same box] --HDMI--> [Display]
|
||||
$25 total, single device
|
||||
```
|
||||
|
||||
This ADR covers Phase 1 (TV box as aggregator) and Phase 2 (custom WiFi firmware for CSI). Phase 2 is speculative and requires per-chipset R&D.
|
||||
|
||||
## Decision
|
||||
|
||||
### Phase 1: TV Box as Aggregator (Armbian)
|
||||
|
||||
1. **Cross-compile the sensing server** for `aarch64-unknown-linux-gnu` using `cross` or Docker-based cross-compilation.
|
||||
|
||||
2. **Create an Armbian deployment package** containing:
|
||||
- Pre-built `wifi-densepose-sensing-server` binary (aarch64)
|
||||
- systemd service file for auto-start on boot
|
||||
- Kiosk-mode Chromium configuration for HDMI dashboard display
|
||||
- Network configuration for ESP32 UDP reception (port 5005)
|
||||
- Optional: `hostapd` config to create a dedicated WiFi AP for the ESP32 mesh
|
||||
|
||||
3. **Define minimum hardware requirements:**
|
||||
|
||||
| Component | Minimum | Recommended |
|
||||
|-----------|---------|-------------|
|
||||
| SoC | Amlogic S905W (A53 quad) | Amlogic S905X3 (A55 quad) |
|
||||
| RAM | 2 GB | 4 GB |
|
||||
| Storage | 8 GB eMMC + 8 GB SD | 16 GB eMMC + 16 GB SD |
|
||||
| WiFi | 802.11n 2.4 GHz | 802.11ac dual-band |
|
||||
| Ethernet | 100 Mbps | Gigabit |
|
||||
| USB | 1x USB 2.0 | 2x USB 3.0 |
|
||||
| HDMI | 1.4 | 2.0 |
|
||||
|
||||
4. **Tested reference devices** (initial target list):
|
||||
|
||||
| Device | SoC | WiFi Chip | Price | Armbian Support |
|
||||
|--------|-----|-----------|-------|-----------------|
|
||||
| T95 Max+ | S905X3 | RTL8822CS | ~$30 | Good (meson-sm1) |
|
||||
| H96 Max X3 | S905X3 | RTL8822CS | ~$35 | Good (meson-sm1) |
|
||||
| X96 Max+ | S905X3 | RTL8822CS | ~$28 | Good (meson-sm1) |
|
||||
| Tanix TX6S | H616 | MT7668 | ~$25 | Moderate (sun50i-h616) |
|
||||
|
||||
5. **New Rust compilation target** in workspace CI:
|
||||
- Add `aarch64-unknown-linux-gnu` to cross-compilation matrix
|
||||
- Binary size target: <15 MB stripped (fits easily in SD card)
|
||||
- No GPU dependency — CPU-only inference using `candle` or ONNX Runtime for ARM
|
||||
|
||||
### Phase 2: Custom WiFi Firmware for CSI Extraction (Future)
|
||||
|
||||
1. **CSI extraction feasibility by chipset:**
|
||||
|
||||
| Chipset | Driver | CSI Support | Monitor Mode | Effort |
|
||||
|---------|--------|-------------|--------------|--------|
|
||||
| Broadcom BCM43455 | brcmfmac | **Proven** (Nexmon CSI) | Yes | Low — patches exist |
|
||||
| Realtek RTL8822CS | rtw88 | **Moderate** — driver is open-source, CSI hooks need adding | Yes (patched) | Medium |
|
||||
| MediaTek MT7661 | mt76 | **Unknown** — MediaTek has released CSI tools for some chips | Yes | Medium-High |
|
||||
|
||||
2. **CSI extraction architecture** (Linux kernel driver modification):
|
||||
|
||||
```
|
||||
[WiFi chipset firmware] → [Modified kernel driver] → [Netlink/procfs CSI export]
|
||||
↓
|
||||
[userspace CSI reader]
|
||||
↓
|
||||
[sensing-server UDP input]
|
||||
```
|
||||
|
||||
The CSI data would be reformatted into the existing ESP32 binary protocol (ADR-018 header, magic `0xC5100001`) so the sensing server treats it identically to ESP32 frames. This means zero changes to the ingestion context.
|
||||
|
||||
3. **Hybrid mode**: When the TV box has both patched WiFi CSI and ESP32 UDP input, the sensing server's multi-node architecture (already supporting multiple `node_id` values) handles both sources transparently. The TV box's own WiFi becomes an additional viewpoint in the multistatic array.
|
||||
|
||||
### Phase 3: Android Companion App (Optional)
|
||||
|
||||
For users who want mobile monitoring without Armbian:
|
||||
|
||||
1. **PWA (Progressive Web App)**: The sensing server already serves a web UI. Adding a PWA manifest with offline caching makes it installable on any Android device. No native app needed.
|
||||
|
||||
2. **Native Android app** (future): Only if PWA proves insufficient. Would use Kotlin + Jetpack Compose, consuming the existing REST API and WebSocket endpoints.
|
||||
|
||||
## Deployment Architecture
|
||||
|
||||
### Single-Room Deployment (Phase 1)
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────┐
|
||||
│ Room │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ ESP32-S3 │ │ ESP32-S3 │ │ ESP32-S3 │ CSI sensor mesh │
|
||||
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ ($10 each) │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ │ │ │ │
|
||||
│ └──────────────┼──────────────┘ │
|
||||
│ │ UDP port 5005 │
|
||||
│ ▼ │
|
||||
│ ┌──────────────────────────────────────┐ │
|
||||
│ │ Android TV Box (Armbian) │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────┐ │ │
|
||||
│ │ │ wifi-densepose-sensing- │ │ │
|
||||
│ │ │ server (aarch64 binary) │ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ │ • CSI ingestion (UDP) │ │ │
|
||||
│ │ │ • Feature extraction │ │ │
|
||||
│ │ │ • NN inference (CPU) │ │ │
|
||||
│ │ │ • WebSocket streaming │ │ │
|
||||
│ │ │ • REST API │ │ │
|
||||
│ │ │ • Web UI (:3000) │ │ │
|
||||
│ │ └──────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────┐ │ │
|
||||
│ │ │ Chromium Kiosk Mode │───│──→ HDMI out │
|
||||
│ │ │ (localhost:3000) │ │ to display │
|
||||
│ │ └──────────────────────────────┘ │ │
|
||||
│ │ │ │
|
||||
│ │ Cost: $25-35 │ │
|
||||
│ │ Power: 5-10W (USB-C or barrel) │ │
|
||||
│ │ Form: fits behind TV/monitor │ │
|
||||
│ └──────────────────────────────────────┘ │
|
||||
│ │
|
||||
└──────────────────────────────────────────────────────────────┘
|
||||
|
||||
Total system cost: $55-65 (3 ESP32 nodes + 1 TV box)
|
||||
```
|
||||
|
||||
### Multi-Room Deployment
|
||||
|
||||
```
|
||||
┌──────────────┐
|
||||
│ Router │
|
||||
│ (WiFi AP) │
|
||||
└──────┬───────┘
|
||||
│ LAN
|
||||
┌──────────────┼──────────────┐
|
||||
│ │ │
|
||||
┌───────▼───────┐ ┌───▼────────┐ ┌──▼──────────┐
|
||||
│ Room A │ │ Room B │ │ Room C │
|
||||
│ TV Box + │ │ TV Box + │ │ TV Box + │
|
||||
│ 3x ESP32 │ │ 3x ESP32 │ │ 3x ESP32 │
|
||||
│ HDMI display │ │ HDMI │ │ HDMI │
|
||||
└───────────────┘ └────────────┘ └─────────────┘
|
||||
|
||||
Each room: self-contained sensing + display
|
||||
Central dashboard: aggregate all rooms via REST API
|
||||
```
|
||||
|
||||
### Standalone Mode (Phase 2 — Custom WiFi FW)
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
│ Android TV Box (Armbian) │
|
||||
│ │
|
||||
│ ┌────────────────────┐ │
|
||||
│ │ Patched WiFi │ │
|
||||
│ │ Driver │ │
|
||||
│ │ (CSI extraction) │ │
|
||||
│ └─────────┬──────────┘ │
|
||||
│ │ CSI frames │
|
||||
│ ▼ │
|
||||
│ ┌────────────────────┐ │
|
||||
│ │ sensing-server │──→ HDMI out │
|
||||
│ │ (inference + │ │
|
||||
│ │ dashboard) │ │
|
||||
│ └────────────────────┘ │
|
||||
│ │
|
||||
│ Single device: $25 │
|
||||
│ No ESP32 nodes needed │
|
||||
└──────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- **10x cost reduction** for aggregator: $25 TV box vs $300+ laptop/PC
|
||||
- **Always-on deployment**: Mains-powered, fanless, designed for 24/7 operation
|
||||
- **HDMI output**: Direct connection to TV/monitor for wall-mounted dashboards
|
||||
- **Familiar hardware**: Available globally, no specialized ordering required
|
||||
- **Armbian ecosystem**: Mature Debian-based distro with package management, systemd, SSH
|
||||
- **Path to standalone**: Custom WiFi firmware could eliminate ESP32 dependency entirely
|
||||
- **PWA for mobile**: No native app development needed for mobile monitoring
|
||||
- **Multi-room scaling**: One TV box per room, each self-contained
|
||||
|
||||
### Negative
|
||||
|
||||
- **ARM cross-compilation**: Adds CI complexity; `candle`/ONNX Runtime ARM builds need testing
|
||||
- **Armbian compatibility**: Not all TV boxes are well-supported; need a tested device list
|
||||
- **Performance uncertainty**: ARM A53 cores are ~3-5x slower than x86 for NN inference; may need model quantization (INT8) for real-time operation
|
||||
- **Phase 2 risk**: Custom WiFi firmware is chipset-specific, may require kernel patches per driver version, and CSI quality varies by chipset
|
||||
- **Support burden**: Different hardware = more configurations to support
|
||||
- **No GPU**: TV boxes lack discrete GPU; inference is CPU-only (but our models are small enough)
|
||||
|
||||
### Neutral
|
||||
|
||||
- **No changes to existing ESP32 firmware** — TV box receives the same UDP frames
|
||||
- **No changes to sensing server protocol** — Phase 2 CSI output uses same binary format
|
||||
- **Existing web UI works as-is** — Chromium kiosk mode or any browser on the LAN
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1 (2-3 weeks)
|
||||
|
||||
1. Add `aarch64-unknown-linux-gnu` cross-compilation target using `cross`
|
||||
2. Build and test sensing-server binary on reference TV box (T95 Max+ / S905X3)
|
||||
3. Create systemd service + Armbian deployment script
|
||||
4. Benchmark: measure inference latency, memory usage, thermal throttling
|
||||
5. Create `docs/deployment/armbian-tv-box.md` setup guide
|
||||
6. Add HDMI kiosk mode configuration (Chromium autostart)
|
||||
|
||||
### Phase 2 (4-8 weeks, R&D)
|
||||
|
||||
1. Acquire TV box with BCM43455 (proven Nexmon CSI support)
|
||||
2. Build Armbian with Nexmon CSI patches for BCM43455
|
||||
3. Write userspace CSI reader → ESP32 binary protocol converter
|
||||
4. Test CSI quality comparison: ESP32 vs BCM43455
|
||||
5. If viable: add RTL8822CS CSI extraction via rtw88 driver modification
|
||||
|
||||
### Phase 3 (1 week)
|
||||
|
||||
1. Add PWA manifest to sensing server web UI
|
||||
2. Test on Android Chrome, iOS Safari
|
||||
3. Add service worker for offline dashboard caching
|
||||
|
||||
## References
|
||||
|
||||
- [Nexmon CSI](https://github.com/seemoo-lab/nexmon_csi) — Broadcom WiFi CSI extraction (BCM43455, BCM4339, BCM4358)
|
||||
- [Armbian](https://www.armbian.com/) — Debian/Ubuntu for ARM SBCs and TV boxes
|
||||
- [rtw88 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/realtek/rtw88) — Mainline Linux driver for Realtek 802.11ac chips
|
||||
- [mt76 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/mediatek/mt76) — Mainline Linux driver for MediaTek WiFi chips
|
||||
- [cross](https://github.com/cross-rs/cross) — Zero-setup Rust cross-compilation
|
||||
- [ADR-018: ESP32 CSI Binary Protocol](ADR-018-dev-implementation.md) — Binary frame format reused for Phase 2 CSI extraction
|
||||
- [ADR-039: Edge Intelligence](ADR-039-esp32-edge-intelligence.md) — On-device processing tiers
|
||||
- [ADR-043: Sensing Server](ADR-043-sensing-server-ui-api-completion.md) — Single-binary deployment target
|
||||
@@ -0,0 +1,152 @@
|
||||
# ADR-047: RuView Observatory — Immersive Three.js WiFi Sensing Visualization
|
||||
|
||||
## Status
|
||||
|
||||
Accepted (Implemented)
|
||||
|
||||
## Date
|
||||
|
||||
2026-03-04
|
||||
|
||||
## Context
|
||||
|
||||
The project has a functional tabbed dashboard UI (`ui/index.html`) with existing Three.js components (body model, gaussian splats, signal visualization, environment). While effective for monitoring, it lacks a cinematic, immersive visualization suitable for demonstrations and stakeholder presentations.
|
||||
|
||||
We need an immersive Three.js room-based visualization with practical WiFi sensing data overlays — human wireframe pose, dot-matrix body mass, vital signs HUD, signal field heatmap — powered by ESP32 CSI data (demo mode with live WebSocket path).
|
||||
|
||||
## Decision
|
||||
|
||||
### Standalone Page Architecture
|
||||
|
||||
`ui/observatory.html` is a standalone full-screen entry point, separate from the tabbed dashboard. Linked via "Observatory" nav tab in `ui/index.html`. No build step — vanilla JS modules with Three.js r160 via CDN importmap.
|
||||
|
||||
### Room-Based Visualization
|
||||
|
||||
Instead of abstract holographic panels, the observatory renders a practical room scene with:
|
||||
|
||||
| Element | Implementation | Data Source |
|
||||
|---------|---------------|-------------|
|
||||
| Human wireframe | COCO 17-keypoint skeleton, CylinderGeometry tube bones, SphereGeometry joints with glow halos | `persons[].position`, `vital_signs.breathing_rate_bpm` |
|
||||
| Dot-matrix mist | 800 Points with per-particle alpha ShaderMaterial, body-shaped distribution | `persons[].position`, `persons[].motion_score` |
|
||||
| Particle trail | 200 Points with age-based fade, emitted from moving person | `persons[].position`, `persons[].motion_score` |
|
||||
| Signal field | 400 floor-level Points with green→amber color ramp | `signal_field.values` (20×20 grid) |
|
||||
| WiFi waves | 5 wireframe SphereGeometry shells, AdditiveBlending, pulsing outward | Always-on animation from router position |
|
||||
| Router | BoxGeometry body, 3 CylinderGeometry antennas, pulsing LED, PointLight | Static scene element |
|
||||
| Room | GridHelper floor, BoxGeometry wireframe boundary, reflective MeshStandardMaterial floor, furniture (table, bed) | Static scene element |
|
||||
|
||||
### HUD Overlay
|
||||
|
||||
Glass-morphism HTML panels overlaid on the 3D canvas:
|
||||
|
||||
- **Left panel (Vital Signs):** Heart rate (BPM), respiration (RPM), confidence (%) with animated bars
|
||||
- **Right panel (WiFi Signal):** RSSI, variance, motion power, person count, 2D RSSI sparkline, presence state badge, fall alert
|
||||
- **Top-right:** Data source badge (DEMO/LIVE), scenario badge, FPS counter, settings gear
|
||||
- **Bottom:** Capability bar (Pose Estimation, Vital Monitoring, Presence Detection)
|
||||
- **Bottom-right:** Keyboard shortcut hints
|
||||
|
||||
### Settings Dialog (4 Tabs)
|
||||
|
||||
Full customization with localStorage persistence and JSON export:
|
||||
|
||||
| Tab | Controls |
|
||||
|-----|----------|
|
||||
| **Rendering** | Bloom strength/radius/threshold, exposure, vignette, film grain, chromatic aberration |
|
||||
| **Wireframe** | Bone thickness, joint size, glow intensity, particle trail, wireframe color, joint color, aura opacity |
|
||||
| **Scene** | Signal field opacity, WiFi wave intensity, room brightness, floor reflection, FOV, orbit speed, grid toggle, room boundary toggle |
|
||||
| **Data** | Scenario selector (auto-cycle or fixed), cycle speed, data source (demo/WebSocket), WS URL, reset camera, export settings |
|
||||
|
||||
### Demo-First with Live Data Path
|
||||
|
||||
Four auto-cycling scenarios (30s default, configurable) with 2s cosine crossfade:
|
||||
|
||||
| Scenario | Description |
|
||||
|----------|-------------|
|
||||
| `empty_room` | Low variance, no presence, flat amplitude, stable RSSI -45dBm |
|
||||
| `single_breathing` | 1 person, breathing 16 BPM, HR 72 BPM, sinusoidal subcarrier modulation |
|
||||
| `two_walking` | 2 persons, high motion, Doppler-like shifts, moving signal field peaks |
|
||||
| `fall_event` | 2s variance spike at t=5s, then stillness, fall flag, confidence drop |
|
||||
|
||||
Data contract matches `SensingUpdate` struct from the Rust sensing server. Live WebSocket connection configurable in settings dialog.
|
||||
|
||||
### Post-Processing Pipeline
|
||||
|
||||
EffectComposer chain: RenderPass → UnrealBloomPass → custom VignetteShader
|
||||
|
||||
- **UnrealBloom:** strength 1.0, radius 0.5, threshold 0.25 (configurable)
|
||||
- **VignetteShader:** warm shadow shift, edge chromatic aberration, film grain
|
||||
- **Adaptive quality:** Auto-degrades when FPS < 25, restores when FPS > 55
|
||||
|
||||
### RuView Foundation Color Palette
|
||||
|
||||
| Role | Color | Hex |
|
||||
|------|-------|-----|
|
||||
| Background | Deep dark | `#080c14` |
|
||||
| Primary wireframe | Green glow | `#00d878` |
|
||||
| Warm accent | Amber | `#ffb020` |
|
||||
| Signal | Blue | `#2090ff` |
|
||||
| Heart / joints | Red | `#ff4060` |
|
||||
| Alert | Crimson | `#ff3040` |
|
||||
|
||||
### Technology Choices
|
||||
|
||||
| Decision | Rationale |
|
||||
|----------|-----------|
|
||||
| Standalone page vs tab | Full-screen immersion, independent loading |
|
||||
| Room-based vs abstract panels | Practical spatial context for WiFi sensing data |
|
||||
| Vanilla JS + CDN, no build step | Matches existing `ui/` pattern, served as static files by Axum |
|
||||
| Custom ShaderMaterial for mist | Per-particle alpha, body-shaped distribution, AdditiveBlending |
|
||||
| CylinderGeometry tube bones | Visible at any zoom vs thin Line geometry |
|
||||
| COCO 17-keypoint skeleton | Standard pose format, 16 bone connections |
|
||||
| localStorage settings | Persistent customization without server round-trip |
|
||||
| Adaptive quality | 3 levels, auto-switches based on FPS measurement |
|
||||
|
||||
### Keyboard Shortcuts
|
||||
|
||||
| Key | Action |
|
||||
|-----|--------|
|
||||
| `A` | Toggle autopilot orbit |
|
||||
| `D` | Cycle demo scenario |
|
||||
| `F` | Toggle FPS counter |
|
||||
| `S` | Open/close settings |
|
||||
| `Space` | Pause/resume data |
|
||||
|
||||
## Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `ui/observatory.html` | Full-screen entry point with HUD overlay + settings dialog |
|
||||
| `ui/observatory/js/main.js` | Scene orchestrator (~1,100 lines): room, wireframe, mist, trails, settings, HUD, animation loop |
|
||||
| `ui/observatory/js/demo-data.js` | 4 scenarios with cosine crossfade, setScenario/setCycleDuration API |
|
||||
| `ui/observatory/js/nebula-background.js` | Procedural fBM nebula + star field background sphere |
|
||||
| `ui/observatory/js/post-processing.js` | EffectComposer: UnrealBloom + VignetteShader (chromatic, grain, warmth) |
|
||||
| `ui/observatory/css/observatory.css` | Foundation color scheme, glass-morphism panels, settings dialog, responsive |
|
||||
| `ui/index.html` | Modified: added Observatory nav link |
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Standalone page does not affect existing dashboard stability
|
||||
- Demo-first allows offline presentations without hardware
|
||||
- Same `SensingUpdate` contract enables seamless live WebSocket switch
|
||||
- Room-based visualization provides intuitive spatial context for WiFi sensing
|
||||
- Dot-matrix mist gives visual body mass without occluding wireframe
|
||||
- Full settings customization without code changes (localStorage + JSON export)
|
||||
- Adaptive quality ensures usability on weaker hardware
|
||||
- ~20 draw calls keeps performance well within budget
|
||||
|
||||
### Negative
|
||||
- Additional static files served by Axum (minimal overhead)
|
||||
- Three.js r160 loaded from CDN (no build step, matches existing pattern)
|
||||
- Settings persistence is per-browser (localStorage, not synced)
|
||||
|
||||
### Risks
|
||||
- CDN dependency for Three.js (mitigated: can vendor locally if needed)
|
||||
- Post-processing may not work on very old GPUs (mitigated: adaptive quality disables bloom)
|
||||
|
||||
## References
|
||||
|
||||
- ADR-045: AMOLED display support
|
||||
- ADR-046: Android TV / Armbian deployment
|
||||
- Existing `ui/components/scene.js` — Three.js scene pattern
|
||||
- Existing `ui/components/gaussian-splats.js` — ShaderMaterial pattern
|
||||
- Existing `ui/services/sensing.service.js` — WebSocket data contract
|
||||
Reference in New Issue
Block a user