Compare commits

...

19 Commits

Author SHA1 Message Date
rUv c82c4fc4ac Update README.md 2026-03-07 23:07:12 -05:00
rUv 0c85d9c86f Update README.md
updated intro
2026-03-07 22:56:18 -05:00
rUv 65c6fa7a34 Update README.md
update intro
2026-03-07 22:51:17 -05:00
rUv 7659b0bbe2 feat: cross-platform WiFi collector factory (ADR-049) (#173)
feat: cross-platform WiFi collector factory (ADR-049)
2026-03-06 15:10:26 -05:00
ruv 75d4685d25 feat: cross-platform WiFi collector factory with graceful degradation (ADR-049)
- Add create_collector() factory function that auto-detects platform and never raises
- Add LinuxWifiCollector.is_available() classmethod for probe-without-exception
- Refactor ws_server.py to use create_collector(), removing ~30 lines of duplicated platform detection
- Add 10 unit tests covering all platform paths and edge cases
- Add ADR-049 documenting the cross-platform detection and fallback chain

Docker, WSL, and headless users now get SimulatedCollector automatically
with a clear WARNING log instead of a RuntimeError crash.

Closes #148
Closes #155

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-06 15:09:32 -05:00
rUv 45c15b77a5 fix: ADR-050 security hardening — HMAC, path traversal, OTA auth (#172)
fix: ADR-050 security hardening — HMAC, path traversal, OTA auth
2026-03-06 14:02:50 -05:00
ruv 47223a98be fix: security hardening — replace fake HMAC, add path traversal protection, OTA auth (ADR-050)
Sprint 1 security fixes from quality engineering analysis (issue #170):

- Replace XOR-fold fake HMAC with real HMAC-SHA256 (hmac + sha2 crates) in secure_tdm.rs
- Add path traversal sanitization on DELETE /api/v1/models/:id and /api/v1/recording/:id
- Default bind address changed from 0.0.0.0 to 127.0.0.1 (configurable via --bind-addr / SENSING_BIND_ADDR)
- Add PSK authentication to ESP32 OTA firmware upload endpoint (ota_update.c)
- Flip WASM signature verification to default-on (CONFIG_WASM_SKIP_SIGNATURE opt-out vs opt-in)
- Add 6 new security tests: HMAC key/message sensitivity, determinism, wrong-key rejection, bit-flip detection, enforcing mode
- Add clap env feature for environment variable configuration

All 106 hardware crate tests pass. Sensing server compiles clean.

Closes #170

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-06 13:11:04 -05:00
ruv c45690ed4e fix: use montserrat_14 for display_ui big label (montserrat_20 not in Kconfig)
Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 11:45:59 -05:00
ruv fb782e0d71 fix: brighten ambient light color and increase multiplier for room brightness slider
The ambient light color 0x446688 (dark blue-gray) was too dim to produce
visible brightness changes. Changed to 0xccccdd (bright neutral) with 5x
multiplier. Bumped SETTINGS_VERSION to force fresh defaults.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:56:37 -05:00
ruv 944076733e fix: room brightness slider now applies 3x multiplier to ambient light
The ambient light was initialized with intensity * 3.0 but the slider
and preset callbacks set raw value without the multiplier, making the
setting appear to do nothing.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:51:41 -05:00
ruv a8f48a7897 docs: make hero image clickable, links to live demo
Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:48:41 -05:00
ruv 7df316f13e docs: make README screenshot clickable, links to live demo
Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:45:53 -05:00
ruv da54ea07d2 fix: reduce default bloom strength, ensure auto-cycle starts on load
- Default bloom: 0.2 → 0.08, radius 0.25 → 0.2, threshold 0.5 → 0.6
- PostProcessing constructor matches new defaults
- Bump SETTINGS_VERSION to '5' to clear stale localStorage (forces auto scenario)

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:42:37 -05:00
rUv bf4d64ad4b docs: add live Observatory demo link to README (#145) 2026-03-05 10:39:58 -05:00
ruv 8b57a6f64c docs: update README with ADR-045–048, Observatory, adaptive classifier, AMOLED display
- Update ADR count from 44 to 48
- Add adaptive classifier (ADR-048) to Intelligence features
- Add Observatory visualization (ADR-047) and AMOLED display (ADR-045) to Deployment features
- Update screenshot to v2-screen.png
- Add ADR-045 (AMOLED), ADR-046 (Android TV), ADR-047 (Observatory), DDD deployment model
- Add AMOLED display firmware (display_hal, display_task, display_ui, LVGL config)
- Add Observatory UI (13 Three.js modules, CSS, HTML entry point)
- Add trained adaptive model JSON
- Update .gitignore for managed_components, recordings, .swarm

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-05 10:20:48 -05:00
rUv 5fa61ba7ea feat: adaptive CSI classifier with signal smoothing pipeline (ADR-048) (#144)
Add environment-tuned activity classification that learns from labeled
ESP32 CSI recordings, replacing brittle static thresholds.

- Adaptive classifier: 15-feature logistic regression trained from JSONL
  recordings (variance, motion band, subcarrier stats: skew, kurtosis,
  entropy, IQR). Trains in <1s, persists as JSON, auto-loads on restart.
- Three-stage signal smoothing: adaptive baseline subtraction (α=0.003),
  EMA + trimmed-mean median filter (21-frame window), hysteresis debounce
  (4 frames). Motion classification now stable across seconds, not frames.
- Vital signs stabilization: outlier rejection (±8 BPM HR, ±2 BPM BR),
  trimmed mean, dead-band (±2 BPM HR), EMA α=0.02. HR holds steady for
  10+ seconds instead of jumping 50 BPM every frame.
- Observatory auto-detect: always probes /health on startup, connects
  WebSocket to live ESP32 data automatically.
- New API endpoints: POST /api/v1/adaptive/train, GET /adaptive/status,
  POST /adaptive/unload for runtime model management.
- Updated user guide with Observatory, adaptive classifier tutorial,
  signal smoothing docs, and new troubleshooting entries.
2026-03-05 10:15:18 -05:00
ruv f771cf8461 docs: add vendor README with submodule setup instructions
Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-04 13:31:19 -05:00
ruv c257e9a215 chore: track upstream main branch for vendor submodules
- Add branch = main to each submodule in .gitmodules
- Add GitHub Actions workflow that checks every 6 hours for
  upstream updates and opens a PR automatically

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-03-04 13:30:48 -05:00
rUv 6e76578dcf Merge pull request #137 from ruvnet/refactor/vendor-submodules
refactor: convert vendor/ to git submodules
2026-03-04 13:23:38 -05:00
55 changed files with 11902 additions and 140 deletions
+50
View File
@@ -0,0 +1,50 @@
name: Update vendor submodules
on:
schedule:
- cron: '0 */6 * * *' # Every 6 hours
workflow_dispatch: # Manual trigger
permissions:
contents: write
pull-requests: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
submodules: true
fetch-depth: 0
token: ${{ secrets.GITHUB_TOKEN }}
- name: Update submodules to latest main
run: git submodule update --remote --merge
- name: Check for changes
id: check
run: |
if git diff --quiet; then
echo "changed=false" >> "$GITHUB_OUTPUT"
else
echo "changed=true" >> "$GITHUB_OUTPUT"
fi
- name: Create PR with updates
if: steps.check.outputs.changed == 'true'
run: |
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
BRANCH="chore/update-submodules-$(date +%Y%m%d-%H%M%S)"
git checkout -b "$BRANCH"
git add vendor/
git commit -m "chore: update vendor submodules to latest main"
git push origin "$BRANCH"
gh pr create \
--title "chore: update vendor submodules" \
--body "Automated submodule update to latest upstream main." \
--base main \
--head "$BRANCH"
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+10
View File
@@ -8,6 +8,16 @@ firmware/esp32-csi-node/sdkconfig.defaults
firmware/esp32-csi-node/sdkconfig.old
# Downloaded WASM3 source (fetched at configure time)
firmware/esp32-csi-node/components/wasm3/wasm3-src/
# ESP-IDF managed components (downloaded at build time)
firmware/esp32-csi-node/managed_components/
firmware/esp32-csi-node/dependencies.lock
firmware/esp32-csi-node/sdkconfig.defaults.bak
# Claude Flow swarm runtime state
.swarm/
# CSI recordings (local training data, machine-specific)
rust-port/wifi-densepose-rs/data/recordings/
# NVS partition images and CSVs (contain WiFi credentials)
nvs.bin
+3
View File
@@ -1,9 +1,12 @@
[submodule "vendor/midstream"]
path = vendor/midstream
url = https://github.com/ruvnet/midstream
branch = main
[submodule "vendor/ruvector"]
path = vendor/ruvector
url = https://github.com/ruvnet/ruvector
branch = main
[submodule "vendor/sublinear-time-solver"]
path = vendor/sublinear-time-solver
url = https://github.com/ruvnet/sublinear-time-solver
branch = main
+31 -6
View File
@@ -1,14 +1,32 @@
# π RuView
<p align="center">
<img src="assets/ruview-small-gemini.jpg" alt="RuView - WiFi DensePose" width="100%">
<a href="https://ruvnet.github.io/RuView/">
<img src="assets/ruview-small-gemini.jpg" alt="RuView - WiFi DensePose" width="100%">
</a>
</p>
**See through walls with WiFi.** No cameras. No wearables. No Internet. Just radio waves.
## **See through walls with WiFi + Ai** ##
WiFi DensePose turns commodity WiFi signals into real-time human pose estimation, vital sign monitoring, and presence detection — all without a single pixel of video.
**Perceive the world through signals.** No cameras. No wearables. No Internet. Just physics.
By analyzing Channel State Information (CSI) disturbances caused by human movement, the system reconstructs body position, breathing rate, and heartbeat using physics-based signal processing and machine learning.
### π RuView is an edge AI perception system that learns directly from the environment around it.
Instead of relying on cameras or cloud models, it observes whatever signals exist in a space such as WiFi, radio waves across the spectrum, motion patterns, vibration, sound, or other sensory inputs and builds an understanding of what is happening locally.
Built on top of [RuVector](https://github.com/ruvnet/ruvector/), the project became widely known for its implementation of WiFi DensePose — a sensing technique first explored in academic research such as Carnegie Mellon University's *DensePose From WiFi* work. That research demonstrated that WiFi signals can be used to reconstruct human pose.
RuView extends that concept into a practical edge system. By analyzing Channel State Information (CSI) disturbances caused by human movement, RuView reconstructs body position, breathing rate, heart rate, and presence in real time using physics-based signal processing and machine learning.
Unlike research systems that rely on synchronized cameras for training, RuView is designed to operate entirely from radio signals and self-learned embeddings at the edge.
The system runs entirely on inexpensive hardware such as an ESP32 sensor mesh (as low as ~$1 per node). Small programmable edge modules analyze signals locally and learn the RF signature of a room over time, allowing the system to separate the environment from the activity happening inside it.
Because RuView learns in proximity to the signals it observes, it improves as it operates. Each deployment develops a local model of its surroundings and continuously adapts without requiring cameras, labeled data, or cloud infrastructure.
In practice this means ordinary environments gain a new kind of spatial awareness. Rooms, buildings, and devices begin to sense presence, movement, and vital activity using the signals that already fill the space.
### Built for low-power edge applications
[Edge modules](#edge-intelligence-adr-041) are small programs that run directly on the ESP32 sensor — no internet needed, no cloud fees, instant response.
@@ -57,15 +75,19 @@ docker run -p 3000:3000 ruvnet/wifi-densepose:latest
|----------|-------------|
| [User Guide](docs/user-guide.md) | Step-by-step guide: installation, first run, API usage, hardware setup, training |
| [Build Guide](docs/build-guide.md) | Building from source (Rust and Python) |
| [Architecture Decisions](docs/adr/README.md) | 44 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Architecture Decisions](docs/adr/README.md) | 48 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Domain Models](docs/ddd/README.md) | 7 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI) — bounded contexts, aggregates, domain events, and ubiquitous language |
---
<img src="assets/screen.png" alt="WiFi DensePose — Live pose detection with setup guide" width="800">
<a href="https://ruvnet.github.io/RuView/">
<img src="assets/v2-screen.png" alt="WiFi DensePose — Live pose detection with setup guide" width="800">
</a>
<br>
<em>Real-time pose skeleton from WiFi CSI signals — no cameras, no wearables</em>
<br>
<a href="https://ruvnet.github.io/RuView/"><strong>▶ Live Observatory Demo</strong></a>
> The [server](#-quick-start) is optional for visualization and aggregation — the ESP32 [runs independently](#esp32-s3-hardware-pipeline) for presence detection, vital signs, and fall alerts.
@@ -98,6 +120,7 @@ The system learns on its own and gets smarter over time — no hand-tuning, no l
| 👁️ | **Cross-Viewpoint Fusion** | AI combines what each sensor sees from its own angle — fills in blind spots and depth ambiguity that no single viewpoint can resolve on its own ([ADR-031](docs/adr/ADR-031-ruview-sensing-first-rf-mode.md)) |
| 🔮 | **Signal-Line Protocol** | A 6-stage processing pipeline transforms raw WiFi signals into structured body representations — from signal cleanup through graph-based spatial reasoning to final pose output ([ADR-033](docs/adr/ADR-033-crv-signal-line-sensing-integration.md)) |
| 🔒 | **QUIC Mesh Security** | All sensor-to-sensor communication is encrypted end-to-end with tamper detection, replay protection, and seamless reconnection if a node moves or drops offline ([ADR-032](docs/adr/ADR-032-multistatic-mesh-security-hardening.md)) |
| 🎯 | **Adaptive Classifier** | Records labeled CSI sessions, trains a 15-feature logistic regression model in pure Rust, and learns your room's unique signal characteristics — replaces hand-tuned thresholds with data-driven classification ([ADR-048](docs/adr/ADR-048-adaptive-csi-classifier.md)) |
### Performance & Deployment
@@ -110,6 +133,8 @@ Fast enough for real-time use, small enough for edge devices, simple enough for
| 🐳 | **One-Command Setup** | `docker pull ruvnet/wifi-densepose:latest` — live sensing in 30 seconds, no toolchain needed (amd64 + arm64 / Apple Silicon) |
| 📡 | **Fully Local** | Runs completely on a $9 ESP32 — no internet connection, no cloud account, no recurring fees. Detects presence, vital signs, and falls on-device with instant response |
| 📦 | **Portable Models** | Trained models package into a single `.rvf` file — runs on edge, cloud, or browser (WASM) |
| 🔭 | **Observatory Visualization** | Cinematic Three.js dashboard with 5 holographic panels — subcarrier manifold, vital signs oracle, presence heatmap, phase constellation, convergence engine — all driven by live or demo CSI data ([ADR-047](docs/adr/ADR-047-psychohistory-observatory-visualization.md)) |
| 📟 | **AMOLED Display** | ESP32-S3 boards with built-in AMOLED screens show real-time presence, vital signs, and room status directly on the sensor — no phone or PC needed ([ADR-045](docs/adr/ADR-045-amoled-display-support.md)) |
---
Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 MiB

+110
View File
@@ -0,0 +1,110 @@
# ADR-045: AMOLED Display Support for ESP32-S3 CSI Node
## Status
Proposed
## Context
The ESP32-S3 board (LilyGO T-Display-S3 AMOLED) has an integrated RM67162 QSPI AMOLED display (536x240) and 8MB octal PSRAM that were unused by the CSI firmware. Users want real-time on-device visualization of CSI statistics, vital signs, and system health without relying on an external server.
### Constraints
- Binary was 947 KB in a 1 MB partition — needed 8MB flash + custom partition table
- SPIRAM was disabled in sdkconfig despite hardware having 8MB PSRAM
- Core 1 is pinned to DSP (edge processing) — display must use Core 0
- Existing CSI pipeline must not be affected
### Available APIs
Thread-safe edge APIs already exist (`edge_get_vitals()`, `edge_get_multi_person()`) — the display task only reads from these, no new synchronization needed.
## Decision
Add optional AMOLED display support with the following architecture:
### Hardware Abstraction Layer
- `display_hal.c/h`: RM67162 QSPI panel driver + CST816S capacitive touch via I2C
- Auto-detect at boot: probe RM67162 and check SPIRAM; log warning and skip if absent
### UI Layer
- `display_ui.c/h`: LVGL 8.3 with 4 swipeable views via tileview widget
- Dark theme (#0a0a0f) with cyan (#00d4ff) accent for three.js-like aesthetic
- Views: Dashboard (CSI amplitude chart + stats), Vitals (breathing + HR line graphs), Presence (4x4 occupancy grid), System (CPU, heap, PSRAM, WiFi, uptime, FPS)
### Task Layer
- `display_task.c/h`: FreeRTOS task on Core 0, priority 1 (lowest)
- LVGL pump loop at configurable FPS (default 30)
- Double-buffered draw buffers allocated in SPIRAM
### Compile-Time Control
- `CONFIG_DISPLAY_ENABLE=y` (default): compiles display code, auto-detects hardware at boot
- `CONFIG_DISPLAY_ENABLE=n`: zero-cost — no display code compiled
- `CONFIG_SPIRAM_IGNORE_NOTFOUND=y`: boots fine on boards without PSRAM
### Flash Layout
8MB partition table (`partitions_display.csv`):
- Dual OTA partitions: 2 x 2MB (supports larger binaries with LVGL)
- SPIFFS: 1.9MB (for future font/asset storage)
- NVS + otadata + phy: standard sizes
### Core/Task Layout
| Task | Core | Priority | Impact |
|------|------|----------|--------|
| WiFi/LwIP | 0 | 18-23 | unchanged |
| OTA httpd | 0 | 5 | unchanged |
| **display_task** | **0** | **1** | **NEW — lowest priority** |
| edge_task (DSP) | 1 | 5 | unchanged |
### Dependencies
- LVGL ~8.3 (via ESP-IDF managed components)
- espressif/esp_lcd_touch_cst816s ^1.0
- espressif/esp_lcd_touch ^1.0
## Consequences
### Positive
- Real-time on-device stats without network dependency
- Zero impact on CSI pipeline (display reads thread-safe APIs, runs at lowest priority)
- Graceful degradation: works on boards without display or PSRAM
- SPIRAM enabled for all boards (benefits WASM runtime too)
- 8MB flash + dual OTA 2MB partitions give headroom for future features
### Negative
- Binary size increase (~200-300 KB with LVGL)
- SPIRAM + 8MB flash config is specific to T-Display-S3 AMOLED boards
- Boards with only 4MB flash need `CONFIG_DISPLAY_ENABLE=n` and the old partition table
### Risks
- RM67162 init sequence is board-specific; other AMOLED panels may need different commands
- QSPI bus conflicts if other peripherals use SPI2_HOST (currently unused)
## New Files
| File | Purpose |
|------|---------|
| `main/display_hal.c/h` | RM67162 QSPI + CST816S touch HAL |
| `main/display_ui.c/h` | LVGL 4-view UI |
| `main/display_task.c/h` | FreeRTOS task, LVGL pump |
| `main/lv_conf.h` | LVGL compile config |
| `partitions_display.csv` | 8MB partition table |
| `idf_component.yml` | Managed component deps |
## Modified Files
| File | Change |
|------|--------|
| `sdkconfig.defaults` | 8MB flash, SPIRAM, custom partitions |
| `main/CMakeLists.txt` | Conditional display sources + deps |
| `main/main.c` | +1 include, +5 lines guarded init |
| `main/Kconfig.projbuild` | "AMOLED Display" menu |
@@ -0,0 +1,263 @@
# ADR-046: Android TV Box / Armbian Deployment Target
## Status
Proposed
## Context
Issue [#138](https://github.com/ruvnet/wifi-densepose/issues/138) requests ESP8266 and mobile device support. The ESP8266 lacks CSI capability and sufficient resources, but the discussion revealed a compelling deployment target: **Android TV boxes** (Amlogic/Allwinner/Rockchip SoCs) running **Armbian** (Debian for ARM).
These devices cost $1535, are always-on mains-powered, include 802.11ac WiFi, 24 GB RAM, quad-core ARM Cortex-A53/A55 CPUs, and HDMI output. They are widely available as consumer "IPTV boxes" (T95, H96 Max, X96, MXQ Pro, etc.) and can boot Armbian from SD card without modifying the factory Android installation.
### Current deployment model
```
[ESP32-S3 nodes] --UDP CSI--> [Laptop/PC running sensing-server] --browser--> [UI]
```
This requires a general-purpose computer ($300+) to run the Rust sensing server, NN inference, and web dashboard. For permanent installations (elder care, smart home, security), dedicating a laptop is impractical.
### Proposed deployment model
```
[ESP32-S3 nodes] --UDP CSI--> [TV Box running Armbian + sensing-server] --HDMI--> [Display]
$25, always-on, fanless
```
### Future: custom WiFi firmware for standalone operation
Many TV box WiFi chipsets (Realtek RTL8822CS, MediaTek MT7661, Broadcom BCM43455) can potentially be patched for CSI extraction when running under Linux with custom drivers. This would eliminate the ESP32 dependency entirely for basic sensing:
```
[TV Box with patched WiFi driver] --CSI extraction--> [sensing-server on same box] --HDMI--> [Display]
$25 total, single device
```
This ADR covers Phase 1 (TV box as aggregator) and Phase 2 (custom WiFi firmware for CSI). Phase 2 is speculative and requires per-chipset R&D.
## Decision
### Phase 1: TV Box as Aggregator (Armbian)
1. **Cross-compile the sensing server** for `aarch64-unknown-linux-gnu` using `cross` or Docker-based cross-compilation.
2. **Create an Armbian deployment package** containing:
- Pre-built `wifi-densepose-sensing-server` binary (aarch64)
- systemd service file for auto-start on boot
- Kiosk-mode Chromium configuration for HDMI dashboard display
- Network configuration for ESP32 UDP reception (port 5005)
- Optional: `hostapd` config to create a dedicated WiFi AP for the ESP32 mesh
3. **Define minimum hardware requirements:**
| Component | Minimum | Recommended |
|-----------|---------|-------------|
| SoC | Amlogic S905W (A53 quad) | Amlogic S905X3 (A55 quad) |
| RAM | 2 GB | 4 GB |
| Storage | 8 GB eMMC + 8 GB SD | 16 GB eMMC + 16 GB SD |
| WiFi | 802.11n 2.4 GHz | 802.11ac dual-band |
| Ethernet | 100 Mbps | Gigabit |
| USB | 1x USB 2.0 | 2x USB 3.0 |
| HDMI | 1.4 | 2.0 |
4. **Tested reference devices** (initial target list):
| Device | SoC | WiFi Chip | Price | Armbian Support |
|--------|-----|-----------|-------|-----------------|
| T95 Max+ | S905X3 | RTL8822CS | ~$30 | Good (meson-sm1) |
| H96 Max X3 | S905X3 | RTL8822CS | ~$35 | Good (meson-sm1) |
| X96 Max+ | S905X3 | RTL8822CS | ~$28 | Good (meson-sm1) |
| Tanix TX6S | H616 | MT7668 | ~$25 | Moderate (sun50i-h616) |
5. **New Rust compilation target** in workspace CI:
- Add `aarch64-unknown-linux-gnu` to cross-compilation matrix
- Binary size target: <15 MB stripped (fits easily in SD card)
- No GPU dependency — CPU-only inference using `candle` or ONNX Runtime for ARM
### Phase 2: Custom WiFi Firmware for CSI Extraction (Future)
1. **CSI extraction feasibility by chipset:**
| Chipset | Driver | CSI Support | Monitor Mode | Effort |
|---------|--------|-------------|--------------|--------|
| Broadcom BCM43455 | brcmfmac | **Proven** (Nexmon CSI) | Yes | Low — patches exist |
| Realtek RTL8822CS | rtw88 | **Moderate** — driver is open-source, CSI hooks need adding | Yes (patched) | Medium |
| MediaTek MT7661 | mt76 | **Unknown** — MediaTek has released CSI tools for some chips | Yes | Medium-High |
2. **CSI extraction architecture** (Linux kernel driver modification):
```
[WiFi chipset firmware] → [Modified kernel driver] → [Netlink/procfs CSI export]
[userspace CSI reader]
[sensing-server UDP input]
```
The CSI data would be reformatted into the existing ESP32 binary protocol (ADR-018 header, magic `0xC5100001`) so the sensing server treats it identically to ESP32 frames. This means zero changes to the ingestion context.
3. **Hybrid mode**: When the TV box has both patched WiFi CSI and ESP32 UDP input, the sensing server's multi-node architecture (already supporting multiple `node_id` values) handles both sources transparently. The TV box's own WiFi becomes an additional viewpoint in the multistatic array.
### Phase 3: Android Companion App (Optional)
For users who want mobile monitoring without Armbian:
1. **PWA (Progressive Web App)**: The sensing server already serves a web UI. Adding a PWA manifest with offline caching makes it installable on any Android device. No native app needed.
2. **Native Android app** (future): Only if PWA proves insufficient. Would use Kotlin + Jetpack Compose, consuming the existing REST API and WebSocket endpoints.
## Deployment Architecture
### Single-Room Deployment (Phase 1)
```
┌──────────────────────────────────────────────────────────────┐
│ Room │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
│ │ ESP32-S3 │ │ ESP32-S3 │ │ ESP32-S3 │ CSI sensor mesh │
│ │ Node 1 │ │ Node 2 │ │ Node 3 │ ($10 each) │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
│ │ │ │ │
│ └──────────────┼──────────────┘ │
│ │ UDP port 5005 │
│ ▼ │
│ ┌──────────────────────────────────────┐ │
│ │ Android TV Box (Armbian) │ │
│ │ │ │
│ │ ┌──────────────────────────────┐ │ │
│ │ │ wifi-densepose-sensing- │ │ │
│ │ │ server (aarch64 binary) │ │ │
│ │ │ │ │ │
│ │ │ • CSI ingestion (UDP) │ │ │
│ │ │ • Feature extraction │ │ │
│ │ │ • NN inference (CPU) │ │ │
│ │ │ • WebSocket streaming │ │ │
│ │ │ • REST API │ │ │
│ │ │ • Web UI (:3000) │ │ │
│ │ └──────────────────────────────┘ │ │
│ │ │ │
│ │ ┌──────────────────────────────┐ │ │
│ │ │ Chromium Kiosk Mode │───│──→ HDMI out │
│ │ │ (localhost:3000) │ │ to display │
│ │ └──────────────────────────────┘ │ │
│ │ │ │
│ │ Cost: $25-35 │ │
│ │ Power: 5-10W (USB-C or barrel) │ │
│ │ Form: fits behind TV/monitor │ │
│ └──────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────┘
Total system cost: $55-65 (3 ESP32 nodes + 1 TV box)
```
### Multi-Room Deployment
```
┌──────────────┐
│ Router │
│ (WiFi AP) │
└──────┬───────┘
│ LAN
┌──────────────┼──────────────┐
│ │ │
┌───────▼───────┐ ┌───▼────────┐ ┌──▼──────────┐
│ Room A │ │ Room B │ │ Room C │
│ TV Box + │ │ TV Box + │ │ TV Box + │
│ 3x ESP32 │ │ 3x ESP32 │ │ 3x ESP32 │
│ HDMI display │ │ HDMI │ │ HDMI │
└───────────────┘ └────────────┘ └─────────────┘
Each room: self-contained sensing + display
Central dashboard: aggregate all rooms via REST API
```
### Standalone Mode (Phase 2 — Custom WiFi FW)
```
┌──────────────────────────────────────┐
│ Android TV Box (Armbian) │
│ │
│ ┌────────────────────┐ │
│ │ Patched WiFi │ │
│ │ Driver │ │
│ │ (CSI extraction) │ │
│ └─────────┬──────────┘ │
│ │ CSI frames │
│ ▼ │
│ ┌────────────────────┐ │
│ │ sensing-server │──→ HDMI out │
│ │ (inference + │ │
│ │ dashboard) │ │
│ └────────────────────┘ │
│ │
│ Single device: $25 │
│ No ESP32 nodes needed │
└──────────────────────────────────────┘
```
## Consequences
### Positive
- **10x cost reduction** for aggregator: $25 TV box vs $300+ laptop/PC
- **Always-on deployment**: Mains-powered, fanless, designed for 24/7 operation
- **HDMI output**: Direct connection to TV/monitor for wall-mounted dashboards
- **Familiar hardware**: Available globally, no specialized ordering required
- **Armbian ecosystem**: Mature Debian-based distro with package management, systemd, SSH
- **Path to standalone**: Custom WiFi firmware could eliminate ESP32 dependency entirely
- **PWA for mobile**: No native app development needed for mobile monitoring
- **Multi-room scaling**: One TV box per room, each self-contained
### Negative
- **ARM cross-compilation**: Adds CI complexity; `candle`/ONNX Runtime ARM builds need testing
- **Armbian compatibility**: Not all TV boxes are well-supported; need a tested device list
- **Performance uncertainty**: ARM A53 cores are ~3-5x slower than x86 for NN inference; may need model quantization (INT8) for real-time operation
- **Phase 2 risk**: Custom WiFi firmware is chipset-specific, may require kernel patches per driver version, and CSI quality varies by chipset
- **Support burden**: Different hardware = more configurations to support
- **No GPU**: TV boxes lack discrete GPU; inference is CPU-only (but our models are small enough)
### Neutral
- **No changes to existing ESP32 firmware** — TV box receives the same UDP frames
- **No changes to sensing server protocol** — Phase 2 CSI output uses same binary format
- **Existing web UI works as-is** — Chromium kiosk mode or any browser on the LAN
## Implementation Plan
### Phase 1 (2-3 weeks)
1. Add `aarch64-unknown-linux-gnu` cross-compilation target using `cross`
2. Build and test sensing-server binary on reference TV box (T95 Max+ / S905X3)
3. Create systemd service + Armbian deployment script
4. Benchmark: measure inference latency, memory usage, thermal throttling
5. Create `docs/deployment/armbian-tv-box.md` setup guide
6. Add HDMI kiosk mode configuration (Chromium autostart)
### Phase 2 (4-8 weeks, R&D)
1. Acquire TV box with BCM43455 (proven Nexmon CSI support)
2. Build Armbian with Nexmon CSI patches for BCM43455
3. Write userspace CSI reader → ESP32 binary protocol converter
4. Test CSI quality comparison: ESP32 vs BCM43455
5. If viable: add RTL8822CS CSI extraction via rtw88 driver modification
### Phase 3 (1 week)
1. Add PWA manifest to sensing server web UI
2. Test on Android Chrome, iOS Safari
3. Add service worker for offline dashboard caching
## References
- [Nexmon CSI](https://github.com/seemoo-lab/nexmon_csi) — Broadcom WiFi CSI extraction (BCM43455, BCM4339, BCM4358)
- [Armbian](https://www.armbian.com/) — Debian/Ubuntu for ARM SBCs and TV boxes
- [rtw88 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/realtek/rtw88) — Mainline Linux driver for Realtek 802.11ac chips
- [mt76 driver](https://github.com/torvalds/linux/tree/master/drivers/net/wireless/mediatek/mt76) — Mainline Linux driver for MediaTek WiFi chips
- [cross](https://github.com/cross-rs/cross) — Zero-setup Rust cross-compilation
- [ADR-018: ESP32 CSI Binary Protocol](ADR-018-dev-implementation.md) — Binary frame format reused for Phase 2 CSI extraction
- [ADR-039: Edge Intelligence](ADR-039-esp32-edge-intelligence.md) — On-device processing tiers
- [ADR-043: Sensing Server](ADR-043-sensing-server-ui-api-completion.md) — Single-binary deployment target
@@ -0,0 +1,152 @@
# ADR-047: RuView Observatory — Immersive Three.js WiFi Sensing Visualization
## Status
Accepted (Implemented)
## Date
2026-03-04
## Context
The project has a functional tabbed dashboard UI (`ui/index.html`) with existing Three.js components (body model, gaussian splats, signal visualization, environment). While effective for monitoring, it lacks a cinematic, immersive visualization suitable for demonstrations and stakeholder presentations.
We need an immersive Three.js room-based visualization with practical WiFi sensing data overlays — human wireframe pose, dot-matrix body mass, vital signs HUD, signal field heatmap — powered by ESP32 CSI data (demo mode with live WebSocket path).
## Decision
### Standalone Page Architecture
`ui/observatory.html` is a standalone full-screen entry point, separate from the tabbed dashboard. Linked via "Observatory" nav tab in `ui/index.html`. No build step — vanilla JS modules with Three.js r160 via CDN importmap.
### Room-Based Visualization
Instead of abstract holographic panels, the observatory renders a practical room scene with:
| Element | Implementation | Data Source |
|---------|---------------|-------------|
| Human wireframe | COCO 17-keypoint skeleton, CylinderGeometry tube bones, SphereGeometry joints with glow halos | `persons[].position`, `vital_signs.breathing_rate_bpm` |
| Dot-matrix mist | 800 Points with per-particle alpha ShaderMaterial, body-shaped distribution | `persons[].position`, `persons[].motion_score` |
| Particle trail | 200 Points with age-based fade, emitted from moving person | `persons[].position`, `persons[].motion_score` |
| Signal field | 400 floor-level Points with green→amber color ramp | `signal_field.values` (20×20 grid) |
| WiFi waves | 5 wireframe SphereGeometry shells, AdditiveBlending, pulsing outward | Always-on animation from router position |
| Router | BoxGeometry body, 3 CylinderGeometry antennas, pulsing LED, PointLight | Static scene element |
| Room | GridHelper floor, BoxGeometry wireframe boundary, reflective MeshStandardMaterial floor, furniture (table, bed) | Static scene element |
### HUD Overlay
Glass-morphism HTML panels overlaid on the 3D canvas:
- **Left panel (Vital Signs):** Heart rate (BPM), respiration (RPM), confidence (%) with animated bars
- **Right panel (WiFi Signal):** RSSI, variance, motion power, person count, 2D RSSI sparkline, presence state badge, fall alert
- **Top-right:** Data source badge (DEMO/LIVE), scenario badge, FPS counter, settings gear
- **Bottom:** Capability bar (Pose Estimation, Vital Monitoring, Presence Detection)
- **Bottom-right:** Keyboard shortcut hints
### Settings Dialog (4 Tabs)
Full customization with localStorage persistence and JSON export:
| Tab | Controls |
|-----|----------|
| **Rendering** | Bloom strength/radius/threshold, exposure, vignette, film grain, chromatic aberration |
| **Wireframe** | Bone thickness, joint size, glow intensity, particle trail, wireframe color, joint color, aura opacity |
| **Scene** | Signal field opacity, WiFi wave intensity, room brightness, floor reflection, FOV, orbit speed, grid toggle, room boundary toggle |
| **Data** | Scenario selector (auto-cycle or fixed), cycle speed, data source (demo/WebSocket), WS URL, reset camera, export settings |
### Demo-First with Live Data Path
Four auto-cycling scenarios (30s default, configurable) with 2s cosine crossfade:
| Scenario | Description |
|----------|-------------|
| `empty_room` | Low variance, no presence, flat amplitude, stable RSSI -45dBm |
| `single_breathing` | 1 person, breathing 16 BPM, HR 72 BPM, sinusoidal subcarrier modulation |
| `two_walking` | 2 persons, high motion, Doppler-like shifts, moving signal field peaks |
| `fall_event` | 2s variance spike at t=5s, then stillness, fall flag, confidence drop |
Data contract matches `SensingUpdate` struct from the Rust sensing server. Live WebSocket connection configurable in settings dialog.
### Post-Processing Pipeline
EffectComposer chain: RenderPass → UnrealBloomPass → custom VignetteShader
- **UnrealBloom:** strength 1.0, radius 0.5, threshold 0.25 (configurable)
- **VignetteShader:** warm shadow shift, edge chromatic aberration, film grain
- **Adaptive quality:** Auto-degrades when FPS < 25, restores when FPS > 55
### RuView Foundation Color Palette
| Role | Color | Hex |
|------|-------|-----|
| Background | Deep dark | `#080c14` |
| Primary wireframe | Green glow | `#00d878` |
| Warm accent | Amber | `#ffb020` |
| Signal | Blue | `#2090ff` |
| Heart / joints | Red | `#ff4060` |
| Alert | Crimson | `#ff3040` |
### Technology Choices
| Decision | Rationale |
|----------|-----------|
| Standalone page vs tab | Full-screen immersion, independent loading |
| Room-based vs abstract panels | Practical spatial context for WiFi sensing data |
| Vanilla JS + CDN, no build step | Matches existing `ui/` pattern, served as static files by Axum |
| Custom ShaderMaterial for mist | Per-particle alpha, body-shaped distribution, AdditiveBlending |
| CylinderGeometry tube bones | Visible at any zoom vs thin Line geometry |
| COCO 17-keypoint skeleton | Standard pose format, 16 bone connections |
| localStorage settings | Persistent customization without server round-trip |
| Adaptive quality | 3 levels, auto-switches based on FPS measurement |
### Keyboard Shortcuts
| Key | Action |
|-----|--------|
| `A` | Toggle autopilot orbit |
| `D` | Cycle demo scenario |
| `F` | Toggle FPS counter |
| `S` | Open/close settings |
| `Space` | Pause/resume data |
## Files
| File | Purpose |
|------|---------|
| `ui/observatory.html` | Full-screen entry point with HUD overlay + settings dialog |
| `ui/observatory/js/main.js` | Scene orchestrator (~1,100 lines): room, wireframe, mist, trails, settings, HUD, animation loop |
| `ui/observatory/js/demo-data.js` | 4 scenarios with cosine crossfade, setScenario/setCycleDuration API |
| `ui/observatory/js/nebula-background.js` | Procedural fBM nebula + star field background sphere |
| `ui/observatory/js/post-processing.js` | EffectComposer: UnrealBloom + VignetteShader (chromatic, grain, warmth) |
| `ui/observatory/css/observatory.css` | Foundation color scheme, glass-morphism panels, settings dialog, responsive |
| `ui/index.html` | Modified: added Observatory nav link |
## Consequences
### Positive
- Standalone page does not affect existing dashboard stability
- Demo-first allows offline presentations without hardware
- Same `SensingUpdate` contract enables seamless live WebSocket switch
- Room-based visualization provides intuitive spatial context for WiFi sensing
- Dot-matrix mist gives visual body mass without occluding wireframe
- Full settings customization without code changes (localStorage + JSON export)
- Adaptive quality ensures usability on weaker hardware
- ~20 draw calls keeps performance well within budget
### Negative
- Additional static files served by Axum (minimal overhead)
- Three.js r160 loaded from CDN (no build step, matches existing pattern)
- Settings persistence is per-browser (localStorage, not synced)
### Risks
- CDN dependency for Three.js (mitigated: can vendor locally if needed)
- Post-processing may not work on very old GPUs (mitigated: adaptive quality disables bloom)
## References
- ADR-045: AMOLED display support
- ADR-046: Android TV / Armbian deployment
- Existing `ui/components/scene.js` — Three.js scene pattern
- Existing `ui/components/gaussian-splats.js` — ShaderMaterial pattern
- Existing `ui/services/sensing.service.js` — WebSocket data contract
+140
View File
@@ -0,0 +1,140 @@
# ADR-048: Adaptive CSI Activity Classifier
| Field | Value |
|-------|-------|
| Status | Accepted |
| Date | 2026-03-05 |
| Deciders | ruv |
| Depends on | ADR-024 (AETHER Embeddings), ADR-039 (Edge Processing), ADR-045 (AMOLED Display) |
## Context
WiFi-based activity classification using ESP32 Channel State Information (CSI) relies on hand-tuned thresholds to distinguish between activity states (absent, present_still, present_moving, active). These static thresholds are brittle — they don't account for:
- **Environment-specific signal patterns**: Room geometry, furniture, wall materials, and ESP32 placement all affect how CSI signals respond to human activity.
- **Temporal noise characteristics**: Real ESP32 CSI data at ~10 FPS has significant frame-to-frame jitter that causes classification to jump between states.
- **Vital signs estimation noise**: Heart rate and breathing rate estimates from Goertzel filter banks produce large swings (50+ BPM frame-to-frame) at low confidence levels.
The existing threshold-based approach produces noisy, unstable classifications that degrade the user experience in the Observatory visualization and the main dashboard.
## Decision
### 1. Three-Stage Signal Smoothing Pipeline
All CSI-derived metrics pass through a three-stage pipeline before reaching the UI:
#### Stage 1: Adaptive Baseline Subtraction
- EMA with α=0.003 (~30s time constant) tracks the "quiet room" noise floor
- Only updates during low-motion periods to avoid inflating baseline during activity
- 50-frame warm-up period for initial baseline learning
- Subtracts 70% of baseline from raw motion score to remove environmental drift
#### Stage 2: EMA + Median Filtering
- **Motion score**: Blended from 4 signals (temporal diff 40%, variance 20%, motion band power 25%, change points 15%), then EMA-smoothed with α=0.15
- **Vital signs**: 21-frame sliding window → trimmed mean (drop top/bottom 25%) → EMA with α=0.02 (~5s time constant)
- **Dead-band**: HR won't update unless trimmed mean differs by >2 BPM; BR needs >0.5 BPM
- **Outlier rejection**: HR jumps >8 BPM/frame and BR jumps >2 BPM/frame are discarded
#### Stage 3: Hysteresis Debounce
- Activity state transitions require 4 consecutive frames (~0.4s) of agreement before committing
- Prevents rapid flickering between states
- Independent candidate tracking resets on new direction changes
### 2. Adaptive Classifier Module (`adaptive_classifier.rs`)
A Rust-native environment-tuned classifier that learns from labeled JSONL recordings:
#### Feature Extraction (15 features)
| # | Feature | Source | Discriminative Power |
|---|---------|--------|---------------------|
| 0 | variance | Server | Medium — temporal CSI spread |
| 1 | motion_band_power | Server | Medium — high-frequency subcarrier energy |
| 2 | breathing_band_power | Server | Low — respiratory band energy |
| 3 | spectral_power | Server | Low — mean squared amplitude |
| 4 | dominant_freq_hz | Server | Low — peak subcarrier index |
| 5 | change_points | Server | Medium — threshold crossing count |
| 6 | mean_rssi | Server | Low — received signal strength |
| 7 | amp_mean | Subcarrier | Medium — mean amplitude across 56 subcarriers |
| 8 | amp_std | Subcarrier | **High** — amplitude spread (motion increases spread) |
| 9 | amp_skew | Subcarrier | Medium — asymmetry of amplitude distribution |
| 10 | amp_kurt | Subcarrier | **High** — peakedness (presence creates peaks) |
| 11 | amp_iqr | Subcarrier | Medium — inter-quartile range |
| 12 | amp_entropy | Subcarrier | **High** — spectral entropy (motion increases disorder) |
| 13 | amp_max | Subcarrier | Medium — peak amplitude value |
| 14 | amp_range | Subcarrier | Medium — amplitude dynamic range |
#### Training Algorithm
- **Multiclass logistic regression** with softmax output
- **Mini-batch SGD** (batch size 32, 200 epochs, linear learning rate decay)
- **Z-score normalisation** using global mean/stddev computed from all training data
- Per-class statistics (mean, stddev) stored for Mahalanobis distance fallback
- Deterministic shuffling (LCG PRNG, seed 42) for reproducible results
#### Training Data Pipeline
1. Record labeled CSI sessions via `POST /api/v1/recording/start {"id":"train_<label>"}`
2. Filename-based label assignment: `*empty*`→absent, `*still*`→present_still, `*walking*`→present_moving, `*active*`→active
3. Train via `POST /api/v1/adaptive/train`
4. Model saved to `data/adaptive_model.json`, auto-loaded on server restart
#### Inference Pipeline
1. Extract 15-feature vector from current CSI frame
2. Z-score normalise using stored global mean/stddev
3. Compute softmax probabilities across 4 classes
4. Blend adaptive model confidence (70%) with smoothed threshold confidence (30%)
5. Override classification only when adaptive model is loaded
### 3. API Endpoints
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/v1/adaptive/train` | Train classifier from `train_*` recordings |
| GET | `/api/v1/adaptive/status` | Check model status, accuracy, class stats |
| POST | `/api/v1/adaptive/unload` | Revert to threshold-based classification |
| POST | `/api/v1/recording/start` | Start recording CSI frames (JSONL) |
| POST | `/api/v1/recording/stop` | Stop recording |
| GET | `/api/v1/recording/list` | List available recordings |
### 4. Vital Signs Smoothing
| Parameter | Value | Rationale |
|-----------|-------|-----------|
| Median window | 21 frames | ~2s of history, robust to transients |
| Aggregation | Trimmed mean (middle 50%) | More stable than pure median, less noisy than raw mean |
| EMA alpha | 0.02 | ~5s time constant — readings change very slowly |
| HR dead-band | ±2 BPM | Prevents display creep from micro-fluctuations |
| BR dead-band | ±0.5 BPM | Same for breathing rate |
| HR max jump | 8 BPM/frame | Outlier rejection threshold |
| BR max jump | 2 BPM/frame | Outlier rejection threshold |
## Consequences
### Benefits
- **Stable UI**: Vital signs readings hold steady for 5-10+ seconds instead of jumping every frame
- **Environment adaptation**: Classifier learns the specific room's signal characteristics
- **Graceful fallback**: If no adaptive model is loaded, threshold-based classification with smoothing still works
- **No external dependencies**: Pure Rust implementation, no Python/ML frameworks needed
- **Fast training**: 3,000+ frames train in <1 second on commodity hardware
- **Portable model**: JSON serialisation, loadable on any platform
### Limitations
- **Single-link**: With one ESP32, the feature space is limited. Multi-AP setups (ADR-029) would dramatically improve separability.
- **No temporal features**: Current frame-level classification doesn't use sequence models (LSTM/Transformer). Could be added later.
- **Label quality**: Training accuracy depends heavily on recording quality (distinct activities, actual room vacancy for "empty").
- **Linear classifier**: Logistic regression may underfit non-linear decision boundaries. Could upgrade to 2-layer MLP if needed.
### Future Work
- **Online learning**: Continuously update model weights from user corrections
- **Sequence models**: Use sliding window of N frames as input for temporal pattern recognition
- **Contrastive pretraining**: Leverage ADR-024 AETHER embeddings for self-supervised feature learning
- **Multi-AP fusion**: Use ADR-029 multistatic sensing for richer feature space
- **Edge deployment**: Export learned thresholds to ESP32 firmware (ADR-039 Tier 2) for on-device classification
## Files
| File | Purpose |
|------|---------|
| `crates/wifi-densepose-sensing-server/src/adaptive_classifier.rs` | Adaptive classifier module (feature extraction, training, inference) |
| `crates/wifi-densepose-sensing-server/src/main.rs` | Smoothing pipeline, API endpoints, integration |
| `ui/observatory/js/hud-controller.js` | UI-side lerp smoothing (4% per frame) |
| `data/adaptive_model.json` | Trained model (auto-created by training endpoint) |
| `data/recordings/train_*.jsonl` | Labeled training recordings |
@@ -0,0 +1,122 @@
# ADR-049: Cross-Platform WiFi Interface Detection and Graceful Degradation
| Field | Value |
|-------|-------|
| Status | Proposed |
| Date | 2026-03-06 |
| Deciders | ruv |
| Depends on | ADR-013 (Feature-Level Sensing), ADR-025 (macOS CoreWLAN) |
| Issue | [#148](https://github.com/ruvnet/wifi-densepose/issues/148) |
## Context
Users report `RuntimeError: Cannot read /proc/net/wireless` when running WiFi DensePose in environments where the Linux wireless proc filesystem is unavailable:
- **Docker containers** on macOS/Windows (Linux kernel detected, but no wireless subsystem)
- **WSL2** without USB WiFi passthrough
- **Headless Linux servers** without WiFi hardware
- **Embedded Linux** boards without wireless-extensions support
The current architecture has two layers of defense:
1. **`ws_server.py`** (line 345-355) checks `os.path.exists("/proc/net/wireless")` before instantiating `LinuxWifiCollector` and falls back to `SimulatedCollector` if missing.
2. **`rssi_collector.py`** `LinuxWifiCollector._validate_interface()` (line 178-196) raises a hard `RuntimeError` if `/proc/net/wireless` is missing or the interface isn't listed.
However, there are gaps:
- **Direct usage**: Any code that instantiates `LinuxWifiCollector` directly (outside `ws_server.py`) hits the unguarded `RuntimeError` with no fallback.
- **Error message**: The RuntimeError message tells users to "use SimulatedCollector instead" but doesn't explain how.
- **No auto-detection**: The collector selection logic is duplicated between `ws_server.py` and `install.sh` with no shared platform-detection utility.
- **Partial `/proc/net/wireless`**: The file may exist (e.g., kernel module loaded) but contain no interfaces, producing a confusing "interface not found" error instead of a clean fallback.
## Decision
### 1. Platform-Aware Collector Factory
Introduce a `create_collector()` factory function in `rssi_collector.py` that encapsulates the platform detection and fallback chain:
```python
def create_collector(
preferred: str = "auto",
interface: str = "wlan0",
sample_rate_hz: float = 10.0,
) -> BaseCollector:
"""
Create the best available WiFi collector for the current platform.
Resolution order (when preferred="auto"):
1. ESP32 CSI (if UDP port 5005 is receiving frames)
2. Platform-native WiFi:
- Linux: LinuxWifiCollector (requires /proc/net/wireless + active interface)
- Windows: WindowsWifiCollector (netsh wlan)
- macOS: MacosWifiCollector (CoreWLAN)
3. SimulatedCollector (always available)
Raises nothing — always returns a usable collector.
"""
```
### 2. Soft Validation in LinuxWifiCollector
Replace the hard `RuntimeError` in `_validate_interface()` with a class method that returns availability status without raising:
```python
@classmethod
def is_available(cls, interface: str = "wlan0") -> tuple[bool, str]:
"""Check if Linux WiFi collection is possible. Returns (available, reason)."""
if not os.path.exists("/proc/net/wireless"):
return False, "/proc/net/wireless not found (Docker, WSL, or no wireless subsystem)"
with open("/proc/net/wireless") as f:
content = f.read()
if interface not in content:
names = cls._parse_interface_names(content)
return False, f"Interface '{interface}' not in /proc/net/wireless. Available: {names}"
return True, "ok"
```
The existing `_validate_interface()` continues to raise `RuntimeError` for direct callers who need fail-fast behavior, but `create_collector()` uses `is_available()` to probe without exceptions.
### 3. Structured Fallback Logging
When auto-detection skips a collector, log at `WARNING` level with actionable context:
```
WiFi collector: LinuxWifiCollector unavailable (/proc/net/wireless not found — likely Docker/WSL).
WiFi collector: Falling back to SimulatedCollector. For real sensing, connect ESP32 nodes via UDP:5005.
```
### 4. Consolidate Platform Detection
Remove duplicated platform-detection logic from `ws_server.py` and `install.sh`. Both should use `create_collector()` (Python) or a shared `detect_wifi_platform()` shell function.
## Consequences
### Positive
- **Zero-crash startup**: `create_collector("auto")` never raises — Docker, WSL, and headless users get `SimulatedCollector` automatically with a clear log message.
- **Single detection path**: Platform logic lives in one place (`rssi_collector.py`), reducing drift between `ws_server.py`, `install.sh`, and future entry points.
- **Better DX**: Error messages explain *why* a collector is unavailable and *what to do* (connect ESP32, install WiFi driver, etc.).
### Negative
- **SimulatedCollector may mask hardware issues**: Users with real WiFi hardware that fails detection might unknowingly run on simulated data. Mitigated by the `WARNING`-level log.
- **Breaking change for direct `LinuxWifiCollector` callers**: Code that catches `RuntimeError` from `_validate_interface()` as a signal needs to migrate to `is_available()` or `create_collector()`. This is a minor change — there are no known external consumers.
### Neutral
- `_validate_interface()` behavior is unchanged for existing direct callers — this is additive.
## Implementation Notes
1. Add `create_collector()` and `BaseCollector.is_available()` to `v1/src/sensing/rssi_collector.py`
2. Refactor `ws_server.py` `_init_collector()` to call `create_collector()`
3. Update `install.sh` `detect_wifi_hardware()` to use shared detection logic
4. Add unit tests for each platform path (mock `/proc/net/wireless` presence/absence)
5. Comment on issue #148 with the fix
## References
- Issue #148: RuntimeError: Cannot read /proc/net/wireless
- ADR-013: Feature-Level Sensing on Commodity Gear
- ADR-025: macOS CoreWLAN WiFi Sensing
- [Linux /proc/net/wireless documentation](https://www.kernel.org/doc/html/latest/networking/statistics.html)
@@ -0,0 +1,100 @@
# ADR-050: Quality Engineering Response — Security Hardening & Code Quality
| Field | Value |
|-------|-------|
| Status | Accepted |
| Date | 2026-03-06 |
| Deciders | ruv |
| Depends on | ADR-032 (Multistatic Mesh Security) |
| Issue | [#170](https://github.com/ruvnet/wifi-densepose/issues/170) |
## Context
An independent quality engineering analysis ([issue #170](https://github.com/ruvnet/wifi-densepose/issues/170)) identified 7 critical findings across the Rust codebase. After verification against the source code, the following findings are confirmed and require action:
### Confirmed Critical Findings
| # | Finding | Location | Verified |
|---|---------|----------|----------|
| 1 | Fake HMAC in `secure_tdm.rs` — XOR fold with hardcoded key | `hardware/src/esp32/secure_tdm.rs:253` | YES — comments say "sufficient for testing" |
| 2 | `sensing-server/main.rs` is 3,741 lines — CC=65, god object | `sensing-server/src/main.rs` | YES — confirmed 3,741 lines |
| 3 | WebSocket server has zero authentication | Rust WS codebase | YES — no auth/token checks found |
| 4 | Zero security tests in Rust codebase | Entire workspace | YES — no auth/injection/tampering tests |
| 5 | 54K fps claim has no supporting benchmark | No criterion benchmarks | YES — no benchmarks exist |
### Findings Requiring Further Investigation
| # | Finding | Status |
|---|---------|--------|
| 6 | Unauthenticated OTA firmware endpoint | Not found in Rust code — may be ESP32 C firmware level |
| 7 | WASM upload without mandatory signatures | Needs review of WASM loader |
| 8 | O(n^2) autocorrelation in heart rate detection | Needs profiling to confirm impact |
## Decision
Address findings in 3 priority sprints as recommended by the report.
### Sprint 1: Security (Blocks Deployment)
1. **Replace fake HMAC with real HMAC-SHA256** in `secure_tdm.rs`
- Use the `hmac` + `sha2` crates (already in `Cargo.lock`)
- Remove XOR fold implementation
- Add key derivation (no more hardcoded keys)
2. **Add WebSocket authentication**
- Token-based auth on WS upgrade handshake
- Optional API key for local-network deployments
- Configurable via environment variable
3. **Add security test suite**
- Auth bypass attempts
- Malformed CSI frame injection
- Protocol tampering (TDM beacon replay, nonce reuse)
### Sprint 2: Code Quality & Testability
4. **Decompose `main.rs`** (3,741 lines -> ~14 focused modules)
- Extract HTTP routes, WebSocket handler, CSI pipeline, config, state
- Target: no file over 500 lines
5. **Add criterion benchmarks**
- CSI frame parsing throughput
- Signal processing pipeline latency
- WebSocket broadcast fanout
### Sprint 3: Functional Verification
6. **Vital sign accuracy verification**
- Reference signal tests with known BPM
- False-negative rate measurement
7. **Fix O(n^2) autocorrelation** (if confirmed by profiling)
- Replace brute-force lag with FFT-based autocorrelation
## Consequences
### Positive
- Addresses all critical security findings before any production deployment
- `main.rs` decomposition enables unit testing of server components
- Criterion benchmarks provide verifiable performance claims
- Security test suite prevents regression
### Negative
- Sprint 1 security changes are breaking for any existing TDM mesh deployments (fake HMAC -> real HMAC requires firmware update)
- `main.rs` decomposition is a large refactor with merge conflict risk
### Neutral
- The report correctly identifies that life-safety claims (disaster detection, vital signs) require rigorous verification — this is an ongoing process, not a single sprint
## Acknowledgment
Thanks to [@proffesor-for-testing](https://github.com/proffesor-for-testing) for the thorough 10-report analysis. The full report is archived at the [original gist](https://gist.github.com/proffesor-for-testing/02321e3f272720aa94484fffec6ab19b).
## References
- Issue #170: Quality Engineering Analysis
- ADR-032: Multistatic Mesh Security Hardening
- ADR-028: ESP32 Capability Audit
@@ -0,0 +1,648 @@
# Deployment Platform Domain Model
The Deployment Platform domain covers everything from cross-compiling the sensing server for ARM targets to managing TV box appliances running Armbian: provisioning devices, deploying binaries, configuring kiosk displays, and coordinating multi-room installations. It bridges the gap between the Sensing Server domain (which produces the binary) and the physical hardware it runs on.
This document defines the system using [Domain-Driven Design](https://martinfowler.com/bliki/DomainDrivenDesign.html) (DDD): bounded contexts that own their data and rules, aggregate roots that enforce invariants, value objects that carry meaning, and domain events that connect everything.
**Bounded Contexts:**
| # | Context | Responsibility | Key ADRs | Code |
|---|---------|----------------|----------|------|
| 1 | [Appliance Management](#1-appliance-management-context) | Device inventory, provisioning, health monitoring, OTA updates for TV box deployments | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `scripts/deploy/`, `config/armbian/` |
| 2 | [Cross-Compilation](#2-cross-compilation-context) | Build pipeline for aarch64, binary packaging, CI/CD release artifacts | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `.github/workflows/`, `Cross.toml` |
| 3 | [Display Kiosk](#3-display-kiosk-context) | HDMI output management, Chromium kiosk mode, screen rotation, auto-start | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `config/armbian/kiosk/` |
| 4 | [WiFi CSI Bridge](#4-wifi-csi-bridge-context) | Custom WiFi driver CSI extraction, protocol translation to ESP32 binary format | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md) | `tools/csi-bridge/` |
| 5 | [Network Topology](#5-network-topology-context) | ESP32 mesh ↔ TV box connectivity, dedicated AP mode, multi-room routing | [ADR-046](../adr/ADR-046-android-tv-box-armbian-deployment.md), [ADR-012](../adr/ADR-012-esp32-csi-sensor-mesh.md) | `config/armbian/network/` |
---
## Domain-Driven Design Specification
### Ubiquitous Language
| Term | Definition |
|------|------------|
| **Appliance** | A TV box running Armbian with the sensing server deployed, treated as a managed device in the fleet |
| **Fleet** | The set of all appliances across a multi-room or multi-site installation |
| **Deployment Package** | A self-contained archive containing the sensing-server binary, systemd unit, configuration, and setup script for a target architecture |
| **Kiosk Mode** | Chromium running in full-screen, no-UI mode pointing at `localhost:3000`, auto-started by systemd on HDMI-connected appliances |
| **CSI Bridge** | A userspace daemon that reads CSI data from a patched WiFi driver and re-encodes it as ESP32-compatible UDP frames for the sensing server |
| **Dedicated AP** | An optional `hostapd`-managed WiFi access point on the TV box that creates an isolated network for ESP32 nodes |
| **OTA Update** | Over-the-air binary replacement: download new sensing-server binary, validate checksum, swap via atomic rename, restart service |
| **Reference Device** | A TV box model that has been tested and validated for Armbian + sensing-server deployment (e.g., T95 Max+ / S905X3) |
| **Provisioning** | First-time setup of an appliance: flash Armbian to SD, deploy package, configure WiFi, start services |
| **Health Beacon** | Periodic JSON payload sent by each appliance to a central coordinator (if multi-room) containing uptime, CPU temp, memory usage, inference latency, connected ESP32 count |
---
## Bounded Contexts
### 1. Appliance Management Context
**Responsibility:** Track deployed TV box appliances, provision new devices, monitor health, and coordinate OTA updates across the fleet.
```
+------------------------------------------------------------+
| Appliance Management Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Device | | Provisioning | |
| | Registry | | Service | |
| | (fleet state) | | (first-time | |
| | | | setup) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Health Monitor | |
| | (beacon receiver,| |
| | thermal alerts, | |
| | connectivity) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | OTA Updater | |
| | (binary swap, | |
| | rollback, | |
| | checksum verify)| |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Aggregates:**
```rust
/// Aggregate Root: A managed TV box appliance in the fleet.
/// Identified by MAC address of the primary Ethernet interface.
pub struct Appliance {
/// Unique device identifier (Ethernet MAC address).
pub device_id: DeviceId,
/// Human-readable name (e.g., "living-room", "bedroom-1").
pub name: String,
/// Hardware model (e.g., "T95 Max+ S905X3").
pub hardware_model: HardwareModel,
/// Current deployment state.
pub state: ApplianceState,
/// Installed sensing-server version.
pub server_version: SemanticVersion,
/// Network configuration.
pub network: NetworkConfig,
/// Last received health beacon.
pub last_health: Option<HealthBeacon>,
/// Provisioning timestamp.
pub provisioned_at: DateTime<Utc>,
/// Connected ESP32 node IDs (from last beacon).
pub connected_nodes: Vec<u8>,
}
/// Lifecycle states for an appliance.
pub enum ApplianceState {
/// SD card prepared, not yet booted.
Provisioned,
/// Booted and running, health beacons received.
Online,
/// No health beacon for >5 minutes.
Unreachable,
/// OTA update in progress.
Updating,
/// Manual maintenance / stopped.
Offline,
/// Thermal throttling or hardware issue detected.
Degraded,
}
```
**Value Objects:**
```rust
/// Hardware model specification for a TV box.
pub struct HardwareModel {
/// Marketing name (e.g., "T95 Max+").
pub name: String,
/// SoC identifier (e.g., "Amlogic S905X3").
pub soc: String,
/// WiFi chipset (e.g., "RTL8822CS").
pub wifi_chipset: String,
/// Total RAM in MB.
pub ram_mb: u32,
/// eMMC storage in GB.
pub emmc_gb: u32,
/// Whether CSI bridge is supported for this WiFi chipset.
pub csi_bridge_supported: bool,
/// Armbian device tree name (e.g., "meson-sm1-sei610").
pub armbian_dtb: String,
}
/// Periodic health report from an appliance.
pub struct HealthBeacon {
pub device_id: DeviceId,
pub timestamp: DateTime<Utc>,
pub uptime_secs: u64,
pub cpu_temp_celsius: f32,
pub cpu_usage_percent: f32,
pub memory_used_mb: u32,
pub memory_total_mb: u32,
pub disk_used_percent: f32,
pub inference_latency_ms: f32,
pub connected_esp32_nodes: Vec<u8>,
pub server_version: SemanticVersion,
pub csi_frames_per_sec: f32,
pub websocket_clients: u32,
}
/// Network configuration for an appliance.
pub struct NetworkConfig {
/// Primary IP address (Ethernet or WiFi client).
pub ip_address: IpAddr,
/// Whether the appliance runs a dedicated AP for ESP32 nodes.
pub dedicated_ap: Option<DedicatedApConfig>,
/// UDP port for ESP32 CSI reception.
pub csi_udp_port: u16, // default: 5005
/// HTTP port for sensing server.
pub http_port: u16, // default: 3000
}
/// Configuration for a dedicated WiFi AP hosted by the appliance.
pub struct DedicatedApConfig {
/// SSID for the ESP32 mesh network.
pub ssid: String,
/// WPA2 passphrase.
pub passphrase: String,
/// Channel (1-11 for 2.4 GHz).
pub channel: u8,
/// DHCP range for connected ESP32 nodes.
pub dhcp_range: (IpAddr, IpAddr),
}
/// Unique device identifier (Ethernet MAC).
pub struct DeviceId(pub [u8; 6]);
/// Semantic version for tracking installed software.
pub struct SemanticVersion {
pub major: u16,
pub minor: u16,
pub patch: u16,
pub pre: Option<String>,
}
```
**Domain Services:**
- `ProvisioningService` — Generates Armbian SD card image with pre-configured deployment package, WiFi credentials, and systemd units
- `HealthMonitorService` — Listens for UDP health beacons from fleet appliances, triggers alerts on thermal throttling (>80°C), unreachable (>5 min), or high memory usage (>90%)
- `OtaUpdateService` — Downloads new binary from release URL, verifies SHA-256 checksum, performs atomic swap (`rename(new, current)`), restarts systemd service, rolls back if health beacon fails within 60s
**Invariants:**
- Device ID (MAC address) is immutable after provisioning
- OTA update refuses to proceed if current CPU temperature >75°C (thermal headroom)
- Rollback is automatic if no healthy beacon is received within 60 seconds of restart
- Dedicated AP SSID must not match the upstream WiFi SSID
---
### 2. Cross-Compilation Context
**Responsibility:** Build the sensing-server binary for ARM64 targets, package deployment archives, and manage CI/CD release artifacts.
```
+------------------------------------------------------------+
| Cross-Compilation Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Cross.toml | | GitHub Actions| |
| | (target cfg) | | CI Matrix | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Build Pipeline | |
| | (cross build | |
| | --target | |
| | aarch64-unknown-| |
| | linux-gnu) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | Binary Packager | |
| | (strip, compress,|---> .tar.gz artifact |
| | bundle assets, | |
| | systemd units) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// A packaged deployment archive for a target platform.
pub struct DeploymentPackage {
/// Target triple (e.g., "aarch64-unknown-linux-gnu").
pub target: String,
/// Sensing server binary (stripped).
pub binary: PathBuf,
/// Binary size in bytes.
pub binary_size: u64,
/// SHA-256 checksum of the binary.
pub checksum: String,
/// Systemd service unit file.
pub service_unit: String,
/// Static web UI assets directory.
pub ui_assets: PathBuf,
/// Armbian configuration files (kiosk, network, etc.).
pub config_files: Vec<PathBuf>,
/// Setup script (runs on first boot).
pub setup_script: PathBuf,
/// Version being packaged.
pub version: SemanticVersion,
}
/// Build target specification.
pub struct BuildTarget {
/// Rust target triple.
pub triple: String,
/// CPU architecture description.
pub arch: String,
/// Whether NEON SIMD is available.
pub has_neon: bool,
/// Cross-compilation Docker image.
pub cross_image: String,
/// Binary size limit in bytes.
pub size_limit: u64,
}
```
**Supported Targets:**
| Target Triple | Architecture | Use Case | Size Limit |
|---------------|-------------|----------|------------|
| `x86_64-unknown-linux-gnu` | x86-64 | PC/laptop (existing) | 30 MB |
| `aarch64-unknown-linux-gnu` | ARM64 | TV box (Armbian) | 15 MB |
| `armv7-unknown-linux-gnueabihf` | ARMv7 | Older TV boxes (32-bit) | 12 MB |
| `x86_64-pc-windows-msvc` | x86-64 | Windows (existing) | 30 MB |
**Invariants:**
- Stripped binary must be under size limit for target
- SHA-256 checksum is computed and included in every deployment package
- UI assets are embedded in binary via `include_dir!` or bundled alongside
- No native GPU dependencies — CPU-only inference (candle or ONNX Runtime)
---
### 3. Display Kiosk Context
**Responsibility:** Manage HDMI output on TV box appliances, running Chromium in kiosk mode to display the sensing dashboard full-screen on boot.
```
+------------------------------------------------------------+
| Display Kiosk Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | systemd | | Chromium | |
| | autologin + | | Kiosk Launch | |
| | X11/Wayland | | (full-screen, | |
| | session | | no-UI bars) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Display Manager | |
| | (resolution, | |
| | rotation, | |
| | overscan, | |
| | sleep/wake) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Display configuration for kiosk mode.
pub struct KioskConfig {
/// URL to display (default: "http://localhost:3000").
pub url: String,
/// Screen rotation in degrees (0, 90, 180, 270).
pub rotation: u16,
/// Whether to hide the mouse cursor.
pub hide_cursor: bool,
/// Auto-refresh interval in seconds (0 = disabled).
pub auto_refresh_secs: u32,
/// Display sleep schedule (e.g., off 23:00-06:00).
pub sleep_schedule: Option<SleepSchedule>,
/// Overscan compensation percentage (0-10).
pub overscan_percent: u8,
}
/// Sleep schedule for display power management.
pub struct SleepSchedule {
/// Time to turn display off (HH:MM local time).
pub sleep_time: String,
/// Time to turn display on (HH:MM local time).
pub wake_time: String,
}
```
**Invariants:**
- Chromium kiosk starts only after sensing-server systemd unit is `active`
- If Chromium crashes, systemd restarts it within 5 seconds (`Restart=always`)
- Display sleep/wake uses CEC commands (HDMI-CEC) to control TV power when available
- No browser UI elements are visible (address bar, scrollbars, etc.)
---
### 4. WiFi CSI Bridge Context
**Responsibility:** Extract CSI data from patched WiFi drivers on the TV box and translate it into ESP32-compatible binary frames for the sensing server. This is the Phase 2 custom firmware path.
```
+------------------------------------------------------------+
| WiFi CSI Bridge Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | Patched WiFi | | CSI Reader | |
| | Driver | | (Netlink / | |
| | (kernel space)| | procfs / | |
| | CSI hooks | | UDP socket) | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Protocol | |
| | Translator | |
| | (chipset CSI → | |
| | ESP32 binary | |
| | 0xC5100001) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | UDP Sender | |
| | (localhost:5005) |---> sensing-server |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Raw CSI extraction from a WiFi chipset.
pub struct ChipsetCsiFrame {
/// Source chipset type.
pub chipset: WifiChipset,
/// Timestamp of extraction (kernel monotonic clock).
pub timestamp_us: u64,
/// Number of subcarriers (varies by chipset and bandwidth).
pub n_subcarriers: u16,
/// Number of spatial streams / antennas.
pub n_streams: u8,
/// Channel frequency in MHz.
pub freq_mhz: u16,
/// Bandwidth (20/40/80/160 MHz).
pub bandwidth_mhz: u16,
/// RSSI in dBm.
pub rssi_dbm: i8,
/// Noise floor estimate in dBm.
pub noise_floor_dbm: i8,
/// Complex CSI values (I/Q pairs) per subcarrier per stream.
pub csi_matrix: Vec<Complex<f32>>,
/// Source MAC address (BSSID of the AP being measured).
pub source_mac: [u8; 6],
}
/// Supported WiFi chipsets for CSI extraction.
pub enum WifiChipset {
/// Broadcom BCM43455 via Nexmon CSI patches.
BroadcomBcm43455,
/// Realtek RTL8822CS via modified rtw88 driver.
RealtekRtl8822cs,
/// MediaTek MT7661 via mt76 driver modification.
MediatekMt7661,
}
/// Translated frame in ESP32 binary protocol (ADR-018).
pub struct Esp32CompatFrame {
/// Magic: 0xC5100001
pub magic: u32,
/// Virtual node ID assigned to this WiFi interface.
pub node_id: u8,
/// Number of antennas / spatial streams.
pub n_antennas: u8,
/// Number of subcarriers (resampled to match ESP32 format).
pub n_subcarriers: u8,
/// Frequency in MHz.
pub freq_mhz: u16,
/// Sequence number (monotonic counter).
pub sequence: u32,
/// RSSI in dBm.
pub rssi: i8,
/// Noise floor in dBm.
pub noise_floor: i8,
/// Amplitude values (extracted from complex CSI).
pub amplitudes: Vec<f32>,
/// Phase values (extracted from complex CSI).
pub phases: Vec<f32>,
}
```
**Domain Services:**
- `CsiExtractionService` — Reads raw CSI from patched driver via Netlink socket (BCM43455), procfs (RTL8822CS), or UDP (MT7661)
- `SubcarrierResamplerService` — Resamples chipset-specific subcarrier counts to match ESP32 format (e.g., 256 → 128 via decimation or interpolation)
- `ProtocolTranslatorService` — Converts `ChipsetCsiFrame` to `Esp32CompatFrame` with ADR-018 binary encoding
- `CalibrationService` — Compensates for chipset-specific phase offsets, antenna spacing, and gain differences relative to ESP32 CSI
**Invariants:**
- Bridge assigns virtual `node_id` in range 200-254 (reserved for non-ESP32 sources) to avoid collision with physical ESP32 node IDs (1-199)
- Subcarrier resampling preserves frequency ordering (lowest to highest)
- Phase values are unwrapped before encoding (continuous, not wrapped to ±π)
- Bridge daemon starts only if a compatible patched driver is detected at boot
---
### 5. Network Topology Context
**Responsibility:** Manage network connectivity between ESP32 sensor nodes and TV box appliances, including optional dedicated AP mode and multi-room routing.
```
+------------------------------------------------------------+
| Network Topology Context |
+------------------------------------------------------------+
| |
| +----------------+ +----------------+ |
| | hostapd | | DHCP Server | |
| | (dedicated AP | | (dnsmasq for | |
| | for ESP32 | | ESP32 nodes) | |
| | mesh) | | | |
| +-------+--------+ +-------+--------+ |
| | | |
| +----------+----------+ |
| v |
| +-------------------+ |
| | Topology Manager | |
| | (node discovery, | |
| | IP assignment, | |
| | route config) | |
| +--------+----------+ |
| v |
| +-------------------+ |
| | Firewall Rules | |
| | (iptables/nft: | |
| | allow UDP 5005, | |
| | block external | |
| | access to ESP32 | |
| | subnet) | |
| +-------------------+ |
| |
+------------------------------------------------------------+
```
**Value Objects:**
```rust
/// Network topology for a single-room deployment.
pub struct RoomTopology {
/// Appliance acting as the aggregator.
pub appliance: DeviceId,
/// Whether the appliance runs a dedicated AP.
pub dedicated_ap: bool,
/// Connected ESP32 nodes with their assigned IPs.
pub nodes: Vec<EspNodeConnection>,
/// Upstream network interface (Ethernet or WiFi client).
pub uplink_interface: String,
/// Sensing network interface (dedicated AP or same as uplink).
pub sensing_interface: String,
}
/// An ESP32 node's network connection to the appliance.
pub struct EspNodeConnection {
/// ESP32 node ID (from firmware NVS).
pub node_id: u8,
/// MAC address of the ESP32.
pub mac: [u8; 6],
/// Assigned IP address (via DHCP or static).
pub ip: IpAddr,
/// Last CSI frame received timestamp.
pub last_seen: DateTime<Utc>,
/// Average CSI frames per second from this node.
pub fps: f32,
}
```
**Domain Services:**
- `DedicatedApService` — Configures `hostapd` to create a WPA2 AP on the TV box's WiFi interface, assigns DHCP range via `dnsmasq`, sets up IP forwarding
- `NodeDiscoveryService` — Monitors UDP port 5005 for new ESP32 node IDs, registers them in the topology, alerts on node departure (no frames for >30s)
- `FirewallService` — Configures `nftables`/`iptables` to isolate the ESP32 subnet from the upstream LAN, allowing only UDP 5005 inbound and HTTP 3000 outbound
**Invariants:**
- Dedicated AP uses a separate WiFi interface or virtual interface (not the uplink)
- ESP32 subnet is isolated from upstream LAN by default (firewall rules)
- If dedicated AP is disabled, ESP32 nodes must be on the same LAN subnet as the appliance
- Node discovery does not require mDNS or any discovery protocol — ESP32 nodes are configured with the appliance's IP via NVS provisioning (ADR-044)
---
## Domain Events
| Event | Published By | Consumed By | Payload |
|-------|-------------|-------------|---------|
| `ApplianceProvisioned` | Appliance Mgmt | Fleet Dashboard | `{ device_id, name, hardware_model, ip }` |
| `ApplianceOnline` | Appliance Mgmt | Fleet Dashboard | `{ device_id, server_version, uptime }` |
| `ApplianceUnreachable` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, last_seen, reason }` |
| `ApplianceDegraded` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, cpu_temp, reason }` |
| `OtaUpdateStarted` | Appliance Mgmt | Fleet Dashboard | `{ device_id, from_version, to_version }` |
| `OtaUpdateCompleted` | Appliance Mgmt | Fleet Dashboard | `{ device_id, new_version, duration_secs }` |
| `OtaUpdateRolledBack` | Appliance Mgmt | Fleet Dashboard, Alerting | `{ device_id, attempted_version, rollback_version, reason }` |
| `BinaryBuilt` | Cross-Compilation | Release Pipeline | `{ target, version, binary_size, checksum }` |
| `DeploymentPackageCreated` | Cross-Compilation | Appliance Mgmt | `{ target, version, package_url }` |
| `KioskStarted` | Display Kiosk | Appliance Mgmt | `{ device_id, url, resolution }` |
| `KioskCrashed` | Display Kiosk | Appliance Mgmt | `{ device_id, exit_code, restart_count }` |
| `CsiBridgeStarted` | WiFi CSI Bridge | Appliance Mgmt, Sensing Server | `{ device_id, chipset, virtual_node_id }` |
| `CsiBridgeFailed` | WiFi CSI Bridge | Appliance Mgmt | `{ device_id, chipset, error }` |
| `EspNodeDiscovered` | Network Topology | Appliance Mgmt | `{ appliance_id, node_id, mac, ip }` |
| `EspNodeLost` | Network Topology | Appliance Mgmt, Alerting | `{ appliance_id, node_id, last_seen }` |
| `DedicatedApStarted` | Network Topology | Appliance Mgmt | `{ appliance_id, ssid, channel }` |
---
## Context Map
```
+-------------------+ +---------------------+
| Appliance |--------->| Fleet Dashboard |
| Management | events | (external UI for |
| (fleet state) | -------> | multi-room mgmt) |
+--------+----------+ +---------------------+
|
| provisions, monitors
v
+-------------------+ +---------------------+
| Cross-Compilation |--------->| GitHub Releases |
| (build pipeline) | uploads | (binary artifacts) |
+-------------------+ +---------------------+
|
| provides binary
v
+-------------------+ +---------------------+
| Display Kiosk |--------->| Sensing Server |
| (Chromium on | loads | (upstream domain, |
| HDMI output) | UI from | produces web UI) |
+-------------------+ +----------+----------+
^
+-------------------+ |
| WiFi CSI Bridge |-----UDP 5005------>|
| (patched driver) | ESP32 compat |
+-------------------+ frames |
|
+-------------------+ |
| Network Topology |-----UDP 5005------>|
| (ESP32 mesh | ESP32 frames |
| connectivity) | |
+-------------------+ |
```
**Relationships:**
| Upstream | Downstream | Relationship | Mechanism |
|----------|-----------|--------------|-----------|
| Cross-Compilation | Appliance Mgmt | Supplier-Consumer | Build produces binary; Appliance Mgmt deploys it |
| Appliance Mgmt | Display Kiosk | Customer-Supplier | Appliance Mgmt starts kiosk after server is healthy |
| WiFi CSI Bridge | Sensing Server (external) | Conformist | Bridge adapts its output to match ESP32 binary protocol (ADR-018) |
| Network Topology | Sensing Server (external) | Shared Kernel | Both depend on UDP port 5005 and ESP32 node ID scheme |
| Appliance Mgmt | Network Topology | Customer-Supplier | Appliance config determines whether dedicated AP is enabled |
---
## Anti-Corruption Layers
### ESP32 Protocol ACL (CSI Bridge)
The WiFi CSI Bridge translates chipset-specific CSI formats (Nexmon, rtw88, mt76) into the ESP32 binary protocol (ADR-018). The sensing server never knows whether frames came from a real ESP32 or a TV box WiFi chipset. Virtual node IDs (200-254) prevent collision with physical ESP32 IDs but are otherwise treated identically by the ingestion context.
### Armbian Platform ACL
Appliance Management abstracts over Armbian specifics (device tree names, boot configuration, dtb overlays) through the `HardwareModel` value object. Higher-level contexts (Cross-Compilation, Display Kiosk) depend only on the target triple (`aarch64-unknown-linux-gnu`) and systemd service interface, not on Amlogic/Allwinner/Rockchip kernel specifics.
### Fleet Coordination ACL
For multi-room deployments, each appliance is self-contained (runs its own sensing server, display, and network). The fleet dashboard reads health beacons but never controls individual appliances directly. OTA updates are pulled by each appliance (not pushed), maintaining the appliance as the authority over its own state.
---
## Related
- [ADR-046: Android TV Box / Armbian Deployment](../adr/ADR-046-android-tv-box-armbian-deployment.md) — Primary architectural decision
- [ADR-012: ESP32 CSI Sensor Mesh](../adr/ADR-012-esp32-csi-sensor-mesh.md) — ESP32 mesh network design
- [ADR-018: Dev Implementation](../adr/ADR-018-dev-implementation.md) — ESP32 binary CSI protocol
- [ADR-039: Edge Intelligence](../adr/ADR-039-esp32-edge-intelligence.md) — On-device processing tiers
- [ADR-044: Provisioning Tool](../adr/ADR-044-provisioning-tool-enhancements.md) — NVS provisioning for ESP32 nodes
- [Hardware Platform Domain Model](hardware-platform-domain-model.md) — Upstream domain (ESP32 hardware)
- [Sensing Server Domain Model](sensing-server-domain-model.md) — Upstream domain (server software)
+214 -25
View File
@@ -26,15 +26,20 @@ WiFi DensePose turns commodity WiFi signals into real-time human pose estimation
7. [Web UI](#web-ui)
8. [Vital Sign Detection](#vital-sign-detection)
9. [CLI Reference](#cli-reference)
10. [Training a Model](#training-a-model)
10. [Observatory Visualization](#observatory-visualization)
11. [Adaptive Classifier](#adaptive-classifier)
- [Recording Training Data](#recording-training-data)
- [Training the Model](#training-the-model)
- [Using the Trained Model](#using-the-trained-model)
12. [Training a Model](#training-a-model)
- [CRV Signal-Line Protocol](#crv-signal-line-protocol)
11. [RVF Model Containers](#rvf-model-containers)
12. [Hardware Setup](#hardware-setup)
13. [RVF Model Containers](#rvf-model-containers)
14. [Hardware Setup](#hardware-setup)
- [ESP32-S3 Mesh](#esp32-s3-mesh)
- [Intel 5300 / Atheros NIC](#intel-5300--atheros-nic)
13. [Docker Compose (Multi-Service)](#docker-compose-multi-service)
14. [Troubleshooting](#troubleshooting)
15. [FAQ](#faq)
15. [Docker Compose (Multi-Service)](#docker-compose-multi-service)
16. [Troubleshooting](#troubleshooting)
17. [FAQ](#faq)
---
@@ -42,12 +47,12 @@ WiFi DensePose turns commodity WiFi signals into real-time human pose estimation
| Requirement | Minimum | Recommended |
|-------------|---------|-------------|
| **OS** | Windows 10, macOS 10.15, Ubuntu 18.04 | Latest stable |
| **OS** | Windows 10/11, macOS 10.15, Ubuntu 18.04 | Latest stable |
| **RAM** | 4 GB | 8 GB+ |
| **Disk** | 2 GB free | 5 GB free |
| **Docker** (for Docker path) | Docker 20+ | Docker 24+ |
| **Rust** (for source build) | 1.70+ | 1.85+ |
| **Python** (for legacy v1) | 3.8+ | 3.11+ |
| **Python** (for legacy v1) | 3.10+ | 3.13+ |
**Hardware for live sensing (optional):**
@@ -82,15 +87,15 @@ cd RuView/rust-port/wifi-densepose-rs
# Build
cargo build --release
# Verify (runs 1,100+ tests)
cargo test --workspace
# Verify (runs 1,400+ tests)
cargo test --workspace --no-default-features
```
The compiled binary is at `target/release/sensing-server`.
### From crates.io (Individual Crates)
All 15 crates are published to crates.io at v0.3.0. Add individual crates to your own Rust project:
All 16 crates are published to crates.io at v0.3.0. Add individual crates to your own Rust project:
```bash
# Core types and traits
@@ -113,6 +118,9 @@ cargo add wifi-densepose-ruvector --features crv
# WebAssembly bindings
cargo add wifi-densepose-wasm
# WASM edge runtime (lightweight, for embedded/IoT)
cargo add wifi-densepose-wasm-edge
```
See the full crate list and dependency order in [CLAUDE.md](../CLAUDE.md#crate-publishing-order).
@@ -206,25 +214,27 @@ Default in Docker. Generates synthetic CSI data exercising the full pipeline.
```bash
# Docker
docker run -p 3000:3000 ruvnet/wifi-densepose:latest
# (--source simulated is the default)
# (--source auto is the default; falls back to simulate when no hardware detected)
# From source
./target/release/sensing-server --source simulated --http-port 3000 --ws-port 3001
./target/release/sensing-server --source simulate --http-port 3000 --ws-port 3001
```
### Windows WiFi (RSSI Only)
Uses `netsh wlan` to capture RSSI from nearby access points. No special hardware needed, but capabilities are limited to coarse presence and motion detection (no pose estimation or vital signs).
Uses `netsh wlan` to capture RSSI from nearby access points. No special hardware needed. Supports presence detection, motion classification, and coarse breathing rate estimation. No pose estimation (requires CSI).
```bash
# From source (Windows only)
./target/release/sensing-server --source windows --http-port 3000 --ws-port 3001 --tick-ms 500
./target/release/sensing-server --source wifi --http-port 3000 --ws-port 3001 --tick-ms 500
# Docker (requires --network host on Windows)
docker run --network host ruvnet/wifi-densepose:latest --source windows --tick-ms 500
docker run --network host ruvnet/wifi-densepose:latest --source wifi --tick-ms 500
```
See [Tutorial #36](https://github.com/ruvnet/RuView/issues/36) for a walkthrough.
> **Community verified:** Tested on Windows 10 (10.0.26200) with Intel Wi-Fi 6 AX201 160MHz, Python 3.14, StormFiber 5 GHz network. All 7 tutorial steps passed with stable RSSI readings at -48 dBm. See [Tutorial #36](https://github.com/ruvnet/RuView/issues/36) for the full walkthrough and test results.
**Vital signs from RSSI:** The sensing server now supports breathing rate estimation from RSSI variance patterns (requires stationary subject near AP) and motion classification with confidence scoring. RSSI-based vital sign detection has lower fidelity than ESP32 CSI — it is best for presence detection and coarse motion classification.
### macOS WiFi (RSSI Only)
@@ -315,6 +325,9 @@ Base URL: `http://localhost:3000` (Docker) or `http://localhost:8080` (binary de
| `GET` | `/api/v1/train/status` | Training run status | `{"phase":"idle"}` |
| `POST` | `/api/v1/train/start` | Start a training run | `{"status":"started"}` |
| `POST` | `/api/v1/train/stop` | Stop the active training run | `{"status":"stopped"}` |
| `POST` | `/api/v1/adaptive/train` | Train adaptive classifier from recordings | `{"success":true,"accuracy":0.85}` |
| `GET` | `/api/v1/adaptive/status` | Adaptive model status and accuracy | `{"loaded":true,"accuracy":0.85}` |
| `POST` | `/api/v1/adaptive/unload` | Unload adaptive model | `{"success":true}` |
### Example: Get Vital Signs
@@ -410,9 +423,16 @@ wscat -c ws://localhost:3001/ws/sensing
## Web UI
The built-in Three.js UI is served at `http://localhost:3000/` (Docker) or the configured HTTP port.
The built-in Three.js UI is served at `http://localhost:3000/ui/` (Docker) or the configured HTTP port.
**What you see:**
**Two visualization modes:**
| Page | URL | Purpose |
|------|-----|---------|
| **Dashboard** | `/ui/index.html` | Tabbed monitoring dashboard with body model, signal heatmap, phase plot, vital signs |
| **Observatory** | `/ui/observatory.html` | Immersive 3D room visualization with cinematic lighting and wireframe figures |
**Dashboard panels:**
| Panel | Description |
|-------|-------------|
@@ -423,7 +443,7 @@ The built-in Three.js UI is served at `http://localhost:3000/` (Docker) or the c
| Vital Signs | Live breathing rate (BPM) and heart rate (BPM) |
| Dashboard | System stats, throughput, connected WebSocket clients |
The UI updates in real-time via the WebSocket connection.
Both UIs update in real-time via WebSocket and auto-detect the sensing server on the same origin.
---
@@ -441,6 +461,8 @@ The system extracts breathing rate and heart rate from CSI signal fluctuations u
- Subject within ~3-5 meters of an access point (up to ~8 m with multistatic mesh)
- Relatively stationary subject (large movements mask vital sign oscillations)
**Signal smoothing:** Vital sign estimates pass through a three-stage smoothing pipeline (ADR-048): outlier rejection (±8 BPM HR, ±2 BPM BR per frame), 21-frame trimmed mean, and EMA with α=0.02. This produces stable readings that hold steady for 5-10+ seconds instead of jumping every frame. See [Adaptive Classifier](#adaptive-classifier) for details.
**Simulated mode** produces synthetic vital sign data for testing.
---
@@ -451,7 +473,7 @@ The Rust sensing server binary accepts the following flags:
| Flag | Default | Description |
|------|---------|-------------|
| `--source` | `auto` | Data source: `auto`, `simulated`, `windows`, `esp32` |
| `--source` | `auto` | Data source: `auto`, `simulate`, `wifi`, `esp32` |
| `--http-port` | `8080` | HTTP port for REST API and UI |
| `--ws-port` | `8765` | WebSocket port |
| `--udp-port` | `5005` | UDP port for ESP32 CSI frames |
@@ -472,13 +494,13 @@ The Rust sensing server binary accepts the following flags:
```bash
# Simulated mode with UI (development)
./target/release/sensing-server --source simulated --http-port 3000 --ws-port 3001 --ui-path ../../ui
./target/release/sensing-server --source simulate --http-port 3000 --ws-port 3001 --ui-path ../../ui
# ESP32 hardware mode
./target/release/sensing-server --source esp32 --udp-port 5005
# Windows WiFi RSSI
./target/release/sensing-server --source windows --tick-ms 500
./target/release/sensing-server --source wifi --tick-ms 500
# Run benchmark
./target/release/sensing-server --benchmark
@@ -492,6 +514,149 @@ The Rust sensing server binary accepts the following flags:
---
## Observatory Visualization
The Observatory is an immersive Three.js visualization that renders WiFi sensing data as a cinematic 3D experience. It features room-scale props, wireframe human figures, WiFi signal animations, and a live data HUD.
**URL:** `http://localhost:3000/ui/observatory.html`
**Features:**
| Feature | Description |
|---------|-------------|
| Room scene | Furniture, walls, floor with emissive materials and 6-point lighting |
| Wireframe figures | Up to 4 human skeletons with joint pulsation synced to breathing |
| Signal field | Volumetric WiFi wave visualization |
| Live HUD | Heart rate, breathing rate, confidence, RSSI, motion level |
| Auto-detect | Automatically connects to live ESP32 data when sensing server is running |
| Scenario cycling | 6 preset scenarios with smooth transitions (demo mode) |
**Keyboard shortcuts:**
| Key | Action |
|-----|--------|
| `1-6` | Switch scenario |
| `A` | Toggle auto-cycle |
| `P` | Pause/resume |
| `S` | Open settings |
| `R` | Reset camera |
**Live data auto-detect:** When served by the sensing server, the Observatory probes `/health` on the same origin and automatically connects via WebSocket. The HUD badge switches from `DEMO` to `LIVE`. No configuration needed.
---
## Adaptive Classifier
The adaptive classifier (ADR-048) learns your environment's specific WiFi signal patterns from labeled recordings. It replaces static threshold-based classification with a trained logistic regression model that uses 15 features (7 server-computed + 8 subcarrier-derived statistics).
### Signal Smoothing Pipeline
All CSI-derived metrics pass through a three-stage pipeline before reaching the UI:
| Stage | What It Does | Key Parameters |
|-------|-------------|----------------|
| **Adaptive baseline** | Learns quiet-room noise floor, subtracts drift | α=0.003, 50-frame warm-up |
| **EMA + median filter** | Smooths motion score and vital signs | Motion α=0.15; Vitals: 21-frame trimmed mean, α=0.02 |
| **Hysteresis debounce** | Prevents rapid state flickering | 4 frames (~0.4s) required for state transition |
Vital signs use additional stabilization:
| Parameter | Value | Effect |
|-----------|-------|--------|
| HR dead-band | ±2 BPM | Prevents micro-drift |
| BR dead-band | ±0.5 BPM | Prevents micro-drift |
| HR max jump | 8 BPM/frame | Rejects noise spikes |
| BR max jump | 2 BPM/frame | Rejects noise spikes |
### Recording Training Data
Record labeled CSI sessions while performing distinct activities. Each recording captures full sensing frames (features + raw subcarrier amplitudes) at ~10-25 FPS.
```bash
# 1. Record empty room (leave the room for 30 seconds)
curl -X POST http://localhost:3000/api/v1/recording/start \
-H "Content-Type: application/json" -d '{"id":"train_empty_room"}'
# ... wait 30 seconds ...
curl -X POST http://localhost:3000/api/v1/recording/stop
# 2. Record sitting still (sit near ESP32 for 30 seconds)
curl -X POST http://localhost:3000/api/v1/recording/start \
-H "Content-Type: application/json" -d '{"id":"train_sitting_still"}'
# ... wait 30 seconds ...
curl -X POST http://localhost:3000/api/v1/recording/stop
# 3. Record walking (walk around the room for 30 seconds)
curl -X POST http://localhost:3000/api/v1/recording/start \
-H "Content-Type: application/json" -d '{"id":"train_walking"}'
# ... wait 30 seconds ...
curl -X POST http://localhost:3000/api/v1/recording/stop
# 4. Record active movement (jumping jacks, arm waving for 30 seconds)
curl -X POST http://localhost:3000/api/v1/recording/start \
-H "Content-Type: application/json" -d '{"id":"train_active"}'
# ... wait 30 seconds ...
curl -X POST http://localhost:3000/api/v1/recording/stop
```
Recordings are saved as JSONL files in `data/recordings/`. Filenames must start with `train_` and contain a class keyword:
| Filename pattern | Class |
|-----------------|-------|
| `*empty*` or `*absent*` | absent |
| `*still*` or `*sitting*` | present_still |
| `*walking*` or `*moving*` | present_moving |
| `*active*` or `*exercise*` | active |
### Training the Model
Train the adaptive classifier from your labeled recordings:
```bash
curl -X POST http://localhost:3000/api/v1/adaptive/train
```
The server trains a multiclass logistic regression on 15 features using mini-batch SGD (200 epochs). Training completes in under 1 second for typical recording sets. The trained model is saved to `data/adaptive_model.json` and automatically loaded on server restart.
**Check model status:**
```bash
curl http://localhost:3000/api/v1/adaptive/status
```
**Unload the model (revert to threshold-based classification):**
```bash
curl -X POST http://localhost:3000/api/v1/adaptive/unload
```
### Using the Trained Model
Once trained, the adaptive model runs automatically:
1. Each CSI frame is classified using the learned weights instead of static thresholds
2. Model confidence is blended with smoothed threshold confidence (70/30 split)
3. The model persists across server restarts (loaded from `data/adaptive_model.json`)
**Tips for better accuracy:**
- Record with clearly distinct activities (actually leave the room for "empty")
- Record 30-60 seconds per activity (more data = better model)
- Re-record and retrain if you move the ESP32 or rearrange the room
- The model is environment-specific — retrain when the physical setup changes
### Adaptive Classifier API
| Method | Endpoint | Description |
|--------|----------|-------------|
| `POST` | `/api/v1/adaptive/train` | Train from `train_*` recordings |
| `GET` | `/api/v1/adaptive/status` | Model status, accuracy, class stats |
| `POST` | `/api/v1/adaptive/unload` | Unload model, revert to thresholds |
| `POST` | `/api/v1/recording/start` | Start recording CSI frames |
| `POST` | `/api/v1/recording/stop` | Stop recording |
| `GET` | `/api/v1/recording/list` | List recordings |
---
## Training a Model
The training pipeline is implemented in pure Rust (7,832 lines, zero external ML dependencies).
@@ -805,13 +970,28 @@ rustc --version
### Windows: RSSI mode shows no data
Run the terminal as Administrator (required for `netsh wlan` access).
Run the terminal as Administrator (required for `netsh wlan` access). Verified working on Windows 10 and 11 with Intel AX201 and Intel BE201 adapters.
### Vital signs show 0 BPM
- Vital sign detection requires CSI-capable hardware (ESP32 or research NIC)
- RSSI-only mode (Windows WiFi) does not have sufficient resolution for vital signs
- In simulated mode, synthetic vital signs are generated after a few seconds of warm-up
- With real ESP32 data, vital signs take ~5 seconds to stabilize (smoothing pipeline warm-up)
### Vital signs jumping around
The server applies a 3-stage smoothing pipeline (ADR-048). If readings are still unstable:
- Ensure the subject is relatively still (large movements mask vital sign oscillations)
- Train the adaptive classifier for your specific environment: `curl -X POST http://localhost:3000/api/v1/adaptive/train`
- Check signal quality: `curl http://localhost:3000/api/v1/sensing/latest` — look for `signal_quality > 0.4`
### Observatory shows DEMO instead of LIVE
- Verify the sensing server is running: `curl http://localhost:3000/health`
- Access Observatory via the server URL: `http://localhost:3000/ui/observatory.html` (not a file:// URL)
- Hard refresh with Ctrl+Shift+R to clear cached settings
- The auto-detect probes `/health` on the same origin — cross-origin won't work
---
@@ -838,11 +1018,20 @@ The system uses WiFi radio signals, not cameras. No images or video are captured
**Q: What's the Python vs Rust difference?**
The Rust implementation (v2) is 810x faster than Python (v1) for the full CSI pipeline. The Docker image is 132 MB vs 569 MB. Rust is the primary and recommended runtime. Python v1 remains available for legacy workflows.
**Q: Can I use an ESP8266 instead of ESP32-S3?**
No. The ESP8266 does not expose WiFi Channel State Information (CSI) through its SDK, has insufficient RAM (~80 KB vs 512 KB), and runs a single-core 80 MHz CPU that cannot handle the signal processing pipeline. The ESP32-S3 is the minimum supported CSI capture device. See [Issue #138](https://github.com/ruvnet/RuView/issues/138) for alternatives including using cheap Android TV boxes as aggregation hubs.
**Q: Does the Windows WiFi tutorial work on Windows 10?**
Yes. Community-tested on Windows 10 (build 26200) with an Intel Wi-Fi 6 AX201 160MHz adapter on a 5 GHz network. All 7 tutorial steps passed with Python 3.14. See [Issue #36](https://github.com/ruvnet/RuView/issues/36) for full test results.
**Q: Can I run the sensing server on an ARM device (Raspberry Pi, TV box)?**
ARM64 deployment is planned ([ADR-046](adr/ADR-046-android-tv-box-armbian-deployment.md)) but not yet available as a pre-built binary. You can cross-compile from source using `cross build --release --target aarch64-unknown-linux-gnu -p wifi-densepose-sensing-server` if you have the Rust cross-compilation toolchain set up.
---
## Further Reading
- [Architecture Decision Records](../docs/adr/) - 43 ADRs covering all design decisions
- [Architecture Decision Records](../docs/adr/) - 48 ADRs covering all design decisions
- [WiFi-Mat Disaster Response Guide](wifi-mat-user-guide.md) - Search & rescue module
- [Build Guide](build-guide.md) - Detailed build instructions
- [RuVector](https://github.com/ruvnet/ruvector) - Signal intelligence crate ecosystem
+18 -5
View File
@@ -1,6 +1,19 @@
idf_component_register(
SRCS "main.c" "csi_collector.c" "stream_sender.c" "nvs_config.c"
"edge_processing.c" "ota_update.c" "power_mgmt.c"
"wasm_runtime.c" "wasm_upload.c" "rvf_parser.c"
INCLUDE_DIRS "."
set(SRCS
"main.c" "csi_collector.c" "stream_sender.c" "nvs_config.c"
"edge_processing.c" "ota_update.c" "power_mgmt.c"
"wasm_runtime.c" "wasm_upload.c" "rvf_parser.c"
)
set(REQUIRES "")
# ADR-045: AMOLED display support (compile-time optional)
if(CONFIG_DISPLAY_ENABLE)
list(APPEND SRCS "display_hal.c" "display_ui.c" "display_task.c")
set(REQUIRES esp_lcd esp_lcd_touch lvgl)
endif()
idf_component_register(
SRCS ${SRCS}
INCLUDE_DIRS "."
REQUIRES ${REQUIRES}
)
@@ -85,6 +85,87 @@ menu "Edge Intelligence (ADR-039)"
endmenu
menu "AMOLED Display (ADR-045)"
config DISPLAY_ENABLE
bool "Enable AMOLED display support"
default y
help
Enable RM67162 QSPI AMOLED display and LVGL UI.
Auto-detects at boot; gracefully skips if no display hardware.
Requires SPIRAM for frame buffers.
config DISPLAY_FPS_LIMIT
int "Display refresh rate limit (FPS)"
default 30
range 10 60
depends on DISPLAY_ENABLE
help
Maximum display refresh rate. Lower values save CPU.
config DISPLAY_BRIGHTNESS
int "Default backlight brightness (%)"
default 80
range 0 100
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_CS
int "QSPI CS GPIO"
default 6
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_CLK
int "QSPI CLK GPIO"
default 47
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D0
int "QSPI D0 GPIO"
default 18
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D1
int "QSPI D1 GPIO"
default 7
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D2
int "QSPI D2 GPIO"
default 48
depends on DISPLAY_ENABLE
config DISPLAY_QSPI_D3
int "QSPI D3 GPIO"
default 5
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_SDA
int "Touch I2C SDA GPIO"
default 3
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_SCL
int "Touch I2C SCL GPIO"
default 2
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_INT
int "Touch INT GPIO"
default 21
depends on DISPLAY_ENABLE
config DISPLAY_TOUCH_RST
int "Touch RST GPIO"
default 17
depends on DISPLAY_ENABLE
config DISPLAY_BL_PIN
int "Backlight PWM GPIO"
default 38
depends on DISPLAY_ENABLE
endmenu
menu "WASM Programmable Sensing (ADR-040)"
config WASM_ENABLE
+382
View File
@@ -0,0 +1,382 @@
/**
* @file display_hal.c
* @brief ADR-045: SH8601 QSPI AMOLED HAL for Waveshare ESP32-S3-Touch-AMOLED-1.8.
*
* Uses ESP-IDF esp_lcd_panel_io_spi in QSPI mode (quad_mode=true, lcd_cmd_bits=32).
* The panel_io layer handles the 0x02/0x32 QSPI command encoding.
*
* Hardware: SH8601 368x448, FT3168 touch, TCA9554 I/O expander for power/reset.
*
* Pin assignments (Waveshare ESP32-S3-Touch-AMOLED-1.8):
* QSPI: CS=12, CLK=11, D0=4, D1=5, D2=6, D3=7
* I2C: SDA=15, SCL=14 (shared: touch FT3168 + TCA9554 expander)
* Touch INT=21
*/
#include "display_hal.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_lcd_panel_io.h"
#include "driver/spi_master.h"
#include "driver/gpio.h"
#include "driver/i2c.h"
#include "esp_heap_caps.h"
static const char *TAG = "disp_hal";
/* ---- QSPI Pin Definitions (Waveshare board) ---- */
#define DISP_QSPI_CS 12
#define DISP_QSPI_CLK 11
#define DISP_QSPI_D0 4
#define DISP_QSPI_D1 5
#define DISP_QSPI_D2 6
#define DISP_QSPI_D3 7
/* ---- I2C (shared: touch + TCA9554 expander) ---- */
#define I2C_SDA 15
#define I2C_SCL 14
#define TOUCH_INT_PIN 21
#define I2C_MASTER_NUM I2C_NUM_0
#define I2C_MASTER_FREQ_HZ 400000
/* ---- TCA9554 I/O expander ---- */
#define TCA9554_ADDR 0x20
#define TCA9554_REG_OUTPUT 0x01
#define TCA9554_REG_CONFIG 0x03
/* ---- FT3168 touch controller ---- */
#define FT3168_ADDR 0x38
/* ---- Display dimensions ---- */
#define DISP_H_RES 368
#define DISP_V_RES 448
/* ---- QSPI opcodes (packed into lcd_cmd bits [31:24]) ---- */
#define LCD_OPCODE_WRITE_CMD 0x02
#define LCD_OPCODE_WRITE_COLOR 0x32
/* ---- State ---- */
static esp_lcd_panel_io_handle_t s_io_handle = NULL;
static bool s_i2c_initialized = false;
static bool s_touch_initialized = false;
/* ---- I2C helpers ---- */
static esp_err_t i2c_write_reg(uint8_t dev_addr, uint8_t reg, const uint8_t *data, size_t len)
{
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, reg, true);
if (data && len > 0) {
i2c_master_write(cmd, data, len, true);
}
i2c_master_stop(cmd);
esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100));
i2c_cmd_link_delete(cmd);
return ret;
}
static esp_err_t i2c_read_reg(uint8_t dev_addr, uint8_t reg, uint8_t *data, size_t len)
{
i2c_cmd_handle_t cmd = i2c_cmd_link_create();
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_WRITE, true);
i2c_master_write_byte(cmd, reg, true);
i2c_master_start(cmd);
i2c_master_write_byte(cmd, (dev_addr << 1) | I2C_MASTER_READ, true);
i2c_master_read(cmd, data, len, I2C_MASTER_LAST_NACK);
i2c_master_stop(cmd);
esp_err_t ret = i2c_master_cmd_begin(I2C_MASTER_NUM, cmd, pdMS_TO_TICKS(100));
i2c_cmd_link_delete(cmd);
return ret;
}
static esp_err_t init_i2c_bus(void)
{
if (s_i2c_initialized) return ESP_OK;
i2c_config_t i2c_cfg = {
.mode = I2C_MODE_MASTER,
.sda_io_num = I2C_SDA,
.scl_io_num = I2C_SCL,
.sda_pullup_en = GPIO_PULLUP_ENABLE,
.scl_pullup_en = GPIO_PULLUP_ENABLE,
.master.clk_speed = I2C_MASTER_FREQ_HZ,
};
esp_err_t ret = i2c_param_config(I2C_MASTER_NUM, &i2c_cfg);
if (ret != ESP_OK) return ret;
ret = i2c_driver_install(I2C_MASTER_NUM, I2C_MODE_MASTER, 0, 0, 0);
if (ret != ESP_OK) return ret;
s_i2c_initialized = true;
ESP_LOGI(TAG, "I2C bus init OK (SDA=%d, SCL=%d)", I2C_SDA, I2C_SCL);
return ESP_OK;
}
/* ---- TCA9554 I/O expander: toggle pins for display power/reset ---- */
static esp_err_t tca9554_init_display_power(void)
{
/* Set pins 0, 1, 2 as outputs */
uint8_t cfg = 0xF8;
esp_err_t ret = i2c_write_reg(TCA9554_ADDR, TCA9554_REG_CONFIG, &cfg, 1);
if (ret != ESP_OK) {
ESP_LOGW(TAG, "TCA9554 not found at 0x%02X: %s", TCA9554_ADDR, esp_err_to_name(ret));
return ret;
}
/* Set pins 0,1,2 LOW (reset state) */
uint8_t out = 0x00;
i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1);
vTaskDelay(pdMS_TO_TICKS(200));
/* Set pins 0,1,2 HIGH (power on + release reset) */
out = 0x07;
i2c_write_reg(TCA9554_ADDR, TCA9554_REG_OUTPUT, &out, 1);
vTaskDelay(pdMS_TO_TICKS(200));
ESP_LOGI(TAG, "TCA9554 display power/reset toggled");
return ESP_OK;
}
/* ---- Panel IO helpers: send commands via esp_lcd QSPI panel IO ---- */
static esp_err_t panel_write_cmd(uint8_t dcs_cmd, const void *data, size_t data_len)
{
/* Pack as 32-bit lcd_cmd: [31:24]=opcode, [23:8]=dcs_cmd, [7:0]=0 */
uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_CMD << 24) | ((uint32_t)dcs_cmd << 8);
return esp_lcd_panel_io_tx_param(s_io_handle, (int)lcd_cmd, data, data_len);
}
static esp_err_t panel_write_color(const void *color_data, size_t data_len)
{
/* RAMWR (0x2C) packed as 32-bit lcd_cmd with quad opcode */
uint32_t lcd_cmd = ((uint32_t)LCD_OPCODE_WRITE_COLOR << 24) | (0x2C << 8);
return esp_lcd_panel_io_tx_color(s_io_handle, (int)lcd_cmd, color_data, data_len);
}
/* ---- SH8601 init sequence (from Waveshare reference) ---- */
typedef struct {
uint8_t cmd;
uint8_t data[4];
uint8_t data_len;
uint16_t delay_ms;
} sh8601_init_cmd_t;
static const sh8601_init_cmd_t sh8601_init_cmds[] = {
{0x11, {0x00}, 0, 120}, /* Sleep Out + 120ms */
{0x44, {0x01, 0xD1}, 2, 0}, /* Partial area */
{0x35, {0x00}, 1, 0}, /* Tearing Effect ON */
{0x53, {0x20}, 1, 10}, /* Write CTRL Display */
{0x2A, {0x00, 0x00, 0x01, 0x6F}, 4, 0}, /* CASET: 0-367 */
{0x2B, {0x00, 0x00, 0x01, 0xBF}, 4, 0}, /* RASET: 0-447 */
{0x51, {0x00}, 1, 10}, /* Brightness: 0 */
{0x29, {0x00}, 0, 10}, /* Display ON */
{0x51, {0xFF}, 1, 0}, /* Brightness: max */
{0x00, {0x00}, 0xFF, 0}, /* End sentinel */
};
static esp_err_t send_init_sequence(void)
{
for (int i = 0; sh8601_init_cmds[i].data_len != 0xFF; i++) {
const sh8601_init_cmd_t *cmd = &sh8601_init_cmds[i];
esp_err_t ret = panel_write_cmd(
cmd->cmd,
cmd->data_len > 0 ? cmd->data : NULL,
cmd->data_len);
if (ret != ESP_OK) {
ESP_LOGE(TAG, "CMD 0x%02X failed: %s", cmd->cmd, esp_err_to_name(ret));
return ret;
}
if (cmd->delay_ms > 0) {
vTaskDelay(pdMS_TO_TICKS(cmd->delay_ms));
}
}
return ESP_OK;
}
/* ---- Public API ---- */
esp_err_t display_hal_init_panel(void)
{
ESP_LOGI(TAG, "Initializing Waveshare AMOLED 1.8\" (SH8601 368x448)...");
/* Step 1: Init I2C bus */
esp_err_t ret = init_i2c_bus();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "I2C bus init failed");
return ESP_ERR_NOT_FOUND;
}
/* Step 2: TCA9554 display power/reset (optional — only present on Waveshare board) */
ret = tca9554_init_display_power();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "TCA9554 not found — assuming display power is always-on (direct wiring)");
/* Continue without TCA9554 — the display may be powered directly */
}
/* Step 3: Initialize SPI bus */
spi_bus_config_t bus_cfg = {
.sclk_io_num = DISP_QSPI_CLK,
.data0_io_num = DISP_QSPI_D0,
.data1_io_num = DISP_QSPI_D1,
.data2_io_num = DISP_QSPI_D2,
.data3_io_num = DISP_QSPI_D3,
.max_transfer_sz = DISP_H_RES * DISP_V_RES * 2,
};
ret = spi_bus_initialize(SPI2_HOST, &bus_cfg, SPI_DMA_CH_AUTO);
if (ret != ESP_OK) {
ESP_LOGW(TAG, "SPI bus init failed: %s", esp_err_to_name(ret));
return ESP_ERR_NOT_FOUND;
}
/* Step 4: Create panel IO with QSPI mode */
esp_lcd_panel_io_spi_config_t io_config = {
.dc_gpio_num = -1, /* No DC pin in QSPI mode */
.cs_gpio_num = DISP_QSPI_CS,
.pclk_hz = 40 * 1000 * 1000,
.lcd_cmd_bits = 32, /* 32-bit command: [opcode|dcs_cmd|0x00] */
.lcd_param_bits = 8,
.spi_mode = 0,
.trans_queue_depth = 10,
.flags = {
.quad_mode = true,
},
};
ret = esp_lcd_new_panel_io_spi((esp_lcd_spi_bus_handle_t)SPI2_HOST, &io_config, &s_io_handle);
if (ret != ESP_OK) {
ESP_LOGE(TAG, "Panel IO init failed: %s", esp_err_to_name(ret));
spi_bus_free(SPI2_HOST);
return ESP_ERR_NOT_FOUND;
}
ESP_LOGI(TAG, "QSPI panel IO created (40MHz, quad mode)");
/* Step 5: Send SH8601 init sequence */
ret = send_init_sequence();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "SH8601 init sequence failed");
esp_lcd_panel_io_del(s_io_handle);
spi_bus_free(SPI2_HOST);
s_io_handle = NULL;
return ESP_ERR_NOT_FOUND;
}
/* Step 6: Draw test pattern — cyan bar at top */
ESP_LOGI(TAG, "Drawing test pattern...");
uint16_t *line_buf = heap_caps_malloc(DISP_H_RES * 2, MALLOC_CAP_DMA);
if (line_buf) {
uint8_t caset[4] = {0, 0, (DISP_H_RES - 1) >> 8, (DISP_H_RES - 1) & 0xFF};
uint8_t raset[4] = {0, 0, (DISP_V_RES - 1) >> 8, (DISP_V_RES - 1) & 0xFF};
panel_write_cmd(0x2A, caset, 4);
panel_write_cmd(0x2B, raset, 4);
for (int y = 0; y < DISP_V_RES; y++) {
uint16_t color = (y < 30) ? 0x07FF : 0x0841;
for (int x = 0; x < DISP_H_RES; x++) {
line_buf[x] = color;
}
panel_write_color(line_buf, DISP_H_RES * 2);
}
free(line_buf);
ESP_LOGI(TAG, "Test pattern drawn");
}
ESP_LOGI(TAG, "SH8601 panel init OK (%dx%d)", DISP_H_RES, DISP_V_RES);
return ESP_OK;
}
void display_hal_draw(int x_start, int y_start, int x_end, int y_end,
const void *color_data)
{
if (!s_io_handle) return;
/* SH8601 requires coordinates divisible by 2 */
x_start &= ~1;
y_start &= ~1;
if (x_end & 1) x_end++;
if (y_end & 1) y_end++;
if (x_end > DISP_H_RES) x_end = DISP_H_RES;
if (y_end > DISP_V_RES) y_end = DISP_V_RES;
uint8_t caset[4] = {
(x_start >> 8) & 0xFF, x_start & 0xFF,
((x_end - 1) >> 8) & 0xFF, (x_end - 1) & 0xFF,
};
panel_write_cmd(0x2A, caset, 4);
uint8_t raset[4] = {
(y_start >> 8) & 0xFF, y_start & 0xFF,
((y_end - 1) >> 8) & 0xFF, (y_end - 1) & 0xFF,
};
panel_write_cmd(0x2B, raset, 4);
size_t len = (x_end - x_start) * (y_end - y_start) * 2;
panel_write_color(color_data, len);
}
esp_err_t display_hal_init_touch(void)
{
ESP_LOGI(TAG, "Probing FT3168 touch controller...");
if (!s_i2c_initialized) {
esp_err_t ret = init_i2c_bus();
if (ret != ESP_OK) return ESP_ERR_NOT_FOUND;
}
gpio_config_t int_cfg = {
.pin_bit_mask = (1ULL << TOUCH_INT_PIN),
.mode = GPIO_MODE_INPUT,
.pull_up_en = GPIO_PULLUP_ENABLE,
.intr_type = GPIO_INTR_DISABLE,
};
gpio_config(&int_cfg);
uint8_t chip_id = 0;
esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0xA8, &chip_id, 1);
if (ret != ESP_OK || chip_id == 0x00 || chip_id == 0xFF) {
ESP_LOGW(TAG, "FT3168 not found (ret=%s, id=0x%02X)", esp_err_to_name(ret), chip_id);
return ESP_ERR_NOT_FOUND;
}
s_touch_initialized = true;
ESP_LOGI(TAG, "FT3168 touch init OK (chip_id=0x%02X)", chip_id);
return ESP_OK;
}
bool display_hal_touch_read(uint16_t *x, uint16_t *y)
{
if (!s_touch_initialized) return false;
uint8_t buf[7] = {0};
esp_err_t ret = i2c_read_reg(FT3168_ADDR, 0x01, buf, 7);
if (ret != ESP_OK) return false;
uint8_t num_points = buf[1];
if (num_points == 0 || num_points > 2) return false;
*x = ((buf[2] & 0x0F) << 8) | buf[3];
*y = ((buf[4] & 0x0F) << 8) | buf[5];
return true;
}
void display_hal_set_brightness(uint8_t percent)
{
if (!s_io_handle) return;
if (percent > 100) percent = 100;
uint8_t val = (uint8_t)((uint32_t)percent * 255 / 100);
panel_write_cmd(0x51, &val, 1);
}
#endif /* CONFIG_DISPLAY_ENABLE */
@@ -0,0 +1,71 @@
/**
* @file display_hal.h
* @brief ADR-045: RM67162 QSPI AMOLED + CST816S touch HAL.
*
* Hardware abstraction for the LilyGO T-Display-S3 AMOLED panel.
* Probes hardware at boot; returns ESP_ERR_NOT_FOUND if absent.
*/
#ifndef DISPLAY_HAL_H
#define DISPLAY_HAL_H
#include <stdbool.h>
#include <stdint.h>
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* Probe and initialize the RM67162 QSPI AMOLED panel.
*
* Configures QSPI bus, sends panel init sequence, and fills
* the screen with dark background to confirm it works.
* Returns ESP_ERR_NOT_FOUND if the panel does not respond.
*
* @return ESP_OK on success, ESP_ERR_NOT_FOUND if no display detected.
*/
esp_err_t display_hal_init_panel(void);
/**
* Draw a rectangle of pixels to the AMOLED.
* Sends CASET + RASET + RAMWR directly via QSPI.
*
* @param x_start Left column (inclusive).
* @param y_start Top row (inclusive).
* @param x_end Right column (exclusive).
* @param y_end Bottom row (exclusive).
* @param color_data RGB565 pixel data, (x_end-x_start)*(y_end-y_start) pixels.
*/
void display_hal_draw(int x_start, int y_start, int x_end, int y_end,
const void *color_data);
/**
* Probe and initialize the CST816S capacitive touch controller.
*
* @return ESP_OK on success, ESP_ERR_NOT_FOUND if no touch IC detected.
*/
esp_err_t display_hal_init_touch(void);
/**
* Read touch point (non-blocking).
*
* @param[out] x Touch X coordinate (0..535).
* @param[out] y Touch Y coordinate (0..239).
* @return true if touch is active, false if released.
*/
bool display_hal_touch_read(uint16_t *x, uint16_t *y);
/**
* Set AMOLED brightness via MIPI DCS command.
*
* @param percent Brightness 0-100.
*/
void display_hal_set_brightness(uint8_t percent);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_HAL_H */
+169
View File
@@ -0,0 +1,169 @@
/**
* @file display_task.c
* @brief ADR-045: FreeRTOS display task LVGL pump on Core 0, priority 1.
*
* Gracefully skips if RM67162 panel or SPIRAM is absent.
* Reads from edge_get_vitals() / edge_get_multi_person() (thread-safe).
*/
#include "display_task.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_log.h"
#include "esp_heap_caps.h"
#include "lvgl.h"
#include "display_hal.h"
#include "display_ui.h"
#define DISP_H_RES 368
#define DISP_V_RES 448
static const char *TAG = "disp_task";
/* ---- Config ---- */
#ifdef CONFIG_DISPLAY_FPS_LIMIT
#define DISP_FPS_LIMIT CONFIG_DISPLAY_FPS_LIMIT
#else
#define DISP_FPS_LIMIT 30
#endif
#define DISP_TASK_STACK (8 * 1024)
#define DISP_TASK_PRIORITY 1
#define DISP_TASK_CORE 0
#define DISP_BUF_LINES 40
/* ---- LVGL flush callback — calls display_hal_draw directly ---- */
static void lvgl_flush_cb(lv_disp_drv_t *drv, const lv_area_t *area, lv_color_t *color_p)
{
display_hal_draw(area->x1, area->y1, area->x2 + 1, area->y2 + 1, color_p);
lv_disp_flush_ready(drv);
}
/* ---- LVGL touch input callback ---- */
static void lvgl_touch_cb(lv_indev_drv_t *drv, lv_indev_data_t *data)
{
uint16_t x, y;
if (display_hal_touch_read(&x, &y)) {
data->point.x = x;
data->point.y = y;
data->state = LV_INDEV_STATE_PRESSED;
} else {
data->state = LV_INDEV_STATE_RELEASED;
}
}
/* ---- Display task ---- */
static void display_task(void *arg)
{
const TickType_t frame_period = pdMS_TO_TICKS(1000 / DISP_FPS_LIMIT);
ESP_LOGI(TAG, "Display task running on Core %d, %d fps limit",
xPortGetCoreID(), DISP_FPS_LIMIT);
display_ui_create(lv_scr_act());
TickType_t last_wake = xTaskGetTickCount();
while (1) {
display_ui_update();
lv_timer_handler();
vTaskDelayUntil(&last_wake, frame_period);
}
}
/* ---- Public API ---- */
esp_err_t display_task_start(void)
{
ESP_LOGI(TAG, "Initializing display subsystem...");
bool use_psram = false;
#if CONFIG_SPIRAM
size_t psram_free = heap_caps_get_free_size(MALLOC_CAP_SPIRAM);
if (psram_free >= 64 * 1024) {
use_psram = true;
ESP_LOGI(TAG, "PSRAM available: %u KB — using PSRAM buffers", (unsigned)(psram_free / 1024));
} else {
ESP_LOGW(TAG, "PSRAM too small (%u bytes) — falling back to internal DMA memory", (unsigned)psram_free);
}
#else
ESP_LOGW(TAG, "SPIRAM not enabled — using internal DMA memory (smaller buffers)");
#endif
/* Probe display hardware */
esp_err_t ret = display_hal_init_panel();
if (ret != ESP_OK) {
ESP_LOGW(TAG, "Display not available — running headless");
return ESP_OK;
}
/* Init touch (optional) */
esp_err_t touch_ret = display_hal_init_touch();
/* Initialize LVGL */
lv_init();
/* Double-buffered draw buffers — prefer PSRAM, fall back to internal DMA */
size_t buf_lines = use_psram ? DISP_BUF_LINES : 10; /* Smaller buffers without PSRAM */
size_t buf_size = DISP_H_RES * buf_lines * sizeof(lv_color_t);
uint32_t alloc_caps = use_psram ? MALLOC_CAP_SPIRAM : (MALLOC_CAP_DMA | MALLOC_CAP_INTERNAL);
lv_color_t *buf1 = heap_caps_malloc(buf_size, alloc_caps);
lv_color_t *buf2 = heap_caps_malloc(buf_size, alloc_caps);
if (!buf1 || !buf2) {
ESP_LOGE(TAG, "Failed to allocate LVGL buffers (%u bytes, caps=0x%lx)",
(unsigned)buf_size, (unsigned long)alloc_caps);
if (buf1) free(buf1);
if (buf2) free(buf2);
return ESP_OK;
}
ESP_LOGI(TAG, "LVGL buffers: 2x %u bytes (%u lines, %s)",
(unsigned)buf_size, (unsigned)buf_lines, use_psram ? "PSRAM" : "internal DMA");
static lv_disp_draw_buf_t draw_buf;
lv_disp_draw_buf_init(&draw_buf, buf1, buf2, DISP_H_RES * buf_lines);
static lv_disp_drv_t disp_drv;
lv_disp_drv_init(&disp_drv);
disp_drv.hor_res = DISP_H_RES;
disp_drv.ver_res = DISP_V_RES;
disp_drv.flush_cb = lvgl_flush_cb;
disp_drv.draw_buf = &draw_buf;
lv_disp_drv_register(&disp_drv);
if (touch_ret == ESP_OK) {
static lv_indev_drv_t indev_drv;
lv_indev_drv_init(&indev_drv);
indev_drv.type = LV_INDEV_TYPE_POINTER;
indev_drv.read_cb = lvgl_touch_cb;
lv_indev_drv_register(&indev_drv);
ESP_LOGI(TAG, "Touch input registered");
}
BaseType_t xret = xTaskCreatePinnedToCore(
display_task, "display", DISP_TASK_STACK,
NULL, DISP_TASK_PRIORITY, NULL, DISP_TASK_CORE);
if (xret != pdPASS) {
ESP_LOGE(TAG, "Failed to create display task");
return ESP_OK;
}
ESP_LOGI(TAG, "Display task started (Core %d, priority %d, %d fps)",
DISP_TASK_CORE, DISP_TASK_PRIORITY, DISP_FPS_LIMIT);
return ESP_OK;
}
#else /* !CONFIG_DISPLAY_ENABLE */
esp_err_t display_task_start(void)
{
return ESP_OK;
}
#endif /* CONFIG_DISPLAY_ENABLE */
@@ -0,0 +1,29 @@
/**
* @file display_task.h
* @brief ADR-045: FreeRTOS display task LVGL pump on Core 0.
*/
#ifndef DISPLAY_TASK_H
#define DISPLAY_TASK_H
#include "esp_err.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* Start the display task on Core 0, priority 1.
*
* Probes for RM67162 panel and SPIRAM. If either is absent,
* logs a warning and returns ESP_OK (graceful skip).
*
* @return ESP_OK always (display is optional).
*/
esp_err_t display_task_start(void);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_TASK_H */
+387
View File
@@ -0,0 +1,387 @@
/**
* @file display_ui.c
* @brief ADR-045: LVGL 4-view swipeable UI Dashboard | Vitals | Presence | System.
*
* Dark theme (#0a0a0f background) with cyan (#00d4ff) accent.
* Glowing line effects via layered semi-transparent chart series.
*/
#include "display_ui.h"
#include "sdkconfig.h"
#if CONFIG_DISPLAY_ENABLE
#include <stdio.h>
#include <string.h>
#include "esp_log.h"
#include "esp_system.h"
#include "esp_timer.h"
#include "esp_heap_caps.h"
#include "edge_processing.h"
static const char *TAG = "disp_ui";
/* ---- Theme colors ---- */
#define COLOR_BG lv_color_make(0x0A, 0x0A, 0x0F)
#define COLOR_CYAN lv_color_make(0x00, 0xD4, 0xFF)
#define COLOR_AMBER lv_color_make(0xFF, 0xB0, 0x00)
#define COLOR_GREEN lv_color_make(0x00, 0xFF, 0x80)
#define COLOR_RED lv_color_make(0xFF, 0x40, 0x40)
#define COLOR_DIM lv_color_make(0x30, 0x30, 0x40)
#define COLOR_TEXT lv_color_make(0xCC, 0xCC, 0xDD)
#define COLOR_TEXT_DIM lv_color_make(0x66, 0x66, 0x77)
/* ---- Chart data points ---- */
#define CHART_POINTS 60
/* ---- View handles ---- */
static lv_obj_t *s_tileview = NULL;
/* Dashboard */
static lv_obj_t *s_dash_chart = NULL;
static lv_chart_series_t *s_csi_series = NULL;
static lv_obj_t *s_dash_persons = NULL;
static lv_obj_t *s_dash_rssi = NULL;
static lv_obj_t *s_dash_motion = NULL;
/* Vitals */
static lv_obj_t *s_vital_chart = NULL;
static lv_chart_series_t *s_breath_series = NULL;
static lv_chart_series_t *s_hr_series = NULL;
static lv_obj_t *s_vital_bpm_br = NULL;
static lv_obj_t *s_vital_bpm_hr = NULL;
/* Presence */
#define GRID_COLS 4
#define GRID_ROWS 4
static lv_obj_t *s_grid_cells[GRID_COLS * GRID_ROWS];
static lv_obj_t *s_presence_label = NULL;
/* System */
static lv_obj_t *s_sys_cpu = NULL;
static lv_obj_t *s_sys_heap = NULL;
static lv_obj_t *s_sys_psram = NULL;
static lv_obj_t *s_sys_rssi = NULL;
static lv_obj_t *s_sys_uptime = NULL;
static lv_obj_t *s_sys_fps = NULL;
static lv_obj_t *s_sys_node = NULL;
/* ---- Style helpers ---- */
static lv_style_t s_style_bg;
static lv_style_t s_style_label;
static lv_style_t s_style_label_big;
static bool s_styles_inited = false;
static void init_styles(void)
{
if (s_styles_inited) return;
s_styles_inited = true;
lv_style_init(&s_style_bg);
lv_style_set_bg_color(&s_style_bg, COLOR_BG);
lv_style_set_bg_opa(&s_style_bg, LV_OPA_COVER);
lv_style_set_border_width(&s_style_bg, 0);
lv_style_set_pad_all(&s_style_bg, 4);
lv_style_init(&s_style_label);
lv_style_set_text_color(&s_style_label, COLOR_TEXT);
lv_style_set_text_font(&s_style_label, &lv_font_montserrat_14);
lv_style_init(&s_style_label_big);
lv_style_set_text_color(&s_style_label_big, COLOR_CYAN);
lv_style_set_text_font(&s_style_label_big, &lv_font_montserrat_14);
}
static lv_obj_t *make_label(lv_obj_t *parent, const char *text, const lv_style_t *style)
{
lv_obj_t *lbl = lv_label_create(parent);
lv_label_set_text(lbl, text);
if (style) lv_obj_add_style(lbl, (lv_style_t *)style, 0);
return lbl;
}
static lv_obj_t *make_tile(lv_obj_t *tv, uint8_t col, uint8_t row)
{
lv_obj_t *tile = lv_tileview_add_tile(tv, col, row, LV_DIR_HOR);
lv_obj_add_style(tile, &s_style_bg, 0);
return tile;
}
/* ---- View 0: Dashboard ---- */
static void create_dashboard(lv_obj_t *tile)
{
make_label(tile, "CSI Dashboard", &s_style_label);
/* CSI amplitude chart */
s_dash_chart = lv_chart_create(tile);
lv_obj_set_size(s_dash_chart, 400, 130);
lv_obj_align(s_dash_chart, LV_ALIGN_TOP_LEFT, 0, 24);
lv_chart_set_type(s_dash_chart, LV_CHART_TYPE_LINE);
lv_chart_set_point_count(s_dash_chart, CHART_POINTS);
lv_chart_set_range(s_dash_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 100);
lv_obj_set_style_bg_color(s_dash_chart, COLOR_BG, 0);
lv_obj_set_style_border_color(s_dash_chart, COLOR_DIM, 0);
lv_obj_set_style_line_width(s_dash_chart, 0, LV_PART_TICKS);
s_csi_series = lv_chart_add_series(s_dash_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y);
/* Stats panel on the right */
lv_obj_t *panel = lv_obj_create(tile);
lv_obj_set_size(panel, 120, 130);
lv_obj_align(panel, LV_ALIGN_TOP_RIGHT, 0, 24);
lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0);
lv_obj_set_style_border_width(panel, 1, 0);
lv_obj_set_style_border_color(panel, COLOR_DIM, 0);
lv_obj_set_style_pad_all(panel, 8, 0);
lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
make_label(panel, "Persons", &s_style_label);
s_dash_persons = make_label(panel, "0", &s_style_label_big);
s_dash_rssi = make_label(panel, "RSSI: --", &s_style_label);
s_dash_motion = make_label(panel, "Motion: 0.0", &s_style_label);
}
/* ---- View 1: Vitals ---- */
static void create_vitals(lv_obj_t *tile)
{
make_label(tile, "Vital Signs", &s_style_label);
s_vital_chart = lv_chart_create(tile);
lv_obj_set_size(s_vital_chart, 480, 150);
lv_obj_align(s_vital_chart, LV_ALIGN_TOP_LEFT, 0, 24);
lv_chart_set_type(s_vital_chart, LV_CHART_TYPE_LINE);
lv_chart_set_point_count(s_vital_chart, CHART_POINTS);
lv_chart_set_range(s_vital_chart, LV_CHART_AXIS_PRIMARY_Y, 0, 120);
lv_obj_set_style_bg_color(s_vital_chart, COLOR_BG, 0);
lv_obj_set_style_border_color(s_vital_chart, COLOR_DIM, 0);
lv_obj_set_style_line_width(s_vital_chart, 0, LV_PART_TICKS);
/* Breathing series (cyan) */
s_breath_series = lv_chart_add_series(s_vital_chart, COLOR_CYAN, LV_CHART_AXIS_PRIMARY_Y);
/* Heart rate series (amber) */
s_hr_series = lv_chart_add_series(s_vital_chart, COLOR_AMBER, LV_CHART_AXIS_PRIMARY_Y);
/* BPM readouts */
s_vital_bpm_br = make_label(tile, "Breathing: -- BPM", &s_style_label);
lv_obj_align(s_vital_bpm_br, LV_ALIGN_BOTTOM_LEFT, 4, -8);
lv_obj_set_style_text_color(s_vital_bpm_br, COLOR_CYAN, 0);
s_vital_bpm_hr = make_label(tile, "Heart Rate: -- BPM", &s_style_label);
lv_obj_align(s_vital_bpm_hr, LV_ALIGN_BOTTOM_RIGHT, -4, -8);
lv_obj_set_style_text_color(s_vital_bpm_hr, COLOR_AMBER, 0);
}
/* ---- View 2: Presence Grid ---- */
static void create_presence(lv_obj_t *tile)
{
make_label(tile, "Occupancy Map", &s_style_label);
int cell_w = 50;
int cell_h = 45;
int x_off = (368 - GRID_COLS * (cell_w + 4)) / 2;
int y_off = 30;
for (int r = 0; r < GRID_ROWS; r++) {
for (int c = 0; c < GRID_COLS; c++) {
lv_obj_t *cell = lv_obj_create(tile);
lv_obj_set_size(cell, cell_w, cell_h);
lv_obj_set_pos(cell, x_off + c * (cell_w + 4), y_off + r * (cell_h + 4));
lv_obj_set_style_bg_color(cell, COLOR_DIM, 0);
lv_obj_set_style_bg_opa(cell, LV_OPA_COVER, 0);
lv_obj_set_style_border_color(cell, COLOR_DIM, 0);
lv_obj_set_style_border_width(cell, 1, 0);
lv_obj_set_style_radius(cell, 4, 0);
s_grid_cells[r * GRID_COLS + c] = cell;
}
}
s_presence_label = make_label(tile, "Persons: 0", &s_style_label);
lv_obj_align(s_presence_label, LV_ALIGN_BOTTOM_MID, 0, -8);
}
/* ---- View 3: System ---- */
static void create_system(lv_obj_t *tile)
{
make_label(tile, "System Info", &s_style_label);
lv_obj_t *panel = lv_obj_create(tile);
lv_obj_set_size(panel, 500, 180);
lv_obj_align(panel, LV_ALIGN_TOP_LEFT, 0, 24);
lv_obj_set_style_bg_color(panel, lv_color_make(0x12, 0x12, 0x1A), 0);
lv_obj_set_style_border_width(panel, 1, 0);
lv_obj_set_style_border_color(panel, COLOR_DIM, 0);
lv_obj_set_style_pad_all(panel, 10, 0);
lv_obj_set_flex_flow(panel, LV_FLEX_FLOW_COLUMN);
lv_obj_set_flex_align(panel, LV_FLEX_ALIGN_SPACE_EVENLY, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
s_sys_node = make_label(panel, "Node: --", &s_style_label);
s_sys_cpu = make_label(panel, "CPU: --%", &s_style_label);
s_sys_heap = make_label(panel, "Heap: -- KB free", &s_style_label);
s_sys_psram = make_label(panel, "PSRAM: -- KB free",&s_style_label);
s_sys_rssi = make_label(panel, "WiFi RSSI: --", &s_style_label);
s_sys_uptime = make_label(panel, "Uptime: --", &s_style_label);
s_sys_fps = make_label(panel, "FPS: --", &s_style_label);
}
/* ---- Public API ---- */
void display_ui_create(lv_obj_t *parent)
{
init_styles();
s_tileview = lv_tileview_create(parent);
lv_obj_add_style(s_tileview, &s_style_bg, 0);
lv_obj_set_style_bg_color(s_tileview, COLOR_BG, 0);
lv_obj_t *t0 = make_tile(s_tileview, 0, 0);
lv_obj_t *t1 = make_tile(s_tileview, 1, 0);
lv_obj_t *t2 = make_tile(s_tileview, 2, 0);
lv_obj_t *t3 = make_tile(s_tileview, 3, 0);
create_dashboard(t0);
create_vitals(t1);
create_presence(t2);
create_system(t3);
ESP_LOGI(TAG, "UI created: 4 views (Dashboard|Vitals|Presence|System)");
}
/* ---- FPS tracking ---- */
static uint32_t s_frame_count = 0;
static uint32_t s_last_fps_time = 0;
static uint32_t s_current_fps = 0;
void display_ui_update(void)
{
/* FPS counter */
s_frame_count++;
uint32_t now_ms = (uint32_t)(esp_timer_get_time() / 1000);
if (now_ms - s_last_fps_time >= 1000) {
s_current_fps = s_frame_count;
s_frame_count = 0;
s_last_fps_time = now_ms;
}
/* Read edge data (thread-safe) */
edge_vitals_pkt_t vitals;
bool has_vitals = edge_get_vitals(&vitals);
edge_person_vitals_t persons[EDGE_MAX_PERSONS];
uint8_t n_active = 0;
edge_get_multi_person(persons, &n_active);
/* ---- Dashboard update ---- */
if (s_dash_chart && has_vitals) {
/* Push motion energy as amplitude proxy (scaled 0-100) */
int val = (int)(vitals.motion_energy * 10.0f);
if (val > 100) val = 100;
if (val < 0) val = 0;
lv_chart_set_next_value(s_dash_chart, s_csi_series, val);
}
if (s_dash_persons) {
char buf[8];
snprintf(buf, sizeof(buf), "%u", has_vitals ? vitals.n_persons : 0);
lv_label_set_text(s_dash_persons, buf);
}
if (s_dash_rssi && has_vitals) {
char buf[16];
snprintf(buf, sizeof(buf), "RSSI: %d", vitals.rssi);
lv_label_set_text(s_dash_rssi, buf);
}
if (s_dash_motion && has_vitals) {
char buf[24];
snprintf(buf, sizeof(buf), "Motion: %.1f", (double)vitals.motion_energy);
lv_label_set_text(s_dash_motion, buf);
}
/* ---- Vitals update ---- */
if (s_vital_chart && has_vitals) {
int br = (int)(vitals.breathing_rate / 100); /* Fixed-point to int BPM */
int hr = (int)(vitals.heartrate / 10000);
if (br > 120) br = 120;
if (hr > 120) hr = 120;
lv_chart_set_next_value(s_vital_chart, s_breath_series, br);
lv_chart_set_next_value(s_vital_chart, s_hr_series, hr);
char buf[32];
snprintf(buf, sizeof(buf), "Breathing: %d BPM", br);
lv_label_set_text(s_vital_bpm_br, buf);
snprintf(buf, sizeof(buf), "Heart Rate: %d BPM", hr);
lv_label_set_text(s_vital_bpm_hr, buf);
}
/* ---- Presence grid update ---- */
if (has_vitals) {
/* Simple visualization: color cells based on motion energy distribution */
float energy = vitals.motion_energy;
uint8_t active_cells = (uint8_t)(energy * 2); /* Scale for visibility */
if (active_cells > GRID_COLS * GRID_ROWS) active_cells = GRID_COLS * GRID_ROWS;
for (int i = 0; i < GRID_COLS * GRID_ROWS; i++) {
if (i < active_cells) {
/* Color gradient: green → amber → red based on intensity */
if (energy > 5.0f) {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_RED, 0);
} else if (energy > 2.0f) {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_AMBER, 0);
} else {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_GREEN, 0);
}
} else {
lv_obj_set_style_bg_color(s_grid_cells[i], COLOR_DIM, 0);
}
}
char buf[20];
snprintf(buf, sizeof(buf), "Persons: %u", vitals.n_persons);
lv_label_set_text(s_presence_label, buf);
}
/* ---- System info update ---- */
{
char buf[48];
#ifdef CONFIG_CSI_NODE_ID
snprintf(buf, sizeof(buf), "Node: %d", CONFIG_CSI_NODE_ID);
#else
snprintf(buf, sizeof(buf), "Node: --");
#endif
lv_label_set_text(s_sys_node, buf);
snprintf(buf, sizeof(buf), "Heap: %lu KB free",
(unsigned long)(esp_get_free_heap_size() / 1024));
lv_label_set_text(s_sys_heap, buf);
#if CONFIG_SPIRAM
snprintf(buf, sizeof(buf), "PSRAM: %lu KB free",
(unsigned long)(heap_caps_get_free_size(MALLOC_CAP_SPIRAM) / 1024));
#else
snprintf(buf, sizeof(buf), "PSRAM: N/A");
#endif
lv_label_set_text(s_sys_psram, buf);
if (has_vitals) {
snprintf(buf, sizeof(buf), "WiFi RSSI: %d dBm", vitals.rssi);
lv_label_set_text(s_sys_rssi, buf);
}
uint32_t uptime_s = (uint32_t)(esp_timer_get_time() / 1000000);
uint32_t h = uptime_s / 3600;
uint32_t m = (uptime_s % 3600) / 60;
uint32_t s = uptime_s % 60;
snprintf(buf, sizeof(buf), "Uptime: %luh %02lum %02lus",
(unsigned long)h, (unsigned long)m, (unsigned long)s);
lv_label_set_text(s_sys_uptime, buf);
snprintf(buf, sizeof(buf), "FPS: %lu", (unsigned long)s_current_fps);
lv_label_set_text(s_sys_fps, buf);
}
}
#endif /* CONFIG_DISPLAY_ENABLE */
+31
View File
@@ -0,0 +1,31 @@
/**
* @file display_ui.h
* @brief ADR-045: LVGL 4-view swipeable UI for CSI node stats.
*
* Views: Dashboard | Vitals | Presence | System
* Dark theme with cyan (#00d4ff) accent.
*/
#ifndef DISPLAY_UI_H
#define DISPLAY_UI_H
#include "lvgl.h"
#ifdef __cplusplus
extern "C" {
#endif
/** Create all LVGL views on the given tileview parent. */
void display_ui_create(lv_obj_t *parent);
/**
* Update all views with latest data. Called every display refresh cycle.
* Reads from edge_get_vitals() and edge_get_multi_person() internally.
*/
void display_ui_update(void);
#ifdef __cplusplus
}
#endif
#endif /* DISPLAY_UI_H */
@@ -0,0 +1,10 @@
## ESP-IDF Managed Component Dependencies (ADR-045)
dependencies:
## LVGL graphics library
lvgl/lvgl: "~8.3"
## CST816S capacitive touch driver
espressif/esp_lcd_touch_cst816s: "^1.0"
## LCD touch abstraction
espressif/esp_lcd_touch: "^1.0"
+94
View File
@@ -0,0 +1,94 @@
/**
* @file lv_conf.h
* @brief LVGL compile-time configuration for ESP32-S3 AMOLED display (ADR-045).
*
* Tuned for RM67162 536x240 QSPI AMOLED with 8MB PSRAM.
* Color depth: RGB565 (16-bit) for QSPI bandwidth.
* Double-buffered in SPIRAM, 30fps target.
*/
#ifndef LV_CONF_H
#define LV_CONF_H
#include <stdint.h>
/* ---- Core ---- */
#define LV_COLOR_DEPTH 16
#define LV_COLOR_16_SWAP 1 /* Byte-swap for SPI/QSPI displays */
#define LV_MEM_CUSTOM 1 /* Use ESP-IDF heap instead of LVGL's internal allocator */
#define LV_MEM_CUSTOM_INCLUDE <stdlib.h>
#define LV_MEM_CUSTOM_ALLOC malloc
#define LV_MEM_CUSTOM_FREE free
#define LV_MEM_CUSTOM_REALLOC realloc
/* ---- Display ---- */
#define LV_HOR_RES_MAX 368
#define LV_VER_RES_MAX 448
#define LV_DPI_DEF 200
/* ---- Tick (provided by esp_timer in display_task.c) ---- */
#define LV_TICK_CUSTOM 1
#define LV_TICK_CUSTOM_INCLUDE "esp_timer.h"
#define LV_TICK_CUSTOM_SYS_TIME_EXPR ((uint32_t)(esp_timer_get_time() / 1000))
/* ---- Drawing ---- */
#define LV_DRAW_COMPLEX 1
#define LV_SHADOW_CACHE_SIZE 0
#define LV_CIRCLE_CACHE_SIZE 4
#define LV_IMG_CACHE_DEF_SIZE 0
/* ---- Fonts ---- */
#define LV_FONT_MONTSERRAT_14 1
#define LV_FONT_MONTSERRAT_20 1
#define LV_FONT_DEFAULT &lv_font_montserrat_14
/* ---- Widgets ---- */
#define LV_USE_ARC 1
#define LV_USE_BAR 1
#define LV_USE_BTN 0
#define LV_USE_BTNMATRIX 0
#define LV_USE_CANVAS 0
#define LV_USE_CHECKBOX 0
#define LV_USE_DROPDOWN 0
#define LV_USE_IMG 0
#define LV_USE_LABEL 1
#define LV_USE_LINE 1
#define LV_USE_ROLLER 0
#define LV_USE_SLIDER 0
#define LV_USE_SWITCH 0
#define LV_USE_TEXTAREA 0
#define LV_USE_TABLE 0
/* ---- Extra widgets ---- */
#define LV_USE_CHART 1
#define LV_CHART_AXIS_TICK_LABEL_MAX_LEN 32
#define LV_USE_METER 0
#define LV_USE_SPINBOX 0
#define LV_USE_SPAN 0
#define LV_USE_TILEVIEW 1 /* Used for swipeable page navigation */
#define LV_USE_TABVIEW 0
#define LV_USE_WIN 0
/* ---- Themes ---- */
#define LV_USE_THEME_DEFAULT 1
#define LV_THEME_DEFAULT_DARK 1
/* ---- Logging ---- */
#define LV_USE_LOG 0
#define LV_USE_ASSERT_NULL 1
#define LV_USE_ASSERT_MALLOC 1
/* ---- GPU / render ---- */
#define LV_USE_GPU_ESP32_S3 0 /* No parallel LCD interface — we use QSPI */
/* ---- Animation ---- */
#define LV_USE_ANIM 1
#define LV_ANIM_DEF_TIME 200
/* ---- Misc ---- */
#define LV_USE_GROUP 1 /* For touch/input device routing */
#define LV_USE_PERF_MONITOR 0
#define LV_USE_MEM_MONITOR 0
#define LV_SPRINTF_CUSTOM 0
#endif /* LV_CONF_H */
+7
View File
@@ -26,6 +26,7 @@
#include "power_mgmt.h"
#include "wasm_runtime.h"
#include "wasm_upload.h"
#include "display_task.h"
#include "esp_timer.h"
@@ -203,6 +204,12 @@ void app_main(void)
/* Initialize power management. */
power_mgmt_init(g_nvs_config.power_duty);
/* ADR-045: Start AMOLED display task (gracefully skips if no display). */
esp_err_t disp_ret = display_task_start();
if (disp_ret != ESP_OK) {
ESP_LOGW(TAG, "Display init returned: %s", esp_err_to_name(disp_ret));
}
ESP_LOGI(TAG, "CSI streaming active → %s:%d (edge_tier=%u, OTA=%s, WASM=%s)",
g_nvs_config.target_ip, g_nvs_config.target_port,
g_nvs_config.edge_tier,
+70
View File
@@ -15,6 +15,8 @@
#include "esp_ota_ops.h"
#include "esp_http_server.h"
#include "esp_app_desc.h"
#include "nvs_flash.h"
#include "nvs.h"
static const char *TAG = "ota_update";
@@ -24,6 +26,52 @@ static const char *TAG = "ota_update";
/** Maximum firmware size (900 KB — matches CI binary size gate). */
#define OTA_MAX_SIZE (900 * 1024)
/** NVS namespace and key for the OTA pre-shared key. */
#define OTA_NVS_NAMESPACE "security"
#define OTA_NVS_KEY "ota_psk"
/** Maximum PSK length (hex-encoded SHA-256). */
#define OTA_PSK_MAX_LEN 65
/** Cached PSK loaded from NVS at init time. Empty = auth disabled. */
static char s_ota_psk[OTA_PSK_MAX_LEN] = {0};
/**
* ADR-050: Verify the Authorization header contains the correct PSK.
* Returns true if auth is disabled (no PSK provisioned) or if the
* Bearer token matches the stored PSK.
*/
static bool ota_check_auth(httpd_req_t *req)
{
if (s_ota_psk[0] == '\0') {
/* No PSK provisioned — auth disabled (permissive for dev). */
return true;
}
char auth_header[128] = {0};
if (httpd_req_get_hdr_value_str(req, "Authorization", auth_header,
sizeof(auth_header)) != ESP_OK) {
return false;
}
/* Expect "Bearer <psk>" */
const char *prefix = "Bearer ";
if (strncmp(auth_header, prefix, strlen(prefix)) != 0) {
return false;
}
const char *token = auth_header + strlen(prefix);
/* Constant-time comparison to prevent timing attacks. */
size_t psk_len = strlen(s_ota_psk);
size_t tok_len = strlen(token);
if (psk_len != tok_len) return false;
volatile uint8_t result = 0;
for (size_t i = 0; i < psk_len; i++) {
result |= (uint8_t)(s_ota_psk[i] ^ token[i]);
}
return result == 0;
}
/**
* GET /ota/status return firmware version and partition info.
*/
@@ -53,6 +101,14 @@ static esp_err_t ota_status_handler(httpd_req_t *req)
*/
static esp_err_t ota_upload_handler(httpd_req_t *req)
{
/* ADR-050: Authenticate before accepting firmware upload. */
if (!ota_check_auth(req)) {
ESP_LOGW(TAG, "OTA upload rejected: authentication failed");
httpd_resp_send_err(req, HTTPD_403_FORBIDDEN,
"Authentication required. Use: Authorization: Bearer <psk>");
return ESP_FAIL;
}
ESP_LOGI(TAG, "OTA update started, content_length=%d", req->content_len);
if (req->content_len <= 0 || req->content_len > OTA_MAX_SIZE) {
@@ -187,6 +243,20 @@ static esp_err_t ota_start_server(httpd_handle_t *out_handle)
esp_err_t ota_update_init(void)
{
/* ADR-050: Load OTA PSK from NVS if provisioned. */
nvs_handle_t nvs;
if (nvs_open(OTA_NVS_NAMESPACE, NVS_READONLY, &nvs) == ESP_OK) {
size_t len = sizeof(s_ota_psk);
if (nvs_get_str(nvs, OTA_NVS_KEY, s_ota_psk, &len) == ESP_OK) {
ESP_LOGI(TAG, "OTA PSK loaded from NVS (%d chars) — authentication enabled", (int)len - 1);
} else {
ESP_LOGW(TAG, "No OTA PSK in NVS — OTA authentication DISABLED (provision with nvs_set)");
}
nvs_close(nvs);
} else {
ESP_LOGW(TAG, "NVS namespace '%s' not found — OTA authentication DISABLED", OTA_NVS_NAMESPACE);
}
return ota_start_server(NULL);
}
+6 -5
View File
@@ -107,8 +107,9 @@ static esp_err_t wasm_upload_handler(httpd_req_t *req)
return ESP_FAIL;
}
/* Verify signature if wasm_verify is enabled. */
#ifdef CONFIG_WASM_VERIFY_SIGNATURE
/* ADR-050: Verify signature (default-on; skip only if
* CONFIG_WASM_SKIP_SIGNATURE is explicitly set for dev/lab). */
#ifndef CONFIG_WASM_SKIP_SIGNATURE
{
/* Load pubkey from NVS config (set via provision.py --wasm-pubkey). */
extern nvs_config_t g_nvs_config;
@@ -173,11 +174,11 @@ static esp_err_t wasm_upload_handler(httpd_req_t *req)
} else if (rvf_is_raw_wasm(buf, (uint32_t)total)) {
/* ── Raw WASM path (dev/lab only) ── */
#ifdef CONFIG_WASM_VERIFY_SIGNATURE
#ifndef CONFIG_WASM_SKIP_SIGNATURE
free(buf);
httpd_resp_send_err(req, HTTPD_403_FORBIDDEN,
"Raw WASM upload rejected (wasm_verify enabled). "
"Use RVF container with signature.");
"Raw WASM upload rejected (signature verification enabled). "
"Use RVF container with signature, or set CONFIG_WASM_SKIP_SIGNATURE for dev.");
return ESP_FAIL;
#else
format = "raw";
@@ -0,0 +1,8 @@
# ESP32-S3 CSI Node — 8MB flash partition table (ADR-045)
# Name, Type, SubType, Offset, Size, Flags
nvs, data, nvs, 0x9000, 0x6000,
otadata, data, ota, 0xf000, 0x2000,
phy_init, data, phy, 0x11000, 0x1000,
ota_0, app, ota_0, 0x20000, 0x200000,
ota_1, app, ota_1, 0x220000, 0x200000,
spiffs, data, spiffs, 0x420000, 0x1E0000,
1 # ESP32-S3 CSI Node — 8MB flash partition table (ADR-045)
2 # Name, Type, SubType, Offset, Size, Flags
3 nvs, data, nvs, 0x9000, 0x6000,
4 otadata, data, ota, 0xf000, 0x2000,
5 phy_init, data, phy, 0x11000, 0x1000,
6 ota_0, app, ota_0, 0x20000, 0x200000,
7 ota_1, app, ota_1, 0x220000, 0x200000,
8 spiffs, data, spiffs, 0x420000, 0x1E0000,
+2
View File
@@ -4579,10 +4579,12 @@ dependencies = [
"chrono",
"clap",
"criterion",
"hmac",
"midstreamer-quic",
"midstreamer-scheduler",
"serde",
"serde_json",
"sha2",
"thiserror 1.0.69",
"tokio",
"tracing",
+1 -1
View File
@@ -101,7 +101,7 @@ csv = "1.3"
indicatif = "0.17"
# CLI
clap = { version = "4.4", features = ["derive"] }
clap = { version = "4.4", features = ["derive", "env"] }
# Testing
criterion = { version = "0.5", features = ["html_reports"] }
@@ -24,6 +24,9 @@ linux-wifi = []
[dependencies]
# CLI argument parsing (for bin/aggregator)
clap = { version = "4.4", features = ["derive"] }
# Cryptographic HMAC (ADR-050: replace fake XOR-fold HMAC)
hmac = "0.12"
sha2 = "0.10"
# Byte parsing
byteorder = "1.5"
# Time
@@ -33,9 +33,13 @@ use super::quic_transport::{
QuicTransportHandle, QuicTransportError, SecurityMode,
};
use super::tdm::{SyncBeacon, TdmCoordinator, TdmSchedule, TdmSlotCompleted};
use hmac::{Hmac, Mac};
use sha2::Sha256;
use std::collections::VecDeque;
use std::fmt;
type HmacSha256 = Hmac<Sha256>;
// ---------------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------------
@@ -245,19 +249,17 @@ impl AuthenticatedBeacon {
})
}
/// Compute the expected HMAC tag for this beacon using the given key.
/// Compute the HMAC-SHA256 tag for this beacon, truncated to 8 bytes.
///
/// Uses a simplified HMAC approximation for testing. In production,
/// this calls mbedtls HMAC-SHA256 via the ESP-IDF hardware accelerator
/// or the `sha2` crate on aggregator nodes.
/// Uses the `hmac` + `sha2` crates for cryptographically secure
/// message authentication (ADR-050, Sprint 1).
pub fn compute_tag(payload_and_nonce: &[u8], key: &[u8; 16]) -> [u8; HMAC_TAG_SIZE] {
// Simplified HMAC: XOR key into payload hash. In production, use
// real HMAC-SHA256 from sha2 crate. This is sufficient for
// testing the protocol structure.
let mut mac = HmacSha256::new_from_slice(key)
.expect("HMAC-SHA256 accepts any key length");
mac.update(payload_and_nonce);
let result = mac.finalize().into_bytes();
let mut tag = [0u8; HMAC_TAG_SIZE];
for (i, byte) in payload_and_nonce.iter().enumerate() {
tag[i % HMAC_TAG_SIZE] ^= byte ^ key[i % 16];
}
tag.copy_from_slice(&result[..HMAC_TAG_SIZE]);
tag
}
@@ -975,6 +977,97 @@ mod tests {
assert_eq!(SecLevel::Enforcing as u8, 2);
}
// ---- Security tests (ADR-050) ----
#[test]
fn test_hmac_different_keys_produce_different_tags() {
let msg = b"test payload with nonce";
let key1: [u8; 16] = [0x01; 16];
let key2: [u8; 16] = [0x02; 16];
let tag1 = AuthenticatedBeacon::compute_tag(msg, &key1);
let tag2 = AuthenticatedBeacon::compute_tag(msg, &key2);
assert_ne!(tag1, tag2, "Different keys must produce different HMAC tags");
}
#[test]
fn test_hmac_different_messages_produce_different_tags() {
let key: [u8; 16] = DEFAULT_TEST_KEY;
let tag1 = AuthenticatedBeacon::compute_tag(b"message one", &key);
let tag2 = AuthenticatedBeacon::compute_tag(b"message two", &key);
assert_ne!(tag1, tag2, "Different messages must produce different HMAC tags");
}
#[test]
fn test_hmac_is_deterministic() {
let key: [u8; 16] = DEFAULT_TEST_KEY;
let msg = b"determinism test";
let tag1 = AuthenticatedBeacon::compute_tag(msg, &key);
let tag2 = AuthenticatedBeacon::compute_tag(msg, &key);
assert_eq!(tag1, tag2, "Same key + message must produce identical tags");
}
#[test]
fn test_wrong_key_fails_verification() {
let beacon = SyncBeacon {
cycle_id: 42,
cycle_period: Duration::from_millis(50),
drift_correction_us: 0,
generated_at: std::time::Instant::now(),
};
let correct_key: [u8; 16] = DEFAULT_TEST_KEY;
let wrong_key: [u8; 16] = [0xFF; 16];
let nonce = 1u32;
let mut msg = [0u8; 20];
msg[..16].copy_from_slice(&beacon.to_bytes());
msg[16..20].copy_from_slice(&nonce.to_le_bytes());
let tag = AuthenticatedBeacon::compute_tag(&msg, &correct_key);
let auth = AuthenticatedBeacon { beacon, nonce, hmac_tag: tag };
assert!(auth.verify(&wrong_key).is_err(), "Wrong key must fail verification");
}
#[test]
fn test_single_bit_flip_in_payload_fails_verification() {
let beacon = SyncBeacon {
cycle_id: 42,
cycle_period: Duration::from_millis(50),
drift_correction_us: 0,
generated_at: std::time::Instant::now(),
};
let key: [u8; 16] = DEFAULT_TEST_KEY;
let nonce = 1u32;
let mut msg = [0u8; 20];
msg[..16].copy_from_slice(&beacon.to_bytes());
msg[16..20].copy_from_slice(&nonce.to_le_bytes());
let tag = AuthenticatedBeacon::compute_tag(&msg, &key);
let auth = AuthenticatedBeacon { beacon, nonce, hmac_tag: tag };
let mut wire = auth.to_bytes();
// Flip one bit in the beacon payload
wire[0] ^= 0x01;
let tampered = AuthenticatedBeacon::from_bytes(&wire).unwrap();
assert!(tampered.verify(&key).is_err(), "Single bit flip must fail verification");
}
#[test]
fn test_enforcing_mode_rejects_unauthenticated() {
let mut cfg = manual_config();
cfg.sec_level = SecLevel::Enforcing;
let mut coord = SecureTdmCoordinator::new(test_schedule(), cfg).unwrap();
// Raw 16-byte beacon without HMAC
let raw = SyncBeacon {
cycle_id: 1,
cycle_period: Duration::from_millis(50),
drift_correction_us: 0,
generated_at: std::time::Instant::now(),
}.to_bytes();
assert!(coord.verify_beacon(&raw).is_err());
}
// ---- Error display tests ----
#[test]
@@ -0,0 +1,461 @@
//! Adaptive CSI Activity Classifier
//!
//! Learns environment-specific classification thresholds from labeled JSONL
//! recordings. Uses a lightweight approach:
//!
//! 1. **Feature statistics**: per-class mean/stddev for each of 7 CSI features
//! 2. **Mahalanobis-like distance**: weighted distance to each class centroid
//! 3. **Logistic regression weights**: learned via gradient descent on the
//! labeled data for fine-grained boundary tuning
//!
//! The trained model is serialised as JSON and hot-loaded at runtime so that
//! the classification thresholds adapt to the specific room and ESP32 placement.
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
use std::path::{Path, PathBuf};
// ── Feature vector ───────────────────────────────────────────────────────────
/// Extended feature vector: 7 server features + 8 subcarrier-derived features = 15.
const N_FEATURES: usize = 15;
/// Activity classes we recognise.
pub const CLASSES: &[&str] = &["absent", "present_still", "present_moving", "active"];
const N_CLASSES: usize = 4;
/// Extract extended feature vector from a JSONL frame (features + raw amplitudes).
pub fn features_from_frame(frame: &serde_json::Value) -> [f64; N_FEATURES] {
let feat = frame.get("features").cloned().unwrap_or(serde_json::Value::Null);
let nodes = frame.get("nodes").and_then(|n| n.as_array());
let amps: Vec<f64> = nodes
.and_then(|ns| ns.first())
.and_then(|n| n.get("amplitude"))
.and_then(|a| a.as_array())
.map(|arr| arr.iter().filter_map(|v| v.as_f64()).collect())
.unwrap_or_default();
// Server-computed features (0-6).
let variance = feat.get("variance").and_then(|v| v.as_f64()).unwrap_or(0.0);
let mbp = feat.get("motion_band_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let bbp = feat.get("breathing_band_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let sp = feat.get("spectral_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let df = feat.get("dominant_freq_hz").and_then(|v| v.as_f64()).unwrap_or(0.0);
let cp = feat.get("change_points").and_then(|v| v.as_f64()).unwrap_or(0.0);
let rssi = feat.get("mean_rssi").and_then(|v| v.as_f64()).unwrap_or(0.0);
// Subcarrier-derived features (7-14).
let (amp_mean, amp_std, amp_skew, amp_kurt, amp_iqr, amp_entropy, amp_max, amp_range) =
subcarrier_stats(&amps);
[
variance, mbp, bbp, sp, df, cp, rssi,
amp_mean, amp_std, amp_skew, amp_kurt, amp_iqr, amp_entropy, amp_max, amp_range,
]
}
/// Also keep a simpler version for runtime (no JSONL, just FeatureInfo + amps).
pub fn features_from_runtime(feat: &serde_json::Value, amps: &[f64]) -> [f64; N_FEATURES] {
let variance = feat.get("variance").and_then(|v| v.as_f64()).unwrap_or(0.0);
let mbp = feat.get("motion_band_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let bbp = feat.get("breathing_band_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let sp = feat.get("spectral_power").and_then(|v| v.as_f64()).unwrap_or(0.0);
let df = feat.get("dominant_freq_hz").and_then(|v| v.as_f64()).unwrap_or(0.0);
let cp = feat.get("change_points").and_then(|v| v.as_f64()).unwrap_or(0.0);
let rssi = feat.get("mean_rssi").and_then(|v| v.as_f64()).unwrap_or(0.0);
let (amp_mean, amp_std, amp_skew, amp_kurt, amp_iqr, amp_entropy, amp_max, amp_range) =
subcarrier_stats(amps);
[
variance, mbp, bbp, sp, df, cp, rssi,
amp_mean, amp_std, amp_skew, amp_kurt, amp_iqr, amp_entropy, amp_max, amp_range,
]
}
/// Compute statistical features from raw subcarrier amplitudes.
fn subcarrier_stats(amps: &[f64]) -> (f64, f64, f64, f64, f64, f64, f64, f64) {
if amps.is_empty() {
return (0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0);
}
let n = amps.len() as f64;
let mean = amps.iter().sum::<f64>() / n;
let var = amps.iter().map(|a| (a - mean).powi(2)).sum::<f64>() / n;
let std = var.sqrt().max(1e-9);
// Skewness (asymmetry).
let skew = amps.iter().map(|a| ((a - mean) / std).powi(3)).sum::<f64>() / n;
// Kurtosis (peakedness).
let kurt = amps.iter().map(|a| ((a - mean) / std).powi(4)).sum::<f64>() / n - 3.0;
// IQR (inter-quartile range).
let mut sorted = amps.to_vec();
sorted.sort_by(|a, b| a.partial_cmp(b).unwrap());
let q1 = sorted[sorted.len() / 4];
let q3 = sorted[3 * sorted.len() / 4];
let iqr = q3 - q1;
// Spectral entropy (normalised).
let total_power: f64 = amps.iter().map(|a| a * a).sum::<f64>().max(1e-9);
let entropy: f64 = amps.iter()
.map(|a| {
let p = (a * a) / total_power;
if p > 1e-12 { -p * p.ln() } else { 0.0 }
})
.sum::<f64>() / n.ln().max(1e-9); // normalise to [0,1]
let max_val = sorted.last().copied().unwrap_or(0.0);
let range = max_val - sorted.first().copied().unwrap_or(0.0);
(mean, std, skew, kurt, iqr, entropy, max_val, range)
}
// ── Per-class statistics ─────────────────────────────────────────────────────
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ClassStats {
pub label: String,
pub count: usize,
pub mean: [f64; N_FEATURES],
pub stddev: [f64; N_FEATURES],
}
// ── Trained model ────────────────────────────────────────────────────────────
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct AdaptiveModel {
/// Per-class feature statistics (centroid + spread).
pub class_stats: Vec<ClassStats>,
/// Logistic regression weights: [N_CLASSES x (N_FEATURES + 1)] (last = bias).
pub weights: Vec<[f64; N_FEATURES + 1]>,
/// Global feature normalisation: mean and stddev across all training data.
pub global_mean: [f64; N_FEATURES],
pub global_std: [f64; N_FEATURES],
/// Training metadata.
pub trained_frames: usize,
pub training_accuracy: f64,
pub version: u32,
}
impl Default for AdaptiveModel {
fn default() -> Self {
Self {
class_stats: Vec::new(),
weights: vec![[0.0; N_FEATURES + 1]; N_CLASSES],
global_mean: [0.0; N_FEATURES],
global_std: [1.0; N_FEATURES],
trained_frames: 0,
training_accuracy: 0.0,
version: 1,
}
}
}
impl AdaptiveModel {
/// Classify a raw feature vector. Returns (class_label, confidence).
pub fn classify(&self, raw_features: &[f64; N_FEATURES]) -> (&'static str, f64) {
if self.weights.is_empty() || self.class_stats.is_empty() {
return ("present_still", 0.5);
}
// Normalise features.
let mut x = [0.0f64; N_FEATURES];
for i in 0..N_FEATURES {
x[i] = (raw_features[i] - self.global_mean[i]) / (self.global_std[i] + 1e-9);
}
// Compute logits: w·x + b for each class.
let mut logits = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES.min(self.weights.len()) {
let w = &self.weights[c];
let mut z = w[N_FEATURES]; // bias
for i in 0..N_FEATURES {
z += w[i] * x[i];
}
logits[c] = z;
}
// Softmax.
let max_logit = logits.iter().cloned().fold(f64::NEG_INFINITY, f64::max);
let exp_sum: f64 = logits.iter().map(|z| (z - max_logit).exp()).sum();
let mut probs = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES {
probs[c] = ((logits[c] - max_logit).exp()) / exp_sum;
}
// Pick argmax.
let (best_c, best_p) = probs.iter().enumerate()
.max_by(|a, b| a.1.partial_cmp(b.1).unwrap())
.unwrap();
let label = if best_c < CLASSES.len() { CLASSES[best_c] } else { "present_still" };
(label, *best_p)
}
/// Save model to a JSON file.
pub fn save(&self, path: &Path) -> std::io::Result<()> {
let json = serde_json::to_string_pretty(self)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))?;
std::fs::write(path, json)
}
/// Load model from a JSON file.
pub fn load(path: &Path) -> std::io::Result<Self> {
let json = std::fs::read_to_string(path)?;
serde_json::from_str(&json)
.map_err(|e| std::io::Error::new(std::io::ErrorKind::Other, e))
}
}
// ── Training ─────────────────────────────────────────────────────────────────
/// A labeled training sample.
struct Sample {
features: [f64; N_FEATURES],
class_idx: usize,
}
/// Load JSONL recording frames and assign a class label based on filename.
fn load_recording(path: &Path, class_idx: usize) -> Vec<Sample> {
let content = match std::fs::read_to_string(path) {
Ok(c) => c,
Err(_) => return Vec::new(),
};
content.lines().filter_map(|line| {
let v: serde_json::Value = serde_json::from_str(line).ok()?;
// Use extended features (server features + subcarrier stats).
Some(Sample {
features: features_from_frame(&v),
class_idx,
})
}).collect()
}
/// Map a recording filename to a class index.
fn classify_recording_name(name: &str) -> Option<usize> {
let lower = name.to_lowercase();
if lower.contains("empty") || lower.contains("absent") { Some(0) }
else if lower.contains("still") || lower.contains("sitting") || lower.contains("standing") { Some(1) }
else if lower.contains("walking") || lower.contains("moving") { Some(2) }
else if lower.contains("active") || lower.contains("exercise") || lower.contains("running") { Some(3) }
else { None }
}
/// Train a model from labeled JSONL recordings in a directory.
///
/// Recordings are matched to classes by filename pattern:
/// - `*empty*` / `*absent*` → absent (0)
/// - `*still*` / `*sitting*` → present_still (1)
/// - `*walking*` / `*moving*` → present_moving (2)
/// - `*active*` / `*exercise*`→ active (3)
pub fn train_from_recordings(recordings_dir: &Path) -> Result<AdaptiveModel, String> {
// Scan for train_* files.
let mut samples: Vec<Sample> = Vec::new();
let entries = std::fs::read_dir(recordings_dir)
.map_err(|e| format!("Cannot read {}: {}", recordings_dir.display(), e))?;
for entry in entries.flatten() {
let fname = entry.file_name().to_string_lossy().to_string();
if !fname.starts_with("train_") || !fname.ends_with(".jsonl") {
continue;
}
if let Some(class_idx) = classify_recording_name(&fname) {
let loaded = load_recording(&entry.path(), class_idx);
eprintln!(" Loaded {}: {} frames → class '{}'",
fname, loaded.len(), CLASSES[class_idx]);
samples.extend(loaded);
}
}
if samples.is_empty() {
return Err("No training samples found. Record data with train_* prefix.".into());
}
let n = samples.len();
eprintln!("Total training samples: {n}");
// ── Compute global normalisation stats ──
let mut global_mean = [0.0f64; N_FEATURES];
let mut global_var = [0.0f64; N_FEATURES];
for s in &samples {
for i in 0..N_FEATURES { global_mean[i] += s.features[i]; }
}
for i in 0..N_FEATURES { global_mean[i] /= n as f64; }
for s in &samples {
for i in 0..N_FEATURES {
global_var[i] += (s.features[i] - global_mean[i]).powi(2);
}
}
let mut global_std = [0.0f64; N_FEATURES];
for i in 0..N_FEATURES {
global_std[i] = (global_var[i] / n as f64).sqrt().max(1e-9);
}
// ── Compute per-class statistics ──
let mut class_sums = vec![[0.0f64; N_FEATURES]; N_CLASSES];
let mut class_sq = vec![[0.0f64; N_FEATURES]; N_CLASSES];
let mut class_counts = vec![0usize; N_CLASSES];
for s in &samples {
let c = s.class_idx;
class_counts[c] += 1;
for i in 0..N_FEATURES {
class_sums[c][i] += s.features[i];
class_sq[c][i] += s.features[i] * s.features[i];
}
}
let mut class_stats = Vec::new();
for c in 0..N_CLASSES {
let cnt = class_counts[c].max(1) as f64;
let mut mean = [0.0; N_FEATURES];
let mut stddev = [0.0; N_FEATURES];
for i in 0..N_FEATURES {
mean[i] = class_sums[c][i] / cnt;
stddev[i] = ((class_sq[c][i] / cnt) - mean[i] * mean[i]).max(0.0).sqrt();
}
class_stats.push(ClassStats {
label: CLASSES[c].to_string(),
count: class_counts[c],
mean,
stddev,
});
}
// ── Normalise all samples ──
let mut norm_samples: Vec<([f64; N_FEATURES], usize)> = samples.iter().map(|s| {
let mut x = [0.0; N_FEATURES];
for i in 0..N_FEATURES {
x[i] = (s.features[i] - global_mean[i]) / (global_std[i] + 1e-9);
}
(x, s.class_idx)
}).collect();
// ── Train logistic regression via mini-batch SGD ──
let mut weights = vec![[0.0f64; N_FEATURES + 1]; N_CLASSES];
let lr = 0.1;
let epochs = 200;
let batch_size = 32;
// Shuffle helper (simple LCG for determinism).
let mut rng_state: u64 = 42;
let mut rng_next = move || -> u64 {
rng_state = rng_state.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
rng_state >> 33
};
for epoch in 0..epochs {
// Shuffle samples.
for i in (1..norm_samples.len()).rev() {
let j = (rng_next() as usize) % (i + 1);
norm_samples.swap(i, j);
}
let mut epoch_loss = 0.0f64;
let mut batch_count = 0;
for batch_start in (0..norm_samples.len()).step_by(batch_size) {
let batch_end = (batch_start + batch_size).min(norm_samples.len());
let batch = &norm_samples[batch_start..batch_end];
// Accumulate gradients.
let mut grad = vec![[0.0f64; N_FEATURES + 1]; N_CLASSES];
for (x, target) in batch {
// Forward: softmax.
let mut logits = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES {
logits[c] = weights[c][N_FEATURES]; // bias
for i in 0..N_FEATURES {
logits[c] += weights[c][i] * x[i];
}
}
let max_l = logits.iter().cloned().fold(f64::NEG_INFINITY, f64::max);
let exp_sum: f64 = logits.iter().map(|z| (z - max_l).exp()).sum();
let mut probs = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES {
probs[c] = ((logits[c] - max_l).exp()) / exp_sum;
}
// Cross-entropy loss.
epoch_loss += -(probs[*target].max(1e-15)).ln();
// Gradient: prob - one_hot(target).
for c in 0..N_CLASSES {
let delta = probs[c] - if c == *target { 1.0 } else { 0.0 };
for i in 0..N_FEATURES {
grad[c][i] += delta * x[i];
}
grad[c][N_FEATURES] += delta; // bias grad
}
}
// Update weights.
let bs = batch.len() as f64;
let current_lr = lr * (1.0 - epoch as f64 / epochs as f64); // linear decay
for c in 0..N_CLASSES {
for i in 0..=N_FEATURES {
weights[c][i] -= current_lr * grad[c][i] / bs;
}
}
batch_count += 1;
}
if epoch % 50 == 0 || epoch == epochs - 1 {
let avg_loss = epoch_loss / n as f64;
eprintln!(" Epoch {epoch:3}: loss = {avg_loss:.4}");
}
}
// ── Evaluate accuracy ──
let mut correct = 0;
for (x, target) in &norm_samples {
let mut logits = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES {
logits[c] = weights[c][N_FEATURES];
for i in 0..N_FEATURES {
logits[c] += weights[c][i] * x[i];
}
}
let pred = logits.iter().enumerate()
.max_by(|a, b| a.1.partial_cmp(b.1).unwrap())
.unwrap().0;
if pred == *target { correct += 1; }
}
let accuracy = correct as f64 / n as f64;
eprintln!("Training accuracy: {correct}/{n} = {accuracy:.1}%");
// ── Per-class accuracy ──
let mut class_correct = vec![0usize; N_CLASSES];
let mut class_total = vec![0usize; N_CLASSES];
for (x, target) in &norm_samples {
class_total[*target] += 1;
let mut logits = [0.0f64; N_CLASSES];
for c in 0..N_CLASSES {
logits[c] = weights[c][N_FEATURES];
for i in 0..N_FEATURES {
logits[c] += weights[c][i] * x[i];
}
}
let pred = logits.iter().enumerate()
.max_by(|a, b| a.1.partial_cmp(b.1).unwrap())
.unwrap().0;
if pred == *target { class_correct[*target] += 1; }
}
for c in 0..N_CLASSES {
let tot = class_total[c].max(1);
eprintln!(" {}: {}/{} ({:.0}%)", CLASSES[c], class_correct[c], tot,
class_correct[c] as f64 / tot as f64 * 100.0);
}
Ok(AdaptiveModel {
class_stats,
weights,
global_mean,
global_std,
trained_frames: n,
training_accuracy: accuracy,
version: 1,
})
}
/// Default path for the saved adaptive model.
pub fn model_path() -> PathBuf {
PathBuf::from("data/adaptive_model.json")
}
@@ -8,6 +8,7 @@
//!
//! Replaces both ws_server.py and the Python HTTP server.
mod adaptive_classifier;
mod rvf_container;
mod rvf_pipeline;
mod vital_signs;
@@ -76,6 +77,10 @@ struct Args {
#[arg(long, default_value = "100")]
tick_ms: u64,
/// Bind address (default 127.0.0.1; set to 0.0.0.0 for network access)
#[arg(long, default_value = "127.0.0.1", env = "SENSING_BIND_ADDR")]
bind_addr: String,
/// Data source: auto, wifi, esp32, simulate
#[arg(long, default_value = "auto")]
source: String,
@@ -299,6 +304,34 @@ struct AppStateInner {
model_loaded: bool,
/// Smoothed person count (EMA) for hysteresis — prevents frame-to-frame jumping.
smoothed_person_score: f64,
// ── Motion smoothing & adaptive baseline (ADR-047 tuning) ────────────
/// EMA-smoothed motion score (alpha ~0.15 for ~10 FPS → ~1s time constant).
smoothed_motion: f64,
/// Current classification state for hysteresis debounce.
current_motion_level: String,
/// How many consecutive frames the *raw* classification has agreed with a
/// *candidate* new level. State only changes after DEBOUNCE_FRAMES.
debounce_counter: u32,
/// The candidate motion level that the debounce counter is tracking.
debounce_candidate: String,
/// Adaptive baseline: EMA of motion score when room is "quiet" (low motion).
/// Subtracted from raw score so slow environmental drift doesn't inflate readings.
baseline_motion: f64,
/// Number of frames processed so far (for baseline warm-up).
baseline_frames: u64,
// ── Vital signs smoothing ────────────────────────────────────────────
/// EMA-smoothed heart rate (BPM).
smoothed_hr: f64,
/// EMA-smoothed breathing rate (BPM).
smoothed_br: f64,
/// EMA-smoothed HR confidence.
smoothed_hr_conf: f64,
/// EMA-smoothed BR confidence.
smoothed_br_conf: f64,
/// Median filter buffer for HR (last N raw values for outlier rejection).
hr_buffer: VecDeque<f64>,
/// Median filter buffer for BR.
br_buffer: VecDeque<f64>,
/// ADR-039: Latest edge vitals packet from ESP32.
edge_vitals: Option<Esp32VitalsPacket>,
/// ADR-040: Latest WASM output packet from ESP32.
@@ -324,6 +357,9 @@ struct AppStateInner {
training_status: String,
/// Training configuration, if any.
training_config: Option<serde_json::Value>,
// ── Adaptive classifier (environment-tuned) ──────────────────────────
/// Trained adaptive model (loaded from data/adaptive_model.json or trained at runtime).
adaptive_model: Option<adaptive_classifier::AdaptiveModel>,
}
/// Number of frames retained in `frame_history` for temporal analysis.
@@ -716,11 +752,12 @@ fn compute_subcarrier_variances(frame_history: &VecDeque<Vec<f64>>, n_sub: usize
/// the amplitude time series.
/// - **Signal quality**: based on SNR estimate (RSSI noise floor) and subcarrier
/// variance stability.
/// Returns (features, raw_classification, breathing_rate_hz, sub_variances, raw_motion_score).
fn extract_features_from_frame(
frame: &Esp32Frame,
frame_history: &VecDeque<Vec<f64>>,
sample_rate_hz: f64,
) -> (FeatureInfo, ClassificationInfo, f64, Vec<f64>) {
) -> (FeatureInfo, ClassificationInfo, f64, Vec<f64>, f64) {
let n_sub = frame.amplitudes.len().max(1);
let n = n_sub as f64;
let mean_amp: f64 = frame.amplitudes.iter().sum::<f64>() / n;
@@ -799,8 +836,11 @@ fn extract_features_from_frame(
};
// Blend temporal motion with variance-based motion for robustness.
// Also factor in motion_band_power and change_points for ESP32 real-world sensitivity.
let variance_motion = (temporal_variance / 10.0).clamp(0.0, 1.0);
let motion_score = (temporal_motion_score * 0.7 + variance_motion * 0.3).clamp(0.0, 1.0);
let mbp_motion = (motion_band_power / 25.0).clamp(0.0, 1.0);
let cp_motion = (change_points as f64 / 15.0).clamp(0.0, 1.0);
let motion_score = (temporal_motion_score * 0.4 + variance_motion * 0.2 + mbp_motion * 0.25 + cp_motion * 0.15).clamp(0.0, 1.0);
// ── Signal quality metric ──
// Based on estimated SNR (RSSI relative to noise floor) and subcarrier consistency.
@@ -823,24 +863,198 @@ fn extract_features_from_frame(
spectral_power,
};
// ── Classification ──
let (motion_level, presence) = if motion_score > 0.4 {
("active".to_string(), true)
} else if motion_score > 0.08 {
("present_still".to_string(), true)
// Return raw motion_score and signal_quality — classification is done by
// `smooth_and_classify()` which has access to EMA state and hysteresis.
let raw_classification = ClassificationInfo {
motion_level: raw_classify(motion_score),
presence: motion_score > 0.04,
confidence: (0.4 + signal_quality * 0.3 + motion_score * 0.3).clamp(0.0, 1.0),
};
(features, raw_classification, breathing_rate_hz, sub_variances, motion_score)
}
/// Simple threshold classification (no smoothing) — used as the "raw" input.
fn raw_classify(score: f64) -> String {
if score > 0.25 { "active".into() }
else if score > 0.12 { "present_moving".into() }
else if score > 0.04 { "present_still".into() }
else { "absent".into() }
}
/// Debounce frames required before state transition (at ~10 FPS = ~0.4s).
const DEBOUNCE_FRAMES: u32 = 4;
/// EMA alpha for motion smoothing (~1s time constant at 10 FPS).
const MOTION_EMA_ALPHA: f64 = 0.15;
/// EMA alpha for slow-adapting baseline (~30s time constant at 10 FPS).
const BASELINE_EMA_ALPHA: f64 = 0.003;
/// Number of warm-up frames before baseline subtraction kicks in.
const BASELINE_WARMUP: u64 = 50;
/// Apply EMA smoothing, adaptive baseline subtraction, and hysteresis debounce
/// to the raw classification. Mutates the smoothing state in `AppStateInner`.
fn smooth_and_classify(state: &mut AppStateInner, raw: &mut ClassificationInfo, raw_motion: f64) {
// 1. Adaptive baseline: slowly track the "quiet room" floor.
// Only update baseline when raw score is below the current smoothed level
// (i.e. during calm periods) so walking doesn't inflate the baseline.
state.baseline_frames += 1;
if state.baseline_frames < BASELINE_WARMUP {
// During warm-up, aggressively learn the baseline.
state.baseline_motion = state.baseline_motion * 0.9 + raw_motion * 0.1;
} else if raw_motion < state.smoothed_motion + 0.05 {
state.baseline_motion = state.baseline_motion * (1.0 - BASELINE_EMA_ALPHA)
+ raw_motion * BASELINE_EMA_ALPHA;
}
// 2. Subtract baseline and clamp.
let adjusted = (raw_motion - state.baseline_motion * 0.7).max(0.0);
// 3. EMA smooth the adjusted score.
state.smoothed_motion = state.smoothed_motion * (1.0 - MOTION_EMA_ALPHA)
+ adjusted * MOTION_EMA_ALPHA;
let sm = state.smoothed_motion;
// 4. Classify from smoothed score.
let candidate = raw_classify(sm);
// 5. Hysteresis debounce: require N consecutive frames agreeing on a new state.
if candidate == state.current_motion_level {
// Already in this state — reset debounce.
state.debounce_counter = 0;
state.debounce_candidate = candidate;
} else if candidate == state.debounce_candidate {
state.debounce_counter += 1;
if state.debounce_counter >= DEBOUNCE_FRAMES {
// Transition accepted.
state.current_motion_level = candidate;
state.debounce_counter = 0;
}
} else {
("absent".to_string(), false)
};
// New candidate — restart counter.
state.debounce_candidate = candidate;
state.debounce_counter = 1;
}
let confidence = (0.4 + signal_quality * 0.3 + motion_score * 0.3).clamp(0.0, 1.0);
// 6. Write the smoothed result back into the classification.
raw.motion_level = state.current_motion_level.clone();
raw.presence = sm > 0.03;
raw.confidence = (0.4 + sm * 0.6).clamp(0.0, 1.0);
}
let classification = ClassificationInfo {
motion_level,
presence,
confidence,
};
/// If an adaptive model is loaded, override the classification with the
/// model's prediction. Uses the full 15-feature vector for higher accuracy.
fn adaptive_override(state: &AppStateInner, features: &FeatureInfo, classification: &mut ClassificationInfo) {
if let Some(ref model) = state.adaptive_model {
// Get current frame amplitudes from the latest history entry.
let amps = state.frame_history.back()
.map(|v| v.as_slice())
.unwrap_or(&[]);
let feat_arr = adaptive_classifier::features_from_runtime(
&serde_json::json!({
"variance": features.variance,
"motion_band_power": features.motion_band_power,
"breathing_band_power": features.breathing_band_power,
"spectral_power": features.spectral_power,
"dominant_freq_hz": features.dominant_freq_hz,
"change_points": features.change_points,
"mean_rssi": features.mean_rssi,
}),
amps,
);
let (label, conf) = model.classify(&feat_arr);
classification.motion_level = label.to_string();
classification.presence = label != "absent";
// Blend model confidence with existing smoothed confidence.
classification.confidence = (conf * 0.7 + classification.confidence * 0.3).clamp(0.0, 1.0);
}
}
(features, classification, breathing_rate_hz, sub_variances)
/// Size of the median filter window for vital signs outlier rejection.
const VITAL_MEDIAN_WINDOW: usize = 21;
/// EMA alpha for vital signs (~5s time constant at 10 FPS).
const VITAL_EMA_ALPHA: f64 = 0.02;
/// Maximum BPM jump per frame before a value is rejected as an outlier.
const HR_MAX_JUMP: f64 = 8.0;
const BR_MAX_JUMP: f64 = 2.0;
/// Minimum change from current smoothed value before EMA updates (dead-band).
/// Prevents micro-drift from creeping in.
const HR_DEAD_BAND: f64 = 2.0;
const BR_DEAD_BAND: f64 = 0.5;
/// Smooth vital signs using median-filter outlier rejection + EMA.
/// Mutates `state.smoothed_hr`, `state.smoothed_br`, etc.
/// Returns the smoothed VitalSigns to broadcast.
fn smooth_vitals(state: &mut AppStateInner, raw: &VitalSigns) -> VitalSigns {
let raw_hr = raw.heart_rate_bpm.unwrap_or(0.0);
let raw_br = raw.breathing_rate_bpm.unwrap_or(0.0);
// -- Outlier rejection: skip values that jump too far from current EMA --
let hr_ok = state.smoothed_hr < 1.0 || (raw_hr - state.smoothed_hr).abs() < HR_MAX_JUMP;
let br_ok = state.smoothed_br < 1.0 || (raw_br - state.smoothed_br).abs() < BR_MAX_JUMP;
// Push into buffer (only non-outlier values)
if hr_ok && raw_hr > 0.0 {
state.hr_buffer.push_back(raw_hr);
if state.hr_buffer.len() > VITAL_MEDIAN_WINDOW { state.hr_buffer.pop_front(); }
}
if br_ok && raw_br > 0.0 {
state.br_buffer.push_back(raw_br);
if state.br_buffer.len() > VITAL_MEDIAN_WINDOW { state.br_buffer.pop_front(); }
}
// Compute trimmed mean: drop top/bottom 25% then average the middle 50%.
// This is more stable than pure median and less noisy than raw mean.
let trimmed_hr = trimmed_mean(&state.hr_buffer);
let trimmed_br = trimmed_mean(&state.br_buffer);
// EMA smooth with dead-band: only update if the trimmed mean differs
// from the current smoothed value by more than the dead-band.
// This prevents the display from constantly creeping by tiny amounts.
if trimmed_hr > 0.0 {
if state.smoothed_hr < 1.0 {
state.smoothed_hr = trimmed_hr;
} else if (trimmed_hr - state.smoothed_hr).abs() > HR_DEAD_BAND {
state.smoothed_hr = state.smoothed_hr * (1.0 - VITAL_EMA_ALPHA)
+ trimmed_hr * VITAL_EMA_ALPHA;
}
// else: within dead-band, hold current value
}
if trimmed_br > 0.0 {
if state.smoothed_br < 1.0 {
state.smoothed_br = trimmed_br;
} else if (trimmed_br - state.smoothed_br).abs() > BR_DEAD_BAND {
state.smoothed_br = state.smoothed_br * (1.0 - VITAL_EMA_ALPHA)
+ trimmed_br * VITAL_EMA_ALPHA;
}
}
// Smooth confidence
state.smoothed_hr_conf = state.smoothed_hr_conf * 0.92 + raw.heartbeat_confidence * 0.08;
state.smoothed_br_conf = state.smoothed_br_conf * 0.92 + raw.breathing_confidence * 0.08;
VitalSigns {
breathing_rate_bpm: if state.smoothed_br > 1.0 { Some(state.smoothed_br) } else { None },
heart_rate_bpm: if state.smoothed_hr > 1.0 { Some(state.smoothed_hr) } else { None },
breathing_confidence: state.smoothed_br_conf,
heartbeat_confidence: state.smoothed_hr_conf,
signal_quality: raw.signal_quality,
}
}
/// Trimmed mean: sort, drop top/bottom 25%, average the middle 50%.
/// More robust than median (uses more data) and less noisy than raw mean.
fn trimmed_mean(buf: &VecDeque<f64>) -> f64 {
if buf.is_empty() { return 0.0; }
let mut sorted: Vec<f64> = buf.iter().copied().collect();
sorted.sort_by(|a, b| a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal));
let n = sorted.len();
let trim = n / 4; // drop 25% from each end
let middle = &sorted[trim..n - trim.max(0)];
if middle.is_empty() {
sorted[n / 2] // fallback to median if too few samples
} else {
middle.iter().sum::<f64>() / middle.len() as f64
}
}
// ── Windows WiFi RSSI collector ──────────────────────────────────────────────
@@ -982,8 +1196,10 @@ async fn windows_wifi_task(state: SharedState, tick_ms: u64) {
s_write_pre.frame_history.pop_front();
}
let sample_rate_hz = 1000.0 / tick_ms as f64;
let (features, classification, breathing_rate_hz, sub_variances) =
let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) =
extract_features_from_frame(&frame, &s_write_pre.frame_history, sample_rate_hz);
smooth_and_classify(&mut s_write_pre, &mut classification, raw_motion);
adaptive_override(&s_write_pre, &features, &mut classification);
drop(s_write_pre);
// ── Step 5: Build enhanced fields from pipeline result ───────
@@ -1025,7 +1241,8 @@ async fn windows_wifi_task(state: SharedState, tick_ms: u64) {
0.05
};
let vitals = s.vital_detector.process_frame(&frame.amplitudes, &frame.phases);
let raw_vitals = s.vital_detector.process_frame(&frame.amplitudes, &frame.phases);
let vitals = smooth_vitals(&mut s, &raw_vitals);
s.latest_vitals = vitals.clone();
let feat_variance = features.variance;
@@ -1132,8 +1349,10 @@ async fn windows_wifi_fallback_tick(state: &SharedState, seq: u32) {
s.frame_history.pop_front();
}
let sample_rate_hz = 2.0_f64; // fallback tick ~ 500 ms => 2 Hz
let (features, classification, breathing_rate_hz, sub_variances) =
let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) =
extract_features_from_frame(&frame, &s.frame_history, sample_rate_hz);
smooth_and_classify(&mut s, &mut classification, raw_motion);
adaptive_override(&s, &features, &mut classification);
s.source = format!("wifi:{ssid}");
s.rssi_history.push_back(rssi_dbm);
@@ -1152,7 +1371,8 @@ async fn windows_wifi_fallback_tick(state: &SharedState, seq: u32) {
0.05
};
let vitals = s.vital_detector.process_frame(&frame.amplitudes, &frame.phases);
let raw_vitals = s.vital_detector.process_frame(&frame.amplitudes, &frame.phases);
let vitals = smooth_vitals(&mut s, &raw_vitals);
s.latest_vitals = vitals.clone();
let feat_variance = features.variance;
@@ -1896,7 +2116,15 @@ async fn delete_model(
State(state): State<SharedState>,
Path(id): Path<String>,
) -> Json<serde_json::Value> {
let path = PathBuf::from("data/models").join(format!("{}.rvf", id));
// ADR-050: Sanitize path to prevent directory traversal
let safe_id = std::path::Path::new(&id)
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("");
if safe_id.is_empty() || safe_id != id {
return Json(serde_json::json!({ "error": "invalid model id", "success": false }));
}
let path = PathBuf::from("data/models").join(format!("{}.rvf", safe_id));
if path.exists() {
if let Err(e) = std::fs::remove_file(&path) {
warn!("Failed to delete model file {:?}: {}", path, e);
@@ -2147,7 +2375,15 @@ async fn delete_recording(
State(state): State<SharedState>,
Path(id): Path<String>,
) -> Json<serde_json::Value> {
let path = PathBuf::from("data/recordings").join(format!("{}.jsonl", id));
// ADR-050: Sanitize path to prevent directory traversal
let safe_id = std::path::Path::new(&id)
.file_name()
.and_then(|f| f.to_str())
.unwrap_or("");
if safe_id.is_empty() || safe_id != id {
return Json(serde_json::json!({ "error": "invalid recording id", "success": false }));
}
let path = PathBuf::from("data/recordings").join(format!("{}.jsonl", safe_id));
if path.exists() {
if let Err(e) = std::fs::remove_file(&path) {
warn!("Failed to delete recording {:?}: {}", path, e);
@@ -2251,6 +2487,77 @@ async fn train_stop(State(state): State<SharedState>) -> Json<serde_json::Value>
}))
}
// ── Adaptive classifier endpoints ────────────────────────────────────────────
/// POST /api/v1/adaptive/train — train the adaptive classifier from recordings.
async fn adaptive_train(State(state): State<SharedState>) -> Json<serde_json::Value> {
let rec_dir = PathBuf::from("data/recordings");
eprintln!("=== Adaptive Classifier Training ===");
match adaptive_classifier::train_from_recordings(&rec_dir) {
Ok(model) => {
let accuracy = model.training_accuracy;
let frames = model.trained_frames;
let stats: Vec<_> = model.class_stats.iter().map(|cs| {
serde_json::json!({
"class": cs.label,
"samples": cs.count,
"feature_means": cs.mean,
})
}).collect();
// Save to disk.
if let Err(e) = model.save(&adaptive_classifier::model_path()) {
warn!("Failed to save adaptive model: {e}");
} else {
info!("Adaptive model saved to {}", adaptive_classifier::model_path().display());
}
// Load into runtime state.
let mut s = state.write().await;
s.adaptive_model = Some(model);
Json(serde_json::json!({
"success": true,
"trained_frames": frames,
"accuracy": accuracy,
"class_stats": stats,
}))
}
Err(e) => {
Json(serde_json::json!({
"success": false,
"error": e,
}))
}
}
}
/// GET /api/v1/adaptive/status — check adaptive model status.
async fn adaptive_status(State(state): State<SharedState>) -> Json<serde_json::Value> {
let s = state.read().await;
match &s.adaptive_model {
Some(model) => Json(serde_json::json!({
"loaded": true,
"trained_frames": model.trained_frames,
"accuracy": model.training_accuracy,
"version": model.version,
"classes": adaptive_classifier::CLASSES,
"class_stats": model.class_stats,
})),
None => Json(serde_json::json!({
"loaded": false,
"message": "No adaptive model. POST /api/v1/adaptive/train to train one.",
})),
}
}
/// POST /api/v1/adaptive/unload — unload the adaptive model (revert to thresholds).
async fn adaptive_unload(State(state): State<SharedState>) -> Json<serde_json::Value> {
let mut s = state.write().await;
s.adaptive_model = None;
Json(serde_json::json!({ "success": true, "message": "Adaptive model unloaded." }))
}
/// Generate a simple timestamp string (epoch seconds) for recording IDs.
fn chrono_timestamp() -> u64 {
std::time::SystemTime::now()
@@ -2492,8 +2799,10 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
}
let sample_rate_hz = 1000.0 / 500.0_f64; // default tick; ESP32 frames arrive as fast as they come
let (features, classification, breathing_rate_hz, sub_variances) =
let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) =
extract_features_from_frame(&frame, &s.frame_history, sample_rate_hz);
smooth_and_classify(&mut s, &mut classification, raw_motion);
adaptive_override(&s, &features, &mut classification);
// Update RSSI history
s.rssi_history.push_back(features.mean_rssi);
@@ -2508,10 +2817,11 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
else if classification.motion_level == "present_still" { 0.3 }
else { 0.05 };
let vitals = s.vital_detector.process_frame(
let raw_vitals = s.vital_detector.process_frame(
&frame.amplitudes,
&frame.phases,
);
let vitals = smooth_vitals(&mut s, &raw_vitals);
s.latest_vitals = vitals.clone();
// Multi-person estimation with temporal smoothing.
@@ -2595,8 +2905,10 @@ async fn simulated_data_task(state: SharedState, tick_ms: u64) {
}
let sample_rate_hz = 1000.0 / tick_ms as f64;
let (features, classification, breathing_rate_hz, sub_variances) =
let (features, mut classification, breathing_rate_hz, sub_variances, raw_motion) =
extract_features_from_frame(&frame, &s.frame_history, sample_rate_hz);
smooth_and_classify(&mut s, &mut classification, raw_motion);
adaptive_override(&s, &features, &mut classification);
s.rssi_history.push_back(features.mean_rssi);
if s.rssi_history.len() > 60 {
@@ -2607,10 +2919,11 @@ async fn simulated_data_task(state: SharedState, tick_ms: u64) {
else if classification.motion_level == "present_still" { 0.3 }
else { 0.05 };
let vitals = s.vital_detector.process_frame(
let raw_vitals = s.vital_detector.process_frame(
&frame.amplitudes,
&frame.phases,
);
let vitals = smooth_vitals(&mut s, &raw_vitals);
s.latest_vitals = vitals.clone();
let frame_amplitudes = frame.amplitudes.clone();
@@ -3264,6 +3577,18 @@ async fn main() {
active_sona_profile: None,
model_loaded,
smoothed_person_score: 0.0,
smoothed_motion: 0.0,
current_motion_level: "absent".to_string(),
debounce_counter: 0,
debounce_candidate: "absent".to_string(),
baseline_motion: 0.0,
baseline_frames: 0,
smoothed_hr: 0.0,
smoothed_br: 0.0,
smoothed_hr_conf: 0.0,
smoothed_br_conf: 0.0,
hr_buffer: VecDeque::with_capacity(8),
br_buffer: VecDeque::with_capacity(8),
edge_vitals: None,
latest_wasm_events: None,
// Model management
@@ -3278,6 +3603,11 @@ async fn main() {
// Training
training_status: "idle".to_string(),
training_config: None,
adaptive_model: adaptive_classifier::AdaptiveModel::load(&adaptive_classifier::model_path()).ok().map(|m| {
info!("Loaded adaptive classifier: {} frames, {:.1}% accuracy",
m.trained_frames, m.training_accuracy * 100.0);
m
}),
}));
// Start background tasks based on source
@@ -3294,6 +3624,10 @@ async fn main() {
}
}
// ADR-050: Parse bind address once, use for all listeners
let bind_ip: std::net::IpAddr = args.bind_addr.parse()
.expect("Invalid --bind-addr (use 127.0.0.1 or 0.0.0.0)");
// WebSocket server on dedicated port (8765)
let ws_state = state.clone();
let ws_app = Router::new()
@@ -3301,7 +3635,7 @@ async fn main() {
.route("/health", get(health))
.with_state(ws_state);
let ws_addr = SocketAddr::from(([0, 0, 0, 0], args.ws_port));
let ws_addr = SocketAddr::from((bind_ip, args.ws_port));
let ws_listener = tokio::net::TcpListener::bind(ws_addr).await
.expect("Failed to bind WebSocket port");
info!("WebSocket server listening on {ws_addr}");
@@ -3364,6 +3698,10 @@ async fn main() {
.route("/api/v1/train/status", get(train_status))
.route("/api/v1/train/start", post(train_start))
.route("/api/v1/train/stop", post(train_stop))
// Adaptive classifier endpoints
.route("/api/v1/adaptive/train", post(adaptive_train))
.route("/api/v1/adaptive/status", get(adaptive_status))
.route("/api/v1/adaptive/unload", post(adaptive_unload))
// Static UI files
.nest_service("/ui", ServeDir::new(&ui_path))
.layer(SetResponseHeaderLayer::overriding(
@@ -3372,7 +3710,7 @@ async fn main() {
))
.with_state(state.clone());
let http_addr = SocketAddr::from(([0, 0, 0, 0], args.http_port));
let http_addr = SocketAddr::from((bind_ip, args.http_port));
let http_listener = tokio::net::TcpListener::bind(http_addr).await
.expect("Failed to bind HTTP port");
info!("HTTP server listening on {http_addr}");
@@ -0,0 +1,267 @@
{
"class_stats": [
{
"label": "absent",
"count": 862,
"mean": [
66.68196972264862,
67.23973219951662,
65.0340640002779,
205.65861248066514,
1.2587006960556917,
8.192575406032482,
0.0,
9.823395623712905,
6.970045450727901,
-0.04488812678641681,
-0.9594767860850162,
10.78889030301701,
0.8330000846014487,
22.47189099978742,
22.47189099978742
],
"stddev": [
64.0493846652119,
90.27545165651007,
40.157907144682206,
161.60550836256004,
1.3807130815029451,
3.2814660018571113,
0.0,
2.219723108446689,
1.6521309619598676,
0.342852106459665,
0.30620004291079783,
3.529722483499124,
0.17574148506941875,
5.519861526721805,
5.519861526721805
]
},
{
"label": "present_still",
"count": 852,
"mean": [
66.39259262094396,
64.42298266818027,
68.34546366405283,
203.34049479166666,
1.1900821596244182,
8.200704225352112,
0.0,
10.032339700775715,
7.234479413048846,
0.027056637948278107,
-0.9161490234231624,
10.991429347401095,
0.8298622589530178,
23.588978503428145,
23.588978503428145
],
"stddev": [
59.144593976065984,
82.61098004853669,
40.08306971525127,
152.89405234329087,
1.2031203046363153,
3.0571012493320526,
0.0,
2.22294769203091,
1.6508044238677446,
0.3315329147240876,
0.29437997092330526,
3.3214071045026303,
0.17096813624285292,
5.622953396738593,
5.622953396738593
]
},
{
"label": "present_moving",
"count": 808,
"mean": [
65.17005228763453,
66.55424930761484,
63.785855267654334,
208.73719832920793,
1.3400990099009942,
7.570544554455446,
0.0,
10.069915394050431,
6.923405617584522,
-0.1440461642917184,
-1.0022460352626226,
10.664608744841848,
0.8384559212414682,
21.798331033369895,
21.798331033369895
],
"stddev": [
66.1800697503931,
93.22042148141067,
42.07226450730718,
164.93282045618218,
1.3706144246607475,
3.1453995481213224,
0.0,
2.431170975696439,
1.672707406405861,
0.35643090355922863,
0.30897080072710387,
3.325911716352165,
0.1806597020966414,
5.418714527442832,
5.418714527442832
]
},
{
"label": "active",
"count": 794,
"mean": [
61.85289600233076,
61.12723986655727,
62.468831971775344,
193.2018524349286,
1.2329974811083138,
8.083123425692696,
0.0,
9.747035051350043,
7.009904234422278,
0.007176072447431498,
-0.9950501087764124,
11.015545839210892,
0.8278984910895401,
22.445656559614797,
22.445656559614797
],
"stddev": [
50.44687370766278,
74.07914900524236,
31.558067649516538,
121.0762294406304,
1.2507304998955402,
3.4503520526220344,
0.0,
2.2730029390882156,
1.6768264387667406,
0.3214256392367928,
0.31003127617615406,
3.1187829194728285,
0.1772099351197549,
5.595050695741912,
5.595050695741912
]
}
],
"weights": [
[
0.9923736589617821,
-0.4600422332552322,
-0.3922101552522972,
-0.1686954616947851,
-0.08471937018349271,
0.033940973559074515,
0.0,
-1.116294981490482,
-0.213861080404439,
-0.41727297566573723,
0.08025552056009382,
0.20864577739519874,
0.36814779033318357,
0.46242679535538855,
0.46242679535538855,
0.09475205040199337
],
[
0.04661470129518883,
0.7974124099989739,
0.3953040913806362,
-1.2708868935843511,
0.10073070355913086,
0.0735810797517633,
0.0,
-0.3957608057630568,
0.22091779039114648,
-0.43105406953304665,
0.24907697332262252,
-0.17604200203759515,
-0.5059663705836186,
0.5740861193153091,
0.5740861193153091,
0.020569218347928304
],
[
-0.5295363836864718,
0.14729609046092632,
0.16131671233151712,
0.15039859740752318,
0.08189110214725194,
-0.1429062024394049,
0.0,
2.459247211223509,
-0.162133339181718,
0.6345474095048843,
0.16626892477248892,
0.2710091094981082,
-0.08197569509399917,
-1.2007197895193034,
-1.2007197895193034,
-0.10027402587742726
],
[
-0.5094519765704947,
-0.48466626720467487,
-0.1644106484598614,
1.2891837578716183,
-0.0979024355228887,
0.0353841491285671,
0.0,
-0.9471914239699604,
0.15507662919500606,
0.2137796356938993,
-0.49560141865520463,
-0.30361288485571664,
0.21979427534444013,
0.16420687484859928,
0.16420687484859928,
-0.015047242872495047
]
],
"global_mean": [
65.08291570815048,
64.88537161757283,
64.96650236787292,
202.8304440905207,
1.25474969843183,
8.016887816646562,
0.0,
9.918865477040464,
7.036167472733628,
-0.038097952045357715,
-0.9672836370393502,
10.86491812646321,
0.8323017200972911,
22.58850497890069,
22.58850497890069
],
"global_std": [
60.376895354908775,
85.49291935872783,
38.814475392686795,
151.54766198012683,
1.3049002582695195,
3.2446975526483737,
1e-9,
2.2904371592847603,
1.667114434239705,
0.34470363318292857,
0.3067332188136679,
3.334427501751985,
0.17614366955910027,
5.577838072123601,
5.577838072123601
],
"trained_frames": 3316,
"training_accuracy": 0.4149577804583836,
"version": 1
}
+1
View File
@@ -29,6 +29,7 @@
<button class="nav-tab" data-tab="applications">Applications</button>
<button class="nav-tab" data-tab="sensing">Sensing</button>
<button class="nav-tab" data-tab="training">Training</button>
<a href="observatory.html" class="nav-tab" style="text-decoration:none">Observatory</a>
</nav>
<!-- Dashboard Tab -->
+340
View File
@@ -0,0 +1,340 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>RuView Observatory — WiFi DensePose</title>
<link rel="stylesheet" href="observatory/css/observatory.css">
</head>
<body>
<canvas id="observatory-canvas"></canvas>
<!-- ======= HUD Overlay ======= -->
<div id="hud">
<!-- Top-left: Branding -->
<div id="brand">
<div id="brand-logo"><span class="pi">&pi;</span> RuView</div>
<div id="brand-tagline">WiFi DensePose Sensing Observatory</div>
</div>
<!-- Top-right: Connection + status + gear -->
<div id="status-bar">
<div id="data-source-badge">
<span class="dot dot--demo"></span>
<span id="data-source-label">DEMO</span>
</div>
<div id="scenario-area">
<span id="autoplay-icon" title="Auto-cycling">&#9654;</span>
<select id="scenario-quick-select" title="Change scenario">
<option value="auto">Auto-Cycle</option>
<option value="empty_room">Empty Room</option>
<option value="single_breathing">Vital Signs</option>
<option value="two_walking">Multi-Person</option>
<option value="fall_event">Fall Detect</option>
<option value="sleep_monitoring">Sleep Monitor</option>
<option value="intrusion_detect">Intrusion</option>
<option value="gesture_control">Gesture Ctrl</option>
<option value="crowd_occupancy">Crowd (4 ppl)</option>
<option value="search_rescue">Search Rescue</option>
<option value="elderly_care">Elderly Care</option>
<option value="fitness_tracking">Fitness</option>
<option value="security_patrol">Security Patrol</option>
</select>
</div>
<div id="scenario-description"></div>
<div id="fps-counter" style="display:none">60 FPS</div>
<button id="settings-btn" title="Settings">&#9881;</button>
</div>
<!-- Left panel: Vital Signs -->
<div id="panel-vitals" class="data-panel">
<div class="panel-header">Vital Signs</div>
<div class="vital-row">
<div class="vital-icon">&#9825;</div>
<div class="vital-data">
<div class="vital-label">Heart Rate</div>
<div class="vital-value"><span id="hr-value">--</span> <span class="vital-unit">BPM</span></div>
<div class="vital-bar"><div id="hr-bar" class="vital-bar-fill vital-bar--hr"></div></div>
</div>
</div>
<div class="vital-row">
<div class="vital-icon">&#9788;</div>
<div class="vital-data">
<div class="vital-label">Respiration</div>
<div class="vital-value"><span id="br-value">--</span> <span class="vital-unit">RPM</span></div>
<div class="vital-bar"><div id="br-bar" class="vital-bar-fill vital-bar--br"></div></div>
</div>
</div>
<div class="vital-row">
<div class="vital-icon">&#9878;</div>
<div class="vital-data">
<div class="vital-label">Confidence</div>
<div class="vital-value"><span id="conf-value">--</span><span class="vital-unit">%</span></div>
<div class="vital-bar"><div id="conf-bar" class="vital-bar-fill vital-bar--conf"></div></div>
</div>
</div>
</div>
<!-- Right panel: Signal & Presence -->
<div id="panel-signal" class="data-panel">
<div class="panel-header">WiFi Signal</div>
<div class="signal-row">
<span class="signal-label">RSSI</span>
<span class="signal-value" id="rssi-value">-- dBm</span>
</div>
<div class="signal-row">
<span class="signal-label">Variance</span>
<span class="signal-value" id="var-value">--</span>
</div>
<div class="signal-row">
<span class="signal-label">Motion</span>
<span class="signal-value" id="motion-value">--</span>
</div>
<div class="signal-row">
<span class="signal-label">Persons</span>
<span class="signal-value" id="persons-value">0</span>
<span id="persons-dots" class="persons-dots"></span>
</div>
<canvas id="rssi-sparkline" width="200" height="48"></canvas>
<div class="panel-header" style="margin-top:12px">Presence</div>
<div id="presence-indicator" class="presence-state presence--absent">
<span id="presence-label">ABSENT</span>
</div>
<div id="fall-alert" class="fall-alert" style="display:none">FALL DETECTED</div>
</div>
<!-- Edge module badges (populated dynamically by HudController) -->
<div id="edge-modules-bar"></div>
<!-- Bottom bar: capabilities -->
<div id="capabilities-bar">
<div class="cap-item"><span class="cap-icon">&#9898;</span><span>Human Pose Estimation</span></div>
<div class="cap-divider"></div>
<div class="cap-item"><span class="cap-icon">&#9829;</span><span>Vital Sign Monitoring</span></div>
<div class="cap-divider"></div>
<div class="cap-item"><span class="cap-icon">&#9784;</span><span>Presence Detection</span></div>
</div>
<!-- Bottom-right: keyboard hints -->
<div id="key-hints">
<span class="key-hint">[A] Orbit</span>
<span class="key-hint">[D] Scenario</span>
<span class="key-hint">[F] FPS</span>
<span class="key-hint">[S] Settings</span>
<span class="key-hint">[Space] Pause</span>
</div>
</div>
<!-- ======= Settings Dialog ======= -->
<div id="settings-overlay" class="settings-overlay" style="display:none">
<div class="settings-dialog">
<div class="settings-header">
<span>Settings</span>
<button id="settings-close">&times;</button>
</div>
<div class="settings-tabs">
<button class="stab active" data-stab="rendering">Rendering</button>
<button class="stab" data-stab="wireframe">Wireframe</button>
<button class="stab" data-stab="scene">Scene</button>
<button class="stab" data-stab="data">Data</button>
</div>
<!-- Rendering tab -->
<div class="stab-content active" id="stab-rendering">
<label class="setting-row">
<span>Bloom Strength</span>
<input type="range" id="opt-bloom" min="0" max="3" step="0.1" value="1.0">
<span class="range-val" id="opt-bloom-val">1.0</span>
</label>
<label class="setting-row">
<span>Bloom Radius</span>
<input type="range" id="opt-bloom-radius" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-bloom-radius-val">0.5</span>
</label>
<label class="setting-row">
<span>Bloom Threshold</span>
<input type="range" id="opt-bloom-thresh" min="0" max="1" step="0.05" value="0.25">
<span class="range-val" id="opt-bloom-thresh-val">0.25</span>
</label>
<label class="setting-row">
<span>Exposure</span>
<input type="range" id="opt-exposure" min="0.3" max="2" step="0.05" value="0.9">
<span class="range-val" id="opt-exposure-val">0.9</span>
</label>
<label class="setting-row">
<span>Vignette</span>
<input type="range" id="opt-vignette" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-vignette-val">0.5</span>
</label>
<label class="setting-row">
<span>Film Grain</span>
<input type="range" id="opt-grain" min="0" max="0.15" step="0.005" value="0.03">
<span class="range-val" id="opt-grain-val">0.03</span>
</label>
<label class="setting-row">
<span>Chromatic Aberration</span>
<input type="range" id="opt-chromatic" min="0" max="0.008" step="0.0005" value="0.0015">
<span class="range-val" id="opt-chromatic-val">0.0015</span>
</label>
</div>
<!-- Wireframe tab -->
<div class="stab-content" id="stab-wireframe">
<label class="setting-row">
<span>Bone Thickness</span>
<input type="range" id="opt-bone-thick" min="0.005" max="0.06" step="0.002" value="0.02">
<span class="range-val" id="opt-bone-thick-val">0.02</span>
</label>
<label class="setting-row">
<span>Joint Size</span>
<input type="range" id="opt-joint-size" min="0.02" max="0.12" step="0.005" value="0.05">
<span class="range-val" id="opt-joint-size-val">0.05</span>
</label>
<label class="setting-row">
<span>Glow Intensity</span>
<input type="range" id="opt-glow" min="0" max="2" step="0.1" value="0.8">
<span class="range-val" id="opt-glow-val">0.8</span>
</label>
<label class="setting-row">
<span>Particle Trail</span>
<input type="range" id="opt-trail" min="0" max="1" step="0.05" value="0.6">
<span class="range-val" id="opt-trail-val">0.6</span>
</label>
<label class="setting-row">
<span>Wireframe Color</span>
<input type="color" id="opt-wire-color" value="#00d878">
</label>
<label class="setting-row">
<span>Joint Color</span>
<input type="color" id="opt-joint-color" value="#ff4060">
</label>
<label class="setting-row">
<span>Aura Opacity</span>
<input type="range" id="opt-aura" min="0" max="0.2" step="0.01" value="0.06">
<span class="range-val" id="opt-aura-val">0.06</span>
</label>
</div>
<!-- Scene tab -->
<div class="stab-content" id="stab-scene">
<label class="setting-row">
<span>Signal Field</span>
<input type="range" id="opt-field" min="0" max="1" step="0.05" value="0.5">
<span class="range-val" id="opt-field-val">0.5</span>
</label>
<label class="setting-row">
<span>WiFi Waves</span>
<input type="range" id="opt-waves" min="0" max="1" step="0.05" value="0.6">
<span class="range-val" id="opt-waves-val">0.6</span>
</label>
<label class="setting-row">
<span>Room Brightness</span>
<input type="range" id="opt-ambient" min="0" max="1" step="0.05" value="0.4">
<span class="range-val" id="opt-ambient-val">0.4</span>
</label>
<label class="setting-row">
<span>Floor Reflection</span>
<input type="range" id="opt-reflect" min="0" max="1" step="0.05" value="0.3">
<span class="range-val" id="opt-reflect-val">0.3</span>
</label>
<label class="setting-row">
<span>FOV</span>
<input type="range" id="opt-fov" min="30" max="90" step="1" value="50">
<span class="range-val" id="opt-fov-val">50</span>
</label>
<label class="setting-row">
<span>Orbit Speed</span>
<input type="range" id="opt-orbit-speed" min="0.02" max="0.5" step="0.02" value="0.15">
<span class="range-val" id="opt-orbit-speed-val">0.15</span>
</label>
<label class="setting-row check-row">
<span>Show Grid</span>
<input type="checkbox" id="opt-grid" checked>
</label>
<label class="setting-row check-row">
<span>Show Room Boundary</span>
<input type="checkbox" id="opt-room" checked>
</label>
</div>
<!-- Data tab -->
<div class="stab-content" id="stab-data">
<label class="setting-row">
<span>Scenario</span>
<select id="opt-scenario">
<option value="auto">Auto-Cycle (30s)</option>
<optgroup label="Core Sensing">
<option value="empty_room">Empty Room</option>
<option value="single_breathing">Vital Signs (Breathing)</option>
<option value="two_walking">Multi-Person Tracking</option>
<option value="fall_event">Fall Detection</option>
</optgroup>
<optgroup label="Medical / Health">
<option value="sleep_monitoring">Sleep Monitoring (Apnea)</option>
<option value="elderly_care">Elderly Care (Gait)</option>
<option value="fitness_tracking">Fitness Tracking</option>
</optgroup>
<optgroup label="Security">
<option value="intrusion_detect">Intrusion Detection</option>
<option value="security_patrol">Security Patrol</option>
</optgroup>
<optgroup label="Building / Retail">
<option value="crowd_occupancy">Crowd Occupancy (4 ppl)</option>
<option value="gesture_control">Gesture Control (DTW)</option>
</optgroup>
<optgroup label="Disaster / Tactical">
<option value="search_rescue">Search &amp; Rescue (WiFi-Mat)</option>
</optgroup>
</select>
</label>
<label class="setting-row">
<span>Cycle Speed (s)</span>
<input type="range" id="opt-cycle" min="10" max="120" step="5" value="30">
<span class="range-val" id="opt-cycle-val">30</span>
</label>
<label class="setting-row">
<span>Style Preset</span>
<select id="opt-preset">
<option value="custom">Custom</option>
<option value="foundation">Foundation (Default)</option>
<option value="cinematic">Cinematic</option>
<option value="minimal">Minimal / Clean</option>
<option value="neon">Neon Glow</option>
<option value="tactical">Tactical / Military</option>
<option value="medical">Medical Monitor</option>
</select>
</label>
<label class="setting-row">
<span>Data Source</span>
<select id="opt-data-source">
<option value="demo" selected>Demo Generator</option>
<option value="ws">Live WebSocket</option>
</select>
</label>
<label class="setting-row" id="ws-url-row" style="display:none">
<span>WS URL</span>
<input type="text" id="opt-ws-url" value="" placeholder="ws://localhost:3000/ws/sensing">
</label>
<button id="btn-reset-camera" class="settings-btn">Reset Camera</button>
<button id="btn-reset-settings" class="settings-btn">Reset to Defaults</button>
<button id="btn-export-settings" class="settings-btn">Export Settings</button>
</div>
</div>
</div>
<!-- Three.js r160 + addons from CDN -->
<script type="importmap">
{
"imports": {
"three": "https://unpkg.com/three@0.160.0/build/three.module.js",
"three/addons/": "https://unpkg.com/three@0.160.0/examples/jsm/"
}
}
</script>
<script type="module" src="observatory/js/main.js"></script>
</body>
</html>
+698
View File
@@ -0,0 +1,698 @@
/* ============================================================
RuView Observatory Foundation Color Scheme
Warm dark background, electric green wireframe, amber data
============================================================ */
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;700&family=JetBrains+Mono:wght@400;600&display=swap');
:root {
--bg-deep: #080c14;
--bg-panel: rgba(8, 16, 28, 0.85);
--bg-panel-border: rgba(0, 210, 120, 0.2);
--green-glow: #00d878;
--green-bright:#3eff8a;
--green-dim: #0a6b3a;
--amber: #ffb020;
--amber-dim: #a06800;
--blue-signal: #2090ff;
--blue-dim: #0a3060;
--red-alert: #ff3040;
--red-heart: #ff4060;
--text-primary: #e8ece0;
--text-secondary: rgba(232,236,224, 0.55);
--text-label: rgba(232,236,224, 0.4);
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
background: var(--bg-deep);
overflow: hidden;
font-family: 'Inter', -apple-system, sans-serif;
color: var(--text-primary);
-webkit-font-smoothing: antialiased;
}
#observatory-canvas {
position: fixed;
top: 0; left: 0;
width: 100vw; height: 100vh;
}
/* ---- HUD Overlay ---- */
#hud {
position: fixed;
top: 0; left: 0;
width: 100%; height: 100%;
pointer-events: none;
z-index: 10;
}
/* ---- Brand ---- */
#brand {
position: absolute;
top: 24px; left: 28px;
}
#brand-logo {
font-family: 'Inter', sans-serif;
font-weight: 700;
font-size: 32px;
color: var(--text-primary);
letter-spacing: -0.5px;
text-shadow: 0 0 30px rgba(0, 216, 120, 0.3);
}
.pi {
color: var(--green-glow);
font-style: italic;
margin-right: 2px;
}
#brand-tagline {
font-size: 11px;
color: var(--text-secondary);
letter-spacing: 1.5px;
text-transform: uppercase;
margin-top: 2px;
}
/* ---- Status bar (top right) ---- */
#status-bar {
position: absolute;
top: 24px; right: 28px;
display: flex;
align-items: center;
gap: 12px;
}
#data-source-badge {
display: flex;
align-items: center;
gap: 6px;
padding: 5px 12px;
border-radius: 20px;
background: rgba(0, 216, 120, 0.1);
border: 1px solid rgba(0, 216, 120, 0.25);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 1px;
color: var(--green-glow);
}
.dot {
width: 7px; height: 7px;
border-radius: 50%;
display: inline-block;
}
.dot--demo { background: var(--amber); box-shadow: 0 0 6px var(--amber); }
.dot--live { background: var(--green-glow); box-shadow: 0 0 6px var(--green-glow); animation: pulse-dot 2s infinite; }
@keyframes pulse-dot {
0%, 100% { opacity: 1; }
50% { opacity: 0.4; }
}
#scenario-area {
display: flex;
align-items: center;
gap: 8px;
padding: 5px 14px;
border-radius: 20px;
background: rgba(255, 176, 32, 0.1);
border: 1px solid rgba(255, 176, 32, 0.25);
pointer-events: auto;
}
#autoplay-icon {
font-size: 10px;
color: var(--green-glow);
animation: pulse-dot 2s infinite;
}
#autoplay-icon.hidden { display: none; }
#scenario-quick-select {
background: none;
border: none;
padding: 0;
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 0.5px;
color: var(--amber);
cursor: pointer;
outline: none;
}
#scenario-quick-select:hover,
#scenario-quick-select:focus { color: var(--green-glow); }
#scenario-quick-select option {
background: #0c1420;
color: var(--text-primary);
font-family: 'JetBrains Mono', monospace;
font-size: 12px;
padding: 4px 8px;
}
#fps-counter {
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
color: var(--text-secondary);
}
/* ---- Data Panels ---- */
.data-panel {
position: absolute;
width: 220px;
background: var(--bg-panel);
border: 1px solid var(--bg-panel-border);
border-radius: 12px;
padding: 16px;
backdrop-filter: blur(12px);
-webkit-backdrop-filter: blur(12px);
pointer-events: auto;
}
.panel-header {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
font-weight: 600;
letter-spacing: 2px;
text-transform: uppercase;
color: var(--text-label);
margin-bottom: 14px;
padding-bottom: 8px;
border-bottom: 1px solid rgba(255,255,255,0.06);
}
/* ---- Vitals Panel (left) ---- */
#panel-vitals {
left: 28px;
top: 50%;
transform: translateY(-50%);
}
.vital-row {
display: flex;
align-items: flex-start;
gap: 12px;
margin-bottom: 18px;
}
.vital-row:last-child { margin-bottom: 0; }
.vital-icon {
font-size: 20px;
line-height: 1;
margin-top: 2px;
width: 24px;
text-align: center;
}
.vital-row:nth-child(2) .vital-icon { color: var(--red-heart); }
.vital-row:nth-child(3) .vital-icon { color: var(--green-glow); }
.vital-row:nth-child(4) .vital-icon { color: var(--amber); }
.vital-data { flex: 1; }
.vital-label {
font-size: 10px;
color: var(--text-label);
letter-spacing: 1px;
text-transform: uppercase;
margin-bottom: 3px;
}
.vital-value {
font-family: 'JetBrains Mono', monospace;
font-size: 26px;
font-weight: 600;
line-height: 1.1;
}
.vital-unit {
font-size: 12px;
font-weight: 400;
color: var(--text-secondary);
}
.vital-bar {
height: 3px;
background: rgba(255,255,255,0.06);
border-radius: 2px;
margin-top: 6px;
overflow: hidden;
}
.vital-bar-fill {
height: 100%;
border-radius: 2px;
transition: width 0.5s ease;
}
.vital-bar--hr { background: var(--red-heart); width: 0%; }
.vital-bar--br { background: var(--green-glow); width: 0%; }
.vital-bar--conf { background: var(--amber); width: 0%; }
/* ---- Signal Panel (right) ---- */
#panel-signal {
right: 28px;
top: 50%;
transform: translateY(-50%);
}
.signal-row {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 8px;
}
.signal-label {
font-size: 11px;
color: var(--text-label);
letter-spacing: 0.5px;
}
.signal-value {
font-family: 'JetBrains Mono', monospace;
font-size: 13px;
font-weight: 600;
color: var(--blue-signal);
}
#rssi-sparkline {
width: 100%;
height: 48px;
margin-top: 8px;
border-radius: 6px;
background: rgba(0,0,0,0.3);
}
/* Presence */
.presence-state {
text-align: center;
padding: 8px;
border-radius: 8px;
font-family: 'JetBrains Mono', monospace;
font-size: 14px;
font-weight: 600;
letter-spacing: 2px;
transition: all 0.5s ease;
}
.presence--absent {
background: rgba(255,255,255,0.03);
color: var(--text-label);
border: 1px solid rgba(255,255,255,0.05);
}
.presence--present {
background: rgba(0, 216, 120, 0.1);
color: var(--green-glow);
border: 1px solid rgba(0, 216, 120, 0.3);
box-shadow: 0 0 20px rgba(0, 216, 120, 0.1);
}
.presence--active {
background: rgba(255, 176, 32, 0.1);
color: var(--amber);
border: 1px solid rgba(255, 176, 32, 0.3);
box-shadow: 0 0 20px rgba(255, 176, 32, 0.1);
}
.fall-alert {
margin-top: 10px;
text-align: center;
padding: 8px;
border-radius: 8px;
font-family: 'JetBrains Mono', monospace;
font-size: 12px;
font-weight: 700;
letter-spacing: 2px;
background: rgba(255, 48, 64, 0.15);
color: var(--red-alert);
border: 1px solid rgba(255, 48, 64, 0.4);
animation: pulse-alert 0.8s infinite;
}
@keyframes pulse-alert {
0%, 100% { opacity: 1; }
50% { opacity: 0.5; }
}
/* ---- Capabilities Bar (bottom center) ---- */
#capabilities-bar {
position: absolute;
bottom: 20px;
left: 50%;
transform: translateX(-50%);
display: flex;
align-items: center;
gap: 0;
background: var(--bg-panel);
border: 1px solid var(--bg-panel-border);
border-radius: 30px;
padding: 8px 24px;
backdrop-filter: blur(12px);
}
.cap-item {
display: flex;
align-items: center;
gap: 8px;
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
padding: 0 16px;
}
.cap-icon {
font-size: 16px;
color: var(--green-glow);
}
.cap-item:nth-child(3) .cap-icon { color: var(--red-heart); }
.cap-item:nth-child(5) .cap-icon { color: var(--blue-signal); }
.cap-divider {
width: 1px;
height: 20px;
background: rgba(255,255,255,0.1);
}
/* ---- Key hints ---- */
#key-hints {
position: absolute;
bottom: 24px;
right: 28px;
display: flex;
gap: 8px;
}
.key-hint {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
color: rgba(255,255,255,0.2);
letter-spacing: 0.5px;
padding: 3px 8px;
border-radius: 4px;
background: rgba(255,255,255,0.03);
border: 1px solid rgba(255,255,255,0.05);
}
/* ---- Settings button ---- */
#settings-btn {
pointer-events: auto;
background: rgba(255,255,255,0.06);
border: 1px solid rgba(255,255,255,0.1);
color: var(--text-secondary);
font-size: 18px;
width: 34px; height: 34px;
border-radius: 50%;
cursor: pointer;
transition: all 0.2s;
display: flex; align-items: center; justify-content: center;
padding: 0;
}
#settings-btn:hover {
background: rgba(0, 216, 120, 0.15);
border-color: var(--green-glow);
color: var(--green-glow);
}
/* ---- Settings Dialog ---- */
.settings-overlay {
position: fixed;
top: 0; left: 0;
width: 100%; height: 100%;
z-index: 100;
background: rgba(0,0,0,0.5);
backdrop-filter: blur(4px);
display: flex;
align-items: center;
justify-content: center;
pointer-events: auto;
}
.settings-dialog {
background: rgba(10, 16, 28, 0.96);
border: 1px solid rgba(0, 216, 120, 0.2);
border-radius: 16px;
width: 440px;
max-height: 80vh;
overflow-y: auto;
padding: 0;
box-shadow: 0 20px 60px rgba(0,0,0,0.6), 0 0 40px rgba(0,216,120,0.05);
}
.settings-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px 20px;
border-bottom: 1px solid rgba(255,255,255,0.06);
font-family: 'JetBrains Mono', monospace;
font-size: 13px;
font-weight: 600;
letter-spacing: 1px;
text-transform: uppercase;
color: var(--text-primary);
}
.settings-header button {
background: none;
border: none;
color: var(--text-secondary);
font-size: 22px;
cursor: pointer;
padding: 0 4px;
line-height: 1;
}
.settings-header button:hover { color: var(--red-alert); }
.settings-tabs {
display: flex;
border-bottom: 1px solid rgba(255,255,255,0.06);
padding: 0 12px;
}
.stab {
background: none;
border: none;
color: var(--text-label);
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
letter-spacing: 1px;
text-transform: uppercase;
padding: 10px 14px;
cursor: pointer;
border-bottom: 2px solid transparent;
transition: all 0.2s;
}
.stab:hover { color: var(--text-secondary); }
.stab.active {
color: var(--green-glow);
border-bottom-color: var(--green-glow);
}
.stab-content {
display: none;
padding: 16px 20px;
}
.stab-content.active { display: block; }
.setting-row {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
margin-bottom: 14px;
font-size: 12px;
color: var(--text-secondary);
}
.setting-row span:first-child {
min-width: 120px;
flex-shrink: 0;
}
.setting-row input[type="range"] {
flex: 1;
height: 4px;
-webkit-appearance: none;
appearance: none;
background: rgba(255,255,255,0.08);
border-radius: 2px;
outline: none;
}
.setting-row input[type="range"]::-webkit-slider-thumb {
-webkit-appearance: none;
width: 14px; height: 14px;
border-radius: 50%;
background: var(--green-glow);
cursor: pointer;
box-shadow: 0 0 6px rgba(0,216,120,0.4);
}
.setting-row input[type="color"] {
-webkit-appearance: none;
width: 36px; height: 24px;
border: 1px solid rgba(255,255,255,0.15);
border-radius: 4px;
background: none;
cursor: pointer;
padding: 0;
}
.setting-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 2px; }
.setting-row input[type="color"]::-webkit-color-swatch { border-radius: 2px; border: none; }
.setting-row select,
.setting-row input[type="text"] {
flex: 1;
background: #0c1420;
border: 1px solid rgba(255,255,255,0.1);
color: var(--text-primary);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
padding: 6px 10px;
border-radius: 6px;
outline: none;
}
.setting-row select:focus,
.setting-row input[type="text"]:focus {
border-color: var(--green-glow);
}
.setting-row select option {
background: #0c1420;
color: var(--text-primary);
padding: 6px 10px;
}
.setting-row select optgroup {
background: #0a1018;
color: var(--green-glow);
font-style: normal;
font-weight: 600;
padding: 4px 0;
}
.setting-row input[type="checkbox"] {
width: 18px; height: 18px;
accent-color: var(--green-glow);
cursor: pointer;
}
.check-row {
flex-direction: row;
}
.range-val {
font-family: 'JetBrains Mono', monospace;
font-size: 10px;
color: var(--green-glow);
min-width: 44px;
text-align: right;
}
.settings-btn {
width: 100%;
padding: 8px;
margin-top: 6px;
background: rgba(0, 216, 120, 0.08);
border: 1px solid rgba(0, 216, 120, 0.2);
color: var(--green-glow);
font-family: 'JetBrains Mono', monospace;
font-size: 11px;
letter-spacing: 1px;
border-radius: 6px;
cursor: pointer;
transition: all 0.2s;
}
.settings-btn:hover {
background: rgba(0, 216, 120, 0.15);
border-color: var(--green-glow);
}
/* ---- Scenario Description ---- */
#scenario-description {
position: absolute;
top: 60px;
right: 28px;
max-width: 340px;
font-size: 11px;
color: var(--text-secondary);
font-style: italic;
letter-spacing: 0.3px;
line-height: 1.4;
pointer-events: none;
opacity: 0.7;
transition: opacity 0.5s ease;
}
/* ---- Edge Module Badges ---- */
#edge-modules-bar {
position: absolute;
bottom: 58px;
left: 50%;
transform: translateX(-50%);
display: flex;
align-items: center;
gap: 6px;
pointer-events: none;
}
.edge-badge {
display: inline-block;
padding: 2px 8px;
border-radius: 10px;
font-family: 'JetBrains Mono', monospace;
font-size: 9px;
font-weight: 600;
letter-spacing: 1px;
color: var(--badge-color, var(--text-secondary));
background: rgba(255,255,255,0.04);
border: 1px solid var(--badge-color, rgba(255,255,255,0.1));
box-shadow: 0 0 6px color-mix(in srgb, var(--badge-color, transparent) 30%, transparent);
}
/* ---- Person Count Dots ---- */
.persons-dots {
display: inline-flex;
align-items: center;
gap: 3px;
margin-left: 6px;
vertical-align: middle;
}
.person-dot {
width: 6px;
height: 6px;
border-radius: 50%;
display: inline-block;
background: rgba(255,255,255,0.08);
border: 1px solid rgba(255,255,255,0.1);
transition: background 0.4s ease, border-color 0.4s ease, box-shadow 0.4s ease;
}
.person-dot--active {
background: var(--green-glow);
border-color: var(--green-glow);
box-shadow: 0 0 4px rgba(0, 216, 120, 0.4);
}
/* ---- Vital Value Color Transitions ---- */
.vital-value span:first-child {
transition: color 0.6s ease;
}
.vital-bar-fill {
transition: width 0.5s ease, background 0.6s ease;
}
/* ---- Responsive ---- */
@media (max-width: 1200px) {
.data-panel { width: 190px; padding: 12px; }
.vital-value { font-size: 22px; }
#capabilities-bar { display: none; }
}
@media (max-width: 800px) {
.data-panel { display: none; }
#key-hints { display: none; }
.settings-dialog { width: 95vw; }
}
+221
View File
@@ -0,0 +1,221 @@
/**
* Module E "Statistical Convergence Engine"
* RSSI waveform, person orbs, classification, fall alert, metric bars
*/
import * as THREE from 'three';
const WAVEFORM_POINTS = 120;
export class ConvergenceEngine {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// --- RSSI Waveform (scrolling line) ---
this._rssiHistory = new Float32Array(WAVEFORM_POINTS);
const waveGeo = new THREE.BufferGeometry();
this._wavePositions = new Float32Array(WAVEFORM_POINTS * 3);
for (let i = 0; i < WAVEFORM_POINTS; i++) {
this._wavePositions[i * 3] = (i / WAVEFORM_POINTS) * 6 - 3; // x: -3 to 3
this._wavePositions[i * 3 + 1] = 0;
this._wavePositions[i * 3 + 2] = 0;
}
waveGeo.setAttribute('position', new THREE.BufferAttribute(this._wavePositions, 3));
const waveMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.8,
blending: THREE.AdditiveBlending,
});
this._waveform = new THREE.Line(waveGeo, waveMat);
this._waveform.position.y = 1.5;
this.group.add(this._waveform);
// Waveform glow (thicker, dimmer duplicate)
const glowMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.2,
linewidth: 2,
blending: THREE.AdditiveBlending,
});
this._waveGlow = new THREE.Line(waveGeo.clone(), glowMat);
this._waveGlow.position.y = 1.5;
this._waveGlow.scale.set(1, 1.3, 1);
this.group.add(this._waveGlow);
// --- Person orbs (up to 4) ---
this._personOrbs = [];
for (let i = 0; i < 4; i++) {
const orbGeo = new THREE.SphereGeometry(0.2, 16, 16);
const orbMat = new THREE.MeshBasicMaterial({
color: 0xff8800,
transparent: true,
opacity: 0,
blending: THREE.AdditiveBlending,
});
const orb = new THREE.Mesh(orbGeo, orbMat);
orb.position.set(-2 + i * 1.2, -0.5, 0);
this.group.add(orb);
const light = new THREE.PointLight(0xff8800, 0, 3);
orb.add(light);
this._personOrbs.push({ mesh: orb, light, mat: orbMat });
}
// --- Classification text sprite ---
this._classCanvas = document.createElement('canvas');
this._classCanvas.width = 256;
this._classCanvas.height = 48;
this._classCtx = this._classCanvas.getContext('2d');
this._classTex = new THREE.CanvasTexture(this._classCanvas);
const classMat = new THREE.SpriteMaterial({
map: this._classTex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._classSprite = new THREE.Sprite(classMat);
this._classSprite.scale.set(3, 0.6, 1);
this._classSprite.position.y = 0.3;
this.group.add(this._classSprite);
// --- Fall alert ring ---
const alertGeo = new THREE.TorusGeometry(2.5, 0.05, 8, 48);
this._alertMat = new THREE.MeshBasicMaterial({
color: 0xff2244,
transparent: true,
opacity: 0,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._alertRing = new THREE.Mesh(alertGeo, this._alertMat);
this._alertRing.rotation.x = Math.PI / 2;
this._alertRing.position.y = -1;
this.group.add(this._alertRing);
// --- Metric bars (3: frame rate, confidence, variance) ---
this._metricBars = [];
const barLabels = ['CONF', 'VAR', 'SPEC'];
for (let i = 0; i < 3; i++) {
const barGeo = new THREE.PlaneGeometry(0.15, 1.5);
const barMat = new THREE.MeshBasicMaterial({
color: [0x00d4ff, 0x8844ff, 0xff8800][i],
transparent: true,
opacity: 0.5,
blending: THREE.AdditiveBlending,
depthWrite: false,
side: THREE.DoubleSide,
});
const bar = new THREE.Mesh(barGeo, barMat);
bar.position.set(2 + i * 0.4, -1.2, 0);
this.group.add(bar);
this._metricBars.push({ mesh: bar, mat: barMat });
}
this._rssiHead = 0;
this._lastClassification = '';
}
update(dt, elapsed, data) {
const features = data?.features || {};
const classification = data?.classification || {};
const persons = data?.persons || [];
const estPersons = data?.estimated_persons || 0;
// --- Update RSSI waveform ---
const rssi = features.mean_rssi || -50;
this._rssiHistory[this._rssiHead] = rssi;
this._rssiHead = (this._rssiHead + 1) % WAVEFORM_POINTS;
for (let i = 0; i < WAVEFORM_POINTS; i++) {
const histIdx = (this._rssiHead + i) % WAVEFORM_POINTS;
const val = this._rssiHistory[histIdx];
// Normalize RSSI (-80 to -20 range) to -1.5 to 1.5
this._wavePositions[i * 3 + 1] = ((val + 50) / 30) * 1.5;
}
this._waveform.geometry.attributes.position.needsUpdate = true;
// Copy to glow
const glowPos = this._waveGlow.geometry.attributes.position;
glowPos.array.set(this._wavePositions);
glowPos.needsUpdate = true;
// --- Person orbs ---
for (let i = 0; i < this._personOrbs.length; i++) {
const { mesh, light, mat } = this._personOrbs[i];
if (i < estPersons) {
mat.opacity = 0.7;
light.intensity = 1.0 + Math.sin(elapsed * 3 + i * 1.5) * 0.5;
const pulse = 1.0 + Math.sin(elapsed * 2 + i) * 0.15;
mesh.scale.set(pulse, pulse, pulse);
} else {
mat.opacity = 0.05;
light.intensity = 0;
mesh.scale.set(0.5, 0.5, 0.5);
}
}
// --- Classification text ---
const motionLevel = classification.motion_level || 'absent';
const label = motionLevel.toUpperCase().replace('_', ' ');
if (label !== this._lastClassification) {
this._lastClassification = label;
const ctx = this._classCtx;
ctx.clearRect(0, 0, 256, 48);
ctx.font = '600 24px "Courier New", monospace';
ctx.textAlign = 'center';
if (motionLevel === 'active') ctx.fillStyle = '#ff8800';
else if (motionLevel.includes('present')) ctx.fillStyle = '#00d4ff';
else ctx.fillStyle = '#445566';
ctx.fillText(label, 128, 32);
this._classTex.needsUpdate = true;
}
// --- Fall alert ---
const fallDetected = classification.fall_detected || false;
if (fallDetected) {
this._alertMat.opacity = 0.3 + Math.abs(Math.sin(elapsed * 6)) * 0.5;
const scale = 1.0 + Math.sin(elapsed * 4) * 0.1;
this._alertRing.scale.set(scale, scale, 1);
} else {
this._alertMat.opacity = 0;
}
// --- Metric bars ---
const confidence = classification.confidence || 0;
const variance = Math.min(1, (features.variance || 0) / 5);
const spectral = Math.min(1, (features.spectral_power || 0) / 0.5);
const values = [confidence, variance, spectral];
for (let i = 0; i < 3; i++) {
const bar = this._metricBars[i];
const v = values[i];
bar.mesh.scale.y = Math.max(0.05, v);
bar.mesh.position.y = -1.2 + v * 0.75;
bar.mat.opacity = 0.3 + v * 0.4;
}
}
dispose() {
this._waveform.geometry.dispose();
this._waveform.material.dispose();
this._waveGlow.geometry.dispose();
this._waveGlow.material.dispose();
this._alertRing.geometry.dispose();
this._alertMat.dispose();
this._classTex.dispose();
for (const { mesh, mat } of this._personOrbs) {
mesh.geometry.dispose();
mat.dispose();
}
for (const { mesh, mat } of this._metricBars) {
mesh.geometry.dispose();
mat.dispose();
}
}
}
File diff suppressed because it is too large Load Diff
+513
View File
@@ -0,0 +1,513 @@
/**
* FigurePool Manages a pool of wireframe human figures for multi-person rendering.
*
* Extracted from main.js Observatory class. Owns the lifecycle of up to MAX_FIGURES
* Three.js figure groups, each containing joints, bones, body segments, and aura.
*
* Improvements over the original inline implementation:
* - Smooth joint interpolation (lerp toward target instead of snapping)
* - Joint pulsation synced with breathing
* - Natural bone thickness taper (thicker at shoulder/hip, thinner at extremities)
* - Secondary motion with slight delay/overshoot for organic feel
* - Pose-adaptive aura shape (wider for exercise, narrower for crouching)
*/
import * as THREE from 'three';
// 17-keypoint COCO skeleton connectivity
export const SKELETON_PAIRS = [
[0, 1], [0, 2], [1, 3], [2, 4],
[5, 6], [5, 7], [7, 9], [6, 8], [8, 10],
[5, 11], [6, 12], [11, 12],
[11, 13], [13, 15], [12, 14], [14, 16],
];
// Body segment cylinders that give volume to the wireframe
export const BODY_SEGMENT_DEFS = [
{ joints: [5, 11], radius: 0.12 }, // left torso
{ joints: [6, 12], radius: 0.12 }, // right torso
{ joints: [5, 6], radius: 0.1 }, // shoulder bar
{ joints: [11, 12], radius: 0.1 }, // hip bar
{ joints: [5, 7], radius: 0.05 }, // left upper arm
{ joints: [6, 8], radius: 0.05 }, // right upper arm
{ joints: [7, 9], radius: 0.04 }, // left forearm
{ joints: [8, 10], radius: 0.04 }, // right forearm
{ joints: [11, 13], radius: 0.07 }, // left thigh
{ joints: [12, 14], radius: 0.07 }, // right thigh
{ joints: [13, 15], radius: 0.05 }, // left shin
{ joints: [14, 16], radius: 0.05 }, // right shin
{ joints: [0, 0], radius: 0.1, isHead: true },
];
// Bone thickness multipliers — thicker at torso, thinner at extremities
const BONE_TAPER = (() => {
const tapers = new Map();
// Torso and shoulder/hip connections are thickest
tapers.set('5-6', 1.4); // shoulder bar
tapers.set('11-12', 1.3); // hip bar
tapers.set('5-11', 1.3); // left torso
tapers.set('6-12', 1.3); // right torso
// Upper limbs
tapers.set('5-7', 1.0); // left upper arm
tapers.set('6-8', 1.0); // right upper arm
tapers.set('11-13', 1.1); // left thigh
tapers.set('12-14', 1.1); // right thigh
// Lower limbs / extremities — thinnest
tapers.set('7-9', 0.7); // left forearm
tapers.set('8-10', 0.7); // right forearm
tapers.set('13-15', 0.8); // left shin
tapers.set('14-16', 0.8); // right shin
// Head connections
tapers.set('0-1', 0.5);
tapers.set('0-2', 0.5);
tapers.set('1-3', 0.4);
tapers.set('2-4', 0.4);
return tapers;
})();
// Secondary motion delay factors per joint — extremities lag more
const SECONDARY_DELAY = [
0.12, // 0 nose
0.10, // 1 left eye
0.10, // 2 right eye
0.08, // 3 left ear
0.08, // 4 right ear
0.18, // 5 left shoulder
0.18, // 6 right shoulder
0.14, // 7 left elbow
0.14, // 8 right elbow
0.10, // 9 left wrist (most lag)
0.10, // 10 right wrist
0.20, // 11 left hip (anchored, fast follow)
0.20, // 12 right hip
0.15, // 13 left knee
0.15, // 14 right knee
0.10, // 15 left ankle
0.10, // 16 right ankle
];
// Overshoot factors — extremities overshoot more for organic feel
const OVERSHOOT = [
0.02, // 0 nose
0.01, // 1 left eye
0.01, // 2 right eye
0.01, // 3 left ear
0.01, // 4 right ear
0.03, // 5 left shoulder
0.03, // 6 right shoulder
0.05, // 7 left elbow
0.05, // 8 right elbow
0.08, // 9 left wrist
0.08, // 10 right wrist
0.02, // 11 left hip
0.02, // 12 right hip
0.04, // 13 left knee
0.04, // 14 right knee
0.06, // 15 left ankle
0.06, // 16 right ankle
];
const MAX_FIGURES = 4;
// Reusable vectors to avoid per-frame allocation
const _vecFrom = new THREE.Vector3();
const _vecTo = new THREE.Vector3();
const _vecTarget = new THREE.Vector3();
export class FigurePool {
/**
* @param {THREE.Scene} scene - The Three.js scene to add figures to
* @param {object} settings - Shared settings object (boneThick, jointSize, glow, etc.)
* @param {object} poseSystem - PoseSystem instance with generateKeypoints(person, elapsed, breathPulse)
*/
constructor(scene, settings, poseSystem) {
this._scene = scene;
this._settings = settings;
this._poseSystem = poseSystem;
this._figures = [];
this._maxFigures = MAX_FIGURES;
this._build();
}
/** @returns {Array} The array of figure objects */
get figures() { return this._figures; }
// ---- Construction ----
_build() {
for (let f = 0; f < this._maxFigures; f++) {
this._figures.push(this._createFigure());
}
}
_createFigure() {
const group = new THREE.Group();
this._scene.add(group);
const wireColor = new THREE.Color(this._settings.wireColor);
const jointColor = new THREE.Color(this._settings.jointColor);
// Joints (17 COCO keypoints)
const joints = [];
for (let i = 0; i < 17; i++) {
const isNose = i === 0;
const size = isNose ? this._settings.jointSize * 0.7 : this._settings.jointSize;
const geo = new THREE.SphereGeometry(size, 12, 12);
const mat = new THREE.MeshStandardMaterial({
color: isNose ? wireColor : jointColor,
emissive: isNose ? wireColor : jointColor,
emissiveIntensity: 0.35,
transparent: true, opacity: 0,
roughness: 0.3, metalness: 0.2,
});
const sphere = new THREE.Mesh(geo, mat);
sphere.castShadow = true;
group.add(sphere);
joints.push(sphere);
// Halo glow on key joints
if ([5, 6, 9, 10, 11, 12, 15, 16].includes(i)) {
const haloGeo = new THREE.SphereGeometry(size * 1.3, 8, 8);
const haloMat = new THREE.MeshBasicMaterial({
color: jointColor,
transparent: true, opacity: 0,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const halo = new THREE.Mesh(haloGeo, haloMat);
sphere.add(halo);
sphere._halo = halo;
sphere._haloMat = haloMat;
const glow = new THREE.PointLight(jointColor, 0, 0.8);
sphere.add(glow);
sphere._glow = glow;
}
}
// Bones — tapered thickness
const bones = [];
for (const [a, b] of SKELETON_PAIRS) {
const taperKey = `${Math.min(a, b)}-${Math.max(a, b)}`;
const taper = BONE_TAPER.get(taperKey) || 1.0;
const thick = this._settings.boneThick * taper;
// Top radius thicker than bottom for natural taper along bone length
const topRadius = thick;
const botRadius = thick * 0.65;
const geo = new THREE.CylinderGeometry(topRadius, botRadius, 1, 8, 1);
geo.translate(0, 0.5, 0);
geo.rotateX(Math.PI / 2);
const mat = new THREE.MeshStandardMaterial({
color: wireColor, emissive: wireColor, emissiveIntensity: 0.3,
transparent: true, opacity: 0, roughness: 0.4, metalness: 0.1,
});
const mesh = new THREE.Mesh(geo, mat);
mesh.castShadow = true;
group.add(mesh);
bones.push({ mesh, a, b, taper });
}
// Body segments (volume cylinders and head sphere)
const bodySegments = [];
for (const seg of BODY_SEGMENT_DEFS) {
const geo = seg.isHead
? new THREE.SphereGeometry(seg.radius, 12, 12)
: new THREE.CylinderGeometry(seg.radius, seg.radius * 0.85, 1, 8, 1);
if (!seg.isHead) {
geo.translate(0, 0.5, 0);
geo.rotateX(Math.PI / 2);
}
const mat = new THREE.MeshStandardMaterial({
color: wireColor, emissive: wireColor, emissiveIntensity: 0.12,
transparent: true, opacity: 0, roughness: 0.5, metalness: 0.1,
side: THREE.DoubleSide,
});
const mesh = new THREE.Mesh(geo, mat);
group.add(mesh);
bodySegments.push({ mesh, mat, a: seg.joints[0], b: seg.joints[1], isHead: seg.isHead });
}
// Aura cylinder
const auraGeo = new THREE.CylinderGeometry(0.4, 0.3, 1.7, 16, 1, true);
const auraMat = new THREE.MeshBasicMaterial({
color: wireColor, transparent: true, opacity: 0,
side: THREE.DoubleSide, blending: THREE.AdditiveBlending, depthWrite: false,
});
const aura = new THREE.Mesh(auraGeo, auraMat);
aura.position.y = 1;
group.add(aura);
// Per-figure point light
const personLight = new THREE.PointLight(wireColor, 0, 6);
personLight.position.y = 1;
group.add(personLight);
// Interpolation state: previous positions for smooth lerp and secondary motion
const prevPositions = [];
const velocities = [];
for (let i = 0; i < 17; i++) {
prevPositions.push(new THREE.Vector3(0, 0, 0));
velocities.push(new THREE.Vector3(0, 0, 0));
}
return {
group, joints, bones, bodySegments, aura, auraMat, personLight,
visible: false,
prevPositions,
velocities,
_initialized: false,
_lastPose: null,
};
}
// ---- Per-frame update ----
/**
* Update all figures based on current data frame.
* @param {object} data - Current sensing data with persons[], vital_signs, classification
* @param {number} elapsed - Elapsed time in seconds
*/
update(data, elapsed) {
const persons = data?.persons || [];
const vs = data?.vital_signs || {};
const isPresent = data?.classification?.presence || false;
const breathBpm = vs.breathing_rate_bpm || 0;
const breathPulse = breathBpm > 0
? Math.sin(elapsed * Math.PI * 2 * (breathBpm / 60)) * 0.012
: 0;
for (let f = 0; f < this._figures.length; f++) {
const fig = this._figures[f];
if (f < persons.length && isPresent) {
const p = persons[f];
const kps = this._poseSystem.generateKeypoints(p, elapsed, breathPulse);
this.applyKeypoints(fig, kps, breathPulse, p.position || [0, 0, 0], elapsed, p.pose);
fig.visible = true;
} else {
if (fig.visible) {
this.hide(fig);
fig.visible = false;
}
}
}
}
/**
* Apply keypoints to a figure with smooth interpolation, pulsation, and secondary motion.
* @param {object} fig - Figure object from the pool
* @param {Array} kps - 17-element array of [x,y,z] keypoint positions
* @param {number} breathPulse - Current breathing pulse value
* @param {Array} pos - Person world position [x,y,z]
* @param {number} elapsed - Elapsed time for pulsation effects
* @param {string} pose - Current pose name for aura adaptation
*/
applyKeypoints(fig, kps, breathPulse, pos, elapsed = 0, pose = 'standing') {
const lerpFactor = fig._initialized ? 0.18 : 1.0;
// Joints with smooth interpolation and secondary motion
for (let i = 0; i < 17 && i < kps.length; i++) {
const j = fig.joints[i];
_vecTarget.set(kps[i][0], kps[i][1], kps[i][2]);
if (fig._initialized) {
// Compute velocity for overshoot
const prev = fig.prevPositions[i];
const vel = fig.velocities[i];
// Smooth lerp with per-joint delay
const delay = SECONDARY_DELAY[i];
const jointLerp = lerpFactor + delay;
j.position.lerp(_vecTarget, Math.min(jointLerp, 0.95));
// Apply subtle overshoot based on velocity change
const overshoot = OVERSHOOT[i];
vel.subVectors(j.position, prev).multiplyScalar(overshoot);
j.position.add(vel);
prev.copy(j.position);
} else {
// First frame: snap to position
j.position.copy(_vecTarget);
fig.prevPositions[i].copy(_vecTarget);
fig.velocities[i].set(0, 0, 0);
}
j.material.opacity = 0.95;
// Joint pulsation synced with breathing
const pulseFactor = 1.0 + Math.abs(breathPulse) * 8.0;
j.material.emissiveIntensity = 0.35 * pulseFactor;
const baseScale = this._settings.jointSize / 0.04;
// Subtle size pulsation on breathing
const pulseScale = baseScale * (1.0 + Math.abs(breathPulse) * 3.0);
j.scale.setScalar(pulseScale);
if (j._haloMat) {
j._haloMat.opacity = 0.04 * this._settings.glow * pulseFactor;
}
if (j._glow) {
j._glow.intensity = this._settings.glow * 0.12 * pulseFactor;
}
}
fig._initialized = true;
// Bones with tapered thickness
for (const bone of fig.bones) {
const pA = kps[bone.a], pB = kps[bone.b];
if (pA && pB) {
_vecFrom.set(pA[0], pA[1], pA[2]);
_vecTo.set(pB[0], pB[1], pB[2]);
const len = _vecFrom.distanceTo(_vecTo);
// Use interpolated joint positions for smooth bone movement
if (fig._initialized) {
const jA = fig.joints[bone.a];
const jB = fig.joints[bone.b];
bone.mesh.position.copy(jA.position);
bone.mesh.scale.set(1, 1, jA.position.distanceTo(jB.position));
bone.mesh.lookAt(jB.position);
} else {
bone.mesh.position.copy(_vecFrom);
bone.mesh.scale.set(1, 1, len);
bone.mesh.lookAt(_vecTo);
}
bone.mesh.material.opacity = 0.85;
bone.mesh.material.emissiveIntensity = 0.3 + Math.abs(breathPulse) * 2.0;
}
}
// Body segments
for (const seg of fig.bodySegments) {
if (seg.isHead) {
const headJoint = fig.joints[seg.a];
seg.mesh.position.set(headJoint.position.x, headJoint.position.y + 0.05, headJoint.position.z);
seg.mat.opacity = 0.15;
} else {
const jA = fig.joints[seg.a];
const jB = fig.joints[seg.b];
if (jA && jB) {
const len = jA.position.distanceTo(jB.position);
seg.mesh.position.copy(jA.position);
seg.mesh.scale.set(1, 1, len);
seg.mesh.lookAt(jB.position);
seg.mat.opacity = 0.12;
}
}
seg.mat.emissiveIntensity = 0.1 + Math.abs(breathPulse) * 0.4;
}
// Aura — adapt shape to pose
const hipY = (fig.joints[11].position.y + fig.joints[12].position.y) / 2;
const cx = (fig.joints[11].position.x + fig.joints[12].position.x) / 2;
const cz = (fig.joints[11].position.z + fig.joints[12].position.z) / 2;
fig.aura.position.set(cx, hipY, cz);
fig.auraMat.opacity = this._settings.aura + Math.abs(breathPulse) * 0.8;
// Pose-adaptive aura: compute from actual keypoint spread
const auraShape = this._computeAuraShape(fig, pose, breathPulse);
fig.aura.scale.set(auraShape.scaleX, auraShape.scaleY, auraShape.scaleZ);
// Person light
fig.personLight.position.set(pos[0], 1.2, pos[2]);
fig.personLight.intensity = this._settings.glow * 0.4;
fig._lastPose = pose;
}
/**
* Compute pose-adaptive aura shape based on actual keypoint spread.
* Wider for exercise/spread poses, narrower for crouching/compact poses.
*/
_computeAuraShape(fig, pose, breathPulse) {
// Measure horizontal spread from shoulders and hips
const lShoulder = fig.joints[5].position;
const rShoulder = fig.joints[6].position;
const lHip = fig.joints[11].position;
const rHip = fig.joints[12].position;
const nose = fig.joints[0].position;
const lAnkle = fig.joints[15].position;
const rAnkle = fig.joints[16].position;
// Horizontal spread (X-Z plane)
const shoulderWidth = Math.sqrt(
(rShoulder.x - lShoulder.x) ** 2 +
(rShoulder.z - lShoulder.z) ** 2
);
const ankleWidth = Math.sqrt(
(rAnkle.x - lAnkle.x) ** 2 +
(rAnkle.z - lAnkle.z) ** 2
);
const maxWidth = Math.max(shoulderWidth, ankleWidth);
// Vertical extent
const headY = nose.y;
const footY = Math.min(lAnkle.y, rAnkle.y);
const height = headY - footY;
// Normalize to base aura dimensions
const baseWidth = 0.44; // default shoulder width
const baseHeight = 1.7; // default standing height
const widthRatio = Math.max(0.6, Math.min(2.0, maxWidth / baseWidth));
const heightRatio = Math.max(0.4, Math.min(1.3, height / baseHeight));
// Breathing modulation
const breathMod = 1 + breathPulse * 2;
return {
scaleX: widthRatio * breathMod,
scaleY: heightRatio * breathMod,
scaleZ: widthRatio * breathMod,
};
}
/**
* Hide a figure by fading all materials to invisible.
* @param {object} fig - Figure object to hide
*/
hide(fig) {
for (const j of fig.joints) {
j.material.opacity = 0;
if (j._haloMat) j._haloMat.opacity = 0;
if (j._glow) j._glow.intensity = 0;
}
for (const b of fig.bones) b.mesh.material.opacity = 0;
for (const seg of fig.bodySegments) seg.mat.opacity = 0;
fig.auraMat.opacity = 0;
fig.personLight.intensity = 0;
fig._initialized = false;
}
/**
* Apply wire and joint colors to all figures in the pool.
* @param {THREE.Color} wireColor
* @param {THREE.Color} jointColor
*/
applyColors(wireColor, jointColor) {
for (const fig of this._figures) {
for (let i = 0; i < fig.joints.length; i++) {
const j = fig.joints[i];
if (i === 0) {
j.material.color.copy(wireColor);
j.material.emissive.copy(wireColor);
} else {
j.material.color.copy(jointColor);
j.material.emissive.copy(jointColor);
}
if (j._haloMat) j._haloMat.color.copy(jointColor);
if (j._glow) j._glow.color.copy(jointColor);
}
for (const b of fig.bones) {
b.mesh.material.color.copy(wireColor);
b.mesh.material.emissive.copy(wireColor);
}
for (const seg of fig.bodySegments) {
seg.mat.color.copy(wireColor);
seg.mat.emissive.copy(wireColor);
}
fig.auraMat.color.copy(wireColor);
fig.personLight.color.copy(wireColor);
}
}
}
+121
View File
@@ -0,0 +1,121 @@
/**
* Holographic Panel Reusable frame with border shader, scan line, title
*/
import * as THREE from 'three';
const BORDER_VERTEX = `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`;
const BORDER_FRAGMENT = `
uniform float uTime;
uniform vec3 uColor;
varying vec2 vUv;
void main() {
// Thin border
float bx = step(vUv.x, 0.015) + step(1.0 - 0.015, vUv.x);
float by = step(vUv.y, 0.02) + step(1.0 - 0.02, vUv.y);
float border = clamp(bx + by, 0.0, 1.0);
// Scan line moving upward
float scan = smoothstep(0.0, 0.02, abs(vUv.y - fract(uTime * 0.15))) ;
scan = 1.0 - (1.0 - scan) * 0.4;
// Corner accents
float corner = 0.0;
float cx = min(vUv.x, 1.0 - vUv.x);
float cy = min(vUv.y, 1.0 - vUv.y);
if (cx < 0.06 && cy < 0.08) corner = 0.6;
// Subtle fill
float fill = 0.03 + corner * 0.05;
float alpha = max(border * 0.7, fill) * scan;
gl_FragColor = vec4(uColor, alpha);
}
`;
export class HolographicPanel {
/**
* @param {Object} opts
* @param {number[]} opts.position - [x, y, z]
* @param {number} opts.width
* @param {number} opts.height
* @param {string} opts.title
* @param {number} [opts.color=0x00d4ff]
*/
constructor(opts) {
this.group = new THREE.Group();
this.group.position.set(...opts.position);
const color = new THREE.Color(opts.color || 0x00d4ff);
// Border plane
this._uniforms = {
uTime: { value: 0 },
uColor: { value: color },
};
const borderGeo = new THREE.PlaneGeometry(opts.width, opts.height);
const borderMat = new THREE.ShaderMaterial({
vertexShader: BORDER_VERTEX,
fragmentShader: BORDER_FRAGMENT,
uniforms: this._uniforms,
transparent: true,
side: THREE.DoubleSide,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._border = new THREE.Mesh(borderGeo, borderMat);
this.group.add(this._border);
// Title sprite
if (opts.title) {
const canvas = document.createElement('canvas');
canvas.width = 512;
canvas.height = 64;
const ctx = canvas.getContext('2d');
ctx.fillStyle = 'transparent';
ctx.fillRect(0, 0, 512, 64);
ctx.font = '600 28px "Courier New", monospace';
ctx.fillStyle = `#${color.getHexString()}`;
ctx.textAlign = 'center';
ctx.fillText(opts.title.toUpperCase(), 256, 42);
const tex = new THREE.CanvasTexture(canvas);
const spriteMat = new THREE.SpriteMaterial({
map: tex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const sprite = new THREE.Sprite(spriteMat);
sprite.scale.set(opts.width * 0.8, opts.width * 0.1, 1);
sprite.position.y = opts.height / 2 + 0.3;
this.group.add(sprite);
this._titleSprite = sprite;
this._titleTex = tex;
}
}
update(dt, elapsed) {
this._uniforms.uTime.value = elapsed;
}
/** Make panel face camera */
lookAt(cameraPos) {
this.group.lookAt(cameraPos);
}
dispose() {
this._border.geometry.dispose();
this._border.material.dispose();
if (this._titleTex) this._titleTex.dispose();
if (this._titleSprite) this._titleSprite.material.dispose();
}
}
+567
View File
@@ -0,0 +1,567 @@
/**
* HudController Extracted HUD update, settings dialog, and scenario UI
*
* Manages all DOM-based HUD elements:
* - Vital sign display with smooth lerp transitions and color coding
* - Signal metrics, sparkline, and presence indicator
* - Scenario description and edge module badges
* - Mini person-count dot visualization
* - Settings dialog (tabs, ranges, presets, data source)
* - Quick-select scenario dropdown
*/
// ---- Constants ----
export const SCENARIO_NAMES = [
'EMPTY ROOM','VITAL SIGNS','MULTI-PERSON','FALL DETECT',
'SLEEP MONITOR','INTRUSION','GESTURE CTRL','CROWD OCCUPANCY',
'SEARCH RESCUE','ELDERLY CARE','FITNESS','SECURITY PATROL',
];
export const DEFAULTS = {
bloom: 0.08, bloomRadius: 0.2, bloomThresh: 0.6,
exposure: 1.3, vignette: 0.25, grain: 0.01, chromatic: 0.0005,
boneThick: 0.018, jointSize: 0.035, glow: 0.3, trail: 0.35,
wireColor: '#00d878', jointColor: '#ff4060', aura: 0.02,
field: 0.45, waves: 0.4, ambient: 0.7, reflect: 0.2,
fov: 50, orbitSpeed: 0.15, grid: true, room: true,
scenario: 'auto', cycle: 30, dataSource: 'demo', wsUrl: '',
};
export const SETTINGS_VERSION = '6';
export const PRESETS = {
foundation: {},
cinematic: {
bloom: 1.2, bloomRadius: 0.5, bloomThresh: 0.2,
exposure: 0.8, vignette: 0.7, grain: 0.04, chromatic: 0.002,
glow: 0.6, trail: 0.8, aura: 0.06, field: 0.4,
waves: 0.7, ambient: 0.25, reflect: 0.5, fov: 40, orbitSpeed: 0.08,
},
minimal: {
bloom: 0.3, bloomRadius: 0.2, bloomThresh: 0.5,
exposure: 1.1, vignette: 0.2, grain: 0, chromatic: 0,
glow: 0.3, trail: 0.2, aura: 0.02, field: 0.7,
waves: 0.3, ambient: 0.6, reflect: 0.1, wireColor: '#40ff90', jointColor: '#4080ff',
},
neon: {
bloom: 2.5, bloomRadius: 0.8, bloomThresh: 0.1,
exposure: 0.6, vignette: 0.6, grain: 0.02, chromatic: 0.004,
glow: 2.0, trail: 1.0, aura: 0.15, field: 0.6,
waves: 1.0, ambient: 0.15, reflect: 0.7, wireColor: '#00ffaa', jointColor: '#ff00ff',
},
tactical: {
bloom: 0.5, bloomRadius: 0.3, bloomThresh: 0.4,
exposure: 0.85, vignette: 0.4, grain: 0.04, chromatic: 0.001,
glow: 0.5, trail: 0.4, aura: 0.03, field: 0.8,
waves: 0.4, ambient: 0.3, reflect: 0.15, wireColor: '#30ff60', jointColor: '#ff8800',
},
medical: {
bloom: 0.6, bloomRadius: 0.4, bloomThresh: 0.35,
exposure: 1.0, vignette: 0.3, grain: 0.01, chromatic: 0.0005,
glow: 0.6, trail: 0.3, aura: 0.04, field: 0.5,
waves: 0.3, ambient: 0.5, reflect: 0.2, wireColor: '#00ccff', jointColor: '#ff3355',
},
};
// Scenario descriptions shown below the dropdown
const SCENARIO_DESCRIPTIONS = {
auto: 'Auto-cycling through all sensing scenarios.',
empty_room: 'Baseline calibration with no human presence in the monitored zone.',
single_breathing: 'Detecting vital signs through WiFi signal micro-variations.',
two_walking: 'Tracking multiple people simultaneously via CSI multiplex separation.',
fall_event: 'Sudden posture-change detection using acceleration feature analysis.',
sleep_monitoring: 'Monitoring breathing patterns and apnea events during sleep.',
intrusion_detect: 'Passive perimeter monitoring -- no cameras, pure RF sensing.',
gesture_control: 'DTW-based gesture recognition from hand/arm motion signatures.',
crowd_occupancy: 'Estimating room occupancy count from aggregate CSI variance.',
search_rescue: 'Through-wall survivor detection using WiFi-MAT multistatic mode.',
elderly_care: 'Continuous gait analysis for early mobility-decline detection.',
fitness_tracking: 'Rep counting and exercise classification from body kinematics.',
security_patrol: 'Multi-zone presence patrol with camera-free motion heatmaps.',
};
// Edge modules active per scenario
const SCENARIO_EDGE_MODULES = {
auto: [],
empty_room: [],
single_breathing: ['VITALS'],
two_walking: ['GAIT', 'TRACKING'],
fall_event: ['FALL', 'VITALS'],
sleep_monitoring: ['VITALS', 'APNEA'],
intrusion_detect: ['PRESENCE', 'ALERT'],
gesture_control: ['GESTURE', 'DTW'],
crowd_occupancy: ['OCCUPANCY'],
search_rescue: ['MAT', 'VITALS', 'PRESENCE'],
elderly_care: ['GAIT', 'VITALS', 'FALL'],
fitness_tracking: ['GESTURE', 'GAIT'],
security_patrol: ['PRESENCE', 'ALERT', 'TRACKING'],
};
// Edge-module badge colors
const MODULE_COLORS = {
VITALS: 'var(--red-heart)',
GAIT: 'var(--green-glow)',
FALL: 'var(--red-alert)',
GESTURE: 'var(--amber)',
PRESENCE: 'var(--blue-signal)',
TRACKING: 'var(--green-bright)',
OCCUPANCY: 'var(--amber)',
ALERT: 'var(--red-alert)',
DTW: 'var(--amber)',
APNEA: 'var(--red-heart)',
MAT: 'var(--blue-signal)',
};
// Vital-sign color-coding thresholds
function vitalColor(type, value) {
if (value <= 0) return 'var(--text-secondary)';
if (type === 'hr') {
if (value < 50 || value > 130) return 'var(--red-alert)';
if (value < 60 || value > 100) return 'var(--amber)';
return 'var(--green-glow)';
}
if (type === 'br') {
if (value < 8 || value > 28) return 'var(--red-alert)';
if (value < 12 || value > 20) return 'var(--amber)';
return 'var(--green-glow)';
}
if (type === 'conf') {
if (value < 40) return 'var(--red-alert)';
if (value < 70) return 'var(--amber)';
return 'var(--green-glow)';
}
return 'var(--text-primary)';
}
function lerp(a, b, t) {
return a + (b - a) * t;
}
// ---- HudController class ----
export class HudController {
constructor(observatory) {
this._obs = observatory;
this._settingsOpen = false;
this._rssiHistory = [];
this._sparklineCtx = document.getElementById('rssi-sparkline')?.getContext('2d');
// Lerp state for smooth vital-sign transitions
this._lerpHr = 0;
this._lerpBr = 0;
this._lerpConf = 0;
// Track current scenario for description/edge updates
this._currentScenarioKey = null;
}
// ============================================================
// Settings dialog
// ============================================================
initSettings() {
const overlay = document.getElementById('settings-overlay');
const btn = document.getElementById('settings-btn');
const closeBtn = document.getElementById('settings-close');
btn.addEventListener('click', () => this.toggleSettings());
closeBtn.addEventListener('click', () => this.toggleSettings());
overlay.addEventListener('click', (e) => { if (e.target === overlay) this.toggleSettings(); });
// Tab switching
document.querySelectorAll('.stab').forEach(tab => {
tab.addEventListener('click', () => {
document.querySelectorAll('.stab').forEach(t => t.classList.remove('active'));
document.querySelectorAll('.stab-content').forEach(c => c.classList.remove('active'));
tab.classList.add('active');
document.getElementById(`stab-${tab.dataset.stab}`).classList.add('active');
});
});
const obs = this._obs;
const s = obs.settings;
// Bind ranges
this._bindRange('opt-bloom', 'bloom', v => { obs._postProcessing._bloomPass.strength = v; });
this._bindRange('opt-bloom-radius', 'bloomRadius', v => { obs._postProcessing._bloomPass.radius = v; });
this._bindRange('opt-bloom-thresh', 'bloomThresh', v => { obs._postProcessing._bloomPass.threshold = v; });
this._bindRange('opt-exposure', 'exposure', v => { obs._renderer.toneMappingExposure = v; });
this._bindRange('opt-vignette', 'vignette', v => { obs._postProcessing._vignettePass.uniforms.uVignetteStrength.value = v; });
this._bindRange('opt-grain', 'grain', v => { obs._postProcessing._vignettePass.uniforms.uGrainStrength.value = v; });
this._bindRange('opt-chromatic', 'chromatic', v => { obs._postProcessing._vignettePass.uniforms.uChromaticStrength.value = v; });
this._bindRange('opt-bone-thick', 'boneThick');
this._bindRange('opt-joint-size', 'jointSize');
this._bindRange('opt-glow', 'glow');
this._bindRange('opt-trail', 'trail');
this._bindRange('opt-aura', 'aura');
this._bindRange('opt-field', 'field', v => { obs._fieldMat.opacity = v; });
this._bindRange('opt-waves', 'waves');
this._bindRange('opt-ambient', 'ambient', v => { obs._ambient.intensity = v * 5.0; });
this._bindRange('opt-reflect', 'reflect', v => {
obs._floorMat.roughness = 1.0 - v * 0.7;
obs._floorMat.metalness = v * 0.5;
});
this._bindRange('opt-fov', 'fov', v => {
obs._camera.fov = v;
obs._camera.updateProjectionMatrix();
});
this._bindRange('opt-orbit-speed', 'orbitSpeed');
this._bindRange('opt-cycle', 'cycle', v => { obs._demoData.setCycleDuration(v); });
// Color pickers
document.getElementById('opt-wire-color').value = s.wireColor;
document.getElementById('opt-wire-color').addEventListener('input', (e) => {
s.wireColor = e.target.value; obs._applyColors(); this.saveSettings();
});
document.getElementById('opt-joint-color').value = s.jointColor;
document.getElementById('opt-joint-color').addEventListener('input', (e) => {
s.jointColor = e.target.value; obs._applyColors(); this.saveSettings();
});
// Checkboxes
document.getElementById('opt-grid').checked = s.grid;
document.getElementById('opt-grid').addEventListener('change', (e) => {
s.grid = e.target.checked; obs._grid.visible = e.target.checked; this.saveSettings();
});
document.getElementById('opt-room').checked = s.room;
document.getElementById('opt-room').addEventListener('change', (e) => {
s.room = e.target.checked; obs._roomWire.visible = e.target.checked; this.saveSettings();
});
// Scenario select
const scenarioSel = document.getElementById('opt-scenario');
scenarioSel.value = s.scenario;
scenarioSel.addEventListener('change', (e) => {
s.scenario = e.target.value;
obs._demoData.setScenario(e.target.value);
this.saveSettings();
});
// Data source
const dsSel = document.getElementById('opt-data-source');
dsSel.value = s.dataSource;
dsSel.addEventListener('change', (e) => {
s.dataSource = e.target.value;
document.getElementById('ws-url-row').style.display = e.target.value === 'ws' ? 'flex' : 'none';
if (e.target.value === 'ws' && s.wsUrl) obs._connectWS(s.wsUrl);
else obs._disconnectWS();
this.updateSourceBadge(s.dataSource, obs._ws);
this.saveSettings();
});
document.getElementById('ws-url-row').style.display = s.dataSource === 'ws' ? 'flex' : 'none';
const wsInput = document.getElementById('opt-ws-url');
wsInput.value = s.wsUrl;
wsInput.addEventListener('change', (e) => {
s.wsUrl = e.target.value;
if (s.dataSource === 'ws') obs._connectWS(e.target.value);
this.saveSettings();
});
// Buttons
document.getElementById('btn-reset-camera').addEventListener('click', () => {
obs._camera.position.set(6, 5, 8);
obs._controls.target.set(0, 1.2, 0);
obs._controls.update();
});
document.getElementById('btn-export-settings').addEventListener('click', () => {
const blob = new Blob([JSON.stringify(s, null, 2)], { type: 'application/json' });
const a = document.createElement('a');
a.href = URL.createObjectURL(blob);
a.download = 'ruview-observatory-settings.json';
a.click();
});
document.getElementById('btn-reset-settings').addEventListener('click', () => {
this.applyPreset(DEFAULTS);
});
const presetSel = document.getElementById('opt-preset');
presetSel.addEventListener('change', (e) => {
const p = PRESETS[e.target.value];
if (p) this.applyPreset({ ...DEFAULTS, ...p });
});
obs._grid.visible = s.grid;
obs._roomWire.visible = s.room;
}
// ============================================================
// Quick-select (top bar scenario dropdown)
// ============================================================
initQuickSelect() {
const sel = document.getElementById('scenario-quick-select');
if (!sel) return;
sel.addEventListener('change', (e) => {
this._obs._demoData.setScenario(e.target.value);
const settingsSel = document.getElementById('opt-scenario');
if (settingsSel) settingsSel.value = e.target.value;
this._obs.settings.scenario = e.target.value;
this.saveSettings();
});
}
// ============================================================
// Toggle / save / preset
// ============================================================
toggleSettings() {
this._settingsOpen = !this._settingsOpen;
document.getElementById('settings-overlay').style.display = this._settingsOpen ? 'flex' : 'none';
}
get settingsOpen() {
return this._settingsOpen;
}
saveSettings() {
try {
localStorage.setItem('ruview-observatory-settings', JSON.stringify(this._obs.settings));
} catch {}
}
applyPreset(preset) {
const obs = this._obs;
Object.assign(obs.settings, preset);
this.saveSettings();
const rangeMap = {
'opt-bloom': 'bloom', 'opt-bloom-radius': 'bloomRadius', 'opt-bloom-thresh': 'bloomThresh',
'opt-exposure': 'exposure', 'opt-vignette': 'vignette', 'opt-grain': 'grain', 'opt-chromatic': 'chromatic',
'opt-bone-thick': 'boneThick', 'opt-joint-size': 'jointSize', 'opt-glow': 'glow', 'opt-trail': 'trail', 'opt-aura': 'aura',
'opt-field': 'field', 'opt-waves': 'waves', 'opt-ambient': 'ambient', 'opt-reflect': 'reflect',
'opt-fov': 'fov', 'opt-orbit-speed': 'orbitSpeed', 'opt-cycle': 'cycle',
};
for (const [id, key] of Object.entries(rangeMap)) {
const el = document.getElementById(id);
const valEl = document.getElementById(`${id}-val`);
if (el) el.value = obs.settings[key];
if (valEl) valEl.textContent = obs.settings[key];
}
const gridEl = document.getElementById('opt-grid');
if (gridEl) { gridEl.checked = obs.settings.grid; obs._grid.visible = obs.settings.grid; }
const roomEl = document.getElementById('opt-room');
if (roomEl) { roomEl.checked = obs.settings.room; obs._roomWire.visible = obs.settings.room; }
document.getElementById('opt-wire-color').value = obs.settings.wireColor;
document.getElementById('opt-joint-color').value = obs.settings.jointColor;
obs._applyPostSettings();
obs._renderer.toneMappingExposure = obs.settings.exposure;
obs._fieldMat.opacity = obs.settings.field;
obs._ambient.intensity = obs.settings.ambient * 5.0;
obs._floorMat.roughness = 1.0 - obs.settings.reflect * 0.7;
obs._floorMat.metalness = obs.settings.reflect * 0.5;
obs._camera.fov = obs.settings.fov;
obs._camera.updateProjectionMatrix();
obs._demoData.setCycleDuration(obs.settings.cycle);
obs._applyColors();
}
// ============================================================
// Source badge
// ============================================================
updateSourceBadge(dataSource, ws) {
const dot = document.querySelector('#data-source-badge .dot');
const label = document.getElementById('data-source-label');
if (dataSource === 'ws' && ws?.readyState === WebSocket.OPEN) {
dot.className = 'dot dot--live'; label.textContent = 'LIVE';
} else {
dot.className = 'dot dot--demo'; label.textContent = 'DEMO';
}
}
// ============================================================
// HUD update (called every frame)
// ============================================================
updateHUD(data, demoData) {
if (!data) return;
const vs = data.vital_signs || {};
const feat = data.features || {};
const cls = data.classification || {};
// Sync scenario dropdown
const quickSel = document.getElementById('scenario-quick-select');
const cur = demoData._autoMode ? 'auto' : demoData.currentScenario;
if (quickSel && quickSel.value !== cur) quickSel.value = cur;
const autoIcon = document.getElementById('autoplay-icon');
if (autoIcon) autoIcon.className = demoData._autoMode ? '' : 'hidden';
const targetHr = vs.heart_rate_bpm || 0;
const targetBr = vs.breathing_rate_bpm || 0;
const targetConf = Math.round((cls.confidence || 0) * 100);
// Smooth lerp transitions (blend 4% per frame toward target — very stable)
const lerpFactor = 0.04;
this._lerpHr = targetHr > 0 ? lerp(this._lerpHr, targetHr, lerpFactor) : 0;
this._lerpBr = targetBr > 0 ? lerp(this._lerpBr, targetBr, lerpFactor) : 0;
this._lerpConf = targetConf > 0 ? lerp(this._lerpConf, targetConf, lerpFactor) : 0;
const dispHr = this._lerpHr > 1 ? Math.round(this._lerpHr) : '--';
const dispBr = this._lerpBr > 1 ? Math.round(this._lerpBr) : '--';
const dispConf = this._lerpConf > 1 ? Math.round(this._lerpConf) : '--';
this._setText('hr-value', dispHr);
this._setText('br-value', dispBr);
this._setText('conf-value', dispConf);
this._setWidth('hr-bar', Math.min(100, this._lerpHr / 120 * 100));
this._setWidth('br-bar', Math.min(100, this._lerpBr / 30 * 100));
this._setWidth('conf-bar', this._lerpConf);
// Color-code vital values
this._setColor('hr-value', vitalColor('hr', this._lerpHr));
this._setColor('br-value', vitalColor('br', this._lerpBr));
this._setColor('conf-value', vitalColor('conf', this._lerpConf));
// Color-code bar fills to match
this._setBarColor('hr-bar', vitalColor('hr', this._lerpHr));
this._setBarColor('br-bar', vitalColor('br', this._lerpBr));
this._setBarColor('conf-bar', vitalColor('conf', this._lerpConf));
this._setText('rssi-value', `${Math.round(feat.mean_rssi || 0)} dBm`);
this._setText('var-value', (feat.variance || 0).toFixed(2));
this._setText('motion-value', (feat.motion_band_power || 0).toFixed(3));
// Mini person-count dots
const personCount = data.estimated_persons || 0;
this._updatePersonDots(personCount);
const presEl = document.getElementById('presence-indicator');
const presLabel = document.getElementById('presence-label');
if (presEl) {
const ml = cls.motion_level || 'absent';
presEl.className = 'presence-state';
if (ml === 'active') { presEl.classList.add('presence--active'); presLabel.textContent = 'ACTIVE'; }
else if (cls.presence) { presEl.classList.add('presence--present'); presLabel.textContent = 'PRESENT'; }
else { presEl.classList.add('presence--absent'); presLabel.textContent = 'ABSENT'; }
}
const fallEl = document.getElementById('fall-alert');
if (fallEl) fallEl.style.display = cls.fall_detected ? 'block' : 'none';
// Scenario description and edge modules
const scenarioKey = demoData._autoMode ? (demoData.currentScenario || 'auto') : (demoData.currentScenario || 'auto');
if (scenarioKey !== this._currentScenarioKey) {
this._currentScenarioKey = scenarioKey;
this._updateScenarioDescription(scenarioKey);
this._updateEdgeModules(scenarioKey);
}
}
// ============================================================
// Sparkline
// ============================================================
updateSparkline(data) {
const rssi = data?.features?.mean_rssi;
if (rssi == null || !this._sparklineCtx) return;
this._rssiHistory.push(rssi);
if (this._rssiHistory.length > 60) this._rssiHistory.shift();
const ctx = this._sparklineCtx;
const w = ctx.canvas.width, h = ctx.canvas.height;
ctx.clearRect(0, 0, w, h);
if (this._rssiHistory.length < 2) return;
ctx.beginPath();
ctx.strokeStyle = '#2090ff';
ctx.lineWidth = 1.5;
ctx.shadowColor = '#2090ff';
ctx.shadowBlur = 4;
for (let i = 0; i < this._rssiHistory.length; i++) {
const x = (i / (this._rssiHistory.length - 1)) * w;
const norm = Math.max(0, Math.min(1, (this._rssiHistory[i] + 80) / 60));
const y = h - norm * h;
i === 0 ? ctx.moveTo(x, y) : ctx.lineTo(x, y);
}
ctx.stroke();
ctx.shadowBlur = 0;
ctx.lineTo(w, h);
ctx.lineTo(0, h);
ctx.closePath();
const grad = ctx.createLinearGradient(0, 0, 0, h);
grad.addColorStop(0, 'rgba(32,144,255,0.15)');
grad.addColorStop(1, 'rgba(32,144,255,0)');
ctx.fillStyle = grad;
ctx.fill();
}
// ============================================================
// Private helpers
// ============================================================
_setText(id, val) {
const e = document.getElementById(id);
if (e) e.textContent = val;
}
_setWidth(id, pct) {
const e = document.getElementById(id);
if (e) e.style.width = `${pct}%`;
}
_setColor(id, color) {
const e = document.getElementById(id);
if (e) e.style.color = color;
}
_setBarColor(id, color) {
const e = document.getElementById(id);
if (e) e.style.background = color;
}
_bindRange(id, key, applyFn) {
const el = document.getElementById(id);
const valEl = document.getElementById(`${id}-val`);
if (!el) return;
el.value = this._obs.settings[key];
if (valEl) valEl.textContent = this._obs.settings[key];
el.addEventListener('input', (e) => {
const v = parseFloat(e.target.value);
this._obs.settings[key] = v;
if (valEl) valEl.textContent = v;
if (applyFn) applyFn(v);
this.saveSettings();
});
}
_updatePersonDots(count) {
const container = document.getElementById('persons-dots');
if (!container) {
// Fall back to text-only display
this._setText('persons-value', count);
return;
}
// Build dot icons: filled for detected persons, dim for empty slots (max 8)
const maxDots = 8;
const clamped = Math.min(count, maxDots);
let html = '';
for (let i = 0; i < maxDots; i++) {
const active = i < clamped;
html += `<span class="person-dot${active ? ' person-dot--active' : ''}"></span>`;
}
container.innerHTML = html;
this._setText('persons-value', count);
}
_updateScenarioDescription(scenarioKey) {
const el = document.getElementById('scenario-description');
if (!el) return;
el.textContent = SCENARIO_DESCRIPTIONS[scenarioKey] || '';
}
_updateEdgeModules(scenarioKey) {
const bar = document.getElementById('edge-modules-bar');
if (!bar) return;
const modules = SCENARIO_EDGE_MODULES[scenarioKey] || [];
if (modules.length === 0) {
bar.innerHTML = '';
bar.style.display = 'none';
return;
}
bar.style.display = 'flex';
bar.innerHTML = modules.map(m => {
const color = MODULE_COLORS[m] || 'var(--text-secondary)';
return `<span class="edge-badge" style="--badge-color:${color}">${m}</span>`;
}).join('');
}
}
+715
View File
@@ -0,0 +1,715 @@
/**
* RuView Observatory Main Scene Orchestrator
*
* Room-based WiFi sensing visualization with:
* - Pool of 4 human wireframe figures (multi-person scenarios)
* - 7 pose types (standing, walking, lying, sitting, fallen, exercising, gesturing, crouching)
* - Scenario-specific room props (chair, exercise mat, door, rubble wall, screen, desk)
* - Dot-matrix mist body mass, particle trails, WiFi waves, signal field
* - Reflective floor, settings dialog, and practical data HUD
*/
import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
import { DemoDataGenerator } from './demo-data.js';
import { NebulaBackground } from './nebula-background.js';
import { PostProcessing } from './post-processing.js';
import { FigurePool, SKELETON_PAIRS } from './figure-pool.js';
import { PoseSystem } from './pose-system.js';
import { ScenarioProps } from './scenario-props.js';
import { HudController, DEFAULTS, SETTINGS_VERSION, PRESETS, SCENARIO_NAMES } from './hud-controller.js';
// ---- Palette ----
const C = {
greenGlow: 0x00d878,
greenBright:0x3eff8a,
greenDim: 0x0a6b3a,
amber: 0xffb020,
blueSignal: 0x2090ff,
redAlert: 0xff3040,
redHeart: 0xff4060,
bgDeep: 0x080c14,
};
// SCENARIO_NAMES, DEFAULTS, SETTINGS_VERSION, PRESETS imported from hud-controller.js
// ---- Main Class ----
class Observatory {
constructor() {
this._canvas = document.getElementById('observatory-canvas');
this.settings = { ...DEFAULTS };
// Load saved settings
try {
const ver = localStorage.getItem('ruview-settings-version');
if (ver === SETTINGS_VERSION) {
const saved = localStorage.getItem('ruview-observatory-settings');
if (saved) Object.assign(this.settings, JSON.parse(saved));
} else {
localStorage.removeItem('ruview-observatory-settings');
localStorage.setItem('ruview-settings-version', SETTINGS_VERSION);
}
} catch {}
// Renderer
this._renderer = new THREE.WebGLRenderer({
canvas: this._canvas,
antialias: true,
powerPreference: 'high-performance',
});
this._renderer.setPixelRatio(Math.min(window.devicePixelRatio, 2));
this._renderer.setSize(window.innerWidth, window.innerHeight);
this._renderer.toneMapping = THREE.ACESFilmicToneMapping;
this._renderer.toneMappingExposure = this.settings.exposure;
this._renderer.shadowMap.enabled = true;
this._renderer.shadowMap.type = THREE.PCFSoftShadowMap;
// Scene
this._scene = new THREE.Scene();
this._scene.background = new THREE.Color(C.bgDeep);
this._scene.fog = new THREE.FogExp2(C.bgDeep, 0.005);
// Camera
this._camera = new THREE.PerspectiveCamera(
this.settings.fov, window.innerWidth / window.innerHeight, 0.1, 300
);
this._camera.position.set(6, 5, 8);
this._camera.lookAt(0, 1.2, 0);
// Controls
this._controls = new OrbitControls(this._camera, this._canvas);
this._controls.enableDamping = true;
this._controls.dampingFactor = 0.08;
this._controls.minDistance = 2;
this._controls.maxDistance = 25;
this._controls.maxPolarAngle = Math.PI * 0.88;
this._controls.target.set(0, 1.2, 0);
this._controls.update();
this._clock = new THREE.Clock();
// Data
this._demoData = new DemoDataGenerator();
this._demoData.setCycleDuration(this.settings.cycle || 30);
if (this.settings.scenario && this.settings.scenario !== 'auto') {
this._demoData.setScenario(this.settings.scenario);
}
this._currentData = null;
this._currentScenario = null;
// Build scene
this._setupLighting();
this._nebula = new NebulaBackground(this._scene);
this._buildRoom();
this._buildRouter();
this._poseSystem = new PoseSystem();
this._figurePool = new FigurePool(this._scene, this.settings, this._poseSystem);
this._scenarioProps = new ScenarioProps(this._scene);
this._buildDotMatrixMist();
this._buildParticleTrail();
this._buildWifiWaves();
this._buildSignalField();
// Post-processing
this._postProcessing = new PostProcessing(this._renderer, this._scene, this._camera);
this._applyPostSettings();
// HUD controller (settings dialog, sparkline, vital displays)
this._hud = new HudController(this);
// State
this._autopilot = false;
this._autoAngle = 0;
this._fpsFrames = 0;
this._fpsTime = 0;
this._fpsValue = 60;
this._showFps = false;
this._qualityLevel = 2;
// WebSocket for live data — always try auto-detect on startup
this._ws = null;
this._liveData = null;
this._autoDetectLive();
// Input
this._initKeyboard();
this._hud.initSettings();
this._hud.initQuickSelect();
window.addEventListener('resize', () => this._onResize());
// Start
this._animate();
}
// ---- Lighting ----
_setupLighting() {
this._ambient = new THREE.AmbientLight(0xccccdd, this.settings.ambient * 5.0);
this._scene.add(this._ambient);
const hemi = new THREE.HemisphereLight(0x6688bb, 0x203040, 1.2);
this._scene.add(hemi);
const key = new THREE.DirectionalLight(0xffeedd, 1.2);
key.position.set(4, 8, 3);
key.castShadow = true;
key.shadow.mapSize.set(1024, 1024);
key.shadow.camera.near = 0.5;
key.shadow.camera.far = 20;
key.shadow.camera.left = -8;
key.shadow.camera.right = 8;
key.shadow.camera.top = 8;
key.shadow.camera.bottom = -8;
this._scene.add(key);
// Fill light from opposite side
const fill = new THREE.DirectionalLight(0x8899bb, 0.7);
fill.position.set(-4, 5, -2);
this._scene.add(fill);
// Rim light from above/behind for edge definition
const rim = new THREE.DirectionalLight(0x6699cc, 0.5);
rim.position.set(0, 6, -5);
this._scene.add(rim);
// Overhead room light — general illumination
const overhead = new THREE.PointLight(0x8899aa, 1.0, 20, 1.0);
overhead.position.set(0, 3.8, 0);
this._scene.add(overhead);
}
// ---- Room ----
_buildRoom() {
this._grid = new THREE.GridHelper(12, 24, 0x1a4830, 0x0c2818);
this._grid.material.opacity = 0.5;
this._grid.material.transparent = true;
this._scene.add(this._grid);
const boxGeo = new THREE.BoxGeometry(12, 4, 10);
const edges = new THREE.EdgesGeometry(boxGeo);
this._roomWire = new THREE.LineSegments(edges, new THREE.LineBasicMaterial({
color: C.greenDim, opacity: 0.3, transparent: true,
}));
this._roomWire.position.y = 2;
this._scene.add(this._roomWire);
// Reflective floor
const floorGeo = new THREE.PlaneGeometry(12, 10);
this._floorMat = new THREE.MeshStandardMaterial({
color: 0x101810,
roughness: 1.0 - this.settings.reflect * 0.7,
metalness: this.settings.reflect * 0.5,
emissive: 0x020404,
emissiveIntensity: 0.08,
});
const floor = new THREE.Mesh(floorGeo, this._floorMat);
floor.rotation.x = -Math.PI / 2;
floor.receiveShadow = true;
this._scene.add(floor);
// Table under router
const tableGeo = new THREE.BoxGeometry(0.8, 0.6, 0.5);
const tableMat = new THREE.MeshStandardMaterial({ color: 0x6b5840, roughness: 0.55, emissive: 0x1a1408, emissiveIntensity: 0.25 });
const table = new THREE.Mesh(tableGeo, tableMat);
table.position.set(-4, 0.3, -3);
table.castShadow = true;
this._scene.add(table);
}
// ---- Router ----
_buildRouter() {
this._routerGroup = new THREE.Group();
this._routerGroup.position.set(-4, 0.92, -3);
const bodyGeo = new THREE.BoxGeometry(0.6, 0.12, 0.35);
const bodyMat = new THREE.MeshStandardMaterial({ color: 0x505060, roughness: 0.2, metalness: 0.7, emissive: 0x101018, emissiveIntensity: 0.2 });
this._routerGroup.add(new THREE.Mesh(bodyGeo, bodyMat));
for (let i = -1; i <= 1; i++) {
const antGeo = new THREE.CylinderGeometry(0.015, 0.015, 0.35);
const antMat = new THREE.MeshStandardMaterial({ color: 0x606068, roughness: 0.3, metalness: 0.6, emissive: 0x101018, emissiveIntensity: 0.15 });
const ant = new THREE.Mesh(antGeo, antMat);
ant.position.set(i * 0.2, 0.24, 0);
ant.rotation.z = i * 0.15;
this._routerGroup.add(ant);
}
const ledGeo = new THREE.SphereGeometry(0.025);
this._routerLed = new THREE.Mesh(ledGeo, new THREE.MeshBasicMaterial({ color: C.greenGlow }));
this._routerLed.position.set(0.22, 0.07, 0.18);
this._routerGroup.add(this._routerLed);
this._routerLight = new THREE.PointLight(C.blueSignal, 1.2, 8);
this._routerLight.position.set(0, 0.3, 0);
this._routerGroup.add(this._routerLight);
this._scene.add(this._routerGroup);
}
// ---- WiFi Waves ----
_buildWifiWaves() {
this._wifiWaves = [];
for (let i = 0; i < 5; i++) {
const radius = 0.8 + i * 1.0;
const geo = new THREE.SphereGeometry(radius, 24, 16, 0, Math.PI * 2, 0, Math.PI * 0.6);
const mat = new THREE.MeshBasicMaterial({
color: C.blueSignal,
transparent: true, opacity: 0,
side: THREE.DoubleSide,
blending: THREE.AdditiveBlending,
depthWrite: false, wireframe: true,
});
const shell = new THREE.Mesh(geo, mat);
shell.position.copy(this._routerGroup.position);
shell.position.y += 0.5;
this._scene.add(shell);
this._wifiWaves.push({ mesh: shell, mat, phase: i * 0.7 });
}
}
// ========================================
// DOT MATRIX MIST
// ========================================
_buildDotMatrixMist() {
const COUNT = 800;
const positions = new Float32Array(COUNT * 3);
const alphas = new Float32Array(COUNT);
for (let i = 0; i < COUNT; i++) {
const angle = Math.random() * Math.PI * 2;
const r = Math.random() * 0.5;
positions[i * 3] = Math.cos(angle) * r;
positions[i * 3 + 1] = Math.random() * 1.8;
positions[i * 3 + 2] = Math.sin(angle) * r;
alphas[i] = 0;
}
const geo = new THREE.BufferGeometry();
geo.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geo.setAttribute('alpha', new THREE.BufferAttribute(alphas, 1));
const mat = new THREE.ShaderMaterial({
vertexShader: `
attribute float alpha;
varying float vAlpha;
void main() {
vAlpha = alpha;
vec4 mv = modelViewMatrix * vec4(position, 1.0);
gl_PointSize = 3.0 * (200.0 / -mv.z);
gl_Position = projectionMatrix * mv;
}
`,
fragmentShader: `
uniform vec3 uColor;
varying float vAlpha;
void main() {
float d = length(gl_PointCoord - 0.5);
if (d > 0.5) discard;
float edge = smoothstep(0.5, 0.2, d);
gl_FragColor = vec4(uColor, edge * vAlpha);
}
`,
uniforms: { uColor: { value: new THREE.Color(this.settings.wireColor) } },
transparent: true, blending: THREE.AdditiveBlending, depthWrite: false,
});
this._mistPoints = new THREE.Points(geo, mat);
this._scene.add(this._mistPoints);
this._mistCount = COUNT;
}
// ---- Particle Trail ----
_buildParticleTrail() {
const COUNT = 200;
const positions = new Float32Array(COUNT * 3);
const ages = new Float32Array(COUNT);
for (let i = 0; i < COUNT; i++) ages[i] = 1;
const geo = new THREE.BufferGeometry();
geo.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geo.setAttribute('age', new THREE.BufferAttribute(ages, 1));
const mat = new THREE.ShaderMaterial({
vertexShader: `
attribute float age;
varying float vAge;
void main() {
vAge = age;
vec4 mv = modelViewMatrix * vec4(position, 1.0);
gl_PointSize = max(1.0, (1.0 - age) * 5.0 * (150.0 / -mv.z));
gl_Position = projectionMatrix * mv;
}
`,
fragmentShader: `
uniform vec3 uColor;
varying float vAge;
void main() {
float d = length(gl_PointCoord - 0.5);
if (d > 0.5) discard;
float alpha = (1.0 - vAge) * 0.6 * smoothstep(0.5, 0.1, d);
gl_FragColor = vec4(uColor, alpha);
}
`,
uniforms: { uColor: { value: new THREE.Color(C.greenGlow) } },
transparent: true, blending: THREE.AdditiveBlending, depthWrite: false,
});
this._trail = new THREE.Points(geo, mat);
this._scene.add(this._trail);
this._trailHead = 0;
this._trailCount = COUNT;
this._trailTimer = 0;
}
// ---- Signal Field ----
_buildSignalField() {
const gridSize = 20;
const count = gridSize * gridSize;
const positions = new Float32Array(count * 3);
this._fieldColors = new Float32Array(count * 3);
this._fieldSizes = new Float32Array(count);
for (let iz = 0; iz < gridSize; iz++) {
for (let ix = 0; ix < gridSize; ix++) {
const idx = iz * gridSize + ix;
positions[idx * 3] = (ix - gridSize / 2) * 0.6;
positions[idx * 3 + 1] = 0.02;
positions[idx * 3 + 2] = (iz - gridSize / 2) * 0.5;
this._fieldSizes[idx] = 8;
}
}
const geo = new THREE.BufferGeometry();
geo.setAttribute('position', new THREE.BufferAttribute(positions, 3));
geo.setAttribute('color', new THREE.BufferAttribute(this._fieldColors, 3));
geo.setAttribute('size', new THREE.BufferAttribute(this._fieldSizes, 1));
this._fieldMat = new THREE.PointsMaterial({
size: 0.35, vertexColors: true, transparent: true,
opacity: this.settings.field, blending: THREE.AdditiveBlending,
depthWrite: false, sizeAttenuation: true,
});
this._fieldPoints = new THREE.Points(geo, this._fieldMat);
this._scene.add(this._fieldPoints);
}
// ---- Keyboard ----
_initKeyboard() {
window.addEventListener('keydown', (e) => {
if (this._hud.settingsOpen) return;
switch (e.key.toLowerCase()) {
case 'a':
this._autopilot = !this._autopilot;
this._controls.enabled = !this._autopilot;
break;
case 'd': this._demoData.cycleScenario(); break;
case 'f':
this._showFps = !this._showFps;
document.getElementById('fps-counter').style.display = this._showFps ? 'block' : 'none';
break;
case 's': this._hud.toggleSettings(); break;
case ' ':
e.preventDefault();
this._demoData.paused = !this._demoData.paused;
break;
}
});
}
// ---- Settings / HUD methods delegated to HudController ----
_applyPostSettings() {
const pp = this._postProcessing;
pp._bloomPass.strength = this.settings.bloom;
pp._bloomPass.radius = this.settings.bloomRadius;
pp._bloomPass.threshold = this.settings.bloomThresh;
pp._vignettePass.uniforms.uVignetteStrength.value = this.settings.vignette;
pp._vignettePass.uniforms.uGrainStrength.value = this.settings.grain;
pp._vignettePass.uniforms.uChromaticStrength.value = this.settings.chromatic;
}
_applyColors() {
const wc = new THREE.Color(this.settings.wireColor);
const jc = new THREE.Color(this.settings.jointColor);
this._figurePool.applyColors(wc, jc);
this._mistPoints.material.uniforms.uColor.value.copy(wc);
}
// ---- WebSocket live data ----
_autoDetectLive() {
// Probe sensing server health on same origin, then common ports
const host = window.location.hostname || 'localhost';
const candidates = [
window.location.origin, // same origin (e.g. :3000)
`http://${host}:8765`, // default WS port
`http://${host}:3000`, // default HTTP port
];
// Deduplicate
const unique = [...new Set(candidates)];
const tryNext = (i) => {
if (i >= unique.length) {
console.log('[Observatory] No sensing server detected, using demo mode');
return;
}
const base = unique[i];
fetch(`${base}/health`, { signal: AbortSignal.timeout(1500) })
.then(r => r.ok ? r.json() : Promise.reject())
.then(data => {
if (data && data.status === 'ok') {
const wsProto = base.startsWith('https') ? 'wss:' : 'ws:';
const urlObj = new URL(base);
const wsUrl = `${wsProto}//${urlObj.host}/ws/sensing`;
console.log('[Observatory] Sensing server detected at', base, '→', wsUrl);
this.settings.dataSource = 'ws';
this.settings.wsUrl = wsUrl;
this._connectWS(wsUrl);
} else {
tryNext(i + 1);
}
})
.catch(() => tryNext(i + 1));
};
tryNext(0);
}
_connectWS(url) {
this._disconnectWS();
try {
this._ws = new WebSocket(url);
this._ws.onopen = () => {
console.log('[Observatory] WebSocket connected');
this._hud.updateSourceBadge('ws', this._ws);
};
this._ws.onmessage = (evt) => { try { this._liveData = JSON.parse(evt.data); } catch {} };
this._ws.onclose = () => {
console.log('[Observatory] WebSocket closed, falling back to demo');
this._ws = null;
this.settings.dataSource = 'demo';
this._hud.updateSourceBadge('demo', null);
};
this._ws.onerror = () => {};
} catch {}
}
_disconnectWS() {
if (this._ws) { this._ws.close(); this._ws = null; }
this._liveData = null;
}
// ========================================
// ANIMATION LOOP
// ========================================
_animate() {
requestAnimationFrame(() => this._animate());
const dt = Math.min(this._clock.getDelta(), 0.1);
const elapsed = this._clock.getElapsedTime();
// Data source
if (this.settings.dataSource === 'ws' && this._liveData) {
this._currentData = this._liveData;
} else {
this._currentData = this._demoData.update(dt);
}
const data = this._currentData;
// Updates
this._nebula.update(dt, elapsed);
this._figurePool.update(data, elapsed);
this._scenarioProps.update(data, this._demoData.currentScenario);
this._updateDotMatrixMist(data, elapsed);
this._updateParticleTrail(data, dt, elapsed);
this._updateWifiWaves(elapsed);
this._updateSignalField(data);
this._hud.updateHUD(data, this._demoData);
this._hud.updateSparkline(data);
// Router LED
this._routerLed.material.opacity = 0.5 + 0.5 * Math.sin(elapsed * 8);
this._routerLight.intensity = 0.3 + 0.2 * Math.sin(elapsed * 3);
// Autopilot orbit
if (this._autopilot) {
this._autoAngle += dt * this.settings.orbitSpeed;
const r = 10;
this._camera.position.set(
Math.sin(this._autoAngle) * r,
4.5 + Math.sin(this._autoAngle * 0.5),
Math.cos(this._autoAngle) * r
);
this._controls.target.set(0, 1.2, 0);
this._controls.update();
}
this._controls.update();
this._postProcessing.update(elapsed);
this._postProcessing.render();
this._updateFPS(dt);
}
// ========================================
// MIST & TRAIL
// ========================================
_updateDotMatrixMist(data, elapsed) {
const persons = data?.persons || [];
const isPresent = data?.classification?.presence || false;
const pos = this._mistPoints.geometry.attributes.position;
const alpha = this._mistPoints.geometry.attributes.alpha;
if (!isPresent || persons.length === 0) {
for (let i = 0; i < this._mistCount; i++) {
alpha.array[i] = Math.max(0, alpha.array[i] - 0.02);
}
alpha.needsUpdate = true;
return;
}
// Follow primary person
const pp = persons[0].position || [0, 0, 0];
const px = pp[0] || 0, pz = pp[2] || 0;
const ms = persons[0].motion_score || 0;
const pose = persons[0].pose || 'standing';
const isLying = pose === 'lying' || pose === 'fallen';
const bodyH = isLying ? 0.4 : 1.7;
const bodyBaseY = isLying ? (pp[1] || 0) + 0.05 : 0.05;
const spread = ms > 50 ? 0.6 : 0.4;
for (let i = 0; i < this._mistCount; i++) {
const drift = Math.sin(elapsed * 0.5 + i * 0.1) * 0.003;
const angle = (i / this._mistCount) * Math.PI * 2 + elapsed * 0.1;
const layerT = (i % 20) / 20;
const layerY = bodyBaseY + layerT * bodyH;
let bodyWidth;
if (isLying) {
bodyWidth = 0.25;
} else {
bodyWidth = layerT > 0.75 ? 0.15 : (layerT > 0.45 ? 0.25 : 0.18);
}
const r = bodyWidth * (0.5 + 0.5 * Math.sin(i * 1.7 + elapsed * 0.3)) * spread;
const tx = px + Math.cos(angle + i * 0.3) * r + drift;
const tz = pz + Math.sin(angle + i * 0.5) * r * 0.6;
pos.array[i * 3] += (tx - pos.array[i * 3]) * 0.05;
pos.array[i * 3 + 1] += (layerY - pos.array[i * 3 + 1]) * 0.05;
pos.array[i * 3 + 2] += (tz - pos.array[i * 3 + 2]) * 0.05;
const targetAlpha = 0.15 + Math.sin(elapsed * 2 + i * 0.5) * 0.08;
alpha.array[i] += (targetAlpha - alpha.array[i]) * 0.08;
}
pos.needsUpdate = true;
alpha.needsUpdate = true;
}
_updateParticleTrail(data, dt, elapsed) {
if (this.settings.trail <= 0) return;
const persons = data?.persons || [];
const isPresent = data?.classification?.presence || false;
const pos = this._trail.geometry.attributes.position;
const ages = this._trail.geometry.attributes.age;
for (let i = 0; i < this._trailCount; i++) {
ages.array[i] = Math.min(1, ages.array[i] + dt * 0.8);
}
// Emit from all active persons
if (isPresent && persons.length > 0) {
this._trailTimer += dt;
const ms = persons[0].motion_score || 0;
const emitRate = ms > 50 ? 0.02 : 0.08;
if (this._trailTimer >= emitRate) {
this._trailTimer = 0;
for (const p of persons) {
const pp = p.position || [0, 0, 0];
const idx = this._trailHead;
pos.array[idx * 3] = (pp[0] || 0) + (Math.random() - 0.5) * 0.15;
pos.array[idx * 3 + 1] = Math.random() * 1.5 + 0.1;
pos.array[idx * 3 + 2] = (pp[2] || 0) + (Math.random() - 0.5) * 0.15;
ages.array[idx] = 0;
this._trailHead = (this._trailHead + 1) % this._trailCount;
}
}
}
pos.needsUpdate = true;
ages.needsUpdate = true;
}
// ---- WiFi Waves ----
_updateWifiWaves(elapsed) {
for (const w of this._wifiWaves) {
const t = (elapsed * 0.8 + w.phase) % 4.5;
const life = t / 4.5;
w.mat.opacity = Math.max(0, this.settings.waves * 0.25 * (1 - life));
const scale = 1 + life * 0.6;
w.mesh.scale.set(scale, scale, scale);
w.mesh.rotation.y = elapsed * 0.05;
}
}
// ---- Signal Field ----
_updateSignalField(data) {
const field = data?.signal_field?.values;
if (!field) return;
const count = Math.min(field.length, 400);
for (let i = 0; i < count; i++) {
const v = field[i] || 0;
let r, g, b;
if (v < 0.3) { r = 0; g = v * 1.5; b = v * 0.3; }
else if (v < 0.6) {
const t = (v - 0.3) / 0.3;
r = t * 0.3; g = 0.45 + t * 0.4; b = 0.09 - t * 0.05;
} else {
const t = (v - 0.6) / 0.4;
r = 0.3 + t * 0.7; g = 0.85 - t * 0.2; b = 0.04;
}
this._fieldColors[i * 3] = r;
this._fieldColors[i * 3 + 1] = g;
this._fieldColors[i * 3 + 2] = b;
this._fieldSizes[i] = 5 + v * 15;
}
this._fieldPoints.geometry.attributes.color.needsUpdate = true;
this._fieldPoints.geometry.attributes.size.needsUpdate = true;
}
// ---- FPS ----
_updateFPS(dt) {
this._fpsFrames++;
this._fpsTime += dt;
if (this._fpsTime >= 1) {
this._fpsValue = Math.round(this._fpsFrames / this._fpsTime);
this._fpsFrames = 0;
this._fpsTime = 0;
if (this._showFps) {
document.getElementById('fps-counter').textContent = `${this._fpsValue} FPS`;
}
this._adaptQuality();
}
}
_adaptQuality() {
let nl = this._qualityLevel;
if (this._fpsValue < 25 && nl > 0) nl--;
else if (this._fpsValue > 55 && nl < 2) nl++;
if (nl !== this._qualityLevel) {
this._qualityLevel = nl;
this._nebula.setQuality(nl);
this._postProcessing.setQuality(nl);
}
}
_onResize() {
const w = window.innerWidth, h = window.innerHeight;
this._camera.aspect = w / h;
this._camera.updateProjectionMatrix();
this._renderer.setSize(w, h);
this._postProcessing.resize(w, h);
}
}
new Observatory();
+115
View File
@@ -0,0 +1,115 @@
/**
* Room Atmosphere Background Warm dark gradient with subtle particles
* Matches RuView Foundation aesthetic: deep blue-black with warm undertones
*/
import * as THREE from 'three';
const BG_VERTEX = `
varying vec3 vWorldPos;
void main() {
vWorldPos = (modelMatrix * vec4(position, 1.0)).xyz;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`;
const BG_FRAGMENT = `
uniform float uTime;
uniform float uOctaves;
varying vec3 vWorldPos;
vec3 hash33(vec3 p) {
p = fract(p * vec3(443.8975, 397.2973, 491.1871));
p += dot(p, p.yxz + 19.19);
return fract(vec3(p.x * p.y, p.y * p.z, p.z * p.x));
}
float noise3d(vec3 p) {
vec3 i = floor(p);
vec3 f = fract(p);
f = f * f * (3.0 - 2.0 * f);
float n = mix(
mix(mix(dot(hash33(i), f), dot(hash33(i + vec3(1,0,0)), f - vec3(1,0,0)), f.x),
mix(dot(hash33(i + vec3(0,1,0)), f - vec3(0,1,0)), dot(hash33(i + vec3(1,1,0)), f - vec3(1,1,0)), f.x), f.y),
mix(mix(dot(hash33(i + vec3(0,0,1)), f - vec3(0,0,1)), dot(hash33(i + vec3(1,0,1)), f - vec3(1,0,1)), f.x),
mix(dot(hash33(i + vec3(0,1,1)), f - vec3(0,1,1)), dot(hash33(i + vec3(1,1,1)), f - vec3(1,1,1)), f.x), f.y),
f.z);
return n * 0.5 + 0.5;
}
float fbm(vec3 p, float octaves) {
float v = 0.0, a = 0.5;
for (float i = 0.0; i < 5.0; i++) {
if (i >= octaves) break;
v += a * noise3d(p);
p *= 2.0;
a *= 0.5;
}
return v;
}
void main() {
vec3 dir = normalize(vWorldPos);
// Warm dark atmosphere with subtle color variation
float n1 = fbm(dir * 2.5 + uTime * 0.008, uOctaves);
float n2 = fbm(dir * 4.0 - uTime * 0.005, max(1.0, uOctaves - 1.0));
// Foundation palette: deep blue-black with warm undertones
vec3 deepBlack = vec3(0.03, 0.04, 0.06);
vec3 warmNavy = vec3(0.04, 0.05, 0.10);
vec3 greenTint = vec3(0.01, 0.06, 0.04);
vec3 bg = mix(deepBlack, warmNavy, n1 * 0.5);
bg = mix(bg, greenTint, n2 * 0.15);
// Subtle top-down gradient (lighter ceiling)
float upFactor = max(0.0, dir.y) * 0.08;
bg += vec3(0.02, 0.03, 0.05) * upFactor;
// Very subtle dim stars (distant)
vec3 c = floor(dir * 200.0);
vec3 h = hash33(c);
float star = step(0.998, h.x) * h.y * 0.15;
star *= 0.7 + 0.3 * sin(uTime * 1.5 + h.z * 80.0);
bg += vec3(0.6, 0.7, 0.8) * star;
gl_FragColor = vec4(bg, 1.0);
}
`;
export class NebulaBackground {
constructor(scene) {
this._octaves = 4;
this.uniforms = {
uTime: { value: 0 },
uOctaves: { value: this._octaves },
};
const geo = new THREE.SphereGeometry(150, 32, 32);
const mat = new THREE.ShaderMaterial({
vertexShader: BG_VERTEX,
fragmentShader: BG_FRAGMENT,
uniforms: this.uniforms,
side: THREE.BackSide,
depthWrite: false,
});
this.mesh = new THREE.Mesh(geo, mat);
scene.add(this.mesh);
}
update(dt, elapsed) {
this.uniforms.uTime.value = elapsed;
}
setQuality(level) {
this._octaves = [2, 3, 4][level] || 4;
this.uniforms.uOctaves.value = this._octaves;
}
dispose() {
this.mesh.geometry.dispose();
this.mesh.material.dispose();
}
}
+170
View File
@@ -0,0 +1,170 @@
/**
* Module D "The Phase Constellation"
* I/Q star map with constellation lines and rotating temporal view
*/
import * as THREE from 'three';
const NUM_SUBCARRIERS = 64;
export class PhaseConstellation {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Star points (current frame)
const starGeo = new THREE.BufferGeometry();
this._positions = new Float32Array(NUM_SUBCARRIERS * 3);
this._colors = new Float32Array(NUM_SUBCARRIERS * 3);
this._sizes = new Float32Array(NUM_SUBCARRIERS);
starGeo.setAttribute('position', new THREE.BufferAttribute(this._positions, 3));
starGeo.setAttribute('color', new THREE.BufferAttribute(this._colors, 3));
starGeo.setAttribute('size', new THREE.BufferAttribute(this._sizes, 1));
const starMat = new THREE.PointsMaterial({
size: 0.12,
vertexColors: true,
transparent: true,
opacity: 0.9,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._stars = new THREE.Points(starGeo, starMat);
this.group.add(this._stars);
// Ghost layer (previous frame)
const ghostGeo = new THREE.BufferGeometry();
this._ghostPos = new Float32Array(NUM_SUBCARRIERS * 3);
ghostGeo.setAttribute('position', new THREE.BufferAttribute(this._ghostPos, 3));
const ghostMat = new THREE.PointsMaterial({
color: 0x00d4ff,
size: 0.06,
transparent: true,
opacity: 0.2,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._ghosts = new THREE.Points(ghostGeo, ghostMat);
this.group.add(this._ghosts);
// Constellation lines (connecting adjacent subcarriers)
const lineGeo = new THREE.BufferGeometry();
this._linePos = new Float32Array(NUM_SUBCARRIERS * 2 * 3); // pairs
lineGeo.setAttribute('position', new THREE.BufferAttribute(this._linePos, 3));
const lineMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.15,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._lines = new THREE.LineSegments(lineGeo, lineMat);
this.group.add(this._lines);
// Axes
this._addAxes();
this._prevIQ = null;
}
_addAxes() {
const axesMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.1,
});
// I axis
const iGeo = new THREE.BufferGeometry().setFromPoints([
new THREE.Vector3(-2.5, 0, 0),
new THREE.Vector3(2.5, 0, 0),
]);
this.group.add(new THREE.Line(iGeo, axesMat));
// Q axis
const qGeo = new THREE.BufferGeometry().setFromPoints([
new THREE.Vector3(0, -2.5, 0),
new THREE.Vector3(0, 2.5, 0),
]);
this.group.add(new THREE.Line(qGeo, axesMat));
}
update(dt, elapsed, data) {
const iq = data?._observatory?.subcarrier_iq;
const variance = data?._observatory?.per_subcarrier_variance;
const amplitude = data?.nodes?.[0]?.amplitude;
// Slow Y rotation for temporal evolution
this.group.rotation.y = elapsed * 0.05;
if (!iq || iq.length < NUM_SUBCARRIERS) return;
// Copy current to ghost
this._ghostPos.set(this._positions);
this._ghosts.geometry.attributes.position.needsUpdate = true;
// Update current positions from I/Q
for (let s = 0; s < NUM_SUBCARRIERS; s++) {
const i3 = s * 3;
const iVal = (iq[s]?.i || 0) * 4; // scale for visibility
const qVal = (iq[s]?.q || 0) * 4;
this._positions[i3] = iVal;
this._positions[i3 + 1] = qVal;
this._positions[i3 + 2] = 0;
// Size from amplitude
const amp = amplitude ? (amplitude[s % amplitude.length] || 0.1) : 0.1;
this._sizes[s] = 0.06 + amp * 0.15;
// Color from variance: blue(low) -> amber(high)
const v = variance ? Math.min(1, (variance[s] || 0) * 2) : 0;
this._colors[i3] = v * 1.0; // R
this._colors[i3 + 1] = 0.5 + v * 0.3; // G
this._colors[i3 + 2] = 1.0 - v * 0.7; // B
}
this._stars.geometry.attributes.position.needsUpdate = true;
this._stars.geometry.attributes.color.needsUpdate = true;
this._stars.geometry.attributes.size.needsUpdate = true;
// Update constellation lines
for (let s = 0; s < NUM_SUBCARRIERS - 1; s++) {
const li = s * 6;
const i3a = s * 3;
const i3b = (s + 1) * 3;
this._linePos[li] = this._positions[i3a];
this._linePos[li + 1] = this._positions[i3a + 1];
this._linePos[li + 2] = this._positions[i3a + 2];
this._linePos[li + 3] = this._positions[i3b];
this._linePos[li + 4] = this._positions[i3b + 1];
this._linePos[li + 5] = this._positions[i3b + 2];
}
// Last pair: wrap around
const lastLi = (NUM_SUBCARRIERS - 1) * 6;
const lastI3 = (NUM_SUBCARRIERS - 1) * 3;
this._linePos[lastLi] = this._positions[lastI3];
this._linePos[lastLi + 1] = this._positions[lastI3 + 1];
this._linePos[lastLi + 2] = this._positions[lastI3 + 2];
this._linePos[lastLi + 3] = this._positions[0];
this._linePos[lastLi + 4] = this._positions[1];
this._linePos[lastLi + 5] = this._positions[2];
this._lines.geometry.attributes.position.needsUpdate = true;
}
dispose() {
this._stars.geometry.dispose();
this._stars.material.dispose();
this._ghosts.geometry.dispose();
this._ghosts.material.dispose();
this._lines.geometry.dispose();
this._lines.material.dispose();
}
}
+567
View File
@@ -0,0 +1,567 @@
/**
* PoseSystem -- Stateless pose keypoint generator for COCO 17-keypoint format.
*
* Keypoint indices:
* 0:nose 1:left_eye 2:right_eye 3:left_ear 4:right_ear
* 5:left_shoulder 6:right_shoulder 7:left_elbow 8:right_elbow
* 9:left_wrist 10:right_wrist 11:left_hip 12:right_hip
* 13:left_knee 14:right_knee 15:left_ankle 16:right_ankle
*
* Every public method is a pure function: parameters in, keypoint array out.
*/
export class PoseSystem {
// ---- Entry point -------------------------------------------------------
generateKeypoints(person, elapsed, breathPulse) {
const pose = person.pose || 'standing';
const pos = person.position || [0, 0, 0];
const facing = person.facing || 0;
const px = pos[0], pz = pos[2];
const ms = person.motion_score || 0;
const bp = breathPulse;
let kps;
switch (pose) {
case 'lying': kps = this.poseLying(px, pos[1] || 0, pz, elapsed, bp); break;
case 'sitting': kps = this.poseSitting(px, pz, elapsed, bp); break;
case 'fallen': kps = this.poseFallen(px, pz, elapsed); break;
case 'falling': kps = this.poseFalling(px, pz, elapsed, person.fallProgress || 0); break;
case 'exercising': kps = this.poseExercising(px, pz, elapsed, person.exerciseType, person.exerciseTime); break;
case 'gesturing': kps = this.poseGesturing(px, pz, elapsed, person.gestureType, person.gestureIntensity || 0); break;
case 'crouching': kps = this.poseCrouching(px, pz, elapsed, bp); break;
case 'walking': kps = this.poseWalking(px, pz, elapsed, ms, bp); break;
case 'standing':
default: kps = this.poseStanding(px, pz, elapsed, ms, bp); break;
}
// Apply facing rotation
if (Math.abs(facing) > 0.01) {
this.rotateKps(kps, px, pz, facing);
}
return kps;
}
// ---- Rotation utility --------------------------------------------------
rotateKps(kps, cx, cz, angle) {
const cos = Math.cos(angle), sin = Math.sin(angle);
for (const kp of kps) {
const dx = kp[0] - cx, dz = kp[2] - cz;
kp[0] = cx + dx * cos - dz * sin;
kp[2] = cz + dx * sin + dz * cos;
}
}
// ---- Standing ----------------------------------------------------------
// Weight shift between feet, idle head look-around, breathing
poseStanding(px, pz, elapsed, ms, bp) {
// Slow weight shift side to side
const weightShift = Math.sin(elapsed * 0.6) * 0.012;
// Idle head look around
const headTurn = Math.sin(elapsed * 0.3) * 0.015;
const headTilt = Math.cos(elapsed * 0.25) * 0.008;
// Slight sway from micro-balance adjustments
const sway = Math.sin(elapsed * 0.8) * 0.005 + weightShift;
// Knee bend alternation with weight shift
const leftKneeBend = Math.max(0, Math.sin(elapsed * 0.6)) * 0.015;
const rightKneeBend = Math.max(0, -Math.sin(elapsed * 0.6)) * 0.015;
return [
[px + sway + headTurn, 1.72 + bp + headTilt, pz], // 0 nose
[px - 0.03 + sway + headTurn, 1.74 + bp + headTilt, pz - 0.02], // 1 left eye
[px + 0.03 + sway + headTurn, 1.74 + bp + headTilt, pz - 0.02], // 2 right eye
[px - 0.07 + headTurn * 0.5, 1.72 + bp, pz], // 3 left ear
[px + 0.07 + headTurn * 0.5, 1.72 + bp, pz], // 4 right ear
[px - 0.22 + weightShift * 0.3, 1.48 + bp, pz], // 5 left shoulder
[px + 0.22 + weightShift * 0.3, 1.48 + bp, pz], // 6 right shoulder
[px - 0.24 + weightShift * 0.2, 1.18 + bp, pz + 0.02], // 7 left elbow
[px + 0.24 + weightShift * 0.2, 1.18 + bp, pz - 0.02], // 8 right elbow
[px - 0.22 + weightShift * 0.15, 0.92 + bp, pz + 0.05], // 9 left wrist
[px + 0.22 + weightShift * 0.15, 0.92 + bp, pz - 0.05], // 10 right wrist
[px - 0.11 + weightShift * 0.5, 0.98 + bp, pz], // 11 left hip
[px + 0.11 + weightShift * 0.5, 0.98 + bp, pz], // 12 right hip
[px - 0.12 + weightShift * 0.3, 0.52 + leftKneeBend, pz], // 13 left knee
[px + 0.12 + weightShift * 0.3, 0.52 + rightKneeBend, pz], // 14 right knee
[px - 0.12 + weightShift * 0.4, 0.04, pz], // 15 left ankle
[px + 0.12 + weightShift * 0.4, 0.04, pz], // 16 right ankle
];
}
// ---- Walking -----------------------------------------------------------
// Torso rotation, head bob, natural arm pendulum with elbow bend
poseWalking(px, pz, elapsed, ms, bp) {
const speed = Math.min(ms / 100, 2.5);
const wp = elapsed * speed * 1.8;
const sFactor = Math.min(speed, 1);
// Leg stride
const legStride = Math.sin(wp) * 0.25 * sFactor;
const legBack = Math.sin(wp + Math.PI) * 0.25 * sFactor;
const kneeAmt = Math.abs(Math.sin(wp)) * 0.08;
// Natural arm pendulum -- opposite to legs, with elbow bend
const armPhase = Math.sin(wp);
const armSwingL = -armPhase * 0.3 * sFactor; // left arm opposite right leg
const armSwingR = armPhase * 0.3 * sFactor;
const elbowBendL = Math.max(0, -armPhase) * 0.12 * sFactor; // bend on backswing
const elbowBendR = Math.max(0, armPhase) * 0.12 * sFactor;
// Torso twist (shoulders rotate opposite to hips)
const torsoTwist = Math.sin(wp) * 0.03 * sFactor;
// Vertical bob (double frequency -- peak at mid-stance)
const bob = Math.abs(Math.sin(wp)) * 0.025;
// Head bob -- slight lag behind body
const headBob = Math.abs(Math.sin(wp - 0.2)) * 0.015;
const headLean = Math.sin(wp) * 0.008;
return [
[px + headLean, 1.72 + bp + bob + headBob, pz], // 0 nose
[px - 0.03 + headLean, 1.74 + bp + bob + headBob, pz - 0.02], // 1 left eye
[px + 0.03 + headLean, 1.74 + bp + bob + headBob, pz - 0.02], // 2 right eye
[px - 0.07, 1.72 + bp + bob + headBob, pz], // 3 left ear
[px + 0.07, 1.72 + bp + bob + headBob, pz], // 4 right ear
[px - 0.22 - torsoTwist, 1.48 + bp + bob, pz], // 5 left shoulder (twist)
[px + 0.22 - torsoTwist, 1.48 + bp + bob, pz], // 6 right shoulder
[px - 0.28 + armSwingL * 0.3, 1.18 + bp + bob - elbowBendL, pz + armSwingL * 0.3], // 7 left elbow
[px + 0.28 + armSwingR * 0.3, 1.18 + bp + bob - elbowBendR, pz + armSwingR * 0.3], // 8 right elbow
[px - 0.26 + armSwingL * 0.6, 0.92 + bp + bob - elbowBendL * 1.5, pz + armSwingL * 0.5], // 9 left wrist
[px + 0.26 + armSwingR * 0.6, 0.92 + bp + bob - elbowBendR * 1.5, pz + armSwingR * 0.5], // 10 right wrist
[px - 0.11 + torsoTwist * 0.5, 0.98 + bp + bob, pz], // 11 left hip (counter-twist)
[px + 0.11 + torsoTwist * 0.5, 0.98 + bp + bob, pz], // 12 right hip
[px - 0.12 + legStride * 0.3, 0.52 + kneeAmt, pz + legStride], // 13 left knee
[px + 0.12 + legBack * 0.3, 0.52 + kneeAmt, pz + legBack], // 14 right knee
[px - 0.12 + legStride * 0.6, 0.04, pz + legStride * 1.5], // 15 left ankle
[px + 0.12 + legBack * 0.6, 0.04, pz + legBack * 1.5], // 16 right ankle
];
}
// ---- Lying -------------------------------------------------------------
// Subtle micro-movements, differentiate supine vs side-lying via elapsed hash
poseLying(px, surfaceY, pz, elapsed, bp) {
const y = (surfaceY || 0) + 0.2;
const chest = bp * 0.015;
// Micro-movements -- tiny random-feeling shifts (deterministic from elapsed)
const microX = Math.sin(elapsed * 0.17) * 0.004;
const microZ = Math.cos(elapsed * 0.13) * 0.003;
const fingerTwitch = Math.sin(elapsed * 0.7) * 0.008;
// Determine supine vs side-lying from a slow oscillation (stays one way for ~20s)
const lyingMode = Math.sin(elapsed * 0.05);
if (lyingMode > 0.3) {
// Side-lying (on left side)
const curl = Math.sin(elapsed * 0.1) * 0.02; // slight fetal curl
return [
[px - 0.72 + microX, y + 0.12, pz - 0.08], // 0 nose (turned)
[px - 0.70, y + 0.14, pz - 0.10], // 1 left eye
[px - 0.70, y + 0.16, pz - 0.06], // 2 right eye (up)
[px - 0.76, y + 0.11, pz - 0.12], // 3 left ear (down)
[px - 0.76, y + 0.14, pz - 0.04], // 4 right ear
[px - 0.45, y + chest + 0.05, pz - 0.12], // 5 left shoulder (down)
[px - 0.45, y + chest + 0.2, pz + 0.04], // 6 right shoulder (up)
[px - 0.38, y + 0.02, pz - 0.28 + curl], // 7 left elbow
[px - 0.35, y + 0.18, pz + 0.15 + fingerTwitch], // 8 right elbow
[px - 0.20, y - 0.01, pz - 0.30 + curl], // 9 left wrist
[px - 0.18, y + 0.12, pz + 0.25 + fingerTwitch], // 10 right wrist
[px + 0.05 + microX, y + chest * 0.4 + 0.03, pz - 0.08], // 11 left hip
[px + 0.05 + microX, y + chest * 0.4 + 0.12, pz + 0.06], // 12 right hip
[px + 0.40 + curl * 2, y + 0.02, pz - 0.14 + curl], // 13 left knee
[px + 0.38 + curl * 2, y + 0.10, pz + 0.10 + curl], // 14 right knee
[px + 0.75, y - 0.01, pz - 0.12], // 15 left ankle
[px + 0.72, y + 0.04, pz + 0.08], // 16 right ankle
];
}
// Supine (face up) -- default
return [
[px - 0.75 + microX, y + 0.08, pz + microZ], // 0 nose
[px - 0.72, y + 0.1, pz - 0.02 + microZ], // 1 left eye
[px - 0.72, y + 0.1, pz + 0.02 + microZ], // 2 right eye
[px - 0.78, y + 0.08, pz - 0.05], // 3 left ear
[px - 0.78, y + 0.08, pz + 0.05], // 4 right ear
[px - 0.45, y + chest, pz - 0.18], // 5 left shoulder
[px - 0.45, y + chest, pz + 0.18], // 6 right shoulder
[px - 0.42, y, pz - 0.35 + fingerTwitch], // 7 left elbow
[px - 0.42, y, pz + 0.35 - fingerTwitch], // 8 right elbow
[px - 0.2, y - 0.02, pz - 0.38 + fingerTwitch], // 9 left wrist
[px - 0.2, y - 0.02, pz + 0.38 - fingerTwitch], // 10 right wrist
[px + 0.05 + microX, y + chest * 0.5, pz - 0.1], // 11 left hip
[px + 0.05 + microX, y + chest * 0.5, pz + 0.1], // 12 right hip
[px + 0.45, y, pz - 0.11], // 13 left knee
[px + 0.45, y, pz + 0.11], // 14 right knee
[px + 0.82, y - 0.02, pz - 0.1], // 15 left ankle
[px + 0.82, y - 0.02, pz + 0.1], // 16 right ankle
];
}
// ---- Sitting -----------------------------------------------------------
// Occasional fidget, breathing chest expansion, weight shift
poseSitting(px, pz, elapsed, bp) {
const sway = Math.sin(elapsed * 0.5) * 0.003;
// Fidget: occasional hand movement (every ~6s a small gesture)
const fidgetCycle = elapsed % 6.0;
const fidgetActive = fidgetCycle > 5.2 && fidgetCycle < 5.8;
const fidgetAmt = fidgetActive ? Math.sin((fidgetCycle - 5.2) * Math.PI / 0.6) * 0.06 : 0;
// Weight shift side to side (slow)
const weightShift = Math.sin(elapsed * 0.25) * 0.008;
// Chest expansion from breathing
const chestExpand = bp * 0.008;
return [
[px + sway + weightShift, 1.15 + bp, pz], // 0 nose
[px - 0.03 + sway + weightShift, 1.17 + bp, pz - 0.02], // 1 left eye
[px + 0.03 + sway + weightShift, 1.17 + bp, pz - 0.02], // 2 right eye
[px - 0.07 + weightShift, 1.15 + bp, pz], // 3 left ear
[px + 0.07 + weightShift, 1.15 + bp, pz], // 4 right ear
[px - 0.20 - chestExpand + weightShift, 0.95 + bp, pz], // 5 left shoulder
[px + 0.20 + chestExpand + weightShift, 0.95 + bp, pz], // 6 right shoulder
[px - 0.25 + weightShift, 0.72 + bp, pz + 0.08], // 7 left elbow
[px + 0.25 + weightShift, 0.72 + bp, pz + 0.08], // 8 right elbow
[px - 0.18 + fidgetAmt, 0.55 + fidgetAmt * 0.3, pz + 0.15], // 9 left wrist (fidgets)
[px + 0.18, 0.55, pz + 0.15], // 10 right wrist
[px - 0.11 + weightShift * 0.5, 0.48, pz + 0.02], // 11 left hip
[px + 0.11 + weightShift * 0.5, 0.48, pz + 0.02], // 12 right hip
[px - 0.12, 0.48, pz + 0.4], // 13 left knee
[px + 0.12, 0.48, pz + 0.4], // 14 right knee
[px - 0.12, 0.04, pz + 0.4], // 15 left ankle
[px + 0.12, 0.04, pz + 0.4], // 16 right ankle
];
}
// ---- Fallen ------------------------------------------------------------
// Occasional twitch/attempt to move, asymmetric breathing
poseFallen(px, pz, elapsed) {
// Irregular twitch -- sharper, less periodic
const twitchArm = Math.sin(elapsed * 0.3) * 0.003 +
Math.sin(elapsed * 1.7) * 0.008 * Math.max(0, Math.sin(elapsed * 0.15));
const twitchLeg = Math.cos(elapsed * 0.4) * 0.005 *
Math.max(0, Math.sin(elapsed * 0.2 + 1.0));
// Asymmetric breathing (one side of chest rises more)
const breathL = Math.sin(elapsed * 0.8) * 0.006;
const breathR = Math.sin(elapsed * 0.8 + 0.3) * 0.004;
// Attempt to move (slow reach every ~10s)
const attemptCycle = elapsed % 10.0;
const attempting = attemptCycle > 8.0 && attemptCycle < 9.5;
const attemptAmt = attempting ? Math.sin((attemptCycle - 8.0) * Math.PI / 1.5) * 0.05 : 0;
return [
[px + 0.35, 0.12, pz + 0.15 + twitchArm], // 0 nose
[px + 0.33, 0.14, pz + 0.13], // 1 left eye
[px + 0.37, 0.14, pz + 0.17], // 2 right eye
[px + 0.38, 0.11, pz + 0.1], // 3 left ear
[px + 0.38, 0.11, pz + 0.2], // 4 right ear
[px + 0.15, 0.15 + breathL, pz - 0.1], // 5 left shoulder
[px + 0.15, 0.2 + breathR, pz + 0.25], // 6 right shoulder
[px - 0.05, 0.08, pz - 0.25 + twitchArm], // 7 left elbow
[px + 0.3, 0.22 + attemptAmt * 0.5, pz + 0.45 + attemptAmt], // 8 right elbow (reaching)
[px - 0.15, 0.05, pz - 0.3 + twitchArm * 1.5], // 9 left wrist
[px + 0.4, 0.15 + attemptAmt, pz + 0.5 + attemptAmt * 1.5], // 10 right wrist (reaching)
[px - 0.05, 0.12, pz - 0.05], // 11 left hip
[px - 0.05, 0.12, pz + 0.15], // 12 right hip
[px - 0.2, 0.08 + twitchLeg, pz - 0.3], // 13 left knee
[px - 0.15, 0.15, pz + 0.35 + twitchLeg], // 14 right knee
[px - 0.35, 0.04, pz - 0.2], // 15 left ankle
[px - 0.3, 0.04, pz + 0.5], // 16 right ankle
];
}
// ---- Falling -----------------------------------------------------------
// Flailing arms, head snap, non-linear easing (cubic ease-in)
poseFalling(px, pz, elapsed, progress) {
const standing = this.poseStanding(px, pz, elapsed, 0, 0);
const fallen = this.poseFallen(px, pz, elapsed);
// Cubic ease-in for realistic acceleration
const t = progress * progress * progress;
// Arm flailing -- sinusoidal perturbation that peaks mid-fall then diminishes
const flailIntensity = Math.sin(progress * Math.PI) * 0.15;
const flailL = Math.sin(elapsed * 8 + progress * 5) * flailIntensity;
const flailR = Math.cos(elapsed * 8 + progress * 5) * flailIntensity;
// Head snaps back early in the fall
const headSnap = progress < 0.4 ? Math.sin(progress * Math.PI / 0.4) * 0.06 : 0;
const kps = [];
for (let i = 0; i < 17; i++) {
kps.push([
standing[i][0] * (1 - t) + fallen[i][0] * t,
standing[i][1] * (1 - t) + fallen[i][1] * t,
standing[i][2] * (1 - t) + fallen[i][2] * t,
]);
}
// Apply head snap (tilt backward)
kps[0][1] += headSnap;
kps[1][1] += headSnap * 0.9;
kps[2][1] += headSnap * 0.9;
// Apply arm flailing
kps[7][0] += flailL; kps[7][2] += flailL * 0.5; // left elbow
kps[8][0] += flailR; kps[8][2] -= flailR * 0.5; // right elbow
kps[9][0] += flailL * 1.5; kps[9][2] += flailL; // left wrist
kps[10][0] += flailR * 1.5; kps[10][2] -= flailR; // right wrist
return kps;
}
// ---- Exercising --------------------------------------------------------
poseExercising(px, pz, elapsed, exerciseType, exerciseTime) {
const et = exerciseTime || elapsed;
if (exerciseType === 'squats') {
return this._poseSquats(px, pz, et);
}
return this._poseJumpingJacks(px, pz, et);
}
// Squats: forward lean, hip hinge, arm counterbalance, depth variation
_poseSquats(px, pz, et) {
const rawPhase = (Math.sin(et * 2.5) + 1) / 2; // 0=up, 1=down
// Depth variation -- every other rep is shallower
const repIndex = Math.floor(et * 2.5 / Math.PI);
const depthMod = (repIndex % 2 === 0) ? 1.0 : 0.7;
const phase = rawPhase * depthMod;
const squat = phase * 0.5;
const armFwd = phase * 0.4;
// Forward lean increases with squat depth
const forwardLean = phase * 0.08;
// Hip hinge -- hips push back
const hipBack = phase * 0.12;
return [
[px + forwardLean * 0.3, 1.72 - squat, pz + forwardLean], // 0 nose
[px - 0.03 + forwardLean * 0.3, 1.74 - squat, pz - 0.02 + forwardLean], // 1 left eye
[px + 0.03 + forwardLean * 0.3, 1.74 - squat, pz - 0.02 + forwardLean], // 2 right eye
[px - 0.07, 1.72 - squat, pz + forwardLean * 0.8], // 3 left ear
[px + 0.07, 1.72 - squat, pz + forwardLean * 0.8], // 4 right ear
[px - 0.22, 1.48 - squat + forwardLean * 0.2, pz + forwardLean * 0.5], // 5 left shoulder
[px + 0.22, 1.48 - squat + forwardLean * 0.2, pz + forwardLean * 0.5], // 6 right shoulder
[px - 0.22, 1.25 - squat * 0.7, pz + armFwd], // 7 left elbow
[px + 0.22, 1.25 - squat * 0.7, pz + armFwd], // 8 right elbow
[px - 0.22, 1.05 - squat * 0.5, pz + armFwd * 1.5], // 9 left wrist (counterbalance)
[px + 0.22, 1.05 - squat * 0.5, pz + armFwd * 1.5], // 10 right wrist
[px - 0.11, 0.98 - squat * 0.3, pz - hipBack], // 11 left hip (pushed back)
[px + 0.11, 0.98 - squat * 0.3, pz - hipBack], // 12 right hip
[px - 0.15, 0.52 - squat * 0.1, pz + squat * 0.3], // 13 left knee
[px + 0.15, 0.52 - squat * 0.1, pz + squat * 0.3], // 14 right knee
[px - 0.13, 0.04, pz + 0.05], // 15 left ankle
[px + 0.13, 0.04, pz + 0.05], // 16 right ankle
];
}
// Jumping jacks: full arm arc, hip sway, landing impact
_poseJumpingJacks(px, pz, et) {
const rawPhase = (Math.sin(et * 3) + 1) / 2; // 0=closed, 1=open
const phase = rawPhase;
// Full arm arc -- from sides to overhead in a smooth arc
const armAngle = phase * Math.PI * 0.85; // 0 to ~153 degrees
const armX = Math.sin(armAngle) * 0.55; // lateral spread
const armY = Math.cos(armAngle) * 0.55; // vertical component
const legSpread = phase * 0.25;
// Landing impact -- brief compression at bottom of cycle
const impact = Math.max(0, -Math.sin(et * 3)) * 0.03;
const jump = Math.max(0, Math.sin(et * 3)) * 0.06;
// Hip sway at apex
const hipSway = Math.sin(et * 3) * 0.015;
return [
[px, 1.72 + jump - impact, pz], // 0 nose
[px - 0.03, 1.74 + jump - impact, pz - 0.02], // 1 left eye
[px + 0.03, 1.74 + jump - impact, pz - 0.02], // 2 right eye
[px - 0.07, 1.72 + jump - impact, pz], // 3 left ear
[px + 0.07, 1.72 + jump - impact, pz], // 4 right ear
[px - 0.22, 1.48 + jump - impact, pz], // 5 left shoulder
[px + 0.22, 1.48 + jump - impact, pz], // 6 right shoulder
[px - 0.22 - armX * 0.6, 1.48 - armY * 0.3 + jump, pz], // 7 left elbow (arc)
[px + 0.22 + armX * 0.6, 1.48 - armY * 0.3 + jump, pz], // 8 right elbow
[px - 0.22 - armX, 1.48 - armY + 0.55 + jump, pz], // 9 left wrist (arc)
[px + 0.22 + armX, 1.48 - armY + 0.55 + jump, pz], // 10 right wrist
[px - 0.11 + hipSway, 0.98 + jump - impact, pz], // 11 left hip
[px + 0.11 + hipSway, 0.98 + jump - impact, pz], // 12 right hip
[px - 0.12 - legSpread, 0.52 + jump * 0.5 - impact * 0.5, pz], // 13 left knee
[px + 0.12 + legSpread, 0.52 + jump * 0.5 - impact * 0.5, pz], // 14 right knee
[px - 0.13 - legSpread * 1.3, 0.04 - impact * 0.3, pz], // 15 left ankle
[px + 0.13 + legSpread * 1.3, 0.04 - impact * 0.3, pz], // 16 right ankle
];
}
// ---- Gesturing ---------------------------------------------------------
poseGesturing(px, pz, elapsed, gestureType, intensity) {
const base = this.poseStanding(px, pz, elapsed, 0, 0);
if (intensity <= 0) return base;
const gt = elapsed;
switch (gestureType) {
case 'wave':
return this._gestureWave(base, px, pz, gt, intensity);
case 'swipe_left':
return this._gestureSwipe(base, px, pz, gt, intensity);
case 'circle':
return this._gestureCircle(base, px, pz, gt, intensity);
case 'point':
return this._gesturePoint(base, px, pz, gt, intensity);
default:
return base;
}
}
// Wave: fluid hand oscillation, elbow pivot, slight shoulder raise
_gestureWave(base, px, pz, gt, intensity) {
const wave = Math.sin(gt * 6) * 0.15 * intensity;
const waveSmooth = Math.sin(gt * 6 + 0.3) * 0.08 * intensity; // secondary harmonic
const shoulderRaise = 0.04 * intensity;
const elbowPivot = Math.sin(gt * 3) * 0.03 * intensity;
// Shoulder rises slightly during wave
base[6][1] += shoulderRaise;
// Elbow raised and pivoting
base[8] = [
px + 0.32 + elbowPivot,
1.55 * intensity + 1.18 * (1 - intensity) + shoulderRaise,
pz + 0.05,
];
// Wrist oscillates fluidly
base[10] = [
px + 0.32 + wave + waveSmooth * 0.3,
1.7 * intensity + 0.92 * (1 - intensity) + shoulderRaise,
pz + 0.08 + waveSmooth,
];
// Slight body lean away from waving arm
base[0][0] -= 0.01 * intensity;
base[5][0] -= 0.008 * intensity;
return base;
}
// Swipe: full body rotation follow-through, arm extension
_gestureSwipe(base, px, pz, gt, intensity) {
const sweep = Math.sin(gt * 2) * intensity;
// Body rotation follows the arm
const bodyRotation = sweep * 0.04;
const shoulderTwist = sweep * 0.025;
// Upper body rotates
for (let i = 0; i <= 4; i++) base[i][0] += bodyRotation * 0.5;
base[5][0] -= shoulderTwist;
base[6][0] += shoulderTwist;
// Arm extends fully during swipe
base[8] = [px + 0.15 + sweep * 0.4, 1.3, pz + 0.3];
base[10] = [px - 0.1 + sweep * 0.6, 1.3, pz + 0.55];
// Hip counter-rotation
base[11][0] += bodyRotation * -0.2;
base[12][0] += bodyRotation * -0.2;
return base;
}
// Circle: smooth circular motion with forearm rotation
_gestureCircle(base, px, pz, gt, intensity) {
const angle = gt * 2.5;
const radius = 0.25 * intensity;
const cx = Math.cos(angle) * radius;
const cy = Math.sin(angle) * radius;
// Forearm rotation -- wrist traces a smaller secondary circle
const forearmAngle = angle * 1.5;
const forearmR = 0.06 * intensity;
base[8] = [
px + 0.3 + cx * 0.5,
1.3 + cy * 0.5,
pz + 0.2 + Math.sin(angle) * 0.05,
];
base[10] = [
px + 0.3 + cx + Math.cos(forearmAngle) * forearmR,
1.3 + cy + Math.sin(forearmAngle) * forearmR,
pz + 0.35 + Math.sin(angle) * 0.08,
];
// Slight shoulder movement following arm
base[6][0] += cx * 0.08;
base[6][1] += cy * 0.04;
return base;
}
// Point: extended index finger simulation with arm sway
_gesturePoint(base, px, pz, gt, intensity) {
const point = intensity;
// Slight arm sway -- breathing/holding still
const sway = Math.sin(gt * 1.5) * 0.01 * intensity;
const vertSway = Math.cos(gt * 1.2) * 0.008 * intensity;
base[8] = [px + 0.15 + sway, 1.35 + vertSway, pz + 0.35 * point];
base[10] = [px + 0.08 + sway * 0.5, 1.38 + vertSway * 0.5, pz + 0.70 * point];
// Lean slightly toward point direction
base[0][2] += 0.02 * point;
base[5][2] += 0.01 * point;
base[6][2] += 0.01 * point;
return base;
}
// ---- Crouching ---------------------------------------------------------
// Stealth-crawl option, weight transfer between legs
poseCrouching(px, pz, elapsed, bp) {
const sway = Math.sin(elapsed * 1.5) * 0.005;
// Weight transfer between legs (slow rocking)
const weightTransfer = Math.sin(elapsed * 0.8) * 0.025;
const leftDown = Math.max(0, weightTransfer) * 0.03;
const rightDown = Math.max(0, -weightTransfer) * 0.03;
// Stealth-crawl micro-movement (slow forward creep every ~4s)
const crawlCycle = elapsed % 4.0;
const crawlActive = crawlCycle > 3.0;
const crawlAmt = crawlActive ? Math.sin((crawlCycle - 3.0) * Math.PI) * 0.02 : 0;
// Arms adjust for balance during weight transfer
const armBalance = weightTransfer * 0.3;
return [
[px + sway, 1.05 + bp, pz + 0.15 + crawlAmt], // 0 nose
[px - 0.03, 1.07 + bp, pz + 0.13 + crawlAmt], // 1 left eye
[px + 0.03, 1.07 + bp, pz + 0.13 + crawlAmt], // 2 right eye
[px - 0.07, 1.05 + bp, pz + 0.12 + crawlAmt], // 3 left ear
[px + 0.07, 1.05 + bp, pz + 0.12 + crawlAmt], // 4 right ear
[px - 0.22, 0.88 + bp, pz + 0.05], // 5 left shoulder
[px + 0.22, 0.88 + bp, pz + 0.05], // 6 right shoulder
[px - 0.28 - armBalance, 0.65 + bp, pz + 0.15 + crawlAmt * 0.5], // 7 left elbow
[px + 0.28 + armBalance, 0.65 + bp, pz + 0.15 + crawlAmt * 0.5], // 8 right elbow
[px - 0.22 - armBalance * 0.5, 0.48, pz + 0.2 + crawlAmt], // 9 left wrist
[px + 0.22 + armBalance * 0.5, 0.48, pz + 0.2 + crawlAmt], // 10 right wrist
[px - 0.12 + weightTransfer, 0.42, pz - 0.05], // 11 left hip
[px + 0.12 + weightTransfer, 0.42, pz - 0.05], // 12 right hip
[px - 0.15 + weightTransfer * 0.5, 0.35 - leftDown, pz + 0.25], // 13 left knee
[px + 0.15 + weightTransfer * 0.5, 0.35 - rightDown, pz + 0.25], // 14 right knee
[px - 0.13, 0.04, pz + 0.1], // 15 left ankle
[px + 0.13, 0.04, pz + 0.1], // 16 right ankle
];
}
}
+125
View File
@@ -0,0 +1,125 @@
/**
* Post-Processing Subtle bloom for green glow wireframe,
* warm vignette, minimal grain. Foundation-style.
*/
import * as THREE from 'three';
import { EffectComposer } from 'three/addons/postprocessing/EffectComposer.js';
import { RenderPass } from 'three/addons/postprocessing/RenderPass.js';
import { UnrealBloomPass } from 'three/addons/postprocessing/UnrealBloomPass.js';
import { ShaderPass } from 'three/addons/postprocessing/ShaderPass.js';
const VignetteShader = {
uniforms: {
tDiffuse: { value: null },
uTime: { value: 0 },
uVignetteStrength: { value: 0.5 },
uChromaticStrength: { value: 0.0015 },
uGrainStrength: { value: 0.03 },
uWarmth: { value: 0.08 },
},
vertexShader: `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`,
fragmentShader: `
uniform sampler2D tDiffuse;
uniform float uTime;
uniform float uVignetteStrength;
uniform float uChromaticStrength;
uniform float uGrainStrength;
uniform float uWarmth;
varying vec2 vUv;
float rand(vec2 co) {
return fract(sin(dot(co, vec2(12.9898, 78.233))) * 43758.5453);
}
void main() {
vec2 uv = vUv;
vec2 center = uv - 0.5;
float dist = length(center);
// Subtle chromatic aberration at edges only
vec2 offset = center * dist * uChromaticStrength;
float r = texture2D(tDiffuse, uv + offset).r;
float g = texture2D(tDiffuse, uv).g;
float b = texture2D(tDiffuse, uv - offset * 0.5).b;
vec3 color = vec3(r, g, b);
// Warm vignette
float vignette = 1.0 - dist * dist * uVignetteStrength * 1.8;
color *= vignette;
// Very subtle warm shift in shadows
float luma = dot(color, vec3(0.299, 0.587, 0.114));
color.r += (1.0 - luma) * uWarmth * 0.5;
color.g += (1.0 - luma) * uWarmth * 0.2;
// Minimal grain
float grain = (rand(uv * uTime * 0.01) - 0.5) * uGrainStrength;
color += grain;
gl_FragColor = vec4(color, 1.0);
}
`,
};
export class PostProcessing {
constructor(renderer, scene, camera) {
const size = renderer.getSize(new THREE.Vector2());
this.composer = new EffectComposer(renderer);
this.composer.addPass(new RenderPass(scene, camera));
// Bloom — tuned for green wireframe glow
this._bloomPass = new UnrealBloomPass(
new THREE.Vector2(size.x, size.y),
0.08, // strength — subtle glow, overridden by settings
0.2, // radius
0.6 // threshold
);
this.composer.addPass(this._bloomPass);
// Vignette + warmth
this._vignettePass = new ShaderPass(VignetteShader);
this.composer.addPass(this._vignettePass);
this._bloomEnabled = true;
}
update(elapsed) {
this._vignettePass.uniforms.uTime.value = elapsed;
}
render() {
this.composer.render();
}
resize(width, height) {
this.composer.setSize(width, height);
this._bloomPass.resolution.set(width, height);
}
setQuality(level) {
if (level === 0) {
this._bloomPass.strength = 0;
this._vignettePass.uniforms.uChromaticStrength.value = 0;
this._vignettePass.uniforms.uGrainStrength.value = 0;
} else if (level === 1) {
this._bloomPass.strength = 0.6;
this._vignettePass.uniforms.uChromaticStrength.value = 0.001;
this._vignettePass.uniforms.uGrainStrength.value = 0.02;
} else {
this._bloomPass.strength = 1.0;
this._vignettePass.uniforms.uChromaticStrength.value = 0.0015;
this._vignettePass.uniforms.uGrainStrength.value = 0.03;
}
}
dispose() {
this.composer.dispose();
}
}
+178
View File
@@ -0,0 +1,178 @@
/**
* Module C "Presence Cartography"
* InstancedMesh 20x4x20 voxel heatmap with person lights
*/
import * as THREE from 'three';
const GRID_X = 20;
const GRID_Y = 4;
const GRID_Z = 20;
const TOTAL_VOXELS = GRID_X * GRID_Y * GRID_Z;
const VOXEL_SIZE = 0.22;
export class PresenceCartography {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Instanced cubes
const cubeGeo = new THREE.BoxGeometry(VOXEL_SIZE, VOXEL_SIZE, VOXEL_SIZE);
const cubeMat = new THREE.MeshBasicMaterial({
color: 0xffffff,
transparent: true,
opacity: 1,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._mesh = new THREE.InstancedMesh(cubeGeo, cubeMat, TOTAL_VOXELS);
this._mesh.instanceMatrix.setUsage(THREE.DynamicDrawUsage);
// Color attribute
this._colors = new Float32Array(TOTAL_VOXELS * 3);
this._mesh.instanceColor = new THREE.InstancedBufferAttribute(this._colors, 3);
// Initialize positions
const dummy = new THREE.Object3D();
const halfX = (GRID_X * VOXEL_SIZE * 1.1) / 2;
const halfZ = (GRID_Z * VOXEL_SIZE * 1.1) / 2;
for (let y = 0; y < GRID_Y; y++) {
for (let z = 0; z < GRID_Z; z++) {
for (let x = 0; x < GRID_X; x++) {
const idx = y * GRID_Z * GRID_X + z * GRID_X + x;
dummy.position.set(
x * VOXEL_SIZE * 1.1 - halfX,
y * VOXEL_SIZE * 1.1,
z * VOXEL_SIZE * 1.1 - halfZ
);
dummy.scale.set(0.01, 0.01, 0.01); // start invisible
dummy.updateMatrix();
this._mesh.setMatrixAt(idx, dummy.matrix);
this._colors[idx * 3] = 0;
this._colors[idx * 3 + 1] = 0.2;
this._colors[idx * 3 + 2] = 0.4;
}
}
}
this._mesh.instanceMatrix.needsUpdate = true;
this._mesh.instanceColor.needsUpdate = true;
this.group.add(this._mesh);
// Room wireframe
const roomW = GRID_X * VOXEL_SIZE * 1.1;
const roomH = GRID_Y * VOXEL_SIZE * 1.1;
const roomD = GRID_Z * VOXEL_SIZE * 1.1;
const boxGeo = new THREE.BoxGeometry(roomW, roomH, roomD);
const edges = new THREE.EdgesGeometry(boxGeo);
const lineMat = new THREE.LineBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.15,
});
const wireframe = new THREE.LineSegments(edges, lineMat);
wireframe.position.y = roomH / 2;
this.group.add(wireframe);
// Person lights (up to 4)
this._personLights = [];
for (let i = 0; i < 4; i++) {
const light = new THREE.PointLight(0xff8800, 0, 3);
this.group.add(light);
this._personLights.push(light);
}
this._dummy = new THREE.Object3D();
this._halfX = halfX;
this._halfZ = halfZ;
}
update(dt, elapsed, data) {
const field = data?.signal_field?.values;
const persons = data?.persons || [];
const dummy = this._dummy;
if (field && field.length >= GRID_X * GRID_Z) {
for (let y = 0; y < GRID_Y; y++) {
for (let z = 0; z < GRID_Z; z++) {
for (let x = 0; x < GRID_X; x++) {
const idx = y * GRID_Z * GRID_X + z * GRID_X + x;
const fieldIdx = z * GRID_X + x;
const val = field[fieldIdx] || 0;
// Extrude vertically: layer 0 = full val, higher layers diminish
const layerFactor = Math.max(0, 1 - y / GRID_Y);
const v = val * layerFactor;
// Scale voxel by value
const s = v > 0.05 ? 0.3 + v * 0.7 : 0.01;
dummy.position.set(
x * VOXEL_SIZE * 1.1 - this._halfX,
y * VOXEL_SIZE * 1.1,
z * VOXEL_SIZE * 1.1 - this._halfZ
);
dummy.scale.set(s, s, s);
dummy.updateMatrix();
this._mesh.setMatrixAt(idx, dummy.matrix);
// Color: blue(low) -> cyan(mid) -> amber(high)
let r, g, b;
if (v < 0.3) {
const t = v / 0.3;
r = 0.02;
g = 0.06 + t * 0.6;
b = 0.2 + t * 0.6;
} else if (v < 0.6) {
const t = (v - 0.3) / 0.3;
r = t * 0.8;
g = 0.66 + t * 0.2;
b = 0.8 - t * 0.5;
} else {
const t = (v - 0.6) / 0.4;
r = 0.8 + t * 0.2;
g = 0.86 - t * 0.5;
b = 0.3 - t * 0.3;
}
this._colors[idx * 3] = r;
this._colors[idx * 3 + 1] = g;
this._colors[idx * 3 + 2] = b;
}
}
}
this._mesh.instanceMatrix.needsUpdate = true;
this._mesh.instanceColor.needsUpdate = true;
}
// Person lights
for (let i = 0; i < this._personLights.length; i++) {
const light = this._personLights[i];
if (i < persons.length) {
const p = persons[i].position || [0, 0, 0];
light.position.set(p[0] * 2, 1.5, p[2] * 2);
light.intensity = 1.5 + Math.sin(elapsed * 3 + i) * 0.5;
light.color.setHex(0xff8800);
} else {
light.intensity = 0;
}
}
}
/** Reduce voxel count for performance */
setQuality(level) {
// For now just toggle visibility of upper layers
// level 0 = show only ground, 2 = show all
this._mesh.count = level === 0
? GRID_X * GRID_Z
: level === 1
? GRID_X * GRID_Z * 2
: TOTAL_VOXELS;
}
dispose() {
this._mesh.geometry.dispose();
this._mesh.material.dispose();
}
}
+739
View File
@@ -0,0 +1,739 @@
/**
* ScenarioProps Scenario-specific room furniture and props
*
* Extracted from main.js. Builds and manages visibility of all physical
* objects that appear/disappear based on the active scenario: bed, chair,
* exercise mat, door, rubble wall, screen/TV, desks, security cameras,
* and the alert light system.
*/
import * as THREE from 'three';
// Scenario-to-prop-name mapping
const SCENARIO_PROPS = {
empty_room: [],
single_breathing: [],
two_walking: [],
fall_event: [],
sleep_monitoring: ['bed'],
intrusion_detect: ['door'],
gesture_control: ['screen'],
crowd_occupancy: ['desk', 'desk2'],
search_rescue: ['rubbleWall'],
elderly_care: ['chair'],
fitness_tracking: ['exerciseMat'],
security_patrol: ['camera1', 'camera2'],
};
export class ScenarioProps {
constructor(scene) {
this._scene = scene;
this._props = {};
this._currentScenario = null;
this._alertLight = null;
this._alertIntensity = 0;
// Animatable references
this._screenGlow = null;
this._camera1Group = null;
this._camera2Group = null;
this._cam1Cone = null;
this._cam2Cone = null;
this._cam1Led = null;
this._cam2Led = null;
this._dustParticles = null;
this._doorSpotlight = null;
this._alarmHousing = null;
this._powerLed = null;
this._build();
}
// ---- helper: positioned box with shadow ----
_box(x, y, z, w, h, d, mat) {
const m = new THREE.Mesh(new THREE.BoxGeometry(w, h, d), mat);
m.position.set(x, y, z);
m.castShadow = true;
m.receiveShadow = true;
return m;
}
// ---- helper: positioned cylinder with shadow ----
_cyl(x, y, z, rTop, rBot, h, segs, mat) {
const m = new THREE.Mesh(new THREE.CylinderGeometry(rTop, rBot, h, segs), mat);
m.position.set(x, y, z);
m.castShadow = true;
m.receiveShadow = true;
return m;
}
// ========================================
// BUILD ALL PROPS
// ========================================
_build() {
const darkMat = new THREE.MeshStandardMaterial({ color: 0x6b5840, roughness: 0.6, emissive: 0x1a1408, emissiveIntensity: 0.25 });
const metalMat = new THREE.MeshStandardMaterial({ color: 0x808088, roughness: 0.3, metalness: 0.7, emissive: 0x1a1a20, emissiveIntensity: 0.2 });
const accentMat = new THREE.MeshStandardMaterial({ color: 0x606070, roughness: 0.4, metalness: 0.4, emissive: 0x101018, emissiveIntensity: 0.15 });
this._buildBed(darkMat);
this._buildChair(darkMat, accentMat);
this._buildExerciseMat();
this._buildDoor();
this._buildRubbleWall();
this._buildScreen(metalMat);
this._buildDesks(darkMat, metalMat, accentMat);
this._buildCameras(metalMat);
this._buildAlertSystem();
}
// ---- BED (sleep monitoring) ----
_buildBed(darkMat) {
const bedGroup = new THREE.Group();
// Bed frame with legs
const frameMat = new THREE.MeshStandardMaterial({ color: 0x7a6448, roughness: 0.55, metalness: 0.25, emissive: 0x181008, emissiveIntensity: 0.25 });
const bedFrame = new THREE.Mesh(new THREE.BoxGeometry(2.2, 0.12, 1.2), frameMat);
bedFrame.position.set(3.5, 0.32, -3.5);
bedFrame.castShadow = true;
bedGroup.add(bedFrame);
// Frame legs (4 short posts)
for (const [lx, lz] of [[2.5, -4.0], [4.5, -4.0], [2.5, -3.0], [4.5, -3.0]]) {
bedGroup.add(this._cyl(lx, 0.13, lz, 0.04, 0.04, 0.26, 6, frameMat));
}
// Headboard — tall panel at head of bed
const headboardMat = new THREE.MeshStandardMaterial({ color: 0x6a5440, roughness: 0.65, emissive: 0x140e08, emissiveIntensity: 0.2 });
const headboard = new THREE.Mesh(new THREE.BoxGeometry(0.08, 0.7, 1.2), headboardMat);
headboard.position.set(2.38, 0.65, -3.5);
headboard.castShadow = true;
bedGroup.add(headboard);
// Mattress
const mattressMat = new THREE.MeshStandardMaterial({ color: 0x484860, roughness: 0.75, emissive: 0x0c0c1a, emissiveIntensity: 0.2 });
const mattress = new THREE.Mesh(new THREE.BoxGeometry(2.0, 0.15, 1.1), mattressMat);
mattress.position.set(3.5, 0.455, -3.5);
mattress.castShadow = true;
bedGroup.add(mattress);
// Wrinkled sheet — wave-displaced plane
const sheetGeo = new THREE.PlaneGeometry(1.4, 1.0, 20, 20);
const posAttr = sheetGeo.getAttribute('position');
for (let i = 0; i < posAttr.count; i++) {
const px = posAttr.getX(i);
const py = posAttr.getY(i);
posAttr.setZ(i, Math.sin(px * 4) * 0.015 + Math.cos(py * 5) * 0.01 + Math.sin(px * py * 3) * 0.008);
}
posAttr.needsUpdate = true;
sheetGeo.computeVertexNormals();
const sheetMat = new THREE.MeshStandardMaterial({
color: 0x506880, roughness: 0.75, side: THREE.DoubleSide, emissive: 0x0c1018, emissiveIntensity: 0.2,
});
const sheet = new THREE.Mesh(sheetGeo, sheetMat);
sheet.rotation.x = -Math.PI / 2;
sheet.position.set(3.7, 0.54, -3.5);
sheet.castShadow = true;
bedGroup.add(sheet);
// Pillow — soft shape using scaled sphere
const pillowGeo = new THREE.SphereGeometry(0.18, 12, 8);
pillowGeo.scale(1, 0.35, 1.4);
const pillowMat = new THREE.MeshStandardMaterial({ color: 0x706868, roughness: 0.7, emissive: 0x141010, emissiveIntensity: 0.2 });
const pillow = new THREE.Mesh(pillowGeo, pillowMat);
pillow.position.set(2.65, 0.52, -3.5);
pillow.castShadow = true;
bedGroup.add(pillow);
// Bedside lamp — small cylinder + sphere shade on a tiny table
const lampBaseMat = new THREE.MeshStandardMaterial({ color: 0x686870, roughness: 0.3, metalness: 0.7, emissive: 0x101018, emissiveIntensity: 0.15 });
// Nightstand
bedGroup.add(this._box(2.15, 0.25, -3.5, 0.35, 0.5, 0.35, darkMat));
// Lamp base
bedGroup.add(this._cyl(2.15, 0.55, -3.5, 0.04, 0.05, 0.1, 8, lampBaseMat));
// Lamp stem
bedGroup.add(this._cyl(2.15, 0.68, -3.5, 0.015, 0.015, 0.2, 6, lampBaseMat));
// Lamp shade (emissive warm glow)
const shadeMat = new THREE.MeshStandardMaterial({
color: 0x705830, emissive: 0x604018, emissiveIntensity: 1.0, roughness: 0.6,
side: THREE.DoubleSide, transparent: true, opacity: 0.9,
});
const shade = new THREE.Mesh(new THREE.ConeGeometry(0.08, 0.1, 8, 1, true), shadeMat);
shade.position.set(2.15, 0.78, -3.5);
shade.rotation.x = Math.PI;
bedGroup.add(shade);
// Warm lamp light
const lampLight = new THREE.PointLight(0xffcc88, 2.0, 6, 1.2);
lampLight.position.set(2.15, 0.78, -3.5);
bedGroup.add(lampLight);
this._props.bed = bedGroup;
bedGroup.visible = false;
this._scene.add(bedGroup);
}
// ---- CHAIR (elderly care) ----
_buildChair(darkMat, accentMat) {
const chairGroup = new THREE.Group();
chairGroup.position.set(1, 0, -1.5);
const cushionMat = new THREE.MeshStandardMaterial({ color: 0x5a5078, roughness: 0.7, emissive: 0x10101a, emissiveIntensity: 0.2 });
// Seat
chairGroup.add(this._box(0, 0.45, 0, 0.5, 0.04, 0.45, darkMat));
// Seat cushion — slightly puffy
const cushionGeo = new THREE.BoxGeometry(0.46, 0.06, 0.42);
// Gentle puff on top vertices
const cPos = cushionGeo.getAttribute('position');
for (let i = 0; i < cPos.count; i++) {
if (cPos.getY(i) > 0) {
const dx = cPos.getX(i) / 0.23;
const dz = cPos.getZ(i) / 0.21;
cPos.setY(i, cPos.getY(i) + 0.015 * (1 - dx * dx) * (1 - dz * dz));
}
}
cPos.needsUpdate = true;
cushionGeo.computeVertexNormals();
const cushion = new THREE.Mesh(cushionGeo, cushionMat);
cushion.position.set(0, 0.50, 0);
cushion.castShadow = true;
chairGroup.add(cushion);
// Back
chairGroup.add(this._box(0, 0.72, -0.22, 0.5, 0.5, 0.04, darkMat));
// Legs
for (const [lx, lz] of [[-0.22, -0.2], [0.22, -0.2], [-0.22, 0.2], [0.22, 0.2]]) {
chairGroup.add(this._box(lx, 0.22, lz, 0.04, 0.44, 0.04, darkMat));
}
// Armrests
chairGroup.add(this._box(-0.28, 0.6, 0, 0.04, 0.04, 0.4, accentMat));
chairGroup.add(this._box(0.28, 0.6, 0, 0.04, 0.04, 0.4, accentMat));
// Armrest supports
chairGroup.add(this._box(-0.28, 0.52, -0.18, 0.04, 0.12, 0.04, accentMat));
chairGroup.add(this._box(0.28, 0.52, -0.18, 0.04, 0.12, 0.04, accentMat));
// Small side table
const tableMat = new THREE.MeshStandardMaterial({ color: 0x685840, roughness: 0.55, emissive: 0x14100a, emissiveIntensity: 0.2 });
chairGroup.add(this._box(0.65, 0.3, 0, 0.35, 0.03, 0.35, tableMat));
// Table legs
for (const [tx, tz] of [[0.5, -0.14], [0.8, -0.14], [0.5, 0.14], [0.8, 0.14]]) {
chairGroup.add(this._cyl(tx, 0.15, tz, 0.015, 0.015, 0.28, 6, tableMat));
}
this._props.chair = chairGroup;
chairGroup.visible = false;
this._scene.add(chairGroup);
}
// ---- EXERCISE MAT (fitness tracking) ----
_buildExerciseMat() {
const matGroup = new THREE.Group();
const matMat = new THREE.MeshStandardMaterial({ color: 0x408858, roughness: 0.75, emissive: 0x0c2010, emissiveIntensity: 0.25 });
// Mat body
const exerciseMat = new THREE.Mesh(new THREE.BoxGeometry(1.8, 0.015, 0.8), matMat);
exerciseMat.position.set(0, 0.008, 0);
exerciseMat.receiveShadow = true;
matGroup.add(exerciseMat);
// Boundary lines on the mat (thin strips)
const lineMat = new THREE.MeshStandardMaterial({ color: 0x50a068, roughness: 0.7, emissive: 0x102818, emissiveIntensity: 0.3 });
// Longitudinal borders
matGroup.add(this._box(0, 0.017, -0.37, 1.7, 0.003, 0.02, lineMat));
matGroup.add(this._box(0, 0.017, 0.37, 1.7, 0.003, 0.02, lineMat));
// Cross lines (exercise area markers)
for (const xOff of [-0.6, 0, 0.6]) {
matGroup.add(this._box(xOff, 0.017, 0, 0.02, 0.003, 0.74, lineMat));
}
// Water bottle (cylinder body + hemisphere cap)
const bottleMat = new THREE.MeshStandardMaterial({ color: 0x4878a8, roughness: 0.2, metalness: 0.7, emissive: 0x0c1828, emissiveIntensity: 0.25 });
const bottleBody = new THREE.Mesh(new THREE.CylinderGeometry(0.035, 0.035, 0.18, 10), bottleMat);
bottleBody.position.set(1.1, 0.09, 0.25);
bottleBody.castShadow = true;
matGroup.add(bottleBody);
const bottleCap = new THREE.Mesh(new THREE.SphereGeometry(0.035, 8, 6, 0, Math.PI * 2, 0, Math.PI / 2), bottleMat);
bottleCap.position.set(1.1, 0.18, 0.25);
matGroup.add(bottleCap);
// Bottle neck
const neckMat = new THREE.MeshStandardMaterial({ color: 0x587088, roughness: 0.3, metalness: 0.6, emissive: 0x0c1420, emissiveIntensity: 0.2 });
matGroup.add(this._cyl(1.1, 0.21, 0.25, 0.018, 0.025, 0.04, 8, neckMat));
// Small towel (flat draped box)
const towelMat = new THREE.MeshStandardMaterial({ color: 0x686890, roughness: 0.75, emissive: 0x101020, emissiveIntensity: 0.2 });
const towel = this._box(1.1, 0.01, -0.25, 0.3, 0.008, 0.15, towelMat);
towel.rotation.y = 0.15;
matGroup.add(towel);
this._props.exerciseMat = matGroup;
matGroup.visible = false;
this._scene.add(matGroup);
}
// ---- DOOR (intrusion detection) ----
_buildDoor() {
const doorGroup = new THREE.Group();
doorGroup.position.set(-5.5, 0, -1);
const doorMat = new THREE.MeshStandardMaterial({ color: 0x7a6040, roughness: 0.5, emissive: 0x18140a, emissiveIntensity: 0.25 });
const hingeMat = new THREE.MeshStandardMaterial({ color: 0x909098, roughness: 0.2, metalness: 0.85, emissive: 0x181820, emissiveIntensity: 0.15 });
// Left jamb
doorGroup.add(this._box(-0.45, 1.1, 0, 0.08, 2.2, 0.15, doorMat));
// Right jamb
doorGroup.add(this._box(0.45, 1.1, 0, 0.08, 2.2, 0.15, doorMat));
// Top
doorGroup.add(this._box(0, 2.2, 0, 0.98, 0.08, 0.15, doorMat));
// Door panel (partially open)
const doorPanel = new THREE.Mesh(new THREE.BoxGeometry(0.85, 2.1, 0.04), doorMat);
doorPanel.position.set(0.2, 1.05, -0.2);
doorPanel.rotation.y = -0.7;
doorPanel.castShadow = true;
doorGroup.add(doorPanel);
// Door handle (torus)
const handleMat = new THREE.MeshStandardMaterial({ color: 0xaaaaB0, roughness: 0.1, metalness: 0.9, emissive: 0x1a1a20, emissiveIntensity: 0.2 });
const handle = new THREE.Mesh(new THREE.TorusGeometry(0.035, 0.008, 6, 12), handleMat);
// Position on the door panel (relative to panel pivot)
handle.position.set(0.48, 1.05, -0.22);
handle.rotation.y = -0.7;
handle.rotation.x = Math.PI / 2;
doorGroup.add(handle);
// Hinge details — small cylinders at jamb
for (const hy of [0.4, 1.1, 1.8]) {
const hinge = new THREE.Mesh(new THREE.CylinderGeometry(0.015, 0.015, 0.06, 6), hingeMat);
hinge.position.set(-0.42, hy, 0.06);
doorGroup.add(hinge);
}
// Light spill through the gap — spotlight from outside
const doorSpot = new THREE.SpotLight(0x88aacc, 3.0, 10, Math.PI / 4, 0.3, 0.6);
doorSpot.position.set(-0.8, 1.2, -0.5);
doorSpot.target.position.set(0.5, 0, 0.5);
doorGroup.add(doorSpot);
doorGroup.add(doorSpot.target);
this._doorSpotlight = doorSpot;
// Window next to door — simple frame with translucent pane
const windowFrame = new THREE.MeshStandardMaterial({ color: 0x686878, roughness: 0.35, metalness: 0.6, emissive: 0x101018, emissiveIntensity: 0.15 });
// Frame
doorGroup.add(this._box(1.2, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.5, 0, 0.6, 0.04, 0.06, windowFrame));
doorGroup.add(this._box(0.92, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.48, 1.5, 0, 0.04, 0.8, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.1, 0, 0.6, 0.04, 0.06, windowFrame));
doorGroup.add(this._box(1.2, 1.9, 0, 0.6, 0.04, 0.06, windowFrame));
// Glass pane
const glassMat = new THREE.MeshStandardMaterial({
color: 0x305880, transparent: true, opacity: 0.4, roughness: 0.05, metalness: 0.3, emissive: 0x0c1830, emissiveIntensity: 0.35,
});
const glass = new THREE.Mesh(new THREE.BoxGeometry(0.52, 0.72, 0.01), glassMat);
glass.position.set(1.2, 1.5, 0);
doorGroup.add(glass);
this._props.door = doorGroup;
doorGroup.visible = false;
this._scene.add(doorGroup);
}
// ---- RUBBLE WALL (search & rescue) ----
_buildRubbleWall() {
const rubbleGroup = new THREE.Group();
const rubbleMat = new THREE.MeshStandardMaterial({ color: 0x807868, roughness: 0.75, emissive: 0x181610, emissiveIntensity: 0.25 });
const rebarMat = new THREE.MeshStandardMaterial({ color: 0x8a7858, roughness: 0.4, metalness: 0.7, emissive: 0x1a1408, emissiveIntensity: 0.2 });
// Broken wall — main slab
rubbleGroup.add(this._box(2, 1, 0, 0.4, 2, 3, rubbleMat));
// Wall crack lines (thin dark boxes embedded in wall surface)
const crackMat = new THREE.MeshStandardMaterial({ color: 0x403828, roughness: 0.9 });
const cracks = [
[1.82, 1.4, -0.3, 0.01, 0.6, 0.02, 0.3],
[1.82, 0.8, 0.5, 0.01, 0.5, 0.02, -0.2],
[1.82, 1.6, 0.8, 0.01, 0.4, 0.02, 0.15],
[1.82, 0.5, -0.7, 0.01, 0.35, 0.02, -0.25],
];
for (const [cx, cy, cz, cw, ch, cd, rot] of cracks) {
const crack = this._box(cx, cy, cz, cw, ch, cd, crackMat);
crack.rotation.z = rot;
rubbleGroup.add(crack);
}
// Rebar — thin metal cylinders protruding from the wall
for (const [rx, ry, rz, rLen, rRot] of [
[1.6, 1.7, -0.4, 0.8, 0.3],
[1.5, 1.2, 0.6, 0.6, -0.2],
[1.7, 0.9, -0.8, 0.5, 0.5],
[1.55, 1.5, 1.0, 0.7, -0.4],
]) {
const rebar = new THREE.Mesh(new THREE.CylinderGeometry(0.012, 0.012, rLen, 6), rebarMat);
rebar.position.set(rx, ry, rz);
rebar.rotation.z = Math.PI / 2 + rRot;
rebar.rotation.y = rRot * 0.5;
rebar.castShadow = true;
rubbleGroup.add(rebar);
}
// Rubble pieces — more varied with random rotations
const rubbleColors = [0x807868, 0x706860, 0x908878, 0x686058];
for (let i = 0; i < 10; i++) {
const s = 0.12 + Math.random() * 0.3;
const rMat = new THREE.MeshStandardMaterial({
color: rubbleColors[i % rubbleColors.length], roughness: 0.7 + Math.random() * 0.15,
emissive: 0x141210, emissiveIntensity: 0.2,
});
const piece = this._box(
1.3 + Math.random() * 1.4, s / 2, -1.5 + Math.random() * 3,
s, s * (0.4 + Math.random() * 0.5), s * (0.6 + Math.random() * 0.4), rMat
);
piece.rotation.x = (Math.random() - 0.5) * 0.6;
piece.rotation.y = (Math.random() - 0.5) * 1.2;
piece.rotation.z = (Math.random() - 0.5) * 0.4;
rubbleGroup.add(piece);
}
// Dust particles near rubble
const dustCount = 60;
const dustGeo = new THREE.BufferGeometry();
const dustPositions = new Float32Array(dustCount * 3);
for (let i = 0; i < dustCount; i++) {
dustPositions[i * 3] = 1.0 + Math.random() * 2.0;
dustPositions[i * 3 + 1] = Math.random() * 2.5;
dustPositions[i * 3 + 2] = -1.5 + Math.random() * 3.0;
}
dustGeo.setAttribute('position', new THREE.BufferAttribute(dustPositions, 3));
const dustMaterial = new THREE.PointsMaterial({
color: 0xaa9988, size: 0.03, transparent: true, opacity: 0.5,
blending: THREE.AdditiveBlending, depthWrite: false,
});
this._dustParticles = new THREE.Points(dustGeo, dustMaterial);
rubbleGroup.add(this._dustParticles);
this._props.rubbleWall = rubbleGroup;
rubbleGroup.visible = false;
this._scene.add(rubbleGroup);
}
// ---- SCREEN / TV (gesture control) ----
_buildScreen(metalMat) {
const screenGroup = new THREE.Group();
const screenFrame = new THREE.MeshStandardMaterial({ color: 0x484850, roughness: 0.2, metalness: 0.7, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
// Frame
screenGroup.add(this._box(0, 1.5, -4.7, 1.8, 1.1, 0.06, screenFrame));
// Screen surface (emissive, color shifts in update())
const screenSurfMat = new THREE.MeshStandardMaterial({
color: 0x1a3868, emissive: 0x1a3868, emissiveIntensity: 1.2, roughness: 0.1,
});
const screenSurf = new THREE.Mesh(new THREE.BoxGeometry(1.6, 0.9, 0.02), screenSurfMat);
screenSurf.position.set(0, 1.5, -4.66);
screenGroup.add(screenSurf);
this._screenGlow = screenSurfMat;
// Stand / mount — neck + base
screenGroup.add(this._box(0, 0.88, -4.7, 0.08, 0.16, 0.08, screenFrame));
screenGroup.add(this._box(0, 0.78, -4.7, 0.4, 0.03, 0.2, metalMat));
// Power LED indicator
const ledMat = new THREE.MeshStandardMaterial({
color: 0x00ff40, emissive: 0x00ff40, emissiveIntensity: 1.0,
});
const powerLed = new THREE.Mesh(new THREE.SphereGeometry(0.012, 6, 4), ledMat);
powerLed.position.set(0.82, 0.96, -4.66);
screenGroup.add(powerLed);
this._powerLed = ledMat;
// Subtle screen glow (point light)
const screenLight = new THREE.PointLight(0x4080e0, 1.5, 6);
screenLight.position.set(0, 1.5, -4.5);
screenGroup.add(screenLight);
// Media console below the screen
const consoleMat = new THREE.MeshStandardMaterial({ color: 0x484858, roughness: 0.45, metalness: 0.5, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
screenGroup.add(this._box(0, 0.55, -4.7, 1.2, 0.35, 0.35, consoleMat));
// Console shelf divider
screenGroup.add(this._box(0, 0.55, -4.54, 1.1, 0.02, 0.01, metalMat));
this._props.screen = screenGroup;
screenGroup.visible = false;
this._scene.add(screenGroup);
}
// ---- DESKS (crowd / office) ----
_buildDesks(darkMat, metalMat, accentMat) {
// Desk 1 (left)
const deskGroup = new THREE.Group();
deskGroup.add(this._box(-2, 0.38, -1, 1.2, 0.04, 0.6, darkMat));
for (const [lx, lz] of [[-2.55, -1.25], [-1.45, -1.25], [-2.55, -0.75], [-1.45, -0.75]]) {
deskGroup.add(this._box(lx, 0.19, lz, 0.04, 0.38, 0.04, darkMat));
}
// Monitor on desk 1
const monitorMat = new THREE.MeshStandardMaterial({ color: 0x484850, roughness: 0.2, metalness: 0.7, emissive: 0x0c0c14, emissiveIntensity: 0.15 });
const monScreenMat = new THREE.MeshStandardMaterial({
color: 0x183858, emissive: 0x183858, emissiveIntensity: 1.0, roughness: 0.1,
});
deskGroup.add(this._box(-2, 0.62, -1.15, 0.5, 0.35, 0.03, monitorMat));
deskGroup.add(this._box(-2, 0.62, -1.13, 0.44, 0.29, 0.01, monScreenMat));
deskGroup.add(this._box(-2, 0.42, -1.1, 0.06, 0.04, 0.06, metalMat)); // stand neck
deskGroup.add(this._box(-2, 0.40, -1.05, 0.18, 0.01, 0.12, metalMat)); // stand base
// Keyboard outline
deskGroup.add(this._box(-2, 0.405, -0.85, 0.35, 0.008, 0.12, accentMat));
// Office chair at desk 1
this._buildOfficeChair(deskGroup, -2, -0.55, darkMat, metalMat);
// Monitor glow light
const monLight = new THREE.PointLight(0x4080e0, 1.2, 4);
monLight.position.set(-2, 0.7, -1.0);
deskGroup.add(monLight);
this._props.desk = deskGroup;
deskGroup.visible = false;
this._scene.add(deskGroup);
// Desk 2 (right)
const desk2Group = new THREE.Group();
desk2Group.add(this._box(2, 0.38, 1, 1.0, 0.04, 0.6, darkMat));
for (const [lx, lz] of [[1.45, 0.75], [2.55, 0.75], [1.45, 1.25], [2.55, 1.25]]) {
desk2Group.add(this._box(lx, 0.19, lz, 0.04, 0.38, 0.04, darkMat));
}
// Monitor on desk 2
desk2Group.add(this._box(2, 0.62, 1.15, 0.5, 0.35, 0.03, monitorMat));
desk2Group.add(this._box(2, 0.62, 1.17, 0.44, 0.29, 0.01, monScreenMat));
desk2Group.add(this._box(2, 0.42, 1.1, 0.06, 0.04, 0.06, metalMat));
desk2Group.add(this._box(2, 0.40, 1.05, 0.18, 0.01, 0.12, metalMat));
// Keyboard
desk2Group.add(this._box(2, 0.405, 0.85, 0.35, 0.008, 0.12, accentMat));
// Office chair at desk 2
this._buildOfficeChair(desk2Group, 2, 0.55, darkMat, metalMat);
// Water cooler / plant between desks area
const plantMat = new THREE.MeshStandardMaterial({ color: 0x2a7838, roughness: 0.7, emissive: 0x0c2810, emissiveIntensity: 0.3 });
const potMat = new THREE.MeshStandardMaterial({ color: 0x706858, roughness: 0.6, emissive: 0x14120c, emissiveIntensity: 0.15 });
desk2Group.add(this._cyl(3.2, 0.15, 0, 0.12, 0.1, 0.3, 8, potMat));
// Foliage — cluster of small spheres
for (const [fx, fy, fz] of [[3.2, 0.45, 0], [3.15, 0.4, 0.06], [3.25, 0.42, -0.05]]) {
const leaf = new THREE.Mesh(new THREE.SphereGeometry(0.08, 6, 5), plantMat);
leaf.position.set(fx, fy, fz);
desk2Group.add(leaf);
}
// Monitor glow light
const monLight2 = new THREE.PointLight(0x4080e0, 1.2, 4);
monLight2.position.set(2, 0.7, 1.0);
desk2Group.add(monLight2);
this._props.desk2 = desk2Group;
desk2Group.visible = false;
this._scene.add(desk2Group);
}
// Helper: small office chair
_buildOfficeChair(parent, x, z, darkMat, metalMat) {
// Seat
parent.add(this._box(x, 0.38, z, 0.35, 0.03, 0.35, darkMat));
// Backrest
parent.add(this._box(x, 0.55, z - 0.16, 0.32, 0.3, 0.03, darkMat));
// Central post
parent.add(this._cyl(x, 0.22, z, 0.025, 0.025, 0.28, 6, metalMat));
// Base star (5 legs)
for (let i = 0; i < 5; i++) {
const angle = (i / 5) * Math.PI * 2;
const legLen = 0.16;
const leg = this._box(
x + Math.cos(angle) * legLen * 0.5, 0.04, z + Math.sin(angle) * legLen * 0.5,
legLen, 0.015, 0.025, metalMat
);
leg.rotation.y = -angle;
parent.add(leg);
}
}
// ---- SECURITY CAMERAS (patrol) ----
_buildCameras(metalMat) {
const camData = [
['camera1', [5, 3.5, -4.5]],
['camera2', [-5, 3.5, 4.5]],
];
for (const [name, pos] of camData) {
const camGroup = new THREE.Group();
camGroup.position.set(...pos);
// Camera body
camGroup.add(this._box(0, 0, 0, 0.15, 0.1, 0.2, metalMat));
// Lens
const lens = new THREE.Mesh(new THREE.CylinderGeometry(0.04, 0.04, 0.08, 8), metalMat);
lens.rotation.x = Math.PI / 2;
lens.position.z = 0.14;
camGroup.add(lens);
// Bracket / mount arm
camGroup.add(this._box(0, 0.1, -0.08, 0.04, 0.2, 0.04, metalMat));
// Rotating motor housing (visible joint)
const motorMat = new THREE.MeshStandardMaterial({ color: 0x686870, roughness: 0.35, metalness: 0.8, emissive: 0x141418, emissiveIntensity: 0.15 });
const motor = new THREE.Mesh(new THREE.CylinderGeometry(0.03, 0.03, 0.04, 8), motorMat);
motor.position.set(0, 0.05, -0.08);
camGroup.add(motor);
// FOV cone (semi-transparent)
const coneMat = new THREE.MeshStandardMaterial({
color: 0xff3040, transparent: true, opacity: 0.15,
side: THREE.DoubleSide, depthWrite: false,
emissive: 0xff2020, emissiveIntensity: 0.3,
});
const cone = new THREE.Mesh(new THREE.ConeGeometry(1.5, 3, 16, 1, true), coneMat);
cone.rotation.x = Math.PI / 2;
cone.position.z = 1.7;
camGroup.add(cone);
// Status LED (blinks in update)
const ledMat = new THREE.MeshStandardMaterial({
color: 0xff2020, emissive: 0xff2020, emissiveIntensity: 1.0,
});
const led = new THREE.Mesh(new THREE.SphereGeometry(0.015, 6, 4), ledMat);
led.position.set(0.08, 0.04, 0.08);
camGroup.add(led);
this._props[name] = camGroup;
camGroup.visible = false;
this._scene.add(camGroup);
// Store references for animation
if (name === 'camera1') {
this._camera1Group = camGroup;
this._cam1Cone = cone;
this._cam1Led = ledMat;
} else {
this._camera2Group = camGroup;
this._cam2Cone = cone;
this._cam2Led = ledMat;
}
}
}
// ---- ALERT SYSTEM ----
_buildAlertSystem() {
// Main alert point light
this._alertLight = new THREE.PointLight(0xff3040, 0, 10);
this._alertLight.position.set(0, 3.5, 0);
this._scene.add(this._alertLight);
// Ceiling-mounted alarm housing
const housingMat = new THREE.MeshStandardMaterial({ color: 0x686878, roughness: 0.35, metalness: 0.6, emissive: 0x101018, emissiveIntensity: 0.15 });
const housing = new THREE.Group();
// Base plate
housing.add(this._box(0, 3.95, 0, 0.2, 0.02, 0.2, housingMat));
// Housing body
housing.add(this._cyl(0, 3.85, 0, 0.08, 0.1, 0.16, 8, housingMat));
// Alarm lens (red when active, dark when inactive)
const lensMat = new THREE.MeshStandardMaterial({
color: 0x330808, emissive: 0x000000, emissiveIntensity: 0, roughness: 0.2,
transparent: true, opacity: 0.8,
});
const alarmLens = new THREE.Mesh(new THREE.SphereGeometry(0.06, 10, 8, 0, Math.PI * 2, 0, Math.PI / 2), lensMat);
alarmLens.position.set(0, 3.76, 0);
alarmLens.rotation.x = Math.PI;
housing.add(alarmLens);
this._alarmHousing = housing;
this._alarmLensMat = lensMat;
this._scene.add(housing);
}
// ========================================
// UPDATE (called every frame)
// ========================================
update(data, currentScenario) {
const scenario = data?.scenario || currentScenario;
const elapsed = Date.now() * 0.001;
// Switch visible props when scenario changes
if (scenario !== this._currentScenario) {
this._currentScenario = scenario;
for (const prop of Object.values(this._props)) prop.visible = false;
const propsToShow = SCENARIO_PROPS[scenario] || [];
for (const name of propsToShow) {
if (this._props[name]) this._props[name].visible = true;
}
}
// --- Alert light (fall / intrusion) ---
const cls = data?.classification || {};
if (cls.fall_detected || cls.intrusion) {
this._alertIntensity = Math.min(2, this._alertIntensity + 0.1);
} else {
this._alertIntensity = Math.max(0, this._alertIntensity - 0.05);
}
// Sawtooth pattern for urgency instead of smooth sine
const alertPhase = (elapsed * 3) % 1.0;
const sawtooth = alertPhase < 0.5 ? alertPhase * 2 : 2 - alertPhase * 2;
this._alertLight.intensity = this._alertIntensity * sawtooth;
// Alarm housing lens glow tracks alert
if (this._alarmLensMat) {
const alertFrac = Math.min(this._alertIntensity / 2, 1);
this._alarmLensMat.emissive.setHex(alertFrac > 0.05 ? 0xff2020 : 0x000000);
this._alarmLensMat.emissiveIntensity = alertFrac * sawtooth;
}
// Subtle ambient color shift during alerts
if (this._alertIntensity > 0.1 && this._alertLight) {
const r = 0.08 + 0.04 * sawtooth * this._alertIntensity;
const g = 0.05 - 0.02 * this._alertIntensity;
const b = 0.10 - 0.04 * this._alertIntensity;
// Shift the alert light color slightly over time
this._alertLight.color.setRGB(
Math.max(0, Math.min(1, 1.0)),
Math.max(0, Math.min(1, 0.15 - 0.1 * sawtooth)),
Math.max(0, Math.min(1, 0.2 - 0.15 * sawtooth))
);
} else if (this._alertLight) {
this._alertLight.color.setHex(0xff3040);
}
// --- Camera rotation animation ---
if (this._camera1Group && this._camera1Group.visible) {
this._camera1Group.rotation.y = Math.sin(elapsed * 0.4) * 0.5;
}
if (this._camera2Group && this._camera2Group.visible) {
this._camera2Group.rotation.y = Math.sin(elapsed * 0.4 + Math.PI) * 0.5;
}
// Camera LED blink
if (this._cam1Led && this._camera1Group?.visible) {
this._cam1Led.emissiveIntensity = (Math.sin(elapsed * 4) > 0.3) ? 1.0 : 0.1;
}
if (this._cam2Led && this._camera2Group?.visible) {
this._cam2Led.emissiveIntensity = (Math.sin(elapsed * 4 + 1) > 0.3) ? 1.0 : 0.1;
}
// --- Screen glow color shift ---
if (this._screenGlow && this._props.screen?.visible) {
const hue = (elapsed * 0.03) % 1;
const r = 0.10 + 0.06 * Math.sin(hue * Math.PI * 2);
const g = 0.16 + 0.08 * Math.sin(hue * Math.PI * 2 + 2.1);
const b = 0.28 + 0.12 * Math.sin(hue * Math.PI * 2 + 4.2);
this._screenGlow.emissive.setRGB(r, g, b);
}
// Power LED gentle pulse
if (this._powerLed && this._props.screen?.visible) {
this._powerLed.emissiveIntensity = 0.5 + 0.5 * Math.sin(elapsed * 2);
}
// --- Dust particle drift near rubble ---
if (this._dustParticles && this._props.rubbleWall?.visible) {
const dPos = this._dustParticles.geometry.getAttribute('position');
for (let i = 0; i < dPos.count; i++) {
let y = dPos.getY(i) + 0.002 * Math.sin(elapsed + i);
if (y > 2.5) y = 0;
dPos.setY(i, y);
dPos.setX(i, dPos.getX(i) + Math.sin(elapsed * 0.5 + i * 0.3) * 0.0005);
}
dPos.needsUpdate = true;
}
}
}
+163
View File
@@ -0,0 +1,163 @@
/**
* Module A "The Subcarrier Manifold"
* 3D scrolling surface: 64 subcarriers x 60 time slots
*/
import * as THREE from 'three';
const MANIFOLD_VERTEX = `
attribute float aHeight;
attribute float aAge; // 0 = newest, 1 = oldest
varying float vHeight;
varying float vAge;
void main() {
vec3 pos = position;
pos.y += aHeight * 2.0;
vHeight = aHeight;
vAge = aAge;
gl_Position = projectionMatrix * modelViewMatrix * vec4(pos, 1.0);
}
`;
const MANIFOLD_FRAGMENT = `
uniform float uTime;
varying float vHeight;
varying float vAge;
void main() {
// Color map: low=deep blue, mid=cyan, high=amber
vec3 lo = vec3(0.02, 0.06, 0.2);
vec3 mid = vec3(0.0, 0.83, 1.0);
vec3 hi = vec3(1.0, 0.53, 0.0);
float h = clamp(vHeight, 0.0, 1.0);
vec3 col = h < 0.5
? mix(lo, mid, h * 2.0)
: mix(mid, hi, (h - 0.5) * 2.0);
// Fade older rows
float alpha = 0.3 + 0.7 * (1.0 - vAge);
gl_FragColor = vec4(col, alpha);
}
`;
const SUBS = 64;
const TIME_SLOTS = 60;
export class SubcarrierManifold {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
this._history = []; // ring buffer of Float32Array[64]
for (let i = 0; i < TIME_SLOTS; i++) {
this._history.push(new Float32Array(SUBS));
}
this._head = 0;
// Build surface geometry
const geo = new THREE.PlaneGeometry(8, 5, SUBS - 1, TIME_SLOTS - 1);
const vertCount = SUBS * TIME_SLOTS;
this._heights = new Float32Array(vertCount);
this._ages = new Float32Array(vertCount);
for (let t = 0; t < TIME_SLOTS; t++) {
for (let s = 0; s < SUBS; s++) {
this._ages[t * SUBS + s] = t / TIME_SLOTS;
}
}
geo.setAttribute('aHeight', new THREE.BufferAttribute(this._heights, 1));
geo.setAttribute('aAge', new THREE.BufferAttribute(this._ages, 1));
// Solid surface
const mat = new THREE.ShaderMaterial({
vertexShader: MANIFOLD_VERTEX,
fragmentShader: MANIFOLD_FRAGMENT,
uniforms: { uTime: { value: 0 } },
transparent: true,
side: THREE.DoubleSide,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._mesh = new THREE.Mesh(geo, mat);
this._mesh.rotation.x = -Math.PI * 0.35;
this.group.add(this._mesh);
// Wireframe overlay
const wireGeo = geo.clone();
wireGeo.setAttribute('aHeight', new THREE.BufferAttribute(this._heights, 1));
wireGeo.setAttribute('aAge', new THREE.BufferAttribute(this._ages, 1));
const wireMat = new THREE.ShaderMaterial({
vertexShader: MANIFOLD_VERTEX,
fragmentShader: `
varying float vHeight;
varying float vAge;
void main() {
float alpha = 0.15 * (1.0 - vAge);
gl_FragColor = vec4(0.0, 0.83, 1.0, alpha);
}
`,
uniforms: { uTime: { value: 0 } },
transparent: true,
wireframe: true,
depthWrite: false,
blending: THREE.AdditiveBlending,
});
this._wire = new THREE.Mesh(wireGeo, wireMat);
this._wire.rotation.x = -Math.PI * 0.35;
this.group.add(this._wire);
this._frameAccum = 0;
this._pushInterval = 1 / 15; // push ~15 rows/sec
}
update(dt, elapsed, data) {
this._mesh.material.uniforms.uTime.value = elapsed;
// Push new amplitude data at regular intervals
this._frameAccum += dt;
if (this._frameAccum >= this._pushInterval && data) {
this._frameAccum = 0;
const amp = data.nodes?.[0]?.amplitude;
const row = new Float32Array(SUBS);
if (amp && amp.length > 0) {
for (let i = 0; i < SUBS; i++) {
row[i] = amp[i % amp.length] || 0;
}
}
this._history[this._head] = row;
this._head = (this._head + 1) % TIME_SLOTS;
this._rebuildHeights();
}
}
_rebuildHeights() {
for (let t = 0; t < TIME_SLOTS; t++) {
const histIdx = (this._head + t) % TIME_SLOTS;
const row = this._history[histIdx];
for (let s = 0; s < SUBS; s++) {
const idx = t * SUBS + s;
this._heights[idx] = row[s];
this._ages[idx] = t / TIME_SLOTS;
}
}
const geo = this._mesh.geometry;
geo.attributes.aHeight.needsUpdate = true;
geo.attributes.aAge.needsUpdate = true;
const wGeo = this._wire.geometry;
wGeo.attributes.aHeight.needsUpdate = true;
wGeo.attributes.aAge.needsUpdate = true;
}
dispose() {
this._mesh.geometry.dispose();
this._mesh.material.dispose();
this._wire.geometry.dispose();
this._wire.material.dispose();
}
}
+187
View File
@@ -0,0 +1,187 @@
/**
* Module B "Vital Signs Oracle"
* Breathing/HR as orbital torus rings with beat markers + trail particles
*/
import * as THREE from 'three';
export class VitalsOracle {
constructor(scene, panelGroup) {
this.group = new THREE.Group();
if (panelGroup) panelGroup.add(this.group);
else scene.add(this.group);
// Outer torus — breathing (violet)
const breathGeo = new THREE.TorusGeometry(1.8, 0.06, 16, 64);
this._breathMat = new THREE.MeshBasicMaterial({
color: 0x8844ff,
transparent: true,
opacity: 0.7,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._breathRing = new THREE.Mesh(breathGeo, this._breathMat);
this._breathRing.rotation.x = Math.PI * 0.4;
this.group.add(this._breathRing);
// Inner torus — heart rate (crimson)
const hrGeo = new THREE.TorusGeometry(1.2, 0.04, 16, 64);
this._hrMat = new THREE.MeshBasicMaterial({
color: 0xff2244,
transparent: true,
opacity: 0.6,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
this._hrRing = new THREE.Mesh(hrGeo, this._hrMat);
this._hrRing.rotation.x = Math.PI * 0.5;
this._hrRing.rotation.z = Math.PI * 0.15;
this.group.add(this._hrRing);
// Center orb
const orbGeo = new THREE.SphereGeometry(0.35, 24, 24);
this._orbMat = new THREE.MeshBasicMaterial({
color: 0x00d4ff,
transparent: true,
opacity: 0.5,
blending: THREE.AdditiveBlending,
});
this._orb = new THREE.Mesh(orbGeo, this._orbMat);
this.group.add(this._orb);
// Bloom point light
this._light = new THREE.PointLight(0x00d4ff, 1.5, 8);
this.group.add(this._light);
// Trail particles along breathing ring
const trailCount = 120;
const trailGeo = new THREE.BufferGeometry();
const trailPos = new Float32Array(trailCount * 3);
const trailSizes = new Float32Array(trailCount);
for (let i = 0; i < trailCount; i++) {
const angle = (i / trailCount) * Math.PI * 2;
trailPos[i * 3] = Math.cos(angle) * 1.8;
trailPos[i * 3 + 1] = 0;
trailPos[i * 3 + 2] = Math.sin(angle) * 1.8;
trailSizes[i] = 3;
}
trailGeo.setAttribute('position', new THREE.BufferAttribute(trailPos, 3));
trailGeo.setAttribute('size', new THREE.BufferAttribute(trailSizes, 1));
const trailMat = new THREE.PointsMaterial({
color: 0x8844ff,
size: 0.08,
transparent: true,
opacity: 0.4,
blending: THREE.AdditiveBlending,
depthWrite: false,
sizeAttenuation: true,
});
this._trails = new THREE.Points(trailGeo, trailMat);
this._trails.rotation.x = Math.PI * 0.4;
this.group.add(this._trails);
// Beat flash sprites
this._beatFlash = this._createBeatSprite(0xff2244);
this.group.add(this._beatFlash);
this._beatTimer = 0;
this._lastBeatTime = 0;
// State
this._breathBpm = 0;
this._hrBpm = 0;
this._breathConf = 0;
this._hrConf = 0;
}
_createBeatSprite(color) {
const canvas = document.createElement('canvas');
canvas.width = 64;
canvas.height = 64;
const ctx = canvas.getContext('2d');
const gradient = ctx.createRadialGradient(32, 32, 0, 32, 32, 32);
gradient.addColorStop(0, `rgba(255, 34, 68, 1)`);
gradient.addColorStop(0.3, `rgba(255, 34, 68, 0.5)`);
gradient.addColorStop(1, `rgba(255, 34, 68, 0)`);
ctx.fillStyle = gradient;
ctx.fillRect(0, 0, 64, 64);
const tex = new THREE.CanvasTexture(canvas);
const mat = new THREE.SpriteMaterial({
map: tex,
transparent: true,
blending: THREE.AdditiveBlending,
depthWrite: false,
});
const sprite = new THREE.Sprite(mat);
sprite.scale.set(0, 0, 0);
return sprite;
}
update(dt, elapsed, data) {
const vs = data?.vital_signs || {};
this._breathBpm = vs.breathing_rate_bpm || 0;
this._hrBpm = vs.heart_rate_bpm || 0;
this._breathConf = vs.breathing_confidence || 0;
this._hrConf = vs.heart_rate_confidence || 0;
// Breathing ring pulsation
const breathFreq = this._breathBpm / 60;
const breathPulse = breathFreq > 0 ? Math.sin(elapsed * Math.PI * 2 * breathFreq) : 0;
const breathScale = 1.0 + breathPulse * 0.08 * this._breathConf;
this._breathRing.scale.set(breathScale, breathScale, 1);
this._breathMat.opacity = 0.3 + this._breathConf * 0.5;
// HR ring pulsation (faster)
const hrFreq = this._hrBpm / 60;
const hrPulse = hrFreq > 0 ? Math.sin(elapsed * Math.PI * 2 * hrFreq) : 0;
const hrScale = 1.0 + hrPulse * 0.06 * this._hrConf;
this._hrRing.scale.set(hrScale, hrScale, 1);
this._hrMat.opacity = 0.2 + this._hrConf * 0.5;
// Slow rotation
this._breathRing.rotation.z = elapsed * 0.1;
this._hrRing.rotation.z = -elapsed * 0.15;
this._trails.rotation.z = elapsed * 0.1;
// Center orb pulse
const orbPulse = 1.0 + breathPulse * 0.1;
this._orb.scale.set(orbPulse, orbPulse, orbPulse);
this._light.intensity = 0.8 + Math.abs(breathPulse) * 1.0;
// Beat flash on HR cycle
if (hrFreq > 0) {
this._beatTimer += dt;
const beatInterval = 1 / hrFreq;
if (this._beatTimer >= beatInterval) {
this._beatTimer -= beatInterval;
this._lastBeatTime = elapsed;
}
const beatAge = elapsed - this._lastBeatTime;
const flashSize = Math.max(0, 1.2 - beatAge * 4) * this._hrConf;
this._beatFlash.scale.set(flashSize, flashSize, 1);
} else {
this._beatFlash.scale.set(0, 0, 0);
}
// Update trail particle sizes based on breathing
const sizes = this._trails.geometry.attributes.size;
if (sizes) {
for (let i = 0; i < sizes.count; i++) {
const phase = (i / sizes.count) * Math.PI * 2 + elapsed * breathFreq * Math.PI * 2;
sizes.array[i] = 0.04 + Math.abs(Math.sin(phase)) * 0.06 * this._breathConf;
}
sizes.needsUpdate = true;
}
}
dispose() {
this._breathRing.geometry.dispose();
this._breathMat.dispose();
this._hrRing.geometry.dispose();
this._hrMat.dispose();
this._orb.geometry.dispose();
this._orbMat.dispose();
this._trails.geometry.dispose();
this._trails.material.dispose();
}
}
+123 -18
View File
@@ -12,13 +12,15 @@ from __future__ import annotations
import logging
import math
import os
import platform
import re
import subprocess
import threading
import time
from collections import deque
from dataclasses import dataclass, field
from typing import Deque, List, Optional, Protocol
from typing import Deque, List, Optional, Protocol, Union
import numpy as np
@@ -173,27 +175,47 @@ class LinuxWifiCollector:
"""Collect a single sample right now (blocking)."""
return self._read_sample()
# -- availability check --------------------------------------------------
@classmethod
def is_available(cls, interface: str = "wlan0") -> tuple[bool, str]:
"""Check if Linux WiFi collection is possible without raising.
Returns
-------
(available, reason) : tuple[bool, str]
``available`` is True when /proc/net/wireless exists and lists
the requested interface. ``reason`` is a human-readable
explanation when unavailable.
"""
if not os.path.exists("/proc/net/wireless"):
return False, (
"/proc/net/wireless not found. "
"This environment has no Linux wireless subsystem "
"(common in Docker, WSL, or headless servers)."
)
try:
with open("/proc/net/wireless", "r") as f:
content = f.read()
except OSError as exc:
return False, f"Cannot read /proc/net/wireless: {exc}"
if interface not in content:
names = cls._parse_interface_names(content)
return False, (
f"Interface '{interface}' not listed in /proc/net/wireless. "
f"Available: {names or '(none)'}. "
f"Ensure the interface is up and associated with an AP."
)
return True, "ok"
# -- internals -----------------------------------------------------------
def _validate_interface(self) -> None:
"""Check that the interface exists on this machine."""
try:
with open("/proc/net/wireless", "r") as f:
content = f.read()
if self._interface not in content:
raise RuntimeError(
f"WiFi interface '{self._interface}' not found in "
f"/proc/net/wireless. Available interfaces may include: "
f"{self._parse_interface_names(content)}. "
f"Ensure the interface is up and associated with an AP."
)
except FileNotFoundError:
raise RuntimeError(
"Cannot read /proc/net/wireless. "
"This collector requires a Linux system with wireless-extensions support. "
"If running in a container or VM without WiFi hardware, use "
"SimulatedCollector instead."
)
available, reason = self.is_available(self._interface)
if not available:
raise RuntimeError(reason)
@staticmethod
def _parse_interface_names(proc_content: str) -> List[str]:
@@ -736,3 +758,86 @@ class MacosWifiCollector:
if self._running:
logger.error("macOS WiFi utility exited unexpectedly. Collector stopped.")
self._running = False
# ---------------------------------------------------------------------------
# Collector factory (ADR-049)
# ---------------------------------------------------------------------------
CollectorType = Union[LinuxWifiCollector, WindowsWifiCollector, MacosWifiCollector, SimulatedCollector]
def create_collector(
preferred: str = "auto",
interface: str = "wlan0",
sample_rate_hz: float = 10.0,
) -> CollectorType:
"""Create the best available WiFi collector for the current platform.
Resolution order (when ``preferred="auto"``):
1. Platform-native WiFi:
- Linux: LinuxWifiCollector (requires /proc/net/wireless + active interface)
- Windows: WindowsWifiCollector (netsh wlan)
- macOS: MacosWifiCollector (CoreWLAN)
2. SimulatedCollector (always available)
This function never raises -- it always returns a usable collector.
Parameters
----------
preferred : str
``"auto"`` for platform detection, or one of ``"linux"``,
``"windows"``, ``"macos"``, ``"simulated"`` to force a specific
collector.
interface : str
WiFi interface name (Linux/Windows only).
sample_rate_hz : float
Target sampling rate.
"""
_VALID_PREFERRED = {"auto", "linux", "windows", "macos", "simulated"}
if preferred not in _VALID_PREFERRED:
logger.warning(
"WiFi collector: unknown preferred=%r (valid: %s). Falling back to auto.",
preferred, ", ".join(sorted(_VALID_PREFERRED)),
)
preferred = "auto"
system = platform.system()
if preferred == "auto":
if system == "Linux":
available, reason = LinuxWifiCollector.is_available(interface)
if available:
logger.info("WiFi collector: using LinuxWifiCollector on %s", interface)
return LinuxWifiCollector(interface=interface, sample_rate_hz=sample_rate_hz)
logger.warning("WiFi collector: LinuxWifiCollector unavailable (%s).", reason)
elif system == "Windows":
try:
win_iface = interface if interface != "wlan0" else "Wi-Fi"
collector = WindowsWifiCollector(interface=win_iface, sample_rate_hz=min(sample_rate_hz, 2.0))
collector.collect_once()
logger.info("WiFi collector: using WindowsWifiCollector on '%s'", interface)
return collector
except Exception as exc:
logger.warning("WiFi collector: WindowsWifiCollector unavailable (%s).", exc)
elif system == "Darwin":
try:
collector = MacosWifiCollector(sample_rate_hz=sample_rate_hz)
logger.info("WiFi collector: using MacosWifiCollector")
return collector
except Exception as exc:
logger.warning("WiFi collector: MacosWifiCollector unavailable (%s).", exc)
elif preferred == "linux":
return LinuxWifiCollector(interface=interface, sample_rate_hz=sample_rate_hz)
elif preferred == "windows":
return WindowsWifiCollector(interface=interface, sample_rate_hz=min(sample_rate_hz, 2.0))
elif preferred == "macos":
return MacosWifiCollector(sample_rate_hz=sample_rate_hz)
elif preferred == "simulated":
return SimulatedCollector(seed=42, sample_rate_hz=sample_rate_hz)
logger.info(
"WiFi collector: falling back to SimulatedCollector. "
"For real sensing, connect ESP32 nodes via UDP:5005 or install platform WiFi drivers."
)
return SimulatedCollector(seed=42, sample_rate_hz=sample_rate_hz)
+18 -42
View File
@@ -24,7 +24,6 @@ import asyncio
import json
import logging
import math
import platform
import signal
import socket
import struct
@@ -38,10 +37,6 @@ import numpy as np
# Sensing pipeline imports
from v1.src.sensing.rssi_collector import (
LinuxWifiCollector,
SimulatedCollector,
WindowsWifiCollector,
MacosWifiCollector,
WifiSample,
RingBuffer,
)
@@ -321,7 +316,13 @@ class SensingWebSocketServer:
self._running = False
def _create_collector(self):
"""Auto-detect data source: ESP32 UDP > Windows WiFi > Linux WiFi > simulated."""
"""Auto-detect data source: ESP32 UDP > platform WiFi > simulated.
Uses the ``create_collector`` factory (ADR-049) for platform WiFi
detection, which never raises and logs actionable fallback messages.
"""
from .rssi_collector import create_collector
# 1. Try ESP32 UDP first
print(" Probing for ESP32 on UDP :5005 ...")
if probe_esp32_udp(ESP32_UDP_PORT, timeout=2.0):
@@ -329,43 +330,18 @@ class SensingWebSocketServer:
self.source = "esp32"
return Esp32UdpCollector(port=ESP32_UDP_PORT, sample_rate_hz=10.0)
# 2. Platform-specific WiFi
system = platform.system()
if system == "Windows":
try:
collector = WindowsWifiCollector(sample_rate_hz=2.0)
collector.collect_once() # test that it works
logger.info("Using WindowsWifiCollector")
self.source = "windows_wifi"
return collector
except Exception as e:
logger.warning("Windows WiFi unavailable (%s), falling back", e)
elif system == "Linux":
# In Docker on Mac, Linux is detected but no wireless extensions exist.
# Force SimulatedCollector if /proc/net/wireless doesn't exist.
import os
if os.path.exists("/proc/net/wireless"):
try:
collector = LinuxWifiCollector(sample_rate_hz=10.0)
self.source = "linux_wifi"
return collector
except RuntimeError:
logger.warning("Linux WiFi unavailable, falling back")
else:
logger.warning("Linux detected but /proc/net/wireless missing (likely Docker). Falling back.")
elif system == "Darwin":
try:
collector = MacosWifiCollector(sample_rate_hz=10.0)
logger.info("Using MacosWifiCollector")
self.source = "macos_wifi"
return collector
except Exception as e:
logger.warning("macOS WiFi unavailable (%s), falling back", e)
# 2. Platform-specific WiFi (auto-detect with graceful fallback)
collector = create_collector(preferred="auto", sample_rate_hz=10.0)
# 3. Simulated
logger.info("Using SimulatedCollector")
self.source = "simulated"
return SimulatedCollector(seed=42, sample_rate_hz=10.0)
# Map collector class to source label
source_map = {
"LinuxWifiCollector": "linux_wifi",
"WindowsWifiCollector": "windows_wifi",
"MacosWifiCollector": "macos_wifi",
"SimulatedCollector": "simulated",
}
self.source = source_map.get(type(collector).__name__, "unknown")
return collector
def _build_message(self, features: RssiFeatures, result: SensingResult) -> str:
"""Build the JSON message to broadcast."""
+103
View File
@@ -702,3 +702,106 @@ class TestBandPower:
# Band 0.21-0.39 has no power
p = _band_power(freqs, psd, 0.21, 0.39)
assert p == 0.0
# ===========================================================================
# LinuxWifiCollector.is_available() tests (ADR-049)
# ===========================================================================
from unittest.mock import patch, mock_open
from v1.src.sensing.rssi_collector import LinuxWifiCollector, create_collector
class TestLinuxWifiCollectorAvailability:
def test_unavailable_when_proc_missing(self):
"""is_available returns False when /proc/net/wireless doesn't exist."""
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=False):
available, reason = LinuxWifiCollector.is_available("wlan0")
assert available is False
assert "/proc/net/wireless not found" in reason
def test_unavailable_when_interface_not_listed(self):
"""is_available returns False when the interface isn't in proc."""
proc_content = (
"Inter-| sta-| Quality | Discarded packets\n"
" face | tus | link level noise | nwid crypt frag retry misc\n"
" wlan1: 0000 60. -50. -95. 0 0 0 0 0\n"
)
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=True):
with patch("builtins.open", mock_open(read_data=proc_content)):
available, reason = LinuxWifiCollector.is_available("wlan0")
assert available is False
assert "wlan0" in reason
assert "wlan1" in reason
def test_available_when_interface_listed(self):
"""is_available returns True when the interface is present."""
proc_content = (
"Inter-| sta-| Quality | Discarded packets\n"
" face | tus | link level noise | nwid crypt frag retry misc\n"
" wlan0: 0000 60. -50. -95. 0 0 0 0 0\n"
)
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=True):
with patch("builtins.open", mock_open(read_data=proc_content)):
available, reason = LinuxWifiCollector.is_available("wlan0")
assert available is True
assert reason == "ok"
def test_unavailable_when_file_unreadable(self):
"""is_available returns False when /proc/net/wireless exists but can't be read."""
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=True):
with patch("builtins.open", side_effect=PermissionError("Permission denied")):
available, reason = LinuxWifiCollector.is_available("wlan0")
assert available is False
assert "Cannot read" in reason
# ===========================================================================
# create_collector() factory tests (ADR-049)
# ===========================================================================
class TestCreateCollector:
def test_returns_simulated_when_no_wifi(self):
"""On Linux without /proc/net/wireless, should return SimulatedCollector."""
with patch("v1.src.sensing.rssi_collector.platform.system", return_value="Linux"):
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=False):
collector = create_collector(preferred="auto")
assert isinstance(collector, SimulatedCollector)
def test_returns_simulated_for_explicit_preference(self):
"""preferred='simulated' always returns SimulatedCollector."""
collector = create_collector(preferred="simulated")
assert isinstance(collector, SimulatedCollector)
def test_returns_linux_collector_when_available(self):
"""On Linux with /proc/net/wireless, should return LinuxWifiCollector."""
proc_content = (
"Inter-| sta-| Quality | Discarded packets\n"
" face | tus | link level noise | nwid crypt frag retry misc\n"
" wlan0: 0000 60. -50. -95. 0 0 0 0 0\n"
)
with patch("v1.src.sensing.rssi_collector.platform.system", return_value="Linux"):
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=True):
with patch("builtins.open", mock_open(read_data=proc_content)):
collector = create_collector(preferred="auto", interface="wlan0")
assert isinstance(collector, LinuxWifiCollector)
def test_never_raises(self):
"""create_collector should never raise, regardless of platform."""
for plat in ["Linux", "Windows", "Darwin", "FreeBSD", "SunOS"]:
with patch("v1.src.sensing.rssi_collector.platform.system", return_value=plat):
with patch("v1.src.sensing.rssi_collector.os.path.exists", return_value=False):
with patch("subprocess.run", side_effect=FileNotFoundError("not found")):
try:
collector = create_collector(preferred="auto")
assert collector is not None
except Exception as exc:
pytest.fail(f"create_collector raised on {plat}: {exc}")
def test_windows_default_interface_mapping(self):
"""On Windows with default interface='wlan0', should map to 'Wi-Fi'."""
with patch("v1.src.sensing.rssi_collector.platform.system", return_value="Windows"):
with patch("subprocess.run", side_effect=FileNotFoundError("netsh not found")):
collector = create_collector(preferred="auto", interface="wlan0")
# Should fall back to SimulatedCollector since netsh isn't available
assert isinstance(collector, SimulatedCollector)
+35
View File
@@ -0,0 +1,35 @@
# vendor/
Third-party dependencies managed as [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules).
| Directory | Upstream | Description |
|-----------|----------|-------------|
| `midstream/` | [ruvnet/midstream](https://github.com/ruvnet/midstream) | Claude Flow middleware and agent orchestration |
| `ruvector/` | [ruvnet/ruvector](https://github.com/ruvnet/ruvector) | RuVector signal processing and ML pipelines |
| `sublinear-time-solver/` | [ruvnet/sublinear-time-solver](https://github.com/ruvnet/sublinear-time-solver) | Sublinear-time optimization solvers |
All submodules track the `main` branch of their upstream repos.
## Setup
After cloning this repo, initialize submodules:
```bash
git submodule update --init --recursive
```
Or clone with submodules in one step:
```bash
git clone --recurse-submodules https://github.com/ruvnet/RuView.git
```
## Update to latest upstream
```bash
git submodule update --remote --merge
git add vendor/
git commit -m "chore: update vendor submodules"
```
A GitHub Actions workflow also checks for updates every 6 hours and opens a PR automatically.