mirror of
https://github.com/ruvnet/RuView
synced 2026-07-24 17:43:20 +00:00
feat: complete vendor repos, add edge intelligence and WASM modules
- Add 154 missing vendor files (gitignore was filtering them) - vendor/midstream: 564 files (was 561) - vendor/sublinear-time-solver: 1190 files (was 1039) - Add ESP32 edge processing (ADR-039): presence, vitals, fall detection - Add WASM programmable sensing (ADR-040/041) with wasm3 runtime - Add firmware CI workflow (.github/workflows/firmware-ci.yml) - Add wifi-densepose-wasm-edge crate for edge WASM modules - Update sensing server, provision.py, UI components Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
@@ -1,299 +1,210 @@
|
||||
# ADR-039: ESP32-S3 Edge Intelligence — On-Device Signal Processing and RuVector Integration
|
||||
# ADR-039: ESP32-S3 Edge Intelligence Pipeline
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed |
|
||||
| **Date** | 2026-03-03 |
|
||||
| **Depends on** | ADR-018 (binary frame format), ADR-014 (SOTA signal processing), ADR-021 (vital sign extraction), ADR-029 (multistatic sensing), ADR-030 (persistent field model), ADR-031 (RuView sensing-first RF) |
|
||||
| **Supersedes** | None |
|
||||
**Status**: Accepted (hardware-validated on RuView ESP32-S3)
|
||||
**Date**: 2026-03-02
|
||||
**Deciders**: @ruvnet
|
||||
|
||||
## Context
|
||||
|
||||
The current ESP32-S3 firmware (1,018 lines, 7 files) is a "dumb sensor" — it captures raw CSI frames and streams them unprocessed over UDP at ~20 Hz. All signal processing, feature extraction, presence detection, vital sign estimation, and pose inference happen server-side in the Rust crates.
|
||||
WiFi-DensePose captures Channel State Information (CSI) from ESP32-S3 nodes and streams raw I/Q data to a host server for processing. This architecture has limitations:
|
||||
|
||||
This creates several limitations:
|
||||
1. **Bandwidth waste** — raw CSI frames are 128-384 bytes each at 20 Hz = ~60 KB/s per node. Most of this is noise.
|
||||
2. **Latency** — round-trip to server adds 5-50ms depending on network.
|
||||
3. **Server dependency** — nodes are useless without an active aggregator.
|
||||
4. **Scalability ceiling** — 6-node mesh at 20 Hz = 120 frames/s = server bottleneck.
|
||||
5. **No local alerting** — fall detection, breathing anomaly, or intrusion must wait for server roundtrip.
|
||||
|
||||
The ESP32-S3 has significant untapped compute:
|
||||
- **Dual-core Xtensa LX7** at 240 MHz
|
||||
- **512 KB SRAM** + optional 8 MB PSRAM (our board has 8 MB flash)
|
||||
- **Vector/DSP instructions** (PIE — Processor Instruction Extensions)
|
||||
- **FPU** — hardware single-precision floating point
|
||||
- **~80% idle CPU** — current firmware uses <20% (WiFi + CSI callback + UDP send)
|
||||
1. **Bandwidth**: Raw CSI at 20 Hz × 128 subcarriers × 2 bytes = ~5 KB/frame = ~100 KB/s per node. Multi-node deployments saturate low-bandwidth links.
|
||||
2. **Latency**: Server-side processing adds network round-trip delay for time-critical signals like fall detection.
|
||||
3. **Power**: Continuous raw streaming prevents duty-cycling for battery-powered deployments.
|
||||
4. **Scalability**: Server CPU scales linearly with node count for basic signal processing that could run on the ESP32-S3's dual cores.
|
||||
|
||||
## Decision
|
||||
|
||||
Implement a **3-tier edge intelligence pipeline** on the ESP32-S3 firmware, progressively offloading signal processing from the server to the device. Each tier is independently toggleable via NVS configuration.
|
||||
Implement a tiered edge processing pipeline on the ESP32-S3 that performs signal processing locally and sends compact results:
|
||||
|
||||
### Tier 1: Smart Filtering & Compression (Firmware C)
|
||||
### Tier 0 — Raw Passthrough (default, backward compatible)
|
||||
No on-device processing. CSI frames streamed as-is (magic `0xC5110001`).
|
||||
|
||||
Lightweight processing in the CSI callback path. Zero additional latency.
|
||||
### Tier 1 — Basic Signal Processing
|
||||
- Phase extraction and unwrapping from I/Q pairs
|
||||
- Welford running variance per subcarrier
|
||||
- Top-K subcarrier selection by variance
|
||||
- Delta compression (XOR + RLE) for 30-50% bandwidth reduction (magic `0xC5110003`)
|
||||
|
||||
| Feature | Source ADR | Algorithm | Memory | CPU |
|
||||
|---------|-----------|-----------|--------|-----|
|
||||
| **Phase sanitization** | ADR-014 | Linear phase unwrap + conjugate multiply | 256 B | <1% |
|
||||
| **Amplitude normalization** | ADR-014 | Per-subcarrier running mean/std (Welford) | 512 B | <1% |
|
||||
| **Subcarrier selection** | ADR-016 (ruvector-mincut) | Top-K variance subcarriers | 128 B | <1% |
|
||||
| **Static environment suppression** | ADR-030 | Exponential moving average subtraction | 512 B | <1% |
|
||||
| **Adaptive frame decimation** | New | Skip frames when CSI variance < threshold | 8 B | <1% |
|
||||
| **Delta compression** | New | XOR + RLE vs. previous frame | 512 B | <2% |
|
||||
### Tier 2 — Full Edge Intelligence
|
||||
All of Tier 1, plus:
|
||||
- Biquad IIR bandpass filters: breathing (0.1-0.5 Hz), heart rate (0.8-2.0 Hz)
|
||||
- Zero-crossing BPM estimation
|
||||
- Presence detection with adaptive threshold calibration (1200 frames, 3-sigma)
|
||||
- Fall detection (phase acceleration exceeding configurable threshold)
|
||||
- Multi-person vitals via subcarrier group clustering (up to 4 persons)
|
||||
- 32-byte vitals packet at configurable interval (magic `0xC5110002`)
|
||||
|
||||
**Bandwidth reduction**: 60-80% (send only changed, high-variance subcarriers).
|
||||
|
||||
**ADR-018 v2 frame extension** (backward-compatible):
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Existing 20-byte header unchanged.
|
||||
New optional trailer (if magic bit set):
|
||||
[N*2] Compressed I/Q (delta-coded, only selected subcarriers)
|
||||
[2] Subcarrier bitmap (which of 64 subcarriers included)
|
||||
[1] Frame flags: bit0=compressed, bit1=phase-sanitized, bit2=amplitude-normed
|
||||
[1] Motion score (0-255)
|
||||
[1] Presence confidence (0-255)
|
||||
[1] Reserved
|
||||
Core 0 (WiFi) Core 1 (DSP)
|
||||
┌─────────────────┐ ┌──────────────────────────┐
|
||||
│ CSI callback │──SPSC ring──▶│ Phase extract + unwrap │
|
||||
│ (wifi_csi_cb) │ buffer │ Welford variance │
|
||||
│ │ │ Top-K selection │
|
||||
│ UDP raw stream │ │ Biquad bandpass filters │
|
||||
│ (0xC5110001) │ │ Zero-crossing BPM │
|
||||
└─────────────────┘ │ Presence detection │
|
||||
│ Fall detection │
|
||||
│ Multi-person clustering │
|
||||
│ Delta compression │
|
||||
│ ──▶ UDP vitals (0xC5110002)│
|
||||
│ ──▶ UDP compressed (0x03) │
|
||||
└──────────────────────────┘
|
||||
```
|
||||
|
||||
### Tier 2: On-Device Vital Signs & Presence (Firmware C + fixed-point DSP)
|
||||
### Wire Protocols
|
||||
|
||||
Runs as a FreeRTOS task on Core 1 (CSI collection on Core 0), processing a sliding window of CSI frames.
|
||||
**Vitals Packet (32 bytes, magic `0xC5110002`)**:
|
||||
|
||||
| Feature | Source ADR | Algorithm | Memory | CPU (Core 1) |
|
||||
|---------|-----------|-----------|--------|--------------|
|
||||
| **Presence detection** | ADR-029 | Variance threshold on amplitude envelope | 2 KB | 5% |
|
||||
| **Motion scoring** | ADR-014 | Subcarrier correlation coefficient | 1 KB | 3% |
|
||||
| **Breathing rate** | ADR-021 | Bandpass 0.1-0.5 Hz + peak detection on CSI phase | 8 KB | 10% |
|
||||
| **Heart rate** | ADR-021 | Bandpass 0.8-2.0 Hz + autocorrelation on CSI phase | 8 KB | 15% |
|
||||
| **Fall detection** | ADR-029 | Sudden variance spike + sustained stillness | 1 KB | 2% |
|
||||
| **Room occupancy count** | ADR-037 | CSI rank estimation (eigenvalue spread) | 4 KB | 8% |
|
||||
| **Coherence gate** | ADR-029 (ruvsense) | Z-score coherence, accept/reject/recalibrate | 1 KB | 2% |
|
||||
| Offset | Type | Field |
|
||||
|--------|------|-------|
|
||||
| 0-3 | u32 LE | Magic `0xC5110002` |
|
||||
| 4 | u8 | Node ID |
|
||||
| 5 | u8 | Flags (bit0=presence, bit1=fall, bit2=motion) |
|
||||
| 6-7 | u16 LE | Breathing rate (BPM × 100) |
|
||||
| 8-11 | u32 LE | Heart rate (BPM × 10000) |
|
||||
| 12 | i8 | RSSI |
|
||||
| 13 | u8 | Number of detected persons |
|
||||
| 14-15 | u8[2] | Reserved |
|
||||
| 16-19 | f32 LE | Motion energy |
|
||||
| 20-23 | f32 LE | Presence score |
|
||||
| 24-27 | u32 LE | Timestamp (ms since boot) |
|
||||
| 28-31 | u32 LE | Reserved |
|
||||
|
||||
**Total memory**: ~25 KB (fits in SRAM, no PSRAM needed).
|
||||
**Total CPU**: ~45% of Core 1.
|
||||
**Compressed Frame (magic `0xC5110003`)**:
|
||||
|
||||
**Output**: Compact vital-signs UDP packet (32 bytes) at 1 Hz:
|
||||
| Offset | Type | Field |
|
||||
|--------|------|-------|
|
||||
| 0-3 | u32 LE | Magic `0xC5110003` |
|
||||
| 4 | u8 | Node ID |
|
||||
| 5 | u8 | WiFi channel |
|
||||
| 6-7 | u16 LE | Original I/Q length |
|
||||
| 8-9 | u16 LE | Compressed length |
|
||||
| 10+ | bytes | RLE-encoded XOR delta |
|
||||
|
||||
```
|
||||
Offset Size Field
|
||||
0 4 Magic: 0xC5110002 (vitals packet)
|
||||
4 1 Node ID
|
||||
5 1 Packet type (0x02 = vitals)
|
||||
6 2 Sequence (LE u16)
|
||||
8 1 Presence (0=empty, 1=present, 2=moving)
|
||||
9 1 Motion score (0-255)
|
||||
10 1 Occupancy estimate (0-8 persons)
|
||||
11 1 Coherence gate (0=reject, 1=predict, 2=accept, 3=recalibrate)
|
||||
12 2 Breathing rate (BPM * 100, LE u16) — 0 if not detected
|
||||
14 2 Heart rate (BPM * 100, LE u16) — 0 if not detected
|
||||
16 2 Breathing confidence (0-10000, LE u16)
|
||||
18 2 Heart rate confidence (0-10000, LE u16)
|
||||
20 1 Fall detected (0/1)
|
||||
21 1 Anomaly flags (bitfield)
|
||||
22 2 Ambient RSSI mean (LE i16)
|
||||
24 4 CSI frame count since last report (LE u32)
|
||||
28 4 Uptime seconds (LE u32)
|
||||
```
|
||||
### Configuration
|
||||
|
||||
### Tier 3: Lightweight Feature Extraction (Firmware C + optional PSRAM)
|
||||
|
||||
Pre-compute features that the server-side neural network needs, reducing server CPU by 60-80%.
|
||||
|
||||
| Feature | Source ADR | Algorithm | Memory | CPU |
|
||||
|---------|-----------|-----------|--------|-----|
|
||||
| **Phase difference matrix** | ADR-014 | Adjacent subcarrier phase diff | 4 KB | 5% |
|
||||
| **Amplitude spectrogram** | ADR-014 | 64-bin FFT on 1s window per subcarrier | 32 KB | 15% |
|
||||
| **Doppler-time map** | ADR-029 | 2D FFT across subcarriers × time | 16 KB | 10% |
|
||||
| **Fresnel zone crossing** | ADR-014 | First Fresnel radius + fade count | 1 KB | 2% |
|
||||
| **Cross-link correlation** | ADR-029 | Pearson correlation between TX-RX pairs | 2 KB | 5% |
|
||||
| **Environment fingerprint** | ADR-027 (MERIDIAN) | PCA-compressed 16-dim CSI signature | 4 KB | 5% |
|
||||
| **Gesture template match** | ADR-029 (ruvsense) | DTW on 8-dim feature vector | 8 KB | 10% |
|
||||
|
||||
**Total memory**: ~67 KB (SRAM) or up to 256 KB with PSRAM.
|
||||
**Total CPU**: ~52% of Core 1.
|
||||
|
||||
**Output**: Feature vector UDP packet (variable size, ~200-500 bytes) at 4 Hz:
|
||||
|
||||
```
|
||||
Offset Size Field
|
||||
0 4 Magic: 0xC5110003 (feature packet)
|
||||
4 1 Node ID
|
||||
5 1 Packet type (0x03 = features)
|
||||
6 2 Feature bitmap (which features included)
|
||||
8 4 Timestamp ms (LE u32)
|
||||
12 N Feature payloads (concatenated, lengths determined by bitmap)
|
||||
```
|
||||
|
||||
## NVS Configuration
|
||||
|
||||
All tiers controllable via NVS without reflashing:
|
||||
Six NVS keys in the `csi_cfg` namespace:
|
||||
|
||||
| NVS Key | Type | Default | Description |
|
||||
|---------|------|---------|-------------|
|
||||
| `edge_tier` | u8 | 0 | 0=raw only, 1=smart filter, 2=+vitals, 3=+features |
|
||||
| `decim_thresh` | u16 | 100 | Adaptive decimation variance threshold |
|
||||
| `subk_count` | u8 | 32 | Top-K subcarriers to keep (Tier 1) |
|
||||
| `vital_window` | u16 | 300 | Vital sign window frames (15s at 20 Hz) |
|
||||
| `vital_interval` | u16 | 1000 | Vital report interval ms |
|
||||
| `feature_hz` | u8 | 4 | Feature extraction rate |
|
||||
| `fall_thresh` | u16 | 500 | Fall detection variance spike threshold |
|
||||
| `presence_thresh` | u16 | 50 | Presence detection threshold |
|
||||
| `edge_tier` | u8 | 2 | Processing tier (0/1/2) |
|
||||
| `pres_thresh` | u16 | 0 | Presence threshold × 1000 (0 = auto) |
|
||||
| `fall_thresh` | u16 | 2000 | Fall threshold × 1000 (rad/s²) |
|
||||
| `vital_win` | u16 | 256 | Phase history window |
|
||||
| `vital_int` | u16 | 1000 | Vitals interval (ms) |
|
||||
| `subk_count` | u8 | 8 | Top-K subcarrier count |
|
||||
|
||||
Provisioning:
|
||||
```bash
|
||||
python firmware/esp32-csi-node/provision.py --port COM7 \
|
||||
--edge-tier 2 --vital-window 300 --presence-thresh 50
|
||||
```
|
||||
All configurable via `provision.py --edge-tier 2 --pres-thresh 0.05 ...`
|
||||
|
||||
## Implementation Plan
|
||||
### Additional Features
|
||||
|
||||
### Phase 1: Infrastructure (1 week)
|
||||
- **OTA Updates**: HTTP server on port 8032 (`POST /ota`, `GET /ota/status`) with rollback support
|
||||
- **Power Management**: WiFi modem sleep + automatic light sleep with configurable duty cycle
|
||||
|
||||
1. **Dual-core task architecture**
|
||||
- Core 0: WiFi + CSI callback (existing)
|
||||
- Core 1: Edge processing task (new FreeRTOS task)
|
||||
- Lock-free ring buffer between cores (producer-consumer)
|
||||
## Consequences
|
||||
|
||||
2. **Ring buffer design**
|
||||
```c
|
||||
#define RING_BUF_FRAMES 64 // ~3.2s at 20 Hz
|
||||
typedef struct {
|
||||
wifi_csi_info_t info;
|
||||
int8_t iq_data[384]; // Max I/Q payload
|
||||
uint32_t timestamp_ms;
|
||||
uint8_t tx_mac[6];
|
||||
} csi_ring_entry_t;
|
||||
```
|
||||
### Positive
|
||||
- Fall detection latency reduced from ~500 ms (network RTT) to <50 ms (on-device)
|
||||
- Bandwidth reduced 30-50% with delta compression, or 95%+ with vitals-only mode
|
||||
- Battery-powered deployments possible with duty-cycled light sleep
|
||||
- Server can handle 10x more nodes (only parses 32-byte vitals instead of ~5 KB CSI)
|
||||
|
||||
3. **NVS config extension** — add `edge_tier` and tier-specific params
|
||||
4. **ADR-018 v2 header** — backward-compatible extension bit
|
||||
### Negative
|
||||
- Firmware complexity increases (edge_processing.c is ~750 lines)
|
||||
- ESP32-S3 RAM usage increases ~12 KB for ring buffer + filter state
|
||||
- Binary size increases from ~550 KB to ~925 KB with full WASM3 Tier 3 (10% free in 1 MB partition — see ADR-040)
|
||||
|
||||
### Phase 2: Tier 1 — Smart Filtering (1 week)
|
||||
### Risks
|
||||
- BPM accuracy depends on subject distance and movement; needs real-world validation
|
||||
- Fall detection heuristic may false-positive on environmental motion (doors, pets)
|
||||
- Multi-person separation via subcarrier clustering is approximate without calibration
|
||||
|
||||
1. **Phase unwrap** — O(N) linear scan, in-place
|
||||
2. **Welford running stats** — per-subcarrier mean/variance, O(1) update
|
||||
3. **Top-K subcarrier selection** — partial sort, O(N) with selection algorithm
|
||||
4. **Delta compression** — XOR vs previous frame, RLE encode
|
||||
5. **Adaptive decimation** — skip frame if total variance < threshold
|
||||
## Implementation
|
||||
|
||||
### Phase 3: Tier 2 — Vital Signs (2 weeks)
|
||||
- `firmware/esp32-csi-node/main/edge_processing.c` — DSP pipeline (~750 lines)
|
||||
- `firmware/esp32-csi-node/main/edge_processing.h` — Types and API
|
||||
- `firmware/esp32-csi-node/main/ota_update.c/h` — HTTP OTA endpoint
|
||||
- `firmware/esp32-csi-node/main/power_mgmt.c/h` — Power management
|
||||
- `rust-port/.../wifi-densepose-sensing-server/src/main.rs` — Vitals parser + REST endpoint
|
||||
- `scripts/provision.py` — Edge config CLI arguments
|
||||
- `.github/workflows/firmware-ci.yml` — CI build + size gate (updated to 950 KB for Tier 3)
|
||||
|
||||
1. **Presence detector** — amplitude variance over 1s window
|
||||
2. **Motion scorer** — correlation coefficient between consecutive frames
|
||||
3. **Breathing extractor** — port from `wifi-densepose-vitals::BreathingExtractor::esp32_default()`
|
||||
- Bandpass via biquad IIR filter (0.1-0.5 Hz)
|
||||
- Peak detection with parabolic interpolation
|
||||
- Fixed-point arithmetic (Q15.16) for efficiency
|
||||
4. **Heart rate extractor** — port from `wifi-densepose-vitals::HeartRateExtractor::esp32_default()`
|
||||
- Bandpass via biquad IIR (0.8-2.0 Hz)
|
||||
- Autocorrelation peak search
|
||||
5. **Fall detection** — variance spike (>5σ) followed by sustained stillness (>3s)
|
||||
6. **Coherence gate** — port from `ruvsense::coherence_gate` (Z-score threshold)
|
||||
### Tier 3 — WASM Programmable Sensing (ADR-040, ADR-041)
|
||||
|
||||
### Phase 4: Tier 3 — Feature Extraction (2 weeks)
|
||||
See [ADR-040](ADR-040-wasm-programmable-sensing.md) for hot-loadable WASM modules
|
||||
compiled from Rust, executed via WASM3 interpreter on-device. Core modules:
|
||||
gesture recognition, coherence monitoring, adversarial detection.
|
||||
|
||||
1. **FFT engine** — fixed-point 64-point FFT (radix-2 DIT, no library needed)
|
||||
2. **Amplitude spectrogram** — 1s sliding window FFT per subcarrier
|
||||
3. **Doppler-time map** — 2D FFT across subcarrier × time dimensions
|
||||
4. **Phase difference matrix** — adjacent subcarrier Δφ
|
||||
5. **Environment fingerprint** — online PCA (incremental SVD, 16 components)
|
||||
6. **Gesture DTW** — 8 stored templates, dynamic time warping on 8-dim feature
|
||||
[ADR-041](ADR-041-wasm-module-collection.md) defines the curated module collection
|
||||
(37 modules across 6 categories). Phase 1 implemented modules:
|
||||
- `vital_trend.rs` — Clinical vital sign trend analysis (bradypnea, tachypnea, apnea)
|
||||
- `intrusion.rs` — State-machine intrusion detection (calibrate-monitor-arm-alert)
|
||||
- `occupancy.rs` — Spatial occupancy zone detection with per-zone variance analysis
|
||||
|
||||
### Phase 5: CI/CD + Testing (1 week)
|
||||
## Hardware Benchmark (RuView ESP32-S3)
|
||||
|
||||
1. **GitHub Actions firmware build** — Docker `espressif/idf:v5.2` on every PR
|
||||
2. **Host-side unit tests** — compile edge processing functions on x86 with mock CSI data
|
||||
3. **Credential leak check** — binary string scan in CI
|
||||
4. **Binary size tracking** — fail CI if firmware exceeds 90% of partition
|
||||
5. **QEMU smoke test** — boot verification, NVS load, task creation
|
||||
Measured on ESP32-S3 (QFN56 rev v0.2, 8 MB flash, 160 MHz, ESP-IDF v5.2).
|
||||
|
||||
## ESP32-S3 Resource Budget
|
||||
### Boot Timing
|
||||
|
||||
| Resource | Available | Tier 1 | Tier 2 | Tier 3 | Remaining |
|
||||
|----------|-----------|--------|--------|--------|-----------|
|
||||
| **SRAM** | 512 KB | 2 KB | 25 KB | 67 KB | 418 KB |
|
||||
| **Core 0 CPU** | 100% | 5% | 0% | 0% | 75% (WiFi uses ~20%) |
|
||||
| **Core 1 CPU** | 100% | 0% | 45% | 52% | 3% (Tier 2+3 exclusive) |
|
||||
| **Flash** | 1 MB partition | 4 KB code | 12 KB code | 20 KB code | 964 KB |
|
||||
| Milestone | Time (ms) |
|
||||
|-----------|-----------|
|
||||
| `app_main()` | 412 |
|
||||
| WiFi STA init | 627 |
|
||||
| WiFi connected + IP | 3,732 |
|
||||
| CSI collection init | 3,754 |
|
||||
| Edge DSP task started | 3,773 |
|
||||
| WASM runtime initialized | 3,857 |
|
||||
| **Total boot → ready** | **~3.9 s** |
|
||||
|
||||
Note: Tier 2 and Tier 3 run on Core 1 but are time-multiplexed — vitals at 1 Hz, features at 4 Hz. Combined peak load is ~60% of Core 1.
|
||||
### CSI Performance
|
||||
|
||||
## Mapping to Existing ADRs
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Frame rate | **28.5 Hz** (measured, ch 5 BW20) |
|
||||
| Frame sizes | 128 / 256 bytes |
|
||||
| RSSI range | -83 to -32 dBm (mean -62 dBm) |
|
||||
| Per-frame interval | 30.6 ms avg |
|
||||
|
||||
| Existing ADR | Capability | Edge Tier | Implementation |
|
||||
|-------------|------------|-----------|----------------|
|
||||
| **ADR-014** (SOTA signal) | Phase sanitization | 1 | Linear unwrap in CSI callback |
|
||||
| **ADR-014** | Amplitude normalization | 1 | Welford running stats |
|
||||
| **ADR-014** | Feature extraction | 3 | FFT spectrogram + phase diff matrix |
|
||||
| **ADR-014** | Fresnel zone detection | 3 | Fade counting + first Fresnel radius |
|
||||
| **ADR-016** (RuVector) | Subcarrier selection | 1 | Top-K variance (simplified mincut) |
|
||||
| **ADR-021** (Vitals) | Breathing rate | 2 | Biquad IIR + peak detect |
|
||||
| **ADR-021** | Heart rate | 2 | Biquad IIR + autocorrelation |
|
||||
| **ADR-021** | Anomaly detection | 2 | Z-score on vital readings |
|
||||
| **ADR-027** (MERIDIAN) | Environment fingerprint | 3 | Online PCA, 16-dim signature |
|
||||
| **ADR-029** (RuvSense) | Coherence gate | 2 | Z-score coherence scoring |
|
||||
| **ADR-029** | Multistatic correlation | 3 | Pearson cross-link correlation |
|
||||
| **ADR-029** | Gesture recognition | 3 | DTW template matching |
|
||||
| **ADR-030** (Field model) | Static suppression | 1 | EMA background subtraction |
|
||||
| **ADR-031** (RuView) | Sensing-first NDP | Existing | Already in firmware (stub) |
|
||||
| **ADR-037** (Multi-person) | Occupancy counting | 2 | CSI rank estimation |
|
||||
### Memory
|
||||
|
||||
## Server-Side Changes
|
||||
| Region | Size |
|
||||
|--------|------|
|
||||
| RAM (main heap) | 256 KiB |
|
||||
| RAM (secondary) | 21 KiB |
|
||||
| DRAM | 32 KiB |
|
||||
| RTC RAM | 7 KiB |
|
||||
| **Total available** | **316 KiB** |
|
||||
| PSRAM | Not populated on test board |
|
||||
| WASM arena fallback | Internal heap (160 KB/slot × 4) |
|
||||
|
||||
The Rust aggregator (`wifi-densepose-hardware`) needs to handle the new packet types:
|
||||
### Firmware Binary
|
||||
|
||||
```rust
|
||||
match magic {
|
||||
0xC5110001 => parse_raw_csi_frame(buf), // Existing
|
||||
0xC5110002 => parse_vitals_packet(buf), // New: Tier 2
|
||||
0xC5110003 => parse_feature_packet(buf), // New: Tier 3
|
||||
_ => Err(ParseError::UnknownMagic(magic)),
|
||||
}
|
||||
```
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Binary size | **925 KB** (0xE7440 bytes) |
|
||||
| Partition size | 1 MB (factory) |
|
||||
| Free space | 10% (99 KB) |
|
||||
| CI size gate | 950 KB (PASS) |
|
||||
| WASM3 interpreter | Included (full, ~100 KB) |
|
||||
| WASM binary (7 modules) | 13.8 KB (wasm32-unknown-unknown release) |
|
||||
|
||||
When edge tier ≥ 1, the server can skip its own phase sanitization and amplitude normalization. When edge tier = 3, the server skips feature extraction entirely and feeds pre-computed features directly to the neural network.
|
||||
### WASM Runtime
|
||||
|
||||
## Testing Strategy
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Init time | **106 ms** |
|
||||
| Module slots | 4 |
|
||||
| Arena per slot | 160 KB |
|
||||
| Frame budget | 10,000 µs (10 ms) |
|
||||
| Timer interval | 1,000 ms (1 Hz) |
|
||||
|
||||
| Test Type | Tool | What |
|
||||
|-----------|------|------|
|
||||
| **Host unit tests** | gcc + Unity + mock CSI data | Phase unwrap, Welford stats, IIR filter, peak detect, DTW |
|
||||
| **QEMU smoke test** | Docker QEMU | Boot, NVS load, task creation, ring buffer |
|
||||
| **Hardware regression** | ESP32-S3 + serial log | Full pipeline: CSI → edge processing → UDP → server |
|
||||
| **Accuracy validation** | Python reference impl | Compare edge vitals vs. server vitals on same CSI data |
|
||||
| **Stress test** | 6-node mesh | Tier 3 at 20 Hz sustained, no frame drops |
|
||||
### Findings
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
1. **Rust on ESP32 (esp-rs)** — More type-safe, could share code with server crates. Rejected: larger binary, longer compile times, less mature ESP-IDF support for CSI APIs.
|
||||
|
||||
2. **MicroPython on ESP32** — Easier prototyping. Rejected: too slow for 20 Hz real-time processing, no fixed-point DSP.
|
||||
|
||||
3. **External co-processor (FPGA/DSP)** — Maximum throughput. Rejected: cost ($50+ per node), defeats the $8 ESP32 value proposition.
|
||||
|
||||
4. **Server-only processing** — Keep firmware dumb. Rejected: doesn't solve bandwidth, latency, or standalone operation requirements.
|
||||
|
||||
## Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Core 1 processing exceeds real-time budget | Adaptive quality: reduce feature_hz or fall back to lower tier |
|
||||
| Fixed-point arithmetic introduces accuracy drift | Validate against Rust f64 reference on same CSI data; track error bounds |
|
||||
| NVS config complexity overwhelms users | Sensible defaults; provision.py presets: `--preset home`, `--preset medical`, `--preset security` |
|
||||
| ADR-018 v2 header breaks old aggregators | Backward-compatible: old magic = old format. New bit in flags field signals extension |
|
||||
| Memory fragmentation from ring buffer | Static allocation only; no malloc in edge processing path |
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Tier 1 reduces bandwidth by ≥60% with <1 dB SNR loss
|
||||
- [ ] Tier 2 breathing rate within ±1 BPM of server-side estimate
|
||||
- [ ] Tier 2 heart rate within ±3 BPM of server-side estimate
|
||||
- [ ] Tier 2 fall detection latency <500ms (vs. ~2s server roundtrip)
|
||||
- [ ] Tier 2 presence detection accuracy ≥95%
|
||||
- [ ] Tier 3 feature extraction matches server output within 5% RMSE
|
||||
- [ ] All tiers: zero frame drops at 20 Hz sustained on single node
|
||||
- [ ] Firmware binary stays under 90% of 1 MB app partition
|
||||
- [ ] SRAM usage stays under 400 KB (leave headroom for WiFi stack)
|
||||
- [ ] CI pipeline: build + host unit tests + binary size check on every PR
|
||||
1. **Fall detection threshold too low** — default `fall_thresh=2000` (2.0 rad/s²) triggers 6.7 false positives/s in static indoor environment. Recommend increasing to 5000-8000 for typical deployments.
|
||||
2. **No PSRAM on test board** — WASM arena falls back to internal heap. Boards with PSRAM would support larger modules.
|
||||
3. **CSI rate exceeds spec** — measured 28.5 Hz vs. expected ~20 Hz. Performance headroom is better than estimated.
|
||||
4. **WiFi-to-Ethernet isolation** — some routers block UDP between WiFi and wired clients. Recommend same-subnet verification in deployment guide.
|
||||
|
||||
@@ -0,0 +1,582 @@
|
||||
# ADR-040: WASM Programmable Sensing (Tier 3)
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-03-02
|
||||
**Deciders**: @ruvnet
|
||||
|
||||
## Context
|
||||
|
||||
ADR-039 implemented Tiers 0-2 of the ESP32-S3 edge intelligence pipeline:
|
||||
- **Tier 0**: Raw CSI passthrough (magic `0xC5110001`)
|
||||
- **Tier 1**: Basic DSP — phase unwrap, Welford stats, top-K, delta compression
|
||||
- **Tier 2**: Full pipeline — vitals, presence, fall detection, multi-person
|
||||
|
||||
The firmware uses ~820 KB of flash, leaving ~80 KB headroom in the 1 MB OTA partition. The ESP32-S3 has 8 MB PSRAM available for runtime data. New sensing algorithms (gesture recognition, signal coherence monitoring, adversarial detection) currently require a full firmware reflash — impractical for deployed sensor networks.
|
||||
|
||||
The project already has 35+ RuVector WASM crates and 28 pre-built `.wasm` binaries, but none are integrated into the ESP32 firmware.
|
||||
|
||||
## Decision
|
||||
|
||||
Add a **Tier 3 WASM programmable sensing layer** that executes hot-loadable algorithms compiled from Rust to `wasm32-unknown-unknown`, interpreted on-device via the WASM3 runtime.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
Core 1 (DSP Task)
|
||||
┌──────────────────────────────────────────────────┐
|
||||
│ Tier 2 Pipeline (existing) │
|
||||
│ Phase extract → Welford → Top-K → Biquad → │
|
||||
│ BPM → Presence → Fall → Multi-person │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────┐ │
|
||||
│ │ Tier 3 WASM Runtime (new) │ │
|
||||
│ │ WASM3 Interpreter (MIT, ~100 KB flash) │ │
|
||||
│ │ ┌────────────┐ ┌────────────┐ │ │
|
||||
│ │ │ Module 0 │ │ Module 1 │ ...×4 │ │
|
||||
│ │ │ gesture.wm │ │ coherence │ │ │
|
||||
│ │ └─────┬──────┘ └─────┬──────┘ │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ Host API ("csi" namespace) │ │
|
||||
│ │ csi_get_phase, csi_get_amplitude, ... │ │
|
||||
│ └──────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
│ UDP output (0xC5110004) │
|
||||
└──────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Components
|
||||
|
||||
| Component | File | Description |
|
||||
|-----------|------|-------------|
|
||||
| WASM3 component | `components/wasm3/CMakeLists.txt` | ESP-IDF managed component, fetches WASM3 from GitHub |
|
||||
| Runtime host | `main/wasm_runtime.c/h` | WASM3 environment, module slots, host API bindings |
|
||||
| HTTP upload | `main/wasm_upload.c/h` | REST endpoints for module management on port 8032 |
|
||||
| Rust WASM crate | `wifi-densepose-wasm-edge/` | `no_std` sensing algorithms compiled to WASM |
|
||||
|
||||
### Host API (namespace "csi")
|
||||
|
||||
| Import | Signature | Description |
|
||||
|--------|-----------|-------------|
|
||||
| `csi_get_phase` | `(i32) -> f32` | Current phase for subcarrier index |
|
||||
| `csi_get_amplitude` | `(i32) -> f32` | Current amplitude |
|
||||
| `csi_get_variance` | `(i32) -> f32` | Welford running variance |
|
||||
| `csi_get_bpm_breathing` | `() -> f32` | Breathing BPM from Tier 2 |
|
||||
| `csi_get_bpm_heartrate` | `() -> f32` | Heart rate BPM from Tier 2 |
|
||||
| `csi_get_presence` | `() -> i32` | Presence flag (0/1) |
|
||||
| `csi_get_motion_energy` | `() -> f32` | Motion energy scalar |
|
||||
| `csi_get_n_persons` | `() -> i32` | Detected person count |
|
||||
| `csi_get_timestamp` | `() -> i32` | Milliseconds since boot |
|
||||
| `csi_emit_event` | `(i32, f32) -> void` | Emit custom event to host |
|
||||
| `csi_log` | `(i32, i32) -> void` | Debug log from WASM memory |
|
||||
| `csi_get_phase_history` | `(i32, i32) -> i32` | Copy phase history ring buffer |
|
||||
|
||||
### Module Lifecycle
|
||||
|
||||
| Export | Called | Description |
|
||||
|--------|--------|-------------|
|
||||
| `on_init()` | Once, when module starts | Initialize module state |
|
||||
| `on_frame(n_sc: i32)` | Per CSI frame (~20 Hz) | Process current frame |
|
||||
| `on_timer()` | At configurable interval | Periodic tasks |
|
||||
|
||||
### Wire Protocol (magic `0xC5110004`)
|
||||
|
||||
| Offset | Type | Field |
|
||||
|--------|------|-------|
|
||||
| 0-3 | u32 LE | Magic `0xC5110004` |
|
||||
| 4 | u8 | Node ID |
|
||||
| 5 | u8 | Module ID (slot index) |
|
||||
| 6-7 | u16 LE | Event count |
|
||||
| 8+ | Event[] | Array of (u8 type, f32 value) tuples |
|
||||
|
||||
### HTTP Endpoints (port 8032)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/wasm/upload` | Upload .wasm binary (max 128 KB) |
|
||||
| `GET` | `/wasm/list` | List loaded modules with status |
|
||||
| `POST` | `/wasm/start/:id` | Start a module |
|
||||
| `POST` | `/wasm/stop/:id` | Stop a module |
|
||||
| `DELETE` | `/wasm/:id` | Unload a module |
|
||||
|
||||
### WASM Crate Modules
|
||||
|
||||
| Module | Source | Events | Description |
|
||||
|--------|--------|--------|-------------|
|
||||
| `gesture.rs` | `ruvsense/gesture.rs` | 1 (Core) | DTW template matching for gesture recognition |
|
||||
| `coherence.rs` | `ruvector/viewpoint/coherence.rs` | 2 (Core) | Phase phasor coherence monitoring |
|
||||
| `adversarial.rs` | `ruvsense/adversarial.rs` | 3 (Core) | Signal anomaly/adversarial detection |
|
||||
| `vital_trend.rs` | ADR-041 Phase 1 | 100-111 (Medical) | Clinical vital sign trend analysis (bradypnea, tachypnea, bradycardia, tachycardia, apnea) |
|
||||
| `occupancy.rs` | ADR-041 Phase 1 | 300-302 (Building) | Spatial occupancy zone detection with per-zone variance analysis |
|
||||
| `intrusion.rs` | ADR-041 Phase 1 | 200-203 (Security) | State-machine intrusion detector (calibrate-monitor-arm-alert) |
|
||||
|
||||
### Memory Budget
|
||||
|
||||
| Component | SRAM | PSRAM | Flash |
|
||||
|-----------|------|-------|-------|
|
||||
| WASM3 interpreter | ~10 KB | — | ~100 KB |
|
||||
| WASM module storage (×4) | — | 512 KB | — |
|
||||
| WASM execution stack | 8 KB | — | — |
|
||||
| Host API bindings | 2 KB | — | ~15 KB |
|
||||
| HTTP upload handler | 1 KB | — | ~8 KB |
|
||||
| RVF parser + verifier | 1 KB | — | ~6 KB |
|
||||
| **Total Tier 3** | **~22 KB** | **512 KB** | **~129 KB** |
|
||||
| **Running total (Tier 0-3)** | **~34 KB** | **512 KB** | **~925 KB** |
|
||||
|
||||
**Measured binary size**: 925 KB (0xE7440 bytes), 10% free in 1 MB OTA partition.
|
||||
|
||||
### NVS Configuration
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `wasm_max` | u8 | 4 | Maximum concurrent WASM modules |
|
||||
| `wasm_verify` | u8 | 1 | Require signature verification (secure-by-default) |
|
||||
| `wasm_pubkey` | blob(32) | — | Signing public key for WASM verification |
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
- Deploy new sensing algorithms to 1000+ nodes without reflashing firmware
|
||||
- 20-year extensibility horizon — new algorithms via .wasm uploads
|
||||
- Algorithms developed/tested in Rust, compiled to portable WASM
|
||||
- PSRAM utilization (previously unused 8 MB) for module storage
|
||||
- Hot-swap algorithms for A/B testing in production deployments
|
||||
- Same `no_std` Rust code runs on ESP32 (WASM3) and in browser (wasm-pack)
|
||||
|
||||
### Negative
|
||||
- WASM3 interpreter overhead: ~10× slower than native C for compute-heavy code
|
||||
- Adds ~123 KB flash footprint (firmware approaches 950 KB of 1 MB limit)
|
||||
- Additional attack surface via WASM module upload endpoint
|
||||
- Debugging WASM modules on ESP32 is harder than native C
|
||||
|
||||
### Risks
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| WASM3 memory management may fragment PSRAM over time | Fixed 160 KB arenas pre-allocated at boot per slot — no runtime malloc/free cycles |
|
||||
| Complex WASM modules (>64 KB) may cause stack overflow in interpreter | `WASM_STACK_SIZE` = 8 KB, `d_m3MaxFunctionStackHeight` = 128; modules validated at load time |
|
||||
| HTTP upload endpoint requires network security | Ed25519 signature verification enabled by default (`wasm_verify=1`); disable only via NVS for lab/dev |
|
||||
| Runaway WASM module blocks DSP pipeline | Per-frame budget guard (10 ms default); module auto-stopped after 10 consecutive faults |
|
||||
| Denial-of-service via rapid upload/unload cycles | Max 4 concurrent slots; upload handler validates size before PSRAM copy |
|
||||
|
||||
## Implementation
|
||||
|
||||
- `firmware/esp32-csi-node/components/wasm3/CMakeLists.txt` — WASM3 ESP-IDF component
|
||||
- `firmware/esp32-csi-node/main/wasm_runtime.c/h` — Runtime host with 12 API bindings + manifest
|
||||
- `firmware/esp32-csi-node/main/wasm_upload.c/h` — HTTP REST endpoints (RVF-aware)
|
||||
- `firmware/esp32-csi-node/main/rvf_parser.c/h` — RVF container parser and verifier
|
||||
- `rust-port/.../wifi-densepose-wasm-edge/` — Rust WASM crate (gesture, coherence, adversarial, rvf, occupancy, vital_trend, intrusion)
|
||||
- `rust-port/.../wifi-densepose-sensing-server/src/main.rs` — `0xC5110004` parser
|
||||
- `docs/adr/ADR-039-esp32-edge-intelligence.md` — Updated with Tier 3 reference
|
||||
|
||||
---
|
||||
|
||||
## Appendix A: Production Hardening
|
||||
|
||||
The initial Tier 3 implementation addresses five production-readiness concerns:
|
||||
|
||||
### A.1 Fixed PSRAM Arenas
|
||||
|
||||
Dynamic `heap_caps_malloc` / `free` cycles on PSRAM fragment memory over days of
|
||||
continuous operation. Instead, each module slot pre-allocates a **160 KB fixed arena**
|
||||
at boot (`WASM_ARENA_SIZE`). The WASM binary and WASM3 runtime heap both live inside
|
||||
this arena. Unloading a module zeroes the arena but never frees it — the slot is
|
||||
reused on the next `wasm_runtime_load()`.
|
||||
|
||||
```
|
||||
Boot: [arena0: 160 KB][arena1: 160 KB][arena2: 160 KB][arena3: 160 KB]
|
||||
Total: 640 KB PSRAM
|
||||
Load: [module0 binary | wasm3 heap | ...padding... ]
|
||||
Unload:[zeroed .......................................] ← slot reusable
|
||||
```
|
||||
|
||||
This eliminates fragmentation at the cost of reserving 640 KB PSRAM at boot
|
||||
(8% of 8 MB). The remaining 7.36 MB is available for future use.
|
||||
|
||||
### A.2 Per-Frame Budget Guard
|
||||
|
||||
Each `on_frame()` call is measured with `esp_timer_get_time()`. If execution
|
||||
exceeds `WASM_FRAME_BUDGET_US` (default 10 ms = 10,000 us), a budget fault is
|
||||
recorded. After **10 consecutive faults**, the module is auto-stopped with
|
||||
`WASM_MODULE_ERROR` state. This prevents a runaway WASM module from blocking the
|
||||
Tier 2 DSP pipeline.
|
||||
|
||||
```c
|
||||
int64_t t_start = esp_timer_get_time();
|
||||
m3_CallV(slot->fn_on_frame, n_sc);
|
||||
uint32_t elapsed_us = (uint32_t)(esp_timer_get_time() - t_start);
|
||||
|
||||
slot->total_us += elapsed_us;
|
||||
if (elapsed_us > slot->max_us) slot->max_us = elapsed_us;
|
||||
|
||||
if (elapsed_us > WASM_FRAME_BUDGET_US) {
|
||||
slot->budget_faults++;
|
||||
if (slot->budget_faults >= 10) {
|
||||
slot->state = WASM_MODULE_ERROR; // auto-stop
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The budget is configurable via `WASM_FRAME_BUDGET_US` (Kconfig or NVS override).
|
||||
|
||||
### A.3 Per-Module Telemetry
|
||||
|
||||
The `/wasm/list` endpoint and `wasm_module_info_t` struct expose per-module
|
||||
telemetry:
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `frame_count` | u32 | Total on_frame calls since start |
|
||||
| `event_count` | u32 | Total csi_emit_event calls |
|
||||
| `error_count` | u32 | WASM3 runtime errors |
|
||||
| `total_us` | u32 | Cumulative execution time (microseconds) |
|
||||
| `max_us` | u32 | Worst-case single frame execution time |
|
||||
| `budget_faults` | u32 | Times frame budget was exceeded |
|
||||
|
||||
Mean execution time = `total_us / frame_count`. This enables remote monitoring
|
||||
of module health and performance regression detection.
|
||||
|
||||
### A.4 Secure-by-Default
|
||||
|
||||
`wasm_verify` defaults to **1** in both Kconfig and the NVS fallback path.
|
||||
Uploaded `.wasm` binaries must include a valid Ed25519 signature (same key as
|
||||
OTA firmware). Disable only for lab/dev use via:
|
||||
|
||||
```bash
|
||||
python provision.py --port COM7 --wasm-verify # NVS: wasm_verify=1 (default)
|
||||
# To disable in dev: write wasm_verify=0 to NVS directly
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Appendix B: Adaptive Budget Architecture (Mincut-Driven)
|
||||
|
||||
### B.1 Design Principle
|
||||
|
||||
One control loop turns **sensing into a bounded compute budget**, spends that
|
||||
budget on **sparse or spiking inference**, and exports **only deltas**. The
|
||||
budget is driven by the **mincut eigenvalue gap** (Δλ = λ₂ − λ₁ of the CSI
|
||||
graph Laplacian), which reflects scene complexity: a quiet room has Δλ ≈ 0,
|
||||
a busy room has large Δλ.
|
||||
|
||||
### B.2 Control Loop
|
||||
|
||||
```
|
||||
┌─────────────────────────────────┐
|
||||
CSI frames ───→ │ Tier 2 DSP (existing) │
|
||||
│ Welford stats, top-K, presence │
|
||||
└──────────┬────────────────────────┘
|
||||
│
|
||||
┌──────────────▼──────────────────────┐
|
||||
│ Budget Controller │
|
||||
│ │
|
||||
│ Inputs: │
|
||||
│ Δλ = mincut eigenvalue gap │
|
||||
│ A = anomaly_score (adversarial) │
|
||||
│ T = thermal_pressure (0.0-1.0) │
|
||||
│ P = battery_pressure (0.0-1.0) │
|
||||
│ │
|
||||
│ Output: │
|
||||
│ B = frame compute budget (μs) │
|
||||
│ │
|
||||
│ B = clamp(B₀ + k₁·max(0,Δλ) │
|
||||
│ + k₂·A │
|
||||
│ − k₃·T │
|
||||
│ − k₄·P, │
|
||||
│ B_min, B_max) │
|
||||
└──────────────┬──────────────────────┘
|
||||
│
|
||||
┌──────────────▼──────────────────────┐
|
||||
│ WASM Module Dispatch │
|
||||
│ Budget B split across active modules│
|
||||
│ Each module gets B/N μs per frame │
|
||||
└──────────────┬──────────────────────┘
|
||||
│
|
||||
┌──────────────▼──────────────────────┐
|
||||
│ Delta Export │
|
||||
│ Only emit events when Δ > threshold │
|
||||
│ Quiet room → near-zero UDP traffic │
|
||||
└─────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### B.3 Budget Formula
|
||||
|
||||
```
|
||||
B = clamp(B₀ + k₁·max(0, Δλ) + k₂·A − k₃·T − k₄·P, B_min, B_max)
|
||||
```
|
||||
|
||||
| Symbol | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| B₀ | 5,000 μs | Base budget (5 ms) |
|
||||
| k₁ | 2,000 | Δλ sensitivity (more scene change → more budget) |
|
||||
| k₂ | 3,000 | Anomaly boost (detected anomaly → more compute) |
|
||||
| k₃ | 4,000 | Thermal penalty (chip hot → less compute) |
|
||||
| k₄ | 3,000 | Battery penalty (low SoC → less compute) |
|
||||
| B_min | 1,000 μs | Floor: always run at least 1 ms |
|
||||
| B_max | 15,000 μs | Ceiling: never exceed 15 ms |
|
||||
|
||||
### B.4 Where Δλ Comes From
|
||||
|
||||
The mincut graph is the **top-K subcarrier correlation graph** already
|
||||
maintained by Tier 1/2 DSP. Subcarriers are nodes; edge weights are
|
||||
pairwise Pearson correlation magnitudes over the Welford window. The
|
||||
algebraic connectivity (Fiedler value λ₂) of this graph's Laplacian
|
||||
approximates the mincut value. On ESP32-S3 with K=8 subcarriers, this
|
||||
is an 8×8 eigenvalue problem — solvable with power iteration in <100 μs.
|
||||
|
||||
### B.5 Spiking and Sparse Optimizations
|
||||
|
||||
When the budget is tight (Δλ ≈ 0, quiet room), WASM modules should:
|
||||
|
||||
1. **Skip on_frame entirely** if Δλ < ε (no scene change → no computation)
|
||||
2. **Sparse inference**: Only process the top-K subcarriers that changed
|
||||
(already tracked by Tier 1 delta compression)
|
||||
3. **Spiking semantics**: Modules emit events only when state transitions
|
||||
occur, not on every frame. The host tracks a per-module "last emitted"
|
||||
state and suppresses duplicate events.
|
||||
|
||||
### B.6 Thermal and Power Hooks
|
||||
|
||||
ESP32-S3 provides:
|
||||
- `temp_sensor_read()` — on-chip temperature (°C)
|
||||
- ADC reading of battery voltage (if wired)
|
||||
|
||||
Thermal pressure: `T = clamp((temp_celsius - 60) / 20, 0, 1)` — ramps
|
||||
from 0 at 60°C to 1.0 at 80°C (thermal throttle zone).
|
||||
|
||||
Battery pressure: `P = clamp((3.3 - battery_volts) / 0.6, 0, 1)` — ramps
|
||||
from 0 at 3.3V to 1.0 at 2.7V (brownout zone).
|
||||
|
||||
### B.7 Transport Strategy
|
||||
|
||||
WASM output packets (`0xC5110004`) adopt **delta-only export**:
|
||||
|
||||
- Events are only emitted when the value changes by more than a
|
||||
configurable dead-band (default: 5% of previous value)
|
||||
- Quiet room = zero WASM UDP packets (only Tier 2 vitals at 1 Hz)
|
||||
- Busy room = bursty WASM events, naturally rate-limited by budget B
|
||||
|
||||
Future work: QUIC-lite transport with 0-RTT connection resumption and
|
||||
congestion-aware pacing, replacing raw UDP for WASM event streams.
|
||||
|
||||
---
|
||||
|
||||
## Appendix C: Hardware Benchmark (RuView ESP32-S3)
|
||||
|
||||
Measured on ESP32-S3 (QFN56 rev v0.2, 8 MB flash, 160 MHz, ESP-IDF v5.2,
|
||||
board without PSRAM). WiFi connected to AP at RSSI -25 dBm, channel 5 BW20.
|
||||
|
||||
### WASM Runtime Performance
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| WASM runtime init | **106 ms** |
|
||||
| Total boot to ready | **3.9 s** (including WiFi connect) |
|
||||
| Module slots | 4 × 160 KB (heap fallback, no PSRAM) |
|
||||
| WASM binary size (7 modules) | **13.8 KB** (wasm32-unknown-unknown release) |
|
||||
| Frame budget | 10,000 µs (10 ms) |
|
||||
| Timer interval | 1,000 ms (1 Hz) |
|
||||
|
||||
### CSI Throughput
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Frame rate | **28.5 Hz** (exceeds 20 Hz estimate) |
|
||||
| Frame sizes | 128 / 256 bytes |
|
||||
| Per-frame interval | 30.6 ms avg |
|
||||
| RSSI range | -83 to -32 dBm (mean -62 dBm) |
|
||||
|
||||
### Rust Test Results
|
||||
|
||||
| Crate | Tests | Status |
|
||||
|-------|-------|--------|
|
||||
| wifi-densepose-wasm-edge (std) | 14 | All pass, 0 warnings |
|
||||
| Full workspace | 1,411 | All pass, 0 failed |
|
||||
|
||||
### Known Issues
|
||||
|
||||
1. **Fall threshold too sensitive** — default 2.0 rad/s² produces 6.7 false positives/s in static environment. Recommend 5.0-8.0 for deployment.
|
||||
2. **No PSRAM on test board** — WASM arenas fall back to internal heap (316 KiB total). Production boards with 8 MB PSRAM will use dedicated PSRAM arenas.
|
||||
3. **WiFi-Ethernet isolation** — some consumer routers block bridging between WiFi and wired clients. Verify network path during deployment.
|
||||
|
||||
### B.8 Implementation Plan
|
||||
|
||||
| Step | Scope | Effort |
|
||||
|------|-------|--------|
|
||||
| 1 | Add `edge_compute_fiedler()` in `edge_processing.c` — power iteration on 8×8 Laplacian | ~50 lines C |
|
||||
| 2 | Add budget controller struct and update formula in `wasm_runtime.c` | ~30 lines C |
|
||||
| 3 | Wire thermal/battery sensors into budget inputs | ~20 lines C |
|
||||
| 4 | Add delta-export dead-band filter in `wasm_runtime_on_frame()` | ~15 lines C |
|
||||
| 5 | NVS keys for k₁-k₄, B_min, B_max, dead-band threshold | ~10 lines C |
|
||||
|
||||
Total: ~125 lines of C, no new files. All constants configurable via NVS.
|
||||
|
||||
### B.9 Failure Modes
|
||||
|
||||
| Failure | Behavior |
|
||||
|---------|----------|
|
||||
| Δλ estimate wrong (correlation noise) | Budget oscillates — clamped by B_min/B_max |
|
||||
| Thermal sensor absent | T defaults to 0 (no throttle) |
|
||||
| Battery ADC not wired | P defaults to 0 (always-on mode) |
|
||||
| All WASM modules budget-faulted | DSP pipeline runs Tier 2 only — graceful degradation |
|
||||
|
||||
---
|
||||
|
||||
## Appendix C: RVF Container Format
|
||||
|
||||
### C.1 Problem
|
||||
|
||||
Raw `.wasm` uploads over HTTP are remote code execution. Signatures solve
|
||||
authenticity, but without a manifest the host has no way to enforce budgets,
|
||||
check API compatibility, or identify what it's running. RVF wraps the WASM
|
||||
payload with governance metadata in a single artifact.
|
||||
|
||||
### C.2 Binary Layout
|
||||
|
||||
```
|
||||
Offset Size Type Field
|
||||
────────────────────────────────────────────
|
||||
0 4 [u8;4] Magic "RVF\x01" (0x01465652 LE)
|
||||
4 2 u16 LE format_version (1)
|
||||
6 2 u16 LE flags (bit 0: has_signature, bit 1: has_test_vectors)
|
||||
8 4 u32 LE manifest_len (always 96)
|
||||
12 4 u32 LE wasm_len
|
||||
16 4 u32 LE signature_len (0 or 64)
|
||||
20 4 u32 LE test_vectors_len (0 if none)
|
||||
24 4 u32 LE total_len (header + manifest + wasm + sig + tvec)
|
||||
28 4 u32 LE reserved (0)
|
||||
────────────────────────────────────────────
|
||||
32 96 struct Manifest (see below)
|
||||
128 N bytes WASM payload ("\0asm" magic)
|
||||
128+N 0|64 bytes Ed25519 signature (signs bytes 0..128+N-1)
|
||||
128+N+S M bytes Test vectors (optional)
|
||||
```
|
||||
|
||||
Total overhead: 32 (header) + 96 (manifest) + 64 (signature) = **192 bytes**.
|
||||
|
||||
### C.3 Manifest (96 bytes, packed)
|
||||
|
||||
| Offset | Size | Type | Field |
|
||||
|--------|------|------|-------|
|
||||
| 0 | 32 | char[] | `module_name` — null-terminated ASCII |
|
||||
| 32 | 2 | u16 | `required_host_api` — version (1 = current) |
|
||||
| 34 | 4 | u32 | `capabilities` — RVF_CAP_* bitmask |
|
||||
| 38 | 4 | u32 | `max_frame_us` — requested per-frame budget (0 = use default) |
|
||||
| 42 | 2 | u16 | `max_events_per_sec` — rate limit (0 = unlimited) |
|
||||
| 44 | 2 | u16 | `memory_limit_kb` — max WASM heap (0 = use default) |
|
||||
| 46 | 2 | u16 | `event_schema_version` — for receiver compatibility |
|
||||
| 48 | 32 | [u8;32] | `build_hash` — SHA-256 of WASM payload |
|
||||
| 80 | 2 | u16 | `min_subcarriers` — minimum required (0 = any) |
|
||||
| 82 | 2 | u16 | `max_subcarriers` — maximum expected (0 = any) |
|
||||
| 84 | 10 | char[] | `author` — null-padded ASCII |
|
||||
| 94 | 2 | [u8;2] | reserved (0) |
|
||||
|
||||
### C.4 Capability Bitmask
|
||||
|
||||
| Bit | Flag | Host API functions |
|
||||
|-----|------|--------------------|
|
||||
| 0 | `READ_PHASE` | `csi_get_phase` |
|
||||
| 1 | `READ_AMPLITUDE` | `csi_get_amplitude` |
|
||||
| 2 | `READ_VARIANCE` | `csi_get_variance` |
|
||||
| 3 | `READ_VITALS` | `csi_get_bpm_*`, `csi_get_presence`, `csi_get_n_persons` |
|
||||
| 4 | `READ_HISTORY` | `csi_get_phase_history` |
|
||||
| 5 | `EMIT_EVENTS` | `csi_emit_event` |
|
||||
| 6 | `LOG` | `csi_log` |
|
||||
|
||||
Modules declare which host APIs they need. Future firmware versions may
|
||||
refuse to link imports that aren't declared in capabilities — defense in
|
||||
depth against supply-chain attacks.
|
||||
|
||||
### C.5 On-Device Flow
|
||||
|
||||
```
|
||||
HTTP POST /wasm/upload
|
||||
│
|
||||
▼
|
||||
┌────────────────────────┐
|
||||
│ Check first 4 bytes │
|
||||
│ "RVF\x01" → RVF path │
|
||||
│ "\0asm" → raw path │
|
||||
└───────┬────────────────┘
|
||||
│
|
||||
┌────▼────┐ ┌───────────┐
|
||||
│ RVF │ │ Raw WASM │
|
||||
│ parse │ │ (dev only,│
|
||||
│ header │ │ verify=0) │
|
||||
└────┬────┘ └─────┬─────┘
|
||||
│ │
|
||||
┌────▼────┐ │
|
||||
│ Verify │ │
|
||||
│ SHA-256 │ │
|
||||
│ hash │ │
|
||||
└────┬────┘ │
|
||||
│ │
|
||||
┌────▼────┐ │
|
||||
│ Verify │ │
|
||||
│ Ed25519 │ │
|
||||
│ sig │ │
|
||||
└────┬────┘ │
|
||||
│ │
|
||||
┌────▼────┐ │
|
||||
│ Check │ │
|
||||
│ host API│ │
|
||||
│ version │ │
|
||||
└────┬────┘ │
|
||||
│ │
|
||||
├────────────────┘
|
||||
▼
|
||||
┌───────────────────┐
|
||||
│ wasm_runtime_load │
|
||||
│ set_manifest │
|
||||
│ start module │
|
||||
└───────────────────┘
|
||||
```
|
||||
|
||||
### C.6 Rollback Support
|
||||
|
||||
Each slot stores the SHA-256 build hash from the manifest. The `/wasm/list`
|
||||
endpoint returns this hash. Fleet management systems can:
|
||||
|
||||
1. Push an RVF to a node
|
||||
2. Verify the installed hash matches via GET `/wasm/list`
|
||||
3. Roll back by pushing the previous RVF (same slot reused after unload)
|
||||
|
||||
Two-slot strategy: maintain slot 0 as "last known good" and slot 1 as
|
||||
"candidate". Promote by stopping slot 0 and starting slot 1.
|
||||
|
||||
### C.7 Rust Builder
|
||||
|
||||
The `wifi-densepose-wasm-edge` crate provides `rvf::builder::build_rvf()`
|
||||
(behind the `std` feature) to package a `.wasm` binary into an `.rvf`:
|
||||
|
||||
```rust
|
||||
use wifi_densepose_wasm_edge::rvf::builder::{build_rvf, RvfConfig};
|
||||
|
||||
let wasm = std::fs::read("target/wasm32-unknown-unknown/release/module.wasm")?;
|
||||
let rvf = build_rvf(&wasm, &RvfConfig {
|
||||
module_name: "gesture".into(),
|
||||
author: "rUv".into(),
|
||||
capabilities: CAP_READ_PHASE | CAP_EMIT_EVENTS,
|
||||
max_frame_us: 5000,
|
||||
..Default::default()
|
||||
});
|
||||
std::fs::write("gesture.rvf", &rvf)?;
|
||||
// Then sign externally with Ed25519 and patch_signature()
|
||||
```
|
||||
|
||||
### C.8 Implementation Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `firmware/.../main/rvf_parser.h` | RVF types, capability flags, parse/verify API |
|
||||
| `firmware/.../main/rvf_parser.c` | Header/manifest parser, SHA-256 hash check |
|
||||
| `wifi-densepose-wasm-edge/src/rvf.rs` | Format constants, builder (std), tests |
|
||||
|
||||
### C.9 Failure Modes
|
||||
|
||||
| Failure | Behavior |
|
||||
|---------|----------|
|
||||
| RVF too large for PSRAM buffer | Rejected at receive with 400 |
|
||||
| Build hash mismatch | Rejected at parse with `ESP_ERR_INVALID_CRC` |
|
||||
| Signature absent when `wasm_verify=1` | Rejected with 403 |
|
||||
| Host API version too new | Rejected with `ESP_ERR_NOT_SUPPORTED` |
|
||||
| Raw WASM when `wasm_verify=1` | Rejected with 403 |
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user