mirror of
https://github.com/ruvnet/RuView
synced 2026-07-31 18:51:42 +00:00
feat: complete vendor repos, add edge intelligence and WASM modules
- Add 154 missing vendor files (gitignore was filtering them) - vendor/midstream: 564 files (was 561) - vendor/sublinear-time-solver: 1190 files (was 1039) - Add ESP32 edge processing (ADR-039): presence, vitals, fall detection - Add WASM programmable sensing (ADR-040/041) with wasm3 runtime - Add firmware CI workflow (.github/workflows/firmware-ci.yml) - Add wifi-densepose-wasm-edge crate for edge WASM modules - Update sensing server, provision.py, UI components Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
ruv
@@ -1,126 +1,158 @@
|
||||
# ESP32-S3 CSI Node Firmware (ADR-018)
|
||||
# ESP32-S3 CSI Node Firmware
|
||||
|
||||
Firmware for ESP32-S3 that collects WiFi Channel State Information (CSI)
|
||||
and streams it as ADR-018 binary frames over UDP to the aggregator.
|
||||
**Turn a $7 microcontroller into a privacy-first human sensing node.**
|
||||
|
||||
Verified working with ESP32-S3-DevKitC-1 (CP2102, MAC 3C:0F:02:EC:C2:28)
|
||||
streaming ~20 Hz CSI to the Rust aggregator binary.
|
||||
This firmware captures WiFi Channel State Information (CSI) from an ESP32-S3 and transforms it into real-time presence detection, vital sign monitoring, and programmable sensing -- all without cameras or wearables. Part of the [WiFi-DensePose](../../README.md) project.
|
||||
|
||||
## Prerequisites
|
||||
[](https://docs.espressif.com/projects/esp-idf/en/v5.2/)
|
||||
[](https://www.espressif.com/en/products/socs/esp32-s3)
|
||||
[](../../LICENSE)
|
||||
[](#memory-budget)
|
||||
[](../../.github/workflows/firmware-ci.yml)
|
||||
|
||||
| Component | Version | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| Docker Desktop | 28.x+ | Cross-compile ESP-IDF firmware |
|
||||
| esptool | 5.x+ | Flash firmware to ESP32 |
|
||||
| ESP32-S3 board | - | Hardware (DevKitC-1 or similar) |
|
||||
| USB-UART driver | CP210x | Silicon Labs driver for serial |
|
||||
> | Capability | Method | Performance |
|
||||
> |------------|--------|-------------|
|
||||
> | **CSI streaming** | Per-subcarrier I/Q capture over UDP | ~20 Hz, ADR-018 binary format |
|
||||
> | **Breathing detection** | Bandpass 0.1-0.5 Hz, zero-crossing BPM | 6-30 BPM |
|
||||
> | **Heart rate** | Bandpass 0.8-2.0 Hz, zero-crossing BPM | 40-120 BPM |
|
||||
> | **Presence sensing** | Phase variance + adaptive calibration | < 1 ms latency |
|
||||
> | **Fall detection** | Phase acceleration threshold | Configurable sensitivity |
|
||||
> | **Programmable sensing** | WASM modules loaded over HTTP | Hot-swap, no reflash |
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Step 1: Configure WiFi credentials
|
||||
For users who want to get running fast. Detailed explanations follow in later sections.
|
||||
|
||||
Create `sdkconfig.defaults` in this directory (it is gitignored):
|
||||
|
||||
```
|
||||
CONFIG_IDF_TARGET="esp32s3"
|
||||
CONFIG_ESP_WIFI_CSI_ENABLED=y
|
||||
CONFIG_CSI_NODE_ID=1
|
||||
CONFIG_CSI_WIFI_SSID="YOUR_WIFI_SSID"
|
||||
CONFIG_CSI_WIFI_PASSWORD="YOUR_WIFI_PASSWORD"
|
||||
CONFIG_CSI_TARGET_IP="192.168.1.20"
|
||||
CONFIG_CSI_TARGET_PORT=5005
|
||||
CONFIG_ESPTOOLPY_FLASHSIZE_4MB=y
|
||||
```
|
||||
|
||||
Replace `YOUR_WIFI_SSID`, `YOUR_WIFI_PASSWORD`, and `CONFIG_CSI_TARGET_IP`
|
||||
with your actual values. The target IP is the machine running the aggregator.
|
||||
|
||||
### Step 2: Build with Docker
|
||||
### 1. Build (Docker -- the only reliable method)
|
||||
|
||||
```bash
|
||||
cd firmware/esp32-csi-node
|
||||
|
||||
# On Linux/macOS:
|
||||
docker run --rm -v "$(pwd):/project" -w /project \
|
||||
espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build"
|
||||
|
||||
# On Windows (Git Bash — MSYS path fix required):
|
||||
MSYS_NO_PATHCONV=1 docker run --rm -v "$(pwd -W)://project" -w //project \
|
||||
espressif/idf:v5.2 bash -c "idf.py set-target esp32s3 && idf.py build"
|
||||
# From the repository root:
|
||||
MSYS_NO_PATHCONV=1 docker run --rm \
|
||||
-v "$(pwd)/firmware/esp32-csi-node:/project" -w /project \
|
||||
espressif/idf:v5.2 bash -c \
|
||||
"rm -rf build sdkconfig && idf.py set-target esp32s3 && idf.py build"
|
||||
```
|
||||
|
||||
Build output: `build/bootloader.bin`, `build/partition_table/partition-table.bin`,
|
||||
`build/esp32-csi-node.bin`.
|
||||
|
||||
### Step 3: Flash to ESP32-S3
|
||||
|
||||
Find your serial port (`COM7` on Windows, `/dev/ttyUSB0` on Linux):
|
||||
### 2. Flash
|
||||
|
||||
```bash
|
||||
cd firmware/esp32-csi-node/build
|
||||
|
||||
python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
|
||||
--before default-reset --after hard-reset \
|
||||
write-flash --flash-mode dio --flash-freq 80m --flash-size 4MB \
|
||||
0x0 bootloader/bootloader.bin \
|
||||
0x8000 partition_table/partition-table.bin \
|
||||
0x10000 esp32-csi-node.bin
|
||||
write_flash --flash_mode dio --flash_size 8MB \
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0x10000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
```
|
||||
|
||||
### Step 4: Run the aggregator
|
||||
### 3. Provision WiFi credentials (no reflash needed)
|
||||
|
||||
```bash
|
||||
cargo run -p wifi-densepose-hardware --bin aggregator -- --bind 0.0.0.0:5005 --verbose
|
||||
python scripts/provision.py --port COM7 \
|
||||
--ssid "YourSSID" --password "YourPass" --target-ip 192.168.1.20
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Listening on 0.0.0.0:5005...
|
||||
[148 bytes from 192.168.1.71:60764]
|
||||
[node:1 seq:0] sc=64 rssi=-49 amp=9.5
|
||||
[276 bytes from 192.168.1.71:60764]
|
||||
[node:1 seq:1] sc=128 rssi=-64 amp=16.0
|
||||
### 4. Start the sensing server
|
||||
|
||||
```bash
|
||||
cargo run -p wifi-densepose-sensing-server -- --http-port 3000 --source auto
|
||||
```
|
||||
|
||||
### Step 5: Verify presence detection
|
||||
### 5. Open the UI
|
||||
|
||||
If you see frames streaming (~20/sec), the system is working. Walk near the
|
||||
ESP32 and observe amplitude variance changes in the CSI data.
|
||||
Navigate to [http://localhost:3000](http://localhost:3000) in your browser.
|
||||
|
||||
## Configuration Reference
|
||||
### 6. (Optional) Upload a WASM sensing module
|
||||
|
||||
Edit via `idf.py menuconfig` or `sdkconfig.defaults`:
|
||||
|
||||
| Setting | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `CSI_NODE_ID` | 1 | Unique node identifier (0-255) |
|
||||
| `CSI_TARGET_IP` | 192.168.1.100 | Aggregator host IP |
|
||||
| `CSI_TARGET_PORT` | 5005 | Aggregator UDP port |
|
||||
| `CSI_WIFI_SSID` | wifi-densepose | WiFi network SSID |
|
||||
| `CSI_WIFI_PASSWORD` | (empty) | WiFi password |
|
||||
| `CSI_WIFI_CHANNEL` | 6 | WiFi channel to monitor |
|
||||
|
||||
## Firewall Note
|
||||
|
||||
On Windows, you may need to allow inbound UDP on port 5005:
|
||||
|
||||
```
|
||||
netsh advfirewall firewall add rule name="ESP32 CSI" dir=in action=allow protocol=UDP localport=5005
|
||||
```bash
|
||||
curl -X POST http://<ESP32_IP>:8032/wasm/upload --data-binary @gesture.rvf
|
||||
curl http://<ESP32_IP>:8032/wasm/list
|
||||
```
|
||||
|
||||
## Architecture
|
||||
---
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
| Component | Specification | Notes |
|
||||
|-----------|---------------|-------|
|
||||
| **SoC** | ESP32-S3 (QFN56) | Dual-core Xtensa LX7, 240 MHz |
|
||||
| **Flash** | 8 MB | ~943 KB used by firmware |
|
||||
| **PSRAM** | 8 MB | 640 KB used for WASM arenas |
|
||||
| **USB bridge** | Silicon Labs CP210x | Install the [CP210x driver](https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers) |
|
||||
| **Recommended boards** | ESP32-S3-DevKitC-1, XIAO ESP32-S3 | Any ESP32-S3 with 8 MB flash works |
|
||||
| **Deployment** | 3-6 nodes per room | Multistatic mesh for 360-degree coverage |
|
||||
|
||||
> **Tip:** A single node provides presence and vital signs along its line of sight. Multiple nodes (3-6) create a multistatic mesh that resolves 3D pose with <30 mm jitter and zero identity swaps.
|
||||
|
||||
---
|
||||
|
||||
## Firmware Architecture
|
||||
|
||||
The firmware implements a tiered processing pipeline. Each tier builds on the previous one. The active tier is selectable at compile time (Kconfig) or at runtime (NVS) without reflashing.
|
||||
|
||||
```
|
||||
ESP32-S3 Host Machine
|
||||
+-------------------+ +-------------------+
|
||||
| WiFi CSI callback | UDP/5005 | aggregator binary |
|
||||
| (promiscuous mode)| ──────────> | (Rust, clap CLI) |
|
||||
| ADR-018 serialize | ADR-018 | Esp32CsiParser |
|
||||
| stream_sender.c | binary frames | CsiFrame output |
|
||||
+-------------------+ +-------------------+
|
||||
ESP32-S3 CSI Node
|
||||
+--------------------------------------------------------------------------+
|
||||
| Core 0 (WiFi) | Core 1 (DSP) |
|
||||
| | |
|
||||
| WiFi STA + CSI callback | SPSC ring buffer consumer |
|
||||
| Channel hopping (ADR-029) | Tier 0: Raw passthrough |
|
||||
| NDP injection | Tier 1: Phase unwrap, Welford, top-K |
|
||||
| TDM slot management | Tier 2: Vitals, presence, fall detect |
|
||||
| | Tier 3: WASM module dispatch |
|
||||
+--------------------------------------------------------------------------+
|
||||
| NVS config | OTA server (8032) | UDP sender | Power management |
|
||||
+--------------------------------------------------------------------------+
|
||||
```
|
||||
|
||||
## Binary Frame Format (ADR-018)
|
||||
### Tier 0 -- Raw CSI Passthrough (Stable)
|
||||
|
||||
The default, production-stable baseline. Captures CSI frames from the WiFi driver and streams them over UDP in the ADR-018 binary format.
|
||||
|
||||
- **Magic:** `0xC5110001`
|
||||
- **Rate:** ~20 Hz per channel
|
||||
- **Payload:** 20-byte header + I/Q pairs (2 bytes per subcarrier per antenna)
|
||||
- **Bandwidth:** ~5 KB/s per node (64 subcarriers, 1 antenna)
|
||||
|
||||
### Tier 1 -- Basic DSP (Stable)
|
||||
|
||||
Adds on-device signal conditioning to reduce bandwidth and improve signal quality.
|
||||
|
||||
- **Phase unwrapping** -- removes 2-pi discontinuities
|
||||
- **Welford running statistics** -- incremental mean and variance per subcarrier
|
||||
- **Top-K subcarrier selection** -- tracks only the K highest-variance subcarriers
|
||||
- **Delta compression** -- XOR + RLE encoding reduces bandwidth by ~70%
|
||||
|
||||
### Tier 2 -- Full Pipeline (Stable)
|
||||
|
||||
Adds real-time health and safety monitoring.
|
||||
|
||||
- **Breathing rate** -- biquad IIR bandpass 0.1-0.5 Hz, zero-crossing BPM (6-30 BPM)
|
||||
- **Heart rate** -- biquad IIR bandpass 0.8-2.0 Hz, zero-crossing BPM (40-120 BPM)
|
||||
- **Presence detection** -- adaptive threshold calibration (60 s ambient learning)
|
||||
- **Fall detection** -- phase acceleration exceeds configurable threshold
|
||||
- **Multi-person estimation** -- subcarrier group clustering (up to 4 persons)
|
||||
- **Vitals packet** -- 32-byte UDP packet at 1 Hz (magic `0xC5110002`)
|
||||
|
||||
### Tier 3 -- WASM Programmable Sensing (Alpha)
|
||||
|
||||
Turns the ESP32 from a fixed-function sensor into a programmable sensing computer. Instead of reflashing firmware to change algorithms, you upload new sensing logic as small WASM modules -- compiled from Rust, packaged in signed RVF containers.
|
||||
|
||||
See the [WASM Programmable Sensing](#wasm-programmable-sensing-tier-3) section for full details.
|
||||
|
||||
---
|
||||
|
||||
## Wire Protocols
|
||||
|
||||
All packets are sent over UDP to the configured aggregator. The magic number in the first 4 bytes identifies the packet type.
|
||||
|
||||
| Magic | Name | Rate | Size | Contents |
|
||||
|-------|------|------|------|----------|
|
||||
| `0xC5110001` | CSI Frame (ADR-018) | ~20 Hz | Variable | Raw I/Q per subcarrier per antenna |
|
||||
| `0xC5110002` | Vitals Packet | 1 Hz | 32 bytes | Presence, breathing BPM, heart rate, fall flag, occupancy |
|
||||
| `0xC5110004` | WASM Output | Event-driven | Variable | Custom events from WASM modules (u8 type + f32 value) |
|
||||
|
||||
### ADR-018 Binary Frame Format
|
||||
|
||||
```
|
||||
Offset Size Field
|
||||
@@ -136,12 +168,397 @@ Offset Size Field
|
||||
20 N*2 I/Q pairs (n_antennas * n_subcarriers * 2 bytes)
|
||||
```
|
||||
|
||||
### Vitals Packet (32 bytes)
|
||||
|
||||
```
|
||||
Offset Size Field
|
||||
0 4 Magic: 0xC5110002
|
||||
4 1 Node ID
|
||||
5 1 Flags (bit0=presence, bit1=fall, bit2=motion)
|
||||
6 2 Breathing rate (BPM * 100, fixed-point)
|
||||
8 4 Heart rate (BPM * 10000, fixed-point)
|
||||
12 1 RSSI (i8)
|
||||
13 1 Number of detected persons
|
||||
14 2 Reserved
|
||||
16 4 Motion energy (f32)
|
||||
20 4 Presence score (f32)
|
||||
24 4 Timestamp (ms since boot)
|
||||
28 4 Reserved
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Building
|
||||
|
||||
### Prerequisites
|
||||
|
||||
| Component | Version | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| Docker Desktop | 28.x+ | Cross-compile firmware in ESP-IDF container |
|
||||
| esptool | 5.x+ | Flash firmware to ESP32 (`pip install esptool`) |
|
||||
| Python 3.10+ | 3.10+ | Provisioning script, serial monitor |
|
||||
| ESP32-S3 board | -- | Target hardware |
|
||||
| CP210x driver | -- | USB-UART bridge driver ([download](https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers)) |
|
||||
|
||||
> **Why Docker?** ESP-IDF does NOT work from Git Bash/MSYS2 on Windows. The `idf.py` script detects the `MSYSTEM` environment variable and skips `main()`. Even removing `MSYSTEM`, the `cmd.exe` subprocess injects `doskey` aliases that break the ninja linker. Docker is the only reliable cross-platform build method.
|
||||
|
||||
### Build Command
|
||||
|
||||
```bash
|
||||
# From the repository root:
|
||||
MSYS_NO_PATHCONV=1 docker run --rm \
|
||||
-v "$(pwd)/firmware/esp32-csi-node:/project" -w /project \
|
||||
espressif/idf:v5.2 bash -c \
|
||||
"rm -rf build sdkconfig && idf.py set-target esp32s3 && idf.py build"
|
||||
```
|
||||
|
||||
The `MSYS_NO_PATHCONV=1` prefix prevents Git Bash from mangling the `/project` path to `C:/Program Files/Git/project`.
|
||||
|
||||
**Build output:**
|
||||
- `build/bootloader/bootloader.bin` -- second-stage bootloader
|
||||
- `build/partition_table/partition-table.bin` -- flash partition layout
|
||||
- `build/esp32-csi-node.bin` -- application firmware
|
||||
|
||||
### Custom Configuration
|
||||
|
||||
To change Kconfig settings before building:
|
||||
|
||||
```bash
|
||||
MSYS_NO_PATHCONV=1 docker run --rm -it \
|
||||
-v "$(pwd)/firmware/esp32-csi-node:/project" -w /project \
|
||||
espressif/idf:v5.2 bash -c \
|
||||
"idf.py set-target esp32s3 && idf.py menuconfig"
|
||||
```
|
||||
|
||||
Or create/edit `sdkconfig.defaults` before building:
|
||||
|
||||
```ini
|
||||
CONFIG_IDF_TARGET="esp32s3"
|
||||
CONFIG_ESP_WIFI_CSI_ENABLED=y
|
||||
CONFIG_CSI_NODE_ID=1
|
||||
CONFIG_CSI_WIFI_SSID="wifi-densepose"
|
||||
CONFIG_CSI_WIFI_PASSWORD=""
|
||||
CONFIG_CSI_TARGET_IP="192.168.1.100"
|
||||
CONFIG_CSI_TARGET_PORT=5005
|
||||
CONFIG_EDGE_TIER=2
|
||||
CONFIG_WASM_MAX_MODULES=4
|
||||
CONFIG_WASM_VERIFY_SIGNATURE=y
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Flashing
|
||||
|
||||
Find your serial port: `COM7` on Windows, `/dev/ttyUSB0` on Linux, `/dev/cu.SLAB_USBtoUART` on macOS.
|
||||
|
||||
```bash
|
||||
python -m esptool --chip esp32s3 --port COM7 --baud 460800 \
|
||||
write_flash --flash_mode dio --flash_size 8MB \
|
||||
0x0 firmware/esp32-csi-node/build/bootloader/bootloader.bin \
|
||||
0x8000 firmware/esp32-csi-node/build/partition_table/partition-table.bin \
|
||||
0x10000 firmware/esp32-csi-node/build/esp32-csi-node.bin
|
||||
```
|
||||
|
||||
### Serial Monitor
|
||||
|
||||
```bash
|
||||
python -m serial.tools.miniterm COM7 115200
|
||||
```
|
||||
|
||||
Expected output after boot:
|
||||
|
||||
```
|
||||
I (321) main: ESP32-S3 CSI Node (ADR-018) -- Node ID: 1
|
||||
I (345) main: WiFi STA initialized, connecting to SSID: wifi-densepose
|
||||
I (1023) main: Connected to WiFi
|
||||
I (1025) main: CSI streaming active -> 192.168.1.100:5005 (edge_tier=2, OTA=ready, WASM=ready)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Runtime Configuration (NVS)
|
||||
|
||||
All settings can be changed at runtime via Non-Volatile Storage (NVS) without reflashing the firmware. NVS values override Kconfig defaults.
|
||||
|
||||
### Provisioning Script
|
||||
|
||||
The easiest way to write NVS settings:
|
||||
|
||||
```bash
|
||||
python scripts/provision.py --port COM7 \
|
||||
--ssid "MyWiFi" \
|
||||
--password "MyPassword" \
|
||||
--target-ip 192.168.1.20
|
||||
```
|
||||
|
||||
### NVS Key Reference
|
||||
|
||||
#### Network Settings
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `ssid` | string | `wifi-densepose` | WiFi SSID |
|
||||
| `password` | string | *(empty)* | WiFi password |
|
||||
| `target_ip` | string | `192.168.1.100` | Aggregator server IP address |
|
||||
| `target_port` | u16 | `5005` | Aggregator UDP port |
|
||||
| `node_id` | u8 | `1` | Unique node identifier (0-255) |
|
||||
|
||||
#### Channel Hopping and TDM (ADR-029)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `hop_count` | u8 | `1` | Number of channels to hop (1 = single-channel mode) |
|
||||
| `chan_list` | blob | `[6]` | WiFi channel numbers for hopping |
|
||||
| `dwell_ms` | u32 | `50` | Dwell time per channel in milliseconds |
|
||||
| `tdm_slot` | u8 | `0` | This node's TDM slot index (0-based) |
|
||||
| `tdm_nodes` | u8 | `1` | Total number of nodes in the TDM schedule |
|
||||
|
||||
#### Edge Intelligence (ADR-039)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `edge_tier` | u8 | `2` | Processing tier: 0=raw, 1=basic DSP, 2=full pipeline |
|
||||
| `pres_thresh` | u16 | *auto* | Presence threshold (x1000). 0 = auto-calibrate from 60 s ambient |
|
||||
| `fall_thresh` | u16 | `2000` | Fall detection threshold (x1000). 2000 = 2.0 rad/s^2 |
|
||||
| `vital_win` | u16 | `256` | Phase history window depth (frames) |
|
||||
| `vital_int` | u16 | `1000` | Vitals packet send interval (ms) |
|
||||
| `subk_count` | u8 | `8` | Top-K subcarrier count for variance tracking |
|
||||
| `power_duty` | u8 | `100` | Power duty cycle percentage (10-100). 100 = always on |
|
||||
|
||||
#### WASM Programmable Sensing (ADR-040)
|
||||
|
||||
| Key | Type | Default | Description |
|
||||
|-----|------|---------|-------------|
|
||||
| `wasm_max` | u8 | `4` | Maximum concurrent WASM module slots (1-8) |
|
||||
| `wasm_verify` | u8 | `1` | Require Ed25519 signature verification for uploads |
|
||||
|
||||
---
|
||||
|
||||
## Kconfig Menus
|
||||
|
||||
Three configuration menus are available via `idf.py menuconfig`:
|
||||
|
||||
### "CSI Node Configuration"
|
||||
|
||||
Basic WiFi and network settings: SSID, password, channel, node ID, aggregator IP/port.
|
||||
|
||||
### "Edge Intelligence (ADR-039)"
|
||||
|
||||
Processing tier selection, vitals interval, top-K subcarrier count, fall detection threshold, power duty cycle.
|
||||
|
||||
### "WASM Programmable Sensing (ADR-040)"
|
||||
|
||||
Maximum module slots, Ed25519 signature verification toggle, timer interval for `on_timer()` callbacks.
|
||||
|
||||
---
|
||||
|
||||
## WASM Programmable Sensing (Tier 3)
|
||||
|
||||
### Overview
|
||||
|
||||
Tier 3 turns the ESP32 from a fixed-function sensor into a programmable sensing computer. Instead of reflashing firmware to change algorithms, you upload new sensing logic as small WASM modules. These modules are:
|
||||
|
||||
- **Compiled from Rust** using the `wasm32-unknown-unknown` target
|
||||
- **Packaged in signed RVF containers** with Ed25519 signatures
|
||||
- **Uploaded over HTTP** to the running device (no physical access needed)
|
||||
- **Executed per-frame** (~20 Hz) by the WASM3 interpreter after Tier 2 DSP completes
|
||||
|
||||
### RVF (RuVector Format)
|
||||
|
||||
RVF is a signed container that wraps a WASM binary with metadata for tamper detection and authenticity.
|
||||
|
||||
```
|
||||
+------------------+-------------------+------------------+------------------+
|
||||
| Header (32 B) | Manifest (96 B) | WASM payload | Ed25519 sig (64B)|
|
||||
+------------------+-------------------+------------------+------------------+
|
||||
```
|
||||
|
||||
**Total overhead:** 192 bytes (32-byte header + 96-byte manifest + 64-byte signature).
|
||||
|
||||
| Field | Size | Contents |
|
||||
|-------|------|----------|
|
||||
| **Header** | 32 bytes | Magic (`RVF\x01`), format version, section sizes, flags |
|
||||
| **Manifest** | 96 bytes | Module name, author, capabilities bitmask, budget request, SHA-256 build hash, event schema version |
|
||||
| **WASM payload** | Variable | The compiled `.wasm` binary (max 128 KB) |
|
||||
| **Signature** | 64 bytes | Ed25519 signature covering header + manifest + WASM |
|
||||
|
||||
### Host API
|
||||
|
||||
WASM modules import functions from the `"csi"` namespace to access sensor data:
|
||||
|
||||
| Function | Signature | Description |
|
||||
|----------|-----------|-------------|
|
||||
| `csi_get_phase` | `(i32) -> f32` | Phase (radians) for subcarrier index |
|
||||
| `csi_get_amplitude` | `(i32) -> f32` | Amplitude for subcarrier index |
|
||||
| `csi_get_variance` | `(i32) -> f32` | Running variance (Welford) for subcarrier |
|
||||
| `csi_get_bpm_breathing` | `() -> f32` | Breathing rate BPM from Tier 2 |
|
||||
| `csi_get_bpm_heartrate` | `() -> f32` | Heart rate BPM from Tier 2 |
|
||||
| `csi_get_presence` | `() -> i32` | Presence flag (0 = empty, 1 = present) |
|
||||
| `csi_get_motion_energy` | `() -> f32` | Motion energy scalar |
|
||||
| `csi_get_n_persons` | `() -> i32` | Number of detected persons |
|
||||
| `csi_get_timestamp` | `() -> i32` | Milliseconds since boot |
|
||||
| `csi_emit_event` | `(i32, f32)` | Emit a typed event to the host (sent over UDP) |
|
||||
| `csi_log` | `(i32, i32)` | Debug log from WASM (pointer + length) |
|
||||
| `csi_get_phase_history` | `(i32, i32) -> i32` | Copy phase ring buffer into WASM memory |
|
||||
|
||||
### Module Lifecycle
|
||||
|
||||
Every WASM module must export these three functions:
|
||||
|
||||
| Export | Called | Purpose |
|
||||
|--------|--------|---------|
|
||||
| `on_init()` | Once, when started | Allocate state, initialize algorithms |
|
||||
| `on_frame(n_subcarriers: i32)` | Per CSI frame (~20 Hz) | Process sensor data, emit events |
|
||||
| `on_timer()` | At configurable interval (default 1 s) | Periodic housekeeping, aggregated output |
|
||||
|
||||
### HTTP Management Endpoints
|
||||
|
||||
All endpoints are served on **port 8032** (shared with the OTA update server).
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/wasm/upload` | Upload an RVF container or raw `.wasm` binary (max 128 KB) |
|
||||
| `GET` | `/wasm/list` | List all module slots with state, telemetry, and RVF metadata |
|
||||
| `POST` | `/wasm/start/:id` | Start a loaded module (calls `on_init`) |
|
||||
| `POST` | `/wasm/stop/:id` | Stop a running module |
|
||||
| `DELETE` | `/wasm/:id` | Unload a module and free its PSRAM arena |
|
||||
|
||||
### Included WASM Modules
|
||||
|
||||
The `wifi-densepose-wasm-edge` Rust crate provides three flagship modules:
|
||||
|
||||
| Module | File | Description |
|
||||
|--------|------|-------------|
|
||||
| **gesture** | `gesture.rs` | DTW template matching for wave, push, pull, and swipe gestures |
|
||||
| **coherence** | `coherence.rs` | Phase phasor coherence monitoring with hysteresis gate |
|
||||
| **adversarial** | `adversarial.rs` | Signal anomaly detection (phase jumps, flatlines, energy spikes) |
|
||||
|
||||
Build all modules:
|
||||
|
||||
```bash
|
||||
cargo build -p wifi-densepose-wasm-edge --target wasm32-unknown-unknown --release
|
||||
```
|
||||
|
||||
### Safety Features
|
||||
|
||||
| Protection | Detail |
|
||||
|------------|--------|
|
||||
| **Memory isolation** | Fixed 160 KB PSRAM arenas per slot (no heap fragmentation) |
|
||||
| **Budget guard** | 10 ms per-frame default; auto-stop after 10 consecutive budget faults |
|
||||
| **Signature verification** | Ed25519 enabled by default; disable with `wasm_verify=0` in NVS for development |
|
||||
| **Hash verification** | SHA-256 of WASM payload checked against RVF manifest |
|
||||
| **Slot limit** | Maximum 4 concurrent module slots (configurable to 8) |
|
||||
| **Per-module telemetry** | Frame count, event count, mean/max execution time, budget faults |
|
||||
|
||||
---
|
||||
|
||||
## Memory Budget
|
||||
|
||||
| Component | SRAM | PSRAM | Flash |
|
||||
|-----------|------|-------|-------|
|
||||
| Base firmware (Tier 0) | ~12 KB | -- | ~820 KB |
|
||||
| Tier 1-2 DSP pipeline | ~10 KB | -- | ~33 KB |
|
||||
| WASM3 interpreter | ~10 KB | -- | ~100 KB |
|
||||
| WASM arenas (x4 slots) | -- | 640 KB | -- |
|
||||
| Host API + HTTP upload | ~3 KB | -- | ~23 KB |
|
||||
| **Total** | **~35 KB** | **640 KB** | **~943 KB** |
|
||||
|
||||
- **PSRAM remaining:** 7.36 MB (available for future use)
|
||||
- **Flash partition:** 1 MB OTA slot (6% headroom at current binary size)
|
||||
- **SRAM remaining:** ~280 KB (FreeRTOS + WiFi stack uses the rest)
|
||||
|
||||
---
|
||||
|
||||
## Source Files
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `main/main.c` | Application entry point: NVS init, WiFi STA, CSI collector, edge pipeline, OTA server, WASM runtime init |
|
||||
| `main/csi_collector.c` / `.h` | WiFi CSI frame capture, ADR-018 binary serialization, channel hopping, NDP injection |
|
||||
| `main/stream_sender.c` / `.h` | UDP socket management and packet transmission to aggregator |
|
||||
| `main/nvs_config.c` / `.h` | Runtime configuration: loads Kconfig defaults, overrides from NVS |
|
||||
| `main/edge_processing.c` / `.h` | Tier 0-2 DSP pipeline: SPSC ring buffer, biquad IIR filters, Welford stats, BPM extraction, presence, fall detection |
|
||||
| `main/ota_update.c` / `.h` | HTTP OTA firmware update server on port 8032 |
|
||||
| `main/power_mgmt.c` / `.h` | Battery-aware light sleep duty cycling |
|
||||
| `main/wasm_runtime.c` / `.h` | WASM3 interpreter: module slots, host API bindings, budget guard, per-frame dispatch |
|
||||
| `main/wasm_upload.c` / `.h` | HTTP endpoints for WASM module upload, list, start, stop, delete |
|
||||
| `main/rvf_parser.c` / `.h` | RVF container parser: header validation, manifest extraction, SHA-256 hash verification |
|
||||
| `components/wasm3/` | WASM3 interpreter library (MIT license, ~100 KB flash, ~10 KB RAM) |
|
||||
|
||||
---
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
ESP32-S3 Node Host Machine
|
||||
+------------------------------------------+ +---------------------------+
|
||||
| Core 0 (WiFi) Core 1 (DSP) | | |
|
||||
| | | |
|
||||
| WiFi STA --------> SPSC Ring Buffer | | |
|
||||
| CSI Callback | | | |
|
||||
| Channel Hop v | | |
|
||||
| NDP Inject +-- Tier 0: Raw ADR-018 ---------> UDP/5005 |
|
||||
| | Tier 1: Phase + Welford | | Sensing Server |
|
||||
| | Tier 2: Vitals + Fall ---------> (vitals) |
|
||||
| | Tier 3: WASM Dispatch ---------> (events) |
|
||||
| + | | | |
|
||||
| NVS Config OTA/WASM HTTP (port 8032) | | v |
|
||||
| Power Mgmt POST /ota | | Web UI (:3000) |
|
||||
| POST /wasm/upload | | Pose + Vitals + Alerts |
|
||||
+------------------------------------------+ +---------------------------+
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CI/CD
|
||||
|
||||
The firmware is continuously verified by [`.github/workflows/firmware-ci.yml`](../../.github/workflows/firmware-ci.yml):
|
||||
|
||||
| Step | Check | Threshold |
|
||||
|------|-------|-----------|
|
||||
| **Docker build** | Full compile with ESP-IDF v5.4 container | Must succeed |
|
||||
| **Binary size gate** | `esp32-csi-node.bin` file size | Must be < 950 KB |
|
||||
| **Flash image integrity** | Partition table magic, bootloader presence, non-padding content | Warnings on failure |
|
||||
| **Artifact upload** | Bootloader + partition table + app binary | 30-day retention |
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| No serial output | Wrong baud rate | Use 115200 |
|
||||
| WiFi won't connect | Wrong SSID/password | Check sdkconfig.defaults |
|
||||
| No UDP frames | Firewall blocking | Add UDP 5005 inbound rule |
|
||||
| CSI callback not firing | Promiscuous mode off | Verify `esp_wifi_set_promiscuous(true)` in csi_collector.c |
|
||||
| Parse errors in aggregator | Firmware/parser mismatch | Rebuild both from same source |
|
||||
| No serial output | Wrong baud rate | Use `115200` in your serial monitor |
|
||||
| WiFi won't connect | Wrong SSID/password | Re-run `provision.py` with correct credentials |
|
||||
| No UDP frames received | Firewall blocking | Allow inbound UDP on port 5005 (see below) |
|
||||
| `idf.py` fails on Windows | Git Bash/MSYS2 incompatibility | Use Docker -- this is the only supported build method on Windows |
|
||||
| CSI callback not firing | Promiscuous mode issue | Verify `esp_wifi_set_promiscuous(true)` in `csi_collector.c` |
|
||||
| WASM upload rejected | Signature verification | Disable with `wasm_verify=0` via NVS for development, or sign with Ed25519 |
|
||||
| High frame drop rate | Ring buffer overflow | Reduce `edge_tier` or increase `dwell_ms` |
|
||||
| Vitals readings unstable | Calibration period | Wait 60 seconds for adaptive threshold to settle |
|
||||
| OTA update fails | Binary too large | Check binary is < 1 MB; current headroom is ~6% |
|
||||
| Docker path error on Windows | MSYS path conversion | Prefix command with `MSYS_NO_PATHCONV=1` |
|
||||
|
||||
### Windows Firewall Rule
|
||||
|
||||
```powershell
|
||||
netsh advfirewall firewall add rule name="ESP32 CSI" dir=in action=allow protocol=UDP localport=5005
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture Decision Records
|
||||
|
||||
This firmware implements or references the following ADRs:
|
||||
|
||||
| ADR | Title | Status |
|
||||
|-----|-------|--------|
|
||||
| [ADR-018](../../docs/adr/ADR-018-csi-binary-frame-format.md) | CSI binary frame format | Accepted |
|
||||
| [ADR-029](../../docs/adr/ADR-029-ruvsense-multistatic-sensing-mode.md) | Channel hopping and TDM protocol | Accepted |
|
||||
| [ADR-039](../../docs/adr/ADR-039-esp32-edge-intelligence.md) | Edge intelligence tiers 0-2 | Accepted |
|
||||
| [ADR-040](../../docs/adr/) | WASM programmable sensing (Tier 3) with RVF container format | Alpha |
|
||||
|
||||
---
|
||||
|
||||
## License
|
||||
|
||||
This firmware is dual-licensed under [MIT](../../LICENSE-MIT) OR [Apache-2.0](../../LICENSE-APACHE), at your option.
|
||||
|
||||
Reference in New Issue
Block a user