mirror of
https://github.com/ruvnet/RuView
synced 2026-08-02 19:11:46 +00:00
deploy: 55f6a74e1e
This commit is contained in:
ruvnet
@@ -0,0 +1,205 @@
|
||||
# Three-Tier Node — Decision Tree
|
||||
|
||||
| Field | Value |
|
||||
|--------------|------------------------------------------------------------------------|
|
||||
| **Status** | Reference — informs whether/how to adopt the three-tier proposal |
|
||||
| **Date** | 2026-04-25 |
|
||||
| **Companion**| `architecture/three-tier-rust-node.md`, `sota/2026-Q2-rf-sensing-and-edge-rust.md` |
|
||||
|
||||
This document maps each load-bearing decision in the three-tier proposal
|
||||
to (a) what it depends on, (b) what evidence would justify yes/no, and
|
||||
(c) which ADR slot would house the decision once made. It is intentionally
|
||||
short — the prose lives in the SOTA survey and the seed exploration.
|
||||
|
||||
---
|
||||
|
||||
## 1. Load-bearing vs independent decisions
|
||||
|
||||
Six decisions are **load-bearing** — they unblock or block other
|
||||
decisions:
|
||||
|
||||
| # | Decision | Blocks |
|
||||
|----|----------------------------------|------------------------------------------|
|
||||
| L1 | Per-node BOM ceiling | Hardware split, Pi shape, all ADRs below |
|
||||
| L2 | Single-MCU vs dual-MCU node | Sensor-MCU runtime, ISR strategy |
|
||||
| L3 | One-Pi-per-node vs one-per-cluster | OTA shape, secure-boot story, BOM |
|
||||
| L4 | CSI no_std maturity gate | Sensor-MCU language choice |
|
||||
| L5 | Mesh control-plane technology | Comms MCU choice (S3 vs C6) |
|
||||
| L6 | Heavy-compute SoC choice | Secure-boot path, ML model class |
|
||||
|
||||
Five decisions are **independent** of the three-tier shape and can be
|
||||
made in parallel:
|
||||
|
||||
| # | Decision |
|
||||
|----|----------------------------------|
|
||||
| I1 | LoRa fallback chip (SX1262 vs LR1121) |
|
||||
| I2 | Charger / PMIC (BQ24074 vs BQ25798) |
|
||||
| I3 | QUIC vs MQTT-over-TLS for backhaul |
|
||||
| I4 | OTA mechanism per die |
|
||||
| I5 | Provisioning protocol (BLE vs USB) |
|
||||
|
||||
---
|
||||
|
||||
## 2. Decision tree (Mermaid)
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
L1{"L1: BOM ceiling per node?"}
|
||||
L1 -->|"<= $15"| KEEP_TODAY["Keep ADR-028 single-S3 node.<br/>Three-tier proposal is out of budget."]
|
||||
L1 -->|"$15-$30"| L3
|
||||
L1 -->|"> $30"| L3
|
||||
|
||||
L3{"L3: Heavy compute per node<br/>or per cluster?"}
|
||||
L3 -->|"per cluster (1 Pi / 3-6 nodes)"| HYBRID["Hybrid path:<br/>single-S3 sensor + cluster Pi.<br/>Cheapest viable upgrade."]
|
||||
L3 -->|"per node"| L2
|
||||
|
||||
L2{"L2: Single-MCU or dual-MCU<br/>per node?"}
|
||||
L2 -->|"single MCU"| L4_SINGLE["ADR-081 already covers this.<br/>Investigate WHY a dual-MCU is needed."]
|
||||
L2 -->|"dual MCU (sensor + comms)"| L4
|
||||
|
||||
L4{"L4: Is no_std CSI capture<br/>production-quality?"}
|
||||
L4 -->|"no / unknown"| L4_NO["Hold dual-MCU shape until<br/>esp-csi-rs / esp-radio matches<br/>esp_wifi_set_csi_rx_cb in jitter & quality."]
|
||||
L4 -->|"yes (benchmarked)"| L5
|
||||
|
||||
L5{"L5: Mesh control plane:<br/>WiFi or 802.15.4?"}
|
||||
L5 -->|"WiFi (ESP-WIFI-MESH)"| L5_WIFI["Comms MCU = ESP32-S3.<br/>Stays on existing ADR-029 shape."]
|
||||
L5 -->|"802.15.4 (Thread)"| L5_THREAD["Comms MCU = ESP32-C6.<br/>Hybrid: WiFi data + Thread control."]
|
||||
|
||||
L6{"L6: Heavy compute SoC?"}
|
||||
L6 -->|"Pi Zero 2W"| L6_ZERO["dm-verity + signed FIT.<br/>NOT immutable-ROM secure boot."]
|
||||
L6 -->|"CM4 / Pi 5"| L6_CM4["RPi-foundation secure boot path.<br/>+~$30-50 BOM."]
|
||||
|
||||
HYBRID --> L6
|
||||
L5_WIFI --> L6
|
||||
L5_THREAD --> L6
|
||||
|
||||
L4_NO -.->|"if gated long-term"| HYBRID
|
||||
|
||||
style KEEP_TODAY fill:#cfe
|
||||
style HYBRID fill:#cfe
|
||||
style L4_NO fill:#fec
|
||||
style L4_SINGLE fill:#cfe
|
||||
```
|
||||
|
||||
The tree's recommended cheapest-first path is:
|
||||
**L1 → L3 (per-cluster) → HYBRID**, which keeps today's ESP32-S3 sensor
|
||||
nodes and adds one Pi per 3–6 nodes. This captures most of the QUIC /
|
||||
ML / secure-boot value without re-spinning the per-node PCB.
|
||||
|
||||
---
|
||||
|
||||
## 3. Decision detail — what evidence justifies each branch
|
||||
|
||||
### L1 — Per-node BOM ceiling
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|-----------------------|--------------------------------------------------------------------|--------------------------------------|
|
||||
| ≤ $15 | Today's $9 BOM, ADR-028 witness; deployment-cost analysis | No new ADR — keep ADR-028 baseline |
|
||||
| $15–$30 | Cost analysis showing single-MCU + cluster-Pi path < $30 | New ADR (e.g., ADR-083) |
|
||||
| > $30 | Deployment-cost analysis showing per-node Pi pays for itself | Two ADRs (per-node Pi, BOM revision) |
|
||||
|
||||
### L2 — Single vs dual MCU per node
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|--------------|--------------------------------------------------------------------------------------------|--------------------------------|
|
||||
| Single MCU | ADR-081 5-layer kernel measurements (already 60 byte feature packets, 0.003% CPU at 5 Hz) | No new ADR — keep ADR-081 |
|
||||
| Dual MCU | Measured ISR-jitter problem on single-MCU node; or no_std-CSI maturity demonstrated | New ADR (firmware split) |
|
||||
|
||||
### L3 — Per-node vs per-cluster heavy compute
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|---------------|-----------------------------------------------------------------------------------------------|--------------------------------|
|
||||
| Per cluster | Throughput math: 6 nodes × 5 Hz × 60 B = 1.8 KB/s per cluster; well within USB/Ethernet to Pi | New ADR (cluster-Pi shape) |
|
||||
| Per node | Need: per-node ML, per-node QUIC, per-node secure boot, deployment without LAN gateway | New ADR (per-node Pi shape) |
|
||||
|
||||
### L4 — CSI no_std maturity gate
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|------------|--------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------|
|
||||
| Mature | esp-csi-rs (or replacement) on real S3 board: matches esp_wifi_set_csi_rx_cb capture rate, frame-loss, ISR-jitter | Phase-4 of ADR-081 + a `no_std` migration ADR |
|
||||
| Not mature | Side-by-side benchmark shows ≥10% drop in capture quality, or ISR-jitter > 100 µs | Defer — remain on ESP-IDF C path |
|
||||
|
||||
### L5 — Mesh control-plane technology
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|-----------------|--------------------------------------------------------------------------------------------------------------|---------------------------------------------|
|
||||
| ESP-WIFI-MESH | ≤ 25-node target; existing ADR-029 + ADR-073 hold | No new ADR — keep ADR-029 |
|
||||
| Thread | ≥ 50-node target; field test showing ESP-WIFI-MESH degradation; comms-MCU change to ESP32-C6 acceptable | New ADR (Thread control plane) |
|
||||
| `esp-mesh-lite` | Wanting IP-layer routing for QUIC + WiFi homogeneity, but staying on S3 | New ADR (mesh-lite migration) |
|
||||
|
||||
### L6 — Heavy-compute SoC choice
|
||||
|
||||
| Branch | Evidence required | ADR slot |
|
||||
|------------|--------------------------------------------------------------------------------------------------------------|-----------------------------------------|
|
||||
| Pi Zero 2W | Buildroot + dm-verity + signed FIT meets the threat model; cost / power matters more than ROM-rooted boot | New ADR (Pi Zero 2W image / OTA) |
|
||||
| CM4 / Pi 5 | True ROM-rooted secure boot is deployment-required (e.g., regulated environment) | New ADR (CM4 image / OTA) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Independent decisions — make in parallel
|
||||
|
||||
Each of these can be evaluated in isolation; none depend on the L-decisions.
|
||||
|
||||
| # | Decision | Default recommendation | ADR slot |
|
||||
|----|---------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------|
|
||||
| I1 | LoRa fallback chip | **SX1262.** LR1121 only if global / 2.4 GHz / satellite roaming is a deployment requirement. (SOTA §6) | ADR (LoRa fallback) |
|
||||
| I2 | PMIC choice | **BQ24074 if panel ≤ 2 W**, **BQ25798 if panel ≥ 5 W or solar-only**. SPV1050 only for sub-watt energy harvesting. (SOTA §7) | ADR (power path) |
|
||||
| I3 | Backhaul protocol | **QUIC (`quinn` + `rustls`)** if bidirectional / large payload / mobile-network handoff matters. **MQTT-over-TLS** for low-rate publish-only. (SOTA §5) | ADR (backhaul) |
|
||||
| I4 | OTA per die | **`embassy-boot` two-slot** on no_std MCUs. **ESP-IDF native OTA** on ESP-IDF MCUs. **A/B + signed FIT** on Pi. (SOTA §3, §9) | ADR (OTA) |
|
||||
| I5 | Provisioning protocol | **BLE provisioning via `esp-idf-svc`** for any in-field reprovisioning; **USB / serial** for factory provisioning only. (No SOTA section — well-trodden ground.) | ADR (provisioning) |
|
||||
|
||||
---
|
||||
|
||||
## 5. Recommended ADR sequence
|
||||
|
||||
If the three-tier proposal is partially adopted, the recommended ADR
|
||||
sequence is **outside-in** — address the cheapest, most independent
|
||||
decisions first, gate the load-bearing ones on real evidence:
|
||||
|
||||
1. **Independent ADRs first** (any order):
|
||||
- I1 LoRa fallback chip choice.
|
||||
- I2 Power-path / PMIC choice (probably BQ24074 if panel stays ≤ 2 W,
|
||||
BQ25798 otherwise).
|
||||
- I3 QUIC vs MQTT-over-TLS (likely MQTT for the heartbeat-only case,
|
||||
QUIC if model updates and fleet sync are real).
|
||||
2. **Per-cluster-Pi ADR** (L3, hybrid branch) — the high-value, low-cost
|
||||
first step. One Pi per 3–6 nodes. Captures most of the ML/QUIC/
|
||||
secure-boot value at minimal per-sensor BOM impact.
|
||||
3. **Mesh control-plane ADR** (L5) — only if deployments target > 25
|
||||
nodes. Otherwise stays on ESP-WIFI-MESH per ADR-029.
|
||||
4. **CSI no_std maturity benchmark ADR** (L4 evidence) — investigate,
|
||||
but do not commit to dual-MCU until benchmarked.
|
||||
5. **Dual-MCU node ADR** (L2) — only after L4 evidence + a clear ML or
|
||||
ISR-jitter problem on the single-MCU node.
|
||||
6. **Three-tier-PCB ADR** (full proposal) — last, only if BOM / threat-
|
||||
model / scale all justify it.
|
||||
|
||||
This ordering deliberately keeps the bulk of the deployable surface on
|
||||
today's ADR-028 / ADR-081 baseline while letting each separable
|
||||
upgrade be evaluated on its own evidence.
|
||||
|
||||
---
|
||||
|
||||
## 6. Out-of-scope for this document
|
||||
|
||||
- **Re-evaluating ADR-029 mesh choices** beyond mentioning Thread as
|
||||
alternative — that belongs in a Mesh-control-plane ADR.
|
||||
- **Specific PCB layout** of any of the candidate boards.
|
||||
- **Cloud-side architecture** (gateway, fleet-sync target, time-series
|
||||
storage). Out of scope of the node architecture proposal.
|
||||
- **Cross-environment domain generalization (ADR-027)** — orthogonal to
|
||||
the hardware shape.
|
||||
- **Multistatic fusion algorithms** (`wifi-densepose-ruvector::viewpoint`)
|
||||
— orthogonal to the hardware shape.
|
||||
|
||||
---
|
||||
|
||||
## 7. References to other documents in this set
|
||||
|
||||
- `architecture/three-tier-rust-node.md` — the seed proposal.
|
||||
- `sota/2026-Q2-rf-sensing-and-edge-rust.md` — SOTA evidence per topic.
|
||||
- `architecture/implementation-plan.md` — earlier (2026-04-02) GOAP plan
|
||||
for ESP32-S3 + Pi Zero 2 W; the three-tier proposal is most usefully
|
||||
read as an extension of this plan.
|
||||
- `architecture/ruvsense-multistatic-fidelity-architecture.md` —
|
||||
multistatic fusion architecture, orthogonal to node hardware shape.
|
||||
@@ -0,0 +1,996 @@
|
||||
# GOAP Implementation Plan: ESP32-S3 + Pi Zero 2 W WiFi Pose Estimation
|
||||
|
||||
**Date:** 2026-04-02
|
||||
**Version:** 1.0
|
||||
**Status:** Proposed
|
||||
**Depends on:** ADR-029, ADR-068, SOTA survey (sota-wifi-sensing-2025.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. Goal State Definition
|
||||
|
||||
### 1.1 Terminal Goal
|
||||
|
||||
A production-ready WiFi-based human pose estimation system where:
|
||||
- **ESP32-S3** nodes capture WiFi CSI at 100 Hz, perform temporal feature extraction, and transmit compressed features via UDP
|
||||
- **Raspberry Pi Zero 2 W** receives features from 1-4 ESP32 nodes, runs neural inference, and outputs 17-keypoint COCO poses at >= 10 Hz
|
||||
- **Single-person MPJPE** < 100mm in trained environments
|
||||
- **End-to-end latency** < 150ms (CSI capture to pose output)
|
||||
- **Total BOM cost** < $30 per sensing zone (1x Pi Zero + 2x ESP32)
|
||||
|
||||
### 1.2 World State Variables
|
||||
|
||||
```
|
||||
current_state:
|
||||
esp32_csi_capture: true # Already implemented
|
||||
multi_node_aggregation: true # ADR-018 UDP aggregator
|
||||
phase_alignment: true # ruvsense/phase_align.rs
|
||||
coherence_gating: true # ruvsense/coherence_gate.rs
|
||||
multistatic_fusion: true # ruvsense/multistatic.rs
|
||||
kalman_pose_tracking: true # ruvsense/pose_tracker.rs
|
||||
onnx_inference_engine: true # wifi-densepose-nn
|
||||
modality_translator: true # wifi-densepose-nn/translator.rs
|
||||
training_pipeline: true # wifi-densepose-train
|
||||
pi_zero_deployment: false # No Pi Zero target
|
||||
lightweight_model: false # No edge-optimized model
|
||||
temporal_conv_module: false # No TCN in inference path
|
||||
csi_compression: false # No ESP32-side compression
|
||||
int8_quantization: false # No quantization pipeline
|
||||
bone_constraint_loss: false # No skeleton physics in loss
|
||||
esp32_pi_protocol: false # No lightweight protocol
|
||||
edge_inference_engine: false # No ARM-optimized inference
|
||||
cross_env_adaptation: false # No domain adaptation
|
||||
multi_person_paf: false # No PAF-based multi-person
|
||||
3d_pose_lifting: false # No Z-axis estimation
|
||||
|
||||
goal_state:
|
||||
esp32_csi_capture: true
|
||||
multi_node_aggregation: true
|
||||
phase_alignment: true
|
||||
coherence_gating: true
|
||||
multistatic_fusion: true
|
||||
kalman_pose_tracking: true
|
||||
onnx_inference_engine: true
|
||||
modality_translator: true
|
||||
training_pipeline: true
|
||||
pi_zero_deployment: true # TARGET
|
||||
lightweight_model: true # TARGET
|
||||
temporal_conv_module: true # TARGET
|
||||
csi_compression: true # TARGET
|
||||
int8_quantization: true # TARGET
|
||||
bone_constraint_loss: true # TARGET
|
||||
esp32_pi_protocol: true # TARGET
|
||||
edge_inference_engine: true # TARGET
|
||||
cross_env_adaptation: true # TARGET (Phase 2)
|
||||
multi_person_paf: true # TARGET (Phase 2)
|
||||
3d_pose_lifting: true # TARGET (Phase 3)
|
||||
```
|
||||
|
||||
## 2. Action Definitions
|
||||
|
||||
Each action has preconditions, effects, estimated cost (developer-days), and priority.
|
||||
|
||||
### Action 1: Define ESP32-Pi Communication Protocol (ADR-069)
|
||||
|
||||
```
|
||||
name: define_esp32_pi_protocol
|
||||
cost: 3 days
|
||||
priority: CRITICAL (blocks all Pi Zero work)
|
||||
preconditions: [esp32_csi_capture]
|
||||
effects: [esp32_pi_protocol := true]
|
||||
```
|
||||
|
||||
**Description:** Design a lightweight binary protocol for ESP32 -> Pi Zero communication over UDP (WiFi) or UART (wired fallback).
|
||||
|
||||
**Protocol specification:**
|
||||
|
||||
```
|
||||
Frame Header (8 bytes):
|
||||
[0:1] magic: 0xCF01 (CSI Frame v1)
|
||||
[2] node_id: u8 (0-255, identifies ESP32 node)
|
||||
[3] frame_type: u8 (0=raw_csi, 1=compressed_features, 2=heartbeat)
|
||||
[4:5] sequence: u16 (monotonic frame counter, wraps at 65535)
|
||||
[6:7] payload_len: u16 (bytes following header)
|
||||
|
||||
Raw CSI Payload (frame_type=0):
|
||||
[0:3] timestamp_us: u32 (microseconds since boot, wraps at ~71 minutes)
|
||||
[4] channel: u8 (WiFi channel 1-13)
|
||||
[5] bandwidth: u8 (0=20MHz, 1=40MHz)
|
||||
[6] rssi: i8 (dBm)
|
||||
[7] noise_floor: i8 (dBm)
|
||||
[8:9] num_sc: u16 (number of subcarriers, typically 52 or 114)
|
||||
[10..] csi_data: [i16; num_sc * 2] (interleaved I/Q, little-endian)
|
||||
|
||||
Compressed Feature Payload (frame_type=1):
|
||||
[0:3] timestamp_us: u32
|
||||
[4] compression: u8 (0=none, 1=pca_16, 2=pca_32, 3=autoencoder)
|
||||
[5] num_features: u8 (number of feature dimensions)
|
||||
[6..] features: [f16; num_features] (half-precision floats)
|
||||
|
||||
Heartbeat Payload (frame_type=2):
|
||||
[0:3] uptime_s: u32
|
||||
[4:7] frames_sent: u32
|
||||
[8:9] free_heap: u16 (KB)
|
||||
[10] wifi_rssi: i8 (connection to AP)
|
||||
[11] battery_pct: u8 (0-100, 0xFF if wired)
|
||||
```
|
||||
|
||||
**Implementation locations:**
|
||||
- ESP32 firmware: `firmware/esp32-csi-node/main/protocol_v2.h`
|
||||
- Rust parser: `wifi-densepose-hardware/src/protocol_v2.rs`
|
||||
|
||||
**Design rationale:**
|
||||
- Fixed 8-byte header with magic number for frame synchronization
|
||||
- Half-precision (f16) for compressed features saves 50% bandwidth vs f32
|
||||
- Heartbeat enables Pi Zero to detect node failures and rebalance
|
||||
- Raw CSI mode for debugging; compressed mode for production
|
||||
|
||||
### Action 2: Implement Lightweight Model Architecture
|
||||
|
||||
```
|
||||
name: implement_lightweight_model
|
||||
cost: 10 days
|
||||
priority: CRITICAL (core inference capability)
|
||||
preconditions: [training_pipeline, onnx_inference_engine]
|
||||
effects: [lightweight_model := true, temporal_conv_module := true]
|
||||
```
|
||||
|
||||
**Architecture: WiFlowPose (hybrid WiFlow + MultiFormer)**
|
||||
|
||||
Based on SOTA analysis, we define a custom architecture combining the best elements:
|
||||
|
||||
```
|
||||
Input: CSI amplitude tensor [B, T, S]
|
||||
B = batch size
|
||||
T = temporal window (20 frames at 20 Hz = 1 second context)
|
||||
S = subcarriers (52 for ESP32-S3 20MHz, 114 for 40MHz)
|
||||
|
||||
Stage 1: Temporal Encoder (runs on ESP32 optionally, or Pi Zero)
|
||||
TCN with 4 layers, dilation [1, 2, 4, 8]
|
||||
Input: [B, T, S] = [B, 20, 52]
|
||||
Output: [B, T', C_t] = [B, 20, 64] (temporal features)
|
||||
|
||||
Stage 2: Spatial Encoder (runs on Pi Zero)
|
||||
Asymmetric convolution blocks (1xk kernels on subcarrier dimension)
|
||||
4 residual blocks: 64 -> 128 -> 128 -> 64 channels
|
||||
Subcarrier compression: 52 -> 26 -> 13 -> 7
|
||||
Output: [B, 64, 7]
|
||||
|
||||
Stage 3: Keypoint Decoder (runs on Pi Zero)
|
||||
Axial self-attention (2-stage, 4 heads)
|
||||
Reshape to [B, 17, 64] (17 keypoints x 64 features)
|
||||
Linear projection: 64 -> 2 (x, y coordinates)
|
||||
Output: [B, 17, 2] (17 COCO keypoints, normalized 0-1)
|
||||
|
||||
Optional Stage 4: Multi-person (Phase 2)
|
||||
PAF branch: predict 19 limb affinity fields
|
||||
Hungarian assignment for person grouping
|
||||
```
|
||||
|
||||
**Estimated model size:**
|
||||
- Temporal encoder: ~0.5M params
|
||||
- Spatial encoder: ~1.2M params
|
||||
- Keypoint decoder: ~0.8M params
|
||||
- Total: ~2.5M params
|
||||
- INT8 size: ~2.5 MB
|
||||
- FP16 size: ~5 MB
|
||||
- Estimated Pi Zero 2 W inference: 30-60ms per frame
|
||||
|
||||
**Rust implementation location:** New module in `wifi-densepose-nn/src/wiflow_pose.rs`
|
||||
|
||||
```rust
|
||||
/// WiFlowPose: Lightweight WiFi CSI to pose estimation model
|
||||
///
|
||||
/// Hybrid architecture combining WiFlow's TCN temporal encoder
|
||||
/// with MultiFormer's dual-token spatial processing and
|
||||
/// axial self-attention for keypoint decoding.
|
||||
pub struct WiFlowPoseConfig {
|
||||
/// Number of input subcarriers (52 for ESP32 20MHz, 114 for 40MHz)
|
||||
pub num_subcarriers: usize,
|
||||
/// Temporal window size in frames (default: 20)
|
||||
pub temporal_window: usize,
|
||||
/// TCN dilation factors (default: [1, 2, 4, 8])
|
||||
pub tcn_dilations: Vec<usize>,
|
||||
/// Number of output keypoints (default: 17, COCO format)
|
||||
pub num_keypoints: usize,
|
||||
/// Hidden dimension for spatial encoder (default: 64)
|
||||
pub hidden_dim: usize,
|
||||
/// Number of attention heads in axial attention (default: 4)
|
||||
pub num_attention_heads: usize,
|
||||
/// Enable multi-person PAF branch (default: false)
|
||||
pub multi_person: bool,
|
||||
}
|
||||
|
||||
impl Default for WiFlowPoseConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
num_subcarriers: 52,
|
||||
temporal_window: 20,
|
||||
tcn_dilations: vec![1, 2, 4, 8],
|
||||
num_keypoints: 17,
|
||||
hidden_dim: 64,
|
||||
num_attention_heads: 4,
|
||||
multi_person: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Action 3: Implement Bone Constraint Loss
|
||||
|
||||
```
|
||||
name: implement_bone_constraint_loss
|
||||
cost: 2 days
|
||||
priority: HIGH
|
||||
preconditions: [training_pipeline, lightweight_model]
|
||||
effects: [bone_constraint_loss := true]
|
||||
```
|
||||
|
||||
**Loss function following WiFlow:**
|
||||
|
||||
```
|
||||
L_total = L_keypoint + lambda_bone * L_bone + lambda_physics * L_physics
|
||||
|
||||
L_keypoint = SmoothL1(pred, gt, beta=0.1)
|
||||
|
||||
L_bone = (1/|B|) * sum_{(i,j) in bones} | ||pred_i - pred_j|| - bone_length_{ij} |
|
||||
|
||||
L_physics = (1/N) * sum_t max(0, ||pred_t - pred_{t-1}|| - v_max * dt)
|
||||
```
|
||||
|
||||
Where:
|
||||
- `bones` = 14 COCO bone connections (e.g., left_shoulder-left_elbow)
|
||||
- `bone_length_{ij}` = average human bone length ratios (normalized to torso length)
|
||||
- `v_max` = maximum physiologically plausible keypoint velocity (2 m/s for walking, 10 m/s for fast gestures)
|
||||
- `lambda_bone = 0.2`, `lambda_physics = 0.1`
|
||||
|
||||
**Bone length ratios (normalized to torso = shoulder_center to hip_center = 1.0):**
|
||||
|
||||
| Bone | Ratio |
|
||||
|------|-------|
|
||||
| shoulder-elbow | 0.55 |
|
||||
| elbow-wrist | 0.50 |
|
||||
| hip-knee | 0.85 |
|
||||
| knee-ankle | 0.80 |
|
||||
| shoulder-hip | 1.00 |
|
||||
| neck-nose | 0.30 |
|
||||
| nose-eye | 0.08 |
|
||||
| eye-ear | 0.12 |
|
||||
|
||||
**Implementation location:** `wifi-densepose-train/src/losses.rs` (add `BoneConstraintLoss`)
|
||||
|
||||
### Action 4: Implement INT8 Quantization Pipeline
|
||||
|
||||
```
|
||||
name: implement_int8_quantization
|
||||
cost: 5 days
|
||||
priority: HIGH
|
||||
preconditions: [lightweight_model, training_pipeline]
|
||||
effects: [int8_quantization := true]
|
||||
```
|
||||
|
||||
**Approach: Post-Training Quantization (PTQ) with calibration**
|
||||
|
||||
1. Train model in FP32 using standard pipeline
|
||||
2. Export to ONNX format
|
||||
3. Run ONNX Runtime quantization tool with calibration dataset:
|
||||
- Collect 1000 representative CSI frames across multiple environments
|
||||
- Run calibration to determine per-layer quantization ranges
|
||||
- Apply symmetric INT8 quantization for weights, asymmetric for activations
|
||||
4. Validate quantized model accuracy (target: <2% PCK@20 degradation)
|
||||
|
||||
**Quantization-aware considerations:**
|
||||
- TCN layers: quantize per-channel (dilated convolutions are sensitive to quantization)
|
||||
- Attention layers: keep attention logits in FP16 (softmax is numerically sensitive)
|
||||
- Output layer: keep in FP32 (final coordinate regression needs precision)
|
||||
|
||||
**Rust implementation:**
|
||||
```rust
|
||||
// In wifi-densepose-nn/src/quantize.rs
|
||||
pub struct QuantizationConfig {
|
||||
/// Quantization method
|
||||
pub method: QuantMethod, // PTQ, QAT, Dynamic
|
||||
/// Per-layer precision overrides
|
||||
pub layer_overrides: HashMap<String, Precision>,
|
||||
/// Calibration dataset path
|
||||
pub calibration_data: PathBuf,
|
||||
/// Number of calibration samples
|
||||
pub num_calibration_samples: usize,
|
||||
/// Target accuracy degradation threshold
|
||||
pub max_accuracy_loss: f32,
|
||||
}
|
||||
|
||||
pub enum Precision {
|
||||
INT8,
|
||||
FP16,
|
||||
FP32,
|
||||
}
|
||||
```
|
||||
|
||||
**ONNX quantization command (for build pipeline):**
|
||||
```bash
|
||||
python -m onnxruntime.quantization.quantize \
|
||||
--input model_fp32.onnx \
|
||||
--output model_int8.onnx \
|
||||
--calibrate \
|
||||
--calibration_data_reader CsiCalibrationReader \
|
||||
--quant_format QDQ \
|
||||
--activation_type QUInt8 \
|
||||
--weight_type QInt8
|
||||
```
|
||||
|
||||
### Action 5: Build Edge Inference Engine for Pi Zero
|
||||
|
||||
```
|
||||
name: build_edge_inference_engine
|
||||
cost: 8 days
|
||||
priority: CRITICAL
|
||||
preconditions: [lightweight_model, int8_quantization, esp32_pi_protocol]
|
||||
effects: [edge_inference_engine := true, pi_zero_deployment := true]
|
||||
```
|
||||
|
||||
**Architecture: Streaming inference with ring buffer**
|
||||
|
||||
```
|
||||
UDP/UART
|
||||
ESP32-S3 ---------> Pi Zero 2 W
|
||||
|
|
||||
v
|
||||
+-- RingBuffer<CsiFrame> --+
|
||||
| (capacity: 64 frames) |
|
||||
+------ | | -------------+
|
||||
v v
|
||||
+-- TemporalWindow --------+
|
||||
| (20 frames, sliding) |
|
||||
+------ | ----------------+
|
||||
v
|
||||
+-- WiFlowPose ONNX ------+
|
||||
| (INT8, XNNPACK accel) |
|
||||
+------ | ----------------+
|
||||
v
|
||||
+-- PoseTracker -----------+
|
||||
| (Kalman + skeleton) |
|
||||
+------ | ----------------+
|
||||
v
|
||||
PoseEstimate output
|
||||
(17 keypoints + confidence)
|
||||
```
|
||||
|
||||
**New Rust binary:** `wifi-densepose-cli/src/bin/edge_infer.rs`
|
||||
|
||||
```rust
|
||||
/// Edge inference daemon for Raspberry Pi Zero 2 W
|
||||
///
|
||||
/// Receives CSI frames from ESP32 nodes via UDP, maintains a temporal
|
||||
/// sliding window, runs INT8 ONNX inference, and outputs pose estimates.
|
||||
///
|
||||
/// Usage:
|
||||
/// wifi-densepose edge-infer \
|
||||
/// --model model_int8.onnx \
|
||||
/// --listen 0.0.0.0:5555 \
|
||||
/// --output-port 5556 \
|
||||
/// --window-size 20 \
|
||||
/// --max-nodes 4
|
||||
|
||||
struct EdgeInferConfig {
|
||||
/// Path to INT8 ONNX model
|
||||
model_path: PathBuf,
|
||||
/// UDP listen address for CSI frames
|
||||
listen_addr: SocketAddr,
|
||||
/// UDP output address for pose results
|
||||
output_addr: Option<SocketAddr>,
|
||||
/// Temporal window size
|
||||
window_size: usize,
|
||||
/// Maximum ESP32 nodes to accept
|
||||
max_nodes: usize,
|
||||
/// Inference thread count (1-4 on Pi Zero 2 W)
|
||||
num_threads: usize,
|
||||
/// Enable XNNPACK acceleration
|
||||
use_xnnpack: bool,
|
||||
}
|
||||
```
|
||||
|
||||
**Cross-compilation for Pi Zero 2 W:**
|
||||
|
||||
```bash
|
||||
# Install cross-compilation toolchain
|
||||
rustup target add aarch64-unknown-linux-gnu
|
||||
sudo apt install gcc-aarch64-linux-gnu
|
||||
|
||||
# Build for Pi Zero 2 W (64-bit Raspberry Pi OS)
|
||||
cross build --target aarch64-unknown-linux-gnu \
|
||||
--release \
|
||||
-p wifi-densepose-cli \
|
||||
--features edge-inference \
|
||||
--no-default-features
|
||||
|
||||
# Or for 32-bit Raspberry Pi OS:
|
||||
# rustup target add armv7-unknown-linux-gnueabihf
|
||||
# cross build --target armv7-unknown-linux-gnueabihf ...
|
||||
```
|
||||
|
||||
**ONNX Runtime linking for ARM:**
|
||||
- Use `ort` crate with `download-binaries` feature for automatic aarch64 binary download
|
||||
- Alternative: build OnnxStream from source for minimal binary size (~2 MB vs ~30 MB for full ONNX Runtime)
|
||||
|
||||
### Action 6: Implement CSI Compression on ESP32
|
||||
|
||||
```
|
||||
name: implement_csi_compression
|
||||
cost: 5 days
|
||||
priority: MEDIUM
|
||||
preconditions: [esp32_csi_capture, esp32_pi_protocol]
|
||||
effects: [csi_compression := true]
|
||||
```
|
||||
|
||||
**Three compression tiers:**
|
||||
|
||||
**Tier 0: No compression (raw CSI)**
|
||||
- Payload: 52 subcarriers x 2 (I/Q) x 2 bytes = 208 bytes per frame
|
||||
- Use case: debugging, maximum fidelity
|
||||
|
||||
**Tier 1: PCA-16 (run on ESP32)**
|
||||
- Pre-computed PCA projection matrix (52 -> 16 dimensions)
|
||||
- Stored in NVS flash during provisioning
|
||||
- Payload: 16 features x 2 bytes (f16) = 32 bytes per frame
|
||||
- Compression: 6.5x
|
||||
- Compute: ~0.1ms on ESP32-S3 (matrix-vector multiply, SIMD)
|
||||
|
||||
**Tier 2: PCA-32 (higher fidelity)**
|
||||
- 52 -> 32 dimensions
|
||||
- Payload: 32 x 2 = 64 bytes
|
||||
- Compression: 3.25x
|
||||
|
||||
**Tier 3: Learned autoencoder (future)**
|
||||
- ESP32-S3 has enough compute for a small encoder (~10K params)
|
||||
- Requires quantized encoder weights in flash
|
||||
- Most bandwidth-efficient but requires training
|
||||
|
||||
**PCA computation (offline, during provisioning):**
|
||||
|
||||
```rust
|
||||
// wifi-densepose-train/src/compression.rs
|
||||
|
||||
/// Compute PCA projection matrix from calibration CSI data
|
||||
pub fn compute_pca_projection(
|
||||
calibration_data: &[CsiFrame],
|
||||
target_dims: usize,
|
||||
) -> PcaProjection {
|
||||
// 1. Stack all CSI amplitude vectors into matrix [N, S]
|
||||
// 2. Center (subtract mean)
|
||||
// 3. Compute covariance matrix [S, S]
|
||||
// 4. Eigendecomposition, take top `target_dims` eigenvectors
|
||||
// 5. Return projection matrix [S, target_dims] and mean vector [S]
|
||||
// ...
|
||||
}
|
||||
|
||||
pub struct PcaProjection {
|
||||
/// Projection matrix [num_subcarriers, target_dims]
|
||||
pub matrix: Vec<f32>,
|
||||
/// Mean vector for centering [num_subcarriers]
|
||||
pub mean: Vec<f32>,
|
||||
/// Number of input subcarriers
|
||||
pub input_dims: usize,
|
||||
/// Number of output features
|
||||
pub output_dims: usize,
|
||||
}
|
||||
```
|
||||
|
||||
**ESP32 firmware integration:**
|
||||
- Store PCA matrix in NVS partition (32x52x4 = 6.5 KB for PCA-32)
|
||||
- Apply projection in CSI callback before UDP transmission
|
||||
- Selectable via provisioning command
|
||||
|
||||
### Action 7: Implement Cross-Environment Adaptation
|
||||
|
||||
```
|
||||
name: implement_cross_env_adaptation
|
||||
cost: 8 days
|
||||
priority: MEDIUM (Phase 2)
|
||||
preconditions: [lightweight_model, training_pipeline, pi_zero_deployment]
|
||||
effects: [cross_env_adaptation := true]
|
||||
```
|
||||
|
||||
**Approach: Rapid environment calibration with few-shot adaptation**
|
||||
|
||||
Inspired by Arena Physica's template-based design space and MERIDIAN (ADR-027):
|
||||
|
||||
1. **Environment fingerprinting (on Pi Zero, at deployment time):**
|
||||
- Collect 60 seconds of "empty room" CSI
|
||||
- Compute room signature: mean amplitude profile, delay spread, K-factor
|
||||
- Match to nearest room template (corridor, office, bedroom, etc.)
|
||||
- Load template-specific model weights
|
||||
|
||||
2. **Few-shot fine-tuning (optional, on workstation):**
|
||||
- Collect 5 minutes of calibration data with known poses
|
||||
- Fine-tune last 2 layers of the model (~50K params)
|
||||
- Transfer updated model back to Pi Zero
|
||||
|
||||
3. **Online adaptation (continuous, on Pi Zero):**
|
||||
- Track CSI statistics over time (sliding window mean/variance)
|
||||
- Detect distribution shift (KL divergence exceeds threshold)
|
||||
- Apply batch normalization statistics update (no gradient computation needed)
|
||||
|
||||
**Implementation location:** `wifi-densepose-train/src/rapid_adapt.rs` (extend existing module)
|
||||
|
||||
### Action 8: Implement Multi-Person PAF Decoding
|
||||
|
||||
```
|
||||
name: implement_multi_person_paf
|
||||
cost: 6 days
|
||||
priority: LOW (Phase 2)
|
||||
preconditions: [lightweight_model, bone_constraint_loss]
|
||||
effects: [multi_person_paf := true]
|
||||
```
|
||||
|
||||
**Architecture (following MultiFormer):**
|
||||
|
||||
Add a PAF branch to the WiFlowPose model:
|
||||
|
||||
```
|
||||
Stage 3 features [B, 64, 7]
|
||||
|
|
||||
+--> Keypoint head: [B, 17, 2] (single-person keypoints)
|
||||
|
|
||||
+--> PAF head: [B, 38, H, W] (19 limb affinity fields)
|
||||
|
|
||||
+--> Confidence head: [B, 19, H, W] (part confidence maps)
|
||||
```
|
||||
|
||||
**Multi-person assignment on Pi Zero:**
|
||||
1. Extract candidate keypoints from confidence maps via NMS
|
||||
2. Compute PAF integral scores between candidate pairs
|
||||
3. Solve bipartite matching with Hungarian algorithm
|
||||
4. Group keypoints into person instances
|
||||
|
||||
**Estimated additional cost:** ~1M parameters, ~10ms additional inference time
|
||||
|
||||
### Action 9: Implement 3D Pose Lifting
|
||||
|
||||
```
|
||||
name: implement_3d_pose_lifting
|
||||
cost: 5 days
|
||||
priority: LOW (Phase 3)
|
||||
preconditions: [lightweight_model, multi_person_paf, multistatic_fusion]
|
||||
effects: [3d_pose_lifting := true]
|
||||
```
|
||||
|
||||
**Approach: Multi-view triangulation + learned depth prior**
|
||||
|
||||
With 2+ ESP32 nodes at known positions, compute 3D pose via:
|
||||
|
||||
1. Each node pair provides a different viewing angle of the WiFi field
|
||||
2. 2D pose from each viewpoint is estimated independently
|
||||
3. Epipolar geometry constrains 3D position from 2D observations
|
||||
4. Learned depth prior resolves ambiguities (front/back confusion)
|
||||
|
||||
This leverages the existing `viewpoint/geometry.rs` module in wifi-densepose-ruvector which already computes GeometricDiversityIndex and Fisher Information for multi-node configurations.
|
||||
|
||||
## 3. Hardware Architecture
|
||||
|
||||
### 3.1 System Topology
|
||||
|
||||
```
|
||||
WiFi AP (existing home router)
|
||||
/ | \
|
||||
/ | \
|
||||
ESP32-S3 #1 ESP32-S3 #2 ESP32-S3 #3
|
||||
(CSI node) (CSI node) (CSI node, optional)
|
||||
| | |
|
||||
+------+------+------+-------+
|
||||
| UDP (WiFi) |
|
||||
v v
|
||||
Raspberry Pi Zero 2 W
|
||||
(edge inference node)
|
||||
|
|
||||
v
|
||||
Pose output (UDP/MQTT/WebSocket)
|
||||
to display / home automation / API
|
||||
```
|
||||
|
||||
### 3.2 Data Flow Timing
|
||||
|
||||
```
|
||||
T=0ms ESP32 #1 captures CSI frame (channel 1)
|
||||
T=2ms ESP32 #1 applies PCA compression (0.1ms compute)
|
||||
T=3ms ESP32 #1 sends UDP packet to Pi Zero (64 bytes)
|
||||
T=5ms ESP32 #2 captures CSI frame (channel 6, TDM slot)
|
||||
T=7ms ESP32 #2 sends UDP packet to Pi Zero
|
||||
T=10ms Pi Zero receives both frames, adds to ring buffer
|
||||
T=10ms Pi Zero checks temporal window (20 frames accumulated?)
|
||||
If yes: run inference
|
||||
T=15ms Temporal encoder processes 20-frame window (5ms)
|
||||
T=35ms Spatial encoder + attention (20ms)
|
||||
T=45ms Keypoint decoder (10ms)
|
||||
T=48ms Kalman filter update + skeleton constraints (3ms)
|
||||
T=50ms Pose estimate emitted (17 keypoints + confidence)
|
||||
```
|
||||
|
||||
**Total latency: ~50ms** (well under 150ms target)
|
||||
**Throughput: 20 Hz** (matching TDMA cycle)
|
||||
|
||||
### 3.3 Hardware Bill of Materials
|
||||
|
||||
| Component | Unit Cost | Quantity | Total |
|
||||
|-----------|----------|----------|-------|
|
||||
| ESP32-S3 DevKit (8MB) | $9 | 2 | $18 |
|
||||
| Raspberry Pi Zero 2 W | $15 | 1 | $15 |
|
||||
| MicroSD card (16GB) | $5 | 1 | $5 |
|
||||
| USB-C power supply | $5 | 1 | $5 |
|
||||
| **Total** | | | **$43** |
|
||||
|
||||
With ESP32-S3 SuperMini ($6 each), total drops to **$37**.
|
||||
|
||||
For minimum viable setup (1 ESP32 + 1 Pi Zero): **$24**.
|
||||
|
||||
### 3.4 Pi Zero 2 W Specifications
|
||||
|
||||
| Parameter | Value |
|
||||
|-----------|-------|
|
||||
| SoC | BCM2710A1 (quad-core Cortex-A53 @ 1 GHz) |
|
||||
| RAM | 512 MB LPDDR2 |
|
||||
| WiFi | 802.11b/g/n (2.4 GHz only) |
|
||||
| Bluetooth | BLE 4.2 |
|
||||
| GPIO | 40-pin header (UART, SPI, I2C) |
|
||||
| Power | 5V/2A USB micro-B |
|
||||
| OS | Raspberry Pi OS Lite (64-bit, headless) |
|
||||
|
||||
**Memory budget for inference:**
|
||||
|
||||
| Component | Memory |
|
||||
|-----------|--------|
|
||||
| OS + services | ~100 MB |
|
||||
| WiFlowPose INT8 model | ~3 MB |
|
||||
| ONNX Runtime / OnnxStream | ~10-30 MB |
|
||||
| Ring buffer (64 frames x 4 nodes) | ~1 MB |
|
||||
| Inference workspace | ~20 MB |
|
||||
| **Total** | ~134-164 MB |
|
||||
| **Available** | ~348-378 MB headroom |
|
||||
|
||||
Comfortable fit within 512 MB RAM.
|
||||
|
||||
## 4. Rust Crate Modifications
|
||||
|
||||
### 4.1 Modified Crates
|
||||
|
||||
#### wifi-densepose-hardware
|
||||
|
||||
**New files:**
|
||||
- `src/protocol_v2.rs` -- Lightweight ESP32-Pi binary protocol parser/serializer
|
||||
- `src/pi_zero.rs` -- Pi Zero UDP receiver with ring buffer management
|
||||
|
||||
**Modified files:**
|
||||
- `src/lib.rs` -- Add `pub mod protocol_v2; pub mod pi_zero;`
|
||||
- `src/aggregator/mod.rs` -- Add support for protocol_v2 frame format
|
||||
|
||||
#### wifi-densepose-nn
|
||||
|
||||
**New files:**
|
||||
- `src/wiflow_pose.rs` -- WiFlowPose model definition (TCN + asymmetric conv + axial attention)
|
||||
- `src/edge_engine.rs` -- Edge-optimized inference engine (streaming, ARM NEON)
|
||||
- `src/quantize.rs` -- INT8 quantization configuration and validation
|
||||
|
||||
**Modified files:**
|
||||
- `src/lib.rs` -- Add new module exports
|
||||
- `src/onnx.rs` -- Add XNNPACK execution provider option, INT8 model loading
|
||||
- `src/translator.rs` -- Add WiFlowPose-compatible input format
|
||||
|
||||
#### wifi-densepose-train
|
||||
|
||||
**New files:**
|
||||
- `src/wiflow_pose_trainer.rs` -- Training loop for WiFlowPose architecture
|
||||
- `src/compression.rs` -- PCA computation for ESP32 CSI compression
|
||||
- `src/bone_loss.rs` -- Bone constraint and physics consistency losses
|
||||
|
||||
**Modified files:**
|
||||
- `src/losses.rs` -- Add `BoneConstraintLoss`, `PhysicsConsistencyLoss`
|
||||
- `src/config.rs` -- Add WiFlowPose training configuration options
|
||||
- `src/dataset.rs` -- Add ESP32-S3 CSI format support (52/114 subcarriers)
|
||||
- `src/rapid_adapt.rs` -- Add few-shot environment calibration
|
||||
|
||||
#### wifi-densepose-signal
|
||||
|
||||
**New files:**
|
||||
- `src/ruvsense/temporal_encoder.rs` -- TCN temporal feature extraction (shared code for ESP32 and Pi)
|
||||
|
||||
**Modified files:**
|
||||
- `src/ruvsense/mod.rs` -- Add `pub mod temporal_encoder;`
|
||||
|
||||
#### wifi-densepose-cli
|
||||
|
||||
**New files:**
|
||||
- `src/bin/edge_infer.rs` -- Pi Zero edge inference daemon
|
||||
- `src/bin/calibrate.rs` -- Environment calibration tool (PCA computation, room fingerprinting)
|
||||
|
||||
#### wifi-densepose-core
|
||||
|
||||
**Modified files:**
|
||||
- `src/types.rs` -- Add `CompressedCsiFrame`, `EdgePoseEstimate` types
|
||||
|
||||
### 4.2 New Feature Flags
|
||||
|
||||
```toml
|
||||
# wifi-densepose-nn/Cargo.toml
|
||||
[features]
|
||||
default = ["onnx"]
|
||||
onnx = ["ort"]
|
||||
edge-inference = ["onnx", "xnnpack"] # NEW: ARM NEON + XNNPACK
|
||||
candle = ["candle-core", "candle-nn"]
|
||||
tch-backend = ["tch"]
|
||||
|
||||
# wifi-densepose-cli/Cargo.toml
|
||||
[features]
|
||||
default = ["full"]
|
||||
full = ["wifi-densepose-nn/onnx", "wifi-densepose-train/tch-backend"]
|
||||
edge-inference = ["wifi-densepose-nn/edge-inference"] # NEW: minimal binary for Pi
|
||||
```
|
||||
|
||||
### 4.3 Cross-Compilation Configuration
|
||||
|
||||
```toml
|
||||
# .cargo/config.toml (add section)
|
||||
[target.aarch64-unknown-linux-gnu]
|
||||
linker = "aarch64-linux-gnu-gcc"
|
||||
rustflags = ["-C", "target-cpu=cortex-a53", "-C", "target-feature=+neon"]
|
||||
```
|
||||
|
||||
## 5. ESP32 Firmware Modifications
|
||||
|
||||
### 5.1 New Files
|
||||
|
||||
- `firmware/esp32-csi-node/main/protocol_v2.h` -- Protocol v2 frame packing
|
||||
- `firmware/esp32-csi-node/main/pca_compress.h` -- PCA compression for CSI
|
||||
- `firmware/esp32-csi-node/main/pca_compress.c` -- PCA implementation with ESP32 SIMD
|
||||
- `firmware/esp32-csi-node/main/pi_zero_mode.c` -- Pi Zero communication mode (lighter than full server mode)
|
||||
|
||||
### 5.2 Modified Files
|
||||
|
||||
- `firmware/esp32-csi-node/main/csi_handler.c` -- Add compression step in CSI callback
|
||||
- `firmware/esp32-csi-node/main/nvs_config.c` -- Store PCA matrix in NVS
|
||||
- `firmware/esp32-csi-node/main/Kconfig.projbuild` -- Add CONFIG_PI_ZERO_MODE, CONFIG_CSI_COMPRESSION options
|
||||
|
||||
### 5.3 Provisioning Updates
|
||||
|
||||
```bash
|
||||
# Provision for Pi Zero mode with PCA-16 compression
|
||||
python firmware/esp32-csi-node/provision.py \
|
||||
--port COM7 \
|
||||
--ssid "MyWiFi" \
|
||||
--password "secret" \
|
||||
--target-ip 192.168.1.50 \ # Pi Zero IP
|
||||
--target-port 5555 \
|
||||
--compression pca-16 \
|
||||
--pca-matrix pca_matrix_16.bin
|
||||
```
|
||||
|
||||
## 6. Training Pipeline
|
||||
|
||||
### 6.1 Training Workflow
|
||||
|
||||
```
|
||||
Phase 1: Pre-train on public datasets (GPU workstation)
|
||||
Dataset: MM-Fi + Wi-Pose (Intel 5300 format, 30 subcarriers)
|
||||
Model: WiFlowPose with 30 subcarriers
|
||||
Loss: L_keypoint + 0.2 * L_bone + 0.1 * L_physics
|
||||
Duration: ~20 hours on single A100
|
||||
|
||||
Phase 2: Domain adaptation for ESP32 CSI (GPU workstation)
|
||||
Dataset: Self-collected ESP32-S3 data (52 subcarriers)
|
||||
Method: Fine-tune all layers with lower learning rate (1e-4)
|
||||
Subcarrier interpolation: 30 -> 52 using existing interpolate_subcarriers()
|
||||
Duration: ~4 hours
|
||||
|
||||
Phase 3: Quantization (CPU workstation)
|
||||
Method: Post-training quantization with 1000 calibration samples
|
||||
Format: ONNX INT8 (QDQ format)
|
||||
Validation: PCK@20 degradation < 2%
|
||||
|
||||
Phase 4: Environment calibration (on Pi Zero)
|
||||
Method: 60-second empty-room CSI collection
|
||||
Output: Room fingerprint + PCA matrix
|
||||
Duration: ~2 minutes total
|
||||
```
|
||||
|
||||
### 6.2 Dataset Collection Protocol
|
||||
|
||||
For self-collected ESP32 training data:
|
||||
|
||||
1. **Setup:** 2 ESP32-S3 nodes at opposite corners of 4x4m room, Pi Zero receiving
|
||||
2. **Ground truth:** Smartphone camera running MediaPipe Pose (30 FPS), synchronized via NTP
|
||||
3. **Activities:** Standing, walking, sitting, waving, falling, idle (2 minutes each)
|
||||
4. **Subjects:** 5+ volunteers with varying body types
|
||||
5. **Environments:** 3+ rooms (bedroom, office, corridor) for generalization
|
||||
6. **Total target:** ~100K synchronized CSI-pose frame pairs
|
||||
|
||||
**Synchronization approach:**
|
||||
- ESP32 and Pi Zero synchronized via NTP (< 10ms accuracy on LAN)
|
||||
- Camera frames timestamped with system clock
|
||||
- Offline alignment via cross-correlation of movement signals
|
||||
|
||||
### 6.3 Transfer Learning Strategy
|
||||
|
||||
Following DensePose-WiFi's proven approach:
|
||||
|
||||
```
|
||||
L_total = lambda_pose * L_pose
|
||||
+ lambda_bone * L_bone
|
||||
+ lambda_transfer * L_transfer
|
||||
+ lambda_physics * L_physics
|
||||
|
||||
L_transfer = MSE(features_student, features_teacher)
|
||||
```
|
||||
|
||||
Where `features_teacher` come from a pre-trained image-based pose model (HRNet or ViTPose) and `features_student` come from the WiFi CSI model at corresponding intermediate layers.
|
||||
|
||||
**Lambda schedule:**
|
||||
- Epochs 1-20: lambda_transfer = 0.5 (heavy transfer guidance)
|
||||
- Epochs 20-50: lambda_transfer = 0.2 (moderate guidance)
|
||||
- Epochs 50-100: lambda_transfer = 0.05 (fine-tuning freedom)
|
||||
|
||||
## 7. Timeline and Milestones
|
||||
|
||||
### Phase 1: Foundation (Weeks 1-4)
|
||||
|
||||
| Week | Actions | Deliverable |
|
||||
|------|---------|-------------|
|
||||
| 1 | Action 1 (protocol), ADR-069 draft | Protocol spec + parser tests |
|
||||
| 2 | Action 2 (model architecture, begin) | WiFlowPose model definition in Rust |
|
||||
| 2 | Action 3 (bone loss) | Loss functions implemented and tested |
|
||||
| 3 | Action 2 (model architecture, complete) | Full model with ONNX export |
|
||||
| 4 | Action 4 (quantization) | INT8 model, accuracy validated |
|
||||
|
||||
**Milestone M1:** WiFlowPose model trained on MM-Fi, exported to INT8 ONNX, PCK@20 > 85% on validation set.
|
||||
|
||||
### Phase 2: Edge Deployment (Weeks 5-8)
|
||||
|
||||
| Week | Actions | Deliverable |
|
||||
|------|---------|-------------|
|
||||
| 5 | Action 5 (edge engine, begin) | Cross-compilation working, model loads on Pi |
|
||||
| 6 | Action 5 (edge engine, complete) | Streaming inference at >= 10 Hz on Pi Zero |
|
||||
| 6 | Action 6 (CSI compression) | PCA compression on ESP32, verified bandwidth reduction |
|
||||
| 7 | Integration testing | ESP32 -> Pi Zero full pipeline working |
|
||||
| 8 | Performance optimization | Latency < 100ms, memory < 200 MB |
|
||||
|
||||
**Milestone M2:** End-to-end demo: ESP32 captures CSI, Pi Zero outputs pose at 10+ Hz.
|
||||
|
||||
### Phase 3: Accuracy and Adaptation (Weeks 9-12)
|
||||
|
||||
| Week | Actions | Deliverable |
|
||||
|------|---------|-------------|
|
||||
| 9 | Data collection (ESP32-S3 training data) | 50K+ synchronized CSI-pose frames |
|
||||
| 10 | Domain adaptation training | ESP32-specific model, MPJPE < 120mm |
|
||||
| 11 | Action 7 (cross-env adaptation) | Room calibration working |
|
||||
| 12 | Validation and documentation | ADR-069 finalized, witness bundle |
|
||||
|
||||
**Milestone M3:** Single-person MPJPE < 100mm in calibrated environment, cross-environment deployment working with 60-second calibration.
|
||||
|
||||
### Phase 4: Multi-Person and 3D (Weeks 13-20)
|
||||
|
||||
| Week | Actions | Deliverable |
|
||||
|------|---------|-------------|
|
||||
| 13-14 | Action 8 (multi-person PAF) | 2-person pose separation working |
|
||||
| 15-16 | Action 9 (3D lifting) | Z-axis estimation from multi-node |
|
||||
| 17-18 | Advanced optimization | Model distillation, QAT |
|
||||
| 19-20 | Production hardening | OTA updates, monitoring, alerting |
|
||||
|
||||
**Milestone M4:** Multi-person 3D pose at 10 Hz on Pi Zero 2 W.
|
||||
|
||||
## 8. Risk Analysis
|
||||
|
||||
### 8.1 Technical Risks
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Pi Zero 2 W inference too slow (> 100ms) | Medium | High | Fall back to activity recognition (smaller model); use Pi 4 instead |
|
||||
| ESP32-S3 CSI quality insufficient for pose | Low | Critical | Already validated in ADR-028; add directional antennas if needed |
|
||||
| INT8 quantization degrades accuracy > 5% | Medium | Medium | Use FP16 instead (2x size, ~1.5x slower); apply QAT |
|
||||
| Cross-environment generalization poor | High | High | Room calibration (Action 7); template-based models; continuous adaptation |
|
||||
| WiFi interference degrades CSI | Medium | Medium | Coherence gating (already implemented); channel hopping; 5 GHz fallback |
|
||||
| ONNX Runtime binary too large for Pi Zero | Low | Medium | Use OnnxStream (2 MB) instead of full ONNX Runtime (30 MB) |
|
||||
| Multi-person association errors | High | Medium | Limit to 2 persons initially; use PAF + Hungarian; AETHER re-ID |
|
||||
|
||||
### 8.2 Hardware Risks
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Pi Zero 2 W supply shortage | Medium | Medium | Design also works with Pi 3A+ or Pi 4 |
|
||||
| ESP32-S3 firmware instability | Low | Medium | Existing firmware battle-tested; OTA rollback |
|
||||
| WiFi AP interference with CSI | Low | Low | Dedicated 2.4 GHz channel; ESP32 channel hopping |
|
||||
| Power supply issues (brownout) | Low | Medium | Proper power supply; ESP32 brownout detection |
|
||||
|
||||
### 8.3 Research Risks
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| WiFlow results don't reproduce | Medium | High | Fall back to CSI-Former or MultiFormer architecture |
|
||||
| ESP32 CSI fundamentally different from Intel 5300 | Medium | High | Collect ESP32-specific training data; subcarrier interpolation |
|
||||
| Bone constraint loss doesn't improve edge accuracy | Low | Low | Remove if no benefit; constraint is simple and cheap |
|
||||
| PCA compression loses critical CSI information | Low | Medium | Validate with ablation study; fall back to raw CSI if needed |
|
||||
|
||||
## 9. Dependency Graph (Action Ordering)
|
||||
|
||||
```
|
||||
[esp32_csi_capture] (DONE)
|
||||
/ \
|
||||
v v
|
||||
[Action 1: Protocol] [training_pipeline] (DONE)
|
||||
| / | \
|
||||
v v v v
|
||||
[Action 6: Compression] [Action 2: Model] [Action 3: Bone Loss]
|
||||
| | |
|
||||
| +------+-------+
|
||||
| v
|
||||
| [Action 4: Quantization]
|
||||
| |
|
||||
+---------------+------------+
|
||||
v
|
||||
[Action 5: Edge Engine]
|
||||
|
|
||||
v
|
||||
[Action 7: Cross-Env] (Phase 2)
|
||||
|
|
||||
v
|
||||
[Action 8: Multi-Person] (Phase 2)
|
||||
|
|
||||
v
|
||||
[Action 9: 3D Lifting] (Phase 3)
|
||||
```
|
||||
|
||||
**Critical path:** Action 1 -> Action 2 -> Action 4 -> Action 5
|
||||
**Parallel path:** Action 3 can proceed concurrently with Action 2
|
||||
**Parallel path:** Action 6 can proceed concurrently with Actions 2-4
|
||||
|
||||
## 10. Success Criteria
|
||||
|
||||
### Phase 1 Exit Criteria
|
||||
|
||||
- [ ] WiFlowPose model trains to convergence on MM-Fi dataset
|
||||
- [ ] PCK@20 >= 85% on MM-Fi validation set
|
||||
- [ ] INT8 ONNX model size < 5 MB
|
||||
- [ ] Bone constraint loss reduces physically implausible predictions by > 50%
|
||||
|
||||
### Phase 2 Exit Criteria
|
||||
|
||||
- [ ] edge_infer binary cross-compiles for aarch64 and runs on Pi Zero 2 W
|
||||
- [ ] End-to-end latency < 150ms (CSI capture to pose output)
|
||||
- [ ] Inference rate >= 10 Hz sustained
|
||||
- [ ] PCA compression reduces bandwidth by >= 3x without > 5% accuracy loss
|
||||
- [ ] Multi-node support (2 ESP32 nodes + 1 Pi Zero) working
|
||||
|
||||
### Phase 3 Exit Criteria
|
||||
|
||||
- [ ] Single-person MPJPE < 100mm in calibrated environment
|
||||
- [ ] Cross-environment deployment works with 60-second calibration
|
||||
- [ ] System runs continuously for 24 hours without crashes
|
||||
- [ ] ESP32 OTA firmware update working for CSI compression parameters
|
||||
|
||||
### Phase 4 Exit Criteria
|
||||
|
||||
- [ ] 2-person pose separation working (MPJPE < 150mm per person)
|
||||
- [ ] 3D pose estimation from 2+ nodes (Z-axis error < 200mm)
|
||||
- [ ] Production monitoring and alerting operational
|
||||
|
||||
## 11. Relationship to Existing ADRs
|
||||
|
||||
| ADR | Relationship |
|
||||
|-----|-------------|
|
||||
| ADR-018 | Protocol v2 (Action 1) extends ADR-018 binary frame format |
|
||||
| ADR-024 | AETHER re-ID embeddings used in multi-person tracking (Action 8) |
|
||||
| ADR-027 | MERIDIAN cross-env generalization informs Action 7 |
|
||||
| ADR-028 | ESP32 capability audit validates CSI quality assumptions |
|
||||
| ADR-029 | RuvSense pipeline stages feed into edge inference (Action 5) |
|
||||
| ADR-068 | Per-node state pipeline directly used by multi-node inference |
|
||||
|
||||
## 12. New ADR Required
|
||||
|
||||
**ADR-069: Edge Inference on Raspberry Pi Zero 2 W**
|
||||
|
||||
This implementation plan should be formalized as ADR-069 covering:
|
||||
- Protocol v2 specification
|
||||
- WiFlowPose architecture selection rationale
|
||||
- Pi Zero deployment constraints and optimizations
|
||||
- INT8 quantization strategy
|
||||
- Cross-compilation approach
|
||||
- Environment calibration protocol
|
||||
|
||||
Status: Proposed, pending this plan's approval.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,434 @@
|
||||
# Three-Tier Rust Node — Exploratory Architecture
|
||||
|
||||
| Field | Value |
|
||||
|--------------|------------------------------------------------------------------------|
|
||||
| **Status** | Exploratory / not yet decided |
|
||||
| **Date** | 2026-04-25 |
|
||||
| **Authors** | ruv (proposal), filed by goal-planner research agent |
|
||||
| **Classifies as** | Speculative architectural alternative to ADR-028 / ADR-081 baseline |
|
||||
| **Companion**| `docs/research/sota/2026-Q2-rf-sensing-and-edge-rust.md` (SOTA), `docs/research/architecture/decision-tree.md` (decisions) |
|
||||
|
||||
> **Reading note.** This document files a long architectural exploration the
|
||||
> author wrote before any commitment. It is intentionally optimistic in places
|
||||
> and will be tempered by the SOTA survey filed alongside it. The decision
|
||||
> tree document maps each load-bearing claim to the evidence that would
|
||||
> justify acting on it. Nothing in this document supersedes ADR-028 (the
|
||||
> capability audit) or ADR-081 (the 5-layer adaptive kernel). Both already
|
||||
> describe a working, single-MCU node; this document describes a
|
||||
> hypothetical *three-tier* node that would replace it on PCBs that ship
|
||||
> Pi-class compute next to two ESP32-class radios on a solar-powered HAT.
|
||||
|
||||
---
|
||||
|
||||
## 1. ADRs this proposal would touch
|
||||
|
||||
If pursued, this proposal evolves the following decisions. None are
|
||||
overturned outright; all need re-read in this light.
|
||||
|
||||
- **ADR-028 — ESP32 Capability Audit.** Today's witnessed node is a single
|
||||
ESP32-S3 streaming raw ADR-018 frames over UDP. A three-tier node changes
|
||||
the audit subject from "one MCU" to "two MCUs + a Pi", with implications
|
||||
for the witness bundle, firmware-manifest hashes, and per-node BOM.
|
||||
- **ADR-081 — Adaptive CSI Mesh Firmware Kernel.** The 5-layer kernel
|
||||
already separates radio abstraction (L1), adaptive control (L2), mesh
|
||||
plane (L3), feature extraction (L4), and Rust handoff (L5). A three-tier
|
||||
node would split L1–L2 onto a no_std sensor MCU, L3 onto an ESP-IDF
|
||||
comms MCU, and Layer-5+ Rust workload onto the Pi. The split is
|
||||
compatible with the kernel; it is a deployment shape rather than a
|
||||
redesign.
|
||||
- **ADR-018 — ESP32 Dev Implementation.** ADR-018 binary CSI frames remain
|
||||
the wire format between the sensor MCU and whoever consumes them. The
|
||||
three-tier proposal tightens the contract: ADR-018 frames flow from
|
||||
sensor MCU into the comms MCU only, never directly off the node.
|
||||
- **ADR-029 / ADR-031 — Multistatic and sensing-first RF mode.** A
|
||||
hardware-gated Pi Zero 2W enables the sensing-first mode to actually
|
||||
hibernate the heavy compute, which ADR-031's power model assumes but the
|
||||
current node cannot deliver because heavy compute lives off-node.
|
||||
- **ADR-032 — Multistatic mesh security hardening.** HMAC-SHA256 beacon
|
||||
auth + SipHash-2-4 frame integrity in ADR-032 already cover the
|
||||
inter-node bus. The proposal adds Secure Boot V2 + flash encryption
|
||||
at-rest on each MCU, and a signed Pi A/B image, which are *complements*
|
||||
to ADR-032, not substitutes.
|
||||
|
||||
---
|
||||
|
||||
## 2. Motivating thesis
|
||||
|
||||
A WiFi/RF sensing node has three jobs that prefer three different
|
||||
runtimes:
|
||||
|
||||
1. **Strict-real-time radio capture and DSP** — sub-millisecond ISR
|
||||
discipline, no allocator surprises, predictable interrupt latency.
|
||||
2. **Networking, OTA, mesh, time sync** — TCP/IP, TLS, BLE provisioning,
|
||||
ESP-WIFI-MESH, OTA bootloaders, NVS. The full battery of WiFi-stack
|
||||
features that come with ESP-IDF and FreeRTOS.
|
||||
3. **Heavy compute, ML inference, storage, fleet sync** — gigabytes of
|
||||
model weights, vision inference, persistent storage, QUIC-based fleet
|
||||
sync, optional cloud APIs.
|
||||
|
||||
Today's RuView node tries to fit jobs 1 and 2 onto one ESP32-S3, and job 3
|
||||
either runs on a separate machine (the "sensing-server" host) or is
|
||||
absent. The thesis of this proposal is that **collapsing all three onto
|
||||
a single PCB but onto three separate dies** captures most of the
|
||||
"single node" simplicity without sacrificing the runtime properties of
|
||||
each layer. Concretely:
|
||||
|
||||
- **Sensor MCU** — ESP32-S3, no_std, `esp-hal` + Embassy + `heapless` +
|
||||
`postcard`. ISR-driven CSI capture, channel hopping, short-window DSP.
|
||||
No WiFi stack of its own (the radio is in the comms MCU); a private
|
||||
UART or SPI link to the comms MCU carries serialized frames. *(See SOTA
|
||||
survey, §3, for the ISR-safety caveat that tempers this.)*
|
||||
- **Comms MCU** — second ESP32-S3, ESP-IDF, `esp-idf-svc` + `esp-idf-sys`,
|
||||
TLS/HTTPS/OTA/ESP-WIFI-MESH, NVS provisioning, BLE provisioning, LoRa
|
||||
fallback. Owns the "outside world."
|
||||
- **Pi Zero 2W** — *normally power-gated*. Wakes on event from the comms
|
||||
MCU, runs heavy ML or fleet-sync work, optionally streams QUIC to a
|
||||
gateway, then power-gates again. `tokio` + `quinn` + `rustls` + `axum`.
|
||||
|
||||
A single PCB, a single 1S Li-ion + 2 W solar + linear charger, a single
|
||||
enclosure. Three separate cores each running the runtime they are
|
||||
actually good at.
|
||||
|
||||
---
|
||||
|
||||
## 3. Hardware shape (proposed)
|
||||
|
||||
### 3.1 Bill of materials (per node, target)
|
||||
|
||||
| Slot | Part | Notes |
|
||||
|---------------------|--------------------------------------------------|---------------------------------------------------|
|
||||
| Sensor MCU | ESP32-S3-WROOM-1 (8 MB flash, 8 MB PSRAM) | no_std, Embassy, esp-radio. Always-on. |
|
||||
| Comms MCU | ESP32-S3-MINI-1 or -WROOM-1 (4 MB flash) | ESP-IDF, ESP-WIFI-MESH, OTA, TLS. Mostly-on. |
|
||||
| Heavy compute | Pi Zero 2W (1 GB RAM) | Power-gated by default. Wake on event. |
|
||||
| LoRa fallback | Semtech SX1262 module | Heartbeat + recovery only. Sub-GHz. |
|
||||
| Charger / PMIC | TI BQ24074 (linear) or BQ25798 (buck-boost MPPT) | See SOTA §7 for trade-off. |
|
||||
| Battery | 1S Li-ion 18650 (3.0 Ah class) | Standard cell, easy to source. |
|
||||
| Solar panel | ~2 W, 6 V, IP-rated | Roof-mount or window-mount. |
|
||||
| Pi power gate | Logic-level P-FET high-side switch + ESP GPIO | Hard-cut when idle (350 mA → ~0 mA). |
|
||||
| Inter-MCU bus | UART or SPI between sensor MCU and comms MCU | Postcard-framed binary on a 4-wire link. |
|
||||
| Comms-to-Pi bus | UART (115200–921600 bps) or SPI | Pi-side `tokio-serial`/`spidev`. |
|
||||
| Enclosure | IP54 or IP65 with antenna pass-through | - |
|
||||
| Estimated BOM | $40–55 | At small build qty; falls with volume. |
|
||||
|
||||
This is roughly 4–6× the ~$9 single-S3 node, which is the largest
|
||||
single mark against the proposal. See §7.4 for whether the cost makes
|
||||
sense.
|
||||
|
||||
### 3.2 Power-state hierarchy (proposed)
|
||||
|
||||
| State | Sensor MCU | Comms MCU | Pi Zero 2W | Approx draw |
|
||||
|----------------|------------------|-----------------|------------------|-----------------|
|
||||
| Deep idle | light sleep | DTIM-modulated | hard-off | < 5 mA |
|
||||
| Sample window | active CSI | passive listen | hard-off | ~80 mA |
|
||||
| Event publish | active CSI | TX burst | hard-off | ~150 mA peak |
|
||||
| Escalation | active CSI | TX + bring-up | booting | ~350 mA peak |
|
||||
| ML in progress | active CSI | passive | inferencing | ~450 mA |
|
||||
| Recovery | sleep | LoRa heartbeat | hard-off | ~30 mA |
|
||||
|
||||
The Pi is treated as the heavyweight worker that **must** be hard-power-
|
||||
gated — not soft-suspended — when not in use. ARM SoCs leak in
|
||||
suspend; a 350 mA "off" leakage destroys solar viability.
|
||||
|
||||
### 3.3 Energy budget sketch
|
||||
|
||||
- **Daily load** (sketch, *not measured*): ~1.4 Wh/day assuming Pi wakes
|
||||
≤ 2 minutes/day on average, sensor MCU light-sleeps when idle, comms
|
||||
MCU DTIM-3 most of the time.
|
||||
- **Daily harvest**: 2 W panel × 4 PSH × 0.7 system efficiency ≈ 5.6
|
||||
Wh/day in the seasonal worst case for mid-latitudes.
|
||||
|
||||
Headroom is roughly 4×. If a deployment skews colder/cloudier, or the
|
||||
inter-MCU bus runs hotter, headroom is 2–3×. SOTA §7 covers whether
|
||||
the linear-charger + supercap-buffered topology actually delivers this
|
||||
math, or whether MPPT is needed on a panel this small.
|
||||
|
||||
---
|
||||
|
||||
## 4. Software shape (proposed)
|
||||
|
||||
### 4.1 Sensor MCU — no_std embedded Rust
|
||||
|
||||
| Concern | Crate(s) |
|
||||
|----------------------|--------------------------------------------------------------|
|
||||
| HAL / async runtime | `esp-hal` 1.x + Embassy executor |
|
||||
| Time / timers | `embassy-time` |
|
||||
| Static allocations | `heapless` (`Vec`, `String`, `Deque`, MPMC channels) |
|
||||
| Wire format | `postcard` over `serde` for compact, schema-stable bytes |
|
||||
| CRC | `crc` crate (already used host-side for the L4 packet check) |
|
||||
| RF capture | `esp-radio` (the rename of `esp-wifi`) — CSI hooks via PR |
|
||||
| Inter-MCU bus | `embassy-uart` or `embedded-hal-async` SPI |
|
||||
| Power management | `esp-hal::system::sleep::*` + light-sleep wake on GPIO/timer |
|
||||
|
||||
Boundary: the sensor MCU does **not** initialize a WiFi stack. It owns
|
||||
the PHY for CSI capture only. All actual WiFi connectivity is on the
|
||||
comms MCU. This is the load-bearing simplification of the proposal: it
|
||||
sidesteps the embassy-on-ESP-IDF ISR-safety question by not running
|
||||
ESP-IDF on this die at all.
|
||||
|
||||
### 4.2 Comms MCU — std + ESP-IDF Rust
|
||||
|
||||
| Concern | Crate(s) |
|
||||
|----------------------|--------------------------------------------------------------------------|
|
||||
| FreeRTOS bindings | `esp-idf-sys` |
|
||||
| Service abstractions | `esp-idf-svc` (HTTPS, OTA, NVS, mDNS, BLE, MQTT, ESP-NOW) |
|
||||
| Async runtime | `esp-idf-svc::timer::EspTaskTimerService` (NOT Embassy directly — see §6)|
|
||||
| TLS | mbedTLS via `esp-idf-svc` |
|
||||
| Mesh | ESP-WIFI-MESH (or ESP-MESH-LITE — see SOTA §8) |
|
||||
| OTA | ESP-IDF native OTA (signed images, A/B partitions) |
|
||||
| LoRa fallback | `lora-phy` or vendor C driver via `esp-idf-sys` |
|
||||
| Inter-MCU bus | UART driver (`esp-idf-svc::uart`) framed with postcard |
|
||||
| BLE provisioning | NimBLE via `esp-idf-svc` |
|
||||
|
||||
The comms MCU is the *only* die that needs the full WiFi-stack security
|
||||
surface. That makes it the obvious place to enforce Secure Boot V2 +
|
||||
flash encryption + signed OTA.
|
||||
|
||||
### 4.3 Pi Zero 2W — std Rust on Linux
|
||||
|
||||
| Concern | Crate(s) |
|
||||
|----------------------|-----------------------------------------------------------------------|
|
||||
| Async runtime | `tokio` |
|
||||
| QUIC | `quinn` + `rustls` |
|
||||
| HTTP server (local) | `axum` |
|
||||
| RPC to comms MCU | `tokio-serial` (UART) or `spidev` (SPI), framed with postcard |
|
||||
| ML inference | `tract` (ONNX), `candle` (Pytorch-flavored), or `ort` (ONNX Runtime) |
|
||||
| Persistent storage | `sled` or `redb` |
|
||||
| OS | Buildroot-based custom image, A/B partitions, dm-verity, signed |
|
||||
|
||||
Crucial constraint: the Pi runs **buildroot**, not Raspberry Pi OS. The
|
||||
Raspberry Pi Foundation does not officially support secure boot on the
|
||||
Pi Zero 2W; the secure-boot path is Pi 4/5-only. The cleanest path on a
|
||||
Pi Zero 2W is buildroot + signed FIT image + dm-verity on the rootfs +
|
||||
A/B partitions for OTA. See SOTA §9 for the realistic version of this.
|
||||
|
||||
### 4.4 OTA on three dies
|
||||
|
||||
| Die | OTA mechanism |
|
||||
|--------------|-----------------------------------------------------------------------|
|
||||
| Sensor MCU | `embassy-boot`-style two-slot OTA, signed images, ed25519 verification|
|
||||
| Comms MCU | ESP-IDF native OTA, signed by project key, dual app partitions |
|
||||
| Pi Zero 2W | A/B rootfs, signed FIT, fwupd or homemade `update-agent` binary |
|
||||
|
||||
OTA is the area where the three-tier shape is most defensible. Each die's
|
||||
update is a separate, independently rollback-able artifact. The comms
|
||||
MCU acts as the *broker* — it pulls signed images for all three dies,
|
||||
verifies them, and pushes them onto the sensor MCU and Pi over their
|
||||
respective buses.
|
||||
|
||||
---
|
||||
|
||||
## 5. Networking shape (proposed)
|
||||
|
||||
Three concentric rings:
|
||||
|
||||
1. **Inner ring — node-local IPC.** Postcard over UART/SPI between the
|
||||
three dies. Length-prefixed, CRC-checked, no encryption (it's on a
|
||||
trace, not a wire).
|
||||
2. **Middle ring — RuView mesh.** ESP-WIFI-MESH (or ESP-MESH-LITE)
|
||||
between comms MCUs across nodes, carrying L3 mesh-plane messages
|
||||
from ADR-081 (TIME_SYNC, ROLE_ASSIGN, CHANNEL_PLAN, FEATURE_DELTA,
|
||||
HEALTH, ANOMALY_ALERT). Authenticated with HMAC-SHA256 per ADR-032.
|
||||
3. **Outer ring — backhaul.** QUIC from the Pi to a gateway/cloud
|
||||
target (`quinn` + `rustls`), with the gateway optionally being
|
||||
another node's Pi acting as a fusion-relay. LoRa is the *fallback*
|
||||
ring for heartbeats and recovery commands when the WiFi mesh is
|
||||
degraded.
|
||||
|
||||
LoRa duty-cycle math (EU868 1% in the relevant sub-band, US915 dwell-
|
||||
time-only) is friendly to "20 bytes every minute" heartbeats; at SF7,
|
||||
125 kHz, the airtime is ~40 ms per packet — far under the 36 s/hour
|
||||
EU868 limit. See SOTA §6 for the citation.
|
||||
|
||||
---
|
||||
|
||||
## 6. Security posture (proposed)
|
||||
|
||||
The proposal layers four mechanisms on each MCU:
|
||||
|
||||
- **Secure Boot V2** — RSA-3072 or ECDSA signed bootloader, immutable
|
||||
primary key digest in eFuse.
|
||||
- **Flash encryption** — AES-XTS-256 with per-device key burned in eFuse,
|
||||
hardware-isolated.
|
||||
- **Disabled ROM download** — `DIS_DOWNLOAD_MODE` fuse blown after
|
||||
provisioning so the device cannot be coerced back into a UART-ROM
|
||||
state.
|
||||
- **Signed OTA images** — separate signing key from the secure-boot key,
|
||||
per-image rollback counter, anti-rollback eFuse counter.
|
||||
|
||||
On the Pi: dm-verity over a read-only rootfs, signed FIT image with the
|
||||
RPi-foundation-blessed (where possible) bootcode, A/B partitions, and a
|
||||
signed manifest of the three dies' image hashes shipped together. The
|
||||
comms MCU validates the manifest before consuming any image.
|
||||
|
||||
This is **complementary** to ADR-032's HMAC-SHA256 + SipHash-2-4 mesh
|
||||
hardening — those protect frames in flight; Secure Boot + flash
|
||||
encryption protect images at rest.
|
||||
|
||||
---
|
||||
|
||||
## 7. Honest critique of this proposal
|
||||
|
||||
This section is required by the project conventions. The companion SOTA
|
||||
survey expands each of these.
|
||||
|
||||
### 7.1 The cost story is bad before volume
|
||||
|
||||
A single ESP32-S3 node is ~$9 today. A three-tier node is closer to
|
||||
$40–55. RuView's design point of "many cheap nodes" rewards low BOM. The
|
||||
three-tier shape is justified only if each node *also* replaces a
|
||||
sensing-server host (i.e., a Pi or laptop running the sensing pipeline)
|
||||
that would have cost more than the marginal Pi-on-each-node. In a
|
||||
deployment with 3 nodes feeding one $80 host, the host already amortizes
|
||||
across the nodes. In a 50-node deployment, the math changes.
|
||||
|
||||
### 7.2 The embassy-on-ESP-IDF ISR-safety question is real
|
||||
|
||||
The proposal *avoids* this question by giving the sensor MCU a no_std
|
||||
runtime instead of putting embassy on top of esp-idf-svc. The reason
|
||||
this matters: per esp-idf-svc maintainers, **embassy-executor is not
|
||||
ISR-safe** in the esp-idf-svc setup (it relies on `critical-section`,
|
||||
which on esp-idf-hal is implemented over FreeRTOS task suspension). On
|
||||
no_std with `esp-hal`, embassy is fine; on top of ESP-IDF, it is not.
|
||||
The two-MCU split is the cleanest engineering answer to the question;
|
||||
the alternative is keeping ESP-IDF on the single MCU (today's design)
|
||||
and not introducing embassy at all. SOTA §3 documents the citation.
|
||||
|
||||
### 7.3 esp-radio replaces esp-wifi, and CSI no_std support is partial
|
||||
|
||||
The crate that the sensor MCU would use to capture CSI (in the
|
||||
`esp-rs/esp-hal` 1.x ecosystem) was renamed to `esp-radio`. Third-party
|
||||
`esp-csi-rs` exists and targets no_std but is described as
|
||||
"early development." The 5-layer kernel today runs on top of ESP-IDF
|
||||
v5.4 in C — a bird in the hand. Migrating CSI capture to no_std is a
|
||||
distinct project, not a side effect of the three-tier shape. SOTA §2
|
||||
covers the maturity matrix.
|
||||
|
||||
### 7.4 The Pi Zero 2W secure-boot story is weaker than the proposal implies
|
||||
|
||||
The Raspberry Pi Foundation's official secure-boot path is **Pi 4 / Pi 5
|
||||
only**, with a USB-rooted RSA chain. There is no official secure-boot
|
||||
bring-up document for the Pi Zero 2W. Buildroot + signed FIT + dm-verity
|
||||
gets you most of the threat surface — but the proposal's "Pi 4 + buildroot
|
||||
is the strongest path" line is not a Pi Zero 2W story. If true secure
|
||||
boot matters for the deployment, the heavy-compute die should arguably
|
||||
be a Pi 4 Compute Module (CM4) and not a Pi Zero 2W. SOTA §9 covers it.
|
||||
|
||||
### 7.5 ESP-WIFI-MESH at 50–500 nodes is an open question
|
||||
|
||||
Espressif documents up to 1,000 nodes and 25 layers as theoretical limits
|
||||
for ESP-WIFI-MESH, with a recommended fan-out of 6 per node. There is
|
||||
limited public evidence of stable 100+ node deployments in adversarial
|
||||
RF environments. Comms-MCU mesh handling at scale is *not free*: the
|
||||
mesh stack runs in the comms MCU's main loop, sharing CPU with TLS, OTA,
|
||||
and BLE. SOTA §8 covers BLE Mesh / Thread / Zigbee comparison. None of
|
||||
those replace WiFi-stack-sharing for CSI capture, but they could replace
|
||||
ESP-WIFI-MESH for control-plane traffic if scale becomes a problem.
|
||||
|
||||
### 7.6 MPPT vs linear charger at 2 W panel
|
||||
|
||||
The proposal's BQ24074-based linear-charger topology is fine for a 2 W
|
||||
panel; the efficiency loss vs MPPT is real but small at this scale.
|
||||
At 2 W, the MPPT die (BQ25798) silicon, inductor, and code complexity
|
||||
costs partly cancel its efficiency gain. SOTA §7 has the math.
|
||||
|
||||
### 7.7 The QUIC outer ring is overkill for the heartbeat case
|
||||
|
||||
QUIC is a strong choice when the Pi has lots of bursty data and is
|
||||
behind a NAT or on flaky cellular. For a node that wakes 2 minutes/day
|
||||
and emits a few KB of summarized features, MQTT-over-TLS or even
|
||||
plain HTTPS is simpler and adequate. QUIC's value goes up if the Pi
|
||||
also runs bidirectional model updates or large-batch fleet sync.
|
||||
SOTA §5.
|
||||
|
||||
---
|
||||
|
||||
## 8. What evidence would justify acting on this proposal
|
||||
|
||||
This section maps to the decision tree in
|
||||
`docs/research/architecture/decision-tree.md`. The short version:
|
||||
|
||||
1. **Per-node cost ceiling.** Decide the BOM ceiling per node. The
|
||||
three-tier shape only makes sense above ~$30/node and at deployments
|
||||
where the host computer is *not* a separate cost.
|
||||
2. **CSI no_std maturity gate.** `esp-csi-rs` (or the replacement under
|
||||
`esp-radio`) must demonstrate equivalent capture quality to today's
|
||||
`esp-wifi-set-csi-rx-cb`-based path on a real ESP32-S3 board, with
|
||||
ISR-jitter measured. Until this is verified, the sensor-MCU Rust
|
||||
story is risk.
|
||||
3. **Inter-MCU bus saturation.** Postcard-framed UART/SPI between the
|
||||
sensor MCU and comms MCU must carry ADR-018 frames at the target
|
||||
capture rate without backpressure-induced drops at the sensor MCU.
|
||||
4. **Pi power-gate budget.** Measured leakage of the gated Pi Zero 2W,
|
||||
with proven cold-boot wake-up under 5 s, is required before the
|
||||
energy budget closes.
|
||||
5. **Mesh scale evidence.** A 12+ node ESP-WIFI-MESH (or alternative)
|
||||
field test at sustained 1–10 Hz `rv_feature_state_t` upload is
|
||||
required to validate the middle ring at >>3 nodes.
|
||||
6. **Secure-boot path on Pi Zero 2W.** Either accept that the Pi cannot
|
||||
be fully secure-booted, or upgrade the heavy-compute die to a CM4 /
|
||||
CM5 / Pi 5 if true secure boot is a deployment requirement.
|
||||
|
||||
---
|
||||
|
||||
## 9. Open questions
|
||||
|
||||
The proposal as written elides answers to these:
|
||||
|
||||
- **Why two ESP32-S3 dies and not one ESP32-S3 plus one ESP32-C6?** The
|
||||
C6 is RISC-V, has 802.15.4 + WiFi 6, and would let the comms MCU
|
||||
handle BLE Mesh / Thread / Zigbee natively. The two-S3 split chose
|
||||
homogeneity and Xtensa toolchain; the C6 split chooses richer
|
||||
protocol coverage on the comms die.
|
||||
- **Is the sensor MCU strictly necessary?** Today, the single-MCU node
|
||||
(ADR-028 / ADR-081) handles CSI capture and ESP-IDF networking on one
|
||||
S3, in C, and works. The two-MCU-on-board case is justified mainly by
|
||||
*ISR purity* and *Rust no_std*, not by a missing capability today.
|
||||
- **Why a Pi Zero 2W rather than the Pi being the gateway?** The
|
||||
proposal puts a Pi *on every node*. A more conservative shape is one
|
||||
Pi per *site* (or per cluster of 3–6 nodes), with the nodes staying
|
||||
single-MCU. That keeps the BOM near today's $9/node for sensors,
|
||||
isolates heavy compute, and concentrates secure boot on a smaller
|
||||
number of more capable dies. This is the deployment shape implicit in
|
||||
ADR-031's sensing-first mode and is worth comparing head-to-head.
|
||||
- **What does a single 50-node deployment cost** under each of: today's
|
||||
shape (one S3 + one host), one-Pi-per-site (one S3 + one Pi per ~6
|
||||
nodes), and the proposal (3-die-per-node)? The cost crossover point
|
||||
determines which architecture is correct.
|
||||
|
||||
---
|
||||
|
||||
## 10. Recommendation
|
||||
|
||||
This document records the proposal accurately. It does not recommend
|
||||
adopting it. The recommendation, if a decision is forced, is:
|
||||
|
||||
1. **Do not build a three-tier-per-node PCB now.** The current shape
|
||||
(single ESP32-S3 + ADR-081 5-layer kernel) is the witnessed system.
|
||||
2. **Investigate one-Pi-per-site as the cheaper variant** (proposal §9
|
||||
bullet 3). It captures most of the heavy-compute and QUIC-backhaul
|
||||
benefits at a fraction of the BOM.
|
||||
3. **Spend the first chunk of effort on the three "evidence" gates from
|
||||
§8** — CSI no_std maturity, ESP-WIFI-MESH at scale, and Pi
|
||||
secure-boot reality — *before* committing to a hardware re-spin.
|
||||
4. **Reserve the three-tier shape** for a future "RuView Pro" SKU
|
||||
targeting deployments where per-node BOM is not the dominant cost
|
||||
and full secure-boot + dm-verity at the edge is mandatory.
|
||||
|
||||
The decision tree document codifies these gates as branch points so
|
||||
they can be checked off independently rather than as one large
|
||||
all-or-nothing ADR.
|
||||
|
||||
---
|
||||
|
||||
## 11. Companion documents
|
||||
|
||||
- **SOTA survey.** `docs/research/sota/2026-Q2-rf-sensing-and-edge-rust.md`
|
||||
— citations, primary sources, what's true in 2026 for each load-bearing
|
||||
claim above.
|
||||
- **Decision tree.** `docs/research/architecture/decision-tree.md` — the
|
||||
Mermaid map from each load-bearing decision to its dependencies and
|
||||
ADR slot.
|
||||
- **Existing implementation plan.** `docs/research/architecture/implementation-plan.md`
|
||||
— the ESP32-S3 + Pi Zero 2W goal-state plan from 2026-04-02. The
|
||||
three-tier proposal is most usefully read as an evolution of *that*
|
||||
plan rather than a replacement of ADR-028.
|
||||
Reference in New Issue
Block a user