docs: update changelog, user guide, and README for ADR-043 changes

- CHANGELOG: add ADR-043 entries (14 new API endpoints, WebSocket fix, mobile WS fix, 25 real mobile tests) - README: update ADR count from 41 to 43 - CLAUDE.md: update ADR count from 32 to 43 - User guide: add 14 new REST endpoints to API reference table, note that /ws/sensing is available on the HTTP port, update ADR count Co-Authored-By: claude-flow <ruv@ruv.net>
2026-07-27 18:11:43 +00:00 · 2026-03-03 15:20:57 -05:00
4 changed files with 25 additions and 352 deletions
@@ -1,214 +0,0 @@
-# ADR-044: Provisioning Tool Enhancements
-
-**Status**: Proposed
-**Date**: 2026-03-03
-**Deciders**: @ruvnet
-**Supersedes**: None
-**Related**: ADR-029, ADR-032, ADR-039, ADR-040
-
---
-
-## Context
-
-The ESP32-S3 CSI node provisioning script (`firmware/esp32-csi-node/provision.py`) is the primary tool for configuring pre-built firmware binaries without recompiling. It writes NVS key-value pairs that the firmware reads at boot.
-
-After #131 added TDM and edge intelligence flags, the script now covers the most-requested NVS keys. However, there remain gaps between what the firmware reads from NVS (`nvs_config.c`, 20 keys) and what the provisioning script can write (13 keys). Additionally, the script lacks usability features that would help field operators deploying multi-node meshes.
-
-### Gap 1: Missing NVS Keys (7 keys)
-
-The firmware reads these NVS keys at boot but the provisioning script has no corresponding CLI flags:
-
-| NVS Key | Type | Firmware Default | Purpose |
-|---------|------|-----------------|---------|
-| `hop_count` | u8 | 1 (no hop) | Number of channels to hop through |
-| `chan_list` | blob (u8[6]) | {1,6,11} | Channel numbers for hopping sequence |
-| `dwell_ms` | u32 | 100 | Time to dwell on each channel before hopping (ms) |
-| `power_duty` | u8 | 100 | Power duty cycle percentage (10-100%) for battery life |
-| `wasm_max` | u8 | 4 | Max concurrent WASM modules (ADR-040) |
-| `wasm_verify` | u8 | 0 | Require Ed25519 signature for WASM uploads (0/1) |
-| `wasm_pubkey` | blob (32B) | zeros | Ed25519 public key for WASM signature verification |
-
-### Gap 2: No Read-Back
-
-There is no way to read the current NVS configuration from a device. Field operators must remember what was provisioned or reflash everything. This is especially problematic for multi-node meshes where each node has different TDM slots.
-
-### Gap 3: No Verification
-
-After flashing, there is no automated check that the device booted successfully with the new configuration. Operators must manually run a serial monitor and inspect logs.
-
-### Gap 4: No Config File Support
-
-Provisioning a 6-node mesh requires running the script 6 times with largely overlapping flags (same SSID, password, target IP) and only TDM slot varying. There is no way to define a mesh configuration in a file.
-
-### Gap 5: No Presets
-
-Common deployment scenarios (single-node basic, 3-node mesh, 6-node mesh with vitals) require operators to know which flags to combine. Named presets would lower the barrier to entry.
-
-### Gap 6: No Auto-Detect
-
-The `--port` flag is required even though the script could auto-detect connected ESP32-S3 devices via `esptool.py`.
-
---
-
-## Decision
-
-Enhance `provision.py` with the following capabilities, implemented incrementally.
-
-### Phase 1: Complete NVS Coverage
-
-Add flags for all remaining firmware NVS keys:
-
-```
--hop-count N          Channel hop count (1=no hop, default: 1)
--channels 1,6,11      Comma-separated channel list for hopping
--dwell-ms N           Dwell time per channel in ms (default: 100)
--power-duty N         Power duty cycle 10-100% (default: 100)
--wasm-max N           Max concurrent WASM modules 1-8 (default: 4)
--wasm-verify          Require Ed25519 signature for WASM uploads
--wasm-pubkey FILE     Path to Ed25519 public key file (32 bytes raw or PEM)
-```
-
-Validation:
- `--channels` length must match `--hop-count`
- `--power-duty` clamped to 10-100
- `--wasm-pubkey` implies `--wasm-verify`
-
-### Phase 2: Config File and Mesh Provisioning
-
-Add `--config FILE` to load settings from a JSON or TOML file:
-
-```json
-{
-  "common": {
-    "ssid": "SensorNet",
-    "password": "secret",
-    "target_ip": "192.168.1.20",
-    "target_port": 5005,
-    "edge_tier": 2
-  },
-  "nodes": [
-    { "port": "COM7", "node_id": 0, "tdm_slot": 0 },
-    { "port": "COM8", "node_id": 1, "tdm_slot": 1 },
-    { "port": "COM9", "node_id": 2, "tdm_slot": 2 }
-  ]
-}
-```
-
-`--config mesh.json` provisions all listed nodes in sequence, computing `tdm_total` automatically from the `nodes` array length.
-
-### Phase 3: Presets
-
-Add `--preset NAME` for common deployment profiles:
-
-| Preset | What It Sets |
-|--------|-------------|
-| `basic` | Single node, edge_tier=0, no TDM, no hopping |
-| `vitals` | Single node, edge_tier=2, vital_int=1000, subk_count=32 |
-| `mesh-3` | 3-node TDM, edge_tier=1, hop_count=3, channels=1,6,11 |
-| `mesh-6-vitals` | 6-node TDM, edge_tier=2, hop_count=3, channels=1,6,11, vital_int=500 |
-
-Presets set defaults that can be overridden by explicit flags.
-
-### Phase 4: Read-Back and Verify
-
-Add `--read` to dump the current NVS configuration from a connected device:
-
-```bash
-python provision.py --port COM7 --read
-# Output:
-#   ssid:         SensorNet
-#   target_ip:    192.168.1.20
-#   tdm_slot:     0
-#   tdm_nodes:    3
-#   edge_tier:    2
-#   ...
-```
-
-Implementation: use `esptool.py read_flash` to read the NVS partition, then parse the NVS binary format to extract key-value pairs.
-
-Add `--verify` to provision and then confirm the device booted:
-
-```bash
-python provision.py --port COM7 --ssid "Net" --password "pass" --target-ip 192.168.1.20 --verify
-# After flash, opens serial monitor for 5 seconds
-# Checks for "CSI streaming active" log line
-# Reports PASS or FAIL
-```
-
-### Phase 5: Auto-Detect Port
-
-When `--port` is omitted, scan for connected ESP32-S3 devices:
-
-```bash
-python provision.py --ssid "Net" --password "pass" --target-ip 192.168.1.20
-# Auto-detected ESP32-S3 on COM7 (Silicon Labs CP210x)
-# Proceed? [Y/n]
-```
-
-Implementation: use `esptool.py` or `serial.tools.list_ports` to enumerate ports.
-
---
-
-## Rationale
-
-### Why incremental phases?
-
-Phase 1 is a small diff that closes the NVS coverage gap immediately. Phases 2-5 add progressively more UX polish. Each phase is independently useful and can be shipped separately.
-
-### Why JSON config over YAML/TOML?
-
-JSON requires no additional Python dependencies (stdlib `json` module). TOML requires `tomllib` (Python 3.11+) or `tomli`. JSON is sufficient for this use case.
-
-### Why not a GUI?
-
-The target users are embedded developers and field operators who are already running `esptool` from the command line. A TUI/GUI would add dependencies and complexity for minimal benefit.
-
---
-
-## Consequences
-
-### Positive
-
- **Complete NVS coverage**: Every firmware-readable key can be set from the provisioning tool
- **Mesh provisioning in one command**: `--config mesh.json` replaces 6 separate invocations
- **Lower barrier to entry**: Presets eliminate the need to know which flags to combine
- **Auditability**: `--read` lets operators inspect and verify deployed configurations
- **Fewer mis-provisions**: `--verify` catches flashing failures before the operator walks away
-
-### Negative
-
- **NVS binary parsing** (Phase 4) requires understanding the ESP-IDF NVS binary format, which is not officially documented as a stable API
- **Auto-detect** (Phase 5) may produce false positives if other ESP32 variants are connected
-
-### Risks
-
-| Risk | Likelihood | Impact | Mitigation |
-|------|-----------|--------|------------|
-| NVS binary format changes in ESP-IDF v6 | Low | Medium | Pin to known ESP-IDF NVS page format; add format version check |
-| `--verify` serial parsing is fragile | Medium | Low | Match on stable log tag `[CSI_MAIN]`; timeout after 10s |
-| Config file credentials in plaintext | Medium | Medium | Document that config files should not be committed; add `.gitignore` pattern |
-
---
-
-## Implementation Priority
-
-| Phase | Effort | Impact | Priority |
-|-------|--------|--------|----------|
-| Phase 1: Complete NVS coverage | Small (1 file, ~50 lines) | High — closes feature gap | P0 |
-| Phase 2: Config file + mesh | Medium (~100 lines) | High — biggest UX win | P1 |
-| Phase 3: Presets | Small (~40 lines) | Medium — convenience | P2 |
-| Phase 4: Read-back + verify | Medium (~150 lines) | Medium — debugging aid | P2 |
-| Phase 5: Auto-detect | Small (~30 lines) | Low — minor convenience | P3 |
-
---
-
-## References
-
- `firmware/esp32-csi-node/main/nvs_config.h` — NVS config struct (20 fields)
- `firmware/esp32-csi-node/main/nvs_config.c` — NVS read logic (20 keys)
- `firmware/esp32-csi-node/provision.py` — Current provisioning script (13 of 20 keys)
- ADR-029: RuvSense multistatic sensing mode (TDM, channel hopping)
- ADR-032: Multistatic mesh security hardening (mesh keys)
- ADR-039: ESP32-S3 edge intelligence (edge tiers, vitals)
- ADR-040: WASM programmable sensing (WASM modules, signature verification)
- Issue #130: Provisioning script doesn't support TDM
@@ -27,16 +27,6 @@ static uint32_t s_sequence = 0;
 static uint32_t s_cb_count = 0;
 static uint32_t s_send_ok = 0;
 static uint32_t s_send_fail = 0;
-static uint32_t s_rate_skip = 0;
-
-/**
- * Minimum interval between UDP sends in microseconds.
- * CSI callbacks can fire hundreds of times per second in promiscuous mode.
- * We cap the send rate to avoid exhausting lwIP packet buffers (ENOMEM).
- * Default: 20 ms = 50 Hz max send rate.
- */
-#define CSI_MIN_SEND_INTERVAL_US  (20 * 1000)
-static int64_t s_last_send_us = 0;

 /* ---- ADR-029: Channel-hop state ---- */

@@ -153,23 +143,14 @@ static void wifi_csi_callback(void *ctx, wifi_csi_info_t *info)
    size_t frame_len = csi_serialize_frame(info, frame_buf, sizeof(frame_buf));

    if (frame_len > 0) {
-        /* Rate-limit UDP sends to avoid ENOMEM from lwIP pbuf exhaustion.
-         * In promiscuous mode, CSI callbacks can fire 100-500+ times/sec.
-         * We only need 20-50 Hz for the sensing pipeline. */
-        int64_t now = esp_timer_get_time();
-        if ((now - s_last_send_us) >= CSI_MIN_SEND_INTERVAL_US) {
-            int ret = stream_sender_send(frame_buf, frame_len);
-            if (ret > 0) {
-                s_send_ok++;
-                s_last_send_us = now;
-            } else {
-                s_send_fail++;
-                if (s_send_fail <= 5) {
-                    ESP_LOGW(TAG, "sendto failed (fail #%lu)", (unsigned long)s_send_fail);
-                }
-            }
+        int ret = stream_sender_send(frame_buf, frame_len);
+        if (ret > 0) {
+            s_send_ok++;
        } else {
-            s_rate_skip++;
+            s_send_fail++;
+            if (s_send_fail <= 5) {
+                ESP_LOGW(TAG, "sendto failed (fail #%lu)", (unsigned long)s_send_fail);
+            }
        }
    }

@@ -9,7 +9,6 @@

 #include <string.h>
 #include "esp_log.h"
-#include "esp_timer.h"
 #include "lwip/sockets.h"
 #include "lwip/netdb.h"
 #include "sdkconfig.h"
@@ -19,17 +18,6 @@ static const char *TAG = "stream_sender";
 static int s_sock = -1;
 static struct sockaddr_in s_dest_addr;

-/**
- * ENOMEM backoff state.
- * When sendto fails with ENOMEM (errno 12), we suppress further sends for
- * a cooldown period to let lwIP reclaim packet buffers.  Without this,
- * rapid-fire CSI callbacks can exhaust the pbuf pool and crash the device.
- */
-static int64_t s_backoff_until_us = 0;       /* esp_timer timestamp to resume */
-#define ENOMEM_COOLDOWN_MS  100              /* suppress sends for 100 ms */
-#define ENOMEM_LOG_INTERVAL 50               /* log every Nth suppressed send */
-static uint32_t s_enomem_suppressed = 0;
-
 static int sender_init_internal(const char *ip, uint16_t port)
 {
    s_sock = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
@@ -69,37 +57,10 @@ int stream_sender_send(const uint8_t *data, size_t len)
        return -1;
    }

-    /* ENOMEM backoff: if we recently exhausted lwIP buffers, skip sends
-     * until the cooldown expires.  This prevents the cascade of failed
-     * sendto calls that leads to a guru meditation crash. */
-    if (s_backoff_until_us > 0) {
-        int64_t now = esp_timer_get_time();
-        if (now < s_backoff_until_us) {
-            s_enomem_suppressed++;
-            if ((s_enomem_suppressed % ENOMEM_LOG_INTERVAL) == 1) {
-                ESP_LOGW(TAG, "sendto suppressed (ENOMEM backoff, %lu dropped)",
-                         (unsigned long)s_enomem_suppressed);
-            }
-            return -1;
-        }
-        /* Cooldown expired — resume sending */
-        ESP_LOGI(TAG, "ENOMEM backoff expired, resuming sends (%lu were suppressed)",
-                 (unsigned long)s_enomem_suppressed);
-        s_backoff_until_us = 0;
-        s_enomem_suppressed = 0;
-    }
-
    int sent = sendto(s_sock, data, len, 0,
                      (struct sockaddr *)&s_dest_addr, sizeof(s_dest_addr));
    if (sent < 0) {
-        if (errno == ENOMEM) {
-            /* Start backoff to let lwIP reclaim buffers */
-            s_backoff_until_us = esp_timer_get_time() +
-                                 (int64_t)ENOMEM_COOLDOWN_MS * 1000;
-            ESP_LOGW(TAG, "sendto ENOMEM — backing off for %d ms", ENOMEM_COOLDOWN_MS);
-        } else {
-            ESP_LOGW(TAG, "sendto failed: errno %d", errno);
-        }
+        ESP_LOGW(TAG, "sendto failed: errno %d", errno);
        return -1;
    }

@@ -30,40 +30,22 @@ NVS_PARTITION_OFFSET = 0x9000
 NVS_PARTITION_SIZE = 0x6000  # 24 KiB


-def build_nvs_csv(args):
+def build_nvs_csv(ssid, password, target_ip, target_port, node_id):
    """Build an NVS CSV string for the csi_cfg namespace."""
    buf = io.StringIO()
    writer = csv.writer(buf)
    writer.writerow(["key", "type", "encoding", "value"])
    writer.writerow(["csi_cfg", "namespace", "", ""])
-    if args.ssid:
-        writer.writerow(["ssid", "data", "string", args.ssid])
-    if args.password is not None:
-        writer.writerow(["password", "data", "string", args.password])
-    if args.target_ip:
-        writer.writerow(["target_ip", "data", "string", args.target_ip])
-    if args.target_port is not None:
-        writer.writerow(["target_port", "data", "u16", str(args.target_port)])
-    if args.node_id is not None:
-        writer.writerow(["node_id", "data", "u8", str(args.node_id)])
-    # TDM mesh settings
-    if args.tdm_slot is not None:
-        writer.writerow(["tdm_slot", "data", "u8", str(args.tdm_slot)])
-    if args.tdm_total is not None:
-        writer.writerow(["tdm_nodes", "data", "u8", str(args.tdm_total)])
-    # Edge intelligence settings (ADR-039)
-    if args.edge_tier is not None:
-        writer.writerow(["edge_tier", "data", "u8", str(args.edge_tier)])
-    if args.pres_thresh is not None:
-        writer.writerow(["pres_thresh", "data", "u16", str(args.pres_thresh)])
-    if args.fall_thresh is not None:
-        writer.writerow(["fall_thresh", "data", "u16", str(args.fall_thresh)])
-    if args.vital_win is not None:
-        writer.writerow(["vital_win", "data", "u16", str(args.vital_win)])
-    if args.vital_int is not None:
-        writer.writerow(["vital_int", "data", "u16", str(args.vital_int)])
-    if args.subk_count is not None:
-        writer.writerow(["subk_count", "data", "u8", str(args.subk_count)])
+    if ssid:
+        writer.writerow(["ssid", "data", "string", ssid])
+    if password is not None:
+        writer.writerow(["password", "data", "string", password])
+    if target_ip:
+        writer.writerow(["target_ip", "data", "string", target_ip])
+    if target_port is not None:
+        writer.writerow(["target_port", "data", "u16", str(target_port)])
+    if node_id is not None:
+        writer.writerow(["node_id", "data", "u8", str(node_id)])
    return buf.getvalue()


@@ -145,37 +127,14 @@ def main():
    parser.add_argument("--target-ip", help="Aggregator host IP (e.g. 192.168.1.20)")
    parser.add_argument("--target-port", type=int, help="Aggregator UDP port (default: 5005)")
    parser.add_argument("--node-id", type=int, help="Node ID 0-255 (default: 1)")
-    # TDM mesh settings
-    parser.add_argument("--tdm-slot", type=int, help="TDM slot index for this node (0-based)")
-    parser.add_argument("--tdm-total", type=int, help="Total number of TDM nodes in mesh")
-    # Edge intelligence settings (ADR-039)
-    parser.add_argument("--edge-tier", type=int, choices=[0, 1, 2],
-                        help="Edge processing tier: 0=off, 1=stats, 2=vitals")
-    parser.add_argument("--pres-thresh", type=int, help="Presence detection threshold (default: 50)")
-    parser.add_argument("--fall-thresh", type=int, help="Fall detection threshold (default: 500)")
-    parser.add_argument("--vital-win", type=int, help="Phase history window in frames (default: 300)")
-    parser.add_argument("--vital-int", type=int, help="Vitals packet interval in ms (default: 1000)")
-    parser.add_argument("--subk-count", type=int, help="Top-K subcarrier count (default: 32)")
    parser.add_argument("--dry-run", action="store_true", help="Generate NVS binary but don't flash")

    args = parser.parse_args()

-    has_value = any([
-        args.ssid, args.password is not None, args.target_ip,
-        args.target_port, args.node_id is not None,
-        args.tdm_slot is not None, args.tdm_total is not None,
-        args.edge_tier is not None, args.pres_thresh is not None,
-        args.fall_thresh is not None, args.vital_win is not None,
-        args.vital_int is not None, args.subk_count is not None,
-    ])
-    if not has_value:
-        parser.error("At least one config value must be specified")
-
-    # Validate TDM: if one is given, both should be
-    if (args.tdm_slot is not None) != (args.tdm_total is not None):
-        parser.error("--tdm-slot and --tdm-total must be specified together")
-    if args.tdm_slot is not None and args.tdm_slot >= args.tdm_total:
-        parser.error(f"--tdm-slot ({args.tdm_slot}) must be less than --tdm-total ({args.tdm_total})")
+    if not any([args.ssid, args.password is not None, args.target_ip,
+                args.target_port, args.node_id is not None]):
+        parser.error("At least one config value must be specified "
+                     "(--ssid, --password, --target-ip, --target-port, --node-id)")

    print("Building NVS configuration:")
    if args.ssid:
@@ -188,23 +147,9 @@ def main():
        print(f"  Target Port:   {args.target_port}")
    if args.node_id is not None:
        print(f"  Node ID:       {args.node_id}")
-    if args.tdm_slot is not None:
-        print(f"  TDM Slot:      {args.tdm_slot} of {args.tdm_total}")
-    if args.edge_tier is not None:
-        tier_desc = {0: "off (raw CSI)", 1: "stats", 2: "vitals"}
-        print(f"  Edge Tier:     {args.edge_tier} ({tier_desc.get(args.edge_tier, '?')})")
-    if args.pres_thresh is not None:
-        print(f"  Pres Thresh:   {args.pres_thresh}")
-    if args.fall_thresh is not None:
-        print(f"  Fall Thresh:   {args.fall_thresh}")
-    if args.vital_win is not None:
-        print(f"  Vital Window:  {args.vital_win} frames")
-    if args.vital_int is not None:
-        print(f"  Vital Interval:{args.vital_int} ms")
-    if args.subk_count is not None:
-        print(f"  Top-K Subcarr: {args.subk_count}")

-    csv_content = build_nvs_csv(args)
+    csv_content = build_nvs_csv(args.ssid, args.password, args.target_ip,
+                                args.target_port, args.node_id)

    try:
        nvs_bin = generate_nvs_binary(csv_content, NVS_PARTITION_SIZE)