mirror of
https://github.com/ruvnet/RuView
synced 2026-07-20 17:03:24 +00:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 rUv
|
04028acd54 |
@@ -52,16 +52,14 @@ jobs:
|
||||
target: esp32s3
|
||||
sdkconfig: sdkconfig.defaults
|
||||
partition_table_name: partitions_display.csv
|
||||
size_warn_kb: 1100
|
||||
size_limit_kb: 1152
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node.bin
|
||||
artifact_pt: partition-table.bin
|
||||
- variant: 4mb
|
||||
target: esp32s3
|
||||
sdkconfig: sdkconfig.defaults.4mb
|
||||
partition_table_name: partitions_4mb.csv
|
||||
size_warn_kb: 1100
|
||||
size_limit_kb: 1152
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node-4mb.bin
|
||||
artifact_pt: partition-table-4mb.bin
|
||||
# ADR-110: ESP32-C6 research target (Wi-Fi 6 / 802.15.4 / TWT / LP-core)
|
||||
@@ -69,8 +67,7 @@ jobs:
|
||||
target: esp32c6
|
||||
sdkconfig: sdkconfig.defaults
|
||||
partition_table_name: partitions_4mb.csv
|
||||
size_warn_kb: 1100
|
||||
size_limit_kb: 1152
|
||||
size_limit_kb: 1100
|
||||
artifact_app: esp32-csi-node-c6.bin
|
||||
artifact_pt: partition-table-c6.bin
|
||||
|
||||
@@ -99,23 +96,18 @@ jobs:
|
||||
make test_adr110
|
||||
./test_adr110
|
||||
|
||||
- name: Verify binary size budget
|
||||
- name: Verify binary size (< ${{ matrix.size_limit_kb }} KB gate)
|
||||
working-directory: firmware/esp32-csi-node
|
||||
run: |
|
||||
BIN=build/esp32-csi-node.bin
|
||||
SIZE=$(stat -c%s "$BIN")
|
||||
MAX=$((${{ matrix.size_limit_kb }} * 1024))
|
||||
WARN=$((${{ matrix.size_warn_kb }} * 1024))
|
||||
echo "Binary size: $SIZE bytes ($(( SIZE / 1024 )) KB)"
|
||||
echo "Warning at: $WARN bytes (${{ matrix.size_warn_kb }} KB)"
|
||||
echo "Size limit: $MAX bytes (${{ matrix.size_limit_kb }} KB)"
|
||||
if [ "$SIZE" -gt "$MAX" ]; then
|
||||
echo "::error::Firmware binary exceeds ${{ matrix.size_limit_kb }} KB size gate ($SIZE > $MAX)"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$SIZE" -gt "$WARN" ]; then
|
||||
echo "::warning::Firmware binary exceeds the ${{ matrix.size_warn_kb }} KB soft budget ($SIZE > $WARN); hard limit is ${{ matrix.size_limit_kb }} KB"
|
||||
fi
|
||||
echo "Binary size OK: $SIZE <= $MAX"
|
||||
|
||||
- name: Verify flash image integrity
|
||||
|
||||
+3
-3
@@ -7,17 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **Multi-actor PoseCode scene crate (`wifi-densepose-posecode`, ADR-266).** Adds a deterministic Rust scene model for multiple persistent RuView actors, synchronized poses, inter-person contacts, confidence and observation provenance. Includes a bounded line parser, canonical serializer, direct `PoseTrack` adapter, confidence-aware phase segmentation, provenance-sensitive range validation, 14 feature-enabled unit tests and Criterion parser/serializer benchmarks. Raw 20 Hz observations remain separate from semantic phases; observed range anomalies are warnings rather than medical safety claims. MEASURED on the implementation container: two-actor parse 3.40 µs, serialization 3.29 µs.
|
||||
|
||||
### Changed
|
||||
- **`@ruvnet/rvagent` startup optimization — stdio time-to-first-response ~242 ms → ~189 ms (−22%; MEASURED, median of repeated `initialize` round-trips against `dist/index.js`, this container, reproduce with a piped-stdin timer).** Two changes: (1) `./http-transport.js` is now imported **lazily** inside the `RVAGENT_HTTP_PORT` branch — it chain-loads the MCP SDK's `streamableHttp` module (~48 ms MEASURED via per-module `import()` timing), which the default stdio path never uses; (2) the advertised JSON Schemas generated from the Zod sources are memoized per tool instead of re-walking the Zod tree on every `tools/list` (matters under the session-per-server HTTP model where each session lists tools). No behavior change: 99/99 jest tests, HTTP session flow re-smoke-tested through the lazy path. The `@ruvnet/ruview` harness CLI was profiled too and left alone — 50 ms vs the ~29 ms bare `node -e ''` floor on the same box (MEASURED), i.e. already near the interpreter floor with zero dependencies.
|
||||
|
||||
### Fixed
|
||||
- **FastAPI health/metrics endpoints event-loop starvation.** Calling `psutil.cpu_percent(interval=1)` blocked the single-threaded async event loop for 1.0 second on every health check or metrics collection tick, stalling all incoming requests and WebSocket operations. Fixed by changing `cpu_percent` to use non-blocking `interval=None` and offloading all blocking OS metrics gathering to background thread pools via `asyncio.to_thread`. Verified event loop responsiveness via concurrency regression tests.
|
||||
- **EngineBridge now honors `WDP_GUARD_INTERVAL_US`/`WDP_SOFT_GUARD_US`/`WDP_TDM_SLOTS`+`WDP_TDM_SLOT_US`** (#1309, PR #1312, @erichkusuki). The governed trust path previously built its multistatic fuser from a hardcoded `MultistaticConfig::default()` (60 ms guard), so multi-node deployments with WiFi/ESP-NOW time sync (10–150 ms drift) failed every governed cycle regardless of configuration — while the startup log claimed the override took effect. New `StreamingEngine::set_multistatic_config()`; `EngineBridge::new()` takes an `Option<MultistaticConfig>` threaded from the same env-derived config as `AppState.multistatic_fuser`. Hardware-verified on a live 2-node ESP32-S3 setup (90 s window, 0 fusion errors; previously every cycle failed).
|
||||
- **`/api/v1/stream/pose` WebSocket reachable with `RUVIEW_API_TOKEN` set + dashboard bearer-token field** (#1310, PR #1313, @erichkusuki). Browsers cannot attach an `Authorization` header to a WS upgrade, so the Live Demo pose stream always failed when auth was on; the path is now on a narrow exact-match exemption list (mirrors `/ws/sensing`), with a regression test pinning that the exemption doesn't leak to other `/api/v1/*` paths. The QuickSettings panel gains an "API Access" field storing the bearer token in `localStorage`; the token is applied at `api.service.js` module load so the very first request carries it.
|
||||
- **Display-less DevKitC-1 boards: `sdkconfig.defaults.devkitc` build overlay** (#1308, PR #1311, @erichkusuki). The ADR-045 runtime display probe false-positives on stock ESP32-S3-DevKitC-1 (floating QSPI pins), which silently skipped the RuView#893 MGMT+DATA CSI upgrade and collapsed CSI yield to 0 pps. The overlay compiles display support out (`has_display` constant-false). Also fixes stale `espressif/idf:v5.2` README references to v5.4 (source requires `esp_driver_uart`, IDF ≥5.3). Hardware-verified on 2× DevKitC-1-N16R8 (0 → 40–45 pps).
|
||||
- **ADR-263/264/265 implemented — the RuView npm surface fixed end-to-end (`@ruvnet/ruview@0.2.0`, `@ruvnet/rvagent@0.2.0`, `@ruv/ruview-cli`).** Harness (ADR-263 O1–O9): `claim-check` now **fails closed** on empty input (CLI exit 2 + `empty_text` tool error); the MCP stdio server dispatches `tools/call` asynchronously over promise-based `spawn` — `ping` answers while a long `verify`/`calibrate` runs (pinned by a new e2e test that runs a 3 s fake proof and asserts sub-second ping); the two `optionalDependencies` are gone so a cold `npx` installs exactly 1 package (MEASURED: was 4 packages / 620 kB / 71 files, `npm i` in a clean prefix); child output is captured as bounded rolling tails (no more 1 MiB `maxBuffer` kills); `node_monitor` passes the port via `sys.argv` instead of splicing it into `python -c` source; the MCP `serverInfo.version` reads package.json; `.claude/skills/*/SKILL.md` are generated from `skills/*.md` by a `prepack` sync script (byte-equality pinned by test); `which()` is a memoized dep-free PATH scan; tools are underscore-canonical (`ruview_claim_check`, …) with the dotted names accepted as call-time aliases, plus `resources/list`/`prompts/list` stubs; the guardrail's `METRIC_TERMS` matching is precision-fixed (word-boundary `map`/`f1`/`auc`/`iou`, code-span + label scrubbing, quantitative-claims-only) — ADR-263/264/265 and both package READMEs now PASS `claim-check` while real untagged claims still flag. 30/30 tests (MEASURED, `node --test`). rvagent (ADR-264 O1–O9): `exports` fixed (types-first, the never-built `dist/index.cjs` `require` target removed — verified broken in the published 0.1.0 tarball); tarball is map-free (127,704 B unpacked / 46 files / 0 maps — MEASURED, `npm pack --dry-run`, down from 188 kB with 44 maps); the Streamable HTTP transport is **actually wired** behind `RVAGENT_HTTP_PORT` with one transport + one MCP server per session (`mcp-session-id` routing), a 1 MiB body cap (413), and a port-aware localhost origin gate — the "dual-transport" description is now true; tools renamed to underscore-canonical with dotted router aliases; ONE Zod validation gate per call with the advertised JSON Schema generated from the same Zod source (`zod-to-json-schema`); `train_count` closes its log fds (was leaking 2/job) and persists job records to `<jobsDir>/<id>.json` so `job_status` survives restarts, with bounded log-tail reads; `detectCogBinary` actually probes its candidate paths; version reads package.json; `@types/express` dropped, `@types/jest` aligned to jest 29; README rewritten to match reality (no phantom `stdio`/`http`/`policy grant` subcommands; unimplemented ADR-124 catalog tools labeled roadmap). 99/99 jest tests (MEASURED); stdio handshake + HTTP session flow + 403/400/404/413 gates smoke-tested live. CLI: bin renamed `ruview-cli` (the `ruview` bin belongs to `@ruvnet/ruview`, ADR-265 D4), version single-sourced. Distribution (ADR-265 D1–D4): new `npm-packages.yml` (3-package × Node 20/22 matrix: tests, version-literal grep gate, pack-content/size gate, tarball-install smoke test incl. the fail-closed claim-check and an ESM-import probe that would have caught the broken `require` export, README claim-check) and `ruview-npm-release.yml` (publish from CI only, `npm publish --provenance`); `ci.yml` NODE_VERSION 18→20.
|
||||
- **Empty-room field-model calibration collected nothing on real HT40 nodes — raw 128-wide frames rejected by the 56-tone model (follow-up to the deadlock fix below).** Once the status-gate deadlock was fixed, `maybe_feed_calibration` reached `feed_calibration`, but a real ESP32 HT40 node streams 128-wide amplitude vectors while the single-link `FieldModel` is the canonical 56-tone grid — so `LinkStats::update` returned `DimensionMismatch`, `feed_calibration` bubbled the error, and `maybe_feed_calibration` swallowed it at `debug` level. Net effect: `frame_count` stayed pinned at 0 on live hardware even though presence/motion/vitals (which read the global history) worked fine. Fixed by resampling each frame onto the model's canonical 56-tone grid via `HardwareNormalizer::resample_to_canonical` before feeding — the same length-only canonicalization the multistatic fusion path uses (#1170). Pinned by `field_bridge::maybe_feed_calibration_resamples_wide_frames_and_accumulates` (128-wide frame → Collecting + count 1; fails on old code). Verified live on the ESP32-S3 deployment.
|
||||
- **Empty-room field-model calibration could never start — `/api/v1/calibration/*` was a dead endpoint (frame count pinned at 0).** `POST /calibration/start` creates the `FieldModel` in `Uncalibrated`, but the per-frame server feed `field_bridge::maybe_feed_calibration` only fed observations while the model was **already** `Collecting` — and the *only* thing that sets `Collecting` is `feed_calibration` itself (on its first fed frame). The two gates deadlocked: nothing ever fed the first frame, so `calibration_frame_count` stayed 0, `status` never left `Uncalibrated`, and the SVD room eigenstructure (eigenvalue-based person counting / localization) could never calibrate — observed live as `{"status":"Uncalibrated","frame_count":0}` never advancing on a streaming ESP32 node. Fixed the guard to feed while `Uncalibrated | Collecting` so the first frame flips the model to `Collecting` and the count advances. Also made `calibration_stop` return a structured `{success:false, frame_count, frames_needed}` (with a new `FieldModel::min_calibration_frames()` accessor) instead of an opaque 500 when finalized before enough empty-room frames accumulate. Pinned by `field_bridge::maybe_feed_calibration_advances_uncalibrated_to_collecting` (asserts `Uncalibrated → Collecting` + count 0 → 1 → 2; fails on old code). Presence/motion/vitals were unaffected — they use the separate auto rolling baseline, not the field model.
|
||||
- **Multistatic fusion never ran on a mixed-mode ESP32 mesh — live bridge fed raw, un-canonicalized per-node CSI to the fuser (#1170).** `node_frame_from_state` (`multistatic_bridge.rs`) wrapped each node's **raw** amplitude vector (HT20 ≈ 64 bins, HT40 ≈ 128/192) into a struct *named* `CanonicalCsiFrame` without ever resampling, so `MultistaticFuser::fuse` tripped `DimensionMismatch` on every cycle, silently fell back to per-node sum/dedup, and spun `total_engine_errors` unbounded. Added `HardwareNormalizer::resample_to_canonical` (resample-only, **no z-score** — preserves the amplitude scale the person-score's `variance/mean²` relies on) and run every node frame through it onto the canonical 56-tone grid before fusion. Heterogeneous meshes now fuse instead of erroring. Pinned by `heterogeneous_node_counts_canonicalize_and_fuse` (mixed 64/192 → fuses), `resample_to_canonical_is_length_only_no_zscore`, and an updated `test_node_frame_conversion`; the pre-existing `engine_bridge::observe_cycle_counts_engine_errors` was retargeted to force a `TimestampMismatch` (its old 56-vs-30 setup now canonicalizes cleanly). `wifi-densepose-signal` 501 / `wifi-densepose-sensing-server` 677 tests, 0 failed.
|
||||
- **`csi_fps_ema` reported the CSI frame rate 40–840× too high under bursty UDP delivery (#1180).** `update_csi_fps_ema` only rejected deltas `≤ 0` or `≥ 1 s`, so a 36 µs intra-burst arrival delta yielded `1/dt ≈ 27 kHz` straight into the EMA — the metric measured server arrival jitter, not the node's ~40 fps production rate. Added a `MIN_PLAUSIBLE_CSI_DT_SEC = 0.005` floor (derived from the firmware's 50 fps `CSI_MIN_SEND_INTERVAL_US` ceiling, ×4 slack) and made `observe_csi_frame_arrival` keep its anchor across sub-floor bursts so the next genuine inter-frame gap measures true cadence. Pinned by `subms_burst_delta_rejected`, `burst_interleaved_with_nominal_stays_in_band`, and `observe_csi_frame_arrival_ignores_subms_bursts`.
|
||||
- **`stream_sender` ENOMEM backoff starved low-rate control packets under a weak uplink (#1183, follow-up to #1135/#1159).** The global `s_backoff_until_us` gate (triggered by the 50 Hz CSI flood at weak RSSI) also suppressed the ≤48 B, ≤1 Hz `feature_state` / mesh `HEALTH` / sync packets that contribute negligible buffer pressure, so telemetry failed essentially every cycle. Added `stream_sender_send_priority()` — bypasses the backoff gate, reports ENOMEM quietly, and never extends/resets the global streak — and routed `feature_state`, HEALTH/anomaly (`rv_mesh_send`), and sync packets through it. Also fixed the misleading `"HEALTH sent"` log that printed unconditionally even when `rv_mesh_send` returned `ESP_FAIL` (now prints `sent`/`FAILED` from the actual return). Firmware builds clean (ESP-IDF v5.4).
|
||||
|
||||
@@ -14,6 +14,7 @@ Dual codebase: Python v1 (`v1/`) and Rust port (`v2/`).
|
||||
| `wifi-densepose-mat` | Mass Casualty Assessment Tool — disaster survivor detection |
|
||||
| `wifi-densepose-hardware` | ESP32 aggregator, TDM protocol, channel hopping firmware; `ieee80211bf/` 802.11bf forward-compat protocol model (ADR-153) |
|
||||
| `wifi-densepose-ruvector` | RuVector v2.0.4 integration + cross-viewpoint fusion (5 modules) |
|
||||
| `wifi-densepose-posecode` | ADR-266 multi-actor semantic motion scenes, PoseTrack adapter, bounded parser, confidence-aware phase segmentation |
|
||||
| `wifi-densepose-wasm` | WebAssembly bindings for browser deployment |
|
||||
| `wifi-densepose-cli` | CLI tool (`wifi-densepose` binary) — `calibrate`/`calibrate-serve`/`enroll`/`train-room`/`room-watch` + MAT (MAT gated behind the `mat` feature; build `--no-default-features` for the aarch64/appliance calibration binary) |
|
||||
| `wifi-densepose-calibration` | ADR-151 per-room calibration & specialist training — `baseline → enroll → extract → train` → bank of small specialists (presence/posture/breathing/heartbeat/restlessness/anomaly) + multistatic fusion; pure Rust, edge-deployable |
|
||||
@@ -81,6 +82,7 @@ All 5 ruvector crates integrated in workspace:
|
||||
- ADR-263: `@ruvnet/ruview` npm harness deep review + optimization strategy (Proposed)
|
||||
- ADR-264: `@ruvnet/rvagent` MCP server + `@ruv/ruview-cli` deep review + optimization strategy (Proposed)
|
||||
- ADR-265: RuView npm distribution strategy — CI gate, provenance, version single-sourcing (Proposed)
|
||||
- ADR-266: Multi-actor PoseCode scenes from persistent RuView tracks (Accepted)
|
||||
|
||||
### Supported Hardware
|
||||
|
||||
@@ -93,8 +95,6 @@ All 5 ruvector crates integrated in workspace:
|
||||
|
||||
**Not supported:** ESP32 (original), ESP32-C3 — single-core, can't run CSI DSP pipeline.
|
||||
|
||||
**⚠️ Compact boards (SuperMini, ESP32-S3-Zero, other coin-sized clones) run hot:** the firmware keeps the WiFi radio on continuously (`WIFI_PS_NONE`) and runs a full DSP pipeline (`edge_tier=2`), which is sustained high current draw. Full-size dev boards handle this fine; coin-sized clones with minimal PCB copper and budget regulators can run uncomfortably hot and, per at least one field report, have failed to power on again after a hot session. Give them airflow and check by touch during the first few minutes. See `firmware/esp32-csi-node/README.md` for details.
|
||||
|
||||
### Build & Test Commands (this repo)
|
||||
```bash
|
||||
# Rust — full workspace tests (1,031+ tests, ~2 min)
|
||||
|
||||
@@ -617,7 +617,8 @@ Verify the plugin structure: `bash plugins/ruview/scripts/smoke.sh`. Full detail
|
||||
| [Semantic Primitives — Precision/Recall](docs/integrations/semantic-primitives-metrics.md) | Per-primitive F1 on the held-out paired-capture set: someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting, bathroom, fall-risk, bed-exit, no-movement, multi-room. |
|
||||
| [Claude Code / Codex Plugin](plugins/ruview/README.md) | The `ruview` plugin + marketplace — skills, `/ruview-*` commands, agents, and the Codex prompt mirror |
|
||||
| [Portable harness — `npx @ruvnet/ruview`](harness/ruview/README.md) | MetaHarness-minted, host-portable RuView operator harness — `ruview.*` MCP tools + the MEASURED-vs-CLAIMED honesty guardrail enforced in code ([ADR-182](docs/adr/ADR-182-npx-ruview-harness-via-metaharness.md)). A lighter, multi-host companion to the in-repo plugin. |
|
||||
| [Architecture Decisions](docs/adr/README.md) | 182 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
|
||||
| [Architecture Decisions](docs/adr/README.md) | 183 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
|
||||
| [Multi-actor PoseCode](v2/crates/wifi-densepose-posecode/) | Rust scene protocol converting persistent RuView pose tracks into confidence-scored actors, synchronized motion phases, contacts and deterministic text. [ADR-266](docs/adr/ADR-266-multi-actor-posecode-scenes.md). |
|
||||
| [Domain Models](docs/ddd/README.md) | 8 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI, rvCSI) — bounded contexts, aggregates, domain events, and ubiquitous language |
|
||||
| [rvCSI — edge RF sensing runtime](https://github.com/ruvnet/rvcsi) | Rust-first / TypeScript-accessible / hardware-abstracted CSI runtime: multi-source ingestion (incl. real nexmon_csi `.pcap` from a **Raspberry Pi 5** / Pi 4 / Pi 3B+ — CYW43455 / BCM43455c0) → validation → DSP → typed events → RuVector RF memory ([ADR-095](docs/adr/ADR-095-rvcsi-edge-rf-sensing-platform.md), [ADR-096](docs/adr/ADR-096-rvcsi-ffi-crate-layout.md), [domain model](docs/ddd/rvcsi-domain-model.md)). Now its own repo — [`ruvnet/rvcsi`](https://github.com/ruvnet/rvcsi) — vendored here under `vendor/rvcsi`; 9 `rvcsi-*` crates on crates.io, `@ruv/rvcsi` on npm, plus a Claude Code plugin. |
|
||||
| [Desktop App](v2/crates/wifi-densepose-desktop/README.md) | **WIP** — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization |
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
Health check API endpoints
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import psutil
|
||||
from typing import Dict, Any, Optional
|
||||
@@ -169,7 +168,7 @@ async def health_check(request: Request):
|
||||
overall_status = "degraded"
|
||||
|
||||
# Get system metrics
|
||||
system_metrics = await asyncio.to_thread(get_system_metrics)
|
||||
system_metrics = get_system_metrics()
|
||||
|
||||
uptime_seconds = (datetime.now() - _APP_START_TIME).total_seconds()
|
||||
|
||||
@@ -264,12 +263,11 @@ async def get_health_metrics(
|
||||
):
|
||||
"""Get detailed system metrics."""
|
||||
try:
|
||||
metrics = await asyncio.to_thread(get_system_metrics)
|
||||
metrics = get_system_metrics()
|
||||
|
||||
# Add additional metrics if authenticated
|
||||
if current_user:
|
||||
detailed = await asyncio.to_thread(get_detailed_metrics)
|
||||
metrics.update(detailed)
|
||||
metrics.update(get_detailed_metrics())
|
||||
|
||||
return {
|
||||
"timestamp": datetime.utcnow().isoformat(),
|
||||
@@ -302,7 +300,7 @@ def get_system_metrics() -> Dict[str, Any]:
|
||||
"""Get basic system metrics."""
|
||||
try:
|
||||
# CPU metrics
|
||||
cpu_percent = psutil.cpu_percent(interval=None)
|
||||
cpu_percent = psutil.cpu_percent(interval=1)
|
||||
cpu_count = psutil.cpu_count()
|
||||
|
||||
# Memory metrics
|
||||
|
||||
@@ -180,24 +180,21 @@ class MetricsService:
|
||||
async def _collect_system_metrics(self):
|
||||
"""Collect system-level metrics."""
|
||||
try:
|
||||
# Query OS metrics in a background thread to prevent blocking the event loop
|
||||
def gather_metrics():
|
||||
return (
|
||||
psutil.cpu_percent(interval=None),
|
||||
psutil.virtual_memory().percent,
|
||||
psutil.disk_usage('/'),
|
||||
psutil.net_io_counters()
|
||||
)
|
||||
|
||||
cpu_percent, mem_percent, disk, network = await asyncio.to_thread(gather_metrics)
|
||||
|
||||
# Record metrics on the main loop
|
||||
# CPU usage
|
||||
cpu_percent = psutil.cpu_percent(interval=1)
|
||||
self._metrics["system_cpu_usage"].add_point(cpu_percent)
|
||||
self._metrics["system_memory_usage"].add_point(mem_percent)
|
||||
|
||||
# Memory usage
|
||||
memory = psutil.virtual_memory()
|
||||
self._metrics["system_memory_usage"].add_point(memory.percent)
|
||||
|
||||
# Disk usage
|
||||
disk = psutil.disk_usage('/')
|
||||
disk_percent = (disk.used / disk.total) * 100
|
||||
self._metrics["system_disk_usage"].add_point(disk_percent)
|
||||
|
||||
# Network I/O
|
||||
network = psutil.net_io_counters()
|
||||
self._metrics["system_network_bytes_sent"].add_point(network.bytes_sent)
|
||||
self._metrics["system_network_bytes_recv"].add_point(network.bytes_recv)
|
||||
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
import asyncio
|
||||
import time
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
# Add project root and archive/v1 to sys.path so we can import src modules
|
||||
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../../../")))
|
||||
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "../../")))
|
||||
|
||||
from archive.v1.src.api.routers.health import get_system_metrics
|
||||
|
||||
async def ticker():
|
||||
"""Asynchronous background ticker to measure event loop latency/freezes."""
|
||||
ticks = []
|
||||
for _ in range(15):
|
||||
ticks.append(time.time())
|
||||
await asyncio.sleep(0.1)
|
||||
return ticks
|
||||
|
||||
async def run_test():
|
||||
print("Starting concurrency verification test...")
|
||||
|
||||
# Start the ticker background task
|
||||
ticker_task = asyncio.create_task(ticker())
|
||||
|
||||
# Let ticker run for a few ticks
|
||||
await asyncio.sleep(0.3)
|
||||
|
||||
print("Calling get_system_metrics offloaded to background thread...")
|
||||
start_time = time.time()
|
||||
|
||||
# Query system metrics using to_thread (simulating FastAPI request)
|
||||
metrics = await asyncio.to_thread(get_system_metrics)
|
||||
|
||||
duration = time.time() - start_time
|
||||
print(f"get_system_metrics took: {duration:.4f}s")
|
||||
|
||||
# Wait for the ticker to complete
|
||||
ticks = await ticker_task
|
||||
|
||||
# Calculate gaps between consecutive ticks to check for event loop freezes
|
||||
gaps = [ticks[i+1] - ticks[i] for i in range(len(ticks)-1)]
|
||||
max_gap = max(gaps)
|
||||
|
||||
print(f"All tick gaps: {[round(g, 3) for g in gaps]}")
|
||||
print(f"Max event loop freeze: {max_gap:.4f}s")
|
||||
|
||||
# In pre-fix code, psutil.cpu_percent(interval=1) blocks for 1.0s,
|
||||
# causing a gap of >1.0s. With our fix, it should be close to 0.1s.
|
||||
return max_gap, duration
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_system_metrics_does_not_starve_event_loop():
|
||||
max_gap, duration = await run_test()
|
||||
# ticker sleeps 0.1s; allow slack for CI, but we should not see ~1s gaps
|
||||
assert max_gap < 0.6
|
||||
assert duration < 0.6
|
||||
@@ -1,171 +0,0 @@
|
||||
# ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform
|
||||
|
||||
- **Status**: proposed
|
||||
- **Date**: 2026-07-18
|
||||
- **Deciders**: ruv
|
||||
- **Tags**: realtek, rtl8720f, ameba, fmcw, radar, cfr, csi, hardware
|
||||
- **Relates to**: ADR-018, ADR-063, ADR-064, ADR-095, ADR-097, ADR-260, ADR-262
|
||||
|
||||
## Context
|
||||
|
||||
Realtek's `RTL8720F-2.4G-Radar-Advantages_EN.pptx` describes an RTL8720F mode that shares the
|
||||
2.4 GHz radio between Wi-Fi, Bluetooth, and an active FMCW radar. It offers two data products that
|
||||
are useful to RuView:
|
||||
|
||||
1. **CFR (Channel Frequency Report)**, described by Realtek as the same concept as Wi-Fi CSI.
|
||||
2. **Near and far Range-FFT reports**, preserving near-field content while extending observation to
|
||||
approximately 5–6 m.
|
||||
|
||||
The proposed radio uses one transmit and one receive antenna, 20/40/70 MHz sweeps, configurable
|
||||
8/16/32/64 microsecond chirp symbols, a maximum 2.56 ms FMCW packet, and a configurable frame
|
||||
interval above 15 ms. The deck recommends 40 MHz outside Japan and 20 MHz in Japan. It also
|
||||
describes EDCCA/CTS channel access, Wi-Fi/BT/radar time division, interference reporting, and
|
||||
priority arbitration in the driver.
|
||||
|
||||
This is not a drop-in replacement for ESP32 CSI:
|
||||
|
||||
- it is **active monostatic FMCW**, while the ESP32 path observes Wi-Fi packet CSI;
|
||||
- one Tx/one Rx has no angle-of-arrival or native multi-target separation;
|
||||
- the stated 40 MHz range resolution is about 3.15 m, despite a finer 0.59 m Range-FFT report step;
|
||||
- the presentation is a capability description, not an SDK contract. It contains no header names,
|
||||
function signatures, callback ABI, binary layouts, toolchain version, licensing terms, or public
|
||||
RTL8720F board package.
|
||||
|
||||
Realtek's public Ameba RTOS repository is the base. Release v1.2.1 includes the CSI API and fixes a
|
||||
CSI application-buffer semaphore issue, but does not expose the radar application surface. Open
|
||||
upstream PR #1336 (2026-07-18 snapshot) adds RTL8720F project artifacts, `AT+RAD`, `AT+RADDBG`, and
|
||||
the public configuration call `wifi_radar_config(struct rtw_radar_action_parm *)`. Its public
|
||||
parameter struct confirms mode, channel, 70/40/20 MHz bandwidth selector, trigger period, and
|
||||
enable/config actions. Report reception still crosses non-public/placeholder HAL symbols such as
|
||||
`wifi_hal_radar_recv_data(frame_num, frame_type, data)`, so the report layout and buffer lifetime
|
||||
remain vendor-gated. Therefore the integration stays split at that boundary.
|
||||
|
||||
## Decision
|
||||
|
||||
RuView will support RTL8720F radar as an **optional, capability-negotiated source**, without
|
||||
replacing the ESP32 firmware or treating radar CFR as byte-compatible with ADR-018 CSI.
|
||||
|
||||
The integration has three layers:
|
||||
|
||||
1. **Realtek device firmware**: a small application built in the vendor-supported Ameba SDK calls
|
||||
the radar API, owns coexistence configuration, and emits versioned reports. This code lives under
|
||||
`firmware/rtl8720f-radar/` only after the redistributable SDK/API is available.
|
||||
2. **Transport-neutral wire contract**: CFR and Range-FFT reports are framed independently from the
|
||||
vendor ABI and sent over UDP, USB CDC, or UART. ADR-264 defines this boundary.
|
||||
3. **Rust host adapter**: `wifi-densepose-hardware` parses reports from bytes and converts CFR into
|
||||
the existing CSI-domain representation, while Range-FFT remains a radar modality and feeds the
|
||||
RuField/RuView cross-modality bridge from ADR-260/262.
|
||||
|
||||
The two report types remain semantically distinct:
|
||||
|
||||
| RTL8720F output | RuView representation | Permitted use |
|
||||
|---|---|---|
|
||||
| CFR | `CsiFrame` through a Realtek calibration adapter | CSI feature extraction after validation |
|
||||
| Range-FFT near/far | `RadarFrame` / RuField `mmwave_radar`-class event with a 2.4 GHz descriptor | range, motion, presence, fusion |
|
||||
| Vendor AI presence probability | derived observation with model/version provenance | advisory input, never ground truth |
|
||||
| Interference report | quality/provenance metadata | reject, down-weight, or mark contaminated frames |
|
||||
|
||||
The modality registry should eventually distinguish `fmcw_radar_2_4ghz` from `mmwave_radar`; until
|
||||
that RuField schema revision is accepted, the adapter must attach `carrier_hz = 2.4e9` and must not
|
||||
claim millimetre-wave provenance.
|
||||
|
||||
## Delivery phases and gates
|
||||
|
||||
### P0 — Vendor enablement
|
||||
|
||||
Obtain the PR #1336-or-newer RTL8720F SDK package, radar API headers/libraries, a supported evaluation
|
||||
board, flashing/debug instructions, report definitions, and written redistribution terms.
|
||||
|
||||
**Gate:** compile and run Realtek's unmodified radar example and capture CFR plus near/far
|
||||
Range-FFT output. Until this passes, device firmware is `VENDOR_BLOCKED`, not implemented.
|
||||
|
||||
### P1 — Host-first contract
|
||||
|
||||
Implement ADR-264 types, parsers, fixtures, fuzz tests, and replay support without linking vendor
|
||||
code. Use the Rust `Rtl8720fSimulator` as the only pre-hardware live source. It emits deterministic
|
||||
CFR, near/far Range-FFT, interference, and capabilities frames through the same ADR-264 encoder and
|
||||
parser used by hardware. Every simulated frame sets `RadarFlags::SYNTHETIC`; simulation results are
|
||||
never reported as device measurements.
|
||||
|
||||
**Gate:** malformed inputs never panic; encode/decode round trips; unknown versions and report
|
||||
types fail closed.
|
||||
|
||||
### P2 — RTL8720F firmware adapter
|
||||
|
||||
Wrap only the minimum vendor API surface: initialization, profile configuration, start/stop,
|
||||
callback acquisition, interference status, and report serialization. Keep vendor types out of the
|
||||
wire protocol.
|
||||
|
||||
**Gate:** 30-minute simultaneous Wi-Fi telemetry and radar capture with no watchdog reset, bounded
|
||||
loss, monotonic sequence numbers, and explicit coexistence/interference statistics.
|
||||
|
||||
### P3 — Calibration and signal validation
|
||||
|
||||
Calibrate CFR phase/amplitude, Range-FFT bin spacing, static leakage, and clock drift. Compare
|
||||
reported range against measured targets at multiple distances and bandwidths.
|
||||
|
||||
**Gate:** publish measured error distributions. Do not infer accuracy from report-bin spacing and
|
||||
do not advertise multi-person pose or vital signs from the vendor deck.
|
||||
|
||||
### P4 — Fusion and productization
|
||||
|
||||
Feed calibrated CFR through the CSI path and Range-FFT through RuField, retaining source, mode,
|
||||
bandwidth, calibration, firmware, and interference provenance.
|
||||
|
||||
**Gate:** ablation shows whether the radar stream improves a named RuView metric over ESP32 CSI
|
||||
alone. If it does not, ship it only as an independent presence/range sensor.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- One low-cost radio can provide active radar and CSI-like CFR while retaining Wi-Fi connectivity.
|
||||
- Range-FFT adds an independent physical measurement for presence/range fusion.
|
||||
- The vendor SDK is isolated from the Rust sensing core and from the stable on-wire contract.
|
||||
- Capability negotiation permits future Realtek parts without another application-level fork.
|
||||
|
||||
### Negative
|
||||
|
||||
- The first implementation is blocked on access to the actual RTL8720F radar SDK/API and hardware.
|
||||
- Active 2.4 GHz transmission changes coexistence, privacy, power, and regional compliance concerns.
|
||||
- 1T1R and limited sweep bandwidth cannot provide the spatial resolution of multi-antenna mmWave.
|
||||
- A second embedded toolchain and firmware release process must be maintained.
|
||||
|
||||
### Neutral
|
||||
|
||||
- ESP32 remains the default CSI node.
|
||||
- Existing consumers receive normalized frames and do not link against Realtek code.
|
||||
- Vendor AI output is optional metadata; RuView retains responsibility for its own validation.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
1. **Map Range-FFT directly to `CsiFrame`.** Rejected because range bins and channel-frequency
|
||||
samples have different axes and physical meaning.
|
||||
2. **Link the Realtek SDK into the Rust server.** Rejected because it couples host builds to a
|
||||
proprietary embedded ABI and toolchain.
|
||||
3. **Wait to define any interface until hardware arrives.** Rejected because the host protocol,
|
||||
parser safety, replay, and provenance can be developed and reviewed independently.
|
||||
4. **Replace ESP32 nodes.** Rejected because the modes are complementary and availability differs.
|
||||
|
||||
## Open vendor questions
|
||||
|
||||
- Exact RTL8720F part/board identifier and production availability.
|
||||
- SDK repository/tag, compiler, RTOS, binary blobs, license, and redistribution permissions.
|
||||
- Radar initialization/configuration/callback API signatures and threading/ISR constraints.
|
||||
- CFR and near/far Range-FFT element type, complex ordering, scaling, endianness, and timestamps.
|
||||
- Whether CFR is calibrated complex data and whether phase remains coherent across frames.
|
||||
- Maximum report rates, buffer ownership, DMA/cache constraints, and Wi-Fi throughput impact.
|
||||
- Region/channel enforcement and whether 70 MHz operation is allowed by the supplied firmware.
|
||||
- Secure boot, signed OTA, unique device identity, and firmware attestation support.
|
||||
|
||||
## Sources
|
||||
|
||||
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 3 and 10–19,
|
||||
supplied 2026-07-18. This is product material, not measured RuView validation.
|
||||
- [Ameba-AIoT/ameba-rtos releases](https://github.com/Ameba-AIoT/ameba-rtos/releases), reviewed
|
||||
2026-07-18; v1.2.1 is the current QC release and includes a CSI buffer-semaphore fix.
|
||||
- [Ameba-AIoT/ameba-rtos PR #1336](https://github.com/Ameba-AIoT/ameba-rtos/pull/1336), reviewed
|
||||
2026-07-18; exposes RTL8720F build assets, `wifi_radar_config`, and radar AT commands while report
|
||||
internals remain in binary/private layers.
|
||||
- ADR-063 (mmWave sensor fusion), ADR-095/097 (source normalization), and ADR-260/262 (RuField
|
||||
multimodal event model and live bridge).
|
||||
@@ -1,148 +0,0 @@
|
||||
# ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports
|
||||
|
||||
- **Status**: proposed
|
||||
- **Date**: 2026-07-18
|
||||
- **Deciders**: ruv
|
||||
- **Tags**: realtek, rtl8720f, protocol, cfr, range-fft, udp, serial
|
||||
- **Depends on**: ADR-263
|
||||
- **Relates to**: ADR-018, ADR-095, ADR-097, ADR-099, ADR-260
|
||||
|
||||
## Context
|
||||
|
||||
ADR-263 adopts RTL8720F radar behind an anti-corruption boundary. The Realtek presentation names
|
||||
CFR, near Range-FFT, far Range-FFT, and interference reports, but does not specify their binary ABI.
|
||||
RuView needs a stable, testable contract that can be implemented before the vendor SDK arrives and
|
||||
that will not expose vendor structs, pointer layouts, padding, or callback lifetime rules over the
|
||||
network.
|
||||
|
||||
ADR-018 already defines ESP32 CSI framing. Reusing its magic or pretending that Realtek radar is an
|
||||
ESP32 packet would make source detection ambiguous and erase radar-specific calibration metadata.
|
||||
|
||||
## Decision
|
||||
|
||||
Define a new little-endian `RtlRadarFrameV1` envelope with its own magic and explicit payload type.
|
||||
This is a RuView protocol, not a claim about Realtek's native memory layout.
|
||||
|
||||
### Envelope
|
||||
|
||||
All integer fields are little-endian. Floating-point payloads use IEEE-754 binary32. No C struct is
|
||||
sent by `memcpy`; firmware serializes each field explicitly.
|
||||
|
||||
| Offset | Size | Field | Meaning |
|
||||
|---:|---:|---|---|
|
||||
| 0 | 4 | magic | ASCII `RTR1` (`0x31525452`) |
|
||||
| 4 | 1 | version | `1` |
|
||||
| 5 | 1 | report_type | 1 CFR, 2 range-near, 3 range-far, 4 interference, 5 capabilities |
|
||||
| 6 | 2 | header_len | complete header size, initially 56 |
|
||||
| 8 | 4 | frame_len | header + payload + CRC |
|
||||
| 12 | 4 | sequence | wraps modulo 2^32 |
|
||||
| 16 | 8 | timestamp_us | monotonic device time at acquisition |
|
||||
| 24 | 8 | device_id | stable pseudonymous identifier, not a MAC address |
|
||||
| 32 | 4 | center_freq_khz | RF centre frequency |
|
||||
| 36 | 2 | bandwidth_mhz | 20, 40, or 70 |
|
||||
| 38 | 2 | flags | calibration/interference/saturation/time-sync flags |
|
||||
| 40 | 2 | element_count | complex samples or range bins |
|
||||
| 42 | 1 | element_format | 0 bytes/TLV, 1 complex-i16, 2 complex-f32, 3 power-u16, 4 power-f32 |
|
||||
| 43 | 1 | antenna_count | expected to be 1 for the deck's 1T1R configuration |
|
||||
| 44 | 4 | scale | quantized-to-physical multiplier; `1.0` for float payloads |
|
||||
| 48 | 4 | bin_spacing | Hz for CFR, metres for Range-FFT |
|
||||
| 52 | 4 | calibration_id | device calibration revision/hash prefix |
|
||||
| 56 | variable | payload | determined by type, count, and format |
|
||||
| final-4 | 4 | crc32 | IEEE CRC-32 over header and payload |
|
||||
|
||||
If vendor evidence shows that 56 bytes is too costly, a later protocol version may introduce a
|
||||
compact header. V1 favors auditable provenance over premature byte savings.
|
||||
|
||||
### Payload semantics
|
||||
|
||||
- **CFR** contains ordered complex channel-frequency samples. The adapter must know the frequency
|
||||
origin/order and must not fabricate missing phase. Uncalibrated frames carry the uncalibrated flag
|
||||
and cannot enter phase-sensitive processing.
|
||||
- **Range-near/range-far** contains ordered range bins. Near and far are separate report types so
|
||||
filtering and leakage behavior are never hidden from consumers.
|
||||
- **Interference** contains a versioned TLV set for channel-busy, detected-during-chirp, estimated
|
||||
interference power, and packet jitter. Unknown TLVs are skipped by length.
|
||||
- **Capabilities** is emitted at boot and on request. It declares supported report types, bandwidths,
|
||||
chirp lengths, maximum elements/report, maximum frame rate, firmware version, and SDK identifier.
|
||||
|
||||
### Transport
|
||||
|
||||
The identical envelope is supported over:
|
||||
|
||||
- UDP datagrams for normal RuView ingestion;
|
||||
- USB CDC or UART with COBS framing and a zero-byte delimiter;
|
||||
- file replay as a length-prefixed sequence of envelopes.
|
||||
|
||||
One envelope must fit one UDP datagram. Fragmentation is not part of V1; firmware rejects a profile
|
||||
whose maximum report exceeds the configured MTU and reports the required size through capabilities.
|
||||
|
||||
### Parser and trust rules
|
||||
|
||||
The host parser:
|
||||
|
||||
1. validates magic, version, lengths, enum values, element count/format multiplication, and CRC
|
||||
before allocating or decoding the payload;
|
||||
2. caps frames at 64 KiB and elements at a configured hardware maximum;
|
||||
3. rejects non-finite float metadata/payload values;
|
||||
4. tracks sequence gaps and timestamp regressions per device;
|
||||
5. preserves unknown flags but never interprets them as trusted;
|
||||
6. attaches transport source, firmware/SDK version, calibration ID, and interference state to
|
||||
provenance;
|
||||
7. labels fixture/generated frames as synthetic.
|
||||
|
||||
No vendor-provided presence probability bypasses RuView privacy, provenance, or quality gates.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Firmware, transport, parser, replay, and fusion can evolve independently.
|
||||
- Fuzzing and golden fixtures require no Realtek SDK or board.
|
||||
- CFR and Range-FFT retain correct axes and calibration provenance.
|
||||
- A boot-time capabilities frame makes SDK/API drift observable.
|
||||
|
||||
### Negative
|
||||
|
||||
- Serialization adds CPU and bandwidth overhead compared with dumping a vendor buffer.
|
||||
- V1 fields may need revision after the actual API and report limits are disclosed.
|
||||
- UDP provides integrity/error detection, not authenticity or confidentiality.
|
||||
|
||||
### Neutral
|
||||
|
||||
- Authentication can be layered with ADR-032 device identity or a signed RuField receipt without
|
||||
changing report semantics.
|
||||
- ESP32 ADR-018 framing remains unchanged.
|
||||
|
||||
## Implementation plan
|
||||
|
||||
1. Add `rtl8720f` types/parser module to `wifi-densepose-hardware` behind no vendor dependency.
|
||||
2. Add golden CFR, near/far Range-FFT, interference, and capabilities fixtures.
|
||||
3. Add property/fuzz tests for length arithmetic, enum handling, CRC, and float validation.
|
||||
4. Add a replay CLI that prints normalized metadata without running inference.
|
||||
5. Once SDK access exists, implement the embedded serializer and verify captured frames against the
|
||||
host golden decoder.
|
||||
6. Revise this proposed ADR with measured element counts, rates, and API names before acceptance.
|
||||
|
||||
Host-side steps 1–3 are implemented in `wifi-densepose-hardware::rtl8720f`: typed report and
|
||||
element enums, semantic type/format validation, bounded length arithmetic, CRC verification,
|
||||
finite-float checks, encode/decode round trips, corruption/truncation tests, and deterministic
|
||||
arbitrary-input panic checks. Cross-language vectors remain blocked on the vendor SDK callback ABI.
|
||||
Bit 15 of `flags` is reserved by RuView as `SYNTHETIC`; the Rust simulator always sets it and real
|
||||
firmware must never set it. The simulator is deterministic by seed and exercises the production
|
||||
encoder/parser rather than a parallel mock representation.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Rust encode/decode round-trip for every report type.
|
||||
- Cross-language golden vector produced by the RTL8720F firmware.
|
||||
- Zero parser panics over the fuzz corpus and arbitrary byte input.
|
||||
- Detection of single-bit corruption, truncation, count overflow, timestamp regression, and gaps.
|
||||
- Captured CFR frequency order and Range-FFT bin spacing verified against vendor documentation and a
|
||||
measured target.
|
||||
|
||||
## Sources
|
||||
|
||||
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 11–19, supplied
|
||||
2026-07-18.
|
||||
- ADR-018 (ESP32 framing), ADR-095/097 (hardware normalization), ADR-260 (multimodal event model),
|
||||
and ADR-263 (platform decision).
|
||||
@@ -0,0 +1,141 @@
|
||||
# ADR-266: Multi-Actor PoseCode Scenes
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| Status | Accepted, crate implementation complete |
|
||||
| Date | 2026-07-15 |
|
||||
| Deciders | ruv |
|
||||
| Crate | `wifi-densepose-posecode` |
|
||||
| Related | ADR-029, ADR-037, ADR-082, ADR-101, ADR-170 |
|
||||
|
||||
## Context
|
||||
|
||||
RuView already maintains multiple persistent `PoseTrack` aggregates. Each track
|
||||
contains 17 COCO keypoints, a lifecycle, an AETHER identity embedding and a
|
||||
stable `TrackId`. The missing layer is a compact, agent-readable representation
|
||||
of what those tracked people do together over time.
|
||||
|
||||
PoseCode 0.1 provides a useful vocabulary for one actor, sequential movement
|
||||
phases, semantic joint actions, contacts and browser rendering. Its current
|
||||
protocol does not represent multiple actors, shared world coordinates,
|
||||
inter-person contact, observation confidence or track provenance.
|
||||
|
||||
Encoding every 20 Hz RuView frame as text would also be incorrect. It would
|
||||
produce 1,200 steps per person per minute, discard observation uncertainty and
|
||||
couple the sensing hot path to a presentation language.
|
||||
|
||||
## Decision
|
||||
|
||||
Create the leaf crate `wifi-densepose-posecode` and define a backwards-minded
|
||||
0.2 scene extension with these invariants:
|
||||
|
||||
1. A `Scene` owns named actors and synchronized phases.
|
||||
2. Each actor can carry the originating RuView `TrackId` and confidence.
|
||||
3. Every joint target and inter-person contact carries confidence.
|
||||
4. Actor positions and travel use one room-calibrated three-dimensional frame.
|
||||
5. Raw `SceneFrame` observations remain separate from compact semantic phases.
|
||||
6. `PhaseSegmenter` emits a phase on actor membership change, motion settling
|
||||
or a bounded maximum duration.
|
||||
7. Serialization is canonical and deterministic.
|
||||
8. The parser has hard document, line, actor, phase and target limits.
|
||||
9. General range-of-motion violations are errors for authored scenes but only
|
||||
warnings for observed WiFi scenes. RF pose estimates are not medical joint
|
||||
measurements and must not create false safety claims.
|
||||
|
||||
The text form is:
|
||||
|
||||
```posecode
|
||||
posecode scene "Assisted squat"
|
||||
source observed_wifi_csi
|
||||
actor patient:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
track 7
|
||||
confidence 0.82
|
||||
position 0 0 0
|
||||
actor therapist:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
track 12
|
||||
confidence 0.76
|
||||
position 1.2 0 0
|
||||
step "Lower" 1.5s flow:
|
||||
patient.knee_left: flex 95 0.8
|
||||
therapist.shoulder_left: flex 30 0.7
|
||||
contact therapist.hand_left patient.shoulder_right 0.7
|
||||
repeat 1
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
`RuViewAdapter` consumes references to existing `PoseTrack` values. It does not
|
||||
duplicate assignment or identity tracking. It calculates elbows and knees from
|
||||
three-point interior angles, estimates hip and shoulder sagittal movement in a
|
||||
calibrated coordinate frame and maps the hip midpoint to actor position.
|
||||
The adapter is exposed by the crate's `ruview` feature so the protocol, parser
|
||||
and segmenter remain usable without pulling the full signal and RuVector graph.
|
||||
|
||||
The adapter uses keypoint confidence when available. Current trackers that have
|
||||
not populated that field receive an explicit configurable fallback confidence,
|
||||
degraded by track staleness. Non-finite coordinates and terminated tracks fail
|
||||
closed.
|
||||
|
||||
`PhaseSegmenter` consumes ordered frames. High-rate frames remain available for
|
||||
live rendering and evidence storage while the resulting scene records only
|
||||
meaningful synchronized targets. This preserves both fidelity and readability.
|
||||
|
||||
## Security and Resource Bounds
|
||||
|
||||
The public parser accepts at most 1 MiB per document and 4,096 bytes per line.
|
||||
The scene validator defaults to 32 actors, 10,000 phases and 64 joint targets
|
||||
per actor per phase. All coordinates, angles and confidence values must be
|
||||
finite. References to undeclared actors are rejected. Durations, repeats and
|
||||
timeline addition are bounded.
|
||||
|
||||
No network, filesystem, model or renderer capability is present in the crate.
|
||||
It is a deterministic transformation leaf over owned data.
|
||||
|
||||
## Consequences
|
||||
|
||||
RuView can now produce privacy-preserving multi-person replays, structured fall
|
||||
and interaction evidence, exercise comparison inputs and agent-readable motion
|
||||
records without video.
|
||||
|
||||
The crate does not solve RF source separation. Its multi-person accuracy is
|
||||
bounded by the upstream pose model and `PoseTracker`. Contact inference is also
|
||||
not fabricated; contacts must be observed by a future calibrated classifier or
|
||||
authored explicitly.
|
||||
|
||||
The current COCO skeleton has no toe keypoints, so ankle rotation and precise
|
||||
foot contact cannot be inferred honestly. Those targets are omitted rather
|
||||
than guessed.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. A scene containing two actors, simultaneous targets and a cross-actor
|
||||
contact parses and serializes deterministically.
|
||||
2. Unknown actors, non-finite values, reversed timestamps and unbounded inputs
|
||||
fail closed.
|
||||
3. One RuView track converts to finite semantic targets with the same TrackId.
|
||||
4. Observed out-of-range angles produce warnings, not medical errors.
|
||||
5. Phase count is smaller than input frame count for a normal motion sequence.
|
||||
6. Focused crate tests and the complete workspace test suite pass before merge.
|
||||
|
||||
## Measured Results
|
||||
|
||||
On the implementation container with Rust 1.88, Criterion measured the canonical
|
||||
two-actor scene parser at 3.40 microseconds and serializer at 3.29 microseconds
|
||||
per operation. The protocol-only dependency surface is three runtime crates:
|
||||
`serde`, `serde_json` and `thiserror`.
|
||||
|
||||
The protocol-only configuration passes 13 unit tests, documentation tests,
|
||||
Rustfmt and Clippy with warnings denied. The `ruview` feature passes 14 unit
|
||||
tests, including native `PoseTrack` conversion, under the repository's RuVector
|
||||
AVX512-capable build configuration.
|
||||
|
||||
## References
|
||||
|
||||
1. PoseCode repository and protocol, MIT licensed:
|
||||
<https://github.com/posecode-dev/posecode>
|
||||
2. RuView multi-person pose decision: ADR-037.
|
||||
3. RuView persistent pose tracking: ADR-029 and `pose_tracker.rs`.
|
||||
+2
-6
@@ -1,11 +1,6 @@
|
||||
# Architecture Decision Records
|
||||
|
||||
Latest proposed decisions:
|
||||
|
||||
- [ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports](ADR-264-rtl8720f-radar-wire-protocol.md)
|
||||
- [ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md)
|
||||
|
||||
This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
|
||||
This folder contains 183 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
|
||||
|
||||
## Why ADRs?
|
||||
|
||||
@@ -128,6 +123,7 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
|
||||
| [ADR-263](ADR-263-ruview-npm-harness-deep-review.md) | `@ruvnet/ruview` npm harness — deep review + optimization strategy | Proposed |
|
||||
| [ADR-264](ADR-264-rvagent-mcp-and-cli-npm-deep-review.md) | `@ruvnet/rvagent` MCP server + `@ruv/ruview-cli` — deep review + optimization strategy | Proposed |
|
||||
| [ADR-265](ADR-265-ruview-npm-distribution-strategy.md) | RuView npm distribution strategy — CI gate, provenance, version single-sourcing, namespace | Proposed |
|
||||
| [ADR-266](ADR-266-multi-actor-posecode-scenes.md) | Multi-actor PoseCode scenes from persistent RuView tracks | Accepted |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,45 +0,0 @@
|
||||
# RuView v0.9.0-realtek-beta.1
|
||||
|
||||
This prerelease introduces the Rust-first RTL8720F 2.4 GHz radar transport and
|
||||
RuView ingestion path. It is intentionally simulator-validated until Realtek
|
||||
hardware and the vendor SDK callback ABI arrive.
|
||||
|
||||
## Included
|
||||
|
||||
- ADR-263 records the upstream Ameba integration and licensing boundary.
|
||||
- ADR-264 defines a versioned, bounded, CRC-protected radar envelope.
|
||||
- `rtl8720f-sim` emits deterministic CFR, near-range, far-range, interference,
|
||||
and capability reports to UDP or replay files.
|
||||
- The sensing server validates RTL8720F datagrams, publishes bounded summaries
|
||||
over `/ws/sensing`, and exposes the latest report at
|
||||
`/api/v1/radar/latest`.
|
||||
- Synthetic provenance is retained end to end as `realtek:simulated`; simulator
|
||||
data is never presented as hardware data.
|
||||
|
||||
## Compatibility
|
||||
|
||||
The adapter tracks the radar control surface proposed by Ameba RTOS pull
|
||||
request #1336 (`wifi_radar_config`, `AT+RAD`, and `AT+RADDBG`). The stable Ameba
|
||||
RTOS v1.2.1 release does not yet expose the complete radar receive callback ABI,
|
||||
so no vendor-private headers or binary libraries are copied into this release.
|
||||
|
||||
## Validation status
|
||||
|
||||
- Rust codec round trips, corruption rejection, size bounds, and deterministic
|
||||
simulator tests pass.
|
||||
- RuView server ingestion, REST reporting, and source provenance were exercised
|
||||
end to end over loopback UDP.
|
||||
- Windows release binaries are built from this branch and accompanied by
|
||||
SHA-256 checksums.
|
||||
|
||||
## Known limitations
|
||||
|
||||
- No physical RTL8720F board has been flashed or measured.
|
||||
- The vendor report callback and exact report layouts remain an SDK/hardware
|
||||
validation gate; the adapter boundary may change when those arrive.
|
||||
- This beta exposes transport and aggregate radar observability. Radar-to-pose,
|
||||
vital-sign inference, RF calibration, and accuracy claims are not enabled.
|
||||
- 2.4 GHz radar reports are not mislabeled as mmWave or Wi-Fi CSI events.
|
||||
|
||||
Do not deploy this prerelease for safety-critical, medical, or occupancy billing
|
||||
uses. It is an integration beta for SDK and hardware bring-up.
|
||||
@@ -1861,23 +1861,6 @@ node scripts/eval-wiflow.js \
|
||||
--data data/paired/*.jsonl
|
||||
```
|
||||
|
||||
> **Model format boundary:** `train-wiflow-supervised.js` produces the
|
||||
> JavaScript WiFlow model `wiflow-v1.json`. There is currently no supported
|
||||
> command that converts that JSON model into the sensing server's binary RVF
|
||||
> container, and renaming the file to `.rvf` does not convert it. Use the JSON
|
||||
> model with the JavaScript evaluation/inference tools. To train a model that
|
||||
> the Rust sensing server can load, use its native training path, which writes
|
||||
> RVF directly:
|
||||
>
|
||||
> ```bash
|
||||
> cargo run -p wifi-densepose-sensing-server --release -- \
|
||||
> --train --dataset data/mmfi --dataset-type mmfi \
|
||||
> --epochs 100 --save-rvf models/room-model.rvf
|
||||
> ```
|
||||
>
|
||||
> The camera+CSI paired JSONL workflow and the native RVF trainer are separate
|
||||
> pipelines today. A JSON-to-RVF exporter is future work.
|
||||
|
||||
**Evaluation protocol matters.** Use `eval-wiflow.js` (torso-normalized
|
||||
PCK@20, the metric comparable to published WiFi-pose results) on a temporal
|
||||
hold-out, and sanity-check that predictions actually vary across frames
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# ESP32 CSI Node Firmware (ADR-018)
|
||||
# Requires ESP-IDF v5.4+
|
||||
# Requires ESP-IDF v5.2+
|
||||
cmake_minimum_required(VERSION 3.16)
|
||||
|
||||
set(EXTRA_COMPONENT_DIRS "")
|
||||
|
||||
@@ -113,8 +113,6 @@ curl http://<ESP32_IP>:8032/wasm/list
|
||||
|
||||
> **Tip:** A single node provides presence and vital signs along its line of sight. Multiple nodes (3-6) create a multistatic mesh that resolves 3D pose with <30 mm jitter and zero identity swaps.
|
||||
|
||||
> **⚠️ Thermal warning — compact boards (ESP32-S3-Zero, SuperMini, other coin-sized clones):** This firmware runs the WiFi radio with modem sleep disabled (`WIFI_PS_NONE`, required for continuous CSI capture) plus a full edge-processing DSP pipeline on Core 1 (`edge_tier=2`) plus, on ADR-183 builds, a continuous 40 Hz onboard LED driver. That's sustained high current draw with no duty-cycling. Full-size dev boards (DevKitC-1, XIAO) have more copper pour and thermal mass around the regulator and tolerate this fine. Coin-sized clones with minimal PCB area and budget regulators may run hot to the touch during normal operation, and in at least one field report, boards that ran hot during a session failed to power on afterward (regulator damage suspected — see issue tracker). Give these boards airflow, don't stack or enclose them, and check them by touch during the first several minutes of a new deployment. If a board is uncomfortably hot (not just warm), power it down and let it cool before continuing.
|
||||
|
||||
---
|
||||
|
||||
## Firmware Architecture
|
||||
|
||||
@@ -67,8 +67,6 @@ static void event_handler(void *arg, esp_event_base_t event_base,
|
||||
if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_START) {
|
||||
esp_wifi_connect();
|
||||
} else if (event_base == WIFI_EVENT && event_id == WIFI_EVENT_STA_DISCONNECTED) {
|
||||
wifi_event_sta_disconnected_t *disc = (wifi_event_sta_disconnected_t *)event_data;
|
||||
ESP_LOGW(TAG, "WiFi disconnected, reason=%d rssi=%d", disc->reason, disc->rssi);
|
||||
if (s_retry_num < MAX_RETRY) {
|
||||
esp_wifi_connect();
|
||||
s_retry_num++;
|
||||
@@ -104,10 +102,7 @@ static void wifi_init_sta(void)
|
||||
|
||||
wifi_config_t wifi_config = {
|
||||
.sta = {
|
||||
/* WPA_PSK (not WPA2_PSK) so routers running WPA/WPA2-mixed
|
||||
* compatibility mode aren't rejected with
|
||||
* WIFI_REASON_NO_AP_FOUND_IN_AUTHMODE_THRESHOLD (#1050). */
|
||||
.threshold.authmode = WIFI_AUTH_WPA_PSK,
|
||||
.threshold.authmode = WIFI_AUTH_WPA2_PSK,
|
||||
},
|
||||
};
|
||||
|
||||
|
||||
@@ -1 +1 @@
|
||||
0.8.4
|
||||
0.7.0
|
||||
|
||||
@@ -18,8 +18,6 @@ Bring a RuView sensing node online: build firmware → flash → provision WiFi
|
||||
|
||||
**Not supported:** original ESP32, ESP32-C3 (single-core).
|
||||
|
||||
**⚠️ Ask about board form factor before flashing.** If the user's board is a coin-sized clone (ESP32-S3-Zero, SuperMini, or similar — not a full DevKitC/XIAO-style board with a real USB connector and visible regulator), warn them before they walk away from it: this firmware runs the WiFi radio continuously (`WIFI_PS_NONE`) plus a full DSP pipeline (`edge_tier=2`), which is sustained high current draw that full-size dev boards handle fine but tiny clones with minimal copper/budget regulators may not. At least one field report: boards ran hot during a normal session and failed to power on again afterward (regulator damage suspected). Tell them to give the board airflow (don't stack/enclose it) and check it by touch during the first several minutes of any new deployment.
|
||||
|
||||
## 1. Build firmware (Windows — Python subprocess, NOT bash directly)
|
||||
|
||||
ESP-IDF v5.4 does not support MSYS2/Git Bash. Use the Espressif Python venv as a subprocess with `MSYSTEM*` env vars stripped. The proven command lives in `CLAUDE.local.md` — reproduce it:
|
||||
|
||||
@@ -57,12 +57,12 @@ def test_heart_rate_extract_per_frame_cost(benchmark) -> None:
|
||||
hr = HeartRateExtractor.esp32_default()
|
||||
rng = Random(43)
|
||||
for i in range(1500):
|
||||
residuals, phases = _synth_frame(56, 100.0, i / 100.0, 1.2, rng)
|
||||
hr.extract(residuals=residuals, phases=phases)
|
||||
residuals, weights = _synth_frame(56, 100.0, i / 100.0, 1.2, rng)
|
||||
hr.extract(residuals=residuals, weights=weights)
|
||||
|
||||
def _one_frame():
|
||||
residuals, phases = _synth_frame(56, 100.0, 16.0, 1.2, rng)
|
||||
return hr.extract(residuals=residuals, phases=phases)
|
||||
residuals, weights = _synth_frame(56, 100.0, 16.0, 1.2, rng)
|
||||
return hr.extract(residuals=residuals, weights=weights)
|
||||
|
||||
benchmark(_one_frame)
|
||||
|
||||
|
||||
@@ -242,22 +242,7 @@ impl PyBreathingExtractor {
|
||||
// ─── HeartRateExtractor ──────────────────────────────────────────────
|
||||
|
||||
/// Extracts heart rate (40–120 BPM) from per-subcarrier amplitude
|
||||
/// residuals and per-subcarrier unwrapped phases (radians) via
|
||||
/// 0.8–2.0 Hz bandpass + autocorrelation peak detection.
|
||||
///
|
||||
/// Python:
|
||||
/// ```python
|
||||
/// from wifi_densepose import HeartRateExtractor
|
||||
///
|
||||
/// hr = HeartRateExtractor.esp32_default() # 56 subcarriers, 100 Hz, 15s window
|
||||
///
|
||||
/// # Feed residuals and matching unwrapped phases from your preprocessor.
|
||||
/// # Unlike BreathingExtractor weights, phases=[] is invalid for heart-rate
|
||||
/// # extraction because the Rust core requires phase data for each subcarrier.
|
||||
/// est = hr.extract(residuals=[0.01, -0.02, …], phases=[0.0, 0.01, …])
|
||||
/// if est is not None:
|
||||
/// print(est.value_bpm, est.confidence)
|
||||
/// ```
|
||||
/// residuals via 0.8–2.0 Hz bandpass + autocorrelation peak detection.
|
||||
#[pyclass(name = "HeartRateExtractor")]
|
||||
pub struct PyHeartRateExtractor {
|
||||
inner: HeartRateExtractor,
|
||||
@@ -280,17 +265,10 @@ impl PyHeartRateExtractor {
|
||||
Self { inner: HeartRateExtractor::esp32_default() }
|
||||
}
|
||||
|
||||
/// Extract heart rate from per-subcarrier residuals and matching
|
||||
/// per-subcarrier unwrapped phases (radians). Empty phases are invalid
|
||||
/// and return `None` because the Rust extractor requires phase data.
|
||||
/// GIL released during DSP.
|
||||
fn extract(
|
||||
&mut self,
|
||||
py: Python<'_>,
|
||||
residuals: Vec<f64>,
|
||||
phases: Vec<f64>,
|
||||
) -> Option<PyVitalEstimate> {
|
||||
let est = py.allow_threads(|| self.inner.extract(&residuals, &phases));
|
||||
/// Extract heart rate from per-subcarrier residuals. GIL released
|
||||
/// during DSP.
|
||||
fn extract(&mut self, py: Python<'_>, residuals: Vec<f64>, weights: Vec<f64>) -> Option<PyVitalEstimate> {
|
||||
let est = py.allow_threads(|| self.inner.extract(&residuals, &weights));
|
||||
est.map(PyVitalEstimate::from_rust)
|
||||
}
|
||||
|
||||
|
||||
@@ -185,44 +185,10 @@ def test_heart_rate_explicit_ctor() -> None:
|
||||
|
||||
def test_heart_rate_extract_returns_none_with_too_few_samples() -> None:
|
||||
hr = HeartRateExtractor.esp32_default()
|
||||
out = hr.extract(residuals=[0.0] * 56, phases=[0.0] * 56)
|
||||
out = hr.extract(residuals=[0.0] * 56, weights=[])
|
||||
assert out is None
|
||||
|
||||
|
||||
def test_heart_rate_extract_rejects_old_weights_keyword() -> None:
|
||||
hr = HeartRateExtractor.esp32_default()
|
||||
with pytest.raises(TypeError):
|
||||
hr.extract(residuals=[0.0] * 56, weights=[0.0] * 56)
|
||||
|
||||
|
||||
def test_heart_rate_extract_with_synthetic_signal_and_phases() -> None:
|
||||
"""Drive the extractor with a synthetic 1.2 Hz sine (72 BPM) plus
|
||||
same-length phase data. This proves the binding feeds Rust's required
|
||||
`phases` slice instead of an empty vector that would keep returning None."""
|
||||
hr = HeartRateExtractor.esp32_default()
|
||||
sample_rate = 100.0
|
||||
target_freq = 1.2 # 72 BPM
|
||||
n_samples = int(60 * sample_rate)
|
||||
phases = [i * 0.01 for i in range(56)]
|
||||
|
||||
produced_estimate = False
|
||||
rng = Random(43)
|
||||
for i in range(n_samples):
|
||||
t = i / sample_rate
|
||||
base = math.sin(2.0 * math.pi * target_freq * t)
|
||||
residuals = [base + rng.gauss(0.0, 0.01) for _ in range(56)]
|
||||
est = hr.extract(residuals=residuals, phases=phases)
|
||||
if est is not None:
|
||||
produced_estimate = True
|
||||
assert math.isfinite(est.value_bpm)
|
||||
assert 0.0 <= est.confidence <= 1.0
|
||||
break
|
||||
|
||||
assert produced_estimate, (
|
||||
"HeartRateExtractor never produced an estimate after 60s of synthetic data"
|
||||
)
|
||||
|
||||
|
||||
# ─── Build feature flag ──────────────────────────────────────────────
|
||||
|
||||
|
||||
|
||||
Generated
+11
@@ -11091,6 +11091,17 @@ dependencies = [
|
||||
"tower-http",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wifi-densepose-posecode"
|
||||
version = "0.3.0"
|
||||
dependencies = [
|
||||
"criterion",
|
||||
"serde",
|
||||
"serde_json",
|
||||
"thiserror 2.0.18",
|
||||
"wifi-densepose-signal",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "wifi-densepose-rufield"
|
||||
version = "0.3.0"
|
||||
|
||||
@@ -46,6 +46,9 @@ members = [
|
||||
# PR #491 slot heuristic with a Candle network + Stoer-Wagner fusion.
|
||||
# Motivated by #499 ghost-skeleton reports.
|
||||
"crates/cog-person-count",
|
||||
# ADR-266: Multi-actor PoseCode scene protocol. Converts persistent
|
||||
# RuvSense PoseTracks into confidence-scored, deterministic motion scenes.
|
||||
"crates/wifi-densepose-posecode",
|
||||
# ADR-116: Home Assistant + Matter Cognitum Seed cog. Wraps the
|
||||
# ADR-115 MQTT publisher as a Seed-installable artifact with
|
||||
# mDNS, embedded broker, RuVector thresholds, Ed25519 witness.
|
||||
@@ -215,6 +218,7 @@ wifi-densepose-hardware = { version = "0.3.0", path = "crates/wifi-densepose-har
|
||||
wifi-densepose-wasm = { version = "0.3.0", path = "crates/wifi-densepose-wasm" }
|
||||
wifi-densepose-mat = { version = "0.3.0", path = "crates/wifi-densepose-mat" }
|
||||
wifi-densepose-ruvector = { version = "0.3.0", path = "crates/wifi-densepose-ruvector" }
|
||||
wifi-densepose-posecode = { version = "0.3.0", path = "crates/wifi-densepose-posecode" }
|
||||
wifi-densepose-worldmodel = { version = "0.3.0", path = "crates/worldgraph/wifi-densepose-worldmodel" }
|
||||
|
||||
[profile.release]
|
||||
|
||||
@@ -13,21 +13,6 @@ hardware sources. All parsing operates on byte buffers with no C FFI or hardware
|
||||
compile time, making the crate fully portable and deterministic -- the same bytes in always produce
|
||||
the same parsed output.
|
||||
|
||||
## RTL8720F radar simulator (ADR-263/264)
|
||||
|
||||
Until Realtek hardware and the radar report SDK arrive, the Rust-only simulator exercises the same
|
||||
versioned CFR/Range-FFT wire codec used by the future device adapter. Every frame is marked
|
||||
`SYNTHETIC`.
|
||||
|
||||
```powershell
|
||||
cargo run -p wifi-densepose-hardware --bin rtl8720f-sim -- `
|
||||
--frames 100 --seed 0x8720f123456789ab `
|
||||
--output rtl8720f-synthetic.rtr
|
||||
```
|
||||
|
||||
Add `--udp 127.0.0.1:5005 --realtime` to stream one ADR-264 frame per UDP datagram. Replay files
|
||||
contain a little-endian `u32` frame length followed by the encoded frame.
|
||||
|
||||
## Features
|
||||
|
||||
- **ESP32 binary parser** -- Parses ADR-018 binary CSI frames streamed over UDP from ESP32 and
|
||||
|
||||
@@ -1,118 +0,0 @@
|
||||
//! Rust-only RTL8720F radar simulator for pre-hardware integration.
|
||||
|
||||
use std::{
|
||||
fs::File,
|
||||
io::{self, Write},
|
||||
net::{SocketAddr, UdpSocket},
|
||||
path::PathBuf,
|
||||
thread,
|
||||
time::Duration,
|
||||
};
|
||||
|
||||
use clap::Parser;
|
||||
use wifi_densepose_hardware::rtl8720f::{
|
||||
simulator::{Rtl8720fSimulator, SimulatorConfig},
|
||||
RadarFrame, ReportType,
|
||||
};
|
||||
|
||||
#[derive(Debug, Parser)]
|
||||
#[command(
|
||||
name = "rtl8720f-sim",
|
||||
about = "Emit synthetic ADR-264 RTL8720F radar frames"
|
||||
)]
|
||||
struct Args {
|
||||
#[arg(long, default_value_t = 100)]
|
||||
frames: u32,
|
||||
#[arg(long, default_value = "0x8720f123456789ab", value_parser = parse_u64)]
|
||||
seed: u64,
|
||||
#[arg(long, default_value_t = 40)]
|
||||
bandwidth: u16,
|
||||
#[arg(long, default_value_t = 15)]
|
||||
interval_ms: u64,
|
||||
/// UDP destination; each frame is one datagram.
|
||||
#[arg(long)]
|
||||
udp: Option<SocketAddr>,
|
||||
/// Replay file; LE u32 length followed by ADR-264 bytes.
|
||||
#[arg(long)]
|
||||
output: Option<PathBuf>,
|
||||
#[arg(long)]
|
||||
realtime: bool,
|
||||
}
|
||||
|
||||
fn parse_u64(value: &str) -> Result<u64, String> {
|
||||
if let Some(hex) = value
|
||||
.strip_prefix("0x")
|
||||
.or_else(|| value.strip_prefix("0X"))
|
||||
{
|
||||
u64::from_str_radix(hex, 16).map_err(|error| error.to_string())
|
||||
} else {
|
||||
value.parse::<u64>().map_err(|error| error.to_string())
|
||||
}
|
||||
}
|
||||
|
||||
fn emit(
|
||||
frame: RadarFrame,
|
||||
socket: Option<&UdpSocket>,
|
||||
destination: Option<SocketAddr>,
|
||||
output: &mut Option<File>,
|
||||
) -> Result<usize, Box<dyn std::error::Error>> {
|
||||
let wire = frame.to_bytes()?;
|
||||
if let (Some(socket), Some(destination)) = (socket, destination) {
|
||||
let sent = socket.send_to(&wire, destination)?;
|
||||
if sent != wire.len() {
|
||||
return Err(io::Error::new(io::ErrorKind::WriteZero, "partial UDP datagram").into());
|
||||
}
|
||||
}
|
||||
if let Some(file) = output {
|
||||
file.write_all(&(wire.len() as u32).to_le_bytes())?;
|
||||
file.write_all(&wire)?;
|
||||
}
|
||||
Ok(wire.len())
|
||||
}
|
||||
|
||||
fn main() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let args = Args::parse();
|
||||
if args.udp.is_none() && args.output.is_none() {
|
||||
return Err("select at least one sink with --udp or --output".into());
|
||||
}
|
||||
let config = SimulatorConfig {
|
||||
seed: args.seed,
|
||||
bandwidth_mhz: args.bandwidth,
|
||||
frame_period_us: args.interval_ms * 1_000,
|
||||
..SimulatorConfig::default()
|
||||
};
|
||||
let mut simulator = Rtl8720fSimulator::new(config)?;
|
||||
let socket = args.udp.map(|_| UdpSocket::bind("0.0.0.0:0")).transpose()?;
|
||||
let mut output = args.output.as_ref().map(File::create).transpose()?;
|
||||
let mut bytes_emitted = emit(
|
||||
simulator.capabilities_frame(),
|
||||
socket.as_ref(),
|
||||
args.udp,
|
||||
&mut output,
|
||||
)?;
|
||||
|
||||
for index in 0..args.frames {
|
||||
let report_type = match index % 16 {
|
||||
15 => ReportType::Interference,
|
||||
value if value % 4 == 1 => ReportType::RangeNear,
|
||||
value if value % 4 == 3 => ReportType::RangeFar,
|
||||
_ => ReportType::Cfr,
|
||||
};
|
||||
bytes_emitted += emit(
|
||||
simulator.next_frame(report_type),
|
||||
socket.as_ref(),
|
||||
args.udp,
|
||||
&mut output,
|
||||
)?;
|
||||
if args.realtime {
|
||||
thread::sleep(Duration::from_millis(args.interval_ms));
|
||||
}
|
||||
}
|
||||
eprintln!(
|
||||
"emitted {} synthetic RTL8720F frames ({} bytes, seed={:#x})",
|
||||
args.frames + 1,
|
||||
bytes_emitted,
|
||||
args.seed
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
@@ -53,9 +53,6 @@ pub mod sync_packet;
|
||||
// coordinator-node Rust code drive the controller stack without
|
||||
// touching any downstream signal/ruvector/train/mat crate.
|
||||
pub mod radio_ops;
|
||||
/// ADR-264 host-side framing for Realtek RTL8720F CFR and FMCW radar reports.
|
||||
/// This module has no dependency on the vendor SDK.
|
||||
pub mod rtl8720f;
|
||||
|
||||
pub use bridge::CsiData;
|
||||
pub use csi_frame::{
|
||||
@@ -67,18 +64,12 @@ pub use esp32_parser::{
|
||||
RUVIEW_FEATURE_MAGIC, RUVIEW_FEATURE_STATE_MAGIC, RUVIEW_FUSED_VITALS_MAGIC,
|
||||
RUVIEW_TEMPORAL_MAGIC, RUVIEW_VITALS_MAGIC,
|
||||
};
|
||||
pub use sync_packet::{
|
||||
SyncPacket, SyncPacketFlags, SYNC_PACKET_MAGIC, SYNC_PACKET_SIZE, SYNC_PACKET_PROTO_VER,
|
||||
};
|
||||
pub use radio_ops::{
|
||||
crc32_ieee, decode_anomaly_alert, decode_mesh, decode_node_status, encode_health, AnomalyAlert,
|
||||
AuthClass, CaptureProfile, MeshError, MeshHeader, MeshMsgType, MeshRole, MockRadio, NodeStatus,
|
||||
RadioError, RadioHealth, RadioMode, RadioOps, MESH_HEADER_SIZE, MESH_MAGIC, MESH_MAX_PAYLOAD,
|
||||
MESH_VERSION,
|
||||
};
|
||||
pub use rtl8720f::{
|
||||
ElementFormat as Rtl8720fElementFormat, RadarFlags as Rtl8720fRadarFlags,
|
||||
RadarFrame as Rtl8720fRadarFrame, RadarParseError as Rtl8720fRadarParseError,
|
||||
RadarPayload as Rtl8720fRadarPayload, ReportType as Rtl8720fReportType,
|
||||
RTL8720F_RADAR_HEADER_LEN, RTL8720F_RADAR_MAGIC, RTL8720F_RADAR_VERSION,
|
||||
};
|
||||
pub use sync_packet::{
|
||||
SyncPacket, SyncPacketFlags, SYNC_PACKET_MAGIC, SYNC_PACKET_PROTO_VER, SYNC_PACKET_SIZE,
|
||||
};
|
||||
|
||||
@@ -1,852 +0,0 @@
|
||||
//! ADR-264 transport-neutral framing for Realtek RTL8720F radar reports.
|
||||
//!
|
||||
//! This is a RuView-owned wire contract around the public Ameba API boundary,
|
||||
//! not a representation of Realtek's private structs. Upstream PR #1336 exposes
|
||||
//! `wifi_radar_config(struct rtw_radar_action_parm *)`; the report callback ABI
|
||||
//! remains vendor-gated. Keeping this codec byte-oriented lets host development,
|
||||
//! replay, and fuzzing proceed without linking the Ameba SDK.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use thiserror::Error;
|
||||
|
||||
use crate::radio_ops::crc32_ieee;
|
||||
|
||||
pub const RTL8720F_RADAR_MAGIC: u32 = 0x3152_5452; // "RTR1" in little endian
|
||||
pub const RTL8720F_RADAR_VERSION: u8 = 1;
|
||||
pub const RTL8720F_RADAR_HEADER_LEN: usize = 56;
|
||||
pub const RTL8720F_RADAR_CRC_LEN: usize = 4;
|
||||
/// Largest payload that can be carried in one IPv4 UDP datagram.
|
||||
pub const RTL8720F_RADAR_MAX_FRAME_LEN: usize = 65_507;
|
||||
pub const RTL8720F_RADAR_MAX_ELEMENTS: usize = 16_384;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[repr(u8)]
|
||||
pub enum ReportType {
|
||||
Cfr = 1,
|
||||
RangeNear = 2,
|
||||
RangeFar = 3,
|
||||
Interference = 4,
|
||||
Capabilities = 5,
|
||||
}
|
||||
|
||||
impl TryFrom<u8> for ReportType {
|
||||
type Error = RadarParseError;
|
||||
|
||||
fn try_from(value: u8) -> Result<Self, Self::Error> {
|
||||
match value {
|
||||
1 => Ok(Self::Cfr),
|
||||
2 => Ok(Self::RangeNear),
|
||||
3 => Ok(Self::RangeFar),
|
||||
4 => Ok(Self::Interference),
|
||||
5 => Ok(Self::Capabilities),
|
||||
_ => Err(RadarParseError::UnknownReportType(value)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[repr(u8)]
|
||||
pub enum ElementFormat {
|
||||
/// TLV/opaque byte payload used by capabilities and interference reports.
|
||||
Bytes = 0,
|
||||
ComplexI16 = 1,
|
||||
ComplexF32 = 2,
|
||||
PowerU16 = 3,
|
||||
PowerF32 = 4,
|
||||
}
|
||||
|
||||
impl ElementFormat {
|
||||
fn bytes_per_element(self) -> usize {
|
||||
match self {
|
||||
Self::Bytes => 1,
|
||||
Self::ComplexI16 | Self::PowerF32 => 4,
|
||||
Self::ComplexF32 => 8,
|
||||
Self::PowerU16 => 2,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl TryFrom<u8> for ElementFormat {
|
||||
type Error = RadarParseError;
|
||||
|
||||
fn try_from(value: u8) -> Result<Self, Self::Error> {
|
||||
match value {
|
||||
0 => Ok(Self::Bytes),
|
||||
1 => Ok(Self::ComplexI16),
|
||||
2 => Ok(Self::ComplexF32),
|
||||
3 => Ok(Self::PowerU16),
|
||||
4 => Ok(Self::PowerF32),
|
||||
_ => Err(RadarParseError::UnknownElementFormat(value)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, Serialize, Deserialize)]
|
||||
pub struct RadarFlags(pub u16);
|
||||
|
||||
impl RadarFlags {
|
||||
pub const CALIBRATED: u16 = 1 << 0;
|
||||
pub const INTERFERENCE_DETECTED: u16 = 1 << 1;
|
||||
pub const SATURATED: u16 = 1 << 2;
|
||||
pub const TIME_SYNCHRONIZED: u16 = 1 << 3;
|
||||
/// Frame was produced by a simulator/replay generator, never real hardware.
|
||||
pub const SYNTHETIC: u16 = 1 << 15;
|
||||
|
||||
pub fn contains(self, flag: u16) -> bool {
|
||||
self.0 & flag != 0
|
||||
}
|
||||
}
|
||||
|
||||
/// Deterministic, Rust-only RTL8720F source used until hardware is available.
|
||||
/// It emits the same [`RadarFrame`] objects and wire bytes as the vendor adapter.
|
||||
pub mod simulator {
|
||||
use super::*;
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct SimulatorConfig {
|
||||
pub seed: u64,
|
||||
pub device_id: u64,
|
||||
pub center_freq_khz: u32,
|
||||
pub bandwidth_mhz: u16,
|
||||
pub frame_period_us: u64,
|
||||
pub cfr_bins: u16,
|
||||
pub range_bins: u16,
|
||||
}
|
||||
|
||||
impl Default for SimulatorConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
seed: 0x8720_F123_4567_89AB,
|
||||
device_id: 0x5254_4C38_3732_3046,
|
||||
center_freq_khz: 2_442_000,
|
||||
bandwidth_mhz: 40,
|
||||
frame_period_us: 15_000,
|
||||
cfr_bins: 128,
|
||||
range_bins: 32,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Rtl8720fSimulator {
|
||||
config: SimulatorConfig,
|
||||
rng: u64,
|
||||
sequence: u32,
|
||||
timestamp_us: u64,
|
||||
target_distance_m: f32,
|
||||
target_velocity_mps: f32,
|
||||
}
|
||||
|
||||
impl Rtl8720fSimulator {
|
||||
pub fn new(config: SimulatorConfig) -> Result<Self, RadarParseError> {
|
||||
if !matches!(config.bandwidth_mhz, 20 | 40 | 70) {
|
||||
return Err(RadarParseError::InvalidBandwidth(config.bandwidth_mhz));
|
||||
}
|
||||
if config.cfr_bins == 0 || config.range_bins == 0 {
|
||||
return Err(RadarParseError::TooManyElements(0));
|
||||
}
|
||||
Ok(Self {
|
||||
rng: config.seed,
|
||||
config,
|
||||
sequence: 0,
|
||||
timestamp_us: 0,
|
||||
target_distance_m: 2.0,
|
||||
target_velocity_mps: 0.20,
|
||||
})
|
||||
}
|
||||
|
||||
pub fn config(&self) -> &SimulatorConfig {
|
||||
&self.config
|
||||
}
|
||||
pub fn target_distance_m(&self) -> f32 {
|
||||
self.target_distance_m
|
||||
}
|
||||
pub fn target_velocity_mps(&self) -> f32 {
|
||||
self.target_velocity_mps
|
||||
}
|
||||
|
||||
/// Emit the boot-time capabilities report as compact TLVs:
|
||||
/// type 1 = bandwidth bitset, 2 = CFR bins, 3 = range bins,
|
||||
/// 4 = minimum frame period in microseconds.
|
||||
pub fn capabilities_frame(&self) -> RadarFrame {
|
||||
let bandwidths = 0b0000_0111u8; // 20, 40, 70 MHz
|
||||
let mut bytes = vec![1, 1, bandwidths, 2, 2];
|
||||
bytes.extend_from_slice(&self.config.cfr_bins.to_le_bytes());
|
||||
bytes.extend_from_slice(&[3, 2]);
|
||||
bytes.extend_from_slice(&self.config.range_bins.to_le_bytes());
|
||||
bytes.extend_from_slice(&[4, 4]);
|
||||
bytes.extend_from_slice(&(self.config.frame_period_us as u32).to_le_bytes());
|
||||
self.frame(
|
||||
ReportType::Capabilities,
|
||||
0,
|
||||
0,
|
||||
RadarPayload::Bytes(bytes),
|
||||
0.0,
|
||||
)
|
||||
}
|
||||
|
||||
pub fn next_frame(&mut self, report_type: ReportType) -> RadarFrame {
|
||||
let sequence = self.sequence;
|
||||
let timestamp_us = self.timestamp_us;
|
||||
self.sequence = self.sequence.wrapping_add(1);
|
||||
self.timestamp_us = self.timestamp_us.wrapping_add(self.config.frame_period_us);
|
||||
self.advance_target();
|
||||
|
||||
match report_type {
|
||||
ReportType::Cfr => {
|
||||
let values = (0..self.config.cfr_bins)
|
||||
.map(|bin| {
|
||||
let phase = bin as f32 * 0.17 + sequence as f32 * 0.05;
|
||||
let noise_i = self.noise_i16(20);
|
||||
let noise_q = self.noise_i16(20);
|
||||
[
|
||||
(phase.cos() * 1800.0) as i16 + noise_i,
|
||||
(phase.sin() * 1800.0) as i16 + noise_q,
|
||||
]
|
||||
})
|
||||
.collect();
|
||||
self.frame(
|
||||
report_type,
|
||||
sequence,
|
||||
timestamp_us,
|
||||
RadarPayload::ComplexI16(values),
|
||||
self.config.bandwidth_mhz as f32 * 1_000_000.0
|
||||
/ self.config.cfr_bins as f32,
|
||||
)
|
||||
}
|
||||
ReportType::RangeNear | ReportType::RangeFar => {
|
||||
let bin_spacing = match self.config.bandwidth_mhz {
|
||||
70 => 0.33,
|
||||
40 => 0.59,
|
||||
_ => 1.18,
|
||||
};
|
||||
let target_bin = (self.target_distance_m / bin_spacing).round() as usize;
|
||||
let values = (0..self.config.range_bins as usize)
|
||||
.map(|bin| {
|
||||
let distance = bin.abs_diff(target_bin) as f32;
|
||||
let peak = 1000.0 * (-0.5 * distance * distance).exp();
|
||||
let leakage = if report_type == ReportType::RangeNear && bin < 2 {
|
||||
250.0
|
||||
} else {
|
||||
0.0
|
||||
};
|
||||
(peak + leakage + self.noise_f32(12.0)).max(0.0)
|
||||
})
|
||||
.collect();
|
||||
self.frame(
|
||||
report_type,
|
||||
sequence,
|
||||
timestamp_us,
|
||||
RadarPayload::PowerF32(values),
|
||||
bin_spacing,
|
||||
)
|
||||
}
|
||||
ReportType::Interference => {
|
||||
// TLV: channel-busy %, detected-during-chirp, signed dBm.
|
||||
let busy = (self.next_u32() % 35) as u8;
|
||||
let detected = u8::from(busy > 25);
|
||||
let dbm = (-90i8 + (self.next_u32() % 25) as i8) as u8;
|
||||
let bytes = vec![1, 1, busy, 2, 1, detected, 3, 1, dbm];
|
||||
let mut frame = self.frame(
|
||||
report_type,
|
||||
sequence,
|
||||
timestamp_us,
|
||||
RadarPayload::Bytes(bytes),
|
||||
0.0,
|
||||
);
|
||||
if detected != 0 {
|
||||
frame.flags.0 |= RadarFlags::INTERFERENCE_DETECTED;
|
||||
}
|
||||
frame
|
||||
}
|
||||
ReportType::Capabilities => self.capabilities_frame(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn next_wire(&mut self, report_type: ReportType) -> Result<Vec<u8>, RadarParseError> {
|
||||
self.next_frame(report_type).to_bytes()
|
||||
}
|
||||
|
||||
fn frame(
|
||||
&self,
|
||||
report_type: ReportType,
|
||||
sequence: u32,
|
||||
timestamp_us: u64,
|
||||
payload: RadarPayload,
|
||||
bin_spacing: f32,
|
||||
) -> RadarFrame {
|
||||
RadarFrame {
|
||||
report_type,
|
||||
sequence,
|
||||
timestamp_us,
|
||||
device_id: self.config.device_id,
|
||||
center_freq_khz: self.config.center_freq_khz,
|
||||
bandwidth_mhz: self.config.bandwidth_mhz,
|
||||
flags: RadarFlags(RadarFlags::CALIBRATED | RadarFlags::SYNTHETIC),
|
||||
antenna_count: 1,
|
||||
scale: 1.0,
|
||||
bin_spacing,
|
||||
calibration_id: 0,
|
||||
payload,
|
||||
}
|
||||
}
|
||||
|
||||
fn advance_target(&mut self) {
|
||||
let dt = self.config.frame_period_us as f32 / 1_000_000.0;
|
||||
self.target_distance_m += self.target_velocity_mps * dt;
|
||||
if self.target_distance_m >= 5.5 || self.target_distance_m <= 0.8 {
|
||||
self.target_velocity_mps = -self.target_velocity_mps;
|
||||
self.target_distance_m = self.target_distance_m.clamp(0.8, 5.5);
|
||||
}
|
||||
}
|
||||
|
||||
fn next_u32(&mut self) -> u32 {
|
||||
// PCG-style state transition with xorshift output; deterministic and dependency-free.
|
||||
self.rng = self
|
||||
.rng
|
||||
.wrapping_mul(6364136223846793005)
|
||||
.wrapping_add(1442695040888963407);
|
||||
let word = (((self.rng >> 18) ^ self.rng) >> 27) as u32;
|
||||
word.rotate_right((self.rng >> 59) as u32)
|
||||
}
|
||||
|
||||
fn noise_i16(&mut self, amplitude: i16) -> i16 {
|
||||
(self.next_u32() % (amplitude as u32 * 2 + 1)) as i16 - amplitude
|
||||
}
|
||||
|
||||
fn noise_f32(&mut self, amplitude: f32) -> f32 {
|
||||
let unit = self.next_u32() as f32 / u32::MAX as f32;
|
||||
(unit * 2.0 - 1.0) * amplitude
|
||||
}
|
||||
}
|
||||
|
||||
impl Iterator for Rtl8720fSimulator {
|
||||
type Item = RadarFrame;
|
||||
|
||||
fn next(&mut self) -> Option<Self::Item> {
|
||||
let report_type = match self.sequence % 4 {
|
||||
0 | 2 => ReportType::Cfr,
|
||||
1 => ReportType::RangeNear,
|
||||
_ => ReportType::RangeFar,
|
||||
};
|
||||
Some(self.next_frame(report_type))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub enum RadarPayload {
|
||||
Bytes(Vec<u8>),
|
||||
ComplexI16(Vec<[i16; 2]>),
|
||||
ComplexF32(Vec<[f32; 2]>),
|
||||
PowerU16(Vec<u16>),
|
||||
PowerF32(Vec<f32>),
|
||||
}
|
||||
|
||||
impl RadarPayload {
|
||||
pub fn format(&self) -> ElementFormat {
|
||||
match self {
|
||||
Self::Bytes(_) => ElementFormat::Bytes,
|
||||
Self::ComplexI16(_) => ElementFormat::ComplexI16,
|
||||
Self::ComplexF32(_) => ElementFormat::ComplexF32,
|
||||
Self::PowerU16(_) => ElementFormat::PowerU16,
|
||||
Self::PowerF32(_) => ElementFormat::PowerF32,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn len(&self) -> usize {
|
||||
match self {
|
||||
Self::Bytes(v) => v.len(),
|
||||
Self::ComplexI16(v) => v.len(),
|
||||
Self::ComplexF32(v) => v.len(),
|
||||
Self::PowerU16(v) => v.len(),
|
||||
Self::PowerF32(v) => v.len(),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn is_empty(&self) -> bool {
|
||||
self.len() == 0
|
||||
}
|
||||
|
||||
fn encoded_len(&self) -> usize {
|
||||
self.len() * self.format().bytes_per_element()
|
||||
}
|
||||
|
||||
fn validate_finite(&self) -> Result<(), RadarParseError> {
|
||||
let valid = match self {
|
||||
Self::ComplexF32(values) => values.iter().flatten().all(|v| v.is_finite()),
|
||||
Self::PowerF32(values) => values.iter().all(|v| v.is_finite()),
|
||||
_ => true,
|
||||
};
|
||||
if valid {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(RadarParseError::NonFiniteValue)
|
||||
}
|
||||
}
|
||||
|
||||
fn encode_into(&self, out: &mut Vec<u8>) {
|
||||
match self {
|
||||
Self::Bytes(values) => out.extend_from_slice(values),
|
||||
Self::ComplexI16(values) => values.iter().for_each(|value| {
|
||||
out.extend_from_slice(&value[0].to_le_bytes());
|
||||
out.extend_from_slice(&value[1].to_le_bytes());
|
||||
}),
|
||||
Self::ComplexF32(values) => values.iter().for_each(|value| {
|
||||
out.extend_from_slice(&value[0].to_le_bytes());
|
||||
out.extend_from_slice(&value[1].to_le_bytes());
|
||||
}),
|
||||
Self::PowerU16(values) => values
|
||||
.iter()
|
||||
.for_each(|value| out.extend_from_slice(&value.to_le_bytes())),
|
||||
Self::PowerF32(values) => values
|
||||
.iter()
|
||||
.for_each(|value| out.extend_from_slice(&value.to_le_bytes())),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct RadarFrame {
|
||||
pub report_type: ReportType,
|
||||
pub sequence: u32,
|
||||
pub timestamp_us: u64,
|
||||
pub device_id: u64,
|
||||
pub center_freq_khz: u32,
|
||||
pub bandwidth_mhz: u16,
|
||||
pub flags: RadarFlags,
|
||||
pub antenna_count: u8,
|
||||
pub scale: f32,
|
||||
pub bin_spacing: f32,
|
||||
pub calibration_id: u32,
|
||||
pub payload: RadarPayload,
|
||||
}
|
||||
|
||||
impl RadarFrame {
|
||||
pub fn to_bytes(&self) -> Result<Vec<u8>, RadarParseError> {
|
||||
self.validate()?;
|
||||
let payload_len = self.payload.encoded_len();
|
||||
let frame_len = RTL8720F_RADAR_HEADER_LEN
|
||||
.checked_add(payload_len)
|
||||
.and_then(|value| value.checked_add(RTL8720F_RADAR_CRC_LEN))
|
||||
.ok_or(RadarParseError::LengthOverflow)?;
|
||||
if frame_len > RTL8720F_RADAR_MAX_FRAME_LEN {
|
||||
return Err(RadarParseError::FrameTooLarge(frame_len));
|
||||
}
|
||||
|
||||
let mut out = Vec::with_capacity(frame_len);
|
||||
out.extend_from_slice(&RTL8720F_RADAR_MAGIC.to_le_bytes());
|
||||
out.push(RTL8720F_RADAR_VERSION);
|
||||
out.push(self.report_type as u8);
|
||||
out.extend_from_slice(&(RTL8720F_RADAR_HEADER_LEN as u16).to_le_bytes());
|
||||
out.extend_from_slice(&(frame_len as u32).to_le_bytes());
|
||||
out.extend_from_slice(&self.sequence.to_le_bytes());
|
||||
out.extend_from_slice(&self.timestamp_us.to_le_bytes());
|
||||
out.extend_from_slice(&self.device_id.to_le_bytes());
|
||||
out.extend_from_slice(&self.center_freq_khz.to_le_bytes());
|
||||
out.extend_from_slice(&self.bandwidth_mhz.to_le_bytes());
|
||||
out.extend_from_slice(&self.flags.0.to_le_bytes());
|
||||
out.extend_from_slice(&(self.payload.len() as u16).to_le_bytes());
|
||||
out.push(self.payload.format() as u8);
|
||||
out.push(self.antenna_count);
|
||||
out.extend_from_slice(&self.scale.to_le_bytes());
|
||||
out.extend_from_slice(&self.bin_spacing.to_le_bytes());
|
||||
out.extend_from_slice(&self.calibration_id.to_le_bytes());
|
||||
debug_assert_eq!(out.len(), RTL8720F_RADAR_HEADER_LEN);
|
||||
self.payload.encode_into(&mut out);
|
||||
let crc = crc32_ieee(&out);
|
||||
out.extend_from_slice(&crc.to_le_bytes());
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
pub fn from_bytes(input: &[u8]) -> Result<(Self, usize), RadarParseError> {
|
||||
if input.len() < RTL8720F_RADAR_HEADER_LEN {
|
||||
return Err(RadarParseError::InsufficientData {
|
||||
needed: RTL8720F_RADAR_HEADER_LEN,
|
||||
got: input.len(),
|
||||
});
|
||||
}
|
||||
let magic = read_u32(input, 0);
|
||||
if magic != RTL8720F_RADAR_MAGIC {
|
||||
return Err(RadarParseError::InvalidMagic(magic));
|
||||
}
|
||||
if input[4] != RTL8720F_RADAR_VERSION {
|
||||
return Err(RadarParseError::UnsupportedVersion(input[4]));
|
||||
}
|
||||
let report_type = ReportType::try_from(input[5])?;
|
||||
let header_len = read_u16(input, 6) as usize;
|
||||
if header_len < RTL8720F_RADAR_HEADER_LEN {
|
||||
return Err(RadarParseError::InvalidHeaderLength(header_len));
|
||||
}
|
||||
let frame_len = read_u32(input, 8) as usize;
|
||||
if frame_len > RTL8720F_RADAR_MAX_FRAME_LEN {
|
||||
return Err(RadarParseError::FrameTooLarge(frame_len));
|
||||
}
|
||||
if frame_len < header_len + RTL8720F_RADAR_CRC_LEN {
|
||||
return Err(RadarParseError::InvalidFrameLength(frame_len));
|
||||
}
|
||||
if input.len() < frame_len {
|
||||
return Err(RadarParseError::InsufficientData {
|
||||
needed: frame_len,
|
||||
got: input.len(),
|
||||
});
|
||||
}
|
||||
|
||||
let element_count = read_u16(input, 40) as usize;
|
||||
if element_count > RTL8720F_RADAR_MAX_ELEMENTS {
|
||||
return Err(RadarParseError::TooManyElements(element_count));
|
||||
}
|
||||
let format = ElementFormat::try_from(input[42])?;
|
||||
validate_type_format(report_type, format)?;
|
||||
let payload_len = element_count
|
||||
.checked_mul(format.bytes_per_element())
|
||||
.ok_or(RadarParseError::LengthOverflow)?;
|
||||
let expected_len = header_len
|
||||
.checked_add(payload_len)
|
||||
.and_then(|value| value.checked_add(RTL8720F_RADAR_CRC_LEN))
|
||||
.ok_or(RadarParseError::LengthOverflow)?;
|
||||
if expected_len != frame_len {
|
||||
return Err(RadarParseError::PayloadLengthMismatch {
|
||||
expected: expected_len,
|
||||
got: frame_len,
|
||||
});
|
||||
}
|
||||
|
||||
let expected_crc = read_u32(input, frame_len - RTL8720F_RADAR_CRC_LEN);
|
||||
let actual_crc = crc32_ieee(&input[..frame_len - RTL8720F_RADAR_CRC_LEN]);
|
||||
if expected_crc != actual_crc {
|
||||
return Err(RadarParseError::CrcMismatch {
|
||||
expected: expected_crc,
|
||||
actual: actual_crc,
|
||||
});
|
||||
}
|
||||
|
||||
let scale = read_f32(input, 44);
|
||||
let bin_spacing = read_f32(input, 48);
|
||||
if !scale.is_finite() || !bin_spacing.is_finite() {
|
||||
return Err(RadarParseError::NonFiniteValue);
|
||||
}
|
||||
let payload = decode_payload(format, &input[header_len..header_len + payload_len])?;
|
||||
let frame = Self {
|
||||
report_type,
|
||||
sequence: read_u32(input, 12),
|
||||
timestamp_us: read_u64(input, 16),
|
||||
device_id: read_u64(input, 24),
|
||||
center_freq_khz: read_u32(input, 32),
|
||||
bandwidth_mhz: read_u16(input, 36),
|
||||
flags: RadarFlags(read_u16(input, 38)),
|
||||
antenna_count: input[43],
|
||||
scale,
|
||||
bin_spacing,
|
||||
calibration_id: read_u32(input, 52),
|
||||
payload,
|
||||
};
|
||||
frame.validate()?;
|
||||
Ok((frame, frame_len))
|
||||
}
|
||||
|
||||
fn validate(&self) -> Result<(), RadarParseError> {
|
||||
if !matches!(self.bandwidth_mhz, 20 | 40 | 70) {
|
||||
return Err(RadarParseError::InvalidBandwidth(self.bandwidth_mhz));
|
||||
}
|
||||
if self.antenna_count == 0 || self.antenna_count > 8 {
|
||||
return Err(RadarParseError::InvalidAntennaCount(self.antenna_count));
|
||||
}
|
||||
if self.payload.len() > RTL8720F_RADAR_MAX_ELEMENTS
|
||||
|| self.payload.len() > u16::MAX as usize
|
||||
{
|
||||
return Err(RadarParseError::TooManyElements(self.payload.len()));
|
||||
}
|
||||
if !self.scale.is_finite() || !self.bin_spacing.is_finite() {
|
||||
return Err(RadarParseError::NonFiniteValue);
|
||||
}
|
||||
validate_type_format(self.report_type, self.payload.format())?;
|
||||
self.payload.validate_finite()
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_type_format(
|
||||
report_type: ReportType,
|
||||
format: ElementFormat,
|
||||
) -> Result<(), RadarParseError> {
|
||||
let valid = match report_type {
|
||||
ReportType::Cfr => matches!(
|
||||
format,
|
||||
ElementFormat::ComplexI16 | ElementFormat::ComplexF32
|
||||
),
|
||||
ReportType::RangeNear | ReportType::RangeFar => {
|
||||
matches!(format, ElementFormat::PowerU16 | ElementFormat::PowerF32)
|
||||
}
|
||||
ReportType::Interference | ReportType::Capabilities => format == ElementFormat::Bytes,
|
||||
};
|
||||
if valid {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(RadarParseError::InvalidTypeFormat {
|
||||
report_type,
|
||||
format,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn decode_payload(format: ElementFormat, bytes: &[u8]) -> Result<RadarPayload, RadarParseError> {
|
||||
let payload = match format {
|
||||
ElementFormat::Bytes => RadarPayload::Bytes(bytes.to_vec()),
|
||||
ElementFormat::ComplexI16 => RadarPayload::ComplexI16(
|
||||
bytes
|
||||
.chunks_exact(4)
|
||||
.map(|c| [read_i16(c, 0), read_i16(c, 2)])
|
||||
.collect(),
|
||||
),
|
||||
ElementFormat::ComplexF32 => RadarPayload::ComplexF32(
|
||||
bytes
|
||||
.chunks_exact(8)
|
||||
.map(|c| [read_f32(c, 0), read_f32(c, 4)])
|
||||
.collect(),
|
||||
),
|
||||
ElementFormat::PowerU16 => {
|
||||
RadarPayload::PowerU16(bytes.chunks_exact(2).map(|c| read_u16(c, 0)).collect())
|
||||
}
|
||||
ElementFormat::PowerF32 => {
|
||||
RadarPayload::PowerF32(bytes.chunks_exact(4).map(|c| read_f32(c, 0)).collect())
|
||||
}
|
||||
};
|
||||
payload.validate_finite()?;
|
||||
Ok(payload)
|
||||
}
|
||||
|
||||
fn read_u16(buf: &[u8], offset: usize) -> u16 {
|
||||
u16::from_le_bytes([buf[offset], buf[offset + 1]])
|
||||
}
|
||||
fn read_i16(buf: &[u8], offset: usize) -> i16 {
|
||||
i16::from_le_bytes([buf[offset], buf[offset + 1]])
|
||||
}
|
||||
fn read_u32(buf: &[u8], offset: usize) -> u32 {
|
||||
u32::from_le_bytes(buf[offset..offset + 4].try_into().unwrap())
|
||||
}
|
||||
fn read_u64(buf: &[u8], offset: usize) -> u64 {
|
||||
u64::from_le_bytes(buf[offset..offset + 8].try_into().unwrap())
|
||||
}
|
||||
fn read_f32(buf: &[u8], offset: usize) -> f32 {
|
||||
f32::from_le_bytes(buf[offset..offset + 4].try_into().unwrap())
|
||||
}
|
||||
|
||||
#[derive(Debug, Error, PartialEq)]
|
||||
pub enum RadarParseError {
|
||||
#[error("insufficient data: need {needed} bytes, got {got}")]
|
||||
InsufficientData { needed: usize, got: usize },
|
||||
#[error("invalid RTL8720F radar magic {0:#010x}")]
|
||||
InvalidMagic(u32),
|
||||
#[error("unsupported RTL8720F radar protocol version {0}")]
|
||||
UnsupportedVersion(u8),
|
||||
#[error("unknown radar report type {0}")]
|
||||
UnknownReportType(u8),
|
||||
#[error("unknown radar element format {0}")]
|
||||
UnknownElementFormat(u8),
|
||||
#[error("invalid header length {0}")]
|
||||
InvalidHeaderLength(usize),
|
||||
#[error("invalid frame length {0}")]
|
||||
InvalidFrameLength(usize),
|
||||
#[error("frame is too large: {0} bytes")]
|
||||
FrameTooLarge(usize),
|
||||
#[error("element count exceeds limit: {0}")]
|
||||
TooManyElements(usize),
|
||||
#[error("length arithmetic overflow")]
|
||||
LengthOverflow,
|
||||
#[error("payload/frame length mismatch: expected {expected}, got {got}")]
|
||||
PayloadLengthMismatch { expected: usize, got: usize },
|
||||
#[error("CRC mismatch: encoded {expected:#010x}, computed {actual:#010x}")]
|
||||
CrcMismatch { expected: u32, actual: u32 },
|
||||
#[error("non-finite floating-point value")]
|
||||
NonFiniteValue,
|
||||
#[error("invalid bandwidth {0} MHz")]
|
||||
InvalidBandwidth(u16),
|
||||
#[error("invalid antenna count {0}")]
|
||||
InvalidAntennaCount(u8),
|
||||
#[error("report {report_type:?} cannot use element format {format:?}")]
|
||||
InvalidTypeFormat {
|
||||
report_type: ReportType,
|
||||
format: ElementFormat,
|
||||
},
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::simulator::{Rtl8720fSimulator, SimulatorConfig};
|
||||
use super::*;
|
||||
|
||||
fn cfr_frame() -> RadarFrame {
|
||||
RadarFrame {
|
||||
report_type: ReportType::Cfr,
|
||||
sequence: 42,
|
||||
timestamp_us: 123_456,
|
||||
device_id: 0x1122_3344_5566_7788,
|
||||
center_freq_khz: 2_442_000,
|
||||
bandwidth_mhz: 40,
|
||||
flags: RadarFlags(RadarFlags::CALIBRATED | RadarFlags::TIME_SYNCHRONIZED),
|
||||
antenna_count: 1,
|
||||
scale: 1.0 / 4096.0,
|
||||
bin_spacing: 312_500.0,
|
||||
calibration_id: 0xAABB_CCDD,
|
||||
payload: RadarPayload::ComplexI16(vec![[12, -7], [2048, -2048], [0, 1]]),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cfr_round_trip_and_stream_consumption() {
|
||||
let frame = cfr_frame();
|
||||
let mut wire = frame.to_bytes().unwrap();
|
||||
let encoded_len = wire.len();
|
||||
wire.extend_from_slice(&[9, 8, 7]);
|
||||
let (decoded, consumed) = RadarFrame::from_bytes(&wire).unwrap();
|
||||
assert_eq!(decoded, frame);
|
||||
assert_eq!(consumed, encoded_len);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_report_family_round_trips() {
|
||||
let payloads = [
|
||||
(
|
||||
ReportType::RangeNear,
|
||||
RadarPayload::PowerU16(vec![1, 2, u16::MAX]),
|
||||
),
|
||||
(
|
||||
ReportType::RangeFar,
|
||||
RadarPayload::PowerF32(vec![0.0, 1.5, 9.25]),
|
||||
),
|
||||
(
|
||||
ReportType::Interference,
|
||||
RadarPayload::Bytes(vec![1, 2, 0x34, 0x12]),
|
||||
),
|
||||
(
|
||||
ReportType::Capabilities,
|
||||
RadarPayload::Bytes(vec![2, 1, 40]),
|
||||
),
|
||||
];
|
||||
for (report_type, payload) in payloads {
|
||||
let mut frame = cfr_frame();
|
||||
frame.report_type = report_type;
|
||||
frame.payload = payload;
|
||||
let (decoded, _) = RadarFrame::from_bytes(&frame.to_bytes().unwrap()).unwrap();
|
||||
assert_eq!(decoded, frame);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn single_bit_corruption_is_detected() {
|
||||
let mut wire = cfr_frame().to_bytes().unwrap();
|
||||
wire[RTL8720F_RADAR_HEADER_LEN + 1] ^= 0x01;
|
||||
assert!(matches!(
|
||||
RadarFrame::from_bytes(&wire),
|
||||
Err(RadarParseError::CrcMismatch { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn truncation_is_reported_without_panicking() {
|
||||
let wire = cfr_frame().to_bytes().unwrap();
|
||||
for end in 0..wire.len() {
|
||||
assert!(RadarFrame::from_bytes(&wire[..end]).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn count_length_mismatch_fails_before_payload_decode() {
|
||||
let mut wire = cfr_frame().to_bytes().unwrap();
|
||||
wire[40..42].copy_from_slice(&100u16.to_le_bytes());
|
||||
let crc_offset = wire.len() - RTL8720F_RADAR_CRC_LEN;
|
||||
let crc = crc32_ieee(&wire[..crc_offset]);
|
||||
wire[crc_offset..].copy_from_slice(&crc.to_le_bytes());
|
||||
assert!(matches!(
|
||||
RadarFrame::from_bytes(&wire),
|
||||
Err(RadarParseError::PayloadLengthMismatch { .. })
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn invalid_semantic_combinations_are_rejected() {
|
||||
let mut frame = cfr_frame();
|
||||
frame.payload = RadarPayload::PowerU16(vec![1]);
|
||||
assert!(matches!(
|
||||
frame.to_bytes(),
|
||||
Err(RadarParseError::InvalidTypeFormat { .. })
|
||||
));
|
||||
frame.report_type = ReportType::RangeNear;
|
||||
frame.bandwidth_mhz = 80;
|
||||
assert_eq!(
|
||||
frame.to_bytes().unwrap_err(),
|
||||
RadarParseError::InvalidBandwidth(80)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn non_finite_values_are_rejected() {
|
||||
let mut frame = cfr_frame();
|
||||
frame.scale = f32::NAN;
|
||||
assert_eq!(
|
||||
frame.to_bytes().unwrap_err(),
|
||||
RadarParseError::NonFiniteValue
|
||||
);
|
||||
frame.scale = 1.0;
|
||||
frame.payload = RadarPayload::ComplexF32(vec![[f32::INFINITY, 0.0]]);
|
||||
assert_eq!(
|
||||
frame.to_bytes().unwrap_err(),
|
||||
RadarParseError::NonFiniteValue
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn arbitrary_short_inputs_never_panic() {
|
||||
let mut state = 0x1234_5678u32;
|
||||
for len in 0..256usize {
|
||||
let mut bytes = vec![0u8; len];
|
||||
for byte in &mut bytes {
|
||||
state = state.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
|
||||
*byte = (state >> 24) as u8;
|
||||
}
|
||||
let result = std::panic::catch_unwind(|| RadarFrame::from_bytes(&bytes));
|
||||
assert!(result.is_ok(), "parser panicked for {len} bytes");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn simulator_is_deterministic_and_uses_real_wire_boundary() {
|
||||
let mut a = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
let mut b = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
for _ in 0..12 {
|
||||
let a_wire = a.next_wire(ReportType::Cfr).unwrap();
|
||||
let b_wire = b.next_wire(ReportType::Cfr).unwrap();
|
||||
assert_eq!(a_wire, b_wire);
|
||||
let (decoded, consumed) = RadarFrame::from_bytes(&a_wire).unwrap();
|
||||
assert_eq!(consumed, a_wire.len());
|
||||
assert!(decoded.flags.contains(RadarFlags::SYNTHETIC));
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn simulator_range_peak_tracks_ground_truth() {
|
||||
let mut sim = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
let frame = sim.next_frame(ReportType::RangeFar);
|
||||
let RadarPayload::PowerF32(power) = frame.payload else {
|
||||
panic!("expected power bins")
|
||||
};
|
||||
let peak = power
|
||||
.iter()
|
||||
.enumerate()
|
||||
.max_by(|a, b| a.1.total_cmp(b.1))
|
||||
.unwrap()
|
||||
.0;
|
||||
let observed_m = peak as f32 * frame.bin_spacing;
|
||||
assert!((observed_m - sim.target_distance_m()).abs() <= frame.bin_spacing);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn simulator_capabilities_are_explicitly_synthetic() {
|
||||
let sim = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
let frame = sim.capabilities_frame();
|
||||
assert_eq!(frame.report_type, ReportType::Capabilities);
|
||||
assert!(frame.flags.contains(RadarFlags::SYNTHETIC));
|
||||
let wire = frame.to_bytes().unwrap();
|
||||
assert_eq!(RadarFrame::from_bytes(&wire).unwrap().0, frame);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
[package]
|
||||
name = "wifi-densepose-posecode"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
authors.workspace = true
|
||||
license.workspace = true
|
||||
repository.workspace = true
|
||||
description = "Multi-actor semantic motion scenes for RuView PoseTrack streams"
|
||||
documentation = "https://docs.rs/wifi-densepose-posecode"
|
||||
keywords = ["wifi", "pose", "motion", "posecode", "multi-person"]
|
||||
categories = ["science", "computer-vision", "parser-implementations"]
|
||||
|
||||
[dependencies]
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
thiserror.workspace = true
|
||||
wifi-densepose-signal = { version = "0.3.5", path = "../wifi-densepose-signal", default-features = false, optional = true }
|
||||
|
||||
[dev-dependencies]
|
||||
criterion.workspace = true
|
||||
|
||||
[features]
|
||||
default = []
|
||||
ruview = ["dep:wifi-densepose-signal"]
|
||||
|
||||
[[bench]]
|
||||
name = "posecode_bench"
|
||||
harness = false
|
||||
@@ -0,0 +1,37 @@
|
||||
use criterion::{black_box, criterion_group, criterion_main, Criterion};
|
||||
use wifi_densepose_posecode::{parse_posecode, to_posecode};
|
||||
|
||||
const TWO_ACTORS: &str = r#"posecode scene "crossing"
|
||||
source observed_wifi_csi
|
||||
actor person_1:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
track 1
|
||||
confidence 0.8
|
||||
position 0 0 0
|
||||
actor person_2:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
track 2
|
||||
confidence 0.8
|
||||
position 1 0 0
|
||||
step "Move" 0.05s linear:
|
||||
person_1.knee_left: flex 30 0.8
|
||||
person_1.hip_left: flex 20 0.8
|
||||
person_2.knee_right: flex 35 0.8
|
||||
person_2.hip_right: flex 25 0.8
|
||||
repeat 1
|
||||
"#;
|
||||
|
||||
fn benchmarks(c: &mut Criterion) {
|
||||
c.bench_function("parse_two_actor_scene", |b| {
|
||||
b.iter(|| parse_posecode(black_box(TWO_ACTORS)).unwrap())
|
||||
});
|
||||
let scene = parse_posecode(TWO_ACTORS).unwrap();
|
||||
c.bench_function("serialize_two_actor_scene", |b| {
|
||||
b.iter(|| to_posecode(black_box(&scene)))
|
||||
});
|
||||
}
|
||||
|
||||
criterion_group!(benches, benchmarks);
|
||||
criterion_main!(benches);
|
||||
@@ -0,0 +1,443 @@
|
||||
//! Conversion from tracked COCO-17 skeletons to semantic joint observations.
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::model::{ActorObservation, JointTarget, SceneFrame, Vec3};
|
||||
|
||||
const NUM_KEYPOINTS: usize = 17;
|
||||
const LEFT_SHOULDER: usize = 5;
|
||||
const RIGHT_SHOULDER: usize = 6;
|
||||
const LEFT_ELBOW: usize = 7;
|
||||
const RIGHT_ELBOW: usize = 8;
|
||||
const LEFT_WRIST: usize = 9;
|
||||
const RIGHT_WRIST: usize = 10;
|
||||
const LEFT_HIP: usize = 11;
|
||||
const RIGHT_HIP: usize = 12;
|
||||
const LEFT_KNEE: usize = 13;
|
||||
const RIGHT_KNEE: usize = 14;
|
||||
const LEFT_ANKLE: usize = 15;
|
||||
const RIGHT_ANKLE: usize = 16;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum TrackState {
|
||||
Tentative,
|
||||
Active,
|
||||
Lost,
|
||||
Terminated,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq)]
|
||||
pub struct KeypointObservation {
|
||||
pub position: Vec3,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct TrackObservation {
|
||||
pub id: u64,
|
||||
pub state: TrackState,
|
||||
pub keypoints: [KeypointObservation; NUM_KEYPOINTS],
|
||||
pub time_since_update: u64,
|
||||
}
|
||||
|
||||
#[cfg(feature = "ruview")]
|
||||
impl From<&wifi_densepose_signal::ruvsense::PoseTrack> for TrackObservation {
|
||||
fn from(track: &wifi_densepose_signal::ruvsense::PoseTrack) -> Self {
|
||||
use wifi_densepose_signal::ruvsense::TrackLifecycleState;
|
||||
Self {
|
||||
id: track.id.0,
|
||||
state: match track.lifecycle {
|
||||
TrackLifecycleState::Tentative => TrackState::Tentative,
|
||||
TrackLifecycleState::Active => TrackState::Active,
|
||||
TrackLifecycleState::Lost => TrackState::Lost,
|
||||
TrackLifecycleState::Terminated => TrackState::Terminated,
|
||||
},
|
||||
keypoints: std::array::from_fn(|i| {
|
||||
let point = track.keypoints[i].position();
|
||||
KeypointObservation {
|
||||
position: Vec3::new(point[0], point[1], point[2]),
|
||||
confidence: track.keypoints[i].confidence,
|
||||
}
|
||||
}),
|
||||
time_since_update: track.time_since_update,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct AdapterConfig {
|
||||
/// Coordinate system up vector after room calibration.
|
||||
pub world_up: Vec3,
|
||||
/// Coordinate system forward vector after room calibration.
|
||||
pub world_forward: Vec3,
|
||||
/// Ignore tracks below this aggregate confidence.
|
||||
pub min_confidence: f32,
|
||||
/// Confidence used until an inference backend populates keypoint confidence.
|
||||
pub unscored_confidence: f32,
|
||||
/// Emit lost tracks as predicted observations.
|
||||
pub include_lost: bool,
|
||||
}
|
||||
|
||||
impl Default for AdapterConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
world_up: Vec3::new(0.0, 1.0, 0.0),
|
||||
world_forward: Vec3::new(0.0, 0.0, 1.0),
|
||||
min_confidence: 0.2,
|
||||
unscored_confidence: 0.5,
|
||||
include_lost: false,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct RuViewAdapter {
|
||||
config: AdapterConfig,
|
||||
}
|
||||
|
||||
impl RuViewAdapter {
|
||||
pub fn new(config: AdapterConfig) -> Result<Self> {
|
||||
if !config.world_up.is_finite() || !config.world_forward.is_finite() {
|
||||
return Err(Error::Observation("coordinate axes must be finite".into()));
|
||||
}
|
||||
if !(0.0..=1.0).contains(&config.min_confidence)
|
||||
|| !(0.0..=1.0).contains(&config.unscored_confidence)
|
||||
{
|
||||
return Err(Error::Observation("confidence must be in [0, 1]".into()));
|
||||
}
|
||||
if norm(config.world_up) < 1e-6 || norm(config.world_forward) < 1e-6 {
|
||||
return Err(Error::Observation(
|
||||
"coordinate axes must be non-zero".into(),
|
||||
));
|
||||
}
|
||||
let alignment = dot(normalize(config.world_up), normalize(config.world_forward)).abs();
|
||||
if alignment > 0.98 {
|
||||
return Err(Error::Observation(
|
||||
"world up and forward axes must not be parallel".into(),
|
||||
));
|
||||
}
|
||||
Ok(Self { config })
|
||||
}
|
||||
|
||||
/// Convert all visible tracks into one synchronized observation frame.
|
||||
pub fn frame(&self, timestamp_ms: u64, tracks: &[TrackObservation]) -> SceneFrame {
|
||||
let actors = tracks
|
||||
.iter()
|
||||
.filter_map(|track| self.actor(track))
|
||||
.collect();
|
||||
SceneFrame {
|
||||
timestamp_ms,
|
||||
actors,
|
||||
}
|
||||
}
|
||||
|
||||
/// Direct bridge for the RuView tracker aggregate. The feature is optional
|
||||
/// so protocol-only consumers do not pull the signal and RuVector graph.
|
||||
#[cfg(feature = "ruview")]
|
||||
pub fn frame_from_pose_tracks(
|
||||
&self,
|
||||
timestamp_ms: u64,
|
||||
tracks: &[&wifi_densepose_signal::ruvsense::PoseTrack],
|
||||
) -> SceneFrame {
|
||||
let observations: Vec<TrackObservation> =
|
||||
tracks.iter().map(|track| (*track).into()).collect();
|
||||
self.frame(timestamp_ms, &observations)
|
||||
}
|
||||
|
||||
pub fn actor(&self, track: &TrackObservation) -> Option<ActorObservation> {
|
||||
if track.state == TrackState::Terminated
|
||||
|| (!self.config.include_lost && track.state == TrackState::Lost)
|
||||
{
|
||||
return None;
|
||||
}
|
||||
|
||||
let confidence = track_confidence(track, self.config.unscored_confidence);
|
||||
if confidence < self.config.min_confidence {
|
||||
return None;
|
||||
}
|
||||
|
||||
let p = |index: usize| track.keypoints[index].position;
|
||||
if track.keypoints.iter().any(|kp| !kp.position.is_finite()) {
|
||||
return None;
|
||||
}
|
||||
|
||||
let left_hip = p(LEFT_HIP);
|
||||
let right_hip = p(RIGHT_HIP);
|
||||
let position = midpoint(left_hip, right_hip);
|
||||
let up = normalize(self.config.world_up);
|
||||
let forward = normalize(sub(
|
||||
self.config.world_forward,
|
||||
scale(up, dot(self.config.world_forward, up)),
|
||||
));
|
||||
|
||||
let mut joints = Vec::with_capacity(10);
|
||||
add_flexion(
|
||||
&mut joints,
|
||||
"elbow_left",
|
||||
p(LEFT_SHOULDER),
|
||||
p(LEFT_ELBOW),
|
||||
p(LEFT_WRIST),
|
||||
joint_confidence(track, &[LEFT_SHOULDER, LEFT_ELBOW, LEFT_WRIST], confidence),
|
||||
);
|
||||
add_flexion(
|
||||
&mut joints,
|
||||
"elbow_right",
|
||||
p(RIGHT_SHOULDER),
|
||||
p(RIGHT_ELBOW),
|
||||
p(RIGHT_WRIST),
|
||||
joint_confidence(
|
||||
track,
|
||||
&[RIGHT_SHOULDER, RIGHT_ELBOW, RIGHT_WRIST],
|
||||
confidence,
|
||||
),
|
||||
);
|
||||
add_flexion(
|
||||
&mut joints,
|
||||
"knee_left",
|
||||
left_hip,
|
||||
p(LEFT_KNEE),
|
||||
p(LEFT_ANKLE),
|
||||
joint_confidence(track, &[LEFT_HIP, LEFT_KNEE, LEFT_ANKLE], confidence),
|
||||
);
|
||||
add_flexion(
|
||||
&mut joints,
|
||||
"knee_right",
|
||||
right_hip,
|
||||
p(RIGHT_KNEE),
|
||||
p(RIGHT_ANKLE),
|
||||
joint_confidence(track, &[RIGHT_HIP, RIGHT_KNEE, RIGHT_ANKLE], confidence),
|
||||
);
|
||||
|
||||
add_signed_flex(
|
||||
&mut joints,
|
||||
"hip_left",
|
||||
sub(p(LEFT_KNEE), left_hip),
|
||||
up,
|
||||
forward,
|
||||
joint_confidence(track, &[LEFT_HIP, LEFT_KNEE], confidence),
|
||||
);
|
||||
add_signed_flex(
|
||||
&mut joints,
|
||||
"hip_right",
|
||||
sub(p(RIGHT_KNEE), right_hip),
|
||||
up,
|
||||
forward,
|
||||
joint_confidence(track, &[RIGHT_HIP, RIGHT_KNEE], confidence),
|
||||
);
|
||||
add_signed_flex(
|
||||
&mut joints,
|
||||
"shoulder_left",
|
||||
sub(p(LEFT_ELBOW), p(LEFT_SHOULDER)),
|
||||
up,
|
||||
forward,
|
||||
joint_confidence(track, &[LEFT_SHOULDER, LEFT_ELBOW], confidence),
|
||||
);
|
||||
add_signed_flex(
|
||||
&mut joints,
|
||||
"shoulder_right",
|
||||
sub(p(RIGHT_ELBOW), p(RIGHT_SHOULDER)),
|
||||
up,
|
||||
forward,
|
||||
joint_confidence(track, &[RIGHT_SHOULDER, RIGHT_ELBOW], confidence),
|
||||
);
|
||||
|
||||
let shoulder_mid = midpoint(p(LEFT_SHOULDER), p(RIGHT_SHOULDER));
|
||||
let torso = sub(shoulder_mid, position);
|
||||
let hinge = angle_deg(torso, up).clamp(0.0, 180.0);
|
||||
joints.push(JointTarget {
|
||||
joint: "pelvis".into(),
|
||||
action: "hinge".into(),
|
||||
degrees: hinge,
|
||||
confidence: joint_confidence(
|
||||
track,
|
||||
&[LEFT_SHOULDER, RIGHT_SHOULDER, LEFT_HIP, RIGHT_HIP],
|
||||
confidence,
|
||||
),
|
||||
});
|
||||
|
||||
Some(ActorObservation {
|
||||
actor_id: format!("person_{}", track.id),
|
||||
track_id: track.id,
|
||||
confidence,
|
||||
position,
|
||||
joints,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
fn track_confidence(track: &TrackObservation, fallback: f32) -> f32 {
|
||||
let scored: Vec<f32> = track
|
||||
.keypoints
|
||||
.iter()
|
||||
.map(|kp| kp.confidence)
|
||||
.filter(|v| v.is_finite() && *v > 0.0)
|
||||
.collect();
|
||||
let measured = if scored.is_empty() {
|
||||
fallback
|
||||
} else {
|
||||
scored.iter().sum::<f32>() / scored.len() as f32
|
||||
};
|
||||
let freshness = 1.0 / (1.0 + track.time_since_update as f32 * 0.2);
|
||||
(measured * freshness).clamp(0.0, 1.0)
|
||||
}
|
||||
|
||||
fn joint_confidence(track: &TrackObservation, indices: &[usize], fallback: f32) -> f32 {
|
||||
indices.iter().fold(1.0_f32, |confidence, &index| {
|
||||
let measured = track.keypoints[index].confidence;
|
||||
confidence.min(if measured.is_finite() && measured > 0.0 {
|
||||
measured
|
||||
} else {
|
||||
fallback
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
fn add_flexion(out: &mut Vec<JointTarget>, name: &str, a: Vec3, b: Vec3, c: Vec3, confidence: f32) {
|
||||
let inner = angle_deg(sub(a, b), sub(c, b));
|
||||
if inner.is_finite() {
|
||||
out.push(JointTarget {
|
||||
joint: name.into(),
|
||||
action: "flex".into(),
|
||||
degrees: (180.0 - inner).clamp(0.0, 180.0),
|
||||
confidence,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn add_signed_flex(
|
||||
out: &mut Vec<JointTarget>,
|
||||
name: &str,
|
||||
limb: Vec3,
|
||||
up: Vec3,
|
||||
forward: Vec3,
|
||||
confidence: f32,
|
||||
) {
|
||||
let down = scale(up, -1.0);
|
||||
let degrees = dot(limb, forward).atan2(dot(limb, down)).to_degrees();
|
||||
if degrees.is_finite() {
|
||||
out.push(JointTarget {
|
||||
joint: name.into(),
|
||||
action: if degrees >= 0.0 { "flex" } else { "extend" }.into(),
|
||||
degrees: degrees.abs().clamp(0.0, 180.0),
|
||||
confidence,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn midpoint(a: Vec3, b: Vec3) -> Vec3 {
|
||||
scale(add(a, b), 0.5)
|
||||
}
|
||||
fn add(a: Vec3, b: Vec3) -> Vec3 {
|
||||
Vec3::new(a.x + b.x, a.y + b.y, a.z + b.z)
|
||||
}
|
||||
fn sub(a: Vec3, b: Vec3) -> Vec3 {
|
||||
Vec3::new(a.x - b.x, a.y - b.y, a.z - b.z)
|
||||
}
|
||||
fn scale(v: Vec3, s: f32) -> Vec3 {
|
||||
Vec3::new(v.x * s, v.y * s, v.z * s)
|
||||
}
|
||||
fn dot(a: Vec3, b: Vec3) -> f32 {
|
||||
a.x * b.x + a.y * b.y + a.z * b.z
|
||||
}
|
||||
fn norm(v: Vec3) -> f32 {
|
||||
dot(v, v).sqrt()
|
||||
}
|
||||
fn normalize(v: Vec3) -> Vec3 {
|
||||
scale(v, 1.0 / norm(v).max(1e-9))
|
||||
}
|
||||
fn angle_deg(a: Vec3, b: Vec3) -> f32 {
|
||||
let denom = norm(a) * norm(b);
|
||||
if denom < 1e-9 {
|
||||
return 0.0;
|
||||
}
|
||||
(dot(a, b) / denom).clamp(-1.0, 1.0).acos().to_degrees()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn standing_track() -> TrackObservation {
|
||||
let mut keypoints = [KeypointObservation::default(); NUM_KEYPOINTS];
|
||||
let mut set = |index, x, y, z| {
|
||||
keypoints[index] = KeypointObservation {
|
||||
position: Vec3::new(x, y, z),
|
||||
confidence: 0.9,
|
||||
};
|
||||
};
|
||||
set(LEFT_SHOULDER, -0.2, 1.5, 0.0);
|
||||
set(RIGHT_SHOULDER, 0.2, 1.5, 0.0);
|
||||
set(LEFT_ELBOW, -0.25, 1.1, 0.0);
|
||||
set(RIGHT_ELBOW, 0.25, 1.1, 0.0);
|
||||
set(LEFT_WRIST, -0.25, 0.75, 0.0);
|
||||
set(RIGHT_WRIST, 0.25, 0.75, 0.0);
|
||||
set(LEFT_HIP, -0.12, 0.9, 0.0);
|
||||
set(RIGHT_HIP, 0.12, 0.9, 0.0);
|
||||
set(LEFT_KNEE, -0.12, 0.5, 0.0);
|
||||
set(RIGHT_KNEE, 0.12, 0.5, 0.0);
|
||||
set(LEFT_ANKLE, -0.12, 0.05, 0.0);
|
||||
set(RIGHT_ANKLE, 0.12, 0.05, 0.0);
|
||||
TrackObservation {
|
||||
id: 7,
|
||||
state: TrackState::Active,
|
||||
keypoints,
|
||||
time_since_update: 0,
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn converts_track_to_named_actor() {
|
||||
let adapter = RuViewAdapter::new(AdapterConfig::default()).unwrap();
|
||||
let actor = adapter.actor(&standing_track()).unwrap();
|
||||
assert_eq!(actor.actor_id, "person_7");
|
||||
assert_eq!(actor.track_id, 7);
|
||||
assert!(actor.joints.len() >= 9);
|
||||
assert!(actor.joints.iter().all(|j| j.degrees.is_finite()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_invalid_axes() {
|
||||
let config = AdapterConfig {
|
||||
world_up: Vec3::default(),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(RuViewAdapter::new(config).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_parallel_coordinate_axes() {
|
||||
let config = AdapterConfig {
|
||||
world_forward: Vec3::new(0.0, 2.0, 0.0),
|
||||
..Default::default()
|
||||
};
|
||||
assert!(RuViewAdapter::new(config).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn filters_stale_low_confidence_tracks() {
|
||||
let mut track = standing_track();
|
||||
track.time_since_update = 100;
|
||||
let adapter = RuViewAdapter::new(AdapterConfig::default()).unwrap();
|
||||
assert!(adapter.actor(&track).is_none());
|
||||
}
|
||||
|
||||
#[cfg(feature = "ruview")]
|
||||
#[test]
|
||||
fn converts_native_ruview_pose_track() {
|
||||
use wifi_densepose_signal::ruvsense::{PoseTrack, TrackId, TrackLifecycleState};
|
||||
|
||||
let source = standing_track();
|
||||
let positions = std::array::from_fn(|i| {
|
||||
let p = source.keypoints[i].position;
|
||||
[p.x, p.y, p.z]
|
||||
});
|
||||
let mut native = PoseTrack::new(TrackId(23), &positions, 0, 128);
|
||||
native.lifecycle = TrackLifecycleState::Active;
|
||||
let observed = TrackObservation::from(&native);
|
||||
assert_eq!(observed.id, 23);
|
||||
assert_eq!(observed.state, TrackState::Active);
|
||||
assert_eq!(
|
||||
observed.keypoints[LEFT_HIP].position,
|
||||
source.keypoints[LEFT_HIP].position
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
//! Error types for parser and adapter boundaries.
|
||||
|
||||
/// Crate result alias.
|
||||
pub type Result<T> = std::result::Result<T, Error>;
|
||||
|
||||
/// Errors are structured and line anchored where input text is involved.
|
||||
#[derive(Debug, thiserror::Error, PartialEq)]
|
||||
pub enum Error {
|
||||
#[error("line {line}: {message}")]
|
||||
Parse { line: usize, message: String },
|
||||
#[error("invalid scene: {0}")]
|
||||
Validation(String),
|
||||
#[error("invalid observation: {0}")]
|
||||
Observation(String),
|
||||
}
|
||||
|
||||
impl Error {
|
||||
pub(crate) fn parse(line: usize, message: impl Into<String>) -> Self {
|
||||
Self::Parse {
|
||||
line,
|
||||
message: message.into(),
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
//! Multi-actor semantic motion scenes for RuView.
|
||||
//!
|
||||
//! This crate uses the PoseCode movement vocabulary and implements the
|
||||
//! RuView 0.2 multi-actor extension from ADR-266. It deliberately separates
|
||||
//! the high-rate observed pose stream from the compact semantic scene:
|
||||
//! [`adapter::RuViewAdapter`] converts persistent RuView tracks into observed
|
||||
//! frames, [`segmenter::PhaseSegmenter`] reduces those frames to synchronized
|
||||
//! phases, and [`serialize::to_posecode`] emits deterministic text.
|
||||
|
||||
pub mod adapter;
|
||||
pub mod error;
|
||||
pub mod model;
|
||||
pub mod parser;
|
||||
pub mod segmenter;
|
||||
pub mod serialize;
|
||||
pub mod validate;
|
||||
|
||||
pub use adapter::{
|
||||
AdapterConfig, KeypointObservation, RuViewAdapter, TrackObservation, TrackState,
|
||||
};
|
||||
pub use error::{Error, Result};
|
||||
pub use model::*;
|
||||
pub use parser::parse_posecode;
|
||||
pub use segmenter::{PhaseSegmenter, SegmenterConfig};
|
||||
pub use serialize::to_posecode;
|
||||
pub use validate::{validate_scene, ValidationConfig, ValidationIssue, ValidationSeverity};
|
||||
|
||||
/// Protocol version implemented by this crate.
|
||||
pub const PROTOCOL_VERSION: &str = "0.2";
|
||||
@@ -0,0 +1,203 @@
|
||||
//! Renderer independent scene model.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
/// Hard limits keep untrusted scene documents bounded in memory and time.
|
||||
pub const MAX_ACTORS: usize = 32;
|
||||
pub const MAX_PHASES: usize = 10_000;
|
||||
pub const MAX_TARGETS_PER_ACTOR: usize = 64;
|
||||
pub const MAX_NAME_BYTES: usize = 128;
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Vec3 {
|
||||
pub x: f32,
|
||||
pub y: f32,
|
||||
pub z: f32,
|
||||
}
|
||||
|
||||
impl Vec3 {
|
||||
pub fn new(x: f32, y: f32, z: f32) -> Self {
|
||||
Self { x, y, z }
|
||||
}
|
||||
|
||||
pub fn is_finite(self) -> bool {
|
||||
self.x.is_finite() && self.y.is_finite() && self.z.is_finite()
|
||||
}
|
||||
|
||||
pub fn distance(self, other: Self) -> f32 {
|
||||
let dx = self.x - other.x;
|
||||
let dy = self.y - other.y;
|
||||
let dz = self.z - other.z;
|
||||
(dx * dx + dy * dy + dz * dz).sqrt()
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Serialize, Deserialize)]
|
||||
pub struct EulerDeg {
|
||||
pub x: f32,
|
||||
pub y: f32,
|
||||
pub z: f32,
|
||||
}
|
||||
|
||||
impl EulerDeg {
|
||||
pub fn is_finite(self) -> bool {
|
||||
self.x.is_finite() && self.y.is_finite() && self.z.is_finite()
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "snake_case")]
|
||||
pub enum SceneSource {
|
||||
Authored,
|
||||
ObservedWifiCsi,
|
||||
Imported,
|
||||
}
|
||||
|
||||
impl Default for SceneSource {
|
||||
fn default() -> Self {
|
||||
Self::Authored
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Actor {
|
||||
pub id: String,
|
||||
pub rig: String,
|
||||
pub start_pose: String,
|
||||
pub track_id: Option<u64>,
|
||||
pub confidence: f32,
|
||||
pub position: Vec3,
|
||||
}
|
||||
|
||||
impl Actor {
|
||||
pub fn humanoid(id: impl Into<String>) -> Self {
|
||||
Self {
|
||||
id: id.into(),
|
||||
rig: "humanoid".into(),
|
||||
start_pose: "standing".into(),
|
||||
track_id: None,
|
||||
confidence: 1.0,
|
||||
position: Vec3::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct JointTarget {
|
||||
pub joint: String,
|
||||
pub action: String,
|
||||
pub degrees: f32,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct ActorTargets {
|
||||
pub actor_id: String,
|
||||
pub joints: Vec<JointTarget>,
|
||||
pub ground_lock: Vec<String>,
|
||||
pub travel: Option<Vec3>,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
impl ActorTargets {
|
||||
pub fn new(actor_id: impl Into<String>) -> Self {
|
||||
Self {
|
||||
actor_id: actor_id.into(),
|
||||
joints: Vec::new(),
|
||||
ground_lock: Vec::new(),
|
||||
travel: None,
|
||||
confidence: 1.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct EffectorRef {
|
||||
pub actor_id: String,
|
||||
pub effector: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Contact {
|
||||
pub from: EffectorRef,
|
||||
pub to: EffectorRef,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "kebab-case")]
|
||||
pub enum Timing {
|
||||
Flow,
|
||||
Settle,
|
||||
Drive,
|
||||
Snap,
|
||||
Linear,
|
||||
EaseIn,
|
||||
EaseOut,
|
||||
EaseInOut,
|
||||
}
|
||||
|
||||
impl Timing {
|
||||
pub fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
Self::Flow => "flow",
|
||||
Self::Settle => "settle",
|
||||
Self::Drive => "drive",
|
||||
Self::Snap => "snap",
|
||||
Self::Linear => "linear",
|
||||
Self::EaseIn => "ease-in",
|
||||
Self::EaseOut => "ease-out",
|
||||
Self::EaseInOut => "ease-in-out",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Phase {
|
||||
pub name: String,
|
||||
pub start_ms: u64,
|
||||
pub duration_ms: u64,
|
||||
pub timing: Timing,
|
||||
pub actors: Vec<ActorTargets>,
|
||||
pub contacts: Vec<Contact>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct Scene {
|
||||
pub version: String,
|
||||
pub name: String,
|
||||
pub source: SceneSource,
|
||||
pub actors: Vec<Actor>,
|
||||
pub phases: Vec<Phase>,
|
||||
pub repeat: u32,
|
||||
}
|
||||
|
||||
impl Scene {
|
||||
pub fn new(name: impl Into<String>) -> Self {
|
||||
Self {
|
||||
version: crate::PROTOCOL_VERSION.into(),
|
||||
name: name.into(),
|
||||
source: SceneSource::Authored,
|
||||
actors: Vec::new(),
|
||||
phases: Vec::new(),
|
||||
repeat: 1,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// One actor at one observed instant. Raw observations remain separate from
|
||||
/// semantic phases so 20 Hz input does not become unreadable scene text.
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct ActorObservation {
|
||||
pub actor_id: String,
|
||||
pub track_id: u64,
|
||||
pub confidence: f32,
|
||||
pub position: Vec3,
|
||||
pub joints: Vec<JointTarget>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
|
||||
pub struct SceneFrame {
|
||||
pub timestamp_ms: u64,
|
||||
pub actors: Vec<ActorObservation>,
|
||||
}
|
||||
@@ -0,0 +1,375 @@
|
||||
//! Bounded line parser for the PoseCode 0.2 multi-actor extension.
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::model::*;
|
||||
use crate::validate::{validate_scene, ValidationConfig, ValidationSeverity};
|
||||
|
||||
const MAX_INPUT_BYTES: usize = 1_048_576;
|
||||
const MAX_LINE_BYTES: usize = 4096;
|
||||
|
||||
pub fn parse_posecode(source: &str) -> Result<Scene> {
|
||||
if source.len() > MAX_INPUT_BYTES {
|
||||
return Err(Error::parse(0, "document exceeds 1 MiB"));
|
||||
}
|
||||
let mut scene: Option<Scene> = None;
|
||||
let mut current_actor: Option<usize> = None;
|
||||
let mut current_phase: Option<usize> = None;
|
||||
let mut elapsed_ms = 0_u64;
|
||||
|
||||
for (index, raw) in source.lines().enumerate() {
|
||||
let line_no = index + 1;
|
||||
if raw.len() > MAX_LINE_BYTES {
|
||||
return Err(Error::parse(line_no, "line exceeds 4096 bytes"));
|
||||
}
|
||||
let line = strip_comment(raw).trim();
|
||||
if line.is_empty() {
|
||||
continue;
|
||||
}
|
||||
|
||||
if scene.is_none() {
|
||||
let rest = line
|
||||
.strip_prefix("posecode scene ")
|
||||
.ok_or_else(|| Error::parse(line_no, "expected posecode scene header"))?;
|
||||
scene = Some(Scene::new(parse_quoted(rest, line_no)?));
|
||||
continue;
|
||||
}
|
||||
let doc = scene.as_mut().expect("initialized above");
|
||||
|
||||
if let Some(value) = line.strip_prefix("source ") {
|
||||
doc.source = match value {
|
||||
"authored" => SceneSource::Authored,
|
||||
"observed_wifi_csi" => SceneSource::ObservedWifiCsi,
|
||||
"imported" => SceneSource::Imported,
|
||||
_ => return Err(Error::parse(line_no, "unknown source")),
|
||||
};
|
||||
continue;
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("actor ") {
|
||||
let id = value
|
||||
.strip_suffix(':')
|
||||
.ok_or_else(|| Error::parse(line_no, "actor declaration must end with ':'"))?;
|
||||
doc.actors.push(Actor::humanoid(id.trim()));
|
||||
current_actor = Some(doc.actors.len() - 1);
|
||||
current_phase = None;
|
||||
continue;
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("step ") {
|
||||
let (name, tail) = take_quoted(value, line_no)?;
|
||||
let fields: Vec<&str> = tail.trim_end_matches(':').split_whitespace().collect();
|
||||
if fields.len() != 2 {
|
||||
return Err(Error::parse(line_no, "step requires duration and timing"));
|
||||
}
|
||||
let duration_ms = parse_duration(fields[0], line_no)?;
|
||||
let timing = parse_timing(fields[1], line_no)?;
|
||||
doc.phases.push(Phase {
|
||||
name,
|
||||
start_ms: elapsed_ms,
|
||||
duration_ms,
|
||||
timing,
|
||||
actors: Vec::new(),
|
||||
contacts: Vec::new(),
|
||||
});
|
||||
elapsed_ms = elapsed_ms
|
||||
.checked_add(duration_ms)
|
||||
.ok_or_else(|| Error::parse(line_no, "timeline overflow"))?;
|
||||
current_phase = Some(doc.phases.len() - 1);
|
||||
current_actor = None;
|
||||
continue;
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("repeat ") {
|
||||
doc.repeat = value
|
||||
.parse()
|
||||
.map_err(|_| Error::parse(line_no, "repeat must be an integer"))?;
|
||||
if doc.repeat == 0 || doc.repeat > 10_000 {
|
||||
return Err(Error::parse(line_no, "repeat must be 1 to 10000"));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if let Some(ai) = current_actor {
|
||||
parse_actor_field(&mut doc.actors[ai], line, line_no)?;
|
||||
continue;
|
||||
}
|
||||
if let Some(pi) = current_phase {
|
||||
parse_phase_field(&mut doc.phases[pi], line, line_no)?;
|
||||
continue;
|
||||
}
|
||||
return Err(Error::parse(line_no, "directive is outside actor or step"));
|
||||
}
|
||||
|
||||
let scene = scene.ok_or_else(|| Error::parse(0, "empty document"))?;
|
||||
if let Some(issue) = validate_scene(&scene, &ValidationConfig::default())
|
||||
.into_iter()
|
||||
.find(|i| i.severity == ValidationSeverity::Error)
|
||||
{
|
||||
return Err(Error::Validation(format!(
|
||||
"{}: {}",
|
||||
issue.path, issue.message
|
||||
)));
|
||||
}
|
||||
Ok(scene)
|
||||
}
|
||||
|
||||
fn parse_actor_field(actor: &mut Actor, line: &str, line_no: usize) -> Result<()> {
|
||||
if let Some(value) = line.strip_prefix("rig ") {
|
||||
actor.rig = value.trim().into();
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("pose start = ") {
|
||||
actor.start_pose = value.trim().into();
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("track ") {
|
||||
actor.track_id = Some(
|
||||
value
|
||||
.parse()
|
||||
.map_err(|_| Error::parse(line_no, "track must be an integer"))?,
|
||||
);
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("confidence ") {
|
||||
actor.confidence = parse_confidence(value, line_no)?;
|
||||
return Ok(());
|
||||
}
|
||||
if let Some(value) = line.strip_prefix("position ") {
|
||||
actor.position = parse_vec3(value, line_no)?;
|
||||
return Ok(());
|
||||
}
|
||||
Err(Error::parse(line_no, "unknown actor directive"))
|
||||
}
|
||||
|
||||
fn parse_phase_field(phase: &mut Phase, line: &str, line_no: usize) -> Result<()> {
|
||||
if let Some(value) = line.strip_prefix("contact ") {
|
||||
let fields: Vec<&str> = value.split_whitespace().collect();
|
||||
if !(fields.len() == 2 || fields.len() == 3) {
|
||||
return Err(Error::parse(
|
||||
line_no,
|
||||
"contact requires two effectors and optional confidence",
|
||||
));
|
||||
}
|
||||
phase.contacts.push(Contact {
|
||||
from: parse_effector(fields[0], line_no)?,
|
||||
to: parse_effector(fields[1], line_no)?,
|
||||
confidence: if fields.len() == 3 {
|
||||
parse_confidence(fields[2], line_no)?
|
||||
} else {
|
||||
1.0
|
||||
},
|
||||
});
|
||||
return Ok(());
|
||||
}
|
||||
let (lhs, rhs) = line
|
||||
.split_once(':')
|
||||
.ok_or_else(|| Error::parse(line_no, "phase directive requires ':'"))?;
|
||||
let (actor_id, target) = lhs
|
||||
.trim()
|
||||
.split_once('.')
|
||||
.ok_or_else(|| Error::parse(line_no, "target must be actor qualified"))?;
|
||||
let actor = phase_actor_mut(phase, actor_id);
|
||||
match target {
|
||||
"ground-lock" => {
|
||||
actor.ground_lock = rhs
|
||||
.split(',')
|
||||
.map(|s| s.trim().to_string())
|
||||
.filter(|s| !s.is_empty())
|
||||
.collect();
|
||||
}
|
||||
"travel" => {
|
||||
actor.travel = Some(parse_vec3_or_xz(rhs, line_no)?);
|
||||
}
|
||||
joint => {
|
||||
let fields: Vec<&str> = rhs.split_whitespace().collect();
|
||||
if !(fields.len() == 2 || fields.len() == 3) {
|
||||
return Err(Error::parse(
|
||||
line_no,
|
||||
"joint target requires action, degrees and optional confidence",
|
||||
));
|
||||
}
|
||||
actor.joints.push(JointTarget {
|
||||
joint: joint.into(),
|
||||
action: fields[0].into(),
|
||||
degrees: fields[1]
|
||||
.parse()
|
||||
.map_err(|_| Error::parse(line_no, "degrees must be numeric"))?,
|
||||
confidence: if fields.len() == 3 {
|
||||
parse_confidence(fields[2], line_no)?
|
||||
} else {
|
||||
1.0
|
||||
},
|
||||
});
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn phase_actor_mut<'a>(phase: &'a mut Phase, id: &str) -> &'a mut ActorTargets {
|
||||
if let Some(index) = phase.actors.iter().position(|a| a.actor_id == id) {
|
||||
return &mut phase.actors[index];
|
||||
}
|
||||
phase.actors.push(ActorTargets::new(id));
|
||||
phase.actors.last_mut().expect("just pushed")
|
||||
}
|
||||
|
||||
fn parse_effector(value: &str, line: usize) -> Result<EffectorRef> {
|
||||
let (actor_id, effector) = value
|
||||
.split_once('.')
|
||||
.ok_or_else(|| Error::parse(line, "effector must be actor qualified"))?;
|
||||
Ok(EffectorRef {
|
||||
actor_id: actor_id.into(),
|
||||
effector: effector.into(),
|
||||
})
|
||||
}
|
||||
|
||||
fn parse_vec3(value: &str, line: usize) -> Result<Vec3> {
|
||||
let f: Vec<&str> = value.split_whitespace().collect();
|
||||
if f.len() != 3 {
|
||||
return Err(Error::parse(line, "position requires x y z"));
|
||||
}
|
||||
Ok(Vec3::new(
|
||||
parse_f32(f[0], line)?,
|
||||
parse_f32(f[1], line)?,
|
||||
parse_f32(f[2], line)?,
|
||||
))
|
||||
}
|
||||
|
||||
fn parse_vec3_or_xz(value: &str, line: usize) -> Result<Vec3> {
|
||||
let f: Vec<&str> = value.split_whitespace().collect();
|
||||
match f.len() {
|
||||
2 => Ok(Vec3::new(
|
||||
parse_f32(f[0], line)?,
|
||||
0.0,
|
||||
parse_f32(f[1], line)?,
|
||||
)),
|
||||
3 => parse_vec3(value, line),
|
||||
_ => Err(Error::parse(line, "travel requires x z or x y z")),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_f32(value: &str, line: usize) -> Result<f32> {
|
||||
let v: f32 = value
|
||||
.parse()
|
||||
.map_err(|_| Error::parse(line, "expected finite number"))?;
|
||||
if !v.is_finite() {
|
||||
return Err(Error::parse(line, "expected finite number"));
|
||||
}
|
||||
Ok(v)
|
||||
}
|
||||
|
||||
fn parse_confidence(value: &str, line: usize) -> Result<f32> {
|
||||
let v = parse_f32(value, line)?;
|
||||
if !(0.0..=1.0).contains(&v) {
|
||||
return Err(Error::parse(line, "confidence must be in [0, 1]"));
|
||||
}
|
||||
Ok(v)
|
||||
}
|
||||
|
||||
fn parse_duration(value: &str, line: usize) -> Result<u64> {
|
||||
let seconds = value
|
||||
.strip_suffix('s')
|
||||
.ok_or_else(|| Error::parse(line, "duration must end in s"))?;
|
||||
let seconds = parse_f32(seconds, line)?;
|
||||
if seconds <= 0.0 || seconds > 300.0 {
|
||||
return Err(Error::parse(line, "duration must be in (0, 300] seconds"));
|
||||
}
|
||||
Ok((seconds * 1000.0).round() as u64)
|
||||
}
|
||||
|
||||
fn parse_timing(value: &str, line: usize) -> Result<Timing> {
|
||||
match value {
|
||||
"flow" => Ok(Timing::Flow),
|
||||
"settle" => Ok(Timing::Settle),
|
||||
"drive" => Ok(Timing::Drive),
|
||||
"snap" => Ok(Timing::Snap),
|
||||
"linear" => Ok(Timing::Linear),
|
||||
"ease-in" => Ok(Timing::EaseIn),
|
||||
"ease-out" => Ok(Timing::EaseOut),
|
||||
"ease-in-out" => Ok(Timing::EaseInOut),
|
||||
_ => Err(Error::parse(line, "unknown timing")),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_quoted(value: &str, line: usize) -> Result<String> {
|
||||
let (quoted, tail) = take_quoted(value, line)?;
|
||||
if !tail.trim().is_empty() {
|
||||
return Err(Error::parse(line, "unexpected text after quoted name"));
|
||||
}
|
||||
Ok(quoted)
|
||||
}
|
||||
|
||||
fn take_quoted(value: &str, line: usize) -> Result<(String, &str)> {
|
||||
let rest = value
|
||||
.strip_prefix('"')
|
||||
.ok_or_else(|| Error::parse(line, "expected quoted name"))?;
|
||||
let end = rest
|
||||
.find('"')
|
||||
.ok_or_else(|| Error::parse(line, "unterminated quoted name"))?;
|
||||
Ok((rest[..end].to_string(), &rest[end + 1..]))
|
||||
}
|
||||
|
||||
fn strip_comment(line: &str) -> &str {
|
||||
let hash = line.find('#');
|
||||
let slash = line.find("//");
|
||||
match (hash, slash) {
|
||||
(Some(a), Some(b)) => &line[..a.min(b)],
|
||||
(Some(a), None) | (None, Some(a)) => &line[..a],
|
||||
(None, None) => line,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
const SCENE: &str = r#"posecode scene "Assisted squat"
|
||||
source observed_wifi_csi
|
||||
actor patient:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
track 7
|
||||
confidence 0.82
|
||||
position 0 0 0
|
||||
actor therapist:
|
||||
rig humanoid
|
||||
pose start = standing
|
||||
position 1.2 0 0
|
||||
step "Lower" 1.5s flow:
|
||||
patient.knee_left: flex 95 0.8
|
||||
patient.ground-lock: feet
|
||||
therapist.shoulder_left: flex 30
|
||||
contact therapist.hand_left patient.shoulder_right 0.7
|
||||
repeat 2
|
||||
"#;
|
||||
|
||||
#[test]
|
||||
fn parses_multiple_actors_and_contacts() {
|
||||
let scene = parse_posecode(SCENE).unwrap();
|
||||
assert_eq!(scene.actors.len(), 2);
|
||||
assert_eq!(scene.phases[0].actors.len(), 2);
|
||||
assert_eq!(scene.phases[0].contacts.len(), 1);
|
||||
assert_eq!(scene.repeat, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_unknown_actor_reference() {
|
||||
let bad = "posecode scene \"x\"\nactor p1:\n rig humanoid\nstep \"x\" 1s flow:\n ghost.knee_left: flex 20";
|
||||
assert!(parse_posecode(bad).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_non_finite_number() {
|
||||
let bad = "posecode scene \"x\"\nactor p1:\n position NaN 0 0";
|
||||
assert!(parse_posecode(bad).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_duplicate_actors() {
|
||||
let bad = "posecode scene \"x\"\nactor p1:\n rig humanoid\nactor p1:\n rig humanoid";
|
||||
assert!(parse_posecode(bad).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_oversized_document_before_parsing() {
|
||||
let bad = "x".repeat(MAX_INPUT_BYTES + 1);
|
||||
assert!(parse_posecode(&bad).is_err());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,212 @@
|
||||
//! Confidence preserving reduction of high-rate observations into phases.
|
||||
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
|
||||
use crate::error::{Error, Result};
|
||||
use crate::model::*;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct SegmenterConfig {
|
||||
pub minimum_phase_ms: u64,
|
||||
pub maximum_phase_ms: u64,
|
||||
pub stable_velocity_deg_s: f32,
|
||||
}
|
||||
|
||||
impl Default for SegmenterConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
minimum_phase_ms: 200,
|
||||
maximum_phase_ms: 3_000,
|
||||
stable_velocity_deg_s: 8.0,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PhaseSegmenter {
|
||||
config: SegmenterConfig,
|
||||
}
|
||||
|
||||
impl PhaseSegmenter {
|
||||
pub fn new(config: SegmenterConfig) -> Result<Self> {
|
||||
if config.minimum_phase_ms == 0
|
||||
|| config.maximum_phase_ms < config.minimum_phase_ms
|
||||
|| !config.stable_velocity_deg_s.is_finite()
|
||||
|| config.stable_velocity_deg_s < 0.0
|
||||
{
|
||||
return Err(Error::Observation("invalid segmenter configuration".into()));
|
||||
}
|
||||
Ok(Self { config })
|
||||
}
|
||||
|
||||
pub fn segment(&self, name: impl Into<String>, frames: &[SceneFrame]) -> Result<Scene> {
|
||||
if frames.is_empty() {
|
||||
return Err(Error::Observation("at least one frame is required".into()));
|
||||
}
|
||||
for pair in frames.windows(2) {
|
||||
if pair[1].timestamp_ms <= pair[0].timestamp_ms {
|
||||
return Err(Error::Observation(
|
||||
"timestamps must be strictly increasing".into(),
|
||||
));
|
||||
}
|
||||
}
|
||||
let mut scene = Scene::new(name);
|
||||
scene.source = SceneSource::ObservedWifiCsi;
|
||||
scene.actors = actors_from_frames(frames);
|
||||
|
||||
if frames.len() == 1 {
|
||||
return Ok(scene);
|
||||
}
|
||||
let mut start = 0_usize;
|
||||
let mut was_stable = true;
|
||||
for i in 1..frames.len() {
|
||||
let dt = frames[i].timestamp_ms - frames[i - 1].timestamp_ms;
|
||||
let velocity = angular_velocity(&frames[i - 1], &frames[i], dt);
|
||||
let stable = velocity <= self.config.stable_velocity_deg_s;
|
||||
let elapsed = frames[i].timestamp_ms - frames[start].timestamp_ms;
|
||||
let membership_changed = actor_ids(&frames[i - 1]) != actor_ids(&frames[i]);
|
||||
let boundary = membership_changed
|
||||
|| elapsed >= self.config.maximum_phase_ms
|
||||
|| (stable && !was_stable && elapsed >= self.config.minimum_phase_ms);
|
||||
if boundary {
|
||||
scene
|
||||
.phases
|
||||
.push(make_phase(scene.phases.len(), &frames[start], &frames[i]));
|
||||
start = i;
|
||||
}
|
||||
was_stable = stable;
|
||||
}
|
||||
let last = frames.len() - 1;
|
||||
if start < last {
|
||||
scene.phases.push(make_phase(
|
||||
scene.phases.len(),
|
||||
&frames[start],
|
||||
&frames[last],
|
||||
));
|
||||
}
|
||||
Ok(scene)
|
||||
}
|
||||
}
|
||||
|
||||
fn actors_from_frames(frames: &[SceneFrame]) -> Vec<Actor> {
|
||||
let mut found = BTreeMap::<u64, Actor>::new();
|
||||
for frame in frames {
|
||||
for observed in &frame.actors {
|
||||
found
|
||||
.entry(observed.track_id)
|
||||
.and_modify(|actor| {
|
||||
actor.confidence = actor.confidence.max(observed.confidence);
|
||||
actor.position = observed.position;
|
||||
})
|
||||
.or_insert_with(|| {
|
||||
let mut actor = Actor::humanoid(observed.actor_id.clone());
|
||||
actor.track_id = Some(observed.track_id);
|
||||
actor.confidence = observed.confidence;
|
||||
actor.position = observed.position;
|
||||
actor
|
||||
});
|
||||
}
|
||||
}
|
||||
found.into_values().collect()
|
||||
}
|
||||
|
||||
fn make_phase(index: usize, start: &SceneFrame, end: &SceneFrame) -> Phase {
|
||||
let actors = end
|
||||
.actors
|
||||
.iter()
|
||||
.map(|observed| ActorTargets {
|
||||
actor_id: observed.actor_id.clone(),
|
||||
joints: observed.joints.clone(),
|
||||
ground_lock: Vec::new(),
|
||||
travel: Some(observed.position),
|
||||
confidence: observed.confidence,
|
||||
})
|
||||
.collect();
|
||||
Phase {
|
||||
name: format!("Observed {}", index + 1),
|
||||
start_ms: start.timestamp_ms,
|
||||
duration_ms: (end.timestamp_ms - start.timestamp_ms).max(1),
|
||||
timing: Timing::Linear,
|
||||
actors,
|
||||
contacts: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn actor_ids(frame: &SceneFrame) -> BTreeSet<u64> {
|
||||
frame.actors.iter().map(|a| a.track_id).collect()
|
||||
}
|
||||
|
||||
fn angular_velocity(previous: &SceneFrame, current: &SceneFrame, dt_ms: u64) -> f32 {
|
||||
let mut sum = 0.0;
|
||||
let mut count = 0_u32;
|
||||
for a in ¤t.actors {
|
||||
let Some(before) = previous.actors.iter().find(|p| p.track_id == a.track_id) else {
|
||||
return f32::INFINITY;
|
||||
};
|
||||
for joint in &a.joints {
|
||||
if let Some(old) = before
|
||||
.joints
|
||||
.iter()
|
||||
.find(|j| j.joint == joint.joint && j.action == joint.action)
|
||||
{
|
||||
sum += (joint.degrees - old.degrees).abs() * joint.confidence.min(old.confidence);
|
||||
count += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
if count == 0 {
|
||||
return 0.0;
|
||||
}
|
||||
(sum / count as f32) / (dt_ms as f32 / 1000.0).max(0.001)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn frame(ms: u64, angle: f32) -> SceneFrame {
|
||||
SceneFrame {
|
||||
timestamp_ms: ms,
|
||||
actors: vec![ActorObservation {
|
||||
actor_id: "p1".into(),
|
||||
track_id: 1,
|
||||
confidence: 0.9,
|
||||
position: Vec3::default(),
|
||||
joints: vec![JointTarget {
|
||||
joint: "knee_left".into(),
|
||||
action: "flex".into(),
|
||||
degrees: angle,
|
||||
confidence: 0.9,
|
||||
}],
|
||||
}],
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn creates_compact_observed_scene() {
|
||||
let frames = vec![
|
||||
frame(0, 0.0),
|
||||
frame(100, 20.0),
|
||||
frame(200, 40.0),
|
||||
frame(300, 40.1),
|
||||
frame(500, 40.1),
|
||||
];
|
||||
let scene = PhaseSegmenter::new(SegmenterConfig::default())
|
||||
.unwrap()
|
||||
.segment("squat", &frames)
|
||||
.unwrap();
|
||||
assert_eq!(scene.actors.len(), 1);
|
||||
assert!(!scene.phases.is_empty());
|
||||
assert!(scene.phases.len() < frames.len());
|
||||
assert_eq!(scene.source, SceneSource::ObservedWifiCsi);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_reversed_time() {
|
||||
let frames = vec![frame(100, 0.0), frame(50, 1.0)];
|
||||
assert!(PhaseSegmenter::new(SegmenterConfig::default())
|
||||
.unwrap()
|
||||
.segment("bad", &frames)
|
||||
.is_err());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,131 @@
|
||||
//! Canonical deterministic PoseCode text serialization.
|
||||
|
||||
use std::fmt::Write;
|
||||
|
||||
use crate::model::*;
|
||||
|
||||
pub fn to_posecode(scene: &Scene) -> String {
|
||||
let mut out = String::new();
|
||||
writeln!(out, "posecode scene \"{}\"", safe_text(&scene.name)).unwrap();
|
||||
writeln!(out, "source {}", source_name(scene.source)).unwrap();
|
||||
for actor in &scene.actors {
|
||||
writeln!(out, "actor {}:", actor.id).unwrap();
|
||||
writeln!(out, " rig {}", actor.rig).unwrap();
|
||||
writeln!(out, " pose start = {}", actor.start_pose).unwrap();
|
||||
if let Some(track) = actor.track_id {
|
||||
writeln!(out, " track {track}").unwrap();
|
||||
}
|
||||
writeln!(out, " confidence {}", number(actor.confidence)).unwrap();
|
||||
writeln!(
|
||||
out,
|
||||
" position {} {} {}",
|
||||
number(actor.position.x),
|
||||
number(actor.position.y),
|
||||
number(actor.position.z)
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
for phase in &scene.phases {
|
||||
writeln!(
|
||||
out,
|
||||
"step \"{}\" {}s {}:",
|
||||
safe_text(&phase.name),
|
||||
number(phase.duration_ms as f32 / 1000.0),
|
||||
phase.timing.as_str()
|
||||
)
|
||||
.unwrap();
|
||||
for actor in &phase.actors {
|
||||
for joint in &actor.joints {
|
||||
writeln!(
|
||||
out,
|
||||
" {}.{}: {} {} {}",
|
||||
actor.actor_id,
|
||||
joint.joint,
|
||||
joint.action,
|
||||
number(joint.degrees),
|
||||
number(joint.confidence)
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
if !actor.ground_lock.is_empty() {
|
||||
writeln!(
|
||||
out,
|
||||
" {}.ground-lock: {}",
|
||||
actor.actor_id,
|
||||
actor.ground_lock.join(", ")
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
if let Some(p) = actor.travel {
|
||||
writeln!(
|
||||
out,
|
||||
" {}.travel: {} {} {}",
|
||||
actor.actor_id,
|
||||
number(p.x),
|
||||
number(p.y),
|
||||
number(p.z)
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
for contact in &phase.contacts {
|
||||
writeln!(
|
||||
out,
|
||||
" contact {}.{} {}.{} {}",
|
||||
contact.from.actor_id,
|
||||
contact.from.effector,
|
||||
contact.to.actor_id,
|
||||
contact.to.effector,
|
||||
number(contact.confidence)
|
||||
)
|
||||
.unwrap();
|
||||
}
|
||||
}
|
||||
writeln!(out, "repeat {}", scene.repeat).unwrap();
|
||||
out
|
||||
}
|
||||
|
||||
fn source_name(source: SceneSource) -> &'static str {
|
||||
match source {
|
||||
SceneSource::Authored => "authored",
|
||||
SceneSource::ObservedWifiCsi => "observed_wifi_csi",
|
||||
SceneSource::Imported => "imported",
|
||||
}
|
||||
}
|
||||
|
||||
fn safe_text(value: &str) -> String {
|
||||
value
|
||||
.chars()
|
||||
.filter(|c| !c.is_control() && *c != '"')
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn number(value: f32) -> String {
|
||||
if value == 0.0 {
|
||||
return "0".into();
|
||||
}
|
||||
let mut text = format!("{value:.4}");
|
||||
while text.ends_with('0') {
|
||||
text.pop();
|
||||
}
|
||||
if text.ends_with('.') {
|
||||
text.pop();
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use crate::parse_posecode;
|
||||
|
||||
#[test]
|
||||
fn canonical_text_round_trips() {
|
||||
let mut scene = Scene::new("two people");
|
||||
scene.actors.push(Actor::humanoid("p1"));
|
||||
scene.actors.push(Actor::humanoid("p2"));
|
||||
let text = to_posecode(&scene);
|
||||
assert_eq!(parse_posecode(&text).unwrap(), scene);
|
||||
assert_eq!(to_posecode(&parse_posecode(&text).unwrap()), text);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,202 @@
|
||||
//! Deterministic scene validation with provenance-aware range policy.
|
||||
|
||||
use std::collections::HashSet;
|
||||
|
||||
use crate::model::*;
|
||||
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub enum ValidationSeverity {
|
||||
Warning,
|
||||
Error,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ValidationIssue {
|
||||
pub severity: ValidationSeverity,
|
||||
pub path: String,
|
||||
pub message: String,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct ValidationConfig {
|
||||
pub max_actors: usize,
|
||||
pub max_phases: usize,
|
||||
/// Authored values outside general ROM are errors. Observed values are
|
||||
/// warnings because RF estimates are not medical measurements.
|
||||
pub authored_rom_is_error: bool,
|
||||
}
|
||||
|
||||
impl Default for ValidationConfig {
|
||||
fn default() -> Self {
|
||||
Self {
|
||||
max_actors: MAX_ACTORS,
|
||||
max_phases: MAX_PHASES,
|
||||
authored_rom_is_error: true,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn validate_scene(scene: &Scene, config: &ValidationConfig) -> Vec<ValidationIssue> {
|
||||
let mut issues = Vec::new();
|
||||
if scene.name.is_empty() || scene.name.len() > MAX_NAME_BYTES {
|
||||
error(
|
||||
&mut issues,
|
||||
"scene.name",
|
||||
"name must contain 1 to 128 bytes",
|
||||
);
|
||||
}
|
||||
if scene.actors.len() > config.max_actors {
|
||||
error(&mut issues, "scene.actors", "actor limit exceeded");
|
||||
}
|
||||
if scene.phases.len() > config.max_phases {
|
||||
error(&mut issues, "scene.phases", "phase limit exceeded");
|
||||
}
|
||||
|
||||
let mut ids = HashSet::new();
|
||||
for (i, actor) in scene.actors.iter().enumerate() {
|
||||
let path = format!("actors[{i}]");
|
||||
if !valid_id(&actor.id) {
|
||||
error(&mut issues, &path, "invalid actor identifier");
|
||||
}
|
||||
if !ids.insert(actor.id.as_str()) {
|
||||
error(&mut issues, &path, "duplicate actor identifier");
|
||||
}
|
||||
check_confidence(&mut issues, &format!("{path}.confidence"), actor.confidence);
|
||||
if !actor.position.is_finite() {
|
||||
error(
|
||||
&mut issues,
|
||||
&format!("{path}.position"),
|
||||
"position must be finite",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
let actor_ids: HashSet<&str> = scene.actors.iter().map(|a| a.id.as_str()).collect();
|
||||
for (pi, phase) in scene.phases.iter().enumerate() {
|
||||
let path = format!("phases[{pi}]");
|
||||
if phase.duration_ms == 0 || phase.duration_ms > 300_000 {
|
||||
error(
|
||||
&mut issues,
|
||||
&format!("{path}.duration_ms"),
|
||||
"duration must be 1 to 300000 ms",
|
||||
);
|
||||
}
|
||||
let mut phase_actors = HashSet::new();
|
||||
for targets in &phase.actors {
|
||||
if !actor_ids.contains(targets.actor_id.as_str()) {
|
||||
error(&mut issues, &path, "phase references unknown actor");
|
||||
}
|
||||
if !phase_actors.insert(targets.actor_id.as_str()) {
|
||||
error(&mut issues, &path, "actor appears more than once in phase");
|
||||
}
|
||||
if targets.joints.len() > MAX_TARGETS_PER_ACTOR {
|
||||
error(&mut issues, &path, "joint target limit exceeded");
|
||||
}
|
||||
check_confidence(&mut issues, &path, targets.confidence);
|
||||
for joint in &targets.joints {
|
||||
check_confidence(&mut issues, &path, joint.confidence);
|
||||
if !joint.degrees.is_finite() {
|
||||
error(&mut issues, &path, "joint angle must be finite");
|
||||
continue;
|
||||
}
|
||||
if let Some((min, max)) = range(&joint.joint, &joint.action) {
|
||||
if joint.degrees < min || joint.degrees > max {
|
||||
let severity = if scene.source == SceneSource::Authored
|
||||
&& config.authored_rom_is_error
|
||||
{
|
||||
ValidationSeverity::Error
|
||||
} else {
|
||||
ValidationSeverity::Warning
|
||||
};
|
||||
issues.push(ValidationIssue {
|
||||
severity,
|
||||
path: path.clone(),
|
||||
message: format!(
|
||||
"{}.{} angle {} outside general range [{}, {}]",
|
||||
joint.joint, joint.action, joint.degrees, min, max
|
||||
),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for contact in &phase.contacts {
|
||||
if !actor_ids.contains(contact.from.actor_id.as_str())
|
||||
|| !actor_ids.contains(contact.to.actor_id.as_str())
|
||||
{
|
||||
error(&mut issues, &path, "contact references unknown actor");
|
||||
}
|
||||
check_confidence(&mut issues, &path, contact.confidence);
|
||||
}
|
||||
}
|
||||
issues
|
||||
}
|
||||
|
||||
fn valid_id(id: &str) -> bool {
|
||||
!id.is_empty()
|
||||
&& id.len() <= 64
|
||||
&& id
|
||||
.bytes()
|
||||
.enumerate()
|
||||
.all(|(i, b)| b.is_ascii_alphanumeric() || b == b'_' || (i > 0 && b == b'-'))
|
||||
}
|
||||
|
||||
fn check_confidence(issues: &mut Vec<ValidationIssue>, path: &str, value: f32) {
|
||||
if !value.is_finite() || !(0.0..=1.0).contains(&value) {
|
||||
error(issues, path, "confidence must be finite and in [0, 1]");
|
||||
}
|
||||
}
|
||||
|
||||
fn error(issues: &mut Vec<ValidationIssue>, path: &str, message: &str) {
|
||||
issues.push(ValidationIssue {
|
||||
severity: ValidationSeverity::Error,
|
||||
path: path.into(),
|
||||
message: message.into(),
|
||||
});
|
||||
}
|
||||
|
||||
fn range(joint: &str, action: &str) -> Option<(f32, f32)> {
|
||||
match (
|
||||
joint.trim_end_matches("_left").trim_end_matches("_right"),
|
||||
action,
|
||||
) {
|
||||
("shoulder", "flex") => Some((0.0, 180.0)),
|
||||
("shoulder", "extend") => Some((0.0, 60.0)),
|
||||
("elbow", "flex") => Some((0.0, 154.0)),
|
||||
("hip", "flex") => Some((0.0, 135.0)),
|
||||
("hip", "extend") => Some((0.0, 20.0)),
|
||||
("knee", "flex") => Some((0.0, 144.0)),
|
||||
("pelvis", "hinge") => Some((0.0, 120.0)),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn observed_rom_is_warning_not_medical_error() {
|
||||
let mut scene = Scene::new("observed");
|
||||
scene.source = SceneSource::ObservedWifiCsi;
|
||||
scene.actors.push(Actor::humanoid("p1"));
|
||||
let mut targets = ActorTargets::new("p1");
|
||||
targets.joints.push(JointTarget {
|
||||
joint: "knee_left".into(),
|
||||
action: "flex".into(),
|
||||
degrees: 170.0,
|
||||
confidence: 0.5,
|
||||
});
|
||||
scene.phases.push(Phase {
|
||||
name: "frame".into(),
|
||||
start_ms: 0,
|
||||
duration_ms: 50,
|
||||
timing: Timing::Linear,
|
||||
actors: vec![targets],
|
||||
contacts: vec![],
|
||||
});
|
||||
let issues = validate_scene(&scene, &ValidationConfig::default());
|
||||
assert_eq!(issues.len(), 1);
|
||||
assert_eq!(issues[0].severity, ValidationSeverity::Warning);
|
||||
}
|
||||
}
|
||||
@@ -7,22 +7,12 @@
|
||||
//! score-based heuristic in `score_to_person_count`.
|
||||
|
||||
use std::collections::VecDeque;
|
||||
use std::sync::LazyLock;
|
||||
use wifi_densepose_signal::hardware_norm::HardwareNormalizer;
|
||||
use wifi_densepose_signal::ruvsense::field_model::{
|
||||
CalibrationStatus, FieldModel, FieldModelConfig,
|
||||
};
|
||||
|
||||
use super::score_to_person_count;
|
||||
|
||||
/// Length-only canonicalizer for calibration frames (issue #1170 pattern,
|
||||
/// shared with `multistatic_bridge`). Raw ESP32 amplitudes arrive at the
|
||||
/// hardware's native width (HT20 ≈ 64, HT40 ≈ 128/192); the FieldModel is
|
||||
/// configured for the canonical 56-tone grid, and `feed_calibration` rejects
|
||||
/// any other width with `DimensionMismatch`. Resampling here (default 56)
|
||||
/// lets real HT40 nodes actually calibrate instead of silently feeding nothing.
|
||||
static CALIB_NORMALIZER: LazyLock<HardwareNormalizer> = LazyLock::new(HardwareNormalizer::new);
|
||||
|
||||
/// Number of recent frames to feed into perturbation extraction.
|
||||
const OCCUPANCY_WINDOW: usize = 50;
|
||||
|
||||
@@ -109,27 +99,15 @@ pub fn occupancy_or_fallback(
|
||||
|
||||
/// Feed the latest frame to the FieldModel during calibration collection.
|
||||
///
|
||||
/// Acts while the model is `Uncalibrated` or `Collecting`. The first fed frame
|
||||
/// flips a freshly-started (`Uncalibrated`) model to `Collecting` inside
|
||||
/// `feed_calibration`; without accepting the `Uncalibrated` state here the two
|
||||
/// gates deadlock and the frame count never leaves 0 (calibration/start yields
|
||||
/// an `Uncalibrated` model that nothing would ever advance). Wraps the latest
|
||||
/// frame as a single-link observation (n_links=1) and feeds it.
|
||||
/// Only acts when the model status is `Collecting`. Wraps the latest frame
|
||||
/// as a single-link observation (n_links=1) and feeds it.
|
||||
pub fn maybe_feed_calibration(field: &mut FieldModel, frame_history: &VecDeque<Vec<f64>>) {
|
||||
if !matches!(
|
||||
field.status(),
|
||||
CalibrationStatus::Uncalibrated | CalibrationStatus::Collecting
|
||||
) {
|
||||
if field.status() != CalibrationStatus::Collecting {
|
||||
return;
|
||||
}
|
||||
if let Some(latest) = frame_history.back() {
|
||||
// Resample the raw amplitude vector onto the FieldModel's canonical
|
||||
// 56-tone grid before feeding. Real HT40 nodes stream 128-wide frames;
|
||||
// feeding those raw made every `feed_calibration` fail DimensionMismatch
|
||||
// (swallowed at debug level), pinning frame_count at 0 even after the
|
||||
// status-gate deadlock was fixed. Single-link observation: [1][56].
|
||||
let canonical = CALIB_NORMALIZER.resample_to_canonical(latest);
|
||||
let observations = vec![canonical];
|
||||
// Single-link observation: [1][n_subcarriers]
|
||||
let observations = vec![latest.clone()];
|
||||
if let Err(e) = field.feed_calibration(&observations) {
|
||||
tracing::debug!("FieldModel calibration feed: {e}");
|
||||
}
|
||||
@@ -202,65 +180,4 @@ mod tests {
|
||||
assert_eq!(positions.len(), 1);
|
||||
assert_eq!(positions[0], [3.0, 4.0, 5.0]);
|
||||
}
|
||||
|
||||
/// Regression: a freshly-started (`Uncalibrated`) field model must begin
|
||||
/// collecting once frames arrive. Before the fix, `maybe_feed_calibration`
|
||||
/// only fed while already `Collecting`, but only `feed_calibration` sets
|
||||
/// `Collecting` — so the first frame was never fed and the count stayed 0.
|
||||
#[test]
|
||||
fn maybe_feed_calibration_advances_uncalibrated_to_collecting() {
|
||||
let mut field = FieldModel::new(single_link_config()).expect("field model");
|
||||
assert_eq!(field.status(), CalibrationStatus::Uncalibrated);
|
||||
assert_eq!(field.calibration_frame_count(), 0);
|
||||
|
||||
// n_subcarriers defaults to 56; one single-link frame of that width.
|
||||
let frame = vec![0.5_f64; 56];
|
||||
let mut history: VecDeque<Vec<f64>> = VecDeque::new();
|
||||
history.push_back(frame);
|
||||
|
||||
maybe_feed_calibration(&mut field, &history);
|
||||
|
||||
assert_eq!(
|
||||
field.status(),
|
||||
CalibrationStatus::Collecting,
|
||||
"first frame must flip Uncalibrated -> Collecting"
|
||||
);
|
||||
assert_eq!(
|
||||
field.calibration_frame_count(),
|
||||
1,
|
||||
"frame count must advance past 0"
|
||||
);
|
||||
|
||||
// Subsequent frames keep accumulating while Collecting.
|
||||
maybe_feed_calibration(&mut field, &history);
|
||||
assert_eq!(field.calibration_frame_count(), 2);
|
||||
}
|
||||
|
||||
/// Regression (#1170 pattern): a real HT40 node streams 128-wide amplitude
|
||||
/// frames, but the FieldModel is a 56-tone grid. Before canonicalization,
|
||||
/// `feed_calibration` rejected every frame with DimensionMismatch (swallowed
|
||||
/// at debug), so frame_count stayed 0 even with the deadlock fixed. The feed
|
||||
/// must resample 128 → 56 and actually accumulate.
|
||||
#[test]
|
||||
fn maybe_feed_calibration_resamples_wide_frames_and_accumulates() {
|
||||
let mut field = FieldModel::new(single_link_config()).expect("field model");
|
||||
|
||||
// 128-wide frame (HT40), NOT the model's 56 — would DimensionMismatch raw.
|
||||
let wide = vec![0.5_f64; 128];
|
||||
let mut history: VecDeque<Vec<f64>> = VecDeque::new();
|
||||
history.push_back(wide);
|
||||
|
||||
maybe_feed_calibration(&mut field, &history);
|
||||
|
||||
assert_eq!(
|
||||
field.status(),
|
||||
CalibrationStatus::Collecting,
|
||||
"128-wide frame must resample to 56 and be accepted"
|
||||
);
|
||||
assert_eq!(
|
||||
field.calibration_frame_count(),
|
||||
1,
|
||||
"wide frame must accumulate, not be silently dropped"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -17,7 +17,6 @@ mod field_bridge;
|
||||
mod field_localize;
|
||||
mod model_format;
|
||||
mod multistatic_bridge;
|
||||
mod realtek_radar;
|
||||
pub mod pose;
|
||||
mod rvf_container;
|
||||
mod rvf_pipeline;
|
||||
@@ -1029,10 +1028,6 @@ struct AppStateInner {
|
||||
source: String,
|
||||
/// Instant of the last ESP32 UDP frame received (for offline detection).
|
||||
last_esp32_frame: Option<std::time::Instant>,
|
||||
/// Latest validated RTL8720F summary; raw radar samples are not retained here.
|
||||
latest_realtek_radar: Option<realtek_radar::RealtekRadarSnapshot>,
|
||||
/// Instant of the last validated RTL8720F UDP frame.
|
||||
last_realtek_frame: Option<std::time::Instant>,
|
||||
tx: broadcast::Sender<String>,
|
||||
// ADR-099 D2/D3/D4: real-time CSI introspection tap. Per-frame state +
|
||||
// a parallel broadcast topic (`/ws/introspection`) running alongside
|
||||
@@ -1204,13 +1199,6 @@ impl AppStateInner {
|
||||
}
|
||||
}
|
||||
}
|
||||
if self.source.starts_with("realtek") {
|
||||
if let Some(last) = self.last_realtek_frame {
|
||||
if last.elapsed() > ESP32_OFFLINE_TIMEOUT {
|
||||
return format!("{}:offline", self.source);
|
||||
}
|
||||
}
|
||||
}
|
||||
self.source.clone()
|
||||
}
|
||||
}
|
||||
@@ -3363,14 +3351,6 @@ async fn latest(State(state): State<SharedState>) -> Json<serde_json::Value> {
|
||||
}
|
||||
}
|
||||
|
||||
async fn latest_realtek_radar(State(state): State<SharedState>) -> Json<serde_json::Value> {
|
||||
let s = state.read().await;
|
||||
match &s.latest_realtek_radar {
|
||||
Some(snapshot) => Json(serde_json::to_value(snapshot).unwrap_or_default()),
|
||||
None => Json(serde_json::json!({"status": "no Realtek radar data yet"})),
|
||||
}
|
||||
}
|
||||
|
||||
/// Generate WiFi-derived pose keypoints from sensing data.
|
||||
///
|
||||
/// Keypoint positions are modulated by real signal features rather than a pure
|
||||
@@ -5011,20 +4991,6 @@ async fn calibration_start(State(state): State<SharedState>) -> Json<serde_json:
|
||||
async fn calibration_stop(State(state): State<SharedState>) -> Json<serde_json::Value> {
|
||||
let mut s = state.write().await;
|
||||
if let Some(ref mut fm) = s.field_model {
|
||||
// Guard: finalizing before enough empty-room frames have accumulated
|
||||
// is a client-side sequencing error, not a server fault. Return a
|
||||
// clear, structured message (with progress) instead of a 500 so the
|
||||
// caller knows to keep the room empty and poll /calibration/status.
|
||||
let have = fm.calibration_frame_count();
|
||||
let need = fm.min_calibration_frames() as u64;
|
||||
if have < need {
|
||||
return Json(serde_json::json!({
|
||||
"success": false,
|
||||
"error": "Not enough calibration frames yet — keep the room empty and poll /calibration/status until frame_count reaches the target.",
|
||||
"frame_count": have,
|
||||
"frames_needed": need,
|
||||
}));
|
||||
}
|
||||
let ts = chrono::Utc::now().timestamp_micros() as u64;
|
||||
match fm.finalize_calibration(ts, 0) {
|
||||
Ok(modes) => {
|
||||
@@ -5064,18 +5030,6 @@ async fn calibration_status(State(state): State<SharedState>) -> Json<serde_json
|
||||
}
|
||||
}
|
||||
|
||||
/// Compatibility surface used by the bundled dashboard. Activity history is
|
||||
/// not persisted by the Rust server yet, so return an honest empty collection
|
||||
/// instead of advertising the endpoint and responding with 404.
|
||||
async fn pose_activities() -> Json<serde_json::Value> {
|
||||
Json(serde_json::json!({
|
||||
"activities": [],
|
||||
"total": 0,
|
||||
"persisted": false,
|
||||
"message": "Activity history is not persisted by the Rust sensing server.",
|
||||
}))
|
||||
}
|
||||
|
||||
/// Generate a simple timestamp string (epoch seconds) for recording IDs.
|
||||
fn chrono_timestamp() -> u64 {
|
||||
std::time::SystemTime::now()
|
||||
@@ -5465,7 +5419,7 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
|
||||
let addr = format!("0.0.0.0:{udp_port}");
|
||||
let socket = match UdpSocket::bind(&addr).await {
|
||||
Ok(s) => {
|
||||
info!("UDP listening on {addr} for ESP32 CSI and RTL8720F radar frames");
|
||||
info!("UDP listening on {addr} for ESP32 CSI frames");
|
||||
s
|
||||
}
|
||||
Err(e) => {
|
||||
@@ -5474,32 +5428,10 @@ async fn udp_receiver_task(state: SharedState, udp_port: u16) {
|
||||
}
|
||||
};
|
||||
|
||||
let mut buf = vec![0u8; wifi_densepose_hardware::rtl8720f::RTL8720F_RADAR_MAX_FRAME_LEN];
|
||||
let mut buf = [0u8; 2048];
|
||||
loop {
|
||||
match socket.recv_from(&mut buf).await {
|
||||
Ok((len, src)) => {
|
||||
if len >= 4
|
||||
&& u32::from_le_bytes(buf[..4].try_into().expect("four-byte slice"))
|
||||
== wifi_densepose_hardware::rtl8720f::RTL8720F_RADAR_MAGIC
|
||||
{
|
||||
match wifi_densepose_hardware::rtl8720f::RadarFrame::from_bytes(&buf[..len]) {
|
||||
Ok((frame, consumed)) if consumed == len => {
|
||||
let snapshot = realtek_radar::RealtekRadarSnapshot::from_frame(&frame);
|
||||
debug!("RTL8720F radar from {src}: type={} seq={} elements={}", snapshot.report_type, snapshot.sequence, snapshot.element_count);
|
||||
let json = serde_json::to_string(&snapshot).ok();
|
||||
let mut s = state.write().await;
|
||||
s.source = snapshot.source.to_string();
|
||||
s.last_realtek_frame = Some(std::time::Instant::now());
|
||||
s.latest_realtek_radar = Some(snapshot);
|
||||
if let Some(json) = json {
|
||||
let _ = s.tx.send(json);
|
||||
}
|
||||
}
|
||||
Ok((_, consumed)) => warn!("RTL8720F radar datagram from {src} has trailing bytes: consumed={consumed} received={len}"),
|
||||
Err(error) => warn!("Rejected RTL8720F radar datagram from {src}: {error}"),
|
||||
}
|
||||
continue;
|
||||
}
|
||||
// ADR-039: Try edge vitals packet first (magic 0xC511_0002).
|
||||
if let Some(vitals) = parse_esp32_vitals(&buf[..len]) {
|
||||
debug!(
|
||||
@@ -7594,8 +7526,6 @@ async fn main() {
|
||||
tick: 0,
|
||||
source: source.into(),
|
||||
last_esp32_frame: None,
|
||||
latest_realtek_radar: None,
|
||||
last_realtek_frame: None,
|
||||
tx,
|
||||
intro: wifi_densepose_sensing_server::introspection::IntrospectionState::new(),
|
||||
intro_tx,
|
||||
@@ -7812,7 +7742,6 @@ async fn main() {
|
||||
.route("/api/v1/metrics", get(health_metrics))
|
||||
// Sensing endpoints
|
||||
.route("/api/v1/sensing/latest", get(latest))
|
||||
.route("/api/v1/radar/latest", get(latest_realtek_radar))
|
||||
// Per-node health endpoint
|
||||
.route("/api/v1/nodes", get(nodes_endpoint))
|
||||
// ADR-110 iter 29 — per-node mesh sync state for HTTP clients.
|
||||
@@ -7839,13 +7768,6 @@ async fn main() {
|
||||
.route("/api/v1/pose/current", get(pose_current))
|
||||
.route("/api/v1/pose/stats", get(pose_stats))
|
||||
.route("/api/v1/pose/zones/summary", get(pose_zones_summary))
|
||||
.route("/api/v1/pose/activities", get(pose_activities))
|
||||
// Dashboard-compatible aliases for the field-model calibration API.
|
||||
.route("/api/v1/pose/calibrate", post(calibration_start))
|
||||
.route(
|
||||
"/api/v1/pose/calibration/status",
|
||||
get(calibration_status),
|
||||
)
|
||||
// Stream endpoints
|
||||
.route("/api/v1/stream/status", get(stream_status))
|
||||
.route("/api/v1/stream/pose", get(ws_pose_handler))
|
||||
|
||||
@@ -1,137 +0,0 @@
|
||||
//! Bounded, privacy-conscious summaries of RTL8720F radar transport frames.
|
||||
|
||||
use serde::Serialize;
|
||||
use wifi_densepose_hardware::rtl8720f::{RadarFlags, RadarFrame, RadarPayload, ReportType};
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Serialize)]
|
||||
pub(crate) struct RealtekRadarSnapshot {
|
||||
pub event_type: &'static str,
|
||||
pub source: &'static str,
|
||||
pub report_type: &'static str,
|
||||
pub sequence: u32,
|
||||
pub timestamp_us: u64,
|
||||
pub device_id: String,
|
||||
pub center_freq_khz: u32,
|
||||
pub bandwidth_mhz: u16,
|
||||
pub antenna_count: u8,
|
||||
pub element_count: usize,
|
||||
pub calibrated: bool,
|
||||
pub synthetic: bool,
|
||||
pub interference_detected: bool,
|
||||
pub saturated: bool,
|
||||
pub time_synchronized: bool,
|
||||
pub calibration_id: u32,
|
||||
pub bin_spacing: f32,
|
||||
pub peak_range_m: Option<f32>,
|
||||
pub peak_power: Option<f32>,
|
||||
pub mean_cfr_amplitude: Option<f32>,
|
||||
}
|
||||
|
||||
impl RealtekRadarSnapshot {
|
||||
pub(crate) fn from_frame(frame: &RadarFrame) -> Self {
|
||||
let synthetic = frame.flags.contains(RadarFlags::SYNTHETIC);
|
||||
let (peak_range_m, peak_power) = range_peak(frame);
|
||||
Self {
|
||||
event_type: "realtek_radar",
|
||||
source: if synthetic {
|
||||
"realtek:simulated"
|
||||
} else {
|
||||
"realtek"
|
||||
},
|
||||
report_type: report_type_name(frame.report_type),
|
||||
sequence: frame.sequence,
|
||||
timestamp_us: frame.timestamp_us,
|
||||
device_id: format!("{:016x}", frame.device_id),
|
||||
center_freq_khz: frame.center_freq_khz,
|
||||
bandwidth_mhz: frame.bandwidth_mhz,
|
||||
antenna_count: frame.antenna_count,
|
||||
element_count: frame.payload.len(),
|
||||
calibrated: frame.flags.contains(RadarFlags::CALIBRATED),
|
||||
synthetic,
|
||||
interference_detected: frame.flags.contains(RadarFlags::INTERFERENCE_DETECTED),
|
||||
saturated: frame.flags.contains(RadarFlags::SATURATED),
|
||||
time_synchronized: frame.flags.contains(RadarFlags::TIME_SYNCHRONIZED),
|
||||
calibration_id: frame.calibration_id,
|
||||
bin_spacing: frame.bin_spacing,
|
||||
peak_range_m,
|
||||
peak_power,
|
||||
mean_cfr_amplitude: mean_cfr_amplitude(frame),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn report_type_name(report_type: ReportType) -> &'static str {
|
||||
match report_type {
|
||||
ReportType::Cfr => "cfr",
|
||||
ReportType::RangeNear => "range_near",
|
||||
ReportType::RangeFar => "range_far",
|
||||
ReportType::Interference => "interference",
|
||||
ReportType::Capabilities => "capabilities",
|
||||
}
|
||||
}
|
||||
|
||||
fn range_peak(frame: &RadarFrame) -> (Option<f32>, Option<f32>) {
|
||||
let max = match &frame.payload {
|
||||
RadarPayload::PowerU16(values) => values
|
||||
.iter()
|
||||
.enumerate()
|
||||
.max_by_key(|(_, value)| *value)
|
||||
.map(|(index, value)| (index, *value as f32 * frame.scale)),
|
||||
RadarPayload::PowerF32(values) => values
|
||||
.iter()
|
||||
.enumerate()
|
||||
.max_by(|(_, a), (_, b)| a.total_cmp(b))
|
||||
.map(|(index, value)| (index, *value * frame.scale)),
|
||||
_ => None,
|
||||
};
|
||||
max.map_or((None, None), |(index, power)| {
|
||||
(Some(index as f32 * frame.bin_spacing), Some(power))
|
||||
})
|
||||
}
|
||||
|
||||
fn mean_cfr_amplitude(frame: &RadarFrame) -> Option<f32> {
|
||||
let (sum, count) = match &frame.payload {
|
||||
RadarPayload::ComplexI16(values) => (
|
||||
values
|
||||
.iter()
|
||||
.map(|[i, q]| ((*i as f32).hypot(*q as f32)) * frame.scale)
|
||||
.sum::<f32>(),
|
||||
values.len(),
|
||||
),
|
||||
RadarPayload::ComplexF32(values) => (
|
||||
values
|
||||
.iter()
|
||||
.map(|[i, q]| i.hypot(*q) * frame.scale)
|
||||
.sum::<f32>(),
|
||||
values.len(),
|
||||
),
|
||||
_ => return None,
|
||||
};
|
||||
(count != 0).then_some(sum / count as f32)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use wifi_densepose_hardware::rtl8720f::simulator::{Rtl8720fSimulator, SimulatorConfig};
|
||||
|
||||
#[test]
|
||||
fn synthetic_range_summary_has_peak_and_provenance() {
|
||||
let mut simulator = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
let snapshot =
|
||||
RealtekRadarSnapshot::from_frame(&simulator.next_frame(ReportType::RangeNear));
|
||||
assert_eq!(snapshot.source, "realtek:simulated");
|
||||
assert!(snapshot.synthetic);
|
||||
assert!(snapshot.peak_range_m.is_some());
|
||||
assert!(snapshot.peak_power.unwrap() > 0.0);
|
||||
assert_eq!(snapshot.mean_cfr_amplitude, None);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn synthetic_cfr_summary_exposes_only_aggregate_amplitude() {
|
||||
let mut simulator = Rtl8720fSimulator::new(SimulatorConfig::default()).unwrap();
|
||||
let snapshot = RealtekRadarSnapshot::from_frame(&simulator.next_frame(ReportType::Cfr));
|
||||
assert!(snapshot.mean_cfr_amplitude.unwrap() > 0.0);
|
||||
assert_eq!(snapshot.peak_power, None);
|
||||
}
|
||||
}
|
||||
@@ -449,11 +449,6 @@ impl FieldModel {
|
||||
.map_or(0, |ls| ls.observation_count())
|
||||
}
|
||||
|
||||
/// Minimum frames required before `finalize_calibration` will succeed.
|
||||
pub fn min_calibration_frames(&self) -> usize {
|
||||
self.config.min_calibration_frames
|
||||
}
|
||||
|
||||
/// Feed a calibration frame (one CSI observation per link during empty room).
|
||||
///
|
||||
/// `observations` is `[n_links][n_subcarriers]` amplitude data.
|
||||
|
||||
Vendored
+1
-1
Submodule vendor/ruvector updated: 6a6c39e662...f3de1724fa
Reference in New Issue
Block a user