diff --git a/docs/adr/ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md b/docs/adr/ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md new file mode 100644 index 00000000..38c3a6d4 --- /dev/null +++ b/docs/adr/ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md @@ -0,0 +1,161 @@ +# ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform + +- **Status**: proposed +- **Date**: 2026-07-18 +- **Deciders**: ruv +- **Tags**: realtek, rtl8720f, ameba, fmcw, radar, cfr, csi, hardware +- **Relates to**: ADR-018, ADR-063, ADR-064, ADR-095, ADR-097, ADR-260, ADR-262 + +## Context + +Realtek's `RTL8720F-2.4G-Radar-Advantages_EN.pptx` describes an RTL8720F mode that shares the +2.4 GHz radio between Wi-Fi, Bluetooth, and an active FMCW radar. It offers two data products that +are useful to RuView: + +1. **CFR (Channel Frequency Report)**, described by Realtek as the same concept as Wi-Fi CSI. +2. **Near and far Range-FFT reports**, preserving near-field content while extending observation to + approximately 5–6 m. + +The proposed radio uses one transmit and one receive antenna, 20/40/70 MHz sweeps, configurable +8/16/32/64 microsecond chirp symbols, a maximum 2.56 ms FMCW packet, and a configurable frame +interval above 15 ms. The deck recommends 40 MHz outside Japan and 20 MHz in Japan. It also +describes EDCCA/CTS channel access, Wi-Fi/BT/radar time division, interference reporting, and +priority arbitration in the driver. + +This is not a drop-in replacement for ESP32 CSI: + +- it is **active monostatic FMCW**, while the ESP32 path observes Wi-Fi packet CSI; +- one Tx/one Rx has no angle-of-arrival or native multi-target separation; +- the stated 40 MHz range resolution is about 3.15 m, despite a finer 0.59 m Range-FFT report step; +- the presentation is a capability description, not an SDK contract. It contains no header names, + function signatures, callback ABI, binary layouts, toolchain version, licensing terms, or public + RTL8720F board package. + +Realtek's public Ameba RTOS repository is a plausible base and its public release history includes +CSI and EDCCA APIs, but the reviewed public material does not establish that the RTL8720F radar +API or firmware blobs are publicly available. Therefore an honest integration must be split at a +vendor boundary. + +## Decision + +RuView will support RTL8720F radar as an **optional, capability-negotiated source**, without +replacing the ESP32 firmware or treating radar CFR as byte-compatible with ADR-018 CSI. + +The integration has three layers: + +1. **Realtek device firmware**: a small application built in the vendor-supported Ameba SDK calls + the radar API, owns coexistence configuration, and emits versioned reports. This code lives under + `firmware/rtl8720f-radar/` only after the redistributable SDK/API is available. +2. **Transport-neutral wire contract**: CFR and Range-FFT reports are framed independently from the + vendor ABI and sent over UDP, USB CDC, or UART. ADR-264 defines this boundary. +3. **Rust host adapter**: `wifi-densepose-hardware` parses reports from bytes and converts CFR into + the existing CSI-domain representation, while Range-FFT remains a radar modality and feeds the + RuField/RuView cross-modality bridge from ADR-260/262. + +The two report types remain semantically distinct: + +| RTL8720F output | RuView representation | Permitted use | +|---|---|---| +| CFR | `CsiFrame` through a Realtek calibration adapter | CSI feature extraction after validation | +| Range-FFT near/far | `RadarFrame` / RuField `mmwave_radar`-class event with a 2.4 GHz descriptor | range, motion, presence, fusion | +| Vendor AI presence probability | derived observation with model/version provenance | advisory input, never ground truth | +| Interference report | quality/provenance metadata | reject, down-weight, or mark contaminated frames | + +The modality registry should eventually distinguish `fmcw_radar_2_4ghz` from `mmwave_radar`; until +that RuField schema revision is accepted, the adapter must attach `carrier_hz = 2.4e9` and must not +claim millimetre-wave provenance. + +## Delivery phases and gates + +### P0 — Vendor enablement + +Obtain a redistributable RTL8720F SDK package, radar API headers/libraries, a supported evaluation +board, flashing/debug instructions, report definitions, and written redistribution terms. + +**Gate:** compile and run Realtek's unmodified radar example and capture CFR plus near/far +Range-FFT output. Until this passes, device firmware is `VENDOR_BLOCKED`, not implemented. + +### P1 — Host-first contract + +Implement ADR-264 types, parsers, fixtures, fuzz tests, and replay support without linking vendor +code. Generate deterministic synthetic fixtures whose provenance is explicitly synthetic. + +**Gate:** malformed inputs never panic; encode/decode round trips; unknown versions and report +types fail closed. + +### P2 — RTL8720F firmware adapter + +Wrap only the minimum vendor API surface: initialization, profile configuration, start/stop, +callback acquisition, interference status, and report serialization. Keep vendor types out of the +wire protocol. + +**Gate:** 30-minute simultaneous Wi-Fi telemetry and radar capture with no watchdog reset, bounded +loss, monotonic sequence numbers, and explicit coexistence/interference statistics. + +### P3 — Calibration and signal validation + +Calibrate CFR phase/amplitude, Range-FFT bin spacing, static leakage, and clock drift. Compare +reported range against measured targets at multiple distances and bandwidths. + +**Gate:** publish measured error distributions. Do not infer accuracy from report-bin spacing and +do not advertise multi-person pose or vital signs from the vendor deck. + +### P4 — Fusion and productization + +Feed calibrated CFR through the CSI path and Range-FFT through RuField, retaining source, mode, +bandwidth, calibration, firmware, and interference provenance. + +**Gate:** ablation shows whether the radar stream improves a named RuView metric over ESP32 CSI +alone. If it does not, ship it only as an independent presence/range sensor. + +## Consequences + +### Positive + +- One low-cost radio can provide active radar and CSI-like CFR while retaining Wi-Fi connectivity. +- Range-FFT adds an independent physical measurement for presence/range fusion. +- The vendor SDK is isolated from the Rust sensing core and from the stable on-wire contract. +- Capability negotiation permits future Realtek parts without another application-level fork. + +### Negative + +- The first implementation is blocked on access to the actual RTL8720F radar SDK/API and hardware. +- Active 2.4 GHz transmission changes coexistence, privacy, power, and regional compliance concerns. +- 1T1R and limited sweep bandwidth cannot provide the spatial resolution of multi-antenna mmWave. +- A second embedded toolchain and firmware release process must be maintained. + +### Neutral + +- ESP32 remains the default CSI node. +- Existing consumers receive normalized frames and do not link against Realtek code. +- Vendor AI output is optional metadata; RuView retains responsibility for its own validation. + +## Rejected alternatives + +1. **Map Range-FFT directly to `CsiFrame`.** Rejected because range bins and channel-frequency + samples have different axes and physical meaning. +2. **Link the Realtek SDK into the Rust server.** Rejected because it couples host builds to a + proprietary embedded ABI and toolchain. +3. **Wait to define any interface until hardware arrives.** Rejected because the host protocol, + parser safety, replay, and provenance can be developed and reviewed independently. +4. **Replace ESP32 nodes.** Rejected because the modes are complementary and availability differs. + +## Open vendor questions + +- Exact RTL8720F part/board identifier and production availability. +- SDK repository/tag, compiler, RTOS, binary blobs, license, and redistribution permissions. +- Radar initialization/configuration/callback API signatures and threading/ISR constraints. +- CFR and near/far Range-FFT element type, complex ordering, scaling, endianness, and timestamps. +- Whether CFR is calibrated complex data and whether phase remains coherent across frames. +- Maximum report rates, buffer ownership, DMA/cache constraints, and Wi-Fi throughput impact. +- Region/channel enforcement and whether 70 MHz operation is allowed by the supplied firmware. +- Secure boot, signed OTA, unique device identity, and firmware attestation support. + +## Sources + +- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 3 and 10–19, + supplied 2026-07-18. This is product material, not measured RuView validation. +- [Ameba-AIoT/ameba-rtos releases](https://github.com/Ameba-AIoT/ameba-rtos/releases), reviewed + 2026-07-18; public release notes mention CSI and EDCCA APIs but do not document the deck's radar ABI. +- ADR-063 (mmWave sensor fusion), ADR-095/097 (source normalization), and ADR-260/262 (RuField + multimodal event model and live bridge). diff --git a/docs/adr/ADR-264-rtl8720f-radar-wire-protocol.md b/docs/adr/ADR-264-rtl8720f-radar-wire-protocol.md new file mode 100644 index 00000000..07730300 --- /dev/null +++ b/docs/adr/ADR-264-rtl8720f-radar-wire-protocol.md @@ -0,0 +1,140 @@ +# ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports + +- **Status**: proposed +- **Date**: 2026-07-18 +- **Deciders**: ruv +- **Tags**: realtek, rtl8720f, protocol, cfr, range-fft, udp, serial +- **Depends on**: ADR-263 +- **Relates to**: ADR-018, ADR-095, ADR-097, ADR-099, ADR-260 + +## Context + +ADR-263 adopts RTL8720F radar behind an anti-corruption boundary. The Realtek presentation names +CFR, near Range-FFT, far Range-FFT, and interference reports, but does not specify their binary ABI. +RuView needs a stable, testable contract that can be implemented before the vendor SDK arrives and +that will not expose vendor structs, pointer layouts, padding, or callback lifetime rules over the +network. + +ADR-018 already defines ESP32 CSI framing. Reusing its magic or pretending that Realtek radar is an +ESP32 packet would make source detection ambiguous and erase radar-specific calibration metadata. + +## Decision + +Define a new little-endian `RtlRadarFrameV1` envelope with its own magic and explicit payload type. +This is a RuView protocol, not a claim about Realtek's native memory layout. + +### Envelope + +All integer fields are little-endian. Floating-point payloads use IEEE-754 binary32. No C struct is +sent by `memcpy`; firmware serializes each field explicitly. + +| Offset | Size | Field | Meaning | +|---:|---:|---|---| +| 0 | 4 | magic | ASCII `RTR1` (`0x31525452`) | +| 4 | 1 | version | `1` | +| 5 | 1 | report_type | 1 CFR, 2 range-near, 3 range-far, 4 interference, 5 capabilities | +| 6 | 2 | header_len | complete header size, initially 56 | +| 8 | 4 | frame_len | header + payload + CRC | +| 12 | 4 | sequence | wraps modulo 2^32 | +| 16 | 8 | timestamp_us | monotonic device time at acquisition | +| 24 | 8 | device_id | stable pseudonymous identifier, not a MAC address | +| 32 | 4 | center_freq_khz | RF centre frequency | +| 36 | 2 | bandwidth_mhz | 20, 40, or 70 | +| 38 | 2 | flags | calibration/interference/saturation/time-sync flags | +| 40 | 2 | element_count | complex samples or range bins | +| 42 | 1 | element_format | 1 complex-i16, 2 complex-f32, 3 power-u16, 4 power-f32 | +| 43 | 1 | antenna_count | expected to be 1 for the deck's 1T1R configuration | +| 44 | 4 | scale | quantized-to-physical multiplier; `1.0` for float payloads | +| 48 | 4 | bin_spacing | Hz for CFR, metres for Range-FFT | +| 52 | 4 | calibration_id | device calibration revision/hash prefix | +| 56 | variable | payload | determined by type, count, and format | +| final-4 | 4 | crc32 | IEEE CRC-32 over header and payload | + +If vendor evidence shows that 56 bytes is too costly, a later protocol version may introduce a +compact header. V1 favors auditable provenance over premature byte savings. + +### Payload semantics + +- **CFR** contains ordered complex channel-frequency samples. The adapter must know the frequency + origin/order and must not fabricate missing phase. Uncalibrated frames carry the uncalibrated flag + and cannot enter phase-sensitive processing. +- **Range-near/range-far** contains ordered range bins. Near and far are separate report types so + filtering and leakage behavior are never hidden from consumers. +- **Interference** contains a versioned TLV set for channel-busy, detected-during-chirp, estimated + interference power, and packet jitter. Unknown TLVs are skipped by length. +- **Capabilities** is emitted at boot and on request. It declares supported report types, bandwidths, + chirp lengths, maximum elements/report, maximum frame rate, firmware version, and SDK identifier. + +### Transport + +The identical envelope is supported over: + +- UDP datagrams for normal RuView ingestion; +- USB CDC or UART with COBS framing and a zero-byte delimiter; +- file replay as a length-prefixed sequence of envelopes. + +One envelope must fit one UDP datagram. Fragmentation is not part of V1; firmware rejects a profile +whose maximum report exceeds the configured MTU and reports the required size through capabilities. + +### Parser and trust rules + +The host parser: + +1. validates magic, version, lengths, enum values, element count/format multiplication, and CRC + before allocating or decoding the payload; +2. caps frames at 64 KiB and elements at a configured hardware maximum; +3. rejects non-finite float metadata/payload values; +4. tracks sequence gaps and timestamp regressions per device; +5. preserves unknown flags but never interprets them as trusted; +6. attaches transport source, firmware/SDK version, calibration ID, and interference state to + provenance; +7. labels fixture/generated frames as synthetic. + +No vendor-provided presence probability bypasses RuView privacy, provenance, or quality gates. + +## Consequences + +### Positive + +- Firmware, transport, parser, replay, and fusion can evolve independently. +- Fuzzing and golden fixtures require no Realtek SDK or board. +- CFR and Range-FFT retain correct axes and calibration provenance. +- A boot-time capabilities frame makes SDK/API drift observable. + +### Negative + +- Serialization adds CPU and bandwidth overhead compared with dumping a vendor buffer. +- V1 fields may need revision after the actual API and report limits are disclosed. +- UDP provides integrity/error detection, not authenticity or confidentiality. + +### Neutral + +- Authentication can be layered with ADR-032 device identity or a signed RuField receipt without + changing report semantics. +- ESP32 ADR-018 framing remains unchanged. + +## Implementation plan + +1. Add `rtl8720f` types/parser module to `wifi-densepose-hardware` behind no vendor dependency. +2. Add golden CFR, near/far Range-FFT, interference, and capabilities fixtures. +3. Add property/fuzz tests for length arithmetic, enum handling, CRC, and float validation. +4. Add a replay CLI that prints normalized metadata without running inference. +5. Once SDK access exists, implement the embedded serializer and verify captured frames against the + host golden decoder. +6. Revise this proposed ADR with measured element counts, rates, and API names before acceptance. + +## Acceptance criteria + +- Rust encode/decode round-trip for every report type. +- Cross-language golden vector produced by the RTL8720F firmware. +- Zero parser panics over the fuzz corpus and arbitrary byte input. +- Detection of single-bit corruption, truncation, count overflow, timestamp regression, and gaps. +- Captured CFR frequency order and Range-FFT bin spacing verified against vendor documentation and a + measured target. + +## Sources + +- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 11–19, supplied + 2026-07-18. +- ADR-018 (ESP32 framing), ADR-095/097 (hardware normalization), ADR-260 (multimodal event model), + and ADR-263 (platform decision). diff --git a/docs/adr/README.md b/docs/adr/README.md index 94c0f218..7c9e510e 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -1,5 +1,10 @@ # Architecture Decision Records +Latest proposed decisions: + +- [ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports](ADR-264-rtl8720f-radar-wire-protocol.md) +- [ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md) + This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.) ## Why ADRs?