mirror of
https://github.com/ruvnet/RuView
synced 2026-08-02 19:11:46 +00:00
docs(adr): plan RTL8720F radar SDK integration
This commit is contained in:
@@ -0,0 +1,161 @@
|
|||||||
|
# ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform
|
||||||
|
|
||||||
|
- **Status**: proposed
|
||||||
|
- **Date**: 2026-07-18
|
||||||
|
- **Deciders**: ruv
|
||||||
|
- **Tags**: realtek, rtl8720f, ameba, fmcw, radar, cfr, csi, hardware
|
||||||
|
- **Relates to**: ADR-018, ADR-063, ADR-064, ADR-095, ADR-097, ADR-260, ADR-262
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Realtek's `RTL8720F-2.4G-Radar-Advantages_EN.pptx` describes an RTL8720F mode that shares the
|
||||||
|
2.4 GHz radio between Wi-Fi, Bluetooth, and an active FMCW radar. It offers two data products that
|
||||||
|
are useful to RuView:
|
||||||
|
|
||||||
|
1. **CFR (Channel Frequency Report)**, described by Realtek as the same concept as Wi-Fi CSI.
|
||||||
|
2. **Near and far Range-FFT reports**, preserving near-field content while extending observation to
|
||||||
|
approximately 5–6 m.
|
||||||
|
|
||||||
|
The proposed radio uses one transmit and one receive antenna, 20/40/70 MHz sweeps, configurable
|
||||||
|
8/16/32/64 microsecond chirp symbols, a maximum 2.56 ms FMCW packet, and a configurable frame
|
||||||
|
interval above 15 ms. The deck recommends 40 MHz outside Japan and 20 MHz in Japan. It also
|
||||||
|
describes EDCCA/CTS channel access, Wi-Fi/BT/radar time division, interference reporting, and
|
||||||
|
priority arbitration in the driver.
|
||||||
|
|
||||||
|
This is not a drop-in replacement for ESP32 CSI:
|
||||||
|
|
||||||
|
- it is **active monostatic FMCW**, while the ESP32 path observes Wi-Fi packet CSI;
|
||||||
|
- one Tx/one Rx has no angle-of-arrival or native multi-target separation;
|
||||||
|
- the stated 40 MHz range resolution is about 3.15 m, despite a finer 0.59 m Range-FFT report step;
|
||||||
|
- the presentation is a capability description, not an SDK contract. It contains no header names,
|
||||||
|
function signatures, callback ABI, binary layouts, toolchain version, licensing terms, or public
|
||||||
|
RTL8720F board package.
|
||||||
|
|
||||||
|
Realtek's public Ameba RTOS repository is a plausible base and its public release history includes
|
||||||
|
CSI and EDCCA APIs, but the reviewed public material does not establish that the RTL8720F radar
|
||||||
|
API or firmware blobs are publicly available. Therefore an honest integration must be split at a
|
||||||
|
vendor boundary.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
RuView will support RTL8720F radar as an **optional, capability-negotiated source**, without
|
||||||
|
replacing the ESP32 firmware or treating radar CFR as byte-compatible with ADR-018 CSI.
|
||||||
|
|
||||||
|
The integration has three layers:
|
||||||
|
|
||||||
|
1. **Realtek device firmware**: a small application built in the vendor-supported Ameba SDK calls
|
||||||
|
the radar API, owns coexistence configuration, and emits versioned reports. This code lives under
|
||||||
|
`firmware/rtl8720f-radar/` only after the redistributable SDK/API is available.
|
||||||
|
2. **Transport-neutral wire contract**: CFR and Range-FFT reports are framed independently from the
|
||||||
|
vendor ABI and sent over UDP, USB CDC, or UART. ADR-264 defines this boundary.
|
||||||
|
3. **Rust host adapter**: `wifi-densepose-hardware` parses reports from bytes and converts CFR into
|
||||||
|
the existing CSI-domain representation, while Range-FFT remains a radar modality and feeds the
|
||||||
|
RuField/RuView cross-modality bridge from ADR-260/262.
|
||||||
|
|
||||||
|
The two report types remain semantically distinct:
|
||||||
|
|
||||||
|
| RTL8720F output | RuView representation | Permitted use |
|
||||||
|
|---|---|---|
|
||||||
|
| CFR | `CsiFrame` through a Realtek calibration adapter | CSI feature extraction after validation |
|
||||||
|
| Range-FFT near/far | `RadarFrame` / RuField `mmwave_radar`-class event with a 2.4 GHz descriptor | range, motion, presence, fusion |
|
||||||
|
| Vendor AI presence probability | derived observation with model/version provenance | advisory input, never ground truth |
|
||||||
|
| Interference report | quality/provenance metadata | reject, down-weight, or mark contaminated frames |
|
||||||
|
|
||||||
|
The modality registry should eventually distinguish `fmcw_radar_2_4ghz` from `mmwave_radar`; until
|
||||||
|
that RuField schema revision is accepted, the adapter must attach `carrier_hz = 2.4e9` and must not
|
||||||
|
claim millimetre-wave provenance.
|
||||||
|
|
||||||
|
## Delivery phases and gates
|
||||||
|
|
||||||
|
### P0 — Vendor enablement
|
||||||
|
|
||||||
|
Obtain a redistributable RTL8720F SDK package, radar API headers/libraries, a supported evaluation
|
||||||
|
board, flashing/debug instructions, report definitions, and written redistribution terms.
|
||||||
|
|
||||||
|
**Gate:** compile and run Realtek's unmodified radar example and capture CFR plus near/far
|
||||||
|
Range-FFT output. Until this passes, device firmware is `VENDOR_BLOCKED`, not implemented.
|
||||||
|
|
||||||
|
### P1 — Host-first contract
|
||||||
|
|
||||||
|
Implement ADR-264 types, parsers, fixtures, fuzz tests, and replay support without linking vendor
|
||||||
|
code. Generate deterministic synthetic fixtures whose provenance is explicitly synthetic.
|
||||||
|
|
||||||
|
**Gate:** malformed inputs never panic; encode/decode round trips; unknown versions and report
|
||||||
|
types fail closed.
|
||||||
|
|
||||||
|
### P2 — RTL8720F firmware adapter
|
||||||
|
|
||||||
|
Wrap only the minimum vendor API surface: initialization, profile configuration, start/stop,
|
||||||
|
callback acquisition, interference status, and report serialization. Keep vendor types out of the
|
||||||
|
wire protocol.
|
||||||
|
|
||||||
|
**Gate:** 30-minute simultaneous Wi-Fi telemetry and radar capture with no watchdog reset, bounded
|
||||||
|
loss, monotonic sequence numbers, and explicit coexistence/interference statistics.
|
||||||
|
|
||||||
|
### P3 — Calibration and signal validation
|
||||||
|
|
||||||
|
Calibrate CFR phase/amplitude, Range-FFT bin spacing, static leakage, and clock drift. Compare
|
||||||
|
reported range against measured targets at multiple distances and bandwidths.
|
||||||
|
|
||||||
|
**Gate:** publish measured error distributions. Do not infer accuracy from report-bin spacing and
|
||||||
|
do not advertise multi-person pose or vital signs from the vendor deck.
|
||||||
|
|
||||||
|
### P4 — Fusion and productization
|
||||||
|
|
||||||
|
Feed calibrated CFR through the CSI path and Range-FFT through RuField, retaining source, mode,
|
||||||
|
bandwidth, calibration, firmware, and interference provenance.
|
||||||
|
|
||||||
|
**Gate:** ablation shows whether the radar stream improves a named RuView metric over ESP32 CSI
|
||||||
|
alone. If it does not, ship it only as an independent presence/range sensor.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
|
||||||
|
- One low-cost radio can provide active radar and CSI-like CFR while retaining Wi-Fi connectivity.
|
||||||
|
- Range-FFT adds an independent physical measurement for presence/range fusion.
|
||||||
|
- The vendor SDK is isolated from the Rust sensing core and from the stable on-wire contract.
|
||||||
|
- Capability negotiation permits future Realtek parts without another application-level fork.
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
|
||||||
|
- The first implementation is blocked on access to the actual RTL8720F radar SDK/API and hardware.
|
||||||
|
- Active 2.4 GHz transmission changes coexistence, privacy, power, and regional compliance concerns.
|
||||||
|
- 1T1R and limited sweep bandwidth cannot provide the spatial resolution of multi-antenna mmWave.
|
||||||
|
- A second embedded toolchain and firmware release process must be maintained.
|
||||||
|
|
||||||
|
### Neutral
|
||||||
|
|
||||||
|
- ESP32 remains the default CSI node.
|
||||||
|
- Existing consumers receive normalized frames and do not link against Realtek code.
|
||||||
|
- Vendor AI output is optional metadata; RuView retains responsibility for its own validation.
|
||||||
|
|
||||||
|
## Rejected alternatives
|
||||||
|
|
||||||
|
1. **Map Range-FFT directly to `CsiFrame`.** Rejected because range bins and channel-frequency
|
||||||
|
samples have different axes and physical meaning.
|
||||||
|
2. **Link the Realtek SDK into the Rust server.** Rejected because it couples host builds to a
|
||||||
|
proprietary embedded ABI and toolchain.
|
||||||
|
3. **Wait to define any interface until hardware arrives.** Rejected because the host protocol,
|
||||||
|
parser safety, replay, and provenance can be developed and reviewed independently.
|
||||||
|
4. **Replace ESP32 nodes.** Rejected because the modes are complementary and availability differs.
|
||||||
|
|
||||||
|
## Open vendor questions
|
||||||
|
|
||||||
|
- Exact RTL8720F part/board identifier and production availability.
|
||||||
|
- SDK repository/tag, compiler, RTOS, binary blobs, license, and redistribution permissions.
|
||||||
|
- Radar initialization/configuration/callback API signatures and threading/ISR constraints.
|
||||||
|
- CFR and near/far Range-FFT element type, complex ordering, scaling, endianness, and timestamps.
|
||||||
|
- Whether CFR is calibrated complex data and whether phase remains coherent across frames.
|
||||||
|
- Maximum report rates, buffer ownership, DMA/cache constraints, and Wi-Fi throughput impact.
|
||||||
|
- Region/channel enforcement and whether 70 MHz operation is allowed by the supplied firmware.
|
||||||
|
- Secure boot, signed OTA, unique device identity, and firmware attestation support.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 3 and 10–19,
|
||||||
|
supplied 2026-07-18. This is product material, not measured RuView validation.
|
||||||
|
- [Ameba-AIoT/ameba-rtos releases](https://github.com/Ameba-AIoT/ameba-rtos/releases), reviewed
|
||||||
|
2026-07-18; public release notes mention CSI and EDCCA APIs but do not document the deck's radar ABI.
|
||||||
|
- ADR-063 (mmWave sensor fusion), ADR-095/097 (source normalization), and ADR-260/262 (RuField
|
||||||
|
multimodal event model and live bridge).
|
||||||
@@ -0,0 +1,140 @@
|
|||||||
|
# ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports
|
||||||
|
|
||||||
|
- **Status**: proposed
|
||||||
|
- **Date**: 2026-07-18
|
||||||
|
- **Deciders**: ruv
|
||||||
|
- **Tags**: realtek, rtl8720f, protocol, cfr, range-fft, udp, serial
|
||||||
|
- **Depends on**: ADR-263
|
||||||
|
- **Relates to**: ADR-018, ADR-095, ADR-097, ADR-099, ADR-260
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
ADR-263 adopts RTL8720F radar behind an anti-corruption boundary. The Realtek presentation names
|
||||||
|
CFR, near Range-FFT, far Range-FFT, and interference reports, but does not specify their binary ABI.
|
||||||
|
RuView needs a stable, testable contract that can be implemented before the vendor SDK arrives and
|
||||||
|
that will not expose vendor structs, pointer layouts, padding, or callback lifetime rules over the
|
||||||
|
network.
|
||||||
|
|
||||||
|
ADR-018 already defines ESP32 CSI framing. Reusing its magic or pretending that Realtek radar is an
|
||||||
|
ESP32 packet would make source detection ambiguous and erase radar-specific calibration metadata.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Define a new little-endian `RtlRadarFrameV1` envelope with its own magic and explicit payload type.
|
||||||
|
This is a RuView protocol, not a claim about Realtek's native memory layout.
|
||||||
|
|
||||||
|
### Envelope
|
||||||
|
|
||||||
|
All integer fields are little-endian. Floating-point payloads use IEEE-754 binary32. No C struct is
|
||||||
|
sent by `memcpy`; firmware serializes each field explicitly.
|
||||||
|
|
||||||
|
| Offset | Size | Field | Meaning |
|
||||||
|
|---:|---:|---|---|
|
||||||
|
| 0 | 4 | magic | ASCII `RTR1` (`0x31525452`) |
|
||||||
|
| 4 | 1 | version | `1` |
|
||||||
|
| 5 | 1 | report_type | 1 CFR, 2 range-near, 3 range-far, 4 interference, 5 capabilities |
|
||||||
|
| 6 | 2 | header_len | complete header size, initially 56 |
|
||||||
|
| 8 | 4 | frame_len | header + payload + CRC |
|
||||||
|
| 12 | 4 | sequence | wraps modulo 2^32 |
|
||||||
|
| 16 | 8 | timestamp_us | monotonic device time at acquisition |
|
||||||
|
| 24 | 8 | device_id | stable pseudonymous identifier, not a MAC address |
|
||||||
|
| 32 | 4 | center_freq_khz | RF centre frequency |
|
||||||
|
| 36 | 2 | bandwidth_mhz | 20, 40, or 70 |
|
||||||
|
| 38 | 2 | flags | calibration/interference/saturation/time-sync flags |
|
||||||
|
| 40 | 2 | element_count | complex samples or range bins |
|
||||||
|
| 42 | 1 | element_format | 1 complex-i16, 2 complex-f32, 3 power-u16, 4 power-f32 |
|
||||||
|
| 43 | 1 | antenna_count | expected to be 1 for the deck's 1T1R configuration |
|
||||||
|
| 44 | 4 | scale | quantized-to-physical multiplier; `1.0` for float payloads |
|
||||||
|
| 48 | 4 | bin_spacing | Hz for CFR, metres for Range-FFT |
|
||||||
|
| 52 | 4 | calibration_id | device calibration revision/hash prefix |
|
||||||
|
| 56 | variable | payload | determined by type, count, and format |
|
||||||
|
| final-4 | 4 | crc32 | IEEE CRC-32 over header and payload |
|
||||||
|
|
||||||
|
If vendor evidence shows that 56 bytes is too costly, a later protocol version may introduce a
|
||||||
|
compact header. V1 favors auditable provenance over premature byte savings.
|
||||||
|
|
||||||
|
### Payload semantics
|
||||||
|
|
||||||
|
- **CFR** contains ordered complex channel-frequency samples. The adapter must know the frequency
|
||||||
|
origin/order and must not fabricate missing phase. Uncalibrated frames carry the uncalibrated flag
|
||||||
|
and cannot enter phase-sensitive processing.
|
||||||
|
- **Range-near/range-far** contains ordered range bins. Near and far are separate report types so
|
||||||
|
filtering and leakage behavior are never hidden from consumers.
|
||||||
|
- **Interference** contains a versioned TLV set for channel-busy, detected-during-chirp, estimated
|
||||||
|
interference power, and packet jitter. Unknown TLVs are skipped by length.
|
||||||
|
- **Capabilities** is emitted at boot and on request. It declares supported report types, bandwidths,
|
||||||
|
chirp lengths, maximum elements/report, maximum frame rate, firmware version, and SDK identifier.
|
||||||
|
|
||||||
|
### Transport
|
||||||
|
|
||||||
|
The identical envelope is supported over:
|
||||||
|
|
||||||
|
- UDP datagrams for normal RuView ingestion;
|
||||||
|
- USB CDC or UART with COBS framing and a zero-byte delimiter;
|
||||||
|
- file replay as a length-prefixed sequence of envelopes.
|
||||||
|
|
||||||
|
One envelope must fit one UDP datagram. Fragmentation is not part of V1; firmware rejects a profile
|
||||||
|
whose maximum report exceeds the configured MTU and reports the required size through capabilities.
|
||||||
|
|
||||||
|
### Parser and trust rules
|
||||||
|
|
||||||
|
The host parser:
|
||||||
|
|
||||||
|
1. validates magic, version, lengths, enum values, element count/format multiplication, and CRC
|
||||||
|
before allocating or decoding the payload;
|
||||||
|
2. caps frames at 64 KiB and elements at a configured hardware maximum;
|
||||||
|
3. rejects non-finite float metadata/payload values;
|
||||||
|
4. tracks sequence gaps and timestamp regressions per device;
|
||||||
|
5. preserves unknown flags but never interprets them as trusted;
|
||||||
|
6. attaches transport source, firmware/SDK version, calibration ID, and interference state to
|
||||||
|
provenance;
|
||||||
|
7. labels fixture/generated frames as synthetic.
|
||||||
|
|
||||||
|
No vendor-provided presence probability bypasses RuView privacy, provenance, or quality gates.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
### Positive
|
||||||
|
|
||||||
|
- Firmware, transport, parser, replay, and fusion can evolve independently.
|
||||||
|
- Fuzzing and golden fixtures require no Realtek SDK or board.
|
||||||
|
- CFR and Range-FFT retain correct axes and calibration provenance.
|
||||||
|
- A boot-time capabilities frame makes SDK/API drift observable.
|
||||||
|
|
||||||
|
### Negative
|
||||||
|
|
||||||
|
- Serialization adds CPU and bandwidth overhead compared with dumping a vendor buffer.
|
||||||
|
- V1 fields may need revision after the actual API and report limits are disclosed.
|
||||||
|
- UDP provides integrity/error detection, not authenticity or confidentiality.
|
||||||
|
|
||||||
|
### Neutral
|
||||||
|
|
||||||
|
- Authentication can be layered with ADR-032 device identity or a signed RuField receipt without
|
||||||
|
changing report semantics.
|
||||||
|
- ESP32 ADR-018 framing remains unchanged.
|
||||||
|
|
||||||
|
## Implementation plan
|
||||||
|
|
||||||
|
1. Add `rtl8720f` types/parser module to `wifi-densepose-hardware` behind no vendor dependency.
|
||||||
|
2. Add golden CFR, near/far Range-FFT, interference, and capabilities fixtures.
|
||||||
|
3. Add property/fuzz tests for length arithmetic, enum handling, CRC, and float validation.
|
||||||
|
4. Add a replay CLI that prints normalized metadata without running inference.
|
||||||
|
5. Once SDK access exists, implement the embedded serializer and verify captured frames against the
|
||||||
|
host golden decoder.
|
||||||
|
6. Revise this proposed ADR with measured element counts, rates, and API names before acceptance.
|
||||||
|
|
||||||
|
## Acceptance criteria
|
||||||
|
|
||||||
|
- Rust encode/decode round-trip for every report type.
|
||||||
|
- Cross-language golden vector produced by the RTL8720F firmware.
|
||||||
|
- Zero parser panics over the fuzz corpus and arbitrary byte input.
|
||||||
|
- Detection of single-bit corruption, truncation, count overflow, timestamp regression, and gaps.
|
||||||
|
- Captured CFR frequency order and Range-FFT bin spacing verified against vendor documentation and a
|
||||||
|
measured target.
|
||||||
|
|
||||||
|
## Sources
|
||||||
|
|
||||||
|
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 11–19, supplied
|
||||||
|
2026-07-18.
|
||||||
|
- ADR-018 (ESP32 framing), ADR-095/097 (hardware normalization), ADR-260 (multimodal event model),
|
||||||
|
and ADR-263 (platform decision).
|
||||||
@@ -1,5 +1,10 @@
|
|||||||
# Architecture Decision Records
|
# Architecture Decision Records
|
||||||
|
|
||||||
|
Latest proposed decisions:
|
||||||
|
|
||||||
|
- [ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports](ADR-264-rtl8720f-radar-wire-protocol.md)
|
||||||
|
- [ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md)
|
||||||
|
|
||||||
This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
|
This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
|
||||||
|
|
||||||
## Why ADRs?
|
## Why ADRs?
|
||||||
|
|||||||
Reference in New Issue
Block a user