mirror of
https://github.com/ruvnet/RuView
synced 2026-07-20 17:03:24 +00:00
docs(adr): plan RTL8720F radar SDK integration
This commit is contained in:
ruv
@@ -0,0 +1,161 @@
|
||||
# ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform
|
||||
|
||||
- **Status**: proposed
|
||||
- **Date**: 2026-07-18
|
||||
- **Deciders**: ruv
|
||||
- **Tags**: realtek, rtl8720f, ameba, fmcw, radar, cfr, csi, hardware
|
||||
- **Relates to**: ADR-018, ADR-063, ADR-064, ADR-095, ADR-097, ADR-260, ADR-262
|
||||
|
||||
## Context
|
||||
|
||||
Realtek's `RTL8720F-2.4G-Radar-Advantages_EN.pptx` describes an RTL8720F mode that shares the
|
||||
2.4 GHz radio between Wi-Fi, Bluetooth, and an active FMCW radar. It offers two data products that
|
||||
are useful to RuView:
|
||||
|
||||
1. **CFR (Channel Frequency Report)**, described by Realtek as the same concept as Wi-Fi CSI.
|
||||
2. **Near and far Range-FFT reports**, preserving near-field content while extending observation to
|
||||
approximately 5–6 m.
|
||||
|
||||
The proposed radio uses one transmit and one receive antenna, 20/40/70 MHz sweeps, configurable
|
||||
8/16/32/64 microsecond chirp symbols, a maximum 2.56 ms FMCW packet, and a configurable frame
|
||||
interval above 15 ms. The deck recommends 40 MHz outside Japan and 20 MHz in Japan. It also
|
||||
describes EDCCA/CTS channel access, Wi-Fi/BT/radar time division, interference reporting, and
|
||||
priority arbitration in the driver.
|
||||
|
||||
This is not a drop-in replacement for ESP32 CSI:
|
||||
|
||||
- it is **active monostatic FMCW**, while the ESP32 path observes Wi-Fi packet CSI;
|
||||
- one Tx/one Rx has no angle-of-arrival or native multi-target separation;
|
||||
- the stated 40 MHz range resolution is about 3.15 m, despite a finer 0.59 m Range-FFT report step;
|
||||
- the presentation is a capability description, not an SDK contract. It contains no header names,
|
||||
function signatures, callback ABI, binary layouts, toolchain version, licensing terms, or public
|
||||
RTL8720F board package.
|
||||
|
||||
Realtek's public Ameba RTOS repository is a plausible base and its public release history includes
|
||||
CSI and EDCCA APIs, but the reviewed public material does not establish that the RTL8720F radar
|
||||
API or firmware blobs are publicly available. Therefore an honest integration must be split at a
|
||||
vendor boundary.
|
||||
|
||||
## Decision
|
||||
|
||||
RuView will support RTL8720F radar as an **optional, capability-negotiated source**, without
|
||||
replacing the ESP32 firmware or treating radar CFR as byte-compatible with ADR-018 CSI.
|
||||
|
||||
The integration has three layers:
|
||||
|
||||
1. **Realtek device firmware**: a small application built in the vendor-supported Ameba SDK calls
|
||||
the radar API, owns coexistence configuration, and emits versioned reports. This code lives under
|
||||
`firmware/rtl8720f-radar/` only after the redistributable SDK/API is available.
|
||||
2. **Transport-neutral wire contract**: CFR and Range-FFT reports are framed independently from the
|
||||
vendor ABI and sent over UDP, USB CDC, or UART. ADR-264 defines this boundary.
|
||||
3. **Rust host adapter**: `wifi-densepose-hardware` parses reports from bytes and converts CFR into
|
||||
the existing CSI-domain representation, while Range-FFT remains a radar modality and feeds the
|
||||
RuField/RuView cross-modality bridge from ADR-260/262.
|
||||
|
||||
The two report types remain semantically distinct:
|
||||
|
||||
| RTL8720F output | RuView representation | Permitted use |
|
||||
|---|---|---|
|
||||
| CFR | `CsiFrame` through a Realtek calibration adapter | CSI feature extraction after validation |
|
||||
| Range-FFT near/far | `RadarFrame` / RuField `mmwave_radar`-class event with a 2.4 GHz descriptor | range, motion, presence, fusion |
|
||||
| Vendor AI presence probability | derived observation with model/version provenance | advisory input, never ground truth |
|
||||
| Interference report | quality/provenance metadata | reject, down-weight, or mark contaminated frames |
|
||||
|
||||
The modality registry should eventually distinguish `fmcw_radar_2_4ghz` from `mmwave_radar`; until
|
||||
that RuField schema revision is accepted, the adapter must attach `carrier_hz = 2.4e9` and must not
|
||||
claim millimetre-wave provenance.
|
||||
|
||||
## Delivery phases and gates
|
||||
|
||||
### P0 — Vendor enablement
|
||||
|
||||
Obtain a redistributable RTL8720F SDK package, radar API headers/libraries, a supported evaluation
|
||||
board, flashing/debug instructions, report definitions, and written redistribution terms.
|
||||
|
||||
**Gate:** compile and run Realtek's unmodified radar example and capture CFR plus near/far
|
||||
Range-FFT output. Until this passes, device firmware is `VENDOR_BLOCKED`, not implemented.
|
||||
|
||||
### P1 — Host-first contract
|
||||
|
||||
Implement ADR-264 types, parsers, fixtures, fuzz tests, and replay support without linking vendor
|
||||
code. Generate deterministic synthetic fixtures whose provenance is explicitly synthetic.
|
||||
|
||||
**Gate:** malformed inputs never panic; encode/decode round trips; unknown versions and report
|
||||
types fail closed.
|
||||
|
||||
### P2 — RTL8720F firmware adapter
|
||||
|
||||
Wrap only the minimum vendor API surface: initialization, profile configuration, start/stop,
|
||||
callback acquisition, interference status, and report serialization. Keep vendor types out of the
|
||||
wire protocol.
|
||||
|
||||
**Gate:** 30-minute simultaneous Wi-Fi telemetry and radar capture with no watchdog reset, bounded
|
||||
loss, monotonic sequence numbers, and explicit coexistence/interference statistics.
|
||||
|
||||
### P3 — Calibration and signal validation
|
||||
|
||||
Calibrate CFR phase/amplitude, Range-FFT bin spacing, static leakage, and clock drift. Compare
|
||||
reported range against measured targets at multiple distances and bandwidths.
|
||||
|
||||
**Gate:** publish measured error distributions. Do not infer accuracy from report-bin spacing and
|
||||
do not advertise multi-person pose or vital signs from the vendor deck.
|
||||
|
||||
### P4 — Fusion and productization
|
||||
|
||||
Feed calibrated CFR through the CSI path and Range-FFT through RuField, retaining source, mode,
|
||||
bandwidth, calibration, firmware, and interference provenance.
|
||||
|
||||
**Gate:** ablation shows whether the radar stream improves a named RuView metric over ESP32 CSI
|
||||
alone. If it does not, ship it only as an independent presence/range sensor.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- One low-cost radio can provide active radar and CSI-like CFR while retaining Wi-Fi connectivity.
|
||||
- Range-FFT adds an independent physical measurement for presence/range fusion.
|
||||
- The vendor SDK is isolated from the Rust sensing core and from the stable on-wire contract.
|
||||
- Capability negotiation permits future Realtek parts without another application-level fork.
|
||||
|
||||
### Negative
|
||||
|
||||
- The first implementation is blocked on access to the actual RTL8720F radar SDK/API and hardware.
|
||||
- Active 2.4 GHz transmission changes coexistence, privacy, power, and regional compliance concerns.
|
||||
- 1T1R and limited sweep bandwidth cannot provide the spatial resolution of multi-antenna mmWave.
|
||||
- A second embedded toolchain and firmware release process must be maintained.
|
||||
|
||||
### Neutral
|
||||
|
||||
- ESP32 remains the default CSI node.
|
||||
- Existing consumers receive normalized frames and do not link against Realtek code.
|
||||
- Vendor AI output is optional metadata; RuView retains responsibility for its own validation.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
1. **Map Range-FFT directly to `CsiFrame`.** Rejected because range bins and channel-frequency
|
||||
samples have different axes and physical meaning.
|
||||
2. **Link the Realtek SDK into the Rust server.** Rejected because it couples host builds to a
|
||||
proprietary embedded ABI and toolchain.
|
||||
3. **Wait to define any interface until hardware arrives.** Rejected because the host protocol,
|
||||
parser safety, replay, and provenance can be developed and reviewed independently.
|
||||
4. **Replace ESP32 nodes.** Rejected because the modes are complementary and availability differs.
|
||||
|
||||
## Open vendor questions
|
||||
|
||||
- Exact RTL8720F part/board identifier and production availability.
|
||||
- SDK repository/tag, compiler, RTOS, binary blobs, license, and redistribution permissions.
|
||||
- Radar initialization/configuration/callback API signatures and threading/ISR constraints.
|
||||
- CFR and near/far Range-FFT element type, complex ordering, scaling, endianness, and timestamps.
|
||||
- Whether CFR is calibrated complex data and whether phase remains coherent across frames.
|
||||
- Maximum report rates, buffer ownership, DMA/cache constraints, and Wi-Fi throughput impact.
|
||||
- Region/channel enforcement and whether 70 MHz operation is allowed by the supplied firmware.
|
||||
- Secure boot, signed OTA, unique device identity, and firmware attestation support.
|
||||
|
||||
## Sources
|
||||
|
||||
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 3 and 10–19,
|
||||
supplied 2026-07-18. This is product material, not measured RuView validation.
|
||||
- [Ameba-AIoT/ameba-rtos releases](https://github.com/Ameba-AIoT/ameba-rtos/releases), reviewed
|
||||
2026-07-18; public release notes mention CSI and EDCCA APIs but do not document the deck's radar ABI.
|
||||
- ADR-063 (mmWave sensor fusion), ADR-095/097 (source normalization), and ADR-260/262 (RuField
|
||||
multimodal event model and live bridge).
|
||||
@@ -0,0 +1,140 @@
|
||||
# ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports
|
||||
|
||||
- **Status**: proposed
|
||||
- **Date**: 2026-07-18
|
||||
- **Deciders**: ruv
|
||||
- **Tags**: realtek, rtl8720f, protocol, cfr, range-fft, udp, serial
|
||||
- **Depends on**: ADR-263
|
||||
- **Relates to**: ADR-018, ADR-095, ADR-097, ADR-099, ADR-260
|
||||
|
||||
## Context
|
||||
|
||||
ADR-263 adopts RTL8720F radar behind an anti-corruption boundary. The Realtek presentation names
|
||||
CFR, near Range-FFT, far Range-FFT, and interference reports, but does not specify their binary ABI.
|
||||
RuView needs a stable, testable contract that can be implemented before the vendor SDK arrives and
|
||||
that will not expose vendor structs, pointer layouts, padding, or callback lifetime rules over the
|
||||
network.
|
||||
|
||||
ADR-018 already defines ESP32 CSI framing. Reusing its magic or pretending that Realtek radar is an
|
||||
ESP32 packet would make source detection ambiguous and erase radar-specific calibration metadata.
|
||||
|
||||
## Decision
|
||||
|
||||
Define a new little-endian `RtlRadarFrameV1` envelope with its own magic and explicit payload type.
|
||||
This is a RuView protocol, not a claim about Realtek's native memory layout.
|
||||
|
||||
### Envelope
|
||||
|
||||
All integer fields are little-endian. Floating-point payloads use IEEE-754 binary32. No C struct is
|
||||
sent by `memcpy`; firmware serializes each field explicitly.
|
||||
|
||||
| Offset | Size | Field | Meaning |
|
||||
|---:|---:|---|---|
|
||||
| 0 | 4 | magic | ASCII `RTR1` (`0x31525452`) |
|
||||
| 4 | 1 | version | `1` |
|
||||
| 5 | 1 | report_type | 1 CFR, 2 range-near, 3 range-far, 4 interference, 5 capabilities |
|
||||
| 6 | 2 | header_len | complete header size, initially 56 |
|
||||
| 8 | 4 | frame_len | header + payload + CRC |
|
||||
| 12 | 4 | sequence | wraps modulo 2^32 |
|
||||
| 16 | 8 | timestamp_us | monotonic device time at acquisition |
|
||||
| 24 | 8 | device_id | stable pseudonymous identifier, not a MAC address |
|
||||
| 32 | 4 | center_freq_khz | RF centre frequency |
|
||||
| 36 | 2 | bandwidth_mhz | 20, 40, or 70 |
|
||||
| 38 | 2 | flags | calibration/interference/saturation/time-sync flags |
|
||||
| 40 | 2 | element_count | complex samples or range bins |
|
||||
| 42 | 1 | element_format | 1 complex-i16, 2 complex-f32, 3 power-u16, 4 power-f32 |
|
||||
| 43 | 1 | antenna_count | expected to be 1 for the deck's 1T1R configuration |
|
||||
| 44 | 4 | scale | quantized-to-physical multiplier; `1.0` for float payloads |
|
||||
| 48 | 4 | bin_spacing | Hz for CFR, metres for Range-FFT |
|
||||
| 52 | 4 | calibration_id | device calibration revision/hash prefix |
|
||||
| 56 | variable | payload | determined by type, count, and format |
|
||||
| final-4 | 4 | crc32 | IEEE CRC-32 over header and payload |
|
||||
|
||||
If vendor evidence shows that 56 bytes is too costly, a later protocol version may introduce a
|
||||
compact header. V1 favors auditable provenance over premature byte savings.
|
||||
|
||||
### Payload semantics
|
||||
|
||||
- **CFR** contains ordered complex channel-frequency samples. The adapter must know the frequency
|
||||
origin/order and must not fabricate missing phase. Uncalibrated frames carry the uncalibrated flag
|
||||
and cannot enter phase-sensitive processing.
|
||||
- **Range-near/range-far** contains ordered range bins. Near and far are separate report types so
|
||||
filtering and leakage behavior are never hidden from consumers.
|
||||
- **Interference** contains a versioned TLV set for channel-busy, detected-during-chirp, estimated
|
||||
interference power, and packet jitter. Unknown TLVs are skipped by length.
|
||||
- **Capabilities** is emitted at boot and on request. It declares supported report types, bandwidths,
|
||||
chirp lengths, maximum elements/report, maximum frame rate, firmware version, and SDK identifier.
|
||||
|
||||
### Transport
|
||||
|
||||
The identical envelope is supported over:
|
||||
|
||||
- UDP datagrams for normal RuView ingestion;
|
||||
- USB CDC or UART with COBS framing and a zero-byte delimiter;
|
||||
- file replay as a length-prefixed sequence of envelopes.
|
||||
|
||||
One envelope must fit one UDP datagram. Fragmentation is not part of V1; firmware rejects a profile
|
||||
whose maximum report exceeds the configured MTU and reports the required size through capabilities.
|
||||
|
||||
### Parser and trust rules
|
||||
|
||||
The host parser:
|
||||
|
||||
1. validates magic, version, lengths, enum values, element count/format multiplication, and CRC
|
||||
before allocating or decoding the payload;
|
||||
2. caps frames at 64 KiB and elements at a configured hardware maximum;
|
||||
3. rejects non-finite float metadata/payload values;
|
||||
4. tracks sequence gaps and timestamp regressions per device;
|
||||
5. preserves unknown flags but never interprets them as trusted;
|
||||
6. attaches transport source, firmware/SDK version, calibration ID, and interference state to
|
||||
provenance;
|
||||
7. labels fixture/generated frames as synthetic.
|
||||
|
||||
No vendor-provided presence probability bypasses RuView privacy, provenance, or quality gates.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Firmware, transport, parser, replay, and fusion can evolve independently.
|
||||
- Fuzzing and golden fixtures require no Realtek SDK or board.
|
||||
- CFR and Range-FFT retain correct axes and calibration provenance.
|
||||
- A boot-time capabilities frame makes SDK/API drift observable.
|
||||
|
||||
### Negative
|
||||
|
||||
- Serialization adds CPU and bandwidth overhead compared with dumping a vendor buffer.
|
||||
- V1 fields may need revision after the actual API and report limits are disclosed.
|
||||
- UDP provides integrity/error detection, not authenticity or confidentiality.
|
||||
|
||||
### Neutral
|
||||
|
||||
- Authentication can be layered with ADR-032 device identity or a signed RuField receipt without
|
||||
changing report semantics.
|
||||
- ESP32 ADR-018 framing remains unchanged.
|
||||
|
||||
## Implementation plan
|
||||
|
||||
1. Add `rtl8720f` types/parser module to `wifi-densepose-hardware` behind no vendor dependency.
|
||||
2. Add golden CFR, near/far Range-FFT, interference, and capabilities fixtures.
|
||||
3. Add property/fuzz tests for length arithmetic, enum handling, CRC, and float validation.
|
||||
4. Add a replay CLI that prints normalized metadata without running inference.
|
||||
5. Once SDK access exists, implement the embedded serializer and verify captured frames against the
|
||||
host golden decoder.
|
||||
6. Revise this proposed ADR with measured element counts, rates, and API names before acceptance.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Rust encode/decode round-trip for every report type.
|
||||
- Cross-language golden vector produced by the RTL8720F firmware.
|
||||
- Zero parser panics over the fuzz corpus and arbitrary byte input.
|
||||
- Detection of single-bit corruption, truncation, count overflow, timestamp regression, and gaps.
|
||||
- Captured CFR frequency order and Range-FFT bin spacing verified against vendor documentation and a
|
||||
measured target.
|
||||
|
||||
## Sources
|
||||
|
||||
- Realtek Semiconductor, `RTL8720F-2.4G-Radar-Advantages_EN.pptx`, slides 11–19, supplied
|
||||
2026-07-18.
|
||||
- ADR-018 (ESP32 framing), ADR-095/097 (hardware normalization), ADR-260 (multimodal event model),
|
||||
and ADR-263 (platform decision).
|
||||
@@ -1,5 +1,10 @@
|
||||
# Architecture Decision Records
|
||||
|
||||
Latest proposed decisions:
|
||||
|
||||
- [ADR-264: Versioned wire protocol for RTL8720F CFR and Range-FFT reports](ADR-264-rtl8720f-radar-wire-protocol.md)
|
||||
- [ADR-263: Adopt RTL8720F 2.4 GHz FMCW radar as an optional RuView sensing platform](ADR-263-rtl8720f-2-4ghz-fmcw-radar-platform.md)
|
||||
|
||||
This folder contains 182 Architecture Decision Records (ADRs) that document every significant technical choice in the RuView / WiFi-DensePose project. (The index tables below list a curated subset per domain; see the directory listing for the full set.)
|
||||
|
||||
## Why ADRs?
|
||||
|
||||
Reference in New Issue
Block a user