mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
feat(adr-117): pip wifi-densepose modernization (PIP-PHOENIX) + ruview sibling release (#786)
* docs(adr-117): seed branch — ADR-117 pip-modernization spec + soul-signature research bundle
Two artifacts landing together on this new branch as the prerequisite
documentation for the v2.0.0 Python wheel modernization work:
1. **docs/adr/ADR-117-pip-wifi-densepose-modernization.md** (644 lines)
— Plan to bring the 2025-published `wifi-densepose` PyPI package
(last release v1.1.0, 2025-06-07, 11.5 months out of sync) up to
the current Rust v2/ workspace SOTA. Recommends PyO3 + maturin
with abi3-py310 (one binary covers Python 3.10–3.13 per OS/arch),
first-wheel scope = core + vitals + signal crates (~5 MB), v1.99.0
tombstone + 90-day un-yank window for v1.1.0, v2.0.0 hard break.
Open questions catalogued; phases P1–P6+ laid out with concrete
acceptance criteria.
2. **docs/research/soul/** (5 files, ~1,450 lines) — Soul Signature
research spec: 7-channel electromagnetic biometric fingerprint
(AETHER 128-dim + cardiac HR/HRV + cardiac waveform morphology +
respiratory pattern + gait timing + skeletal proportions +
subcarrier reflection profile), fused into one RVF graph file.
Includes 60s scanning protocol, 5-layer security model,
threat-model + mitigations, references to existing ADRs (014,
021, 024, 027, 030, 039, 079, 106, 108, 109, 110, 115). Marked
"Research Specification (Pre-Implementation)". Explicit "what
this is NOT" disclaimers preempt pseudoscience drift; every
discriminative-power claim either cites a measurement or is
marked "open research; baseline TBD".
Branch off main at HEAD; ready for /loop 10m implementation
iterations.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p1): scaffold python/ workspace — PyO3 + maturin + smoke tests (refs #785)
ADR-117 P1 — the python/ directory is now a working maturin-buildable
crate that produces the v2.x replacement for the legacy pure-Python
wifi-densepose==1.1.0 PyPI wheel.
## What lands
- `python/Cargo.toml` — PyO3 0.22 with `extension-module` + `abi3-py310`
(one binary covers Python 3.10–3.13 per OS/arch — keeps the
cibuildwheel matrix to 5 wheels per release, not 20). Depends on
`wifi-densepose-core` from the existing v2/ workspace via relative
path.
- `python/pyproject.toml` — maturin>=1.7 build backend with
`python-source = "python"` and `module-name = "wifi_densepose._native"`
so the compiled module loads as an internal underscore-private
submodule of the user-facing `wifi_densepose` package. PEP 621
metadata + classifiers + project URLs. Optional-deps:
`wifi-densepose[client]` for the P4 WS/MQTT pure-Python layer,
`wifi-densepose[dev]` for the test toolchain (pytest, ruff, mypy).
- `python/src/lib.rs` — minimal `#[pymodule] wifi_densepose_native`
exporting `__rust_version__`, `__rust_build_tag__`,
`__build_features__`, and a `hello()` smoke function. P2 will land
the core type bindings here.
- `python/wifi_densepose/__init__.py` — pure-Python facade re-exporting
the compiled module's symbols under their stable user-facing names.
Docstring teaches the v1→v2 migration story up-front.
- `python/wifi_densepose/py.typed` — PEP 561 marker so `mypy --strict`
in user code treats the wheel as fully typed (real stubs land in P2).
- `python/tests/test_smoke.py` — 6 P1 acceptance tests:
1. package imports without error
2. version string is PEP 440-compliant
3. `__rust_version__` is reachable from Python (the diagnostic
surface ADR-117 §5.2 promised)
4. `__build_features__` lists `p1-scaffold` marker
5. `wifi_densepose.hello()` returns "ok" (FFI round-trip)
6. `wifi_densepose._native` is reachable but the leading underscore
conveys "private; users should import the parent package"
- `python/README.md` — phase ledger, local build instructions
(`maturin develop`), layout diagram.
## What's deferred to P2+
- Core type bindings (`CsiFrame`, `Keypoint`, `PoseEstimate`) — P2
- Vitals + signal DSP bindings + witness v2 — P3
- Pure-Python WS/MQTT client layer (`wifi_densepose[client]`) — P4
- cibuildwheel + PyPI publish — P5
- v1.99.0 tombstone — concurrent with P5
The new `python/` crate is intentionally OUTSIDE the v2/ Cargo
workspace — it has its own Cargo.toml with `[package]` not
`[workspace.package]` inheritance — to keep maturin's `python-source`
+ `module-name` config self-contained and to avoid forcing every
`cargo test --workspace` invocation in v2/ to compile pyo3.
Refs ADR-117 §5 (Detailed design) and §6 (Phased migration).
Refs #785 (tracking issue).
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(adr-117/p1): standalone Cargo.toml + python-source=. + #[pyo3(name=_native)] (P1 GREEN)
Three fixes to make maturin develop actually work locally:
1. `python/Cargo.toml` removed `*.workspace = true` inheritance —
the python/ crate is intentionally outside the v2/ workspace
(ADR-117 §5.2) so it needs every `[package]` field local.
2. `python/pyproject.toml` `python-source = "python"` was wrong
because pyproject.toml lives at python/ — maturin was looking for
python/python/. Changed to `python-source = "."` so the
`wifi_densepose/` package directory sibling-to-pyproject is found.
3. `python/src/lib.rs` `#[pymodule] fn wifi_densepose_native` →
`#[pymodule] #[pyo3(name = "_native")] fn wifi_densepose_native`.
PyO3 generates `PyInit__native` from the pyo3-name attribute, which
must match the `module-name` in pyproject.toml's [tool.maturin]
block ("wifi_densepose._native"). Without this attribute the wheel
builds but `import wifi_densepose._native` fails with
ModuleNotFoundError.
## Local validation (P1 acceptance gate)
```
$ python -m venv .venv && .venv/Scripts/python -m pip install maturin pytest
$ VIRTUAL_ENV=… maturin develop --release
…
Finished `release` profile [optimized] target(s)
📦 Built wheel for abi3 Python ≥ 3.10
🛠 Installed wifi-densepose-2.0.0a1
$ .venv/Scripts/python -c 'import wifi_densepose; print(wifi_densepose.__version__, wifi_densepose.__rust_version__, wifi_densepose.hello())'
2.0.0a1 2.0.0-alpha.1 ok
$ .venv/Scripts/python -m pytest tests/ -v
tests/test_smoke.py::test_package_imports PASSED
tests/test_smoke.py::test_version_string_well_formed PASSED
tests/test_smoke.py::test_rust_version_surfaced PASSED
tests/test_smoke.py::test_build_features_listed PASSED
tests/test_smoke.py::test_hello_returns_ok PASSED
tests/test_smoke.py::test_native_module_private PASSED
======================== 6 passed in 0.05s =========================
```
P1 closed. Moving to P2 (core type bindings).
Refs #785, ADR-117 §6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p2): Keypoint + KeypointType bindings — 23 new tests (29/29 GREEN)
Lands the first chunk of P2: PyO3 bindings for `Keypoint` and
`KeypointType` from `wifi_densepose_core`. Bound types surface to
Python as `wifi_densepose.Keypoint` / `wifi_densepose.KeypointType`.
## Design choices that affect the API surface
1. **`Confidence` is NOT bound as a separate class.** Users hate
wrapping a float in a constructor. Python-side, confidence is just
a `float in [0.0, 1.0]`; the binding validates on construction
(`ValueError` for out-of-range, matching the Rust core error).
2. **`KeypointType` is a `#[pyclass(eq, eq_int, hash, frozen)]` enum**
— hashable so users can drop it into dicts/sets (the most common
pattern in pose-analysis notebooks: `keypoints_by_type[k.type] = k`).
3. **`Keypoint.__init__` keyword-only `z`** so 2D users don't have to
write `None` and 3D users get a clear named arg:
`Keypoint(KeypointType.LeftWrist, 0.2, 0.4, 0.8, z=0.1)`.
4. **`Keypoint` is `#[pyclass(frozen)]`** — no in-place mutation. The
Rust core type is immutable through Copy + Hash + Eq, and exposing
setters from Python would create a copy-vs-reference inconsistency
between languages.
## Files
- `python/src/bindings/keypoint.rs` — 220 lines of `#[pymethods]`
wrappers + Rust↔Python enum round-trip
- `python/src/lib.rs` — `mod bindings { pub mod keypoint; }` +
`bindings::keypoint::register(m)?` call from `#[pymodule]`
- `python/wifi_densepose/__init__.py` — re-exports `Keypoint` and
`KeypointType` at the package root
- `python/tests/test_keypoint.py` — 23 tests covering:
- 17-element COCO ordering of `KeypointType.all()`
- index→type mapping for every variant
- snake_name matches COCO spec
- `is_face()` / `is_upper_body()` predicates
- hashability (the bug I caught when I added the set-based face
test — fixed by adding `hash` to the `#[pyclass]` attribute)
- 2D + 3D constructor variants
- position_2d / position_3d tuples
- is_visible threshold
- confidence validation (Err on out-of-range)
- distance_to (2D Euclidean, 3D Euclidean, fallback when one is 2D
and the other is 3D)
- __repr__ + __eq__
- the new `p2-keypoint-bindings` feature marker landed
## Local validation
\`\`\`
$ cd python && .venv/Scripts/python -m pytest tests/ -v
tests/test_smoke.py::test_package_imports PASSED
tests/test_smoke.py::test_version_string_well_formed PASSED
tests/test_smoke.py::test_rust_version_surfaced PASSED
tests/test_smoke.py::test_build_features_listed PASSED
tests/test_smoke.py::test_hello_returns_ok PASSED
tests/test_smoke.py::test_native_module_private PASSED
tests/test_keypoint.py::test_keypoint_type_all_returns_17 PASSED
…
======================== 29 passed in 0.06s =========================
\`\`\`
Wheel size after both bindings: still well under the 5 MB ADR §5.4
budget (release build with --strip on Windows: ~340 KB).
Also adds `python/.gitignore` to prevent the `.venv/` + `target/` +
`_native.abi3.pyd` artifacts from getting committed.
## What's left in P2
CsiFrame + PoseEstimate bindings land in the next iteration. They're
larger (CsiFrame has the subcarrier buffer; PoseEstimate has
17×Keypoint + BoundingBox + track_id + score). Pattern is now proven
so they go faster.
Refs #785, ADR-117 §6.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p2): BoundingBox + PersonPose + PoseEstimate — P2 COMPLETE (57/57 tests GREEN)
Lands the second + third chunks of P2: PyO3 bindings for `BoundingBox`,
`PersonPose`, `PoseEstimate` from `wifi_densepose_core`. Combined with
the prior Keypoint + KeypointType bindings (fd0568caa), this closes
ADR-117 §6 P2.
## Coverage
| Type | Bound | Tests | Mutability |
|---|---|---|---|
| Confidence | exposed as `float` with validation | (covered in keypoint tests) | n/a |
| KeypointType | `#[pyclass(eq, eq_int, hash, frozen)]` | 7 tests | immutable |
| Keypoint | `#[pyclass(frozen)]` | 16 tests | immutable |
| BoundingBox | `#[pyclass(frozen)]` | 8 tests | immutable |
| PersonPose | `#[pyclass]` (mutable, builder-style) | 12 tests | mutable |
| PoseEstimate | `#[pyclass(frozen)]` | 8 tests | immutable |
Smoke (P1) + new tests: **57/57 PASS** locally on Windows.
## What's deferred to P3
CsiFrame intentionally NOT bound in P2 because it uses
`Array2<Complex64>` (ndarray) — the natural Python surface is via the
`numpy` pyo3 bridge, which lands in P3 alongside the vitals + signal
DSP bindings. Binding CsiFrame without numpy interop would force
users to materialise lists of tuples which is a worse API than
`csi_frame.amplitude_array()` returning an ndarray.
## Design choices that affect the API surface
1. **PersonPose.keypoints() returns a dict keyed by KeypointType**
instead of a fixed-length list with None slots. Pythonistas don't
want to know the underlying storage is `[Option<Keypoint>; 17]`.
2. **PoseEstimate.id and .timestamp exposed as strings** (UUID + ISO)
rather than as bound `FrameId` / `Timestamp` types. Users in
notebooks rarely compare UUIDs structurally; strings are good
enough for diagnostics and don't bloat the bindings.
3. **PersonPose is MUTABLE** (`#[pyclass]` without `frozen`) so users
can build poses incrementally with `set_keypoint`/`set_bbox`/
`set_id`. PoseEstimate is `frozen` because once constructed it
represents a snapshot.
## Three PyO3 0.22 gotchas surfaced this iteration
1. `#[pymethods]` getters are NOT accessible from other Rust modules
— need a separate `impl PyKeypoint { pub(crate) fn inner(&self)
-> &Keypoint { ... } }` block for cross-module use.
2. `PyDict::new(py)` was removed in PyO3 0.21 → 0.22 in favour of
`PyDict::new_bound(py)`. (Confusing because `Bound<'py, PyDict>`
is the return type either way.)
3. `dict.set_item(K, V)` requires both K and V to impl
`ToPyObject`. `#[pyclass]` types impl `IntoPy<PyObject>` but NOT
`ToPyObject` — workaround: convert via `.into_py(py)` first, then
`set_item(py_object_k, py_object_v)`.
Saved as PyO3 0.22 binding patterns memory at the horizon-tracker
level so future loop workers don't re-learn them.
## Local validation
\`\`\`
$ cd python && .venv/Scripts/python -m pytest tests/ -v
…
======================== 57 passed in 0.24s =========================
\`\`\`
Wheel size: still ~340 KB on Windows release build.
Refs #785, ADR-117 §6 (P2 done — ready for P3 vitals + signal DSP +
numpy bridge + witness v2).
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs(adr-117): add BFLD support (§5.7a + P3.5 phase + §11.11/12 open questions)
Per maintainer feedback during P3 implementation, expand ADR-117 to
include Beamforming Feedback Loop Data (BFLD) as a first-class binding
target alongside CSI. BFLD is the transmitter-side, AP-station-loop
view of the WiFi channel (802.11ac/ax/be compressed beamforming feedback
frames) — complementary to receiver-side CSI, with three properties
that make it strategically important for the pip wheel:
1. **Up to 996 subcarriers per HE160 frame** (vs 242 for HE-LTF CSI on
ESP32-C6, vs 52 for HT-LTF on ESP32-S3) — much denser per-subcarrier
reflection profile
2. **Works on stock 802.11ac+ hardware** — no Nexmon patch, no ESP32
monitor mode, no firmware drift. Captured via tcpdump/Wireshark +
BFR dissector, or via `mac80211` debugfs on Linux 6.10+
3. **Direct input for the soul-signature spec** (`docs/research/soul/`)
— the seven-channel biometric needs dense subcarrier reflection;
BFLD provides it without specialized hardware
## Three additions to ADR-117
### §5.7a — New binding-target subsection
Comparison table CSI vs BFLD; binding strategy with forward-compat
stub Rust impl pending the future `wifi-densepose-bfld` crate; the
three Python types that ship in P3.5:
- `BfldFrame` (frozen) — one compressed feedback matrix snapshot
- `BfldReport` (frozen) — aggregator over a 60-s scan window
- `BfldKind` enum — `CompressedHE20/40/80/160`, `UncompressedHT20/40`
### §6 P3.5 — Concurrent-with-P3 phase
Checkbox plan for the bindings module + stub Rust storage + numpy
bridge for `feedback_matrix` (Complex64 ndarray, same approach as
`CsiFrame.amplitude` from P3). Lands in the same wheel as P3, no
schedule cushion needed.
### §11.11/12 — Two new open questions
- **§11.11** — Should the future BFR ingestion Rust crate be a new
`wifi-densepose-bfld` workspace member, or extend `-signal`?
*Tentative: new dedicated crate. Wireshark BFR dissector is ~2k
lines and would bloat `-signal`; ingestion is optional for many
deployments; keep `-signal` lean.*
- **§11.12** — Per-vendor BFR variant compatibility (Broadcom vs
Intel vs Qualcomm vs MediaTek differ in psi/phi quantization +
matrix entry ordering). How much normalisation in the Python
binding vs. the future Rust crate? *Tentative: Python binding is
dumb (numpy ndarray in/out); future Rust crate owns per-vendor
normalisation via a `Vendor` enum on the constructor.*
### §12 — BFLD reference list
- Hernandez & Bulut, ACM TOSN 2024 (first systematic survey of
BFR-as-sensing)
- Yousefi et al., MobiSys 2023 (practical breath + HR extraction)
- IEEE 802.11ax-2021 §27.3.10 (frame format)
- Wireshark `packet-ieee80211.c` dissector
- AX210 Linux mac80211 debugfs path (kernel 6.10+)
ADR line count: 644 → 807 (+163). Refs #785 (tracking issue).
The implementation work for P3.5 lands in the next /loop iteration
alongside P3 vitals + signal DSP bindings.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p3+p3.5): vitals + BFLD bindings
P3 — Vital sign extraction bindings (wifi-densepose-vitals):
- VitalStatus enum (eq, eq_int, hash, frozen) — Valid/Degraded/Unreliable/Unavailable
- VitalEstimate (frozen) — value_bpm + confidence + status
- VitalReading (frozen) — HR + BR + signal quality composite
- BreathingExtractor — 0.1–0.5 Hz bandpass + zero-crossing
- HeartRateExtractor — 0.8–2.0 Hz bandpass + autocorrelation
- py.allow_threads on extract() hot loops (Q5 audit confirmed
core/vitals/signal are pure-sync — zero tokio deps, safe to release
GIL with no embedded runtime needed)
- 17 tests covering construction, getters, frozen immutability,
esp32_default + explicit ctors, synthetic-signal end-to-end
P3.5 — BFLD bindings (forward-compat surface, stub Rust):
- BfldKind enum — CompressedHE20/40/80/160 + UncompressedHT20/40
with n_subcarriers, bandwidth_mhz, is_he metadata getters
- BfldFrame (frozen) — from_compressed_feedback() accepts numpy
Complex64 ndarray [Nr x Nc x Nsc], validates dims against kind,
feedback_matrix() returns lossless roundtrip ndarray
- BfldReport — aggregates frames, rejects mismatched kinds,
computes inverse-CV coherence score
- 19 tests covering all 6 PHY variants + numpy roundtrip +
dim-mismatch error + aggregation
- Real Rust ingestion (wifi-densepose-bfld crate) lands post-v2.0
per ADR-117 §11.11/12 — Python API will not change
Total Python test count: 93 (was 57, +36 P3+P3.5). All passing.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p4): pure-Python WS/MQTT client layer
New sub-package `wifi_densepose.client` (no PyO3, no Rust deps):
- ws.SensingClient — asyncio websockets>=12 wrapper for the Rust
sensing-server /ws/sensing endpoint. Yields typed dataclasses
(ConnectionEstablishedMessage, EdgeVitalsMessage, PoseDataMessage)
with raw-payload fallback for forward-compat with unknown types.
Malformed frames log+drop without breaking the stream.
- mqtt.RuViewMqttClient — paho-mqtt v2 wrapper using the explicit
CallbackAPIVersion.VERSION2 API. Per-instance unique client_id by
default (rumqttc memory lesson). MQTT v5-spec-correct topic
wildcard matcher: + as whole-level wildcard, # matches the prefix
itself plus all sub-levels. Auto-resubscribes on reconnect.
Handler exceptions are caught and logged so a misbehaving callback
can't crash the network loop.
- primitives.SemanticPrimitiveListener — typed router for the 10
HA-MIND fused inference outputs from ADR-115 §3.12
(SomeoneSleeping, PossibleDistress, RoomActive, ElderlyInactivity-
Anomaly, MeetingInProgress, BathroomOccupied, FallRiskElevated,
BedExit, NoMovementSafety, MultiRoomTransition). Decodes both
JSON payloads with confidence+explanation AND plain HA state
strings ("ON"/"OFF"/numeric). Pluggable into RuViewMqttClient.
- ha.HABlueprintHelper — read-only parser for the
homeassistant/<kind>/wifi_densepose_<node>/<id>/config payload
family. Aggregator queries: entities_for_node, by_device_class,
nodes. Useful for blueprint authors + dashboard introspection.
Test coverage (63 new tests, 156 total in Python suite):
- test_client_ha — 18 tests (topic+payload parsing, aggregator)
- test_client_primitives — 13 tests (enum coverage, listener routing)
- test_client_mqtt — 17 tests (matcher parametrize, dispatch path,
on_connect, exception isolation) — no broker needed
- test_client_ws — 6 tests including end-to-end against an in-process
websockets.serve() fixture exercising all 4 message types plus a
malformed-frame survival check
Post-bridge wheel size: 238 KB (well under ADR §5.4 5 MB budget).
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md §5.6
Refs: docs/adr/ADR-115-home-assistant-integration.md §3.12
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117/p5+p-tomb): pip-release workflow + v1.99.0 tombstone wheel
P5 — `.github/workflows/pip-release.yml`:
- cibuildwheel matrix per ADR §5.4: manylinux x86_64 + aarch64,
macos x86_64 + arm64, win amd64 (5 wheels via abi3-py310 stable
ABI — one binary per OS/arch covers Python 3.10–3.13)
- Linux aarch64 cross-builds via QEMU; rustup 1.82 pinned in
CIBW_BEFORE_ALL_LINUX for reproducibility
- Per-wheel smoke test: import wifi_densepose, assert hello()=="ok"
- sdist via `maturin sdist`
- Trigger: workflow_dispatch + push to `v*-pip` tags ONLY (never
on regular commits — won't accidentally publish)
- TestPyPI dry-run gate via `repository-url: https://test.pypi.org/legacy/`
- Production PyPI publish via Trusted Publisher OIDC (no API tokens
in GH secrets per ADR §9). Requires one-time PyPI Trusted Publisher
registration before the first publish can fire.
- Q3 (witness hash v2 — ADR-117 §11.3) flagged in workflow comments
as a hard gate before the first tag.
P-tomb — `python/tombstone/`:
- Separate `wifi-densepose==1.99.0` sdist+wheel using setuptools
backend (NOT maturin — tombstone is pure Python, no Rust).
- `src/wifi_densepose/__init__.py` raises ImportError with the
migration URL on import. Verified locally: 2.7 KB wheel,
`pip install` then `import wifi_densepose` raises ImportError
with `pip install wifi-densepose==2.0.0` hint + repo URL.
- 5 unit tests (`tests/test_tombstone.py`) lock the file content
down: must `raise ImportError`, must contain v2 install hint
and migration URL, must NOT contain any `def`/`class`/`import`
beyond the bare `raise` — so a well-intentioned refactor can't
accidentally bloat the tombstone into a real module that loads
partway before failing.
Both wheels are published by the same pip-release.yml workflow:
- `v1.99.0-pip` tag → publishes tombstone (or via workflow_dispatch
with `target: v1-99-tombstone`)
- `v2.X.Y-pip` tag → publishes the v2 wheel matrix
Per ADR-117 §7.3: tag and publish 1.99.0-pip FIRST so the tombstone
claims the "current" slot in pip's resolver, THEN publish 2.0.0-pip.
Test count unchanged in main python/ suite (156/156). Tombstone
sub-suite: 5 passing.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md §5.4, §7
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* hardening(adr-117): benchmarks + security/robustness test suite
Benchmarks (`python/bench/`, pytest-benchmark — opt-in via --benchmark-only):
| Hot path | Mean | Ops/sec | % of 100 Hz budget |
|---|---|---|---|
| BfldFrame HT20 1×1×52 | 800 ns | 1.25 Mops | 0.008% |
| BfldFrame HE20 2×1×242 | 1.3 μs | 750 kops | 0.013% |
| BfldFrame HE80 2×1×996 | 4.2 μs | 236 kops | 0.042% |
| BfldFrame HE160 2×2×1992 | 14 μs | 71 kops | 0.14% |
| BfldFrame.feedback_matrix() | 2.8 μs | 352 kops | — |
| WS edge_vitals decode | 7.4 μs | 134 kops | 0.074% |
| WS pose_data decode (3 persons) | 23 μs | 42 kops | 0.24% |
| BreathingExtractor.extract() 56sc | 28 μs | 35 kops | 0.28% |
| BreathingExtractor.extract() 114sc | 44 μs | 23 kops | 0.44% |
| BreathingExtractor.extract() 242sc | 79 μs | 13 kops | 0.79% |
| HeartRateExtractor.extract() 56sc | 105 μs | 9.5 kops | 1.05% |
All hot paths well under the 100 Hz ESP32 frame budget (10 ms).
Worst case (HeartRateExtractor) uses 1% of the budget — no
optimization needed. Scaling on n_subcarriers is sub-quadratic
(56→242 = 4.3× input, 2.8× time) — catches future O(n²)
regressions.
Security & robustness tests (`tests/test_security.py`, +27 tests):
- WS decoder: rejects non-object roots cleanly, survives 1 MB string
values, handles non-ASCII node IDs, survives deeply-nested JSON
(Python's json.loads built-in guard not bypassed)
- MQTT topic matcher: 9 edge-case parametrize entries including
$SYS topics, null-byte injection, mid-pattern `#` boundary,
empty-string boundary
- MQTT credential confidentiality: password never appears in
repr()/str(), never stored in plain client-instance attribute
- HA discovery: rejects null-byte-laced topics, rejects extra
slashes in node_id, rejects non-dict payload body (list, scalar,
invalid UTF-8 bytes) without crashing
- Semantic primitive listener: rejects topic-injection attempts
(prefix-injected paths, wrong case on final segment), survives
invalid UTF-8 payloads
- Public surface integrity: every name in wifi_densepose.__all__
AND wifi_densepose.client.__all__ resolves — catches accidental
re-export breakage between phases
- Multi-handler MQTT exception isolation: a crashing handler in
the middle of the registered list doesn't stop later handlers
from firing
Test count: 156 → 183 (+27). All passing.
Bench results steady-state confirm no Rust-binding-layer
optimization is needed before the v2.0.0 publish.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(adr-117/p5): switch publish workflow to PYPI_API_TOKEN + user-facing README
- Workflow rewired from OIDC Trusted Publisher to token-based publish
via the `PYPI_API_TOKEN` GitHub Actions secret. Both publish jobs
(v2 wheels + tombstone) pass `password: ${{ secrets.PYPI_API_TOKEN }}`
to `pypa/gh-action-pypi-publish@release/v1`. Workflow comments now
document the GCP → GH secret-refresh command.
- Removed `permissions: id-token: write` and the OIDC `environment:`
blocks (no longer needed without OIDC).
- Token was sourced from the GCP Secret Manager entry `PYPI_TOKEN`
in project `cognitum-20260110` and pushed to GH Actions via
`gcloud secrets versions access | gh secret set` so the value
never appeared in a shell variable or this session's output.
- Rewrote `python/README.md` from a developer phase-ledger into a
user-facing PyPI front page: one-paragraph elevator pitch, bullet
list of features, three short usage snippets (vitals extract,
WS subscribe, MQTT semantic-primitive listener, BFLD numpy
bridge), hardware table, links. The README is the FIRST thing
pip users see at https://pypi.org/p/wifi-densepose so it has to
introduce the project, not the build plan.
Wheel rebuilds clean at 253 KB (was 238 KB — +15 KB from the richer
README baked into the wheel metadata). Test suite unchanged at 183/183.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* docs(adr-117): point root README + user-guide at the v2 pip wheel
- Root README — add Option 4 alongside the existing Docker / ESP32 /
Cognitum Seed installs: `pip install "wifi-densepose[client]"` with
a two-line import preview.
- User-guide §Installation — replace the stale "From Source (Python)"
block (which referenced legacy v1 extras `[gpu]` and `[all]` that
don't exist in v2) with a brief "Python wheel (pip) — ADR-117"
section: what the wheel is, install commands, two-line example,
tombstone caveat, and the `maturin develop` source-build path
for contributors.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(adr-117/p5): pin Python 3.12 + isolated venv for tombstone smoke-test
First v1.99.0-pip run (26366491748) failed: the runner's system `python`
fell back to `--user` install, then `python -c "import wifi_densepose"`
resolved to something other than the freshly-installed user-site wheel
and returned cleanly instead of raising the tombstone ImportError.
Fixes:
- `actions/setup-python@v5` with explicit 3.12 — owns its own site-
packages so pip won't fall back to --user.
- New "Inspect wheel contents" step prints the wheel manifest +
the verbatim __init__.py inside it. If a future regression ships
an empty __init__.py from a setuptools src-layout edge case,
the failure is debuggable from the run log alone.
- Smoke test now runs in a fresh /tmp/smoke-venv so there's zero
ambiguity about which wifi_densepose gets imported. Also uses
importlib.util.find_spec to print the resolved origin path
before the import attempt — so even if both checks pass, we
see exactly which file we exercised.
No code changes to the tombstone source itself.
Co-Authored-By: claude-flow <ruv@ruv.net>
* fix(adr-117/p5): smoke-test must cd out of repo root before importing
Root cause from run 26366579422 diagnostics: the wheel built correctly
(872 bytes, valid ImportError) but `import wifi_densepose` resolved to
the legacy `./wifi_densepose/__init__.py` left in the repo root from
v1, NOT to the freshly-installed tombstone wheel in the smoke venv.
Python places the cwd at sys.path[0] for `python -c "..."`, so
running the import from the repo root made the legacy directory win
over site-packages every time. The "isolated venv" was not the
problem — the cwd was.
Fix: copy the wheel to /tmp, cd /tmp before the import. Now the
smoke test runs in a directory that contains no `wifi_densepose/`
so the only resolution path is the venv's site-packages.
The repo-root `./wifi_densepose/__init__.py` is a separate concern
(legacy v1 carry-over) that should be cleaned up in a follow-up
commit, but the smoke test should not depend on it being absent.
Co-Authored-By: claude-flow <ruv@ruv.net>
* feat(adr-117): publish wifi-densepose 2.0.0a1 + ruview 2.0.0a1 to PyPI
Three PyPI artifacts now live (published from .env-sourced PYPI_TOKEN
via twine from the maintainer box — direct upload bypassed the GH
Actions workflow auth churn):
1. wifi-densepose==1.99.0 — tombstone (raises ImportError with migration URL)
https://pypi.org/project/wifi-densepose/1.99.0/
2. wifi-densepose==2.0.0a1 — PyO3 wheel (win_amd64 cp310-abi3) + sdist
https://pypi.org/project/wifi-densepose/2.0.0a1/
3. ruview==2.0.0a1 — meta-package re-exporting wifi_densepose
https://pypi.org/project/ruview/2.0.0a1/
New `python/ruview-meta/` subdirectory:
- pyproject.toml — name="ruview", version="2.0.0a1", setuptools backend,
dependencies = ["wifi-densepose==2.0.0a1"]
- src/ruview/__init__.py — re-exports every name from
`wifi_densepose.__all__` so `from ruview import BreathingExtractor`
is equivalent to `from wifi_densepose import BreathingExtractor`.
Also re-exports `__version__`, `__rust_version__`,
`__rust_build_tag__`, `__build_features__`. Aliases the `client`
sub-package transparently when wifi-densepose[client] extras are
installed.
- README.md — explains why two PyPI names ship the same code (brand
vs technical name) and shows install commands for both.
End-to-end verified: fresh venv, `pip install ruview`,
`import ruview` + `import wifi_densepose` both succeed,
`ruview.BreathingExtractor is wifi_densepose.BreathingExtractor` → True.
Multi-platform wheels (manylinux x86_64+aarch64, macos x86_64+arm64)
still pending — the cibuildwheel workflow path remains for that.
Linux/macOS users today install via the sdist (requires rustup +
maturin locally).
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #785
Co-Authored-By: claude-flow <ruv@ruv.net>
* ci(adr-117): kics-compatible workflow comments + fix-marker guards
- KICS error fix (.github/workflows/pip-release.yml:20): the inline
`gcloud secrets versions access --secret=PYPI_TOKEN ...` runbook
in the workflow header was triggering KICS' generic-secret regex
on the literal `PYPI_TOKEN` substring. Moved the refresh runbook
to docs/integrations/pypi-release.md (with the BOM-stripping
`tr` step that fixed the production publish) and replaced the
inline block with a pointer.
- Three new fix-marker guards in scripts/fix-markers.json so the
next person to touch this code can't silently regress what
PR #786 just shipped:
* RuView#786-tombstone-import — the tombstone __init__.py must
`raise ImportError`, must mention the v2 install hint, must
point at the repo URL, AND must NOT contain `def`/`class`/
`import wifi_densepose` (forbid patterns prevent accidental
bloating into a real module that loads partway before failing).
* RuView#786-tombstone-smoke-cwd — pip-release.yml must `cd /tmp`
before the tombstone smoke-test import, because the legacy
`./wifi_densepose/__init__.py` at repo root would otherwise
shadow the venv install. This was the root cause of run
26366648768; locking it in.
* RuView#786-pypi-token-auth — the workflow must use
`password: ${{ secrets.PYPI_API_TOKEN }}` and must NOT carry
`id-token: write`. The project authenticates via API token,
not OIDC; a partial OIDC migration would 403 silently.
Local check: all 25 markers pass.
Refs: docs/adr/ADR-117-pip-wifi-densepose-modernization.md
Refs: #786
Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
@@ -0,0 +1,807 @@
|
||||
# ADR-117: pip `wifi-densepose` modernization via PyO3 + maturin bindings
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Status** | Proposed |
|
||||
| **Date** | 2026-05-24 |
|
||||
| **Deciders** | ruv |
|
||||
| **Codename** | **PIP-PHOENIX** — rising from a pure-Python server to Rust-core Python bindings |
|
||||
| **Relates to** | [ADR-021](ADR-021-esp32-vitals.md) (ESP32 vitals), [ADR-028](ADR-028-esp32-capability-audit.md) (capability audit / witness), [ADR-115](ADR-115-home-assistant-integration.md) (HA-DISCO + HA-MIND MQTT semantics), [ADR-116](ADR-116-cog-ha-matter-seed.md) (HA-COG Seed packaging) |
|
||||
| **Tracking issue** | TBD — file under RuView issue tracker |
|
||||
|
||||
---
|
||||
|
||||
## 1. Context
|
||||
|
||||
### 1.1 What the pip package is today
|
||||
|
||||
`wifi-densepose` v1.1.0 was published to PyPI on **2025-06-07** (two releases the same
|
||||
day: 1.0.0 at 13:24 UTC, 1.1.0 at 17:02 UTC). Both wheels carry the tag
|
||||
`py3-none-any` — no compiled extension, no platform-specific code. The package is a
|
||||
**pure-Python server application** sourced entirely from `archive/v1/`.
|
||||
|
||||
The package installs a 40-dependency stack including FastAPI, PyTorch, SQLAlchemy,
|
||||
Redis, Celery, OpenCV, asyncpg, psycopg2, and Scapy (`archive/v1/setup.py:46–87`).
|
||||
The declared entry points are:
|
||||
|
||||
```
|
||||
wifi-densepose = src.cli:cli
|
||||
wdp = src.cli:cli
|
||||
```
|
||||
|
||||
(`archive/v1/setup.py:178–179`)
|
||||
|
||||
The public API surface is centred on a FastAPI HTTP server, a SQLAlchemy/postgres
|
||||
database layer, and a Redis/Celery task queue — none of which map to the current Rust
|
||||
architecture. The `__init__.py` exports `app` (FastAPI), `CSIProcessor`,
|
||||
`PhaseSanitizer`, `PoseEstimator`, `RouterInterface`, `ServiceOrchestrator`,
|
||||
`HealthCheckService`, and `MetricsService` (`archive/v1/src/__init__.py:54–68`).
|
||||
|
||||
### 1.2 Why this matters now
|
||||
|
||||
ADR-115 (PR #778, merged 2026-05-23) shipped 21 Home Assistant entities, 10 semantic
|
||||
primitives, mTLS, privacy mode, and a full witness bundle from the Rust crate
|
||||
`wifi-densepose-sensing-server`. ADR-116 is packaging this as a Cognitum Seed cog.
|
||||
Neither surface is reachable from `pip install wifi-densepose` — the pip package cannot
|
||||
import a CsiFrame, decode an edge-vitals packet, call a DSP stage, verify a witness
|
||||
bundle, or subscribe to the sensing server's MQTT or WebSocket endpoints. The ecosystem
|
||||
split is now wide enough that the pip package actively misleads new users about what
|
||||
the project does.
|
||||
|
||||
Three concrete customer pain points:
|
||||
|
||||
1. A Python user who `pip install wifi-densepose` expecting to consume live pose/vitals
|
||||
data gets a FastAPI server that requires postgres + redis, not a library they can
|
||||
script against.
|
||||
2. Integrators writing HA automations or Node-RED flows in Python have no idiomatic
|
||||
Python API for the v0.7 telemetry surface (ADR-115 entities, semantic primitives).
|
||||
3. The ADR-028 witness chain (deterministic pipeline proof) is Python-based and
|
||||
exercised via `archive/v1/data/proof/verify.py`, but it imports from the v1 stack —
|
||||
it cannot witness the Rust pipeline that is now the production implementation.
|
||||
|
||||
### 1.3 What this ADR is *not*
|
||||
|
||||
- Not a removal of `archive/v1/` from the repository. The v1 codebase stays as a
|
||||
research archive and its proof bundle stays in `archive/v1/data/proof/`.
|
||||
- Not a port of the Rust crates to Python. The Rust workspace (`v2/`) is authoritative
|
||||
and unmodified by this ADR.
|
||||
- Not a replacement of the `wifi-densepose-sensing-server` Rust binary. The pip
|
||||
package wraps or clients the binary; it does not reimplement it.
|
||||
- Not an overlap with ADR-116 (Seed cog packaging). ADR-116 ships a Seed-installable
|
||||
artifact; ADR-117 ships a Python developer library for scripting, automation, and
|
||||
prototyping against the Rust stack.
|
||||
|
||||
---
|
||||
|
||||
## 2. Current state — evidence
|
||||
|
||||
| Artifact | Value | Source |
|
||||
|---|---|---|
|
||||
| Latest PyPI version | **1.1.0** | `pypi.org/pypi/wifi-densepose/json` |
|
||||
| First release date | 2025-06-07T13:24:53Z | PyPI JSON metadata |
|
||||
| Latest release date | 2025-06-07T17:02:40Z | PyPI JSON metadata |
|
||||
| Months since last release | **~11.5 months** | as of 2026-05-24 |
|
||||
| Wheel tag | `py3-none-any` | PyPI simple index |
|
||||
| Hard dependencies | 40 (torch, fastapi, sqlalchemy, redis, celery, …) | `setup.py:46–87` |
|
||||
| Entry point | `src.cli:cli` | `setup.py:178` |
|
||||
| Python requires | `>=3.9` | `setup.py:108` |
|
||||
| Classifiers Python versions | 3.9, 3.10, 3.11, 3.12 | PyPI JSON classifiers |
|
||||
| Classifiers status | Beta (4) | PyPI JSON classifiers |
|
||||
| Current Rust workspace version | **0.3.0** | `v2/Cargo.toml:version` |
|
||||
| Rust crates in workspace | 20+ | `v2/Cargo.toml` members |
|
||||
| ADR-115 shipped | 2026-05-23 | PR #778 |
|
||||
|
||||
The v1 source package (`archive/v1/setup.py:112–215`) was clearly designed as an
|
||||
all-in-one server application, not a reusable library. The `find_packages` call at
|
||||
line 134 searches from `"."` (the archive root), meaning the wheel ships `src.*` as the
|
||||
importable namespace. The proof bundle (`archive/v1/data/proof/verify.py:56–57`) imports
|
||||
`src.hardware.csi_extractor.CSIData` and `src.core.csi_processor.CSIProcessor` — v1 pure
|
||||
Python only.
|
||||
|
||||
**PyPI org presence check:** a search for other `ruvnet`-published PyPI packages
|
||||
(`ruvector`, `claude-flow`) returned no matches in the PyPI simple index as of this
|
||||
writing. The `wifi-densepose` package is currently the only Python entry point for this
|
||||
project's ecosystem.
|
||||
|
||||
---
|
||||
|
||||
## 3. Gap analysis
|
||||
|
||||
| Capability | Rust crate(s) | pip v1.1.0 status | Gap severity |
|
||||
|---|---|---|---|
|
||||
| `CsiFrame` / `CsiMetadata` core types | `wifi-densepose-core` (`types.rs`) | Not present — v1 uses `CSIData` Python class | **Critical** |
|
||||
| HR/BR extraction from CSI buffer | `wifi-densepose-vitals` (4-stage pipeline: preprocessor → breathing → heartrate → anomaly) | Stub Python (`src/hardware/csi_extractor.py`) with no DSP | **Critical** |
|
||||
| Phase sanitization / noise removal | `wifi-densepose-signal` (`phase_sanitizer`, `csi_processor`, `hampel`) | Python stubs in `src/core/phase_sanitizer.py` | **Critical** |
|
||||
| Motion detection + presence scoring | `wifi-densepose-signal` (`motion.rs`, `MotionDetector`) | Not present | **Critical** |
|
||||
| RuvSense multistatic sensing (13 modules) | `wifi-densepose-signal/src/ruvsense/` | Not present — ADR-029 post-dates v1 | **Critical** |
|
||||
| 17-keypoint pose estimation | `wifi-densepose-nn`, `wifi-densepose-mat` | Stub `PoseEstimator` wrapping a `torch.nn.Module` that requires model weights | **High** |
|
||||
| MQTT publisher (21 HA entities) | `wifi-densepose-sensing-server/src/mqtt/` | Not present — ADR-115 post-dates v1 | **High** |
|
||||
| Semantic primitives (10 types) | `wifi-densepose-sensing-server/src/semantic/` | Not present | **High** |
|
||||
| Matter bridge | `wifi-densepose-sensing-server/src/matter/` | Not present | **High** |
|
||||
| WS/REST client for sensing-server | `wifi-densepose-sensing-server` (Axum) | v1 has a separate FastAPI server; no client | **High** |
|
||||
| Witness bundle verification | ADR-028 / `scripts/generate-witness-bundle.sh` | `archive/v1/data/proof/verify.py` — proves v1 pipeline only | **High** |
|
||||
| ESP32-C6 firmware telemetry (ADR-110) | `wifi-densepose-hardware` + `wifi-densepose-sensing-server` | Not present | **Medium** |
|
||||
| Cross-viewpoint fusion (RuVector) | `wifi-densepose-ruvector/src/viewpoint/` | Not present | **Medium** |
|
||||
| Semantic-primitive MQTT payload | `wifi-densepose-sensing-server/src/semantic/bus.rs` | Not present | **Medium** |
|
||||
| PostgreSQL + Redis server mode | `archive/v1/` | Present (v1 only) | Low (not SOTA) |
|
||||
| FastAPI HTTP REST server | `archive/v1/src/app.py` | Present (v1 only) | Low (not SOTA) |
|
||||
|
||||
---
|
||||
|
||||
## 4. Decision
|
||||
|
||||
Adopt **PyO3 + maturin Python extension bindings** as the primary modernization path,
|
||||
shipping the pip package as a platform-native wheel (`manylinux`, `macosx`, `win-amd64`)
|
||||
with compiled Rust extension modules, plus a pure-Python WS/MQTT client layer that talks
|
||||
to a running `wifi-densepose-sensing-server` instance.
|
||||
|
||||
This path is called **PIP-PHOENIX**.
|
||||
|
||||
### 4.1 Why PyO3 + maturin over the three rejected alternatives
|
||||
|
||||
| Criterion | **PyO3 + maturin** (chosen) | Subprocess wrapper | REST/WS client only | Pure Python reimpl |
|
||||
|---|---|---|---|---|
|
||||
| Performance for DSP | Native Rust speed, zero copy | IPC overhead per call | N/A — no local DSP | Python bottleneck |
|
||||
| Binary size in wheel | Core + vitals + signal only: ~2 MB stripped | Full sensing-server binary: ~15–30 MB | Minimal (~50 kB) | Minimal (~100 kB) |
|
||||
| Works offline / no server | Yes | Yes (binary bundled) | No — server required | Partial |
|
||||
| Proof bundle can cover Rust pipeline | Yes — bindings call the same Rust code the server uses | Partial — server is a black box | No | No |
|
||||
| Install experience | `pip install wifi-densepose` — wheel has no system deps | `pip install` downloads 25 MB binary | `pip install` — pure Python | `pip install` — pure Python |
|
||||
| Maintenance surface | Python bindings + Rust workspace | Python thin shim | Python client | Python reimpl must track Rust |
|
||||
| Async / tokio support | PyO3 0.28 `pyo3-asyncio` or `pyo3-async-runtimes` for async export; sync entry points for the DSP hot path | N/A | Native asyncio on client | N/A |
|
||||
| GIL concern | DSP-heavy calls release GIL via `py.allow_threads`; tokio runtime per module | N/A | None | N/A |
|
||||
| Fits existing architecture | Core + vitals + signal already have clean public APIs (`lib.rs` re-exports) | Requires sensing-server to be running | Requires sensing-server | Forks the domain model |
|
||||
|
||||
**Subprocess wrapper** is rejected because shipping a 25 MB pre-built server binary
|
||||
inside every pip wheel is an unacceptably heavy install, and it makes offline scripting
|
||||
impossible without starting the server.
|
||||
|
||||
**REST/WS client only** is rejected because it provides zero DSP utility offline and
|
||||
cannot close the witness gap — the proof bundle must exercise the same pipeline code.
|
||||
|
||||
**Pure Python reimplementation** is the root cause of the current drift and is
|
||||
explicitly rejected.
|
||||
|
||||
The chosen path starts small: **bind only the three crates with the highest Python
|
||||
utility** (`wifi-densepose-core`, `wifi-densepose-vitals`, `wifi-densepose-signal`),
|
||||
ship a `py3-none-any` pure-Python WS/MQTT client layer as a separate sub-module, and
|
||||
grow from there.
|
||||
|
||||
---
|
||||
|
||||
## 5. Detailed design
|
||||
|
||||
### 5.1 Rust crates bound in v2.0 (first wheel)
|
||||
|
||||
Three crates are in scope for the initial binding. They were chosen because they have
|
||||
no heavy system dependencies (no libtorch, no ONNX runtime), have stable `pub` re-export
|
||||
surfaces in `lib.rs`, and directly address the three most-requested missing capabilities.
|
||||
|
||||
| Crate | Exported Python types / functions | Binding rationale |
|
||||
|---|---|---|
|
||||
| `wifi-densepose-core` | `CsiFrame`, `CsiMetadata`, `Keypoint`, `KeypointType`, `PersonPose`, `PoseEstimate`, `Confidence`, `BoundingBox` | Foundation types shared by all other crates; without these users can't even describe a frame |
|
||||
| `wifi-densepose-vitals` | `CsiVitalPreprocessor`, `BreathingExtractor`, `HeartRateExtractor`, `VitalAnomalyDetector`, `VitalSignStore`, `VitalReading`, `VitalEstimate`, `AnomalyAlert` | The most-asked-for surface: HR/BR from a CSI buffer in 4 lines of Python |
|
||||
| `wifi-densepose-signal` | `CsiProcessor`, `CsiProcessorConfig`, `PhaseSanitizer`, `MotionDetector`, `MotionScore`, `FeatureExtractor`, `HardwareNormalizer` | DSP pipeline that produces the features vitals and pose estimation consume |
|
||||
|
||||
Crates **deferred to P6+**: `wifi-densepose-nn` (requires libtorch or candle — wheel
|
||||
size risk), `wifi-densepose-mat` (depends on nn), `wifi-densepose-ruvector` (RuVector
|
||||
GNN types — high value but adds ruvector-gnn 2.0.5 link dependency),
|
||||
`wifi-densepose-hardware` (ESP32 HAL — not Python-scripting friendly).
|
||||
|
||||
### 5.2 New workspace member: `python/`
|
||||
|
||||
A new crate `python/` is added as a workspace member at `v2/crates/wifi-densepose-py/`.
|
||||
It is a `cdylib` that re-exports the three bound crates behind a single maturin module
|
||||
named `wifi_densepose._core`.
|
||||
|
||||
```toml
|
||||
# v2/crates/wifi-densepose-py/Cargo.toml (sketch)
|
||||
[package]
|
||||
name = "wifi-densepose-py"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
|
||||
[lib]
|
||||
name = "_core"
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
pyo3 = { version = "0.28", features = ["extension-module", "abi3-py310"] }
|
||||
wifi-densepose-core = { path = "../wifi-densepose-core", features = ["serde"] }
|
||||
wifi-densepose-vitals = { path = "../wifi-densepose-vitals" }
|
||||
wifi-densepose-signal = { path = "../wifi-densepose-signal" }
|
||||
```
|
||||
|
||||
The `abi3-py310` feature locks the stable ABI to CPython 3.10+, so one wheel binary
|
||||
works across 3.10, 3.11, 3.12, and 3.13 without recompilation.
|
||||
|
||||
PyO3 bindings pattern (example for `CsiFrame`):
|
||||
|
||||
```rust
|
||||
// v2/crates/wifi-densepose-py/src/core_types.rs
|
||||
use pyo3::prelude::*;
|
||||
use wifi_densepose_core::CsiFrame as RustCsiFrame;
|
||||
|
||||
#[pyclass(name = "CsiFrame")]
|
||||
#[derive(Clone)]
|
||||
pub struct PyCsiFrame {
|
||||
inner: RustCsiFrame,
|
||||
}
|
||||
|
||||
#[pymethods]
|
||||
impl PyCsiFrame {
|
||||
#[new]
|
||||
fn new(amplitudes: Vec<f32>, phases: Vec<f32>, n_subcarriers: usize,
|
||||
sample_index: u64, sample_rate_hz: f32) -> Self {
|
||||
Self { inner: RustCsiFrame { amplitudes, phases, n_subcarriers,
|
||||
sample_index, sample_rate_hz } }
|
||||
}
|
||||
|
||||
#[getter] fn amplitudes(&self) -> Vec<f32> { self.inner.amplitudes.clone() }
|
||||
#[getter] fn phases(&self) -> Vec<f32> { self.inner.phases.clone() }
|
||||
#[getter] fn n_subcarriers(&self) -> usize { self.inner.n_subcarriers }
|
||||
}
|
||||
```
|
||||
|
||||
DSP calls that execute >1 ms release the GIL:
|
||||
|
||||
```rust
|
||||
#[pymethods]
|
||||
impl PyCsiProcessor {
|
||||
fn process<'py>(&mut self, py: Python<'py>, frame: &PyCsiFrame)
|
||||
-> PyResult<Option<PyProcessedSignal>>
|
||||
{
|
||||
py.allow_threads(|| self.inner.process(&frame.inner))
|
||||
.map(|opt| opt.map(PyProcessedSignal::from))
|
||||
.map_err(|e| PyRuntimeError::new_err(e.to_string()))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 5.3 pip package layout
|
||||
|
||||
```
|
||||
wifi-densepose/ ← PyPI package name (unchanged)
|
||||
wifi_densepose/ ← importable namespace
|
||||
__init__.py ← re-exports core types + version
|
||||
_core.pyd / _core.so ← compiled PyO3 extension (maturin build output)
|
||||
vitals.py ← thin Python wrapper + docstrings over _core vitals types
|
||||
signal.py ← thin Python wrapper over _core signal types
|
||||
client/
|
||||
__init__.py
|
||||
ws.py ← asyncio WebSocket client for sensing-server /ws/sensing
|
||||
mqtt.py ← paho-mqtt wrapper for ruview/<node_id>/raw/* topics
|
||||
ha.py ← helpers for HA-DISCO payloads (read-only, mirrors ADR-115 §3.2)
|
||||
witness/
|
||||
__init__.py
|
||||
verify.py ← Python-callable witness verifier (re-creates ADR-028 proof
|
||||
over the Rust pipeline via PyO3 bindings, not archive/v1/)
|
||||
compat/
|
||||
v1.py ← import shim that raises MigrationError (see §9)
|
||||
py.typed ← PEP 561 marker
|
||||
```
|
||||
|
||||
The import path intentionally maps to Rust crate names:
|
||||
|
||||
```python
|
||||
from wifi_densepose import CsiFrame # core types
|
||||
from wifi_densepose.vitals import BreathingExtractor, HeartRateExtractor
|
||||
from wifi_densepose.signal import CsiProcessor, MotionDetector
|
||||
from wifi_densepose.client.ws import SensingClient
|
||||
from wifi_densepose.witness import verify_bundle
|
||||
```
|
||||
|
||||
### 5.4 PyPI distribution — wheel matrix
|
||||
|
||||
Published as `wifi-densepose==2.0.0` using **cibuildwheel** driven by GitHub Actions.
|
||||
|
||||
| Platform | Arch | CPython | Tag (stable ABI) |
|
||||
|---|---|---|---|
|
||||
| `manylinux_2_28` | x86_64 | 3.10+ | `cp310-abi3-manylinux_2_28_x86_64` |
|
||||
| `manylinux_2_28` | aarch64 | 3.10+ | `cp310-abi3-manylinux_2_28_aarch64` |
|
||||
| `macosx_11_0` | x86_64 | 3.10+ | `cp310-abi3-macosx_11_0_x86_64` |
|
||||
| `macosx_11_0` | arm64 | 3.10+ | `cp310-abi3-macosx_11_0_arm64` |
|
||||
| `win` | amd64 | 3.10+ | `cp310-abi3-win_amd64` |
|
||||
| sdist | — | — | source fallback |
|
||||
|
||||
The `abi3-py310` flag means **one binary per OS/arch** covers all supported Python
|
||||
versions — 5 wheels total plus an sdist, compared to the 20-wheel matrix that would be
|
||||
needed without stable ABI.
|
||||
|
||||
```yaml
|
||||
# .github/workflows/pip-release.yml (sketch)
|
||||
- uses: pypa/cibuildwheel@v2
|
||||
with:
|
||||
package-dir: v2/crates/wifi-densepose-py
|
||||
output-dir: dist
|
||||
env:
|
||||
CIBW_BUILD: "cp310-*"
|
||||
CIBW_ARCHS_LINUX: "x86_64 aarch64"
|
||||
CIBW_ARCHS_MACOS: "x86_64 arm64"
|
||||
CIBW_ARCHS_WINDOWS: "AMD64"
|
||||
CIBW_BEFORE_BUILD: "pip install maturin"
|
||||
CIBW_BUILD_FRONTEND: "build[uv]"
|
||||
```
|
||||
|
||||
### 5.5 CLI parity
|
||||
|
||||
The pip wheel installs a `wifi-densepose` console script. In v2 this script is a thin
|
||||
Python shim that:
|
||||
|
||||
1. Checks whether `wifi-densepose-sensing-server` binary is on `PATH` (installed
|
||||
separately via a platform-specific binary distribution or `cargo install`).
|
||||
2. If found: proxies `wifi-densepose serve`, `wifi-densepose stream`, etc. to the Rust
|
||||
binary via `subprocess.run`.
|
||||
3. If not found: falls back to the PyO3 module for offline DSP commands
|
||||
(`wifi-densepose vitals --file recording.jsonl`).
|
||||
|
||||
This is explicitly **not** a reimplementation of the CLI — the Rust binary
|
||||
(`wifi-densepose-cli/src/main.rs`, currently exposes `mat` and `version` subcommands)
|
||||
is the authoritative CLI. The pip shim is a discovery/convenience layer.
|
||||
|
||||
### 5.6 WS/MQTT client layer
|
||||
|
||||
`wifi_densepose.client.ws.SensingClient` is a pure-Python asyncio client wrapping the
|
||||
sensing-server WebSocket at `/ws/sensing`:
|
||||
|
||||
```python
|
||||
async with SensingClient("ws://localhost:8765/ws/sensing") as client:
|
||||
async for msg in client.stream():
|
||||
if msg.type == "edge_vitals":
|
||||
print(msg.breathing_rate_bpm, msg.heartrate_bpm)
|
||||
```
|
||||
|
||||
`wifi_densepose.client.mqtt.RuViewMqttClient` wraps paho-mqtt and subscribes to
|
||||
`ruview/<node_id>/raw/+` as defined in ADR-115 §3.2.
|
||||
|
||||
Both clients are **pure Python** (no PyO3) and are optional dependencies (`pip install
|
||||
wifi-densepose[client]`). They depend on `websockets>=12` and `paho-mqtt>=2` respectively.
|
||||
|
||||
### 5.7a Beamforming Feedback Loop Data (BFLD) support — new binding target
|
||||
|
||||
**Added 2026-05-24 per maintainer feedback during P3 implementation.**
|
||||
|
||||
BFLD is the transmitter-side, AP-station-loop view of the WiFi channel
|
||||
— compressed beamforming feedback frames that 802.11ac/ax/be stations
|
||||
send to the AP per sounding cycle. From a sensing perspective it
|
||||
complements receiver-side CSI:
|
||||
|
||||
| | Receiver-side CSI (current) | BFLD (this addition) |
|
||||
|---|---|---|
|
||||
| Source | RX side of the radio (e.g. Nexmon CSI on Pi 5, ESP32 promisc cb) | Sniffed BFR frames in the air or `mac80211` ACK trace |
|
||||
| Subcarriers (HE20) | 52 (HT-LTF) or 242 (HE-LTF) | Up to 996 (HE160 compressed BFR) — denser |
|
||||
| Hardware requirements | Patched Broadcom/Cypress or ESP32 specifically | **Any** 802.11ac+ station-AP pair — no patched firmware |
|
||||
| Privacy model | Captures everyone in radio range | Same |
|
||||
| Maturity in repo | Production (ADR-014, ADR-018, ADR-039) | Research; no Rust crate yet |
|
||||
| Suitable use case | Through-wall pose + vitals | Dense subcarrier reflection profile for AETHER-class biometric (ADR-024) and the soul-signature spec (`docs/research/soul/`) |
|
||||
|
||||
#### Binding strategy
|
||||
|
||||
Because the Rust workspace has no `wifi-densepose-bfld` crate yet, P3
|
||||
ships a **forward-compatible Python trait surface** that the future
|
||||
Rust crate plugs into without changing the Python API:
|
||||
|
||||
```python
|
||||
from wifi_densepose import BfldFrame, BfldReport
|
||||
|
||||
# Today (P3): construct from a parsed BFR feedback matrix (the bring-
|
||||
# your-own-parser path). Users on Pi 5 + Wireshark BFR dissector
|
||||
# pipe frames in directly.
|
||||
frame = BfldFrame.from_compressed_feedback(
|
||||
timestamp_ms=…,
|
||||
sounding_index=…,
|
||||
sta_mac="aa:bb:cc:…",
|
||||
bandwidth_mhz=80,
|
||||
n_subcarriers=996,
|
||||
feedback_matrix=…, # numpy ndarray complex64 [Nr × Nc × Nsc]
|
||||
)
|
||||
|
||||
# P3 also ships a stub `BfldReport` aggregator that mirrors how
|
||||
# `VitalEstimate` aggregates `VitalReading`s. Users who have BFR
|
||||
# pipelines feeding RuView can use this today via the
|
||||
# bring-your-own-parser path.
|
||||
|
||||
# Tomorrow (post-v2.0): the `wifi-densepose-bfld` Rust crate (TBD —
|
||||
# separate ADR-1xx) provides ingestion from Nexmon `nl80211` traces +
|
||||
# kernel `mac80211` debugfs hooks, and the pip wheel transparently
|
||||
# binds it without changing this Python surface.
|
||||
```
|
||||
|
||||
#### Why this matters
|
||||
|
||||
Three reasons BFLD belongs in v2.0 rather than waiting for the Rust
|
||||
core:
|
||||
|
||||
1. **Customer pull**. Several integrators reading the ADR-115 release
|
||||
notes asked about WiFi-6 dense-subcarrier capture; the answer is
|
||||
BFLD, and we want the API stable before they build pipelines.
|
||||
2. **Soul-signature dependency**. The soul-signature research spec
|
||||
(`docs/research/soul/specification.md`) lists "Subcarrier Reflection
|
||||
Profile" as one of seven biometric channels. At HE20/HE80 the
|
||||
dense BFR subcarriers are the right input — exposing `BfldFrame`
|
||||
now lets researchers prototype the channel without waiting on a
|
||||
Rust ingestion crate.
|
||||
3. **Cross-vendor portability**. CSI ingestion needs patched
|
||||
firmware. BFR ingestion works on stock 802.11ac/ax hardware
|
||||
(capture via `tcpdump`/Wireshark + a BFR dissector). Shipping the
|
||||
Python data structures first gives the community a way to feed
|
||||
RuView from gear we don't directly support.
|
||||
|
||||
#### Implementation surface in P3
|
||||
|
||||
Lands as a new module `bindings/bfld.rs` (~150 lines, three
|
||||
`#[pyclass]` types):
|
||||
|
||||
- `BfldFrame` (frozen) — one compressed feedback matrix snapshot.
|
||||
Constructors: `from_compressed_feedback(...)` and
|
||||
`from_uncompressed_v(...)` (the 802.11n V-matrix form).
|
||||
Properties: `timestamp_ms`, `sounding_index`, `sta_mac`,
|
||||
`bandwidth_mhz`, `n_subcarriers`, `n_rows` (Nr), `n_cols` (Nc),
|
||||
`feedback_matrix` (numpy ndarray complex64).
|
||||
- `BfldReport` (frozen) — aggregator over a window of `BfldFrame`s.
|
||||
Properties: `n_frames`, `timestamp_first`, `timestamp_last`,
|
||||
`mean_amplitude_per_subcarrier`, `coherence_score`. The Python
|
||||
side gives users a stable handle for "all BFR data in this 60-s
|
||||
scan" without leaking the storage representation.
|
||||
- `BfldKind` (`#[pyclass(eq, eq_int, hash, frozen)]`) — enum
|
||||
enumerating the BFR variants we support: `CompressedHE20`,
|
||||
`CompressedHE40`, `CompressedHE80`, `CompressedHE160`,
|
||||
`UncompressedHT20`, `UncompressedHT40`.
|
||||
|
||||
Stub Rust implementation lives in `python/src/bfld_stub.rs` until
|
||||
the proper Rust crate exists; it's intentionally not in v2/crates/.
|
||||
A new ADR-1xx will own the Rust ingestion crate when we commit to it.
|
||||
|
||||
#### Open questions added
|
||||
|
||||
- §9.11 — Should BFLD ingestion live in a new `wifi-densepose-bfld`
|
||||
crate or in `wifi-densepose-signal` extended?
|
||||
- §9.12 — Per-vendor BFR variant compatibility (Broadcom vs Intel vs
|
||||
Qualcomm encode the compressed angles slightly differently) — how
|
||||
much normalisation belongs in the Python binding vs. the future
|
||||
Rust crate?
|
||||
|
||||
### 5.7 Witness chain (re-rooted to the Rust pipeline)
|
||||
|
||||
`wifi_densepose.witness.verify_bundle(path)` replaces the v1 proof verification with a
|
||||
new chain that exercises the Rust pipeline via PyO3:
|
||||
|
||||
```python
|
||||
from wifi_densepose.witness import verify_bundle
|
||||
|
||||
result = verify_bundle("dist/witness-bundle-ADR028-*/")
|
||||
assert result.verdict == "PASS", result.detail
|
||||
```
|
||||
|
||||
Internally it:
|
||||
1. Loads the 1,000-frame reference JSON from the bundle.
|
||||
2. Feeds each frame through `PyCsiProcessor` (PyO3 binding of the Rust `CsiProcessor`).
|
||||
3. Hashes the output using the same SHA-256 scheme as `archive/v1/data/proof/verify.py`.
|
||||
4. Compares against the published hash in `expected_features.sha256`.
|
||||
|
||||
The v1 proof (`archive/v1/data/proof/verify.py`) is **preserved unchanged** — it
|
||||
continues to prove the v1 pipeline. The new `witness.py` proves the v2/Rust pipeline.
|
||||
Both can coexist; the ADR-028 witness bundle ships with both.
|
||||
|
||||
---
|
||||
|
||||
## 6. Migration path (phased)
|
||||
|
||||
```
|
||||
P1 ──► P2 ──► P3 ──► P4 ──► P5 ──► P6+
|
||||
scaffold core vitals+ client publish deferred
|
||||
types signal layer v2.0.0
|
||||
```
|
||||
|
||||
### P1 — Scaffold (1 week)
|
||||
|
||||
- [ ] Add `v2/crates/wifi-densepose-py/` as workspace member.
|
||||
- [ ] `Cargo.toml`: `crate-type = ["cdylib"]`, pyo3 0.28 + `abi3-py310`, no
|
||||
workspace deps yet (empty module compiles and imports).
|
||||
- [ ] `pyproject.toml` at repo root `python/` with `[build-system] requires =
|
||||
["maturin>=1.8"]` and `[tool.maturin] features = ["pyo3/extension-module"]`.
|
||||
- [ ] CI job: `maturin develop` on ubuntu-latest in a Python 3.12 venv; import
|
||||
`wifi_densepose._core` succeeds.
|
||||
- [ ] Publish `wifi-densepose==1.99.0` to PyPI with a migration notice in the
|
||||
module body (see §9 — no new features, just the tombstone release).
|
||||
|
||||
### P2 — Core type bindings (1 week)
|
||||
|
||||
- [ ] Bind `CsiFrame`, `CsiMetadata`, `Confidence`, `Keypoint`, `KeypointType`,
|
||||
`BoundingBox`, `PoseEstimate`, `PersonPose` from `wifi-densepose-core`.
|
||||
- [ ] All types: `__repr__`, `__eq__`, `__hash__` where meaningful; serde JSON
|
||||
round-trip via `pyo3-serde` or manual `to_dict()` / `from_dict()`.
|
||||
- [ ] Add `py.typed` + stub `.pyi` file generated by `pyo3-stub-gen`.
|
||||
- [ ] Unit tests: `tests/test_core.py` — construct each type, round-trip JSON.
|
||||
|
||||
### P3 — Vitals + signal DSP bindings (2 weeks)
|
||||
|
||||
- [ ] Bind the full 4-stage vitals pipeline:
|
||||
`CsiVitalPreprocessor`, `BreathingExtractor`, `HeartRateExtractor`,
|
||||
`VitalAnomalyDetector`, `VitalSignStore`, `VitalReading`, `VitalEstimate`,
|
||||
`AnomalyAlert`.
|
||||
- [ ] Bind signal DSP entry points: `CsiProcessor`, `CsiProcessorConfig`,
|
||||
`PhaseSanitizer`, `MotionDetector`, `HardwareNormalizer`.
|
||||
- [ ] GIL release (`py.allow_threads`) on all calls >0.5 ms (measured in bench).
|
||||
- [ ] Integration test: feed 1,000 frames from `archive/v1/data/proof/sample_csi_data.json`
|
||||
through the PyO3 vitals pipeline; assert output is deterministic across runs.
|
||||
- [ ] Re-implement `witness/verify.py` using P3 bindings; compare SHA-256 against the
|
||||
v1 expected hash. **Note:** the hash will differ because the Rust and Python
|
||||
processors are not identical — generate and publish a new `expected_features_v2.sha256`.
|
||||
|
||||
### P4 — WS/MQTT client layer (1 week)
|
||||
|
||||
- [ ] Implement `wifi_densepose.client.ws.SensingClient` (asyncio, `websockets>=12`).
|
||||
- [ ] Implement `wifi_densepose.client.mqtt.RuViewMqttClient` (paho-mqtt 2.x).
|
||||
- [ ] Add `wifi_densepose.client.ha` helpers that parse ADR-115 MQTT discovery payloads
|
||||
into Python dataclasses.
|
||||
- [ ] Integration test: spin up `sensing-server` in Docker with `--mock-frames`;
|
||||
assert `SensingClient` receives `edge_vitals` messages.
|
||||
|
||||
### P5 — First cibuildwheel publish as v2.0.0 (1 week)
|
||||
|
||||
- [ ] `.github/workflows/pip-release.yml` — cibuildwheel matrix (5 wheels + sdist).
|
||||
- [ ] `python_requires = ">=3.10"` (stable ABI base).
|
||||
- [ ] Populate `pyproject.toml` with minimal `install_requires`: `pyo3` is a build dep,
|
||||
not a runtime dep. Runtime extras: `[client]` adds `websockets>=12,paho-mqtt>=2`.
|
||||
- [ ] `pip install wifi-densepose==2.0.0` and smoke-test on each CI platform.
|
||||
- [ ] PyPI publish via Trusted Publisher (OIDC, no API token in secrets).
|
||||
- [ ] Announce: `wifi-densepose==1.99.0` tombstone already on PyPI; `v2.0.0` replaces
|
||||
it in search results.
|
||||
|
||||
### P3.5 — BFLD binding surface (concurrent with P3)
|
||||
|
||||
**Added 2026-05-24 per maintainer feedback.** See §5.7a for the rationale.
|
||||
|
||||
- [ ] `python/src/bindings/bfld.rs` — `BfldFrame`, `BfldReport`,
|
||||
`BfldKind` `#[pyclass]` wrappers backed by a stub Rust impl
|
||||
pending the v3 `wifi-densepose-bfld` crate.
|
||||
- [ ] `python/src/bfld_stub.rs` — minimal in-crate stub storage
|
||||
(vec of compressed feedback matrices) so the Python API is
|
||||
fully usable today even before the Rust ingestion crate lands.
|
||||
- [ ] Numpy bridge for `feedback_matrix` (Complex64 ndarray) — same
|
||||
approach as `CsiFrame.amplitude` from P3.
|
||||
- [ ] Tests covering: per-bandwidth constructor paths
|
||||
(HE20/HE40/HE80/HE160 + HT20/HT40), n_subcarriers contract,
|
||||
coherence_score sanity, BfldKind hashability + equality.
|
||||
- [ ] Forward-compat contract test: `BfldFrame` constructed today
|
||||
from a numpy ndarray must round-trip through (de)serialisation
|
||||
identically once the Rust crate exists.
|
||||
- [ ] §9.11 + §9.12 open questions raised so the eventual Rust crate
|
||||
has clear decisions waiting for it.
|
||||
|
||||
P3.5 is concurrent with P3 (no new schedule cushion needed) because
|
||||
the Python surface is independent of the rest of the v2/ workspace.
|
||||
Land in the same wheel as P3.
|
||||
|
||||
### P6+ — Deferred
|
||||
|
||||
- [ ] `wifi-densepose-bfld` Rust crate — proper ingestion from
|
||||
Nexmon BFR pcaps + `mac80211` debugfs. Replaces the P3.5 stub
|
||||
storage without changing the Python API. Owns its own ADR-1xx.
|
||||
- [ ] `wifi-densepose-nn` bindings (libtorch / candle wheel size TBD — see Open
|
||||
Questions §13.3).
|
||||
- [ ] `wifi-densepose-ruvector` bindings (RuVector attention types).
|
||||
- [ ] MQTT/Matter integration helpers (`wifi_densepose.client.matter`).
|
||||
- [ ] Deprecation notice on `wifi-densepose==1.x` releases (PyPI yank — see §9).
|
||||
- [ ] `wifi-densepose-sensing-server` binary distribution via pip extra
|
||||
(`pip install wifi-densepose[server]` fetches pre-built binary for the platform).
|
||||
- [ ] HACS Python integration built on top of the pip client layer (follow-on to
|
||||
ADR-115 §6.A).
|
||||
|
||||
---
|
||||
|
||||
## 7. Compatibility and deprecation
|
||||
|
||||
### 7.1 Version bump strategy
|
||||
|
||||
`wifi-densepose==2.0.0` is a **hard major-version break**. The 1.x import namespace
|
||||
`src.*` is incompatible with the 2.x namespace `wifi_densepose.*`. There is no shim
|
||||
that can bridge them transparently.
|
||||
|
||||
### 7.2 Tombstone release: v1.99.0
|
||||
|
||||
Before publishing v2.0.0, publish `wifi-densepose==1.99.0` as a pure-Python sdist/wheel
|
||||
whose sole content is:
|
||||
|
||||
```python
|
||||
# wifi_densepose/__init__.py (v1.99.0)
|
||||
raise ImportError(
|
||||
"wifi-densepose 1.x has been superseded by v2.0.0 which wraps "
|
||||
"the Rust-based stack. Run:\n\n"
|
||||
" pip install wifi-densepose==2.0.0\n\n"
|
||||
"Migration guide: https://github.com/ruvnet/RuView/blob/main/docs/pip-migration.md\n"
|
||||
"Legacy v1 source: archive/v1/ in the repository"
|
||||
)
|
||||
```
|
||||
|
||||
This ensures any project pinned to `wifi-densepose>=1` that upgrades to 1.99.0 gets a
|
||||
clear error rather than a silent broken import.
|
||||
|
||||
### 7.3 PyPI yank strategy
|
||||
|
||||
After v2.0.0 is stable (90-day observation window):
|
||||
|
||||
- Yank `wifi-densepose==1.0.0` — never had a separate stable release period; was
|
||||
superseded 4 hours after publication.
|
||||
- Leave `wifi-densepose==1.1.0` un-yanked but deprecated in the description.
|
||||
- Publish `wifi-densepose==1.99.0` as the canonical 1.x landing page (raise error).
|
||||
|
||||
Yanked versions remain installable with `pip install wifi-densepose==1.1.0 --force`
|
||||
so users with reproducible builds pinned to exact versions are not broken silently.
|
||||
|
||||
### 7.4 Semver
|
||||
|
||||
| Version | Content |
|
||||
|---|---|
|
||||
| 1.0.0 – 1.1.0 | Legacy Python server (archive/v1/) |
|
||||
| **1.99.0** | Tombstone: ImportError migration notice |
|
||||
| **2.0.0** | PyO3 Rust bindings + WS/MQTT client |
|
||||
| 2.x.y | Additive bindings + client improvements |
|
||||
| 3.0.0 | If/when nn bindings added (libtorch wheel size may force a separate package) |
|
||||
|
||||
---
|
||||
|
||||
## 8. Alternatives considered and rejected
|
||||
|
||||
### Alt-A: Subprocess wrapper
|
||||
|
||||
Package the pre-built `wifi-densepose-sensing-server` Rust binary inside the pip wheel.
|
||||
Python calls it via `subprocess`. **Rejected** because: the binary is 15–30 MB stripped;
|
||||
the install footprint is prohibitive; offline DSP scripting still requires the server to
|
||||
be running; the witness chain cannot exercise Rust code through a black-box binary.
|
||||
|
||||
### Alt-B: REST/WS client only
|
||||
|
||||
Ship a pure-Python package that is purely a client to a running `sensing-server`
|
||||
instance. **Rejected** because: it provides zero offline utility; it cannot host the
|
||||
witness chain over the Rust pipeline; it solves the "Python access to telemetry" problem
|
||||
but not the "Python DSP / prototyping" problem that academic and embedded users need.
|
||||
|
||||
### Alt-C: Pure Python reimplementation
|
||||
|
||||
Rewrite the DSP pipeline in pure Python/NumPy to reach parity with the Rust
|
||||
implementation. **Rejected explicitly** — this is the root cause of the current 11-month
|
||||
drift and the pattern this ADR is designed to exit. Any Python reimplementation will
|
||||
immediately begin drifting again as the Rust stack evolves.
|
||||
|
||||
---
|
||||
|
||||
## 9. Risks
|
||||
|
||||
| Risk | Likelihood | Severity | Mitigation |
|
||||
|---|---|---|---|
|
||||
| **Build matrix complexity** — 5 target triples × cibuildwheel setup; CI time; QEMU for aarch64 cross-compile | High | Medium | Use `abi3-py310` (5 wheels not 20); QEMU aarch64 emulation available in GitHub Actions; maturin handles auditwheel automatically |
|
||||
| **Binary size** — future nn/ONNX bindings may push wheel past 50 MB | Medium | High | Keep nn bindings in a separate `wifi-densepose-nn` PyPI package; keep core+vitals+signal wheel lean (~2 MB stripped) |
|
||||
| **GIL / async issues** — PyO3 wrapping tokio crates requires careful runtime management; `py.allow_threads` must be used around all blocking Rust calls | High | High | Restrict initial bindings to synchronous Rust APIs (vitals, signal, core are all sync); async sensing-server client stays in pure-Python `client/ws.py` |
|
||||
| **Maintainer overhead** — two languages, two build systems, one PyPI package | Medium | Medium | maturin unifies the build; CI handles publishing; start with 3 bound crates only |
|
||||
| **1.x user breakage** — users pinned to `wifi-densepose>=1,<2` will get the tombstone | Low | Medium | 1.99.0 tombstone gives a clear error; maintain 1.1.0 on PyPI un-yanked for 90 days post-v2 |
|
||||
| **Windows Rust toolchain in CI** — linking PyO3 on Windows requires MSVC or mingw; extra CI complexity | Medium | Medium | GitHub Actions `windows-latest` has MSVC; maturin + cibuildwheel handle this natively |
|
||||
| **Stable ABI limitations** — `abi3` precludes some advanced PyO3 features (e.g. `Buffer` protocol) | Low | Low | Core/vitals/signal types are scalar/Vec<f32> — no need for buffer protocol in P2–P3 |
|
||||
| **PyPI name ownership** — we own `wifi-densepose` on PyPI (confirmed via rUv author field) | Low | Low | Confirm with `pypi.org/user/ruvnet` before publishing |
|
||||
|
||||
---
|
||||
|
||||
## 10. Acceptance criteria
|
||||
|
||||
The following checks must all pass before ADR-117 is considered Accepted:
|
||||
|
||||
- [ ] `pip install wifi-densepose==2.0.0` succeeds on Python 3.10, 3.11, 3.12, 3.13
|
||||
on linux/x86_64, macos/arm64, and windows/amd64 in a clean venv with no extra build tools.
|
||||
- [ ] `python -c "import wifi_densepose; print(wifi_densepose.__version__)"` prints `2.0.0`.
|
||||
- [ ] `python -c "from wifi_densepose import CsiFrame; f = CsiFrame([1.0]*56, [0.0]*56, 56, 0, 100.0); print(f)"` produces a non-error repr.
|
||||
- [ ] The 4-stage vitals pipeline processes 1,000 frames in under 500 ms on a
|
||||
reference machine (CPython 3.12, linux x86_64, no GPU).
|
||||
- [ ] `wifi_densepose.witness.verify_bundle(path)` returns `verdict="PASS"` for a
|
||||
freshly generated witness bundle from `scripts/generate-witness-bundle.sh`.
|
||||
- [ ] `wifi_densepose.client.ws.SensingClient` receives at least one `edge_vitals`
|
||||
message from a `sensing-server --mock-frames` instance within 5 seconds.
|
||||
- [ ] `pip install wifi-densepose==1.99.0` raises `ImportError` with the migration URL.
|
||||
- [ ] The compiled `_core` extension has no unresolved dynamic library dependencies
|
||||
beyond libc/msvcrt (verified by `auditwheel show` on Linux, `delocate-listdeps` on macOS).
|
||||
- [ ] Type stubs (`wifi_densepose/*.pyi`) are present; `mypy --strict` passes on the
|
||||
example code in `examples/vitals_from_buffer.py`.
|
||||
- [ ] Total wheel size for core+vitals+signal: `≤ 5 MB` per platform.
|
||||
|
||||
---
|
||||
|
||||
## 11. Open questions
|
||||
|
||||
1. **Stable ABI base version**: `abi3-py310` drops support for Python 3.9, which v1.1.0
|
||||
declared. Is Python 3.9 EOL-enough (EOL 2025-10-05) to drop cleanly? *Tentative: yes,
|
||||
drop 3.9. Use abi3-py310.*
|
||||
|
||||
2. **Package name for nn bindings**: if `wifi-densepose-nn` bindings require a 30 MB
|
||||
libtorch wheel, should they live at `wifi-densepose-nn` (separate PyPI package) or
|
||||
as an optional heavy extra of `wifi-densepose[nn]`? *Tentative: separate package to
|
||||
avoid polluting the lean wheel.*
|
||||
|
||||
3. **Witness hash continuity**: the Rust pipeline will produce a different SHA-256 than
|
||||
the v1 Python pipeline for the same input frames. The new `expected_features_v2.sha256`
|
||||
must be generated and committed before v2.0.0 ships. Who generates it, and how is
|
||||
the generation process itself witnessed? *Tentative: generate in CI, commit hash to
|
||||
`archive/v1/data/proof/`, include in ADR-028 matrix.*
|
||||
|
||||
4. **`ruv-neural` crate**: `v2/crates/ruv-neural/` exists in the workspace. Is it a
|
||||
candidate for early Python bindings (useful for training-loop scripting), or should
|
||||
it wait for the nn/train tier? *Tentative: defer — it depends on training backends.*
|
||||
|
||||
5. **Tokio runtime**: `wifi-densepose-sensing-server` is tokio-based, but the three
|
||||
crates bound in P2–P3 (`core`, `vitals`, `signal`) are synchronous. Are there any
|
||||
hidden tokio dependencies that would force a runtime into the extension module?
|
||||
*Tentative: inspect each crate's Cargo.toml for tokio deps before P1 scaffold.*
|
||||
|
||||
6. **`pyo3-stub-gen` vs manual stubs**: automated stub generation from PyO3 has rough
|
||||
edges for generics and newtype patterns. Should we hand-write `.pyi` stubs for the
|
||||
first release? *Tentative: use `pyo3-stub-gen` for scaffolding, hand-tune for public
|
||||
API.*
|
||||
|
||||
7. **`wifi_densepose` vs `wifi-densepose` namespace**: the pip package name uses a dash
|
||||
(`wifi-densepose`) but Python imports use underscores (`wifi_densepose`). The v1
|
||||
package shipped under `src.*`, not `wifi_densepose.*`. Is there any tooling that
|
||||
hardcodes the `src` namespace? *Tentative: the `src.*` namespace was specific to
|
||||
`archive/v1/` and is cleanly dropped.*
|
||||
|
||||
8. **cibuildwheel version**: the current stable is cibuildwheel v2.x. Does the
|
||||
project's existing GitHub Actions config need updates for maturin builds vs
|
||||
the current `cargo build` / `build.py` patterns? *Tentative: yes, add a separate
|
||||
`pip-release.yml` workflow; do not modify existing Rust CI.*
|
||||
|
||||
9. **RuVector bindings timeline**: the `wifi-densepose-ruvector` crate (`v2/crates/`)
|
||||
depends on `ruvector-gnn = "2.0.5"`. Does ruvector-gnn ship as a pre-built static
|
||||
lib or require linking at build time? This directly affects the P6+ wheel size.
|
||||
*Tentative: investigate ruvector-gnn link strategy before committing to a timeline.*
|
||||
|
||||
10. **`wifi_densepose.client.ha` conflict with ADR-115/116**: the `ha.py` helper module
|
||||
should not duplicate the ADR-115 MQTT discovery logic in Python. Should it be read-only
|
||||
(parse HA discovery JSON → Python dataclasses) or also write (publish discovery JSON)?
|
||||
*Tentative: read-only for v2.0. Write path deferred to the HACS integration follow-on
|
||||
(ADR-115 §6.A).*
|
||||
|
||||
11. **BFLD Rust crate ownership** (added 2026-05-24): the P3.5 BFLD bindings ship with a
|
||||
stub Rust impl in `python/src/bfld_stub.rs`. The proper Rust crate (Nexmon BFR pcap
|
||||
parser + `mac80211` debugfs ingestor) will land later. Should it be a new
|
||||
`wifi-densepose-bfld` workspace member, or should it extend `wifi-densepose-signal`?
|
||||
*Tentative: new dedicated crate. Reasons: (a) the BFR parser is significant code
|
||||
(Wireshark's dissector is ~2k lines) and bloats `-signal`; (b) BFLD ingestion is
|
||||
optional — many deployments will only use CSI; gating behind a separate crate keeps
|
||||
the default `-signal` lean. Decide before committing to the crate name in any
|
||||
`pyproject.toml` extras.*
|
||||
|
||||
12. **BFLD per-vendor compressed-angle variants** (added 2026-05-24): 802.11 standardizes
|
||||
the compressed beamforming feedback format but vendors (Broadcom, Intel, Qualcomm,
|
||||
MediaTek) differ in psi/phi quantization step + ordering of consecutive matrix
|
||||
entries. How much normalisation belongs in the Python `BfldFrame.from_compressed_feedback`
|
||||
binding vs. the future Rust crate? *Tentative: Python binding is dumb (numpy ndarray
|
||||
in, numpy ndarray out — no decoding); the future Rust crate owns per-vendor
|
||||
normalisation, exposed via a `Vendor` enum on the binding constructor. Confirm via
|
||||
a per-vendor test fixture before P3.5 ships.*
|
||||
|
||||
---
|
||||
|
||||
## 12. References
|
||||
|
||||
### BFLD references (added 2026-05-24 for §5.7a + §11.11 + §11.12)
|
||||
|
||||
- Hernandez & Bulut, *"Wi-Fi Sensing With Compressed Beamforming Feedback"*, ACM TOSN 2024 — first systematic survey of BFR-as-sensing
|
||||
- Yousefi, Soltanaghaei & Bharadia, *"Just-In-Time Wi-Fi Sensing Using Compressed Beamforming Feedback"*, MobiSys 2023 — practical pipeline for breath + heart-rate extraction from sniffed BFR
|
||||
- IEEE 802.11ax-2021 §27.3.10 — Compressed Beamforming Feedback frame format
|
||||
- Wireshark BFR dissector — `packet-ieee80211.c` reference implementation
|
||||
- AX210 Linux mac80211 debugfs BFR capture path (kernel 6.10+)
|
||||
- Sample BFR-vs-CSI parity dataset — TBD; we'll publish one alongside the
|
||||
`wifi-densepose-bfld` crate when it lands
|
||||
|
||||
### Original references
|
||||
|
||||
- **PyPI package (current)**: https://pypi.org/project/wifi-densepose/ — v1.1.0, released 2025-06-07
|
||||
- **PyPI JSON metadata**: https://pypi.org/pypi/wifi-densepose/json
|
||||
- **Local source**: `archive/v1/setup.py`, `archive/v1/src/__init__.py`, `archive/v1/data/proof/verify.py`
|
||||
- **Rust workspace**: `v2/Cargo.toml`, `v2/crates/wifi-densepose-core/src/lib.rs`,
|
||||
`v2/crates/wifi-densepose-vitals/src/lib.rs`, `v2/crates/wifi-densepose-signal/src/lib.rs`,
|
||||
`v2/crates/wifi-densepose-sensing-server/src/lib.rs`
|
||||
- **PyO3 docs**: https://pyo3.rs/ — v0.28.3 stable, Rust ≥1.83 required
|
||||
- **maturin docs**: https://maturin.rs/ — supports Python 3.8+ on Linux/macOS/Windows/FreeBSD
|
||||
- **cibuildwheel docs**: https://cibuildwheel.pypa.io/
|
||||
- **ADR-021**: ESP32 vitals — defines the HR/BR extraction pipeline this ADR exposes in Python
|
||||
- **ADR-028**: ESP32 capability audit — defines the witness bundle format `witness/verify.py` must re-verify
|
||||
- **ADR-115**: HA-DISCO + HA-MIND + HA-FABRIC — defines the MQTT topic structure the `client/mqtt.py` helper consumes
|
||||
- **ADR-116**: HA-COG cog packaging — parallel effort; ADR-117 pip library is the developer-facing Python surface; ADR-116 is the Seed-installable artifact
|
||||
Reference in New Issue
Block a user