bc0e8fd0 added `serde_json` to the aether crate's [dev-dependencies] for the
native golden-parity test, but v2/Cargo.lock was not regenerated then (that
worktree had no submodules, so the workspace could not build). CI passed only
because it does not use `--locked`; committing the lockfile keeps Cargo.toml and
Cargo.lock in sync and lets a `--locked`/`--frozen` build succeed.
Co-Authored-By: Ruflo & AQE
Closes the two findings from the adversarial review that were verified but left
unfixed. Both now have proper tests, each proven to fail against the old code.
1. PyO3 bindings panicked / over-allocated on caller input (aether.rs).
`EmbeddingExtractor(n_heads=0)` reached `d_model % n_heads` in the transformer
and panicked (surfacing to Python as an opaque PanicException); a non-divisor
head count tripped the native assert; and `AetherConfig(d_model=100_000)`
allocated multi-gigabyte weight matrices that abort the interpreter. The
binding did no validation.
Now both constructors return `PyResult` and validate at the boundary —
positive dims, `d_model % n_heads == 0`, and a generous MAX_DIM/MAX_LAYERS cap
— raising `ValueError`. Proven: `n_heads=0` -> "n_heads must be positive",
`d_model=100_000` -> a clean ValueError, both previously a
PanicException / abort. +10 pytest cases (test_aether.py); a valid config
still constructs and embeds.
2. The single-training-job guard was a TOCTOU race (training_api.rs).
`spawn_training_job` checked `is_active()` under a `state` READ lock, released
it, then set `active` later. A tokio RwLock read lock is SHARED, so two
concurrent `POST /train/start` could both hold it, both see the slot free, and
both spawn jobs — sharing/overwriting one status+cancel and orphaning a task
handle.
Extracted `claim_training_slot`, which does the check-and-set in ONE `status`
mutex scope — the atomicity lives on the status mutex, not the coarse state
lock — so concurrent starts serialise and exactly one wins. This also makes it
unit-testable without a full AppState.
Test: 32 threads hit a barrier and race to claim; asserts EXACTLY ONE wins.
Mutation-proven — reverting to the split check-then-set makes it fail
(`left: 3, right: 1`), and it returns to 1 with the fix.
Verified on aarch64/macOS: training_api 28 pass (26 existing + 2 new), full
python/tests suite 237 pass (227 + 10). The native module keeps its internal
assert as a defence-in-depth invariant; the binding now enforces it at the edge.
Co-Authored-By: Ruflo & AQE
A cross-vendor review of the parity rework (2febbb81) found three issues; all
verified and fixed here.
1. HIGH — the native≈golden half never ran in CI, so a binding marshalling bug
could pass. The golden vectors were regenerated through the Python binding,
and only pytest (binding vs that golden) runs in CI — the native reference
tests in `python/tests/aether_parity.rs` link against the PyO3 crate and no
workflow runs them. A stable PyO3 conversion defect present at regeneration
would therefore be baked into the golden and go undetected.
Fix: a real native parity test IN the `wifi-densepose-aether` crate
(`tests/golden_parity.rs`), which is std-only and a member of the v2
workspace, so it runs under the existing `cargo test --workspace`. It
recomputes the embedding with no PyO3/marshalling and asserts it matches the
SAME committed golden within tolerance. Now native≈golden AND binding≈golden
both run in CI ⇒ binding≈native, and a binding-specific artifact in the
golden surfaces here as a native mismatch. serde_json added as a
dev-dependency only (test-only; never linked into the lib or the wheel, so
the crate stays runtime-dependency-free).
Independently proven at this commit: native output equals the committed
golden BIT-FOR-BIT (128/128 exact bits, Δ=0) for both base and loaded — the
golden is native-faithful today; this test keeps it that way.
2. MEDIUM — the Python parity helper passed non-finite output. `abs(nan - b) >
tol` is False, so an all-NaN embedding slipped through. Now every element
must be `math.isfinite` first. Proven: an all-NaN vector is rejected
("element 0 is not finite (nan)"); the Rust helpers already caught it via
`<=`.
3. LOW — a coherent shift inside the per-element tolerance could move the whole
vector undetected. Added a whole-vector cosine-similarity bound (≥ 1 - 1e-6)
alongside the per-element check.
Verified on aarch64/macOS: `test_aether` 13, full `python/tests/` suite 227
passed against a `--features sota` build; the new native test passes
(2/2) run standalone (the in-worktree `cargo test -p` only failed on an
un-checked-out submodule; ci.yml checks out submodules recursively).
Co-Authored-By: Ruflo & AQE
Reverses the release-wheel packaging change from 3ed43e9a after review.
That commit built the published wheels with `--features sota` so
`pip install wifi-densepose[aether]` would work. But it did so by abandoning a
DELIBERATE, guarded design: `default = []`, the
RuView#1387-default-wheel-budget-config fix-marker, and the wheel-size-budget
job exist specifically to keep the base wheel small and make SOTA opt-in
(ADR-117 §5.4). Building SOTA into every published wheel makes the base fat,
turns the `[aether]`/`[mat]`/`[meridian]` extras into inert no-ops, and — worse
— leaves the PUBLISHED wheel un-budget-checked (the budget job only builds the
no-features wheel), so the ONNX Runtime dependency the fix-marker anticipates
for `mat` would later balloon the shipped wheel past 5 MiB silently.
A pip extra cannot enable a cargo feature on a prebuilt wheel, so "small base +
opt-in SOTA in binary wheels" is only achievable by splitting into separate
packages — tracked as #1412. Until that lands, P6 is source-build-only, and the
honest thing is to say so.
This commit:
- Restores pip-release.yml to its original state (no --features in the release
build; original smoke test). Net packaging change across the branch is now
ZERO — the small-base design is fully intact.
- Points the aether/mat/meridian import shims at the real interim path: a source
build with `--features <x>` (or `--features sota`), and #1412 for the binary
wheels follow-up. The earlier revision wrongly claimed "the official wheels
include it", which is only true once #1412 ships.
KEPT from 3ed43e9a (correct regardless of packaging): the AETHER parity tests
are now arch-portable — they compare a golden VECTOR within tolerance instead of
a SHA-256 of raw f32 bytes, which only ever matched the one architecture that
generated it and failed on the aarch64/macOS-arm wheels this project builds.
Verified on aarch64/macOS with a `--features sota` build (as python-ci does):
test_aether 13, test_mat 7, test_meridian 13 pass; full suite unaffected.
Co-Authored-By: Ruflo & AQE
Three coupled defects, all found by building the wheel and running the real
suite on Apple Silicon rather than trusting green CI.
1. THE P6 SOTA BINDINGS NEVER REACHED USERS (release blocker).
`pip-release.yml`'s cibuildwheel built the DEFAULT feature set, so published
wheels contained none of aether/mat/meridian. `pip install
wifi-densepose[aether]` then raised ImportError — and the extra is empty, so
its own error message ("install the [aether] extra") sent users in a circle.
A pip extra cannot enable a Rust cargo feature on an already-built wheel, so
the only way P6 reaches PyPI is to compile it in.
Fix: the RELEASE build opts in via `MATURIN_PEP517_ARGS="--features sota"`
(per-platform, since CIBW_ENVIRONMENT_LINUX overrides CIBW_ENVIRONMENT).
`default = []` in Cargo.toml, the RuView#1387-default-wheel-budget-config
fix-marker, and the wheel-size-budget job are all left UNTOUCHED — they keep
guarding the small base compile. Measured published wheel: 1.68 MiB, well
under the ADR-117 §5.4 5 MiB budget. Proven: built via the exact PEP517 path,
`import wifi_densepose.aether/mat/meridian` all succeed.
2. THE WHEEL SMOKE TEST COULD NOT SEE #1.
CIBW_TEST_COMMAND only asserted `hello()` and PRINTED `__build_features__`.
The one signal that would reveal missing bindings was dumped to stdout and
ignored, so a featureless wheel published green. It now ASSERTS the p6
features are present and imports the three modules. Proven red→green: fails
on the old featureless wheel ("missing SOTA bindings: [...]"), passes on the
fixed one.
3. THE AETHER PARITY TESTS WERE NOT PORTABLE ACROSS THE WHEEL MATRIX.
`test_aether.py` and the native `aether_parity.rs`/`aether_weights_parity.rs`
hashed the raw f32 embedding bytes (SHA-256) against a committed golden. The
embedding is pure f32 with transcendental ops (ln/sqrt/cos) that are not
bit-reproducible across CPUs/libm, so the hash only ever matched the one arch
that generated it. This passed in python-ci (x86) but fails on the aarch64 /
macOS-arm wheels this same project ships — latent until #1 is fixed and the
bindings actually load. Every tolerance/behavioral assertion already passed;
only the two byte-hash tests failed, which is the signature of a non-portable
golden, not a logic bug.
Fix: compare to a committed golden VECTOR within tolerance
(atol=rtol=1e-4 — ~100x cross-arch f32 drift, ~100x under any real algorithm
change), on both the Python and native sides against the SAME golden. Native
≈ golden and binding ≈ golden together prove binding ≈ native, portably.
`.sha256` goldens replaced by `.json` vectors.
Also: the aether/mat/meridian import shims told users to `pip install
wifi-densepose[<x>]` on a missing feature — an empty extra that cannot help.
Corrected to name the real fix (rebuild with `--features <x>`); message tests
updated to assert the honest message and forbid the misleading one.
Verified on aarch64/macOS: full `python/tests/` suite 227 passed against a wheel
built with the new mechanism; `cargo check --tests --features aether` clean.
The native `.rs` parity tests were converted by inspection and cargo-checked but
not executed — they link against the PyO3 crate and no workflow runs them today
(a pre-existing gap; wiring `cd python && cargo test --features sota` into
python-ci would make the native anchor actually run).
Co-Authored-By: Ruflo & AQE
SensingClient now accepts a `token=` argument, defaulting to the
RUVIEW_API_TOKEN env var, and sets `Authorization: Bearer <token>`
directly on the WebSocket upgrade — no browser-style ticket workaround,
since Python can set the header on the handshake. Empty/unset token
means no auth (auth-disabled path unchanged).
websockets version-compat: the header keyword was renamed in the
`>=12.0` range this package pins (`extra_headers` <= 13,
`additional_headers` >= 14). Resolved by runtime detection —
`_select_header_kwarg()` inspects `websockets.connect`'s signature and
passes the correct keyword — rather than raising the floor to `>=14`.
Chosen deliberately so existing users on older websockets are not forced
to upgrade; keeps the `websockets>=12.0` pin valid.
Scope: only ws.py connects to the auth-enabled sensing-server. mqtt.py
already supports username/password (its own auth convention); ha.py and
primitives.py do no network I/O. OAuth (ADR-271) is out of scope.
Tests: constructor token, env-var token, constructor-overrides-env,
no-token/empty-token (no header), a parametrized test across both
websockets kwarg conventions, and an end-to-end test asserting the
bearer reaches the in-process server's upgrade request.
The python/ PyO3+maturin wheel (wifi-densepose, ADR-117/185) had ZERO
per-PR CI coverage: pip-release.yml only fires on release triggers, so a
PR could break the wheel build, a native-Rust parity test, or the wheel
budget and nothing in the gating CI would catch it until release day.
New .github/workflows/python-ci.yml (path-scoped to python/** + the
backing crates, following the repo convention cf. firmware-ci.yml --
GitHub has no per-job paths filter and no workflow here uses a
change-detection action, so a dedicated path-scoped workflow is the
idiomatic, no-new-dependency choice):
* parity-tests: cargo build of the aether/mat/train backing crates
(fast-fail attribution), then `maturin develop --features sota` +
`pytest python/tests/ -q`. sota builds the aether/meridian/mat
compiled submodules so their SHA-256 parity tests against native
Rust actually run (those test modules import the submodules at
collection time -- without the features they error, not skip).
* wheel-size-budget: builds the DEFAULT (no-features) wheel and fails
if it exceeds the ADR-117 5.4 5 MiB budget -- the real numeric
enforcement a string fix-marker cannot provide.
Two fix-marker guards (scripts/fix-markers.json):
* RuView#1387-rvf-filename-collision: training_api.rs next_model_id()
must keep the microsecond timestamp (%6f) + AtomicU64 counter; the
forbid uses a negative lookahead so it fires only on a bare
second-resolution format lacking the disambiguator. Guards the
CI-caught collision fixed on this branch in 8409f434c.
* RuView#1387-default-wheel-budget-config: default Cargo features
empty, SOTA deps optional, maturin strip=true -- the config that
keeps the default wheel small. Bytes are enforced by the CI job
above, not a string marker (per the marker system's limits).
Verification:
- python scripts/check_fix_markers.py -> All 27 fix markers present (0 regressed)
- all 26 .github/workflows/*.yml parse (yaml.safe_load, utf-8)
- cargo build -p wifi-densepose-aether -> Finished (exit 0)
- forbid regexes confirmed: fire on a simulated regression, clean on the current tree
Co-Authored-By: claude-flow <ruv@ruv.net>
Reverts the OIDC-only publish path from cc153e8b5. Restores password:
${{ secrets.PYPI_API_TOKEN }} on all four publish steps and drops the
id-token: write / environment: pypi blocks from publish-v2 and
publish-tombstone. OIDC-only is a net regression until the manual
pypi.org Trusted Publisher registration exists — publishing would fail
'no trusted publisher configured'. Token auth is the active working path.
OIDC design retained as documented TODO(ADR-184 P1b) in the header; the
switch becomes a dedicated gated follow-up once the owner confirms
registration. Restores compliance with fix-marker RuView#786-pypi-token-auth.
Refs ADR-184 P1.
Closes the ADR-185 §13.a follow-up (the tractable, data-independent one of
the three §6.7 gaps): the bound EmbeddingExtractor was random-Xavier-init
only, with NO way to load real weights — so it was structurally untrained.
This adds the *capability* to load weights whenever a trained checkpoint
exists. It does NOT itself produce trained/SOTA embeddings and does NOT
close §6.7 (still no trained checkpoint, no labeled data, no eval harness).
Serialization: not greenfield — EmbeddingExtractor already had
flatten_weights()/unflatten_weights() (flat Vec<f32>). wifi-densepose-aether
is a deliberately dependency-free std-only leaf crate (ADR-185 §13), so
rather than add safetensors/serde/bincode (which would undo the zero-dep
property), the on-disk format is raw little-endian f32 with a 12-byte header
(magic "AETHERW1" + u32 param count) — zero new deps.
Native (v2/crates/wifi-densepose-aether/src/embedding.rs):
- EmbeddingExtractor::save_weights(path) / load_weights(path). load_weights
never panics: errors on unreadable file, short/oversized payload, bad
magic, or a param-count mismatch (delegated to unflatten_weights).
- Default (no weights) construction is UNCHANGED — still random init,
clearly labeled untrained. Purely additive.
Python binding (python/src/bindings/aether.rs, .pyi):
- EmbeddingExtractor.load_weights(path) / save_weights(path) / param_count,
GIL-released, ValueError on bad input.
Tests (both Rust and Python, per the task):
- Rust unit (embedding.rs): load_weights_actually_replaces_weights_and_
round_trips proves the loaded weights MOVE the embedding away from the
random-init baseline (not a silent no-op), match the source extractor
(round-trip), and are bit-identical after the file round-trip; plus a
bad-magic/wrong-count rejection test.
- Cross-language golden (aether_weights_parity.rs + test_aether.py): a
shared deterministic weight formula (w[i]=k/65536-0.5, exact in f32+f64)
written to the AETHER format; both native Rust and the Python binding load
it and must produce the byte-identical embedding SHA-256
(tests/golden/aether_loaded_embedding.sha256) — proving the binding's
load path is bit-identical to native, and (vs the random baseline) that
the loaded weights are actually used.
Verified:
cargo test -p wifi-densepose-aether 98 passed, 0 failed
cargo test --features aether --test aether_parity
--test aether_weights_parity 3 passed, 0 failed
maturin develop --features aether + pytest test_aether.py 13/13 pass
default cargo build (no aether feature) clean
Root cause: the trained-model filename was `trained-{type}-{%Y%m%d_%H%M%S}` — a
**second-resolution** timestamp. Two training runs of the same type that finish
in the same wall-clock second compute the SAME path and silently overwrite each
other's artifact. This is a real robustness gap (a second rapid run clobbers the
first model), and it surfaced as a flaky test on the Linux CI runner:
`http_train_start_produces_model_and_streams` asserts (via a before/after
directory diff) that a new `.rvf` appeared, but the concurrent
`training_job_streams_real_progress_and_writes_model` test — same `supervised`
type, same second — wrote to the identical path and then deleted it in its own
cleanup, so the diff saw no new file. Windows scheduling happened to avoid the
same-second overlap; Linux CI hit it (the sibling `TRAIN_ENV_LOCK` holders then
failed via std::sync::Mutex poison-cascade after the first panic).
Reproduced locally by isolating the three model-writing tests (which forces the
overlap): ~10/20 runs flaked before; 0/30 after.
Fix: `next_model_id()` appends microseconds (`%6f`) plus a monotonic per-process
`AtomicU64` counter, so every run gets a distinct path even for same-microsecond
concurrent completions. Pinned by `model_ids_are_unique_per_call` (1000 ids, all
distinct) so the scheme can't regress to non-unique. No test-level band-aid
(no added sleeps / serialization) — the collision is removed at the source.
Verified: cargo test -p wifi-densepose-sensing-server -p wifi-densepose-train
--no-default-features — bin 218 passed / 0 failed, all suites 0 failed; the
isolated-tests stress loop is 0/15 flakes post-fix.
ADR-185 §13 flagged the `[aether]` wheel as linking the whole
`wifi-densepose-sensing-server` Axum/tokio/worldgraph/ruvector tree via the
non-optional deps of that crate. Hoist the pure-compute AETHER stack out so the
binding depends only on pure Rust.
New crate `wifi-densepose-aether` (std-only, ZERO external deps) holds the four
self-contained modules the AETHER surface needs — `embedding`, `graph_transformer`,
`sona`, `sparse_inference` (verified: no serde/tokio/axum/crate:: refs outside the
set; the four form a closed dependency graph). `wifi-densepose-sensing-server`
now `pub use`s them from the leaf crate, so its own code (`crate::embedding`, …)
and public API (`wifi_densepose_sensing_server::embedding`, …) are unchanged —
`main.rs`/`trainer.rs` untouched. The Python binding + parity test + `[aether]`
feature repoint at the leaf crate; the sensing-server dep is dropped from python.
MEASURED wheel size (maturin build --release --features aether --strip; each
wheel install-verified to actually contain `AetherConfig`):
BEFORE (aether -> sensing-server): 369,782 B (~361 KB)
AFTER (aether -> leaf crate): 319,719 B (~312 KB)
HONEST CORRECTION to the §9/§13 premise: the pre-hoist wheel was NOT over the
ADR-117 §5.4 5 MB budget — it was ~361 KB, ~14x under. Linker dead-code
elimination (--gc-sections on the pyo3 cdylib) already strips the unreached
server tree because the binding only reaches pure-compute symbols. So this is
"blows the budget" CLAIMED, not MEASURED. The hoist is still worthwhile and
lands the real, measured wins:
- `[aether]` build time 71s -> 12s (no server tree to compile),
- python/Cargo.lock -1238 lines (server tree no longer resolved),
- ~50 KB smaller wheel,
- honest, minimal declared dependency surface + removes the latent risk that a
future change makes some server code reachable and *then* bloats the wheel.
Verified this session (real counts):
- cargo test --features aether --test aether_parity: 2 passed / 0 failed
- pytest tests/test_aether.py (maturin develop --features aether): 9 passed
- cargo test -p wifi-densepose-sensing-server --no-default-features: 217 bin +
388 lib, 0 failed (no regression)
- cargo test -p wifi-densepose-aether: 96 passed (the relocated unit tests)
The §9/§13 wheel-size narrative in ADR-185 (owned by the ADR author) should be
corrected to reflect the measured reality — flagged to the team lead.
ADR-184 P2 (version promotion + changelog) — version-metadata prep only;
triggers NO publish (P3's real PyPI upload is still gated on the manual
Trusted Publisher registration on pypi.org).
- python/pyproject.toml: version 2.0.0a1 -> 2.0.0; classifier
"Development Status :: 3 - Alpha" -> "5 - Production/Stable".
- python/ruview-meta/pyproject.toml: version 2.0.0a1 -> 2.0.0; same
classifier bump; repointed the wifi-densepose dependency pins
(base + [client]) from ==2.0.0a1 to ==2.0.0 so the stable ruview
meta-package depends on the stable wifi-densepose, not the alpha.
- python/wifi_densepose/__init__.py: runtime __version__ 2.0.0a1 -> 2.0.0
(caught during the sanity check — it would otherwise contradict the
stable wheel version reported by `wifi_densepose.__version__`).
- CHANGELOG.md: [Unreleased] entry recording the promotion.
Sanity-checked that the core package genuinely deserves "Production/Stable"
(not just a version-string change):
maturin build --release --strip -> default wheel 279 KB
pytest python/tests/ (excluding [aether]/[meridian]/[mat] extra modules)
-> 185 passed, 0 failed (smoke/keypoint/pose/vitals/bfld/security/
WS+MQTT client)
post-bump: wheel installs as 2.0.0, __version__ = 2.0.0, smoke 6/6 pass.
No publish/tag/release commands were run.
Root cause (investigated, same rigor as the MAT fix): `wifi-densepose-train`
declared `wifi-densepose-nn` as a dependency but NEVER imported it anywhere
in src/ or bin/ — a dead dependency. (The tch-backend model path uses the
`tch` crate directly, not wifi-densepose-nn; `grep -r wifi_densepose_nn
src/` returns nothing.) That dead dep pulled `ort` (ONNX Runtime) +
reqwest/hyper into every downstream consumer, including the ADR-185
`[meridian]` wheel, which binds only train's pure-compute geometry /
rapid_adapt / eval modules + signal::hardware_norm.
`cargo tree -i wifi-densepose-nn` confirmed train was the SOLE path pulling
nn into the wheel; after removal, the package is absent from the graph.
Fix: remove the dead dep line from train's Cargo.toml (no code change, no
feature-gating needed — nothing referenced it). Not a hoist: the coupling
turned out to be nonexistent, not deep.
Measured (maturin build --release --features meridian --strip):
BEFORE: 1.8 MB AFTER: 1.7 MB (both already within the ADR-117 §5.4
<=5 MB budget). Honest note: unlike MAT — which instantiates OnnxSession
and thus bundles the ort native runtime (8.4 MB) — train never references
ort, so the native lib was dead-stripped and the wheel was already under
budget. The real win is removing ort/reqwest/hyper from train's dependency
graph and compile for ALL consumers, not a dramatic wheel shrink.
Verified:
cargo build -p wifi-densepose-train (default=[]) OK
cargo tree -i wifi-densepose-nn -> no longer in the graph
cargo test --features meridian --test meridian_parity 2/2 pass
maturin develop --features meridian + pytest test_meridian 13/13 pass
Root cause (investigated, not assumed): the heavy deps in a [mat] wheel
were NOT on the survivor-detection/triage path the Python binding uses.
`wifi-densepose-mat` pulled `wifi-densepose-nn` as a NON-optional dep, and
nn's default `onnx` feature drags in `ort` (ONNX Runtime) + its
download/reqwest/hyper stack. But nn is used ONLY by mat's `src/ml/`
module (debris + vital-signs ONNX classifiers), which is an OPTIONAL
runtime enhancement: `DetectionPipeline.ml_pipeline` is `Option<_>`,
created only when `DetectionConfig::enable_ml` is set, and
`from_disaster_config` (the DisasterResponse path) never enables it. So
the ONNX stack was compiled-and-linked but never executed by the binding.
Fix = feature-gating (not a hoist — a hoist was unnecessary):
- `wifi-densepose-nn` made optional; new `ml` Cargo feature gates it.
- `default = [..., "ml"]` so existing consumers are unchanged; `api`
implies `ml` (the REST surface exposes `ml_ready`).
- `#[cfg(feature = "ml")]` on the `ml` module, its crate-root + prelude
re-exports, the `MatError::Ml` variant, and every ml touchpoint in
detection/pipeline.rs (ml_config/ml_pipeline fields, constructor,
process_zone enhancement block, enhance_with_ml/get_ml_results/
initialize_ml/ml_ready/ml_pipeline accessors, update_config).
- Fixed a latent feature-unification bug surfaced by dropping nn:
integration/hardware_adapter.rs uses `tokio::select!`, which needs
`tokio/macros`. That was only satisfied transitively via nn; now
declared explicitly on mat's own tokio dep so --no-default-features
builds compile.
The ADR-185 [mat] wheel already depended on mat with
`default-features = false, features = ["std"]`, so no python/ change was
needed — dropping `ml` from the default now actually removes ort.
Measured (maturin build --release --features mat --strip):
BEFORE (ml/ort in wheel): 8.4 MB (over the ADR-117 §5.4 <=5 MB budget)
AFTER (ml gated off): 2.0 MB (under budget) — 76% smaller
Verified:
cargo build -p wifi-densepose-mat (default+ml) OK
cargo build -p wifi-densepose-mat --no-default-features --features std OK
cargo test --features mat --test mat_parity 2/2 pass
maturin develop --features mat + pytest tests/test_mat.py 7/7 pass
(same detection result: 1 survivor, triage Delayed — behavior-transparent)
Completes ADR-186 phases P5 (dashboard honesty / fallback guarantee) and P6
(tests + witness) on top of the P1–P4 reconnection, and flips the ADR to Accepted.
P5 — dashboard honesty:
- Backend (training_api): a runtime enablement gate (`RUVIEW_DISABLE_SERVER_TRAINING`)
makes the three start endpoints return a structured `{enabled:false, reason,
cli:"wifi-densepose train-room"}` HTTP 409 (never a silent success) when server
training is disabled; `/api/v1/train/status` now carries an `enabled` flag. A runtime
flag (not a Cargo feature) so `--no-default-features` builds keep training ON (resolves
§9.4). Handlers return `Response` to carry the 409.
- Frontend (TrainingPanel.js): reads `enabled` off the status payload and disables the
Start/Pretrain/LoRA buttons with a CLI tooltip when server training is off; a rejected
start tears down the optimistically-opened WS and refreshes. The enabled path was
already fully wired (opens the WS before POST, renders live epoch/loss/ETA + terminal
state) — verified.
P6 — tests & witness:
- New HTTP-level tests in a `#[cfg(test)] mod adr186_http_tests` built on a minimal
`AppStateInner::minimal()` test-state helper: a live-socket test that completes a
genuine 101 WebSocket handshake (tokio-tungstenite) and receives a real progress frame
after a POST start; a 426-not-404 route-wired check; a full POST→poll-status→`.rvf`-exists
round-trip; and the disabled-409 structured-response test. Added `tokio-tungstenite`
dev-dep (version already in the workspace lock).
- CHANGELOG updated; ADR §6 ledger + §7 acceptance criteria checked off with evidence;
Status → Accepted. README/CLAUDE have no training route table (no edit needed).
Fixed a test-only parallelism race found under `cargo test --workspace`: two
model-writing tests deleted `.rvf`s by directory-diff and could remove a file a third
test asserted existed. Each test now cleans only its own artifact (data/models is
gitignored).
Verified this session: `cargo test -p wifi-densepose-sensing-server -p wifi-densepose-train
--no-default-features` — sensing-server bin 217 passed / 0 failed, all train suites 0
failed. Full `--workspace` re-run in progress to reconfirm untouched crates.
ADR-185 §4.2 pytest-benchmark micro-benchmarks + runnable examples +
README extras table for the aether/meridian/mat bindings.
- python/bench/test_bench_{aether,meridian,mat}.py — follow the existing
test_bench_vitals.py pattern (skipped by default; --benchmark-only).
- python/examples/{reid_from_csi,cross_room_calibrate,mat_triage}.py —
typed, runnable, mypy --strict clean.
- python/README.md — SOTA extras table + example links.
Measured on a RELEASE wheel (maturin develop --release --features sota),
reference machine per ADR-117 §10:
AETHER embed() mean ~150 us/window (target <2 ms) PASS
batch scaling 1/8/64: 140 / 1091 / 8509 us (linear, no O(n^2)) PASS
MERIDIAN normalize() mean ~2.2 us/frame (target <200 us) PASS
MERIDIAN encode() mean ~6.9 us (target <200 us) PASS
MAT ingest+scan_once() mean ~40 ms/256-frame (< 500 ms interval) PASS
Acceptance self-verification (ADR-185 §6), all run just now:
§6.1 default wheel 279 KB (<=5 MB); build_features has no p6-* feature PASS
§6.2 pytest tests/test_aether.py 9/9 PASS
§6.3 pytest tests/test_meridian.py 13/13 PASS
§6.4 pytest tests/test_mat.py 7/7 PASS
§6.5 benchmarks meet all targets (above) PASS
§6.6 parity harness: 3/3 SHA golden gates green (cargo test --features
sota, 6/6); CI *wiring* as a release gate is out of python/ scope PARTIAL
§6.7 SOTA accuracy bars on labeled fixtures: NOT met (no labeled
fixtures / trained models available; parity proves path-equality,
not accuracy) OPEN
§6.8 .pyi stubs present for all three; mypy --strict on the 3 examples PASS
§6.9 base wheel `import wifi_densepose.{aether,meridian,mat}` raises a
clear ImportError naming the extra PASS
No regression: 76 pre-existing tests pass on the default wheel.
Status NOT flipped to Accepted: §6.7 (accuracy bars) is unmet, §6.6 CI
wiring is pending, and the per-extra wheel-size hoists (sensing-server /
train / mat leaf crates) remain follow-ups. docs/adr/ is owned by another
agent this session, so the ADR ledger edit is deferred to that owner.
Bind the ADR-024 MAT (Mass Casualty Assessment Tool) disaster-survivor
detection + START triage surface into the wheel behind a gated [mat]
extra / Cargo `mat` feature, mirroring the upstream disaster/ML gating.
Also adds the [sota] superset extra (aether+meridian+mat).
Surface (bound against the REAL code at HEAD, not the ADR wishlist):
- DisasterType (9 variants) / TriageStatus (5, START) enums
- DisasterConfig (builder-backed, continuous_monitoring forced off)
- DisasterResponse: initialize_event / add_zone / push_csi_data /
scan_once / survivors / survivors_by_triage
- Survivor (id, triage_status, confidence, location, latest_vitals)
- VitalSignsReading (breathing/heartbeat rate, movement, confidence)
- ScanZone.rectangle / ScanZone.circle
push_csi_data + scan_once are GIL-released.
Honest deviations from ADR section 3.4 (documented in module header):
- ADR proposed adding a Rust-side sync scan_once() (section 11.3). That was
UNNECESSARY: the public async start_scanning() runs exactly one
scan_cycle and returns when continuous_monitoring == false. The binding
forces that flag off and drives one cycle on a private current-thread
tokio runtime -- NO change to wifi-densepose-mat.
- scan_cycle requires an active event + Active zone, which the ADR surface
omitted; initialize_event + add_zone are bound as required additions.
- Survivor.vital_signs is a *history* in the real code; bound as
Survivor.latest_vitals -> Optional[VitalSignsReading].
- DisasterType has 9 variants at HEAD (adds Landslide/MineCollapse/
Industrial/TunnelCollapse); all bound.
Parity (section 4.1, release-blocking): committed fixture mat_input.json
(synthetic breathing-modulated CSI stream) -> native Rust reference
(tests/mat_parity.rs, drives DisasterResponse directly) locks
tests/golden/mat_result.sha256 over a canonical
`count=<K>;triage_priorities=<sorted>` string (survivor UUIDs/timestamps
excluded as non-deterministic); pytest (tests/test_mat.py) runs the same
stream through the binding and asserts the identical hash. Both detect
exactly 1 survivor, triage Delayed. Honest: synthetic fixture proves
binding==native path equality, NOT live detection accuracy.
Verified:
cargo test --features mat --test mat_parity -> 2/2 pass
maturin develop --features mat + pytest tests/test_mat.py -> 7/7 pass
default cargo build clean, 0 mat/tokio refs in the default dep graph.
WHEEL-SIZE FINDING (ADR-185 section 9): default-features=false drops MAT's
`api` (axum) and `ruvector` features, but MAT still carries NON-optional
tokio (rt/sync/time), wifi-densepose-nn (ort/ONNX + reqwest/hyper),
rustfft, geo, ndarray. So a [mat] wheel exceeds the ADR-117 section 5.4
<=5 MB budget -- same leaf-crate-hoist follow-up as AETHER/MERIDIAN. The
default wheel is untouched (feature-gated).
Bind the ADR-027 MERIDIAN cross-environment domain-generalization surface
into the wheel behind a gated [meridian] extra / Cargo `meridian` feature.
Inference/adaptation path only (tch-free), per ADR-185 section 3.3.
Surface (bound against the REAL code at HEAD, not the ADR wishlist):
- HardwareType / HardwareNormalizer / CanonicalCsiFrame (from
wifi-densepose-signal::hardware_norm)
- MeridianGeometryConfig / GeometryEncoder (64-dim, permutation-invariant)
- RapidAdaptation / AdaptationResult (push_frame + adapt, LoRA deltas)
- CrossDomainEvaluator + mpjpe (from wifi-densepose-train, NO tch-backend)
All compute paths GIL-released (py.allow_threads).
Honest deviations from ADR section 3.3 (documented in the module header):
- ADR's RapidAdaptation.calibrate(windows) and AdaptationResult.converged
DO NOT EXIST. Real API is push_frame + adapt(); result carries
{lora_weights, final_loss, frames_used, adaptation_epochs}. Bound as-is.
- ADR's HardwareType.detect exposed as a staticmethod delegating to the
real HardwareNormalizer::detect_hardware.
- ADR's normalize(frame: CsiFrame, hw) is really normalize(amplitude,
phase, hw) over f64 vectors returning Result; bound faithfully.
- CanonicalCsiFrame fields are singular amplitude/phase (ADR said plural).
Training-time types (DomainFactorizer, GradientReversalLayer,
VirtualDomainAugmentor) are out of P6 scope (need the libtorch tier).
Parity (section 4.1, release-blocking): committed fixture
meridian_input.json -> native Rust reference (tests/meridian_parity.rs,
calls hardware_norm + geometry + rapid_adapt directly) locks
tests/golden/meridian_output.sha256 over the concatenated f32 outputs
(esp32+intel canonical frames, 64-dim geometry vector, rapid-adapt LoRA
weights); pytest (tests/test_meridian.py) runs the same fixture through
the binding and asserts the identical SHA-256. Both pass.
Verified:
cargo test --features meridian --test meridian_parity -> 2/2 pass
maturin develop --features meridian + pytest tests/test_meridian.py
-> 13/13 pass
default cargo build clean, 0 train/signal/sensing-server refs in the
default dep graph (gate keeps the base wheel lean).
WHEEL-SIZE FINDING (ADR-185 section 9 / section 1.2): the libtorch risk
the ADR feared is AVOIDED -- wifi-densepose-train's `tch` dep is properly
optional (feature tch-backend, OFF), so no libtorch links. BUT train
still carries NON-optional deps: tokio (rt subset), the five ruvector-*
crates, and wifi-densepose-nn (which itself pulls `ort` / ONNX Runtime +
reqwest/hyper). So a [meridian] wheel exceeds the ADR-117 section 5.4
<=5 MB budget (though lighter than AETHER's axum/tokio server tree). The
clean fix is the same leaf-crate hoist: move the pure inference modules
(geometry, rapid_adapt, eval, hardware_norm) into a tch/tokio/ort-free
leaf crate. A required pre-release follow-up, not a functional blocker;
P2 binds real code and proves parity today.
The training pipeline in `training_api.rs` was written and committed but never
declared as a module (no `mod training_api;`), so it was dead, uncompiled code.
The live `/api/v1/train/start` was a stub that flipped a status string and
logged one line without starting any job, and `/ws/train/progress` 404'd
(issue #1233). This wires the real trainer into the live server.
- main.rs: declare `mod training_api;` (+ `mod path_safety;` for its load
guard), replace the `training_status`/`training_config` stub fields on
AppStateInner with `training_state: training_api::TrainingState` +
`training_progress_tx`, delete the three stub handlers, and
`.merge(training_api::routes())` before `.with_state` so `/api/v1/train/*`
(bearer-gated) and `/ws/train/progress` resolve against the shared state.
- training_api.rs: decouple the training core (`run_training_job`) from the
~60-field AppStateInner via a shared status handle (`Arc<Mutex<TrainingStatus>>`)
+ cooperative cancel flag (`Arc<AtomicBool>`) + an owned frame_history
snapshot, making it unit-testable. Self-contained input schema (local
`RecordedFrame`/`RECORDINGS_DIR`) instead of coupling to the still-orphaned
`recording.rs`. Consolidate the single-job guard + spawn into
`spawn_training_job`.
- Tests: real end-to-end test (synthetic CSI dataset -> real progress frames
stream + an actual `.rvf` artifact is written), plus path-traversal-rejection
and cancellation tests.
Verified: cargo test -p wifi-densepose-sensing-server -p wifi-densepose-train
--no-default-features -> 0 failed.
P5 (feature-flagged disabled-build honesty) deferred as follow-up; the
default/enabled build no longer silently no-ops.
Bind the ADR-024 contrastive CSI embedding surface into the wheel behind
a gated [aether] extra / Cargo `aether` feature, per ADR-185 section 3.2.
Surface (bound against the REAL code at HEAD, not the ADR wishlist):
- AetherConfig -> EmbeddingConfig {d_model,d_proj,temperature,normalize}
- CsiAugmenter -> augment_pair(window, seed)
- EmbeddingExtractor -> embed(csi): 128-dim L2-normed, GIL-released
- info_nce_loss, cosine_similarity (module fns)
ADR-185 section 3.2 also names aether_loss/VICReg components,
alignment_metric, uniformity_metric, forward_dual, and vicreg_* config
fields. None exist in embedding.rs at HEAD, so they are intentionally NOT
bound (a Rust-side gap, not a binding gap) rather than fabricated.
Parity (section 4.1, release-blocking): committed fixture
aether_input.json -> native Rust reference (tests/aether_parity.rs, calls
sensing-server EmbeddingExtractor directly) locks
tests/golden/aether_embedding.sha256; pytest (tests/test_aether.py) runs
the same fixture through the binding and asserts the identical SHA-256 of
the LE f32 bytes. Both pass.
Verified: cargo test --features aether --test aether_parity -> 2/2 pass;
maturin develop --features aether + pytest tests/test_aether.py -> 9/9
pass; default cargo build clean with 0 sensing-server refs in the dep
graph (gate keeps the base wheel lean).
HONEST WHEEL-SIZE FINDING (ADR-185 section 9 / Open Q 11.1, unresolved):
wifi-densepose-sensing-server is an Axum/tokio crate whose tokio/axum/
worldgraph/ruvector deps are NON-optional (only mqtt/matter are
features), so default-features=false does NOT drop them. An [aether]
wheel therefore links the full server tree and BREAKS the ADR-117 section
5.4 <=5 MB budget. The fix is the ADR-sanctioned hoist of embedding.rs
(+ its graph_transformer/sona siblings) into a leaf crate so the wheel
links pure compute only -- a change INSIDE wifi-densepose-sensing-server,
owned by another agent this session, so deferred as a required
pre-release follow-up. P1 binds real code and proves parity today; the
hoist is a wheel-size optimization, not a functional blocker.
Note: building required initializing the worldgraph/rufield/ruvector
submodules (absent in the fresh worktree).
Verified all 6 acceptance criteria against current file contents and checked them.
Set Status to Accepted; marked P1/P2/P3 DONE (1fb5397dd); left P4 as ACCEPTED-FUTURE (#645).
To make criterion 4 genuinely true (live ESP32 17-keypoint feature nowhere advertised
without the first-cut/below-target/runtime-stub caveat), added the caveat to the three
remaining live-pose advertisements in README.md: the recommended-hardware capability
table, the hero image caption, and the Live-ESP32-pipeline note.
Refs #509, #1125
Drop all four PYPI_API_TOKEN password inputs from the publish-v2 and
publish-tombstone jobs, grant id-token: write, and bind both jobs to the
pypi GitHub environment so pypa/gh-action-pypi-publish uses secretless
Trusted Publishing. Rewrite the header + add step comments documenting
the blocking manual pypi.org trusted-publisher registration (ADR-184
§3.1) and the token fallback (§3.2). Build matrix untouched.
Refs ADR-184 P1.
Follow-up to the calibration deadlock fix. With the status gate unstuck,
maybe_feed_calibration reached feed_calibration, but a real ESP32 HT40 node
streams 128-wide amplitude frames while the single-link FieldModel is the
canonical 56-tone grid. LinkStats::update returned DimensionMismatch,
feed_calibration bubbled it, and maybe_feed_calibration swallowed it at debug
level — so frame_count stayed pinned at 0 on live hardware (presence/vitals,
which read the global history, were unaffected).
Resample each frame onto the model's canonical 56-tone grid via
HardwareNormalizer::resample_to_canonical before feeding — the same length-only
canonicalization the multistatic fusion path uses (#1170).
Pinned by maybe_feed_calibration_resamples_wide_frames_and_accumulates
(128-wide -> Collecting + count 1). sensing-server bin: 173 passed, 0 failed.
Co-Authored-By: claude-flow <ruv@ruv.net>
POST /api/v1/calibration/start created the FieldModel in Uncalibrated, but
field_bridge::maybe_feed_calibration only fed frames while already Collecting
— and the only thing that sets Collecting is feed_calibration on its first
fed frame. The two gates deadlocked: no first frame was ever fed, so
calibration_frame_count stayed 0 and status never left Uncalibrated. Observed
live on a streaming ESP32 node as {"status":"Uncalibrated","frame_count":0}
that never advanced.
- field_bridge::maybe_feed_calibration: feed while Uncalibrated | Collecting
so the first frame flips the model to Collecting and the count advances.
- calibration_stop: return structured {success:false, frame_count,
frames_needed} instead of an opaque 500 when finalized with too few frames.
- FieldModel::min_calibration_frames() accessor for the guard above.
- Regression test: maybe_feed_calibration_advances_uncalibrated_to_collecting.
Presence/motion/vitals were unaffected (separate auto rolling baseline).
Co-Authored-By: claude-flow <ruv@ruv.net>
Dragan Spiridonov