5 Commits

Author	SHA1	Message	Date
Dragan Spiridonov	788d685b9f	fix(python,training): validate PyO3 inputs; make the single-training-job guard atomic ... Closes the two findings from the adversarial review that were verified but left unfixed. Both now have proper tests, each proven to fail against the old code. 1. PyO3 bindings panicked / over-allocated on caller input (aether.rs). `EmbeddingExtractor(n_heads=0)` reached `d_model % n_heads` in the transformer and panicked (surfacing to Python as an opaque PanicException); a non-divisor head count tripped the native assert; and `AetherConfig(d_model=100_000)` allocated multi-gigabyte weight matrices that abort the interpreter. The binding did no validation. Now both constructors return `PyResult` and validate at the boundary — positive dims, `d_model % n_heads == 0`, and a generous MAX_DIM/MAX_LAYERS cap — raising `ValueError`. Proven: `n_heads=0` -> "n_heads must be positive", `d_model=100_000` -> a clean ValueError, both previously a PanicException / abort. +10 pytest cases (test_aether.py); a valid config still constructs and embeds. 2. The single-training-job guard was a TOCTOU race (training_api.rs). `spawn_training_job` checked `is_active()` under a `state` READ lock, released it, then set `active` later. A tokio RwLock read lock is SHARED, so two concurrent `POST /train/start` could both hold it, both see the slot free, and both spawn jobs — sharing/overwriting one status+cancel and orphaning a task handle. Extracted `claim_training_slot`, which does the check-and-set in ONE `status` mutex scope — the atomicity lives on the status mutex, not the coarse state lock — so concurrent starts serialise and exactly one wins. This also makes it unit-testable without a full AppState. Test: 32 threads hit a barrier and race to claim; asserts EXACTLY ONE wins. Mutation-proven — reverting to the split check-then-set makes it fail (`left: 3, right: 1`), and it returns to 1 with the fix. Verified on aarch64/macOS: training_api 28 pass (26 existing + 2 new), full python/tests suite 237 pass (227 + 10). The native module keeps its internal assert as a defence-in-depth invariant; the binding now enforces it at the edge. Co-Authored-By: Ruflo & AQE	2026-07-24 14:45:55 +02:00
Dragan Spiridonov	bc0e8fd031	test(python): close the three parity-review findings (native anchor in CI, NaN, cosine) ... A cross-vendor review of the parity rework (2febbb81) found three issues; all verified and fixed here. 1. HIGH — the native≈golden half never ran in CI, so a binding marshalling bug could pass. The golden vectors were regenerated through the Python binding, and only pytest (binding vs that golden) runs in CI — the native reference tests in `python/tests/aether_parity.rs` link against the PyO3 crate and no workflow runs them. A stable PyO3 conversion defect present at regeneration would therefore be baked into the golden and go undetected. Fix: a real native parity test IN the `wifi-densepose-aether` crate (`tests/golden_parity.rs`), which is std-only and a member of the v2 workspace, so it runs under the existing `cargo test --workspace`. It recomputes the embedding with no PyO3/marshalling and asserts it matches the SAME committed golden within tolerance. Now native≈golden AND binding≈golden both run in CI ⇒ binding≈native, and a binding-specific artifact in the golden surfaces here as a native mismatch. serde_json added as a dev-dependency only (test-only; never linked into the lib or the wheel, so the crate stays runtime-dependency-free). Independently proven at this commit: native output equals the committed golden BIT-FOR-BIT (128/128 exact bits, Δ=0) for both base and loaded — the golden is native-faithful today; this test keeps it that way. 2. MEDIUM — the Python parity helper passed non-finite output. `abs(nan - b) > tol` is False, so an all-NaN embedding slipped through. Now every element must be `math.isfinite` first. Proven: an all-NaN vector is rejected ("element 0 is not finite (nan)"); the Rust helpers already caught it via `<=`. 3. LOW — a coherent shift inside the per-element tolerance could move the whole vector undetected. Added a whole-vector cosine-similarity bound (≥ 1 - 1e-6) alongside the per-element check. Verified on aarch64/macOS: `test_aether` 13, full `python/tests/` suite 227 passed against a `--features sota` build; the new native test passes (2/2) run standalone (the in-worktree `cargo test -p` only failed on an un-checked-out submodule; ci.yml checks out submodules recursively). Co-Authored-By: Ruflo & AQE	2026-07-24 12:29:23 +02:00
Dragan Spiridonov	3ed43e9a2f	fix(python): ship SOTA bindings in release wheels; make parity tests arch-portable ... Three coupled defects, all found by building the wheel and running the real suite on Apple Silicon rather than trusting green CI. 1. THE P6 SOTA BINDINGS NEVER REACHED USERS (release blocker). `pip-release.yml`'s cibuildwheel built the DEFAULT feature set, so published wheels contained none of aether/mat/meridian. `pip install wifi-densepose[aether]` then raised ImportError — and the extra is empty, so its own error message ("install the [aether] extra") sent users in a circle. A pip extra cannot enable a Rust cargo feature on an already-built wheel, so the only way P6 reaches PyPI is to compile it in. Fix: the RELEASE build opts in via `MATURIN_PEP517_ARGS="--features sota"` (per-platform, since CIBW_ENVIRONMENT_LINUX overrides CIBW_ENVIRONMENT). `default = []` in Cargo.toml, the RuView#1387-default-wheel-budget-config fix-marker, and the wheel-size-budget job are all left UNTOUCHED — they keep guarding the small base compile. Measured published wheel: 1.68 MiB, well under the ADR-117 §5.4 5 MiB budget. Proven: built via the exact PEP517 path, `import wifi_densepose.aether/mat/meridian` all succeed. 2. THE WHEEL SMOKE TEST COULD NOT SEE #1. CIBW_TEST_COMMAND only asserted `hello()` and PRINTED `__build_features__`. The one signal that would reveal missing bindings was dumped to stdout and ignored, so a featureless wheel published green. It now ASSERTS the p6 features are present and imports the three modules. Proven red→green: fails on the old featureless wheel ("missing SOTA bindings: [...]"), passes on the fixed one. 3. THE AETHER PARITY TESTS WERE NOT PORTABLE ACROSS THE WHEEL MATRIX. `test_aether.py` and the native `aether_parity.rs`/`aether_weights_parity.rs` hashed the raw f32 embedding bytes (SHA-256) against a committed golden. The embedding is pure f32 with transcendental ops (ln/sqrt/cos) that are not bit-reproducible across CPUs/libm, so the hash only ever matched the one arch that generated it. This passed in python-ci (x86) but fails on the aarch64 / macOS-arm wheels this same project ships — latent until #1 is fixed and the bindings actually load. Every tolerance/behavioral assertion already passed; only the two byte-hash tests failed, which is the signature of a non-portable golden, not a logic bug. Fix: compare to a committed golden VECTOR within tolerance (atol=rtol=1e-4 — ~100x cross-arch f32 drift, ~100x under any real algorithm change), on both the Python and native sides against the SAME golden. Native ≈ golden and binding ≈ golden together prove binding ≈ native, portably. `.sha256` goldens replaced by `.json` vectors. Also: the aether/mat/meridian import shims told users to `pip install wifi-densepose[<x>]` on a missing feature — an empty extra that cannot help. Corrected to name the real fix (rebuild with `--features <x>`); message tests updated to assert the honest message and forbid the misleading one. Verified on aarch64/macOS: full `python/tests/` suite 227 passed against a wheel built with the new mechanism; `cargo check --tests --features aether` clean. The native `.rs` parity tests were converted by inspection and cargo-checked but not executed — they link against the PyO3 crate and no workflow runs them today (a pre-existing gap; wiring `cd python && cargo test --features sota` into python-ci would make the native anchor actually run). Co-Authored-By: Ruflo & AQE	2026-07-24 10:45:00 +02:00
ruv	65da488add	feat(adr-185): add weight-loading capability to AETHER EmbeddingExtractor (§13.a) ... Closes the ADR-185 §13.a follow-up (the tractable, data-independent one of the three §6.7 gaps): the bound EmbeddingExtractor was random-Xavier-init only, with NO way to load real weights — so it was structurally untrained. This adds the *capability* to load weights whenever a trained checkpoint exists. It does NOT itself produce trained/SOTA embeddings and does NOT close §6.7 (still no trained checkpoint, no labeled data, no eval harness). Serialization: not greenfield — EmbeddingExtractor already had flatten_weights()/unflatten_weights() (flat Vec<f32>). wifi-densepose-aether is a deliberately dependency-free std-only leaf crate (ADR-185 §13), so rather than add safetensors/serde/bincode (which would undo the zero-dep property), the on-disk format is raw little-endian f32 with a 12-byte header (magic "AETHERW1" + u32 param count) — zero new deps. Native (v2/crates/wifi-densepose-aether/src/embedding.rs): - EmbeddingExtractor::save_weights(path) / load_weights(path). load_weights never panics: errors on unreadable file, short/oversized payload, bad magic, or a param-count mismatch (delegated to unflatten_weights). - Default (no weights) construction is UNCHANGED — still random init, clearly labeled untrained. Purely additive. Python binding (python/src/bindings/aether.rs, .pyi): - EmbeddingExtractor.load_weights(path) / save_weights(path) / param_count, GIL-released, ValueError on bad input. Tests (both Rust and Python, per the task): - Rust unit (embedding.rs): load_weights_actually_replaces_weights_and_ round_trips proves the loaded weights MOVE the embedding away from the random-init baseline (not a silent no-op), match the source extractor (round-trip), and are bit-identical after the file round-trip; plus a bad-magic/wrong-count rejection test. - Cross-language golden (aether_weights_parity.rs + test_aether.py): a shared deterministic weight formula (w[i]=k/65536-0.5, exact in f32+f64) written to the AETHER format; both native Rust and the Python binding load it and must produce the byte-identical embedding SHA-256 (tests/golden/aether_loaded_embedding.sha256) — proving the binding's load path is bit-identical to native, and (vs the random baseline) that the loaded weights are actually used. Verified: cargo test -p wifi-densepose-aether 98 passed, 0 failed cargo test --features aether --test aether_parity --test aether_weights_parity 3 passed, 0 failed maturin develop --features aether + pytest test_aether.py 13/13 pass default cargo build (no aether feature) clean	2026-07-21 18:20:23 -07:00
ruv	d060998e3b	feat(adr-185): P1 AETHER bindings (wifi_densepose.aether) + parity harness ... Bind the ADR-024 contrastive CSI embedding surface into the wheel behind a gated [aether] extra / Cargo `aether` feature, per ADR-185 section 3.2. Surface (bound against the REAL code at HEAD, not the ADR wishlist): - AetherConfig -> EmbeddingConfig {d_model,d_proj,temperature,normalize} - CsiAugmenter -> augment_pair(window, seed) - EmbeddingExtractor -> embed(csi): 128-dim L2-normed, GIL-released - info_nce_loss, cosine_similarity (module fns) ADR-185 section 3.2 also names aether_loss/VICReg components, alignment_metric, uniformity_metric, forward_dual, and vicreg_* config fields. None exist in embedding.rs at HEAD, so they are intentionally NOT bound (a Rust-side gap, not a binding gap) rather than fabricated. Parity (section 4.1, release-blocking): committed fixture aether_input.json -> native Rust reference (tests/aether_parity.rs, calls sensing-server EmbeddingExtractor directly) locks tests/golden/aether_embedding.sha256; pytest (tests/test_aether.py) runs the same fixture through the binding and asserts the identical SHA-256 of the LE f32 bytes. Both pass. Verified: cargo test --features aether --test aether_parity -> 2/2 pass; maturin develop --features aether + pytest tests/test_aether.py -> 9/9 pass; default cargo build clean with 0 sensing-server refs in the dep graph (gate keeps the base wheel lean). HONEST WHEEL-SIZE FINDING (ADR-185 section 9 / Open Q 11.1, unresolved): wifi-densepose-sensing-server is an Axum/tokio crate whose tokio/axum/ worldgraph/ruvector deps are NON-optional (only mqtt/matter are features), so default-features=false does NOT drop them. An [aether] wheel therefore links the full server tree and BREAKS the ADR-117 section 5.4 <=5 MB budget. The fix is the ADR-sanctioned hoist of embedding.rs (+ its graph_transformer/sona siblings) into a leaf crate so the wheel links pure compute only -- a change INSIDE wifi-densepose-sensing-server, owned by another agent this session, so deferred as a required pre-release follow-up. P1 binds real code and proves parity today; the hoist is a wheel-size optimization, not a functional blocker. Note: building required initializing the worldgraph/rufield/ruvector submodules (absent in the fresh worktree).	2026-07-21 16:28:14 -07:00