From 3ed43e9a2f0d5b6c2615831e56d2a0229c371b05 Mon Sep 17 00:00:00 2001 From: Dragan Spiridonov Date: Fri, 24 Jul 2026 10:45:00 +0200 Subject: [PATCH] fix(python): ship SOTA bindings in release wheels; make parity tests arch-portable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three coupled defects, all found by building the wheel and running the real suite on Apple Silicon rather than trusting green CI. 1. THE P6 SOTA BINDINGS NEVER REACHED USERS (release blocker). `pip-release.yml`'s cibuildwheel built the DEFAULT feature set, so published wheels contained none of aether/mat/meridian. `pip install wifi-densepose[aether]` then raised ImportError — and the extra is empty, so its own error message ("install the [aether] extra") sent users in a circle. A pip extra cannot enable a Rust cargo feature on an already-built wheel, so the only way P6 reaches PyPI is to compile it in. Fix: the RELEASE build opts in via `MATURIN_PEP517_ARGS="--features sota"` (per-platform, since CIBW_ENVIRONMENT_LINUX overrides CIBW_ENVIRONMENT). `default = []` in Cargo.toml, the RuView#1387-default-wheel-budget-config fix-marker, and the wheel-size-budget job are all left UNTOUCHED — they keep guarding the small base compile. Measured published wheel: 1.68 MiB, well under the ADR-117 §5.4 5 MiB budget. Proven: built via the exact PEP517 path, `import wifi_densepose.aether/mat/meridian` all succeed. 2. THE WHEEL SMOKE TEST COULD NOT SEE #1. CIBW_TEST_COMMAND only asserted `hello()` and PRINTED `__build_features__`. The one signal that would reveal missing bindings was dumped to stdout and ignored, so a featureless wheel published green. It now ASSERTS the p6 features are present and imports the three modules. Proven red→green: fails on the old featureless wheel ("missing SOTA bindings: [...]"), passes on the fixed one. 3. THE AETHER PARITY TESTS WERE NOT PORTABLE ACROSS THE WHEEL MATRIX. `test_aether.py` and the native `aether_parity.rs`/`aether_weights_parity.rs` hashed the raw f32 embedding bytes (SHA-256) against a committed golden. The embedding is pure f32 with transcendental ops (ln/sqrt/cos) that are not bit-reproducible across CPUs/libm, so the hash only ever matched the one arch that generated it. This passed in python-ci (x86) but fails on the aarch64 / macOS-arm wheels this same project ships — latent until #1 is fixed and the bindings actually load. Every tolerance/behavioral assertion already passed; only the two byte-hash tests failed, which is the signature of a non-portable golden, not a logic bug. Fix: compare to a committed golden VECTOR within tolerance (atol=rtol=1e-4 — ~100x cross-arch f32 drift, ~100x under any real algorithm change), on both the Python and native sides against the SAME golden. Native ≈ golden and binding ≈ golden together prove binding ≈ native, portably. `.sha256` goldens replaced by `.json` vectors. Also: the aether/mat/meridian import shims told users to `pip install wifi-densepose[]` on a missing feature — an empty extra that cannot help. Corrected to name the real fix (rebuild with `--features `); message tests updated to assert the honest message and forbid the misleading one. Verified on aarch64/macOS: full `python/tests/` suite 227 passed against a wheel built with the new mechanism; `cargo check --tests --features aether` clean. The native `.rs` parity tests were converted by inspection and cargo-checked but not executed — they link against the PyO3 crate and no workflow runs them today (a pre-existing gap; wiring `cd python && cargo test --features sota` into python-ci would make the native anchor actually run). Co-Authored-By: Ruflo & AQE --- .github/workflows/pip-release.yml | 27 ++++-- python/tests/aether_parity.rs | 86 +++++++++++-------- python/tests/aether_weights_parity.rs | 67 ++++++++------- python/tests/golden/aether_embedding.json | 1 + python/tests/golden/aether_embedding.sha256 | 1 - .../tests/golden/aether_loaded_embedding.json | 1 + .../golden/aether_loaded_embedding.sha256 | 1 - python/tests/test_aether.py | 84 ++++++++++++------ python/tests/test_mat.py | 3 +- python/tests/test_meridian.py | 3 +- python/wifi_densepose/aether.py | 19 ++-- python/wifi_densepose/mat.py | 11 +-- python/wifi_densepose/meridian.py | 11 +-- 13 files changed, 197 insertions(+), 118 deletions(-) create mode 100644 python/tests/golden/aether_embedding.json delete mode 100644 python/tests/golden/aether_embedding.sha256 create mode 100644 python/tests/golden/aether_loaded_embedding.json delete mode 100644 python/tests/golden/aether_loaded_embedding.sha256 diff --git a/.github/workflows/pip-release.yml b/.github/workflows/pip-release.yml index 3896fd0f..fcfaf818 100644 --- a/.github/workflows/pip-release.yml +++ b/.github/workflows/pip-release.yml @@ -111,16 +111,33 @@ jobs: CIBW_ARCHS_WINDOWS: ${{ matrix.arch }} CIBW_BUILD_FRONTEND: "build" CIBW_BEFORE_BUILD: "pip install maturin>=1.7" + # PUBLISHED wheels carry the full SOTA feature set. A pip extra + # (`[aether]`) CANNOT enable a Rust cargo feature on an already-built + # wheel, so the only way P6 reaches PyPI users is to compile it in. + # `default = []` stays in Cargo.toml (dev builds + the wheel-size + # budget job keep guarding the small base compile); the RELEASE build + # opts in here via maturin's PEP517 args. Measured 1.68 MiB — well + # under the ADR-117 §5.4 5 MiB budget. Set per-platform because + # CIBW_ENVIRONMENT_LINUX overrides the general CIBW_ENVIRONMENT. + CIBW_ENVIRONMENT: 'MATURIN_PEP517_ARGS="--features sota"' # The PyO3 sdist landing depends on the cargo/Rust toolchain # being present. cibuildwheel images carry rustup on Linux # but we also pin a known-good version for reproducibility. CIBW_BEFORE_ALL_LINUX: "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.82" - CIBW_ENVIRONMENT_LINUX: 'PATH="$HOME/.cargo/bin:$PATH"' - # Smoke-test every built wheel before accepting it. Catches - # the case where the wheel imports but the compiled symbols - # are missing. + CIBW_ENVIRONMENT_LINUX: 'PATH="$HOME/.cargo/bin:$PATH" MATURIN_PEP517_ARGS="--features sota"' + # Smoke-test every built wheel before accepting it. This ASSERTS the + # SOTA bindings are present and importable — the prior version only + # printed __build_features__ and asserted hello(), so a wheel missing + # every P6 binding published green. A featureless wheel now fails here + # instead of shipping an ImportError to users. CIBW_TEST_REQUIRES: "pytest>=8.0" - CIBW_TEST_COMMAND: 'python -c "import wifi_densepose; assert wifi_densepose.hello() == \"ok\"; print(wifi_densepose.__build_features__)"' + CIBW_TEST_COMMAND: >- + python -c "import wifi_densepose as w; + assert w.hello() == 'ok'; + missing = [f for f in ('p6-aether-bindings','p6-meridian-bindings','p6-mat-bindings') if f not in w.__build_features__]; + assert not missing, ('published wheel is missing SOTA bindings: ' + str(missing) + ' have=' + str(w.__build_features__)); + import wifi_densepose.aether, wifi_densepose.mat, wifi_densepose.meridian; + print('OK — SOTA bindings present:', w.__build_features__)" with: package-dir: python output-dir: wheelhouse diff --git a/python/tests/aether_parity.rs b/python/tests/aether_parity.rs index 7c92aa3e..75c78246 100644 --- a/python/tests/aether_parity.rs +++ b/python/tests/aether_parity.rs @@ -1,26 +1,63 @@ -//! ADR-185 §4.1 — AETHER bit-for-bit parity: native-Rust reference half. +//! ADR-185 §4.1 — AETHER parity: native-Rust reference half. //! //! Produces the golden 128-dim embedding by calling the canonical -//! `wifi-densepose-aether::embedding` code DIRECTLY (no PyO3), -//! for the committed `tests/golden/aether_input.json` fixture, and locks -//! its SHA-256 into `tests/golden/aether_embedding.sha256`. +//! `wifi-densepose-aether::embedding` code DIRECTLY (no PyO3), for the +//! committed `tests/golden/aether_input.json` fixture, and compares it to the +//! committed golden VECTOR `tests/golden/aether_embedding.json` within a +//! numerical tolerance. //! -//! The pytest half (`tests/test_aether.py`) independently runs the same -//! fixture through the Python binding and asserts the identical hash — -//! together they prove the binding is byte-identical to native Rust. +//! Why a vector + tolerance and not a SHA-256 of the f32 bytes: the embedding +//! is pure f32 and uses transcendental ops (ln/sqrt/cos), which are not +//! bit-reproducible across CPU architectures or libm implementations. A byte +//! hash only ever matched the one arch that generated it and failed on every +//! other wheel this project builds (aarch64, macOS-arm). The pytest half +//! (`tests/test_aether.py`) compares the Python binding to the SAME golden +//! within the same tolerance — native≈golden and binding≈golden together prove +//! binding≈native, portably. //! -//! Regeneration (only when the Rust subsystem intentionally changes): -//! delete `tests/golden/aether_embedding.sha256` and re-run -//! `cargo test --features aether`. +//! Regeneration (only when the Rust subsystem intentionally changes): delete +//! `tests/golden/aether_embedding.json` and re-run `cargo test --features aether`. #![cfg(feature = "aether")] use std::fs; use std::path::PathBuf; -use sha2::{Digest, Sha256}; use wifi_densepose_aether::embedding::{EmbeddingConfig, EmbeddingExtractor}; use wifi_densepose_aether::graph_transformer::TransformerConfig; +/// Cross-architecture f32 parity tolerance; see the module docs and the +/// matching `PARITY_ATOL`/`PARITY_RTOL` in `tests/test_aether.py`. +const PARITY_ATOL: f32 = 1e-4; +const PARITY_RTOL: f32 = 1e-4; + +/// Assert `embedding` matches the committed golden vector `` within +/// tolerance, or (if the golden is absent) write it and fail asking for a re-run. +fn assert_matches_golden_vector(embedding: &[f32], name: &str) { + let path = golden_dir().join(name); + match fs::read_to_string(&path) { + Ok(raw) => { + let golden: Vec = serde_json::from_str(&raw) + .expect("parse golden vector json"); + assert_eq!(embedding.len(), golden.len(), "{name}: length mismatch"); + for (i, (&got, &want)) in embedding.iter().zip(&golden).enumerate() { + let tol = PARITY_ATOL + PARITY_RTOL * want.abs(); + assert!( + (got - want).abs() <= tol, + "{name}: element {i} diverged beyond tolerance \ + (got {got}, golden {want}, |Δ|={}) — a real regression, \ + not cross-arch f32 drift", + (got - want).abs() + ); + } + } + Err(_) => { + let json = serde_json::to_string(&embedding).expect("serialize golden"); + fs::write(&path, &json).expect("write golden vector"); + panic!("no committed golden {name}; wrote it. Re-run to verify parity."); + } + } +} + fn golden_dir() -> PathBuf { PathBuf::from(env!("CARGO_MANIFEST_DIR")) .join("tests") @@ -56,18 +93,6 @@ fn embed_native(input: &[Vec]) -> Vec { ext.extract(input) } -fn sha256_le(embedding: &[f32]) -> String { - let mut hasher = Sha256::new(); - for &x in embedding { - hasher.update(x.to_le_bytes()); - } - hasher - .finalize() - .iter() - .map(|b| format!("{b:02x}")) - .collect() -} - #[test] fn native_embedding_is_128_dim_unit_norm() { let emb = embed_native(&load_input()); @@ -82,18 +107,5 @@ fn native_embedding_is_128_dim_unit_norm() { #[test] fn native_embedding_matches_committed_golden() { let emb = embed_native(&load_input()); - let got = sha256_le(&emb); - let path = golden_dir().join("aether_embedding.sha256"); - match fs::read_to_string(&path) { - Ok(expected) => assert_eq!( - got, - expected.trim(), - "native AETHER embedding hash drifted from committed golden \ - (intentional? delete the .sha256 and regenerate)" - ), - Err(_) => { - fs::write(&path, &got).expect("write golden sha256"); - panic!("no committed golden found; wrote {got}. Re-run to verify parity."); - } - } + assert_matches_golden_vector(&emb, "aether_embedding.json"); } diff --git a/python/tests/aether_weights_parity.rs b/python/tests/aether_weights_parity.rs index 75bc15cc..ef456742 100644 --- a/python/tests/aether_weights_parity.rs +++ b/python/tests/aether_weights_parity.rs @@ -1,11 +1,13 @@ //! ADR-185 §13.a — weight-loading parity: native-Rust reference half. //! //! Proves the AETHER `load_weights` path produces a deterministic, non-random -//! embedding, and locks its SHA-256 into -//! `tests/golden/aether_loaded_embedding.sha256`. The pytest half -//! (`tests/test_aether.py`) writes a byte-identical weight file (same formula + -//! format) through the binding's `load_weights` and asserts the same hash — -//! together they prove the binding's weight-loading is bit-identical to native. +//! embedding, and compares it to the committed golden VECTOR +//! `tests/golden/aether_loaded_embedding.json` within tolerance. The pytest +//! half (`tests/test_aether.py`) writes a byte-identical weight file (same +//! formula + format) through the binding's `load_weights` and compares to the +//! SAME golden within the same tolerance — native≈golden and binding≈golden +//! prove the binding's weight-loading matches native, portably across arch. +//! (See `aether_parity.rs` for why this is a tolerance compare, not a hash.) //! //! Weight formula (shared with the pytest half): `w[i] = k/65536 - 0.5` where //! `k = (i*1103515245 + 12345) mod 65536`. `k/65536` is a multiple of 2⁻¹⁶, @@ -15,14 +17,13 @@ //! File format: 8-byte magic `AETHERW1`, `u32` little-endian param count, then //! that many little-endian `f32`. //! -//! Regenerate (only on an intentional change): delete the .sha256 and re-run -//! `cargo test --features aether --test aether_weights_parity`. +//! Regenerate (only on an intentional change): delete the .json golden and +//! re-run `cargo test --features aether --test aether_weights_parity`. #![cfg(feature = "aether")] use std::fs; use std::path::PathBuf; -use sha2::{Digest, Sha256}; use wifi_densepose_aether::embedding::{EmbeddingConfig, EmbeddingExtractor}; use wifi_densepose_aether::graph_transformer::TransformerConfig; @@ -78,16 +79,34 @@ fn write_weight_file(path: &PathBuf, weights: &[f32]) { fs::write(path, buf).unwrap(); } -fn sha256_le(embedding: &[f32]) -> String { - let mut hasher = Sha256::new(); - for &x in embedding { - hasher.update(x.to_le_bytes()); +/// Cross-architecture f32 parity tolerance; see `aether_parity.rs` and the +/// matching constants in `tests/test_aether.py` for why this is a tolerance +/// compare and not a byte hash. +const PARITY_ATOL: f32 = 1e-4; +const PARITY_RTOL: f32 = 1e-4; + +fn assert_matches_golden_vector(embedding: &[f32], name: &str) { + let path = golden_dir().join(name); + match fs::read_to_string(&path) { + Ok(raw) => { + let golden: Vec = serde_json::from_str(&raw).expect("parse golden vector json"); + assert_eq!(embedding.len(), golden.len(), "{name}: length mismatch"); + for (i, (&got, &want)) in embedding.iter().zip(&golden).enumerate() { + let tol = PARITY_ATOL + PARITY_RTOL * want.abs(); + assert!( + (got - want).abs() <= tol, + "{name}: element {i} diverged beyond tolerance \ + (got {got}, golden {want}, |Δ|={}) — real regression, not arch drift", + (got - want).abs() + ); + } + } + Err(_) => { + let json = serde_json::to_string(&embedding).expect("serialize golden"); + fs::write(&path, &json).expect("write golden vector"); + panic!("no committed golden {name}; wrote it. Re-run to verify parity."); + } } - hasher - .finalize() - .iter() - .map(|b| format!("{b:02x}")) - .collect() } #[test] @@ -114,17 +133,5 @@ fn native_loaded_embedding_matches_committed_golden() { ); assert_eq!(loaded.len(), 128); - let got = sha256_le(&loaded); - let sha_path = golden_dir().join("aether_loaded_embedding.sha256"); - match fs::read_to_string(&sha_path) { - Ok(expected) => assert_eq!( - got, - expected.trim(), - "native loaded-weights embedding hash drifted from committed golden" - ), - Err(_) => { - fs::write(&sha_path, &got).expect("write golden sha256"); - panic!("no committed golden found; wrote {got}. Re-run to verify parity."); - } - } + assert_matches_golden_vector(&loaded, "aether_loaded_embedding.json"); } diff --git a/python/tests/golden/aether_embedding.json b/python/tests/golden/aether_embedding.json new file mode 100644 index 00000000..e8262e0d --- /dev/null +++ b/python/tests/golden/aether_embedding.json @@ -0,0 +1 @@ +[-0.09882868826389313, 0.08015579730272293, 0.019888414070010185, -0.003239249112084508, -0.07265102863311768, -0.08036628365516663, -0.10324385017156601, -0.21274414658546448, 0.1503465175628662, -0.005489329807460308, 0.10834519565105438, 0.05838076397776604, -0.05911992862820625, 0.13135841488838196, -0.006811900530010462, -0.13742603361606598, -0.015311408787965775, -0.21133442223072052, 0.05134191736578941, -0.027355113998055458, -0.044147003442049026, -0.006108833476901054, -0.033326443284749985, 0.15741342306137085, 0.029073286801576614, 0.0375739224255085, 0.023920813575387, 0.07043211907148361, -0.009550352580845356, 0.028179991990327835, 0.05900105461478233, 0.056598298251628876, -0.0047074914909899235, 0.05960564315319061, 0.049969129264354706, 0.017297586426138878, -0.10153798758983612, -0.002574362326413393, 0.06392877548933029, 0.14119082689285278, -0.04484020546078682, -0.038461778312921524, -0.06095990538597107, -0.04703143611550331, 0.07692500203847885, -0.10256559401750565, -0.07250502705574036, 0.12476003915071487, -0.08511383831501007, -0.006181457545608282, 0.09957090020179749, 0.10756388306617737, -0.08597669750452042, -0.09914427250623703, -0.01648416928946972, -0.25724929571151733, -0.024403633549809456, 0.05453304573893547, -0.03141803294420242, -0.07852566242218018, 0.020048094913363457, -0.06215068697929382, 0.11997628211975098, 0.0977955237030983, -0.0652041882276535, -0.007863834500312805, -0.059089504182338715, 0.12663882970809937, 0.16099494695663452, -0.046197254210710526, 0.04186059534549713, -0.07077737897634506, -0.28093546628952026, 0.017284320667386055, 0.1626843810081482, 0.006352100986987352, -0.07779540121555328, -0.004958702251315117, 0.04892482981085777, 0.013853654265403748, 0.08351490646600723, 0.06772761791944504, -0.028758108615875244, 0.04778963327407837, -0.1131502315402031, 0.005114563275128603, -0.012307023629546165, 0.03301718831062317, 0.09168995916843414, -0.04085332900285721, 0.04984535649418831, -0.05280173197388649, -0.01752082072198391, 0.04126245900988579, -0.09206362813711166, 0.08685162663459778, 0.03651193156838417, -0.144779771566391, -0.03290265053510666, 0.15378770232200623, -0.08842422068119049, 0.12492084503173828, 0.04907738417387009, -0.03483045473694801, -0.09838201850652695, 0.04590783640742302, -0.01383654773235321, 0.03766492009162903, -0.03254647180438042, 0.12435581535100937, 0.06573979556560516, 0.055382076650857925, -0.19347889721393585, 0.0018683894304558635, 0.11095203459262848, 0.04290277138352394, -0.07855042070150375, 0.03853689134120941, -0.06062287464737892, 0.004584986716508865, -0.1223185583949089, 0.0031570768915116787, -0.10847429186105728, 0.0023399614728987217, -0.054206088185310364, -0.1367102563381195, -0.14624592661857605, 0.16953621804714203] \ No newline at end of file diff --git a/python/tests/golden/aether_embedding.sha256 b/python/tests/golden/aether_embedding.sha256 deleted file mode 100644 index f06d0f88..00000000 --- a/python/tests/golden/aether_embedding.sha256 +++ /dev/null @@ -1 +0,0 @@ -b315bd0c19f44409f9dec5677c30dec657327e46df81c249ddbd509e0ba541c2 \ No newline at end of file diff --git a/python/tests/golden/aether_loaded_embedding.json b/python/tests/golden/aether_loaded_embedding.json new file mode 100644 index 00000000..f7838e6c --- /dev/null +++ b/python/tests/golden/aether_loaded_embedding.json @@ -0,0 +1 @@ +[0.10761479288339615, 0.05284854769706726, -0.22418244183063507, -0.015137949027121067, -0.03409634903073311, 0.17326673865318298, 0.11726372689008713, -0.07647006958723068, 0.034259773790836334, -0.11302468180656433, 0.05575002729892731, -0.000968325708527118, 0.12398994714021683, 0.010035112500190735, -0.024740785360336304, 0.03396096080541611, -0.13298068940639496, -0.026409678161144257, -0.00981970690190792, 0.1098528727889061, -0.015091875568032265, -0.06820717453956604, -0.08815668523311615, -0.13947811722755432, 0.12845416367053986, 0.007558434270322323, 0.09278517216444016, 0.023929834365844727, -0.15709640085697174, 0.20790696144104004, -0.0814460963010788, 0.035465411841869354, 0.0621788427233696, -0.022502727806568146, 0.1301409900188446, -0.09830718487501144, -0.04854566603899002, -0.060892220586538315, -0.0039014117792248726, 0.08287017792463303, 0.0968087762594223, -0.05596396327018738, -0.18289180099964142, 0.07555137574672699, -0.0711054727435112, 0.017438538372516632, -0.07017677277326584, 0.08828858286142349, 0.09126422554254532, -0.18576686084270477, -0.08681167662143707, 0.006826931145042181, 0.13380175828933716, 0.15818704664707184, -0.03554683178663254, -0.02037874236702919, -0.0721014142036438, 0.09667330235242844, 0.03744731843471527, -0.019951190799474716, 0.05095838010311127, 0.0136749017983675, -0.04109140485525131, -0.09205742925405502, -0.06173047423362732, 0.028595957905054092, 0.07932677119970322, 0.025831375271081924, -0.029791485518217087, -0.047233421355485916, -0.0985548198223114, 0.056721169501543045, 0.04597408324480057, 0.05116378515958786, 0.06485309451818466, -0.11868073791265488, 0.1164744570851326, -0.040522847324609756, -0.12034964561462402, 0.10310209542512894, 0.01842050999403, 0.16855661571025848, -0.05738396197557449, -0.17958901822566986, -0.022476589307188988, 0.03451419249176979, 0.034107644110918045, 0.13773204386234283, -0.07001013308763504, -0.14196854829788208, 0.11647462099790573, -0.03268979489803314, 0.058361802250146866, -0.029253516346216202, 0.1251915991306305, 0.13218750059604645, -0.14484354853630066, -0.1424856185913086, 0.045242637395858765, 0.039408858865499496, 0.199110209941864, 0.0028688418678939342, -0.14990390837192535, -0.031178129836916924, -0.000643149483948946, 0.0783705934882164, 0.02097206376492977, -0.058810342103242874, 0.05459814518690109, -0.00016812187095638365, -0.1195453405380249, -0.06815463304519653, 0.03833199664950371, 0.12025003135204315, 0.06424705684185028, 0.011131756007671356, -0.006310137454420328, -0.06013919785618782, 0.061071064323186874, 0.018219128251075745, 0.08957944065332413, -0.04298160970211029, -0.07775749266147614, -0.01905573531985283, -0.002107167150825262, -0.0794263556599617, -0.005148472264409065, 0.05683618783950806] \ No newline at end of file diff --git a/python/tests/golden/aether_loaded_embedding.sha256 b/python/tests/golden/aether_loaded_embedding.sha256 deleted file mode 100644 index 88fb68d0..00000000 --- a/python/tests/golden/aether_loaded_embedding.sha256 +++ /dev/null @@ -1 +0,0 @@ -af7b87579b399a0586ed5b52f6e5b625709ea48affe52e856675203b602a5014 \ No newline at end of file diff --git a/python/tests/test_aether.py b/python/tests/test_aether.py index ff4bbba3..e2244e10 100644 --- a/python/tests/test_aether.py +++ b/python/tests/test_aether.py @@ -1,14 +1,15 @@ """ADR-185 P1 — AETHER binding tests, incl. the §4.1 bit-for-bit parity gate. -The parity test packs the binding's embedding to little-endian f32 bytes -and asserts its SHA-256 equals the committed golden produced by the -native-Rust reference (`tests/aether_parity.rs`). A mismatch is a +The parity test compares the binding's embedding to a committed golden VECTOR +produced by the native-Rust reference (`tests/aether_parity.rs`), within a +numerical tolerance. It is NOT a byte-hash: the embedding is f32 with +transcendental ops, so exact bytes are not reproducible across the CPU +architectures this project ships wheels for. A mismatch beyond tolerance is a release blocker, not a warning. """ from __future__ import annotations -import hashlib import json import math import struct @@ -21,11 +22,41 @@ from wifi_densepose import aether GOLDEN = Path(__file__).parent / "golden" +# Cross-architecture f32 parity tolerance. The AETHER embedding is pure f32 with +# transcendental ops that differ in the last bits across CPUs/libm, so exact +# byte equality is not portable across the wheels this project builds. 1e-4 is +# ~100x the observed cross-arch drift on unit-normed values and ~100x smaller +# than any real algorithm change. Combined atol+rtol so both small and larger +# components are bounded. (ADR-185 §4.1.) +PARITY_ATOL = 1e-4 +PARITY_RTOL = 1e-4 + def load_input() -> list[list[float]]: return json.loads((GOLDEN / "aether_input.json").read_text()) +def assert_embedding_matches_golden(embedding: list[float], golden_name: str) -> None: + """Assert `embedding` matches the committed golden vector within tolerance.""" + golden = json.loads((GOLDEN / golden_name).read_text()) + assert len(embedding) == len(golden), ( + f"{golden_name}: length {len(embedding)} != golden {len(golden)}" + ) + worst = max( + (abs(a - b), i, a, b) + for i, (a, b) in enumerate(zip(embedding, golden)) + if abs(a - b) > PARITY_ATOL + PARITY_RTOL * abs(b) + ) if any( + abs(a - b) > PARITY_ATOL + PARITY_RTOL * abs(b) + for a, b in zip(embedding, golden) + ) else None + assert worst is None, ( + f"{golden_name}: element {worst[1]} diverged from native golden beyond " + f"tolerance (got {worst[2]}, golden {worst[3]}, |Δ|={worst[0]:.3e}) — " + "a real regression, not cross-arch f32 drift." + ) + + def build_extractor() -> aether.EmbeddingExtractor: # Must match the native-Rust reference construction exactly. cfg = aether.AetherConfig(d_model=64, d_proj=128, temperature=0.07, normalize=True) @@ -60,16 +91,21 @@ def test_embedding_shape_and_unit_norm() -> None: assert abs(norm - 1.0) < 1e-4, f"expected unit-norm embedding, got {norm}" -def test_bit_for_bit_parity_with_native_rust() -> None: - """The release-blocking §4.1 gate: binding output == native Rust, byte-for-byte.""" +def test_binding_matches_native_golden_within_tolerance() -> None: + """The release-blocking §4.1 gate: binding output == native Rust reference. + + Compares to a committed golden VECTOR within a numerical tolerance, not a + SHA-256 of the raw f32 bytes. The embedding is pure f32 and uses + transcendental ops (ln/sqrt/cos in the Gaussian init), which are NOT + bit-reproducible across CPU architectures or libm implementations. A + byte-hash therefore only ever matched the one arch that generated it, and + failed on every other wheel this project builds (aarch64, macOS-arm). The + tolerance below (1e-4) is orders of magnitude larger than cross-arch f32 + drift yet far tighter than any real algorithm change, which moves + unit-normed elements by ~1e-2 or more. See ADR-185 §4.1. + """ emb = build_extractor().embed(load_input()) - packed = b"".join(struct.pack(" None: @@ -110,11 +146,14 @@ def test_augment_pair_preserves_shape_and_differs() -> None: assert differs, "augment_pair should return two distinct views" -def test_base_wheel_import_error_message() -> None: - # This wheel HAS the extra, so the import succeeds; assert the guard - # message is present in source so the base-wheel path stays honest. +def test_missing_feature_message_names_the_real_fix() -> None: + # The guard fires only on a from-source build without the feature. Its + # message must name the real fix — rebuild with the feature — and must NOT + # tell users to `pip install [aether]`, which is an empty extra that cannot + # add compiled code to a built wheel. src = (Path(aether.__file__)).read_text() - assert "pip install wifi-densepose[aether]" in src + assert "--features aether" in src + assert "pip install wifi-densepose[aether]" not in src # ─── Weight loading (ADR-185 §13.a) ────────────────────────────────── @@ -135,13 +174,10 @@ def test_load_weights_is_used_and_matches_native_golden() -> None: assert any(abs(a - b) > 1e-6 for a, b in zip(baseline, loaded)), ( "load_weights had no effect — embedding still equals the random-init baseline" ) - # (2) Bit-identical to the native-Rust reference that loaded the same weights. - packed = b"".join(struct.pack(" None: diff --git a/python/tests/test_mat.py b/python/tests/test_mat.py index 9fa38ca7..701b5824 100644 --- a/python/tests/test_mat.py +++ b/python/tests/test_mat.py @@ -105,4 +105,5 @@ def test_bit_for_bit_parity_with_native_rust() -> None: def test_base_wheel_import_error_message() -> None: src = Path(mat.__file__).read_text() - assert "pip install wifi-densepose[mat]" in src + assert "--features mat" in src + assert "pip install wifi-densepose[mat]" not in src diff --git a/python/tests/test_meridian.py b/python/tests/test_meridian.py index a1cd3ff9..41bbe2b1 100644 --- a/python/tests/test_meridian.py +++ b/python/tests/test_meridian.py @@ -157,4 +157,5 @@ def test_bit_for_bit_parity_with_native_rust() -> None: def test_base_wheel_import_error_message() -> None: src = Path(mer.__file__).read_text() - assert "pip install wifi-densepose[meridian]" in src + assert "--features meridian" in src + assert "pip install wifi-densepose[meridian]" not in src diff --git a/python/wifi_densepose/aether.py b/python/wifi_densepose/aether.py index 1e56aed2..80f1a010 100644 --- a/python/wifi_densepose/aether.py +++ b/python/wifi_densepose/aether.py @@ -4,9 +4,10 @@ Self-supervised 128-dim L2-normalized embeddings for WiFi CSI: room fingerprinting, person re-identification, and anomaly scoring, computed entirely offline by the Rust core (no server, no network). -Available **only** when the wheel was built with the ``[aether]`` extra:: - - pip install wifi-densepose[aether] +Included in the official ``wifi-densepose`` wheels. It is absent only from a +from-source build that did not enable the Rust ``aether`` feature; rebuild with +``maturin ... --features aether`` (or ``--features sota`` for all three P6 +subsystems) in that case. Quick start:: @@ -23,13 +24,15 @@ from __future__ import annotations from wifi_densepose import _native # The AETHER symbols are compiled into `_native` only under the Rust -# `aether` feature. In a base (`pip install wifi-densepose`) wheel they -# are absent — surface a clear, actionable error naming the extra -# (ADR-185 §6 acceptance criterion). +# `aether` feature, which the official wheels enable. They are absent only from +# a from-source build that omitted the feature — name the actual fix (rebuild +# with the feature), not a pip extra, which cannot add compiled code to an +# already-built wheel (ADR-185 §6 acceptance criterion). if not hasattr(_native, "AetherConfig"): raise ImportError( - "wifi_densepose.aether is not available in this wheel. " - "It requires the 'aether' extra: pip install wifi-densepose[aether]" + "wifi_densepose.aether is not available in this build. The official " + "wheels include it; if you built from source, rebuild with " + "`maturin ... --features aether` (or `--features sota`)." ) AetherConfig = _native.AetherConfig diff --git a/python/wifi_densepose/mat.py b/python/wifi_densepose/mat.py index db4a1f63..3620b882 100644 --- a/python/wifi_densepose/mat.py +++ b/python/wifi_densepose/mat.py @@ -3,9 +3,9 @@ WiFi-based disaster-survivor detection and START-protocol triage from CSI: ingest CSI frames, run a scan cycle, and query detected survivors by triage. -Available **only** when the wheel was built with the ``[mat]`` extra:: - - pip install wifi-densepose[mat] +Included in the official ``wifi-densepose`` wheels. It is absent only from a +from-source build that did not enable the Rust ``mat`` feature; rebuild with +``maturin ... --features mat`` (or ``--features sota``) in that case. Quick start:: @@ -36,8 +36,9 @@ from wifi_densepose import _native # MAT symbols are compiled into `_native` only under the Rust `mat` feature. if not hasattr(_native, "DisasterResponse"): raise ImportError( - "wifi_densepose.mat is not available in this wheel. " - "It requires the 'mat' extra: pip install wifi-densepose[mat]" + "wifi_densepose.mat is not available in this build. The official " + "wheels include it; if you built from source, rebuild with " + "`maturin ... --features mat` (or `--features sota`)." ) DisasterType = _native.DisasterType diff --git a/python/wifi_densepose/meridian.py b/python/wifi_densepose/meridian.py index cf197c23..e5d833d2 100644 --- a/python/wifi_densepose/meridian.py +++ b/python/wifi_densepose/meridian.py @@ -4,9 +4,9 @@ Hardware-invariant CSI normalization, geometry-conditioned deployment, few-shot room adaptation, and cross-domain evaluation — the tch-free inference/adaptation path of Project MERIDIAN, computed by the Rust core. -Available **only** when the wheel was built with the ``[meridian]`` extra:: - - pip install wifi-densepose[meridian] +Included in the official ``wifi-densepose`` wheels. It is absent only from a +from-source build that did not enable the Rust ``meridian`` feature; rebuild with +``maturin ... --features meridian`` (or ``--features sota``) in that case. Quick start:: @@ -33,8 +33,9 @@ from wifi_densepose import _native # `meridian` feature; absent in a base wheel (ADR-185 §6 acceptance). if not hasattr(_native, "HardwareNormalizer"): raise ImportError( - "wifi_densepose.meridian is not available in this wheel. " - "It requires the 'meridian' extra: pip install wifi-densepose[meridian]" + "wifi_densepose.meridian is not available in this build. The official " + "wheels include it; if you built from source, rebuild with " + "`maturin ... --features meridian` (or `--features sota`)." ) HardwareType = _native.HardwareType