fix(python): ship SOTA bindings in release wheels; make parity tests arch-portable

Three coupled defects, all found by building the wheel and running the real
suite on Apple Silicon rather than trusting green CI.

1. THE P6 SOTA BINDINGS NEVER REACHED USERS (release blocker).
   `pip-release.yml`'s cibuildwheel built the DEFAULT feature set, so published
   wheels contained none of aether/mat/meridian. `pip install
   wifi-densepose[aether]` then raised ImportError — and the extra is empty, so
   its own error message ("install the [aether] extra") sent users in a circle.
   A pip extra cannot enable a Rust cargo feature on an already-built wheel, so
   the only way P6 reaches PyPI is to compile it in.

   Fix: the RELEASE build opts in via `MATURIN_PEP517_ARGS="--features sota"`
   (per-platform, since CIBW_ENVIRONMENT_LINUX overrides CIBW_ENVIRONMENT).
   `default = []` in Cargo.toml, the RuView#1387-default-wheel-budget-config
   fix-marker, and the wheel-size-budget job are all left UNTOUCHED — they keep
   guarding the small base compile. Measured published wheel: 1.68 MiB, well
   under the ADR-117 §5.4 5 MiB budget. Proven: built via the exact PEP517 path,
   `import wifi_densepose.aether/mat/meridian` all succeed.

2. THE WHEEL SMOKE TEST COULD NOT SEE #1.
   CIBW_TEST_COMMAND only asserted `hello()` and PRINTED `__build_features__`.
   The one signal that would reveal missing bindings was dumped to stdout and
   ignored, so a featureless wheel published green. It now ASSERTS the p6
   features are present and imports the three modules. Proven red→green: fails
   on the old featureless wheel ("missing SOTA bindings: [...]"), passes on the
   fixed one.

3. THE AETHER PARITY TESTS WERE NOT PORTABLE ACROSS THE WHEEL MATRIX.
   `test_aether.py` and the native `aether_parity.rs`/`aether_weights_parity.rs`
   hashed the raw f32 embedding bytes (SHA-256) against a committed golden. The
   embedding is pure f32 with transcendental ops (ln/sqrt/cos) that are not
   bit-reproducible across CPUs/libm, so the hash only ever matched the one arch
   that generated it. This passed in python-ci (x86) but fails on the aarch64 /
   macOS-arm wheels this same project ships — latent until #1 is fixed and the
   bindings actually load. Every tolerance/behavioral assertion already passed;
   only the two byte-hash tests failed, which is the signature of a non-portable
   golden, not a logic bug.

   Fix: compare to a committed golden VECTOR within tolerance
   (atol=rtol=1e-4 — ~100x cross-arch f32 drift, ~100x under any real algorithm
   change), on both the Python and native sides against the SAME golden. Native
   ≈ golden and binding ≈ golden together prove binding ≈ native, portably.
   `.sha256` goldens replaced by `.json` vectors.

Also: the aether/mat/meridian import shims told users to `pip install
wifi-densepose[<x>]` on a missing feature — an empty extra that cannot help.
Corrected to name the real fix (rebuild with `--features <x>`); message tests
updated to assert the honest message and forbid the misleading one.

Verified on aarch64/macOS: full `python/tests/` suite 227 passed against a wheel
built with the new mechanism; `cargo check --tests --features aether` clean.
The native `.rs` parity tests were converted by inspection and cargo-checked but
not executed — they link against the PyO3 crate and no workflow runs them today
(a pre-existing gap; wiring `cd python && cargo test --features sota` into
python-ci would make the native anchor actually run).

Co-Authored-By: Ruflo & AQE
This commit is contained in:
Dragan Spiridonov
2026-07-24 10:45:00 +02:00
parent 408caf35fa
commit 3ed43e9a2f
13 changed files with 197 additions and 118 deletions
+22 -5
View File
@@ -111,16 +111,33 @@ jobs:
CIBW_ARCHS_WINDOWS: ${{ matrix.arch }}
CIBW_BUILD_FRONTEND: "build"
CIBW_BEFORE_BUILD: "pip install maturin>=1.7"
# PUBLISHED wheels carry the full SOTA feature set. A pip extra
# (`[aether]`) CANNOT enable a Rust cargo feature on an already-built
# wheel, so the only way P6 reaches PyPI users is to compile it in.
# `default = []` stays in Cargo.toml (dev builds + the wheel-size
# budget job keep guarding the small base compile); the RELEASE build
# opts in here via maturin's PEP517 args. Measured 1.68 MiB — well
# under the ADR-117 §5.4 5 MiB budget. Set per-platform because
# CIBW_ENVIRONMENT_LINUX overrides the general CIBW_ENVIRONMENT.
CIBW_ENVIRONMENT: 'MATURIN_PEP517_ARGS="--features sota"'
# The PyO3 sdist landing depends on the cargo/Rust toolchain
# being present. cibuildwheel images carry rustup on Linux
# but we also pin a known-good version for reproducibility.
CIBW_BEFORE_ALL_LINUX: "curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.82"
CIBW_ENVIRONMENT_LINUX: 'PATH="$HOME/.cargo/bin:$PATH"'
# Smoke-test every built wheel before accepting it. Catches
# the case where the wheel imports but the compiled symbols
# are missing.
CIBW_ENVIRONMENT_LINUX: 'PATH="$HOME/.cargo/bin:$PATH" MATURIN_PEP517_ARGS="--features sota"'
# Smoke-test every built wheel before accepting it. This ASSERTS the
# SOTA bindings are present and importable — the prior version only
# printed __build_features__ and asserted hello(), so a wheel missing
# every P6 binding published green. A featureless wheel now fails here
# instead of shipping an ImportError to users.
CIBW_TEST_REQUIRES: "pytest>=8.0"
CIBW_TEST_COMMAND: 'python -c "import wifi_densepose; assert wifi_densepose.hello() == \"ok\"; print(wifi_densepose.__build_features__)"'
CIBW_TEST_COMMAND: >-
python -c "import wifi_densepose as w;
assert w.hello() == 'ok';
missing = [f for f in ('p6-aether-bindings','p6-meridian-bindings','p6-mat-bindings') if f not in w.__build_features__];
assert not missing, ('published wheel is missing SOTA bindings: ' + str(missing) + ' have=' + str(w.__build_features__));
import wifi_densepose.aether, wifi_densepose.mat, wifi_densepose.meridian;
print('OK — SOTA bindings present:', w.__build_features__)"
with:
package-dir: python
output-dir: wheelhouse