mirror of
https://github.com/ruvnet/RuView
synced 2026-07-24 17:43:20 +00:00
test(python): close the three parity-review findings (native anchor in CI, NaN, cosine)
A cross-vendor review of the parity rework (2febbb81) found three issues; all
verified and fixed here.
1. HIGH — the native≈golden half never ran in CI, so a binding marshalling bug
could pass. The golden vectors were regenerated through the Python binding,
and only pytest (binding vs that golden) runs in CI — the native reference
tests in `python/tests/aether_parity.rs` link against the PyO3 crate and no
workflow runs them. A stable PyO3 conversion defect present at regeneration
would therefore be baked into the golden and go undetected.
Fix: a real native parity test IN the `wifi-densepose-aether` crate
(`tests/golden_parity.rs`), which is std-only and a member of the v2
workspace, so it runs under the existing `cargo test --workspace`. It
recomputes the embedding with no PyO3/marshalling and asserts it matches the
SAME committed golden within tolerance. Now native≈golden AND binding≈golden
both run in CI ⇒ binding≈native, and a binding-specific artifact in the
golden surfaces here as a native mismatch. serde_json added as a
dev-dependency only (test-only; never linked into the lib or the wheel, so
the crate stays runtime-dependency-free).
Independently proven at this commit: native output equals the committed
golden BIT-FOR-BIT (128/128 exact bits, Δ=0) for both base and loaded — the
golden is native-faithful today; this test keeps it that way.
2. MEDIUM — the Python parity helper passed non-finite output. `abs(nan - b) >
tol` is False, so an all-NaN embedding slipped through. Now every element
must be `math.isfinite` first. Proven: an all-NaN vector is rejected
("element 0 is not finite (nan)"); the Rust helpers already caught it via
`<=`.
3. LOW — a coherent shift inside the per-element tolerance could move the whole
vector undetected. Added a whole-vector cosine-similarity bound (≥ 1 - 1e-6)
alongside the per-element check.
Verified on aarch64/macOS: `test_aether` 13, full `python/tests/` suite 227
passed against a `--features sota` build; the new native test passes
(2/2) run standalone (the in-worktree `cargo test -p` only failed on an
un-checked-out submodule; ci.yml checks out submodules recursively).
Co-Authored-By: Ruflo & AQE
This commit is contained in:
+31
-13
@@ -37,23 +37,41 @@ def load_input() -> list[list[float]]:
|
||||
|
||||
|
||||
def assert_embedding_matches_golden(embedding: list[float], golden_name: str) -> None:
|
||||
"""Assert `embedding` matches the committed golden vector within tolerance."""
|
||||
"""Assert `embedding` matches the committed golden vector.
|
||||
|
||||
Two independent checks, because a per-element tolerance alone is not enough:
|
||||
- **Per element**, within atol+rtol — catches a single component drifting.
|
||||
- **Whole-vector cosine** ≥ 1 - 1e-6 — catches a *coherent* shift that stays
|
||||
inside the per-element bound on every component yet moves the vector as a
|
||||
whole (the failure mode a loose element tolerance would hide).
|
||||
|
||||
NaN/inf are rejected explicitly: `abs(nan - b) > tol` is False, so a bare
|
||||
tolerance check would silently PASS an all-NaN embedding. Every value must be
|
||||
finite first.
|
||||
"""
|
||||
golden = json.loads((GOLDEN / golden_name).read_text())
|
||||
assert len(embedding) == len(golden), (
|
||||
f"{golden_name}: length {len(embedding)} != golden {len(golden)}"
|
||||
)
|
||||
worst = max(
|
||||
(abs(a - b), i, a, b)
|
||||
for i, (a, b) in enumerate(zip(embedding, golden))
|
||||
if abs(a - b) > PARITY_ATOL + PARITY_RTOL * abs(b)
|
||||
) if any(
|
||||
abs(a - b) > PARITY_ATOL + PARITY_RTOL * abs(b)
|
||||
for a, b in zip(embedding, golden)
|
||||
) else None
|
||||
assert worst is None, (
|
||||
f"{golden_name}: element {worst[1]} diverged from native golden beyond "
|
||||
f"tolerance (got {worst[2]}, golden {worst[3]}, |Δ|={worst[0]:.3e}) — "
|
||||
"a real regression, not cross-arch f32 drift."
|
||||
for i, x in enumerate(embedding):
|
||||
assert math.isfinite(x), f"{golden_name}: element {i} is not finite ({x})"
|
||||
|
||||
for i, (a, b) in enumerate(zip(embedding, golden)):
|
||||
tol = PARITY_ATOL + PARITY_RTOL * abs(b)
|
||||
assert abs(a - b) <= tol, (
|
||||
f"{golden_name}: element {i} diverged from native golden beyond "
|
||||
f"tolerance (got {a}, golden {b}, |Δ|={abs(a - b):.3e}) — "
|
||||
"a real regression, not cross-arch f32 drift."
|
||||
)
|
||||
|
||||
dot = sum(a * b for a, b in zip(embedding, golden))
|
||||
na = math.sqrt(sum(a * a for a in embedding))
|
||||
nb = math.sqrt(sum(b * b for b in golden))
|
||||
cosine = dot / (na * nb) if na > 0 and nb > 0 else 0.0
|
||||
assert cosine >= 1.0 - 1e-6, (
|
||||
f"{golden_name}: whole-vector cosine similarity to the golden is "
|
||||
f"{cosine:.9f} (< 1 - 1e-6) — a coherent shift the per-element "
|
||||
"tolerance did not catch."
|
||||
)
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user