mirror of
https://github.com/ruvnet/RuView
synced 2026-07-20 17:03:24 +00:00
9c49ff1a38ee8960c19e3e418ab65e2ab9099266
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ruv
|
dbcbac1d43 |
feat(adr-110): Python SyncPacket API parity with Rust (apply_to_local + interpolation)
Iter 26 — closes the ABI gap between the Python and Rust SyncPacket
decoders. Before this, Python could decode the wire but had no helpers
to apply offsets or recover per-frame mesh time; any Python-side tooling
(host scripts, replay analysers, notebooks) would have to re-implement
the math from scratch and could drift from Rust silently.
New methods on the Python SyncPacket dataclass:
local_minus_epoch_us() -> int
Signed local-vs-mesh offset. Matches Rust byte-for-byte.
apply_to_local(local_at_frame_us: int) -> int
offset = epoch_us - local_us
return local_at_frame_us + offset
Identity at local_at_frame_us == self.local_us returns epoch_us.
mesh_aligned_us_for_sequence(frame_seq: int, fps_hz: float) -> int
Sequence-based interpolation matching Rust's identical method.
Includes u32 wraparound handling via masked-subtract — verified
against Rust's iter 17 `mesh_aligned_for_sequence_handles_seq_wraparound`.
3 new Python tests (10 total in TestSyncPacketParser, all green in 0.24s):
test_apply_to_local_recovers_epoch_at_sync_point
Identity at the sync point. Also verifies local_minus_epoch_us()
matches §A0.10's measured 1,163,565 µs bench number.
test_apply_to_local_preserves_inter_frame_delta
Frame arriving 5 s after the sync on the follower's local clock
produces mesh time exactly 5 s after sync.epoch_us.
test_mesh_aligned_us_for_sequence_matches_rust
Cross-language parity with Rust's
`end_to_end_sync_decode_then_frame_mesh_recovery` (iter 20):
100 frames after sync.sequence at 20 fps = sync.epoch_us + 5 s.
Cross-checks via apply_to_local — both paths must agree.
Test count after iter 26:
Python TestSyncPacketParser: 10/10 (was 7/7)
Rust sync_packet::tests: 15/15
Combined: 25 unit tests defending the SyncPacket contract across
the two host language stacks.
Co-Authored-By: claude-flow <ruv@ruv.net>
|
||
|
ruv
|
82be960de5 |
test(adr-110): cross-language wire-format conformance gate
Iter 21 — ultra-opt for protocol correctness across the two production decoders. Pin the same 32-byte canonical hex in both Python and Rust tests; if either decoder drifts from the wire, ONE of the tests starts failing — and it's clear which side moved. Canonical packet: COM9 sync-pkt #1 from §A0.12 live capture, expressed as exact little-endian bytes: 10a111c5 09 01 06 00 magic + node + ver + flags + rsvd f26db70100000000 local_us = 28_798_450 c5aca50100000000 epoch_us = 27_634_885 1400000000000000 sequence = 20 + reserved Python test: archive/v1/tests/unit/test_esp32_binary_parser.py::TestSyncPacketParser ::test_canonical_wire_bytes_match_rust_decoder — decodes the pinned hex, asserts every field including the §A0.10 1,163,565 µs offset. Rust test: v2/crates/wifi-densepose-hardware/src/sync_packet.rs::tests ::canonical_wire_bytes_match_python_decoder — decodes the same bytes, asserts the same fields, then re-encodes via to_bytes() and asserts the round-trip produces the EXACT same 32 bytes. So this also catches drift in the Rust encoder. Test counts after this iter: Rust sync_packet: 15/15 green (was 14) Python SyncPacketParser: 7/7 green (was 6) Branch contract: if a future PR changes the firmware wire format, BOTH tests must be updated atomically with the new canonical hex. CI will gate this naturally. Co-Authored-By: claude-flow <ruv@ruv.net> |
||
|
ruv
|
3a6648c290 |
test+docs(adr-110): 6 SyncPacketParser tests + README/user-guide for v0.7.0
Iter 13 — solidifies v0.7.0 as a real, reviewable release.
Tests (archive/v1/tests/unit/test_esp32_binary_parser.py):
- TestSyncPacketParser (6 tests, all passing in 0.27s):
* test_follower_typical_packet_roundtrips — matches the COM9-witnessed
sync-pkt #1 byte-for-byte, including the 1,163,565 µs offset that
§A0.10 measured for the COM9-vs-COM12 boot-time delta
* test_leader_packet_has_local_close_to_epoch — COM12 leader case,
flags=0x03, epoch ≈ local
* test_magic_mismatch_raises — non-sync datagrams don't silently decode
* test_short_packet_raises — early error vs silent truncation
* test_all_flag_combinations — every (leader, valid, smoothed) triple
round-trips independently
* test_dispatch_distinguishes_csi_from_sync — CSI vs sync magics differ
so a host can dispatch by leading u32
Docs:
- README C6 hardware row now headlines v0.7.0 (was v0.6.7), names the
measured 99.56% match / 104 µs stdev / 3.95× suppression numbers, and
acknowledges the firmware-side ADR-110 substrate closure.
- docs/user-guide.md firmware release table now lists v0.7.0 / v0.6.9 /
v0.6.8 / v0.6.7 chain with one-liner highlights so 4MB-flash users +
multistatic operators know which release brings what.
Co-Authored-By: claude-flow <ruv@ruv.net>
|
||
|
ruv
|
d199279caa |
release(firmware): v0.7.0-esp32 major — ADR-110 firmware-side substrate closed
Marks the end of the firmware-side ADR-110 push. Everything the firmware
can deliver toward §B multistatic alignment without hardware-blocked
dependencies is shipped, measured, and witnessed:
§A0.7–§A0.10 ESP-NOW mesh quantified: 99.43-99.56% cross-board match,
104.1 µs smoothed offset stdev, 1.4 ppm crystal-skew
tracking, ≤100 µs alignment target empirically met.
§A0.12 32-byte UDP sync packet emits with mesh-aligned epoch
+ sequence high-water; verified live both boards.
§A0.13 (new) bit-4 wire-fix: byte 19 bit 4 sourced from
c6_sync_espnow_is_valid() too. Mixed S3+C6 fleets now
correctly advertise mesh-sync.
Host-side enabler (Python):
archive/v1/src/hardware/csi_extractor.py grows SyncPacketParser +
SyncPacket dataclass. ESP32BinaryParser docstring acknowledges the
sibling sync packet. Sets up wifi-densepose-sensing-server to
consume the §A0.12 stream without inventing the parser.
Build artifacts (IDF v5.4, both RC=0):
S3 8 MB: 1094 KB, 47% partition slack
C6 4 MB: 1019 KB, 45% partition slack
Tag v0.7.0-esp32. Branch adr-110-esp32c6. PR #764.
What remains is outside the firmware: host-side parser wiring,
multistatic CSI fusion in wifi-densepose-signal, 11ax-cooperative AP
(or future IDF AP-HE API), INA226 for ≤5 µA LP-core.
Co-Authored-By: claude-flow <ruv@ruv.net>
|
||
|
ruv
|
8eaa92cf21 |
feat(python): host-side decode for ADR-018 byte 18-19 (ADR-110)
Python ESP32BinaryParser was using struct format '<IBBHIIBB2x' — the
'2x' skipped bytes 18-19 as reserved. After the Rust-side decoder was
extended to surface PPDU type + flags, the Python pipeline (which
archive/v1 still uses for testing + the proof verifier) needs the same
update so its consumers see the HE metadata too.
csi_extractor.py:
- HEADER_FMT now '<IBBHIIBBBB' (captures bytes 18-19)
- New metadata fields: ppdu_type ('ht_legacy'|'he_su'|'he_mu'|'he_tb'|'unknown'),
ppdu_type_raw, he_capable, bw40, stbc, ldpc, ieee802154_sync_valid,
adr018_flags_raw
- Class constants PPDU_HT_LEGACY..PPDU_UNKNOWN mirror the firmware
test_esp32_binary_parser.py:
- build_binary_frame() takes optional ppdu_byte + flags_byte (default 0)
- New TestAdr110ByteEncoding class with 5 tests:
- Pre-ADR-110 zeros decode as 'ht_legacy' + all-flags-false
- HE-SU / HE-MU / HE-TB decode correctly
- 0xFF decodes as 'unknown'
- All-flags-set round-trip (0x1D)
11/11 parser tests pass (6 existing + 5 new). Backwards compat verified.
Pairs with the Rust-side decoder in commit
|
||
|
rUv
|
50131b2519 |
fix(verify): cross-platform deterministic proof — 6-decimal quantize + thread-pinning (closes #560) (#609)
* fix(verify): quantize features before SHA-256 for cross-platform hash stability (#560) ## The bug archive/v1/data/proof/verify.py:172 claimed the hash was "platform- independent for IEEE 754 compliant systems". That claim is empirically false. scipy.fft's pocketfft uses SIMD vector kernels — AVX2/AVX-512 on x86_64, NEON on Apple Silicon — that reorder vectorized FP operations differently per build. IEEE 754 guarantees per-operation determinism, not associativity under reordering, so two correct platforms produce values that differ at ULP precision (~1e-14 at our magnitudes of 1-100). The SHA-256 of features_to_bytes() then explodes that ULP-level divergence into a totally different hash, which is what bug report #560 caught on macOS arm64: | Platform | numpy/scipy | sha256 (legacy) | |----------|-------------|-----------------| | Windows (Intel AVX-512) | 2.4.2 / 1.17.1 | 78b3fb… | | ruvultra (Linux x86_64) | 1.26.4 / 1.14.1 | 41dc56… | | ruv-mac-mini (Apple Silicon NEON) | 2.4.4 / 1.17.1 | 9b5e19… | ## The fix features_to_bytes() now np.round(.., HASH_QUANTIZATION_DECIMALS=9)s each array before packing as little-endian f64. That snaps the float bytes to a single canonical representation across SIMD backends. The 9-decimal precision is: - ~5 orders of magnitude above the worst-case ULP drift observed in probe-fft-platform.py measurements - Many orders of magnitude below any meaningful signal change (CSI phase precision is ~1e-3 rad; PSD bins differ by orders of magnitude) - Conservative — could tighten to 11-12 decimals if needed, but 9 leaves comfortable headroom for future scipy SIMD changes ## Probe-side verification scripts/probe-fft-platform.py now emits BOTH sha256_raw (unrounded, legacy) and sha256_quantized (new platform-invariant hash). Running it on Windows here produced: sha256_raw = 78b3fb4acb8cc18c3e870f92e29ee98143c7cac4767f2f71b0fc384a82b92f6e sha256_quantized = a587792c050cf697366b9bef4611050f9dc3af56624915ab2452c3c11362e79a quantization_decimals = 9 On Linux and macOS arm64 the maintainer should observe the SAME sha256_quantized value (and a different sha256_raw) — that's the fix working. ## What this PR does NOT do The published archive/v1/data/proof/expected_features.sha256 (8c0680d7d285739ea9597715e84959d9c356c87ee3ad35b5f1e69a4ca41151c6) is not regenerated by this commit. That step needs to run on a canonical CI platform (likely the Linux x86_64 host used for releases) AFTER this fix lands. The regeneration command is: python archive/v1/data/proof/verify.py --generate-hash After regeneration, every platform running ./verify will produce the same hash and the proof replay will be honestly cross-platform — which is what the ADR-028 trust-kill-switch promised. ## Files - archive/v1/data/proof/verify.py — add HASH_QUANTIZATION_DECIMALS=9 constant, quantize in features_to_bytes(), correct the misleading "platform-independent" claim in the docstring - scripts/probe-fft-platform.py — emit both raw and quantized hashes - scripts/fix-markers.json — RuView#560 marker prevents removing the np.round() call without explicit intent - CHANGELOG.md — Fixed entry under [Unreleased] documenting the change and flagging the expected_features.sha256 regeneration as a follow-up Co-Authored-By: claude-flow <ruv@ruv.net> * ci: fix verify-pipeline.yml working-directory from v1/ to archive/v1/ The verify-pipeline workflow's "Run pipeline verification" and "Run verification twice to confirm determinism" steps use `working-directory: v1` but `v1/` was archived to `archive/v1/` long ago. The workflow fails before verify.py even runs: ##[error]An error occurred trying to start process '/usr/bin/bash' with working directory '/home/runner/work/RuView/RuView/v1'. No such file or directory Same v1 → archive/v1 path correction that already shipped for the ./verify wrapper (RuView#559 / PR #590) and the other lint workflows (RuView#489). Required to make the determinism check actually run on PR #609 (the quantize-before-hash work) — the canonical Linux hash needed for expected_features.sha256 will fall out of the next CI log once this fix lands. * fix(proof): regenerate expected_features.sha256 with the quantized canonical hash The hash on the previous line was the legacy pre-quantization value (8c0680d7d28573…), which by definition cannot match the quantized output that this branch's verify.py now produces. Replaced with the canonical Linux x86_64 hash captured from the CI run on this branch: d9985569b3ab833c74b7c9254df568bbb144879e2222edb0bcf2605bfd4c155b Source of truth: run 26005976495 / "Verify Pipeline Determinism (3.11)" on Ubuntu 24.04, Python 3.11.15, exercising the full verify.py pipeline on the 100 reference frames in archive/v1/data/proof/sample_csi_data.json. Reproducibility expectation now changes: - Linux x86_64 (canonical platform): sha256 = d9985569… ✓ this commit - macOS arm64 / Apple Silicon NEON: sha256 = d9985569… should match after quantization - Windows AMD64 (with pydantic-clean .env): sha256 = d9985569… should match after quantization If macOS arm64 still mismatches after this, the quantization decimals need to be tightened from 9 to 11 or 12 (HASH_QUANTIZATION_DECIMALS in verify.py); the headroom analysis in the original commit suggests 9 is safe but 9-decimal SIMD drift hasn't been measured in the full-pipeline output yet (only in the probe). Closes the maintainer-action-required item on PR #609. * fix(proof): bump quantization to 6 decimals (9 wasn't enough across Azure CI microarchs) Two back-to-back Ubuntu 24.04 / Python 3.11 / scipy 1.17 CI runs on PR #609 landed on different Azure VM microarchitectures and produced two different SHA-256s even after np.round(.., 9): Run 1: d9985569b3ab833c74b7c9254df568bbb144879e2222edb0bcf2605bfd4c155b Run 2: 37c49a1f6b87207fa9fc67f2d6a85c4417dd4a536573605fd175510d1dce7cbe Same JSON input, same byte count hashed (294,400), same Python version, same scipy version. The only variable is the underlying CPU pocketfft SIMD kernel. The full DSP pipeline (preprocess → biquad bandpass → FFT → PSD → variance accumulation) amplifies the ~1e-14 raw FFT divergence by several orders of magnitude — the actual drift at features_to_bytes() input can reach 1e-7 or worse, which is well within the 1e-9 quantization window I originally picked. Bumping to 6 decimals = parts per million. ~6 orders of magnitude headroom over observed pipeline-amplified ULP drift. Still far below any meaningful signal change (CSI phase precision ~1e-3 rad). Kept the probe constant in sync. Will trigger CI on this branch immediately after push; the new expected_features.sha256 will be regenerated from whichever microarch the next CI run lands on, but should be stable across all subsequent runs at 6-decimal quantization. * chore(probe): keep HASH_QUANTIZATION_DECIMALS in sync with verify.py (now 6) * fix(proof): regenerate expected_features.sha256 for 6-decimal quantization * ci: pin thread count to 1 for proof verification (scipy.fft threading non-determinism) |
||
|
rUv
|
50136c920d |
fix(archive/v1/pose-service): call sanitize_phase, not sanitize (closes #612) (#614)
Reported by @bannned-bit. archive/v1/src/services/pose_service.py:223:
sanitized_phase = self.phase_sanitizer.sanitize(phase_data)
PhaseSanitizer exposes the full-pipeline entry point as `sanitize_phase`
(unwrap_phase + remove_outliers + smooth_phase), not `sanitize`. The
shorter name doesn't exist on the class, so any path that reaches this
branch raises AttributeError mid-frame and crashes the pose service.
archive/v1/src/core/phase_sanitizer.py:266 is the canonical name:
def sanitize_phase(self, phase_data: np.ndarray) -> np.ndarray:
"""Sanitize phase data through complete pipeline."""
One-line rename. No other call sites use the wrong name; verified with
grep -rn 'phase_sanitizer\.sanitize\b' archive/v1/src/.
This is v1 archived code, but the proof verify path still exercises it
(./verify reaches into archive/v1/src/), so the bug was a latent
regression risk for the trust-kill-switch flow.
|
||
|
DavidKrame
|
68b042faf6 | fix(archive/v1): middleware inherits BaseHTTPMiddleware to fix 500 errors (#570) | ||
|
rUv
|
81cc241b9e |
chore(repo): move v1/ → archive/v1/ + add archive/README.md (#430)
The Rust port at v2/ has been the primary codebase since the rename in #427. The Python implementation at v1/ is no longer the active target; the only load-bearing path is the deterministic proof bundle at v1/data/proof/ (per ADR-011 / ADR-028 witness verification). Move the whole Python tree into archive/v1/ and document the policy in archive/README.md: no new features, bug fixes only when they affect a still-load-bearing path (currently just the proof), CI continues to verify the proof on every push and PR. Path references updated in 26 files via path-pattern sed (only matches v1/<known-child> patterns, never bare v1 or API URLs like /api/v1/). Two double-prefix typos (archive/archive/v1/) caught and hand-fixed in verify-pipeline.yml and ADR-011. Validated: - Python proof verify.py imports cleanly at archive/v1/data/proof/ (numpy/scipy still required; CI installs requirements-lock.txt from archive/v1/ now) - cargo test --workspace --no-default-features → 1,539 passed, 0 failed, 8 ignored (unaffected by Python tree relocation) - ESP32-S3 on COM7 untouched (no firmware paths changed) After-merge: contributors should re-run any local `python v1/...` commands as `python archive/v1/...` (CLAUDE.md and CHANGELOG already updated). |