mirror of
https://github.com/ruvnet/RuView
synced 2026-07-26 18:01:48 +00:00
f783df234eec22929b88e256422cbfc50579196b
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dragan Spiridonov
|
788d685b9f |
fix(python,training): validate PyO3 inputs; make the single-training-job guard atomic
Closes the two findings from the adversarial review that were verified but left unfixed. Both now have proper tests, each proven to fail against the old code. 1. PyO3 bindings panicked / over-allocated on caller input (aether.rs). `EmbeddingExtractor(n_heads=0)` reached `d_model % n_heads` in the transformer and panicked (surfacing to Python as an opaque PanicException); a non-divisor head count tripped the native assert; and `AetherConfig(d_model=100_000)` allocated multi-gigabyte weight matrices that abort the interpreter. The binding did no validation. Now both constructors return `PyResult` and validate at the boundary — positive dims, `d_model % n_heads == 0`, and a generous MAX_DIM/MAX_LAYERS cap — raising `ValueError`. Proven: `n_heads=0` -> "n_heads must be positive", `d_model=100_000` -> a clean ValueError, both previously a PanicException / abort. +10 pytest cases (test_aether.py); a valid config still constructs and embeds. 2. The single-training-job guard was a TOCTOU race (training_api.rs). `spawn_training_job` checked `is_active()` under a `state` READ lock, released it, then set `active` later. A tokio RwLock read lock is SHARED, so two concurrent `POST /train/start` could both hold it, both see the slot free, and both spawn jobs — sharing/overwriting one status+cancel and orphaning a task handle. Extracted `claim_training_slot`, which does the check-and-set in ONE `status` mutex scope — the atomicity lives on the status mutex, not the coarse state lock — so concurrent starts serialise and exactly one wins. This also makes it unit-testable without a full AppState. Test: 32 threads hit a barrier and race to claim; asserts EXACTLY ONE wins. Mutation-proven — reverting to the split check-then-set makes it fail (`left: 3, right: 1`), and it returns to 1 with the fix. Verified on aarch64/macOS: training_api 28 pass (26 existing + 2 new), full python/tests suite 237 pass (227 + 10). The native module keeps its internal assert as a defence-in-depth invariant; the binding now enforces it at the edge. Co-Authored-By: Ruflo & AQE |
||
|
ruv
|
65da488add |
feat(adr-185): add weight-loading capability to AETHER EmbeddingExtractor (§13.a)
Closes the ADR-185 §13.a follow-up (the tractable, data-independent one of
the three §6.7 gaps): the bound EmbeddingExtractor was random-Xavier-init
only, with NO way to load real weights — so it was structurally untrained.
This adds the *capability* to load weights whenever a trained checkpoint
exists. It does NOT itself produce trained/SOTA embeddings and does NOT
close §6.7 (still no trained checkpoint, no labeled data, no eval harness).
Serialization: not greenfield — EmbeddingExtractor already had
flatten_weights()/unflatten_weights() (flat Vec<f32>). wifi-densepose-aether
is a deliberately dependency-free std-only leaf crate (ADR-185 §13), so
rather than add safetensors/serde/bincode (which would undo the zero-dep
property), the on-disk format is raw little-endian f32 with a 12-byte header
(magic "AETHERW1" + u32 param count) — zero new deps.
Native (v2/crates/wifi-densepose-aether/src/embedding.rs):
- EmbeddingExtractor::save_weights(path) / load_weights(path). load_weights
never panics: errors on unreadable file, short/oversized payload, bad
magic, or a param-count mismatch (delegated to unflatten_weights).
- Default (no weights) construction is UNCHANGED — still random init,
clearly labeled untrained. Purely additive.
Python binding (python/src/bindings/aether.rs, .pyi):
- EmbeddingExtractor.load_weights(path) / save_weights(path) / param_count,
GIL-released, ValueError on bad input.
Tests (both Rust and Python, per the task):
- Rust unit (embedding.rs): load_weights_actually_replaces_weights_and_
round_trips proves the loaded weights MOVE the embedding away from the
random-init baseline (not a silent no-op), match the source extractor
(round-trip), and are bit-identical after the file round-trip; plus a
bad-magic/wrong-count rejection test.
- Cross-language golden (aether_weights_parity.rs + test_aether.py): a
shared deterministic weight formula (w[i]=k/65536-0.5, exact in f32+f64)
written to the AETHER format; both native Rust and the Python binding load
it and must produce the byte-identical embedding SHA-256
(tests/golden/aether_loaded_embedding.sha256) — proving the binding's
load path is bit-identical to native, and (vs the random baseline) that
the loaded weights are actually used.
Verified:
cargo test -p wifi-densepose-aether 98 passed, 0 failed
cargo test --features aether --test aether_parity
--test aether_weights_parity 3 passed, 0 failed
maturin develop --features aether + pytest test_aether.py 13/13 pass
default cargo build (no aether feature) clean
|
||
|
ruv
|
a47bb71b22 |
fix(adr-185): hoist AETHER compute surface into a std-only leaf crate
ADR-185 §13 flagged the `[aether]` wheel as linking the whole
`wifi-densepose-sensing-server` Axum/tokio/worldgraph/ruvector tree via the
non-optional deps of that crate. Hoist the pure-compute AETHER stack out so the
binding depends only on pure Rust.
New crate `wifi-densepose-aether` (std-only, ZERO external deps) holds the four
self-contained modules the AETHER surface needs — `embedding`, `graph_transformer`,
`sona`, `sparse_inference` (verified: no serde/tokio/axum/crate:: refs outside the
set; the four form a closed dependency graph). `wifi-densepose-sensing-server`
now `pub use`s them from the leaf crate, so its own code (`crate::embedding`, …)
and public API (`wifi_densepose_sensing_server::embedding`, …) are unchanged —
`main.rs`/`trainer.rs` untouched. The Python binding + parity test + `[aether]`
feature repoint at the leaf crate; the sensing-server dep is dropped from python.
MEASURED wheel size (maturin build --release --features aether --strip; each
wheel install-verified to actually contain `AetherConfig`):
BEFORE (aether -> sensing-server): 369,782 B (~361 KB)
AFTER (aether -> leaf crate): 319,719 B (~312 KB)
HONEST CORRECTION to the §9/§13 premise: the pre-hoist wheel was NOT over the
ADR-117 §5.4 5 MB budget — it was ~361 KB, ~14x under. Linker dead-code
elimination (--gc-sections on the pyo3 cdylib) already strips the unreached
server tree because the binding only reaches pure-compute symbols. So this is
"blows the budget" CLAIMED, not MEASURED. The hoist is still worthwhile and
lands the real, measured wins:
- `[aether]` build time 71s -> 12s (no server tree to compile),
- python/Cargo.lock -1238 lines (server tree no longer resolved),
- ~50 KB smaller wheel,
- honest, minimal declared dependency surface + removes the latent risk that a
future change makes some server code reachable and *then* bloats the wheel.
Verified this session (real counts):
- cargo test --features aether --test aether_parity: 2 passed / 0 failed
- pytest tests/test_aether.py (maturin develop --features aether): 9 passed
- cargo test -p wifi-densepose-sensing-server --no-default-features: 217 bin +
388 lib, 0 failed (no regression)
- cargo test -p wifi-densepose-aether: 96 passed (the relocated unit tests)
The §9/§13 wheel-size narrative in ADR-185 (owned by the ADR author) should be
corrected to reflect the measured reality — flagged to the team lead.
|
||
|
ruv
|
d060998e3b |
feat(adr-185): P1 AETHER bindings (wifi_densepose.aether) + parity harness
Bind the ADR-024 contrastive CSI embedding surface into the wheel behind
a gated [aether] extra / Cargo `aether` feature, per ADR-185 section 3.2.
Surface (bound against the REAL code at HEAD, not the ADR wishlist):
- AetherConfig -> EmbeddingConfig {d_model,d_proj,temperature,normalize}
- CsiAugmenter -> augment_pair(window, seed)
- EmbeddingExtractor -> embed(csi): 128-dim L2-normed, GIL-released
- info_nce_loss, cosine_similarity (module fns)
ADR-185 section 3.2 also names aether_loss/VICReg components,
alignment_metric, uniformity_metric, forward_dual, and vicreg_* config
fields. None exist in embedding.rs at HEAD, so they are intentionally NOT
bound (a Rust-side gap, not a binding gap) rather than fabricated.
Parity (section 4.1, release-blocking): committed fixture
aether_input.json -> native Rust reference (tests/aether_parity.rs, calls
sensing-server EmbeddingExtractor directly) locks
tests/golden/aether_embedding.sha256; pytest (tests/test_aether.py) runs
the same fixture through the binding and asserts the identical SHA-256 of
the LE f32 bytes. Both pass.
Verified: cargo test --features aether --test aether_parity -> 2/2 pass;
maturin develop --features aether + pytest tests/test_aether.py -> 9/9
pass; default cargo build clean with 0 sensing-server refs in the dep
graph (gate keeps the base wheel lean).
HONEST WHEEL-SIZE FINDING (ADR-185 section 9 / Open Q 11.1, unresolved):
wifi-densepose-sensing-server is an Axum/tokio crate whose tokio/axum/
worldgraph/ruvector deps are NON-optional (only mqtt/matter are
features), so default-features=false does NOT drop them. An [aether]
wheel therefore links the full server tree and BREAKS the ADR-117 section
5.4 <=5 MB budget. The fix is the ADR-sanctioned hoist of embedding.rs
(+ its graph_transformer/sona siblings) into a leaf crate so the wheel
links pure compute only -- a change INSIDE wifi-densepose-sensing-server,
owned by another agent this session, so deferred as a required
pre-release follow-up. P1 binds real code and proves parity today; the
hoist is a wheel-size optimization, not a functional blocker.
Note: building required initializing the worldgraph/rufield/ruvector
submodules (absent in the fresh worktree).
|