mirror of
https://github.com/ruvnet/RuView
synced 2026-06-20 12:03:19 +00:00
ruv
49e57efcec
The structural advantage that's the entire point of ADR-096: O(log T) per new token via decode_step against an accumulated KvCache, vs O(N²) recompute for dense MHA. This commit lands the API and proves the numerical equivalence at the last position. API: - AetherTemporalHead::step(q_new, k_new, v_new, &mut cache) Single-token decode. Appends (k_new, v_new) to cache, runs decode_step(q_new) against the now-updated cache, returns the new position's output. - AetherTemporalHead::make_cache(capacity) Convenience constructor — caller doesn't need to import ruvllm_sparse_attention to size a cache. Per ADR-096 §8.5 the natural lifetime is per-PoseTrack (re-ID) or per-session (online classification); when the track drops, drop the cache. - KvCache re-exported at the crate root. Contract: - q_new/k_new/v_new must each have seq == 1. Multi-token q is the prefill path (forward), not decode_step. - Cache lifetime is the caller's. The crate enforces shape via make_cache so callers can't mismatch kv_heads / head_dim / block_size. - KvCache fill is the caller's problem. Upstream H2O heavy-hitter eviction is opt-in; this crate's wrapper doesn't pre-pick a policy. Tests (18/18 total now passing): - streaming_step_matches_forward_at_last_position — central claim: 16-token sequence, append k/v one at a time via step(), compare the streamed last-token output to forward(full Q,K,V)[N-1]. max_abs_err < 1e-3 (currently passes well under that bound for the 0.1-magnitude activations the test uses). - step_rejects_multi_token_q — contract enforcement. - make_cache_returns_kvcache_with_correct_shape — wiring smoke, confirms (capacity, kv_heads, dim, block_size) ordering is correct through the make_cache wrapper. Test config uses MHA shape (q_heads == kv_heads) because the upstream decode_step is wired to the MHA branch; the GQA decode path is on upstream's roadmap and lands in a separate ADR-096 follow-up when it does. Co-Authored-By: claude-flow <ruv@ruv.net>