mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
ruv
a6808568a2
AetherArena ("AA") — the official, project-agnostic Spatial-Intelligence Benchmark
(ADR-149, Accepted). Iteration 1 of the long-horizon build:
- ADR-149 accepted: name locked (ruvnet/aether-arena), v0 metrics locked
(pose/presence/latency/determinism), dataset legality resolved (MM-Fi CC BY-NC
only; Wi-Pose excluded). Adds four-part framing, threat model, arena_score
formula, submission state machine, neutrality/governance, and the §7 acceptance test.
- aa_score_runner: deterministic scorer bin reusing the real ruview_metrics pose
harness on a fixed seed=42 fixture → RuViewTier-style verdict + cross-platform
SHA-256 proof hash. Builds --no-default-features (no torch/GPU). VERDICT: PASS.
- CI harness gate: .github/workflows/aether-arena-harness.yml runs the scorer on
every PR — the "PR that runs the harness as part of the build" requirement.
- Scaffold: aether-arena/{README,VERIFY,STATUS}.md + schema/aa-submission.toml.
- Horizon record persisted (.claude-flow/horizons/aether-arena-aa.json).
Infra = the deliverable; model SOTA (MM-Fi PCK@20) is a separate effort blocked on
ADR-079 data collection, tracked as a stretch goal, not an infra exit.
Co-Authored-By: claude-flow <ruv@ruv.net>
1.6 KiB
1.6 KiB
AetherArena — Build Status
Tracks ADR-149 implementation milestones. "Complete" = benchmark infrastructure done, tested, CI-gated, deploy-ready, RuView baseline entered, §7 acceptance test passing. Model SOTA (e.g. MM-Fi PCK@20 ~72%) is a separate long-running ML effort, blocked on ADR-079 camera-ground-truth collection — not an infra-completion blocker.
| # | Milestone | Status |
|---|---|---|
| M1 | ADR-149 Accepted + committed | ✅ done |
| M2 | Deterministic scorer runner (aa_score_runner) → tier + proof hash |
✅ done — builds --no-default-features, hash stable, VERDICT: PASS |
| M3 | CI harness-gate workflow (PR runs the scorer) | ✅ done — .github/workflows/aether-arena-harness.yml |
| M4 | Scaffold: README + submission schema + VERIFY (acceptance test) | ✅ done |
| M5 | Public smoke split (committed) + private MM-Fi held-out split prep | ⏳ next |
| M6 | HF Space (Gradio) submission flow + sandboxed scorer container | ⛔ blocked — needs HF token / maintainer authorization to deploy |
| M7 | Signed append-only Parquet results ledger | ⏳ |
| M8 | RuView baseline entry (honest PCK@20) + public launch | ⏳ |
Blockers / decisions needed
- HF deploy (M6) needs an HF token and authorization to create the public
ruvnet/aether-arenaSpace. - MM-Fi is CC BY-NC → AA must stay non-commercial / legally distinct from the commercial RuView product.
- Realism of M2 fixture: current fixture is a determinism fixture (stable hash), not a realistic baseline; M5 swaps in real MM-Fi held-out scoring.