mirror of
https://github.com/ruvnet/RuView
synced 2026-07-24 17:43:20 +00:00
feat(aether-arena): benchmark-first scorer + witness chain + repeatability (M2/M5/M7)
Per direction "remove the initial number, optimize for benchmark first" + "include witness chain capabilities for proof and repeatability analysis": - Empty board, no seeded numbers: ledger seeds to genesis only. Every result is a real scoring-pipeline witness; RuView gets no hand-entered baseline. - Real model scoring: aa_score_runner now loads predictions + an eval split (--split/--pred) and scores them through the real ruview_metrics pose harness — not just a synthetic fixture. Committed public smoke split (fixtures/smoke_*.json). - Witness chain: each score emits a witness = inputs_sha256 (binds it to the exact inputs) + proof_sha256 (cross-platform-stable score hash) + harness_version. - Repeatability analysis: --repeat N runs the harness N× and fails if it ever yields >=2 distinct proof hashes (16/16 identical locally). - Witness ledger: ledger/ledger_tools.py — append-only, hash-chained, tamper- evident (seed/append/verify); editing any past row breaks the chain. - CI gate extended: determinism + repeatability(16) + real-scoring smoke + ledger chain verify on every PR. Co-Authored-By: claude-flow <ruv@ruv.net>
This commit is contained in:
@@ -55,17 +55,38 @@ jobs:
|
||||
- name: Run determinism gate
|
||||
run: cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features
|
||||
|
||||
# 3. Emit the score row into the PR run summary (leaderboard-ledger shape).
|
||||
- name: Score row → job summary
|
||||
# 3. Repeatability analysis (witness chain): the harness must produce one
|
||||
# identical proof hash across many runs — any nondeterminism fails here.
|
||||
- name: Repeatability analysis (16 runs)
|
||||
run: cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --repeat 16
|
||||
|
||||
# 4. Real-scoring smoke: score a sample prediction against the public smoke
|
||||
# split, exercising the actual model-scoring path (not just the fixture).
|
||||
- name: Real-scoring smoke test
|
||||
run: |
|
||||
cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- \
|
||||
--split ../aether-arena/fixtures/smoke_split.json \
|
||||
--pred ../aether-arena/fixtures/smoke_pred.json --json
|
||||
|
||||
# 5. Witness ledger chain integrity: the append-only results ledger must
|
||||
# verify (every prev_hash link + row_hash intact = no silent edits).
|
||||
- name: Verify witness ledger chain
|
||||
working-directory: aether-arena/ledger
|
||||
run: python3 ledger_tools.py verify
|
||||
|
||||
# 6. Emit the witness row + repeatability into the PR run summary.
|
||||
- name: Witness row → job summary
|
||||
if: always()
|
||||
run: |
|
||||
ROW=$(cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --json)
|
||||
REP=$(cargo run -q -p wifi-densepose-train --bin aa_score_runner --no-default-features -- --repeat 16)
|
||||
{
|
||||
echo "## AetherArena harness gate"
|
||||
echo "## AetherArena harness gate (witness chain)"
|
||||
echo ""
|
||||
echo "Deterministic score row (ADR-149 §2.2):"
|
||||
echo "Deterministic witness (ADR-149 §2.2 / proof + repeatability):"
|
||||
echo '```json'
|
||||
echo "$ROW"
|
||||
echo "$REP"
|
||||
echo '```'
|
||||
echo ""
|
||||
echo "If the determinism gate failed, the scoring maths changed: regenerate with"
|
||||
|
||||
Reference in New Issue
Block a user