feat: add bounded nightly SOTA research agent

This commit is contained in:
ruv
2026-07-29 16:13:28 -04:00
parent a34bfc246e
commit dc03d174ee
7 changed files with 3214 additions and 0 deletions
+83
View File
@@ -0,0 +1,83 @@
# Nightly SOTA research agent
`nightly-sota-agent.yml` turns recent public research into at most one
repository issue and, for low-risk topics, one draft offline-prototype pull
request. It is intentionally not a general-purpose autonomous coding agent.
## Enablement
The committed schedule is `03:17 UTC` every day. Scheduled runs stay disabled
until both repository settings exist:
1. Actions secret `COGNITUM_NIGHTLY_API_KEY`, issued with only the Cognitum
`completions:mid` scope.
2. Actions variable `RUVIEW_NIGHTLY_SOTA_ENABLED=true`.
The key must not receive guidance-write, evolve, pods, brain, Flywheel-write,
or administrative scopes. First run the workflow manually in `dry-run` mode;
that mode only collects a bounded evidence artifact and never reads the secret
or writes an issue. Manual `live` mode is restricted to the repository owner.
The repository must also allow GitHub Actions to create pull requests. Normal
branch protection must require at least one approving review and the
`Verify contributor harness` status check. The publisher requires that exact
job-name check to be bound to the GitHub Actions app,
uses GitHub's effective-active-rules endpoint, and stops before prototype
generation when either requirement is absent. It does not request an
administrative token to inspect hidden ruleset bypass actors; safety does not
depend on that metadata because the publisher has no merge or `main`-push path.
## Authority split
| Job | External credential | Repository authority | Result |
|---|---|---|---|
| `collect` | none | contents read | Normalized public Cognitum registry and recent arXiv evidence |
| `propose` | Cognitum completions key | contents read | One schema-checked proposal |
| `score` | none | contents read | Frozen Darwin digest, completeness score, honest-null Flywheel replay |
| `issue` | GitHub token | issue write, PR read | One deduplicated issue |
| `implement` | Cognitum completions key | contents read | Declarative transform and test vectors |
| `validate` | none | contents read | Schema, template, syntax, claim, path, digest, and replay checks |
| `publish` | GitHub token | branch/issue/draft-PR/Actions write | One draft PR and an explicit read-only harness-verifier dispatch |
The Cognitum key and a write-capable GitHub token never coexist in one job.
Model output is never executable code. Repository-owned templates emit the
prototype module and tests, which this workflow syntax-checks but never runs.
## Hard boundaries
- Public HTTPS sources are fixed to the Cognitum application registry and the
arXiv Atom API. Redirects, oversized responses, unexpected media types, and
schema drift fail closed.
- Retrieved text is `CLAIMED`, untrusted evidence. It is quoted inside a fixed
trusted prompt and cannot grant authority.
- The Darwin genome is read-only. Scheduled jobs never invoke Darwin evolution.
- Flywheel runs a separate committed honest-null canary. A valid canary stays
root-only, rejects its candidate, and reports zero verified improvements and
no promotion. It does not evaluate the nightly proposal. The workflow's
static authority split and artifact gates are what prevent nightly learning
or promotion.
- High-risk topics stop at an issue. This includes production, security,
authentication, release/deployment, workflows, dependencies, firmware,
hardware, networking, native plugins, HomeKit pairing, and voice protocols.
- Low-risk model output is a closed transform DSL: bounded scalar test vectors
and 1-8 allowlisted operations (`center`, `normalize-peak`, `absolute`,
`square`, `difference`, `moving-average`, or `clip`). Local trusted templates
emit exactly five `.md`, `.json`, and `.mjs` files below
`examples/research-sota/nightly/<fingerprint>/`. Existing files, symlinked
parents, dependencies, binaries, executable modes, and more than 400 lines
are rejected.
- Publication is a draft PR. The agent cannot approve, merge, release, promote,
or modify the reviewed shared brain.
## Deduplication and failure behavior
The stable fingerprint hashes sorted evidence IDs, finding class, and subsystem.
Issues and PRs carry an exact hidden marker. Only markers on
`github-actions[bot]` records with the automation label are trusted for
deduplication, so copied issue text cannot suppress future runs.
A failure leaves the last completed bounded artifact for seven days. Model,
protection-preflight, or validation failures may leave an issue without a PR;
maintainers can inspect the run and decide whether to continue manually. The
workflow does not retry a failed model call, force-push a branch, close an
issue, or delete a branch.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+345
View File
@@ -0,0 +1,345 @@
name: Nightly SOTA research agent
on:
schedule:
- cron: '17 3 * * *'
workflow_dispatch:
inputs:
mode:
description: 'dry-run collects evidence only; live may create one issue and one draft prototype PR'
required: true
default: dry-run
type: choice
options:
- dry-run
- live
permissions: {}
concurrency:
group: nightly-sota-agent
cancel-in-progress: false
env:
NODE_VERSION: '20'
jobs:
collect:
name: Collect public evidence
if: >-
github.repository == 'ruvnet/RuView' &&
github.ref == 'refs/heads/main' &&
(github.event_name == 'workflow_dispatch' || vars.RUVIEW_NIGHTLY_SOTA_ENABLED == 'true')
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- name: Collect bounded public evidence
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs collect
--out "${RUNNER_TEMP}/nightly-sota/evidence.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/evidence.json
if-no-files-found: error
retention-days: 7
propose:
name: Synthesize bounded proposal
if: >-
needs.collect.result == 'success' &&
(
github.event_name == 'schedule' ||
(inputs.mode == 'live' && github.actor == github.repository_owner)
)
needs: collect
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- name: Synthesize one proposal with Cognitum
env:
COGNITUM_NIGHTLY_API_KEY: ${{ secrets.COGNITUM_NIGHTLY_API_KEY }}
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs propose
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--repo-root "${GITHUB_WORKSPACE}"
--proposal-out "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--receipt-out "${RUNNER_TEMP}/nightly-sota/propose/cognitum-receipt.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose/
if-no-files-found: error
retention-days: 7
score:
name: Verify frozen Darwin and Flywheel score
needs: propose
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose
- name: Install exact-pinned Flywheel development dependencies
working-directory: harness/ruview
run: npm ci --ignore-scripts --omit=optional
- name: Audit Flywheel dependency graph
working-directory: harness/ruview
run: npm audit --omit=optional
- name: Score with frozen Darwin policy and honest-null Flywheel replay
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs score
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--proposal "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--repo-root "${GITHUB_WORKSPACE}"
--score-out "${RUNNER_TEMP}/nightly-sota/score/score.json"
--replay-out "${RUNNER_TEMP}/nightly-sota/score/replay.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-score-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/score/
if-no-files-found: error
retention-days: 7
issue:
name: Deduplicate and create issue
needs: score
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
issues: write # Create the single labelled research issue.
pull-requests: read # Stop before spending on a fingerprint with an existing bot PR.
outputs:
should_implement: ${{ steps.triage.outputs.should_implement }}
issue_number: ${{ steps.triage.outputs.issue_number }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-score-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/score
- name: Deduplicate or create one issue
id: triage
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs issue
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--proposal "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--proposal-receipt "${RUNNER_TEMP}/nightly-sota/propose/cognitum-receipt.json"
--score "${RUNNER_TEMP}/nightly-sota/score/score.json"
--replay "${RUNNER_TEMP}/nightly-sota/score/replay.json"
--repo-root "${GITHUB_WORKSPACE}"
--out "${RUNNER_TEMP}/nightly-sota/issue/issue.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-issue-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/issue/
if-no-files-found: error
retention-days: 7
implement:
name: Generate offline prototype bundle
if: needs.issue.outputs.should_implement == 'true'
needs: issue
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose
- name: Generate a bounded offline prototype with Cognitum
env:
COGNITUM_NIGHTLY_API_KEY: ${{ secrets.COGNITUM_NIGHTLY_API_KEY }}
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs implement
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--proposal "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--repo-root "${GITHUB_WORKSPACE}"
--bundle-out "${RUNNER_TEMP}/nightly-sota/implement/bundle.json"
--receipt-out "${RUNNER_TEMP}/nightly-sota/implement/cognitum-receipt.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-implementation-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/implement/
if-no-files-found: error
retention-days: 7
validate:
name: Validate without external credentials
needs: [score, implement]
runs-on: ubuntu-latest
timeout-minutes: 15
permissions:
contents: read
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
persist-credentials: false
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-score-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/score
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-implementation-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/implement
- name: Install exact-pinned Flywheel verification dependency
working-directory: harness/ruview
run: npm ci --ignore-scripts --omit=optional
- name: Validate without model or GitHub write credentials
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs validate
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--proposal "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--proposal-receipt "${RUNNER_TEMP}/nightly-sota/propose/cognitum-receipt.json"
--score "${RUNNER_TEMP}/nightly-sota/score/score.json"
--replay "${RUNNER_TEMP}/nightly-sota/score/replay.json"
--bundle "${RUNNER_TEMP}/nightly-sota/implement/bundle.json"
--implementation-receipt "${RUNNER_TEMP}/nightly-sota/implement/cognitum-receipt.json"
--repo-root "${GITHUB_WORKSPACE}"
--out "${RUNNER_TEMP}/nightly-sota/validate/validation.json"
- uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
with:
name: nightly-sota-validation-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/validate/
if-no-files-found: error
retention-days: 7
publish:
name: Publish draft prototype PR
needs: [issue, validate]
runs-on: ubuntu-latest
timeout-minutes: 10
permissions:
actions: write # Dispatch the read-only contributor-harness verifier for the generated branch.
contents: write # Push the one new prototype-only branch.
issues: write # Label the draft PR and link it from the issue.
pull-requests: write # Create a draft PR; the script has no approve or merge path.
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4
with:
ref: ${{ github.sha }}
fetch-depth: 1
persist-credentials: true
submodules: false
- uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4
with:
node-version: ${{ env.NODE_VERSION }}
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-evidence-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/collect
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-proposal-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/propose
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-score-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/score
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-issue-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/issue
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-implementation-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/implement
- uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4
with:
name: nightly-sota-validation-${{ github.run_id }}
path: ${{ runner.temp }}/nightly-sota/validate
- name: Publish one draft PR and dispatch the read-only verifier
env:
GITHUB_TOKEN: ${{ github.token }}
run: >-
node --disable-proto=throw .github/scripts/nightly-sota/agent.mjs publish
--evidence "${RUNNER_TEMP}/nightly-sota/collect/evidence.json"
--proposal "${RUNNER_TEMP}/nightly-sota/propose/proposal.json"
--proposal-receipt "${RUNNER_TEMP}/nightly-sota/propose/cognitum-receipt.json"
--score "${RUNNER_TEMP}/nightly-sota/score/score.json"
--replay "${RUNNER_TEMP}/nightly-sota/score/replay.json"
--issue "${RUNNER_TEMP}/nightly-sota/issue/issue.json"
--bundle "${RUNNER_TEMP}/nightly-sota/implement/bundle.json"
--implementation-receipt "${RUNNER_TEMP}/nightly-sota/implement/cognitum-receipt.json"
--validation "${RUNNER_TEMP}/nightly-sota/validate/validation.json"
--repo-root "${GITHUB_WORKSPACE}"
@@ -4,7 +4,10 @@ on:
pull_request:
paths:
- 'harness/ruview/**'
- '.github/scripts/nightly-sota/**'
- '.github/workflows/nightly-sota-agent.yml'
- '.github/workflows/ruview-harness-flywheel.yml'
- 'docs/adr/ADR-284-bounded-nightly-sota-agent.md'
workflow_dispatch:
inputs:
run_darwin:
@@ -16,8 +19,13 @@ on:
permissions:
contents: read
concurrency:
group: ruview-harness-flywheel-${{ github.ref }}
cancel-in-progress: false
jobs:
verify:
name: Verify contributor harness
runs-on: ubuntu-latest
defaults:
run:
@@ -41,6 +49,7 @@ jobs:
- run: npm pack --dry-run
darwin-proposal:
name: Generate untrusted Darwin proposal
if: github.event_name == 'workflow_dispatch' && inputs.run_darwin
needs: verify
runs-on: ubuntu-latest