Compare commits

...

56 Commits

Author SHA1 Message Date
ruv 450b98345e chore: republish wifi-densepose-vitals 0.3.3 to crates.io
The published 0.3.2 predates the #1449 HeartRateExtractor/
BreathingExtractor fixes (weights=[] silent None, stale-lock
recovery) -- this bumps and republishes so crates.io reflects
current main. No other crate in the documented publish order has
unpublished code changes: wifi-densepose-sensing-server/-cli remain
blocked on the ruview-auth publish=false decision from #1439;
wifi-densepose-calibration (the #1440 fix) was never in the
crates.io publish list.

64 tests passing, 0 failed.

Co-Authored-By: claude-flow <ruv@ruv.net>
2026-07-27 22:51:48 -04:00
rUv 535043731c fix(homecore): review findings from PR #1451 — HAP secret redaction, REST cap, event_type, migration --force (#1452)
HAP accessory signing seed no longer reachable via derived Debug. /api/history/period and /api/logbook no longer break the default (unfiltered) call shape above 32 entities. fire_event's event_type validation relaxed to match real HA's contract. homecore-migrate gained a --force flag for re-running imports. Public v2051 release notes corrected. 110 tests across the 3 touched crates, 0 failed, clippy clean.
2026-07-27 17:01:25 -07:00
rUv 42d56fc1a5 Merge pull request #1451 from ruvnet/feat/homecore-platform-parity
feat(homecore): complete platform runtime capabilities
2026-07-27 12:37:42 -07:00
ruv 5bf820700c fix(homecore-plugins): use patched Wasmtime runtime 2026-07-27 15:23:33 -04:00
ruv 546081e628 docs(homecore): record platform runtime completion 2026-07-27 15:11:03 -04:00
ruv e7c598e64c fix(homecore-server): provision stable HAP identity securely 2026-07-27 15:08:52 -04:00
ruv 42684a7a1e feat(hap): implement authenticated pairing and transport 2026-07-27 15:06:26 -04:00
ruv b41b8c8a82 fix(homecore-api): bound multi-entity history responses 2026-07-27 14:47:04 -04:00
ruv e47d40c5c4 feat(homecore-api): add logbook and integration REST routes 2026-07-27 14:45:35 -04:00
ruv bc690ff309 feat(homecore-api): serve bounded recorder history 2026-07-27 14:43:48 -04:00
ruv fbd5cfa242 feat(homecore-server): wire HAP runtime and registry APIs 2026-07-27 14:41:25 -04:00
ruv 5b5c7f323d fix(homecore): wire plugin lifecycle and preserve MSRV 2026-07-27 14:30:51 -04:00
ruv d8dcccda28 feat(homecore): load signed native and Wasmtime plugins 2026-07-27 14:23:01 -04:00
ruv ec2c64cb62 fix(homecore-server): separate HA and dashboard event routes 2026-07-27 14:19:51 -04:00
ruv c2abe53e92 feat(homecore-hap): add fail-closed network foundation 2026-07-27 14:15:59 -04:00
ruv 0a8e72e762 feat(homecore): complete migration and startup restore 2026-07-27 14:14:56 -04:00
ruv 3136f1305b feat(homecore-api): negotiate modern websocket clients 2026-07-27 14:09:17 -04:00
ruv 273bd449c8 feat(homecore-api): expose loaded components 2026-07-27 14:07:06 -04:00
ruv ac1fdfb725 feat(homecore): add API compatibility and voice protocols 2026-07-27 13:59:36 -04:00
rUv 581af67fbc fix(homecore): harden runtime and publish truthful capabilities (#1450) 2026-07-27 10:41:43 -07:00
rUv 13015c9d36 fix(vitals): HeartRateExtractor weights=[] silent None + BreathingExtractor stale-lock recovery (#1422, #1423) (#1449)
Fixes #1422, Fixes #1423. HeartRateExtractor no longer truncates subcarrier count on empty phases; BreathingExtractor resets immediately on first out-of-band rejection instead of passively draining a stale window (recovery 28.6s -> 6.0s). 64 tests passing, clippy clean, zero regressions workspace-wide.
2026-07-27 08:46:58 -07:00
rUv 931a38abdb fix(calibration): PresenceSpecialist reads empty room as present when occ_var < empty_var (#1440) (#1448)
Fixes the calibration-path bug in #1440. Disables the variance channel when occupied-anchor variance doesn't genuinely exceed the empty-room baseline, mirroring the existing mean-shift channel's fallback. 64 tests passing, zero regressions.
2026-07-27 08:46:47 -07:00
rUv 4e720540d8 fix(sensing-server): classification.presence contradicting motion_level (#1442) (#1447)
Fixes #1442. Extracted classify_vitals() so presence is always derived from the label, matching the convention already used elsewhere in the codebase. 4 new regression tests, 707 tests passing, zero regressions.
2026-07-27 08:46:36 -07:00
rUv 2cc378c12f chore: version-bump and republish 10 of 12 documented crates to crates.io (#1439)
Published: wifi-densepose-core 0.3.2, -vitals 0.3.2, -wifiscan 0.3.2, -hardware 0.3.2, -signal 0.3.6, -nn 0.3.2, -ruvector 0.3.3, -train 0.3.3, -mat 0.3.2, -wasm 0.3.1.

Not published (blocked, needs a decision): wifi-densepose-sensing-server and wifi-densepose-cli both path-depend on ruview-auth, which is publish = false. Version bumps reserved (0.3.5, 0.3.2) but not published.
2026-07-26 15:44:24 -07:00
rUv 2e018f4f19 feat(ruview-unified): Unified RF spatial world model — ADR-273..282 (#1437)
Native frame contract, universal RF encoder, RF-aware Gaussian spatial memory, physics-guided synthetic RF worlds, edge sensing control plane, BLE-CS + factorized pose. All 10 ADRs (273-282) fully implemented and tested (99 tests); ADR-278 (radar inverse rendering) honestly gated with zero code as a future research program.

Deep-reviewed and hardware-tested against a live ESP32-C6 CSI node before merge: fixed a reachable panic, a silent NaN-corruption path, a cross-entity Gaussian conflation bug, and a wrong-center-frequency bug in the WiFi adapter (confirmed live: was misreporting channel 4 as 2437 MHz, now correctly reports 2427 MHz matching the hardware parser exactly). Added a standing hardware-in-the-loop test (examples/esp32_live_hardware_test.rs). Also fixed unrelated pre-existing issues surfaced during validation (wifi-densepose-core clippy warnings, a ruview-auth Windows build break, a sensing-server test flake).

Full review: https://gist.github.com/ruvnet/89795f3c4b8ea166cff5ac35ae4c7651
2026-07-26 14:37:56 -07:00
rUv f783df234e fix release publishing pipelines (#1417)
Make deployment consume the exact published sensing-server image, publish the two Python packages in lock-step, and enforce the production witness gate.
2026-07-24 09:10:52 -07:00
rUv e1e10ad7be fix release publishing pipelines (#1417)
Make deployment consume the exact published sensing-server image, publish the two Python packages in lock-step, and enforce the production witness gate.
2026-07-24 08:57:36 -07:00
Dragan Spiridonov 99700c7851 Merge #1397: Cognitum OAuth resource server, sign-in, and WebSocket auth (ADR-271/272)
Cognitum OAuth for RuView: resource server, sign-in, and WebSocket authentication (ADR-271/272)
2026-07-24 16:01:06 +02:00
Dragan Spiridonov 89babb00a9 Merge remote-tracking branch 'origin/main' into feat/ruview-auth-cognitum-oauth-verifier
# Conflicts:
#	v2/crates/wifi-densepose-sensing-server/src/main.rs
2026-07-24 15:26:15 +02:00
Dragan Spiridonov 56327d0931 decide: browser sessions are read-only permanently; drop the dead step-up client
Decision: browser-side admin is not wanted. `BROWSER_SIGNIN_SCOPE` stays
`sensing:read`, and the escalate-on-demand design sketched while this was open
is not being built. Destructive operations — training, model delete, recording
delete — keep their home in the CLI, where `--admin` is explicit and typed by a
person. Routing them through a browser would mean either asking every user to
consent to delete capability in order to watch a stream, or building a second
consent flow to avoid that.

Consequences, now settled rather than open:

- The UI's admin controls are unreachable from a Cognitum browser session, and
  that is intended. The manual token-paste field is unchanged and still carries
  whatever authority the pasted token has, so nothing that worked before stops
  working.

- REMOVED the client-side step-up redirect from ui/services/api.service.js. It
  caught an RFC 6750 challenge that can never be issued to a browser, and it
  ended in `return new Promise(() => {})` — so if any other 401 had ever grown
  that header, every caller awaiting it would have hung forever with no error
  and no timeout. Dead code with a trap in it is worse than no code.

- KEPT ADMIN_REVERIFY_SECS as a server-side backstop. Fail-closed and free, so
  if the requested scope is ever widened the freshness requirement is already
  in place. Documented at its definition as a backstop specifically so nobody
  reads its passing tests as evidence the control is exercised — the tests
  reach it through a crate-internal seam that mints an admin cookie the real
  flow does not produce.

ADR-271 also stops hedging on the session TTL: chosen is A at one hour. Option B
(server-side refresh-token store) is not built, and the ADR now names the
residual instead of implying it is closed — within one hour a revoked Cognitum
grant still reads sensing data through an existing browser session. There is no
introspection endpoint, so nothing short of B closes that, and one hour is the
size of the hole we accepted.

Adds `the_cookie_max_age_matches_the_session_expiry`, which the previous ADR
revision asked for and nobody had written: Max-Age and the payload's `exp` are
two independent expressions of one lifetime, and drift means either the browser
presents a session we reject or we hold authority the browser discarded.

Verified: workspace 176 suites clean, UI 22, api.service.js parses.

Co-Authored-By: Ruflo & AQE
2026-07-23 14:47:31 +02:00
Dragan Spiridonov 1ed0bc57ef docs(adr): correct a false claim about browser scope; pin the decision in code
A cross-vendor pre-merge sweep found no merge blockers, but did surface an
error in ADR-271 that I wrote — and a coherence gap behind it.

ADR-271 P2 stated that capping the browser session at `sensing:read` was
"considered and rejected, because the dashboard genuinely performs admin
operations". That is wrong. `/oauth/start` (main.rs:9206) already requests
`SENSING_READ` and nothing else, deliberately, with a comment saying so. The
browser session is ALREADY read-only, so the breakage I claimed capping would
cause is simply the current behaviour.

Two consequences, now stated in the ADR instead of left to be discovered:

1. The UI's admin controls do not work from a browser OAuth session.
   `model.service.js:136` issues DELETE /api/v1/models/{id}, which 401s. Admin
   work needs the CLI (`login --admin`) or a pasted admin bearer. A gap in a new
   feature, not a regression — the token-paste path is unchanged.

2. The ADMIN_REVERIFY_SECS step-up control added in the previous commit guards
   a case that cannot currently arise. No browser session holds `sensing:admin`,
   so the freshness branch never fires in production. Its tests pass because the
   crate-internal seam mints an admin cookie the real flow never produces.

That second point is worth being blunt about: it is the same shape as several
defects this branch already fixed — correct code, green tests, unreachable call
site. The difference is that here the guard is deliberately ahead of the need
rather than mistakenly behind it, and saying which one it is matters.

So the constant is now named `BROWSER_SIGNIN_SCOPE` rather than inlined, with
the cost of widening it documented at the definition, and two tests:
`browser_sign_in_stays_read_only_until_someone_decides_otherwise` pins the
value, and `the_authorize_url_actually_carries_that_scope` proves it reaches the
wire — asserting on the constant alone would pass even if `begin` were called
with something else, which is exactly the isolation failure being guarded
against.

The ADR also records the coherent way to add browser-side admin if wanted:
escalate-on-demand via the RFC 6750 challenge, keeping least privilege by
default rather than asking every user to consent to delete capability to watch
a stream. Not bundled here — it needs a scope parameter and a UI affordance.

Sweep verdict: no merge blockers. Two other non-blocking risks it raised are
accurate and unchanged: concurrent JWKS refresh at the stale boundary is not
atomic (a duplicated idempotent GET, already documented as an accepted cost),
and the service worker's SHELL_ASSETS use root-relative paths while the UI
mounts under /ui, so offline shell precaching is incomplete — pre-existing,
unrelated to this branch.

Verified: workspace 176 suites clean.

Co-Authored-By: Ruflo & AQE
2026-07-23 14:31:01 +02:00
Dragan Spiridonov f7cc68bd5c fix(auth): check scope before step-up, so read-only callers are not sent in a circle
Found by an empirical route sweep during the pre-merge pass, not by a test.

The step-up gate fired BEFORE the scope check, so a caller holding only
`sensing:read` who attempted a privileged action with a session older than
ADMIN_REVERIFY_SECS received the RFC 6750 "reauthentication required"
challenge. `api.service.js` acts on that by redirecting through /oauth/start —
and the user returns with exactly the same scopes and is refused again. One
wasted round trip, and a reason for the refusal that was simply untrue: they
were not refused for staleness, they were refused for capability.

Now the scope check comes first, so only a caller who actually HOLDS
`sensing:admin` is ever asked to prove the session is fresh. Not a loop before
(the second refusal is a plain 401 with no challenge) and not a security issue
either way — both paths refuse. It was misleading, and it cost a redirect.

Also confirms the deny-by-default gate does not over-reach. Probed the real
binary with auth off vs on:

  /, /ui/*, /health, /health/{live,ready,metrics,version},
  /oauth/{status,start}    unchanged
  /api/field, /api/v1/*    200 -> 401   (intended)
  /metrics, /favicon.ico   404 -> 401   (never existed; now uniform)

Nothing that returned 2xx before returns 401 now, which is the property that
matters for existing deployments.

Tests: +2. `a_read_only_user_is_not_sent_to_reauthenticate_pointlessly` fails
against the old ordering with the challenge header present; its counterpart
`an_admin_holder_with_a_stale_session_does_get_the_challenge` ensures the fix
does not simply disable the signal.

Verified: workspace 176 suites clean, ruview-auth 62+25+2 --all-features.

Co-Authored-By: Ruflo & AQE
2026-07-23 14:03:54 +02:00
Dragan Spiridonov 89cceaf835 fix(auth): close P1/P2/P3 — JWKS stall, 12h session, cookie shadowing
All three deferred findings from the qe-court round. Each fix is guarded by a
test confirmed to FAIL against the old behaviour.

P1 — JWKS: self-inflicted stall, and a blocking fetch on a tokio worker.

`fetched_at` advances only on SUCCESS, and the only rate limiter sat behind
`if fresh`. So once the TTL elapsed after the last successful fetch, `fresh`
was permanently false, the limiter was never consulted, and EVERY request
performed its own blocking 3s-timeout fetch. A Pi that loses WAN stalled
itself 300s later with no attacker present; an attacker could force the same
state by flooding tokens with an unknown `kid`.

Now: `last_attempt_at` is recorded BEFORE every fetch regardless of outcome,
and gates the stale path too; a stale-but-present key is served rather than
erroring, which is the offline tolerance this module always claimed.
Measured by the new test: 26 outbound fetches before, 1 after.

Kept as TWO independent limiters. Merging them looks tidy and is wrong — a
routine refetch would then suppress the unknown-`kid` path for 30s and delay
pickup of a key rotation inside the TTL. I made that mistake first; two
existing tests caught it.

The blocking call also now runs in `spawn_blocking` at the verify boundary,
matching what `main.rs` already does for the token exchange, where the comment
reads "the same mistake this codebase had to fix in jwks.rs". The hot
verification path had never been given the same treatment. A panicked task
fails closed.

P2 — session lifetime, per decision: 1 hour, plus step-up.

SESSION_TTL_SECS 12h -> 1h, and privileged (`sensing:admin`) actions now
require the user to have authenticated within ADMIN_REVERIFY_SECS (5 min),
tracked by a new `auth_time` claim. Reads ride the full session; only the
routes where a stale session does damage are re-verified, so a dashboard whose
main use is watching a live stream does not re-auth hourly.

`auth_time` is `#[serde(default)]`, so a cookie issued before the field existed
reads as 0 — infinitely stale. Such a session keeps working for reads and
cannot perform privileged actions. Fail-closed and self-healing on next sign-in.

The refusal carries an RFC 6750 `WWW-Authenticate` error code, because the
client's correct response differs from a plain 401: the user IS signed in and
needs to prove it again. `api.service.js` acts on that and redirects through
`/oauth/start` — otherwise a stale-session delete surfaces as a generic
"Request failed" with no hint that signing in again fixes it.

P3 — cookie shadowing.

`read_cookie` returned the FIRST match, and RFC 6265 §5.4 sends longer-`Path`
cookies first. Cookies are not isolated by port or scheme, so any other service
on the host — or a plain-HTTP MITM injecting Set-Cookie — could plant
`ruview_session=<their own validly signed session>; Path=/ui`. The victim sent
both, the attacker's first, and it verified because it genuinely was signed:
silent session takeover, with `/oauth/status` reporting the attacker's account.

The signature was doing its job throughout, which is why "it's signed" never
answered this. `__Host-` would, but requires `Secure`, and RuView is routinely
reached over plain HTTP on a LAN.

So both credential paths now accept only when EXACTLY ONE candidate verifies.
An attacker can still cause a refusal by planting a second valid cookie — a
nuisance — but no longer a takeover. Planting junk changes nothing, so this
does not become a trivial DoS.

Tests: +1 jwks (26-vs-1 fetch amplification), +4 step-up, +4 shadowing, +1
duplicate-name reader. Mutation-verified: reverting the stale-path guard gives
26 fetches; reverting to first-match cookie reads fails
`a_shadowing_cookie_cannot_silently_take_over_the_session`.

Verified: workspace 176 suites clean under CI flags, ruview-auth 62+25+2 with
--all-features, UI 22.

ADR-271: P1/P2/P3 marked RESOLVED with the analysis retained, since it explains
why each fix has the shape it does.

Co-Authored-By: Ruflo & AQE
2026-07-23 13:13:42 +02:00
Dragan Spiridonov 6ce50d5158 test(auth): cover browser sign-in; ADR-271/272 corrections and remediation plans
Browser sign-in was the newest security surface in this PR and had no
executable evidence behind it: browser_session.rs was 534 lines with 13 tests,
every one of which hit a private helper (sign, unsign, cookie, read_cookie,
is_live, has_scope). No test called issue, from_cookie_header, begin,
verifier_for_callback or is_configured, and no test anywhere presented a session
cookie to the gate.

+10 tests in browser_session, +6 in bearer_auth. Three mutants the adversarial
review named, each now verified dead by actually applying the mutation:

  (a) delete the `state` comparison in verifier_for_callback
      -> a_callback_whose_state_does_not_match_is_refused FAILED
      Without it the callback accepts a code from a flow the user never
      started: login CSRF, victim silently lands in the attacker's session.

  (b) `session.is_live().then_some(session)` -> `Some(session)`
      -> an_expired_session_cookie_does_not_authenticate FAILED
      -> an_expired_browser_session_is_refused FAILED
      `is_live` was already unit-tested; nothing asserted the CALLER consults
      it. Same "tested in isolation, call site untested" shape as the earlier
      refresh-never-invoked defect.

  (c) `session.has_scope(required)` -> `true`
      -> a_read_scoped_browser_session_cannot_delete_or_train FAILED
      Without it any browser session could delete models and start training.

Mutant (c) initially appeared to SURVIVE. It did not — there are two
has_scope call sites and the first substitution only hit one. Mutating the
one in `session_or_unauthorized` kills the test. That accident confirmed a
separate finding: the cookie branch inside require_bearer is unreachable when
an Authorization header is present, because the OAuth step returns on both
arms. It fails closed, so it is not a hole, but "try the next credential" is
what the code reads like. Pinned by
a_bad_bearer_beats_a_good_cookie_rather_than_falling_back.

Adds two crate-internal test seams (init_secret_for_tests, test_cookie_value).
test_cookie_value signs through the same path as `issue`, so tests presenting a
cookie exercise real verification rather than a test-only bypass.

ADR-271:
- The "browser cannot obtain an OAuth token" section asserted
  `grep -ril "oauth|cognitum|pkce" ui/` returns nothing. It now returns three
  files, invalidated by commits in this same PR. Marked superseded, original
  retained under a fold, replaced with what actually ships.
- Records the two deferred decisions with designs rather than patches: P1 the
  blocking JWKS fetch on a tokio worker (whose rate limiter is bypassed on
  exactly the stale path that matters, because fetched_at updates only on
  success — so after the TTL every request fetches, and a Pi that loses WAN
  stalls itself with no attacker present); P2 the 12-hour session from a
  15-minute token, with three costed options. Capping the session to
  sensing:read was considered and rejected: the dashboard genuinely issues
  DELETE /api/v1/models/{id}.
- P3 records that dropping `__Host-` costs origin-integrity, not just Secure —
  read_cookie takes the FIRST match and cookies are not port-scoped, so a
  same-host writer can shadow a session. Forgery was never the threat that
  prefix addresses.
- Documents redirect_uri's hardcoded default and the unconsumed CLI credential
  as known-incomplete, per decision to leave both as-is.

ADR-272: corrected a claim that would mislead users into a 401. It stated the
Python client DOES send Authorization: Bearer on the handshake; ws.py passes no
headers at all (zero occurrences of extra_headers or Authorization), so every
published client 401s once auth is enabled. Server-side decision unchanged.

Verified: sensing-server 566 + 179 + 7 + 5 + 8 + 4 + 16 pass under CI flags,
ruview-auth 61 + 25 + 2 with --all-features, auth_wiring 7.

Co-Authored-By: Ruflo & AQE
2026-07-23 12:08:13 +02:00
Dragan Spiridonov c72bbc15dd fix(auth): close /api/field bypass, two fail-opens, and a self-disarming test
Findings from a qe-court adversarial round (4 prosecutors across 2 vendors).
Each was verified against the code before being accepted; the ones below
reproduced, the rest are reported in the PR thread rather than acted on.

FATAL — `/api/field` was reachable with no credential, on both listeners.
The gate protected `/api/v1/*` by prefix. `/api/field` is the REST sibling of
`/ws/field` and serves the same signed FieldEvent stream — live presence, pose,
vitals. `/ws/field` was gated in this PR; its twin one path segment over was
not. Measured with RUVIEW_API_TOKEN set and no credential supplied:

    /api/v1/models  401      (control)
    /ws/field       401      (gated by this PR)
    /api/field      200      on :8080 AND :8765

Fixed by inverting the gate to deny-by-default with an explicit anonymous
allowlist (`/`, `/ui`, `/health`, `/oauth/`). A route added at a new path is
now gated because nobody exposed it, rather than exposed because nobody
protected it — the same inversion already applied to the scope gate.

FATAL — the wiring test disarmed itself exactly when it mattered.
`Server::start` returned an Option that all five tests turned into `return`,
so a server that failed to boot produced "5 passed" with zero assertions run,
and cargo swallows the skip line without --nocapture. The one test that
observes real wiring — the guard against both shipped bypasses — was silent
for any change that breaks startup, including a boot panic in the auth path.
It now panics with the child's stderr.

MAJOR — a malformed client-id list silently disabled the audience check.
An empty allowlist is the opt-out sentinel in verify.rs. `RUVIEW_OAUTH_CLIENT_IDS=","`
is non-empty, passes the guard, then filters to an empty Vec — turning the
audience boundary off with no log and admitting a token minted for any other
Cognitum product. Only a literal `*` may opt out now; anything else that parses
to nothing warns and falls back to the default. Same fail-open shape as the
scope denylist this PR already had to invert.

MAJOR — credentials were world-readable for a window on every refresh.
`fs::write` creates at 0666 & !umask (0644 by default), and both writers
chmodded afterwards. The existing permissions test asserted on the FINAL file
and passed throughout. Affected the CLI refresh token (rotated with reuse
detection — a thief who presents it first takes the session family) and the
browser session secret (the HMAC key for every session; stealing it forges any
account at any scope). Both now create with mode 0600 via OpenOptions.

MAJOR — ui/sw.js cached authenticated API responses.
Closing the /oauth/ leg left the /api/ leg open. `networkFirst` cached every
successful response, keyed by URL alone, purged by nothing at sign-out: sign in
as A, load sensing data, sign out, sign in as B, lose the network, and B is
served A's data with no authorization check. API responses are now network-only
— which is also the correct behaviour for a live sensing dashboard, where
replaying a stale reading can show a room occupied after the person left — plus
a cache purge on sign-out.

CI — 40 of ruview-auth's 87 tests never ran.
The workspace runs --no-default-features, which switches off the `login` and
`pkce` features. Measured: 47 tests vs 87. The whole interactive sign-in path —
credential storage, single-flight refresh, the file lock, the loopback callback
— was green locally and never executed in CI. Added an --all-features step.
(Checked the sensing-server for the same problem and did NOT find it:
bearer_auth's 49 and browser_session's 15 do run under CI flags.)

Tests: +2 wiring tests (one fails against the old gate with
"http port served /api/field to an anonymous caller", passes after), +3 UI
service-worker tests, +3 CLI scope tests, +1 temp-file permission test, +1
client-id parsing test. wifi-densepose-cli/src/auth.rs had zero tests and
builds its own scope string, so the library's least-privilege test said nothing
about what the CLI requests.

Verified: ruview-auth 61+25+2 pass (--all-features), sensing-server bearer_auth
50, browser_session 15, auth_wiring 7, workspace 25 suites clean, UI 22.

Co-Authored-By: Ruflo & AQE
2026-07-23 11:48:59 +02:00
Dragan Spiridonov 9b9754778f fix(ui): service worker cached /oauth/status, freezing browser sign-in
Browser sign-in worked, but the settings panel kept offering "Sign in with
Cognitum" afterwards; only a hard reload showed the true state. The server was
correct throughout.

Root cause: `ui/sw.js` routed cache-first as its CATCH-ALL for every path
outside `/api/` and `/health/`. `/oauth/status` therefore had its first
(signed-out) response stored in the Cache API and replayed to the page forever.
The Cache API is not the HTTP cache and ignores `Cache-Control` entirely, so the
`no-store, no-cache, must-revalidate` the server already sends could not prevent
it. A hard reload bypasses the service worker, which is why that alone appeared
to fix it, and why signing out re-poisoned the entry.

Fixes, in order of blast radius:

- `/oauth/` is never handled by the worker. Not network-first — that still
  writes a copy, which would be replayed the moment the server is briefly
  unreachable, silently reinstating a stale sign-in state.
- Cache-first is now an ALLOWLIST (navigations and static asset extensions)
  rather than the catch-all. This is the underlying defect: any endpoint added
  outside `/api/` was frozen on its first response. Unrecognised paths now go
  to the network untouched.
- `CACHE_NAME` bumped to `ruview-v2`, so `activate` evicts the poisoned
  `ruview-v1` from browsers that already ran the old worker. Verified: the
  cache list went from `[ruview-v1]` to `[ruview-v2]` on update.
- Shell lookups use `ignoreSearch` and store a search-less key, so the
  `?signed_in=<ms>` the callback redirects to does not mint a fresh, never-hit
  cache entry per sign-in.

Also: re-check sign-in state on `pageshow` (bfcache restore, where no script
re-runs) and on `visibilitychange` (signed in or out in another tab). Opening
the panel alone is not sufficient — it may already be open.

Verification, in a real browser driven end-to-end:
- Reproduced with a valid session cookie: server returned `signed_in: true` to
  curl while the page's own fetch got a cached `signed_in: false`, and the
  request never appeared in the server log at all.
- Confirmed the cached entry existed: `caches.open('ruview-v1').match(
  '/oauth/status')` returned the signed-out body.
- After the fix, on a NORMAL (not hard) reload the panel reads
  "Signed in as <account> - sensing:read" with Sign out shown.

Tests: `ui/sw.test.mjs` loads the real `sw.js` with stubbed worker globals and
asserts the routing decision per path. 4 of its 10 tests fail against the
pre-fix worker and pass after; the other 6 pin pre-existing guards (non-GET,
websocket upgrade, cross-origin, API paths, static assets, navigation) and pass
in both, so they track behaviour rather than the rewrite.

CI: adds a `ui-tests` job. Nothing ran the UI JavaScript before, so both this
suite and the ADR-272 ws-ticket suite would have rotted unexecuted — which is
the same blind spot that let this defect ship.

Removes the temporary `/oauth/status` cookie logging and the panel console
diagnostic added while tracking this down.

Co-Authored-By: Ruflo & AQE
2026-07-23 11:20:13 +02:00
Dragan Spiridonov 43737941cb fix(auth): two Set-Cookie headers were collapsing into one; clear the spent transaction
Two bugs, one of which I introduced while fixing the other and caught only by
checking the actual response.

1. THE SPENT TRANSACTION COOKIE WAS NEVER CLEARED ON SUCCESS.
   `clear_transaction` was only used on error paths, so after a successful
   sign-in `ruview_oauth_txn` lingered for its full 10-minute TTL and every
   subsequent request carried a dead cookie. Visible in the logs as
   `names=ruview_oauth_txn,ruview_session`. Now cleared alongside issuing the
   session, and on logout too.

2. THE FIX FOR (1) BROKE SIGN-IN, AND THE TEST CAUGHT IT.
   Axum's array-of-tuples response form REPLACES same-name headers rather than
   appending. Adding a second `Set-Cookie` silently overwrote the first, so the
   logout response emitted only the transaction clear and — had this shipped —
   the CALLBACK would have emitted only the transaction clear too, dropping the
   session cookie and making a successful OAuth round-trip a no-op.

   Caught by looking at the real response headers rather than trusting the
   change: `curl -D-` showed one Set-Cookie where there should have been two.

   Now uses `axum::response::AppendHeaders`. Both cookies verified present.

   Two tests pin this: one DOCUMENTS the footgun (the array form collapses two
   Set-Cookie headers into one) and one asserts AppendHeaders emits both. The
   first exists so the next person to reach for the tidier-looking array form
   finds out here instead of in production.

Also adds a cache-busting query to both redirect targets
(`/ui/?signed_in=<ms>` and `?signed_out=<ms>`). Landing on the same URL let the
browser restore the page from the back/forward cache with a stale panel, which
is why signing in appeared not to work until a hard refresh — the session was
established correctly every time, the page simply was not re-fetched.

Tests: 549 sensing-server lib.

Co-Authored-By: Ruflo & AQE
2026-07-23 10:46:43 +02:00
Dragan Spiridonov 4a704acc02 fix(ui): actually call refreshSignInPanel — VERIFIED IN A REAL BROWSER
`refreshSignInPanel` was exported and never invoked. The Cognitum Account panel
would have rendered "Checking…" forever and the Sign in button would never have
reflected a completed sign-in.

That is the FOURTH defect of this exact shape today — implemented, unit-tested,
and never called. The others: the refresh path no command invoked, scope
resolved only at write time, and a WebSocket listener with no auth layer. The
pattern is consistent: the logic was fine, the caller was missing, and a green
suite could not see it.

Now refreshed on every panel open — not once at construction — because the
session may have been established in another tab or expired since page load.
Buttons are also bound at construction so a click works before the first status
fetch resolves.

VERIFIED END TO END IN A REAL BROWSER (Chrome, http://127.0.0.1:8099/ui/), which
is the gap that has been open all session:

  1. panel showed "This server requires sign-in." + Sign in with Cognitum
  2. clicked -> auth.cognitum.one -> approved
  3. server: "browser sign-in complete" sub=ed3efb51-…
  4. DevTools: ruview_session cookie on 127.0.0.1, Path=/, HttpOnly, SameSite=Lax
  5. after reload the panel reads:
       "Signed in as UyShaIh1B1gL7km9Xli9kCDSdRT2 — sensing:read"  + Sign out

So the browser half of gate G-3 is now proven, not argued: PKCE, consent, the
server-side code exchange, ES256 verification with audience and scope checks,
the signed session cookie, and the UI reading it back.

Diagnosis note worth keeping: the first attempt appeared to fail because the
browser tab had been open since BEFORE the fix — an already-loaded ES module
stays in memory regardless of `Cache-Control: no-store`. The cookie had been
stored correctly the whole time; the panel simply never re-queried. The server
log showed zero /oauth/status probes, which is what distinguished "fetch never
fired" from "fetch got the wrong answer". Temporary cookie diagnostics added to
find that are removed here.

Tests: 547 sensing-server lib, unchanged.

Co-Authored-By: Ruflo & AQE
2026-07-23 10:41:11 +02:00
Dragan Spiridonov 9299a3b137 feat(ui): Sign in with Cognitum, zero-config session secret, executed JS tests
Three loose ends from the browser sign-in work.

--- 1. The last mile: a button ---

The server endpoints existed but nothing in ui/ linked to them, so the feature
was unreachable. QuickSettings gains a "Cognitum Account" panel that renders
from `GET /oauth/status` and offers Sign in / Sign out.

`/oauth/status` is a new endpoint and is deliberately UNGATED — a signed-OUT
browser cannot ask a gated API whether sign-in is available. It returns only
capability flags and, when a session exists, who it belongs to. Never a
credential.

The panel distinguishes four states rather than showing a button that might
404: signed in; sign-in available; auth on but OAuth off (points at the
existing static-token panel); no auth required. A 404 from /oauth/status means
a server predating this work and says so plainly.

Sign-in is a full-page navigation, not fetch(): the server replies 302 and the
browser must follow it carrying the transaction cookie. An XHR would follow the
redirect invisibly and land nowhere.

--- 2. RUVIEW_SESSION_SECRET no longer required ---

Previously `/oauth/start` returned 503 unless an operator invented a secret —
a footgun, since they set RUVIEW_OAUTH_ISSUER, expect sign-in, and get a 503
naming an env var they have never heard of.

Now resolved in order: env var, then `<data_dir>/session-secret`, then generate
one and persist it 0600 (temp file, chmod before rename — the discipline used
for the CLI's credentials). Persisted rather than in-memory so a restart does
not silently sign everyone out.

The env var still wins, which is what a multi-instance deployment needs: several
servers must share a secret or a session issued by one is rejected by the next.
If the file cannot be written we log the reason and continue with an in-memory
secret rather than refusing sign-in outright.

Verified with NO configuration at all: secret generated, file mode 0600,
/oauth/start returns 302, /oauth/status reports browser_signin: true.

--- 3. The JavaScript is now executed by tests ---

`ui/services/ws-ticket.test.mjs`, 9 tests, node:test built-in — no new
dependency and no package.json needed. Run: `node --test ui/services/`.

Covers: no token means no fetch and an unchanged URL; the bearer reaches the
Authorization header and NEVER the URL; `?` vs `&` when a query already exists;
ticket URL-encoding; 404 treated as a pre-ADR-272 server (the property that
lets one UI work against old and new servers, and therefore lets the legacy
escape hatch be removed later); 503 and network failure swallowed rather than
breaking the connect path; and that a fresh ticket is minted per call, since
tickets are single-use and caching one fails on the second reconnect.

STATED PRECISELY, because the distinction matters: this EXECUTES the module in
Node with stubbed fetch/localStorage. It is more than the `node --check` it
replaces and less than a browser — no real WebSocket upgrade, no real cookies,
no page wiring. "The UI JavaScript has never been run" is no longer true of
this module. "Browser-tested" still is not.

Tests: 547 sensing-server lib, 5 wiring integration, 87 ruview-auth, 9 JS.

Co-Authored-By: Ruflo & AQE
2026-07-23 10:07:49 +02:00
Dragan Spiridonov 347698b67c feat(auth): browser sign-in — /oauth/start, /oauth/callback, session cookie
Closes the gap adversarial review found: `wifi-densepose login` writes
~/.ruview/credentials.json, which a BROWSER CANNOT READ. The UI therefore had no
way to obtain a Cognitum token at all, and the WebSocket ticket mechanism
ADR-272 built "for browsers" was only exercisable with the legacy static shared
secret OAuth was meant to replace. The ADRs described a browser story that did
not exist.

Ported from cognitum-one/freetokens (src/auth/oauth.ts, live at
freetokens.cognitum.one), whose shape is not the obvious one and is the whole
point:

  THE BROWSER NEVER HOLDS AN OAUTH TOKEN.

The server generates the PKCE verifier and state, keeps them in an HMAC-signed
cookie, performs the code exchange itself, verifies the token, and issues its
OWN session cookie carrying an assertion — subject, account, scope, expiry — not
a credential. So the access token cannot be read by an XSS, cannot sit in
localStorage, and cannot leak through a URL. A stolen session cookie is useless
against Cognitum or any sibling service.

  GET /oauth/start    -> 302 to auth.cognitum.one + signed transaction cookie
  GET /oauth/callback -> constant-time state check, exchange, verify, session
  GET /oauth/logout   -> clears the local session (not the Cognitum session)

Verified against the running binary: /oauth/start returns the same 302 +
HttpOnly/SameSite=Lax/Max-Age=600 shape freetokens does live; a forged `state`
is refused 400; a forged session cookie is refused 401.

DELIBERATE DEVIATION from freetokens: no `__Host-` cookie prefix. That prefix
REQUIRES `Secure`, and RuView is routinely reached at http://localhost or over
plain HTTP on a LAN, where such a cookie is never sent and sign-in would fail
silently. `Secure` is set only when the request actually arrived over TLS
(direct or via x-forwarded-proto). Every other attribute matches; the HMAC is
what protects the value.

Other decisions worth stating:
- The callback verifies through the SAME `verify_access_token` every other
  request uses — signature, audience (client_id), typ, expiry, scope. A sign-in
  path must not be a softer path.
- The session cookie is checked LAST in the middleware, after bearer and ticket:
  it is the weakest-bound credential, so a presented bearer should win.
- A browser session requests `sensing:read` only. Admin work goes through the
  CLI's explicit `--admin`.
- The token exchange runs in `spawn_blocking` — `ureq` is blocking, and parking
  an async worker is the mistake this codebase just had to fix in jwks.rs.
- `/oauth/*` sits outside `/api/v1/*` on purpose: gating the routes you use to
  obtain a credential would deadlock.

PKCE moved out from behind the `login` feature into its own light `pkce` feature
(rand + sha2 + base64, no HTTP stack), so the server can build an authorize URL
without pulling in the client-side login machinery. `login` now implies `pkce`.

Tests: 13 new browser_session unit tests — signature round-trip, tampered
payload, wrong secret, malformed cookie values, HttpOnly/SameSite/Secure
attributes, exact scope matching with no implied escalation, a cookie name that
merely ends with the target not matching, multi-scope URL encoding, and the
core property that a session cookie never contains the access token.
Totals: 547 sensing-server lib + 5 wiring integration, 87 ruview-auth.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:57:15 +02:00
Dragan Spiridonov 705a167ffe test(auth): boot the real binary and probe BOTH listeners (the wiring gap)
Two authentication bypasses shipped in this PR and 526 green unit tests could
not see either, because every auth test in the crate builds its OWN Router with
a hand-picked subset of routes. A synthetic router cannot observe how the real
one is assembled — and both defects were assembly, not logic:

  1. the dedicated --ws-port listener had no `require_bearer` at all
  2. `/ws/field` was .merge()d AFTER the auth layer, which in axum exempts it

This spawns the actual `sensing-server` binary (via CARGO_BIN_EXE) on ephemeral
ports and speaks raw HTTP/1.1 and real WebSocket upgrades to BOTH listeners. No
mocks, no synthetic router, no in-process shortcuts.

PROVEN TO HAVE TEETH, which matters more than it passing: with main.rs reverted
to eb68e07a (the vulnerable commit), the suite fails —

  assertion `left != right` failed: http port ACCEPTED an unauthenticated
  upgrade to /ws/field — this is the bypass that shipped twice

and passes again once restored. A regression test that has never been shown to
fail is a comment.

Five cases:
- with auth ON, NO listener accepts an unauthenticated upgrade — all four WS
  paths x both ports, with a REST 401 control first so the WS assertions cannot
  pass for the wrong reason
- a bearer on the upgrade is accepted on both listeners (native clients are not
  browser-constrained and must not need the ticket round-trip)
- with auth OFF both listeners stay open — the compatibility promise
- the legacy escape hatch opens WebSockets WITHOUT weakening REST — scoped, or
  it is a bypass wearing a migration label
- /health stays anonymous on both listeners — a documented exemption, pinned so
  it stays a decision rather than an accident

The child inherits no RUVIEW_* variables (env_remove), or a developer's local
export would silently change what the test proves. Ports are reserved by
binding :0 and releasing, so a collision surfaces as a boot failure rather than
a false pass.

Tests: 5 new integration, 534 lib, 87 ruview-auth.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:50:43 +02:00
Dragan Spiridonov b09625ece7 fix(auth): close the four residual findings from the adversarial review
(a) Test fixture still invented an `iss` claim.
    `bearer_auth.rs`'s `token_with_scope` added `"iss": ISSUER` to its tokens.
    Harmless today because the verifier ignores `iss` — but it is the same
    fixture-invents-reality pattern that hid the original `iss` bug for a day,
    sitting in the second-largest auth test suite. Removed, with a pointer to
    ruview-auth's regression test.

(b) Cross-process refresh race -> session revocation.
    The single-flight guarantee was per-PROCESS. Every CLI invocation is a new
    process with its own Session and mutex over one shared credential file, so
    two commands run close together inside the 60s refresh window would each
    present the same rotating refresh token — and the second is replay, which
    identity answers by revoking the whole session family. The user gets logged
    out for running two commands at once.

    Now guarded by an advisory file lock, taken NON-BLOCKING. A busy lock means
    another process is already refreshing, so we wait and re-read its result
    rather than race it (20 x 150ms, then proceed anyway — the lock is advisory,
    not a correctness barrier, and a dead holder must not wedge us). Blocking on
    the lock would have parked the async executor, which is the exact mistake
    just fixed in jwks.rs.

    Unix only. On other platforms it is a documented no-op — a lock that does
    nothing while claiming to protect is worse than none.

(c) One principal could exhaust the global ticket pool.
    The 512 cap was global with no per-caller quota, so a single authenticated
    `sensing:read` client looping on POST /api/v1/ws-ticket could hold every
    slot for 30s and 503 everyone else — denial of service by the
    lowest-privilege account the product issues. Added a 16-ticket
    per-principal cap; a page needs a handful. Test asserts a noisy user hits
    its own cap while a second user is still served and the global pool is
    never exhausted.

(d) Debug impls printed live credentials.
    `AuthState` derived Debug over the raw RUVIEW_API_TOKEN and
    `StoredCredentials` over both OAuth tokens. Not leaking today — I checked
    every call site — but this PR had already hand-written redacting Debug for
    `OAuthState` and `TicketStore` for exactly this reason, and the two types
    actually holding secrets were the ones that missed out. Both now redact.

Tests: 87 ruview-auth (2 new: lock exclusivity + non-blocking, Debug
redaction), 534 sensing-server (1 new: per-principal quota).

Co-Authored-By: Ruflo & AQE
2026-07-23 09:48:09 +02:00
Dragan Spiridonov 0547fd7344 fix(auth): enforce client_id as the audience — Cognitum's stand-in for aud
Found by reading cognitum-one/freetokens, a live sibling service whose browser
OAuth landed while this PR was open. Its integration contract states the
platform rule outright:

  "Cognitum access tokens intentionally use custom `client_id` rather than a
   registered JWT `aud` claim."  -- freetokens docs/AUTH_INTEGRATION.md

and `src/auth/oauth.ts` enforces it on every sign-in:

  payload.client_id !== config.OAUTH_CLIENT_ID  -> reject

RuView did not. An earlier revision here removed the `client_id` check and kept
it only for logging, reasoning that clients borrow one another's registrations
(musica shipped as `meta-proxy` while its own was pending) and that scope alone
must therefore carry the boundary. That reasoned from a TRANSITIONAL state:
RuView has its own registered client (identity migration 0017), and the platform
does have an audience mechanism — it is simply spelled `client_id`.

Consequence of the old behaviour: a Cognitum access token minted for ANY product
— meta-proxy, musica, metaharness, freetokens — was accepted by a RuView server
provided it carried a sensing scope. Scope was the only thing standing between
another product's token and this one. Now there are two boundaries, audience and
capability, which is what the platform intends.

- `VerifierConfig.allowed_client_ids`; empty = accept any (explicit opt-out).
- `RUVIEW_OAUTH_CLIENT_IDS` env, default `ruview`, `*` to disable with a loud
  warning naming what is being given up. Comma-separated for the migration case
  where a borrowed registration must be accepted alongside our own.
- New `VerifyError::WrongAudience`, checked BEFORE scope, so the failure names
  the real reason rather than blaming the scope.

The existing cross-product test now asserts `WrongAudience` rather than
`MissingScope` — the token is refused for the stronger reason. Three new tests:
a correctly-scoped token from another product is still refused; the empty-list
opt-out accepts anything (pinned so it stays deliberate); multiple allowed
clients work.

This also corrects the module docs and ADR-271, which claimed "scope is the ONLY
capability boundary" — true of the code as written, but not of the platform.

Tests: 85 ruview-auth, 533 sensing-server.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:31:24 +02:00
Dragan Spiridonov f67a880a1a docs: correct three doc-vs-code contradictions found by adversarial review
All three are my own drift — the exact failure this session criticised in other
repos' ADRs and then reproduced.

1. ADR-271 §2 still listed "`iss` matches the configured issuer verbatim" in the
   accept-rule. That rule was removed from the code in 12635a85 because Cognitum
   issues no `iss` and it rejected every real token. The ADR's own "Facts about
   the tokens" section 30 lines above already said tokens carry no `iss` — the
   document contradicted itself for a day. Also removed a claim that tests cover
   "issuer mismatch including a trailing-slash-only difference"; that test was
   deleted with the rule and no longer exists.

2. `ruview-auth/src/lib.rs:59` said "**No login flow.** ... this crate only
   verifies" — 15 lines above `pub mod login;`. Now states the accurate thing:
   the login flow is behind the non-default `login` feature, so a verifying
   server never compiles it.

3. ADR-271's scope table still described the old prefix-denylist admin set.
   Replaced with the fail-closed rule that actually ships, including why:
   `/api/v1/adaptive/train` was reachable with `sensing:read`.

Also records a KNOWN INCOMPLETE that the ADRs previously implied was done: the
browser cannot obtain an OAuth token at all. `wifi-densepose login` writes
~/.ruview/credentials.json, which a browser cannot read; the UI reads
localStorage['ruview-api-token'], populated only by the manual-paste
QuickSettings panel. `grep -ril "oauth|cognitum|pkce" ui/` returns nothing.

So the WebSocket ticket mechanism ADR-272 introduces "for browsers" is today
only exercisable with the legacy static shared secret OAuth was meant to
replace. Server-side gating is correct and complete; the browser half of the
story these ADRs tell is not built. Recorded rather than left implied.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:20:31 +02:00
Dragan Spiridonov 7a05417493 fix(auth): scope gate was fail-OPEN; JWKS lock held across a blocking fetch
Two charges from an adversarial multi-vendor review (qe-court). Both verified
in source before fixing.

--- 1. AUTHORIZATION BYPASS: POST /api/v1/adaptive/train reachable with read ---

`required_scope_for` enumerated admin routes by prefix (`/api/v1/train/`, plus
DELETE on models/recording) and let EVERYTHING ELSE fall through to
`sensing:read`.

`POST /api/v1/adaptive/train` (main.rs:8112 -> handler main.rs:5028) calls
`adaptive_classifier::train_from_recordings()`, writes the model to disk with
`model.save()`, and swaps `state.adaptive_model` used by the live inference
pipeline. It does not start with `/api/v1/train/`, so it landed on
`sensing:read` — the scope `wifi-densepose login` requests BY DEFAULT. Exactly
the blast radius the read/admin split exists to prevent, reachable by the
lowest-privilege token the product issues.

Fixed by inverting the polarity, which is the real defect — a denylist for a
security gate keeps missing routes as routes keep being added:

  GET/HEAD/OPTIONS            -> sensing:read
  any other method            -> sensing:admin
  ...unless the exact path is in READ_SAFE_MUTATIONS (an explicit allowlist of
     mutations that change runtime state but destroy nothing: ws-ticket,
     model load/unload/activate, calibration start/stop, recording start/stop,
     vendor event ingest)

A mutating route added tomorrow is now admin-gated by default. Pinned by
`an_unknown_mutating_route_defaults_to_admin`. `POST /api/v1/ws-ticket` is
allowlisted deliberately and has its own test: were it admin, the read scope
could never open a stream from a browser at all.

Enumerated all 17 mutating /api/v1 routes to build the allowlist rather than
guessing. `POST /api/v1/config/ground-truth` now requires admin — a deliberate
tightening, it writes config.

--- 2. DoS: blocking JWKS fetch under std::sync::Mutex in async middleware ---

Filed INDEPENDENTLY by two prosecutors, which is why it gets fixed rather than
argued about.

`decoding_key_for` locked a `std::sync::Mutex` and held it across a blocking
`ureq` call (3s timeout, longer on a dead link), invoked from inside the async
`require_bearer`. `Mutex::lock()` in an async fn is a blocking syscall, not a
yield point — so one slow JWKS fetch blocked EVERY concurrent request on that
mutex, including ones carrying already-cached valid tokens, and parked the
tokio workers running them. On Pi-class hardware with few workers that stalls
the whole server. It fires on the routine 300s TTL rollover whenever the link
is degraded — the exact offline case the module exists to tolerate. Reachable
by anyone able to send a syntactically valid ES256 header with an unknown kid,
since kid lookup precedes signature verification.

Now three phases: read under the lock, RELEASE, network, re-take only to
install. Cost is a possible duplicated idempotent GET during a rollover, which
is strictly better than serialising every request behind one socket. The
codebase already uses spawn_blocking for outbound I/O elsewhere
(main.rs:2490, 5272); moving verification fully onto spawn_blocking remains a
follow-up — this removes the amplification, not every blocking millisecond.

Tests: 533 sensing-server (7 new scope-polarity), 82 ruview-auth.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:18:33 +02:00
Dragan Spiridonov 3347e258e6 fix(ws): gate the dedicated WS port and the merged field routes (ADR-272 was incomplete)
The previous commit claimed WebSocket upgrades were gated. They were not. Found
by an adversarial cross-vendor review, reproduced, and fixed here.

MEASURED BEFORE (auth ON via RUVIEW_API_TOKEN, no credential presented):

  HTTP port :3991  /ws/sensing  -> 401   (what the previous fix covered)
  HTTP port :3991  /ws/field    -> 101   HOLE
  WS   port :3990  /ws/sensing  -> 101   HOLE
  WS   port :3990  /ws/field    -> 101   HOLE
  control   :3991  /api/v1/models -> 401

Two independent defects, both from the same root cause — routes registered
AFTER an axum `.layer()` are silently exempt from it:

1. The dedicated WebSocket server on `--ws-port` was built with ONLY
   `host_validation::require_allowed_host`. `require_bearer` was never applied
   to it at all. My earlier verification only ever probed the HTTP port and I
   generalised from it.

   This is the worse of the two, because it is the port the UI actually uses:
   `ui/services/sensing.service.js` maps HTTP 8080 -> WS 8765. So the previous
   fix protected a path the browser never takes, while the path it does take
   stayed open.

2. On the HTTP router, `/ws/field` (ADR-262) was `.merge()`d AFTER the
   `require_bearer` layer, so it bypassed authentication entirely. The auth
   layer is now applied after the merge, and the comment says why the ordering
   is load-bearing.

MEASURED AFTER, same conditions:

  :3989 /ws/sensing -> 401 · :3989 /ws/field -> 401
  :3988 /ws/sensing -> 401 · :3988 /ws/field -> 401
  with bearer on the upgrade: 101 on both ports
  ticket minted at POST /api/v1/ws-ticket on the HTTP port and redeemed on the
    WS port: 101  (AuthState shares its TicketStore via Arc)
  auth OFF: 101 — unchanged, no regression

526 sensing-server tests still pass.

TEST GAP, stated rather than papered over: no automated test covers this. The
defect is in router WIRING in main.rs, not in logic a unit test reaches — both
holes were invisible to 526 green tests and to the ws_gate_tests suite, which
builds its own Router and therefore cannot see how the real one is assembled.
Catching this class needs an integration test that boots the binary and probes
BOTH ports; that is the honest follow-up.

Co-Authored-By: Ruflo & AQE
2026-07-23 09:14:07 +02:00
Dragan Spiridonov eb68e07a2c feat(ui): fetch WebSocket tickets, prefix-match WS paths, and write ADR-272
Completes ADR-272. Three parts.

1. PREFIX MATCHING, not an allowlist. Anything under `/ws/` is a WebSocket path,
   plus `/api/v1/stream/pose` which lives outside it. An allowlist means every
   WebSocket route added later ships ungated until someone remembers to extend
   it — the same bug reintroduced on a delay. Not hypothetical:
   `/ws/train/progress` (ADR-186, arriving with PR #1387) is ALREADY referenced
   by ui/services/training.service.js and would have shipped unauthenticated.
   Pinned by a test that asserts it is gated before it exists.

2. UI wiring. A shared `withWsTicket()` helper mints a ticket immediately before
   each connection attempt — never cached, because a ticket is single-use and
   expires in seconds, so reusing one across reconnects fails on the second
   attempt. Wired into the three sites that open gated sockets:
   sensing.service.js, websocket-client.js, observatory/js/main.js.

   It degrades in both directions on purpose: no stored token means auth is off
   and no ticket is needed; a 404 from /api/v1/ws-ticket means a server
   predating this ADR, which still exempts WebSockets, so connecting without a
   ticket is correct there. The same UI therefore works against old and new
   servers, which is what makes the escape hatch removable later rather than
   permanent.

   The long-lived bearer token is still never put in a URL — only the ticket is.

3. ADR-272 itself. Previously cited in five places without existing; the same
   dangling-reference mistake made with ADR-271 earlier today, so it is written
   before this lands rather than after someone notices.

   It records the measured before/after, why a credential in a URL is
   acceptable here specifically (single use, seconds-long, not the credential),
   why the escape hatch exists and why it is deliberately uncomfortable, and
   what is deliberately NOT done — including that /health/metrics stays ungated,
   with the caveat that this should be revisited if metrics ever carry
   occupancy-derived values, since that would make them sensing data wearing an
   ops label.

Tests: 526 sensing-server (4 new path-matching), 82 ruview-auth. JS
syntax-checked with `node --check`; there is no UI test suite to extend.

Co-Authored-By: Ruflo & AQE
2026-07-22 19:15:20 +02:00
Dragan Spiridonov 6300b1cbd2 feat(ws): gate WebSocket upgrades behind bearer-or-ticket (ADR-272)
Closes the hole measured in 7d6d6694. Before, with RUVIEW_API_TOKEN set, a real
handshake carrying no credential:

  /ws/sensing 101 · /ws/introspection 101 · /api/v1/stream/pose 101
  (/api/v1/models correctly 401)

After, same server, same handshake:

  /ws/sensing 401 · /ws/introspection 401 · /api/v1/stream/pose 401
  bearer on the upgrade                         -> 101
  POST /api/v1/ws-ticket then ?ticket=<value>   -> 101
  the same ticket replayed                      -> 401
  a bogus ticket                                -> 401
  POST /api/v1/ws-ticket unauthenticated        -> 401

Two ways in, matching what each client can actually do:

- Native clients (Python, Rust CLI, TS MCP) send a normal Authorization header
  on the upgrade. They were never browser-constrained; forcing them through a
  ticket would add a round-trip and a second credential path for nothing.
- Browsers cannot set that header, so they exchange their credential at
  POST /api/v1/ws-ticket — an ordinary request, where they can — for a 30s
  single-use ticket passed as ?ticket=.

A ticket inherits the issuing principal's scopes, so a sensing:read session
cannot mint one that outranks itself, and it is not a REST credential: pinned
by a test that `?ticket=` on /api/v1/models is still 401.

ESCAPE HATCH (RUVIEW_WS_LEGACY_UNAUTHENTICATED=1) restores the old behaviour
for deployments that cannot update server and UI in lockstep. It is a migration
aid, not a supported configuration, and it says so on every boot in a warning
that names the actual exposure ("the live sensing stream — presence, pose and
vital signs — is readable by anyone who can reach this port"). Its blast radius
is exactly the WebSocket paths: a test pins that it does not weaken REST.

The flag is read once at construction, so a mid-flight environment change
cannot silently open these paths on a running server.

SUPERSEDES the PR #1313 test `enabled_exempts_pose_stream_websocket`, which
asserted the old exemption. Its reasoning about browsers was correct; the
conclusion was not. Renamed and inverted rather than deleted, with the history
in the doc comment — and the half that still matters (the WebSocket rule must
not leak to other /api/v1/* paths) is kept.

Deployments with auth OFF see no change at all — pinned by a test.

Tests: 522 passed in the sensing server (9 new WS-gating, 12 ticket-store).

STILL OUTSTANDING for ADR-272: the browser UI does not yet fetch a ticket, so
it needs the escape hatch until ui/services/api.service.js is updated. That is
the next commit, not a permanent state.

Co-Authored-By: Ruflo & AQE
2026-07-22 19:04:44 +02:00
Dragan Spiridonov 7d6d66941a feat(ws): single-use WebSocket ticket store (ADR-272 groundwork)
Audit finding this exists to close — verified empirically, not inferred. With
RUVIEW_API_TOKEN set (operator believes auth is ON), a real WebSocket handshake
carrying NO credential:

  /ws/sensing         -> 101 Switching Protocols
  /ws/introspection   -> 101 Switching Protocols
  /api/v1/stream/pose -> 101 Switching Protocols
  /api/v1/models      -> 401   (control: REST is correctly gated)

The REST control plane is locked while the DATA plane — live presence, pose and
vitals — is open to anyone who can reach the port. Two of those paths sit
outside PROTECTED_PREFIX entirely; the third is the documented EXEMPT_PATHS
entry. The exemption was reasonable when added (a browser cannot set
Authorization on an upgrade) but its blast radius is larger than it looks, and
phase 3 sharpened the contrast by making REST genuinely strong.

(To be precise about what was proven: the handshake is accepted. A payload
frame was not captured in that window, so this is "the connection is
established without a credential", not "data was read".)

This commit adds only the store; wiring it into the middleware is a breaking
change for browser clients and lands with the UI update.

Design notes:
- Single use. `consume` REMOVES the entry, so a replay of the same URL fails
  even inside the TTL. This is what makes a credential-in-a-query tolerable:
  by the time it reaches an access log or a Referer header, it is spent.
- 30-second TTL. Long enough for a page to open a socket; too short to harvest.
- It is not the credential. It authorizes one WebSocket. It cannot be replayed
  against /api/v1/*, cannot be refreshed, and carries no reusable identity.
- The grant captures the ISSUING principal's scopes, so a WebSocket inherits
  exactly the authority of the credential that asked for it — a sensing:read
  session cannot mint a ticket that outranks itself.
- Capped at 512 outstanding, self-healing as tickets expire, so an
  authenticated but misbehaving caller cannot grow the map without bound.
- In-memory because that is correct, not merely convenient: a ticket surviving
  a restart would outlive the server that vouched for it.

Native clients (Python, Rust CLI, TS MCP) are NOT browsers and will send a
normal Authorization header on the upgrade instead — tickets would add a
round-trip and a second credential path for no benefit.

12 tests: single-use enforced, replay refused, expiry refused AND pruned,
unknown ticket refused, 256-bit unpredictability, grant carries issuer scopes,
cap enforced and self-healing, and query parsing including `?myticket=x` not
being read as `?ticket=x`.

Co-Authored-By: Ruflo & AQE
2026-07-22 18:41:34 +02:00
Dragan Spiridonov 6d3fb88677 fix(cli): resolve scope from the token on read, and add whoami --refresh
Two gaps that only appear with a credential file written by an earlier build —
i.e. exactly the case a fresh-login test never exercises.

1. `whoami` printed "Scope: (not reported)" for an existing file. The previous
   commit resolved scope from the token claim at WRITE time only, so files
   written before it stayed blank forever. `effective_scope()` now falls back at
   READ time, which fixes existing files and any client that stored only what
   the token response carried. The token is authoritative either way; the stored
   field is a convenience copy.

2. `whoami` said the token "will refresh on next use" — a promise nothing kept,
   because no command called `ensure_fresh`. The refresh path was implemented
   and unit-tested but never actually run. `--refresh` exercises it, and goes
   through `Session::ensure_fresh` rather than reimplementing a second, subtly
   different refresh, so it inherits the single-flight guarantee and the
   persist-before-return ordering.

   It is a flag, not silent behaviour: refreshing rotates the stored refresh
   token — identity spends the old one — so it is a state change, not a read.

VERIFIED AGAINST PRODUCTION, end to end:
  whoami on a pre-existing file        -> Scope: sensing:read (was "not reported")
  whoami --refresh                     -> both tokens rotated
                                          refresh sha256[:12] 50cfa06b -> 2d37617a
                                          access  sha256[:12] 61eebe8d -> 14d8e8de
                                          status: expired -> valid
  refreshed token GET  /api/v1/models      -> 200
  refreshed token POST /api/v1/train/start -> 401 (still read-scoped)

That exercises the rotating-refresh path against identity's real reuse
detection — the one place a bug costs the user their session rather than a
retry — and confirms the scope gate survives a refresh.

Tests: 82 with --features login, 44 default, 501 sensing-server.

Co-Authored-By: Ruflo & AQE
2026-07-22 18:21:49 +02:00
Dragan Spiridonov 12635a85b2 fix(auth): stop requiring an iss claim Cognitum never issues (G-3 blocker)
The verifier called `Validation::set_issuer` and listed `iss` in
`set_required_spec_claims`. Cognitum access tokens have **no `iss` claim** — the
real claim set is typ, sub, account_id, org_id, workspace_id, client_id, scope,
family_id, jti, iat, exp, setup, workload. So the verifier rejected every
genuine token with a flat 401.

The whole 41-test suite was green throughout, because the test fixtures included
an `iss` the real thing does not have. The tests validated my assumption instead
of the platform. Found only by pointing a real token at a real server; no amount
of additional unit testing against the same wrong fixture would have caught it.

`valid_claims()` now mirrors production exactly, with a comment saying why, so
the next person cannot reintroduce the drift by "fixing" an incomplete-looking
fixture.

What binds a token to its issuer, then: the JWKS. We accept only signatures made
by a key served from the configured jwks_uri, so a valid signature IS proof of
issuer. meta-llm's verifier — the org's only other resource-side verifier —
checks neither `iss` nor `aud`, for the same reason. `VerifierConfig.issuer`
stays, now documented as JWKS-derivation and logging rather than a claim
assertion.

Three tests replace the two that asserted issuer behaviour that cannot exist:
a real-shaped (iss-less) token verifies; an unrelated `iss` changes nothing (so
identity adding one later cannot silently start failing tokens); and a token
signed by a different key is still refused — the complement that proves removing
the issuer check did not remove the boundary.

Also: report the granted scope from the token's `scope` claim when the
/oauth/token envelope omits it, which identity's does. `login` and `whoami` said
"(not reported by the server)" while the authoritative answer sat inside the
token they had just stored. The claim-peek helper is display-only and documented
at length as NOT verification — a client reading its own freshly issued token is
a different situation from a server reading a stranger's.

VERIFIED END TO END against production (gate G-3, previously open):
  no credential            GET  /api/v1/models        -> 401
  garbage bearer           GET  /api/v1/models        -> 401
  real sensing:read token  GET  /api/v1/models        -> 200
  real sensing:read token  GET  /api/v1/recording/list-> 200
  real sensing:read token  POST /api/v1/train/start   -> 401
  real sensing:read token  POST /api/v1/train/stop    -> 401
Token obtained by a human through `wifi-densepose login` against
auth.cognitum.one; server ran with RUVIEW_OAUTH_ISSUER set, JWKS fetched live
at boot (key_count=1).

Tests: 82 with --features login (58 unit + 22 matrix + 2 doctests).

Co-Authored-By: Ruflo & AQE
2026-07-22 18:11:21 +02:00
Dragan Spiridonov 714dae9a2c docs(adr): amend ADR-271 — the login flow lives in ruview-auth behind a feature
The ADR said the login flow was "not in this crate". It is now, gated behind a
non-default `login` feature. The original line existed to keep the sensing
server lean; a feature gate achieves that without forcing the Tauri desktop app
to grow a second copy of a PKCE + rotating-refresh implementation — the kind of
duplication that drifts and then disagrees about something subtle.

Recording the change rather than letting the ADR quietly go stale, which is the
exact failure this session hit twice in other repos' ADRs.

Co-Authored-By: Ruflo & AQE
2026-07-22 17:54:13 +02:00
Dragan Spiridonov 31fb3d53f6 feat(cli): wifi-densepose login — Cognitum sign-in (ADR-271 phase 2)
Phase 1 could verify a token and phase 3 could gate on one, but there was no
way for a user to OBTAIN one. This closes that: sign in with a Cognitum account
and get a token a RuView sensing server accepts, instead of everyone sharing
one static RUVIEW_API_TOKEN string.

Lives in `ruview-auth` behind a non-default `login` feature rather than in the
CLI, so the Tauri desktop app can reuse it instead of growing a second copy.
A server built with default features still gets the verifier and nothing else —
no reqwest, no tokio net, no browser launcher. (This amends ADR-271's "no login
flow in this crate" note; the reason for that line was to keep the server lean,
and a feature gate achieves it without duplication.)

Ported from meta-proxy's src/oauth/, cross-checked against musica's
cognitum_provider.rs — two independent implementations against this same AS.
Where they agree, this follows both: redirect path EXACTLY /oauth/callback,
60-second refresh skew, OOB fallback on SSH/CONTAINER//.dockerenv.

Refresh is the part with teeth. Identity rotates refresh tokens with reuse
detection, so presenting a spent one revokes the whole session family. Both
obvious implementations are wrong: refreshing concurrently looks like replay,
and retrying a failed refresh with the same token IS the replay. So
`Session::ensure_fresh` holds an async mutex across the await, re-checks expiry
after acquiring it (the waiter usually finds the work already done), persists
the rotated token BEFORE returning it, and never retries. A missing expires_at
counts as expired rather than being given a guessed default.

Least scope by default: `login` requests `sensing:read`. `--admin` adds
`sensing:admin` explicitly, and requests both because there is no scope
hierarchy server-side. A session that streams poses should not casually hold
the capability to delete the model it streams through.

Credentials are written atomically and 0600 (temp file, chmod BEFORE rename) —
the same discipline the seed applies to its cloud key. `logout` is local-only
and says so: it makes this machine unable to act as you, but revoking the
session everywhere is an account-level action.

Also `whoami`, which reports whether the stored token is live — an
expired-looking session is the most common reason a command starts 401ing, and
it should be visible directly rather than inferred from a failure elsewhere.

Verified against PRODUCTION, not just locally: authorize URLs built by this
exact code path return HTTP 200 from auth.cognitum.one for both
`sensing:read` and `sensing:read sensing:admin`, which exercises the real
client_id, scope encoding, PKCE parameters and redirect_uri shape.

Tests: 74 with --features login (51 unit + 21 verifier matrix + 2 doctests),
including the RFC 7636 Appendix B vector, multi-scope URL encoding (a space
that is hand-formatted rather than encoded silently truncates the request), a
real TCP callback round-trip, callback timeout, 0600 permissions asserted on
disk, atomic-save leaving no temp file, and refresh-window boundaries.
Unchanged: 43 with default features, 501 in the sensing server.

Co-Authored-By: Ruflo & AQE
2026-07-22 17:53:42 +02:00
Dragan Spiridonov c2bd33e649 feat(sensing-server): accept Cognitum OAuth on /api/v1/*, scope-gated (ADR-271 phase 3)
Wires `ruview-auth` into `bearer_auth.rs`. `RUVIEW_OAUTH_ISSUER` enables it;
unset, nothing changes.

Layering, in order:
  1. `RUVIEW_API_TOKEN` set and the bearer matches exactly -> allow. Byte-for-
     byte today's behaviour.
  2. Otherwise, if OAuth is configured, verify the bearer as a Cognitum access
     token and require the scope the route needs.
  3. Otherwise 401.

The static compare goes first for compatibility, not security: a matching
static token is not a JWT and a JWT never matches the static token. It means an
existing deployment behaves identically even with OAuth switched on.

Scope gate (`required_scope_for`), split by blast radius per ADR-060 —
"can this destroy something", not how many routes it covers:
  sensing:admin  /api/v1/train/* (hours of Pi CPU, writes models)
                 DELETE /api/v1/models/{id}      (irreversible)
                 DELETE /api/v1/recording/{id}   (irreversible)
  sensing:read   everything else
Deliberately NOT admin: model load/unload and recording start. They mutate
server state but destroy nothing, and gating them would push routine dashboard
use into requesting delete capability — the opposite of least privilege.

The legacy static token stays un-scope-gated. It predates scopes and carries no
claims, so narrowing it would be a silent breaking change to deployments using
it; migrating to OAuth is how an operator opts into the finer split.

FAIL CLOSED at boot. If OAuth is requested but cannot work — empty issuer, or a
JWKS we cannot fetch — the server logs why and exits rather than serving.
Starting anyway would silently downgrade an operator who asked for OAuth to
either an open API or a shared-secret one, with no signal it happened. The JWKS
is warmed eagerly for the same reason: a bad `jwks_uri` should die at boot with
a legible message, not surface as a puzzling 401 an hour later.

The verified `Principal` is attached to request extensions, so handlers and
audit logs can attribute a request (`sub`, `account_id`, `org_id`,
`workspace_id`, `jti`) instead of knowing only "someone had the secret". That
is the point of moving off a shared bearer.

Verification failures are logged with the reason and returned as a flat 401 —
the reason is useful to an operator and equally useful to an attacker probing
for which claim to forge next.

Also aligns `ruview_auth::extract_bearer` to match the scheme
case-insensitively (RFC 7235 §2.1). The sensing server has always done this
deliberately, with a comment saying why; the two layers disagreeing about what
a valid header looks like would be a latent bug.

Tests: 16 new in `bearer_auth::oauth_tests`, driving a real Router end to end
(request -> middleware -> verifier -> handler) with ES256 tokens signed by a
runtime-generated key. Covers the scope policy as a pure function, read-scoped
tokens refused on delete and train, admin-scoped tokens allowed, an
`inference`-only token from another Cognitum product refused on every route,
garbage and absent bearers, both legacy-token layering directions, the
principal reaching a handler, and the unset case remaining a no-op.

`cargo test -p wifi-densepose-sensing-server --lib --no-default-features`:
501 passed, 0 failed. `ruview-auth`: 43 passed across both feature configs.

Co-Authored-By: Ruflo & AQE
2026-07-22 15:33:22 +02:00
Dragan Spiridonov 92cbeb0c34 docs(adr): ADR-271 — RuView as a Cognitum OAuth resource server
The previous commit referenced ADR-271 in five places without the ADR existing.
This writes it.

Records the decision and, more importantly, the direction — RuView verifies
tokens users present, it does not obtain tokens to call Cognitum. Every other
Cognitum OAuth integration in the org is the client side of a plane RuView does
not have; the sole relevant precedent is meta-llm's oauthBearer.ts.

Covers: why offline verification is a requirement rather than an optimisation
(Pi-class hosts lose WAN, and no introspection endpoint exists); why the
accept-rule is a port and not a design; why long-lived setup/workload
credentials are refused (no database to check revocation); why scope — not
`aud`, not `client_id` — is the capability boundary; and the alternatives,
including the `cog_`-minting approach that was tried org-wide and found broken
against production.

Co-Authored-By: Ruflo & AQE
2026-07-22 15:10:48 +02:00
Dragan Spiridonov 499cec7914 feat(auth): ruview-auth — offline Cognitum OAuth access-token verification (ADR-271)
RuView's `/api/v1/*` is gated today by `RUVIEW_API_TOKEN`: one shared secret,
no expiry, no per-user attribution. This adds the verifier half of replacing
that with Cognitum identity — RuView as an OAuth **resource server**, so a user
signs in to their own sensing server with their Cognitum account.

Note the direction: RuView does not call Cognitum. It never obtains a token for
its own use; it verifies tokens users present. Every other Cognitum OAuth
integration in the org (meta-proxy, musica, metaharness, the dashboard CLI) is
the client side of a plane RuView doesn't have. The one existing resource-server
verifier is `meta-llm/src/auth/oauthBearer.ts`, and this crate is a port of its
accept-rule — divergence would be a bug, not a preference: a token meta-llm
rejects must not be one RuView accepts.

Verification is OFFLINE, and that is a requirement rather than an optimisation.
RuView runs on Pi-class hardware that loses WAN, and identity publishes no
introspection endpoint even when the network is up. So: ES256 over identity's
published JWKS, cached by `kid`.

What it accepts, mirroring oauthBearer.ts:
  typ == "access" AND NOT setup AND NOT workload AND account_id non-empty AND
  exp in the future AND iss matches verbatim AND the required scope is held.

Long-lived setup (365-day) and workload credentials are refused outright for
identity's own stated reason: their revocation lives in `oauth_setup_tokens`,
and RuView — like meta-llm — has no database to check it. A 15-minute access
token needs no revocation round-trip because it expires faster than revocation
propagates; a 365-day one does.

Scope is the capability boundary, and it has to be. Cognitum access tokens carry
no `aud`, and `client_id` cannot substitute because clients borrow each other's
registrations (musica ships DEFAULT_CLIENT_ID = "meta-proxy"). `sensing:read`
covers streams and inference; `sensing:admin` covers training, model delete and
recording delete. No hierarchy — admin does not imply read; consent means what
it said. Registered in identity migration 0016 (cognitum-one/dashboard#116).

Design notes:
- `ureq`, not `reqwest`: the sensing server deliberately chose ureq as "the
  smallest" HTTP client, and pulling reqwest in here would reverse that for the
  whole graph. Transport sits behind a `JwksFetcher` trait, so a host can supply
  its own and take no HTTP dependency at all (feature `ureq-transport`, default).
- A JWKS refetch failure is survivable while a key set is already cached: a key
  that verified a minute ago has not stopped being valid because the network
  blipped, and failing closed there logs every user out of their own sensing
  server whenever their internet wobbles. We fail closed in exactly one case —
  no key set has ever been fetched.
- Unknown `kid` forces one refetch so rotation is picked up without waiting out
  the TTL, rate-limited so junk-kid tokens can't become a request amplifier
  aimed at identity.
- Expiry is reported distinctly from a bad signature: on an RTC-less Pi that is
  usually a clock-sync fault, and an operator must be able to tell them apart.
- Test keypairs are generated at runtime, never committed. This repo tracks zero
  `.pem` files and committed key material shouldn't start here — it also means
  no fixture can drift out of sync with the JWKS it's served by.

Tests: 41 green (19 unit + 21 matrix + 1 doctest) under BOTH
`cargo test --no-default-features` (the repo's canonical gate) and default
features. The matrix signs real ES256 tokens rather than asserting on strings,
and covers alg:none, forged signature, spliced payload, unknown kid, expired,
leeway boundaries both sides, wrong issuer, issuer differing only by a trailing
slash, missing/!=access typ, setup and workload smuggled onto typ=access,
missing and empty account_id, and scope escalation.

The highest-value case is `g2_a_genuinely_valid_token_from_another_cognitum_
product_cannot_reach_the_sensing_surface`: a correctly signed, unexpired,
right-issuer, right-typ token with client_id=meta-proxy and scope=inference is
rejected. Nothing about its signature or identity claims distinguishes it — only
scope does. A naive verifier accepts it, and an `inference` token becomes a key
to someone's home sensor.

No wiring into the sensing server yet; this crate is verification only, with no
login flow and no outbound Cognitum calls.

Co-Authored-By: Ruflo & AQE
2026-07-22 15:08:40 +02:00
169 changed files with 29494 additions and 1652 deletions
+34 -22
View File
@@ -1,14 +1,10 @@
name: Continuous Deployment
on:
push:
branches: [ main ]
tags: [ 'v*' ]
workflow_run:
workflows: ["Continuous Integration"]
workflows: ["wifi-densepose sensing-server → Docker Hub + ghcr.io"]
types:
- completed
branches: [ main ]
workflow_dispatch:
inputs:
environment:
@@ -19,6 +15,11 @@ on:
options:
- staging
- production
image_tag:
description: 'Existing ghcr.io/ruvnet/wifi-densepose tag to deploy'
required: true
default: 'latest'
type: string
force_deploy:
description: 'Force deployment (skip checks)'
required: false
@@ -27,7 +28,7 @@ on:
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
IMAGE_NAME: ruvnet/wifi-densepose
KUBE_CONFIG_DATA: ${{ secrets.KUBE_CONFIG_DATA }}
jobs:
@@ -35,7 +36,9 @@ jobs:
pre-deployment:
name: Pre-deployment Checks
runs-on: ubuntu-latest
if: github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch'
if: |
(github.event_name == 'workflow_run' && github.event.workflow_run.conclusion == 'success') ||
github.event_name == 'workflow_dispatch'
outputs:
deploy_env: ${{ steps.determine-env.outputs.environment }}
image_tag: ${{ steps.determine-tag.outputs.tag }}
@@ -43,6 +46,7 @@ jobs:
- name: Checkout code
uses: actions/checkout@v4
with:
ref: ${{ github.event.workflow_run.head_sha || github.sha }}
submodules: recursive
- name: Determine deployment environment
@@ -50,14 +54,12 @@ jobs:
env:
# Use environment variable to prevent shell injection
GITHUB_EVENT_NAME: ${{ github.event_name }}
GITHUB_REF: ${{ github.ref }}
PUBLISHED_REF: ${{ github.event.workflow_run.head_branch }}
GITHUB_INPUT_ENVIRONMENT: ${{ github.event.inputs.environment }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]]; then
echo "environment=$GITHUB_INPUT_ENVIRONMENT" >> $GITHUB_OUTPUT
elif [[ "$GITHUB_REF" == "refs/heads/main" ]]; then
echo "environment=staging" >> $GITHUB_OUTPUT
elif [[ "$GITHUB_REF" == refs/tags/v* ]]; then
elif [[ "$PUBLISHED_REF" == v* ]]; then
echo "environment=production" >> $GITHUB_OUTPUT
else
echo "environment=staging" >> $GITHUB_OUTPUT
@@ -65,16 +67,23 @@ jobs:
- name: Determine image tag
id: determine-tag
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PUBLISHED_REF: ${{ github.event.workflow_run.head_branch }}
PUBLISHED_SHA: ${{ github.event.workflow_run.head_sha }}
INPUT_IMAGE_TAG: ${{ github.event.inputs.image_tag }}
run: |
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
echo "tag=${GITHUB_REF#refs/tags/}" >> $GITHUB_OUTPUT
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" ]]; then
echo "tag=$INPUT_IMAGE_TAG" >> $GITHUB_OUTPUT
elif [[ "$PUBLISHED_REF" == v* ]]; then
echo "tag=$PUBLISHED_REF" >> $GITHUB_OUTPUT
else
echo "tag=${{ github.sha }}" >> $GITHUB_OUTPUT
echo "tag=sha-${PUBLISHED_SHA:0:7}" >> $GITHUB_OUTPUT
fi
- name: Verify image exists
run: |
docker manifest inspect ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.determine-tag.outputs.tag }}
docker manifest inspect "${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.determine-tag.outputs.tag }}"
# Deploy to staging
deploy-staging:
@@ -129,7 +138,10 @@ jobs:
name: Deploy to Production
runs-on: ubuntu-latest
needs: [pre-deployment, deploy-staging]
if: needs.pre-deployment.outputs.deploy_env == 'production' || (github.ref == 'refs/tags/v*' && needs.deploy-staging.result == 'success')
if: |
always() &&
needs.pre-deployment.result == 'success' &&
needs.pre-deployment.outputs.deploy_env == 'production'
environment:
name: production
url: https://wifi-densepose.com
@@ -210,7 +222,7 @@ jobs:
# kubectl scale rs -n wifi-densepose -l app=wifi-densepose,version!=green --replicas=0
- name: Upload deployment artifacts
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: production-deployment-${{ github.run_number }}
path: |
@@ -260,7 +272,7 @@ jobs:
post-deployment:
name: Post-deployment Monitoring
runs-on: ubuntu-latest
needs: [deploy-staging, deploy-production]
needs: [pre-deployment, deploy-staging, deploy-production]
if: always() && (needs.deploy-staging.result == 'success' || needs.deploy-production.result == 'success')
steps:
- name: Monitor deployment health
@@ -281,7 +293,7 @@ jobs:
done
- name: Update deployment status
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
script: |
const deployEnv = '${{ needs.pre-deployment.outputs.deploy_env }}';
@@ -300,7 +312,7 @@ jobs:
notify:
name: Notify Deployment Status
runs-on: ubuntu-latest
needs: [deploy-staging, deploy-production, post-deployment]
needs: [pre-deployment, deploy-staging, deploy-production, post-deployment]
if: always()
steps:
- name: Notify Slack on success
@@ -332,7 +344,7 @@ jobs:
- name: Create deployment issue on failure
if: needs.deploy-production.result == 'failure'
uses: actions/github-script@v6
uses: actions/github-script@v7
with:
script: |
github.rest.issues.create({
@@ -355,4 +367,4 @@ jobs:
**Logs:** Check the workflow run for detailed error messages.
`,
labels: ['deployment', 'production', 'urgent']
})
})
+35
View File
@@ -171,6 +171,41 @@ jobs:
- name: ADR-135 calibration witness proof (determinism guard)
run: bash scripts/verify-calibration-proof.sh
# The workspace runs with --no-default-features, which switches OFF
# ruview-auth's `login` and `pkce` features. That silently excluded 40 of
# its 87 tests — the whole interactive sign-in path: credential storage,
# single-flight refresh, the advisory file lock, the loopback callback, and
# PKCE generation. They were green locally and never executed here.
# Measured: 47 tests with --no-default-features, 87 with --all-features.
- name: Run ruview-auth tests with all features (ADR-271 login path)
working-directory: v2
env:
CARGO_PROFILE_DEV_DEBUG: "0"
CARGO_PROFILE_TEST_DEBUG: "0"
run: cargo test -p ruview-auth --all-features
# Browser-facing JavaScript.
#
# These run the dashboard's own modules in Node with stubbed browser globals.
# They exist because the Rust suite cannot see them at all: two ADR-271/272
# defects (a service worker caching /oauth/status, and the WebSocket ticket
# helper) lived entirely in `ui/` and were invisible to a fully green
# workspace. Blocking, and fast — no browser, no install step.
ui-tests:
name: UI JavaScript Tests
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Run UI unit tests
run: node --test ui/sw.test.mjs ui/services/ws-ticket.test.mjs
# Unit and Integration Tests
# Python pytest matrix — runs against the archived v1 Python tree.
# `continue-on-error: true` for the same reason as code-quality above:
+59 -8
View File
@@ -34,9 +34,8 @@
# dedicated follow-up commit (drop `password:`, add the OIDC id-token
# permission + `environment: pypi`) so there is no capability gap between.
#
# Q3 (witness hash v2 — open in ADR-117 §11.3) MUST be resolved
# before the first v2.0.0 publish. When v2 lands, add a parallel
# step that verifies the v2 hash against the Rust pipeline.
# Production publishing fails closed until the ADR-117 §11.3 v2 witness
# hash exists. TestPyPI remains usable to validate release artifacts.
name: pip-release
@@ -83,7 +82,7 @@ jobs:
arch: x86_64
- os: ubuntu-latest
arch: aarch64
- os: macos-13 # x86_64 runner
- os: macos-15-intel # x86_64 runner
arch: x86_64
- os: macos-14 # arm64 runner
arch: arm64
@@ -152,6 +151,46 @@ jobs:
path: sdist/*.tar.gz
if-no-files-found: error
build-ruview:
name: Build ruview meta-package
if: |
github.event_name == 'workflow_dispatch' && inputs.target == 'v2-wheels' ||
startsWith(github.ref, 'refs/tags/v2.')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v6
with:
python-version: '3.12'
- name: Verify lock-step package versions
shell: python
run: |
import pathlib
import tomllib
root = pathlib.Path("python")
core = tomllib.loads((root / "pyproject.toml").read_text(encoding="utf-8"))
meta = tomllib.loads((root / "ruview-meta" / "pyproject.toml").read_text(encoding="utf-8"))
core_version = core["project"]["version"]
meta_version = meta["project"]["version"]
expected_dependency = f"wifi-densepose=={core_version}"
if meta_version != core_version:
raise SystemExit(
f"package versions differ: wifi-densepose={core_version}, ruview={meta_version}"
)
if expected_dependency not in meta["project"]["dependencies"]:
raise SystemExit(f"ruview must depend on {expected_dependency}")
print(f"lock-step version: {core_version}")
- name: Build ruview wheel and sdist
run: |
python -m pip install --upgrade pip build
python -m build python/ruview-meta --outdir ruview-dist
- uses: actions/upload-artifact@v4
with:
name: ruview
path: ruview-dist/*
if-no-files-found: error
# ────────────────────────────────────────────────────────────────
# v1.99.0 — tombstone wheel (pure Python, single sdist + wheel)
# ────────────────────────────────────────────────────────────────
@@ -236,18 +275,29 @@ jobs:
# ────────────────────────────────────────────────────────────────
publish-v2:
name: Publish v2 wheels
needs: [build-wheels, build-sdist]
name: Publish wifi-densepose + ruview
needs: [build-wheels, build-sdist, build-ruview]
if: |
always() &&
needs.build-wheels.result == 'success' &&
needs.build-sdist.result == 'success' &&
needs.build-ruview.result == 'success' &&
(
github.event_name == 'workflow_dispatch' && inputs.target == 'v2-wheels' ||
startsWith(github.ref, 'refs/tags/v2.')
)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Enforce production witness gate
if: |
startsWith(github.ref, 'refs/tags/v2.') ||
(github.event_name == 'workflow_dispatch' && inputs.publish_to == 'pypi')
run: |
test -s archive/v1/data/proof/expected_features_v2.sha256 || {
echo "::error::ADR-117 §11.3 release gate is incomplete: archive/v1/data/proof/expected_features_v2.sha256 is missing or empty"
exit 1
}
- name: Gather all artifacts into dist/
uses: actions/download-artifact@v4
with:
@@ -264,7 +314,7 @@ jobs:
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
password: ${{ secrets.PYPI_API_TOKEN }}
password: ${{ secrets.TESTPYPI_API_TOKEN }}
packages-dir: dist
skip-existing: true
- name: Publish to PyPI
@@ -275,6 +325,7 @@ jobs:
with:
password: ${{ secrets.PYPI_API_TOKEN }}
packages-dir: dist
verbose: true
publish-tombstone:
name: Publish v1.99 tombstone
@@ -299,7 +350,7 @@ jobs:
uses: pypa/gh-action-pypi-publish@release/v1
with:
repository-url: https://test.pypi.org/legacy/
password: ${{ secrets.PYPI_API_TOKEN }}
password: ${{ secrets.TESTPYPI_API_TOKEN }}
packages-dir: dist
skip-existing: true
- name: Publish to PyPI
+5
View File
@@ -291,3 +291,8 @@ harness/**/ruvector.db
# ruvector runtime/hook DB — never tracked (any depth)
ruvector.db
**/ruvector.db
# sensing-server runtime artifacts written by its test suite (trained model
# snapshots + the generated session-secret) — never tracked
v2/crates/wifi-densepose-sensing-server/data/
*.proptest-regressions
+10 -2
View File
@@ -7,9 +7,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
### Added
- **HOMECORE platform runtime completion — secure native/Wasmtime plugins, authenticated HAP IP, expanded Home Assistant APIs, durable restoration/migration, and voice protocols.** `homecore-server` now owns deterministic compiled-in native plugin registration plus explicitly configured, path-bounded, Ed25519 publisher-verified Wasm packages executed through Wasmtime with setup/state-change/teardown lifecycle; arbitrary native dynamic libraries remain intentionally unsupported. The optional HAP server implements persisted accessory identity and controller records, SRP-6a Pair-Setup M1M6, X25519/Ed25519 Pair-Verify M1M4, HKDF-SHA512/ChaCha20-Poly1305 record framing, authenticated/admin endpoint gates, replay/tamper closure, live entity synchronization, and paired-state `_hap._tcp` mDNS updates (45 focused tests; external Apple certification is not claimed). Startup restores device/entity registries and deterministic latest recorder states before plugins, and migration now atomically preserves forward-compatible device/config-entry fields. The HA-compatible surface adds events, templates, config checks, components, registries, history/logbook with SQL-enforced global response bounds, calendar/camera provider routes, and modern WebSocket negotiation while retaining a machine-readable limitations matrix for integration-specific behavior. Assist adds bounded PCM16, async STT/TTS contracts, an end-to-end speech pipeline, and an authenticated satellite session protocol; real deployments still provide the speech engines.
- **`ruview-unified` increment 3 — Gaussian update-loop completion, separable delay-Doppler, and property-tested boundary hardening.** (1) `GaussianMap::merge_overlapping` (ADR-275 step 5: mutual-Mahalanobis + semantic-compatibility dedup catching drift the insert-time gate misses) and lifetime-aware decay (`τ_eff = τ·(1+ln(1+lifetime/τ))` — confirmed structures outlive transients at equal nominal τ). (2) `delay_doppler_map` reimplemented separably (`O(B²S+S²B)`), proven equivalent to the direct reference to <1e-10 and **measured 8.3× faster** (520 µs vs 4.34 ms at 56×8). (3) `tests/security_boundaries.rs` — 8 `proptest` properties over the boundary surfaces (arbitrary values incl. NaN/±inf via `f64::from_bits`) that found and fixed three input-controlled defects: a BLE-CS phase-unwrap infinite loop on non-finite phases and an ~1e299-iteration loop on finite-huge phases (now O(1) modular unwrap + plausibility bound), and a subnormal Gaussian scale overflowing `1/σ²` to NaN density (now physical σ/occupancy bounds). (4) New criterion benches for all increment-2 hot paths (`to_canonical` 38 µs, `ble_cs_range` 481 ns, AoI planner 647 ns/200 regions, coherent fusion 1.5 µs/32 members, factorized pose 521 ns). ruview-unified now 98 tests (87 lib + 3 acceptance + 8 security), 0 failed, clippy-clean.
- **`ruview-unified` increment 2 — native frame contract + programmable perception (ADR-279..282).** (1) `RfFrameV2` becomes the authoritative RF record: native complex IQ with explicit validity masks, declared `PhaseState`, TX/RX poses + antenna geometry in one building frame, calibration/quality state, and a provenance rule enforced at construction — `Synthetic ⇒ L0Simulation` and `Measured ⇒ ≥ L1CapturedReplay` can never alias (the public L0L5 evidence ladder is now a type); the 56-bin canonical tensor is demoted to a derived compatibility view (`to_canonical`, mask-aware gap-filling through the same normalization path as every adapter; native samples proven byte-untouched). (2) Active sensing control plane (`control.rs`): ETSI-ISAC-vocabulary `SensingTask` admission (raw export always refused; identity requires consent), `SensingAction`/`InformationGoal`, an age-of-information `ActiveSensingPlanner` (priority = uncertainty × change rate × criticality ÷ cost; **measured 95% sensing-traffic reduction** vs uniform refresh on a 20-region scenario), fail-closed `CoherentSensorGroup` fusion gates (time/phase/geometry bounds; five denial paths tested), policy-authorized RIS/movable-antenna actuation receipts, and purpose-scoped `TaskSufficientRepresentation` leakage validation. (3) New modality surfaces: BLE Channel Sounding adapter + `ble_cs_range` treating phase-slope and RTT as **separate cross-validated evidence** (exact distance recovery on synthetic tones; relay-style divergence flagged, never averaged), delay-Doppler-native `FieldAxis` + `delay_doppler_map` (unit-peak tone test), IEEE P3162 synthetic-aperture import profile. (4) RePos-factorized pose head (relative skeleton on the content representation, root on the geometry-conditioned one, calibrated per-joint uncertainties): held-out-room MPJPE 0.0003 m vs 0.2534 m for the monolithic baseline in the room-shortcut leakage experiment; ≤2% structured-adapter budget (740 params). (5) Age gate input now `log(1+age_ms)` per the age-aware-CSI recipe (gradient check re-proven); Gaussian primitives gained `first_seen_ns`/`doppler_variance`/bounded `source_receipts` lineage; `PartitionKey` gained a `session` dimension and `SplitManifest` certifies disjointness across all seven dimensions. 87 tests, 0 failed; crate clippy-clean. Docker images unaffected (no shipped binary consumes the crate yet); Python proof re-verified PASS.
- **`ruview-unified` — unified RF spatial world model, P1 (ADR-273..278).** New v2 workspace leaf crate implementing the five-pillar architecture: (1) canonical `RfTensor` (`links × 56 bins × 8 snapshots`, complex, validated at the boundary) plus a fail-closed hardware adapter registry with reference adapters for 802.11 CSI (consumes `wifi-densepose-core::CsiFrame`), FMCW radar cubes (fast-time DFT), UWB CIR, and 5G SRS (comb de-interleave); (2) a universal RF foundation encoder — window-median + CFO-aligned tokenizer, masked-reconstruction pretraining with a hand-derived backward pass verified against central finite differences (174 params sampled, max rel err 1.31e-5), the ADR-273 fusion contract `z = Enc(CSI) ⊙ σ(AgeEnc) + GeomEnc(pose)`, and ≤1% task adapters (presence 129 / activity 268 / localization 387 / anomaly 2 vs a 40,856-param backbone); (3) an RF-aware Gaussian spatial memory — anisotropic primitives with per-band×angle reflectivity, confidence-weighted fusion, exponential decay, spatial-hash + semantic queries, closed-form (erf) BeerLambert channel-gain queries that degrade to exact Friis on an empty map, inverse gain updates that learn an unseen 6 dB obstruction to <0.5 dB in 20 link observations, and a JITOMA-style task-gated scene graph; (4) a physics-guided synthetic RF world generator — AllenBerkley image method (order ≤2), complex-permittivity Fresnel materials, bistatic person scattering with *emergent* Doppler (proven against the analytic phase rate), seeded ChaCha20 domain randomization of physics + hardware nuisances (gain/CFO/phase noise/packet loss/interference); (5) an edge sensing control plane — 802.11bf/ETSI-ISAC-aligned purposes and zones, fail-closed authorization, a double-gated identity purpose, retention bounds, and a `BoundedEvent`-only trust boundary that makes raw RF export unrepresentable. Anti-leakage evaluation (`StrictSplit` by room/day/person/chipset/firmware/layout with an independent disjointness verifier, ECE, selective risk, degradation) plus an end-to-end acceptance pipeline: presence F1 1.00 on held-out rooms *and* held-out chipset, degradation 0.0, ECE 0.012, p95 tokenize+encode 2.0 ms debug / 105 µs release — **all SYNTHETIC** (honest labeling propagates from `RfModality::Synthetic` through `Provenance.synthetic`). Criterion benches with an optimization pass: segment-corridor candidate search took `channel_gain` from 139 µs → 27 µs (O(1) in map size; hash/linear crossover at ~4k Gaussians reported honestly), `observe_link` 305 µs → 74 µs, precomputed DFT twiddles 4.9×. 66 unit + 3 acceptance tests, 0 failed.
### Changed
- **`wifi-densepose` promoted to `2.0.0` stable; `ruview` `2.0.0` first stable publish (ADR-184 P2).** Dropped the `a1` alpha suffix on both sibling packages (`python/pyproject.toml`, `python/ruview-meta/pyproject.toml`) and flipped their trove classifier `Development Status :: 3 - Alpha``5 - Production/Stable`; the `ruview` meta-package's `wifi-densepose==2.0.0a1` dependency pins (base + `[client]`) were repointed to `==2.0.0`. Pip-release now authenticates via Trusted Publishing (see the entry below). **Version-metadata prep only — nothing is published by this change**: the actual PyPI upload (ADR-184 P3) is still gated on the one-time manual Trusted Publisher registration on pypi.org that only the repo owner can perform. Justified as "stable": the default (no-extras) wheel builds at 279 KB (`maturin build --release --strip`) and the base non-SOTA suite is green — `pytest python/tests/` (excluding the `[aether]`/`[meridian]`/`[mat]` extra modules) = **185 passed, 0 failed** (smoke / keypoint / pose / vitals / bfld / security / WS+MQTT client).
- **CI (ADR-184): `pip-release.yml` publish job migrated to PyPI OIDC Trusted Publishing** (commit `cc153e8b5`; refs #785, completes ADR-117). The release workflow now authenticates to PyPI via short-lived OIDC tokens (`id-token: write`) instead of a long-lived `PYPI_API_TOKEN` secret. **Not yet active**: publishing will fail until the matching Trusted Publisher is registered manually on pypi.org (a one-time, per-project step that cannot be automated from CI) — ADR-184 P1 tracks this as the remaining gate (status recorded in `dfc4c1abd`).
- **crates.io release batch — 10 of the 12 documented crates republished at their next patch version.** `wifi-densepose-core` 0.3.2, `-vitals` 0.3.2, `-wifiscan` 0.3.2, `-hardware` 0.3.2 (picks up the ADR-273..282 review-fix commit's clippy fixes), `-signal` 0.3.6, `-nn` 0.3.2, `-ruvector` 0.3.3, `-train` 0.3.3, `-mat` 0.3.2, `-wasm` 0.3.1 — all published and verified live on crates.io. **`wifi-densepose-sensing-server` and `wifi-densepose-cli` were bumped locally (0.3.5, 0.3.2) but NOT published**: both now path-depend on `ruview-auth`, which is deliberately `publish = false` and not on crates.io — `cargo publish` correctly refuses to publish a crate with an unversioned/unpublishable path dependency. This is a pre-existing gap (the dependency predates this batch); resolving it is a deliberate call for whoever owns whether `ruview-auth` becomes public, not something to route around silently.
- **`wifi-densepose` promoted to `2.0.0` stable; `ruview` `2.0.0` prepared for its first stable publish (ADR-184 P2).** Dropped the `a1` alpha suffix on both sibling packages (`python/pyproject.toml`, `python/ruview-meta/pyproject.toml`) and flipped their trove classifier `Development Status :: 3 - Alpha``5 - Production/Stable`; the `ruview` meta-package's `wifi-densepose==2.0.0a1` dependency pins (base + `[client]`) were repointed to `==2.0.0`. **Version-metadata prep only — nothing is published by this change**: the actual PyPI upload remains gated on the ADR-117 v2 witness hash. Justified as "stable": the default (no-extras) wheel builds at 279 KB (`maturin build --release --strip`) and the base non-SOTA suite is green — `pytest python/tests/` (excluding the `[aether]`/`[meridian]`/`[mat]` extra modules) = **185 passed, 0 failed** (smoke / keypoint / pose / vitals / bfld / security / WS+MQTT client).
- **CI (ADR-184): `pip-release.yml` keeps token-based PyPI authentication until Trusted Publishing is registered.** An OIDC migration was attempted in `cc153e8b5` and reverted in `82d5c7339` so releases would not enter a half-configured state. Production currently uses `PYPI_API_TOKEN`; TestPyPI uses its independent `TESTPYPI_API_TOKEN`. The workflow now builds and publishes `wifi-densepose` and `ruview` together, verifies their versions and dependency pin match, and fails closed before production upload when `expected_features_v2.sha256` is absent.
- **`@ruvnet/rvagent` startup optimization — stdio time-to-first-response ~242 ms → ~189 ms (22%; MEASURED, median of repeated `initialize` round-trips against `dist/index.js`, this container, reproduce with a piped-stdin timer).** Two changes: (1) `./http-transport.js` is now imported **lazily** inside the `RVAGENT_HTTP_PORT` branch — it chain-loads the MCP SDK's `streamableHttp` module (~48 ms MEASURED via per-module `import()` timing), which the default stdio path never uses; (2) the advertised JSON Schemas generated from the Zod sources are memoized per tool instead of re-walking the Zod tree on every `tools/list` (matters under the session-per-server HTTP model where each session lists tools). No behavior change: 99/99 jest tests, HTTP session flow re-smoke-tested through the lazy path. The `@ruvnet/ruview` harness CLI was profiled too and left alone — 50 ms vs the ~29 ms bare `node -e ''` floor on the same box (MEASURED), i.e. already near the interpreter floor with zero dependencies.
### Deprecated
+12 -1
View File
@@ -25,6 +25,7 @@ Dual codebase: Python v1 (`v1/`) and Rust port (`v2/`).
| `vendor/rufield` (submodule) | **RuField MFS** — the open spec for camera-free multimodal field sensing (ADR-260). A common `FieldEvent`/`FieldTensor`/`FusionGraph`/`PrivacyClass`/`ProvenanceReceipt` model *above* WiFi CSI/CIR/BFLD, UWB, BLE Channel Sounding, mmWave radar, ultrasound, subsonic, infrared, and quantum sensors. Lives in its own repo ([github.com/ruvnet/rufield](https://github.com/ruvnet/rufield)), vendored here under `vendor/rufield`. Not a `v2/` workspace member. v0.1 reference stack = 7 crates (`rufield-core`/`-provenance`/`-privacy`/`-adapters`/`-fusion`/`-bench`/`-viewer`), 72 tests/0 failed; `rufield-viewer` is an Axum + vanilla-JS read-only dashboard (`cargo run -p rufield-viewer`) completing ADR-260 §27.9. The WiFi-CSI modality is now **real-replay-backed** via `CsiReplayAdapter` (ingests real captured `.csi.jsonl` → fused presence/breathing inferences; replay-from-file, unlabeled CSI-variance proxy, not validated accuracy); mmWave/thermal + all synthetic-bench F1 numbers remain **SYNTHETIC** (no live hardware — live streaming + labeled accuracy are roadmap). |
| `wifi-densepose-rufield` | ADR-262 P1 **anti-corruption bridge** — converts RuView WiFi-CSI sensing output (`SensingSnapshot` mirroring `SensingUpdate` + `TrustedOutput`, owned primitives, no dep on `wifi-densepose-sensing-server`) into **signed RuField `FieldEvent`s** (`Modality::WifiCsi`, real `timestamp_ns`, sha256 + ed25519 provenance, `synthetic=false`). The single coupling point between RuView and the standalone RuField MFS spec (§5.4); path-deps the `vendor/rufield` submodule crates (`rufield-core`/`-provenance`/`-privacy`/`-fusion`). **Critical §3.3 privacy mapping** (`map_privacy`): maps RuView class → RuField P0P5 by **information content, never byte value**, fail-closed (`Derived → P4/P5`, never P1; `demoted` floors to ≥ P2). 15 tests / 0 failed (round-trip / `is_fusable` / fusion-ingest / privacy-safety / determinism). P1 plumbing — not wired into the live server (P3), no accuracy claim. |
| `ruview-swarm` | Drone swarm control system (ADR-148) — hierarchical-mesh topology, Raft consensus, MARL, CSI sensing payload, MAVLink/PX4 compat, Ruflo AI-agent integration |
| `ruview-unified` | ADR-273..282 **unified RF spatial world model**: authoritative native `RfFrameV2` frame contract (native IQ never overwritten, phase-state/evidence-ladder/provenance invariants) with the canonical `RfTensor` as a derived view; fail-closed hardware adapter registry (WiFi CSI / FMCW cube / UWB CIR / 5G SRS / BLE Channel Sounding with phase-vs-RTT cross-validated ranging); universal RF foundation encoder (masked-reconstruction pretraining with finite-difference-verified backprop, `z = Enc ⊙ σ(AgeEnc(log age)) + Geom` fusion, ≤1% scalar / <2% structured task adapters incl. RePos-factorized pose); RF-aware Gaussian spatial memory (fusion/decay/channel-gain queries + inverse updates, lineage receipts, task-gated scene graph); physics-guided synthetic RF world generator (image-method multipath, Fresnel materials, emergent Doppler, seeded domain randomization); edge sensing control plane (802.11bf/ETSI-ISAC purposes/zones/tasks, AoI active-sensing planner, fail-closed coherent-aperture fusion, governed RIS actuation; raw RF structurally unexportable); delay-Doppler-native transforms. Pure Rust leaf; all accuracy numbers SYNTHETIC (evidence level L0) until real-data validation. |
### RuvSense Modules (`signal/src/ruvsense/`)
| Module | Purpose |
@@ -62,7 +63,7 @@ All 5 ruvector crates integrated in workspace:
- `ruvector-attention``model.rs` (apply_spatial_attention) + `bvp.rs`
### Architecture Decisions
182 ADRs in `docs/adr/` (numbered ADR-001 through ADR-265, with gaps). Key ones:
205 ADRs in `docs/adr/` (numbered ADR-001 through ADR-282, with gaps). Key ones:
- ADR-014: SOTA signal processing (Accepted)
- ADR-015: MM-Fi + Wi-Pose training datasets (Accepted)
- ADR-016: RuVector training pipeline integration (Accepted — complete)
@@ -81,6 +82,16 @@ All 5 ruvector crates integrated in workspace:
- ADR-263: `@ruvnet/ruview` npm harness deep review + optimization strategy (Proposed)
- ADR-264: `@ruvnet/rvagent` MCP server + `@ruv/ruview-cli` deep review + optimization strategy (Proposed)
- ADR-265: RuView npm distribution strategy — CI gate, provenance, version single-sourcing (Proposed)
- ADR-273: Unified RF spatial world model — umbrella + anti-leakage evaluation protocol + acceptance gates (Accepted — P1 implemented in `ruview-unified`)
- ADR-274: Universal RF foundation encoder + hardware adapter registry (Accepted — P1 implemented)
- ADR-275: RF-aware Gaussian spatial memory — fusion, decay, channel-gain queries, inverse updates, task-gated scene graph (Accepted — P1 implemented)
- ADR-276: Physics-guided synthetic RF world generator — randomize physics, not textures (Accepted — P1 implemented)
- ADR-277: Edge sensing control plane — purposes/zones/retention/identity double-gate; raw RF unexportable (Accepted — P1 implemented)
- ADR-278: Radar inverse rendering + differentiable RF SLAM research program — RISE/DiffRadar/GeRaF reproduction gates (Proposed)
- ADR-279: Native RF frame contract — `RfFrameV2` authoritative, canonical tensor demoted to derived view; 7 invariants; split manifest with session dimension (Accepted — implemented)
- ADR-280: Active sensing & programmable perception — sensing tasks/actions, AoI freshness scheduler (95% traffic reduction measured), fail-closed coherent-aperture fusion, governed RIS actuation, task-sufficient representations (Accepted — implemented)
- ADR-281: BLE Channel Sounding (phase vs RTT cross-validated ranging), delay-Doppler-native tensors, IEEE P3162 import profile, RePos factorized pose (Accepted — implemented)
- ADR-282: Ecosystem positioning — RuView as edge RF perception runtime; RuField/RuVector/MetaHarness layering; mandatory L0L5 evidence ladder (Accepted)
### Supported Hardware
+2 -1
View File
@@ -638,11 +638,12 @@ Verify the plugin structure: `bash plugins/ruview/scripts/smoke.sh`. Full detail
| [Semantic Primitives — Precision/Recall](docs/integrations/semantic-primitives-metrics.md) | Per-primitive F1 on the held-out paired-capture set: someone-sleeping, possible-distress, room-active, elderly-inactivity-anomaly, meeting, bathroom, fall-risk, bed-exit, no-movement, multi-room. |
| [Claude Code / Codex Plugin](plugins/ruview/README.md) | The `ruview` plugin + marketplace — skills, `/ruview-*` commands, agents, and the Codex prompt mirror |
| [Portable harness — `npx @ruvnet/ruview`](harness/ruview/README.md) | MetaHarness-minted, host-portable RuView operator harness — `ruview.*` MCP tools + the MEASURED-vs-CLAIMED honesty guardrail enforced in code ([ADR-182](docs/adr/ADR-182-npx-ruview-harness-via-metaharness.md)). A lighter, multi-host companion to the in-repo plugin. |
| [Architecture Decisions](docs/adr/README.md) | 182 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Architecture Decisions](docs/adr/README.md) | 205 ADRs — why each technical choice was made, organized by domain (hardware, signal processing, ML, platform, infrastructure) |
| [Domain Models](docs/ddd/README.md) | 8 DDD models (RuvSense, Signal Processing, Training Pipeline, Hardware Platform, Sensing Server, WiFi-Mat, CHCI, rvCSI) — bounded contexts, aggregates, domain events, and ubiquitous language |
| [rvCSI — edge RF sensing runtime](https://github.com/ruvnet/rvcsi) | Rust-first / TypeScript-accessible / hardware-abstracted CSI runtime: multi-source ingestion (incl. real nexmon_csi `.pcap` from a **Raspberry Pi 5** / Pi 4 / Pi 3B+ — CYW43455 / BCM43455c0) → validation → DSP → typed events → RuVector RF memory ([ADR-095](docs/adr/ADR-095-rvcsi-edge-rf-sensing-platform.md), [ADR-096](docs/adr/ADR-096-rvcsi-ffi-crate-layout.md), [domain model](docs/ddd/rvcsi-domain-model.md)). Now its own repo — [`ruvnet/rvcsi`](https://github.com/ruvnet/rvcsi) — vendored here under `vendor/rvcsi`; 9 `rvcsi-*` crates on crates.io, `@ruv/rvcsi` on npm, plus a Claude Code plugin. |
| [Desktop App](v2/crates/wifi-densepose-desktop/README.md) | **WIP** — Tauri v2 desktop app for node management, OTA updates, WASM deployment, and mesh visualization |
| `ruview-swarm` | Drone swarm control system (ADR-148) — hierarchical-mesh topology, Raft consensus, MARL, CSI sensing payload, MAVLink/PX4/ArduPilot compatibility, Ruflo AI-agent integration |
| `ruview-unified` | Unified RF spatial world model ([ADR-273](docs/adr/ADR-273-unified-rf-spatial-world-model.md)..[277](docs/adr/ADR-277-edge-sensing-control-plane.md)) — canonical RF tensor + hardware adapters (WiFi CSI / FMCW radar / UWB / 5G SRS), universal RF foundation encoder with ≤1% task adapters, RF-aware Gaussian spatial memory with channel-gain queries + inverse updates, physics-guided synthetic RF worlds, and an 802.11bf/ETSI-ISAC-aligned sensing policy plane (raw RF structurally unexportable). All accuracy numbers SYNTHETIC until real-data validation. |
| [Medical Examples](examples/medical/README.md) | Contactless blood pressure, heart rate, breathing rate via 60 GHz mmWave radar — $15 hardware, no wearable |
| [Extended Documentation](docs/readme-details.md) | Latest additions, key features, installation, quick start, signal processing, training, CLI, testing, deployment, and changelog |
@@ -82,6 +82,11 @@ The entity registry is a `RwLock<HashMap<EntityId, EntityEntry>>` backed by an a
`DeviceRegistry` mirrors HA's `core.device_registry` schema (version 13). Devices are identified by a set of `(id_type, id_value)` tuples (the `identifiers` field), which matches HA's pattern of accepting multiple identifier types per device (MAC address, serial number, integration-specific ID).
`DeviceEntry` and the in-memory `DeviceRegistry` are implemented. On server
startup, entity and device registry files are restored in deterministic key
order with a configurable hard row bound; malformed individual entries are
isolated and reported.
---
## 3. HA-side reference table
@@ -148,6 +148,12 @@ correctness, fail-closed write integrity, semantic-store NaN poisoning, and PII
- **Memory-DoS — `get_state_history` was unbounded.** No `LIMIT`, so a wide time window over a
high-frequency entity loaded an unbounded row set into memory. Now capped at
`MAX_HISTORY_ROWS` (1,000,000); sibling search paths were already `k`-bounded.
- **Startup state restoration.** `latest_states(limit)` selects one newest row
per entity with `(last_updated_ts, state_id)` tie-breaking, orders results by
entity ID, and caps requests at 100,000. Malformed rows are skipped with
typed warnings. `restore_latest` preserves recorded timestamps and installs
snapshots with a `homecore.restore` context before the recorder listener and
automation engine start.
- **Disk-DoS / documented-but-missing `purge`.** The README advertised `Recorder::purge`, but
no retention path existed → unbounded disk growth. Added a **transactional** `purge(older_than)`
with an **exclusive** cutoff (idempotent, no off-by-one) that deletes old `states`/`events` and
@@ -224,8 +224,9 @@ touched:
SHA-256-checks the module, Ed25519-verifies the signature against
`publisher_key`, and enforces a `PluginPolicy` trust allowlist
(secure-default rejects unsigned/untrusted/tampered modules).
- **HAP real pairing (P2)** — SRP/HKDF pairing + encrypted sessions; current
bridge is an accessory-mapping surface. **ACCEPTED-FUTURE (honestly stubbed).**
- **HAP real pairing (P2)** — **DONE (2026-07-27 addendum below).** SRP/HKDF
Pair-Setup, transcript-authenticated Pair-Verify, encrypted sessions, and
administrator-only pairing management now land as one fail-closed boundary.
- **`RunMode::Queued`/`Restart`/`max` ordering** — ~~`Single`/`Parallel` are
honored; bounded queueing, restart-kill, and `max` concurrency are not yet
wired (every non-Single mode is parallel).~~ **DONE — ADR-162 §A5.** Restart
@@ -336,3 +337,35 @@ is still delivered (old code: 5s-timeout panic).
+1 api-root accept-guard, +1 WS lag-survival), 0 failed. Workspace green.
Python deterministic proof unchanged (homecore-api is off the signal proof
path).
## Addendum — HAP cryptographic boundary completed (2026-07-27)
The P2 HAP deferral recorded above is closed as a single security boundary in
`homecore-hap`; it was not replaced with a success-shaped partial protocol.
- Pair-Setup M1-M6 uses RustCrypto SRP-6a with the RFC 5054 3072-bit group,
SHA-512 and HAP proof compatibility, followed by the specified
HKDF-SHA512, ChaCha20-Poly1305, and Ed25519 transcript construction.
- Pair-Verify M1-M4 uses ephemeral X25519, strict Ed25519 transcript
verification, and separately derived directional control keys.
- The TCP server changes to authenticated HAP record framing only after the
plaintext M4 response is written. Record lengths are authenticated, plaintext
is capped at 1024 bytes, counters are independent and monotonic, and any
authentication/replay/framing failure closes without an oracle response.
- Accessory identity, signing seed, SRP verifier, and controller pairings share
one versioned, bounded, permission-checked, atomically replaced store. The raw
setup code is disclosed only on first provisioning and is not persisted.
- Protected endpoints require an encrypted Pair-Verify session. Pairing
management rechecks current persisted administrator authority, handles the
last-admin invariant, updates mDNS paired state, and revokes live sessions.
Evidence includes a deterministic HAP SRP vector, complete in-process
Pair-Setup and Pair-Verify ceremonies, malformed/proof/transcript tests, record
tamper/replay/oversize tests, persistence lifecycle tests, and a real TCP test
that verifies Pair-Verify, accesses `/accessories` over encrypted records, then
proves replay closes the connection.
This closes the cryptographic implementation item, not the entire Apple Home
product surface. Current-Apple/MFi interoperability has not been certified;
transient/split Pair-Setup, writable/timed characteristics, resource endpoints,
and persisted AID/IID allocation remain explicitly unsupported.
@@ -2,7 +2,7 @@
| Field | Value |
|-------|-------|
| **Status** | Accepted — P1 scaffold (full conversion deferred to P2) |
| **Status** | Accepted — registry/config persistence implemented |
| **Date** | 2026-05-25 |
| **Deciders** | ruv |
| **Codename** | **HOMECORE-MIGRATE** |
@@ -44,8 +44,8 @@ files are read, how schema versions are validated, and what happens on an unknow
## 2. Decision
Ship `homecore-migrate` as a CLI + library that reads an existing HA filesystem and imports
its configuration into HOMECORE. P1 is a **scaffold**: it parses and inspects everything and
converts the entity registry; full conversion of the remaining artifacts is deferred to P2.
its configuration into HOMECORE. Registry and config-entry conversion are durable; automation
conversion and secret-reference resolution remain deferred.
### 2.1 Storage reader + versioned format gate (P1, shipped)
@@ -57,22 +57,25 @@ converts the entity registry; full conversion of the remaining artifacts is defe
unknown `minor_version` is a **hard error** (`MigrateError::UnsupportedSchemaVersion`),
never a silent best-effort parse. Better to refuse than to corrupt.
### 2.2 Per-artifact parsers (P1, shipped)
### 2.2 Per-artifact conversion (shipped)
- `entity_registry::load()``core.entity_registry``Vec<homecore::EntityEntry>`
(ready for import).
- `device_registry::load()``core.device_registry` `Vec<DeviceImport>` (P1 diagnostic;
full conversion P2).
- `config_entries::load()``core.config_entries` → domain counts + integration names
(the format is undocumented per §6 Q5; treated diagnostically).
- `device_registry::read_device_registry()` converts the supported v13 device fields into
`homecore::DeviceEntry`; `write_device_registry()` emits an HA-compatible v13 envelope.
- `config_entries::convert_config_entries()` emits versioned `homecore.config_entries`
storage. Original rows are retained verbatim, while unsupported domains and fields produce
typed warnings instead of being discarded.
- `secrets::load_secrets()``secrets.yaml``HashMap<String, String>` (resolution P2).
- `automations::load()``automations.yaml` → count + ID/alias list (conversion P2).
### 2.3 CLI (P1, shipped)
### 2.3 CLI
- `homecore-migrate inspect <ha-dir>` previews what will be migrated (entity/device/config
counts, redacted secret/automation lists) (`src/cli.rs`, `src/main.rs`).
- `import-entities` and `export-for-sidecar` are declared but their full behaviour is P2.
- `import-entities`, `import-devices`, and `import-config-entries` write destination files and
emit one-line JSON summaries. Writes use synced same-directory temporary files and atomic
no-clobber publication; an existing destination is never implicitly replaced.
### 2.4 Structured errors (P1, shipped)
@@ -88,27 +91,25 @@ converts the entity registry; full conversion of the remaining artifacts is defe
file path and a coarse location (`serde_yaml::Error::location()`), never the scalar content.
Pinned by `secrets::tests::malformed_secrets_error_never_contains_secret_value` (asserts the
rendered error **and its full `#[source]` chain** never contain the secret value).
**Review dimensions confirmed clean with evidence:** source is never mutated (no
`fs::write`/`remove`/`create` anywhere — P1 reads source, writes nothing); paths are
**Review dimensions confirmed clean with evidence:** source is never mutated; destination
writes are explicit `--to` paths and no-clobber; paths are
user-supplied dirs joined with fixed filenames (no `..`/absolute traversal beyond the
user's own privileges); malformed/typed/truncated `.storage` JSON and YAML **error, never
panic** (every production `unwrap`/`expect` is test-only); unknown schema `minor_version`
hard-errors fail-closed; no SQL/shell/path injection surface (the tool emits diagnostics
only, persists nothing in P1).
hard-errors fail-closed; no SQL/shell injection surface.
### 2.5 Deferred to P2+ (NOT built — honestly labelled)
- Convert `config_entries` HOMECORE plugin manifests.
- Execute imported config entries (a matching HOMECORE plugin must claim the preserved domain).
- Convert `automations.yaml``homecore-automation` YAML.
- Side-by-side runtime mode (requires `homecore-recorder`, ADR-132; behind the `recorder`
Cargo feature, currently a no-op stub).
- `!secret` reference resolution in non-secrets YAML files.
### 2.6 Test evidence (as shipped)
### 2.6 Test evidence
- 21 tests (`cargo test -p homecore-migrate`) — 19 as originally shipped plus 2 added by the
2026-06 security review (`secrets::tests::malformed_secrets_error_never_contains_secret_value`,
`malformed_secrets_error_reports_location`).
- Targeted tests cover registry round trips, unknown versions, lossless unsupported config
fields/domains, malformed input, and crash-safe/no-overwrite destination behaviour.
## 3. Consequences
@@ -118,13 +119,12 @@ converts the entity registry; full conversion of the remaining artifacts is defe
schema drift fails loudly instead of corrupting an imported home.
- Reusing HA's own `.storage` and YAML formats means no intermediate export step; the tool
reads a live HA install directly.
- P1 `inspect` gives users a no-risk dry run before any write.
- `inspect` gives users a no-risk dry run before any write.
**Negative / honest limits.**
- P1 is a **scaffold**: only the entity registry is conversion-ready. Device registry,
config-entry→plugin, automation, and secret-resolution conversions are P2 and **not yet
built** — the Status field and crate docs say so.
- Imported config entries are durable but do not install or execute Python HA integrations.
- Automation conversion and secret-reference resolution are not built.
- The side-by-side recorder export depends on ADR-132 and is currently a feature-gated
no-op.
- Performance figures in the README (envelope parse < 5 ms, 1 000-entity load < 50 ms) are
@@ -0,0 +1,487 @@
# ADR-271: RuView as a Cognitum OAuth resource server
- **Status**: accepted
- **Date**: 2026-07-22
- **Deciders**: RuView maintainers
- **Tags**: auth, oauth, cognitum, security, sensing-server
- **Related**: ADR-055 (integrated sensing server), ADR-102 (edge module registry), ADR-066 (ESP32 seed pairing), cognitum-one/dashboard ADR-060 (OAuth scopes beyond `inference`), cognitum-one/meta-llm ADR-045 (Bearer at completions)
## Context
`/api/v1/*` on `wifi-densepose-sensing-server` is gated by `RUVIEW_API_TOKEN`
(`bearer_auth.rs`): a single shared secret, compared in constant time, with no
expiry, no rotation and no per-user attribution. `homecore-api` has a second,
unrelated scheme (`LongLivedTokenStore` over `HOMECORE_TOKENS`) whose own doc
comment describes it as "no expiry, no rotation, no per-user attribution yet".
That is proportionate for the ADR-055 topology — server bundled in the desktop
app, spawned as a child, localhost only. It is not proportionate for the other
deployment RuView actually has: a sensing server on a Pi or hub, reachable on a
LAN, potentially serving more than one person, exposing live presence, pose,
breathing and heart-rate data plus destructive operations (model training,
model delete, recording delete).
Cognitum operates a live OAuth 2.1 authorization server at `auth.cognitum.one`.
Users of RuView are already Cognitum account holders. The obvious question is
whether RuView can accept that identity instead of a shared string.
### The direction of the integration is the thing most likely to be misread
Every existing Cognitum OAuth integration in the org — meta-proxy, musica,
metaharness, the dashboard CLI — is an OAuth **client**: it obtains a token so
the application can *call* a Cognitum service (the completions plane).
RuView is the opposite. It makes **no authenticated calls to any Cognitum API**.
Its only outbound Cognitum dependency is the ADR-102 registry fetch, which is an
anonymous GET against a public GCS bucket. What RuView wants is to be a
**resource server**: a user signs in to their *own* RuView instance with their
Cognitum identity, and RuView verifies the token they present.
So the client-side prior art in the org, while useful for a future `ruview
login` command, addresses a plane RuView does not have. The only relevant
precedent is `meta-llm/src/auth/oauthBearer.ts` (ADR-045) — the org's sole
resource-server-side verifier of these tokens. It is TypeScript; **RuView is the
first Rust one.**
### Facts about the tokens, verified against a live production token
- **ES256 JWT**, signed by a single P-256 key published at
`https://auth.cognitum.one/.well-known/jwks.json`.
- **15-minute lifetime**, with an opaque refresh token that **rotates with reuse
detection** (presenting a spent one ends the session).
- Claims: `typ`, `sub`, `account_id`, `org_id`, `workspace_id`, `client_id`,
`scope`, `family_id`, `jti`, `iat`, `exp`, `setup`, `workload`.
- **No `aud` claim.** No `/oauth/introspect`. No `/userinfo`. It is an OAuth 2.1
authorization server, not an OpenID Provider, deliberately.
## Decision
Verify Cognitum access tokens **offline**, in a new `ruview-auth` crate, and
gate RuView's own API surface on the **scope** they carry.
### 1. Offline verification is a requirement, not an optimisation
RuView runs on Pi-class hardware that loses WAN, and there is no introspection
endpoint to call even when the network is up. Verification is therefore an
ES256 signature check against a `kid`-indexed JWKS cache. Two consequences we
accept explicitly:
- **Revocation window = token lifetime.** A compromised access token stays
usable until `exp`. This is the same position meta-llm takes, for the same
reason, and it is why §3 refuses long-lived credentials.
- **A JWKS refetch failure is survivable while a key set is cached.** A key that
verified a minute ago has not stopped being valid because the network blipped;
failing closed there would log every user out of their own sensing server
whenever their internet wobbled. We fail closed in exactly one case: no key
set has *ever* been fetched.
### 2. The accept-rule is ported from meta-llm, not designed
```
typ == "access" AND NOT setup AND NOT workload
AND account_id is a non-empty string
AND exp is in the future
AND the scope required by the route is held
```
**Note there is no `iss` check.** An earlier revision of this section listed
"`iss` matches the configured issuer verbatim" — that rule was implemented,
shipped, and rejected EVERY real token, because Cognitum access tokens carry no
`iss` claim (see §"Facts about the tokens" above, which contradicted this
paragraph for a day). Removed in the code; removed here. The JWKS is the issuer
binding.
Divergence from `oauthBearer.ts` would be a bug rather than a preference: a
token meta-llm rejects must not be one RuView accepts. The algorithm is **fixed
to ES256 by our code** — the header's `alg` is only ever compared against that
allowlist, never used to select an algorithm.
### 3. Long-lived setup and workload credentials are refused outright
Identity also issues 365-day *setup* and machine *workload* credentials. Their
revocation state lives in identity's `oauth_setup_tokens` table. RuView — like
meta-llm — has no database and no way to check it, so accepting one would mean
honouring a credential that may already have been revoked. A 15-minute token
needs no revocation round-trip because it expires faster than revocation
propagates; a 365-day one does.
### 4. Scope is the capability boundary, because nothing else can be
Tokens carry no `aud`, so RuView cannot verify a token was minted *for* RuView.
`client_id` cannot substitute: clients borrow each other's registrations when
their own has not been deployed (musica ships `DEFAULT_CLIENT_ID = "meta-proxy"`).
This is not a defect to route around. Cross-product **identity** is intended —
one Cognitum account, every Cognitum product. Cross-product **capability** is
not, and scope is what carries the difference.
RuView registers two scopes (dashboard ADR-060, identity migration `0016`):
| Scope | Grants |
|---|---|
| `sensing:read` | sensing/pose streams, one-shot inference, reading model and recording metadata |
| `sensing:admin` | every mutating route not explicitly allowlisted as read-safe — training (`/api/v1/train/*` AND `/api/v1/adaptive/train`), model and recording deletion, config writes |
**The gate is fail-closed for writes, and that polarity is load-bearing.** An
earlier revision enumerated admin routes by prefix and let everything else fall
through to `sensing:read`. `POST /api/v1/adaptive/train` — which trains a
classifier, overwrites the on-disk model and swaps the live one — does not match
`/api/v1/train/`, so it was reachable with `sensing:read`, the scope
`wifi-densepose login` requests by default. Found by adversarial review. Now:
reads are open, writes require admin unless the exact path is on a short
allowlist of non-destructive mutations. A route added tomorrow is admin-gated
until someone classifies it.
**No hierarchy**: `sensing:admin` does not imply `sensing:read`. Consent means
exactly what it said, and a token needing both must have consented to both.
`client_id` is retained on the principal for logging and attribution only —
never as an authorization input.
### 5. Additive and fail-closed, never a silent downgrade
`RUVIEW_API_TOKEN` and `HOMECORE_TOKENS` deployments keep working unchanged.
OAuth is opt-in; with it unconfigured, behaviour is byte-identical to today.
When OAuth *is* configured but unusable (JWKS unreachable at boot, required
scope not registered), the server must refuse to serve `/api/v1/*` rather than
fall through to an open or single-secret state.
### 6. `ureq`, and a transport seam
`wifi-densepose-sensing-server` deliberately chose `ureq` as "the smallest" HTTP
client. Introducing `reqwest` for a JWKS fetch would silently reverse that for
the whole dependency graph. The fetch sits behind a `JwksFetcher` trait — the
`ureq` implementation is a default-on feature, and a host may supply its own and
take no HTTP dependency at all.
## Consequences
- Requests become attributable: `sub`, `account_id`, `org_id`, `workspace_id`,
`jti`. This closes the gap `homecore-api`'s `tokens.rs` has been deferring as
"P3", using claims rather than new RuView machinery.
- Destructive operations can be separated from observation for the first time.
- **The 15-minute lifetime is the main operational cost.** A long-running client
must refresh, and because refresh tokens rotate with reuse detection, a
concurrent or naively retried refresh **ends the session** — single-flight is a
correctness requirement, not an optimisation. This lands with the login flow,
not this crate.
- Hosts without a battery-backed clock will fail `exp`/`iat` until NTP lands.
The verifier reports that distinguishably so it is diagnosable rather than
presenting as a generic 401.
- A new dependency, `jsonwebtoken` — the same crate, same major version, that
identity itself uses to sign these tokens.
## ~~Known incomplete: the browser cannot obtain an OAuth token~~ — CLOSED 2026-07-23
> **Superseded within this same PR.** The text below described the state when
> this ADR was first written. It is retained because the reasoning still
> explains *why* the browser half was built, but every factual claim in it is
> now false — in particular `grep -ril "oauth|cognitum|pkce" ui/` now returns
> `ui/sw.js`, `ui/sw.test.mjs` and `ui/utils/quick-settings.js`. An adversarial
> review caught the ADR still asserting the old state; see "Browser sign-in"
> below for what actually ships.
<details>
<summary>Original text (no longer accurate)</summary>
`wifi-densepose login` writes to `~/.ruview/credentials.json` — a file a browser
cannot read. The UI's `ws-ticket.js` reads a bearer from
`localStorage['ruview-api-token']`, which is populated **only** by the
QuickSettings manual-paste panel. There is no "Sign in with Cognitum" control,
no redirect flow, and `grep -ril "oauth|cognitum|pkce" ui/` returns nothing.
So a user who signs in via the CLI gets **no benefit in the browser UI**, and
the WebSocket ticket mechanism this ADR's sibling (ADR-272) introduces "for
browsers" is today only exercisable with the legacy static shared secret that
OAuth was meant to replace. The server-side gating is correct and complete; the
browser half of the story these ADRs tell is not built.
</details>
## Browser sign-in
`/oauth/start`, `/oauth/callback`, `/oauth/logout` and `/oauth/status`, plus a
"Cognitum Account" panel in QuickSettings. The server runs the authorization
code + PKCE flow itself and hands the browser a **signed session cookie**
never the access token. The browser gets an assertion that this server already
verified a token, which is nothing replayable anywhere else.
Three things about it are load-bearing and were each found the hard way:
- **The cookie carries the granted scope**, and the gate re-checks it per
request. A `sensing:read` session cannot delete a model.
- **`__Host-` is deliberately NOT used.** That prefix requires `Secure`, and
RuView is routinely reached over plain HTTP on a LAN; a cookie the browser
refuses to set is worse than one without the prefix. The cost is real and is
recorded as P3 under "Open problems" below.
- **The service worker must never cache `/oauth/*` or authenticated `/api/*`.**
The Cache API is not the HTTP cache and ignores `Cache-Control` entirely, so
a cached `/oauth/status` froze sign-in until a hard reload, and cached API
responses could be replayed to a different user after sign-out. `ui/sw.js` is
now deny-by-default with an allowlist.
### Still incomplete
`redirect_uri` defaults to `http://127.0.0.1:8080/oauth/callback` and is
overridden only by `RUVIEW_PUBLIC_BASE_URL`. Browser sign-in therefore works
only on a host reached at exactly that origin: an operator browsing
`http://localhost:8080` or `http://192.168.1.50:8080` cannot complete the flow
(PKCE keeps the code unexchangeable, so this is a broken flow, not a token
leak). Deriving it from the request is the fix; deferred deliberately, since
deriving a redirect URI from attacker-controllable headers is its own class of
bug and deserves its own decision.
The credential `wifi-densepose login` stores is also **not yet consumed by any
shipped client** — no CLI subcommand, MCP server or Python client reads
`~/.ruview/credentials.json`. The token is obtainable and verifiable; wiring the
clients to send it is separate work.
## Open problems — RESOLVED 2026-07-23
Three findings from the 2026-07-23 adversarial review. All three are now
**fixed**; the analysis is retained because it explains why each fix has the
shape it does, and each is guarded by a test that was confirmed to fail against
the old behaviour.
### P1 — the JWKS fetch blocks a tokio worker, and the stale path is unbounded — **FIXED**
`verify.rs:182` calls `JwksCache::decoding_key_for`, which performs a blocking
`ureq` request (`jwks.rs:181`, 3s connect + 3s read) directly on the async
worker running `require_bearer`. The same codebase already knows this is wrong:
`main.rs:9265` wraps the token exchange in `spawn_blocking`, commenting "the
same mistake this codebase had to fix in `jwks.rs`". The hot verification path
did not get the same treatment.
Worse, the rate limiter does not cover the case that matters.
`state.fetched_at` is updated **only on success** (`jwks.rs:188`); the error arm
leaves it untouched. So once the TTL elapses after the last *successful* fetch,
`fresh` is permanently `false`, the `may_force` guard at `:170` is never
consulted, and **every** request performs its own blocking fetch attempt.
This fires with no attacker present. On a Pi that loses WAN — the documented
deployment reality — 300 seconds later every API call and every UI poll starts a
blocking outbound attempt, and with few tokio workers the whole server stalls,
including `/health`. An attacker can reach the same state deliberately by
flooding tokens carrying an unknown `kid`.
**Proposed fix, in dependency order:**
1. **Rate-limit attempts, not successes.** Add `last_attempt_at`, recorded
before the fetch regardless of outcome, and consult it on the stale path too.
This alone converts "every request fetches" into "one request per interval".
2. **Get the blocking call off the runtime.** Either wrap the call in
`spawn_blocking` at the `verify` boundary, or give `JwksCache` an async
transport behind the existing transport seam. The seam already exists —
`JwksCache::new` takes a boxed transport — so this is an added
implementation, not a redesign.
3. **Single-flight the refresh.** Concurrent misses for the same `kid` should
await one shared fetch rather than each issuing their own.
4. **Refresh ahead of expiry** from a background task, so the request path
normally never fetches at all.
Steps 1 and 2 are the ones that remove the stall; 3 and 4 are optimisations.
The test that must accompany this: a transport whose fetch blocks on a barrier,
asserting that a second concurrent verification is not serialised behind it —
the current suite is entirely single-threaded and could not observe a
reintroduction (`jwks::tests` contains no concurrency primitive at all).
### P2 — a 15-minute access token becomes a 12-hour session — **FIXED**
`issue()` sets `exp: now() + SESSION_TTL_SECS` with `SESSION_TTL_SECS = 12 *
3600`, deliberately not inheriting the access token's ~15-minute lifetime. The
session cookie is an assertion that this server verified a token, so it is not
*wrong* for it to outlive the token — but 12 hours is a long time to hold an
authority that cannot be revoked. Cognitum publishes no introspection endpoint
(see "Facts about the tokens"), so RuView has no way to ask whether the grant
behind a session still stands. A disabled account keeps sensing access, and
`sensing:admin` if it had it, until the cookie expires on its own.
**Correction.** An earlier revision of this section said capping the session at
`sensing:read` was "considered and rejected, because the dashboard genuinely
performs admin operations". That was wrong, and a cross-vendor pre-merge sweep
caught it: `/oauth/start` (`main.rs:9206`) already requests `SENSING_READ` and
nothing else, deliberately — "admin work goes through the CLI, which requires an
explicit `--admin`". So a browser session is **already** read-only, and the
consequence I claimed capping would cause is simply the current behaviour.
Two things follow, and both are stated here rather than left for the next reader
to trip over:
1. **The UI's admin controls do not work from a browser OAuth session.**
`model.service.js:136` issues `DELETE /api/v1/models/{id}`; from a
Cognitum-signed-in browser that returns 401. Admin work requires either the
CLI (`wifi-densepose login --admin`) or a manually pasted admin bearer in the
QuickSettings token field. This is a gap in the browser feature, not a
regression — browser sign-in is new here, and the token-paste path still
carries whatever authority the pasted token has.
2. **The step-up control below is therefore a guard ahead of need, not an active
one.** No browser session currently holds `sensing:admin`, so
`session.has_scope(SENSING_ADMIN)` is false and the freshness branch never
fires in production. Its tests pass because the crate-internal test seam
mints an admin cookie the real flow does not produce. That is worth naming
plainly: it is correct code guarding a case that cannot yet arise, and it
becomes load-bearing the moment anyone widens the requested scope — which is
the right time for the guard to already exist, but it is not evidence that
the control is exercised today.
### Decision, 2026-07-23: the browser is read-only, permanently
**Browser-side admin is not wanted.** `BROWSER_SIGNIN_SCOPE` stays
`sensing:read`, and the escalate-on-demand design sketched while this was still
open is **not** being built.
The reasoning holds up on its own terms rather than being a concession to
scope: the destructive operations — training, model delete, recording delete —
already have a home in the CLI, where `--admin` is explicit, typed by a person,
and scoped to the session that needed it. Routing them through a browser would
mean either asking every user to consent to delete capability in order to watch
a stream, or building a second consent flow to avoid that. Neither is worth it
for operations that are administrative by nature and rare by frequency.
What this settles:
- **The UI's admin controls are unreachable from a Cognitum browser session**
and that is now intended, not a gap. `model.service.js` issuing
`DELETE /api/v1/models/{id}` returns 401. The manual token-paste field still
works and carries whatever authority the pasted token has, so nothing that
worked before this change stops working.
- **The client-side step-up redirect has been removed** from
`ui/services/api.service.js`. It caught a challenge that can never be issued,
and it ended in a promise that never settles — so had any other 401 ever grown
that header, every caller would have hung forever. Dead code with a trap in it
is worse than no code.
- **`ADMIN_REVERIFY_SECS` stays as a server-side backstop.** It is fail-closed
and costs nothing, so if the requested scope is ever widened the freshness
requirement is already there rather than something to remember. It is
documented at its definition as a backstop, so nobody mistakes its passing
tests for evidence that it is exercised.
**Three options, with the tradeoff each carries:**
| Option | Effect | Cost |
|---|---|---|
| **A. Shorten the TTL** (e.g. 12h → 4h) | Bounds exposure by a factor of 3, one constant | Re-auth is a full-page navigation, which interrupts a live streaming dashboard. Mostly silent while the Cognitum session is alive, but not free. |
| **B. Server-side session store** with the refresh token, revalidated periodically | Real revocation: a disabled grant fails at the next refresh | The server now stores refresh tokens — a new and higher-value secret at rest — and refresh rotates with reuse detection, so a bug logs users out. |
| **C. Re-verify on privileged operations only** | `sensing:admin` requires a fresh token; reads keep the long session | Best blast-radius-per-unit-cost, but needs a UI affordance for step-up auth that does not exist. |
**Chosen: A, at one hour**`SESSION_TTL_SECS` is 3600, down from 12 hours.
C was implemented too, and then the browser-read-only decision above made it a
backstop rather than an active control: with no browser session holding
`sensing:admin`, there is no privileged operation to re-verify. It is kept
because it is fail-closed and free, not because it is doing work today.
B is not built. It is only worth its cost — storing refresh tokens at rest,
against an authorization server that rotates them with reuse detection — if
RuView later needs true cross-device sign-out. Shortening the window addresses
the same risk for a fraction of the exposure.
That leaves a residual this ADR should not pretend away: **within one hour, a
revoked Cognitum grant still reads sensing data through an existing browser
session.** Cognitum publishes no introspection endpoint, so nothing short of B
closes that, and one hour is the size of the hole we accepted.
### P3 — dropping `__Host-` costs cookie origin-integrity, not just `Secure` — **FIXED**
The decision above frames omitting `__Host-` as trading away a `Secure`
requirement that RuView cannot meet on a plain-HTTP LAN. That framing is
incomplete: `__Host-` also guarantees the cookie was set by *this* origin with
`Path=/` and no `Domain`. Without it, cookies are not port-scoped and are not
integrity-protected against a same-host writer.
`read_cookie` returns the **first** match in the header, and RFC 6265 §5.4 sends
longer-`Path` cookies first. So an attacker who can set a cookie on the same
host — any other service on any port on that appliance, or a plain-HTTP MITM
injecting `Set-Cookie` — can plant `ruview_session=<their own validly signed
session>; Path=/ui`. The victim's browser then sends both, the attacker's first,
and it verifies correctly because it *is* genuinely signed. The victim ends up
operating inside the attacker's session; `/oauth/status` reports the attacker's
account, and anything the victim records is attributed to them.
Note the shape: the signature is doing its job. Forgery was never the threat
`__Host-` addresses, so "the signature is what protects the value" does not
answer this.
**Proposed fix (cheap, no prefix needed):** have `read_cookie` collect *all*
values for the name and accept only if exactly one verifies — or, more strictly,
reject outright when more than one `ruview_session` is present, since a browser
should never legitimately send two. Add `Secure` and the `__Host-` prefix
conditionally when the server knows it is behind TLS, keeping the plain-HTTP LAN
case working.
## Alternatives considered
**Keep `RUVIEW_API_TOKEN` only.** Zero work, and adequate for a single-user
localhost install. Rejected because it cannot express who did what, cannot be
revoked without a restart, and cannot separate "watch the stream" from "delete
the model" — all of which matter the moment the server is on a LAN.
**Exchange the OAuth token for a `cog_` key.** The pattern ADR-316 (meta-proxy)
and ADR-119 (metaharness) originally described. Rejected: it cannot work.
`/v1/me/keys` requires a *Firebase* ID token, not an OAuth token — meta-proxy
hit the resulting 401 in production, replaced the approach with Bearer-direct
under ADR-045, and deleted `mint.rs` as dead code.
**Call identity to introspect each token.** Rejected: no introspection endpoint
exists, and a network round-trip per request would be wrong for an edge sensing
server regardless.
**Wait for an `aud` claim before shipping.** Rejected as sequencing. `aud` would
touch every issued token and every verifier in the org; scope is additive and
independently correct. Tracked separately; adding `aud` later strengthens this
design rather than invalidating it.
**Use OAuth for the ESP32 device plane too.** Rejected as a category error.
Devices have no browser, no user and no human present; they already pair with a
`seed_token` bearer (ADR-066) plus a device-bound PSK. Cognitum OAuth is for the
API plane only.
## Implementation
`v2/crates/ruview-auth``jwks` (fetch, TTL cache, `kid` index, one
rate-limited forced refetch on an unknown `kid` so rotation is picked up without
waiting out the TTL), `verify` (the §2 accept-rule), `principal` (the verified
caller and its scopes).
41 tests pass under both `cargo test --no-default-features` (the repo's
canonical gate) and default features. The matrix signs real ES256 tokens with a
runtime-generated key — no key material is committed — and covers `alg:none`,
forged signatures, spliced payloads, unknown `kid`, expiry on both sides of the
leeway, `typ` confusion, `setup`/`workload` smuggled onto a `typ=access` token, missing and
empty `account_id`, and scope escalation.
The load-bearing case is
`g2_a_genuinely_valid_token_from_another_cognitum_product_cannot_reach_the_sensing_surface`:
a correctly signed, unexpired, right-issuer, right-`typ` token bearing
`client_id=meta-proxy` and `scope=inference` is rejected. Nothing about its
signature or identity claims distinguishes it — only scope does. A naive
verifier accepts it, and an `inference` token becomes a key to someone's home
sensor.
**Not in this crate**: WebSocket authentication (ADR-272) and any outbound
Cognitum call.
### Amendment, 2026-07-22 — the login flow lives here after all, behind a feature
The paragraph above originally also excluded the login flow. That was written
to keep the sensing server lean, which is the right goal but not a reason to put
the code somewhere else: the Tauri desktop app needs the same flow, and a second
copy of a PKCE + rotating-refresh implementation is exactly the kind of
duplication that drifts apart and then disagrees about something subtle.
So `login` is a **non-default feature** of this crate. A server built with
default features gets the verifier and nothing more — no `reqwest`, no tokio
networking, no browser launcher. The CLI opts in with
`features = ["login"]`, and the desktop app can do the same.
Shipped as `wifi-densepose login` / `logout` / `whoami`. Two properties worth
restating because they are easy to get wrong:
* **Refresh is serialised and never retried.** Identity rotates refresh tokens
with reuse detection, so a concurrent refresh looks like replay and a retry
*is* replay — either revokes the session family. `Session::ensure_fresh`
holds an async mutex across the network call, re-checks expiry after
acquiring it, and persists the rotated token before returning it.
* **Least scope by default.** `login` requests `sensing:read`; `--admin` is an
explicit escalation and requests both scopes, since there is no hierarchy
server-side.
@@ -0,0 +1,185 @@
# ADR-272: WebSocket authentication tickets
- **Status**: accepted
- **Date**: 2026-07-22
- **Deciders**: RuView maintainers
- **Tags**: auth, websocket, security, sensing-server
- **Related**: ADR-271 (Cognitum OAuth resource server), ADR-055 (integrated sensing server), PR #1313 (the exemption this supersedes), cognitum-one/dashboard ADR-060
## Context
`bearer_auth` gates `/api/v1/*`. WebSocket upgrade endpoints were exempt, for a
real reason: a browser's `WebSocket` constructor cannot attach an
`Authorization` header to the handshake, so a gated socket is simply
unreachable from page JavaScript. `/ws/sensing` and `/ws/introspection` sat
outside `PROTECTED_PREFIX` entirely; `/api/v1/stream/pose` was added to an
explicit `EXEMPT_PATHS` list by PR #1313.
The reasoning was sound. The consequence was not, and it was measured rather
than argued. On a server with `RUVIEW_API_TOKEN` set — an operator who believes
authentication is ON — a real WebSocket handshake carrying **no credential at
all**:
```
/ws/sensing -> 101 Switching Protocols
/ws/introspection -> 101 Switching Protocols
/api/v1/stream/pose -> 101 Switching Protocols
/api/v1/models -> 401 Unauthorized (control)
```
**The control plane was locked and the data plane was open.** `/ws/sensing`
carries the live sensing output — presence, pose, breathing and heart rate.
`/ws/introspection` exposes internal pipeline state. For the ADR-055 desktop
topology (server bundled in the app, loopback only) that is bounded. For the
LAN/hub deployment RuView also supports, anyone who can reach the port can
watch the sensor.
ADR-271 sharpened the contrast rather than causing it: the REST surface is now
genuinely strong — offline-verified Cognitum tokens, scope-separated
destructive routes — which makes an ungated data plane the obvious way in.
*Precision about the evidence:* the handshake completing was verified. A
payload frame was not captured in that window, so the finding is "the
connection is established without a credential", not "data was read".
## Decision
Gate every WebSocket upgrade. Accept **either** of two credentials, chosen to
match what each kind of client can actually do.
### 1. Native clients send a bearer on the upgrade
The Python client, the Rust CLI and the TypeScript MCP client are not browsers
and have never been subject to the header limitation. They **can** send a normal
`Authorization: Bearer` on the handshake, so the server accepts one there;
routing them through a ticket would add a round-trip and a second credential
path for no benefit.
> **Correction, 2026-07-23.** This section previously stated that those clients
> **do** send a bearer. The published Python client does not:
> `python/wifi_densepose/client/ws.py` calls `websockets.connect(url,
> ping_interval, ping_timeout, max_size)` and passes no headers at all — the
> file contains zero occurrences of `extra_headers` or `Authorization`. So every
> `wifi-densepose[client]` consumer **401s the moment an operator enables
> auth**, and this ADR told them they would be fine.
>
> The server side of the decision stands — a bearer on the upgrade is accepted,
> and that is the right contract for a non-browser client. What is missing is
> the client implementing it, tracked as ruvnet/RuView#1395. Until then the only
> remedy available to those users is
> `RUVIEW_WS_LEGACY_UNAUTHENTICATED=1`, which reopens the exposure this ADR
> exists to close — so it is a migration aid with a deadline, not an answer.
### 2. Browsers exchange their credential for a single-use ticket
`POST /api/v1/ws-ticket` is an ordinary authenticated request — where headers
*do* work — and returns an opaque ticket the page appends as
`?ticket=<value>` on the socket URL.
**A credential in a URL is normally a mistake.** URLs reach access logs,
`Referer` headers and browser history. Three properties bound this one, and all
three are load-bearing:
| Property | Why it matters |
|---|---|
| **Single use** — consumed on the first upgrade attempt, valid or not | A ticket found in a log is already spent |
| **~30 second TTL** | Long enough to open a socket; not long enough to harvest |
| **Not the credential** — authorizes one WebSocket | Cannot be replayed against `/api/v1/*`, cannot be refreshed, carries no reusable identity |
The long-lived bearer token is still never placed in a URL.
A ticket **inherits the issuing principal's scopes**, so a `sensing:read`
session cannot mint one that outranks itself, and a ticket from a token without
`sensing:read` is refused at the upgrade.
### 3. WebSocket paths are matched by **prefix**, not by an allowlist
Anything under `/ws/` is treated as an upgrade path, plus the one endpoint that
lives outside it (`/api/v1/stream/pose`).
This is the most important detail in the ADR. An allowlist means every
WebSocket route added later is ungated until someone remembers to extend it —
the same bug, reintroduced on a delay. It is not hypothetical:
`/ws/train/progress` (ADR-186, arriving with PR #1387) is already referenced by
`ui/services/training.service.js` and would have shipped unauthenticated under
an allowlist. Prefix matching gates it on arrival.
New WebSocket routes should live under `/ws/` and inherit gating for free.
### 4. A migration escape hatch, deliberately uncomfortable
`RUVIEW_WS_LEGACY_UNAUTHENTICATED=1` restores the previous behaviour. Gating
these paths **breaks a browser UI that has not yet been updated to fetch a
ticket**, and not every deployment can update server and UI in lockstep.
It is a migration aid, not a supported configuration:
- It logs a warning on every boot naming the actual exposure — "the live
sensing stream — presence, pose and vital signs — is readable by anyone who
can reach this port" — rather than something an operator can skim past.
- Its blast radius is exactly the WebSocket paths. A test pins that it does not
weaken `/api/v1/*`.
- It is read **once at construction**, so changing the environment cannot
silently open the paths on a running server.
The alternative — a clean break with no hatch — was considered and rejected as
sequencing, not principle: a hard break tempts an operator into turning auth off
entirely, which is strictly worse than a narrow, loudly-announced exception.
The hatch should be removed once the shipped UI fetches tickets.
### 5. Deployments with auth off are unchanged
No credential configured ⇒ the middleware is the same no-op it has always been.
Pinned by a test.
## Consequences
- The measured hole is closed: all three paths now return `401` to a
credential-less handshake, while a bearer or a valid ticket returns `101`.
- Browser UIs need updating. Shipped in the same change for
`sensing.service.js`, `websocket-client.js` and `observatory/js/main.js` via
a shared `withWsTicket()` helper; a ticket is minted per connection attempt
and never cached, because it is single-use and short-lived.
- A UI running against a server that predates this ADR still works: the helper
treats `404` from `/api/v1/ws-ticket` as "no ticket needed".
- One more round-trip before a browser opens a socket. Negligible against a
stream that then runs for minutes.
- Tickets live in memory, capped at 512 outstanding and self-healing as they
expire, so an authenticated but misbehaving caller cannot grow the store
without bound. In-memory is correct rather than convenient: a ticket
surviving a restart would outlive the server that vouched for it.
## Supersedes
PR #1313's `enabled_exempts_pose_stream_websocket`, which asserted the
exemption. Its premise about browsers was correct and is preserved here; its
conclusion is replaced. The test was renamed and inverted rather than deleted,
with the history in its doc comment, and the half that still matters — the
WebSocket rule must not leak to other `/api/v1/*` paths — is kept.
## Deliberately not done
- **`/health*` stays ungated.** Orchestrator probes hit it anonymously, and
that is the point of a liveness endpoint. `/health/metrics` is included in
that exemption; if metrics ever carry occupancy-derived values this should be
revisited, because that would make them sensing data wearing an ops label.
- **`/ui/*` stays ungated.** It is static assets; the data behind them is
gated.
- **No revocation of an issued ticket.** It expires in seconds and is
single-use; a revocation path would be more machinery than the exposure
justifies.
- **No ticket for native clients.** They can send a header, so they should.
## Implementation
`v2/crates/wifi-densepose-sensing-server/src/ws_ticket.rs` (store),
`src/bearer_auth.rs` (gating), `src/main.rs` (`POST /api/v1/ws-ticket`),
`ui/services/ws-ticket.js` plus the three call sites.
Tests: 12 store, 9 gating, 4 path-matching. Store coverage includes single-use
enforcement, replay refusal, expiry refusal *and* pruning, 256-bit
unpredictability, cap enforcement and self-healing, and `?myticket=x` not being
read as `?ticket=x`. Gating coverage includes every known WS path refusing an
unauthenticated upgrade, bearer acceptance, ticket single-use, a ticket being
useless against REST, the escape hatch working *and* not weakening REST, and
auth-off behaviour unchanged.
@@ -0,0 +1,123 @@
# ADR-273: Unified RF Spatial World Model — one shared representation, not another isolated RF classifier
| Field | Value |
|-------|-------|
| **Status** | Accepted — **P1 implemented** (new v2 workspace crate `ruview-unified`; 66 unit + 3 acceptance-pipeline tests, 0 failed; criterion benches) |
| **Date** | 2026-07-26 |
| **Deciders** | ruv |
| **Codebase target** | `v2/crates/ruview-unified/` (new leaf crate; single internal dep on `wifi-densepose-core` for `CsiFrame`) |
| **Sub-ADRs** | ADR-274 (universal RF encoder + adapter registry), ADR-275 (RF-aware Gaussian spatial memory), ADR-276 (physics-guided synthetic RF worlds), ADR-277 (edge sensing control plane), ADR-278 (radar inverse rendering research program) |
| **Relates to** | ADR-152 (WiFi-Pose SOTA intake: geometry conditioning), ADR-153 (802.11bf protocol model), ADR-260/262 (RuField MFS + bridge), ADR-135/136 (calibration + canonical frame provenance), ADR-024 (AETHER), ADR-027 (MERIDIAN domain generalization) |
| **Scope** | Decide the target architecture for RuView + RuVector sensing through 2026-H2: one persistent, queryable spatial world model that vision, WiFi CSI, cellular CFR/SRS, radar, geometry, semantics, uncertainty, and time all update — and the priority order for building it. |
---
## 0. PROOF discipline
Every number in this ADR family is one of:
- **MEASURED-SYNTHETIC** — produced by this repo's tests/benches on data from the ADR-276 physics generator. Reproducible: `cd v2 && cargo test -p ruview-unified` / `cargo bench -p ruview-unified`. **No claim of real-world accuracy is made or implied.**
- **MEASURED-CODE** — a structural property of the implementation (parameter counts, gradient-check error, determinism), verified by a named test.
- **EXTERNAL-UNVERIFIED** — a number reported by an external paper/preprint (WiFo-2, WiLHPE, RISE, DiffRadar, HybridSim, OAI SRS demo, …) that this repo has **not** reproduced. These motivated design choices; they are never presented as our results.
## 1. Context
Through mid-2026 the field moved decisively away from task-specific RF classifiers:
1. **RF foundation models** (WiFo-2 scaling across 11.6 B CSI points/12 tasks; WiLLM's dataset adapters + shared self-supervised transformer; age-aware CSI fusion) — the architectural signal: *standardize heterogeneous CSI, pretrain with masked reconstruction, attach small task adapters* (all EXTERNAL-UNVERIFIED).
2. **Gaussian fields as spatial memory** (EmbodiedSplat online semantic 3-D Gaussian mapping; TGSFormer bounded temporal Gaussian memory; July's physics-informed channel-gain mapping with incremental Gaussian insertion) — the missing bridge between RuView sensing and a queryable digital twin.
3. **Synthetic RF worlds** (WaveVerse phase-coherent ray tracing; HybridSim's 92 % vs 54 % synthetic-to-real gap when *physics parameters*, not textures, are randomized) — the fastest path out of data scarcity.
4. **Standards became actionable**: IEEE 802.11bf-2025 published (2025-09), 802.11bk (320 MHz positioning), ETSI ISAC architecture (2026-02) + security report (19 privacy/security issue classes), 3GPP Rel-20 sensing studies, OAI SRS xApp localization demo.
5. **Generalization lessons**: PerceptAlign (condition on TX/RX geometry), RePos (factor root-relative pose from absolute localization), JITOMA (task-gated scene memory).
RuView already has the ingredients (calibration ADR-151, canonical frames ADR-136, ruvsense multistatic stack, RuField bridge ADR-262) but they update **separate** state. The decision is to converge on **one shared representation with persistent scene memory**.
## 2. Decision
Build the unified model as five pillars in strict priority order (scored 35 % business value / 25 % readiness / 20 % defensibility / 20 % strategic learning):
| # | Pillar | Score | Sub-ADR | P1 status |
|---|--------|-------|---------|-----------|
| 1 | Universal RF foundation encoder + hardware adapter registry | 4.7 | ADR-274 | **implemented** |
| 2 | RF-aware Gaussian spatial memory | 4.5 | ADR-275 | **implemented** |
| 3 | Age/geometry/uncertainty-aware inference (folded into the encoder contract) | 4.4 | ADR-274 §3 | **implemented** |
| 4 | Physics-guided synthetic RF world generator | 4.1 | ADR-276 | **implemented** |
| 5 | Edge sensing control plane (802.11bf / ETSI ISAC aligned) | 3.9* | ADR-277 | **implemented** (policy engine; O-RAN xApp is roadmap) |
| 6 | Radar inverse rendering + differentiable RF SLAM | 3.6 | ADR-278 | research program (not implemented) |
\* the 3.9-scored item is the O-RAN SRS xApp; its *policy plane* and its *SRS adapter seam* ship in P1 because they are cheap and gate everything else.
The representation contract every pillar shares:
```text
z = Encoder(RF tokens) ⊙ σ(AgeEncoder(age)) + GeometryEncoder(sensor_pose)
```
served from one canonical tensor (`RfTensor`, ADR-274 §2) and persisted into one scene memory (`GaussianMap` + task-gated `SceneGraph`, ADR-275).
## 3. Architecture (implemented, `v2/crates/ruview-unified/src/`)
```text
vendor captures ──▶ adapters.rs (WiFi CSI / FMCW cube / UWB CIR / 5G SRS)
│ normalize: layout → gain → phase (ADR-274 §2.3)
tensor.rs RfTensor (links × 56 bins × 8 snapshots, complex)
tokenizer.rs amplitude/delay/Doppler/phase/age/geometry/
│ clock/uncertainty tokens (CFO-aligned,
│ median-scale-normalized)
encoder.rs + pretrain.rs masked-reconstruction pretraining,
│ exact hand-derived backprop (gradient-checked)
┌── heads.rs ≤1 % task adapters (presence/activity/localization/anomaly)
├── gaussian/ RF-aware Gaussian memory: fusion, decay, channel-gain
│ queries, inverse updates, task-gated scene graph
└── policy.rs purposes/zones/retention/identity gating; BoundedEvent
is the only exportable type (raw RF unrepresentable)
```
`synth/` (ADR-276) generates the labeled physics worlds that train and gate all of it; `eval.rs` implements the anti-leakage protocol below.
## 4. The non-negotiable evaluation protocol (anti-leakage)
The biggest failure mode in this field is **domain leakage disguised as accuracy**: random frame splits let a model recognize the room, session, person, device, or trajectory. Bigger models make it worse. Therefore:
- **No result counts unless the test set holds out complete** rooms, days, people, chipsets, firmware versions, and antenna layouts. `eval::StrictSplit` constructs such splits and `verify()` independently proves disjointness (`eval.rs`; test `verify_catches_a_manufactured_leak`).
- Track **relative degradation** known→unknown (`relative_degradation`, gate < 20 %), **calibration** (`expected_calibration_error`), and **abstention quality** (`selective_metrics` — an uncertain result must become *no decision*, not a confident guess).
- Every synthetic number is labeled SYNTHETIC in test output and in these ADRs.
## 5. Acceptance gates — P1 (synthetic analogue) results
The ADR's acceptance test (frozen shared encoder, adapters < 1 % of backbone, unseen rooms/chipsets/layouts) is implemented end-to-end in `tests/e2e_acceptance.rs`. **MEASURED-SYNTHETIC** results on the ADR-276 generator (8 rooms × 20 windows × 3 links, seed 273273):
| Gate (ADR target) | P1 synthetic result | Verdict |
|---|---|---|
| Presence F1 ≥ 0.90, unseen rooms | **1.0000** (rooms 67 held out of pretraining *and* head training) | pass |
| Presence F1 ≥ 0.90, unseen chipset | **1.0000** (`chip-2` held out; per-room random gain/phase/CFO/noise) | pass |
| Cross-environment degradation < 20 % | **0.0000** | pass |
| Adapter budget < 1 % of backbone | presence 129 / activity 268 / localization 387 / anomaly 2 params vs 40,856-param backbone (< 408) | pass (MEASURED-CODE) |
| Edge latency p95 < 50 ms | **2.0 ms** debug profile (tokenize+encode); 105 µs encode / 67 µs tokenize release (criterion) | pass |
| Held-out ECE | **0.0122**; abstention risk monotone in threshold | pass |
| Raw RF never crosses the trust boundary | structural: only `policy::BoundedEvent` exports (no tensor-carrying variant exists) | pass |
| Every output carries uncertainty, provenance, model version, purpose | enforced at `BoundedEvent::new` (construction fails otherwise) | pass |
**Honest reading**: a synthetic world where presence ⇔ a moving scatterer is *separable by construction*; F1 = 1.0 here validates the **pipeline and the anti-leakage machinery**, not real-world performance. The real-data gate (5 unseen rooms, 2 unseen chipsets, 2 unseen layouts, measured CSI) is P2 and remains open.
## 6. Consequences
- RuView gains a single, tested substrate that all future sensing work (vision fusion, SRS xApp, radar) updates instead of forking.
- The synthetic-first discipline means every accuracy claim is grade-labeled; publishing an unlabeled number is now a process violation.
- The Gaussian memory becomes the integration point for RuVector (vector retrieval → graph constraints → geometric verification; the LLM plans the query, the renderer verifies the answer).
- Cost: a new crate to maintain (~4.6 k lines incl. tests); mitigations: zero heavy deps, deterministic tests, files < 500 lines each.
## 7. Roadmap after P1
| Phase | Content | Gate |
|-------|---------|------|
| P2 | Replay real `.csi.jsonl` (rvCSI / ADR-262 corpus) through the WiFi adapter; calibrate the anomaly head on real empty-room captures | strict-split F1/ECE on measured data, reported with degradation vs synthetic |
| P3 | Wire `GaussianMap` into `wifi-densepose-sensing-server` behind the ADR-277 boundary; RuVector embedding of Gaussian clusters | live map consistency + bounded-event-only egress audit |
| P4 | OAI SRS xApp feeding `CellularSrsAdapter` (the adapter + registry seam already exists) | 0.5 m p90 localization under *non-random* splits |
| P5 | ADR-278 radar inverse rendering reproduction (RISE first) |
@@ -0,0 +1,95 @@
# ADR-274: Universal RF foundation encoder + hardware adapter registry
| Field | Value |
|-------|-------|
| **Status** | Accepted — **P1 implemented** (`ruview-unified`: `tensor.rs`, `adapters.rs`, `tokenizer.rs`, `encoder.rs`, `pretrain.rs`, `heads.rs`, `eval.rs`) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 |
| **Relates to** | ADR-136 (`CanonicalFrame` provenance — the WiFi adapter consumes `wifi-densepose-core::CsiFrame` directly), ADR-152 §2 (geometry conditioning intake), ADR-016/017 (ruvector integration points) |
## 0. PROOF discipline
Grades as in ADR-273 §0. Every number below is MEASURED-CODE or MEASURED-SYNTHETIC unless marked EXTERNAL-UNVERIFIED.
## 1. Context
WiFo-2 and WiLLM (EXTERNAL-UNVERIFIED) demonstrated that heterogeneous CSI standardization + masked-reconstruction pretraining + small task adapters beats per-task models, and the age-aware CSI line showed a cheap win from encoding sample freshness multiplicatively. RuView has four incompatible capture families today (802.11 CSI, FMCW radar cubes, UWB CIR, and — via O-RAN — 5G SRS). Each previously implied its own model.
## 2. Decision — canonical tensor + adapter registry
### 2.1 Canonical tensor
All modalities normalize to `RfTensor` (`tensor.rs`): complex `(links × 56 bins × 8 snapshots)` plus carrier/bandwidth, per-link `LinkGeometry`, `sample_age_s`, `clock_quality ∈ [0,1]`, `uncertainty ∈ [0,1]`, `device_id`, and a `CalibrationMeta` contract. 56 bins = usable 20 MHz 802.11n subcarriers (and the existing 114→56 interpolation in `wifi-densepose-train`), so the most common source resamples trivially.
**Boundary rule**: `RfTensor::new` is the only constructor and validates every field (finite samples, geometry/link arity, ranges). Downstream code assumes validity. Tests: `tensor.rs::tests` (4).
### 2.2 Normalization pipeline (every adapter, 3 stages)
1. **Layout** — vendor shape → `(links, bins, snapshots)`; FMCW gets a fast-time DFT to range bins; SRS gets comb de-interleaving; then linear complex resampling to canonical dims.
2. **Amplitude** — per-link division by median amplitude (chipset gain invariance; offset recorded in `CalibrationMeta.gain_offset_db`).
3. **Phase** — per (link, snapshot), remove constant offset + least-squares linear ramp across bins (CFO residual + sampling-time offset), with unwrapping. Skipped for delay-domain modalities (radar range profiles, UWB taps) where a detrend would erase ToF structure.
Measured (test `wifi_adapter_normalizes_shape_gain_and_phase`): a synthetic capture with per-link gains ×3.7/×7.4 and phase ramp `0.9 + 0.11·bin` comes out with median amplitude 1.0 ± 1e-9 and residual phase < 1e-4 rad (the ~7 µrad residue is second-order chord-vs-arc error from complex resampling). The radar adapter localizes a fast-time beat tone to the analytically expected canonical range bin (`radar_adapter_localizes_beat_tone_to_range_bin`).
### 2.3 Registry
`AdapterRegistry` maps hardware id → `dyn RfAdapter`, **fail-closed** (unknown hardware is an error; wrong modality is a typed `ModalityMismatch`). Reference adapters ship for `esp32s3-csi`, `mr60bha2` (FMCW), `dw3000` (UWB), `oai-srs-xapp` (5G SRS) — the last being the ADR-273 P4 seam.
## 3. Decision — encoder, fusion contract, adapters
### 3.1 Tokenizer
One token per (link, 8-bin subcarrier group); 24 features: log-amplitudes, delay-spectrum DFT (4), Doppler DFT bins 14 (log-compressed `ln(1+100·mag)`), temporal amplitude deviation (`ln(1+20·std)`), phase velocity, sample age, link distance/height/azimuth, clock quality, uncertainty (`tokenizer.rs`, layout table on `RfToken`).
Two hardware-invariance steps precede feature extraction, and both were *forced by measurement*, not aesthetics (see §5 evidence trail):
- **window-median amplitude normalization** — raw Friis-scale features (~1e-3) left every head unable to learn;
- **CFO alignment** — per link, each snapshot is de-rotated by `arg Σ_b H[b,s]·H̄[b,0]`; carrier-frequency-offset drift is a *common* rotation and cancels, while a moving scatterer's frequency-selective perturbation survives (test `motion_raises_doppler_and_variance_features` uses a bin-dependent perturbation precisely so alignment cannot cancel it).
### 3.2 Encoder + pretraining
Pure-Rust, exactly differentiable (`encoder.rs`):
```text
h_i = tanh(W1·x_i + b1) token embedding
c = mean_i h_i permutation-invariant pool
m = tanh(W2·c + b2); g = tanh(W2b·m + b2b)
gate = σ(age_w·age + age_b) multiplicative freshness gate
z = g ⊙ gate + Wg·geo + bg ← the ADR-273 fusion contract, verbatim
```
Masked-reconstruction pretraining (`pretrain.rs`): mask 25 % of tokens, reconstruct each from `[z ; sinusoidal-position]` via a linear head discarded at deployment; SGD.
**Proof of the backward pass** (MEASURED-CODE, `gradients_match_finite_differences`): analytic gradients of **all 12 parameter groups** vs central finite differences — 174 sampled parameters, max relative error **1.31e-5**, with the absolute floor at central-difference roundoff (≈5e-11). Training halves masked loss and beats the constant-predictor variance baseline (`0.2757 → 0.0966` vs baseline `0.1550`; `pretraining_reduces_masked_loss_and_beats_mean_baseline`). Same seed ⇒ bit-identical weights (`training_is_deterministic`).
Backbone at deployment config (d_model 128): **40,856 parameters** (hand-count asserted in `param_count_matches_hand_computation`).
### 3.3 Two representation views (the PerceptAlign lesson, applied)
- `encode()` → full `z` (geometry-conditioned) — for localization/channel-prediction heads where sensor pose is signal.
- `encode_content()``[g ⊙ gate ; mean token features]` — for environment-invariant heads (presence/activity/anomaly). The additive `Wg·geo` term is a **room-specific offset a linear adapter would memorize** — measured: with it, held-out-room presence F1 was 0.00 while training F1 fit; without it plus the pooled-statistics skip connection, held-out F1 is 1.00 (SYNTHETIC, ADR-273 §5).
### 3.4 Task adapters, ≤ 1 % budget
`heads.rs`: presence (logistic, 129 params), activity (rank-2 LoRA-style factorized softmax, 268), localization (linear ℝ³, 387), anomaly (2 calibration statistics on reconstruction error). All < 408 = 1 % of the 40,856-param backbone, asserted in `every_head_fits_the_one_percent_budget_at_deployment_config`. Convex heads train full-batch (deterministic); tests show they fit separable/multiclass toys to ≥ 95 %.
### 3.5 Anti-leakage evaluation (ADR-273 §4)
`eval.rs`: `PartitionKey` (room/day/person/chipset/firmware/layout), `StrictSplit::holdout` + independent `verify()`, ECE, coverage/selective-risk, degradation ratio, F1. Six unit tests including a manufactured-leak detection test.
## 4. Alternatives considered
- **Candle/ONNX backbone now** — rejected for P1: the deliverable is a *proven contract* (gradient-checked fusion formula, budget enforcement, leakage protocol); porting to `wifi-densepose-nn` backends is mechanical once real-data P2 justifies scale.
- **Per-modality encoders with late fusion** — rejected: reproduces the isolated-classifier status quo ADR-273 exists to end.
- **Full transformer attention** — deferred: mean-pool + 2 mixing layers passed every P1 gate; attention is a P2 measurement question, not a default.
## 5. Evidence trail (what the measurements changed)
P1 development falsified two comfortable assumptions, recorded here because the *fixes are the ADR*:
1. Raw-scale tokens: presence head stuck at F1 0.47 even on training rooms → window-median normalization + CFO alignment (train F1 → 0.76).
2. Geometry-additive `z` for invariant tasks: held-out-room F1 0.00 → content view + pooled-statistic skip (held-out F1 → 1.00) — i.e. *the leak the eval protocol was designed to catch, caught in our own architecture first*.
## 6. Consequences
One encoder now serves presence, activity, localization, respiration-class, channel prediction, and anomaly through < 1 % adapters; new hardware lands as an adapter, not a model. Cost: the pure-Rust trainer is CPU-bound (fine at 40 k params; a P2 scale-up moves to `wifi-densepose-nn`).
@@ -0,0 +1,79 @@
# ADR-275: RF-aware Gaussian spatial memory — the persistent scene representation
| Field | Value |
|-------|-------|
| **Status** | Accepted — **P1 implemented** (`ruview-unified/src/gaussian/`: `primitive.rs`, `map.rs`, `gain.rs`, `graph.rs`; 16 unit tests, criterion benches) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 |
| **Relates to** | ADR-030 (persistent field model — superseded in direction by this), ADR-134 (CIR/ISTA), ADR-147 (OccWorld priors), ADR-261 (RuVector graph-ANN — the retrieval layer this memory will index into) |
## 0. PROOF discipline
Grades per ADR-273 §0. The July 2026 external motivators (EmbodiedSplat ~5 fps online semantic Gaussian mapping, ~67× memory efficiency; TGSFormer bounded temporal Gaussian memory; physics-informed channel-gain mapping with incremental Gaussian insertion; JITOMA task-gated activation) are EXTERNAL-UNVERIFIED throughout.
## 1. Context
RuView's spatial state is currently scattered (pose tracker state, field-model eigenstructure, worldgraph tracks). Vision-side SOTA converged on Gaussian fields as the common continuous scene memory, and — the July signal that matters here — the representation crossed into RF: propagation geometry, opacity, attenuation, and scattering as Gaussian primitives, updated *incrementally* when the environment changes. That is exactly the bridge from RuView sensing to a queryable digital twin: one store that answers both geometric questions ("what is near the sofa") and RF questions ("which object caused the channel anomaly", "where did multipath change").
## 2. Decision — the primitive
`RfGaussian` (`primitive.rs`) carries all six ADR-273 attribute groups:
1. **Geometry**: position, per-axis scale (σ), unit-quaternion orientation → anisotropic metric `Σ⁻¹ = R·diag(1/σ²)·Rᵀ`.
2. **Semantics**: 16-d embedding (RuVector-alignable).
3. **RF response**: reflectivity `[4 bands × 4 incident-angle bins]` (2.4/5/6/60 GHz), plus `occupancy` = peak extinction coefficient (nepers/m) used by the gain model.
4. **Motion**: signed Doppler m/s + `{Static, Slow, Fast}` class.
5. **Trust/lifecycle**: confidence ∈ [0,1], timestamp, decay τ, `Provenance {device, model_version, synthetic}`.
6. **Links**: typed references into the scene graph / RuVector entities.
Validated constructor (quaternion normalized, ranges checked); anisotropy and rotation are proven behaviorally (thin axis decays ≥ 80× faster at 0.3 m — the analytic ratio is 86; a 90° quaternion rotates the metric with it).
## 3. Decision — the map
`GaussianMap` (`map.rs`): spatial-hash grid (1 m default pitch) over a flat store.
- **Fusion, not accumulation**: an insert within Mahalanobis² 9 of a same-entity-kind Gaussian merges — confidence-weighted position/scale/occupancy/semantics/reflectivity/Doppler, noisy-OR confidence (`c₁+c₂−c₁c₂`), newest provenance wins, links union. Test: two 0.5-confidence observations 0.1 m apart fuse to one Gaussian at the weighted midpoint with confidence 0.75.
- **Decay + static persistence** (update-loop step 7): exponential confidence decay per Gaussian τ, **stretched by observed lifetime**`τ_eff = τ·(1 + ln(1 + lifetime/τ))` with `lifetime = last_seen first_seen` — so a wall confirmed over 30 min outlives a once-seen transient at equal nominal τ (test `long_lived_structure_outlives_transients_at_equal_tau`); prune below 0.02; deterministic (replay test).
- **Merge pass** (update-loop step 5): `merge_overlapping` collapses pairs that are *mutually* inside each other's Mahalanobis gate **and** semantically compatible (cosine ≥ 0.7, or both unlabeled) — orthogonal-semantic overlaps stay separate (test `merge_pass_collapses_mutual_overlaps_but_respects_semantics`). This catches drift the insert-time gate (±1 cell neighborhood only) misses.
- **Queries**: radius (hash + linear reference impl, equivalence-tested on 100-Gaussian grids), kNN (expanding ring), semantic cosine top-k, and the segment-corridor query below.
## 4. Decision — channel gain as a first-class query + inverse update
`gain.rs` implements the RF query surface:
```text
H(tx,rx,f) = (λ/4πd)·e^{-j2πd/λ} · exp(−Σ_g occ_g·I_g)
```
with `I_g` the **closed-form** line integral of each Gaussian's density along the TX→RX segment (1-D Gaussian integral via erf; derivation in the module doc).
**Exactness anchors (MEASURED-CODE):**
- Empty map ⇒ **exact Friis** amplitude (< 1e-15) and propagation phase (`empty_map_returns_exact_friis`).
- Closed-form line integral matches 1 mm trapezoid quadrature through a rotated anisotropic Gaussian to < 1e-6 (`line_integral_matches_numeric_quadrature`).
- On-path absorber attenuates strictly monotonically in occupancy; a 10σ off-path absorber changes LoS gain < 1e-6 dB.
**Inverse update** (`observe_link`) — the incremental-mapping move: measured link amplitude → target optical depth `τ* = ln(friis/measured)`; a projected-gradient step distributes the residual over intersected Gaussians proportional to their path integrals (exact Newton along the link at lr = 1), clamped at occupancy ≥ 0; if nothing intersects and attenuation is demanded, a compact absorber is spawned at the midpoint sized to close the residual. **Measured**: from an empty map, 20 observations of a link with an unseen 0.7-neper (≈6.1 dB) obstruction converge to < 0.06 neper residual and < 0.5 dB prediction error (`inverse_update_learns_a_wall_from_link_residuals`).
## 5. Decision — task-gated scene graph
`graph.rs`: sparse typed nodes (`Object/Room/PersonClass/Device/Event` — person *classes* only; identity lives behind ADR-277's double gate) and relations (`Contains/Near/CausedBy/ObservedBy`). The only sanctioned read is `activate(relevant_kinds, seeds, max_nodes)` — bounded BFS that reports truncation instead of silently scanning (the JITOMA lesson). Tests: an "which object caused the anomaly" activation pulls exactly {event, object, room} and gates out devices/person-classes; the node budget is enforced and truncation is flagged.
## 6. Performance (criterion, release, this machine)
| Benchmark | Result | Note |
|---|---|---|
| `channel_gain`, 1 k Gaussians | **26.9 µs** | was 139 µs with the midpoint-ball candidate query |
| `channel_gain`, 16 k Gaussians | **27.7 µs** | ~O(1) in map size after the corridor rewrite |
| segment corridor query, hash vs linear | 24 µs vs 6 µs (1 k) / 24 µs vs **163 µs** (16 k) | crossover ≈ 4 k Gaussians — reported honestly; both paths kept + equivalence-tested |
| radius query, hash vs linear | 4.3 µs vs 101 µs @ 16 k (23×) | hash loses at 1 k (4.0 vs 1.9 µs) — small maps are brute-force territory |
| `observe_link` inverse update | **74 µs** | was 305 µs pre-optimization |
| map insert+fuse (64 Gaussians, in observe bench setup) | included above | |
The optimization pass replaced a midpoint-ball candidate search (`(2·(L/2+3)+1)³ ≈ 9,300` cell lookups on a 14 m link) with an AABB sweep prefiltered by cell-centre-to-segment distance (bound `margin + √3/2·cell`), after a first corridor attempt (per-sample cube inserts into a BTreeSet) measured *worse* (1.2 ms) and was discarded — kept in this record as the honest negative result.
## 7. Consequences
- The map answers "where is a person likely", "where did multipath change", and "which object caused a channel anomaly" (gain residual → `CausedBy` edge) from one store.
- RuVector integration (ADR-261) becomes: vector search retrieves candidate Gaussians/nodes → graph traversal enforces relations → the gain model *verifies* answers against geometry. The LLM plans the query; it never invents the spatial answer.
- Not yet done (P3): live wiring into `wifi-densepose-sensing-server`, visual/depth Gaussian ingestion, and RuVector index sync.
@@ -0,0 +1,68 @@
# ADR-276: Physics-guided synthetic RF world generator — randomize physics, not textures
| Field | Value |
|-------|-------|
| **Status** | Accepted — **P1 implemented** (`ruview-unified/src/synth/`: `room.rs`, `raytrace.rs`, `generator.rs`; 10 unit tests + the ADR-273 acceptance pipeline consumes it end-to-end) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 |
| **Relates to** | ADR-015 (MM-Fi/Wi-Pose datasets), ADR-089 (nvsim — the determinism pattern this follows), ADR-135 (empty-room baselines the generator can emulate) |
## 0. PROOF discipline
Grades per ADR-273 §0. WaveVerse (released simulator, phase-coherent ray tracing) and HybridSim (92.07 % vs 54.22 % synthetic-only→real activity recognition when physics is modeled explicitly) are EXTERNAL-UNVERIFIED motivators. Every output of this generator is stamped `RfModality::Synthetic` and every number derived from it is labeled SYNTHETIC — that stamp survives into `Provenance.synthetic` at the ADR-277 export boundary.
## 1. Context
RuView's scarcest resource is labeled, *diverse* RF data: rooms, materials, antenna placements, people, chipsets. The 2026 evidence says synthetic RF transfers **when the physics is explicit and the randomization hits physical parameters** (permittivity, geometry, kinematics, hardware nuisances) rather than cosmetic noise. A physics generator also gives the ADR-273 acceptance machinery something it can never get from captures alone: *ground truth by construction* and unlimited strict-split diversity.
## 2. Decision — physics core
### 2.1 Rooms and materials (`room.rs`)
Shoebox rooms `[0,Lx]×[0,Ly]×[0,Lz]`, one wall material with **complex permittivity** `ε = ε_r j·σ/(ωε₀)` and normal-incidence Fresnel reflection `Γ = (1−√ε)/(1+√ε)`. Presets (concrete/drywall/glass, ITU-R P.2040 ballpark) plus a perfect absorber for test isolation. Measured sanity: concrete at 2.4 GHz gives |Γ| ≈ 0.390.45 with phase inversion; |Γ| < 1 for all passive presets; ε_r = 1, σ = 0 gives Γ = 0 exactly. People are validated-in-room point scatterers with constant velocity and RCS.
### 2.2 Multipath (`raytrace.rs`)
AllenBerkley image method, reflection order ≤ 2 (per-axis images `±x + 2nL`, bounce count `|2n|` / `|2n1|`), plus single-bounce bistatic person scattering with amplitude `√(σ_rcs/4π)/(d₁·d₂)` (bistatic radar equation, amplitude form):
```text
H(f) = Σ_paths Γ^order · (c/f)/(4π) · s_p · e^{j2πf·d_p/c}
```
**Doppler is never injected** — it emerges from the person's path length changing between snapshots.
**Physics gates (MEASURED-CODE):**
| Gate | Test | Result |
|---|---|---|
| Direct path ≡ Friis | `direct_path_is_exact_friis` | < 1e-15 per subcarrier (absorber walls) |
| Reciprocity `H(a→b) = H(b→a)` | `channel_is_reciprocal` | < 1e-12, with person + concrete walls |
| Image geometry | `first_order_reflection_matches_mirror_geometry` | floor/ceiling bounce at exactly the mirror distance; 1 direct + 6 first-order + second-order set |
| Doppler | `moving_person_produces_the_analytic_doppler_phase_rate` | residual-phase rotation matches `2πf·Δd/c` to < 1e-6 rad across 4 steps |
## 3. Decision — domain randomization (`generator.rs`)
Per room, seeded ChaCha20 (nvsim discipline — same seed ⇒ byte-identical corpus, cross-machine):
- **Physics**: dimensions 410 × 38 × 2.43.2 m; ε_r ∈ [2,7], σ ∈ [0.002,0.1] S/m; random TX/RX placements; person start/heading/speed/RCS.
- **Hardware nuisances** (what breaks naive models in the field): per-room gain ×0.52, static phase offset, **CFO drift** ±0.3 rad/snapshot, thermal noise, 5 % packet loss (snapshot re-delivery), 3 % wideband interference bursts.
- **Provenance for strict splits**: every window carries a full `PartitionKey` (room/day/person/chipset/firmware/layout) so ADR-273 §4 holdouts exist by construction.
Measured: byte-determinism per seed (and divergence across seeds); presence windows carry > 5× the temporal amplitude variance of empty windows (actual measured ratio on the test corpus is far higher); labels/keys complete.
The CFO nuisance earned its keep immediately: it *defeated the first tokenizer* (empty rooms looked like motion) and forced the CFO-alignment step now documented in ADR-274 §3.1 — exactly the class of failure a physics-parameter randomizer exists to surface before real deployments do.
## 4. What this generator is NOT
- Not a WaveVerse replacement: order-2 specular + point scatterers, no diffraction, no diffuse scattering, no angle-dependent Fresnel, no antenna patterns. These are refinements to add *when a P2 real-data gap analysis demands them*, not before.
- Not evidence of real-world accuracy: the ADR-273 acceptance numbers on this data validate the pipeline; the synthetic→real transfer claim (HybridSim-style) is untested here and stays EXTERNAL-UNVERIFIED until P2 replay experiments.
## 5. Performance
Criterion (release): 1 room × 4 windows × 3 links generates in **3.1 ms** (≈ 260 µs/window) — corpus generation is never the bottleneck; the 8-room acceptance corpus builds in well under a second even in debug.
## 6. Consequences
- Every pipeline stage gains a deterministic, physics-proven test bed; regressions in adapters/tokenizer/encoder now fail loudly against ground truth.
- Data scarcity stops gating architecture work: strict-split experiments (rooms/chipsets/layouts) run in CI.
- The honest-labeling chain (`RfModality::Synthetic``Provenance.synthetic` → SYNTHETIC-graded ADR claims) is structural, not editorial.
@@ -0,0 +1,61 @@
# ADR-277: Edge sensing control plane — purposes, zones, retention, and a trust boundary raw RF cannot cross
| Field | Value |
|-------|-------|
| **Status** | Accepted — **P1 implemented** (`ruview-unified/src/policy.rs`; 5 unit tests + the acceptance-pipeline export test) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 |
| **Relates to** | ADR-153 (802.11bf protocol model), ADR-141/120 (BFLD privacy control plane + privacy classes), ADR-262 §3.3 (RuField P0P5 fail-closed mapping — the same philosophy, applied to sensing outputs), ADR-032 (mesh security hardening) |
## 0. PROOF discipline
Grades per ADR-273 §0. Standards status (EXTERNAL, checkable): IEEE 802.11bf-2025 published 2025-09; IEEE 802.11bk addresses ≤ 320 MHz positioning; ETSI published an ISAC architecture 2026-02 (monostatic/bistatic/multistatic/network/device sensing) followed by a security report identifying **19 privacy and security issue classes**; 3GPP Release 20 sensing studies are active. The OpenAirInterface SRS-xApp demo (0.12 m MAE under a **random** split) is EXTERNAL-UNVERIFIED and its split methodology is exactly the leakage ADR-273 §4 rejects — we cite the *implementation path*, not the number.
## 1. Context
Sensing purposes and sensing zones are becoming first-class authorization objects in the standards (802.11bf sensing sessions; ETSI ISAC purposes/exposure). Meanwhile the ETSI security report's issue classes make one thing clear: a sensing stack without a policy plane is a liability. RuView already fails closed at other boundaries (ADR-262 §3.3 maps privacy by information content, never byte value); this ADR gives sensing *outputs* the same discipline, on-device, before any transport.
## 2. Decision — three structural rules
### 2.1 Raw RF never leaves the trust boundary
The only exportable type is `BoundedEvent` — typed verdicts only (`Presence(bool)`, `ActivityClass(u8)`, `RespirationBpm(f64)`, `Location([f64;3])`, `AnomalyScore(f64)`). **No variant can carry RF samples, so raw CSI/radar export is unrepresentable, not merely forbidden**; `TrustBoundary::export` is the single egress and there is deliberately no API that serializes an `RfTensor` outward. External systems receive bounded events + uncertainty, never signal history.
### 2.2 Fail closed, everywhere
`PolicyEngine::authorize`: unknown zone ⇒ deny; purpose not granted in the zone ⇒ deny; **identity recognition is double-gated** — it must be in the zone's `allowed_purposes` *and* the zone must set `identity_explicitly_enabled` (either alone denies). Retention: an event older than the zone's `retention_s` at export time is dropped with a typed `PolicyDenied`. Tests cover every branch, including the manufactured cases (identity granted-but-not-enabled; enabled-but-not-granted; stale event).
### 2.3 Every output is accountable (ADR-273 acceptance item 8)
`BoundedEvent::new` is the only constructor and *fails* without: uncertainty ∈ [0,1], provenance (device + `synthetic` flag — the ADR-276 honest label survives export), a non-zero model version, timestamp, purpose, and zone. The acceptance test (`outputs_leave_only_through_the_policy_boundary_fully_attributed`) runs the full pipeline — synthetic world → encoder → presence head → event → export — and asserts the attribution and the denial of an ungranted purpose on the same zone.
## 3. Purpose taxonomy
`SensingPurpose`: `Presence, Activity, Vitals, Localization, PoseTracking, IdentityRecognition, ChannelDiagnostics` — deliberately aligned with the ETSI ISAC sensing-service classes and WLAN-sensing use cases so a future 802.11bf sensing-session negotiation or ISAC exposure API maps 1:1 onto zone grants. Person *identity* is additionally kept out of the ADR-275 scene graph by type (`EntityKind::PersonClass`, never a person id) — the graph cannot leak what it cannot store.
## 4. O-RAN / cellular path (roadmap, seams shipped)
The P1 control plane is transport-agnostic and already fronts the cellular seam:
- `CellularSrsAdapter` (`oai-srs-xapp`, ADR-274 §2.3) normalizes comb-sampled SRS frequency responses into the canonical tensor — the data-plane contract an OAI xApp needs.
- P4 (ADR-273 §7) places the sensing application beside the DU for sub-ms I/QCSISRS access, with the xApp performing wider-area fusion; **every output of that path still exits through this ADR's `TrustBoundary`**, and its localization claims will be reported only under strict splits (the OAI demo's random split is the cautionary example, not the target).
## 5. Alternatives considered
- **Reuse BFLD's privacy classes directly** — rejected: BFLD (ADR-120) classifies *captures*; this plane authorizes *outputs by purpose and zone*. They compose (a BFLD-classified capture feeding a head still exits through `TrustBoundary`), and ADR-262's `map_privacy` remains the capture-side mapping.
- **Config-file allow-lists without types** — rejected: the 19 ETSI issue classes are mostly "the code path existed" failures; unrepresentability beats configuration.
## 5.5 Boundary hardening (property-tested)
`tests/security_boundaries.rs` drives every validated constructor and every authorization gate with `proptest` over arbitrary values — including NaN/±inf smuggled via `f64::from_bits` — and asserts the *contract* (valid object **or** typed error, never a panic, never a permissive default). Three real defects surfaced and were fixed, all input-controlled denial-of-service or NaN-propagation:
1. `ble_cs_range` unwrap looped forever on a **non-finite** phase (`+inf x = +inf`); a **finite-but-huge** phase (1e300 rad) made the same loop run ~1e299 iterations. Fixed by rejecting implausible phases (> 1e6 rad) and replacing the loop-based unwrap with O(1) modular arithmetic.
2. A **subnormal** Gaussian scale (5e-324) passed `> 0` but overflowed `1/σ²` to ∞, making the density at the primitive's own centre NaN. Fixed with physical plausibility bounds (σ ∈ [1e-6, 1e4] m, occupancy ∈ [0, 1e6] nepers/m).
The eight properties now proven: tensor/Gaussian/BoundedEvent constructors never panic; `ble_cs_range` never panics and yields only finite non-negative distances; the policy engine is fail-closed for every (purpose, grants, zone) triple; raw export is unreachable for every task configuration; coherent fusion rejects every non-finite or out-of-bounds sync state; occupancy representations can never retain identity.
## 6. Consequences
- Enterprise/telecom conversations get a concrete artifact: a privacy manifest is a serialization of zones + purposes + retention (all types already `serde`).
- Every future surface (sensing-server WS, RuField bridge, SRS xApp, MCP tools) must route sensing outputs through `TrustBoundary` — added to the pre-merge security-review checklist item 12.
- Cost: purposes are coarse (no per-consumer grants yet); P3 adds consumer identity when the sensing-server wiring lands.
@@ -0,0 +1,46 @@
# ADR-278: Radar inverse rendering + differentiable RF SLAM — a gated research program, not a dependency
| Field | Value |
|-------|-------|
| **Status** | Proposed — research program (deliberately **no code in P1**) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 (pillar 6, score 3.6 — highest strategic value, highest hardware + reproduction risk) |
| **Relates to** | ADR-275 (the Gaussian memory these methods would write into), ADR-263/264 (RTL8720F radar platform + wire protocol), ADR-021 (mmWave vitals hardware), ADR-276 (synthetic worlds as the reproduction sandbox) |
## 0. PROOF discipline
Everything numeric in this ADR is **EXTERNAL-UNVERIFIED** — reported by fresh papers/preprints that this repo has not reproduced. That is the point of this ADR: to fix the reproduction gates *before* any of these numbers are allowed to influence the roadmap as if they were ours.
## 1. Context — what the field reports (July 2026)
| System | Claim (theirs) | Availability | Risk read |
|---|---|---|---|
| **RISE** | Single static mmWave radar + multipath inversion → joint room layout + furniture; 16 cm scene Chamfer (baseline 40 cm), 58 % furniture IoU | code available | most reproducible; static sensor matches our appliance posture |
| **DiffRadar** | Radar SLAM + Gaussian fields + differentiable rendering; 0.129 m vs 0.823 m ATE, 94.78 % vs 42.59 % map consistency, 70 fps, 40 MB maps | fresh preprint | treat as **reproduction target, not component** — numbers are single-team, single-venue |
| **GeRaF** | Differentiable RF renderer + SDF + reflectivity, near-range reconstruction; ~32 h on one H100 for 50 k iterations | published setup | offline calibration / digital-twin tool only; unsuitable for continuous adaptation |
The strategic pull is real: all three converge on *inverse rendering into continuous scene representations* — exactly the ADR-275 memory. The risks are equally real: single-source numbers, mmWave hardware variance, and compute profiles (GeRaF) incompatible with edge deployment.
## 2. Decision
1. **No production dependency** on any of these systems or their claims. ADR-275's gain model + inverse update is the only RF-inverse machinery in the deployment path.
2. **Reproduction order: RISE → DiffRadar → GeRaF-lite**, each on one controlled test site, each gated (§3) before the next starts. RISE first because a static radar matches the RuView appliance posture and its inversion writes naturally into `RfGaussian` (occupancy + reflectivity fields already exist for it).
3. **Sandbox-first**: before hardware, each method's core inversion is exercised against ADR-276 synthetic worlds extended with a radar-cube output mode (the `FmcwRadarCube` adapter already normalizes such cubes), so failures separate into "our reimplementation" vs "their claim" cleanly.
4. **Integration contract**: any reproduced system emits into `GaussianMap` via the existing primitive — no parallel scene store. SLAM trajectories, if any, become `Provenance`-stamped map updates subject to ADR-277 export rules like everything else.
## 3. Gates (each phase passes all or the program pauses)
| Gate | Threshold | Split discipline |
|---|---|---|
| G1 RISE-repro (synthetic) | layout Chamfer within 2× of paper's on our synthetic rooms | held-out room geometries |
| G2 RISE-repro (one real site) | qualitative layout recovery + quantified Chamfer vs measured floor plan; report *our* number, whatever it is | site never used in tuning |
| G3 DiffRadar-repro | ATE and map consistency on our trajectory rig; publish the delta vs paper | held-out trajectories |
| G4 Edge viability | inversion or map-update loop ≤ 50 ms p95 on target hardware, or explicit reclassification as offline-calibration tooling (GeRaF's honest category) | — |
A gate failure is a *result*, recorded in this ADR's log — the program exists to convert EXTERNAL-UNVERIFIED into MEASURED, in either direction.
## 4. Consequences
- The roadmap cannot silently absorb preprint numbers; anything radar-inverse must pass through §3.
- ADR-275's primitive already reserves the fields (per-band × angle reflectivity, occupancy, motion) these methods need, so a successful reproduction integrates without schema churn.
- Cost of delay is accepted: pillar 6 scored lowest on readiness, and P1P4 (encoder, memory, synth, control plane, SRS) do not depend on it.
@@ -0,0 +1,54 @@
# ADR-279: Native RF frame contract — `RfFrameV2` is authoritative, the canonical tensor is a derived view
| Field | Value |
|-------|-------|
| **Status** | Accepted — **implemented** (`ruview-unified/src/frame.rs`; 5 invariant tests) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 (amends ADR-274 §2) |
| **Relates to** | ADR-136 (`CanonicalFrame` — extended, not replaced), ADR-262 (provenance discipline), ADR-282 (evidence ladder policy) |
## 0. PROOF discipline
Grades per ADR-273 §0. This ADR is a **correction** to ADR-274 §2, adopted before any measured-data debt accumulates.
## 1. Context — the architectural correction
ADR-274 made the 56-bin × 8-snapshot canonical `RfTensor` the adapter output, which is right for *compatibility* but wrong as the *authoritative* format: resampling every device into one fixed tensor discards bandwidth (a 320 MHz 802.11bk capture and a 20 MHz 802.11n capture become indistinguishable), antenna structure, phase state, and hardware-specific information a foundation encoder should learn from (the WiLLM lesson: lightweight per-device adapters into a shared *latent*, not a shared *tensor*). RuView's own history proves the cost of premature canonicalization — MERIDIAN's normalizer is useful precisely because the native data was still around.
## 2. Decision — `RfFrameV2`
The authoritative record preserves the native capture. Fields per the implementation: schema version, frame id, timestamp, modality (now including `WifiCir`, `WifiBfReport`, `FmcwRangeAzimuth`, `FmcwDopplerAzimuth` alongside CSI/SRS/FMCW/UWB/BLE-CS), **declared native axes** (`FieldAxis`: time/frequency/delay/Doppler/range/azimuth/elevation/antenna/polarization), centre frequency, bandwidth, sample rate, arbitrary-rank `native_shape` + `native_iq` + explicit `valid_mask`, TX/RX `Pose3` in one building frame, `AntennaElement` geometry, `sample_age_ns`, `CalibrationState` with a **declared `PhaseState`** (`Raw | Sanitized | Calibrated | Unavailable`), `SignalQuality`, and `FrameProvenance`.
Seven required invariants, each enforced in the validated constructor or proven by a test:
1. **Native samples are never overwritten**`to_canonical(&self)` is read-only; `canonical_view_is_derived_and_native_is_untouched` asserts byte-identical native IQ + mask after derivation.
2. Subcarrier/antenna masks are explicit (`valid_mask`, arity-checked).
3. Phase declares its state — consumers branch on `PhaseState` instead of guessing whether detrending happened.
4. TX/RX geometry uses one building coordinate system (`Pose3`).
5. Results retain source identity via `receipt_id` (consumed by the Gaussian memory's `source_receipts` lineage, ADR-275).
6. **Synthetic and measured frames can never share a provenance class**, strengthened to an evidence rule: `Synthetic ⇒ exactly L0Simulation`, `Measured ⇒ ≥ L1CapturedReplay` — both directions rejected at construction (`synthetic_and_measured_provenance_can_never_alias`).
7. Sample age is carried through the whole path (frame → tensor → age gate → `BoundedEvent`).
## 3. The canonical tensor is demoted to a compatibility view
`RfFrameV2::to_canonical()` derives the ADR-274 tensor **through the exact same normalization code path as every adapter** (`adapters::normalize_grid` — one normalization, many entry points), after mask-aware gap-filling (invalid bins interpolated from nearest valid neighbors on the complex plane). Rank ≠ 3 frames have no canonical projection and say so with a typed error. The existing ESP32/Intel/Atheros 114→56 projections stay as-is; they simply stop being the storage format.
## 4. The mandatory split manifest
The brief's leakage rule is now code: `PartitionKey` gains a `session` dimension (packet-session leakage is as real as room leakage) and `eval::SplitManifest` certifies per-dimension disjointness across **all seven** dimensions (room/day/person/chipset/firmware/layout/session):
```text
train_rooms ∩ test_rooms = ∅ … train_sessions ∩ test_sessions = ∅
```
`fully_disjoint()` is the bar for reporting a result as leakage-resistant; a room-holdout split that still shares people *says so* in its manifest instead of masquerading (test `split_manifest_certifies_per_dimension_disjointness`). The hidden real-world test set requirement (never accessible to synthetic generation/calibration) is process, recorded in ADR-282 §4.
## 5. Consequences
- New hardware (PicoScenes, Intel, Atheros, Realtek radar, 320 MHz 802.11bk) lands as an `RfFrameV2` producer + latent adapter; nothing is lost at ingest. Vendor conformance receipt = the constructor's invariants (native shape preserved, phase state declared, timestamps monotonic, geometry present, loss measured, synthetic flag correct).
- The encoder input contract (ADR-274) is unchanged *today* (it consumes the derived view); migrating the tokenizer to native-resolution tokens is the flagged follow-up once real multi-bandwidth data exists (P2).
- Storage cost rises (native + derived); accepted — the derived view can always be recomputed, the native never can be.
## 6. Verification
`cargo test -p ruview-unified frame::` — 5 tests: provenance aliasing, shape/mask/axes arity, derived-view purity + gap-filling, rank/geometry rejection, P3162 import-profile validation (`SyntheticApertureSoundingDataset`, ADR-281 §5). All MEASURED-CODE.
@@ -0,0 +1,59 @@
# ADR-280: Active sensing and programmable perception — tasks, freshness, coherence, and governed actuation
| Field | Value |
|-------|-------|
| **Status** | Accepted — **implemented** (`ruview-unified/src/control.rs`; 6 test suites incl. a measured ≥70 % traffic-reduction gate) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273; extends ADR-277 |
| **Relates to** | ADR-277 (policy engine — every contract here composes with it), ADR-262 (P0P5 privacy classes, reused verbatim), ADR-148 (`ruview-swarm` — the mobile-agent consumer of sensing actions) |
## 0. PROOF discipline
Grades per ADR-273 §0. External motivators — ESI-Bench's act-to-uncover formalization, LuLIS's 256-coherent-RF-chain distributed aperture, ETSI's cooperative-ISAC and AI/data-handling work items, age-of-information digital-twin scheduling, semantic/task-sufficient communication architectures — are all EXTERNAL-UNVERIFIED. Everything asserted about *our* behavior is a named test.
## 1. Context
The important shift is from **passive sensing** (accept whatever measurements arrive) to **programmable perception**: the system chooses where, when, how, and at what fidelity to sense, then changes the radio environment or moves sensing agents to resolve uncertainty. Simultaneously, the dominant failure mode across the emerging systems is **hidden synchronization and calibration dependence** — shared clocks, known antenna poses, stable phase silently assumed, confidently wrong when violated. Both belong in the control plane, fail-closed, before capture begins.
## 2. Decision — the evidence-aware sensing task (`SensingTask`)
ETSI-ISAC-vocabulary contract: purpose, target zone, modalities, requested resolution, latency bound, minimum confidence (below which results become *no decision*), raw + result retention, authorized consumers, consent reference. `admit_task` composes with the ADR-277 engine and is fail-closed on every branch; two rules deserve record:
- `raw_export_allowed` **exists in the contract** (ISAC vocabulary compatibility) but is **always refused** (`task_admission_is_fail_closed`): ADR-277 §2.1 made raw export unrepresentable, and a config flag does not reopen it.
- Identity-purpose tasks without a consent reference are refused before the zone check even runs.
## 3. Decision — sensing actions (`SensingAction` + `InformationGoal`)
An action is a deliberate act of evidence-gathering against a stated hypothesis ("the east corridor holds one stationary person or two closely spaced people"), bounded by latency, energy, and a **privacy ceiling** (`PrivacyClass` P0P5, the ADR-262 ladder). Actions are what the planner (§4), a MetaHarness agent, or a swarm drone consume.
## 4. Decision — age-of-information scheduler (`ActiveSensingPlanner`)
A spatial twin is only useful when it knows which parts are stale. Per region: `SpatialStateFreshness` (last observation, expected change rate, uncertainty growth, business criticality, sensing cost), with
```text
priority = uncertainty(age) × change_rate × criticality ÷ cost
```
The planner emits at most the highest-priority action above threshold per cycle. **Measured** (`planner_reduces_sensing_traffic_versus_uniform_refresh`): 20 regions / 100 ticks, one hot region — 100 observations vs 2,000 under uniform refresh = **95 % sensing-traffic reduction** while the hot region stays observed. (The brief's "5090 %" was an architectural estimate; this is a synthetic-scenario measurement, sensitive to how concentrated change is.) Priority ordering is proven separately (`planner_prioritizes_stale_critical_regions`: emergency-exit > server-room > storage).
## 5. Decision — coherent distributed apertures fail closed (`CoherentSensorGroup`)
No coherent fusion unless the group can *prove* compatibility: every member must report sync state, be within the group's time-error and phase-error bounds, and match the calibrated baseline geometry hash; unknown reporters are rejected too. Five denial paths, each tested (`coherent_fusion_fails_closed`): missing member, clock drift, phase drift, geometry change since calibration, non-member injection. This is the antidote to the hidden-synchronization failure mode — a building-scale WiFi aperture (the LuLIS direction) degrades to incoherent processing rather than producing confident nonsense.
## 6. Decision — programmable radio environments are governed actuators
RIS / movable / fluid antennas change **which rooms and people are observable**, so actuation is governed like sensing: `request_actuation` is the only way to obtain an `ActuationReceipt`, it verifies the state is supported *and* that the affected zone grants the purpose under the ADR-277 engine (`actuation_requires_policy_authorization`: steering a beam for an ungranted purpose is denied). Receipts carry requested/applied state, time, controller, purpose — the audit trail the RIS governance requirement demands.
## 7. Decision — task-sufficient representations are leakage-checked
Semantic compression ("transmit occupancy uncertainty, not CSI") must remain **task-scoped**: a representation sufficient for anonymous occupancy may not retain identity. `TaskSufficientRepresentation` carries source lineage, an information bound, an explicit `excluded_information` list, and a privacy class; `validate_representation` enforces per-purpose ceilings (Presence/Diagnostics ≤ P2 excluding identity+vitals; Activity/Localization ≤ P3 excluding identity; Vitals/Pose ≤ P4; Identity = P5) and refuses lineage-free orphans (`task_sufficient_representation_is_leakage_checked`).
## 8. Standards alignment (the strongest strategic seam)
The vocabulary here — sensing task/service/entity, measurement configuration, sensing data/result/consumer/purpose, retention, result exposure — is deliberately the emerging ETSI ISAC data-plane vocabulary, positioning this crate as an open reference implementation candidate for ISAC data handling rather than a parallel dialect. Charging/mobility management are explicitly out of scope until a cellular deployment exists.
## 9. Consequences
- MetaHarness/OaK-style agents get a typed surface: read freshness, plan actions, receive receipts — spatial memory meets agentic planning without touching raw RF.
- Distributed-aperture work (P4+) inherits a fusion gate that already fails closed.
- Not implemented (honest scope): information-gain *estimation* is caller-supplied (the planner uses staleness heuristics, not mutual information); RIS drivers, actual multi-AP coherence measurement, and OTFS waveform control are hardware-dependent roadmap items.
@@ -0,0 +1,49 @@
# ADR-281: New modality surfaces — BLE Channel Sounding, delay-Doppler-native tensors, P3162 import, and factorized pose
| Field | Value |
|-------|-------|
| **Status** | Accepted — **implemented** (`adapters.rs` BLE CS + ranging evidence, `tensor.rs::delay_doppler_map`, `frame.rs` P3162 import profile, `heads.rs` factorized pose; 8 new test suites) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273; extends ADR-274 |
| **Relates to** | ADR-279 (`FieldAxis` native axes), ADR-152 (geometry conditioning intake), ADR-021/263 (radar hardware) |
## 0. PROOF discipline
Grades per ADR-273 §0. Bluetooth SIG cm-level claims vs the ~2050 cm practical review, OTFS ISAC field trials, IEEE P3162, PerceptAlign's >60 % cross-domain error reduction, and RePos's 1021 % MPJPE gains are EXTERNAL-UNVERIFIED design inputs. Our numbers below are MEASURED-CODE / MEASURED-SYNTHETIC.
## 1. BLE Channel Sounding (§2) — likely the fastest path to consumer-scale spatial anchoring
`BleCsFrame` carries per-frequency-step round-trip tone phases plus optional RTT. Two rules:
- **Phase-based ranging and RTT are separate evidence sources.** `ble_cs_range` computes both — `d_phase = |dθ/df|·c/4π` from the unwrapped phase-vs-frequency slope, `d_rtt = rtt·c/2` — and *cross-validates* instead of averaging. Agreement raises confidence; divergence beyond 0.5 m yields `RangingAnomaly::Divergent` (multipath bias, relay attack, timing fault, or calibration problem) with confidence capped ≤ 0.2. Measured: exact recovery at 1.5/5/12 m (< 1 µm error on clean synthetic phases, 40 steps × 1 MHz); a relay-style RTT inflation to ~51 m against a 5 m phase estimate is flagged, not blended (`ble_cs_flags_relay_style_divergence_instead_of_averaging`).
- The tensor view (`BleCsAdapter`, `nrf54-cs` in the registry) **never detrends phase** — the ranging ramp *is* the measurement; the preserved ramp is asserted in test.
Single-source evidence (no RTT) is capped at confidence 0.5 — one mechanism alone is never high-trust ranging.
## 2. Delay-Doppler-native support (§3)
`FieldAxis` (ADR-279) makes delay/Doppler first-class native axes so OTFS-style captures are stored natively, and `RfTensor::delay_doppler_map` provides the standard transform for frequency-time tensors: IDFT over bins (→ delay) × DFT over snapshots (→ Doppler). Measured: a synthetic scatterer at (delay 7, Doppler 3) produces a unit peak with < 1e-9 leakage everywhere else. The transform is implemented **separably** (delay IDFT per snapshot, then Doppler DFT per delay row — `O(B²S + S²B)` vs the direct form's `O(B²S²)`), proven equivalent to the direct reference to < 1e-10 and **measured 8.3× faster** (520 µs vs 4.34 ms at 56×8 in the criterion bench). Rule: derived features may be small, but delay-Doppler maps are not collapsed into scalar motion energy before provenance and local storage.
## 3. IEEE P3162 synthetic-aperture import (§5)
`SyntheticApertureSoundingDataset` (frequency range, aperture poses, directional PDP, coordinate system, processing-manifest hash) is the validated import profile — the calibration bridge between measured environments, Sionna-class simulators, and learned RF scene models. Schema + validation only; parsers arrive with the first real dataset.
## 4. Factorized pose (RePos) + log-age gating
`FactorizedPoseHead` separates what generalizes from what conditions:
- **relative skeleton** branch reads the environment-invariant content representation (cannot learn room-position shortcuts);
- **root localization** branch reads the geometry-conditioned representation (sensor pose is signal there — the PerceptAlign lesson);
- `absolute = root + relative` (`PoseOutput::absolute_joints_m`), with **calibrated per-joint and root residual σ** so every pose output carries uncertainty (ADR-273 item 8).
**The leakage experiment** (`factorized_pose_resists_room_shortcut_leakage`): training rooms where room position *correlates* with body scale (the trap real deployments set), held-out room breaking the correlation — factorized MPJPE **0.0003 m** vs monolithic absolute-head **0.2534 m** (845× worse), on a toy that isolates the mechanism. MEASURED-CODE for the mechanism; not a pose-accuracy claim.
Budget: the structured pose head is the largest adapter at **740 params vs the 40,856-param backbone (1.8 %)** — documented ceiling for structured heads is **< 2 %** (scalar heads keep the 1 % gate), both asserted in `every_head_fits_the_one_percent_budget_at_deployment_config`.
Age gating now matches the age-aware-CSI recipe exactly: the freshness gate input is `log(1 + sample_age_ms)` (`encoder::age_feature`), giving millisecond and multi-second staleness comparable input scale; the finite-difference gradient check re-proves the backward pass through the changed input.
## 5. Consequences
- Bluetooth/UWB anchors slot in as *geometric* evidence while WiFi carries ambient activity — the complement strategy, in code.
- The Gaussian primitive gained the lifecycle fields the update-loop spec requires (`first_seen_ns`, `doppler_variance`, bounded `source_receipts` lineage merged on fusion) — static structure is distinguishable from transients by lifetime, and every primitive traces to source frames.
- Roadmap, explicitly not done: real nRF54 CS capture path, OTFS waveform generation, P3162 file parsing, pose heads on real MM-Fi-style data.
@@ -0,0 +1,62 @@
# ADR-282: Ecosystem positioning — RuView is the camera-free RF perception runtime, not the whole spatial OS
| Field | Value |
|-------|-------|
| **Status** | Accepted (positioning + evidence-ladder policy; ladder implemented as `frame::EvidenceLevel`) |
| **Date** | 2026-07-26 |
| **Parent** | ADR-273 |
| **Relates to** | ADR-260/262 (RuField), ADR-261 (RuVector), ADR-182 (MetaHarness-minted harness), ADR-279 (provenance/evidence types), ADR-187 (honest labeling precedent) |
## 1. Context
RuView currently occupies a valuable but ambiguous position: the README's breadth invites reading every capability as field-validated, and the platform sometimes speaks as if it were the complete spatial intelligence operating system. The defensible identity is narrower and stronger.
## 2. Decision — the layered identity
> **RuView is an open, edge-native RF perception runtime that turns heterogeneous radio measurements into governed spatial observations.**
It is *not* positioned as a complete world model, robotics platform, digital twin, or universal spatial OS. The stack divides:
| Layer | Responsibility | Owner |
|---|---|---|
| Applications | healthcare, buildings, robotics, security, retail, industrial | application systems |
| Agent & decision | query planning, active sensing, automation, policy | **MetaHarness** |
| Spatial memory & reasoning | persistent objects, Gaussian fields, scene graphs, temporal memory | **RuVector** (fed by `ruview-unified::gaussian`) |
| Governed sensing plane | evidence, privacy, calibration, lineage, sensing tasks | **RuField** (bridged per ADR-262; contracts in ADR-277/279/280) |
| Perception & edge inference | native capture, adapters, shared encoder, task heads, uncertainty, P0 containment | **RuView** |
| Radio & physical sensors | WiFi CSI/CIR/BF, radar, UWB, BLE CS, cellular SRS | hardware |
Competitive posture follows from the layer: **complement vision platforms** (coverage where cameras are unavailable, unwanted, or ineffective — never "replaces cameras universally"); one shared encoder + spatial field across CSI and radar; BLE/UWB as geometric anchors with WiFi for ambient sensing; and against 6G ISAC, be the practical open implementation of the sensing data plane on hardware that exists today.
## 3. Decision — strengths to invest, weaknesses to fix
Invest (already differentiated): low-cost ambient perception on commodity radios; camera-free coverage (with the explicit caveat that camera-free ≠ privacy-preserving — that is what ADR-277/280 gates are for); edge-first execution; existing application surfaces (HA/Matter/HomeKit), to be extended toward ROS 2, OpenUSD, MQTT Sparkplug, OPC UA, BIM/digital-twin connectors as demand proves out.
Fix (each has a concrete ADR): platform/world-model claim mixing → this ADR's ladder; no persistent spatial representation → ADR-275 (feed RuVector, don't contain everything in the sensing server); ESP32-specific pipeline risk → ADR-279 adapters; stream-only operation → ADR-280 sensing tasks.
## 4. Decision — the public evidence ladder (mandatory)
`frame::EvidenceLevel` is now a type, and its use is policy:
| Level | Meaning |
|---|---|
| L0 | Simulation only |
| L1 | Captured replay |
| L2 | Controlled laboratory |
| L3 | Held-out room + subject validation |
| L4 | Multi-site field pilot |
| L5 | Production operational evidence |
Rules: (a) every capability row in README/registry carries exactly one level; (b) `ProvenanceClass::Synthetic` frames are L0 *by type* and measured frames are ≥ L1 — the constructor rejects both aliasing directions (ADR-279 invariant 6); (c) a level upgrade requires the corresponding artifact (a replay corpus, a lab protocol, a strict-split manifest per ADR-279 §4, a pilot report); (d) the hidden real-world test set used for L3+ claims is never accessible to synthetic generation, augmentation, or calibration. Everything shipped in ADR-273..281 is **L0** except the adapter/contract layers, which are code-level (no accuracy claim to grade).
## 5. Commercial focus (bounded claims per vertical)
Elder care (decision support and anomaly escalation, **not** diagnosis); smart buildings (occupancy/utilization; value = energy + space + safety cost); industrial safety (works in dust/darkness/occlusion; **not** a certified safety system until field-validated); security (through-wall occupancy with the surveillance-governance gates of ADR-277/280 as a feature, not friction); robotics (RuView is probabilistic exteroception, never ground truth).
## 6. The moat
Not any single detector: the *combination* of broad hardware support (ADR-279 adapters), heterogeneous data with provenance, cross-environment pretrained encoders under anti-leakage evaluation (ADR-273 §4), calibration/uncertainty discipline, privacy-preserving edge execution (ADR-277/280), cryptographic evidence (RuField bridge), persistent spatial memory (ADR-275 → RuVector), and open integration. Harder to reproduce than any model.
## 7. Acceptance test (ecosystem-fit)
RuView fits the mature stack when a **frozen** encoder ingests WiFi CSI, radar, and Bluetooth measurements from previously unseen hardware, emits RuField-compliant observations, updates a persistent RuVector spatial model, and supports an agent query with: ≤ 0.5 m p90 localization; < 20 % degradation across unseen rooms; explicit uncertainty on every result; complete calibration + provenance lineage; no P0 RF leaving the edge; replay/lab/live evidence clearly separated; successful fusion with a standard robotics or digital-twin platform. Tracked as the L4 gate; the synthetic analogue machinery already exists (`tests/e2e_acceptance.rs`).
+10
View File
@@ -132,6 +132,16 @@ Statuses: **Proposed** (under discussion), **Accepted** (approved and/or impleme
| [ADR-263](ADR-263-ruview-npm-harness-deep-review.md) | `@ruvnet/ruview` npm harness — deep review + optimization strategy | Proposed |
| [ADR-264](ADR-264-rvagent-mcp-and-cli-npm-deep-review.md) | `@ruvnet/rvagent` MCP server + `@ruv/ruview-cli` — deep review + optimization strategy | Proposed |
| [ADR-265](ADR-265-ruview-npm-distribution-strategy.md) | RuView npm distribution strategy — CI gate, provenance, version single-sourcing, namespace | Proposed |
| [ADR-273](ADR-273-unified-rf-spatial-world-model.md) | Unified RF spatial world model — umbrella, anti-leakage protocol, acceptance gates | Accepted (P1 implemented) |
| [ADR-274](ADR-274-universal-rf-encoder-adapter-registry.md) | Universal RF foundation encoder + hardware adapter registry | Accepted (P1 implemented) |
| [ADR-275](ADR-275-rf-aware-gaussian-spatial-memory.md) | RF-aware Gaussian spatial memory | Accepted (P1 implemented) |
| [ADR-276](ADR-276-physics-guided-synthetic-rf-worlds.md) | Physics-guided synthetic RF world generator | Accepted (P1 implemented) |
| [ADR-277](ADR-277-edge-sensing-control-plane.md) | Edge sensing control plane (802.11bf / ETSI ISAC aligned) | Accepted (P1 implemented) |
| [ADR-278](ADR-278-radar-inverse-rendering-research-program.md) | Radar inverse rendering + differentiable RF SLAM research program | Proposed |
| [ADR-279](ADR-279-native-rf-frame-contract.md) | Native RF frame contract — `RfFrameV2` authoritative, canonical tensor derived | Accepted (implemented) |
| [ADR-280](ADR-280-active-sensing-programmable-perception.md) | Active sensing & programmable perception control plane | Accepted (implemented) |
| [ADR-281](ADR-281-ble-cs-delay-doppler-pose-factorization.md) | BLE Channel Sounding, delay-Doppler tensors, P3162 import, factorized pose | Accepted (implemented) |
| [ADR-282](ADR-282-ruview-ecosystem-positioning.md) | Ecosystem positioning + mandatory L0L5 evidence ladder | Accepted |
---
+16 -10
View File
@@ -4,9 +4,12 @@ Operations doc for the `.github/workflows/pip-release.yml` CI workflow.
## Auth
The workflow uses one GitHub Actions secret named `PYPI_API_TOKEN`.
It's a project-token issued by the rUv PyPI account with upload
scope for both `wifi-densepose` and `ruview`.
Production uses the GitHub Actions secret `PYPI_API_TOKEN`. It is a
project token issued by the rUv PyPI account with upload scope for both
`wifi-densepose` and `ruview`.
TestPyPI uses a separate `TESTPYPI_API_TOKEN` secret issued by
test.pypi.org. PyPI and TestPyPI accounts and tokens are independent.
## Refreshing the token
@@ -47,16 +50,19 @@ Per ADR-117 §7.3, the tombstone publishes first so it claims the
tombstone live at `https://pypi.org/project/wifi-densepose/1.99.0/`
2. Verify: `pip install wifi-densepose==1.99.0; python -c "import
wifi_densepose"` → ImportError with migration URL.
3. `git tag v2.0.0-pip && git push origin v2.0.0-pip` → v2 wheel
matrix live at `https://pypi.org/project/wifi-densepose/2.0.0/`.
4. (Optional, in lock-step) build + publish a matching `ruview`
release from `python/ruview-meta/` so the meta-package version
stays pinned to the same wifi-densepose version.
3. Confirm `archive/v1/data/proof/expected_features_v2.sha256` is
committed and non-empty. Production publishing fails closed without it.
4. `git tag v2.0.0-pip && git push origin v2.0.0-pip` → the v2
`wifi-densepose` wheel matrix and matching `ruview` wheel/sdist are
published together. Their versions and dependency pin are checked in CI.
5. Verify both `https://pypi.org/project/wifi-densepose/2.0.0/` and
`https://pypi.org/project/ruview/2.0.0/`.
## Off-loop manual gates
- **Q3** (ADR-117 §11.3) — generate `expected_features_v2.sha256`
from the v2 Rust pipeline before any v2 publish.
- **Q3** (ADR-117 §11.3) — generate
`archive/v1/data/proof/expected_features_v2.sha256` from the v2 Rust
pipeline before a production v2 publish. The workflow enforces this gate.
- **OIDC Trusted Publisher** — not used. The workflow is token-based;
this is a deliberate choice to keep the secret refresh entirely in
GCP. If the project migrates to OIDC later, remove `password:`
+8 -5
View File
@@ -252,8 +252,9 @@ impl PyBreathingExtractor {
/// hr = HeartRateExtractor.esp32_default() # 56 subcarriers, 100 Hz, 15s window
///
/// # Feed residuals and matching unwrapped phases from your preprocessor.
/// # Unlike BreathingExtractor weights, phases=[] is invalid for heart-rate
/// # extraction because the Rust core requires phase data for each subcarrier.
/// # Like BreathingExtractor's weights, phases=[] means "no per-subcarrier
/// # coherence information available" and falls back to equal weighting
/// # across all subcarriers -- it does NOT silently drop every frame.
/// est = hr.extract(residuals=[0.01, -0.02, …], phases=[0.0, 0.01, …])
/// if est is not None:
/// print(est.value_bpm, est.confidence)
@@ -281,9 +282,11 @@ impl PyHeartRateExtractor {
}
/// Extract heart rate from per-subcarrier residuals and matching
/// per-subcarrier unwrapped phases (radians). Empty phases are invalid
/// and return `None` because the Rust extractor requires phase data.
/// GIL released during DSP.
/// per-subcarrier unwrapped phases (radians). A short or empty `phases`
/// slice falls back to equal weighting for any subcarrier missing phase
/// data (issue #1423) -- it does not truncate the number of subcarriers
/// fused, and does not silently return `None` for every frame. GIL
/// released during DSP.
fn extract(
&mut self,
py: Python<'_>,
+32
View File
@@ -223,6 +223,38 @@ def test_heart_rate_extract_with_synthetic_signal_and_phases() -> None:
)
def test_heart_rate_extract_with_empty_phases_produces_estimates() -> None:
"""Issue #1423 regression: `phases=[]` must fall back to equal
weighting (mirroring `BreathingExtractor`'s `weights=[]`), not silently
return `None` for every frame.
Reproduces the GH-issue repro: a noiseless 1.2 Hz (72 BPM) sine
identical across all 56 subcarriers, fed frame-by-frame with an empty
`phases` list. Before the fix this produced 0/4000 estimates; the same
signal with `phases=[1.0] * 56` already produced thousands.
"""
hr = HeartRateExtractor.esp32_default()
sample_rate = 100.0
target_freq = 1.2 # 72 BPM
n_samples = 4000
produced = 0
for i in range(n_samples):
t = i / sample_rate
base = math.sin(2.0 * math.pi * target_freq * t)
residuals = [base] * 56
est = hr.extract(residuals=residuals, phases=[])
if est is not None:
produced += 1
assert math.isfinite(est.value_bpm)
assert 0.0 <= est.confidence <= 1.0
assert produced > 0, (
"HeartRateExtractor.extract(residuals=..., phases=[]) must not silently "
"return None for every frame of a clean 72 BPM signal (issue #1423)"
)
# ─── Build feature flag ──────────────────────────────────────────────
+7 -3
View File
@@ -8,6 +8,7 @@
* - Dot-matrix mist body mass, particle trails, WiFi waves, signal field
* - Reflective floor, settings dialog, and practical data HUD
*/
import { withWsTicket } from '../../services/ws-ticket.js';
import * as THREE from 'three';
import { OrbitControls } from 'three/addons/controls/OrbitControls.js';
@@ -462,7 +463,7 @@ class Observatory {
console.log('[Observatory] Sensing server detected at', base, '→', wsUrl);
this.settings.dataSource = 'ws';
this.settings.wsUrl = wsUrl;
this._connectWS(wsUrl);
void this._connectWS(wsUrl);
} else {
tryNext(i + 1);
}
@@ -472,10 +473,13 @@ class Observatory {
tryNext(0);
}
_connectWS(url) {
// async: `/ws/sensing` is gated (ADR-272); mint a single-use ticket first.
async _connectWS(url) {
this._disconnectWS();
let wsUrl = url;
try { wsUrl = await withWsTicket(url); } catch { /* auth off or pre-ADR-272 server */ }
try {
this._ws = new WebSocket(url);
this._ws = new WebSocket(wsUrl);
this._ws.onopen = () => {
console.log('[Observatory] WebSocket connected');
this._hud.updateSourceBadge('ws', this._ws);
+15
View File
@@ -86,6 +86,21 @@ export class ApiService {
// Process response through interceptors
const processedResponse = await this.processResponse(response, url);
// NOTE: there is deliberately no step-up re-authentication branch here.
//
// An earlier revision caught the server's RFC 6750 "reauthentication
// required" challenge and redirected to /oauth/start. That challenge can
// never be issued to a browser: browser sign-in requests `sensing:read`
// only and always will (see BROWSER_SIGNIN_SCOPE), so no browser session
// holds `sensing:admin`, so the freshness gate the challenge announces is
// never reached. Admin work goes through the CLI or a pasted bearer.
//
// Removed rather than left inert, because it was not merely dead — it
// ended in a promise that never settles. If any other 401 ever grew that
// header, every caller awaiting this would hang forever with no error.
// The server-side guard stays as a fail-closed backstop; the client has
// nothing to do about a flow that does not exist.
// Handle errors
if (!processedResponse.ok) {
const error = await processedResponse.json().catch(() => ({
+18 -4
View File
@@ -1,3 +1,4 @@
import { withWsTicket } from './ws-ticket.js';
/**
* Sensing WebSocket Service
*
@@ -65,7 +66,7 @@ class SensingService {
/** Start the service (connect or simulate). */
start() {
this._connect();
void this._connect();
}
/** Stop the service entirely. */
@@ -120,13 +121,26 @@ class SensingService {
// ---- Connection --------------------------------------------------------
_connect() {
// async because the server gates `/ws/sensing` (ADR-272) and a browser
// cannot set an Authorization header on an upgrade — so we mint a
// single-use ticket first. Minted per connect attempt, never cached: a
// ticket is valid once and expires in seconds, so reusing one across
// reconnects would fail on the second attempt.
async _connect() {
if (this._ws && this._ws.readyState <= WebSocket.OPEN) return;
this._setState('connecting');
let url = SENSING_WS_URL;
try {
this._ws = new WebSocket(SENSING_WS_URL);
url = await withWsTicket(SENSING_WS_URL);
} catch {
// Ticket minting is best-effort: against a server with auth off, or one
// predating ADR-272, connecting without a ticket is correct.
}
try {
this._ws = new WebSocket(url);
} catch (err) {
console.warn('[Sensing] WebSocket constructor failed:', err.message);
this._fallbackToSimulation();
@@ -184,7 +198,7 @@ class SensingService {
this._reconnectTimer = setTimeout(() => {
this._reconnectTimer = null;
this._connect();
void this._connect();
}, delay);
// Only start simulation after several failed attempts so a brief hiccup
+10 -3
View File
@@ -1,3 +1,4 @@
import { withWsTicket } from './ws-ticket.js';
// WebSocket Client for Three.js Visualization - WiFi DensePose
// Default endpoint is `/ws/sensing` on the same host the page was served from.
// Callers (e.g. viz.html) usually pass an explicit `url` derived from
@@ -47,7 +48,9 @@ export class WebSocketClient {
}
// Attempt to connect
connect() {
// async: `/ws/*` is gated (ADR-272) and a browser cannot set an
// Authorization header on an upgrade, so mint a single-use ticket first.
async connect() {
if (this.state === 'connecting' || this.state === 'connected') {
console.warn('[WS-VIZ] Already connected or connecting');
return;
@@ -56,8 +59,12 @@ export class WebSocketClient {
this._setState('connecting');
console.log(`[WS-VIZ] Connecting to ${this.url}`);
// Per attempt, never cached — a ticket is single-use and short-lived.
let url = this.url;
try { url = await withWsTicket(this.url); } catch { /* auth off or pre-ADR-272 server */ }
try {
this.ws = new WebSocket(this.url);
this.ws = new WebSocket(url);
this.ws.binaryType = 'arraybuffer';
this.ws.onopen = () => this._handleOpen();
@@ -235,7 +242,7 @@ export class WebSocketClient {
console.log(`[WS-VIZ] Reconnecting in ${delay}ms (attempt ${this.reconnectAttempts}/${this.maxReconnectAttempts})`);
this.reconnectTimer = setTimeout(() => {
this.connect();
void this.connect();
}, delay);
}
+77
View File
@@ -0,0 +1,77 @@
// Single-use WebSocket tickets (ADR-272).
//
// A browser's WebSocket constructor cannot set an `Authorization` header on the
// upgrade request. That used to mean the sensing WebSocket stayed reachable
// with no credential even when the server had auth switched on — the REST
// control plane was locked while the live sensing stream was open.
//
// The server now gates `/ws/*` and `/api/v1/stream/pose`. Browsers exchange
// their stored bearer token at `POST /api/v1/ws-ticket` — an ordinary request,
// where headers DO work — for a short-lived, single-use ticket, and pass that
// as `?ticket=` on the socket URL.
//
// Why a token in a URL is acceptable here when it normally is not: the ticket
// is consumed on first use, lives ~30 seconds, and authorizes one WebSocket
// and nothing else. It cannot be replayed against /api/v1/*. The long-lived
// bearer token is still never put in a URL.
import { API_TOKEN_STORAGE_KEY } from './api.service.js';
function storedToken() {
try {
return localStorage.getItem(API_TOKEN_STORAGE_KEY) || null;
} catch {
// Private browsing / storage disabled — treat as "no token configured".
return null;
}
}
/**
* Mint a ticket. Returns null when no token is configured (auth is off, so no
* ticket is needed) or when the server does not offer the endpoint.
*/
async function mintTicket() {
const token = storedToken();
if (!token) return null;
try {
const resp = await fetch('/api/v1/ws-ticket', {
method: 'POST',
headers: { Authorization: `Bearer ${token}` },
});
// 404 means a server predating ADR-272: it still exempts WebSockets, so
// connecting without a ticket is correct there. Treated as "no ticket
// needed" rather than as an error, so the UI works against both.
if (resp.status === 404) return null;
if (!resp.ok) {
console.warn('[ws-ticket] mint failed:', resp.status);
return null;
}
const body = await resp.json();
return body.ticket || null;
} catch (err) {
// Offline, or the server is down. The socket attempt will fail on its own
// and the caller's reconnect logic handles it; failing loudly here would
// just duplicate that.
console.warn('[ws-ticket] mint error:', err.message);
return null;
}
}
/**
* Return `url` with a freshly minted ticket appended, or unchanged when no
* ticket is available or needed.
*
* Always call this immediately before opening the socket — a ticket expires in
* seconds and is valid exactly once, so one must never be cached or reused
* across reconnects.
*
* @param {string} url ws:// or wss:// URL
* @returns {Promise<string>}
*/
export async function withWsTicket(url) {
const ticket = await mintTicket();
if (!ticket) return url;
const sep = url.includes('?') ? '&' : '?';
return `${url}${sep}ticket=${encodeURIComponent(ticket)}`;
}
+113
View File
@@ -0,0 +1,113 @@
// Executed tests for the WebSocket ticket helper (ADR-272).
//
// Run: node --test ui/sw.test.mjs ui/services/ws-ticket.test.mjs
// (a directory argument does not work — Node resolves it as a module)
//
// WHAT THIS IS AND IS NOT.
// This EXECUTES the module in Node with stubbed `fetch` and `localStorage`. It
// is strictly more than the `node --check` syntax pass this file replaces, and
// strictly less than a browser: it does not exercise a real WebSocket upgrade,
// real cookie handling, or the page wiring. The claim "the UI JavaScript has
// never been run" is no longer true of this module; "browser-tested" still is
// not. Both statements matter and neither should be rounded up.
import { test, beforeEach } from 'node:test';
import assert from 'node:assert/strict';
const STORAGE_KEY = 'ruview-api-token';
// --- stubs installed before the module under test is imported ---------------
let stored = {};
let fetchCalls = [];
let fetchImpl = async () => ({ ok: true, status: 200, json: async () => ({ ticket: 'T' }) });
globalThis.localStorage = {
getItem: (k) => (k in stored ? stored[k] : null),
setItem: (k, v) => { stored[k] = String(v); },
removeItem: (k) => { delete stored[k]; },
};
globalThis.fetch = async (...args) => { fetchCalls.push(args); return fetchImpl(...args); };
const { withWsTicket } = await import('./ws-ticket.js');
beforeEach(() => {
stored = {};
fetchCalls = [];
fetchImpl = async () => ({ ok: true, status: 200, json: async () => ({ ticket: 'T' }) });
});
test('with no stored token the URL is returned unchanged and nothing is fetched', async () => {
// Auth is off, so no ticket is needed. Minting one would be a pointless
// round-trip on every reconnect.
const url = await withWsTicket('ws://host/ws/sensing');
assert.equal(url, 'ws://host/ws/sensing');
assert.equal(fetchCalls.length, 0);
});
test('a minted ticket is appended and the bearer is sent only in the header', async () => {
stored[STORAGE_KEY] = 'secret-bearer';
const url = await withWsTicket('ws://host/ws/sensing');
assert.equal(url, 'ws://host/ws/sensing?ticket=T');
// The long-lived bearer must never reach a URL — only the bounded ticket does.
assert.ok(!url.includes('secret-bearer'), `bearer leaked into URL: ${url}`);
const [path, init] = fetchCalls[0];
assert.equal(path, '/api/v1/ws-ticket');
assert.equal(init.method, 'POST');
assert.equal(init.headers.Authorization, 'Bearer secret-bearer');
});
test('an existing query string gets & rather than a second ?', async () => {
stored[STORAGE_KEY] = 'b';
const url = await withWsTicket('ws://host/ws/sensing?foo=1');
assert.equal(url, 'ws://host/ws/sensing?foo=1&ticket=T');
});
test('the ticket value is URL-encoded', async () => {
stored[STORAGE_KEY] = 'b';
fetchImpl = async () => ({ ok: true, status: 200, json: async () => ({ ticket: 'a+b/c=d' }) });
const url = await withWsTicket('ws://host/ws/sensing');
assert.ok(url.endsWith('ticket=a%2Bb%2Fc%3Dd'), url);
});
test('404 means a server predating ADR-272, so connect without a ticket', async () => {
// That server still exempts WebSockets, so an unticketed connect is correct.
// This is what lets one UI work against both old and new servers, which is
// what makes the legacy escape hatch removable rather than permanent.
stored[STORAGE_KEY] = 'b';
fetchImpl = async () => ({ ok: false, status: 404, json: async () => ({}) });
assert.equal(await withWsTicket('ws://host/ws/sensing'), 'ws://host/ws/sensing');
});
test('a 503 does not append a ticket and does not throw', async () => {
// Ticket store exhausted. The socket attempt will fail on its own and the
// caller's reconnect logic handles it; throwing here would duplicate that.
stored[STORAGE_KEY] = 'b';
fetchImpl = async () => ({ ok: false, status: 503, json: async () => ({}) });
assert.equal(await withWsTicket('ws://host/ws/sensing'), 'ws://host/ws/sensing');
});
test('a network failure is swallowed rather than breaking the connect path', async () => {
stored[STORAGE_KEY] = 'b';
fetchImpl = async () => { throw new Error('offline'); };
assert.equal(await withWsTicket('ws://host/ws/sensing'), 'ws://host/ws/sensing');
});
test('a 200 with no ticket field yields no ticket', async () => {
stored[STORAGE_KEY] = 'b';
fetchImpl = async () => ({ ok: true, status: 200, json: async () => ({}) });
assert.equal(await withWsTicket('ws://host/ws/sensing'), 'ws://host/ws/sensing');
});
test('a fresh ticket is minted per call and never reused', async () => {
// Tickets are single-use and expire in ~30s, so caching one across reconnects
// fails on the second attempt.
stored[STORAGE_KEY] = 'b';
let n = 0;
fetchImpl = async () => ({ ok: true, status: 200, json: async () => ({ ticket: `T${++n}` }) });
assert.equal(await withWsTicket('ws://h/ws/sensing'), 'ws://h/ws/sensing?ticket=T1');
assert.equal(await withWsTicket('ws://h/ws/sensing'), 'ws://h/ws/sensing?ticket=T2');
assert.equal(fetchCalls.length, 2, 'each connect attempt must mint its own');
});
+91 -13
View File
@@ -1,7 +1,27 @@
// RuView Service Worker - Offline caching for the dashboard shell
// Strategy: Network-first for API calls, Cache-first for static assets
const CACHE_NAME = 'ruview-v1';
// Bumped from v1: an older SW cached `/oauth/status` cache-first, so browsers
// that ran it hold a permanently signed-out answer. `activate` deletes every
// cache whose name is not CACHE_NAME, so bumping is what evicts it from clients
// already in the field. Bump again if a future change poisons the cache.
const CACHE_NAME = 'ruview-v2';
// Requests whose response depends on the caller's credentials. These must never
// be served from the Cache API.
//
// The Cache API is NOT the HTTP cache: it ignores `Cache-Control` completely, so
// the `no-store, no-cache, must-revalidate` the server already sends on
// `/oauth/status` has no effect here. A cached signed-out response was returned
// to the page forever, and only a hard reload — which bypasses the service
// worker entirely — showed the true state. (ADR-271.)
const NEVER_CACHE_PREFIXES = ['/oauth/'];
// What may be served cache-first. Previously cache-first was the *catch-all* for
// everything outside `/api/` and `/health/`, which meant any endpoint added at a
// new path was frozen on first response. An allowlist fails safe instead: an
// unrecognised path goes to the network untouched.
const STATIC_ASSET = /\.(?:js|mjs|css|html|json|png|jpe?g|gif|svg|ico|webp|woff2?|ttf|map)$/i;
const SHELL_ASSETS = [
'/',
'/index.html',
@@ -74,25 +94,49 @@ self.addEventListener('fetch', (event) => {
// Skip cross-origin requests
if (url.origin !== self.location.origin) return;
// Credentialed endpoints: hands off entirely. Not networkFirst — that still
// writes a copy into the cache, which would be replayed the moment the server
// is briefly unreachable, silently reinstating a stale sign-in state.
if (NEVER_CACHE_PREFIXES.some((prefix) => url.pathname.startsWith(prefix))) {
// Signing out is the one moment we know cached data belongs to a session
// that is ending. Observed, not intercepted — the request itself still goes
// straight to the network. `waitUntil` keeps the worker alive for the purge
// even though the navigation is what the browser is really waiting on.
if (url.pathname === '/oauth/logout') {
event.waitUntil(purgeNonShell());
}
return;
}
// API calls: network-first with cache fallback
if (url.pathname.startsWith('/api/') || url.pathname.startsWith('/health/')) {
event.respondWith(networkFirst(request));
return;
}
// Static assets: cache-first with network fallback
event.respondWith(cacheFirst(request));
// Static assets and the app shell: cache-first with network fallback.
if (request.mode === 'navigate' || STATIC_ASSET.test(url.pathname)) {
event.respondWith(cacheFirst(request));
}
// Anything else is left alone and goes to the network as normal.
});
async function cacheFirst(request) {
const cached = await caches.match(request);
// `ignoreSearch` so the shell is a single entry. Sign-in redirects back to
// `/ui/?signed_in=<ms>`, which would otherwise mint a fresh cache entry per
// sign-in and never hit any of them again.
const cached = await caches.match(request, { ignoreSearch: true });
if (cached) return cached;
try {
const response = await fetch(request);
if (response.ok) {
const cache = await caches.open(CACHE_NAME);
cache.put(request, response.clone());
// Store under the search-less URL to match how it is looked up above.
const key = new URL(request.url);
key.search = '';
cache.put(key.toString(), response.clone());
}
return response;
} catch {
@@ -105,20 +149,54 @@ async function cacheFirst(request) {
}
}
/**
* Network-only, with an explicit offline signal.
*
* This used to cache every successful `/api/` response and replay it whenever
* the network failed. Two things are wrong with that now:
*
* 1. **Authorization.** API responses are per-user once auth is on, but the
* cache is keyed by URL alone and nothing purges it at sign-out. Sign in as
* A, load sensing data, sign out, sign in as B, lose the network — B is
* served A's data with no authorization check at all. That is the same
* defect class as the cached `/oauth/status`: the Cache API happily outlives
* the session that produced its contents.
* 2. **Correctness.** This is a live sensing dashboard. Replaying a stale pose
* or presence reading as if it were current is its own defect — it can show
* a room as occupied after the person has left.
*
* The offline shell (HTML/CSS/JS) is still cached; only the data is not. If
* offline data replay is wanted back, it needs a per-session cache key and a
* purge on sign-out, not a URL-keyed shared cache.
*/
async function networkFirst(request) {
try {
const response = await fetch(request);
if (response.ok) {
const cache = await caches.open(CACHE_NAME);
cache.put(request, response.clone());
}
return response;
return await fetch(request);
} catch {
const cached = await caches.match(request);
if (cached) return cached;
return new Response(JSON.stringify({ error: 'offline' }), {
status: 503,
headers: { 'Content-Type': 'application/json' }
});
}
}
/**
* Drop everything except the static shell.
*
* Called when the user signs out. Belt-and-braces: nothing user-specific should
* be in the cache after the `networkFirst` change above, but a cache populated
* by an OLDER worker on this browser can still hold API responses, and that
* worker's entries survive into this one under the same name.
*/
async function purgeNonShell() {
const cache = await caches.open(CACHE_NAME);
const keys = await cache.keys();
await Promise.all(
keys
.filter((req) => {
const p = new URL(req.url).pathname;
return p.startsWith('/api/') || p.startsWith('/health/');
})
.map((req) => cache.delete(req))
);
}
+228
View File
@@ -0,0 +1,228 @@
// Executed tests for the service worker's request routing (ADR-271/272).
//
// WHY THIS EXISTS.
// Browser sign-in appeared to fail: after a successful OAuth round-trip the
// settings panel still offered "Sign in with Cognitum", and only a hard reload
// showed the truth. The server was correct throughout — `/oauth/status` fell
// through to the service worker's cache-first catch-all, so the first
// (signed-out) response was stored in the Cache API and replayed forever.
//
// The Cache API is not the HTTP cache. It ignores `Cache-Control` entirely, so
// the `no-store` the server already sent could not prevent this. Nothing in the
// Rust suite, the UI unit tests, or a curl probe could observe it: curl has no
// service worker, and a hard reload bypasses one. It was only visible by
// driving a real browser.
//
// These tests load the REAL `sw.js` with stubbed worker globals and assert on
// which strategy each path is routed to.
//
// Run: node --test ui/sw.test.mjs ui/services/ws-ticket.test.mjs
// (a directory argument does not work — Node resolves it as a module)
import { test, beforeEach } from 'node:test';
import assert from 'node:assert/strict';
import { readFileSync } from 'node:fs';
import { fileURLToPath } from 'node:url';
import vm from 'node:vm';
const SW_SOURCE = readFileSync(fileURLToPath(new URL('./sw.js', import.meta.url)), 'utf8');
const ORIGIN = 'http://127.0.0.1:8099';
/** Load sw.js in a fresh context and return its registered `fetch` listener. */
function loadServiceWorker() {
const listeners = {};
const cachePuts = [];
// A real in-memory cache, so purge and put behaviour can be observed rather
// than assumed.
const entries = new Map();
const cacheStub = {
addAll: async () => {},
put: async (req, res) => {
const url = String(req.url ?? req);
cachePuts.push(url);
entries.set(url, res);
},
keys: async () => Array.from(entries.keys()).map((url) => ({ url })),
delete: async (req) => entries.delete(String(req.url ?? req)),
match: async () => undefined,
};
const sandbox = {
self: {
addEventListener: (name, fn) => { listeners[name] = fn; },
location: { origin: ORIGIN },
skipWaiting: () => {},
clients: { claim: () => {} },
},
caches: {
open: async () => cacheStub,
keys: async () => [],
delete: async () => true,
match: async () => undefined,
},
// Never actually reached: every assertion below inspects the routing
// decision, not the network.
fetch: async () => ({ ok: true, clone: () => ({}) }),
URL,
Response: class { constructor(body, init) { this.body = body; Object.assign(this, init); } },
console,
};
vm.createContext(sandbox);
vm.runInContext(SW_SOURCE, sandbox);
return { listeners, cachePuts, sandbox, entries };
}
/**
* Route one request and report the decision.
* `handled: false` means the SW called neither respondWith nor cache — the
* request goes to the network untouched, which is the only safe outcome for a
* credentialed endpoint.
*/
function route(path, opts = {}) {
return dispatch(path, opts).handled;
}
/** Route one request and expose everything the worker did with it. */
function dispatch(path, { method = 'GET', mode = 'cors', headers = {}, sw = null } = {}) {
const worker = sw ?? loadServiceWorker();
let handled = false;
let responded = null;
const waited = [];
const request = {
url: `${ORIGIN}${path}`,
method,
mode,
headers: { get: (k) => headers[k] ?? headers[k.toLowerCase()] ?? null },
};
worker.listeners.fetch({
request,
respondWith: (p) => { handled = true; responded = p; },
waitUntil: (p) => { waited.push(p); },
});
return { handled, responded, waited, worker };
}
let sw;
beforeEach(() => { sw = loadServiceWorker(); });
// --- the defect this file exists for ----------------------------------------
test('/oauth/status is never handled by the service worker', () => {
// The exact request whose cached signed-out copy made sign-in look broken.
assert.equal(route('/oauth/status'), false);
});
test('every /oauth/ path bypasses the worker, not just status', () => {
// start/callback/logout all carry or clear credentials. A cached redirect or
// Set-Cookie replayed later is worse than a cached status.
for (const p of ['/oauth/start', '/oauth/callback?code=x&state=y', '/oauth/logout']) {
assert.equal(route(p), false, `${p} must go straight to the network`);
}
});
// --- the underlying defect: cache-first was the catch-all -------------------
test('an unrecognised path is left to the network rather than cached', () => {
// This is what made the OAuth bug possible in the first place: any endpoint
// added outside /api/ was silently frozen on its first response. An
// allowlist means the next such endpoint is safe by default.
assert.equal(route('/some/future/endpoint'), false);
});
test('static assets are still served cache-first', () => {
// The offline shell is the point of the worker; the fix must not disable it.
for (const p of ['/app.js', '/style.css', '/components/TabManager.js', '/icons/logo.svg']) {
assert.equal(route(p), true, `${p} should be cache-first`);
}
});
test('a navigation request is still served cache-first', () => {
assert.equal(route('/ui/', { mode: 'navigate' }), true);
});
test('API paths are still routed through the worker', () => {
// Handled, but network-only — see the "not written to the cache" test below.
assert.equal(route('/api/v1/models'), true);
assert.equal(route('/health/live'), true);
});
// --- pre-existing guards, pinned so the rewrite did not drop them -----------
test('non-GET requests are ignored', () => {
assert.equal(route('/api/v1/models', { method: 'POST' }), false);
});
test('websocket upgrades are ignored', () => {
assert.equal(route('/ws/sensing', { headers: { Upgrade: 'websocket' } }), false);
});
test('cross-origin requests are ignored', () => {
const { listeners } = loadServiceWorker();
let handled = false;
listeners.fetch({
request: {
url: 'https://auth.cognitum.one/oauth/authorize',
method: 'GET',
mode: 'cors',
headers: { get: () => null },
},
respondWith: () => { handled = true; },
});
assert.equal(handled, false);
});
// --- authenticated API responses must not be retained ------------------------
// Filed by the cross-vendor prosecutor in the qe-court round after the
// /oauth/status fix: closing the /oauth/ leg left the /api/ leg open.
test('a successful API response is NOT written to the cache', async () => {
// The leak: cache keys are URLs, nothing partitions them by session, and
// nothing purged them at sign-out. Sign in as A, fetch sensing data, sign
// out, sign in as B, lose the network -> B is served A's data.
const { responded, worker } = dispatch('/api/v1/sensing/latest');
await responded;
assert.deepEqual(worker.cachePuts, [], 'API responses must not be cached');
});
test('an API request with no network returns 503 rather than stale data', async () => {
// Also a correctness property, not only an authorization one: replaying a
// stale pose reading as current can show a room occupied after the person
// has left.
const worker = loadServiceWorker();
worker.sandbox.fetch = async () => { throw new Error('offline'); };
worker.entries.set(`${ORIGIN}/api/v1/sensing/latest`, { stale: true });
const { responded } = dispatch('/api/v1/sensing/latest', { sw: worker });
const res = await responded;
assert.equal(res.status, 503);
assert.match(String(res.body), /offline/);
});
test('signing out purges cached API data but keeps the offline shell', async () => {
const worker = loadServiceWorker();
worker.entries.set(`${ORIGIN}/api/v1/sensing/latest`, {});
worker.entries.set(`${ORIGIN}/health/live`, {});
worker.entries.set(`${ORIGIN}/app.js`, {});
const { handled, waited } = dispatch('/oauth/logout', { sw: worker });
// Observed, not intercepted — the logout request itself must still reach the
// server, or signing out would not actually sign anyone out.
assert.equal(handled, false, '/oauth/logout must still go to the network');
assert.equal(waited.length, 1, 'the purge must be kept alive via waitUntil');
await Promise.all(waited);
const left = Array.from(worker.entries.keys());
assert.deepEqual(left, [`${ORIGIN}/app.js`], 'only the static shell should survive');
});
// --- cache hygiene -----------------------------------------------------------
test('the cache name is bumped so clients holding the poisoned v1 evict it', () => {
// `activate` deletes every cache whose name !== CACHE_NAME. Browsers that
// already ran the old worker hold a signed-out /oauth/status in `ruview-v1`;
// only a name change removes it for them.
assert.ok(!/['"]ruview-v1['"]/.test(SW_SOURCE), 'CACHE_NAME must not still be ruview-v1');
assert.ok(/CACHE_NAME\s*=\s*['"]ruview-v[2-9]/.test(SW_SOURCE));
});
+95
View File
@@ -74,6 +74,16 @@ export class QuickSettings {
<span class="qs-switch"></span>
</label>
</div>
<div class="qs-section">
<div class="qs-section-title">Cognitum Account</div>
<div class="qs-row" style="flex-direction: column; align-items: stretch; gap: 6px;">
<span id="qs-signin-status" style="font-size: 0.9em; opacity: 0.85;">Checking...</span>
<div style="display: flex; gap: 8px;">
<button class="qs-btn" id="qs-signin" hidden>Sign in with Cognitum</button>
<button class="qs-btn-danger" id="qs-signout" hidden>Sign out</button>
</div>
</div>
</div>
<div class="qs-section">
<div class="qs-section-title">API Access</div>
<div class="qs-row" style="flex-direction: column; align-items: stretch; gap: 6px;">
@@ -103,6 +113,23 @@ export class QuickSettings {
// Bind events
this.panel.querySelector('.qs-close').addEventListener('click', () => this.close());
// Re-check sign-in state whenever the page could be showing a stale view:
// `pageshow` fires on a back/forward-cache restore (where no script re-runs
// and no fetch would otherwise happen), and `visibilitychange` covers
// signing in or out in another tab. Opening the panel alone is not enough —
// the panel may already be open, or the page may be restored wholesale.
window.addEventListener('pageshow', () => { void refreshSignInPanel(this.panel); });
document.addEventListener('visibilitychange', () => {
if (!document.hidden) void refreshSignInPanel(this.panel);
});
// ADR-271 sign-in. Bound here as well as in refreshSignInPanel so a click
// works even if the status fetch has not resolved yet.
this.panel.querySelector('#qs-signin')
.addEventListener('click', () => { window.location.href = '/oauth/start'; });
this.panel.querySelector('#qs-signout')
.addEventListener('click', () => { window.location.href = '/oauth/logout'; });
this.panel.querySelector('#qs-reduced-motion').addEventListener('change', (e) => {
document.body.classList.toggle('reduced-motion', e.target.checked);
this.saveSetting('reduced-motion', e.target.checked);
@@ -211,6 +238,11 @@ export class QuickSettings {
open() {
this.isOpen = true;
this.panel.classList.add('open');
// Refresh on every open, not once at construction: the session may have
// been established in another tab, or expired since the page loaded.
// Fire-and-forget — a failure renders as a message in the panel, and must
// not stop the panel opening.
void refreshSignInPanel(this.panel);
}
close() {
@@ -233,3 +265,66 @@ export class QuickSettings {
this.panel?.remove();
}
}
// ---- Cognitum browser sign-in (ADR-271) -------------------------------------
//
// `/oauth/status` is intentionally UNGATED: a signed-out browser cannot ask a
// gated endpoint whether sign-in is available. It returns capability flags and,
// when a session exists, who it belongs to — never a credential.
//
// Sign-in is a full-page navigation, not fetch(): the server replies 302 to
// auth.cognitum.one, and the browser must follow it and carry the transaction
// cookie. An XHR would follow the redirect invisibly and land nowhere useful.
export async function refreshSignInPanel(root = document) {
const status = root.querySelector('#qs-signin-status');
const signIn = root.querySelector('#qs-signin');
const signOut = root.querySelector('#qs-signout');
if (!status || !signIn || !signOut) return null;
let info;
try {
const resp = await fetch('/oauth/status', { credentials: 'same-origin' });
// 404 = a server predating ADR-271. Say so plainly rather than offering a
// button that will 404.
if (resp.status === 404) {
status.textContent = 'This server does not support Cognitum sign-in.';
signIn.hidden = true;
signOut.hidden = true;
return null;
}
if (!resp.ok) throw new Error(`status ${resp.status}`);
info = await resp.json();
} catch (err) {
status.textContent = `Could not reach the server (${err.message}).`;
signIn.hidden = true;
signOut.hidden = true;
return null;
}
if (info.signed_in) {
status.textContent = `Signed in${info.account ? ` as ${info.account}` : ''}${
info.scope ? ` - ${info.scope}` : ''
}`;
signIn.hidden = true;
signOut.hidden = false;
} else if (info.browser_signin) {
status.textContent = info.auth_required
? 'This server requires sign-in.'
: 'Optional: sign in to use your Cognitum account.';
signIn.hidden = false;
signOut.hidden = true;
} else if (info.auth_required) {
// Auth is on but OAuth is not — the static-token panel below is the path.
status.textContent = 'This server uses a shared API token (see API Access below).';
signIn.hidden = true;
signOut.hidden = true;
} else {
status.textContent = 'This server does not require sign-in.';
signIn.hidden = true;
signOut.hidden = true;
}
signIn.onclick = () => { window.location.href = '/oauth/start'; };
signOut.onclick = () => { window.location.href = '/oauth/logout'; };
return info;
}
Generated
+670 -286
View File
File diff suppressed because it is too large Load Diff
+12 -1
View File
@@ -29,6 +29,12 @@ members = [
# geo + worldgraph extracted to ruvnet/worldgraph submodule (see crates/worldgraph)
"crates/wifi-densepose-engine", # ADR-135..146 integration/composition layer
"crates/wifi-densepose-calibration", # ADR-151 — per-room calibration & specialist training
# ADR-271 — Cognitum OAuth access-token verification. RuView as an OAuth
# RESOURCE SERVER: offline ES256/JWKS verification of tokens issued by
# auth.cognitum.one, so a user signs in to their own sensing server with
# their Cognitum identity instead of a shared static bearer. No login flow
# and no outbound Cognitum API calls live here — verification only.
"crates/ruview-auth",
"crates/nvsim",
"crates/nvsim-server",
"crates/homecore", # ADR-127 — HOMECORE state machine
@@ -72,6 +78,11 @@ members = [
"crates/homecore-assist", # ADR-133 — HOMECORE voice assistant + ruflo bridge
"crates/homecore-server", # iter-9 — HOMECORE integration binary (all 8 crates wired together)
"crates/ruview-swarm", # ADR-148 — drone swarm control system
# ADR-273..277 — unified RF spatial world model: canonical RF tensor +
# hardware adapters, universal foundation encoder (masked-reconstruction
# pretraining, ≤1% task adapters), RF-aware Gaussian spatial memory,
# physics-guided synthetic RF worlds, edge sensing control plane.
"crates/ruview-unified",
# ADR-262 P1 — anti-corruption bridge converting RuView WiFi-CSI sensing
# output into signed RuField FieldEvents. Path-deps the `vendor/rufield`
# submodule crates (rufield-core/-provenance/-privacy/-fusion); single
@@ -92,7 +103,7 @@ exclude = [
]
[workspace.package]
version = "0.3.0"
version = "0.3.1"
edition = "2021"
authors = ["rUv <ruv@ruv.net>", "WiFi-DensePose Contributors"]
license = "MIT OR Apache-2.0"
+2
View File
@@ -17,6 +17,8 @@ path = "src/bin/server.rs"
[dependencies]
homecore = { path = "../homecore", version = "0.1.0-alpha.0" }
homecore-automation = { path = "../homecore-automation", version = "0.1.0-alpha.0" }
homecore-recorder = { path = "../homecore-recorder", version = "0.1.0-alpha.0" }
axum = { version = "0.7", features = ["ws", "json", "macros"] }
tokio = { version = "1", features = ["full"] }
+22 -6
View File
@@ -27,6 +27,7 @@ pub fn router(state: SharedState) -> Router {
Router::new()
.route("/api/", get(rest::api_root))
.route("/api/config", get(rest::get_config))
.route("/api/components", get(rest::get_components))
.route("/api/states", get(rest::get_states))
.route(
"/api/states/:entity_id",
@@ -36,6 +37,22 @@ pub fn router(state: SharedState) -> Router {
)
.route("/api/services", get(rest::get_services))
.route("/api/services/:domain/:service", post(rest::call_service))
.route("/api/events", get(rest::get_events))
.route("/api/events/:event_type", post(rest::fire_event))
.route("/api/template", post(rest::render_template))
.route("/api/config/core/check_config", post(rest::check_config))
.route("/api/error_log", get(rest::error_log))
.route("/api/history/period", get(rest::get_history))
.route(
"/api/history/period/:start_time",
get(rest::get_history_period),
)
.route("/api/logbook", get(rest::get_logbook))
.route("/api/logbook/:start_time", get(rest::get_logbook_period))
.route("/api/calendars", get(rest::get_calendars))
.route("/api/calendars/:entity_id", get(rest::get_calendar_events))
.route("/api/camera_proxy/:entity_id", get(rest::get_camera_proxy))
.route("/api/homecore/compatibility", get(rest::compatibility))
.route("/api/websocket", get(ws::websocket_handler))
.layer(cors)
.layer(TraceLayer::new_for_http())
@@ -58,11 +75,7 @@ pub fn build_cors_layer() -> CorsLayer {
CorsLayer::new()
.allow_origin(AllowOrigin::list(origins))
.allow_methods([Method::GET, Method::POST, Method::OPTIONS, Method::DELETE])
.allow_headers([
header::AUTHORIZATION,
header::CONTENT_TYPE,
header::ACCEPT,
])
.allow_headers([header::AUTHORIZATION, header::CONTENT_TYPE, header::ACCEPT])
.allow_credentials(false)
}
@@ -108,7 +121,10 @@ mod tests {
#[test]
fn env_override_via_homecore_cors_origins() {
let _env = ENV_LOCK.lock().unwrap_or_else(|e| e.into_inner());
std::env::set_var("HOMECORE_CORS_ORIGINS", "https://example.com,https://other.example.com");
std::env::set_var(
"HOMECORE_CORS_ORIGINS",
"https://example.com,https://other.example.com",
);
// build_cors_layer() returns a CorsLayer which doesn't expose
// its origin list; we test the parse path indirectly by
// confirming no panic + at least one origin would parse.
+13 -4
View File
@@ -40,13 +40,22 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
// Token provisioning (HC-WS-08). Prefer the HOMECORE_TOKENS env
// whitelist; fall back to DEV mode (warn-logged) only when unset.
let tokens = if std::env::var("HOMECORE_TOKENS")
let has_tokens = std::env::var("HOMECORE_TOKENS")
.map(|v| !v.trim().is_empty())
.unwrap_or(false)
{
.unwrap_or(false);
let insecure_dev_auth = std::env::var("HOMECORE_INSECURE_DEV_AUTH").as_deref() == Ok("1");
if !has_tokens && !insecure_dev_auth {
return Err(
"HOMECORE_TOKENS is required (or set HOMECORE_INSECURE_DEV_AUTH=1 for loopback-only development)"
.into(),
);
}
let tokens = if has_tokens {
let s = LongLivedTokenStore::from_env();
let n = s.len().await;
tracing::info!("LongLivedTokenStore provisioned with {n} bearer token(s) from HOMECORE_TOKENS");
tracing::info!(
"LongLivedTokenStore provisioned with {n} bearer token(s) from HOMECORE_TOKENS"
);
s
} else {
tracing::warn!(
+6 -1
View File
@@ -14,6 +14,8 @@ pub enum ApiError {
Unauthorized,
#[error("service not registered: {domain}.{service}")]
ServiceNotRegistered { domain: String, service: String },
#[error("service unavailable: {0}")]
Unavailable(String),
#[error("internal error: {0}")]
Internal(String),
}
@@ -21,7 +23,9 @@ pub enum ApiError {
pub type ApiResult<T> = Result<T, ApiError>;
#[derive(Serialize)]
struct ErrorPayload { message: String }
struct ErrorPayload {
message: String,
}
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
@@ -30,6 +34,7 @@ impl IntoResponse for ApiError {
Self::BadRequest(_) => (StatusCode::BAD_REQUEST, self.to_string()),
Self::Unauthorized => (StatusCode::UNAUTHORIZED, self.to_string()),
Self::ServiceNotRegistered { .. } => (StatusCode::BAD_REQUEST, self.to_string()),
Self::Unavailable(_) => (StatusCode::SERVICE_UNAVAILABLE, self.to_string()),
Self::Internal(_) => (StatusCode::INTERNAL_SERVER_ERROR, self.to_string()),
};
(status, Json(ErrorPayload { message })).into_response()
+605 -23
View File
@@ -1,5 +1,6 @@
use axum::extract::{Path, State};
use axum::extract::{Path, Query, State};
use axum::http::{HeaderMap, StatusCode};
use axum::response::IntoResponse;
use axum::Json;
use serde::{Deserialize, Serialize};
@@ -10,7 +11,9 @@ use crate::error::{ApiError, ApiResult};
use crate::state::SharedState;
#[derive(Serialize)]
pub struct ApiRunning { message: &'static str }
pub struct ApiRunning {
message: &'static str,
}
/// `GET /api/` — the HA `APIStatusView` ("API running." ping).
///
@@ -23,9 +26,14 @@ pub struct ApiRunning { message: &'static str }
/// HOMECORE-API endpoint. The P2 handler skipped the bearer gate that
/// every sibling route applies; this restores wire-compat by validating
/// the bearer like `get_config`/`get_states` before replying.
pub async fn api_root(headers: HeaderMap, State(s): State<SharedState>) -> ApiResult<Json<ApiRunning>> {
pub async fn api_root(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<ApiRunning>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok(Json(ApiRunning { message: "API running." }))
Ok(Json(ApiRunning {
message: "API running.",
}))
}
#[derive(Serialize)]
@@ -36,16 +44,44 @@ pub struct ApiConfig {
components: Vec<String>,
}
pub async fn get_config(headers: HeaderMap, State(s): State<SharedState>) -> ApiResult<Json<ApiConfig>> {
const LOADED_COMPONENTS: &[&str] = &[
"api",
"automation",
"config",
"homecore",
"recorder",
"websocket_api",
];
pub async fn get_config(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<ApiConfig>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok(Json(ApiConfig {
location_name: s.location_name().to_string(),
version: s.version().to_string(),
state: "RUNNING",
components: vec![],
components: LOADED_COMPONENTS
.iter()
.map(|component| (*component).to_owned())
.collect(),
}))
}
pub async fn get_components(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<Vec<String>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok(Json(
LOADED_COMPONENTS
.iter()
.map(|component| (*component).to_owned())
.collect(),
))
}
#[derive(Serialize)]
pub struct StateView {
pub entity_id: String,
@@ -56,6 +92,328 @@ pub struct StateView {
pub context: ContextView,
}
#[derive(Debug, Deserialize)]
pub struct HistoryQuery {
filter_entity_id: Option<String>,
end_time: Option<String>,
#[serde(default)]
minimal_response: bool,
#[serde(default)]
no_attributes: bool,
#[serde(default)]
significant_changes_only: bool,
}
const MAX_HISTORY_ENTITIES: usize = 32;
const MAX_API_HISTORY_ROWS: usize = 100_000;
pub async fn get_history(
headers: HeaderMap,
State(s): State<SharedState>,
Query(query): Query<HistoryQuery>,
) -> ApiResult<Json<Vec<Vec<StateView>>>> {
history_response(headers, s, None, query).await
}
pub async fn get_history_period(
headers: HeaderMap,
State(s): State<SharedState>,
Path(start_time): Path<String>,
Query(query): Query<HistoryQuery>,
) -> ApiResult<Json<Vec<Vec<StateView>>>> {
history_response(headers, s, Some(start_time), query).await
}
async fn history_response(
headers: HeaderMap,
state: SharedState,
start_time: Option<String>,
query: HistoryQuery,
) -> ApiResult<Json<Vec<Vec<StateView>>>> {
let _ = BearerAuth::from_headers(&headers, state.tokens()).await?;
let recorder = state
.recorder()
.ok_or_else(|| ApiError::Unavailable("recorder is disabled".into()))?;
let now = chrono::Utc::now();
let start = match start_time {
Some(value) => parse_history_time(&value)?,
None => now - chrono::Duration::days(1),
};
let end = match query.end_time.as_deref() {
Some(value) => parse_history_time(value)?,
None => now,
};
if end < start {
return Err(ApiError::BadRequest(
"end_time must not precede start_time".into(),
));
}
let explicit_filter = query.filter_entity_id.is_some();
let entity_ids = match query.filter_entity_id.as_deref() {
Some(raw) => raw
.split(',')
.map(str::trim)
.filter(|value| !value.is_empty())
.map(|value| {
EntityId::parse(value)
.map_err(|error| ApiError::BadRequest(format!("invalid entity_id: {error}")))
})
.collect::<ApiResult<Vec<_>>>()?,
None => state
.homecore()
.states()
.all()
.into_iter()
.map(|snapshot| snapshot.entity_id.clone())
.collect(),
};
// Only reject an explicit, unusually-large `filter_entity_id` list. The
// real HA frontend's history page calls this endpoint with NO filter by
// design (meaning "all entities") — a real install routinely has 50-500+
// entities, so applying this cap there rejected the single most common
// call shape outright. The `MAX_API_HISTORY_ROWS` total-row budget below
// already bounds the actual work regardless of entity count.
if explicit_filter && entity_ids.len() > MAX_HISTORY_ENTITIES {
return Err(ApiError::BadRequest(format!(
"history queries are limited to {MAX_HISTORY_ENTITIES} explicitly filtered entities"
)));
}
let mut result = Vec::with_capacity(entity_ids.len());
let mut remaining = MAX_API_HISTORY_ROWS;
for entity_id in entity_ids {
let rows = recorder
.get_state_history_limited(&entity_id, start, end, remaining)
.await
.map_err(|error| ApiError::Internal(format!("history query failed: {error}")))?;
remaining = remaining.saturating_sub(rows.len());
let mut previous_state: Option<String> = None;
let states = rows
.into_iter()
.filter_map(|row| {
if query.significant_changes_only
&& previous_state.as_deref() == Some(row.state.as_str())
{
return None;
}
previous_state = Some(row.state.clone());
let changed = history_timestamp(row.last_changed_ts);
let updated = history_timestamp(row.last_updated_ts);
Some(StateView {
entity_id: row.entity_id.as_str().to_owned(),
state: row.state,
attributes: if query.no_attributes || query.minimal_response {
serde_json::json!({})
} else {
row.attributes
},
last_changed: changed,
last_updated: updated,
context: ContextView {
id: row.context_id.unwrap_or_default(),
user_id: None,
parent_id: None,
},
})
})
.collect();
result.push(states);
}
Ok(Json(result))
}
fn parse_history_time(value: &str) -> ApiResult<chrono::DateTime<chrono::Utc>> {
chrono::DateTime::parse_from_rfc3339(value)
.map(|value| value.with_timezone(&chrono::Utc))
.map_err(|_| ApiError::BadRequest("history timestamps must be RFC 3339".into()))
}
fn history_timestamp(seconds: f64) -> String {
let whole = seconds.floor() as i64;
let nanos = ((seconds - seconds.floor()) * 1_000_000_000.0).round() as u32;
chrono::DateTime::<chrono::Utc>::from_timestamp(whole, nanos.min(999_999_999))
.unwrap_or(chrono::DateTime::<chrono::Utc>::UNIX_EPOCH)
.to_rfc3339()
}
#[derive(Debug, Deserialize)]
pub struct LogbookQuery {
end_time: Option<String>,
entity: Option<String>,
}
pub async fn get_logbook(
headers: HeaderMap,
State(s): State<SharedState>,
Query(query): Query<LogbookQuery>,
) -> ApiResult<Json<Vec<serde_json::Value>>> {
logbook_response(headers, s, None, query).await
}
pub async fn get_logbook_period(
headers: HeaderMap,
State(s): State<SharedState>,
Path(start_time): Path<String>,
Query(query): Query<LogbookQuery>,
) -> ApiResult<Json<Vec<serde_json::Value>>> {
logbook_response(headers, s, Some(start_time), query).await
}
async fn logbook_response(
headers: HeaderMap,
state: SharedState,
start_time: Option<String>,
query: LogbookQuery,
) -> ApiResult<Json<Vec<serde_json::Value>>> {
let _ = BearerAuth::from_headers(&headers, state.tokens()).await?;
let recorder = state
.recorder()
.ok_or_else(|| ApiError::Unavailable("recorder is disabled".into()))?;
let now = chrono::Utc::now();
let start = match start_time {
Some(value) => parse_history_time(&value)?,
None => now - chrono::Duration::days(1),
};
let end = match query.end_time.as_deref() {
Some(value) => parse_history_time(value)?,
None => now,
};
if end < start {
return Err(ApiError::BadRequest(
"end_time must not precede start_time".into(),
));
}
let explicit_filter = query.entity.is_some();
let entity_ids = match query.entity.as_deref() {
Some(raw) => raw
.split(',')
.map(str::trim)
.filter(|value| !value.is_empty())
.map(|value| {
EntityId::parse(value)
.map_err(|error| ApiError::BadRequest(format!("invalid entity_id: {error}")))
})
.collect::<ApiResult<Vec<_>>>()?,
None => state
.homecore()
.states()
.all()
.into_iter()
.map(|snapshot| snapshot.entity_id.clone())
.collect(),
};
// See the matching comment in `history_response`: only reject an
// explicit, unusually-large filter — the default (no filter, "all
// entities") is the real HA frontend's normal call shape, and the
// `MAX_API_HISTORY_ROWS` row budget below already bounds the work.
if explicit_filter && entity_ids.len() > MAX_HISTORY_ENTITIES {
return Err(ApiError::BadRequest(format!(
"logbook queries are limited to {MAX_HISTORY_ENTITIES} explicitly filtered entities"
)));
}
let mut entries = Vec::new();
let mut remaining = MAX_API_HISTORY_ROWS;
for entity_id in entity_ids {
let rows = recorder
.get_state_history_limited(&entity_id, start, end, remaining)
.await
.map_err(|error| ApiError::Internal(format!("logbook query failed: {error}")))?;
remaining = remaining.saturating_sub(rows.len());
for row in rows {
entries.push(serde_json::json!({
"when": history_timestamp(row.last_updated_ts),
"name": row.entity_id.as_str(),
"state": row.state,
"entity_id": row.entity_id.as_str(),
"context_id": row.context_id
}));
}
}
entries.sort_by(|left, right| {
left["when"]
.as_str()
.cmp(&right["when"].as_str())
.then_with(|| left["entity_id"].as_str().cmp(&right["entity_id"].as_str()))
});
Ok(Json(entries))
}
#[derive(Serialize)]
pub struct CalendarView {
entity_id: String,
name: String,
}
pub async fn get_calendars(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<Vec<CalendarView>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let calendars = s
.homecore()
.states()
.all_by_domain("calendar")
.into_iter()
.map(|state| CalendarView {
entity_id: state.entity_id.as_str().to_owned(),
name: state
.attributes
.get("friendly_name")
.and_then(serde_json::Value::as_str)
.unwrap_or(state.entity_id.as_str())
.to_owned(),
})
.collect();
Ok(Json(calendars))
}
#[derive(Debug, Deserialize)]
pub struct CalendarQuery {
start: String,
end: String,
}
pub async fn get_calendar_events(
headers: HeaderMap,
State(s): State<SharedState>,
Path(entity_id): Path<String>,
Query(query): Query<CalendarQuery>,
) -> ApiResult<Json<Vec<serde_json::Value>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let id =
EntityId::parse(&entity_id).map_err(|error| ApiError::BadRequest(error.to_string()))?;
if id.domain() != "calendar" || s.homecore().states().get(&id).is_none() {
return Err(ApiError::NotFound(entity_id));
}
let start = parse_history_time(&query.start)?;
let end = parse_history_time(&query.end)?;
if end < start {
return Err(ApiError::BadRequest(
"end must not precede start".to_owned(),
));
}
// Calendar integrations may expose their current entity without an event
// provider. An empty list is the valid response for that interval.
Ok(Json(Vec::new()))
}
pub async fn get_camera_proxy(
headers: HeaderMap,
State(s): State<SharedState>,
Path(entity_id): Path<String>,
) -> ApiResult<StatusCode> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let id =
EntityId::parse(&entity_id).map_err(|error| ApiError::BadRequest(error.to_string()))?;
if id.domain() != "camera" || s.homecore().states().get(&id).is_none() {
return Err(ApiError::NotFound(entity_id));
}
Err(ApiError::Unavailable(
"camera integration has no image provider".into(),
))
}
#[derive(Serialize)]
pub struct ContextView {
pub id: String,
@@ -80,10 +438,15 @@ impl StateView {
}
}
pub async fn get_states(headers: HeaderMap, State(s): State<SharedState>) -> ApiResult<Json<Vec<StateView>>> {
pub async fn get_states(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<Vec<StateView>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let snapshots = s.homecore().states().all();
Ok(Json(snapshots.iter().map(|x| StateView::from_state(x)).collect()))
Ok(Json(
snapshots.iter().map(|x| StateView::from_state(x)).collect(),
))
}
pub async fn get_state(
@@ -93,7 +456,11 @@ pub async fn get_state(
) -> ApiResult<Json<StateView>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let id = EntityId::parse(entity_id.clone()).map_err(|e| ApiError::BadRequest(e.to_string()))?;
let st = s.homecore().states().get(&id).ok_or_else(|| ApiError::NotFound(entity_id))?;
let st = s
.homecore()
.states()
.get(&id)
.ok_or(ApiError::NotFound(entity_id))?;
Ok(Json(StateView::from_state(&st)))
}
@@ -128,9 +495,20 @@ pub async fn set_state(
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let id = EntityId::parse(entity_id).map_err(|e| ApiError::BadRequest(e.to_string()))?;
let existed = s.homecore().states().get(&id).is_some();
let attrs = if body.attributes.is_null() { serde_json::json!({}) } else { body.attributes };
let snap = s.homecore().states().set(id, body.state, attrs, Context::new());
let status = if existed { StatusCode::OK } else { StatusCode::CREATED };
let attrs = if body.attributes.is_null() {
serde_json::json!({})
} else {
body.attributes
};
let snap = s
.homecore()
.states()
.set(id, body.state, attrs, Context::new());
let status = if existed {
StatusCode::OK
} else {
StatusCode::CREATED
};
Ok((status, Json(StateView::from_state(&snap))))
}
@@ -140,17 +518,31 @@ pub struct ServiceDomainView {
pub services: serde_json::Value,
}
pub async fn get_services(headers: HeaderMap, State(s): State<SharedState>) -> ApiResult<Json<Vec<ServiceDomainView>>> {
pub async fn get_services(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<Vec<ServiceDomainView>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let services = s.homecore().services().registered_services().await;
let mut by_domain: std::collections::HashMap<String, serde_json::Map<String, serde_json::Value>> =
std::collections::HashMap::new();
let mut by_domain: std::collections::HashMap<
String,
serde_json::Map<String, serde_json::Value>,
> = std::collections::HashMap::new();
for sv in services {
by_domain.entry(sv.domain.clone()).or_default().insert(sv.service.clone(), serde_json::json!({}));
by_domain
.entry(sv.domain.clone())
.or_default()
.insert(sv.service.clone(), serde_json::json!({}));
}
Ok(Json(by_domain.into_iter().map(|(domain, services)| ServiceDomainView {
domain, services: serde_json::Value::Object(services),
}).collect()))
Ok(Json(
by_domain
.into_iter()
.map(|(domain, services)| ServiceDomainView {
domain,
services: serde_json::Value::Object(services),
})
.collect(),
))
}
pub async fn call_service(
@@ -166,9 +558,199 @@ pub async fn call_service(
data: body,
context: Context::new(),
};
let resp = s.homecore().services().call(call).await.map_err(|e| match e {
homecore::ServiceError::NotRegistered { .. } => ApiError::ServiceNotRegistered { domain, service },
other => ApiError::Internal(other.to_string()),
})?;
let resp = s
.homecore()
.services()
.call(call)
.await
.map_err(|e| match e {
homecore::ServiceError::NotRegistered { .. } => {
ApiError::ServiceNotRegistered { domain, service }
}
other => ApiError::Internal(other.to_string()),
})?;
Ok(Json(resp))
}
#[derive(Serialize)]
pub struct EventView {
pub event: String,
pub listener_count: usize,
}
/// Event types whose wire shape is implemented by the core event bridge.
const CORE_EVENT_TYPES: &[&str] = &[
"state_changed",
"call_service",
"homeassistant_start",
"homeassistant_stop",
];
pub async fn get_events(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<Vec<EventView>>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok(Json(
CORE_EVENT_TYPES
.iter()
.map(|event| EventView {
event: (*event).to_owned(),
// Tokio broadcast intentionally does not expose a stable
// per-filter count. Zero is HA-compatible and honest.
listener_count: 0,
})
.collect(),
))
}
/// Whether `event_type` is acceptable to fire on the domain bus.
///
/// Real Home Assistant places essentially no format restriction on event
/// types beyond "non-empty string" — integrations commonly fire types with
/// mixed case, dots, or hyphens (e.g. `mobile_app.notification_action`,
/// `ios.action_fired`). The original check here only accepted
/// `[a-z0-9_]+`, silently rejecting any of those — a real behavioral gap
/// versus the documented contract, not a security boundary (this endpoint is
/// already bearer-authenticated). We keep only the bounds that protect the
/// server itself: non-empty, a sane length cap, and no control characters
/// (which could otherwise corrupt log lines or downstream storage).
pub(crate) fn is_valid_event_type(event_type: &str) -> bool {
!event_type.is_empty()
&& event_type.len() <= 255
&& event_type.chars().all(|ch| !ch.is_control())
}
pub async fn fire_event(
headers: HeaderMap,
State(s): State<SharedState>,
Path(event_type): Path<String>,
Json(body): Json<serde_json::Value>,
) -> ApiResult<Json<serde_json::Value>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
if !is_valid_event_type(&event_type) {
return Err(ApiError::BadRequest("invalid event_type".into()));
}
if !body.is_object() && !body.is_null() {
return Err(ApiError::BadRequest("event data must be an object".into()));
}
let data = if body.is_null() {
serde_json::json!({})
} else {
body
};
s.homecore().bus().fire_domain(homecore::DomainEvent::new(
event_type.clone(),
data,
Context::new(),
));
Ok(Json(
serde_json::json!({"message": format!("Event {event_type} fired.")}),
))
}
#[derive(Deserialize)]
pub struct TemplateRequest {
pub template: String,
}
pub async fn render_template(
headers: HeaderMap,
State(s): State<SharedState>,
Json(body): Json<TemplateRequest>,
) -> ApiResult<String> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
let environment = homecore_automation::TemplateEnvironment::new(std::sync::Arc::new(
s.homecore().states().clone(),
));
environment
.render(&body.template)
.map_err(|error| ApiError::BadRequest(error.to_string()))
}
pub async fn check_config(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<serde_json::Value>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
// Runtime configuration has already passed HOMECORE's typed loaders.
Ok(Json(serde_json::json!({
"result": "valid",
"errors": null,
"warnings": null
})))
}
pub async fn error_log(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<impl IntoResponse> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok((
[("content-type", "text/plain; charset=utf-8")],
String::new(),
))
}
/// Machine-readable support matrix. This prevents clients from confusing
/// core protocol compatibility with every optional HA integration.
pub async fn compatibility(
headers: HeaderMap,
State(s): State<SharedState>,
) -> ApiResult<Json<serde_json::Value>> {
let _ = BearerAuth::from_headers(&headers, s.tokens()).await?;
Ok(Json(serde_json::json!({
"baseline": "Home Assistant Core 2025.1",
"rest": {
"core": "implemented",
"events": "implemented",
"template": "implemented",
"check_config": "implemented",
"error_log": "implemented",
"history": "implemented_when_recorder_enabled",
"logbook": "implemented_when_recorder_enabled",
"calendar": "implemented_with_integration_supplied_events",
"camera": "implemented_with_integration_supplied_images",
"media": "integration_dependent"
},
"websocket": {
"auth": "implemented",
"states_services_config": "implemented",
"events": "implemented",
"render_template": "implemented",
"feature_negotiation_and_panels": "implemented",
"registry_lists": {
"entity": "implemented",
"device": "implemented",
"area": "implemented_empty",
"mutations": "requires_persistent_registry_backend"
},
"lovelace_media": "integration_dependent"
}
})))
}
#[cfg(test)]
mod tests {
use super::is_valid_event_type;
/// Real HA integrations commonly fire event types with mixed case, dots,
/// or hyphens (e.g. `mobile_app.notification_action`). The original
/// `[a-z0-9_]+`-only check rejected all of these; only non-empty,
/// length, and control-character bounds should remain.
#[test]
fn realistic_ha_event_types_are_accepted() {
assert!(is_valid_event_type("mobile_app.notification_action"));
assert!(is_valid_event_type("ios.action_fired"));
assert!(is_valid_event_type("Custom-Event.2"));
assert!(is_valid_event_type("state_changed"));
}
#[test]
fn empty_oversized_or_control_char_event_types_are_rejected() {
assert!(!is_valid_event_type(""));
assert!(!is_valid_event_type(&"a".repeat(256)));
assert!(!is_valid_event_type("bad\nevent"));
assert!(!is_valid_event_type("bad\tevent"));
}
}
+34 -10
View File
@@ -1,5 +1,6 @@
use std::sync::Arc;
use homecore::HomeCore;
use homecore_recorder::Recorder;
use std::sync::Arc;
use crate::tokens::LongLivedTokenStore;
@@ -13,6 +14,7 @@ struct SharedStateInner {
pub homecore_version: String,
pub location_name: String,
pub tokens: LongLivedTokenStore,
pub recorder: Option<Recorder>,
}
impl SharedState {
@@ -28,15 +30,13 @@ impl SharedState {
location_name: impl Into<String>,
homecore_version: impl Into<String>,
) -> Self {
// P2 default: dev-mode token store (accepts any non-empty
// bearer) so existing smoke tests still work; the
// `homecore-server` binary uses with_tokens() to provision a
// real store at boot.
// Fail closed by default. Tests and explicitly insecure local
// development must opt into `allow_any_non_empty()` themselves.
Self::with_tokens(
homecore,
location_name,
homecore_version,
LongLivedTokenStore::allow_any_non_empty(),
LongLivedTokenStore::empty(),
)
}
@@ -52,12 +52,36 @@ impl SharedState {
homecore_version: homecore_version.into(),
location_name: location_name.into(),
tokens,
recorder: None,
}),
}
}
pub fn homecore(&self) -> &HomeCore { &self.inner.homecore }
pub fn version(&self) -> &str { &self.inner.homecore_version }
pub fn location_name(&self) -> &str { &self.inner.location_name }
pub fn tokens(&self) -> &LongLivedTokenStore { &self.inner.tokens }
pub fn with_recorder(self, recorder: Option<Recorder>) -> Self {
Self {
inner: Arc::new(SharedStateInner {
homecore: self.inner.homecore.clone(),
homecore_version: self.inner.homecore_version.clone(),
location_name: self.inner.location_name.clone(),
tokens: self.inner.tokens.clone(),
recorder,
}),
}
}
pub fn homecore(&self) -> &HomeCore {
&self.inner.homecore
}
pub fn version(&self) -> &str {
&self.inner.homecore_version
}
pub fn location_name(&self) -> &str {
&self.inner.location_name
}
pub fn tokens(&self) -> &LongLivedTokenStore {
&self.inner.tokens
}
pub fn recorder(&self) -> Option<&Recorder> {
self.inner.recorder.as_ref()
}
}
+4
View File
@@ -127,6 +127,10 @@ impl LongLivedTokenStore {
self.inner.read().await.tokens.len()
}
pub async fn is_empty(&self) -> bool {
self.inner.read().await.tokens.is_empty()
}
/// Is the store accepting any non-empty bearer (DEV mode)?
pub async fn is_dev_mode(&self) -> bool {
self.inner.read().await.allow_any
+162 -25
View File
@@ -20,7 +20,6 @@
//! drains the response channel onto the socket (HC-WS-02 closed the prior
//! reply-theater where responses were logged and discarded).
use std::sync::atomic::{AtomicU64, Ordering};
use std::sync::Arc;
use axum::extract::ws::{Message, WebSocket, WebSocketUpgrade};
@@ -28,6 +27,10 @@ use axum::extract::State;
use axum::response::IntoResponse;
use serde::{Deserialize, Serialize};
use tokio::sync::broadcast;
/// Per-connection outbound queue. A bounded queue prevents a client that
/// stops reading from turning event fan-out into unbounded process memory.
const OUTBOUND_QUEUE_CAPACITY: usize = 256;
use tracing::warn;
use homecore::{Context, ServiceCall, ServiceName, SystemEvent};
@@ -49,7 +52,11 @@ async fn handle_socket(mut socket: WebSocket, state: SharedState) {
"type": "auth_required",
"ha_version": state.version(),
});
if socket.send(Message::Text(auth_req.to_string())).await.is_err() {
if socket
.send(Message::Text(auth_req.to_string()))
.await
.is_err()
{
return;
}
@@ -59,7 +66,8 @@ async fn handle_socket(mut socket: WebSocket, state: SharedState) {
_ => {
let _ = socket
.send(Message::Text(
serde_json::json!({"type":"auth_invalid","message":"expected auth"}).to_string(),
serde_json::json!({"type":"auth_invalid","message":"expected auth"})
.to_string(),
))
.await;
return;
@@ -85,7 +93,11 @@ async fn handle_socket(mut socket: WebSocket, state: SharedState) {
return;
}
let auth_ok = serde_json::json!({"type":"auth_ok","ha_version": state.version()});
if socket.send(Message::Text(auth_ok.to_string())).await.is_err() {
if socket
.send(Message::Text(auth_ok.to_string()))
.await
.is_err()
{
return;
}
@@ -118,6 +130,10 @@ struct WsCommand {
service: Option<String>,
#[serde(default)]
service_data: Option<serde_json::Value>,
#[serde(default)]
event_data: Option<serde_json::Value>,
#[serde(default)]
template: Option<String>,
}
#[derive(Serialize)]
@@ -140,7 +156,6 @@ struct ErrorView<'a> {
struct Connection {
state: SharedState,
next_sub_id: AtomicU64,
subs: Arc<dashmap::DashMap<u64, SubscriptionHandle>>,
}
@@ -152,7 +167,6 @@ impl Connection {
fn new(state: SharedState) -> Self {
Self {
state,
next_sub_id: AtomicU64::new(1),
subs: Arc::new(dashmap::DashMap::new()),
}
}
@@ -168,7 +182,7 @@ impl Connection {
// DISCARDED every message — so no `result`/`pong`/`event` ever
// reached the client. Now `rx` feeds `socket.send`.
let (mut sink, mut stream) = socket.split();
let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel::<String>();
let (tx, mut rx) = tokio::sync::mpsc::channel::<String>(OUTBOUND_QUEUE_CAPACITY);
// Writer task: drain replies onto the socket. A `__pong:<n>`
// sentinel maps to a binary Pong control frame; everything else
@@ -205,7 +219,7 @@ impl Connection {
conn.handle_cmd(cmd, &reader_tx).await;
}
Ok(Message::Ping(p)) => {
let _ = reader_tx.send(format!("__pong:{}", p.len()));
let _ = reader_tx.try_send(format!("__pong:{}", p.len()));
}
Ok(Message::Close(_)) | Err(_) => break,
_ => {}
@@ -224,15 +238,22 @@ impl Connection {
let _ = writer_task.await;
}
async fn handle_cmd(&self, cmd: WsCommand, tx: &tokio::sync::mpsc::UnboundedSender<String>) {
async fn handle_cmd(&self, cmd: WsCommand, tx: &tokio::sync::mpsc::Sender<String>) {
match cmd.kind.as_str() {
"supported_features" => {
// HOMECORE currently emits individual messages. Accepting the
// negotiation command keeps modern HA clients compatible while
// deliberately declining optional coalescing.
self.ack(tx, cmd.id, true, None);
}
"ping" => {
let msg = serde_json::json!({"id": cmd.id, "type": "pong"});
let _ = tx.send(msg.to_string());
let _ = tx.try_send(msg.to_string());
}
"get_states" => {
let snapshots = self.state.homecore().states().all();
let views: Vec<StateView> = snapshots.iter().map(|s| StateView::from_state(s)).collect();
let views: Vec<StateView> =
snapshots.iter().map(|s| StateView::from_state(s)).collect();
self.ack(tx, cmd.id, true, Some(serde_json::to_value(views).unwrap()));
}
"get_config" => {
@@ -243,19 +264,53 @@ impl Connection {
});
self.ack(tx, cmd.id, true, Some(payload));
}
"get_panels" => {
// Panels are frontend integration resources. An empty map is
// the valid shape for a headless server.
self.ack(tx, cmd.id, true, Some(serde_json::json!({})));
}
"get_services" => {
let services = self.state.homecore().services().registered_services().await;
let mut by_domain: std::collections::HashMap<String, serde_json::Map<String, serde_json::Value>> =
std::collections::HashMap::new();
let mut by_domain: std::collections::HashMap<
String,
serde_json::Map<String, serde_json::Value>,
> = std::collections::HashMap::new();
for s in services {
by_domain.entry(s.domain).or_default().insert(s.service, serde_json::json!({}));
by_domain
.entry(s.domain)
.or_default()
.insert(s.service, serde_json::json!({}));
}
let payload = serde_json::to_value(by_domain).unwrap();
self.ack(tx, cmd.id, true, Some(payload));
}
"config/entity_registry/list" | "get_entity_registry" => {
let entries = self.state.homecore().entities().all().await;
let payload =
serde_json::to_value(entries).unwrap_or_else(|_| serde_json::json!([]));
self.ack(tx, cmd.id, true, Some(payload));
}
"config/device_registry/list" | "get_device_registry" => {
let entries = self.state.homecore().devices().all().await;
let payload =
serde_json::to_value(entries).unwrap_or_else(|_| serde_json::json!([]));
self.ack(tx, cmd.id, true, Some(payload));
}
"config/area_registry/list" | "get_area_registry" => {
// HOMECORE does not yet model named areas. Returning the valid
// empty-list shape lets clients distinguish that from an
// unsupported command.
self.ack(tx, cmd.id, true, Some(serde_json::json!([])));
}
"call_service" => {
let (Some(domain), Some(service)) = (cmd.domain.clone(), cmd.service.clone()) else {
self.err(tx, cmd.id, "missing_domain_service", "domain and service are required");
let (Some(domain), Some(service)) = (cmd.domain.clone(), cmd.service.clone())
else {
self.err(
tx,
cmd.id,
"missing_domain_service",
"domain and service are required",
);
return;
};
let call = ServiceCall {
@@ -268,8 +323,53 @@ impl Connection {
Err(e) => self.err(tx, cmd.id, "service_error", &e.to_string()),
}
}
"fire_event" => {
let Some(event_type) = cmd.event_type.clone() else {
self.err(tx, cmd.id, "invalid_format", "event_type is required");
return;
};
if !crate::rest::is_valid_event_type(&event_type) {
self.err(tx, cmd.id, "invalid_format", "invalid event_type");
return;
}
let event_data = cmd.event_data.unwrap_or_else(|| serde_json::json!({}));
if !event_data.is_object() {
self.err(tx, cmd.id, "invalid_format", "event_data must be an object");
return;
}
self.state
.homecore()
.bus()
.fire_domain(homecore::DomainEvent::new(
event_type,
event_data,
Context::new(),
));
self.ack(tx, cmd.id, true, None);
}
"render_template" => {
let Some(template) = cmd.template.as_deref() else {
self.err(tx, cmd.id, "invalid_format", "template is required");
return;
};
let environment = homecore_automation::TemplateEnvironment::new(Arc::new(
self.state.homecore().states().clone(),
));
match environment.render(template) {
Ok(rendered) => {
self.ack(tx, cmd.id, true, Some(serde_json::Value::String(rendered)))
}
Err(error) => self.err(tx, cmd.id, "template_error", &error.to_string()),
}
}
"subscribe_events" => {
let sub_id = self.next_sub_id.fetch_add(1, Ordering::Relaxed);
// HA uses the subscribing command ID as the subscription ID
// in every emitted event and in `unsubscribe_events`.
let sub_id = cmd.id;
if self.subs.contains_key(&sub_id) {
self.err(tx, cmd.id, "id_reused", "subscription id is already active");
return;
}
let filter = cmd.event_type.clone();
let tx_clone = tx.clone();
let mut domain_rx = self.state.homecore().bus().subscribe_domain();
@@ -294,7 +394,27 @@ impl Connection {
"time_fired": sc.fired_at.to_rfc3339(),
}
});
if tx_clone.send(payload.to_string()).is_err() { break; }
if tx_clone.try_send(payload.to_string()).is_err() { break; }
}
}
Ok(SystemEvent::ServiceCalled { domain, service, data, context }) => {
if filter.as_deref() == Some("call_service") || filter.is_none() {
let payload = serde_json::json!({
"id": sub_id,
"type": "event",
"event": {
"event_type": "call_service",
"data": {
"domain": domain,
"service": service,
"service_data": data,
},
"origin": "LOCAL",
"time_fired": chrono::Utc::now().to_rfc3339(),
"context": context,
}
});
if tx_clone.try_send(payload.to_string()).is_err() { break; }
}
}
Ok(_) => {}
@@ -321,9 +441,10 @@ impl Connection {
"data": de.event_data,
"origin": format!("{:?}", de.origin).to_uppercase(),
"time_fired": de.fired_at.to_rfc3339(),
"context": de.context,
}
});
if tx_clone.send(payload.to_string()).is_err() { break; }
if tx_clone.try_send(payload.to_string()).is_err() { break; }
}
}
// Same recoverable-lag handling as the system arm
@@ -353,11 +474,21 @@ impl Connection {
self.err(tx, cmd.id, "not_found", "subscription_id not found");
}
} else {
self.err(tx, cmd.id, "missing_subscription", "subscription is required");
self.err(
tx,
cmd.id,
"missing_subscription",
"subscription is required",
);
}
}
other => {
self.err(tx, cmd.id, "unknown_command", &format!("unknown ws command: {other}"));
self.err(
tx,
cmd.id,
"unknown_command",
&format!("unknown ws command: {other}"),
);
}
}
// entity_id is reserved for future per-entity subscribes
@@ -366,7 +497,7 @@ impl Connection {
fn ack(
&self,
tx: &tokio::sync::mpsc::UnboundedSender<String>,
tx: &tokio::sync::mpsc::Sender<String>,
id: u64,
success: bool,
result: Option<serde_json::Value>,
@@ -378,10 +509,16 @@ impl Connection {
result,
error: None,
};
let _ = tx.send(serde_json::to_string(&msg).unwrap());
let _ = tx.try_send(serde_json::to_string(&msg).unwrap());
}
fn err(&self, tx: &tokio::sync::mpsc::UnboundedSender<String>, id: u64, code: &'static str, message: &str) {
fn err(
&self,
tx: &tokio::sync::mpsc::Sender<String>,
id: u64,
code: &'static str,
message: &str,
) {
let msg = ResultMessage {
id,
kind: "result",
@@ -389,7 +526,7 @@ impl Connection {
result: None,
error: Some(ErrorView { code, message }),
};
let _ = tx.send(serde_json::to_string(&msg).unwrap());
let _ = tx.try_send(serde_json::to_string(&msg).unwrap());
}
}
@@ -0,0 +1,193 @@
use axum::body::Body;
use axum::http::{Request, StatusCode};
use homecore::HomeCore;
use homecore_api::{router, LongLivedTokenStore, SharedState};
use http_body_util::BodyExt;
use tower::ServiceExt;
async fn app() -> (axum::Router, HomeCore) {
let homecore = HomeCore::new();
let tokens = LongLivedTokenStore::empty();
tokens.register("test-token").await;
let state = SharedState::with_tokens(homecore.clone(), "Test", "test", tokens);
(router(state), homecore)
}
fn post(uri: &str, body: &str) -> Request<Body> {
Request::builder()
.method("POST")
.uri(uri)
.header("authorization", "Bearer test-token")
.header("content-type", "application/json")
.body(Body::from(body.to_owned()))
.unwrap()
}
#[tokio::test]
async fn rest_event_is_delivered_to_domain_bus() {
let (app, homecore) = app().await;
let mut receiver = homecore.bus().subscribe_domain();
let response = app
.oneshot(post("/api/events/test_event", r#"{"answer":42}"#))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let event = receiver.recv().await.unwrap();
assert_eq!(event.event_type, "test_event");
assert_eq!(event.event_data["answer"], 42);
}
#[tokio::test]
async fn rest_template_uses_live_state_environment() {
let (app, _) = app().await;
let response = app
.oneshot(post("/api/template", r#"{"template":"{{ 6 * 7 }}"}"#))
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let bytes = response.into_body().collect().await.unwrap().to_bytes();
assert_eq!(&bytes[..], b"42");
}
#[tokio::test]
async fn compatibility_matrix_is_authenticated() {
let (app, _) = app().await;
let response = app
.oneshot(
Request::builder()
.uri("/api/homecore/compatibility")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::UNAUTHORIZED);
}
#[tokio::test]
async fn history_reads_real_recorder_rows() {
use homecore::{Context, EntityId};
use homecore_recorder::Recorder;
let homecore = HomeCore::new();
let recorder = Recorder::open("sqlite::memory:").await.unwrap();
let mut changes = homecore.states().subscribe();
homecore.states().set(
EntityId::parse("light.history_probe").unwrap(),
"on",
serde_json::json!({"brightness": 123}),
Context::new(),
);
let change = changes.recv().await.unwrap();
recorder.record_state(&change).await.unwrap();
let tokens = LongLivedTokenStore::empty();
tokens.register("test-token").await;
let state =
SharedState::with_tokens(homecore, "Test", "test", tokens).with_recorder(Some(recorder));
let response = router(state)
.oneshot(
Request::builder()
.uri("/api/history/period?filter_entity_id=light.history_probe")
.header("authorization", "Bearer test-token")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
let bytes = response.into_body().collect().await.unwrap().to_bytes();
let body: serde_json::Value = serde_json::from_slice(&bytes).unwrap();
assert_eq!(body[0][0]["entity_id"], "light.history_probe");
assert_eq!(body[0][0]["state"], "on");
assert_eq!(body[0][0]["attributes"]["brightness"], 123);
}
/// The real HA frontend's history/logbook pages call these endpoints with NO
/// entity filter by design (meaning "all entities") — a real install
/// routinely has 50-500+ entities. The 32-entity cap must only reject an
/// explicit, unusually-large `filter_entity_id`/`entity` list, never the
/// default unfiltered "all entities" shape.
#[tokio::test]
async fn history_and_logbook_unfiltered_are_not_capped_by_entity_count() {
use homecore::{Context, EntityId};
use homecore_recorder::Recorder;
let homecore = HomeCore::new();
let recorder = Recorder::open("sqlite::memory:").await.unwrap();
for i in 0..40 {
homecore.states().set(
EntityId::parse(&format!("sensor.probe_{i}")).unwrap(),
"on",
serde_json::json!({}),
Context::new(),
);
}
assert_eq!(homecore.states().all().len(), 40, "sanity: more than MAX_HISTORY_ENTITIES");
let tokens = LongLivedTokenStore::empty();
tokens.register("test-token").await;
let state =
SharedState::with_tokens(homecore, "Test", "test", tokens).with_recorder(Some(recorder));
let app = router(state);
let history_response = app
.clone()
.oneshot(
Request::builder()
.uri("/api/history/period")
.header("authorization", "Bearer test-token")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
history_response.status(),
StatusCode::OK,
"unfiltered history with >32 known entities must not be rejected"
);
let logbook_response = app
.oneshot(
Request::builder()
.uri("/api/logbook")
.header("authorization", "Bearer test-token")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(
logbook_response.status(),
StatusCode::OK,
"unfiltered logbook with >32 known entities must not be rejected"
);
}
/// An explicit, unusually-large `filter_entity_id` list is still rejected —
/// only the default "no filter" shape is exempt from the cap.
#[tokio::test]
async fn history_explicit_oversized_filter_is_still_rejected() {
use homecore_recorder::Recorder;
let homecore = HomeCore::new();
let recorder = Recorder::open("sqlite::memory:").await.unwrap();
let tokens = LongLivedTokenStore::empty();
tokens.register("test-token").await;
let state =
SharedState::with_tokens(homecore, "Test", "test", tokens).with_recorder(Some(recorder));
let filter: String = (0..40).map(|i| format!("sensor.probe_{i}")).collect::<Vec<_>>().join(",");
let response = router(state)
.oneshot(
Request::builder()
.uri(format!("/api/history/period?filter_entity_id={filter}"))
.header("authorization", "Bearer test-token")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
}
+127 -3
View File
@@ -80,7 +80,10 @@ async fn wrong_token_is_rejected() {
resp["type"], "auth_invalid",
"wrong token must be rejected with auth_invalid, got: {resp}"
);
assert_ne!(resp["type"], "auth_ok", "wrong token must NOT receive auth_ok");
assert_ne!(
resp["type"], "auth_ok",
"wrong token must NOT receive auth_ok"
);
}
#[tokio::test]
@@ -99,7 +102,10 @@ async fn correct_token_is_accepted() {
.unwrap();
let resp = next_json(&mut ws).await;
assert_eq!(resp["type"], "auth_ok", "correct token should be accepted, got: {resp}");
assert_eq!(
resp["type"], "auth_ok",
"correct token should be accepted, got: {resp}"
);
}
#[tokio::test]
@@ -133,7 +139,10 @@ async fn result_reply_is_received() {
let reply = tokio::time::timeout(std::time::Duration::from_secs(5), next_json(&mut ws))
.await
.expect("did not receive a reply within 5s — reply theater (HC-WS-02)");
assert_eq!(reply["type"], "result", "expected a result reply, got: {reply}");
assert_eq!(
reply["type"], "result",
"expected a result reply, got: {reply}"
);
assert_eq!(reply["id"], 1);
assert_eq!(reply["success"], true);
}
@@ -263,3 +272,118 @@ async fn subscription_survives_broadcast_lag() {
);
assert_eq!(got["event"]["data"]["marker"], "post-lag");
}
#[tokio::test]
async fn real_state_change_uses_client_subscription_id() {
use homecore::{Context, EntityId};
let (addr, hc) = spawn_server_returning_homecore("good_token_abc").await;
let url = format!("ws://{addr}/api/websocket");
let (mut ws, _resp) = connect_async(&url).await.unwrap();
let _ = next_json(&mut ws).await;
ws.send(Message::Text(
serde_json::json!({"type":"auth","access_token":"good_token_abc"}).to_string(),
))
.await
.unwrap();
let _ = next_json(&mut ws).await;
ws.send(Message::Text(
serde_json::json!({
"id": 41,
"type": "subscribe_events",
"event_type": "state_changed"
})
.to_string(),
))
.await
.unwrap();
let ack = next_json(&mut ws).await;
assert_eq!(ack["id"], 41);
assert_eq!(ack["success"], true);
hc.states().set(
EntityId::parse("light.integration_probe").unwrap(),
"on",
serde_json::json!({"source":"test"}),
Context::new(),
);
let event = tokio::time::timeout(std::time::Duration::from_secs(5), next_json(&mut ws))
.await
.expect("state change was not bridged to the WebSocket system-event subscription");
assert_eq!(
event["id"], 41,
"HA events must use the subscribe command id"
);
assert_eq!(event["type"], "event");
assert_eq!(event["event"]["event_type"], "state_changed");
assert_eq!(
event["event"]["data"]["entity_id"],
"light.integration_probe"
);
ws.send(Message::Text(
serde_json::json!({
"id": 42,
"type": "unsubscribe_events",
"subscription": 41
})
.to_string(),
))
.await
.unwrap();
let unsub = next_json(&mut ws).await;
assert_eq!(unsub["id"], 42);
assert_eq!(unsub["success"], true);
}
#[tokio::test]
async fn registry_list_commands_return_ha_result_shapes() {
use homecore::{EntityEntry, EntityId};
let (addr, hc) = spawn_server_returning_homecore("good_token_abc").await;
hc.entities()
.register(EntityEntry {
entity_id: EntityId::parse("light.registry_probe").unwrap(),
unique_id: Some("probe-1".into()),
platform: "test".into(),
name: Some("Registry probe".into()),
disabled_by: None,
area_id: None,
device_id: None,
entity_category: None,
config_entry_id: None,
})
.await;
let url = format!("ws://{addr}/api/websocket");
let (mut ws, _response) = connect_async(&url).await.unwrap();
let _ = next_json(&mut ws).await;
ws.send(Message::Text(
serde_json::json!({"type":"auth","access_token":"good_token_abc"}).to_string(),
))
.await
.unwrap();
let _ = next_json(&mut ws).await;
for (id, command) in [
(71, "config/entity_registry/list"),
(72, "config/device_registry/list"),
(73, "config/area_registry/list"),
] {
ws.send(Message::Text(
serde_json::json!({"id": id, "type": command}).to_string(),
))
.await
.unwrap();
let reply = next_json(&mut ws).await;
assert_eq!(reply["id"], id);
assert_eq!(reply["success"], true);
assert!(reply["result"].is_array());
if command == "config/entity_registry/list" {
assert_eq!(reply["result"][0]["entity_id"], "light.registry_probe");
}
}
}
+82
View File
@@ -0,0 +1,82 @@
//! Bounded audio types shared by STT, TTS, and satellite transports.
use serde::{Deserialize, Serialize};
use thiserror::Error;
/// Maximum audio accepted in one chunk (256 KiB).
pub const MAX_AUDIO_CHUNK_BYTES: usize = 256 * 1024;
/// Maximum audio accepted in a single utterance (16 MiB).
pub const MAX_UTTERANCE_AUDIO_BYTES: usize = 16 * 1024 * 1024;
/// Audio encodings supported by the native voice pipeline.
#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum AudioCodec {
/// Signed, little-endian, 16-bit PCM.
PcmS16Le,
}
/// A validated audio stream format.
#[derive(Clone, Copy, Debug, Eq, PartialEq, Serialize, Deserialize)]
pub struct AudioFormat {
pub codec: AudioCodec,
pub sample_rate: u32,
pub channels: u8,
}
impl AudioFormat {
pub fn validate(self) -> Result<Self, AudioError> {
if !(8_000..=48_000).contains(&self.sample_rate) {
return Err(AudioError::InvalidSampleRate(self.sample_rate));
}
if !(1..=2).contains(&self.channels) {
return Err(AudioError::InvalidChannels(self.channels));
}
Ok(self)
}
}
/// One bounded audio packet.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct AudioChunk {
bytes: Vec<u8>,
}
impl AudioChunk {
pub fn new(bytes: Vec<u8>) -> Result<Self, AudioError> {
if bytes.is_empty() {
return Err(AudioError::Empty);
}
if bytes.len() > MAX_AUDIO_CHUNK_BYTES {
return Err(AudioError::ChunkTooLarge(bytes.len()));
}
if bytes.len() % 2 != 0 {
return Err(AudioError::UnalignedPcm);
}
Ok(Self { bytes })
}
pub fn as_bytes(&self) -> &[u8] {
&self.bytes
}
pub fn into_bytes(self) -> Vec<u8> {
self.bytes
}
}
#[derive(Debug, Error, Eq, PartialEq)]
pub enum AudioError {
#[error("audio chunk is empty")]
Empty,
#[error("audio chunk is {0} bytes; maximum is {MAX_AUDIO_CHUNK_BYTES}")]
ChunkTooLarge(usize),
#[error("16-bit PCM data must contain an even number of bytes")]
UnalignedPcm,
#[error("sample rate {0} Hz is outside 8000..=48000")]
InvalidSampleRate(u32),
#[error("channel count {0} is outside 1..=2")]
InvalidChannels(u8),
#[error("utterance audio exceeds {MAX_UTTERANCE_AUDIO_BYTES} bytes")]
UtteranceTooLarge,
}
+24 -10
View File
@@ -35,27 +35,41 @@
//! honest path until it ships.
//! - STT/TTS bridge and satellite protocol (P3).
pub mod intent;
pub mod recognizer;
pub mod semantic_recognizer;
pub mod audio;
pub mod handler;
pub mod runner;
pub mod intent;
pub mod pipeline;
pub mod recognizer;
pub mod runner;
pub mod satellite;
pub mod semantic_recognizer;
pub mod speech;
pub mod voice;
/// Deterministic text embedding used by [`semantic_recognizer::SemanticIntentRecognizer`].
#[cfg(feature = "semantic")]
pub mod embedding;
pub use intent::{Card, Intent, IntentName, IntentResponse};
pub use recognizer::{
IntentRecognizer, RecognizerError, RegexIntentRecognizer, MAX_UTTERANCE_BYTES,
};
pub use semantic_recognizer::{SemanticIntentRecognizer, DEFAULT_SIMILARITY_THRESHOLD};
pub use audio::{AudioChunk, AudioCodec, AudioError, AudioFormat};
pub use handler::{
HandlerError, HassCancelAll, HassLightSet, HassNevermind, HassTurnOff, HassTurnOn,
IntentHandler,
};
pub use intent::{Card, Intent, IntentName, IntentResponse};
pub use pipeline::AssistPipeline;
pub use recognizer::{
IntentRecognizer, RecognizerError, RegexIntentRecognizer, MAX_UTTERANCE_BYTES,
};
pub use runner::{
AssistError, LocalRunner, NoopRunner, RufloResponse, RufloRunner, RufloRunnerOpts,
};
pub use pipeline::AssistPipeline;
pub use satellite::{
SatelliteClientMessage, SatelliteError, SatelliteServerMessage, SatelliteSession,
SatelliteState, SATELLITE_PROTOCOL_VERSION,
};
pub use semantic_recognizer::{SemanticIntentRecognizer, DEFAULT_SIMILARITY_THRESHOLD};
pub use speech::{
DisabledStt, DisabledTts, SpeechError, SpeechToText, SynthesizedSpeech, TextToSpeech,
Transcript,
};
pub use voice::{VoiceError, VoicePipeline, VoiceResponse, DEFAULT_VOICE_TIMEOUT};
+250
View File
@@ -0,0 +1,250 @@
//! Transport-independent satellite voice session protocol.
//!
//! Text frames use [`SatelliteClientMessage`] / [`SatelliteServerMessage`].
//! Binary frames are accepted only while a stream is active.
use serde::{Deserialize, Serialize};
use thiserror::Error;
use crate::audio::{AudioChunk, AudioError, AudioFormat, MAX_UTTERANCE_AUDIO_BYTES};
pub const SATELLITE_PROTOCOL_VERSION: u16 = 1;
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum SatelliteClientMessage {
Hello {
version: u16,
token: String,
},
Start {
language: String,
format: AudioFormat,
},
End,
Cancel,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum SatelliteServerMessage {
Ready { version: u16 },
Started,
Transcript { text: String, language: String },
Intent { response: crate::IntentResponse },
Audio { format: AudioFormat, bytes: usize },
Finished,
Error { code: String, message: String },
}
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum SatelliteState {
AwaitingHello,
Idle,
Streaming,
Closed,
}
/// Strict state machine used by WebSocket or native satellite transports.
pub struct SatelliteSession {
state: SatelliteState,
authenticated: bool,
audio: Vec<u8>,
format: Option<AudioFormat>,
language: Option<String>,
}
impl SatelliteSession {
pub fn new() -> Self {
Self {
state: SatelliteState::AwaitingHello,
authenticated: false,
audio: Vec::new(),
format: None,
language: None,
}
}
pub fn state(&self) -> SatelliteState {
self.state
}
/// Handle a control message. `authenticate` must perform constant-time
/// credential comparison; credentials are never retained by this session.
pub fn control(
&mut self,
message: SatelliteClientMessage,
authenticate: impl FnOnce(&str) -> bool,
) -> Result<SatelliteServerMessage, SatelliteError> {
match (self.state, message) {
(SatelliteState::AwaitingHello, SatelliteClientMessage::Hello { version, token }) => {
if version != SATELLITE_PROTOCOL_VERSION {
self.state = SatelliteState::Closed;
return Err(SatelliteError::UnsupportedVersion(version));
}
if !authenticate(&token) {
self.state = SatelliteState::Closed;
return Err(SatelliteError::Unauthorized);
}
self.authenticated = true;
self.state = SatelliteState::Idle;
Ok(SatelliteServerMessage::Ready { version })
}
(SatelliteState::Idle, SatelliteClientMessage::Start { language, format })
if self.authenticated =>
{
if language.is_empty() || language.len() > 35 {
return Err(SatelliteError::InvalidLanguage);
}
self.format = Some(format.validate()?);
self.language = Some(language);
self.audio.clear();
self.state = SatelliteState::Streaming;
Ok(SatelliteServerMessage::Started)
}
(SatelliteState::Streaming, SatelliteClientMessage::End) => {
if self.audio.is_empty() {
return Err(SatelliteError::EmptyStream);
}
self.state = SatelliteState::Idle;
Ok(SatelliteServerMessage::Finished)
}
(SatelliteState::Streaming, SatelliteClientMessage::Cancel) => {
self.reset_stream();
Ok(SatelliteServerMessage::Finished)
}
(_, SatelliteClientMessage::Cancel) => {
self.reset_stream();
Ok(SatelliteServerMessage::Finished)
}
_ => Err(SatelliteError::InvalidSequence),
}
}
pub fn audio(&mut self, chunk: AudioChunk) -> Result<(), SatelliteError> {
if self.state != SatelliteState::Streaming {
return Err(SatelliteError::InvalidSequence);
}
if self.audio.len().saturating_add(chunk.as_bytes().len()) > MAX_UTTERANCE_AUDIO_BYTES {
self.reset_stream();
return Err(SatelliteError::AudioLimit);
}
self.audio.extend_from_slice(chunk.as_bytes());
Ok(())
}
pub fn take_utterance(&mut self) -> Option<(Vec<u8>, AudioFormat, String)> {
if self.state != SatelliteState::Idle || self.audio.is_empty() {
return None;
}
let audio = std::mem::take(&mut self.audio);
Some((audio, self.format.take()?, self.language.take()?))
}
fn reset_stream(&mut self) {
self.audio.clear();
self.format = None;
self.language = None;
self.state = if self.authenticated {
SatelliteState::Idle
} else {
SatelliteState::AwaitingHello
};
}
}
impl Default for SatelliteSession {
fn default() -> Self {
Self::new()
}
}
#[derive(Debug, Error)]
pub enum SatelliteError {
#[error("satellite message is invalid in the current session state")]
InvalidSequence,
#[error("satellite protocol version {0} is unsupported")]
UnsupportedVersion(u16),
#[error("satellite authentication failed")]
Unauthorized,
#[error("invalid language tag")]
InvalidLanguage,
#[error("audio stream is empty")]
EmptyStream,
#[error("audio stream exceeded its size limit")]
AudioLimit,
#[error(transparent)]
Audio(#[from] AudioError),
}
#[cfg(test)]
mod tests {
use super::*;
use crate::audio::AudioCodec;
fn format() -> AudioFormat {
AudioFormat {
codec: AudioCodec::PcmS16Le,
sample_rate: 16_000,
channels: 1,
}
}
#[test]
fn happy_path_preserves_audio_and_metadata() {
let mut session = SatelliteSession::new();
session
.control(
SatelliteClientMessage::Hello {
version: 1,
token: "secret".into(),
},
|token| token == "secret",
)
.unwrap();
session
.control(
SatelliteClientMessage::Start {
language: "en-CA".into(),
format: format(),
},
|_| false,
)
.unwrap();
session
.audio(AudioChunk::new(vec![1, 0, 2, 0]).unwrap())
.unwrap();
session
.control(SatelliteClientMessage::End, |_| false)
.unwrap();
let (audio, stored_format, language) = session.take_utterance().unwrap();
assert_eq!(audio, vec![1, 0, 2, 0]);
assert_eq!(stored_format, format());
assert_eq!(language, "en-CA");
}
#[test]
fn unauthenticated_stream_is_rejected_and_closed() {
let mut session = SatelliteSession::new();
let error = session
.control(
SatelliteClientMessage::Hello {
version: 1,
token: "wrong".into(),
},
|_| false,
)
.unwrap_err();
assert!(matches!(error, SatelliteError::Unauthorized));
assert_eq!(session.state(), SatelliteState::Closed);
}
#[test]
fn binary_before_start_is_rejected() {
let mut session = SatelliteSession::new();
let error = session
.audio(AudioChunk::new(vec![0, 0]).unwrap())
.unwrap_err();
assert!(matches!(error, SatelliteError::InvalidSequence));
}
}
+87
View File
@@ -0,0 +1,87 @@
//! Provider-neutral speech-to-text and text-to-speech contracts.
use async_trait::async_trait;
use thiserror::Error;
use crate::audio::{AudioFormat, MAX_UTTERANCE_AUDIO_BYTES};
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct Transcript {
pub text: String,
pub language: String,
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct SynthesizedSpeech {
pub audio: Vec<u8>,
pub format: AudioFormat,
}
#[derive(Debug, Error)]
pub enum SpeechError {
#[error("{0} provider is not configured")]
NotConfigured(&'static str),
#[error("speech provider rejected the request: {0}")]
Provider(String),
#[error("speech provider returned invalid data: {0}")]
InvalidOutput(String),
#[error("speech operation timed out")]
Timeout,
}
#[async_trait]
pub trait SpeechToText: Send + Sync {
async fn transcribe(
&self,
audio: &[u8],
format: AudioFormat,
language: &str,
) -> Result<Transcript, SpeechError>;
}
#[async_trait]
pub trait TextToSpeech: Send + Sync {
async fn synthesize(
&self,
text: &str,
language: &str,
) -> Result<SynthesizedSpeech, SpeechError>;
}
/// Fail-closed provider used until an STT integration is configured.
pub struct DisabledStt;
#[async_trait]
impl SpeechToText for DisabledStt {
async fn transcribe(
&self,
_audio: &[u8],
_format: AudioFormat,
_language: &str,
) -> Result<Transcript, SpeechError> {
Err(SpeechError::NotConfigured("STT"))
}
}
/// Fail-closed provider used until a TTS integration is configured.
pub struct DisabledTts;
#[async_trait]
impl TextToSpeech for DisabledTts {
async fn synthesize(
&self,
_text: &str,
_language: &str,
) -> Result<SynthesizedSpeech, SpeechError> {
Err(SpeechError::NotConfigured("TTS"))
}
}
pub(crate) fn validate_provider_audio(audio: &[u8]) -> Result<(), SpeechError> {
if audio.is_empty() || audio.len() > MAX_UTTERANCE_AUDIO_BYTES || audio.len() % 2 != 0 {
return Err(SpeechError::InvalidOutput(
"audio must be non-empty, bounded, aligned PCM".into(),
));
}
Ok(())
}
+183
View File
@@ -0,0 +1,183 @@
//! End-to-end STT → intent → TTS pipeline.
use std::time::Duration;
use homecore::HomeCore;
use thiserror::Error;
use tokio::time::timeout;
use crate::audio::{AudioFormat, MAX_UTTERANCE_AUDIO_BYTES};
use crate::pipeline::AssistPipeline;
use crate::recognizer::IntentRecognizer;
use crate::speech::{
validate_provider_audio, SpeechError, SpeechToText, SynthesizedSpeech, TextToSpeech, Transcript,
};
use crate::{AssistError, IntentResponse};
pub const DEFAULT_VOICE_TIMEOUT: Duration = Duration::from_secs(30);
#[derive(Clone, Debug)]
pub struct VoiceResponse {
pub transcript: Transcript,
pub intent: IntentResponse,
pub speech: SynthesizedSpeech,
}
#[derive(Debug, Error)]
pub enum VoiceError {
#[error("input audio is empty or exceeds the utterance limit")]
InvalidAudio,
#[error(transparent)]
Speech(#[from] SpeechError),
#[error(transparent)]
Assist(#[from] AssistError),
#[error("voice pipeline timed out")]
Timeout,
}
pub struct VoicePipeline<S, T, R: IntentRecognizer> {
stt: S,
tts: T,
assist: AssistPipeline<R>,
timeout: Duration,
}
impl<S, T, R> VoicePipeline<S, T, R>
where
S: SpeechToText,
T: TextToSpeech,
R: IntentRecognizer,
{
pub fn new(stt: S, tts: T, assist: AssistPipeline<R>) -> Self {
Self {
stt,
tts,
assist,
timeout: DEFAULT_VOICE_TIMEOUT,
}
}
pub fn with_timeout(mut self, value: Duration) -> Self {
self.timeout = value;
self
}
pub async fn process(
&self,
audio: &[u8],
format: AudioFormat,
language: &str,
hc: &HomeCore,
) -> Result<VoiceResponse, VoiceError> {
if audio.is_empty() || audio.len() > MAX_UTTERANCE_AUDIO_BYTES || audio.len() % 2 != 0 {
return Err(VoiceError::InvalidAudio);
}
let format = format.validate().map_err(|_| VoiceError::InvalidAudio)?;
timeout(self.timeout, async {
let transcript = self.stt.transcribe(audio, format, language).await?;
let intent = self
.assist
.process(&transcript.text, &transcript.language, hc)
.await?;
let speech = self
.tts
.synthesize(&intent.speech, &transcript.language)
.await?;
speech
.format
.validate()
.map_err(|error| SpeechError::InvalidOutput(error.to_string()))?;
validate_provider_audio(&speech.audio)?;
Ok(VoiceResponse {
transcript,
intent,
speech,
})
})
.await
.map_err(|_| VoiceError::Timeout)?
}
}
#[cfg(test)]
mod tests {
use async_trait::async_trait;
use super::*;
use crate::audio::AudioCodec;
use crate::recognizer::RegexIntentRecognizer;
use crate::speech::{SpeechToText, TextToSpeech};
struct FixedStt;
#[async_trait]
impl SpeechToText for FixedStt {
async fn transcribe(
&self,
_audio: &[u8],
_format: AudioFormat,
language: &str,
) -> Result<Transcript, SpeechError> {
Ok(Transcript {
text: "never mind".into(),
language: language.into(),
})
}
}
struct FixedTts;
#[async_trait]
impl TextToSpeech for FixedTts {
async fn synthesize(
&self,
text: &str,
_language: &str,
) -> Result<SynthesizedSpeech, SpeechError> {
assert!(!text.is_empty());
Ok(SynthesizedSpeech {
audio: vec![0, 0, 1, 0],
format: format(),
})
}
}
fn format() -> AudioFormat {
AudioFormat {
codec: AudioCodec::PcmS16Le,
sample_rate: 16_000,
channels: 1,
}
}
#[tokio::test]
async fn runs_stt_intent_and_tts() {
let recognizer = RegexIntentRecognizer::new();
recognizer
.register("HassNevermind", r"never ?mind", "*")
.await
.unwrap();
let pipeline = crate::pipeline::default_pipeline(recognizer);
let voice = VoicePipeline::new(FixedStt, FixedTts, pipeline);
let result = voice
.process(&[0, 0, 1, 0], format(), "en-CA", &HomeCore::new())
.await
.unwrap();
assert_eq!(result.transcript.text, "never mind");
assert_eq!(result.speech.audio.len(), 4);
}
#[tokio::test]
async fn rejects_unaligned_audio_before_provider_call() {
let voice = VoicePipeline::new(
FixedStt,
FixedTts,
crate::pipeline::default_pipeline(RegexIntentRecognizer::new()),
);
let error = voice
.process(&[0], format(), "en", &HomeCore::new())
.await
.unwrap_err();
assert!(matches!(error, VoiceError::InvalidAudio));
}
}
@@ -74,7 +74,7 @@ impl Condition {
Condition::State { entity_id, state } => {
ctx.states
.get(entity_id)
.map_or(false, |s| s.state == *state)
.is_some_and(|s| s.state == *state)
}
Condition::NumericState { entity_id, above, below } => {
let value: Option<f64> = ctx
@@ -84,16 +84,13 @@ impl Condition {
match value {
None => false,
Some(v) => {
above.map_or(true, |a| v > a) && below.map_or(true, |b| v < b)
above.is_none_or(|a| v > a) && below.is_none_or(|b| v < b)
}
}
}
Condition::Template { value_template } => {
if let Some(env) = &ctx.template_env {
match env.render_bool(value_template) {
Ok(v) => v,
Err(_) => false,
}
env.render_bool(value_template).unwrap_or_default()
} else {
false
}
+2 -2
View File
@@ -106,7 +106,7 @@ impl Trigger {
pub fn matches_sync(&self, ctx: &TriggerContext) -> bool {
match self {
Trigger::State { entity_id, from, to } => {
let eid_match = ctx.entity_id.as_ref().map_or(false, |e| e == entity_id);
let eid_match = ctx.entity_id.as_ref() == Some(entity_id);
if !eid_match {
return false;
}
@@ -125,7 +125,7 @@ impl Trigger {
true
}
Trigger::NumericState { entity_id, above, below } => {
let eid_match = ctx.entity_id.as_ref().map_or(false, |e| e == entity_id);
let eid_match = ctx.entity_id.as_ref() == Some(entity_id);
if !eid_match {
return false;
}
+17 -5
View File
@@ -10,7 +10,7 @@ version = "0.1.0-alpha.0"
edition = "2021"
license = "MIT"
authors = ["rUv <ruv@ruv.net>", "HOMECORE Contributors"]
description = "Apple Home HomeKit Accessory Protocol bridge — ADR-125 P1 scaffold"
description = "Fail-closed HomeKit Accessory Protocol network foundation for HOMECORE"
repository = "https://github.com/ruvnet/wifi-densepose"
[lib]
@@ -19,18 +19,30 @@ path = "src/lib.rs"
[features]
default = []
# P2: gates the actual hap = "0.1" crate integration + real mDNS via mdns-sd
hap-server = []
# Enables the bounded TCP/HTTP listener and real `_hap._tcp` mDNS advertiser.
hap-server = ["dep:httparse", "dep:mdns-sd"]
[dependencies]
homecore = { path = "../homecore" }
tokio = { version = "1", features = ["sync", "rt", "rt-multi-thread", "time", "macros"] }
tokio = { version = "1", features = ["fs", "io-util", "macros", "net", "rt", "rt-multi-thread", "sync", "time"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2"
tracing = "0.1"
async-trait = "0.1"
uuid = { version = "1", features = ["v4", "serde"] }
ed25519-dalek = "2.1"
tempfile = "3"
chacha20poly1305 = "0.10"
getrandom = "0.2"
hkdf = "0.12"
sha2 = "0.10"
sha2_11 = { package = "sha2", version = "0.11" }
srp = "=0.7.0-rc.3"
x25519-dalek = { version = "2", features = ["static_secrets"] }
zeroize = { version = "1", features = ["derive"] }
httparse = { version = "1", optional = true }
mdns-sd = { version = "0.11", optional = true }
[dev-dependencies]
tokio = { version = "1", features = ["sync", "rt", "rt-multi-thread", "time", "macros", "test-util"] }
tokio = { version = "1", features = ["fs", "io-util", "macros", "net", "rt", "rt-multi-thread", "sync", "time", "test-util"] }
+98 -105
View File
@@ -1,121 +1,114 @@
# homecore-hap
Apple Home HomeKit Accessory Protocol bridge for HOMECORE with HAP-1.1 trait surface and mDNS advertisement (P2).
`homecore-hap` is HOMECORE's bounded, fail-closed HAP IP accessory server
(ADR-125). It implements the HAP R2 cryptographic pairing and transport
boundary without relying on the broken `hap` 0.1 pre-release crate.
[![Crates.io](https://img.shields.io/crates/v/homecore-hap.svg)](https://crates.io/crates/homecore-hap)
![License](https://img.shields.io/badge/license-MIT-blue.svg)
![MSRV: 1.89+](https://img.shields.io/badge/MSRV-1.89%2B-purple.svg)
[![Tests](https://img.shields.io/badge/tests-17%20passing-brightgreen.svg)](https://github.com/ruvnet/RuView)
[![ADR-125](https://img.shields.io/badge/ADR-125-orange.svg)](../../docs/adr/ADR-125-homecore-apple-home-homekit-bridge.md)
## Security and protocol coverage
**P1 scaffold**: trait surface for HAP accessories + characteristics, entity→HAP mapping rules, and bridge ownership. The actual HAP-1.1 TLS server and real mDNS integration are gated behind `--features hap-server` (P2).
- Pair-Setup M1-M6 uses the RFC 5054 3072-bit group with SHA-512, the HAP
compatibility proof construction, HKDF-SHA512, ChaCha20-Poly1305, and
Ed25519 long-term keys.
- Pair-Verify M1-M4 uses ephemeral X25519, strict Ed25519 transcript
verification, HKDF-SHA512, and authenticated encrypted sub-TLVs.
- After successful M4, all HTTP and `EVENT/1.0` traffic uses HAP records:
two-byte little-endian authenticated lengths, at most 1024 plaintext bytes,
independent directional keys, and monotonic 64-bit nonces. Authentication,
replay, truncation, and oversize failures close the connection without an
oracle response.
- `/accessories`, `/characteristics`, and `/pairings` are inaccessible until
Pair-Verify succeeds on that TCP connection. Pairings add/remove/list is
restricted to a currently persisted administrator.
- Accessory identity, Ed25519 seed, SRP salt/verifier, and controller records
are stored in a versioned file using same-directory atomic replacement.
Created Unix directories use mode `0700`, files use `0600`, and permissive,
oversized, symlinked, legacy, or malformed stores fail closed.
- The raw setup code is returned only during first provisioning. Only its SRP
verifier is persisted; `SetupCode` redacts `Debug` output and zeroizes on
drop.
- Removing the last administrator atomically clears every pairing. Live
sessions observe pairing revisions and are revoked, while the removal
response is delivered before the requesting session closes.
## What this crate does
The server also bounds connections, headers, bodies, request time, shutdown,
TLV sizes, controller counts, setup attempts, and concurrent Pair-Setup.
mDNS advertises the persisted accessory identifier and updates `sf` after
pairing or unpairing.
`homecore-hap` bridges HOMECORE entity state to Apple HomeKit Accessory Protocol (HAP-1.1), allowing HomeKit-native apps (Home, Control Center, Siri) to control HOMECORE devices. It provides:
## Provisioning and server integration
- **HapAccessoryType enum** — 11 accessory types matching HA's HomeKit integration (`Light`, `Switch`, `Thermostat`, `Lock`, `Door`, etc.)
- **HapCharacteristic enum** — HAP characteristic types (`On`, `Brightness`, `Temperature`, `TargetLockState`, etc.)
- **EntityToAccessoryMapper** — bidirectional rules for mapping HOMECORE entities to HAP accessories (e.g., `light.kitchen``Light` accessory + `On` + `Brightness` characteristics)
- **HapBridge** — owns and exposes a collection of mapped accessories over HAP
- **MdnsAdvertiser trait** — abstraction over mDNS advertisement; P1 ships `NullAdvertiser` (no-op), P2 adds real mDNS via `mdns-sd`
- **RuViewToHapMapper** — bridges RuView sensing data (temperature, humidity, occupancy) to HAP characteristics
```rust,no_run
use std::{net::IpAddr, sync::Arc};
use homecore_hap::{
start_server, HapBridge, HapServerConfig, HapServiceRecord,
MdnsSdAdvertiser, PairingStore,
};
The bridge itself is a HAP Accessory Bridge (HAP-1.1 spec §8.3), advertising a single service with characteristic slots for each exposed accessory.
## Features
- **11 accessory types** — Light, Switch, Thermostat, Door, Lock, Window, Blind, Outlet, Fan, Sensor, SecuritySystem
- **Bi-directional mapping** — HOMECORE entity state ↔ HAP characteristic values with type-safe enums
- **HAP-1.1 spec compliance** — characteristic types and permissions match HomeKit's published spec
- **Trait-based advertisement** — `MdnsAdvertiser` abstraction; swappable implementations (null, real mDNS, etc.)
- **RuView integration** — maps WiFi sensing data (occupancy, temperature, vital signs) to HomeKit sensor accessories
- **No TLS server in P1** — bridge compiles and tests pass with `--no-default-features`; real server lands in P2 with `--features hap-server`
- **Home.app compatible** — exposed accessories appear in Home app on any HomeKit hub (Apple TV, HomePod, HomePod mini)
## Capabilities
| Capability | Type | Method | Notes |
|------------|------|--------|-------|
| Define accessory type | Trait | `HapAccessoryType::Light` etc. (11 variants) | Enum; no instantiation yet (P1) |
| Define characteristic | Trait | `HapCharacteristic::On`, `Brightness`, etc. | Enum; values encoded as HAP TLV |
| Map entity to accessory | Mapping | `EntityToAccessoryMapper::map_light()` | Takes `EntityId` + `State`; returns `HapAccessory` |
| Expose accessory | Bridge | `HapBridge::expose(accessory)` | Adds to the bridge's characteristic list |
| Advertise bridge | mDNS | `NullAdvertiser::advertise()` (P1) | No-op stub; real mDNS in P2 |
| Advertise bridge (P2) | mDNS | `mdns_sd::ServiceInstanceBuilder` | Real mDNS via `--features hap-server` |
| Bridge state query | Bridge | `HapBridge::list_accessories()` | Returns exposed accessories + their characteristics |
| Characteristic write | Characteristic | HAP `WriteRequest` TLV (P2) | Home.app button press → service call |
| Characteristic read | Characteristic | HAP `ReadResponse` TLV (P2) | Home.app query → current entity state |
## Comparison to Home Assistant
| Aspect | Home Assistant | homecore-hap |
|--------|----------------|--------------|
| Framework | HA's `hap-python` (pure Python) | Rust 1.89+ with HAP trait abstraction |
| Server type | Python asyncio HAP-1.1 server | TLS server trait (P2); stub in P1 |
| Accessory types | 30+ (Light, Switch, Thermostat, etc.) | 11 (Light, Switch, Thermostat, Door, Lock, Window, Blind, Outlet, Fan, Sensor, SecuritySystem) |
| mDNS | mdns-py broadcast via asyncio | Abstraction + real mDNS (P2) or no-op stub (P1) |
| Entity filtering | YAML `include_domains` + `exclude_entities` | Mapper rules (planned P2) |
| HomeKit hub requirement | Yes (for remote access) | Yes (same as HomeKit) |
| Pairing code generation | Automatic (HA web UI) | Manual setup code (P2) |
| Characteristic persistence | HomeKit cloud only | Paired with homecore state machine |
## Performance
- **Entity→HAP mapping** — < 100 μs per entity (enum lookups + type conversions)
- **HAP write latency** — ~10 ms (TLS decrypt + characteristic parse + entity state set); bounded by homecore state machine lock contention
- **mDNS advertisement** (P2) — ~50 ms multicast broadcast; periodic rediscovery on network change
- **Memory overhead per accessory** — ~500 bytes (enum + characteristic slots + metadata)
- **No per-crate benchmarks yet** — a follow-up issue tracks baseline measurements
## Usage
Mapping an entity (P1):
```rust
use homecore_hap::{EntityToAccessoryMapper, HapBridge, HapAccessoryType};
use homecore::{EntityId, State};
use std::collections::HashMap;
#[tokio::main]
async fn main() {
let light_id = EntityId::parse("light.kitchen").unwrap();
let state = State::new("on", HashMap::new());
// Map the entity to a HAP Light accessory
let mut mapper = EntityToAccessoryMapper::new();
if let Ok(accessory) = mapper.map_light(&light_id, &state) {
println!("Mapped to HAP: {:?}", accessory.accessory_type);
// Expose it via the bridge
let mut bridge = HapBridge::new();
bridge.expose(accessory);
println!("Exposed {} accessories", bridge.list_accessories().len());
}
# async fn run() -> Result<(), Box<dyn std::error::Error>> {
let provisioned = PairingStore::load_or_create(
"/var/lib/homecore-hap/security.json",
)?;
if let Some(setup_code) = provisioned.setup_code.as_ref() {
// Send this once to a trusted local display or provisioning boundary.
println!("HAP setup code: {}", setup_code.expose());
}
let pairings = Arc::new(provisioned.store);
let record = HapServiceRecord::bridge(
"HOMECORE Bridge",
51826,
pairings.accessory_id()?,
);
let bridge = HapBridge::new(record);
let advertiser = Arc::new(MdnsSdAdvertiser::new(
"homecore",
"192.168.1.50".parse::<IpAddr>()?,
)?);
let server = start_server(
HapServerConfig::default(),
bridge.clone(),
pairings,
advertiser,
).await?;
// Feed HOMECORE StateChanged events through bridge.update_accessory(...).
server.shutdown().await?;
# Ok(())
# }
```
Real HAP server (P2, via `--features hap-server`):
The mDNS device ID must equal the persisted accessory ID; startup rejects a
mismatch. Real mDNS also requires a LAN-routable advertised address and
multicast access.
## Validation and remaining interoperability limits
The deterministic suite covers the HAP SRP session-key vector, complete
Pair-Setup and Pair-Verify ceremonies, transcript tampering, wrong proofs,
malformed/replayed/oversized records, atomic restart, last-admin removal, and
a real TCP lifecycle from Pair-Verify through encrypted `/accessories`.
This is protocol-level HAP R2 coverage, not a claim of Apple certification:
- It has not yet been exercised against a current Apple Home controller or
the current commercial MFi specification.
- Transient and split Pair-Setup flags are rejected as `Unavailable`.
- Writable characteristic service calls, timed writes, resource endpoints,
and stable persisted AID/IID allocation are not implemented. The present
characteristic surface is read and event subscription only.
- Operational hardening still depends on protecting the host and the
`0600` security file; no hardware-backed key store is integrated.
Build and test:
```bash
cargo build -p homecore-hap --features hap-server
# The server will advertise over mDNS and accept HomeKit pairing requests
cargo test -p homecore-hap --no-default-features
cargo test -p homecore-hap --features hap-server
cargo clippy -p homecore-hap --all-targets --features hap-server -- -D warnings
```
## Relation to other HOMECORE crates
## Decisions
```
homecore-hap (HomeKit bridge)
├─ homecore (state machine; bridge reads entity states)
├─ homecore-api (exposes HAP state via REST /api for remote debugging)
├─ homecore-server (starts the bridge on homecore init)
└─ homecore-automation (can trigger state changes via service calls)
```
## References
- [ADR-125: HOMECORE Apple Home / HomeKit Bridge](../../docs/adr/ADR-125-homecore-apple-home-homekit-bridge.md)
- [ADR-126: HOMECORE Home Assistant Port (master)](../../docs/adr/ADR-126-homecore-home-assistant-port.md)
- [HomeKit Accessory Protocol Specification (HAP-1.1)](https://developer.apple.com/homekit/)
- [user-guide-apple-homepod.md](../../docs/user-guide-apple-homepod.md)
- [README — wifi-densepose](../../../README.md)
- [ADR-125 — native Apple Home HAP bridge](../../../docs/adr/ADR-125-ruview-apple-home-native-hap-bridge.md)
- [ADR-130 — bounded async REST/WebSocket server patterns](../../../docs/adr/ADR-130-homecore-rest-websocket-api.md)
- [ADR-161 — server-layer security and honest labeling](../../../docs/adr/ADR-161-homecore-server-layer-security.md)
+96 -23
View File
@@ -1,15 +1,15 @@
//! `HapBridge` — owns the set of HOMECORE entities exposed as HAP accessories.
//!
//! P1 does not start a real HAP-1.1 server; it ships the API surface so other
//! crates (and P2's `hap-server` feature) can register accessories and query
//! their current mapping. The actual mDNS + HAP pairing is gated to P2.
//! The bridge owns mappings and their event stream. The feature-gated network
//! lifecycle is started separately with `start_server`.
use std::collections::HashMap;
use std::sync::{Arc, RwLock};
use homecore::entity::EntityId;
use tokio::sync::broadcast;
use crate::accessory::HapAccessoryType;
use crate::accessory::{HapAccessoryType, HapCharacteristic, HapCharacteristicValue};
use crate::error::HapError;
use crate::mapping::{AccessoryMapping, EntityToAccessoryMapper};
use crate::mdns::{HapServiceRecord, MdnsAdvertiser, NullAdvertiser};
@@ -22,37 +22,49 @@ pub struct ExposedAccessory {
pub mapping: AccessoryMapping,
}
/// A characteristic snapshot emitted after a registered entity changes.
#[derive(Debug, Clone)]
pub struct CharacteristicEvent {
pub entity_id: EntityId,
pub accessory_type: HapAccessoryType,
pub characteristics: Vec<(HapCharacteristic, HapCharacteristicValue)>,
}
struct BridgeInner {
accessories: HashMap<EntityId, ExposedAccessory>,
}
/// The P1 HAP bridge.
/// HOMECORE-to-HAP accessory bridge state.
///
/// Call [`HapBridge::add_accessory`] to register entities and
/// [`HapBridge::running_accessories`] to read back what is currently
/// registered. In P2, `start()` will spawn the `hap` server task.
/// registered. Use `start_server` for the bounded TCP lifecycle.
#[derive(Clone)]
pub struct HapBridge {
inner: Arc<RwLock<BridgeInner>>,
advertiser: Arc<dyn MdnsAdvertiser>,
events: broadcast::Sender<CharacteristicEvent>,
pub service_record: HapServiceRecord,
}
impl HapBridge {
/// Create a bridge with the given service record and a `NullAdvertiser`
/// (P1 default — real mDNS lands in P2).
/// Create a bridge with the given service record and a `NullAdvertiser`.
pub fn new(service_record: HapServiceRecord) -> Self {
Self::with_advertiser(service_record, Arc::new(NullAdvertiser))
}
/// Create a bridge with a custom `MdnsAdvertiser` (used in tests and P2).
/// Create a bridge with a custom `MdnsAdvertiser`.
pub fn with_advertiser(
service_record: HapServiceRecord,
advertiser: Arc<dyn MdnsAdvertiser>,
) -> Self {
let (events, _) = broadcast::channel(128);
Self {
inner: Arc::new(RwLock::new(BridgeInner { accessories: HashMap::new() })),
inner: Arc::new(RwLock::new(BridgeInner {
accessories: HashMap::new(),
})),
advertiser,
events,
service_record,
}
}
@@ -97,9 +109,46 @@ impl HapBridge {
Ok(())
}
/// Refresh a registered accessory and notify event subscribers.
pub fn update_accessory(
&self,
entity_id: &EntityId,
state: &homecore::entity::State,
) -> Result<(), HapError> {
let mapping = EntityToAccessoryMapper::map(entity_id, state)?;
let accessory_type = mapping.accessory_type;
{
let mut inner = self.inner.write().unwrap();
let accessory = inner
.accessories
.get_mut(entity_id)
.ok_or_else(|| HapError::EntityNotFound(entity_id.as_str().to_owned()))?;
accessory.accessory_type = accessory_type;
accessory.mapping = mapping.clone();
}
let _ = self.events.send(CharacteristicEvent {
entity_id: entity_id.clone(),
accessory_type,
characteristics: mapping.characteristics,
});
Ok(())
}
/// Subscribe to bounded characteristic updates. Lagging receivers receive
/// Tokio's explicit `Lagged` error and must resynchronize from a snapshot.
pub fn subscribe_events(&self) -> broadcast::Receiver<CharacteristicEvent> {
self.events.subscribe()
}
/// Snapshot all currently registered accessories.
pub fn running_accessories(&self) -> Vec<ExposedAccessory> {
self.inner.read().unwrap().accessories.values().cloned().collect()
self.inner
.read()
.unwrap()
.accessories
.values()
.cloned()
.collect()
}
/// Number of registered accessories.
@@ -111,21 +160,25 @@ impl HapBridge {
self.len() == 0
}
/// P2 stub — will start the HAP-1.1 server + mDNS advertisement.
/// In P1 this only fires the null advertiser.
/// Start advertisement only.
///
/// This legacy lifecycle does not bind a TCP listener. New integrations
/// should call `start_server`, which advertises only after binding.
pub async fn start(&self) -> Result<(), HapError> {
self.advertiser.advertise(&self.service_record).await?;
tracing::info!(
instance = %self.service_record.instance_name,
port = self.service_record.port,
"HapBridge started (P1 — no real HAP server; mDNS stub only)"
"HAP advertisement started without a TCP server"
);
Ok(())
}
/// Graceful shutdown — retracts mDNS advertisement.
pub async fn stop(&self) -> Result<(), HapError> {
self.advertiser.retract(&self.service_record.instance_name).await?;
self.advertiser
.retract(&self.service_record.instance_name)
.await?;
Ok(())
}
}
@@ -137,18 +190,22 @@ mod tests {
use homecore::event::Context;
fn make_bridge() -> HapBridge {
HapBridge::new(HapServiceRecord {
instance_name: "RuView Sense".into(),
port: 51826,
setup_code: "111-22-333".into(),
device_id: "AA:BB:CC:DD:EE:FF".into(),
})
HapBridge::new(HapServiceRecord::bridge(
"RuView Sense",
51826,
"AA:BB:CC:DD:EE:FF",
))
}
fn light_state(name: &str, on: bool, brightness: u8) -> (EntityId, State) {
let eid = EntityId::parse(&format!("light.{name}")).unwrap();
let eid = EntityId::parse(format!("light.{name}")).unwrap();
let attrs = serde_json::json!({"brightness": brightness});
let s = State::new(eid.clone(), if on { "on" } else { "off" }, attrs, Context::default());
let s = State::new(
eid.clone(),
if on { "on" } else { "off" },
attrs,
Context::default(),
);
(eid, s)
}
@@ -187,6 +244,22 @@ mod tests {
assert!(matches!(err, HapError::EntityNotFound(_)));
}
#[tokio::test]
async fn update_emits_characteristic_event() {
let bridge = make_bridge();
let (eid, initial) = light_state("kitchen", false, 10);
bridge.add_accessory(&eid, &initial).unwrap();
let mut events = bridge.subscribe_events();
let (_, updated) = light_state("kitchen", true, 200);
bridge.update_accessory(&eid, &updated).unwrap();
let event = events.recv().await.unwrap();
assert_eq!(event.entity_id, eid);
assert!(event
.characteristics
.contains(&(HapCharacteristic::On, HapCharacteristicValue::Bool(true))));
}
#[tokio::test]
async fn start_stop_with_null_advertiser() {
let bridge = make_bridge();
+253
View File
@@ -0,0 +1,253 @@
//! HAP cryptographic composition over RustCrypto primitives.
#![cfg_attr(not(feature = "hap-server"), allow(dead_code))]
use chacha20poly1305::aead::{Aead, Payload};
use chacha20poly1305::{ChaCha20Poly1305, KeyInit, Nonce};
use hkdf::Hkdf;
use sha2::Sha512;
use zeroize::{Zeroize, ZeroizeOnDrop};
use crate::error::HapError;
pub(crate) const MAX_RECORD_PLAINTEXT: usize = 1024;
pub(crate) const RECORD_TAG_BYTES: usize = 16;
pub(crate) fn hkdf_sha512(
salt: &[u8],
input_key: &[u8],
info: &[u8],
) -> Result<[u8; 32], HapError> {
let mut output = [0u8; 32];
Hkdf::<Sha512>::new(Some(salt), input_key)
.expand(info, &mut output)
.map_err(|_| HapError::Protocol("HKDF output length is invalid".into()))?;
Ok(output)
}
fn label_nonce(label: &[u8; 8]) -> [u8; 12] {
let mut nonce = [0u8; 12];
nonce[4..].copy_from_slice(label);
nonce
}
pub(crate) fn seal_labeled(
key: &[u8; 32],
label: &[u8; 8],
plaintext: &[u8],
) -> Result<Vec<u8>, HapError> {
seal(key, &label_nonce(label), plaintext, &[])
}
pub(crate) fn open_labeled(
key: &[u8; 32],
label: &[u8; 8],
ciphertext_and_tag: &[u8],
) -> Result<Vec<u8>, HapError> {
open(key, &label_nonce(label), ciphertext_and_tag, &[])
}
fn seal(
key: &[u8; 32],
nonce: &[u8; 12],
plaintext: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, HapError> {
ChaCha20Poly1305::new(key.into())
.encrypt(
Nonce::from_slice(nonce),
Payload {
msg: plaintext,
aad,
},
)
.map_err(|_| HapError::Protocol("ChaCha20-Poly1305 encryption failed".into()))
}
fn open(
key: &[u8; 32],
nonce: &[u8; 12],
ciphertext_and_tag: &[u8],
aad: &[u8],
) -> Result<Vec<u8>, HapError> {
ChaCha20Poly1305::new(key.into())
.decrypt(
Nonce::from_slice(nonce),
Payload {
msg: ciphertext_and_tag,
aad,
},
)
.map_err(|_| HapError::Protocol("ChaCha20-Poly1305 authentication failed".into()))
}
#[derive(Zeroize, ZeroizeOnDrop)]
pub(crate) struct SessionKeys {
accessory_to_controller: [u8; 32],
controller_to_accessory: [u8; 32],
}
impl SessionKeys {
pub(crate) fn derive(shared_secret: &[u8; 32]) -> Result<Self, HapError> {
Ok(Self {
accessory_to_controller: hkdf_sha512(
b"Control-Salt",
shared_secret,
b"Control-Read-Encryption-Key",
)?,
controller_to_accessory: hkdf_sha512(
b"Control-Salt",
shared_secret,
b"Control-Write-Encryption-Key",
)?,
})
}
#[cfg(test)]
pub(crate) fn controller_view(&self) -> Self {
Self {
accessory_to_controller: self.controller_to_accessory,
controller_to_accessory: self.accessory_to_controller,
}
}
}
/// Stateful HAP IP record protection. A failed decryption is terminal: callers
/// must close the connection and must never retry with the same counter.
#[derive(Zeroize, ZeroizeOnDrop)]
pub(crate) struct RecordLayer {
read_key: [u8; 32],
write_key: [u8; 32],
read_counter: u64,
write_counter: u64,
}
impl RecordLayer {
pub(crate) fn accessory(keys: SessionKeys) -> Self {
Self {
read_key: keys.controller_to_accessory,
write_key: keys.accessory_to_controller,
read_counter: 0,
write_counter: 0,
}
}
#[cfg(test)]
pub(crate) fn controller(keys: SessionKeys) -> Self {
Self {
read_key: keys.controller_to_accessory,
write_key: keys.accessory_to_controller,
read_counter: 0,
write_counter: 0,
}
}
pub(crate) fn encrypt(&mut self, plaintext: &[u8]) -> Result<Vec<u8>, HapError> {
let mut output = Vec::with_capacity(
plaintext.len() + plaintext.len().div_ceil(MAX_RECORD_PLAINTEXT) * 18,
);
for chunk in plaintext.chunks(MAX_RECORD_PLAINTEXT) {
let length = u16::try_from(chunk.len())
.expect("HAP record chunks never exceed the u16 range")
.to_le_bytes();
let nonce = record_nonce(self.write_counter);
let encrypted = seal(&self.write_key, &nonce, chunk, &length)?;
self.write_counter = self
.write_counter
.checked_add(1)
.ok_or_else(|| HapError::Protocol("HAP write nonce exhausted".into()))?;
output.extend_from_slice(&length);
output.extend_from_slice(&encrypted);
}
Ok(output)
}
pub(crate) fn decrypt(
&mut self,
length_bytes: [u8; 2],
ciphertext_and_tag: &[u8],
) -> Result<Vec<u8>, HapError> {
let length = u16::from_le_bytes(length_bytes) as usize;
if length > MAX_RECORD_PLAINTEXT {
return Err(HapError::Protocol(
"encrypted HAP record exceeds 1024 bytes".into(),
));
}
if ciphertext_and_tag.len() != length + RECORD_TAG_BYTES {
return Err(HapError::Protocol(
"encrypted HAP record length does not match framing".into(),
));
}
let nonce = record_nonce(self.read_counter);
let plaintext = open(&self.read_key, &nonce, ciphertext_and_tag, &length_bytes)?;
self.read_counter = self
.read_counter
.checked_add(1)
.ok_or_else(|| HapError::Protocol("HAP read nonce exhausted".into()))?;
Ok(plaintext)
}
}
fn record_nonce(counter: u64) -> [u8; 12] {
let mut nonce = [0u8; 12];
nonce[4..].copy_from_slice(&counter.to_le_bytes());
nonce
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn deterministic_record_vector_and_multiframe_roundtrip() {
let shared = [0x42; 32];
let keys = SessionKeys::derive(&shared).unwrap();
let mut vector_accessory = RecordLayer::accessory(keys);
let vector = vector_accessory.encrypt(b"HAP").unwrap();
assert_eq!(
vector,
[
0x03, 0x00, 0xa2, 0x37, 0x30, 0x29, 0xba, 0xe2, 0xa9, 0xa6, 0xbb, 0x5b, 0xff, 0xed,
0x6a, 0x29, 0x74, 0x12, 0xd1, 0x6d, 0x7a,
]
);
let mut accessory = RecordLayer::accessory(SessionKeys::derive(&shared).unwrap());
let controller_keys = SessionKeys::derive(&shared).unwrap().controller_view();
let mut controller = RecordLayer::controller(controller_keys);
let plaintext = vec![0x5a; 2050];
let encrypted = accessory.encrypt(&plaintext).unwrap();
assert_eq!(&encrypted[..2], &[0, 4]);
let mut offset = 0;
let mut decrypted = Vec::new();
while offset < encrypted.len() {
let length_bytes: [u8; 2] = encrypted[offset..offset + 2].try_into().unwrap();
let length = u16::from_le_bytes(length_bytes) as usize;
let end = offset + 2 + length + RECORD_TAG_BYTES;
decrypted.extend(
controller
.decrypt(length_bytes, &encrypted[offset + 2..end])
.unwrap(),
);
offset = end;
}
assert_eq!(decrypted, plaintext);
}
#[test]
fn replay_tamper_and_oversize_fail_closed() {
let shared = [7; 32];
let mut sender = RecordLayer::accessory(SessionKeys::derive(&shared).unwrap());
let frame = sender.encrypt(b"authenticated").unwrap();
let length: [u8; 2] = frame[..2].try_into().unwrap();
let mut receiver =
RecordLayer::controller(SessionKeys::derive(&shared).unwrap().controller_view());
assert_eq!(
receiver.decrypt(length, &frame[2..]).unwrap(),
b"authenticated"
);
assert!(receiver.decrypt(length, &frame[2..]).is_err());
assert!(receiver
.decrypt(1025u16.to_le_bytes(), &vec![0; 1025 + RECORD_TAG_BYTES])
.is_err());
}
}
+31
View File
@@ -1,5 +1,6 @@
//! Unified error type for `homecore-hap`.
use std::path::PathBuf;
use thiserror::Error;
/// Errors produced by the HAP bridge and its sub-components.
@@ -19,4 +20,34 @@ pub enum HapError {
#[error("bridge not running")]
NotRunning,
#[error("pairing store error: {0}")]
PairingStore(String),
#[error("invalid pairing record: {0}")]
InvalidPairingRecord(String),
#[error("controller pairing already exists: {0}")]
PairingAlreadyExists(String),
#[error("controller pairing not found: {0}")]
PairingNotFound(String),
#[error("maximum controller pairings reached")]
PairingCapacity,
#[error("insecure permissions on {path}: mode {mode:o}; expected no group/other access")]
InsecurePermissions { path: PathBuf, mode: u32 },
#[error("invalid HAP session transition from {from} to {to}")]
InvalidSessionTransition {
from: &'static str,
to: &'static str,
},
#[error("HAP protocol error: {0}")]
Protocol(String),
#[error("HAP server error: {0}")]
Server(String),
}
+25 -8
View File
@@ -1,12 +1,11 @@
//! `homecore-hap` — Apple Home HomeKit Accessory Protocol bridge (ADR-125).
//!
//! # P1 scope
//! # Network foundation scope
//!
//! Ships the trait surface and type definitions needed to map HOMECORE entity
//! states onto HAP accessory / characteristic values. The actual HAP-1.1 TLS
//! server and real mDNS advertisement are gated behind the `hap-server`
//! feature (P2). P1 ships `NullAdvertiser` (no-op) so the bridge compiles and
//! all tests pass with `--no-default-features`.
//! The crate provides persisted accessory/controller identity, SRP-6a
//! Pair-Setup, X25519/Ed25519 Pair-Verify, encrypted HAP IP framing, bounded
//! TLV8/HTTP parsing, characteristic event flow, and (with `hap-server`) a
//! bounded TCP listener plus real mDNS.
//!
//! # Module layout
//!
@@ -16,19 +15,37 @@
//! | [`mapping`] | `EntityToAccessoryMapper` — HOMECORE entity → HAP |
//! | [`bridge`] | `HapBridge` — owns exposed accessories |
//! | [`mdns`] | `MdnsAdvertiser` trait + `NullAdvertiser` stub |
//! | [`pairing`] | Atomic accessory identity, setup, and pairing persistence |
//! | [`protocol`] | Bounded TLV8 protocol primitives |
//! | [`ruview`] | `RuViewToHapMapper` — sensing primitives → HAP |
//! | [`session`] | Authenticated request-gating state machine |
//! | `server` | Feature-gated bounded TCP/HTTP lifecycle |
//! | [`error`] | Unified `HapError` type |
pub mod accessory;
pub mod bridge;
mod crypto;
pub mod error;
pub mod mapping;
pub mod mdns;
mod pair_setup;
mod pair_verify;
pub mod pairing;
pub mod protocol;
pub mod ruview;
#[cfg(feature = "hap-server")]
pub mod server;
pub mod session;
pub use accessory::{HapAccessoryType, HapCharacteristic, HapCharacteristicValue};
pub use bridge::{ExposedAccessory, HapBridge};
pub use bridge::{CharacteristicEvent, ExposedAccessory, HapBridge};
pub use error::HapError;
pub use mapping::EntityToAccessoryMapper;
pub use mdns::{MdnsAdvertiser, NullAdvertiser};
#[cfg(feature = "hap-server")]
pub use mdns::MdnsSdAdvertiser;
pub use mdns::{HapServiceRecord, MdnsAdvertiser, NullAdvertiser};
pub use pairing::{ControllerPairing, PairingStore, PairingStoreProvisioning, SetupCode};
pub use ruview::RuViewToHapMapper;
#[cfg(feature = "hap-server")]
pub use server::{start_server, HapServerConfig, HapServerHandle};
pub use session::{Session, SessionState};
+194 -37
View File
@@ -1,61 +1,206 @@
//! mDNS advertisement trait and P1 no-op stub.
//!
//! Real mDNS via the `mdns-sd` crate (https://crates.io/crates/mdns-sd)
//! lands in P2 behind the `hap-server` feature flag. P1 ships `NullAdvertiser`
//! so the bridge compiles and tests pass without any mDNS infrastructure.
//! HAP `_hap._tcp` advertisement.
use async_trait::async_trait;
use crate::error::HapError;
/// Service record advertised over mDNS for HAP discovery.
#[derive(Debug, Clone)]
/// HAP service record advertised over mDNS.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HapServiceRecord {
/// Service instance name shown in Apple Home ("RuView Sense").
/// Service instance shown in discovery UI.
pub instance_name: String,
/// TCP port the HAP server listens on (default 51826).
/// Bound HAP TCP port.
pub port: u16,
/// HAP pairing setup code (8 digits, formatted as XXX-XX-XXX).
pub setup_code: String,
/// Unique device ID (colon-separated MAC-like hex, required by HAP §5.4).
/// Stable colon-separated accessory device identifier.
pub device_id: String,
/// Accessory model (`md` TXT key).
pub model: String,
/// Configuration number (`c#`), incremented when accessory layout changes.
pub configuration_number: u32,
/// Current state number (`s#`).
pub state_number: u32,
/// HAP accessory category identifier. Bridges use `2`.
pub category: u16,
/// Whether a controller pairing already exists.
pub paired: bool,
}
/// Advertise (and retract) a HAP accessory over mDNS (`_hap._tcp`).
///
/// Implementors register the `_hap._tcp` service so HomePod / Apple TV can
/// discover the bridge and initiate pairing. P1 provides only `NullAdvertiser`.
impl HapServiceRecord {
pub fn bridge(
instance_name: impl Into<String>,
port: u16,
device_id: impl Into<String>,
) -> Self {
Self {
instance_name: instance_name.into(),
port,
device_id: device_id.into(),
model: "HOMECORE Bridge".into(),
configuration_number: 1,
state_number: 1,
category: 2,
paired: false,
}
}
fn validate(&self) -> Result<(), HapError> {
if self.instance_name.is_empty() || self.instance_name.len() > 63 {
return Err(HapError::MdnsError(
"instance_name must contain 1..=63 bytes".into(),
));
}
if self.port == 0 {
return Err(HapError::MdnsError("advertised port cannot be zero".into()));
}
let parts: Vec<&str> = self.device_id.split(':').collect();
if parts.len() != 6
|| parts
.iter()
.any(|part| part.len() != 2 || !part.bytes().all(|byte| byte.is_ascii_hexdigit()))
{
return Err(HapError::MdnsError(
"device_id must be six colon-separated hexadecimal octets".into(),
));
}
if self.model.is_empty() || self.model.len() > 64 {
return Err(HapError::MdnsError(
"model must contain 1..=64 bytes".into(),
));
}
if self.configuration_number == 0 || self.state_number == 0 {
return Err(HapError::MdnsError(
"configuration and state numbers must be non-zero".into(),
));
}
Ok(())
}
/// Standard HAP Bonjour TXT keys. Setup codes and controller data are
/// intentionally never included because TXT records are plaintext.
pub fn txt_records(&self) -> Vec<(String, String)> {
vec![
("c#".into(), self.configuration_number.to_string()),
("ci".into(), self.category.to_string()),
("ff".into(), "0".into()),
("id".into(), self.device_id.to_ascii_uppercase()),
("md".into(), self.model.clone()),
("pv".into(), "1.1".into()),
("s#".into(), self.state_number.to_string()),
("sf".into(), if self.paired { "0" } else { "1" }.into()),
]
}
}
/// Advertise and retract a HAP service.
#[async_trait]
pub trait MdnsAdvertiser: Send + Sync {
/// Begin advertising the service. Idempotent.
async fn advertise(&self, record: &HapServiceRecord) -> Result<(), HapError>;
/// Stop advertising. Called on bridge shutdown.
async fn retract(&self, instance_name: &str) -> Result<(), HapError>;
}
/// No-op advertiser for P1 / test environments.
///
/// All calls succeed without touching the network.
/// Deterministic no-network advertiser for tests and disabled deployments.
#[derive(Debug, Default, Clone)]
pub struct NullAdvertiser;
#[async_trait]
impl MdnsAdvertiser for NullAdvertiser {
async fn advertise(&self, record: &HapServiceRecord) -> Result<(), HapError> {
record.validate()?;
tracing::debug!(
instance = %record.instance_name,
port = record.port,
"NullAdvertiser: skipping mDNS advertisement (P1 stub)"
"HAP mDNS advertisement disabled"
);
Ok(())
}
async fn retract(&self, instance_name: &str) -> Result<(), HapError> {
tracing::debug!(
instance = %instance_name,
"NullAdvertiser: skipping mDNS retract (P1 stub)"
);
tracing::debug!(instance = %instance_name, "HAP mDNS retraction disabled");
Ok(())
}
}
/// Network-backed Bonjour advertiser.
#[cfg(feature = "hap-server")]
pub struct MdnsSdAdvertiser {
daemon: mdns_sd::ServiceDaemon,
hostname: String,
address: String,
registrations: std::sync::Mutex<std::collections::HashMap<String, String>>,
}
#[cfg(feature = "hap-server")]
impl std::fmt::Debug for MdnsSdAdvertiser {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("MdnsSdAdvertiser")
.field("hostname", &self.hostname)
.field("address", &self.address)
.finish_non_exhaustive()
}
}
#[cfg(feature = "hap-server")]
impl MdnsSdAdvertiser {
/// Bind the mDNS daemon for a LAN-routable host/address.
pub fn new(hostname: impl Into<String>, address: std::net::IpAddr) -> Result<Self, HapError> {
let mut hostname = hostname.into();
if !hostname.ends_with(".local.") {
hostname = format!("{}.local.", hostname.trim_end_matches('.'));
}
let daemon = mdns_sd::ServiceDaemon::new()
.map_err(|error| HapError::MdnsError(error.to_string()))?;
Ok(Self {
daemon,
hostname,
address: address.to_string(),
registrations: std::sync::Mutex::new(std::collections::HashMap::new()),
})
}
fn service_info(&self, record: &HapServiceRecord) -> Result<mdns_sd::ServiceInfo, HapError> {
record.validate()?;
let properties: std::collections::HashMap<String, String> =
record.txt_records().into_iter().collect();
mdns_sd::ServiceInfo::new(
"_hap._tcp.local.",
&record.instance_name,
&self.hostname,
self.address.as_str(),
record.port,
Some(properties),
)
.map_err(|error| HapError::MdnsError(error.to_string()))
}
}
#[cfg(feature = "hap-server")]
#[async_trait]
impl MdnsAdvertiser for MdnsSdAdvertiser {
async fn advertise(&self, record: &HapServiceRecord) -> Result<(), HapError> {
let info = self.service_info(record)?;
let fullname = info.get_fullname().to_owned();
self.daemon
.register(info)
.map_err(|error| HapError::MdnsError(error.to_string()))?;
self.registrations
.lock()
.map_err(|_| HapError::MdnsError("registration lock poisoned".into()))?
.insert(record.instance_name.clone(), fullname);
Ok(())
}
async fn retract(&self, instance_name: &str) -> Result<(), HapError> {
let fullname = self
.registrations
.lock()
.map_err(|_| HapError::MdnsError("registration lock poisoned".into()))?
.remove(instance_name);
if let Some(fullname) = fullname {
self.daemon
.unregister(&fullname)
.map_err(|error| HapError::MdnsError(error.to_string()))?;
}
Ok(())
}
}
@@ -64,16 +209,28 @@ impl MdnsAdvertiser for NullAdvertiser {
mod tests {
use super::*;
#[test]
fn txt_record_is_hap_shaped_and_contains_no_setup_secret() {
let record = HapServiceRecord::bridge("RuView Sense", 51826, "AA:BB:CC:DD:EE:FF");
let txt: std::collections::HashMap<_, _> = record.txt_records().into_iter().collect();
assert_eq!(txt.get("pv").map(String::as_str), Some("1.1"));
assert_eq!(txt.get("sf").map(String::as_str), Some("1"));
assert_eq!(txt.get("ci").map(String::as_str), Some("2"));
assert!(!txt
.keys()
.any(|key| key.contains("pin") || key.contains("code")));
}
#[tokio::test]
async fn null_advertiser_is_noop() {
let adv = NullAdvertiser;
let rec = HapServiceRecord {
instance_name: "RuView Sense".into(),
port: 51826,
setup_code: "111-22-333".into(),
device_id: "AA:BB:CC:DD:EE:FF".into(),
};
adv.advertise(&rec).await.unwrap();
adv.retract(&rec.instance_name).await.unwrap();
async fn null_advertiser_validates_without_network_io() {
let record = HapServiceRecord::bridge("RuView Sense", 51826, "AA:BB:CC:DD:EE:FF");
NullAdvertiser.advertise(&record).await.unwrap();
NullAdvertiser.retract(&record.instance_name).await.unwrap();
}
#[tokio::test]
async fn malformed_record_is_rejected() {
let record = HapServiceRecord::bridge("RuView Sense", 0, "not-an-id");
assert!(NullAdvertiser.advertise(&record).await.is_err());
}
}
+478
View File
@@ -0,0 +1,478 @@
//! Server-side HAP Pair-Setup M1-M6.
#![cfg_attr(not(feature = "hap-server"), allow(dead_code))]
use std::sync::Arc;
use ed25519_dalek::{Signature, Signer, VerifyingKey};
use sha2_11::Sha512;
use srp::ServerG3072;
use zeroize::{Zeroize, Zeroizing};
use crate::crypto::{hkdf_sha512, open_labeled, seal_labeled};
use crate::error::HapError;
use crate::pairing::{ControllerPairing, PairingStore};
use crate::protocol::{
encode_items, error_response, Tlv8, TLV_ENCRYPTED_DATA, TLV_ERROR_AUTHENTICATION,
TLV_ERROR_BUSY, TLV_ERROR_MAX_TRIES, TLV_ERROR_UNAVAILABLE, TLV_FLAGS, TLV_IDENTIFIER,
TLV_METHOD, TLV_PROOF, TLV_PUBLIC_KEY, TLV_SALT, TLV_SIGNATURE, TLV_STATE,
};
const SRP_USERNAME: &[u8] = b"Pair-Setup";
const PAIR_SETUP_METHOD: u8 = 0;
enum Phase {
Idle,
AwaitM3 {
salt: [u8; 16],
verifier: Vec<u8>,
server_secret: Zeroizing<Vec<u8>>,
},
AwaitM5 {
session_key: Zeroizing<Vec<u8>>,
},
}
pub(crate) struct PairSetup {
store: Arc<PairingStore>,
phase: Phase,
owns_global_slot: bool,
}
pub(crate) struct PairSetupResponse {
pub(crate) body: Vec<u8>,
pub(crate) paired: bool,
pub(crate) terminal: bool,
}
impl PairSetup {
pub(crate) fn new(store: Arc<PairingStore>) -> Self {
Self {
store,
phase: Phase::Idle,
owns_global_slot: false,
}
}
pub(crate) fn handle(&mut self, request: &[u8]) -> Result<PairSetupResponse, HapError> {
let tlv = Tlv8::parse(request)?;
let state = tlv
.byte(TLV_STATE)
.ok_or_else(|| HapError::Protocol("Pair-Setup requires one-byte State".into()))?;
match state {
1 => self.m1(&tlv),
3 => self.m3(&tlv),
5 => self.m5(&tlv),
_ => Err(HapError::Protocol(
"Pair-Setup state is out of sequence".into(),
)),
}
}
fn m1(&mut self, tlv: &Tlv8) -> Result<PairSetupResponse, HapError> {
if !matches!(self.phase, Phase::Idle) {
return Err(HapError::Protocol("Pair-Setup M1 was replayed".into()));
}
if tlv.byte(TLV_METHOD) != Some(PAIR_SETUP_METHOD) {
return Ok(response(
error_response(2, TLV_ERROR_UNAVAILABLE),
false,
true,
));
}
if tlv.get(TLV_FLAGS).is_some() {
// Transient/split setup changes the session-key lifecycle and is
// deliberately rejected instead of being partially implemented.
return Ok(response(
error_response(2, TLV_ERROR_UNAVAILABLE),
false,
true,
));
}
if self.store.is_paired()? {
return Ok(response(
error_response(2, TLV_ERROR_UNAVAILABLE),
false,
true,
));
}
if self.store.pair_setup_locked_out() {
return Ok(response(
error_response(2, TLV_ERROR_MAX_TRIES),
false,
true,
));
}
if !self.store.try_begin_pair_setup() {
return Ok(response(error_response(2, TLV_ERROR_BUSY), false, true));
}
self.owns_global_slot = true;
let (salt, verifier) = self.store.setup_record()?;
let mut server_secret = Zeroizing::new(vec![0u8; 64]);
getrandom::getrandom(server_secret.as_mut_slice())
.map_err(|error| HapError::Protocol(format!("generate SRP secret: {error}")))?;
let server = ServerG3072::<Sha512>::new_with_options(false);
let public_key = server.compute_public_ephemeral(&server_secret, &verifier);
self.phase = Phase::AwaitM3 {
salt,
verifier,
server_secret,
};
Ok(response(
encode_items([
(TLV_STATE, [2].as_slice()),
(TLV_PUBLIC_KEY, public_key.as_slice()),
(TLV_SALT, salt.as_slice()),
]),
false,
false,
))
}
fn m3(&mut self, tlv: &Tlv8) -> Result<PairSetupResponse, HapError> {
let Phase::AwaitM3 {
salt,
verifier,
server_secret,
} = std::mem::replace(&mut self.phase, Phase::Idle)
else {
return Err(HapError::Protocol(
"Pair-Setup M3 arrived without M1".into(),
));
};
let result = (|| {
let client_public = required_bounded(tlv, TLV_PUBLIC_KEY, 384, 384, "SRP public key")?;
let client_proof = required_bounded(tlv, TLV_PROOF, 64, 64, "SRP proof")?;
let server = ServerG3072::<Sha512>::new_with_options(false);
let verifier_state = server
.process_reply(
SRP_USERNAME,
&salt,
&server_secret,
&verifier,
client_public,
)
.map_err(|_| HapError::Protocol("invalid SRP public key".into()))?;
let session_key = verifier_state
.verify_client(client_proof)
.map_err(|_| HapError::Protocol("SRP proof rejected".into()))?
.to_vec();
let proof = verifier_state.proof().to_vec();
Ok::<_, HapError>((session_key, proof))
})();
match result {
Ok((session_key, proof)) => {
self.phase = Phase::AwaitM5 {
session_key: Zeroizing::new(session_key),
};
Ok(response(
encode_items([(TLV_STATE, [4].as_slice()), (TLV_PROOF, proof.as_slice())]),
false,
false,
))
}
Err(_) => {
self.store.record_pair_setup_failure();
self.release_slot();
Ok(response(
error_response(4, TLV_ERROR_AUTHENTICATION),
false,
true,
))
}
}
}
fn m5(&mut self, tlv: &Tlv8) -> Result<PairSetupResponse, HapError> {
let Phase::AwaitM5 { session_key } = std::mem::replace(&mut self.phase, Phase::Idle) else {
return Err(HapError::Protocol(
"Pair-Setup M5 arrived without authenticated M3".into(),
));
};
let result = self.finish_m5(tlv, &session_key);
if result.is_err() {
self.store.record_pair_setup_failure();
}
self.release_slot();
match result {
Ok(body) => Ok(response(body, true, true)),
Err(_) => Ok(response(
error_response(6, TLV_ERROR_AUTHENTICATION),
false,
true,
)),
}
}
fn finish_m5(&self, tlv: &Tlv8, session_key: &[u8]) -> Result<Vec<u8>, HapError> {
let encrypted = required_bounded(
tlv,
TLV_ENCRYPTED_DATA,
17,
4096,
"Pair-Setup encrypted data",
)?;
let encryption_key = hkdf_sha512(
b"Pair-Setup-Encrypt-Salt",
session_key,
b"Pair-Setup-Encrypt-Info",
)?;
let mut plaintext = Zeroizing::new(open_labeled(&encryption_key, b"PS-Msg05", encrypted)?);
let sub_tlv = Tlv8::parse(&plaintext)?;
let controller_id_bytes =
required_bounded(&sub_tlv, TLV_IDENTIFIER, 1, 64, "controller identifier")?;
let controller_id = std::str::from_utf8(controller_id_bytes)
.map_err(|_| HapError::Protocol("controller identifier is not UTF-8".into()))?
.to_owned();
let controller_key: [u8; 32] =
required_bounded(&sub_tlv, TLV_PUBLIC_KEY, 32, 32, "controller LTPK")?
.try_into()
.expect("length checked");
let signature_bytes: [u8; 64] =
required_bounded(&sub_tlv, TLV_SIGNATURE, 64, 64, "controller signature")?
.try_into()
.expect("length checked");
let verifying_key = VerifyingKey::from_bytes(&controller_key)
.map_err(|_| HapError::Protocol("controller LTPK is invalid".into()))?;
let controller_x = hkdf_sha512(
b"Pair-Setup-Controller-Sign-Salt",
session_key,
b"Pair-Setup-Controller-Sign-Info",
)?;
let mut controller_info =
Zeroizing::new(Vec::with_capacity(32 + controller_id_bytes.len() + 32));
controller_info.extend_from_slice(&controller_x);
controller_info.extend_from_slice(controller_id_bytes);
controller_info.extend_from_slice(&controller_key);
verifying_key
.verify_strict(&controller_info, &Signature::from_bytes(&signature_bytes))
.map_err(|_| HapError::Protocol("controller signature rejected".into()))?;
let signing_key = self.store.signing_key()?;
let accessory_id = self.store.accessory_id()?;
let accessory_public = signing_key.verifying_key().to_bytes();
let accessory_x = hkdf_sha512(
b"Pair-Setup-Accessory-Sign-Salt",
session_key,
b"Pair-Setup-Accessory-Sign-Info",
)?;
let mut accessory_info = Zeroizing::new(Vec::with_capacity(32 + accessory_id.len() + 32));
accessory_info.extend_from_slice(&accessory_x);
accessory_info.extend_from_slice(accessory_id.as_bytes());
accessory_info.extend_from_slice(&accessory_public);
let accessory_signature = signing_key.sign(&accessory_info).to_bytes();
let mut response_plaintext = Zeroizing::new(encode_items([
(TLV_IDENTIFIER, accessory_id.as_bytes()),
(TLV_PUBLIC_KEY, accessory_public.as_slice()),
(TLV_SIGNATURE, accessory_signature.as_slice()),
]));
let response_encrypted = seal_labeled(&encryption_key, b"PS-Msg06", &response_plaintext)?;
// Commit only after every authentication and response construction
// step has succeeded, and before emitting success-shaped M6.
self.store.add_initial(ControllerPairing {
controller_id,
public_key: controller_key,
admin: true,
})?;
plaintext.zeroize();
response_plaintext.zeroize();
Ok(encode_items([
(TLV_STATE, [6].as_slice()),
(TLV_ENCRYPTED_DATA, response_encrypted.as_slice()),
]))
}
fn release_slot(&mut self) {
if self.owns_global_slot {
self.store.end_pair_setup();
self.owns_global_slot = false;
}
}
}
impl Drop for PairSetup {
fn drop(&mut self) {
self.release_slot();
}
}
fn response(body: Vec<u8>, paired: bool, terminal: bool) -> PairSetupResponse {
PairSetupResponse {
body,
paired,
terminal,
}
}
fn required_bounded<'a>(
tlv: &'a Tlv8,
kind: u8,
min: usize,
max: usize,
name: &str,
) -> Result<&'a [u8], HapError> {
let value = tlv
.get(kind)
.ok_or_else(|| HapError::Protocol(format!("missing {name}")))?;
if !(min..=max).contains(&value.len()) {
return Err(HapError::Protocol(format!(
"{name} must contain {min}..={max} bytes"
)));
}
Ok(value)
}
#[cfg(test)]
mod tests {
use super::*;
use ed25519_dalek::SigningKey;
use srp::ClientG3072;
fn setup() -> (tempfile::TempDir, Arc<PairingStore>) {
let directory = tempfile::tempdir().unwrap();
let store = PairingStore::create(
directory.path().join("pairings.json"),
crate::pairing::SetupCode::parse("518-26-003").unwrap(),
Some("AA:BB:CC:DD:EE:FF".into()),
)
.unwrap();
(directory, Arc::new(store))
}
#[test]
fn apple_hap_srp_3072_sha512_session_key_vector() {
let decode = |value: &str| {
value
.as_bytes()
.chunks_exact(2)
.map(|pair| u8::from_str_radix(std::str::from_utf8(pair).unwrap(), 16).unwrap())
.collect::<Vec<_>>()
};
let salt = decode("BEB25379D1A8581EB5A727673A2441EE");
let a = decode("60975527035CF2AD1989806F0407210BC81EDC04E2762A56AFD529DDDA2D4393");
let b = decode("E487CB59D31AC550471E81F00F6928E01DDA08E974A004F49E61F5D105284D20");
let expected_key = decode(
"5CBC219DB052138EE1148C71CD4498963D682549CE91CA24F098468F06015BEB\
6AF245C2093F98C3651BCA83AB8CAB2B580BBF02184FEFDF26142F73DF95AC50",
);
let client = ClientG3072::<Sha512>::new();
let server = ServerG3072::<Sha512>::new_with_options(false);
let verifier = client.compute_verifier(b"alice", b"password123", &salt);
let a_public = client.compute_public_ephemeral(&a);
let b_public = server.compute_public_ephemeral(&b, &verifier);
let client_state = client
.process_reply(&a, b"alice", b"password123", &salt, &b_public)
.unwrap();
let server_state = server
.process_reply(b"alice", &salt, &b, &verifier, &a_public)
.unwrap();
assert_eq!(
server_state.verify_client(client_state.proof()).unwrap(),
expected_key
);
assert_eq!(
client_state.verify_server(server_state.proof()).unwrap(),
expected_key
);
}
#[test]
fn full_m1_through_m6_persists_only_authenticated_controller() {
let (_directory, store) = setup();
let mut server = PairSetup::new(store.clone());
let m1 = encode_items([(TLV_STATE, [1].as_slice()), (TLV_METHOD, [0].as_slice())]);
let m2 = Tlv8::parse(&server.handle(&m1).unwrap().body).unwrap();
let salt = m2.get(TLV_SALT).unwrap();
let server_public = m2.get(TLV_PUBLIC_KEY).unwrap();
let client = ClientG3072::<Sha512>::new();
let client_secret = [0x31; 48];
let client_state = client
.process_reply(
&client_secret,
SRP_USERNAME,
b"518-26-003",
salt,
server_public,
)
.unwrap();
let client_public = client.compute_public_ephemeral(&client_secret);
let m3 = encode_items([
(TLV_STATE, [3].as_slice()),
(TLV_PUBLIC_KEY, client_public.as_slice()),
(TLV_PROOF, client_state.proof()),
]);
let m4 = Tlv8::parse(&server.handle(&m3).unwrap().body).unwrap();
let session_key = client_state
.verify_server(m4.get(TLV_PROOF).unwrap())
.unwrap();
let controller_signing = SigningKey::from_bytes(&[0x22; 32]);
let controller_public = controller_signing.verifying_key().to_bytes();
let controller_id = b"deterministic-controller";
let controller_x = hkdf_sha512(
b"Pair-Setup-Controller-Sign-Salt",
session_key,
b"Pair-Setup-Controller-Sign-Info",
)
.unwrap();
let mut info = Vec::new();
info.extend_from_slice(&controller_x);
info.extend_from_slice(controller_id);
info.extend_from_slice(&controller_public);
let signature = controller_signing.sign(&info).to_bytes();
let sub_tlv = encode_items([
(TLV_IDENTIFIER, controller_id.as_slice()),
(TLV_PUBLIC_KEY, controller_public.as_slice()),
(TLV_SIGNATURE, signature.as_slice()),
]);
let encryption_key = hkdf_sha512(
b"Pair-Setup-Encrypt-Salt",
session_key,
b"Pair-Setup-Encrypt-Info",
)
.unwrap();
let encrypted = seal_labeled(&encryption_key, b"PS-Msg05", &sub_tlv).unwrap();
let m5 = encode_items([
(TLV_STATE, [5].as_slice()),
(TLV_ENCRYPTED_DATA, encrypted.as_slice()),
]);
let result = server.handle(&m5).unwrap();
assert!(result.paired);
let m6 = Tlv8::parse(&result.body).unwrap();
assert!(open_labeled(
&encryption_key,
b"PS-Msg06",
m6.get(TLV_ENCRYPTED_DATA).unwrap()
)
.is_ok());
assert_eq!(
store
.get("deterministic-controller")
.unwrap()
.unwrap()
.public_key,
controller_public
);
}
#[test]
fn malformed_replayed_and_wrong_proof_requests_fail_closed() {
let (_directory, store) = setup();
let mut server = PairSetup::new(store.clone());
let m1 = encode_items([(TLV_STATE, [1].as_slice()), (TLV_METHOD, [0].as_slice())]);
assert!(!server.handle(&m1).unwrap().paired);
assert!(server.handle(&m1).is_err());
let bad_m3 = encode_items([
(TLV_STATE, [3].as_slice()),
(TLV_PUBLIC_KEY, [1].as_slice()),
(TLV_PROOF, [0u8; 64].as_slice()),
]);
let response = Tlv8::parse(&server.handle(&bad_m3).unwrap().body).unwrap();
assert_eq!(
response.byte(crate::protocol::TLV_ERROR),
Some(TLV_ERROR_AUTHENTICATION)
);
assert!(!store.is_paired().unwrap());
}
}
+332
View File
@@ -0,0 +1,332 @@
//! Server-side HAP Pair-Verify M1-M4.
#![cfg_attr(not(feature = "hap-server"), allow(dead_code))]
use std::sync::Arc;
use ed25519_dalek::{Signature, Signer, VerifyingKey};
use x25519_dalek::{PublicKey, StaticSecret};
use zeroize::{Zeroize, Zeroizing};
use crate::crypto::{hkdf_sha512, open_labeled, seal_labeled, SessionKeys};
use crate::error::HapError;
use crate::pairing::PairingStore;
use crate::protocol::{
encode_items, error_response, Tlv8, TLV_ENCRYPTED_DATA, TLV_ERROR_AUTHENTICATION,
TLV_IDENTIFIER, TLV_PUBLIC_KEY, TLV_SIGNATURE, TLV_STATE,
};
struct AwaitM3 {
controller_public: [u8; 32],
accessory_public: [u8; 32],
shared_secret: Zeroizing<[u8; 32]>,
session_key: Zeroizing<[u8; 32]>,
}
enum Phase {
Idle,
AwaitM3(AwaitM3),
}
pub(crate) struct AuthenticatedSession {
pub(crate) controller_id: String,
pub(crate) admin: bool,
pub(crate) keys: SessionKeys,
}
pub(crate) struct PairVerifyResponse {
pub(crate) body: Vec<u8>,
pub(crate) authenticated: Option<AuthenticatedSession>,
pub(crate) terminal: bool,
}
pub(crate) struct PairVerify {
store: Arc<PairingStore>,
phase: Phase,
}
impl PairVerify {
pub(crate) fn new(store: Arc<PairingStore>) -> Self {
Self {
store,
phase: Phase::Idle,
}
}
pub(crate) fn handle(&mut self, request: &[u8]) -> Result<PairVerifyResponse, HapError> {
let tlv = Tlv8::parse(request)?;
let state = tlv
.byte(TLV_STATE)
.ok_or_else(|| HapError::Protocol("Pair-Verify requires one-byte State".into()))?;
match state {
1 => self.m1(&tlv),
3 => self.m3(&tlv),
_ => Err(HapError::Protocol(
"Pair-Verify state is out of sequence".into(),
)),
}
}
fn m1(&mut self, tlv: &Tlv8) -> Result<PairVerifyResponse, HapError> {
if !matches!(self.phase, Phase::Idle) {
return Err(HapError::Protocol("Pair-Verify M1 was replayed".into()));
}
if !self.store.is_paired()? {
return Ok(PairVerifyResponse {
body: error_response(2, TLV_ERROR_AUTHENTICATION),
authenticated: None,
terminal: true,
});
}
let controller_public: [u8; 32] =
required_exact(tlv, TLV_PUBLIC_KEY, 32, "controller Curve25519 public key")?
.try_into()
.expect("length checked");
let mut secret_bytes = Zeroizing::new([0u8; 32]);
getrandom::getrandom(secret_bytes.as_mut())
.map_err(|error| HapError::Protocol(format!("generate X25519 secret: {error}")))?;
let secret = StaticSecret::from(*secret_bytes);
let accessory_public = PublicKey::from(&secret).to_bytes();
let shared = secret.diffie_hellman(&PublicKey::from(controller_public));
if !shared.was_contributory() {
return Ok(PairVerifyResponse {
body: error_response(2, TLV_ERROR_AUTHENTICATION),
authenticated: None,
terminal: true,
});
}
let shared_secret = Zeroizing::new(shared.to_bytes());
let accessory_id = self.store.accessory_id()?;
let signing_key = self.store.signing_key()?;
let mut accessory_info = Zeroizing::new(Vec::with_capacity(32 + accessory_id.len() + 32));
accessory_info.extend_from_slice(&accessory_public);
accessory_info.extend_from_slice(accessory_id.as_bytes());
accessory_info.extend_from_slice(&controller_public);
let signature = signing_key.sign(&accessory_info).to_bytes();
let plaintext = Zeroizing::new(encode_items([
(TLV_IDENTIFIER, accessory_id.as_bytes()),
(TLV_SIGNATURE, signature.as_slice()),
]));
let session_key = Zeroizing::new(hkdf_sha512(
b"Pair-Verify-Encrypt-Salt",
shared_secret.as_ref(),
b"Pair-Verify-Encrypt-Info",
)?);
let encrypted = seal_labeled(&session_key, b"PV-Msg02", &plaintext)?;
self.phase = Phase::AwaitM3(AwaitM3 {
controller_public,
accessory_public,
shared_secret,
session_key,
});
Ok(PairVerifyResponse {
body: encode_items([
(TLV_STATE, [2].as_slice()),
(TLV_PUBLIC_KEY, accessory_public.as_slice()),
(TLV_ENCRYPTED_DATA, encrypted.as_slice()),
]),
authenticated: None,
terminal: false,
})
}
fn m3(&mut self, tlv: &Tlv8) -> Result<PairVerifyResponse, HapError> {
let Phase::AwaitM3(state) = std::mem::replace(&mut self.phase, Phase::Idle) else {
return Err(HapError::Protocol(
"Pair-Verify M3 arrived without M1".into(),
));
};
match self.finish_m3(tlv, state) {
Ok(authenticated) => Ok(PairVerifyResponse {
body: encode_items([(TLV_STATE, [4].as_slice())]),
authenticated: Some(authenticated),
terminal: true,
}),
Err(_) => Ok(PairVerifyResponse {
body: error_response(4, TLV_ERROR_AUTHENTICATION),
authenticated: None,
terminal: true,
}),
}
}
fn finish_m3(&self, tlv: &Tlv8, state: AwaitM3) -> Result<AuthenticatedSession, HapError> {
let encrypted = tlv
.get(TLV_ENCRYPTED_DATA)
.filter(|value| (17..=4096).contains(&value.len()))
.ok_or_else(|| HapError::Protocol("invalid Pair-Verify encrypted data".into()))?;
let mut plaintext =
Zeroizing::new(open_labeled(&state.session_key, b"PV-Msg03", encrypted)?);
let sub_tlv = Tlv8::parse(&plaintext)?;
let controller_id_bytes = sub_tlv
.get(TLV_IDENTIFIER)
.filter(|value| !value.is_empty() && value.len() <= 64)
.ok_or_else(|| HapError::Protocol("invalid controller identifier".into()))?;
let controller_id = std::str::from_utf8(controller_id_bytes)
.map_err(|_| HapError::Protocol("controller identifier is not UTF-8".into()))?
.to_owned();
let signature_bytes: [u8; 64] =
required_exact(&sub_tlv, TLV_SIGNATURE, 64, "controller signature")?
.try_into()
.expect("length checked");
let pairing = self
.store
.get(&controller_id)?
.ok_or_else(|| HapError::Protocol("unknown controller pairing".into()))?;
let key = VerifyingKey::from_bytes(&pairing.public_key)
.map_err(|_| HapError::Protocol("persisted controller key is invalid".into()))?;
let mut controller_info =
Zeroizing::new(Vec::with_capacity(32 + controller_id_bytes.len() + 32));
controller_info.extend_from_slice(&state.controller_public);
controller_info.extend_from_slice(controller_id_bytes);
controller_info.extend_from_slice(&state.accessory_public);
key.verify_strict(&controller_info, &Signature::from_bytes(&signature_bytes))
.map_err(|_| HapError::Protocol("controller transcript signature rejected".into()))?;
let keys = SessionKeys::derive(&state.shared_secret)?;
plaintext.zeroize();
Ok(AuthenticatedSession {
controller_id,
admin: pairing.admin,
keys,
})
}
}
fn required_exact<'a>(
tlv: &'a Tlv8,
kind: u8,
length: usize,
name: &str,
) -> Result<&'a [u8], HapError> {
tlv.get(kind)
.filter(|value| value.len() == length)
.ok_or_else(|| HapError::Protocol(format!("{name} must contain {length} bytes")))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::crypto::RecordLayer;
use crate::pairing::{ControllerPairing, SetupCode};
use ed25519_dalek::SigningKey;
fn setup() -> (tempfile::TempDir, Arc<PairingStore>, SigningKey) {
let directory = tempfile::tempdir().unwrap();
let store = PairingStore::create(
directory.path().join("pairings.json"),
SetupCode::parse("518-26-003").unwrap(),
Some("AA:BB:CC:DD:EE:FF".into()),
)
.unwrap();
let controller = SigningKey::from_bytes(&[0x44; 32]);
store
.add_initial(ControllerPairing {
controller_id: "controller-1".into(),
public_key: controller.verifying_key().to_bytes(),
admin: true,
})
.unwrap();
(directory, Arc::new(store), controller)
}
#[test]
fn full_m1_through_m4_authenticates_transcript_and_derives_record_keys() {
let (_directory, store, controller_signing) = setup();
let mut server = PairVerify::new(store.clone());
let controller_secret = StaticSecret::from([0x33; 32]);
let controller_public = PublicKey::from(&controller_secret).to_bytes();
let m1 = encode_items([
(TLV_STATE, [1].as_slice()),
(TLV_PUBLIC_KEY, controller_public.as_slice()),
]);
let m2 = Tlv8::parse(&server.handle(&m1).unwrap().body).unwrap();
let accessory_public: [u8; 32] = m2.get(TLV_PUBLIC_KEY).unwrap().try_into().unwrap();
let shared = controller_secret.diffie_hellman(&PublicKey::from(accessory_public));
let session_key = hkdf_sha512(
b"Pair-Verify-Encrypt-Salt",
shared.as_bytes(),
b"Pair-Verify-Encrypt-Info",
)
.unwrap();
let accessory_sub_tlv = Tlv8::parse(
&open_labeled(
&session_key,
b"PV-Msg02",
m2.get(TLV_ENCRYPTED_DATA).unwrap(),
)
.unwrap(),
)
.unwrap();
let accessory_id = accessory_sub_tlv.get(TLV_IDENTIFIER).unwrap();
let accessory_signature: [u8; 64] = accessory_sub_tlv
.get(TLV_SIGNATURE)
.unwrap()
.try_into()
.unwrap();
let mut accessory_info = Vec::new();
accessory_info.extend_from_slice(&accessory_public);
accessory_info.extend_from_slice(accessory_id);
accessory_info.extend_from_slice(&controller_public);
VerifyingKey::from_bytes(&store.accessory_public_key().unwrap())
.unwrap()
.verify_strict(
&accessory_info,
&Signature::from_bytes(&accessory_signature),
)
.unwrap();
let controller_id = b"controller-1";
let mut controller_info = Vec::new();
controller_info.extend_from_slice(&controller_public);
controller_info.extend_from_slice(controller_id);
controller_info.extend_from_slice(&accessory_public);
let signature = controller_signing.sign(&controller_info).to_bytes();
let sub_tlv = encode_items([
(TLV_IDENTIFIER, controller_id.as_slice()),
(TLV_SIGNATURE, signature.as_slice()),
]);
let encrypted = seal_labeled(&session_key, b"PV-Msg03", &sub_tlv).unwrap();
let m3 = encode_items([
(TLV_STATE, [3].as_slice()),
(TLV_ENCRYPTED_DATA, encrypted.as_slice()),
]);
let result = server.handle(&m3).unwrap();
assert_eq!(Tlv8::parse(&result.body).unwrap().byte(TLV_STATE), Some(4));
let authenticated = result.authenticated.unwrap();
assert_eq!(authenticated.controller_id, "controller-1");
let mut accessory_records = RecordLayer::accessory(authenticated.keys);
let encrypted_record = accessory_records.encrypt(b"response").unwrap();
assert!(!encrypted_record
.windows(8)
.any(|window| window == b"response"));
}
#[test]
fn all_zero_key_replay_and_tamper_fail_closed() {
let (_directory, store, _) = setup();
let mut server = PairVerify::new(store);
let zero_key = encode_items([
(TLV_STATE, [1].as_slice()),
(TLV_PUBLIC_KEY, [0u8; 32].as_slice()),
]);
let response = Tlv8::parse(&server.handle(&zero_key).unwrap().body).unwrap();
assert_eq!(
response.byte(crate::protocol::TLV_ERROR),
Some(TLV_ERROR_AUTHENTICATION)
);
let controller_secret = StaticSecret::from([9; 32]);
let controller_public = PublicKey::from(&controller_secret).to_bytes();
let m1 = encode_items([
(TLV_STATE, [1].as_slice()),
(TLV_PUBLIC_KEY, controller_public.as_slice()),
]);
let _ = server.handle(&m1).unwrap();
assert!(server.handle(&m1).is_err());
let tampered = encode_items([
(TLV_STATE, [3].as_slice()),
(TLV_ENCRYPTED_DATA, [0u8; 17].as_slice()),
]);
let response = server.handle(&tampered).unwrap();
assert!(response.authenticated.is_none());
}
}
+776
View File
@@ -0,0 +1,776 @@
//! Durable HAP accessory identity, setup verifier, and controller pairings.
#![cfg_attr(not(feature = "hap-server"), allow(dead_code))]
use std::collections::BTreeMap;
use std::fs::{self, File};
use std::io::{Read, Write};
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicBool, AtomicU16, Ordering};
use std::sync::RwLock;
use ed25519_dalek::{SigningKey, VerifyingKey};
use serde::{Deserialize, Serialize};
use sha2_11::Sha512;
use srp::ClientG3072;
use tempfile::Builder;
use tokio::sync::watch;
use zeroize::{Zeroize, ZeroizeOnDrop, Zeroizing};
use crate::error::HapError;
const STORE_VERSION: u32 = 2;
const MAX_STORE_BYTES: u64 = 1024 * 1024;
const MAX_CONTROLLER_ID_BYTES: usize = 64;
const MAX_CONTROLLERS: usize = 16;
const MAX_PAIR_SETUP_FAILURES: u16 = 100;
const SRP_USERNAME: &[u8] = b"Pair-Setup";
/// A controller authorized by a completed HAP pairing ceremony.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct ControllerPairing {
/// Case-sensitive HAP controller pairing identifier.
pub controller_id: String,
/// Controller Ed25519 long-term public key.
pub public_key: [u8; 32],
/// Whether this controller may manage other pairings.
pub admin: bool,
}
impl ControllerPairing {
/// Validate bounded identifiers and the encoded Ed25519 point.
pub fn validate(&self) -> Result<(), HapError> {
let len = self.controller_id.len();
if len == 0
|| len > MAX_CONTROLLER_ID_BYTES
|| self.controller_id.chars().any(char::is_control)
{
return Err(HapError::InvalidPairingRecord(
"controller_id must contain 1..=64 bytes and no control characters".into(),
));
}
VerifyingKey::from_bytes(&self.public_key).map_err(|_| {
HapError::InvalidPairingRecord("controller public key is not valid Ed25519".into())
})?;
Ok(())
}
}
/// A newly provisioned HAP setup code.
///
/// The code is returned only when a store is created. The persisted file holds
/// an SRP verifier instead of the raw code. Call [`SetupCode::expose`] only at
/// the operator-facing provisioning boundary.
#[derive(Zeroize, ZeroizeOnDrop)]
pub struct SetupCode {
bytes: [u8; 10],
}
impl SetupCode {
pub fn parse(value: &str) -> Result<Self, HapError> {
let bytes: [u8; 10] = value.as_bytes().try_into().map_err(|_| {
HapError::InvalidPairingRecord("setup code must use the XXX-XX-XXX format".into())
})?;
if bytes[3] != b'-'
|| bytes[6] != b'-'
|| bytes
.iter()
.enumerate()
.any(|(index, byte)| index != 3 && index != 6 && !byte.is_ascii_digit())
{
return Err(HapError::InvalidPairingRecord(
"setup code must use the XXX-XX-XXX format".into(),
));
}
let digits: Vec<u8> = bytes.iter().copied().filter(u8::is_ascii_digit).collect();
if is_trivial_setup_code(&digits) {
return Err(HapError::InvalidPairingRecord(
"setup code is prohibited because it is trivial".into(),
));
}
Ok(Self { bytes })
}
/// Explicitly expose the code for a display, label, or provisioning UI.
pub fn expose(&self) -> &str {
std::str::from_utf8(&self.bytes).expect("validated setup code is ASCII")
}
pub(crate) fn as_bytes(&self) -> &[u8] {
&self.bytes
}
fn generate() -> Result<Self, HapError> {
const RANGE: u32 = 100_000_000;
const ACCEPT_BELOW: u32 = (u32::MAX / RANGE) * RANGE;
loop {
let mut random = [0u8; 4];
getrandom::getrandom(&mut random)
.map_err(|error| HapError::PairingStore(format!("generate setup code: {error}")))?;
let sample = u32::from_le_bytes(random);
if sample >= ACCEPT_BELOW {
continue;
}
let digits = format!("{:08}", sample % RANGE);
if is_trivial_setup_code(digits.as_bytes()) {
continue;
}
return Self::parse(&format!(
"{}-{}-{}",
&digits[..3],
&digits[3..5],
&digits[5..]
));
}
}
}
impl std::fmt::Debug for SetupCode {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("SetupCode([REDACTED])")
}
}
fn is_trivial_setup_code(digits: &[u8]) -> bool {
digits == b"12345678"
|| digits == b"87654321"
|| (digits.len() == 8 && digits.iter().all(|digit| *digit == digits[0]))
}
/// Result of opening an existing store or provisioning a new one.
pub struct PairingStoreProvisioning {
pub store: PairingStore,
/// Present only on first creation. It is never recoverable from the store.
pub setup_code: Option<SetupCode>,
}
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct StoredAccessory {
device_id: String,
signing_seed: [u8; 32],
}
// Manual, redacted impl: `signing_seed` is the accessory's permanent Ed25519
// identity key, used to sign every Pair-Setup/Pair-Verify transcript for the
// device's whole lifetime with no rotation mechanism. A derived `Debug` would
// print it in plaintext the first time anything formats this struct (a log
// line, a panic message) — same rationale as `SetupCode`'s manual impl below.
impl std::fmt::Debug for StoredAccessory {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter
.debug_struct("StoredAccessory")
.field("device_id", &self.device_id)
.field("signing_seed", &"[REDACTED]")
.finish()
}
}
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct StoredSetup {
salt: [u8; 16],
verifier: Vec<u8>,
}
// Manual, redacted impl: `salt`/`verifier` are the SRP-6a material derived
// from the setup code. Printing them would hand an attacker exactly what an
// offline dictionary attack against the (8-digit) setup code needs.
impl std::fmt::Debug for StoredSetup {
fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
formatter.write_str("StoredSetup([REDACTED])")
}
}
impl Drop for StoredAccessory {
fn drop(&mut self) {
self.signing_seed.zeroize();
}
}
impl Drop for StoredSetup {
fn drop(&mut self) {
self.salt.zeroize();
self.verifier.zeroize();
}
}
#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
struct PairingFile {
version: u32,
accessory: StoredAccessory,
setup: StoredSetup,
controllers: Vec<ControllerPairing>,
}
#[derive(Debug, Clone)]
struct StoreState {
accessory: StoredAccessory,
setup: StoredSetup,
controllers: BTreeMap<String, ControllerPairing>,
}
/// Thread-safe, atomically persisted HAP security store.
#[derive(Debug)]
pub struct PairingStore {
path: PathBuf,
state: RwLock<StoreState>,
pair_setup_active: AtomicBool,
pair_setup_failures: AtomicU16,
changes: watch::Sender<u64>,
}
impl PairingStore {
/// Open an existing v2 store.
pub fn open(path: impl Into<PathBuf>) -> Result<Self, HapError> {
let path = path.into();
if !path.exists() {
return Err(HapError::PairingStore(format!(
"{} does not exist; use PairingStore::load_or_create for first provisioning",
path.display()
)));
}
Self::from_state(path.clone(), load_file(&path)?)
}
/// Open a store, or securely create identity/setup material and return the
/// one-time setup code.
pub fn load_or_create(path: impl Into<PathBuf>) -> Result<PairingStoreProvisioning, HapError> {
let path = path.into();
if path.exists() {
return Ok(PairingStoreProvisioning {
store: Self::open(path)?,
setup_code: None,
});
}
let setup_code = SetupCode::generate()?;
let state = new_state(&setup_code, None)?;
persist_file(&path, &state, false)?;
Ok(PairingStoreProvisioning {
store: Self::from_state(path, state)?,
setup_code: Some(setup_code),
})
}
/// Deterministic provisioning entry point for an externally generated
/// per-accessory setup code and optional device ID.
pub fn create(
path: impl Into<PathBuf>,
setup_code: SetupCode,
device_id: Option<String>,
) -> Result<Self, HapError> {
let path = path.into();
if path.exists() {
return Err(HapError::PairingStore(format!(
"{} already exists",
path.display()
)));
}
let state = new_state(&setup_code, device_id)?;
persist_file(&path, &state, false)?;
Self::from_state(path, state)
}
fn from_state(path: PathBuf, state: StoreState) -> Result<Self, HapError> {
validate_state(&state)?;
let (changes, _) = watch::channel(0);
Ok(Self {
path,
state: RwLock::new(state),
pair_setup_active: AtomicBool::new(false),
pair_setup_failures: AtomicU16::new(0),
changes,
})
}
pub fn path(&self) -> &Path {
&self.path
}
pub fn accessory_id(&self) -> Result<String, HapError> {
Ok(self.read_state()?.accessory.device_id.clone())
}
pub fn accessory_public_key(&self) -> Result<[u8; 32], HapError> {
Ok(self.signing_key()?.verifying_key().to_bytes())
}
pub fn list(&self) -> Result<Vec<ControllerPairing>, HapError> {
Ok(self.read_state()?.controllers.values().cloned().collect())
}
pub fn get(&self, controller_id: &str) -> Result<Option<ControllerPairing>, HapError> {
Ok(self.read_state()?.controllers.get(controller_id).cloned())
}
pub fn is_paired(&self) -> Result<bool, HapError> {
Ok(!self.read_state()?.controllers.is_empty())
}
/// Add the first administrator after a fully authenticated Pair-Setup M5.
pub(crate) fn add_initial(&self, pairing: ControllerPairing) -> Result<(), HapError> {
pairing.validate()?;
if !pairing.admin {
return Err(HapError::InvalidPairingRecord(
"the first controller must be an administrator".into(),
));
}
self.update(|state| {
if !state.controllers.is_empty() {
return Err(HapError::PairingAlreadyExists(
pairing.controller_id.clone(),
));
}
state
.controllers
.insert(pairing.controller_id.clone(), pairing);
Ok(())
})?;
self.pair_setup_failures.store(0, Ordering::Release);
Ok(())
}
/// Add or update a pairing according to HAP Add Pairing semantics.
pub(crate) fn upsert(&self, pairing: ControllerPairing) -> Result<(), HapError> {
pairing.validate()?;
self.update(|state| {
if let Some(existing) = state.controllers.get(&pairing.controller_id) {
if existing.public_key != pairing.public_key {
return Err(HapError::InvalidPairingRecord(
"existing pairing public key cannot be replaced".into(),
));
}
} else if state.controllers.len() >= MAX_CONTROLLERS {
return Err(HapError::PairingCapacity);
}
state
.controllers
.insert(pairing.controller_id.clone(), pairing);
Ok(())
})
}
/// Remove a pairing idempotently. Removing the final administrator clears
/// every pairing as required by HAP.
pub(crate) fn remove_hap(&self, controller_id: &str) -> Result<bool, HapError> {
let mut removed = false;
self.update(|state| {
removed = state.controllers.remove(controller_id).is_some();
if removed
&& !state.controllers.is_empty()
&& !state.controllers.values().any(|pairing| pairing.admin)
{
state.controllers.clear();
}
Ok(())
})?;
Ok(removed)
}
pub(crate) fn signing_key(&self) -> Result<SigningKey, HapError> {
Ok(SigningKey::from_bytes(
&self.read_state()?.accessory.signing_seed,
))
}
pub(crate) fn setup_record(&self) -> Result<([u8; 16], Vec<u8>), HapError> {
let state = self.read_state()?;
Ok((state.setup.salt, state.setup.verifier.clone()))
}
pub(crate) fn try_begin_pair_setup(&self) -> bool {
self.pair_setup_active
.compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
.is_ok()
}
pub(crate) fn end_pair_setup(&self) {
self.pair_setup_active.store(false, Ordering::Release);
}
pub(crate) fn pair_setup_locked_out(&self) -> bool {
self.pair_setup_failures.load(Ordering::Acquire) >= MAX_PAIR_SETUP_FAILURES
}
pub(crate) fn record_pair_setup_failure(&self) {
let _ = self.pair_setup_failures.fetch_update(
Ordering::AcqRel,
Ordering::Acquire,
|attempts| Some(attempts.saturating_add(1)),
);
}
pub(crate) fn subscribe_changes(&self) -> watch::Receiver<u64> {
self.changes.subscribe()
}
fn read_state(&self) -> Result<std::sync::RwLockReadGuard<'_, StoreState>, HapError> {
self.state
.read()
.map_err(|_| HapError::PairingStore("pairing store lock poisoned".into()))
}
fn update(
&self,
mutate: impl FnOnce(&mut StoreState) -> Result<(), HapError>,
) -> Result<(), HapError> {
let mut guard = self
.state
.write()
.map_err(|_| HapError::PairingStore("pairing store lock poisoned".into()))?;
let mut next = guard.clone();
mutate(&mut next)?;
validate_state(&next)?;
persist_file(&self.path, &next, true)?;
*guard = next;
self.changes.send_modify(|revision| *revision += 1);
Ok(())
}
}
fn new_state(setup_code: &SetupCode, device_id: Option<String>) -> Result<StoreState, HapError> {
let mut signing_seed = [0u8; 32];
let mut salt = [0u8; 16];
getrandom::getrandom(&mut signing_seed)
.and_then(|_| getrandom::getrandom(&mut salt))
.map_err(|error| HapError::PairingStore(format!("generate accessory identity: {error}")))?;
let device_id = match device_id {
Some(device_id) => device_id,
None => generate_device_id()?,
};
let verifier =
ClientG3072::<Sha512>::new().compute_verifier(SRP_USERNAME, setup_code.as_bytes(), &salt);
let state = StoreState {
accessory: StoredAccessory {
device_id,
signing_seed,
},
setup: StoredSetup { salt, verifier },
controllers: BTreeMap::new(),
};
validate_state(&state)?;
Ok(state)
}
fn generate_device_id() -> Result<String, HapError> {
let mut bytes = [0u8; 6];
getrandom::getrandom(&mut bytes)
.map_err(|error| HapError::PairingStore(format!("generate device ID: {error}")))?;
Ok(bytes
.iter()
.map(|byte| format!("{byte:02X}"))
.collect::<Vec<_>>()
.join(":"))
}
fn validate_device_id(device_id: &str) -> Result<(), HapError> {
let parts: Vec<&str> = device_id.split(':').collect();
if parts.len() != 6
|| parts
.iter()
.any(|part| part.len() != 2 || !part.bytes().all(|byte| byte.is_ascii_hexdigit()))
{
return Err(HapError::InvalidPairingRecord(
"accessory device ID must be six colon-separated hexadecimal octets".into(),
));
}
Ok(())
}
fn validate_state(state: &StoreState) -> Result<(), HapError> {
validate_device_id(&state.accessory.device_id)?;
SigningKey::from_bytes(&state.accessory.signing_seed);
if state.setup.verifier.len() != 384 {
return Err(HapError::InvalidPairingRecord(
"SRP verifier must contain exactly 384 bytes".into(),
));
}
if state.controllers.len() > MAX_CONTROLLERS {
return Err(HapError::InvalidPairingRecord(
"too many persisted controller pairings".into(),
));
}
for (id, pairing) in &state.controllers {
pairing.validate()?;
if id != &pairing.controller_id {
return Err(HapError::InvalidPairingRecord(
"controller map key does not match identifier".into(),
));
}
}
if !state.controllers.is_empty() && !state.controllers.values().any(|pairing| pairing.admin) {
return Err(HapError::InvalidPairingRecord(
"persisted pairings have no administrator".into(),
));
}
Ok(())
}
fn load_file(path: &Path) -> Result<StoreState, HapError> {
let metadata = fs::symlink_metadata(path)
.map_err(|error| HapError::PairingStore(format!("metadata {}: {error}", path.display())))?;
if metadata.file_type().is_symlink() || !metadata.is_file() {
return Err(HapError::PairingStore(format!(
"{} must be a regular, non-symlink file",
path.display()
)));
}
if metadata.len() > MAX_STORE_BYTES {
return Err(HapError::PairingStore(format!(
"{} exceeds the {MAX_STORE_BYTES}-byte limit",
path.display()
)));
}
validate_permissions(path, &metadata)?;
let mut bytes = Zeroizing::new(Vec::with_capacity(metadata.len() as usize));
File::open(path)
.and_then(|file| file.take(MAX_STORE_BYTES + 1).read_to_end(bytes.as_mut()))
.map_err(|error| HapError::PairingStore(format!("read {}: {error}", path.display())))?;
if bytes.len() as u64 > MAX_STORE_BYTES {
return Err(HapError::PairingStore(
"pairing store exceeds size limit".into(),
));
}
let file: PairingFile = serde_json::from_slice(bytes.as_slice())
.map_err(|error| HapError::PairingStore(format!("parse {}: {error}", path.display())))?;
if file.version != STORE_VERSION {
return Err(HapError::PairingStore(format!(
"unsupported pairing store version {}; v1 controller-only stores cannot be used because they lack accessory identity and setup material",
file.version
)));
}
let mut controllers = BTreeMap::new();
for pairing in file.controllers {
pairing.validate()?;
let id = pairing.controller_id.clone();
if controllers.insert(id.clone(), pairing).is_some() {
return Err(HapError::InvalidPairingRecord(format!(
"duplicate controller_id {id}"
)));
}
}
let state = StoreState {
accessory: file.accessory,
setup: file.setup,
controllers,
};
validate_state(&state)?;
Ok(state)
}
fn persist_file(path: &Path, state: &StoreState, overwrite: bool) -> Result<(), HapError> {
let parent = path
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or(Path::new("."));
create_private_dir(parent)?;
let payload = Zeroizing::new(
serde_json::to_vec_pretty(&PairingFile {
version: STORE_VERSION,
accessory: state.accessory.clone(),
setup: state.setup.clone(),
controllers: state.controllers.values().cloned().collect(),
})
.map_err(|error| HapError::PairingStore(format!("serialize pairings: {error}")))?,
);
let mut temp = Builder::new()
.prefix(".homecore-hap-security-")
.tempfile_in(parent)
.map_err(|error| HapError::PairingStore(format!("create temporary store: {error}")))?;
set_private_file_permissions(temp.as_file())?;
temp.write_all(&payload)
.and_then(|_| temp.flush())
.and_then(|_| temp.as_file().sync_all())
.map_err(|error| HapError::PairingStore(format!("write temporary store: {error}")))?;
if overwrite {
temp.persist(path).map_err(|error| {
HapError::PairingStore(format!("replace {}: {}", path.display(), error.error))
})?;
} else {
temp.persist_noclobber(path).map_err(|error| {
HapError::PairingStore(format!("create {}: {}", path.display(), error.error))
})?;
}
#[cfg(unix)]
File::open(parent)
.and_then(|directory| directory.sync_all())
.map_err(|error| HapError::PairingStore(format!("sync {}: {error}", parent.display())))?;
Ok(())
}
fn create_private_dir(path: &Path) -> Result<(), HapError> {
let created = !path.exists();
if created {
fs::create_dir_all(path).map_err(|error| {
HapError::PairingStore(format!("create {}: {error}", path.display()))
})?;
}
#[cfg(unix)]
if created {
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(path, fs::Permissions::from_mode(0o700)).map_err(|error| {
HapError::PairingStore(format!("chmod {}: {error}", path.display()))
})?;
}
Ok(())
}
fn set_private_file_permissions(_file: &File) -> Result<(), HapError> {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
_file
.set_permissions(fs::Permissions::from_mode(0o600))
.map_err(|error| HapError::PairingStore(format!("chmod temporary store: {error}")))?;
}
Ok(())
}
fn validate_permissions(path: &Path, metadata: &fs::Metadata) -> Result<(), HapError> {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
let mode = metadata.permissions().mode();
if mode & 0o077 != 0 {
return Err(HapError::InsecurePermissions {
path: path.to_path_buf(),
mode: mode & 0o777,
});
}
}
let _ = (path, metadata);
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
fn store(directory: &tempfile::TempDir) -> PairingStore {
PairingStore::create(
directory.path().join("pairings.json"),
SetupCode::parse("518-26-003").unwrap(),
Some("AA:BB:CC:DD:EE:FF".into()),
)
.unwrap()
}
fn pairing(id: &str, byte: u8, admin: bool) -> ControllerPairing {
ControllerPairing {
controller_id: id.into(),
public_key: SigningKey::from_bytes(&[byte; 32])
.verifying_key()
.to_bytes(),
admin,
}
}
#[test]
fn first_provisioning_returns_code_but_restart_does_not() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("pairings.json");
let provisioned = PairingStore::load_or_create(&path).unwrap();
let id = provisioned.store.accessory_id().unwrap();
assert!(provisioned.setup_code.is_some());
drop(provisioned);
let reopened = PairingStore::load_or_create(path).unwrap();
assert!(reopened.setup_code.is_none());
assert_eq!(reopened.store.accessory_id().unwrap(), id);
}
#[test]
fn identity_and_pairings_survive_atomic_restart() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("pairings.json");
let store = store(&directory);
let public_key = store.accessory_public_key().unwrap();
store.add_initial(pairing("controller-1", 7, true)).unwrap();
drop(store);
let reopened = PairingStore::open(path).unwrap();
assert_eq!(reopened.accessory_public_key().unwrap(), public_key);
assert_eq!(
reopened.list().unwrap(),
vec![pairing("controller-1", 7, true)]
);
}
#[test]
fn prohibited_setup_codes_are_rejected_and_debug_is_redacted() {
assert!(SetupCode::parse("111-11-111").is_err());
assert!(SetupCode::parse("123-45-678").is_err());
let code = SetupCode::parse("518-26-003").unwrap();
assert_eq!(format!("{code:?}"), "SetupCode([REDACTED])");
}
/// A stray `format!("{store:?}")` (a future debug log line, a panic
/// message) must never print the accessory's permanent Ed25519 signing
/// seed or the SRP salt/verifier — both are compromise-forever secrets
/// with no rotation mechanism.
#[test]
fn stored_accessory_and_setup_debug_never_print_secret_material() {
let accessory = StoredAccessory {
device_id: "AA:BB:CC:DD:EE:FF".into(),
signing_seed: [0x42; 32],
};
let rendered = format!("{accessory:?}");
assert!(rendered.contains("device_id"));
assert!(rendered.contains("AA:BB:CC:DD:EE:FF"));
assert!(!rendered.contains("66"), "hex of 0x42 must not leak: {rendered}");
assert_eq!(
rendered,
"StoredAccessory { device_id: \"AA:BB:CC:DD:EE:FF\", signing_seed: \"[REDACTED]\" }"
);
let setup = StoredSetup { salt: [0x7a; 16], verifier: vec![0x13; 8] };
assert_eq!(format!("{setup:?}"), "StoredSetup([REDACTED])");
// The redaction must propagate through every derived-Debug container
// that embeds these structs, with no further code changes needed.
let state = StoreState {
accessory,
setup,
controllers: std::collections::BTreeMap::new(),
};
let rendered_state = format!("{state:?}");
assert!(!rendered_state.contains("0x42"));
assert!(rendered_state.contains("[REDACTED]"));
}
#[test]
fn removing_last_admin_clears_all_pairings() {
let directory = tempfile::tempdir().unwrap();
let store = store(&directory);
store.add_initial(pairing("admin", 1, true)).unwrap();
store.upsert(pairing("member", 2, false)).unwrap();
assert!(store.remove_hap("admin").unwrap());
assert!(store.list().unwrap().is_empty());
}
#[test]
fn malformed_or_legacy_records_fail_closed() {
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("pairings.json");
fs::write(&path, br#"{"version":1,"controllers":[]}"#).unwrap();
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(&path, fs::Permissions::from_mode(0o600)).unwrap();
}
assert!(PairingStore::open(path).is_err());
}
#[cfg(unix)]
#[test]
fn permissive_existing_file_is_rejected() {
use std::os::unix::fs::PermissionsExt;
let directory = tempfile::tempdir().unwrap();
let path = directory.path().join("pairings.json");
let _ = store(&directory);
fs::set_permissions(&path, fs::Permissions::from_mode(0o644)).unwrap();
assert!(matches!(
PairingStore::open(path),
Err(HapError::InsecurePermissions { .. })
));
}
}
+141
View File
@@ -0,0 +1,141 @@
//! Bounded HAP TLV8 primitives.
use std::collections::BTreeMap;
use crate::error::HapError;
pub const TLV_METHOD: u8 = 0x00;
pub const TLV_IDENTIFIER: u8 = 0x01;
pub const TLV_SALT: u8 = 0x02;
pub const TLV_PUBLIC_KEY: u8 = 0x03;
pub const TLV_PROOF: u8 = 0x04;
pub const TLV_ENCRYPTED_DATA: u8 = 0x05;
pub const TLV_STATE: u8 = 0x06;
pub const TLV_ERROR: u8 = 0x07;
pub const TLV_SIGNATURE: u8 = 0x0a;
pub const TLV_PERMISSIONS: u8 = 0x0b;
pub const TLV_FLAGS: u8 = 0x13;
pub const TLV_SEPARATOR: u8 = 0xff;
pub const TLV_ERROR_UNKNOWN: u8 = 0x01;
pub const TLV_ERROR_AUTHENTICATION: u8 = 0x02;
pub const TLV_ERROR_MAX_PEERS: u8 = 0x04;
pub const TLV_ERROR_MAX_TRIES: u8 = 0x05;
pub const TLV_ERROR_UNAVAILABLE: u8 = 0x06;
pub const TLV_ERROR_BUSY: u8 = 0x07;
const MAX_TLV_BYTES: usize = 4096;
const MAX_TLV_TYPES: usize = 32;
/// Parsed TLV8 values. Repeated fragments of a type are concatenated.
#[derive(Debug, Clone, Default, PartialEq, Eq)]
pub struct Tlv8 {
values: BTreeMap<u8, Vec<u8>>,
}
impl Tlv8 {
pub fn parse(input: &[u8]) -> Result<Self, HapError> {
if input.len() > MAX_TLV_BYTES {
return Err(HapError::Protocol("TLV8 body exceeds 4096 bytes".into()));
}
let mut values: BTreeMap<u8, Vec<u8>> = BTreeMap::new();
let mut offset = 0usize;
while offset < input.len() {
if input.len() - offset < 2 {
return Err(HapError::Protocol("truncated TLV8 header".into()));
}
let kind = input[offset];
let len = input[offset + 1] as usize;
offset += 2;
let end = offset
.checked_add(len)
.filter(|end| *end <= input.len())
.ok_or_else(|| HapError::Protocol("truncated TLV8 value".into()))?;
if !values.contains_key(&kind) && values.len() == MAX_TLV_TYPES {
return Err(HapError::Protocol("too many TLV8 types".into()));
}
values
.entry(kind)
.or_default()
.extend_from_slice(&input[offset..end]);
offset = end;
}
Ok(Self { values })
}
pub fn get(&self, kind: u8) -> Option<&[u8]> {
self.values.get(&kind).map(Vec::as_slice)
}
pub fn byte(&self, kind: u8) -> Option<u8> {
let value = self.get(kind)?;
(value.len() == 1).then_some(value[0])
}
pub fn insert(&mut self, kind: u8, value: impl Into<Vec<u8>>) {
self.values.insert(kind, value.into());
}
pub fn encode(&self) -> Vec<u8> {
encode_items(
self.values
.iter()
.map(|(&kind, value)| (kind, value.as_slice())),
)
}
}
/// Encode ordered TLV8 items, including repeated items separated by a
/// zero-length `Separator` for `/pairings` list responses.
pub fn encode_items<'a>(items: impl IntoIterator<Item = (u8, &'a [u8])>) -> Vec<u8> {
let mut encoded = Vec::new();
for (kind, value) in items {
if value.is_empty() {
encoded.extend_from_slice(&[kind, 0]);
continue;
}
for chunk in value.chunks(u8::MAX as usize) {
encoded.push(kind);
encoded.push(chunk.len() as u8);
encoded.extend_from_slice(chunk);
}
}
encoded
}
pub fn error_response(state: u8, error: u8) -> Vec<u8> {
encode_items([
(TLV_STATE, [state].as_slice()),
(TLV_ERROR, [error].as_slice()),
])
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tlv8_roundtrip_supports_fragmented_values() {
let mut tlv = Tlv8::default();
tlv.insert(TLV_PUBLIC_KEY, vec![3; 300]);
tlv.insert(TLV_STATE, vec![1]);
let encoded = tlv.encode();
let decoded = Tlv8::parse(&encoded).unwrap();
assert_eq!(decoded, tlv);
}
#[test]
fn malformed_or_oversized_tlv_is_rejected() {
assert!(Tlv8::parse(&[TLV_STATE]).is_err());
assert!(Tlv8::parse(&[TLV_STATE, 2, 1]).is_err());
assert!(Tlv8::parse(&vec![0; MAX_TLV_BYTES + 1]).is_err());
}
#[test]
fn error_response_is_not_success_shaped() {
let response = error_response(2, TLV_ERROR_UNAVAILABLE);
let decoded = Tlv8::parse(&response).unwrap();
assert_eq!(decoded.byte(TLV_STATE), Some(2));
assert_eq!(decoded.byte(TLV_ERROR), Some(TLV_ERROR_UNAVAILABLE));
}
}
File diff suppressed because it is too large Load Diff
+171
View File
@@ -0,0 +1,171 @@
//! Per-connection HAP authentication state.
#![cfg_attr(not(feature = "hap-server"), allow(dead_code))]
use crate::crypto::SessionKeys;
use crate::error::HapError;
/// Authentication phase of one TCP connection.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum SessionState {
Connected,
PairSetup,
PairVerify,
Authenticated { controller_id: String, admin: bool },
Closing,
}
impl SessionState {
fn name(&self) -> &'static str {
match self {
Self::Connected => "connected",
Self::PairSetup => "pair-setup",
Self::PairVerify => "pair-verify",
Self::Authenticated { .. } => "authenticated",
Self::Closing => "closing",
}
}
pub fn is_authenticated(&self) -> bool {
matches!(self, Self::Authenticated { .. })
}
}
/// Fail-closed state machine for one HAP connection.
pub struct Session {
state: SessionState,
pending_keys: Option<SessionKeys>,
}
impl Default for Session {
fn default() -> Self {
Self::new()
}
}
impl Session {
pub fn new() -> Self {
Self {
state: SessionState::Connected,
pending_keys: None,
}
}
pub fn state(&self) -> &SessionState {
&self.state
}
pub fn begin_pair_setup(&mut self) -> Result<(), HapError> {
self.transition(SessionState::PairSetup)
}
pub fn begin_pair_verify(&mut self) -> Result<(), HapError> {
self.transition(SessionState::PairVerify)
}
pub(crate) fn authenticate(
&mut self,
controller_id: String,
admin: bool,
keys: SessionKeys,
) -> Result<(), HapError> {
if !matches!(self.state, SessionState::PairVerify) {
return Err(HapError::InvalidSessionTransition {
from: self.state.name(),
to: "authenticated",
});
}
self.state = SessionState::Authenticated {
controller_id,
admin,
};
self.pending_keys = Some(keys);
Ok(())
}
pub(crate) fn take_session_keys(&mut self) -> Option<SessionKeys> {
self.pending_keys.take()
}
pub fn reset_pairing(&mut self) -> Result<(), HapError> {
match self.state {
SessionState::PairSetup | SessionState::PairVerify => {
self.state = SessionState::Connected;
self.pending_keys = None;
Ok(())
}
_ => Err(HapError::InvalidSessionTransition {
from: self.state.name(),
to: "connected",
}),
}
}
pub fn close(&mut self) {
self.pending_keys = None;
self.state = SessionState::Closing;
}
pub(crate) fn controller_id(&self) -> Option<&str> {
match &self.state {
SessionState::Authenticated { controller_id, .. } => Some(controller_id),
_ => None,
}
}
#[cfg(all(test, feature = "hap-server"))]
pub(crate) fn authenticated_for_test(admin: bool) -> Self {
Self {
state: SessionState::Authenticated {
controller_id: "test-controller".into(),
admin,
},
pending_keys: None,
}
}
fn transition(&mut self, next: SessionState) -> Result<(), HapError> {
if matches!(self.state, SessionState::Connected) {
self.state = next;
Ok(())
} else {
Err(HapError::InvalidSessionTransition {
from: self.state.name(),
to: next.name(),
})
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn authentication_requires_pair_verify_and_yields_keys_once() {
let mut session = Session::new();
let keys = SessionKeys::derive(&[7; 32]).unwrap();
assert!(session
.authenticate("controller".into(), true, keys)
.is_err());
session.begin_pair_verify().unwrap();
session
.authenticate(
"controller".into(),
true,
SessionKeys::derive(&[7; 32]).unwrap(),
)
.unwrap();
assert!(session.state().is_authenticated());
assert!(session.take_session_keys().is_some());
assert!(session.take_session_keys().is_none());
}
#[test]
fn pairing_phases_cannot_overlap() {
let mut session = Session::new();
session.begin_pair_setup().unwrap();
assert!(session.begin_pair_verify().is_err());
session.reset_pairing().unwrap();
session.begin_pair_verify().unwrap();
}
}
+60 -126
View File
@@ -1,143 +1,77 @@
# homecore-migrate
Migration tooling for importing Home Assistant configuration, entities, and secrets into HOMECORE.
Migration tooling for importing Home Assistant filesystem storage into
HOMECORE. The implementation follows
[ADR-165](../../docs/adr/ADR-165-homecore-migrate-from-home-assistant.md).
[![Crates.io](https://img.shields.io/crates/v/homecore-migrate.svg)](https://crates.io/crates/homecore-migrate)
![License](https://img.shields.io/badge/license-MIT-blue.svg)
![MSRV: 1.89+](https://img.shields.io/badge/MSRV-1.89%2B-purple.svg)
[![Tests](https://img.shields.io/badge/tests-19%20passing-brightgreen.svg)](https://github.com/ruvnet/RuView)
[![ADR-165](https://img.shields.io/badge/ADR-165-orange.svg)](../../docs/adr/ADR-165-homecore-migrate-from-home-assistant.md)
## Implemented
Parse and inspect Home Assistant's `.storage/` directory, entity registry, device registry, secrets, and automations. Convert existing HA configurations for import into HOMECORE (full conversion in P2).
- Reads versioned HA `.storage` JSON and rejects unknown schema versions.
- Converts entity registry metadata to `homecore::EntityEntry`.
- Converts the supported HA v13 device-registry fields to
`homecore::DeviceEntry`, including identifiers, connections, versions,
serial number, labels, topology, and config-entry links.
- Converts `core.config_entries` to a versioned
`homecore.config_entries` file. Each original row is retained verbatim.
Unsupported domains and non-portable fields produce typed warnings.
- Publishes destination JSON through a synced same-directory temporary file
and an atomic no-clobber link. Existing destination files are never replaced.
- Emits one-line JSON summaries from every import command.
- Parses secrets with redacted errors and inspects automations.
## What this crate does
## CLI
`homecore-migrate` reads Home Assistant's filesystem state and provides tooling to analyze and migrate it to HOMECORE. It includes:
- **HaStorageDir** — reads HA's `.homeassistant/.storage/` directory and parses versioned JSON envelopes
- **Entity registry parser** — converts `core.entity_registry` JSON to HOMECORE `EntityEntry` types
- **Device registry parser** — reads `core.device_registry` (P1 diagnostic only; full conversion in P2)
- **Config entries parser** — reads `core.config_entries` to list active integrations
- **Secrets parser** — reads `secrets.yaml` as `HashMap<String, String>` for reference resolution (P2)
- **Automations parser** — reads `automations.yaml` and counts/lists automations (full conversion in P2)
- **CLI binary** — `homecore-migrate inspect` to preview what will be migrated
The tool enforces version schema compatibility: unknown HA schema versions are rejected (hard error per ADR-165 §6 Q5) rather than silently corrupting data.
## Features
- **Entity registry import** — `core.entity_registry` → HOMECORE entity definitions (ready for import)
- **Device registry inspection** — read HA device metadata; full conversion deferred to P2
- **Config entries analysis** — list active integrations by domain (enables gap analysis)
- **Secrets extraction** — read `secrets.yaml` references for annotation (resolution in P2)
- **Automations counting** — list automation IDs and aliases without conversion (conversion in P2)
- **Schema version validation** — explicit rejection of unknown HA versions (no silent corruption)
- **Structured error reporting** — `MigrateError` enum with context (file path, line number)
- **CLI subcommands** — `inspect` to preview, `import-entities` to load (P2), `export-for-sidecar` (P2)
## Capabilities
| Capability | Type | Method | Notes |
|------------|------|--------|-------|
| Read storage envelope | Parser | `storage::read_envelope(path)` | Deserialize `.storage/*.json` |
| Parse entity registry | Parser | `entity_registry::load(storage_dir)` | → `Vec<homecore::EntityEntry>` |
| Inspect device registry | Parser | `device_registry::load(storage_dir)` | → `Vec<DeviceImport>` (P1 diagnostic) |
| List config entries | Parser | `config_entries::load(storage_dir)` | → domain counts + names |
| Load secrets | Parser | `secrets::load_secrets(path)` | → `HashMap<String, String>` |
| Count automations | Parser | `automations::load(path)` | → count + ID list |
| Validate schema version | Validator | `storage_format::validate_version(major, minor)` | Hard error if unknown |
| Convert to HOMECORE | Converter | `entity_registry::to_homecore_entries()` (P2) | → `homecore::EntityRegistry` |
| Export side-by-side DB | Exporter | `recorder::export_states()` (P2, `--features recorder`) | → `.homecore/home.db` |
## Comparison to Home Assistant
| Aspect | Home Assistant | homecore-migrate |
|--------|----------------|-----------------|
| State source | Python `.homeassistant/` directory | Same HA filesystem format |
| Entity registry format | JSON envelope in `.storage/core.entity_registry` | Identical format, schema v13 |
| Schema versioning | `version` + optional `minor_version` | Explicit version struct validation |
| Secrets resolution | `!secret` YAML references via loader | Planned P2 (reads `secrets.yaml`) |
| Automation conversion | Python → HA YAML (internal) | P2: convert to `homecore-automation` format |
| Device registry import | Python device types | P1 diagnostic; full conversion P2 |
| Side-by-side runtime | N/A (HA doesn't side-by-side migrate) | P2 feature: run old + new in parallel |
| CLI tooling | HA doesn't export | `homecore-migrate` binary with subcommands |
## Performance
- **Storage envelope parse** — < 5 ms per file (serde_json)
- **Entity registry load** — < 50 ms for 1,000 entities
- **Storage directory scan** — < 100 ms for full `.storage/` directory
- **Secrets file parse** — < 10 ms (YAML)
- **No per-crate benchmarks yet** — a follow-up issue tracks baseline measurements
## Usage
CLI inspection (P1):
The import commands take the HA `.storage` directory and the HOMECORE storage
destination:
```bash
# Inspect what will be migrated from an existing HA installation
homecore-migrate inspect ~/.homeassistant
homecore-migrate import-entities \
--storage ~/.homeassistant/.storage \
--to ~/.homecore/storage
# Output:
# Entity Registry: 47 entities
# light: 12
# sensor: 20
# binary_sensor: 10
# switch: 5
# Device Registry: 8 devices
# Config Entries: 6 integrations (mqtt, rest, zeroconf, ...)
# Secrets: 3 defined (redacted)
# Automations: 5 automations (redacted)
homecore-migrate import-devices \
--storage ~/.homeassistant/.storage \
--to ~/.homecore/storage
homecore-migrate import-config-entries \
--storage ~/.homeassistant/.storage \
--to ~/.homecore/storage
```
Programmatic entity import (P1):
Successful imports print a machine-readable JSON object:
```rust
use homecore_migrate::entity_registry;
use homecore::HomeCore;
#[tokio::main]
async fn main() {
let storage_dir = std::path::Path::new("/home/user/.homeassistant/.storage");
// Load HA entities
let entries = entity_registry::load(storage_dir)
.expect("load entity registry");
println!("Loaded {} entities", entries.len());
// Import into HOMECORE (P2 when EntityRegistry::import() lands)
let homecore = HomeCore::new();
for entry in entries {
println!("Entity: {} ({})", entry.entity_id, entry.name);
}
}
```json
{"kind":"device_registry","imported":8,"warning_count":0,"warnings":[],"destination":"/home/user/.homecore/storage/core.device_registry"}
```
Full migration (P2 onwards, via `--features recorder`):
`inspect`, `inspect-config-entries`, `inspect-secrets`, and
`inspect-automations` are read-only.
## Destination files
| Source | Destination | Format |
|---|---|---|
| `core.entity_registry` | `core.entity_registry` | HA-compatible v1/minor 13 envelope |
| `core.device_registry` | `core.device_registry` | HA-compatible v1/minor 13 envelope |
| `core.config_entries` | `homecore.config_entries` | HOMECORE v1/minor 0 envelope |
Config entries are storage-compatible, not runtime-compatible: importing an
entry does not install or execute its HA integration. A HOMECORE plugin must
explicitly claim the domain and consume the preserved source payload.
## Remaining limitations
- Automation conversion is not implemented; the tool only inspects
`automations.yaml`.
- `!secret` reference resolution in other YAML files is not implemented.
- Deleted entity/device tombstones are not imported.
- Device fields newer than HA registry minor version 13 require an explicit
parser update; unknown versions fail closed.
- No side-by-side HA recorder database exporter is provided.
## Validation
```bash
# Side-by-side: old HA continues running while HOMECORE reads the DB
homecore-migrate export-for-sidecar \
--ha-dir ~/.homeassistant \
--homecore-db ~/.homecore/home.db \
--keep-automations true # Don't stop HA automations during test period
cargo test -p homecore-migrate
cargo clippy -p homecore-migrate --all-targets -- -D warnings
```
## Relation to other HOMECORE crates
```
homecore-migrate (import from HA)
├─ homecore (EntityEntry → EntityRegistry; config entry imports)
├─ homecore-automation (automations.yaml → automation rules, P2)
├─ homecore-recorder (side-by-side state export, P2, `--features recorder`)
├─ homecore-plugins (config_entries → plugin manifests, P2)
└─ homecore-server (can auto-import at startup with --import-ha flag, P2)
```
## References
- [ADR-165: HOMECORE Migration from Python Home Assistant](../../docs/adr/ADR-165-homecore-migrate-from-home-assistant.md)
- [ADR-126: HOMECORE Home Assistant Port (master)](../../docs/adr/ADR-126-homecore-home-assistant-port.md)
- [Home Assistant .storage/ format](https://developers.home-assistant.io/docs/storage/)
- [homecore-migrate CLI source](src/main.rs)
- [README — wifi-densepose](../../../README.md)
+32 -2
View File
@@ -21,10 +21,12 @@ pub enum Command {
Inspect(InspectArgs),
/// Import entity registry from HA into a HOMECORE storage directory.
ImportEntities(ImportEntitiesArgs),
/// Import device registry (P1: parses and reports; wiring to HOMECORE P2).
/// Import the device registry into HOMECORE storage.
ImportDevices(ImportDevicesArgs),
/// Inspect config entries (P1: count + domain list; conversion is P2).
/// Inspect config entries without writing.
InspectConfigEntries(InspectConfigEntriesArgs),
/// Import config entries losslessly into versioned HOMECORE storage.
ImportConfigEntries(ImportConfigEntriesArgs),
/// Parse secrets.yaml and report secret names (values redacted).
InspectSecrets(InspectSecretsArgs),
/// Count and list automations from automations.yaml (conversion is P2).
@@ -46,6 +48,11 @@ pub struct ImportEntitiesArgs {
/// Path to the HOMECORE storage directory (destination).
#[arg(long)]
pub to: PathBuf,
/// Overwrite an existing destination file instead of refusing. Use this
/// to re-run an import after fixing a bad source row, or to re-import
/// after further changes on the HA side.
#[arg(long)]
pub force: bool,
}
#[derive(Debug, clap::Args)]
@@ -53,6 +60,29 @@ pub struct ImportDevicesArgs {
/// Path to the HA `.storage/` directory.
#[arg(long)]
pub storage: PathBuf,
/// Path to the HOMECORE storage directory (destination).
#[arg(long)]
pub to: PathBuf,
/// Overwrite an existing destination file instead of refusing. Use this
/// to re-run an import after fixing a bad source row, or to re-import
/// after further changes on the HA side.
#[arg(long)]
pub force: bool,
}
#[derive(Debug, clap::Args)]
pub struct ImportConfigEntriesArgs {
/// Path to the HA `.storage/` directory.
#[arg(long)]
pub storage: PathBuf,
/// Path to the HOMECORE storage directory (destination).
#[arg(long)]
pub to: PathBuf,
/// Overwrite an existing destination file instead of refusing. Use this
/// to re-run an import after fixing a bad source row, or to re-import
/// after further changes on the HA side.
#[arg(long)]
pub force: bool,
}
#[derive(Debug, clap::Args)]
+278 -85
View File
@@ -1,84 +1,247 @@
//! Parser for `core.config_entries` (HA storage schema v1, minor_version varies).
//! Lossless conversion of HA `core.config_entries` into HOMECORE storage.
//!
//! Per ADR-165 §6 Q5, `.storage/core.config_entries` format is undocumented
//! and version-gated. P1 reads the envelope and emits:
//! - count of config entries
//! - list of integration domains represented
//!
//! Conversion to HOMECORE plugin manifests is P2.
//!
//! Note: `config_entries` uses a different `minor_version` track from
//! `entity_registry`. As of HA 2025.1 it is typically minor_version=1 or 2.
//! We accept any minor_version ≤ MAX_SUPPORTED_MINOR and hard-error above it.
//! HOMECORE cannot yet execute arbitrary HA integrations. Every source row is
//! therefore retained verbatim while portable identity/config fields are
//! projected for future plugin setup. Typed warnings make the unsupported
//! surface machine-readable instead of silently dropping it.
use std::path::Path;
use std::collections::{BTreeMap, BTreeSet};
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use crate::{storage::read_envelope, MigrateError};
use crate::{
storage::{read_envelope, write_json_atomic, HaStorageEnvelope},
MigrateError,
};
/// Maximum `minor_version` we claim to understand for config_entries.
const SOURCE_KEY: &str = "core.config_entries";
pub const DESTINATION_KEY: &str = "homecore.config_entries";
const MAX_SUPPORTED_MINOR: u32 = 4;
const HOMECORE_CONFIG_VERSION: u32 = 1;
/// Diagnostic summary produced by P1 inspection.
#[derive(Clone, Debug, Serialize)]
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct HomeCoreConfigEntry {
pub entry_id: String,
pub domain: String,
pub title: String,
#[serde(default)]
pub data: serde_json::Value,
/// Exact HA row, including unsupported and future fields.
pub source: serde_json::Value,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq, Eq)]
#[serde(tag = "code", rename_all = "snake_case")]
pub enum MigrationWarning {
UnsupportedDomain { entry_id: String, domain: String },
UnsupportedField { entry_id: String, field: String },
UnsupportedRootField { field: String },
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct HomeCoreConfigData {
pub source_version: u32,
pub source_minor_version: u32,
/// Exact non-entry fields from the HA `data` object.
#[serde(default)]
pub source_extra: BTreeMap<String, serde_json::Value>,
pub entries: Vec<HomeCoreConfigEntry>,
pub warnings: Vec<MigrationWarning>,
}
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct HomeCoreConfigEnvelope {
pub version: u32,
pub minor_version: u32,
pub key: String,
pub data: HomeCoreConfigData,
}
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct ConfigEntriesSummary {
pub count: usize,
pub domains: Vec<String>,
pub warning_count: usize,
pub destination: Option<PathBuf>,
}
/// Minimal fields we read from each config-entry row.
#[derive(Debug, Deserialize)]
struct HaConfigEntryRow {
domain: String,
#[allow(dead_code)]
entry_id: String,
/// Title shown in HA UI (informational only in P1).
#[serde(default)]
#[allow(dead_code)]
title: Option<String>,
/// Source of the entry: "user" | "discovery" | "import" etc.
#[serde(default)]
#[allow(dead_code)]
source: Option<String>,
/// State: "loaded" | "setup_error" etc.
#[serde(default)]
#[allow(dead_code)]
state: Option<String>,
}
#[derive(Debug, Deserialize)]
struct HaConfigEntriesData {
entries: Vec<HaConfigEntryRow>,
}
/// Read `core.config_entries` from `path` and return a diagnostic summary.
pub fn inspect_config_entries(path: &Path) -> Result<ConfigEntriesSummary, MigrateError> {
let env = read_envelope(path)?;
let file_str = path.display().to_string();
// config_entries has version=1 and minor_version in 1..MAX_SUPPORTED_MINOR.
fn validate_source(env: &HaStorageEnvelope, path: &Path) -> Result<(), MigrateError> {
if env.version != 1 || env.minor_version > MAX_SUPPORTED_MINOR {
return Err(MigrateError::UnsupportedSchemaVersion {
file: file_str.clone(),
file: path.display().to_string(),
version: env.version,
minor_version: env.minor_version,
});
}
if env.key != SOURCE_KEY {
return Err(MigrateError::UnexpectedStorageKey {
path: path.display().to_string(),
expected: SOURCE_KEY.to_owned(),
actual: env.key.clone(),
});
}
Ok(())
}
let data: HaConfigEntriesData =
serde_json::from_value(env.data).map_err(|e| MigrateError::JsonParse {
path: file_str,
source: e,
pub fn convert_config_entries(path: &Path) -> Result<HomeCoreConfigEnvelope, MigrateError> {
let env = read_envelope(path)?;
validate_source(&env, path)?;
let entries = env
.data
.get("entries")
.and_then(serde_json::Value::as_array)
.ok_or_else(|| MigrateError::MissingField {
field: "entries".to_owned(),
context: path.display().to_string(),
})?;
let source_extra: BTreeMap<String, serde_json::Value> = env
.data
.as_object()
.into_iter()
.flat_map(|object| object.iter())
.filter(|(field, _)| field.as_str() != "entries")
.map(|(field, value)| (field.clone(), value.clone()))
.collect();
let mut domains: Vec<String> = data.entries.iter().map(|e| e.domain.clone()).collect();
domains.sort();
domains.dedup();
let portable = BTreeSet::from(["entry_id", "domain", "title", "data"]);
let mut converted = Vec::with_capacity(entries.len());
let mut warnings = source_extra
.keys()
.map(|field| MigrationWarning::UnsupportedRootField {
field: field.clone(),
})
.collect::<Vec<_>>();
for (index, source) in entries.iter().enumerate() {
let object = source
.as_object()
.ok_or_else(|| MigrateError::MissingField {
field: "object row".to_owned(),
context: format!("{} data.entries[{index}]", path.display()),
})?;
let string_field = |field: &str| -> Result<String, MigrateError> {
object
.get(field)
.and_then(serde_json::Value::as_str)
.map(str::to_owned)
.ok_or_else(|| MigrateError::MissingField {
field: field.to_owned(),
context: format!("{} data.entries[{index}]", path.display()),
})
};
let entry_id = string_field("entry_id")?;
let domain = string_field("domain")?;
let title = object
.get("title")
.and_then(serde_json::Value::as_str)
.unwrap_or(&domain)
.to_owned();
let data = object
.get("data")
.cloned()
.unwrap_or_else(|| serde_json::json!({}));
// No HA domain is implicitly claimed executable by HOMECORE. The
// source is retained so a matching plugin can consume it later.
warnings.push(MigrationWarning::UnsupportedDomain {
entry_id: entry_id.clone(),
domain: domain.clone(),
});
for field in object
.keys()
.filter(|field| !portable.contains(field.as_str()))
{
warnings.push(MigrationWarning::UnsupportedField {
entry_id: entry_id.clone(),
field: field.clone(),
});
}
converted.push(HomeCoreConfigEntry {
entry_id,
domain,
title,
data,
source: source.clone(),
});
}
warnings.sort_by_key(|warning| serde_json::to_string(warning).unwrap_or_default());
Ok(HomeCoreConfigEnvelope {
version: HOMECORE_CONFIG_VERSION,
minor_version: 0,
key: DESTINATION_KEY.to_owned(),
data: HomeCoreConfigData {
source_version: env.version,
source_minor_version: env.minor_version,
source_extra,
entries: converted,
warnings,
},
})
}
pub fn write_config_entries(
storage_dir: &Path,
envelope: &HomeCoreConfigEnvelope,
) -> Result<PathBuf, MigrateError> {
write_config_entries_with(storage_dir, envelope, false)
}
/// As [`write_config_entries`], but `force = true` atomically replaces an
/// existing destination instead of refusing — the escape hatch for
/// re-running an import after fixing a bad source row.
pub fn write_config_entries_with(
storage_dir: &Path,
envelope: &HomeCoreConfigEnvelope,
force: bool,
) -> Result<PathBuf, MigrateError> {
let target = storage_dir.join(DESTINATION_KEY);
write_json_atomic(&target, envelope, force)
}
pub fn read_homecore_config_entries(path: &Path) -> Result<HomeCoreConfigEnvelope, MigrateError> {
let raw = std::fs::read_to_string(path).map_err(|source| MigrateError::Io {
path: path.display().to_string(),
source,
})?;
let envelope: HomeCoreConfigEnvelope =
serde_json::from_str(&raw).map_err(|source| MigrateError::JsonParse {
path: path.display().to_string(),
source,
})?;
if envelope.version != HOMECORE_CONFIG_VERSION || envelope.minor_version != 0 {
return Err(MigrateError::UnsupportedSchemaVersion {
file: path.display().to_string(),
version: envelope.version,
minor_version: envelope.minor_version,
});
}
if envelope.key != DESTINATION_KEY {
return Err(MigrateError::UnexpectedStorageKey {
path: path.display().to_string(),
expected: DESTINATION_KEY.to_owned(),
actual: envelope.key,
});
}
Ok(envelope)
}
pub fn inspect_config_entries(path: &Path) -> Result<ConfigEntriesSummary, MigrateError> {
let converted = convert_config_entries(path)?;
let domains: BTreeMap<&str, usize> =
converted
.data
.entries
.iter()
.fold(BTreeMap::new(), |mut map, entry| {
*map.entry(entry.domain.as_str()).or_default() += 1;
map
});
Ok(ConfigEntriesSummary {
count: data.entries.len(),
domains,
count: converted.data.entries.len(),
domains: domains.keys().map(|value| (*value).to_owned()).collect(),
warning_count: converted.data.warnings.len(),
destination: None,
})
}
@@ -89,40 +252,70 @@ mod tests {
use tempfile::NamedTempFile;
const FIXTURE: &str = r#"{
"version": 1,
"minor_version": 1,
"key": "core.config_entries",
"data": {
"entries": [
{"domain": "hue", "entry_id": "ce_001", "title": "Philips Hue", "source": "user", "state": "loaded"},
{"domain": "zha", "entry_id": "ce_002", "title": "ZHA", "source": "user", "state": "loaded"},
{"domain": "hue", "entry_id": "ce_003", "title": "Hue 2", "source": "user", "state": "setup_error"}
]
}
"version":1,"minor_version":2,"key":"core.config_entries",
"data":{"future_root":{"kept":true},"entries":[{
"domain":"future_hub","entry_id":"ce_001","title":"Future Hub",
"source":"user","state":"loaded","data":{"host":"10.0.0.2"},
"options":{"scan":true},"future_field":{"nested":[1,2,3]}
}]}
}"#;
#[test]
fn inspect_emits_count_and_domains() {
let mut f = NamedTempFile::new().unwrap();
f.write_all(FIXTURE.as_bytes()).unwrap();
let summary = inspect_config_entries(f.path()).unwrap();
assert_eq!(summary.count, 3);
assert_eq!(summary.domains, vec!["hue", "zha"]);
fn fixture() -> NamedTempFile {
let mut file = NamedTempFile::new().unwrap();
file.write_all(FIXTURE.as_bytes()).unwrap();
file
}
#[test]
fn unknown_minor_version_hard_errors() {
let json = r#"{
"version": 1, "minor_version": 99,
"key": "core.config_entries",
"data": {"entries": []}
}"#;
let mut f = NamedTempFile::new().unwrap();
f.write_all(json.as_bytes()).unwrap();
let err = inspect_config_entries(f.path()).unwrap_err();
fn unknown_domain_and_fields_are_lossless_with_typed_warnings() {
let source = fixture();
let converted = convert_config_entries(source.path()).unwrap();
assert_eq!(
converted.data.entries[0].source["future_field"]["nested"][2],
3
);
assert_eq!(converted.data.source_extra["future_root"]["kept"], true);
assert!(converted.data.warnings.iter().any(|warning| matches!(
warning,
MigrationWarning::UnsupportedDomain { domain, .. } if domain == "future_hub"
)));
assert!(converted.data.warnings.iter().any(|warning| matches!(
warning,
MigrationWarning::UnsupportedField { field, .. } if field == "future_field"
)));
let dir = tempfile::tempdir().unwrap();
let path = write_config_entries(dir.path(), &converted).unwrap();
let restored = read_homecore_config_entries(&path).unwrap();
assert_eq!(restored, converted);
}
#[test]
fn unknown_destination_version_is_rejected() {
let dir = tempfile::tempdir().unwrap();
let path = dir.path().join(DESTINATION_KEY);
std::fs::write(
&path,
r#"{"version":99,"minor_version":0,"key":"homecore.config_entries","data":{"source_version":1,"source_minor_version":1,"entries":[],"warnings":[]}}"#,
)
.unwrap();
assert!(matches!(
err,
MigrateError::UnsupportedSchemaVersion { minor_version: 99, .. }
read_homecore_config_entries(&path),
Err(MigrateError::UnsupportedSchemaVersion { version: 99, .. })
));
}
#[test]
fn malformed_entry_is_an_error_not_a_partial_write() {
let mut source = NamedTempFile::new().unwrap();
source
.write_all(
br#"{"version":1,"minor_version":1,"key":"core.config_entries","data":{"entries":[{"domain":"hue"}]}}"#,
)
.unwrap();
assert!(matches!(
convert_config_entries(source.path()),
Err(MigrateError::MissingField { .. })
));
}
}
+185 -71
View File
@@ -1,60 +1,142 @@
//! Parser for `core.device_registry` (HA storage schema v1, minor_version 113).
//!
//! P1: deserializes the envelope and returns `Vec<DeviceImport>`.
//! HOMECORE's device registry isn't fully wired yet (ADR-127 §2.5 deferred
//! to P2), so `DeviceImport` is a staging type for the future hand-off.
//! Conversion for HA `core.device_registry` schema v1/minor 1-13.
use std::path::Path;
use std::collections::{BTreeMap, HashSet};
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use homecore::DeviceEntry;
use serde::Deserialize;
use crate::{storage::read_envelope, storage_format::v13, MigrateError};
use crate::{
storage::{read_envelope, write_json_atomic},
storage_format::v13,
MigrateError,
};
/// Staging type for a device imported from HA. Not yet wired to HOMECORE's
/// device registry (ADR-127 §2.5 — deferred to P2).
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct DeviceImport {
pub id: String,
pub config_entries: Vec<String>,
#[serde(default)]
pub manufacturer: Option<String>,
#[serde(default)]
pub model: Option<String>,
#[serde(default)]
pub name: Option<String>,
/// `identifiers` — list of `[integration, id]` pairs. Preserved as raw
/// JSON for P2 consumption; not yet mapped to HOMECORE DeviceEntry.
#[serde(default)]
pub identifiers: Vec<Vec<String>>,
#[serde(default)]
pub connections: Vec<Vec<String>>,
#[serde(default)]
pub via_device_id: Option<String>,
#[serde(default)]
pub area_id: Option<String>,
}
const FILE_KEY: &str = "core.device_registry";
#[derive(Debug, Deserialize)]
struct HaDeviceRegistryData {
devices: Vec<DeviceImport>,
/// Deleted device tombstones — ignored in P1.
devices: Vec<HaDeviceRow>,
#[serde(default)]
#[allow(dead_code)]
deleted_devices: Vec<serde_json::Value>,
}
/// Read `core.device_registry` from `path` and return the raw import list.
pub fn read_device_registry(path: &Path) -> Result<Vec<DeviceImport>, MigrateError> {
let env = read_envelope(path)?;
let file_str = path.display().to_string();
v13::require_supported(&file_str, env.version, env.minor_version)?;
#[derive(Debug, Deserialize)]
struct HaDeviceRow {
id: String,
#[serde(default)]
config_entries: HashSet<String>,
#[serde(default)]
identifiers: HashSet<(String, String)>,
#[serde(default)]
connections: HashSet<(String, String)>,
#[serde(default)]
manufacturer: Option<String>,
#[serde(default)]
model: Option<String>,
#[serde(default)]
model_id: Option<String>,
#[serde(default)]
name: Option<String>,
#[serde(default)]
name_by_user: Option<String>,
#[serde(default)]
sw_version: Option<String>,
#[serde(default)]
hw_version: Option<String>,
#[serde(default)]
serial_number: Option<String>,
#[serde(default)]
via_device_id: Option<String>,
#[serde(default)]
area_id: Option<String>,
#[serde(default)]
entry_type: Option<String>,
#[serde(default)]
disabled_by: Option<String>,
#[serde(default)]
configuration_url: Option<String>,
#[serde(default)]
labels: HashSet<String>,
#[serde(default)]
primary_config_entry: Option<String>,
#[serde(default, flatten)]
extra: BTreeMap<String, serde_json::Value>,
}
impl From<HaDeviceRow> for DeviceEntry {
fn from(row: HaDeviceRow) -> Self {
Self {
id: row.id,
config_entries: row.config_entries,
identifiers: row.identifiers,
connections: row.connections,
manufacturer: row.manufacturer,
model: row.model,
model_id: row.model_id,
name: row.name,
name_by_user: row.name_by_user,
sw_version: row.sw_version,
hw_version: row.hw_version,
serial_number: row.serial_number,
via_device_id: row.via_device_id,
area_id: row.area_id,
entry_type: row.entry_type,
disabled_by: row.disabled_by,
configuration_url: row.configuration_url,
labels: row.labels,
primary_config_entry: row.primary_config_entry,
extra: row.extra,
}
}
}
pub fn read_device_registry(path: &Path) -> Result<Vec<DeviceEntry>, MigrateError> {
let env = read_envelope(path)?;
let file = path.display().to_string();
v13::require_supported(&file, env.version, env.minor_version)?;
if env.key != FILE_KEY {
return Err(MigrateError::UnexpectedStorageKey {
path: file,
expected: FILE_KEY.to_owned(),
actual: env.key,
});
}
let data: HaDeviceRegistryData =
serde_json::from_value(env.data).map_err(|e| MigrateError::JsonParse {
path: file_str,
source: e,
serde_json::from_value(env.data).map_err(|source| MigrateError::JsonParse {
path: path.display().to_string(),
source,
})?;
Ok(data.devices)
let _preserved_tombstone_count = data.deleted_devices.len();
Ok(data.devices.into_iter().map(DeviceEntry::from).collect())
}
pub fn write_device_registry(
storage_dir: &Path,
devices: &[DeviceEntry],
) -> Result<PathBuf, MigrateError> {
write_device_registry_with(storage_dir, devices, false)
}
/// As [`write_device_registry`], but `force = true` atomically replaces an
/// existing destination instead of refusing — the escape hatch for
/// re-running an import after fixing a bad source row.
pub fn write_device_registry_with(
storage_dir: &Path,
devices: &[DeviceEntry],
force: bool,
) -> Result<PathBuf, MigrateError> {
let target = storage_dir.join(FILE_KEY);
let payload = serde_json::json!({
"version": 1,
"minor_version": 13,
"key": FILE_KEY,
"data": {
"devices": devices,
"deleted_devices": []
}
});
write_json_atomic(&target, &payload, force)
}
#[cfg(test)]
@@ -64,36 +146,68 @@ mod tests {
use tempfile::NamedTempFile;
const FIXTURE: &str = r#"{
"version": 1,
"minor_version": 13,
"key": "core.device_registry",
"data": {
"devices": [
{
"id": "dev_abc",
"config_entries": ["ce_001"],
"manufacturer": "Philips",
"model": "Hue Bridge",
"name": "Philips Hue Bridge",
"identifiers": [["hue", "001788FFFE3D4B13"]],
"connections": [["mac", "00:17:88:ff:fe:3d:4b:13"]],
"via_device_id": null,
"area_id": null
}
],
"deleted_devices": []
}
"version":1,"minor_version":13,"key":"core.device_registry",
"data":{"devices":[{
"id":"dev_abc","config_entries":["ce_001"],
"manufacturer":"Philips","model":"Hue Bridge","model_id":"BSB002",
"name":"Hue","name_by_user":"Downstairs Hue",
"sw_version":"1.2","hw_version":"3","serial_number":"SN42",
"identifiers":[["hue","001788FFFE3D4B13"]],
"connections":[["mac","00:17:88:ff:fe:3d:4b:13"]],
"via_device_id":"gateway","area_id":"living_room",
"entry_type":"service","disabled_by":"user",
"configuration_url":"http://hue.local","labels":["lighting"],
"primary_config_entry":"ce_001","created_at":1735689600.0
}],"deleted_devices":[]}
}"#;
#[test]
fn parses_device_registry() {
let mut f = NamedTempFile::new().unwrap();
f.write_all(FIXTURE.as_bytes()).unwrap();
let devices = read_device_registry(f.path()).unwrap();
assert_eq!(devices.len(), 1);
let d = &devices[0];
assert_eq!(d.id, "dev_abc");
assert_eq!(d.manufacturer.as_deref(), Some("Philips"));
assert_eq!(d.identifiers, vec![vec!["hue", "001788FFFE3D4B13"]]);
fn all_supported_fields_round_trip() {
let mut source = NamedTempFile::new().unwrap();
source.write_all(FIXTURE.as_bytes()).unwrap();
let devices = read_device_registry(source.path()).unwrap();
let destination = tempfile::tempdir().unwrap();
let path = write_device_registry(destination.path(), &devices).unwrap();
let imported = read_device_registry(&path).unwrap();
assert_eq!(imported, devices);
assert_eq!(imported[0].serial_number.as_deref(), Some("SN42"));
assert!(imported[0].labels.contains("lighting"));
assert_eq!(imported[0].extra["created_at"], 1735689600.0);
}
#[test]
fn destination_is_never_overwritten() {
let mut source = NamedTempFile::new().unwrap();
source.write_all(FIXTURE.as_bytes()).unwrap();
let devices = read_device_registry(source.path()).unwrap();
let destination = tempfile::tempdir().unwrap();
write_device_registry(destination.path(), &devices).unwrap();
let error = write_device_registry(destination.path(), &[]).unwrap_err();
assert!(error.to_string().contains("refusing to overwrite"));
assert_eq!(
read_device_registry(&destination.path().join(FILE_KEY))
.unwrap()
.len(),
1
);
}
/// `force = true` is the operator escape hatch: re-running an import
/// after fixing a bad source row (or re-importing after further HA-side
/// changes) must not require manually deleting prior output first.
#[test]
fn force_overwrites_existing_destination() {
let mut source = NamedTempFile::new().unwrap();
source.write_all(FIXTURE.as_bytes()).unwrap();
let devices = read_device_registry(source.path()).unwrap();
let destination = tempfile::tempdir().unwrap();
write_device_registry(destination.path(), &devices).unwrap();
write_device_registry_with(destination.path(), &[], true).unwrap();
assert_eq!(
read_device_registry(&destination.path().join(FILE_KEY))
.unwrap()
.len(),
0
);
}
}
@@ -26,14 +26,14 @@
//! }
//! ```
use std::path::Path;
use std::path::{Path, PathBuf};
use serde::{Deserialize, Serialize};
use homecore::{registry::DisabledBy, EntityCategory, EntityEntry, EntityId};
use crate::{
storage::read_envelope,
storage::{read_envelope, write_json_atomic},
storage_format::v13,
MigrateError,
};
@@ -75,21 +75,21 @@ struct HaEntityRow {
// Fields present in v13 that we capture but do not yet map to HOMECORE.
// Forwarded as Q5 items.
#[serde(default)]
hidden_by: Option<String>, // v13: "user" | "integration"
hidden_by: Option<String>, // v13: "user" | "integration"
#[serde(default)]
has_entity_name: Option<bool>, // v13: HA naming convention flag
has_entity_name: Option<bool>, // v13: HA naming convention flag
#[serde(default)]
original_name: Option<String>, // v13: integration-provided default name
original_name: Option<String>, // v13: integration-provided default name
#[serde(default)]
icon: Option<String>, // v13: mdi:xxx icon override
icon: Option<String>, // v13: mdi:xxx icon override
#[serde(default)]
original_icon: Option<String>, // v13: integration-provided icon
original_icon: Option<String>, // v13: integration-provided icon
#[serde(default)]
aliases: Option<Vec<String>>, // v13: user-set aliases for voice assist
aliases: Option<Vec<String>>, // v13: user-set aliases for voice assist
#[serde(default)]
capabilities: Option<serde_json::Value>, // v13: integration-specific caps
#[serde(default)]
supported_features: Option<u64>, // v13: bitmask
supported_features: Option<u64>, // v13: bitmask
}
#[derive(Debug, Serialize, Deserialize, PartialEq, Eq)]
@@ -166,6 +166,39 @@ pub fn read_entity_registry(path: &Path) -> Result<Vec<EntityEntry>, MigrateErro
Ok(entries)
}
/// Persist imported entries using the HA-compatible v13 storage envelope.
///
/// The destination is created if needed and the file is written through a
/// same-directory temporary file followed by an atomic rename. Existing
/// registries are never overwritten implicitly.
pub fn write_entity_registry(
storage_dir: &Path,
entries: &[EntityEntry],
) -> Result<PathBuf, MigrateError> {
write_entity_registry_with(storage_dir, entries, false)
}
/// As [`write_entity_registry`], but `force = true` atomically replaces an
/// existing destination instead of refusing — the escape hatch for
/// re-running an import after fixing a bad source row.
pub fn write_entity_registry_with(
storage_dir: &Path,
entries: &[EntityEntry],
force: bool,
) -> Result<PathBuf, MigrateError> {
let target = storage_dir.join(FILE_KEY);
let payload = serde_json::json!({
"version": 1,
"minor_version": 13,
"key": FILE_KEY,
"data": {
"entities": entries,
"deleted_entities": []
}
});
write_json_atomic(&target, &payload, force)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -227,7 +260,10 @@ mod tests {
fn entity_fields_round_trip_correctly() {
let f = write_fixture(FIXTURE_V13);
let entries = read_entity_registry(f.path()).unwrap();
let light = entries.iter().find(|e| e.entity_id.as_str() == "light.kitchen").unwrap();
let light = entries
.iter()
.find(|e| e.entity_id.as_str() == "light.kitchen")
.unwrap();
assert_eq!(light.unique_id.as_deref(), Some("hue_lamp_42"));
assert_eq!(light.platform, "hue");
assert_eq!(light.name.as_deref(), Some("Kitchen lamp"));
@@ -238,6 +274,21 @@ mod tests {
assert_eq!(light.config_entry_id.as_deref(), Some("ce_001"));
}
#[test]
fn writes_atomic_compatible_registry_without_overwrite() {
let source = write_fixture(FIXTURE_V13);
let entries = read_entity_registry(source.path()).unwrap();
let destination = tempfile::tempdir().unwrap();
let path = write_entity_registry(destination.path(), &entries).unwrap();
let imported = read_entity_registry(&path).unwrap();
assert_eq!(imported.len(), entries.len());
assert_eq!(imported[0].entity_id, entries[0].entity_id);
let second = write_entity_registry(destination.path(), &entries).unwrap_err();
assert!(second.to_string().contains("refusing to overwrite"));
}
#[test]
fn disabled_by_maps_to_homecore() {
let f = write_fixture(FIXTURE_V13);
@@ -260,7 +311,13 @@ mod tests {
let f = write_fixture(json);
let err = read_entity_registry(f.path()).unwrap_err();
assert!(
matches!(err, MigrateError::UnsupportedSchemaVersion { minor_version: 99, .. }),
matches!(
err,
MigrateError::UnsupportedSchemaVersion {
minor_version: 99,
..
}
),
"got: {err}"
);
let msg = err.to_string();
+13 -5
View File
@@ -4,20 +4,21 @@
//! (HOMECORE-MIGRATE; ADR-126 §4 series map labels the role "ADR-134 HOMECORE-MIGRATE",
//! but on-disk ADR-134 is CIR — the migrate decision was renumbered to ADR-165. See ADR-164).
//!
//! ## P1 scope
//! ## Implemented scope
//!
//! - [`storage`] — `HaStorageDir`, `HaStorageEnvelope`; `read_envelope(path)`
//! - [`storage_format`] — versioned format parsers (`v13`); unknown minor_version → hard error
//! - [`entity_registry`] — `core.entity_registry` → `Vec<homecore::EntityEntry>`
//! - [`device_registry`] — `core.device_registry` → `Vec<DeviceImport>` (P1 stub)
//! - [`config_entries`] — `core.config_entries` diagnostic (count + domain list; P2 converts)
//! - [`device_registry`] — full supported HA v13 device fields → `homecore::DeviceEntry`
//! - [`config_entries`] — lossless, versioned HOMECORE representation + typed warnings
//! - [`secrets`] — `secrets.yaml` → `HashMap<String, String>`
//! - [`automations`] — `automations.yaml` count + ID list (P2 converts)
//! - [`cli`] — `clap`-derived subcommand types shared between `src/main.rs` and tests
//!
//! ## What is NOT here yet (deferred to P2+)
//! ## Remaining limitations
//!
//! - Conversion of `config_entries` to HOMECORE plugin manifests
//! - Imported config entries are durable but do not make an HA integration executable;
//! a matching HOMECORE plugin must consume the preserved source payload.
//! - Conversion of `automations.yaml` to `homecore-automation` YAML
//! - Side-by-side runtime mode (requires `homecore-recorder`, ADR-132)
//! - `!secret` reference resolution in non-secrets YAML files
@@ -88,6 +89,13 @@ pub enum MigrateError {
minor_version: u32,
},
#[error("unexpected storage key in {path}: expected {expected}, got {actual}")]
UnexpectedStorageKey {
path: String,
expected: String,
actual: String,
},
#[error("missing required field '{field}' in {context}")]
MissingField { field: String, context: String },
+71 -18
View File
@@ -2,6 +2,21 @@
use clap::Parser;
use homecore_migrate::cli::{Cli, Command};
use serde::Serialize;
#[derive(Serialize)]
struct ImportSummary {
kind: &'static str,
imported: usize,
warning_count: usize,
warnings: Vec<serde_json::Value>,
destination: std::path::PathBuf,
}
fn print_summary(summary: &ImportSummary) -> anyhow::Result<()> {
println!("{}", serde_json::to_string(summary)?);
Ok(())
}
fn main() -> anyhow::Result<()> {
tracing_subscriber::fmt::init();
@@ -9,7 +24,10 @@ fn main() -> anyhow::Result<()> {
match cli.command {
Command::Inspect(args) => {
println!("Inspecting HA .storage directory: {}", args.storage.display());
println!(
"Inspecting HA .storage directory: {}",
args.storage.display()
);
// Probe entity_registry
let entity_path = args.storage.join("core.entity_registry");
if entity_path.exists() {
@@ -42,31 +60,41 @@ fn main() -> anyhow::Result<()> {
Command::ImportEntities(args) => {
let entity_path = args.storage.join("core.entity_registry");
let entries =
homecore_migrate::entity_registry::read_entity_registry(&entity_path)?;
println!("Imported {} entity entries (P1: in-memory only)", entries.len());
println!(" Destination: {} (P2 persistence)", args.to.display());
for e in &entries {
println!(
" {} ({}{})",
e.entity_id.as_str(),
e.platform,
if e.disabled_by.is_some() { " DISABLED" } else { "" }
);
}
let entries = homecore_migrate::entity_registry::read_entity_registry(&entity_path)?;
let destination = homecore_migrate::entity_registry::write_entity_registry_with(
&args.to,
&entries,
args.force,
)?;
print_summary(&ImportSummary {
kind: "entity_registry",
imported: entries.len(),
warning_count: 0,
warnings: vec![],
destination,
})?;
}
Command::ImportDevices(args) => {
let device_path = args.storage.join("core.device_registry");
let devices =
homecore_migrate::device_registry::read_device_registry(&device_path)?;
println!("Parsed {} device entries (P1: staging only, wiring to HOMECORE is P2)", devices.len());
let devices = homecore_migrate::device_registry::read_device_registry(&device_path)?;
let destination = homecore_migrate::device_registry::write_device_registry_with(
&args.to,
&devices,
args.force,
)?;
print_summary(&ImportSummary {
kind: "device_registry",
imported: devices.len(),
warning_count: 0,
warnings: vec![],
destination,
})?;
}
Command::InspectConfigEntries(args) => {
let ce_path = args.storage.join("core.config_entries");
let summary =
homecore_migrate::config_entries::inspect_config_entries(&ce_path)?;
let summary = homecore_migrate::config_entries::inspect_config_entries(&ce_path)?;
println!(
"config_entries: {} total, domains: {}",
summary.count,
@@ -74,6 +102,31 @@ fn main() -> anyhow::Result<()> {
);
}
Command::ImportConfigEntries(args) => {
let source = args.storage.join("core.config_entries");
let converted = homecore_migrate::config_entries::convert_config_entries(&source)?;
let imported = converted.data.entries.len();
let warning_count = converted.data.warnings.len();
let warnings = converted
.data
.warnings
.iter()
.map(serde_json::to_value)
.collect::<Result<Vec<_>, _>>()?;
let destination = homecore_migrate::config_entries::write_config_entries_with(
&args.to,
&converted,
args.force,
)?;
print_summary(&ImportSummary {
kind: "config_entries",
imported,
warning_count,
warnings,
destination,
})?;
}
Command::InspectSecrets(args) => {
let secrets_path = args.config_dir.join("secrets.yaml");
let secrets = homecore_migrate::secrets::read_secrets(&secrets_path)?;
+100
View File
@@ -15,12 +15,17 @@
//! left as `serde_json::Value` — version-specific parsers in `storage_format`
//! are responsible for further deserialization.
use std::fs::{self, OpenOptions};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::atomic::{AtomicU64, Ordering};
use serde::{Deserialize, Serialize};
use crate::MigrateError;
static TEMP_SEQUENCE: AtomicU64 = AtomicU64::new(0);
/// Points to a HA `.storage/` directory.
#[derive(Clone, Debug)]
pub struct HaStorageDir {
@@ -66,6 +71,101 @@ pub fn read_envelope(path: &Path) -> Result<HaStorageEnvelope, MigrateError> {
})
}
/// Durably publish JSON at `target` without ever replacing an existing file.
///
/// Bytes are synced in a same-directory temporary file, then exposed with an
/// atomic hard-link create. `hard_link` fails with `AlreadyExists` if another
/// process won the destination race, unlike a POSIX rename which would replace
/// the destination after a check-then-rename sequence.
pub fn write_json_atomic_noclobber<T: Serialize>(
target: &Path,
value: &T,
) -> Result<PathBuf, MigrateError> {
write_json_atomic(target, value, false)
}
/// Same atomic write (temp file → `sync_all` → publish), but when `force` is
/// true an existing destination is atomically replaced via `rename` instead
/// of refusing via `hard_link`'s `AlreadyExists`. Without `force`, an
/// operator who re-runs an import after fixing a bad source row (or wants a
/// fresh pass) had no way to overwrite prior output short of deleting it by
/// hand first — this is the escape hatch for that, opt-in so the default
/// no-clobber safety is unchanged.
pub fn write_json_atomic<T: Serialize>(
target: &Path,
value: &T,
force: bool,
) -> Result<PathBuf, MigrateError> {
let parent = target.parent().ok_or_else(|| MigrateError::Io {
path: target.display().to_string(),
source: std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"destination has no parent directory",
),
})?;
fs::create_dir_all(parent).map_err(|source| MigrateError::Io {
path: parent.display().to_string(),
source,
})?;
let bytes = serde_json::to_vec_pretty(value).map_err(|source| MigrateError::JsonParse {
path: target.display().to_string(),
source,
})?;
let sequence = TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let name = target
.file_name()
.and_then(|value| value.to_str())
.unwrap_or("storage");
let temp = parent.join(format!(".{name}.{}.{}.tmp", std::process::id(), sequence));
let result = (|| -> std::io::Result<()> {
let mut file = OpenOptions::new()
.create_new(true)
.write(true)
.open(&temp)?;
file.write_all(&bytes)?;
file.write_all(b"\n")?;
file.sync_all()?;
if force {
// `rename`-over-existing hits real sharing-violation flakiness
// on Windows (ERROR_ACCESS_DENIED even with no other open
// handle in this process). Pre-clear the destination instead,
// then publish through the same hard_link step the no-clobber
// path uses. This briefly widens the crash window (a crash
// between remove and hard_link leaves no destination file
// rather than the old one), which is the accepted, opt-in
// tradeoff of explicitly requesting an overwrite — the default
// (non-force) path keeps its full no-clobber atomicity.
match fs::remove_file(target) {
Ok(()) => {}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {}
Err(error) => return Err(error),
}
}
fs::hard_link(&temp, target)?;
fs::remove_file(&temp)?;
Ok(())
})();
if let Err(source) = result {
let _ = fs::remove_file(&temp);
let source = if source.kind() == std::io::ErrorKind::AlreadyExists {
std::io::Error::new(
std::io::ErrorKind::AlreadyExists,
"destination exists; refusing to overwrite",
)
} else {
source
};
return Err(MigrateError::Io {
path: target.display().to_string(),
source,
});
}
Ok(target.to_path_buf())
}
#[cfg(test)]
mod tests {
use super::*;
+5 -10
View File
@@ -5,9 +5,7 @@
# - PluginRuntime trait + InProcessRuntime (native Rust, first-party plugins)
# - PluginRegistry (load / unload / list)
#
# P2 will add the `wasmtime` feature (gated below, default-off) for the real
# Wasmtime JIT sandbox. wasm3 interpretation mode lands behind `--features wasm3`
# in P3 for constrained-hardware targets.
# The `wasmtime` feature gates the audited server-side WebAssembly sandbox.
[package]
name = "homecore-plugins"
@@ -27,8 +25,6 @@ default = []
# P2: real Wasmtime JIT sandbox (Cranelift; ~15 MB binary delta on Pi 5).
# Do not enable in production until the host ABI is frozen (ADR-128 §8 risk).
wasmtime = ["dep:wasmtime"]
# P3: wasm3 interpretation mode for constrained hardware (~50 kB).
wasm3 = ["dep:wasm3"]
[dependencies]
# HOMECORE state machine — local path (ADR-127).
@@ -60,12 +56,11 @@ hex = "0.4"
base64 = "0.22"
# Optional Wasmtime runtime (P2, default-off — 30 MB dep).
# Bumped from 25.0.3 → 42 to remediate RUSTSEC-2026-0095 and RUSTSEC-2026-0096
# (Cranelift/Winch sandbox-escape CVEs, CVSS 9.0 — iter-11 security sprint HC-03/04).
wasmtime = { version = "42", optional = true }
# Upgraded from 25.0.3 to remediate the Cranelift/Winch sandbox advisories;
# 40.0.4 was subsequently replaced because of RUSTSEC-2026-0114.
# Patched 36.x retains HOMECORE's Rust 1.89 MSRV.
wasmtime = { version = "36.0.8", optional = true }
# Optional wasm3 interpretation runtime (P3, default-off).
wasm3 = { version = "0.3", optional = true }
[dev-dependencies]
tokio = { version = "1", features = ["sync", "rt", "rt-multi-thread", "time", "macros", "test-util"] }
+272
View File
@@ -0,0 +1,272 @@
//! Secure filesystem discovery for packaged WASM plugins.
//!
//! A plugin directory has exactly this shape:
//!
//! ```text
//! <configured-root>/<package>/manifest.json
//! <module named by wasm_module>
//! ```
//!
//! Only immediate child directories of explicitly configured roots are
//! inspected. Symlinks are rejected and the module must be a plain filename
//! in the same package directory. This deliberately avoids recursive search,
//! implicit current-directory loading, and path traversal.
use std::collections::BTreeSet;
use std::fs;
use std::path::{Component, Path, PathBuf};
use crate::{PluginError, PluginManifest};
/// Conservative input limits applied before allocating file contents.
#[derive(Debug, Clone, Copy)]
pub struct DiscoveryLimits {
pub max_plugins: usize,
pub max_manifest_bytes: u64,
pub max_module_bytes: u64,
}
impl Default for DiscoveryLimits {
fn default() -> Self {
Self {
max_plugins: 128,
max_manifest_bytes: 256 * 1024,
max_module_bytes: 16 * 1024 * 1024,
}
}
}
/// A validated package whose files are safe to read.
#[derive(Debug, Clone)]
pub struct DiscoveredPlugin {
pub manifest: PluginManifest,
pub package_dir: PathBuf,
pub manifest_path: PathBuf,
pub module_path: PathBuf,
}
impl DiscoveredPlugin {
/// Read the module after re-checking its type and size. Re-checking closes
/// the common accidental replacement window between discovery and load;
/// signature verification remains the authoritative tamper gate.
pub fn read_module(&self, limits: DiscoveryLimits) -> Result<Vec<u8>, PluginError> {
bounded_regular_file(&self.module_path, limits.max_module_bytes, "WASM module")
}
}
/// Discover packages in deterministic root/path order.
pub fn discover_plugins(
roots: &[PathBuf],
limits: DiscoveryLimits,
) -> Result<Vec<DiscoveredPlugin>, PluginError> {
if roots.is_empty() {
return Ok(Vec::new());
}
if limits.max_plugins == 0 || limits.max_manifest_bytes == 0 || limits.max_module_bytes == 0 {
return Err(PluginError::ResourceLimit(
"discovery limits must all be greater than zero".into(),
));
}
let mut packages = BTreeSet::new();
for configured_root in roots {
let metadata = fs::symlink_metadata(configured_root).map_err(|e| {
PluginError::Discovery(format!(
"cannot inspect configured plugin root {}: {e}",
configured_root.display()
))
})?;
if metadata.file_type().is_symlink() || !metadata.is_dir() {
return Err(PluginError::Discovery(format!(
"configured plugin root must be a real directory, not a symlink: {}",
configured_root.display()
)));
}
let root = fs::canonicalize(configured_root)?;
let entries = fs::read_dir(&root)?;
for entry in entries {
let entry = entry?;
let ty = entry.file_type()?;
if ty.is_symlink() {
return Err(PluginError::Discovery(format!(
"symlink found in plugin root: {}",
entry.path().display()
)));
}
if ty.is_dir() && entry.path().join("manifest.json").exists() {
packages.insert(entry.path());
}
}
}
if packages.len() > limits.max_plugins {
return Err(PluginError::ResourceLimit(format!(
"discovered {} plugins, maximum is {}",
packages.len(),
limits.max_plugins
)));
}
let mut result = Vec::with_capacity(packages.len());
let mut domains = BTreeSet::new();
for package_dir in packages {
let package_dir = fs::canonicalize(package_dir)?;
let manifest_path = package_dir.join("manifest.json");
let manifest_bytes =
bounded_regular_file(&manifest_path, limits.max_manifest_bytes, "manifest")?;
let manifest_text = std::str::from_utf8(&manifest_bytes).map_err(|e| {
PluginError::InvalidManifest(format!("{} is not UTF-8: {e}", manifest_path.display()))
})?;
let manifest = PluginManifest::parse_json(manifest_text)?;
if !domains.insert(manifest.domain.clone()) {
return Err(PluginError::Discovery(format!(
"duplicate plugin domain `{}`",
manifest.domain
)));
}
let module_name = manifest.wasm_module.as_deref().ok_or_else(|| {
PluginError::InvalidManifest(format!(
"WASM package `{}` has no wasm_module",
manifest.domain
))
})?;
let relative = Path::new(module_name);
if relative.components().count() != 1
|| !matches!(relative.components().next(), Some(Component::Normal(_)))
|| relative.extension().and_then(|v| v.to_str()) != Some("wasm")
{
return Err(PluginError::InvalidManifest(format!(
"plugin `{}` wasm_module must be a .wasm filename in its package directory",
manifest.domain
)));
}
let module_path = package_dir.join(relative);
let metadata = fs::symlink_metadata(&module_path).map_err(|e| {
PluginError::Discovery(format!("cannot inspect {}: {e}", module_path.display()))
})?;
if metadata.file_type().is_symlink() || !metadata.is_file() {
return Err(PluginError::Discovery(format!(
"plugin module must be a regular non-symlink file: {}",
module_path.display()
)));
}
if metadata.len() > limits.max_module_bytes {
return Err(PluginError::ResourceLimit(format!(
"{} is {} bytes; maximum is {}",
module_path.display(),
metadata.len(),
limits.max_module_bytes
)));
}
result.push(DiscoveredPlugin {
manifest,
package_dir,
manifest_path,
module_path,
});
}
Ok(result)
}
fn bounded_regular_file(path: &Path, maximum: u64, kind: &str) -> Result<Vec<u8>, PluginError> {
let metadata = fs::symlink_metadata(path)?;
if metadata.file_type().is_symlink() || !metadata.is_file() {
return Err(PluginError::Discovery(format!(
"{kind} must be a regular non-symlink file: {}",
path.display()
)));
}
if metadata.len() > maximum {
return Err(PluginError::ResourceLimit(format!(
"{} is {} bytes; maximum is {}",
path.display(),
metadata.len(),
maximum
)));
}
let bytes = fs::read(path)?;
if bytes.len() as u64 > maximum {
return Err(PluginError::ResourceLimit(format!(
"{} grew beyond the {maximum}-byte limit while being read",
path.display()
)));
}
Ok(bytes)
}
#[cfg(test)]
mod tests {
use super::*;
use std::time::{SystemTime, UNIX_EPOCH};
fn root() -> PathBuf {
let path = std::env::temp_dir().join(format!(
"homecore-plugin-discovery-{}-{}",
std::process::id(),
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.as_nanos()
));
fs::create_dir(&path).unwrap();
path
}
fn package(root: &Path, directory: &str, domain: &str, module: &str) {
let dir = root.join(directory);
fs::create_dir(&dir).unwrap();
fs::write(
dir.join("manifest.json"),
format!(
r#"{{"domain":"{domain}","name":"Test","version":"1","wasm_module":"{module}"}}"#
),
)
.unwrap();
if !module.contains('/') && !module.contains('\\') && module.ends_with(".wasm") {
fs::write(dir.join(module), b"\0asm\x01\0\0\0").unwrap();
}
}
#[test]
fn discovery_is_sorted_and_only_reads_explicit_roots() {
let root = root();
package(&root, "z-last", "zeta", "z.wasm");
package(&root, "a-first", "alpha", "a.wasm");
let found =
discover_plugins(std::slice::from_ref(&root), DiscoveryLimits::default()).unwrap();
assert_eq!(
found
.iter()
.map(|p| p.manifest.domain.as_str())
.collect::<Vec<_>>(),
["alpha", "zeta"]
);
fs::remove_dir_all(root).unwrap();
}
#[test]
fn traversal_module_path_is_rejected() {
let root = root();
package(&root, "bad", "bad", "../bad.wasm");
let error =
discover_plugins(std::slice::from_ref(&root), DiscoveryLimits::default()).unwrap_err();
assert!(matches!(error, PluginError::InvalidManifest(_)));
fs::remove_dir_all(root).unwrap();
}
#[test]
fn size_limits_are_enforced_before_read() {
let root = root();
package(&root, "large", "large", "large.wasm");
let error = discover_plugins(
std::slice::from_ref(&root),
DiscoveryLimits {
max_module_bytes: 4,
..DiscoveryLimits::default()
},
)
.unwrap_err();
assert!(matches!(error, PluginError::ResourceLimit(_)));
fs::remove_dir_all(root).unwrap();
}
}
+8
View File
@@ -21,6 +21,14 @@ pub enum PluginError {
#[error("runtime error: {0}")]
RuntimeError(String),
/// Plugin discovery or filesystem layout is invalid.
#[error("plugin discovery failed: {0}")]
Discovery(String),
/// A configured manifest, module, or ABI payload exceeds its limit.
#[error("plugin resource limit exceeded: {0}")]
ResourceLimit(String),
/// The plugin's `setup` hook returned an error.
#[error("plugin setup failed: {0}")]
SetupFailed(String),
+7 -8
View File
@@ -14,14 +14,12 @@
//! - [`registry`] — `PluginRegistry<R>`: load / unload / list plugins.
//! - [`error`] — `PluginError` typed error enum.
//!
//! ## What's NOT here yet (deferred)
//! ## Runtime scope
//!
//! - `WasmtimeRuntime` (P2, `--features wasmtime`): Cranelift JIT sandbox on
//! Pi 5 / x86_64. The runtime-selection question (Wasmtime vs wasm3) is still
//! open (ADR-128 §8) and will be resolved in Q2 before P2 begins.
//! - Host ABI wiring: `hc_state_get`, `hc_state_set`, `hc_event_fire`, etc.
//! (P2 — requires ADR-127 state machine API freeze first).
//! - Config entry lifecycle + hot-load (P3).
//! - `WasmtimeRuntime` (`--features wasmtime`) provides the Cranelift sandbox.
//! - Native Rust plugins are compiled into the server; arbitrary native
//! dynamic libraries are not loaded.
//! - The host ABI is deliberately narrow and resource-bounded.
//!
//! ## Now enforced (ADR-162)
//!
@@ -38,9 +36,9 @@
//! | Feature | Default | Description |
//! |---------|---------|-------------|
//! | `wasmtime` | off | Wasmtime Cranelift JIT runtime (P2) |
//! | `wasm3` | off | wasm3 interpreter runtime for constrained hardware (P3) |
pub mod error;
pub mod discovery;
pub mod host_abi;
pub mod manifest;
pub mod permissions;
@@ -53,6 +51,7 @@ pub mod verify;
pub mod wasmtime_runtime;
pub use error::PluginError;
pub use discovery::{discover_plugins, DiscoveredPlugin, DiscoveryLimits};
pub use host_abi::{ConfigEntryJson, StateChangedEventJson};
pub use manifest::{IotClass, IntegrationType, PluginManifest};
pub use permissions::PermissionSet;
+8 -1
View File
@@ -11,10 +11,11 @@ use async_trait::async_trait;
use homecore::HomeCore;
use crate::error::PluginError;
use crate::StateChangedEventJson;
/// Unique identifier for a loaded plugin — mirrors the `domain` field of
/// the plugin's `PluginManifest` (e.g. `"mqtt"`, `"homecore_lights"`).
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
#[derive(Debug, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PluginId(pub String);
impl PluginId {
@@ -52,6 +53,12 @@ pub trait HomeCorePlugin: Send + Sync + 'static {
/// register its entities, services, and event subscriptions here.
async fn setup(&self, hc: HomeCore) -> Result<(), PluginError>;
/// Receive a committed state change. The default is a no-op so built-in
/// plugins only opt into event work they need.
async fn state_changed(&self, _event: &StateChangedEventJson) -> Result<(), PluginError> {
Ok(())
}
/// Called when the plugin is being removed from the registry.
///
/// The plugin should clean up subscriptions and deregister its entities.
+54 -16
View File
@@ -5,7 +5,7 @@
//! the `InProcessRuntime` (P1) for a `WasmtimeRuntime` (P2) without
//! changing registry code.
use std::collections::HashMap;
use std::collections::{BTreeMap, BTreeSet};
use std::sync::Arc;
use homecore::HomeCore;
@@ -15,6 +15,7 @@ use crate::error::PluginError;
use crate::manifest::PluginManifest;
use crate::plugin::{HomeCorePlugin, PluginId};
use crate::runtime::{LoadedPlugin, PluginRuntime};
use crate::StateChangedEventJson;
/// Holds all loaded plugins keyed by `PluginId`.
///
@@ -22,7 +23,8 @@ use crate::runtime::{LoadedPlugin, PluginRuntime};
/// unload) take an exclusive lock only while mutating the map.
pub struct PluginRegistry<R: PluginRuntime> {
runtime: R,
plugins: RwLock<HashMap<PluginId, LoadedPlugin>>,
plugins: RwLock<BTreeMap<PluginId, LoadedPlugin>>,
loading: RwLock<BTreeSet<PluginId>>,
}
impl<R: PluginRuntime> PluginRegistry<R> {
@@ -30,7 +32,8 @@ impl<R: PluginRuntime> PluginRegistry<R> {
pub fn new(runtime: R) -> Self {
Self {
runtime,
plugins: RwLock::new(HashMap::new()),
plugins: RwLock::new(BTreeMap::new()),
loading: RwLock::new(BTreeSet::new()),
}
}
@@ -48,22 +51,31 @@ impl<R: PluginRuntime> PluginRegistry<R> {
{
let guard = self.plugins.read().await;
if guard.contains_key(&id) {
let mut loading = self.loading.write().await;
if guard.contains_key(&id) || !loading.insert(id.clone()) {
return Err(PluginError::AlreadyLoaded(id.to_string()));
}
}
let loaded = self
let result = self
.runtime
.load(id.clone(), manifest, plugin)
.await?;
loaded
.setup(hc)
.await
.map_err(|e| PluginError::SetupFailed(e.to_string()))?;
.await;
let loaded = match result {
Ok(loaded) => loaded,
Err(error) => {
self.loading.write().await.remove(&id);
return Err(error);
}
};
if let Err(error) = loaded.setup(hc).await {
// Best-effort rollback for partially initialized plugins.
let _ = loaded.unload().await;
self.loading.write().await.remove(&id);
return Err(PluginError::SetupFailed(error.to_string()));
}
self.plugins.write().await.insert(id.clone(), loaded);
self.loading.write().await.remove(&id);
Ok(id)
}
@@ -78,10 +90,11 @@ impl<R: PluginRuntime> PluginRegistry<R> {
.ok_or_else(|| PluginError::NotFound(id.to_string()))?
};
loaded
.unload()
.await
.map_err(|e| PluginError::UnloadFailed(e.to_string()))?;
if let Err(error) = loaded.unload().await {
// Keep the handle reachable so teardown can be retried.
self.plugins.write().await.insert(id.clone(), loaded);
return Err(PluginError::UnloadFailed(error.to_string()));
}
Ok(())
}
@@ -99,4 +112,29 @@ impl<R: PluginRuntime> PluginRegistry<R> {
pub async fn contains(&self, id: &PluginId) -> bool {
self.plugins.read().await.contains_key(id)
}
/// Dispatch a state change in stable plugin-domain order.
pub async fn state_changed(&self, event: &StateChangedEventJson) -> Vec<(PluginId, PluginError)> {
let guard = self.plugins.read().await;
let mut errors = Vec::new();
for (id, plugin) in guard.iter() {
if let Err(error) = plugin.state_changed(event).await {
errors.push((id.clone(), error));
}
}
errors
}
/// Tear down all plugins in reverse domain order. Failures are collected
/// while remaining plugins still receive teardown.
pub async fn shutdown(&self) -> Vec<(PluginId, PluginError)> {
let ids: Vec<_> = self.plugins.read().await.keys().cloned().rev().collect();
let mut errors = Vec::new();
for id in ids {
if let Err(error) = self.unload(&id).await {
errors.push((id, error));
}
}
errors
}
}
+8 -3
View File
@@ -1,8 +1,8 @@
//! `PluginRuntime` trait + `InProcessRuntime` (P1).
//!
//! Abstracts over Wasmtime (P2, `--features wasmtime`) and native in-process
//! Rust plugins (P1, always-on). A third backend, wasm3 (P3), will provide
//! interpretation mode for constrained hardware.
//! Rust plugins (P1, always-on). Constrained builds use the compiled-in native
//! registry rather than an unaudited interpreter backend.
//!
//! # Architecture
//!
@@ -12,7 +12,6 @@
//! ▼
//! PluginRuntime ◄─── InProcessRuntime (P1, native Rust, <1 µs call)
//! ◄─── WasmtimeRuntime (P2, Cranelift JIT, ~5 ms cold start)
//! ◄─── Wasm3Runtime (P3, interpreter, ~50 kB, Pi Zero)
//! ```
use std::sync::Arc;
@@ -23,6 +22,7 @@ use homecore::HomeCore;
use crate::error::PluginError;
use crate::manifest::PluginManifest;
use crate::plugin::{HomeCorePlugin, PluginId};
use crate::StateChangedEventJson;
/// A loaded plugin handle — returned by [`PluginRuntime::load`].
pub struct LoadedPlugin {
@@ -42,6 +42,11 @@ impl LoadedPlugin {
pub async fn unload(&self) -> Result<(), PluginError> {
self.instance.unload().await
}
/// Dispatch a committed state change to this plugin.
pub async fn state_changed(&self, event: &StateChangedEventJson) -> Result<(), PluginError> {
self.instance.state_changed(event).await
}
}
/// Abstraction over the WASM (and native) plugin execution environment.
+9 -2
View File
@@ -5,6 +5,7 @@
//! and PluginError variants.
#[cfg(test)]
#[allow(clippy::module_inception)]
mod tests {
use std::sync::Arc;
@@ -144,7 +145,10 @@ mod tests {
.expect("load should succeed");
assert_eq!(id.as_str(), "lights");
assert!(*plugin.setup_called.lock().await, "setup should have been called");
assert!(
*plugin.setup_called.lock().await,
"setup should have been called"
);
let listing = registry.list().await;
assert_eq!(listing.len(), 1);
@@ -163,7 +167,10 @@ mod tests {
.expect("load should succeed");
registry.unload(&id).await.expect("unload should succeed");
assert!(*plugin.unload_called.lock().await, "unload should have been called");
assert!(
*plugin.unload_called.lock().await,
"unload should have been called"
);
assert_eq!(registry.list().await.len(), 0);
}
@@ -26,7 +26,7 @@
use std::sync::{Arc, Mutex};
use homecore::HomeCore;
use wasmtime::{Engine, Linker, Module, Store};
use wasmtime::{Config, Engine, Linker, Module, Store, StoreLimits, StoreLimitsBuilder};
use crate::error::PluginError;
use crate::host_abi::{LogLevel, StateChangedEventJson, MAX_ABI_BUFFER_BYTES};
@@ -34,6 +34,13 @@ use crate::manifest::PluginManifest;
use crate::permissions::PermissionSet;
use crate::verify::{verify_module, PluginPolicy};
/// Hard ceiling for every module accepted by the runtime, including callers
/// that bypass filesystem discovery.
pub const MAX_WASM_MODULE_BYTES: usize = 16 * 1024 * 1024;
const MAX_SUBSCRIPTIONS: usize = 4096;
const MAX_LINEAR_MEMORY_BYTES: usize = 16 * 1024 * 1024;
const FUEL_PER_CALL: u64 = 10_000_000;
// ── Store data ─────────────────────────────────────────────────────────────
/// Per-plugin state stored inside the Wasmtime [`Store`].
@@ -51,6 +58,7 @@ pub struct PluginStoreData {
/// [`WasmtimeRuntime::load_plugin`] path installs the manifest's
/// declared set.
pub permissions: PermissionSet,
limits: StoreLimits,
}
// ── WasmtimeRuntime ────────────────────────────────────────────────────────
@@ -66,7 +74,10 @@ pub struct WasmtimeRuntime {
impl WasmtimeRuntime {
/// Create a new runtime with default Cranelift config.
pub fn new() -> Result<Self, PluginError> {
let engine = Engine::default();
let mut config = Config::new();
config.consume_fuel(true);
let engine = Engine::new(&config)
.map_err(|e| PluginError::RuntimeError(format!("Wasmtime engine: {e}")))?;
Ok(Self { engine })
}
@@ -83,6 +94,7 @@ impl WasmtimeRuntime {
wasm_bytes: &[u8],
hc: HomeCore,
) -> Result<WasmPlugin, PluginError> {
check_module_size(wasm_bytes)?;
self.instantiate(wasm_bytes, hc, PermissionSet::allow_all())
}
@@ -104,6 +116,7 @@ impl WasmtimeRuntime {
hc: HomeCore,
policy: &PluginPolicy,
) -> Result<WasmPlugin, PluginError> {
check_module_size(wasm_bytes)?;
// P4: verify before instantiation.
verify_module(manifest, wasm_bytes, policy)?;
// P5: scope write authority to the manifest's declared permissions.
@@ -128,8 +141,17 @@ impl WasmtimeRuntime {
hc,
subscriptions: Vec::new(),
permissions,
limits: StoreLimitsBuilder::new()
.memory_size(MAX_LINEAR_MEMORY_BYTES)
.instances(1)
.memories(1)
.build(),
};
let mut store = Store::new(&self.engine, store_data);
store.limiter(|data| &mut data.limits);
store
.set_fuel(FUEL_PER_CALL)
.map_err(|e| PluginError::RuntimeError(format!("set instantiation fuel: {e}")))?;
let instance = linker
.instantiate(&mut store, &module)
@@ -179,6 +201,12 @@ fn register_hc_state_get(
out_ptr: i32,
out_cap: i32|
-> i32 {
if out_ptr < 0
|| out_cap < 0
|| out_cap as usize > MAX_ABI_BUFFER_BYTES
{
return -1;
}
// Phase 1: read the entity key from guest memory.
let key: String = {
let mem = match caller.get_export("memory") {
@@ -216,7 +244,9 @@ fn register_hc_state_get(
Some(wasmtime::Extern::Memory(m)) => m,
_ => return -1,
};
let end = out_ptr as usize + json_bytes.len();
let Some(end) = (out_ptr as usize).checked_add(json_bytes.len()) else {
return -1;
};
let out = match mem.data_mut(&mut caller).get_mut(out_ptr as usize..end) {
Some(s) => s,
None => return -1,
@@ -331,7 +361,15 @@ fn register_hc_state_subscribe(
None => return -1,
}
};
caller.data_mut().subscriptions.push(eid);
if homecore::EntityId::parse(&eid).is_err() {
return -1;
}
if caller.data().subscriptions.len() >= MAX_SUBSCRIPTIONS {
return -2;
}
if !caller.data().subscriptions.contains(&eid) {
caller.data_mut().subscriptions.push(eid);
}
0
},
)
@@ -374,6 +412,7 @@ fn register_hc_log(
///
/// The `Arc<Mutex<_>>` allows the handle to be `Clone` + `Send` while
/// maintaining exclusive access for calls into the WASM module.
#[derive(Clone)]
pub struct WasmPlugin {
pub inner: Arc<Mutex<(Store<PluginStoreData>, wasmtime::Instance)>>,
}
@@ -394,6 +433,9 @@ impl WasmPlugin {
.lock()
.map_err(|e| PluginError::RuntimeError(format!("lock: {e}")))?;
let (store, instance) = &mut *guard;
store
.set_fuel(FUEL_PER_CALL)
.map_err(|e| PluginError::RuntimeError(format!("set call fuel: {e}")))?;
call_export_str(store, instance, "plugin_setup", config_entry_json)
}
@@ -409,20 +451,46 @@ impl WasmPlugin {
.lock()
.map_err(|e| PluginError::RuntimeError(format!("lock: {e}")))?;
let (store, instance) = &mut *guard;
store
.set_fuel(FUEL_PER_CALL)
.map_err(|e| PluginError::RuntimeError(format!("set call fuel: {e}")))?;
call_export_str(store, instance, "plugin_handle_state_changed", &json)
}
/// Call an optional `plugin_teardown() -> i32` export. Older modules that
/// do not export teardown remain compatible; new modules can release
/// guest-owned resources deterministically.
pub fn call_teardown(&self) -> Result<i32, PluginError> {
let mut guard = self
.inner
.lock()
.map_err(|e| PluginError::RuntimeError(format!("lock: {e}")))?;
let (store, instance) = &mut *guard;
store
.set_fuel(FUEL_PER_CALL)
.map_err(|e| PluginError::RuntimeError(format!("set call fuel: {e}")))?;
let Some(func) = instance.get_func(&mut *store, "plugin_teardown") else {
return Ok(0);
};
let func = func.typed::<(), i32>(&*store).map_err(|e| {
PluginError::RuntimeError(format!("plugin_teardown has invalid signature: {e}"))
})?;
func.call(&mut *store, ())
.map_err(|e| PluginError::RuntimeError(format!("call plugin_teardown: {e}")))
}
}
// ── Memory helpers ─────────────────────────────────────────────────────────
/// Read a UTF-8 string from guest linear memory.
fn read_str(mem: &[u8], ptr: i32, len: i32) -> Option<&str> {
if len < 0 || len as usize > MAX_ABI_BUFFER_BYTES {
if ptr < 0 || len < 0 || len as usize > MAX_ABI_BUFFER_BYTES {
return None;
}
let ptr = ptr as usize;
let len = len as usize;
let slice = mem.get(ptr..ptr + len)?;
let end = ptr.checked_add(len)?;
let slice = mem.get(ptr..end)?;
std::str::from_utf8(slice).ok()
}
@@ -435,6 +503,13 @@ fn call_export_str(
payload: &str,
) -> Result<i32, PluginError> {
let payload_bytes = payload.as_bytes().to_vec(); // owned copy avoids reborrow issues
if payload_bytes.len() > MAX_ABI_BUFFER_BYTES {
return Err(PluginError::ResourceLimit(format!(
"ABI payload is {} bytes; maximum is {}",
payload_bytes.len(),
MAX_ABI_BUFFER_BYTES
)));
}
let payload_len = payload_bytes.len() as i32;
// 1. Allocate guest buffer.
@@ -444,15 +519,23 @@ fn call_export_str(
let ptr = alloc
.call(&mut *store, payload_len)
.map_err(|e| PluginError::RuntimeError(format!("call alloc: {e}")))?;
if ptr < 0 {
return Err(PluginError::RuntimeError(
"guest alloc returned a negative pointer".into(),
));
}
// 2. Write payload into guest memory.
{
let mem = instance
.get_memory(&mut *store, "memory")
.ok_or_else(|| PluginError::RuntimeError("no memory export".into()))?;
let end = (ptr as usize)
.checked_add(payload_bytes.len())
.ok_or_else(|| PluginError::RuntimeError("guest allocation overflow".into()))?;
let guest_slice = mem
.data_mut(&mut *store)
.get_mut(ptr as usize..ptr as usize + payload_bytes.len())
.get_mut(ptr as usize..end)
.ok_or_else(|| PluginError::RuntimeError("guest memory OOB".into()))?;
guest_slice.copy_from_slice(&payload_bytes);
}
@@ -476,6 +559,17 @@ fn call_export_str(
Ok(result)
}
fn check_module_size(wasm_bytes: &[u8]) -> Result<(), PluginError> {
if wasm_bytes.len() > MAX_WASM_MODULE_BYTES {
return Err(PluginError::ResourceLimit(format!(
"WASM module is {} bytes; maximum is {}",
wasm_bytes.len(),
MAX_WASM_MODULE_BYTES
)));
}
Ok(())
}
// ── Unit tests (using inline WAT) ──────────────────────────────────────────
#[cfg(test)]
+3
View File
@@ -30,6 +30,8 @@ Data persists in `.homecore/home.db` (by default; configurable). Queries work vi
- **Attribute persistence** — JSON attributes for entities stored in separate table (HA pattern)
- **Automatic deduplication** — skip writes when state hasn't changed (detect via hash)
- **Recorder runs table** — track purge cycles and migration events (HA `recorder_runs` equivalent)
- **Startup restoration** — deterministic newest row per entity, bounded at
100,000 rows, with malformed rows isolated as typed warnings
- **Semantic search** (P2, `--features ruvector`) — embed state attributes + query by meaning
- **HNSW index** (P2) — k-NN search for "all warm rooms" via ruvector
- **No data export overhead** — SQLite is queryable directly; no proprietary format
@@ -41,6 +43,7 @@ Data persists in `.homecore/home.db` (by default; configurable). Queries work vi
| Record state change | Listener | `RecorderListener::on_state_changed(event)` | Fires on homecore event bus; writes to SQLite |
| Query state history | SQL | `SELECT * FROM states WHERE entity_id = ? ORDER BY last_changed DESC` | Standard SQLite; can be queried from anywhere |
| Purge old states | Maintenance | `Recorder::purge(older_than)` | Deletes states older than specified timestamp |
| Restore latest states | Startup | `Recorder::restore_latest(states, limit)` | Entity-id ordered, bounded, malformed-row isolation |
| Deduplicate write | Dedup | `DedupEngine::should_record(old_state, new_state)` | Skip if state hash unchanged |
| Create semantic index | Index | `SemanticIndex::index_state(entity_id, state)` (P2, opt-in) | Hash-based embeddings; real embeddings in P3 |
| Search by meaning | Search | `SemanticIndex::search(query, k)` (P2, opt-in) | "warm rooms" → k-NN search in ruvector HNSW |
+525 -87
View File
@@ -20,11 +20,24 @@ use tokio::sync::RwLock;
use tracing::debug;
use homecore::entity::{EntityId, State};
use homecore::event::{DomainEvent, StateChangedEvent};
use homecore::event::{Context, DomainEvent, StateChangedEvent};
use homecore::StateMachine;
use crate::dedup::fnv64a_hash;
use crate::schema::ALL_DDL;
type SearchStateRecord = (
i64,
String,
String,
Option<String>,
f64,
f64,
Option<String>,
);
type StateRecord = (String, String, Option<String>, f64, f64, Option<String>);
type HistoryStateRecord = (i64, String, Option<String>, f64, f64, Option<String>);
/// Hard upper bound on rows returned by [`Recorder::get_state_history`].
///
/// Without this cap a wide `[since, until]` window over a high-frequency entity
@@ -33,6 +46,8 @@ use crate::schema::ALL_DDL;
/// history-graph query, small enough to bound the worst case. Callers needing a
/// wider span page by narrowing the window.
pub const MAX_HISTORY_ROWS: i64 = 1_000_000;
/// Absolute cap for one startup restore query.
pub const MAX_RESTORE_STATES: usize = 100_000;
/// Errors returned by `Recorder` operations.
#[derive(Error, Debug)]
@@ -289,9 +304,8 @@ impl Recorder {
.replace('_', "\\_");
let pattern = format!("%{escaped}%");
let rows: Vec<(i64, String, String, Option<String>, f64, f64, Option<String>)> =
sqlx::query_as(
"SELECT s.state_id, s.entity_id, s.state, sa.shared_attrs, \
let rows: Vec<SearchStateRecord> = sqlx::query_as(
"SELECT s.state_id, s.entity_id, s.state, sa.shared_attrs, \
s.last_changed_ts, s.last_updated_ts, s.context_id \
FROM states s \
LEFT JOIN state_attributes sa ON s.attributes_id = sa.attributes_id \
@@ -301,48 +315,57 @@ impl Recorder {
OR sa.shared_attrs LIKE ?2 ESCAPE '\\' \
ORDER BY s.last_updated_ts DESC \
LIMIT ?3",
)
.bind(query)
.bind(&pattern)
.bind(k as i64)
.fetch_all(&self.pool)
.await?;
)
.bind(query)
.bind(&pattern)
.bind(k as i64)
.fetch_all(&self.pool)
.await?;
rows.into_iter()
.map(|(state_id, entity_id, state, shared_attrs, last_changed_ts, last_updated_ts, context_id)| {
let eid = EntityId::parse(&entity_id)
.unwrap_or_else(|_| EntityId::parse("unknown.unknown").unwrap());
let attributes = shared_attrs
.as_deref()
.map(serde_json::from_str)
.transpose()?
.unwrap_or(serde_json::Value::Object(Default::default()));
Ok(StateRow {
.map(
|(
state_id,
entity_id: eid,
entity_id,
state,
attributes,
shared_attrs,
last_changed_ts,
last_updated_ts,
context_id,
})
})
)| {
let eid = EntityId::parse(&entity_id)
.unwrap_or_else(|_| EntityId::parse("unknown.unknown").unwrap());
let attributes = shared_attrs
.as_deref()
.map(serde_json::from_str)
.transpose()?
.unwrap_or(serde_json::Value::Object(Default::default()));
Ok(StateRow {
state_id,
entity_id: eid,
state,
attributes,
last_changed_ts,
last_updated_ts,
context_id,
})
},
)
.collect()
}
/// Fetch a single `StateRow` by its `state_id`, joining attributes.
async fn fetch_state_row(&self, state_id: i64) -> Result<Option<StateRow>, RecorderError> {
let row: Option<(String, String, Option<String>, f64, f64, Option<String>)> =
sqlx::query_as(
"SELECT s.entity_id, s.state, sa.shared_attrs, \
let row: Option<StateRecord> = sqlx::query_as(
"SELECT s.entity_id, s.state, sa.shared_attrs, \
s.last_changed_ts, s.last_updated_ts, s.context_id \
FROM states s \
LEFT JOIN state_attributes sa ON s.attributes_id = sa.attributes_id \
WHERE s.state_id = ?",
)
.bind(state_id)
.fetch_optional(&self.pool)
.await?;
)
.bind(state_id)
.fetch_optional(&self.pool)
.await?;
let Some((entity_id, state, shared_attrs, last_changed_ts, last_updated_ts, context_id)) =
row
@@ -406,10 +429,28 @@ impl Recorder {
since: DateTime<Utc>,
until: DateTime<Utc>,
) -> Result<Vec<StateRow>, RecorderError> {
self.get_state_history_limited(entity_id, since, until, MAX_HISTORY_ROWS as usize)
.await
}
/// Query state history with a caller-selected limit capped by
/// [`MAX_HISTORY_ROWS`]. The bound is applied in SQL, before rows are
/// materialized.
pub async fn get_state_history_limited(
&self,
entity_id: &EntityId,
since: DateTime<Utc>,
until: DateTime<Utc>,
requested_limit: usize,
) -> Result<Vec<StateRow>, RecorderError> {
let limit = requested_limit.min(MAX_HISTORY_ROWS as usize);
if limit == 0 {
return Ok(Vec::new());
}
let since_ts = since.timestamp_micros() as f64 / 1_000_000.0;
let until_ts = until.timestamp_micros() as f64 / 1_000_000.0;
let rows: Vec<(i64, String, Option<String>, f64, f64, Option<String>)> = sqlx::query_as(
let rows: Vec<HistoryStateRecord> = sqlx::query_as(
"SELECT s.state_id, s.state, sa.shared_attrs, \
s.last_changed_ts, s.last_updated_ts, s.context_id \
FROM states s \
@@ -423,31 +464,189 @@ impl Recorder {
.bind(entity_id.as_str())
.bind(since_ts)
.bind(until_ts)
.bind(MAX_HISTORY_ROWS)
.bind(limit as i64)
.fetch_all(&self.pool)
.await?;
rows.into_iter()
.map(|(state_id, state, shared_attrs, last_changed_ts, last_updated_ts, context_id)| {
let attributes = shared_attrs
.as_deref()
.map(serde_json::from_str)
.transpose()?
.unwrap_or(serde_json::Value::Object(Default::default()));
.map(
|(state_id, state, shared_attrs, last_changed_ts, last_updated_ts, context_id)| {
let attributes = shared_attrs
.as_deref()
.map(serde_json::from_str)
.transpose()?
.unwrap_or(serde_json::Value::Object(Default::default()));
Ok(StateRow {
state_id,
entity_id: entity_id.clone(),
state,
attributes,
last_changed_ts,
last_updated_ts,
context_id,
})
})
Ok(StateRow {
state_id,
entity_id: entity_id.clone(),
state,
attributes,
last_changed_ts,
last_updated_ts,
context_id,
})
},
)
.collect()
}
/// Read the newest row for each entity in deterministic entity-id order.
///
/// The query is bounded and uses `(last_updated_ts, state_id)` as a stable
/// newest-row tie-break. Malformed rows are reported and skipped
/// individually so one corrupt entity cannot prevent the rest from
/// starting.
pub async fn latest_states(
&self,
requested_limit: usize,
) -> Result<LatestStates, RecorderError> {
let limit = requested_limit.min(MAX_RESTORE_STATES);
if limit == 0 {
return Ok(LatestStates::default());
}
type RawRestoreRow = (
i64,
String,
Option<String>,
Option<String>,
Option<f64>,
Option<f64>,
Option<String>,
);
let rows: Vec<RawRestoreRow> = sqlx::query_as(
"SELECT state_id, entity_id, state, shared_attrs, \
last_changed_ts, last_updated_ts, context_id \
FROM ( \
SELECT s.state_id, s.entity_id, s.state, sa.shared_attrs, \
s.last_changed_ts, s.last_updated_ts, s.context_id, \
ROW_NUMBER() OVER ( \
PARTITION BY s.entity_id \
ORDER BY s.last_updated_ts DESC, s.state_id DESC \
) AS newest \
FROM states s \
LEFT JOIN state_attributes sa ON s.attributes_id = sa.attributes_id \
) \
WHERE newest = 1 \
ORDER BY entity_id ASC \
LIMIT ?",
)
.bind((limit + 1) as i64)
.fetch_all(&self.pool)
.await?;
let truncated = rows.len() > limit;
let mut states = Vec::with_capacity(rows.len().min(limit));
let mut warnings = Vec::new();
for (
state_id,
raw_entity_id,
raw_state,
raw_attributes,
last_changed_ts,
last_updated_ts,
context_id,
) in rows.into_iter().take(limit)
{
let entity_id = match EntityId::parse(&raw_entity_id) {
Ok(value) => value,
Err(error) => {
warnings.push(RestoreWarning::MalformedEntityId {
state_id,
entity_id: raw_entity_id,
reason: error.to_string(),
});
continue;
}
};
let Some(state) = raw_state else {
warnings.push(RestoreWarning::MissingState {
state_id,
entity_id: raw_entity_id,
});
continue;
};
let attributes = match raw_attributes {
Some(value) => match serde_json::from_str(&value) {
Ok(value) => value,
Err(error) => {
warnings.push(RestoreWarning::MalformedAttributes {
state_id,
entity_id: raw_entity_id,
reason: error.to_string(),
});
continue;
}
},
None => serde_json::json!({}),
};
let Some(last_changed) = last_changed_ts.and_then(timestamp_from_seconds) else {
warnings.push(RestoreWarning::MalformedTimestamp {
state_id,
entity_id: raw_entity_id,
field: "last_changed_ts",
});
continue;
};
let Some(last_updated) = last_updated_ts.and_then(timestamp_from_seconds) else {
warnings.push(RestoreWarning::MalformedTimestamp {
state_id,
entity_id: raw_entity_id,
field: "last_updated_ts",
});
continue;
};
let parent_id = match context_id {
Some(value) => match value.parse() {
Ok(value) => Some(value),
Err(_) => {
warnings.push(RestoreWarning::MalformedContext {
state_id,
entity_id: raw_entity_id.clone(),
});
None
}
},
None => None,
};
states.push(State {
entity_id,
state,
attributes,
last_changed,
last_updated,
context: Context::restoration(parent_id),
});
}
Ok(LatestStates {
states,
warnings,
truncated,
})
}
/// Load latest durable snapshots into a state machine without producing
/// fresh recorder events.
pub async fn restore_latest(
&self,
states: &StateMachine,
limit: usize,
) -> Result<RestoreReport, RecorderError> {
let batch = self.latest_states(limit).await?;
let mut restored = 0;
for state in batch.states {
// latest_states constructs the required restoration marker.
if states.restore(state).is_ok() {
restored += 1;
}
}
Ok(RestoreReport {
restored,
warnings: batch.warnings,
truncated: batch.truncated,
})
}
/// Purge history older than `older_than`, returning a [`PurgeStats`] summary.
///
/// Deletes:
@@ -511,6 +710,58 @@ impl Recorder {
}
}
fn timestamp_from_seconds(value: f64) -> Option<DateTime<Utc>> {
if !value.is_finite() {
return None;
}
let micros = (value * 1_000_000.0).round();
if micros < i64::MIN as f64 || micros > i64::MAX as f64 {
return None;
}
DateTime::from_timestamp_micros(micros as i64)
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum RestoreWarning {
MalformedEntityId {
state_id: i64,
entity_id: String,
reason: String,
},
MissingState {
state_id: i64,
entity_id: String,
},
MalformedAttributes {
state_id: i64,
entity_id: String,
reason: String,
},
MalformedTimestamp {
state_id: i64,
entity_id: String,
field: &'static str,
},
MalformedContext {
state_id: i64,
entity_id: String,
},
}
#[derive(Debug, Default)]
pub struct LatestStates {
pub states: Vec<State>,
pub warnings: Vec<RestoreWarning>,
pub truncated: bool,
}
#[derive(Debug)]
pub struct RestoreReport {
pub restored: usize,
pub warnings: Vec<RestoreWarning>,
pub truncated: bool,
}
/// Summary of a [`Recorder::purge`] run.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct PurgeStats {
@@ -554,14 +805,20 @@ mod tests {
use super::*;
async fn open_memory() -> Recorder {
Recorder::open("sqlite::memory:").await.expect("open in-memory DB")
Recorder::open("sqlite::memory:")
.await
.expect("open in-memory DB")
}
fn entity(s: &str) -> EntityId {
EntityId::parse(s).unwrap()
}
fn make_state_event(entity_id: &str, state_val: &str, attrs: serde_json::Value) -> StateChangedEvent {
fn make_state_event(
entity_id: &str,
state_val: &str,
attrs: serde_json::Value,
) -> StateChangedEvent {
let eid = entity(entity_id);
let ctx = Context::new();
let s = Arc::new(State::new(eid.clone(), state_val, attrs, ctx));
@@ -585,7 +842,10 @@ mod tests {
.await
.unwrap();
let names: Vec<&str> = tables.iter().map(|(n,)| n.as_str()).collect();
assert!(names.contains(&"state_attributes"), "missing state_attributes");
assert!(
names.contains(&"state_attributes"),
"missing state_attributes"
);
assert!(names.contains(&"states"), "missing states");
assert!(names.contains(&"events"), "missing events");
assert!(names.contains(&"recorder_runs"), "missing recorder_runs");
@@ -595,7 +855,10 @@ mod tests {
async fn schema_idempotent_double_open() {
// Applying schema twice (on the same pool) must not panic or error.
let recorder = open_memory().await;
recorder.apply_schema().await.expect("second apply_schema must be a no-op");
recorder
.apply_schema()
.await
.expect("second apply_schema must be a no-op");
}
// ── record_state ──────────────────────────────────────────────────────────
@@ -603,7 +866,11 @@ mod tests {
#[tokio::test]
async fn record_state_inserts_row() {
let recorder = open_memory().await;
let event = make_state_event("light.kitchen", "on", serde_json::json!({"brightness": 200}));
let event = make_state_event(
"light.kitchen",
"on",
serde_json::json!({"brightness": 200}),
);
let state_id = recorder.record_state(&event).await.unwrap();
assert!(state_id.is_some(), "expected a state_id");
@@ -642,19 +909,20 @@ mod tests {
recorder.record_state(&e1).await.unwrap();
recorder.record_state(&e2).await.unwrap();
let attr_count: (i64,) =
sqlx::query_as("SELECT COUNT(*) FROM state_attributes")
.fetch_one(&recorder.pool)
.await
.unwrap();
let attr_count: (i64,) = sqlx::query_as("SELECT COUNT(*) FROM state_attributes")
.fetch_one(&recorder.pool)
.await
.unwrap();
// Both events share identical attrs → only one state_attributes row.
assert_eq!(attr_count.0, 1, "identical attrs must share one state_attributes row");
assert_eq!(
attr_count.0, 1,
"identical attrs must share one state_attributes row"
);
let state_count: (i64,) =
sqlx::query_as("SELECT COUNT(*) FROM states")
.fetch_one(&recorder.pool)
.await
.unwrap();
let state_count: (i64,) = sqlx::query_as("SELECT COUNT(*) FROM states")
.fetch_one(&recorder.pool)
.await
.unwrap();
assert_eq!(state_count.0, 2, "two states rows expected");
}
@@ -667,11 +935,10 @@ mod tests {
recorder.record_state(&e1).await.unwrap();
recorder.record_state(&e2).await.unwrap();
let attr_count: (i64,) =
sqlx::query_as("SELECT COUNT(*) FROM state_attributes")
.fetch_one(&recorder.pool)
.await
.unwrap();
let attr_count: (i64,) = sqlx::query_as("SELECT COUNT(*) FROM state_attributes")
.fetch_one(&recorder.pool)
.await
.unwrap();
assert_eq!(attr_count.0, 2);
}
@@ -691,7 +958,10 @@ mod tests {
let since = Utc::now() - chrono::Duration::seconds(10);
let until = Utc::now() + chrono::Duration::seconds(10);
let rows = recorder.get_state_history(&eid, since, until).await.unwrap();
let rows = recorder
.get_state_history(&eid, since, until)
.await
.unwrap();
assert_eq!(rows.len(), 3, "expected 3 history rows");
// Verify ascending order by last_updated_ts.
@@ -705,6 +975,36 @@ mod tests {
assert_eq!(rows[2].state, "22.0");
}
#[tokio::test]
async fn history_caller_limit_is_applied_before_materialization() {
let recorder = open_memory().await;
let eid = entity("sensor.bounded");
for value in ["1", "2", "3"] {
recorder
.record_state(&make_state_event(
"sensor.bounded",
value,
serde_json::json!({}),
))
.await
.unwrap();
}
let since = Utc::now() - chrono::Duration::seconds(10);
let until = Utc::now() + chrono::Duration::seconds(10);
let rows = recorder
.get_state_history_limited(&eid, since, until, 2)
.await
.unwrap();
assert_eq!(rows.len(), 2);
assert_eq!(rows[0].state, "1");
assert_eq!(rows[1].state, "2");
assert!(recorder
.get_state_history_limited(&eid, since, until, 0)
.await
.unwrap()
.is_empty());
}
// ── record_event ──────────────────────────────────────────────────────────
#[tokio::test]
@@ -739,11 +1039,19 @@ mod tests {
// FAILS against the old always-empty path: asserts real rows come back.
let recorder = open_memory().await;
recorder
.record_state(&make_state_event("light.kitchen", "on", serde_json::json!({})))
.record_state(&make_state_event(
"light.kitchen",
"on",
serde_json::json!({}),
))
.await
.unwrap();
recorder
.record_state(&make_state_event("light.bedroom", "off", serde_json::json!({})))
.record_state(&make_state_event(
"light.bedroom",
"off",
serde_json::json!({}),
))
.await
.unwrap();
recorder
@@ -779,7 +1087,10 @@ mod tests {
))
.await
.unwrap();
let rows = recorder.search_states_by_text("portland", 10).await.unwrap();
let rows = recorder
.search_states_by_text("portland", 10)
.await
.unwrap();
assert_eq!(rows.len(), 1);
assert_eq!(rows[0].entity_id.as_str(), "sensor.weather");
assert_eq!(rows[0].attributes["location"], "portland");
@@ -806,7 +1117,11 @@ mod tests {
async fn text_search_no_match_returns_empty() {
let recorder = open_memory().await;
recorder
.record_state(&make_state_event("light.kitchen", "on", serde_json::json!({})))
.record_state(&make_state_event(
"light.kitchen",
"on",
serde_json::json!({}),
))
.await
.unwrap();
let rows = recorder
@@ -829,7 +1144,11 @@ mod tests {
// EntityId::parse permits this, so it reaches the bind path as data.
let evil = "light.x_drop_table_states_select";
recorder
.record_state(&make_state_event(evil, "'; DROP TABLE states; --", serde_json::json!({})))
.record_state(&make_state_event(
evil,
"'; DROP TABLE states; --",
serde_json::json!({}),
))
.await
.unwrap();
@@ -884,11 +1203,14 @@ mod tests {
// public method (whose cap is MAX_HISTORY_ROWS) and run the *same* SQL
// shape with a small bind to demonstrate the LIMIT term is effective —
// and separately assert the constant is a sane positive bound.
assert!(MAX_HISTORY_ROWS > 0, "history cap must be positive");
let recorder = open_memory().await;
for v in &["1", "2", "3", "4", "5"] {
recorder
.record_state(&make_state_event("sensor.bounded", v, serde_json::json!({})))
.record_state(&make_state_event(
"sensor.bounded",
v,
serde_json::json!({}),
))
.await
.unwrap();
tokio::time::sleep(std::time::Duration::from_millis(2)).await;
@@ -905,12 +1227,20 @@ mod tests {
.fetch_all(&recorder.pool)
.await
.unwrap();
assert_eq!(capped.len(), 2, "LIMIT term effectively bounds the result set");
assert_eq!(
capped.len(),
2,
"LIMIT term effectively bounds the result set"
);
// And the real method returns all rows when under the cap.
let eid = entity("sensor.bounded");
let rows = recorder
.get_state_history(&eid, Utc::now() - chrono::Duration::seconds(10), Utc::now() + chrono::Duration::seconds(10))
.get_state_history(
&eid,
Utc::now() - chrono::Duration::seconds(10),
Utc::now() + chrono::Duration::seconds(10),
)
.await
.unwrap();
assert_eq!(rows.len(), 5, "all rows under the cap return");
@@ -938,7 +1268,10 @@ mod tests {
// Read back the actual timestamps so the cutoff is exact.
let since = Utc::now() - chrono::Duration::seconds(60);
let until = Utc::now() + chrono::Duration::seconds(60);
let all = recorder.get_state_history(&eid, since, until).await.unwrap();
let all = recorder
.get_state_history(&eid, since, until)
.await
.unwrap();
assert_eq!(all.len(), 3);
// Cut off exactly at the middle row's timestamp.
let mid_ts = all[1].last_updated_ts;
@@ -947,8 +1280,15 @@ mod tests {
let stats = recorder.purge(cutoff).await.unwrap();
assert_eq!(stats.states_deleted, 1, "only the strictly-older 'old' row");
let remaining = recorder.get_state_history(&eid, since, until).await.unwrap();
assert_eq!(remaining.len(), 2, "boundary 'mid' row is KEPT (exclusive cutoff)");
let remaining = recorder
.get_state_history(&eid, since, until)
.await
.unwrap();
assert_eq!(
remaining.len(),
2,
"boundary 'mid' row is KEPT (exclusive cutoff)"
);
assert_eq!(remaining[0].state, "mid");
assert_eq!(remaining[1].state, "new");
}
@@ -986,18 +1326,28 @@ mod tests {
// referenced by sensor.b, so it must survive.
let eid_b = entity("sensor.b");
let rows_b = recorder
.get_state_history(&eid_b, Utc::now() - chrono::Duration::seconds(60), Utc::now() + chrono::Duration::seconds(60))
.get_state_history(
&eid_b,
Utc::now() - chrono::Duration::seconds(60),
Utc::now() + chrono::Duration::seconds(60),
)
.await
.unwrap();
let b_ts = rows_b[0].last_updated_ts;
let cutoff = DateTime::<Utc>::from_timestamp_micros((b_ts * 1_000_000.0) as i64).unwrap();
let stats = recorder.purge(cutoff).await.unwrap();
assert_eq!(stats.states_deleted, 1, "sensor.a purged");
assert_eq!(stats.attributes_deleted, 0, "shared blob still referenced — kept");
assert_eq!(
stats.attributes_deleted, 0,
"shared blob still referenced — kept"
);
assert_eq!(attr_count(&recorder).await, 1, "blob survives");
// Now purge everything → sensor.b gone, blob orphaned → GC'd.
let stats2 = recorder.purge(Utc::now() + chrono::Duration::seconds(120)).await.unwrap();
let stats2 = recorder
.purge(Utc::now() + chrono::Duration::seconds(120))
.await
.unwrap();
assert_eq!(stats2.states_deleted, 1, "sensor.b purged");
assert_eq!(stats2.attributes_deleted, 1, "now-orphaned blob GC'd");
assert_eq!(attr_count(&recorder).await, 0, "no blobs remain");
@@ -1008,7 +1358,11 @@ mod tests {
let recorder = open_memory().await;
let ctx = Context::new();
recorder
.record_event(&DomainEvent::new("call_service", serde_json::json!({}), ctx))
.record_event(&DomainEvent::new(
"call_service",
serde_json::json!({}),
ctx,
))
.await
.unwrap();
// Purge with a far-future cutoff removes the event.
@@ -1030,11 +1384,95 @@ mod tests {
// real rows via the text fallback — proving it's no longer always-empty.
let recorder = open_memory().await;
recorder
.record_state(&make_state_event("light.kitchen", "on", serde_json::json!({})))
.record_state(&make_state_event(
"light.kitchen",
"on",
serde_json::json!({}),
))
.await
.unwrap();
let rows = recorder.search_semantic("kitchen", 5).await.unwrap();
assert_eq!(rows.len(), 1, "fallback must surface the kitchen row");
assert_eq!(rows[0].entity_id.as_str(), "light.kitchen");
}
#[tokio::test]
async fn latest_states_is_deterministic_newest_per_entity() {
let recorder = open_memory().await;
recorder
.record_state(&make_state_event("sensor.z", "old", serde_json::json!({})))
.await
.unwrap();
recorder
.record_state(&make_state_event("sensor.a", "only", serde_json::json!({})))
.await
.unwrap();
recorder
.record_state(&make_state_event("sensor.z", "new", serde_json::json!({})))
.await
.unwrap();
let batch = recorder.latest_states(10).await.unwrap();
assert_eq!(batch.states.len(), 2);
assert_eq!(batch.states[0].entity_id.as_str(), "sensor.a");
assert_eq!(batch.states[1].entity_id.as_str(), "sensor.z");
assert_eq!(batch.states[1].state, "new");
assert!(batch
.states
.iter()
.all(|state| state.context.is_restoration()));
}
#[tokio::test]
async fn latest_states_isolates_malformed_rows_and_honours_bound() {
let recorder = open_memory().await;
recorder
.record_state(&make_state_event(
"sensor.good",
"ok",
serde_json::json!({"x": 1}),
))
.await
.unwrap();
sqlx::query(
"INSERT INTO states \
(entity_id, state, attributes_id, last_changed_ts, last_updated_ts, context_id) \
VALUES ('INVALID', 'bad', NULL, 1.0, 2.0, NULL)",
)
.execute(&recorder.pool)
.await
.unwrap();
let attrs_id = sqlx::query(
"INSERT INTO state_attributes (shared_attrs, hash) VALUES ('not-json', 424242)",
)
.execute(&recorder.pool)
.await
.unwrap()
.last_insert_rowid();
sqlx::query(
"INSERT INTO states \
(entity_id, state, attributes_id, last_changed_ts, last_updated_ts, context_id) \
VALUES ('sensor.badattrs', 'bad', ?, 1.0, 2.0, NULL)",
)
.bind(attrs_id)
.execute(&recorder.pool)
.await
.unwrap();
let batch = recorder.latest_states(10).await.unwrap();
assert_eq!(batch.states.len(), 1);
assert_eq!(batch.warnings.len(), 2);
assert!(batch
.warnings
.iter()
.any(|warning| matches!(warning, RestoreWarning::MalformedEntityId { .. })));
assert!(batch
.warnings
.iter()
.any(|warning| matches!(warning, RestoreWarning::MalformedAttributes { .. })));
let bounded = recorder.latest_states(1).await.unwrap();
assert!(bounded.truncated);
assert!(bounded.states.len() <= 1);
}
}
+7 -1
View File
@@ -30,9 +30,15 @@ pub mod schema;
pub mod semantic;
// Re-export the primary public API surface.
pub use db::{PurgeStats, Recorder, RecorderError, StateRow, MAX_HISTORY_ROWS};
pub use db::{
LatestStates, PurgeStats, Recorder, RecorderError, RestoreReport, RestoreWarning,
SemanticIndex, StateRow, MAX_HISTORY_ROWS, MAX_RESTORE_STATES,
};
pub use listener::RecorderListener;
/// Null semantic index used when the `ruvector` feature is off.
/// Satisfies the [`db::SemanticIndex`] trait bound without any allocation.
pub use db::NullSemanticIndex;
#[cfg(feature = "ruvector")]
pub use semantic::RuvectorSemanticIndex;
+8 -3
View File
@@ -23,12 +23,13 @@ path = "src/main.rs"
# The 8 HOMECORE crates this binary integrates
homecore = { path = "../homecore", version = "0.1.0-alpha.0" }
homecore-api = { path = "../homecore-api", version = "0.1.0-alpha.0" }
homecore-plugins = { path = "../homecore-plugins", version = "0.1.0-alpha.0" }
homecore-hap = { path = "../homecore-hap", version = "0.1.0-alpha.0" }
homecore-plugins = { path = "../homecore-plugins", version = "0.1.0-alpha.0" }
homecore-recorder = { path = "../homecore-recorder", version = "0.1.0-alpha.0" }
# Reuse version-gated registry envelope parsing in the isolated restore module.
homecore-migrate = { path = "../homecore-migrate", version = "0.1.0-alpha.0" }
homecore-automation = { path = "../homecore-automation", version = "0.1.0-alpha.0" }
homecore-assist = { path = "../homecore-assist", version = "0.1.0-alpha.0" }
# Migration crate is CLI-only; not linked here.
homecore-hap = { path = "../homecore-hap", version = "0.1.0-alpha.0", optional = true }
tokio = { version = "1", features = ["full"] }
tracing = "0.1"
@@ -50,6 +51,7 @@ tower-http = { version = "0.6", features = ["fs", "trace", "cors"] }
# CHANGELOG / ADR-131 security note).
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
serde = { version = "1", features = ["derive"] }
serde_yaml = "0.9"
# Concurrent fan-out of per-bank RoomState fetches in the gateway (§11 perf).
futures = "0.3"
@@ -57,6 +59,7 @@ futures = "0.3"
# Drive the assembled router in integration tests via ServiceExt::oneshot.
tower = { version = "0.5", features = ["util"] }
http-body-util = "0.1"
tempfile = "3"
[features]
default = []
@@ -64,3 +67,5 @@ default = []
ruvector = ["homecore-recorder/ruvector"]
# Pull in real Wasmtime plugin runtime (vs InProcessRuntime).
wasmtime = ["homecore-plugins/wasmtime"]
# Bind the HAP TCP listener and publish `_hap._tcp` over mDNS.
hap-server = ["dep:homecore-hap", "homecore-hap/hap-server"]
+64 -180
View File
@@ -1,204 +1,88 @@
# homecore-server
Integrated HOMECORE server binary that wires state machine, API, recorder, plugins, automations, intent assistant, and HomeKit bridge into one process.
`homecore-server` is the alpha integration binary for HOMECORE. It runs one
shared state machine, event bus, service registry, REST/WebSocket API, optional
SQLite recorder, automation engine, intent endpoint, BFF gateway, and static
dashboard.
[![Crates.io](https://img.shields.io/badge/crates.io-workspace%20binary-inactive)](.)
![License](https://img.shields.io/badge/license-MIT-blue.svg)
![MSRV: 1.89+](https://img.shields.io/badge/MSRV-1.89%2B-purple.svg)
[![ADR-126](https://img.shields.io/badge/ADR-126-orange.svg)](../../docs/adr/ADR-126-homecore-home-assistant-port.md)
It is not a drop-in replacement for the complete Home Assistant API. The exact
implemented and deferred surfaces are listed in
[`docs/homecore-capabilities.md`](../../docs/homecore-capabilities.md).
The production-ready HOMECORE binary — boots all 7 subsystems (core, API, recorder, plugins, automation, assist, HAP bridge) in a single process listening on `:8123`.
## Security defaults
## What this crate does
`homecore-server` is the integration point for the entire HOMECORE ecosystem. It orchestrates:
1. **HomeCore runtime** — state machine, event bus, service registry
2. **REST + WebSocket API** — Axum server on `:8123` (HA-compatible)
3. **SQLite Recorder** — persists all state changes to disk
4. **Plugin Registry** — loads and manages integrations (InProcessRuntime by default)
5. **Automation Engine** — evaluates triggers, conditions, and actions
6. **Assist Pipeline** — intent recognition and execution
7. **HAP Bridge** — exposes accessories to HomeKit
All subsystems share the same `HomeCore` instance, so state changes flow through the event bus and trigger automations, record history, and notify WebSocket subscribers in lockstep.
## Features
- **Single unified process** — no external microservices; run with `cargo run -p homecore-server`
- **HA-compatible REST API** — drop-in replacement for Home Assistant's `/api/` on `:8123`
- **SQLite state history** — persistent recording of all state changes
- **Automation engine** — YAML-driven trigger→condition→action execution
- **Intent assistant** — regex-based (P1) intent recognition + service calling
- **HomeKit bridge** — exposes HOMECORE entities as HomeKit accessories
- **Plugin system** — load first-party Rust plugins; Wasmtime WASM plugins (P2, `--features wasmtime`)
- **Configurable via CLI + env vars** — no YAML required; sensible defaults
- **Structured logging** — tracing output with `RUST_LOG` filtering
- **Feature-gated subsystems** — disable recorder (`--no-recorder`), enable ruvector/wasmtime as needed
## Subsystems
| Subsystem | Crate | Role | Notes |
|-----------|-------|------|-------|
| State Machine | `homecore` | Core domain model | All other subsystems depend on this |
| REST API | `homecore-api` | HTTP boundary | Listens on `:8123`; Axum framework |
| Recorder | `homecore-recorder` | Persistence | SQLite; optional `--no-recorder` |
| Plugins | `homecore-plugins` | Extension system | InProcessRuntime default; Wasmtime w/ feature |
| Automation | `homecore-automation` | Trigger execution | Subscribes to event bus; YAML-driven |
| Assist | `homecore-assist` | Intent pipeline | Regex recognizer (P1); semantic (P2) |
| HAP Bridge | `homecore-hap` | HomeKit export | Accessories + characteristics; mDNS (P2) |
## Usage
**Basic startup** (in-memory recorder):
Authentication fails closed. Set one or more comma-separated bearer tokens:
```bash
cargo build -p homecore-server
./target/debug/homecore-server
# Listens on http://localhost:8123
set HOMECORE_TOKENS=replace-with-a-long-random-token
cargo run -p homecore-server
```
**With persistent SQLite**:
`--insecure-dev-auth` explicitly allows any non-empty bearer token and must only
be used in an isolated development environment. The browser UI does not contain
a default token. Configure allowed browser origins with
`HOMECORE_CORS_ORIGINS`.
## Runtime behavior
- SQLite recording is enabled by default at `sqlite://homecore.db`.
- Entity/device registries are restored from `.homecore/storage` before
recorder states, listeners, automations, and API startup.
- The latest recorder state for each entity is restored deterministically.
Restored snapshots retain their timestamps and carry a `homecore.restore`
context marker; malformed rows are logged and isolated.
- Synthetic entities are disabled by default; opt in with
`--seed-demo-entities`.
- Automations can be loaded with `--automations <file>` or
`HOMECORE_AUTOMATIONS`.
- `Ctrl-C` initiates graceful HTTP shutdown and emits `HomeCoreStop`.
- The `ruvector` feature enables the recorder's semantic index.
- Plugin and HAP crates are libraries in this workspace, but this binary does
not claim to load plugins or advertise a HomeKit server yet.
## API
All API requests require `Authorization: Bearer <token>`.
```bash
./target/debug/homecore-server \
--bind 0.0.0.0:8123 \
--db sqlite:~/.homecore/home.db \
--location-name "My Home"
```
curl -H "Authorization: Bearer $HOMECORE_TOKEN" \
http://127.0.0.1:8123/api/states
**Full feature build** (ruvector semantic search + Wasmtime plugins):
```bash
cargo build -p homecore-server --features ruvector,wasmtime --release
```
**Via Docker** (Dockerfile planned P2):
```bash
docker run -p 8123:8123 \
-e HOMECORE_DB=sqlite:///data/home.db \
-v ~/.homecore:/data \
homecore-server:latest
```
**Test the API**:
```bash
# List all entities
curl http://localhost:8123/api/states
# Set a light to "on"
curl -X POST \
-H "Authorization: Bearer $HOMECORE_TOKEN" \
-H "Content-Type: application/json" \
-d '{"state":"on","attributes":{"brightness":200}}' \
http://localhost:8123/api/states/light.kitchen
-d '{"entity_id":"light.kitchen"}' \
http://127.0.0.1:8123/api/services/light/turn_on
# WebSocket subscription (real-time state changes)
wscat -c ws://localhost:8123/api/websocket
curl -X POST \
-H "Authorization: Bearer $HOMECORE_TOKEN" \
-H "Content-Type: application/json" \
-d '{"utterance":"turn on light.kitchen","language":"en"}' \
http://127.0.0.1:8123/api/intent/handle
```
**Configuration via env**:
Built-in executable services are `homecore.ping`,
`homecore.snapshot_state`, and `turn_on`/`turn_off`/`toggle` for the
`homeassistant`, `light`, and `switch` domains. Unsupported services are left
unregistered and return an error rather than a false acknowledgement.
```bash
export HOMECORE_BIND="0.0.0.0:8123"
export HOMECORE_DB="sqlite:~/.homecore/home.db"
export HOMECORE_LOCATION="Living Room"
export RUST_LOG="homecore=debug,homecore_api=info"
./target/debug/homecore-server
```
## Configuration
## CLI Options
| Flag | Environment | Default |
|---|---|---|
| `--bind` | `HOMECORE_BIND` | `0.0.0.0:8123` |
| `--db` | `HOMECORE_DB` | `sqlite://homecore.db` |
| `--storage-dir` | `HOMECORE_STORAGE_DIR` | `.homecore/storage` |
| `--restore-limit` | `HOMECORE_RESTORE_LIMIT` | `100000` |
| `--location-name` | `HOMECORE_LOCATION` | `Home` |
| `--automations` | `HOMECORE_AUTOMATIONS` | unset |
| `--insecure-dev-auth` | `HOMECORE_INSECURE_DEV_AUTH` | `false` |
| `--seed-demo-entities` | — | `false` |
| `--no-recorder` | — | `false` |
| Flag | Env Var | Default | Description |
|------|---------|---------|-------------|
| `--bind` | `HOMECORE_BIND` | `0.0.0.0:8123` | REST API listen address |
| `--db` | `HOMECORE_DB` | `sqlite::memory:` | SQLite path (`:memory:` for ephemeral) |
| `--location-name` | `HOMECORE_LOCATION` | `Home` | Friendly name returned by `/api/config` |
| `--no-recorder` | — | off | Disable SQLite recorder (low-resource deployments) |
| `--ui-dir` | `HOMECORE_UI_DIR` | `<crate>/ui` | HOMECORE-UI asset dir served at `/homecore` (ADR-131); empty disables the mount |
## HOMECORE-UI dashboard (ADR-131)
This binary also serves the **HOMECORE-UI** — the complete operational dashboard
for the two-tier Cognitum stack (v0 Appliance → SEEDs → ESP32 nodes) — at
`/homecore`, alongside the HA-compat `/api` surface. It is a zero-dependency,
no-build-step vanilla TS/JS + CSS frontend living in `ui/`:
```bash
cargo run -p homecore-server # then open http://localhost:8123/homecore/
```
It drives the live `/api` + `/api/websocket` (`subscribe_events`) endpoints; panels
backed by services not in this binary (SEED HTTPS API, calibration ADR-151,
federation ADR-105) render against a DEMO-flagged contract-conformant mock until
those endpoints land (ADR-131 §7.1). Frontend tests + benchmark run under plain
`node` (no `npm install`):
```bash
cd ui && npm test # import graph + render-smoke + interaction (24 checks)
cd ui && npm run bench # bundle budget (~137 KB, ~37× smaller than HA) + render timing
```
## Comparison to Home Assistant
| Aspect | Home Assistant | homecore-server |
|--------|----------------|-----------------|
| Architecture | Python asyncio monolith | Rust async Tokio + component traits |
| API protocol | `/api/` REST (HA wire format) | Identical HA wire format |
| Persistence | SQLite + YAML files | SQLite (P1); Redis (P2) |
| Plugins | Python integrations in `homeassistant/components/` | Rust (P1) + WASM (P2) |
| Automation execution | Python asyncio event loop | Tokio async tasks + trait-based |
| HomeKit bridge | Via `homekit` integration | Built-in `homecore-hap` subsystem |
| CLI | `hass` command with config YAML | `homecore-server` with feature flags |
| Scalability | Single instance (HA Cloud for scale) | Can be load-balanced (future) |
| Binary size | ~200 MB (Python + deps) | ~50 MB (Rust, release build; 200 MB w/ wasmtime) |
## Performance Targets (unreleased; TBD)
- **Startup time** — < 2s to listen on `:8123`
- **REST endpoint latency** — p50 < 1 ms; p99 < 10 ms
- **Event bus throughput** — 10,000+ events/sec
- **Automation evaluation** — < 100 μs per trigger
- **Concurrent WebSocket connections** — 10,000+
- **Memory footprint** — ~100 MB (idle); ~500 MB with 1,000 recorded states
## Development
**Run tests**:
## Validation
```bash
cargo test -p homecore-server
cargo clippy -p homecore-server --all-targets --all-features -- -D warnings
```
**Enable debug logging**:
```bash
RUST_LOG=debug cargo run -p homecore-server -- --bind 127.0.0.1:8123
```
**Build documentation**:
```bash
cargo doc -p homecore-server --open
```
## Relation to other HOMECORE crates
```
homecore-server (orchestration binary)
├── homecore (state machine)
├── homecore-api (REST + WS)
├── homecore-recorder (SQLite persistence)
├── homecore-plugins (extension system)
├── homecore-automation (trigger execution)
├── homecore-assist (intent pipeline)
└── homecore-hap (HomeKit bridge)
```
## References
- [ADR-126: HOMECORE Home Assistant Port (master)](../../docs/adr/ADR-126-homecore-home-assistant-port.md)
- [README — wifi-densepose](../../../README.md)
- [Dockerfile (planned P2)](Dockerfile.planned)
- [Docker Hub image (planned P2)](https://hub.docker.com/r/ruvnet/homecore-server)
+189 -37
View File
@@ -29,12 +29,16 @@ use axum::body::Bytes;
use axum::extract::{Path, RawQuery, State};
use axum::http::{header, HeaderMap, HeaderValue, StatusCode};
use axum::response::{IntoResponse, Response};
use axum::routing::get;
use axum::routing::{get, post};
use axum::{Json, Router};
use serde::Deserialize;
use serde_json::{json, Value};
use homecore_api::auth::BearerAuth;
use homecore_api::SharedState;
use homecore_assist::{AssistPipeline, RegexIntentRecognizer};
#[cfg(test)]
use homecore_assist::{HassCancelAll, HassLightSet, HassNevermind, HassTurnOff, HassTurnOn};
/// Static gateway configuration (from CLI/env in `main`).
pub struct GatewayConfig {
@@ -54,15 +58,36 @@ pub struct GatewayState {
pub shared: SharedState,
pub http: reqwest::Client,
pub cfg: Arc<GatewayConfig>,
pub assist: Arc<AssistPipeline<RegexIntentRecognizer>>,
}
impl GatewayState {
#[cfg(test)]
pub fn new(shared: SharedState, cfg: GatewayConfig) -> Self {
let mut assist = AssistPipeline::new(RegexIntentRecognizer::new());
assist.register_handler(HassTurnOn);
assist.register_handler(HassTurnOff);
assist.register_handler(HassLightSet);
assist.register_handler(HassNevermind);
assist.register_handler(HassCancelAll);
Self::with_assist(shared, cfg, assist)
}
pub fn with_assist(
shared: SharedState,
cfg: GatewayConfig,
assist: AssistPipeline<RegexIntentRecognizer>,
) -> Self {
let http = reqwest::Client::builder()
.timeout(cfg.timeout)
.build()
.unwrap_or_else(|_| reqwest::Client::new());
Self { shared, http, cfg: Arc::new(cfg) }
Self {
shared,
http,
cfg: Arc::new(cfg),
assist: Arc::new(assist),
}
}
}
@@ -76,6 +101,7 @@ pub fn gateway_router(state: GatewayState) -> Router {
.route("/api/homecore/rooms", get(rooms))
.route("/api/homecore/cogs", get(cogs_list))
.route("/api/homecore/appliance", get(appliance))
.route("/api/intent/handle", post(handle_intent))
// ── upstream-dependent stubs (W3 / W5 / W6): typed 503 ───────
.route("/api/homecore/seeds", get(stub_503))
.route("/api/homecore/seeds/:id", get(stub_503))
@@ -89,10 +115,47 @@ pub fn gateway_router(state: GatewayState) -> Router {
.route("/api/homecore/cogs/updates", get(empty_list))
.route("/api/homecore/hailo", get(stub_503))
.route("/api/homecore/tokens", get(stub_503))
.route("/api/events", get(stub_503))
.route("/api/homecore/events", get(stub_503))
.with_state(state)
}
#[derive(Deserialize)]
struct IntentRequest {
#[serde(alias = "text")]
utterance: String,
#[serde(default = "default_language")]
language: String,
}
fn default_language() -> String {
"en".to_owned()
}
async fn handle_intent(
State(st): State<GatewayState>,
headers: HeaderMap,
Json(request): Json<IntentRequest>,
) -> Response {
if let Err(response) = require_auth(&headers, &st).await {
return response;
}
if request.utterance.trim().is_empty() {
return bad_request("utterance cannot be empty");
}
match st
.assist
.process(&request.utterance, &request.language, st.shared.homecore())
.await
{
Ok(response) => Json(response).into_response(),
Err(error) => typed(
StatusCode::UNPROCESSABLE_ENTITY,
"intent_failed",
&error.to_string(),
),
}
}
// ── auth + typed errors ─────────────────────────────────────────────
async fn require_auth(headers: &HeaderMap, st: &GatewayState) -> Result<(), Response> {
@@ -106,7 +169,11 @@ fn typed(status: StatusCode, error: &str, detail: &str) -> Response {
(status, Json(json!({ "error": error, "detail": detail }))).into_response()
}
fn upstream_unavailable(detail: &str) -> Response {
typed(StatusCode::SERVICE_UNAVAILABLE, "upstream_unavailable", detail)
typed(
StatusCode::SERVICE_UNAVAILABLE,
"upstream_unavailable",
detail,
)
}
fn upstream_timeout(detail: &str) -> Response {
typed(StatusCode::GATEWAY_TIMEOUT, "upstream_timeout", detail)
@@ -130,33 +197,32 @@ fn bad_request(detail: &str) -> Response {
/// like `%252e` decodes once to `%2e` and is caught here).
///
/// Legitimate `v1/...` paths (the only shape the UI sends) pass unchanged.
fn validate_proxy_path(path: &str) -> Result<(), Response> {
fn validate_proxy_path(path: &str) -> Result<(), &'static str> {
// 1. Reject on the raw form first (cheap; catches backslash + leading `/`).
if path.starts_with('/') {
return Err(bad_request("proxied path must be relative (leading '/' not allowed)"));
return Err("proxied path must be relative (leading '/' not allowed)");
}
if path.contains('\\') {
return Err(bad_request("proxied path must not contain a backslash"));
return Err("proxied path must not contain a backslash");
}
// 2. Percent-decode once and re-check; reject if decoding is invalid.
let decoded = percent_decode_once(path)
.ok_or_else(|| bad_request("proxied path has invalid percent-encoding"))?;
let decoded = percent_decode_once(path).ok_or("proxied path has invalid percent-encoding")?;
if decoded.starts_with('/') || decoded.contains('\\') {
return Err(bad_request("proxied path resolves to an absolute/traversal path"));
return Err("proxied path resolves to an absolute/traversal path");
}
// 3. Reject any `.`/`..` segment on BOTH the raw and decoded forms so an
// encoded `%2e%2e%2f` cannot slip a dot-segment past the split.
for form in [path, decoded.as_str()] {
for seg in form.split(['/', '\\']) {
if seg == "." || seg == ".." {
return Err(bad_request("proxied path must not contain '.' or '..' segments"));
return Err("proxied path must not contain '.' or '..' segments");
}
}
// Defence in depth: a residual encoded traversal marker survived the
// single decode (e.g. originally double-encoded). Reject it outright.
let lower = form.to_ascii_lowercase();
if lower.contains("%2e") || lower.contains("%2f") || lower.contains("%5c") {
return Err(bad_request("proxied path must not contain encoded traversal markers"));
return Err("proxied path must not contain encoded traversal markers");
}
}
Ok(())
@@ -194,7 +260,9 @@ async fn stub_503(State(st): State<GatewayState>, headers: HeaderMap) -> Respons
if let Err(r) = require_auth(&headers, &st).await {
return r;
}
upstream_unavailable("endpoint not yet wired — see ADR-131 §11/§12 (SEED device / appliance upstream)")
upstream_unavailable(
"endpoint not yet wired — see ADR-131 §11/§12 (SEED device / appliance upstream)",
)
}
/// Auth-gated empty-array response (e.g. OTA updates with no feed wired).
@@ -216,12 +284,14 @@ async fn cal_proxy_get(
if let Err(r) = require_auth(&headers, &st).await {
return r;
}
if let Err(r) = validate_proxy_path(&path) {
return r;
if let Err(detail) = validate_proxy_path(&path) {
return bad_request(detail);
}
let base = match &st.cfg.calibration_url {
Some(u) => u,
None => return upstream_unavailable("calibration service not configured (set --calibration-url / HOMECORE_CALIBRATION_URL)"),
None => return upstream_unavailable(
"calibration service not configured (set --calibration-url / HOMECORE_CALIBRATION_URL)",
),
};
let qs = q.map(|s| format!("?{s}")).unwrap_or_default();
// The wildcard already carries the `v1/...` segment (the UI calls
@@ -239,12 +309,14 @@ async fn cal_proxy_post(
if let Err(r) = require_auth(&headers, &st).await {
return r;
}
if let Err(r) = validate_proxy_path(&path) {
return r;
if let Err(detail) = validate_proxy_path(&path) {
return bad_request(detail);
}
let base = match &st.cfg.calibration_url {
Some(u) => u,
None => return upstream_unavailable("calibration service not configured (set --calibration-url / HOMECORE_CALIBRATION_URL)"),
None => return upstream_unavailable(
"calibration service not configured (set --calibration-url / HOMECORE_CALIBRATION_URL)",
),
};
let url = format!("{}/api/{}", base.trim_end_matches('/'), path);
let rb = st
@@ -264,7 +336,8 @@ async fn proxy(st: &GatewayState, mut rb: reqwest::RequestBuilder) -> Response {
}
match rb.send().await {
Ok(resp) => {
let status = StatusCode::from_u16(resp.status().as_u16()).unwrap_or(StatusCode::BAD_GATEWAY);
let status =
StatusCode::from_u16(resp.status().as_u16()).unwrap_or(StatusCode::BAD_GATEWAY);
let ct = resp
.headers()
.get(reqwest::header::CONTENT_TYPE)
@@ -325,7 +398,10 @@ async fn rooms(State(st): State<GatewayState>, headers: HeaderMap) -> Response {
let base = base.as_str();
async move {
let url = format!("{base}/api/v1/room/state?bank={bank}");
fetch_json(st, &url).await.ok().map(|v| adapt_room_state(&bank, &v))
fetch_json(st, &url)
.await
.ok()
.map(|v| adapt_room_state(&bank, &v))
}
});
let out: Vec<Value> = futures::future::join_all(fetches)
@@ -352,10 +428,7 @@ fn bank_names(v: &Value) -> Vec<String> {
_ => None,
})
.collect(),
Value::Object(o) => o
.get("baselines")
.map(|b| bank_names(b))
.unwrap_or_default(),
Value::Object(o) => o.get("baselines").map(bank_names).unwrap_or_default(),
_ => Vec::new(),
}
}
@@ -456,7 +529,11 @@ async fn cogs_list(State(st): State<GatewayState>, headers: HeaderMap) -> Respon
}
fn read_pid(dir: &std::path::Path, id: &str) -> Option<i64> {
for name in [format!("{id}.pid"), "pid".to_string(), "app.pid".to_string()] {
for name in [
format!("{id}.pid"),
"pid".to_string(),
"app.pid".to_string(),
] {
if let Ok(s) = std::fs::read_to_string(dir.join(&name)) {
if let Ok(p) = s.trim().parse::<i64>() {
return Some(p);
@@ -515,7 +592,9 @@ async fn appliance(State(st): State<GatewayState>, headers: HeaderMap) -> Respon
}
fn read_first_line(path: &str) -> Option<String> {
std::fs::read_to_string(path).ok().and_then(|s| s.lines().next().map(str::to_string))
std::fs::read_to_string(path)
.ok()
.and_then(|s| s.lines().next().map(str::to_string))
}
fn uptime_secs() -> Option<u64> {
@@ -551,7 +630,9 @@ fn cpu_load_pct() -> Option<f64> {
.next()?
.parse::<f64>()
.ok()?;
let ncpu = std::thread::available_parallelism().map(|n| n.get() as f64).unwrap_or(1.0);
let ncpu = std::thread::available_parallelism()
.map(|n| n.get() as f64)
.unwrap_or(1.0);
Some(((load / ncpu * 100.0).min(100.0) * 10.0).round() / 10.0)
}
@@ -606,7 +687,9 @@ mod tests {
.await
.unwrap();
let status = resp.status();
let b = axum::body::to_bytes(resp.into_body(), 1 << 20).await.unwrap();
let b = axum::body::to_bytes(resp.into_body(), 1 << 20)
.await
.unwrap();
(status, String::from_utf8_lossy(&b).into_owned())
}
@@ -614,7 +697,12 @@ mod tests {
async fn unauthenticated_is_rejected() {
let app = gateway_router(gw());
let resp = app
.oneshot(Request::builder().uri("/api/homecore/cogs").body(Body::empty()).unwrap())
.oneshot(
Request::builder()
.uri("/api/homecore/cogs")
.body(Body::empty())
.unwrap(),
)
.await
.unwrap();
assert_eq!(resp.status(), StatusCode::UNAUTHORIZED);
@@ -636,7 +724,12 @@ mod tests {
#[tokio::test]
async fn seed_tier_routes_are_typed_503() {
for p in ["/api/homecore/seeds", "/api/homecore/federation", "/api/homecore/witness", "/api/events"] {
for p in [
"/api/homecore/seeds",
"/api/homecore/federation",
"/api/homecore/witness",
"/api/homecore/events",
] {
let (status, body) = send(gateway_router(gw()), "GET", p).await;
assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE, "{p} should be 503");
assert!(body.contains("upstream_unavailable"), "{p} typed body");
@@ -651,6 +744,46 @@ mod tests {
assert!(body.contains("\"ram_pct\""));
}
#[tokio::test]
async fn intent_endpoint_executes_a_real_service() {
let state = gw();
let hc = state.shared.homecore().clone();
let id = homecore::EntityId::parse("light.kitchen").unwrap();
hc.states().set(
id.clone(),
"off",
serde_json::json!({}),
homecore::Context::new(),
);
crate::register_builtin_services(&hc).await;
let assist = crate::build_assist_pipeline().await.unwrap();
let state = GatewayState::with_assist(
state.shared,
GatewayConfig {
calibration_url: None,
calibration_token: None,
apps_dir: PathBuf::from("/nonexistent-apps-dir"),
timeout: Duration::from_millis(200),
},
assist,
);
let response = gateway_router(state)
.oneshot(
Request::builder()
.method("POST")
.uri("/api/intent/handle")
.header("authorization", "Bearer dev")
.header("content-type", "application/json")
.body(Body::from(r#"{"utterance":"turn on light.kitchen"}"#))
.unwrap(),
)
.await
.unwrap();
assert_eq!(response.status(), StatusCode::OK);
assert_eq!(hc.states().get(&id).unwrap().state, "on");
}
#[test]
fn adapt_room_state_maps_fields_and_preserves_null() {
// breathing/heartbeat rename; None → null; anomaly gets a threshold.
@@ -668,7 +801,10 @@ mod tests {
assert_eq!(ui["stale"], true);
assert_eq!(ui["presence"]["value"], "occupied");
assert_eq!(ui["breathing_bpm"]["value"], 12.0);
assert!(ui["heart_bpm"].is_null(), "None heartbeat must map to null (not trained)");
assert!(
ui["heart_bpm"].is_null(),
"None heartbeat must map to null (not trained)"
);
// §6 invariant 3: upstream RoomState carries no threshold here, so the
// adapter must emit null (withheld) — NOT a fabricated constant.
assert!(
@@ -685,7 +821,10 @@ mod tests {
"anomaly": {"kind":"Anomaly","value":0.2,"confidence":0.5,"threshold":0.73},
});
let ui = adapt_room_state("bedroom_1", &cal);
assert_eq!(ui["anomaly"]["threshold"], 0.73, "real threshold must pass through");
assert_eq!(
ui["anomaly"]["threshold"], 0.73,
"real threshold must pass through"
);
}
#[test]
@@ -731,8 +870,15 @@ mod tests {
("POST", "/api/cal/v1/../../x"),
] {
let (status, body) = send(gateway_router(gw()), method, path).await;
assert_eq!(status, StatusCode::BAD_REQUEST, "{method} {path} must be 400");
assert!(body.contains("bad_request"), "{method} {path} typed 400 body");
assert_eq!(
status,
StatusCode::BAD_REQUEST,
"{method} {path} must be 400"
);
assert!(
body.contains("bad_request"),
"{method} {path} typed 400 body"
);
assert!(
!body.contains("upstream_unavailable"),
"{method} {path} must NOT reach the upstream-config branch"
@@ -746,13 +892,19 @@ mod tests {
// "not configured" 503 (proving it was NOT blocked as traversal).
let (status, body) = send(gateway_router(gw()), "GET", "/api/cal/v1/room/state").await;
assert_eq!(status, StatusCode::SERVICE_UNAVAILABLE);
assert!(body.contains("upstream_unavailable"), "legit path should reach upstream branch");
assert!(
body.contains("upstream_unavailable"),
"legit path should reach upstream branch"
);
}
#[test]
fn bank_names_accepts_strings_and_objects() {
assert_eq!(bank_names(&json!(["a", "b"])), vec!["a", "b"]);
assert_eq!(bank_names(&json!([{"name":"x"}, {"id":"y"}])), vec!["x", "y"]);
assert_eq!(
bank_names(&json!([{"name":"x"}, {"id":"y"}])),
vec!["x", "y"]
);
assert_eq!(bank_names(&json!({"baselines":["z"]})), vec!["z"]);
}
}
+178
View File
@@ -0,0 +1,178 @@
//! Optional network HomeKit Accessory Protocol lifecycle.
use std::net::{IpAddr, SocketAddr};
use std::path::PathBuf;
use anyhow::Result;
use homecore::HomeCore;
#[derive(Debug, Clone)]
#[cfg_attr(not(feature = "hap-server"), allow(dead_code))]
pub(crate) struct HapRuntimeConfig {
pub bind_addr: Option<SocketAddr>,
pub device_id: Option<String>,
pub setup_code: Option<String>,
pub advertise_addr: Option<IpAddr>,
pub hostname: String,
pub instance_name: String,
pub pairing_store: PathBuf,
}
pub(crate) struct HapRuntime {
#[cfg(feature = "hap-server")]
handle: Option<homecore_hap::HapServerHandle>,
#[cfg(feature = "hap-server")]
state_task: Option<tokio::task::JoinHandle<()>>,
}
impl HapRuntime {
pub(crate) async fn shutdown(self) -> Result<()> {
#[cfg(feature = "hap-server")]
{
let mut runtime = self;
if let Some(task) = runtime.state_task.take() {
task.abort();
let _ = task.await;
}
if let Some(handle) = runtime.handle.take() {
handle.shutdown().await?;
}
}
Ok(())
}
}
pub(crate) async fn start(hc: &HomeCore, config: HapRuntimeConfig) -> Result<HapRuntime> {
let Some(bind_addr) = config.bind_addr else {
tracing::info!("HAP network server disabled");
return Ok(HapRuntime {
#[cfg(feature = "hap-server")]
handle: None,
#[cfg(feature = "hap-server")]
state_task: None,
});
};
#[cfg(not(feature = "hap-server"))]
{
let _ = (hc, bind_addr);
anyhow::bail!(
"HAP was requested but this binary was built without the `hap-server` feature"
);
}
#[cfg(feature = "hap-server")]
{
use std::sync::Arc;
use homecore_hap::{
start_server, HapBridge, HapServerConfig, HapServiceRecord, MdnsSdAdvertiser,
PairingStore,
};
let device_id = config
.device_id
.as_deref()
.ok_or_else(|| anyhow::anyhow!("--hap-device-id is required when HAP is enabled"))?;
let advertise_addr = config.advertise_addr.ok_or_else(|| {
anyhow::anyhow!("--hap-advertise-addr is required when HAP is enabled")
})?;
if let Some(parent) = config.pairing_store.parent() {
std::fs::create_dir_all(parent)?;
}
let advertiser = Arc::new(MdnsSdAdvertiser::new(
config.hostname.clone(),
advertise_addr,
)?);
let pairings = if config.pairing_store.exists() {
if config.setup_code.is_some() {
tracing::warn!(
"ignoring --hap-setup-code because the pairing store already exists"
);
}
PairingStore::open(&config.pairing_store)?
} else {
let setup_code = config.setup_code.as_deref().ok_or_else(|| {
anyhow::anyhow!("--hap-setup-code is required when creating a HAP pairing store")
})?;
PairingStore::create(
&config.pairing_store,
homecore_hap::SetupCode::parse(setup_code)?,
Some(device_id.to_owned()),
)?
};
let pairings = Arc::new(pairings);
let record =
HapServiceRecord::bridge(config.instance_name.clone(), bind_addr.port(), device_id);
let bridge = HapBridge::new(record);
let mut state_rx = hc.states().subscribe();
synchronize(&bridge, hc);
let sync_bridge = bridge.clone();
let sync_hc = hc.clone();
let state_task = tokio::spawn(async move {
loop {
match state_rx.recv().await {
Ok(change) => match change.new_state {
Some(state) => {
if sync_bridge
.update_accessory(&change.entity_id, &state)
.is_err()
{
let _ = sync_bridge.add_accessory(&change.entity_id, &state);
}
}
None => {
let _ = sync_bridge.remove_accessory(&change.entity_id);
}
},
Err(tokio::sync::broadcast::error::RecvError::Lagged(skipped)) => {
tracing::warn!(
skipped,
"HAP state listener lagged; rebuilding accessory snapshot"
);
synchronize(&sync_bridge, &sync_hc);
}
Err(tokio::sync::broadcast::error::RecvError::Closed) => break,
}
}
});
let handle = start_server(
HapServerConfig {
bind_addr,
..HapServerConfig::default()
},
bridge,
pairings,
advertiser,
)
.await?;
tracing::info!(
address = %handle.local_addr(),
pairing_store = %config.pairing_store.display(),
"HAP network server and mDNS advertisement started"
);
Ok(HapRuntime {
handle: Some(handle),
state_task: Some(state_task),
})
}
}
#[cfg(feature = "hap-server")]
fn synchronize(bridge: &homecore_hap::HapBridge, hc: &HomeCore) {
for accessory in bridge.running_accessories() {
let _ = bridge.remove_accessory(&accessory.entity_id);
}
for state in hc.states().all() {
if let Err(error) = bridge.add_accessory(&state.entity_id, &state) {
tracing::debug!(
entity = %state.entity_id,
%error,
"entity is not exposed through HAP"
);
}
}
}
+559 -159
View File
@@ -5,10 +5,8 @@
//! - HomeCore runtime (state machine + event bus + service registry)
//! - SQLite recorder writing every state_changed event
//! - REST + WebSocket API on :8123 (HA wire-compat)
//! - Plugin runtime (InProcessRuntime by default; Wasmtime with --features wasmtime)
//! - Automation engine subscribed to the state machine
//! - Assist pipeline (intent recognizer + handler set)
//! - HAP bridge surface (accessories registered via the API)
//!
//! Run with:
//!
@@ -19,27 +17,29 @@
//! cargo run -p homecore-server --features ruvector,wasmtime -- ...
use std::net::SocketAddr;
use std::sync::Arc;
use anyhow::Result;
use clap::Parser;
use tracing::{info, warn};
use homecore::{Context, EntityId, HomeCore, ServiceCall, ServiceError, ServiceName};
use homecore::service::FnHandler;
use homecore::{Context, EntityId, HomeCore, ServiceCall, ServiceError, ServiceName};
use homecore_api::{build_cors_layer, router, LongLivedTokenStore, SharedState};
use homecore_assist::pipeline::default_pipeline;
use homecore_assist::RegexIntentRecognizer;
use homecore_assist::{
AssistPipeline, HassCancelAll, HassLightSet, HassNevermind, HassTurnOff, HassTurnOn,
RegexIntentRecognizer,
};
use homecore_automation::AutomationEngine;
use homecore_hap::{bridge::HapBridge, mdns::HapServiceRecord};
use homecore_plugins::{InProcessRuntime, PluginRegistry};
use homecore_recorder::Recorder;
use homecore_recorder::{Recorder, RecorderListener};
use axum::Router;
use tower_http::services::ServeDir;
use tower_http::trace::TraceLayer;
mod gateway;
mod hap;
mod plugins;
mod restore;
use gateway::{GatewayConfig, GatewayState};
/// Compile-time default location of the HOMECORE-UI assets (ADR-131).
@@ -71,7 +71,11 @@ struct Cli {
calibration_token: Option<String>,
/// COG install directory the gateway's supervisor reads (ADR-131 §11.6).
#[arg(long, env = "HOMECORE_APPS_DIR", default_value = "/var/lib/cognitum/apps")]
#[arg(
long,
env = "HOMECORE_APPS_DIR",
default_value = "/var/lib/cognitum/apps"
)]
apps_dir: String,
/// Per-upstream proxy timeout in milliseconds (ADR-131 §11.1).
@@ -79,9 +83,21 @@ struct Cli {
gateway_timeout_ms: u64,
/// SQLite recorder DB path. Use `:memory:` for an ephemeral run.
#[arg(long, env = "HOMECORE_DB", default_value = "sqlite::memory:")]
#[arg(long, env = "HOMECORE_DB", default_value = "sqlite://homecore.db")]
db: String,
/// HOMECORE registry storage directory restored at startup.
#[arg(
long,
env = "HOMECORE_STORAGE_DIR",
default_value = ".homecore/storage"
)]
storage_dir: std::path::PathBuf,
/// Maximum registry rows and latest entity states restored at startup.
#[arg(long, env = "HOMECORE_RESTORE_LIMIT", default_value_t = 100_000)]
restore_limit: usize,
/// Friendly location name surfaced via `/api/config`.
#[arg(long, env = "HOMECORE_LOCATION", default_value = "Home")]
location_name: String,
@@ -90,71 +106,177 @@ struct Cli {
#[arg(long)]
no_recorder: bool,
/// Skip the boot-time entity seeding (10 demo entities including
/// 4 RuView-derived sensors). Use this when wiring real
/// integrations that will populate the state machine themselves.
/// Explicitly allow any non-empty bearer token. Development only.
#[arg(long, env = "HOMECORE_INSECURE_DEV_AUTH", default_value_t = false)]
insecure_dev_auth: bool,
/// Seed synthetic demo entities. Disabled by default so simulated
/// biometric readings are never mistaken for live sensor data.
#[arg(long)]
no_seed_entities: bool,
seed_demo_entities: bool,
/// Optional Home Assistant-style automations YAML file to load at boot.
#[arg(long, env = "HOMECORE_AUTOMATIONS")]
automations: Option<std::path::PathBuf>,
/// Explicit directories containing packaged WebAssembly plugins.
#[arg(
long = "plugin-dir",
env = "HOMECORE_PLUGIN_DIRS",
value_delimiter = ','
)]
plugin_dirs: Vec<std::path::PathBuf>,
/// Base64 Ed25519 publisher keys trusted to sign WebAssembly packages.
#[arg(
long = "plugin-trusted-publisher",
env = "HOMECORE_PLUGIN_TRUSTED_PUBLISHERS",
value_delimiter = ','
)]
plugin_trusted_publishers: Vec<String>,
/// Permit unsigned WebAssembly plugins. Unsafe; development only.
#[arg(long, env = "HOMECORE_PLUGIN_ALLOW_UNSIGNED", default_value_t = false)]
plugin_allow_unsigned: bool,
/// Bind address for the optional HomeKit Accessory Protocol server.
/// The server remains disabled unless this option is supplied.
#[arg(long, env = "HOMECORE_HAP_BIND")]
hap_bind: Option<SocketAddr>,
/// Stable six-octet HAP accessory identifier (for example
/// `AA:BB:CC:DD:EE:FF`). Required when HAP is enabled.
#[arg(long, env = "HOMECORE_HAP_DEVICE_ID")]
hap_device_id: Option<String>,
/// HAP setup code in `XXX-XX-XXX` form. Required only when creating a
/// pairing store for the first time and never persisted in plaintext.
#[arg(long, env = "HOMECORE_HAP_SETUP_CODE", hide_env_values = true)]
hap_setup_code: Option<String>,
/// LAN address published in the HAP mDNS record. Required when HAP is enabled.
#[arg(long, env = "HOMECORE_HAP_ADVERTISE_ADDR")]
hap_advertise_addr: Option<std::net::IpAddr>,
/// DNS hostname published by mDNS for the HAP bridge.
#[arg(long, env = "HOMECORE_HAP_HOSTNAME", default_value = "homecore")]
hap_hostname: String,
/// HAP discovery instance shown to controller applications.
#[arg(
long,
env = "HOMECORE_HAP_INSTANCE_NAME",
default_value = "HOMECORE Bridge"
)]
hap_instance_name: String,
/// Durable controller pairing database.
#[arg(
long,
env = "HOMECORE_HAP_PAIRING_STORE",
default_value = ".homecore/hap/pairings.json"
)]
hap_pairing_store: std::path::PathBuf,
}
#[tokio::main]
async fn main() -> Result<()> {
init_tracing();
let cli = Cli::parse();
let mut cli = Cli::parse();
let has_tokens = std::env::var("HOMECORE_TOKENS")
.map(|value| !value.trim().is_empty())
.unwrap_or(false);
if !has_tokens && !cli.insecure_dev_auth {
anyhow::bail!(
"HOMECORE_TOKENS is required; use --insecure-dev-auth only for isolated development"
);
}
let tokens = if has_tokens {
let store = LongLivedTokenStore::from_env();
info!(
"Provisioned {} bearer token(s) from HOMECORE_TOKENS",
store.len().await
);
store
} else {
warn!(
"Insecure development authentication enabled: any non-empty bearer token is accepted"
);
LongLivedTokenStore::allow_any_non_empty()
};
info!("HOMECORE booting — bind={}, db={}, location={:?}",
cli.bind, cli.db, cli.location_name);
info!(
"HOMECORE booting — bind={}, db={}, location={:?}",
cli.bind, cli.db, cli.location_name
);
// ── 1. HomeCore runtime ─────────────────────────────────────────
let hc = HomeCore::new();
info!("HomeCore state machine + event bus + service registry online");
let recorder = if cli.no_recorder {
None
} else {
match open_recorder(&cli.db).await {
Ok(recorder) => Some(recorder),
Err(error) => {
warn!("Recorder failed to open ({error}) — continuing without persistence");
None
}
}
};
let restored =
restore::restore_startup(&hc, recorder.as_ref(), &cli.storage_dir, cli.restore_limit).await;
info!(
entities = restored.entity_entries,
devices = restored.device_entries,
states = restored.states,
truncated = restored.truncated,
"Startup restoration complete"
);
for warning in restored.warnings {
warn!("{warning}");
}
// Seed a representative set of built-in services so the web UI
// and HA-wire-compat clients see a populated /api/services on
// first boot. These are no-op handlers (they just echo back the
// call as JSON for observability) — integrations override them
// by registering the same ServiceName later.
seed_default_services(&hc).await;
register_builtin_services(&hc).await;
// Seed 10 representative entities so the web UI's Dashboard +
// States pages have content out of the box. Operators registering
// real integrations / plugins overwrite these by writing the same
// entity_id with new values. Opt out with `--no-seed-entities`.
if !cli.no_seed_entities {
if cli.seed_demo_entities {
seed_default_entities(&hc);
} else {
info!("Entity seeding disabled by --no-seed-entities");
info!("Synthetic demo entities disabled (use --seed-demo-entities to opt in)");
}
// ── 2. Recorder (optional) ──────────────────────────────────────
if !cli.no_recorder {
match Recorder::open(&cli.db).await {
Ok(recorder) => {
let recorder = Arc::new(recorder);
let rec_clone = Arc::clone(&recorder);
let mut state_rx = hc.states().subscribe();
tokio::spawn(async move {
while let Ok(event) = state_rx.recv().await {
if let Err(e) = rec_clone.record_state(&event).await {
warn!("recorder write failed: {e}");
}
}
});
info!("Recorder open at {} — state_changed events being persisted", cli.db);
}
Err(e) => {
warn!("Recorder failed to open ({e}) — continuing without persistence");
}
}
if let Some(recorder) = recorder.clone() {
let _recorder_task = RecorderListener::new(hc.states(), recorder).spawn();
info!(
"Recorder open at {} — state_changed events being persisted",
cli.db
);
} else {
info!("Recorder disabled by --no-recorder");
info!("Recorder unavailable or disabled");
}
// ── 3. Plugin runtime ───────────────────────────────────────────
let plugin_runtime = InProcessRuntime;
let plugin_registry: PluginRegistry<InProcessRuntime> = PluginRegistry::new(plugin_runtime);
info!("Plugin registry ready (runtime: InProcess; Wasmtime available with --features wasmtime)");
let _ = plugin_registry; // wired-but-empty at boot; integrations register here
let server_plugins = plugins::ServerPlugins::start(
hc.clone(),
plugins::PluginConfig {
directories: cli.plugin_dirs.clone(),
trusted_publishers: cli.plugin_trusted_publishers.clone(),
allow_unsigned: cli.plugin_allow_unsigned,
limits: homecore_plugins::DiscoveryLimits::default(),
},
)
.await?;
// ── 4. Automation engine ────────────────────────────────────────
// Construct AND start the engine (HC-WS-03, ADR-161). `start()`
@@ -166,6 +288,14 @@ async fn main() -> Result<()> {
// boot yet (YAML loader is P-next); integrations register via
// `engine.register(..)`.
let automation_engine = AutomationEngine::new(hc.clone());
if let Some(path) = &cli.automations {
let raw = tokio::fs::read_to_string(path).await?;
let automations: Vec<homecore_automation::Automation> = serde_yaml::from_str(&raw)
.map_err(|e| anyhow::anyhow!("invalid automations file {}: {e}", path.display()))?;
for automation in automations {
automation_engine.register(automation);
}
}
let _automation_task = automation_engine.start();
info!(
"Automation engine started ({} automations registered) — \
@@ -174,46 +304,41 @@ async fn main() -> Result<()> {
);
// ── 5. Assist pipeline ──────────────────────────────────────────
let recognizer = RegexIntentRecognizer::new();
let pipeline = default_pipeline(recognizer);
info!("Assist pipeline ready (5 built-in intent handlers via default_pipeline)");
let _ = pipeline; // wired-but-idle at boot; voice WS plugs in here
// ── 6. HAP bridge surface ───────────────────────────────────────
let hap_record = HapServiceRecord {
instance_name: "HOMECORE".to_string(),
port: 51826,
setup_code: "123-45-678".to_string(),
device_id: "AA:BB:CC:DD:EE:FF".to_string(),
};
let hap_bridge = HapBridge::new(hap_record);
info!("HAP bridge surface ready ({} accessories registered)", hap_bridge.running_accessories().len());
let _ = hap_bridge;
// ── 7. REST + WS API ────────────────────────────────────────────
// Token provisioning closes audit findings HC-01/HC-02. If
// HOMECORE_TOKENS is set in the env, populate the store from
// its comma-separated list. Otherwise fall back to DEV mode
// (warn-on-each-request) so existing smoke tests still work.
let tokens = if std::env::var("HOMECORE_TOKENS").map(|v| !v.trim().is_empty()).unwrap_or(false) {
let s = LongLivedTokenStore::from_env();
let n = s.len().await;
info!("LongLivedTokenStore provisioned with {} bearer token(s) from HOMECORE_TOKENS", n);
s
} else {
warn!("HOMECORE_TOKENS not set — token store in DEV mode (any non-empty bearer accepted). Provision real tokens before exposing to the network.");
LongLivedTokenStore::allow_any_non_empty()
};
let api_state = SharedState::with_tokens(
hc.clone(),
cli.location_name,
env!("CARGO_PKG_VERSION"),
tokens,
);
)
.with_recorder(recorder);
// BFF gateway (ADR-131 §11): single-origin aggregation of the
// calibration API + SEED/appliance tiers. Shares the same token store
// for auth; upstream credentials stay server-side.
let gw = GatewayState::new(
let assist = build_assist_pipeline().await?;
info!(
"Assist intent endpoint ready with {} handlers",
assist.handler_count()
);
let hap_runtime = hap::start(
&hc,
hap::HapRuntimeConfig {
bind_addr: cli.hap_bind,
device_id: cli.hap_device_id.clone(),
setup_code: cli.hap_setup_code.take(),
advertise_addr: cli.hap_advertise_addr,
hostname: cli.hap_hostname.clone(),
instance_name: cli.hap_instance_name.clone(),
pairing_store: cli.hap_pairing_store.clone(),
},
)
.await?;
let gw = GatewayState::with_assist(
api_state.clone(),
GatewayConfig {
calibration_url: cli.calibration_url.clone(),
@@ -221,6 +346,7 @@ async fn main() -> Result<()> {
apps_dir: std::path::PathBuf::from(&cli.apps_dir),
timeout: std::time::Duration::from_millis(cli.gateway_timeout_ms),
},
assist,
);
// Merge the HA-compat API + UI mount with the BFF gateway, THEN apply the
// audited CORS allowlist + request tracing to the WHOLE surface. The
@@ -233,19 +359,37 @@ async fn main() -> Result<()> {
.layer(build_cors_layer())
.layer(TraceLayer::new_for_http());
let listener = tokio::net::TcpListener::bind(cli.bind).await?;
info!("HOMECORE-API listening on http://{} (HA-compat /api + /api/websocket)", cli.bind);
info!(
"HOMECORE-API listening on http://{} (HA-compat /api + /api/websocket)",
cli.bind
);
info!(
"HOMECORE BFF gateway active: /api/homecore/* + /api/cal/* (calibration_url={:?})",
cli.calibration_url
);
if !cli.ui_dir.trim().is_empty() {
info!("HOMECORE-UI (ADR-131) served at http://{}/homecore/ from {}", cli.bind, cli.ui_dir);
info!(
"HOMECORE-UI (ADR-131) served at http://{}/homecore/ from {}",
cli.bind, cli.ui_dir
);
} else {
info!("HOMECORE-UI mount disabled (--ui-dir empty)");
}
// Run forever (until SIGINT). axum::serve handles graceful shutdown.
axum::serve(listener, app).await?;
let shutdown_hc = hc.clone();
axum::serve(listener, app)
.with_graceful_shutdown(async move {
if let Err(error) = tokio::signal::ctrl_c().await {
warn!("failed to install Ctrl-C handler: {error}");
}
shutdown_hc
.bus()
.fire_system(homecore::SystemEvent::HomeCoreStop);
info!("Shutdown requested; draining active HTTP connections");
})
.await?;
hap_runtime.shutdown().await?;
server_plugins.shutdown().await;
Ok(())
}
@@ -263,11 +407,68 @@ fn build_app(api_state: SharedState, ui_dir: &str) -> Router {
app.nest_service("/homecore", ServeDir::new(ui_dir))
}
#[cfg(not(feature = "ruvector"))]
async fn open_recorder(path: &str) -> anyhow::Result<Recorder> {
Ok(Recorder::open(path).await?)
}
async fn build_assist_pipeline() -> anyhow::Result<AssistPipeline<RegexIntentRecognizer>> {
let recognizer = RegexIntentRecognizer::new();
recognizer
.register(
"HassTurnOn",
r"^turn on (?:the )?(?P<entity_id>[a-z0-9_]+\.[a-z0-9_]+)$",
"*",
)
.await?;
recognizer
.register(
"HassTurnOff",
r"^turn off (?:the )?(?P<entity_id>[a-z0-9_]+\.[a-z0-9_]+)$",
"*",
)
.await?;
recognizer
.register(
"HassLightSet",
r"^set (?P<entity_id>light\.[a-z0-9_]+) to (?P<brightness>[0-9]{1,3})$",
"*",
)
.await?;
recognizer
.register("HassNevermind", r"^(?:never ?mind|cancel that)$", "*")
.await?;
recognizer
.register("HassCancelAll", r"^cancel all automations$", "*")
.await?;
let mut pipeline = AssistPipeline::new(recognizer);
pipeline.register_handler(HassTurnOn);
pipeline.register_handler(HassTurnOff);
pipeline.register_handler(HassLightSet);
pipeline.register_handler(HassNevermind);
pipeline.register_handler(HassCancelAll);
Ok(pipeline)
}
#[cfg(feature = "ruvector")]
async fn open_recorder(path: &str) -> anyhow::Result<Recorder> {
use homecore_recorder::{RuvectorSemanticIndex, SemanticIndex};
use tokio::sync::RwLock;
let index = RuvectorSemanticIndex::new(100_000)
.map_err(|error| anyhow::anyhow!("failed to initialize ruvector index: {error}"))?;
let semantic: std::sync::Arc<RwLock<dyn SemanticIndex>> =
std::sync::Arc::new(RwLock::new(index));
Ok(Recorder::open_with_index(path, semantic).await?)
}
fn init_tracing() {
tracing_subscriber::fmt()
.with_env_filter(
tracing_subscriber::EnvFilter::try_from_default_env()
.unwrap_or_else(|_| "info,homecore=debug,homecore_server=debug,tower_http=info".into()),
tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| {
"info,homecore=debug,homecore_server=debug,tower_http=info".into()
}),
)
.init();
}
@@ -281,42 +482,125 @@ fn init_tracing() {
/// light / switch / scene / automation domains) plus a `homecore.*`
/// domain so operators can see HOMECORE-native services distinguished
/// from the HA-compat ones.
async fn seed_default_services(hc: &HomeCore) {
let echo = || FnHandler(|call: ServiceCall| async move {
Ok(serde_json::json!({
"called": format!("{}.{}", call.name.domain, call.name.service),
"service_data": call.data,
"acknowledged": true,
}))
});
async fn register_builtin_services(hc: &HomeCore) {
hc.services()
.register(
ServiceName::new("homecore", "ping"),
FnHandler(|_call| async { Ok(serde_json::json!({ "pong": true })) }),
)
.await;
let svcs = [
// Conventional HA wire-compat services
("homeassistant", "restart"),
("homeassistant", "stop"),
("homeassistant", "reload_core_config"),
("light", "turn_on"),
("light", "turn_off"),
("light", "toggle"),
("switch", "turn_on"),
("switch", "turn_off"),
("switch", "toggle"),
("scene", "apply"),
("automation", "trigger"),
// HOMECORE-native services
("homecore", "ping"),
("homecore", "snapshot_state"),
];
let snapshot_states = hc.states().clone();
hc.services()
.register(
ServiceName::new("homecore", "snapshot_state"),
FnHandler(move |_call| {
let states = snapshot_states.clone();
async move { Ok(serde_json::to_value(states.all()).unwrap_or_default()) }
}),
)
.await;
for (domain, service) in svcs {
hc.services()
.register(ServiceName::new(domain, service), echo())
.await;
for domain in ["homeassistant", "light", "switch"] {
for action in ["turn_on", "turn_off", "toggle"] {
let states = hc.states().clone();
let required_domain = (domain != "homeassistant").then(|| domain.to_owned());
hc.services()
.register(
ServiceName::new(domain, action),
FnHandler(move |call: ServiceCall| {
let states = states.clone();
let required_domain = required_domain.clone();
async move {
let ids = service_entity_ids(&call.data)?;
let mut changed = Vec::with_capacity(ids.len());
for id in ids {
if let Some(domain) = required_domain.as_deref() {
if id.domain() != domain {
return Err(ServiceError::HandlerFailed(format!(
"{}.{} only accepts {domain} entities",
call.name.domain, call.name.service
)));
}
}
let current = states.get(&id).ok_or_else(|| {
ServiceError::HandlerFailed(format!(
"entity not found: {}",
id.as_str()
))
})?;
let next = match call.name.service.as_str() {
"turn_on" => "on",
"turn_off" => "off",
"toggle" if current.state == "on" => "off",
"toggle" => "on",
_ => unreachable!("only registered actions are dispatched"),
};
let mut attributes = current.attributes.clone();
if next == "on" {
if let (Some(object), Some(brightness)) =
(attributes.as_object_mut(), call.data.get("brightness"))
{
object.insert("brightness".into(), brightness.clone());
}
}
states.set(
id.clone(),
next,
attributes,
Context::child_of(&call.context),
);
changed.push(id.as_str().to_owned());
}
Ok(serde_json::json!({ "changed": changed }))
}
}),
)
.await;
}
}
let count = hc.services().registered_services().await.len();
let _ = ServiceError::NotRegistered { domain: String::new(), service: String::new() };
info!("Service registry seeded with {} default service(s)", count);
info!(
"Registered {} executable built-in services",
hc.services().registered_services().await.len()
);
}
fn service_entity_ids(data: &serde_json::Value) -> Result<Vec<EntityId>, ServiceError> {
let value = data
.get("entity_id")
.or_else(|| {
data.get("target")
.and_then(|target| target.get("entity_id"))
})
.ok_or_else(|| ServiceError::HandlerFailed("missing entity_id".into()))?;
let raw_ids: Vec<&str> = match value {
serde_json::Value::String(value) => vec![value],
serde_json::Value::Array(values) => values
.iter()
.map(|value| {
value.as_str().ok_or_else(|| {
ServiceError::HandlerFailed("entity_id array must contain strings".into())
})
})
.collect::<Result<_, _>>()?,
_ => {
return Err(ServiceError::HandlerFailed(
"entity_id must be a string or string array".into(),
));
}
};
if raw_ids.is_empty() {
return Err(ServiceError::HandlerFailed(
"entity_id array cannot be empty".into(),
));
}
raw_ids
.into_iter()
.map(|value| {
EntityId::parse(value).map_err(|error| ServiceError::HandlerFailed(error.to_string()))
})
.collect()
}
/// Register 10 representative entities so a fresh `--db :memory:`
@@ -325,45 +609,85 @@ async fn seed_default_services(hc: &HomeCore) {
/// they stay in sync.
fn seed_default_entities(hc: &HomeCore) {
let entities: Vec<(&str, &str, serde_json::Value)> = vec![
("sensor.living_room_presence", "false", serde_json::json!({
"friendly_name": "Living Room Presence", "device_class": "occupancy",
"source": "RuView ESP32-C6 BFLD"
})),
("sensor.living_room_motion_score", "0.0", serde_json::json!({
"friendly_name": "Living Room Motion Score", "unit_of_measurement": "score",
"icon": "mdi:motion-sensor"
})),
("sensor.bedroom_breathing_rate", "14.5", serde_json::json!({
"friendly_name": "Bedroom Breathing Rate", "unit_of_measurement": "BPM",
"device_class": "frequency", "source": "Seeed MR60BHA2 mmWave"
})),
("sensor.bedroom_heart_rate", "68.0", serde_json::json!({
"friendly_name": "Bedroom Heart Rate", "unit_of_measurement": "BPM",
"device_class": "frequency", "source": "Seeed MR60BHA2 mmWave"
})),
("light.kitchen_ceiling", "on", serde_json::json!({
"friendly_name": "Kitchen Ceiling", "brightness": 230,
"color_temp_kelvin": 4000, "supported_color_modes": ["color_temp"]
})),
("light.living_room_lamp", "off", serde_json::json!({
"friendly_name": "Living Room Lamp", "brightness": 0,
"supported_color_modes": ["brightness"]
})),
("switch.coffee_maker", "off", serde_json::json!({
"friendly_name": "Coffee Maker", "device_class": "outlet"
})),
("binary_sensor.front_door", "off", serde_json::json!({
"friendly_name": "Front Door", "device_class": "door"
})),
("climate.thermostat", "heat", serde_json::json!({
"friendly_name": "Thermostat", "current_temperature": 21.5,
"temperature": 22.0, "hvac_modes": ["off", "heat", "cool", "auto"],
"supported_features": 387
})),
("sensor.air_quality_index", "42", serde_json::json!({
"friendly_name": "Air Quality Index", "unit_of_measurement": "AQI",
"device_class": "aqi"
})),
(
"sensor.living_room_presence",
"false",
serde_json::json!({
"friendly_name": "Living Room Presence", "device_class": "occupancy",
"source": "RuView ESP32-C6 BFLD"
}),
),
(
"sensor.living_room_motion_score",
"0.0",
serde_json::json!({
"friendly_name": "Living Room Motion Score", "unit_of_measurement": "score",
"icon": "mdi:motion-sensor"
}),
),
(
"sensor.bedroom_breathing_rate",
"14.5",
serde_json::json!({
"friendly_name": "Bedroom Breathing Rate", "unit_of_measurement": "BPM",
"device_class": "frequency", "source": "Seeed MR60BHA2 mmWave"
}),
),
(
"sensor.bedroom_heart_rate",
"68.0",
serde_json::json!({
"friendly_name": "Bedroom Heart Rate", "unit_of_measurement": "BPM",
"device_class": "frequency", "source": "Seeed MR60BHA2 mmWave"
}),
),
(
"light.kitchen_ceiling",
"on",
serde_json::json!({
"friendly_name": "Kitchen Ceiling", "brightness": 230,
"color_temp_kelvin": 4000, "supported_color_modes": ["color_temp"]
}),
),
(
"light.living_room_lamp",
"off",
serde_json::json!({
"friendly_name": "Living Room Lamp", "brightness": 0,
"supported_color_modes": ["brightness"]
}),
),
(
"switch.coffee_maker",
"off",
serde_json::json!({
"friendly_name": "Coffee Maker", "device_class": "outlet"
}),
),
(
"binary_sensor.front_door",
"off",
serde_json::json!({
"friendly_name": "Front Door", "device_class": "door"
}),
),
(
"climate.thermostat",
"heat",
serde_json::json!({
"friendly_name": "Thermostat", "current_temperature": 21.5,
"temperature": 22.0, "hvac_modes": ["off", "heat", "cool", "auto"],
"supported_features": 387
}),
),
(
"sensor.air_quality_index",
"42",
serde_json::json!({
"friendly_name": "Air Quality Index", "unit_of_measurement": "AQI",
"device_class": "aqi"
}),
),
];
for (id, state, attrs) in entities {
@@ -381,8 +705,11 @@ fn seed_default_entities(hc: &HomeCore) {
context: Context::new(),
};
let total = hc.states().all().len();
info!("State machine seeded with {} default entit{}", total,
if total == 1 { "y" } else { "ies" });
info!(
"State machine seeded with {} default entit{}",
total,
if total == 1 { "y" } else { "ies" }
);
}
#[cfg(test)]
@@ -419,9 +746,19 @@ mod ui_tests {
async fn ui_index_is_served_at_homecore() {
let app = build_app(test_state(), DEFAULT_UI_DIR);
let (status, body) = get(app, "/homecore/").await;
assert_eq!(status, StatusCode::OK, "GET /homecore/ should serve index.html");
assert!(body.contains("HOMECORE"), "index.html should mention HOMECORE");
assert!(body.contains("./js/app.js"), "index.html should bootstrap app.js");
assert_eq!(
status,
StatusCode::OK,
"GET /homecore/ should serve index.html"
);
assert!(
body.contains("HOMECORE"),
"index.html should mention HOMECORE"
);
assert!(
body.contains("./js/app.js"),
"index.html should bootstrap app.js"
);
}
#[tokio::test]
@@ -437,8 +774,18 @@ mod ui_tests {
#[tokio::test]
async fn ui_panels_are_served() {
let app = build_app(test_state(), DEFAULT_UI_DIR);
for p in ["dashboard", "rooms", "calibration", "fleet", "seed-detail",
"entities", "cogs", "events", "audit", "settings"] {
for p in [
"dashboard",
"rooms",
"calibration",
"fleet",
"seed-detail",
"entities",
"cogs",
"events",
"audit",
"settings",
] {
let (status, _) = get(app.clone(), &format!("/homecore/js/panels/{p}.js")).await;
assert_eq!(status, StatusCode::OK, "panel {p}.js should be served");
}
@@ -459,9 +806,15 @@ mod ui_tests {
.await
.unwrap();
let status = resp.status();
let bytes = axum::body::to_bytes(resp.into_body(), 1 << 20).await.unwrap();
let bytes = axum::body::to_bytes(resp.into_body(), 1 << 20)
.await
.unwrap();
let body = String::from_utf8_lossy(&bytes);
assert_eq!(status, StatusCode::OK, "the HA-compat API must coexist with the UI mount");
assert_eq!(
status,
StatusCode::OK,
"the HA-compat API must coexist with the UI mount"
);
assert!(body.contains("API running"));
}
@@ -469,12 +822,16 @@ mod ui_tests {
async fn ui_mount_can_be_disabled() {
let app = build_app(test_state(), "");
let (status, _) = get(app, "/homecore/").await;
assert_eq!(status, StatusCode::NOT_FOUND, "empty --ui-dir disables the mount");
assert_eq!(
status,
StatusCode::NOT_FOUND,
"empty --ui-dir disables the mount"
);
}
/// Build the SAME merged + layered surface `main()` serves: API + UI mount
/// + BFF gateway, with the audited CORS allowlist + tracing applied to the
/// whole thing. Used to prove the gateway routes are CORS-covered.
/// whole thing. Used to prove the gateway routes are CORS-covered.
fn full_app(state: SharedState) -> Router {
use crate::gateway::{GatewayConfig, GatewayState};
let gw = GatewayState::new(
@@ -527,4 +884,47 @@ mod ui_tests {
"gateway route must echo the allowlisted dev origin"
);
}
#[tokio::test]
async fn builtin_light_service_changes_real_state() {
let hc = HomeCore::new();
let id = EntityId::parse("light.kitchen").unwrap();
hc.states().set(
id.clone(),
"off",
serde_json::json!({"friendly_name": "Kitchen"}),
Context::new(),
);
register_builtin_services(&hc).await;
let result = hc
.services()
.call(ServiceCall {
name: ServiceName::new("light", "turn_on"),
data: serde_json::json!({"entity_id": "light.kitchen", "brightness": 123}),
context: Context::new(),
})
.await
.unwrap();
assert_eq!(result["changed"][0], "light.kitchen");
let state = hc.states().get(&id).unwrap();
assert_eq!(state.state, "on");
assert_eq!(state.attributes["brightness"], 123);
}
#[tokio::test]
async fn unsupported_builtin_service_is_not_success_shaped() {
let hc = HomeCore::new();
register_builtin_services(&hc).await;
let result = hc
.services()
.call(ServiceCall {
name: ServiceName::new("homeassistant", "restart"),
data: serde_json::json!({}),
context: Context::new(),
})
.await;
assert!(matches!(result, Err(ServiceError::NotRegistered { .. })));
}
}
+234
View File
@@ -0,0 +1,234 @@
//! Server ownership for plugin discovery, setup, event dispatch, and teardown.
//!
//! Native Rust plugins are never discovered from disk. They must be compiled
//! into this binary and exposed through [`NativePluginRegistration`]. External
//! packages are WebAssembly only and require the `wasmtime` feature.
use std::path::PathBuf;
use std::sync::Arc;
use anyhow::{Context as _, Result};
use homecore::HomeCore;
use homecore_plugins::{
DiscoveryLimits, HomeCorePlugin, InProcessRuntime, PluginError, PluginManifest, PluginRegistry,
StateChangedEventJson,
};
use tokio::task::JoinHandle;
use tracing::{error, info, warn};
#[cfg(feature = "wasmtime")]
use homecore_plugins::{
discover_plugins, ConfigEntryJson, PluginId, PluginPolicy, WasmPlugin, WasmtimeRuntime,
};
/// Factory entry for a trusted plugin linked into the server at compile time.
type NativePluginFactory = fn() -> Result<(PluginManifest, Arc<dyn HomeCorePlugin>), PluginError>;
pub struct NativePluginRegistration {
pub create: NativePluginFactory,
}
/// This is intentionally empty in the generic server. Appliance builds can
/// add first-party registrations here; arbitrary Rust cdylibs are unsupported.
const NATIVE_PLUGINS: &[NativePluginRegistration] = &[];
#[derive(Debug, Clone)]
pub struct PluginConfig {
pub directories: Vec<PathBuf>,
pub trusted_publishers: Vec<String>,
pub allow_unsigned: bool,
#[cfg_attr(not(feature = "wasmtime"), allow(dead_code))]
pub limits: DiscoveryLimits,
}
pub struct ServerPlugins {
native: Arc<PluginRegistry<InProcessRuntime>>,
dispatcher: JoinHandle<()>,
#[cfg(feature = "wasmtime")]
wasm: Vec<(PluginId, WasmPlugin)>,
}
impl ServerPlugins {
/// Start the complete plugin subsystem. Any configured package that fails
/// discovery, trust verification, compilation, or setup aborts startup.
pub async fn start(hc: HomeCore, config: PluginConfig) -> Result<Self> {
let native = Arc::new(PluginRegistry::new(InProcessRuntime));
for registration in NATIVE_PLUGINS {
let (manifest, plugin) =
(registration.create)().context("compiled-in native plugin factory failed")?;
native
.load(manifest, plugin, hc.clone())
.await
.context("compiled-in native plugin setup failed")?;
}
#[cfg(not(feature = "wasmtime"))]
if !config.directories.is_empty() {
anyhow::bail!(
"plugin directories were configured, but homecore-server was built without \
--features wasmtime"
);
}
#[cfg(not(feature = "wasmtime"))]
if config.allow_unsigned || !config.trusted_publishers.is_empty() {
anyhow::bail!(
"plugin trust options were configured, but homecore-server was built without \
--features wasmtime"
);
}
#[cfg(feature = "wasmtime")]
let wasm = load_wasm_plugins(&hc, &config).await?;
let mut receiver = hc.states().subscribe();
let dispatch_native = native.clone();
#[cfg(feature = "wasmtime")]
let dispatch_wasm = wasm.clone();
let dispatcher = tokio::spawn(async move {
loop {
let change = match receiver.recv().await {
Ok(change) => change,
Err(tokio::sync::broadcast::error::RecvError::Lagged(skipped)) => {
warn!(
skipped,
"plugin state dispatcher lagged; events were dropped"
);
continue;
}
Err(tokio::sync::broadcast::error::RecvError::Closed) => break,
};
let event = StateChangedEventJson::state_changed(
change.entity_id.as_str(),
change.new_state.as_ref().map(|state| state.state.as_str()),
change
.new_state
.as_ref()
.map(|state| state.attributes.clone())
.unwrap_or_else(|| serde_json::json!({})),
);
for (id, error) in dispatch_native.state_changed(&event).await {
error!(plugin = %id, %error, "native plugin state_changed failed");
}
#[cfg(feature = "wasmtime")]
for (id, plugin) in &dispatch_wasm {
if !plugin
.subscriptions()
.iter()
.any(|entity| entity == change.entity_id.as_str())
{
continue;
}
let plugin = plugin.clone();
let id = id.clone();
let event = StateChangedEventJson::state_changed(
&event.entity_id,
event.new_state.as_deref(),
event.attributes.clone(),
);
match tokio::task::spawn_blocking(move || plugin.call_state_changed(&event))
.await
{
Ok(Ok(0)) => {}
Ok(Ok(code)) => {
error!(plugin = %id, code, "WASM state_changed returned failure")
}
Ok(Err(error)) => {
error!(plugin = %id, %error, "WASM state_changed trapped")
}
Err(error) => error!(plugin = %id, %error, "WASM dispatch task failed"),
}
}
}
});
#[cfg(feature = "wasmtime")]
info!(
native = NATIVE_PLUGINS.len(),
wasm = wasm.len(),
"plugin subsystem started"
);
#[cfg(not(feature = "wasmtime"))]
info!(native = NATIVE_PLUGINS.len(), "plugin subsystem started");
Ok(Self {
native,
dispatcher,
#[cfg(feature = "wasmtime")]
wasm,
})
}
/// Stop dispatch first, then tear down WASM and native plugins in reverse
/// deterministic order. All teardown failures are reported.
pub async fn shutdown(self) {
self.dispatcher.abort();
let _ = self.dispatcher.await;
#[cfg(feature = "wasmtime")]
for (id, plugin) in self.wasm.into_iter().rev() {
match tokio::task::spawn_blocking(move || plugin.call_teardown()).await {
Ok(Ok(0)) => {}
Ok(Ok(code)) => error!(plugin = %id, code, "WASM teardown returned failure"),
Ok(Err(error)) => error!(plugin = %id, %error, "WASM teardown trapped"),
Err(error) => error!(plugin = %id, %error, "WASM teardown task failed"),
}
}
for (id, error) in self.native.shutdown().await {
error!(plugin = %id, %error, "native plugin teardown failed");
}
}
}
#[cfg(feature = "wasmtime")]
async fn load_wasm_plugins(
hc: &HomeCore,
config: &PluginConfig,
) -> Result<Vec<(PluginId, WasmPlugin)>> {
let policy = if config.allow_unsigned {
warn!("INSECURE plugin policy enabled: unsigned plugins may execute");
PluginPolicy::AllowUnsigned
} else {
let keys: Vec<_> = config
.trusted_publishers
.iter()
.map(String::as_str)
.collect();
PluginPolicy::trusted(&keys).context("invalid trusted plugin publisher key")?
};
let discovered =
discover_plugins(&config.directories, config.limits).context("plugin discovery failed")?;
let runtime = WasmtimeRuntime::new().context("Wasmtime initialization failed")?;
let mut loaded: Vec<(PluginId, WasmPlugin)> = Vec::with_capacity(discovered.len());
for package in discovered {
let id = PluginId::new(&package.manifest.domain);
let bytes = package
.read_module(config.limits)
.with_context(|| format!("failed reading plugin `{id}`"))?;
let plugin = runtime
.load_plugin(&package.manifest, &bytes, hc.clone(), &policy)
.with_context(|| format!("plugin `{id}` rejected before setup"))?;
let config_entry = serde_json::to_string(&ConfigEntryJson::bootstrap(id.as_str()))?;
let setup_plugin = plugin.clone();
let result = tokio::task::spawn_blocking(move || setup_plugin.call_setup(&config_entry))
.await
.with_context(|| format!("plugin `{id}` setup task failed"))?
.with_context(|| format!("plugin `{id}` setup trapped"))?;
if result != 0 {
let _ = plugin.call_teardown();
teardown_wasm(loaded).await;
anyhow::bail!("plugin `{id}` setup returned failure code {result}");
}
info!(
plugin = %id,
package = %package.package_dir.display(),
"signed WASM plugin loaded"
);
loaded.push((id, plugin));
}
Ok(loaded)
}
#[cfg(feature = "wasmtime")]
async fn teardown_wasm(plugins: Vec<(PluginId, WasmPlugin)>) {
for (_, plugin) in plugins.into_iter().rev() {
let _ = tokio::task::spawn_blocking(move || plugin.call_teardown()).await;
}
}
+219
View File
@@ -0,0 +1,219 @@
//! Bounded startup restoration for registries and latest recorder states.
use std::path::Path;
use homecore::{DeviceEntry, EntityEntry, HomeCore};
use homecore_migrate::storage::read_envelope;
use homecore_recorder::{Recorder, RestoreWarning};
use serde::de::DeserializeOwned;
pub const MAX_REGISTRY_ENTRIES: usize = 100_000;
pub const MAX_REGISTRY_FILE_BYTES: u64 = 64 * 1024 * 1024;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum RestorePhase {
EntityRegistry,
DeviceRegistry,
States,
}
#[derive(Debug, Default)]
pub struct StartupRestoreReport {
pub entity_entries: usize,
pub device_entries: usize,
pub states: usize,
pub warnings: Vec<String>,
pub truncated: bool,
pub phases: Vec<RestorePhase>,
}
pub async fn restore_startup(
homecore: &HomeCore,
recorder: Option<&Recorder>,
storage_dir: &Path,
limit: usize,
) -> StartupRestoreReport {
let limit = limit.min(MAX_REGISTRY_ENTRIES);
let mut report = StartupRestoreReport::default();
report.phases.push(RestorePhase::EntityRegistry);
let entities_path = storage_dir.join("core.entity_registry");
if entities_path.exists() {
match load_rows::<EntityEntry>(&entities_path, "entities", limit, |entry| {
entry.entity_id.as_str().to_owned()
}) {
Ok(batch) => {
report.truncated |= batch.truncated;
report.warnings.extend(batch.warnings);
for entry in batch.rows {
homecore.entities().register(entry).await;
report.entity_entries += 1;
}
}
Err(error) => report.warnings.push(error),
}
}
report.phases.push(RestorePhase::DeviceRegistry);
let devices_path = storage_dir.join("core.device_registry");
if devices_path.exists() {
match load_rows::<DeviceEntry>(&devices_path, "devices", limit, |entry| entry.id.clone()) {
Ok(batch) => {
report.truncated |= batch.truncated;
report.warnings.extend(batch.warnings);
for entry in batch.rows {
homecore.devices().register(entry).await;
report.device_entries += 1;
}
}
Err(error) => report.warnings.push(error),
}
}
report.phases.push(RestorePhase::States);
if let Some(recorder) = recorder {
match recorder.restore_latest(homecore.states(), limit).await {
Ok(state_report) => {
report.states = state_report.restored;
report.truncated |= state_report.truncated;
report
.warnings
.extend(state_report.warnings.into_iter().map(format_state_warning));
}
Err(error) => report
.warnings
.push(format!("state restore failed: {error}")),
}
}
report
}
struct RowBatch<T> {
rows: Vec<T>,
warnings: Vec<String>,
truncated: bool,
}
fn load_rows<T: DeserializeOwned>(
path: &Path,
field: &str,
limit: usize,
key: impl Fn(&T) -> String,
) -> Result<RowBatch<T>, String> {
let file_size = std::fs::metadata(path)
.map_err(|error| format!("{}: {error}", path.display()))?
.len();
if file_size > MAX_REGISTRY_FILE_BYTES {
return Err(format!(
"{}: registry file is {file_size} bytes, exceeding the {} byte startup bound",
path.display(),
MAX_REGISTRY_FILE_BYTES
));
}
let envelope = read_envelope(path).map_err(|error| error.to_string())?;
if envelope.version != 1 || envelope.minor_version > 13 {
return Err(format!(
"{}: unsupported registry version {}.{}",
path.display(),
envelope.version,
envelope.minor_version
));
}
let values = envelope
.data
.get(field)
.and_then(serde_json::Value::as_array)
.ok_or_else(|| format!("{}: missing data.{field} array", path.display()))?;
let scan_limit = limit.saturating_add(1).min(MAX_REGISTRY_ENTRIES + 1);
let mut rows = Vec::with_capacity(values.len().min(scan_limit));
let mut warnings = Vec::new();
for (index, value) in values.iter().take(scan_limit).enumerate() {
match serde_json::from_value::<T>(value.clone()) {
Ok(row) => rows.push(row),
Err(error) => warnings.push(format!(
"{}: isolated malformed data.{field}[{index}]: {error}",
path.display()
)),
}
}
rows.sort_by_key(&key);
let truncated = values.len() > limit || rows.len() > limit;
rows.truncate(limit);
Ok(RowBatch {
rows,
warnings,
truncated,
})
}
fn format_state_warning(warning: RestoreWarning) -> String {
format!("isolated malformed recorder row: {warning:?}")
}
#[cfg(test)]
mod tests {
use super::*;
use homecore::{Context, EntityId};
use homecore_recorder::Recorder;
#[tokio::test]
async fn restores_registries_before_states_and_isolates_bad_rows() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(
dir.path().join("core.entity_registry"),
r#"{"version":1,"minor_version":13,"key":"core.entity_registry","data":{"entities":[
{"entity_id":"sensor.good","unique_id":null,"platform":"test","name":null,"disabled_by":null,"area_id":null,"device_id":"dev1","entity_category":null,"config_entry_id":null},
{"entity_id":"INVALID","platform":"bad"}
]}}"#,
)
.unwrap();
std::fs::write(
dir.path().join("core.device_registry"),
r#"{"version":1,"minor_version":13,"key":"core.device_registry","data":{"devices":[
{"id":"dev1","config_entries":[],"identifiers":[],"connections":[],"manufacturer":null,"model":null,"model_id":null,"name":"Device","name_by_user":null,"sw_version":null,"hw_version":null,"serial_number":null,"via_device_id":null,"area_id":null,"entry_type":null,"disabled_by":null,"configuration_url":null,"labels":[],"primary_config_entry":null}
]}}"#,
)
.unwrap();
let recorder = Recorder::open("sqlite::memory:").await.unwrap();
let source = HomeCore::new();
source.states().set(
EntityId::parse("sensor.good").unwrap(),
"on",
serde_json::json!({"restorable": true}),
Context::new(),
);
let mut receiver = source.states().subscribe();
// Force one recorder row from a real state event.
source.states().set(
EntityId::parse("sensor.good").unwrap(),
"off",
serde_json::json!({"restorable": true}),
Context::new(),
);
recorder
.record_state(&receiver.recv().await.unwrap())
.await
.unwrap();
let target = HomeCore::new();
let report =
restore_startup(&target, Some(&recorder), dir.path(), MAX_REGISTRY_ENTRIES).await;
assert_eq!(
report.phases,
vec![
RestorePhase::EntityRegistry,
RestorePhase::DeviceRegistry,
RestorePhase::States
]
);
assert_eq!(report.entity_entries, 1);
assert_eq!(report.device_entries, 1);
assert_eq!(report.states, 1);
assert!(!report.warnings.is_empty());
let state = target
.states()
.get(&EntityId::parse("sensor.good").unwrap())
.unwrap();
assert!(state.context.is_restoration());
}
}
+3 -3
View File
@@ -31,7 +31,7 @@ export function demoMode() {
export const api = {
base: '',
token: () => { try { return localStorage.getItem('homecore_token') || 'dev-token'; } catch { return 'dev-token'; } },
token: () => { try { return localStorage.getItem('homecore_token') || ''; } catch { return ''; } },
isDemo: (key) => !!demoFlags[key],
anyDemo: () => demoMode() && Object.keys(demoFlags).length > 0,
demoMode,
@@ -76,7 +76,7 @@ export const api = {
async callService(domain, service, data) { return this._post(`/api/services/${domain}/${service}`, data); },
async setState(entityId, state, attributes) { return this._post(`/api/states/${entityId}`, { state, attributes: attributes || {} }); },
// ── gateway /api/homecore/* + /api/events (§11.2) ─────────────────
// ── gateway /api/homecore/* (§11.2) ───────────────────────────────
async appliance() { return this._data('appliance', '/api/homecore/appliance', (m) => m.applianceHealth()); },
async seeds() { return this._data('fleet', '/api/homecore/seeds', (m) => m.seeds()); },
async seed(id) { return this._data('fleet', '/api/homecore/seeds/' + encodeURIComponent(id), (m) => m.seed(id)); },
@@ -93,7 +93,7 @@ export const api = {
async witnessLog(page = 0, size = 12) { return this._data('audit', `/api/homecore/witness?page=${page}&size=${size}`, (m) => m.witnessLog(page, size)); },
async privacyModes() { return this._data('audit', '/api/homecore/privacy', (m) => m.privacyModes()); },
async setPrivacy(seed, modeValue) { if (demoMode()) return { seed, mode: modeValue }; return this._post('/api/homecore/privacy', { seed, mode: modeValue }); },
async eventHistory(n = 40) { return this._data('events', `/api/events?limit=${n}`, (m) => m.recentEvents(n)); },
async eventHistory(n = 40) { return this._data('events', `/api/homecore/events?limit=${n}`, (m) => m.recentEvents(n)); },
recentEvents(n) { return this.eventHistory(n); }, // back-compat alias (async)
async settings() { return this._data('settings', '/api/homecore/settings', (m) => m.settings()); },
async automations() { return this._data('automations', '/api/homecore/automations', () => []); },
@@ -1,6 +1,6 @@
// §4.8 Event Bus & Automation Feed — ADR-131 / ADR-129.
//
// Live event stream (seeded from /api/events, then prepended live from
// Live event stream (seeded from /api/homecore/events, then prepended live from
// the shared WS bus — never polled, §2/§4.4), a context-causality
// breadcrumb on row expand (Context.id → parent_id → grandparent_id),
// and a trigger→condition→action automation builder (ADR-129 scope:
@@ -50,7 +50,7 @@ export default {
root.appendChild(sectionHeader('Event Bus & Automation', 'Live entity events + causality + automation builder (ADR-131 §4.8, ADR-129)'));
if (api.isDemo('events')) {
root.appendChild(banner('DEMO — event history is contract-conformant mock data until the live /api/events feed lands (§7.1). New rows still arrive over the WS bus.', 'amber'));
root.appendChild(banner('DEMO — event history is contract-conformant mock data until the live /api/homecore/events feed lands (§7.1). New rows still arrive over the WS bus.', 'amber'));
}
// ── live lag indicator (top, fed by the shared WS bus) ──────────
+34 -3
View File
@@ -47,6 +47,9 @@ pub struct Context {
}
impl Context {
/// Marker stored on snapshots loaded from durable state at startup.
pub const RESTORE_USER_ID: &'static str = "homecore.restore";
pub fn new() -> Self {
Self::default()
}
@@ -66,6 +69,20 @@ impl Context {
parent_id: Some(parent.id),
}
}
/// Create a fresh context that identifies a startup restoration. The
/// persisted context, when valid, is retained as the causal parent.
pub fn restoration(parent_id: Option<Uuid>) -> Self {
Self {
id: Uuid::new_v4(),
user_id: Some(Self::RESTORE_USER_ID.to_owned()),
parent_id,
}
}
pub fn is_restoration(&self) -> bool {
self.user_id.as_deref() == Some(Self::RESTORE_USER_ID)
}
}
impl Default for Context {
@@ -84,9 +101,23 @@ impl Default for Context {
#[derive(Clone, Debug)]
pub enum SystemEvent {
StateChanged(StateChangedEvent),
ServiceRegistered { domain: String, service: String },
ServiceRemoved { domain: String, service: String },
ComponentLoaded { component: String },
ServiceCalled {
domain: String,
service: String,
data: serde_json::Value,
context: Context,
},
ServiceRegistered {
domain: String,
service: String,
},
ServiceRemoved {
domain: String,
service: String,
},
ComponentLoaded {
component: String,
},
HomeCoreStart,
HomeCoreStarted,
HomeCoreStop,
+74 -6
View File
@@ -6,7 +6,7 @@
use std::sync::Arc;
use crate::bus::EventBus;
use crate::registry::EntityRegistry;
use crate::registry::{DeviceRegistry, EntityRegistry};
use crate::service::ServiceRegistry;
use crate::state::StateMachine;
@@ -20,16 +20,19 @@ struct HomeCoreInner {
pub states: StateMachine,
pub services: ServiceRegistry,
pub entities: EntityRegistry,
pub devices: DeviceRegistry,
}
impl HomeCore {
pub fn new() -> Self {
let bus = EventBus::new();
Self {
inner: Arc::new(HomeCoreInner {
bus: EventBus::new(),
states: StateMachine::new(),
services: ServiceRegistry::new(),
states: StateMachine::with_event_bus(bus.clone()),
services: ServiceRegistry::with_event_bus(bus.clone()),
bus,
entities: EntityRegistry::new(),
devices: DeviceRegistry::new(),
}),
}
}
@@ -49,6 +52,10 @@ impl HomeCore {
pub fn entities(&self) -> &EntityRegistry {
&self.inner.entities
}
pub fn devices(&self) -> &DeviceRegistry {
&self.inner.devices
}
}
impl Default for HomeCore {
@@ -61,15 +68,76 @@ impl Default for HomeCore {
mod tests {
use super::*;
use crate::entity::EntityId;
use crate::event::Context;
use crate::event::{Context, SystemEvent};
use crate::service::{FnHandler, ServiceCall, ServiceName};
#[tokio::test]
async fn end_to_end_set_then_get() {
let hc = HomeCore::new();
let id = EntityId::parse("light.kitchen").unwrap();
hc.states().set(id.clone(), "on", serde_json::json!({"brightness": 200}), Context::new());
hc.states().set(
id.clone(),
"on",
serde_json::json!({"brightness": 200}),
Context::new(),
);
let snap = hc.states().get(&id).unwrap();
assert_eq!(snap.state, "on");
assert_eq!(snap.attributes["brightness"], 200);
}
#[tokio::test]
async fn state_changes_are_published_on_shared_system_bus() {
let hc = HomeCore::new();
let mut rx = hc.bus().subscribe_system();
let id = EntityId::parse("light.kitchen").unwrap();
hc.states()
.set(id.clone(), "on", serde_json::json!({}), Context::new());
let event = rx.recv().await.unwrap();
match event {
SystemEvent::StateChanged(change) => {
assert_eq!(change.entity_id, id);
assert_eq!(change.new_state.unwrap().state, "on");
}
other => panic!("expected StateChanged, got {other:?}"),
}
}
#[tokio::test]
async fn service_calls_are_published_on_shared_system_bus() {
let hc = HomeCore::new();
let service = ServiceName::new("light", "turn_on");
hc.services()
.register(
service.clone(),
FnHandler(|_| async { Ok(serde_json::json!({})) }),
)
.await;
let mut rx = hc.bus().subscribe_system();
hc.services()
.call(ServiceCall {
name: service,
data: serde_json::json!({"brightness": 42}),
context: Context::new(),
})
.await
.unwrap();
match rx.recv().await.unwrap() {
SystemEvent::ServiceCalled {
domain,
service,
data,
..
} => {
assert_eq!(domain, "light");
assert_eq!(service, "turn_on");
assert_eq!(data["brightness"], 42);
}
other => panic!("expected ServiceCalled, got {other:?}"),
}
}
}
+8 -9
View File
@@ -11,7 +11,7 @@
//! - [`state`] — `StateMachine`: DashMap-backed concurrent state store
//! - [`bus`] — `EventBus`: tokio broadcast wiring for system + domain events
//! - [`service`] — `ServiceRegistry` (stub; full mpsc dispatch lands in P2)
//! - [`registry`] — `EntityRegistry` (in-memory P1; persistence lands in P2)
//! - [`registry`] — in-memory entity and device registries, restored by the server
//! - [`homecore`] — `HomeCore` runtime coordinator: holds bus + states + services
//!
//! ## Threading model
@@ -23,31 +23,30 @@
//!
//! ## What's NOT here yet (deferred to P2+)
//!
//! - Persistence of entity registry to `.homecore/storage/core.entity_registry`
//! - Automatic persistence of registry mutations (startup restoration exists)
//! - Schema validation (`schemas` module from §3 stub)
//! - Service handler mpsc dispatch (`service::ServiceRegistry::call`)
//! - Device registry (mirror of HA's `core.device_registry`)
//! - Witness chain integration (ADR-028)
//!
//! Each is marked `// TODO P2:` at the relevant call site.
pub mod bus;
pub mod entity;
pub mod event;
pub mod state;
pub mod bus;
pub mod service;
pub mod registry;
pub mod service;
pub mod state;
mod homecore;
pub use homecore::HomeCore;
pub use bus::EventBus;
pub use entity::{EntityId, EntityIdError, State};
pub use event::{Context, DomainEvent, EventType, StateChangedEvent, SystemEvent};
pub use state::StateMachine;
pub use bus::EventBus;
pub use registry::{DeviceEntry, DeviceRegistry, EntityCategory, EntityEntry, EntityRegistry};
pub use service::{ServiceCall, ServiceError, ServiceName, ServiceRegistry};
pub use registry::{EntityCategory, EntityEntry, EntityRegistry};
pub use state::StateMachine;
/// HOMECORE protocol/data-model version. Bumped when the public surface
/// or on-disk persistence schema changes in a backwards-incompatible way.

Some files were not shown because too many files have changed in this diff Show More