Files
ruvnet--RuView/v2/crates/cog-pose-estimation/cog/artifacts/train_results.json
T
rUv 3314c8db8d feat(cog-pose-estimation): scaffold first Cog from this repo (ADR-100 + ADR-101) (#642)
* feat(cog-pose-estimation): scaffold first Cog from this repo (ADR-100 + ADR-101)

Adds the foundation for the pose-estimation Cog that ships from this
repo into Cognitum V0 appliances. Companion ADR-225 + crate land in
cognitum-one/v0-appliance.

ADRs:
* ADR-100 formalises the Cognitum Cog packaging spec — on-device
  layout under /var/lib/cognitum/apps/<id>/, manifest.json schema
  (incl. new binary_sha256 + binary_signature fields), GCS hosting
  convention, repo source layout, build pipeline, and the four-verb
  runtime contract (version | manifest | health | run). Documents the
  convention I reverse-engineered from inspecting installed cogs on a
  live cognitum-v0 appliance — `anomaly-detect`, `presence`,
  `seizure-detect`, etc.
* ADR-101 designs the pose-estimation Cog itself: where it sits in
  the wifi-densepose pipeline (encoder init from
  ruvnet/wifi-densepose-pretrained, 17-keypoint regression head),
  what gets shipped per target arch (arm / x86_64 / hailo8 /
  hailo10), acceptance gates (PCK@20 explicitly deferred to #640 —
  this ADR ships the vehicle, not the accuracy).

Crate v2/crates/cog-pose-estimation/:
* Cargo.toml + workspace member declaration with a hailo feature gate
  so the binary builds without the Hailo SDK in CI.
* main.rs implements the four-verb CLI exactly per ADR-100.
* config.rs / manifest.rs / publisher.rs / inference.rs / runtime.rs —
  small modules, each <100 lines.
* publisher.rs emits ADR-100 structured JSON events.
* inference.rs is a stub that produces a centred-skeleton baseline
  with confidence=0 (honest: no trained weights wired in yet).
* runtime.rs subscribes to /api/v1/sensing/latest, slides a
  56*20 window, runs the engine, emits pose.frame events.
* cog/manifest.template.json + cog/config.schema.json define the
  release artifact + runtime config schemas.
* cog/Makefile holds build / sign / upload targets.
* tests/smoke.rs covers manifest roundtrip + engine I/O surface.

Verified locally:
* cargo check -p cog-pose-estimation: clean.
* cargo test  -p cog-pose-estimation: 4/4 pass.
* ./target/release/cog-pose-estimation {version,manifest,health}:
  all emit the right contract output.

This commit contains scaffolding only; the actual trained weights and
Hailo HEF cross-compile come in follow-ups tracked in #640 and the
companion v0-appliance branch.

* feat(cog-pose-estimation): first measured run — Candle CUDA on RTX 5080

Trained pose_v1 on ruvultra (RTX 5080) via Candle 0.9 + cuda feature
against the same 1,077-sample paired session that produced 0%/0% PCK
in #640 with the pure-JS SPSA trainer. First real numbers:

  PCK@20 = 3.0%   (up from 0.0%)
  PCK@50 = 18.5%  (up from 0.0%)
  MPJPE  = 0.093  (down from 0.66, ~7x improvement)

400 epochs in 2.1 s wall time, full-batch, ~5 ms/epoch. Loss curve
0.181 -> 0.014 over the run, eval 0.010. Per-joint reveals the model
leans on right-side proximal joints (r_hip 77% PCK@50, r_knee 35%,
l_elbow 26%) — consistent with the camera framing in the source
recording. Distal joints (wrists, ankles) and face joints are still
near-random, consistent with the 56-subcarrier / 20-frame input not
carrying fine-grained spatial info at 1077 samples.

This commit:

* Adds v2/crates/cog-pose-estimation/cog/artifacts/{pose_v1.safetensors,
  train_results.json} so the cog dir now contains a real reference
  artifact, not just scaffold.
* Updates cog/README.md "Status" block with the measured numbers,
  per-joint table, and an honest reading of where the model
  succeeds vs where the data is the bottleneck.
* Adds docs/benchmarks/pose-estimation-cog.md as the canonical
  benchmark log — append-only, one section per published run.
* Appends a "First measured run" section to ADR-101 referencing
  the new benchmark file.

Still pending in the follow-up:
* Wire pose_v1.safetensors into src/inference.rs (replace stub).
* ONNX export (Candle lacks a writer — needs external conversion).
* Hailo HEF cross-compile + cluster deploy.

The data-bound gap to PCK@20 >= 35% is tracked in #640.

* feat(cog-pose-estimation): wire real weights — cog is no longer a stub

Replaces the centred-skeleton stub in src/inference.rs with a real
Candle-based loader that reads cog/artifacts/pose_v1.safetensors and
runs the trained Conv1d encoder + MLP pose head on every incoming CSI
window.

What changes:

* src/inference.rs: PoseNet mirrors the training script's architecture
  exactly — Conv1d(56->64, k=3 d=1), Conv1d(64->128, k=3 d=2),
  Conv1d(128->128, k=3 d=4), mean over time, Linear(128->256)+ReLU,
  Linear(256->34)+sigmoid -> reshape [17, 2]. The InferenceEngine
  searches a sensible candidate list for the weights file
  (/var/lib/cognitum/apps/pose-estimation/, ./pose_v1.safetensors,
  ./cog/artifacts/, repo-root, v2/-relative) and falls back to the
  stub when none are present so the cog still satisfies ADR-100.
* Cargo.toml: adds candle-core 0.9 + candle-nn 0.9 (no-default-features,
  CPU build by default) + safetensors 0.4. New `cuda` feature opt-in
  for GPU inference on hosts that have it. Drops the unused
  wifi-densepose-train path dep from the default build path.
* src/main.rs + src/publisher.rs: health.ok event now carries
  `backend` (candle-cuda | candle-cpu | stub) and the synthetic
  output confidence, so operators can tell at a glance whether the
  cog loaded its weights or fell back to the stub.
* tests/smoke.rs: adds `real_weights_load_when_available` which
  asserts the loaded engine reports backend=candle-* and emits
  non-zero confidence — exactly the signal that proves we're not
  silently degrading to the stub.

Verified locally:

* `cargo check -p cog-pose-estimation --no-default-features` — clean
* `cargo test  -p cog-pose-estimation --no-default-features` — 5/5 pass
* `./target/release/cog-pose-estimation health` emits:
  {"event":"health.ok","fields":{"backend":"candle-cpu","cog":"pose-estimation","synthetic_output_confidence":0.185}}
  — 0.185 is the published PCK@50 from cog/artifacts/train_results.json,
  emitted by the real Candle inference path (would be 0.0 if it had
  fallen back to the stub).

The cog now runs the trained pose_v1 model end-to-end. Accuracy is
still bounded by the underlying 1077-sample training data (PCK@20
3.0%, PCK@50 18.5% per docs/benchmarks/pose-estimation-cog.md) — that
gap is data-bound and tracked in #640. ONNX export + Hailo HEF
cross-compile remain follow-ups.

* docs(benchmarks): measure cog-pose-estimation cold-start latency

100 sequential `cog-pose-estimation health` invocations average 76.2 ms
each on a Windows x86_64 host using the `candle-cpu` backend. Each
invocation re-loads pose_v1.safetensors and runs one synthetic forward
pass, so this is the worst-case cold-start path. Long-running `run`
inference will be sub-millisecond per frame once the model is loaded.

Updates the benchmarks doc accordingly.

* feat(cog-pose-estimation): ONNX export — pose_v1.onnx + scripts/export-onnx.py

Adds the canonical ONNX artifact that unblocks downstream Hailo HEF
cross-compile + ONNX Runtime benchmarks. Generated on ruvultra (torch
2.12.0 + CUDA), 12,059 bytes, opset 18, dynamic batch axis.

* scripts/export-onnx.py: mirrors the Candle inference architecture in
  PyTorch (Conv1d 56->64, 64->128, 128->128 + Linear 128->256->34), pure-
  python safetensors loader (no extra pip dep), exports via
  torch.onnx.export, then verifies via onnx.checker.check_model and
  numerical parity against the torch reference.
* Verified parity vs torch: max |torch - onnx| = 8.94e-8 (1e-5
  threshold). Effectively bit-perfect.
* v2/crates/cog-pose-estimation/cog/artifacts/pose_v1.onnx — the
  artifact itself, 12 KB.
* docs/benchmarks/pose-estimation-cog.md — adds an ONNX export
  section with the verification numbers.

Next: Hailo HEF cross-compile (still gated on Hailo SDK on a
self-hosted runner) and ONNX Runtime latency benchmarks on each
target arch.

* feat(cog-pose-estimation): release v0.0.1 — signed aarch64 binary on GCS

End-to-end deploy: cross-compiled to aarch64-unknown-linux-gnu on
ruvultra, ran via qemu-aarch64-static, then smoke-tested on a real
cognitum-v0 Pi 5. Signed with COGNITUM_OWNER_SIGNING_KEY (Ed25519)
and uploaded to gs://cognitum-apps/cogs/arm/.

Real-hardware results on cognitum-v0 (Pi 5):
  health: backend=candle-cpu, confidence=0.185, real weights loaded
  30x sequential `health`: 0.251 s total -> 8.4 ms / invocation (cold)

GCS release artifacts (publicly downloadable):
  binary:  3,741,976 bytes
    sha256 1e1a7d3dd01ca05d5bfc5dbb142a5941b7866ed9f3224a21edc04d3f09a99bf5
  weights:   507,032 bytes
    sha256 eb249b9a6b2e10130437a10976ed0230b0d085f86a0553d7226e1ae6eae4b9e5
  signature (Ed25519, b64): LUN7xqLPYD3MFzm5dKB5MnYU0LvoRtek5ci5KiKPHBg+Xo6xuazwokn2Dw2JPMaLYJzmWn/SpT4djuR7hYvVDw==

Adds:
* v2/crates/cog-pose-estimation/cog/artifacts/manifest.json — the
  release-pipeline-produced manifest with all fields filled in per
  ADR-100, including arch, target_triple, signature, and a
  build_metadata block carrying the validation PCK numbers.
* docs/benchmarks/pose-estimation-cog.md — new sections covering
  the real Pi 5 smoke (8.4 ms cold-start) and the signed GCS
  release artifacts.

Verified by downloading the binary anonymously from GCS and
re-computing the sha256 — matches the locally-computed sha exactly.
Signature decoded to the expected 64-byte Ed25519 length.

Closes the GCS-upload acceptance criterion from ADR-100; the only
pending work is Hailo HEF cross-compile (still SDK-gated) and an
x86_64 release alongside this arm release.

* docs(benchmarks): record live cognitum-v0 install + 5-sec smoke run

Adds the "Live appliance install" section documenting what happened
when the signed v0.0.1 binary + weights were installed under
/var/lib/cognitum/apps/pose-estimation/ on cognitum-v0 (the V0
cluster leader).

* Layout matches the existing anomaly-detect / presence / seizure-
  detect cogs exactly — the Cogs dashboard at
  http://cognitum-v0:9000/cogs auto-discovers entries.
* `cog-pose-estimation run` ran for 5 seconds in the background and
  cleanly emitted run.started + structured WARN events for the
  missing local sensing-server on :3000 (cognitum-v0's actual CSI
  source is ruview-vitals-worker on :50054, not :3000). No crashes,
  no NaN, no leaks.
* Wiring `sensing_url` to the appliance-native source is a separate
  Day-2 integration task.
2026-05-19 17:03:09 -04:00

573 lines
13 KiB
JSON

{
"backend": "candle-cuda",
"data": {
"eval_samples": 216,
"split": "temporal_80_20",
"split_timestamp": "2026-05-19T17:38:45.486Z",
"total_samples": 1077,
"train_samples": 861
},
"encoder_init": "random",
"epoch_losses": [
0.1808941662311554,
0.16265815496444702,
0.13955898582935333,
0.12225159257650375,
0.10377667844295502,
0.08922480046749115,
0.076103076338768,
0.06308665871620178,
0.049426380544900894,
0.039140596985816956,
0.030129408463835716,
0.025303713977336884,
0.022442471235990524,
0.02088615857064724,
0.02010779082775116,
0.01956109143793583,
0.01948179118335247,
0.019212622195482254,
0.019074730575084686,
0.018810957670211792,
0.01868920773267746,
0.01838303543627262,
0.018172571435570717,
0.017943259328603745,
0.01760796643793583,
0.01735210232436657,
0.016929639503359795,
0.01662956178188324,
0.016312643885612488,
0.016049085184931755,
0.015733029693365097,
0.01548701710999012,
0.015283167362213135,
0.014983722940087318,
0.014812562614679337,
0.01465131901204586,
0.014480160549283028,
0.014315342530608177,
0.014290803112089634,
0.014210136607289314,
0.014109139330685139,
0.014035886153578758,
0.014050519093871117,
0.013955573551356792,
0.013999568298459053,
0.014035838656127453,
0.013971822336316109,
0.013921688310801983,
0.013923658058047295,
0.014015297405421734,
0.014005525968968868,
0.013793034479022026,
0.014398499391973019,
0.016041349619627,
0.018437474966049194,
0.019666751846671104,
0.01953406259417534,
0.018313558772206306,
0.016403522342443466,
0.014824355952441692,
0.014008168131113052,
0.013724717311561108,
0.013581405393779278,
0.013707487843930721,
0.01353893056511879,
0.013217244297266006,
0.012987865135073662,
0.012728189118206501,
0.01254442147910595,
0.012492014095187187,
0.012401513755321503,
0.012278808280825615,
0.012222359888255596,
0.012228039093315601,
0.012238679453730583,
0.012207139283418655,
0.012071969918906689,
0.012182669714093208,
0.011957147158682346,
0.011931930668652058,
0.011995002627372742,
0.012032398954033852,
0.011852897703647614,
0.011876476928591728,
0.011844047345221043,
0.011939700692892075,
0.011796612292528152,
0.01177540048956871,
0.011741355061531067,
0.011779669672250748,
0.011744190007448196,
0.011707762256264687,
0.011584608815610409,
0.011752696707844734,
0.011729150079190731,
0.011659013107419014,
0.011693276464939117,
0.011864989064633846,
0.011667383834719658,
0.011718816123902798,
0.01166768092662096,
0.011662120930850506,
0.011931229382753372,
0.012049584649503231,
0.012037307024002075,
0.01206426601856947,
0.012293326668441296,
0.012212480418384075,
0.01250689011067152,
0.012488565407693386,
0.012466518208384514,
0.012616620399057865,
0.012812258675694466,
0.013071495108306408,
0.013044825755059719,
0.01321423426270485,
0.013319150544703007,
0.013587700203061104,
0.013670523650944233,
0.01378133799880743,
0.014047945849597454,
0.013731345534324646,
0.014244080521166325,
0.014112128876149654,
0.014279313385486603,
0.014710888266563416,
0.01515843067318201,
0.014713115990161896,
0.014796034432947636,
0.01475681271404028,
0.014950357377529144,
0.015005035325884819,
0.014768424443900585,
0.015024485997855663,
0.015059541910886765,
0.015051408670842648,
0.015090585686266422,
0.015175160020589828,
0.015102844685316086,
0.015151201747357845,
0.015226155519485474,
0.015032590366899967,
0.015155772678554058,
0.01507557276636362,
0.015160820446908474,
0.015019215643405914,
0.015037509612739086,
0.015222272835671902,
0.015005122870206833,
0.015173210762441158,
0.015132835134863853,
0.027589134871959686,
0.07165955752134323,
0.06373818218708038,
0.06655537337064743,
0.07562592625617981,
0.06909485161304474,
0.05691340193152428,
0.048039719462394714,
0.040047839283943176,
0.034030981361866,
0.02623862214386463,
0.02114911563694477,
0.018268009647727013,
0.01640227809548378,
0.01537158153951168,
0.014892393723130226,
0.014505675993859768,
0.014186820015311241,
0.013841629028320312,
0.013426804915070534,
0.013020739890635014,
0.012673602439463139,
0.012330775149166584,
0.01226764265447855,
0.012166578322649002,
0.012095688842236996,
0.012270377948880196,
0.012516235001385212,
0.012700744904577732,
0.012992565520107746,
0.013367722742259502,
0.013592609204351902,
0.013607893139123917,
0.013697323389351368,
0.013854263350367546,
0.013832741416990757,
0.01367993839085102,
0.013867720030248165,
0.013601685874164104,
0.013631370849907398,
0.013577244244515896,
0.013414927758276463,
0.013450143858790398,
0.013431857340037823,
0.01343410275876522,
0.013244441710412502,
0.013297016732394695,
0.01346137747168541,
0.01331599336117506,
0.014807604253292084,
0.014646961353719234,
0.014483925886452198,
0.014267523773014545,
0.014087164774537086,
0.013921936973929405,
0.013723043724894524,
0.013571077957749367,
0.013395787216722965,
0.013234280981123447,
0.013133431784808636,
0.013057147152721882,
0.012962305918335915,
0.012835373170673847,
0.012728667818009853,
0.012636503204703331,
0.012564707547426224,
0.01253308542072773,
0.012460188008844852,
0.012445810250937939,
0.01240697130560875,
0.012377945706248283,
0.012340536341071129,
0.01233599055558443,
0.012312998063862324,
0.012278364971280098,
0.012224015779793262,
0.012239382602274418,
0.012242404744029045,
0.012323223985731602,
0.012205271050333977,
0.012227945029735565,
0.012205214239656925,
0.012209423817694187,
0.01217598281800747,
0.012150637805461884,
0.01217078510671854,
0.01225175429135561,
0.012216047383844852,
0.012195242568850517,
0.012198278680443764,
0.012190825305879116,
0.012173629365861416,
0.012157510966062546,
0.012140096165239811,
0.012207810766994953,
0.012194979004561901,
0.01217165682464838,
0.01216792967170477,
0.01218471210449934,
0.012194857932627201,
0.012163667008280754,
0.012145694345235825,
0.012135420925915241,
0.012164837680757046,
0.01216159388422966,
0.012148530222475529,
0.012224133126437664,
0.012155838310718536,
0.012177230790257454,
0.012110436335206032,
0.012090248055756092,
0.012101170606911182,
0.012153848074376583,
0.012173553928732872,
0.012172674760222435,
0.012157287448644638,
0.012172986753284931,
0.012137886136770248,
0.012157085351645947,
0.012121357955038548,
0.012135915458202362,
0.012176922522485256,
0.012193577364087105,
0.012180276215076447,
0.012223861180245876,
0.012179303914308548,
0.012176022864878178,
0.012092312797904015,
0.012138010933995247,
0.01214117556810379,
0.012276227585971355,
0.012187770567834377,
0.012211603112518787,
0.012213931418955326,
0.012225016951560974,
0.012142234481871128,
0.012134073302149773,
0.012163194827735424,
0.01223068218678236,
0.012200715951621532,
0.012191612273454666,
0.01220244076102972,
0.01220419630408287,
0.012142208404839039,
0.012142272666096687,
0.01212950050830841,
0.012169948779046535,
0.012184932827949524,
0.012199781835079193,
0.012189080938696861,
0.012251517735421658,
0.012228423729538918,
0.012237711809575558,
0.012216192670166492,
0.012263692915439606,
0.012285872362554073,
0.012329400517046452,
0.012345477007329464,
0.012416589073836803,
0.012419192120432854,
0.012471407651901245,
0.012412074953317642,
0.012433832511305809,
0.01246955618262291,
0.012568573467433453,
0.012632711790502071,
0.01270760502666235,
0.012691991403698921,
0.012749818153679371,
0.012748819775879383,
0.01276922132819891,
0.012770597822964191,
0.012830909341573715,
0.012891922146081924,
0.012974675744771957,
0.01295324694365263,
0.01304001547396183,
0.0130251320078969,
0.013028905726969242,
0.012945529073476791,
0.013016759417951107,
0.013065450824797153,
0.013240920379757881,
0.013167147524654865,
0.013239633291959763,
0.013240372762084007,
0.013296829536557198,
0.01322928350418806,
0.013259101659059525,
0.013233119621872902,
0.013339969329535961,
0.013323795981705189,
0.013341942802071571,
0.013390406966209412,
0.013395088724792004,
0.013347778469324112,
0.013323097489774227,
0.013308844529092312,
0.01338045671582222,
0.013418255373835564,
0.013455703854560852,
0.01349731907248497,
0.013548982329666615,
0.013543978333473206,
0.013514911755919456,
0.013511871919035912,
0.01351082045584917,
0.01348851714283228,
0.013556062243878841,
0.013558348640799522,
0.013616240583360195,
0.013577889651060104,
0.013577991165220737,
0.013531915843486786,
0.013514644466340542,
0.01348655391484499,
0.013568769209086895,
0.013610766269266605,
0.013646356761455536,
0.013650151900947094,
0.013662545941770077,
0.013631481677293777,
0.013629746623337269,
0.01362497080117464,
0.013645497150719166,
0.013664674945175648,
0.013721015304327011,
0.013627894222736359,
0.013688581064343452,
0.013681283220648766,
0.013655297458171844,
0.013539095409214497,
0.013555340468883514,
0.013566684909164906,
0.013745179399847984,
0.013687034137547016,
0.013702981173992157,
0.01367457490414381,
0.013732061721384525,
0.01364122238010168,
0.013664795085787773,
0.013612691313028336,
0.013709086924791336,
0.013684045523405075,
0.013670985586941242,
0.013698549009859562,
0.013667520135641098,
0.013631648384034634,
0.013607441447675228
],
"epochs": 400,
"final_eval_loss": 0.010066533461213112,
"hyperparameters": {
"augmentation": "subcarrier_dropout_10pct (last 50 epochs)",
"base_lr": 0.001,
"batch_mode": "full_batch",
"loss": "SmoothL1 (Huber beta=0.1)",
"optimizer": "AdamW",
"schedule": "cosine",
"weight_decay": 0.01
},
"model": {
"encoder": "Conv1d(56->64->128->128, k=3, dilation=[1,2,4]) + GlobalMeanPool",
"head": "Linear(128->256)->ReLU->Linear(256->34)->Sigmoid",
"parameters": 126562
},
"mpjpe_normalized": 0.09310426687050756,
"pck_at_20": 2.968409586056645,
"pck_at_50": 18.51851851851852,
"per_joint_pck20": [
{
"joint": "nose",
"pck20": 0.4629629629629629
},
{
"joint": "l_eye",
"pck20": 2.7777777777777777
},
{
"joint": "r_eye",
"pck20": 1.8518518518518516
},
{
"joint": "l_ear",
"pck20": 0.0
},
{
"joint": "r_ear",
"pck20": 1.8518518518518516
},
{
"joint": "l_shoulder",
"pck20": 4.62962962962963
},
{
"joint": "r_shoulder",
"pck20": 1.8518518518518516
},
{
"joint": "l_elbow",
"pck20": 1.8518518518518516
},
{
"joint": "r_elbow",
"pck20": 0.0
},
{
"joint": "l_wrist",
"pck20": 3.2407407407407405
},
{
"joint": "r_wrist",
"pck20": 1.3888888888888888
},
{
"joint": "l_hip",
"pck20": 0.0
},
{
"joint": "r_hip",
"pck20": 25.0
},
{
"joint": "l_knee",
"pck20": 2.314814814814815
},
{
"joint": "r_knee",
"pck20": 0.9259259259259258
},
{
"joint": "l_ankle",
"pck20": 1.3888888888888888
},
{
"joint": "r_ankle",
"pck20": 0.9259259259259258
}
],
"per_joint_pck50": [
{
"joint": "nose",
"pck50": 5.092592592592593
},
{
"joint": "l_eye",
"pck50": 8.333333333333332
},
{
"joint": "r_eye",
"pck50": 15.74074074074074
},
{
"joint": "l_ear",
"pck50": 3.2407407407407405
},
{
"joint": "r_ear",
"pck50": 9.722222222222223
},
{
"joint": "l_shoulder",
"pck50": 8.796296296296296
},
{
"joint": "r_shoulder",
"pck50": 19.90740740740741
},
{
"joint": "l_elbow",
"pck50": 26.38888888888889
},
{
"joint": "r_elbow",
"pck50": 4.166666666666666
},
{
"joint": "l_wrist",
"pck50": 24.074074074074073
},
{
"joint": "r_wrist",
"pck50": 12.037037037037036
},
{
"joint": "l_hip",
"pck50": 27.314814814814813
},
{
"joint": "r_hip",
"pck50": 76.85185185185185
},
{
"joint": "l_knee",
"pck50": 20.833333333333336
},
{
"joint": "r_knee",
"pck50": 35.18518518518518
},
{
"joint": "l_ankle",
"pck50": 7.87037037037037
},
{
"joint": "r_ankle",
"pck50": 9.25925925925926
}
],
"train_time_s": 2.058459526
}