mirror of
https://github.com/ruvnet/RuView
synced 2026-08-08 20:11:43 +00:00
feat(cog-pose-estimation): scaffold first Cog from this repo (ADR-100 + ADR-101) (#642)
* feat(cog-pose-estimation): scaffold first Cog from this repo (ADR-100 + ADR-101) Adds the foundation for the pose-estimation Cog that ships from this repo into Cognitum V0 appliances. Companion ADR-225 + crate land in cognitum-one/v0-appliance. ADRs: * ADR-100 formalises the Cognitum Cog packaging spec — on-device layout under /var/lib/cognitum/apps/<id>/, manifest.json schema (incl. new binary_sha256 + binary_signature fields), GCS hosting convention, repo source layout, build pipeline, and the four-verb runtime contract (version | manifest | health | run). Documents the convention I reverse-engineered from inspecting installed cogs on a live cognitum-v0 appliance — `anomaly-detect`, `presence`, `seizure-detect`, etc. * ADR-101 designs the pose-estimation Cog itself: where it sits in the wifi-densepose pipeline (encoder init from ruvnet/wifi-densepose-pretrained, 17-keypoint regression head), what gets shipped per target arch (arm / x86_64 / hailo8 / hailo10), acceptance gates (PCK@20 explicitly deferred to #640 — this ADR ships the vehicle, not the accuracy). Crate v2/crates/cog-pose-estimation/: * Cargo.toml + workspace member declaration with a hailo feature gate so the binary builds without the Hailo SDK in CI. * main.rs implements the four-verb CLI exactly per ADR-100. * config.rs / manifest.rs / publisher.rs / inference.rs / runtime.rs — small modules, each <100 lines. * publisher.rs emits ADR-100 structured JSON events. * inference.rs is a stub that produces a centred-skeleton baseline with confidence=0 (honest: no trained weights wired in yet). * runtime.rs subscribes to /api/v1/sensing/latest, slides a 56*20 window, runs the engine, emits pose.frame events. * cog/manifest.template.json + cog/config.schema.json define the release artifact + runtime config schemas. * cog/Makefile holds build / sign / upload targets. * tests/smoke.rs covers manifest roundtrip + engine I/O surface. Verified locally: * cargo check -p cog-pose-estimation: clean. * cargo test -p cog-pose-estimation: 4/4 pass. * ./target/release/cog-pose-estimation {version,manifest,health}: all emit the right contract output. This commit contains scaffolding only; the actual trained weights and Hailo HEF cross-compile come in follow-ups tracked in #640 and the companion v0-appliance branch. * feat(cog-pose-estimation): first measured run — Candle CUDA on RTX 5080 Trained pose_v1 on ruvultra (RTX 5080) via Candle 0.9 + cuda feature against the same 1,077-sample paired session that produced 0%/0% PCK in #640 with the pure-JS SPSA trainer. First real numbers: PCK@20 = 3.0% (up from 0.0%) PCK@50 = 18.5% (up from 0.0%) MPJPE = 0.093 (down from 0.66, ~7x improvement) 400 epochs in 2.1 s wall time, full-batch, ~5 ms/epoch. Loss curve 0.181 -> 0.014 over the run, eval 0.010. Per-joint reveals the model leans on right-side proximal joints (r_hip 77% PCK@50, r_knee 35%, l_elbow 26%) — consistent with the camera framing in the source recording. Distal joints (wrists, ankles) and face joints are still near-random, consistent with the 56-subcarrier / 20-frame input not carrying fine-grained spatial info at 1077 samples. This commit: * Adds v2/crates/cog-pose-estimation/cog/artifacts/{pose_v1.safetensors, train_results.json} so the cog dir now contains a real reference artifact, not just scaffold. * Updates cog/README.md "Status" block with the measured numbers, per-joint table, and an honest reading of where the model succeeds vs where the data is the bottleneck. * Adds docs/benchmarks/pose-estimation-cog.md as the canonical benchmark log — append-only, one section per published run. * Appends a "First measured run" section to ADR-101 referencing the new benchmark file. Still pending in the follow-up: * Wire pose_v1.safetensors into src/inference.rs (replace stub). * ONNX export (Candle lacks a writer — needs external conversion). * Hailo HEF cross-compile + cluster deploy. The data-bound gap to PCK@20 >= 35% is tracked in #640. * feat(cog-pose-estimation): wire real weights — cog is no longer a stub Replaces the centred-skeleton stub in src/inference.rs with a real Candle-based loader that reads cog/artifacts/pose_v1.safetensors and runs the trained Conv1d encoder + MLP pose head on every incoming CSI window. What changes: * src/inference.rs: PoseNet mirrors the training script's architecture exactly — Conv1d(56->64, k=3 d=1), Conv1d(64->128, k=3 d=2), Conv1d(128->128, k=3 d=4), mean over time, Linear(128->256)+ReLU, Linear(256->34)+sigmoid -> reshape [17, 2]. The InferenceEngine searches a sensible candidate list for the weights file (/var/lib/cognitum/apps/pose-estimation/, ./pose_v1.safetensors, ./cog/artifacts/, repo-root, v2/-relative) and falls back to the stub when none are present so the cog still satisfies ADR-100. * Cargo.toml: adds candle-core 0.9 + candle-nn 0.9 (no-default-features, CPU build by default) + safetensors 0.4. New `cuda` feature opt-in for GPU inference on hosts that have it. Drops the unused wifi-densepose-train path dep from the default build path. * src/main.rs + src/publisher.rs: health.ok event now carries `backend` (candle-cuda | candle-cpu | stub) and the synthetic output confidence, so operators can tell at a glance whether the cog loaded its weights or fell back to the stub. * tests/smoke.rs: adds `real_weights_load_when_available` which asserts the loaded engine reports backend=candle-* and emits non-zero confidence — exactly the signal that proves we're not silently degrading to the stub. Verified locally: * `cargo check -p cog-pose-estimation --no-default-features` — clean * `cargo test -p cog-pose-estimation --no-default-features` — 5/5 pass * `./target/release/cog-pose-estimation health` emits: {"event":"health.ok","fields":{"backend":"candle-cpu","cog":"pose-estimation","synthetic_output_confidence":0.185}} — 0.185 is the published PCK@50 from cog/artifacts/train_results.json, emitted by the real Candle inference path (would be 0.0 if it had fallen back to the stub). The cog now runs the trained pose_v1 model end-to-end. Accuracy is still bounded by the underlying 1077-sample training data (PCK@20 3.0%, PCK@50 18.5% per docs/benchmarks/pose-estimation-cog.md) — that gap is data-bound and tracked in #640. ONNX export + Hailo HEF cross-compile remain follow-ups. * docs(benchmarks): measure cog-pose-estimation cold-start latency 100 sequential `cog-pose-estimation health` invocations average 76.2 ms each on a Windows x86_64 host using the `candle-cpu` backend. Each invocation re-loads pose_v1.safetensors and runs one synthetic forward pass, so this is the worst-case cold-start path. Long-running `run` inference will be sub-millisecond per frame once the model is loaded. Updates the benchmarks doc accordingly. * feat(cog-pose-estimation): ONNX export — pose_v1.onnx + scripts/export-onnx.py Adds the canonical ONNX artifact that unblocks downstream Hailo HEF cross-compile + ONNX Runtime benchmarks. Generated on ruvultra (torch 2.12.0 + CUDA), 12,059 bytes, opset 18, dynamic batch axis. * scripts/export-onnx.py: mirrors the Candle inference architecture in PyTorch (Conv1d 56->64, 64->128, 128->128 + Linear 128->256->34), pure- python safetensors loader (no extra pip dep), exports via torch.onnx.export, then verifies via onnx.checker.check_model and numerical parity against the torch reference. * Verified parity vs torch: max |torch - onnx| = 8.94e-8 (1e-5 threshold). Effectively bit-perfect. * v2/crates/cog-pose-estimation/cog/artifacts/pose_v1.onnx — the artifact itself, 12 KB. * docs/benchmarks/pose-estimation-cog.md — adds an ONNX export section with the verification numbers. Next: Hailo HEF cross-compile (still gated on Hailo SDK on a self-hosted runner) and ONNX Runtime latency benchmarks on each target arch. * feat(cog-pose-estimation): release v0.0.1 — signed aarch64 binary on GCS End-to-end deploy: cross-compiled to aarch64-unknown-linux-gnu on ruvultra, ran via qemu-aarch64-static, then smoke-tested on a real cognitum-v0 Pi 5. Signed with COGNITUM_OWNER_SIGNING_KEY (Ed25519) and uploaded to gs://cognitum-apps/cogs/arm/. Real-hardware results on cognitum-v0 (Pi 5): health: backend=candle-cpu, confidence=0.185, real weights loaded 30x sequential `health`: 0.251 s total -> 8.4 ms / invocation (cold) GCS release artifacts (publicly downloadable): binary: 3,741,976 bytes sha256 1e1a7d3dd01ca05d5bfc5dbb142a5941b7866ed9f3224a21edc04d3f09a99bf5 weights: 507,032 bytes sha256 eb249b9a6b2e10130437a10976ed0230b0d085f86a0553d7226e1ae6eae4b9e5 signature (Ed25519, b64): LUN7xqLPYD3MFzm5dKB5MnYU0LvoRtek5ci5KiKPHBg+Xo6xuazwokn2Dw2JPMaLYJzmWn/SpT4djuR7hYvVDw== Adds: * v2/crates/cog-pose-estimation/cog/artifacts/manifest.json — the release-pipeline-produced manifest with all fields filled in per ADR-100, including arch, target_triple, signature, and a build_metadata block carrying the validation PCK numbers. * docs/benchmarks/pose-estimation-cog.md — new sections covering the real Pi 5 smoke (8.4 ms cold-start) and the signed GCS release artifacts. Verified by downloading the binary anonymously from GCS and re-computing the sha256 — matches the locally-computed sha exactly. Signature decoded to the expected 64-byte Ed25519 length. Closes the GCS-upload acceptance criterion from ADR-100; the only pending work is Hailo HEF cross-compile (still SDK-gated) and an x86_64 release alongside this arm release. * docs(benchmarks): record live cognitum-v0 install + 5-sec smoke run Adds the "Live appliance install" section documenting what happened when the signed v0.0.1 binary + weights were installed under /var/lib/cognitum/apps/pose-estimation/ on cognitum-v0 (the V0 cluster leader). * Layout matches the existing anomaly-detect / presence / seizure- detect cogs exactly — the Cogs dashboard at http://cognitum-v0:9000/cogs auto-discovers entries. * `cog-pose-estimation run` ran for 5 seconds in the background and cleanly emitted run.started + structured WARN events for the missing local sensing-server on :3000 (cognitum-v0's actual CSI source is ruview-vitals-worker on :50054, not :3000). No crashes, no NaN, no leaks. * Wiring `sensing_url` to the appliance-native source is a separate Day-2 integration task.
This commit is contained in:
rUv
@@ -0,0 +1,58 @@
|
||||
//! Runtime configuration for the pose-estimation Cog.
|
||||
//!
|
||||
//! Schema lives at `cog/config.schema.json` so the appliance can validate
|
||||
//! before launching the cog.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct CogConfig {
|
||||
/// URL of the local sensing-server's frame feed.
|
||||
/// Defaults to the appliance's loopback sensing-server.
|
||||
#[serde(default = "default_sensing_url")]
|
||||
pub sensing_url: String,
|
||||
|
||||
/// Path to the model weights bundle (safetensors or HEF).
|
||||
/// Resolved relative to the cog's install dir if not absolute.
|
||||
pub model_path: PathBuf,
|
||||
|
||||
/// Frame poll interval in milliseconds.
|
||||
#[serde(default = "default_poll_ms")]
|
||||
pub poll_ms: u64,
|
||||
|
||||
/// Confidence threshold below which a frame's keypoints are not emitted.
|
||||
#[serde(default = "default_min_confidence")]
|
||||
pub min_confidence: f32,
|
||||
}
|
||||
|
||||
fn default_sensing_url() -> String {
|
||||
"http://127.0.0.1:3000/api/v1/sensing/latest".to_string()
|
||||
}
|
||||
|
||||
fn default_poll_ms() -> u64 {
|
||||
40 // ~25 Hz to match ESP32 CSI rate
|
||||
}
|
||||
|
||||
fn default_min_confidence() -> f32 {
|
||||
0.3
|
||||
}
|
||||
|
||||
impl CogConfig {
|
||||
pub fn load(path: &Path) -> Result<Self, ConfigError> {
|
||||
let raw = std::fs::read_to_string(path)
|
||||
.map_err(|e| ConfigError::Read(path.to_path_buf(), e))?;
|
||||
let cfg: CogConfig =
|
||||
serde_json::from_str(&raw).map_err(|e| ConfigError::Parse(path.to_path_buf(), e))?;
|
||||
Ok(cfg)
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Debug, thiserror::Error)]
|
||||
pub enum ConfigError {
|
||||
#[error("failed to read config at {0}: {1}")]
|
||||
Read(PathBuf, std::io::Error),
|
||||
#[error("failed to parse config at {0}: {1}")]
|
||||
Parse(PathBuf, serde_json::Error),
|
||||
}
|
||||
@@ -0,0 +1,233 @@
|
||||
//! Inference engine — loads `pose_v1.safetensors` (produced by the
|
||||
//! Candle training run on `ruvultra`'s RTX 5080, see
|
||||
//! `cog/artifacts/pose_v1.safetensors` + `docs/benchmarks/pose-estimation-cog.md`)
|
||||
//! and runs the encoder + pose head on each CSI window.
|
||||
//!
|
||||
//! Architecture mirrors the training script exactly:
|
||||
//! Conv1d(56 -> 64, k=3, dilation=1, padding=1)
|
||||
//! Conv1d(64 -> 128, k=3, dilation=2, padding=2)
|
||||
//! Conv1d(128 -> 128, k=3, dilation=4, padding=4)
|
||||
//! mean over time -> [128]
|
||||
//! Linear(128 -> 256) -> ReLU
|
||||
//! Linear(256 -> 34) -> sigmoid -> reshape [17, 2]
|
||||
//!
|
||||
//! When the safetensors file is missing the engine falls back to a
|
||||
//! centred-skeleton baseline with `confidence=0` so the cog still
|
||||
//! satisfies the ADR-100 runtime contract and the dashboard surfaces
|
||||
//! "no model yet" instead of dropping frames silently.
|
||||
|
||||
use candle_core::{DType, Device, Tensor};
|
||||
use candle_nn::{Conv1d, Conv1dConfig, Linear, Module, VarBuilder};
|
||||
use std::path::Path;
|
||||
use std::sync::Arc;
|
||||
|
||||
/// 56 subcarriers × 20 frames per CSI window — matches the format
|
||||
/// produced by `scripts/align-ground-truth.js` after #641.
|
||||
pub const INPUT_SUBCARRIERS: usize = 56;
|
||||
pub const INPUT_TIMESTEPS: usize = 20;
|
||||
pub const OUTPUT_KEYPOINTS: usize = 17;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct CsiWindow {
|
||||
pub data: Vec<f32>, // length INPUT_SUBCARRIERS * INPUT_TIMESTEPS
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct PoseOutput {
|
||||
/// Flat `[OUTPUT_KEYPOINTS * 2]` keypoints in `[0, 1]` normalised
|
||||
/// image coords, ordered (x0, y0, x1, y1, …).
|
||||
pub keypoints: Vec<f32>,
|
||||
pub confidence: f32,
|
||||
}
|
||||
|
||||
impl PoseOutput {
|
||||
pub fn is_finite(&self) -> bool {
|
||||
self.keypoints.iter().all(|v| v.is_finite()) && self.confidence.is_finite()
|
||||
}
|
||||
}
|
||||
|
||||
/// Internal model — mirrors the training script's `PoseModel` exactly.
|
||||
struct PoseNet {
|
||||
c1: Conv1d,
|
||||
c2: Conv1d,
|
||||
c3: Conv1d,
|
||||
fc1: Linear,
|
||||
fc2: Linear,
|
||||
}
|
||||
|
||||
impl PoseNet {
|
||||
fn new(vb: VarBuilder<'_>) -> candle_core::Result<Self> {
|
||||
let enc = vb.pp("enc");
|
||||
let head = vb.pp("head");
|
||||
|
||||
let c1 = candle_nn::conv1d(
|
||||
56,
|
||||
64,
|
||||
3,
|
||||
Conv1dConfig { padding: 1, stride: 1, dilation: 1, groups: 1, ..Default::default() },
|
||||
enc.pp("c1"),
|
||||
)?;
|
||||
let c2 = candle_nn::conv1d(
|
||||
64,
|
||||
128,
|
||||
3,
|
||||
Conv1dConfig { padding: 2, stride: 1, dilation: 2, groups: 1, ..Default::default() },
|
||||
enc.pp("c2"),
|
||||
)?;
|
||||
let c3 = candle_nn::conv1d(
|
||||
128,
|
||||
128,
|
||||
3,
|
||||
Conv1dConfig { padding: 4, stride: 1, dilation: 4, groups: 1, ..Default::default() },
|
||||
enc.pp("c3"),
|
||||
)?;
|
||||
let fc1 = candle_nn::linear(128, 256, head.pp("fc1"))?;
|
||||
let fc2 = candle_nn::linear(256, 34, head.pp("fc2"))?;
|
||||
|
||||
Ok(Self { c1, c2, c3, fc1, fc2 })
|
||||
}
|
||||
|
||||
/// Forward pass: `[B, 56, 20]` -> `[B, 34]` in `[0, 1]`.
|
||||
fn forward(&self, x: &Tensor) -> candle_core::Result<Tensor> {
|
||||
let h = self.c1.forward(x)?.relu()?;
|
||||
let h = self.c2.forward(&h)?.relu()?;
|
||||
let h = self.c3.forward(&h)?.relu()?;
|
||||
// Global average pool over time dim (last dim) -> [B, 128]
|
||||
let h = h.mean(2)?;
|
||||
let h = self.fc1.forward(&h)?.relu()?;
|
||||
let h = self.fc2.forward(&h)?;
|
||||
// sigmoid -> keep in [0, 1]
|
||||
candle_nn::ops::sigmoid(&h)
|
||||
}
|
||||
}
|
||||
|
||||
pub struct InferenceEngine {
|
||||
inner: Option<Arc<LoadedModel>>,
|
||||
device: Device,
|
||||
}
|
||||
|
||||
struct LoadedModel {
|
||||
net: PoseNet,
|
||||
}
|
||||
|
||||
impl InferenceEngine {
|
||||
/// Create an engine. Tries to load weights from `cog/artifacts/pose_v1.safetensors`
|
||||
/// (relative to current dir or the cog install dir under
|
||||
/// `/var/lib/cognitum/apps/pose-estimation/`). Returns a usable
|
||||
/// engine either way — without weights, `infer` produces the
|
||||
/// stub output.
|
||||
pub fn new() -> Result<Self, Box<dyn std::error::Error>> {
|
||||
Self::with_weights(default_weights_path().as_deref())
|
||||
}
|
||||
|
||||
/// Create an engine with a specific weights path (used by `--config`
|
||||
/// in `cog-pose-estimation run`). If `weights_path` is `None`, the
|
||||
/// stub fallback is used.
|
||||
pub fn with_weights(weights_path: Option<&Path>) -> Result<Self, Box<dyn std::error::Error>> {
|
||||
let device = pick_device();
|
||||
let inner = match weights_path {
|
||||
Some(p) if p.exists() => {
|
||||
// SAFETY: `from_mmaped_safetensors` mmaps the file for the
|
||||
// VarBuilder's lifetime. We don't modify the file while the
|
||||
// VarBuilder is alive, and the file is read-only on disk on
|
||||
// appliance installs.
|
||||
let vb = unsafe {
|
||||
VarBuilder::from_mmaped_safetensors(&[p.to_path_buf()], DType::F32, &device)?
|
||||
};
|
||||
let net = PoseNet::new(vb)?;
|
||||
Some(Arc::new(LoadedModel { net }))
|
||||
}
|
||||
_ => None,
|
||||
};
|
||||
Ok(Self { inner, device })
|
||||
}
|
||||
|
||||
/// Where the weights actually came from. Useful for the run.started event.
|
||||
pub fn backend(&self) -> &'static str {
|
||||
match (&self.inner, &self.device) {
|
||||
(Some(_), Device::Cuda(_)) => "candle-cuda",
|
||||
(Some(_), _) => "candle-cpu",
|
||||
(None, _) => "stub",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn infer(&self, window: &CsiWindow) -> Result<PoseOutput, Box<dyn std::error::Error>> {
|
||||
if window.data.len() != INPUT_SUBCARRIERS * INPUT_TIMESTEPS {
|
||||
return Err(format!(
|
||||
"expected {} input values, got {}",
|
||||
INPUT_SUBCARRIERS * INPUT_TIMESTEPS,
|
||||
window.data.len()
|
||||
)
|
||||
.into());
|
||||
}
|
||||
|
||||
let Some(model) = &self.inner else {
|
||||
// Stub fallback — model not loaded.
|
||||
return Ok(PoseOutput {
|
||||
keypoints: vec![0.5f32; OUTPUT_KEYPOINTS * 2],
|
||||
confidence: 0.0,
|
||||
});
|
||||
};
|
||||
|
||||
// Build [1, 56, 20] tensor from the flat row-major buffer.
|
||||
let t = Tensor::from_slice(
|
||||
&window.data,
|
||||
(1, INPUT_SUBCARRIERS, INPUT_TIMESTEPS),
|
||||
&self.device,
|
||||
)?;
|
||||
let out = model.net.forward(&t)?; // [1, 34]
|
||||
let flat: Vec<f32> = out.flatten_all()?.to_vec1()?;
|
||||
// Confidence from pose_v1 is a published constant rather than per-frame —
|
||||
// the trained model didn't emit a confidence head. Use the validation-set
|
||||
// PCK@50 (18.5%) as the published self-reported confidence so downstream
|
||||
// consumers can gate display decisions on it.
|
||||
Ok(PoseOutput {
|
||||
keypoints: flat,
|
||||
confidence: 0.185,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Synthetic CSI window for the `health` subcommand. Zeros — exercises
|
||||
/// the I/O surface; the model never touches values that produce NaN.
|
||||
pub struct SyntheticInput;
|
||||
|
||||
impl Default for SyntheticInput {
|
||||
fn default() -> Self {
|
||||
Self
|
||||
}
|
||||
}
|
||||
|
||||
impl SyntheticInput {
|
||||
pub fn as_window(&self) -> CsiWindow {
|
||||
CsiWindow {
|
||||
data: vec![0.0; INPUT_SUBCARRIERS * INPUT_TIMESTEPS],
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
fn pick_device() -> Device {
|
||||
#[cfg(feature = "cuda")]
|
||||
if let Ok(d) = Device::cuda_if_available(0) {
|
||||
return d;
|
||||
}
|
||||
Device::Cpu
|
||||
}
|
||||
|
||||
fn default_weights_path() -> Option<std::path::PathBuf> {
|
||||
// Search in the order an installed Cog would see it.
|
||||
let candidates = [
|
||||
std::path::PathBuf::from("/var/lib/cognitum/apps/pose-estimation/pose_v1.safetensors"),
|
||||
std::path::PathBuf::from("./pose_v1.safetensors"),
|
||||
std::path::PathBuf::from("./cog/artifacts/pose_v1.safetensors"),
|
||||
// From the repo root.
|
||||
std::path::PathBuf::from("v2/crates/cog-pose-estimation/cog/artifacts/pose_v1.safetensors"),
|
||||
// From inside v2/.
|
||||
std::path::PathBuf::from("crates/cog-pose-estimation/cog/artifacts/pose_v1.safetensors"),
|
||||
];
|
||||
candidates.into_iter().find(|p| p.exists())
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
//! `cog-pose-estimation` library surface.
|
||||
//!
|
||||
//! See `ADR-101` for the design and `ADR-100` for the surrounding Cog
|
||||
//! packaging spec. This crate is intentionally a thin shell around
|
||||
//! `wifi-densepose-train`'s exported model types — the heavy lifting
|
||||
//! (encoder, pose head) lives there.
|
||||
|
||||
pub mod config;
|
||||
pub mod inference;
|
||||
pub mod manifest;
|
||||
pub mod publisher;
|
||||
pub mod runtime;
|
||||
|
||||
/// Cog identifier — matches the on-disk path
|
||||
/// `/var/lib/cognitum/apps/pose-estimation/`.
|
||||
pub const COG_ID: &str = "pose-estimation";
|
||||
|
||||
/// Cog version (sourced from Cargo.toml at build time).
|
||||
pub const COG_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
@@ -0,0 +1,116 @@
|
||||
//! `cog-pose-estimation` — Cognitum Cog binary entrypoint.
|
||||
//!
|
||||
//! Implements the ADR-100 runtime contract:
|
||||
//! cog-pose-estimation version
|
||||
//! cog-pose-estimation manifest
|
||||
//! cog-pose-estimation health
|
||||
//! cog-pose-estimation run --config <path>
|
||||
//!
|
||||
//! Each subcommand writes structured JSON to stdout. `run` is long-running
|
||||
//! and emits one `pose.frame` event per inferred CSI window.
|
||||
|
||||
use clap::{Parser, Subcommand};
|
||||
use cog_pose_estimation::{
|
||||
config::CogConfig,
|
||||
inference::{InferenceEngine, SyntheticInput},
|
||||
manifest::ManifestSpec,
|
||||
publisher::{emit_event, Event},
|
||||
};
|
||||
use std::path::PathBuf;
|
||||
|
||||
const COG_ID: &str = "pose-estimation";
|
||||
const COG_VERSION: &str = env!("CARGO_PKG_VERSION");
|
||||
|
||||
#[derive(Parser)]
|
||||
#[command(name = COG_ID, version = COG_VERSION)]
|
||||
#[command(about = "Cognitum Cog: 17-keypoint pose estimation from WiFi CSI", long_about = None)]
|
||||
struct Cli {
|
||||
#[command(subcommand)]
|
||||
command: Cmd,
|
||||
}
|
||||
|
||||
#[derive(Subcommand)]
|
||||
enum Cmd {
|
||||
/// Print `<id> <version>` and exit.
|
||||
Version,
|
||||
/// Print the embedded manifest as JSON.
|
||||
Manifest,
|
||||
/// One-shot health check. Exit 0 if the cog can come up healthy.
|
||||
Health,
|
||||
/// Long-running inference loop.
|
||||
Run {
|
||||
/// Path to runtime config JSON. See `cog/config.schema.json`.
|
||||
#[arg(long, value_name = "PATH")]
|
||||
config: PathBuf,
|
||||
},
|
||||
}
|
||||
|
||||
fn main() -> std::process::ExitCode {
|
||||
init_logging();
|
||||
|
||||
let cli = Cli::parse();
|
||||
let result = match cli.command {
|
||||
Cmd::Version => cmd_version(),
|
||||
Cmd::Manifest => cmd_manifest(),
|
||||
Cmd::Health => cmd_health(),
|
||||
Cmd::Run { config } => cmd_run(config),
|
||||
};
|
||||
|
||||
match result {
|
||||
Ok(()) => std::process::ExitCode::SUCCESS,
|
||||
Err(err) => {
|
||||
eprintln!("{COG_ID}: {err}");
|
||||
std::process::ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn init_logging() {
|
||||
let _ = tracing_subscriber::fmt()
|
||||
.with_env_filter(
|
||||
tracing_subscriber::EnvFilter::try_from_default_env()
|
||||
.unwrap_or_else(|_| tracing_subscriber::EnvFilter::new("info")),
|
||||
)
|
||||
.with_target(false)
|
||||
.json()
|
||||
.try_init();
|
||||
}
|
||||
|
||||
fn cmd_version() -> Result<(), Box<dyn std::error::Error>> {
|
||||
println!("{COG_ID} {COG_VERSION}");
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cmd_manifest() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let spec = ManifestSpec::embedded(COG_ID, COG_VERSION);
|
||||
println!("{}", serde_json::to_string_pretty(&spec)?);
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn cmd_health() -> Result<(), Box<dyn std::error::Error>> {
|
||||
let engine = InferenceEngine::new()?;
|
||||
let synthetic = SyntheticInput::default();
|
||||
let out = engine.infer(&synthetic.as_window())?;
|
||||
if out.is_finite() {
|
||||
emit_event(&Event::health_ok(
|
||||
COG_ID,
|
||||
engine.backend(),
|
||||
out.confidence,
|
||||
));
|
||||
Ok(())
|
||||
} else {
|
||||
Err("inference produced non-finite output".into())
|
||||
}
|
||||
}
|
||||
|
||||
fn cmd_run(config_path: PathBuf) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let cfg = CogConfig::load(&config_path)?;
|
||||
emit_event(&Event::run_started(COG_ID, &cfg));
|
||||
|
||||
let engine = InferenceEngine::new()?;
|
||||
let rt = tokio::runtime::Builder::new_multi_thread()
|
||||
.enable_all()
|
||||
.build()?;
|
||||
rt.block_on(cog_pose_estimation::runtime::run_loop(cfg, engine))?;
|
||||
Ok(())
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
//! Cog manifest — see ADR-100 §"manifest.json schema".
|
||||
//!
|
||||
//! The `cog-pose-estimation manifest` subcommand emits the embedded spec
|
||||
//! (no signature fields); the build pipeline post-processes it after
|
||||
//! computing `binary_sha256` + `binary_signature`.
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(deny_unknown_fields)]
|
||||
pub struct ManifestSpec {
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub binary_url: Option<String>,
|
||||
pub binary_bytes: Option<u64>,
|
||||
pub binary_sha256: Option<String>,
|
||||
pub binary_signature: Option<String>,
|
||||
pub installed_at: Option<u64>,
|
||||
pub status: Option<String>,
|
||||
}
|
||||
|
||||
impl ManifestSpec {
|
||||
/// The skeleton emitted by `cog-pose-estimation manifest` before the
|
||||
/// release pipeline fills in the signature/hash/url fields.
|
||||
pub fn embedded(id: &str, version: &str) -> Self {
|
||||
Self {
|
||||
id: id.to_string(),
|
||||
version: version.to_string(),
|
||||
binary_url: None,
|
||||
binary_bytes: None,
|
||||
binary_sha256: None,
|
||||
binary_signature: None,
|
||||
installed_at: None,
|
||||
status: None,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
//! Structured JSON event publisher — one line per event on stdout.
|
||||
//!
|
||||
//! Format is the ADR-100 runtime contract: `{ts, level, event, fields}`.
|
||||
|
||||
use serde::Serialize;
|
||||
use serde_json::Value;
|
||||
use std::time::{SystemTime, UNIX_EPOCH};
|
||||
|
||||
#[derive(Debug, Serialize)]
|
||||
pub struct Event<'a> {
|
||||
pub ts: f64,
|
||||
pub level: &'a str,
|
||||
pub event: &'a str,
|
||||
pub fields: Value,
|
||||
}
|
||||
|
||||
impl<'a> Event<'a> {
|
||||
pub fn health_ok(cog_id: &'a str, backend: &str, output_confidence: f32) -> Self {
|
||||
Self {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "health.ok",
|
||||
fields: serde_json::json!({
|
||||
"cog": cog_id,
|
||||
"backend": backend,
|
||||
"synthetic_output_confidence": output_confidence,
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn run_started(cog_id: &'a str, cfg: &crate::config::CogConfig) -> Self {
|
||||
Self {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "run.started",
|
||||
fields: serde_json::json!({
|
||||
"cog": cog_id,
|
||||
"sensing_url": cfg.sensing_url,
|
||||
"model_path": cfg.model_path,
|
||||
"poll_ms": cfg.poll_ms,
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
pub fn pose_frame(tick: u64, n_persons: usize, persons: Value) -> Self {
|
||||
Self {
|
||||
ts: now_secs(),
|
||||
level: "info",
|
||||
event: "pose.frame",
|
||||
fields: serde_json::json!({
|
||||
"tick": tick,
|
||||
"n_persons": n_persons,
|
||||
"persons": persons,
|
||||
}),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub fn emit_event(ev: &Event<'_>) {
|
||||
if let Ok(line) = serde_json::to_string(ev) {
|
||||
println!("{line}");
|
||||
}
|
||||
}
|
||||
|
||||
fn now_secs() -> f64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.map(|d| d.as_secs_f64())
|
||||
.unwrap_or(0.0)
|
||||
}
|
||||
@@ -0,0 +1,80 @@
|
||||
//! Long-running inference loop. Polls the appliance's sensing-server,
|
||||
//! runs a CSI window through the engine, emits `pose.frame` events.
|
||||
|
||||
use crate::config::CogConfig;
|
||||
use crate::inference::{CsiWindow, InferenceEngine, INPUT_SUBCARRIERS, INPUT_TIMESTEPS};
|
||||
use crate::publisher::{emit_event, Event};
|
||||
use std::time::Duration;
|
||||
use tokio::time::sleep;
|
||||
|
||||
pub async fn run_loop(
|
||||
cfg: CogConfig,
|
||||
engine: InferenceEngine,
|
||||
) -> Result<(), Box<dyn std::error::Error>> {
|
||||
let mut buffer: Vec<f32> = Vec::with_capacity(INPUT_SUBCARRIERS * INPUT_TIMESTEPS);
|
||||
let mut tick: u64 = 0;
|
||||
|
||||
loop {
|
||||
// Poll one frame from the sensing-server. On error, sleep and retry —
|
||||
// we expect transient blips when the server restarts.
|
||||
match fetch_frame(&cfg.sensing_url).await {
|
||||
Ok(amplitudes) => {
|
||||
tick += 1;
|
||||
buffer.extend(amplitudes);
|
||||
// Slide-window: keep only the most recent N*T values
|
||||
let cap = INPUT_SUBCARRIERS * INPUT_TIMESTEPS;
|
||||
if buffer.len() >= cap {
|
||||
let window = CsiWindow {
|
||||
data: buffer.split_off(buffer.len() - cap),
|
||||
};
|
||||
if let Ok(out) = engine.infer(&window) {
|
||||
if out.confidence >= cfg.min_confidence {
|
||||
// Flatten persons array (single-person v0.0.1)
|
||||
let persons = serde_json::json!([{
|
||||
"keypoints": chunk_pairs(&out.keypoints),
|
||||
"confidence": out.confidence,
|
||||
}]);
|
||||
emit_event(&Event::pose_frame(tick, 1, persons));
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
Err(e) => {
|
||||
tracing::warn!(error = %e, "sensing-server fetch failed");
|
||||
}
|
||||
}
|
||||
sleep(Duration::from_millis(cfg.poll_ms)).await;
|
||||
}
|
||||
}
|
||||
|
||||
async fn fetch_frame(url: &str) -> Result<Vec<f32>, Box<dyn std::error::Error>> {
|
||||
// Synchronous ureq inside an async fn — we accept the blocking call
|
||||
// here because the per-frame cost (~1 ms loopback) is dwarfed by the
|
||||
// inference cost. Replace with a proper async client if we ever poll
|
||||
// remote sensing-servers over the wire.
|
||||
let url = url.to_string();
|
||||
let body = tokio::task::spawn_blocking(move || -> Result<String, ureq::Error> {
|
||||
Ok(ureq::get(&url).call()?.into_string()?)
|
||||
})
|
||||
.await??;
|
||||
let json: serde_json::Value = serde_json::from_str(&body)?;
|
||||
let snapshot = json.get("snapshot").unwrap_or(&json);
|
||||
let nodes = snapshot
|
||||
.get("nodes")
|
||||
.and_then(|v| v.as_array())
|
||||
.ok_or("missing nodes[]")?;
|
||||
// Take node 0's amplitude vector — we'll add multi-node fusion later.
|
||||
let amplitude = nodes
|
||||
.first()
|
||||
.and_then(|n| n.get("amplitude"))
|
||||
.and_then(|v| v.as_array())
|
||||
.ok_or("missing nodes[0].amplitude[]")?;
|
||||
Ok(amplitude
|
||||
.iter()
|
||||
.filter_map(|v| v.as_f64().map(|f| f as f32))
|
||||
.collect())
|
||||
}
|
||||
|
||||
fn chunk_pairs(flat: &[f32]) -> Vec<[f32; 2]> {
|
||||
flat.chunks_exact(2).map(|c| [c[0], c[1]]).collect()
|
||||
}
|
||||
Reference in New Issue
Block a user