mirror of
https://github.com/ruvnet/RuView
synced 2026-06-09 10:13:17 +00:00
27a6edba8b
* feat(examples/three.js): cinematic skinned realtime pose demo + ESP32 CSI bridge
Five-stage example progression exploring three.js helpers (ADR-097 surface) as
a viewer for live RuView sensor data:
1. helpers-demo.html — clean ADR-097 helper reference (GridHelper,
PolarGridHelper, BoxHelper, AxesHelper),
file://-safe, no backend
2. helpers-cinematic.html — same scene + UnrealBloomPass + pseudo-CSI
sonar pings + tomography sweep + procedural
cyber floor + ambient drift particles
3. helpers-skinned.html — replaces sphere skeleton with Mixamo X Bot
via GLTFLoader from threejs.org CDN, plays
bundled animations with additive blending
4. helpers-skinned-fbx.html — same but loads a local Mixamo FBX (needs
serve-demo.py — file:// can't fetch local
siblings). Drop X Bot.fbx alongside.
5. helpers-skinned-realtime.html — webcam → MediaPipe Pose Heavy →
poseWorldLandmarks → direct quaternion
retargeting onto the Mixamo skeleton.
Real ESP32-S3 CSI streamed over WebSocket
from ruvultra (Tailscale, port 8766).
Supporting:
- serve-demo.py threaded HTTP server with no-cache headers
(fixes net::ERR_EMPTY_RESPONSE on the FBX path)
- ruvultra-csi-bridge.py ESP32 RuView firmware tick → WebSocket bridge,
runs as systemd-run unit on ruvultra
Bugs found + fixed along the way (all documented in code comments):
- FBX exports yield TWO parallel Bone trees with identical names; only the
SkinnedMesh.skeleton.bones one drives visible deformation. model.traverse
finds orphans.
- Mixamo FBX nests a zero-length wrapper bone above the real bone, same name.
bone.children[0].getWorldPosition == bone.getWorldPosition → restDir is
(0,0,0) → setFromUnitVectors collapses to identity. Walk past same-named
same-position wrappers when computing tail.
- AnimationMixer.update() with a "stopped" action still mutates bones unless
enabled=false is set.
Retargeting layer in helpers-skinned-realtime.html:
- 12 bones direct quaternion retarget (arms × 2, legs × 2, spine × 3, neck)
- Hips root rotation from shoulder/hip line basis (torso twist + lean)
- Neck aims at ear-midpoint (kp 7+8), not nose (kp 0), to remove the
forward bias of the protruding-nose anchor
- One Euro Filter per landmark per axis (Casiez 2012) — adaptive low-pass
- Visibility-weighted per-bone slerp gain — occluded limbs relax to rest
- URL toggles: ?mirror= ?yflip= ?zflip= ?cnn=0/1/2 ?csi=ws://...
Live CSI integration:
- Bridge parses adaptive_ctrl tick lines (motion/presence/rssi/yield)
- Browser fans single ESP32 reading across 4 UI nodes with phase-shifted
wobble (0.88–1.00 × sin(t·0.55 + offsetᵢ))
- EMA α=0.06 (~3 sec time constant), HUD update throttled 3 Hz
Co-Authored-By: claude-flow <ruv@ruv.net>
* refactor(examples/three.js): organize into demos/screenshots/server/assets + add README
Flatten the 13-file flat layout into purposeful subfolders so the demo
collection has a clean top-level entry point (README.md) and the file roles
are obvious from a directory listing.
Layout:
demos/ 01..05 — numbered for the progression (helpers → cinematic →
skinned → skinned-fbx → skinned-realtime)
screenshots/ one PNG per demo, matching the demo's filename prefix
server/ serve-demo.py + ruvultra-csi-bridge.py
assets/ X Bot.fbx (gitignored, used by demos 04 and 05)
Touched files (beyond the renames):
- 04-skinned-fbx.html, 05-skinned-realtime.html: MODEL_URL now resolves
'../assets/X%20Bot.fbx' instead of './X%20Bot.fbx'
- server/serve-demo.py: chdir() walks 3 levels up to repo root (was 2), and
the URL banner now lists all 5 demos
- .gitignore: comment refresh — points at assets/ and screenshots/
- 05-skinned-realtime.html also picks up in-flight fps-tune work from this
branch (Holistic script, SMOOTH_K URL param, slerp gain scaling) since
those edits and the rename hit the same file
Verified end-to-end:
- python examples/three.js/server/serve-demo.py
- all 5 demos return 200, X Bot.fbx returns 200 from new asset/ path
- demos 04 + 05 render the X Bot mesh; 0 JS errors via browser eval
- screenshots reproduced match the originals
Co-Authored-By: claude-flow <ruv@ruv.net>
154 lines
4.8 KiB
Python
154 lines
4.8 KiB
Python
#!/usr/bin/env python3
|
|
"""ruvultra → browser CSI bridge.
|
|
|
|
Reads adaptive_ctrl tick lines from the ESP32-S3 RuView firmware on
|
|
/dev/ttyACM0 and forwards normalized per-node metrics over a WebSocket
|
|
that the helpers-skinned-realtime demo can subscribe to via Tailscale.
|
|
|
|
Sample serial line (1 Hz cadence from firmware):
|
|
I (22890561) adaptive_ctrl: medium tick: state=6 yield=15pps motion=1.00 presence=5.35 rssi=-33
|
|
|
|
Output JSON (per tick):
|
|
{
|
|
"ts": 1716830400.123,
|
|
"node": 0, # always 0 (single node), client expands to 4
|
|
"motion": 1.00, # raw firmware metric
|
|
"presence": 5.35,
|
|
"rssi": -33,
|
|
"yield_pps": 15,
|
|
"amp": 0.78 # synthesized CSI amplitude in [0..1] for the bar
|
|
}
|
|
|
|
Run on ruvultra:
|
|
python3 -u ruvultra-csi-bridge.py
|
|
"""
|
|
import asyncio
|
|
import builtins
|
|
import json
|
|
import re
|
|
import sys
|
|
import time
|
|
from contextlib import suppress
|
|
|
|
# Force every print to flush — we're often piped to a log file
|
|
_orig_print = builtins.print
|
|
def _print(*a, **kw):
|
|
kw.setdefault("flush", True)
|
|
return _orig_print(*a, **kw)
|
|
builtins.print = _print
|
|
|
|
import serial
|
|
import websockets
|
|
|
|
PORT = "/dev/ttyACM0"
|
|
BAUD = 115200
|
|
WS_HOST = "0.0.0.0"
|
|
WS_PORT = 8766
|
|
|
|
TICK_RE = re.compile(
|
|
r"adaptive_ctrl:\s*\w+\s+tick:\s*"
|
|
r"state=(?P<state>\d+)\s+"
|
|
r"yield=(?P<yield>\d+)pps\s+"
|
|
r"motion=(?P<motion>[\d.]+)\s+"
|
|
r"presence=(?P<presence>[\d.]+)\s+"
|
|
r"rssi=(?P<rssi>-?\d+)"
|
|
)
|
|
|
|
clients = set()
|
|
last_payload = None
|
|
|
|
|
|
def amp_from_metrics(motion, presence, rssi):
|
|
"""Map firmware metrics to a [0..1] CSI-style amplitude."""
|
|
rssi_norm = max(0.0, min(1.0, (rssi + 80) / 50)) # -80..-30 → 0..1
|
|
presence_norm = max(0.0, min(1.0, presence / 8.0)) # cap at 8
|
|
motion_norm = max(0.0, min(1.0, motion)) # already 0..1ish
|
|
return 0.40 * rssi_norm + 0.35 * presence_norm + 0.25 * motion_norm
|
|
|
|
|
|
async def serial_reader_loop():
|
|
global last_payload
|
|
print(f"[bridge] opening {PORT} @ {BAUD}…")
|
|
while True:
|
|
try:
|
|
ser = serial.Serial(PORT, BAUD, timeout=1)
|
|
except (serial.SerialException, OSError) as e:
|
|
print(f"[bridge] serial open failed ({e}); retry in 3s")
|
|
await asyncio.sleep(3)
|
|
continue
|
|
|
|
print(f"[bridge] connected to {PORT}")
|
|
loop = asyncio.get_event_loop()
|
|
try:
|
|
while True:
|
|
line = await loop.run_in_executor(None, ser.readline)
|
|
if not line:
|
|
continue
|
|
try:
|
|
text = line.decode(errors="replace").strip()
|
|
except Exception:
|
|
continue
|
|
m = TICK_RE.search(text)
|
|
if not m:
|
|
continue
|
|
motion = float(m["motion"])
|
|
presence = float(m["presence"])
|
|
rssi = int(m["rssi"])
|
|
payload = {
|
|
"ts": time.time(),
|
|
"node": 0,
|
|
"state": int(m["state"]),
|
|
"yield_pps": int(m["yield"]),
|
|
"motion": motion,
|
|
"presence": presence,
|
|
"rssi": rssi,
|
|
"amp": amp_from_metrics(motion, presence, rssi),
|
|
}
|
|
last_payload = payload
|
|
msg = json.dumps(payload)
|
|
if clients:
|
|
dead = []
|
|
for ws in list(clients):
|
|
try:
|
|
await ws.send(msg)
|
|
except websockets.ConnectionClosed:
|
|
dead.append(ws)
|
|
for d in dead:
|
|
clients.discard(d)
|
|
print(
|
|
f"[tick] motion={motion:.2f} presence={presence:5.2f} "
|
|
f"rssi={rssi:+d} yield={int(m['yield']):3d}pps "
|
|
f"amp={payload['amp']:.2f} clients={len(clients)}"
|
|
)
|
|
except (serial.SerialException, OSError) as e:
|
|
print(f"[bridge] serial error ({e}); reopen in 1s")
|
|
with suppress(Exception):
|
|
ser.close()
|
|
await asyncio.sleep(1)
|
|
|
|
|
|
async def ws_handler(ws):
|
|
addr = ws.remote_address
|
|
clients.add(ws)
|
|
print(f"[ws] client connected: {addr} total={len(clients)}")
|
|
try:
|
|
if last_payload is not None:
|
|
await ws.send(json.dumps(last_payload))
|
|
await ws.wait_closed()
|
|
finally:
|
|
clients.discard(ws)
|
|
print(f"[ws] client gone: {addr} total={len(clients)}")
|
|
|
|
|
|
async def main():
|
|
print(f"[bridge] websocket on ws://{WS_HOST}:{WS_PORT}")
|
|
async with websockets.serve(ws_handler, WS_HOST, WS_PORT):
|
|
await serial_reader_loop()
|
|
|
|
|
|
if __name__ == "__main__":
|
|
try:
|
|
asyncio.run(main())
|
|
except KeyboardInterrupt:
|
|
pass
|