mirror of
https://github.com/ruvnet/RuView
synced 2026-08-04 19:31:42 +00:00
feat(training): two-checkerboard camera↔room calibration for ADR-079 labels (ADR-152 §2.1.3)
Defends the camera-supervised pipeline against PerceptAlign's "coordinate overfitting": MediaPipe keypoints were emitted in raw camera coordinates with no shared frame and no transceiver-geometry metadata — the exact label shape that memorizes deployment layout and collapses cross-layout. - scripts/calibrate-camera-room.py + calibration_lib.py: OpenCV two-checkerboard calibration → versioned bundle JSON (intrinsics, camera→room extrinsics, checkerboard spec, transceiver geometry, sha256 calibration_id). Intrinsics resolve from file > cache > multi-view computation > loud-warning 2-view fallback. - collect-ground-truth.py --calibration <bundle>: every sample gains keypoints_room (unit bearing rays from the camera center in the room frame — documented projective alignment; raw image coords preserved so training chooses), camera_origin_room, calibration_id, and the transceiver geometry stamp. Without the flag, output is byte-identical to before (tested) + a one-line ADR-152 warning. Design finding (recorded for ADR-152): a single planar checkerboard's corner grid is centrosymmetric — the reversed corner ordering fits a ghost camera pose with IDENTICAL reprojection error, so per-board flip disambiguation is mathematically ill-posed. solve_two_board_extrinsics solves the joint wall+floor set over all 4 flip combinations, where the minimum is unique — an independent reason the TWO-checkerboard method is required, beyond what PerceptAlign states. 15 headless pytest tests green (synthetic corners: extrinsics recovery incl. ghost resolution, bundle round-trip + hash stability, ray transforms w/ distortion + cross-resolution, no-calibration byte identity). Co-Authored-By: RuFlo <ruv@ruv.net>
This commit is contained in:
@@ -6,9 +6,19 @@ synchronizes with ESP32 CSI recording from the sensing server.
|
||||
|
||||
Output: JSONL file in data/ground-truth/ with per-frame 17-keypoint COCO poses.
|
||||
|
||||
With --calibration <bundle.json> (produced by scripts/calibrate-camera-room.py,
|
||||
ADR-152 S2.1.3), every record is additionally stamped with room-frame bearing
|
||||
rays for each keypoint, the calibration_id, and the transceiver geometry --
|
||||
the PerceptAlign-style defense against coordinate overfitting. Raw image
|
||||
coordinates are always kept; without depth the room-frame representation is
|
||||
a projective alignment (rays, not 3D points) -- see scripts/calibration_lib.py.
|
||||
Without --calibration the output is byte-identical to the original ADR-079
|
||||
format.
|
||||
|
||||
Usage:
|
||||
python scripts/collect-ground-truth.py --preview --duration 60
|
||||
python scripts/collect-ground-truth.py --server http://192.168.1.10:3000
|
||||
python scripts/collect-ground-truth.py --calibration data/calibration/camera-room.json
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -168,8 +178,23 @@ def main():
|
||||
default="data/ground-truth",
|
||||
help="Output directory (default: data/ground-truth)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--calibration",
|
||||
default=None,
|
||||
help="Camera-room calibration bundle JSON from scripts/calibrate-camera-room.py "
|
||||
"(ADR-152 S2.1.3); adds room-frame keypoint rays + transceiver geometry "
|
||||
"to every record",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if not args.calibration:
|
||||
print(
|
||||
"WARNING: no --calibration bundle; labels stay in raw camera coordinates "
|
||||
"and are layout-brittle (coordinate overfitting, ADR-152 S2.1.3) -- run "
|
||||
"scripts/calibrate-camera-room.py first.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
|
||||
# --- Resolve paths relative to repo root ---
|
||||
repo_root = Path(__file__).resolve().parent.parent
|
||||
output_dir = repo_root / args.output
|
||||
@@ -193,6 +218,25 @@ def main():
|
||||
frame_h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
|
||||
print(f"Camera opened: {frame_w}x{frame_h}")
|
||||
|
||||
# --- Load calibration bundle (ADR-152 S2.1.3) ---
|
||||
calib_ctx = None
|
||||
if args.calibration:
|
||||
# Lazy import keeps the no-calibration path identical to the original.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import calibration_lib
|
||||
|
||||
try:
|
||||
calib_ctx = calibration_lib.load_calibration_context(
|
||||
Path(args.calibration), frame_w, frame_h
|
||||
)
|
||||
except (OSError, ValueError, json.JSONDecodeError) as exc:
|
||||
print(f"ERROR: Cannot load calibration bundle {args.calibration}: {exc}",
|
||||
file=sys.stderr)
|
||||
sys.exit(1)
|
||||
n_nodes = len(calib_ctx.transceiver_geometry.get("nodes", []))
|
||||
print(f"Calibration: {calib_ctx.calibration_id[:23]}... "
|
||||
f"({n_nodes} transceiver node(s)); emitting room-frame keypoint rays")
|
||||
|
||||
# --- Create PoseLandmarker ---
|
||||
options = PoseLandmarkerOptions(
|
||||
base_options=BaseOptions(model_asset_path=str(model_path)),
|
||||
@@ -287,6 +331,10 @@ def main():
|
||||
"n_visible": n_visible,
|
||||
"n_persons": n_persons,
|
||||
}
|
||||
if calib_ctx is not None:
|
||||
# Adds keypoints_room (bearing rays), camera_origin_room,
|
||||
# calibration_id, transceiver_geometry (ADR-152 S2.1.3).
|
||||
record = calibration_lib.augment_record(record, calib_ctx)
|
||||
out_file.write(json.dumps(record) + "\n")
|
||||
frame_count += 1
|
||||
total_confidence += confidence
|
||||
|
||||
Reference in New Issue
Block a user