mirror of
https://github.com/ruvnet/RuView
synced 2026-07-22 17:23:19 +00:00
ruv
8409f434c9
Root cause: the trained-model filename was `trained-{type}-{%Y%m%d_%H%M%S}` — a
**second-resolution** timestamp. Two training runs of the same type that finish
in the same wall-clock second compute the SAME path and silently overwrite each
other's artifact. This is a real robustness gap (a second rapid run clobbers the
first model), and it surfaced as a flaky test on the Linux CI runner:
`http_train_start_produces_model_and_streams` asserts (via a before/after
directory diff) that a new `.rvf` appeared, but the concurrent
`training_job_streams_real_progress_and_writes_model` test — same `supervised`
type, same second — wrote to the identical path and then deleted it in its own
cleanup, so the diff saw no new file. Windows scheduling happened to avoid the
same-second overlap; Linux CI hit it (the sibling `TRAIN_ENV_LOCK` holders then
failed via std::sync::Mutex poison-cascade after the first panic).
Reproduced locally by isolating the three model-writing tests (which forces the
overlap): ~10/20 runs flaked before; 0/30 after.
Fix: `next_model_id()` appends microseconds (`%6f`) plus a monotonic per-process
`AtomicU64` counter, so every run gets a distinct path even for same-microsecond
concurrent completions. Pinned by `model_ids_are_unique_per_call` (1000 ids, all
distinct) so the scheme can't regress to non-unique. No test-level band-aid
(no added sleeps / serialization) — the collision is removed at the source.
Verified: cargo test -p wifi-densepose-sensing-server -p wifi-densepose-train
--no-default-features — bin 218 passed / 0 failed, all suites 0 failed; the
isolated-tests stress loop is 0/15 flakes post-fix.