fix(ci): perf job gates on the real frame-budget guard, not TDD stubs (#915 )

After #914 fixed collection, the perf job actually ran the suite and exposed that test_api_throughput.py / test_inference_speed.py are TDD red-phase stubs (every test suffixed `_should_fail_initially`) that time a *mock that sleeps* — not a real perf signal. They carry machine- dependent wall-clock asserts (actual_rps >= 40, batch_time < individual_time) that are inherently flaky on shared CI runners, plus a cross-class fixture-scope bug (`fixture 'standard_model' not found`). Result: 3 failed, 10 errored — by design, not a regression. Forcing those green would manufacture a false signal. Instead, gate only on test_frame_budget.py, which times the *real* CSIProcessor pipeline against the ADR 50 ms per-frame budget (single-frame, p95/100-frames, +Doppler) — a genuine regression guard. Verified locally: 3 passed. The stub files remain in-repo for local TDD; they re-enter CI when their features are implemented and the mock-timing asserts are made deterministic.
fix(ci): perf tests — use python -m pytest so src import resolves (#914 )
2026-06-09 10:13:17 +00:00 · 2026-06-02 18:31:55 +02:00 · 2026-06-02 18:12:00 +02:00 · 2026-06-02 17:36:43 +02:00
1 changed files with 20 additions and 7 deletions
@@ -282,13 +282,26 @@ jobs:
      env:
        MOCK_POSE_DATA: "true"
      run: |
-        # The repo's performance suite is pytest (test_api_throughput.py,
-        # test_frame_budget.py, test_inference_speed.py) — there is no
-        # locustfile.py, so the old `locust -f tests/performance/locustfile.py`
-        # command always failed with "Could not find ...". Run the real suite.
-        # -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100
-        # flags (pytest-cov isn't installed here and 100% cov is for unit tests).
-        pytest tests/performance/ -o addopts="" -v --junitxml=perf-junit.xml
+        # Gate only on the genuine, deterministic perf guard:
+        # test_frame_budget.py times the *real* CSIProcessor pipeline against
+        # the ADR 50 ms per-frame budget (single-frame, p95 over 100 frames,
+        # +Doppler) — a true regression signal.
+        #
+        # test_api_throughput.py / test_inference_speed.py are excluded: every
+        # test there is a TDD red-phase stub (suffix `_should_fail_initially`)
+        # that times a *mock that sleeps* — meaningless as a perf signal, with
+        # machine-dependent wall-clock asserts (e.g. `actual_rps >= 40`,
+        # `batch_time < individual_time`) that are inherently flaky on shared
+        # CI runners, plus a cross-class fixture-scope bug. Forcing them green
+        # would be manufacturing a false signal; they stay in-repo for local
+        # TDD but do not gate CI until the underlying features are implemented.
+        #
+        # `python -m pytest` (not the bare `pytest` script) puts the cwd
+        # (archive/v1) on sys.path so `from src.core...` resolves — the bare
+        # script omits cwd and raises ModuleNotFoundError: No module named 'src'.
+        # -o addopts="" drops the root pyproject's --cov/--cov-fail-under=100.
+        python -m pytest tests/performance/test_frame_budget.py \
+          -o addopts="" -v --junitxml=perf-junit.xml

    - name: Upload performance results
      if: always()