docs: update ROADMAP.md + USD plan after Phase B completion

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

ROADMAP.md

@@ -10,7 +10,7 @@
 
 | Area | Detail |
 |---|---|
-| Phase A | Flamenco removed, blender-renderer → render-worker Celery container, threejs-renderer removed, MinIO added |
+| Phase A runtime | Flamenco removed from the active pipeline, `render-worker` replaced the old Blender service path, MinIO added; legacy `blender-renderer/` and `threejs-renderer/` directories still remain in the repo |
 | Phase B | Domain-driven project structure, Tenant model + RLS migrations 035/036, Tenant management UI |
 | Phase C | WorkflowDefinition model, standard workflows seeded, React Flow Workflow Editor |
 | Phase D | OCC mesh attributes extraction, Blender integration |
@@ -20,6 +20,26 @@
 | Media cache-bust | `?v={file_size_bytes}` in download URLs + `Cache-Control: no-cache` headers |
 | GPU activation order | Fix: `_activate_gpu()` called before AND after `open_mainfile` to survive engine reset |
 | Material system | Aliases-first lookup, `get_material_library_path()` via AssetLibrary |
+| Pipeline task split | `backend/app/tasks/step_tasks.py` is now a 23-line compatibility shim; active task implementations live in `backend/app/domains/pipeline/tasks/` |
+| Render job tracking | `RenderJobDocument`, `PipelineLogger`, and cancel-via-real-`celery_task_id` are already wired into the render pipeline |
+| Tenant isolation baseline | `TenantContextMiddleware`, JWT `tenant_id`, and the `global_admin` / `tenant_admin` role hierarchy are in place for HTTP requests |
+| Hash groundwork | `compute_step_hash()` exists and `CadFile.step_file_hash` is already persisted during thumbnail processing |
+
+---
+
+## 🔎 Status Snapshot
+
+Verified against the repository on `2026-03-11`.
+
+| Priority | Status | Re-evaluated state |
+|---|---|---|
+| 1. Pipeline Cleanup Foundation | In progress | `step_tasks.py` decomposition is done; dead-code cleanup and `blender_render.py` decomposition are still open |
+| 2. USD Foundation Without Viewer Regression | Not started in code | Decisions are documented, but there is no `export_step_to_usd.py`, `usd_master`, `scene-manifest`, or `partKey` implementation yet |
+| 3. Tessellation and Topology Quality | Not started in code | No `gmsh` install/wiring, no `tessellation_engine` setting, no Admin dropdown yet |
+| 7. Render Job Tracking and Structured Logging | Done | `RenderJobDocument`, migration `048`, `PipelineLogger`, and revoke-by-real-task-id are present |
+| 8. Tenant Isolation Completion | In progress | HTTP-side RLS context is wired; Celery task-side `set_tenant_context()` propagation still needs to be added |
+| 9. Hash-Based Scene Conversion Caching | Partial foundation | Existing `step_file_hash` and STL-cache utilities should be extended, not rebuilt from scratch |
+| 10. UI/UX Polish | Partial | Admin help tooltips, mobile nav, and some empty states exist; notification batching and remaining polish items are still open |
 
 ---
 
@@ -41,9 +61,11 @@ That removes the old assumption that USD work must wait for a Three.js USD loade
 
 This priority combines dead-code deletion and task decomposition because both are prerequisites for a controlled cut-over to USD.
 
+**Status:** In progress. M2 is complete; M1 and M3 remain open.
+
 **Milestones:**
 - M1: Dead code deleted — Pillow block, STL settings, orphaned directories
-- M2: `step_tasks.py` decomposed into `backend/app/tasks/pipeline/` submodules
+- M2: `step_tasks.py` decomposed into `backend/app/domains/pipeline/tasks/` submodules
 - M3: `blender_render.py` decomposed into `render-worker/scripts/_blender_*.py` submodules
 
 **File targets:**
@@ -55,13 +77,13 @@ This priority combines dead-code deletion and task decomposition because both ar
 | DELETE | `threejs-renderer/` directory |
 | DELETE | `flamenco/` directory |
 | DELETE | `renderproblems_tmp/` |
-| DELETE lines 705–1050 | `backend/app/tasks/step_tasks.py` (`render_order_line_task` — duplicates `rendering/tasks`) |
+| DONE | `backend/app/tasks/step_tasks.py` — reduced to compatibility shim only |
 | REMOVE settings | `admin.py`: `VALID_STL_QUALITIES`, `stl_quality`, `generate-missing-stls` endpoint |
 | REMOVE endpoint | `cad.py`: `POST /cad/{id}/generate-stl/{quality}` |
-| CREATE | `backend/app/tasks/pipeline/extract.py` — metadata extraction (OCC parsing, < 2s) |
+| DONE | `backend/app/domains/pipeline/tasks/extract_metadata.py` — metadata extraction |
-| CREATE | `backend/app/tasks/pipeline/thumbnail.py` — Blender thumbnail render |
+| DONE | `backend/app/domains/pipeline/tasks/render_thumbnail.py` — thumbnail render task |
-| CREATE | `backend/app/tasks/pipeline/stills.py` — still render task |
+| DONE (combined) | `backend/app/domains/pipeline/tasks/render_order_line.py` — still/dispatch pipeline entry |
-| CREATE | `backend/app/tasks/pipeline/turntable.py` — turntable render task |
+| OPEN | turntable-specific pipeline task split still needs to be carved out explicitly if kept as a separate concern |
 | THIN (< 80 lines) | `backend/app/tasks/step_tasks.py` — dispatch only |
 | CREATE | `render-worker/scripts/_blender_gpu.py` |
 | CREATE | `render-worker/scripts/_blender_import.py` |
@@ -81,6 +103,8 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 **Goal:** Introduce canonical part identity and the three-layer material assignment model while keeping the current GLB-based browser UX working end-to-end.
 
+**Status:** Not started in code. Architecture decisions are documented, but repo work has not begun.
+
 **Milestones:**
 - M1: `export_step_to_usd.py` produces valid USD with part hierarchy and `schaeffler:partKey` on every prim
 - M2: `usd_master` MediaAsset type exists in DB and is stored after each export
@@ -106,8 +130,7 @@ This priority combines dead-code deletion and task decomposition because both ar
 | CREATE | `frontend/src/api/sceneManifest.ts` — `SceneManifest` interface, `fetchSceneManifest()` |
 
 **Open questions to decide before M1:**
-- USD authoring library: `pxr` (OpenUSD full Python SDK) vs. plain USDA text templating vs. `usd-core` pip package
+- None blocking at the architecture level. The roadmap decisions for `usd-core` and index-space seam/sharp primvars are already captured in `docs/plans/0001-step-to-usd-implementation.md`.
-- seam/sharp payload encoding: custom primvars (`primvars:schaeffler:seamEdgeVertexPairs`) or a separate JSON sidecar
 
 **Acceptance gates:**
 - `python3 export_step_to_usd.py --step_path 81113-l_cut.stp` → valid `.usd` file, 25 part prims, each has `schaeffler:partKey` attribute
@@ -124,6 +147,8 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 **Goal:** Eliminate fan triangles on cylindrical surfaces (rings, bearings) and produce clean seams for UV unwrap.
 
+**Status:** Not started in code. This is still a pure planning workstream at the moment.
+
 **Milestones:**
 - M1: GMSH 4.15+ installed in render-worker container
 - M2: `export_step_to_gltf.py --tessellation_engine gmsh` produces fan-free GLB
@@ -232,6 +257,8 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 **Goal:** Fix broken render job cancellation (synthetic `render-{line_id}` ID never matches real Celery task ID) and establish structured per-step logging.
 
+**Status:** Done, aside from any follow-up polish.
+
 **Milestones:**
 - M1: `RenderJobDocument` schema + migration; tasks write real `self.request.id` to DB
 - M2: Cancel endpoint reads `celery_task_id` from job doc and calls `revoke()` — actually stops task
@@ -243,7 +270,7 @@ This priority combines dead-code deletion and task decomposition because both ar
 |---|---|
 | CREATE | `backend/app/domains/rendering/job_document.py` — `RenderJobDocument` Pydantic model, `update_step()`, `set_state()` |
 | CREATE | `backend/app/core/pipeline_logger.py` — `PipelineLogger(step_start/done/error)` writing to logging + Redis SSE |
-| CREATE migration | `backend/alembic/versions/062_render_job_document.py` — add `render_job_doc JSONB` to `order_lines` |
+| DONE | `backend/alembic/versions/048_render_job_document.py` — adds `render_job_doc JSONB` to `order_lines` |
 | MODIFY | `backend/app/domains/pipeline/tasks/render_order_line.py` — write `celery_task_id` + step events to job doc |
 | MODIFY | `backend/app/domains/pipeline/tasks/render_thumbnail.py` — same |
 | MODIFY | `backend/app/api/routers/orders.py` — cancel reads `render_job_doc.celery_task_id`, calls `celery.control.revoke()` |
@@ -255,7 +282,9 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 ### Priority 8 — Tenant Isolation Completion
 
-**Goal:** Make PostgreSQL RLS enforcement real. Currently `build_tenant_db_dep()` yields `db` without calling `SET LOCAL app.current_tenant_id`, making all tenant isolation a silent no-op.
+**Goal:** Finish tenant isolation hardening, especially for non-HTTP execution paths.
+
+**Status:** In progress. HTTP-side RLS enforcement is now real; task-side propagation is the remaining gap.
 
 **Milestones:**
 - M1: `TenantContextMiddleware` registered; all HTTP requests set RLS context from JWT
@@ -270,8 +299,8 @@ This priority combines dead-code deletion and task decomposition because both ar
 | MODIFY | `backend/app/main.py` — `app.add_middleware(TenantContextMiddleware)` |
 | MODIFY | `backend/app/utils/auth.py` — `create_access_token()` embeds `tenant_id` in JWT claims |
 | MODIFY | `backend/app/tasks/pipeline/thumbnail.py`, `extract.py`, `stills.py`, `turntable.py` — `set_tenant_context()` at start |
-| CREATE migration | `backend/alembic/versions/063_role_hierarchy.py` — rename `admin` → `global_admin`, add `tenant_admin` |
+| DONE | `backend/alembic/versions/049_role_hierarchy.py` — adds `global_admin` and `tenant_admin` |
-| MODIFY | All routers using `require_admin()` → `require_global_admin()` |
+| PARTIAL | Routers are mixed between new `require_global_admin()` usage and backward-compatible `require_admin()` aliases |
 
 **Acceptance gates:**
 - Login as tenant A user → `GET /api/products` returns 0 results when tenant A has no products, even if tenant B has 50
@@ -281,10 +310,12 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 ### Priority 9 — Hash-Based Scene Conversion Caching
 
-**Goal:** Skip re-tessellation when the STEP file has not changed. Cache canonical scene + preview derivatives by `SHA256(step_file)`.
+**Goal:** Extend the existing STEP-hash plumbing so canonical scene + preview derivatives can skip unnecessary re-tessellation.
+
+**Status:** Partial foundation already exists.
 
 **Milestones:**
-- M1: `cad_files.step_hash` column in DB; hash computed and stored on each export
+- M1: Existing `cad_files.step_file_hash` is reused or renamed for canonical-scene caching
 - M2: Export task checks hash before processing — returns cached asset UUID on hit
 - M3: Hash invalidated correctly when admin forces reprocess or deflection settings change
 
@@ -292,10 +323,10 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 | Action | Path |
 |---|---|
-| ADD column | `backend/app/domains/products/models.py` — `CadFile.step_hash: str \| None` |
+| DONE | `backend/app/domains/products/models.py` — `CadFile.step_file_hash: str \| None` already exists |
-| CREATE migration | `backend/alembic/versions/064_step_hash.py` — `ADD COLUMN step_hash VARCHAR(64)` |
+| DONE | `backend/app/domains/products/cache_service.py` — `compute_step_hash(file_path)` already exists |
-| MODIFY | `backend/app/domains/pipeline/tasks/export_glb.py` (or future USD task) — hash check before subprocess call |
+| OPEN | `backend/app/domains/pipeline/tasks/export_glb.py` (or future USD task) — hash check before subprocess call |
-| ADD util | `backend/app/services/step_processor.py` — `compute_step_hash(file_path) -> str` |
+| OPEN | optional migration to rename `step_file_hash` → `step_hash` only if naming consistency is worth the churn |
 
 **Acceptance gates:**
 - Upload same STEP file twice → second task completes in < 2s (cache hit logged: `[CACHE] hash match, skipping tessellation`)
@@ -306,11 +337,13 @@ This priority combines dead-code deletion and task decomposition because both ar
 
 **Goal:** Address independent UI items from `visual-audit-report.md` that require no backend changes.
 
+**Status:** Partial. Some milestones are already shipped and should be removed from the active queue.
+
 **Milestones:**
-- M1: Tooltip/help text on every Admin settings input
+- M1: Tooltip/help text on every Admin settings input — mostly done
-- M2: Empty state messages in MediaBrowser, ProductLibrary, Orders
+- M2: Empty state messages in MediaBrowser, ProductLibrary, Orders — partially done
 - M3: Notification batching — group per-render noise into job summaries
-- M4: Mobile navigation — hamburger menu at < 768px
+- M4: Mobile navigation — hamburger menu at < 768px — done
 - M5: Kanban rejection flow — drag-to-reject with reason field
 
 **File targets:**
@@ -345,9 +378,8 @@ Priority 1 (Pipeline Cleanup Foundation)
 Priority 3 (Tessellation and Topology Quality)
   └── Priority 5 (Canonical USD Export and Render Migration)
 
-Priority 7 (Render Job Tracking and Structured Logging) — can run in parallel
+Priority 8 remaining work (Celery tenant context)      — can run in parallel
-Priority 8 (Tenant Isolation Completion)               — can run in parallel
+Priority 10 remaining polish                           — independent
-Priority 10 (UI/UX Polish)                             — independent
 ```
 
 ---
@@ -355,19 +387,20 @@ Priority 10 (UI/UX Polish)                             — independent
 ## What To Do Next
 
 **Recommended execution path:**
-1. Do Priority 1 first: clean up and split the current pipeline.
+1. Finish the remaining Priority 1 work first: remove STL-era dead code and split `blender_render.py`.
-2. Start Priority 2 immediately after: add `partKey`, assignment-layer semantics, and scene manifest without changing the browser UX.
+2. Start Priority 2 immediately after that cleanup baseline is stable: add `partKey`, assignment layers, and scene manifest without changing browser UX.
-3. Run Priority 3 in parallel or immediately after, depending on whether current tessellation quality blocks scene-authoring confidence.
+3. Run Priority 3 in parallel only if cylinder tessellation is actively blocking confidence in seam/sharp payload work; otherwise keep it behind Priority 2.
-4. Use the implementation plan in `docs/plans/0001-step-to-usd-implementation.md` as the execution checklist for the USD workstream.
+4. Treat Priority 8 as a short parallel hardening task: add Celery-side tenant context propagation.
+5. Use `docs/plans/0001-step-to-usd-implementation.md` as the execution checklist for the USD workstream.
 
 **Parallel sprint option (2 agents):**
-- Agent 1: Priority 1 (pipeline cleanup foundation)
+- Agent 1: Priority 1 remainder (dead-code cleanup + `blender_render.py` split)
-- Agent 2: Priority 3 (tessellation and topology quality)
+- Agent 2: Priority 8 remainder or Priority 3, depending on whether tessellation quality is currently blocking work
 
 **Parallel sprint option (3 agents):**
-- Agent 1: Priority 1
+- Agent 1: Priority 1 remainder
-- Agent 2: Priority 3
+- Agent 2: Priority 2 groundwork (`usd_master`, `part_key_service`, `scene-manifest`)
-- Agent 3: Priority 7 or Priority 8
+- Agent 3: Priority 8 remainder or targeted Priority 10 polish
 
 **Do not defer anymore:**
 - canonical `partKey`
@@ -383,8 +416,8 @@ These are now considered implementation prerequisites for the long-term refactor
 Old planning files are kept for reference but superseded by this document:
 - `PLAN.md` — original Phase A–F plan (Phases A–E complete, Phase F = Priority 9 here)
 - `PLAN_REFACTOR.md` — 1,173-line architectural plan (Phases 1–8 mapped to Priorities 2–8 above)
-- `plan.md` — active GMSH implementation plan (Priority 1)
+- `plan.md` — GMSH tessellation implementation plan (Priority 3)
-- `docs/rfcs/0001-step-to-usd-workflow.md` — USD RFC (Priority 10)
+- `docs/rfcs/0001-step-to-usd-workflow.md` — USD RFC (Priorities 2, 4, and 5)
 - `docs/plans/0001-step-to-usd-implementation.md` — actionable USD implementation plan
 - `review-report.md` — latest code review results
 - `visual-audit-report.md` — UX audit results

docs/plans/0001-step-to-usd-implementation.md

@@ -8,12 +8,21 @@
 
 ## Prerequisites
 
-- [ ] Priority 1 complete (step_tasks.py decomposed, blender_render.py decomposed)
+- [x] `step_tasks.py` decomposition is already done; `backend/app/tasks/step_tasks.py` is now a compatibility shim over `backend/app/domains/pipeline/tasks/*`
+- [ ] `blender_render.py` decomposition is still pending; current file remains monolithic
+- [ ] Legacy STL-era cleanup is still pending (`stl_quality`, STL endpoints, orphaned directories)
 - [x] Decision: USD authoring library → **`usd-core` (pip)** — provides `pxr` module, no GPU tools needed, pip-installable in render-worker
 - [x] Decision: seam/sharp payload encoding → **index-space primvars** (`primvars:schaeffler:seamEdgeVertexPairs`, `primvars:schaeffler:sharpEdgeVertexPairs`) — survives transforms, no KD-tree needed
 - [x] Decision: preview GLB derivation → **co-author from same tessellation pass** during migration (avoid round-trip loss from USD→GLB export)
 - [x] Decision: single-file vs override layers → **Option B: canonical geometry layer + material override layer, flattened via `UsdUtils.FlattenLayerStack()` for delivery** — preserves hierarchy AND allows instancing later (`FlattenLayerStack` keeps `instanceable` prims; `UsdStage.Flatten` would expand them). Note: Phase 1 uses no instancing (matching current GLB pipeline), but the delivery path is already instancing-safe.
 
+## Current Baseline
+
+- No USD exporter exists yet in `render-worker/scripts/`
+- No `usd_master` asset type or scene-manifest endpoint exists yet in backend/frontend code
+- No `partKey` payload exists yet in the GLB export/viewer contract
+- No `gmsh` tessellation wiring exists yet; Phase 1 should assume current OCC tessellation unless Priority 3 is pulled forward
+
 ---
 
 ## Phase 1 — Dual-Write USD Beside GLB (Priority 2, M1–M2)