docs: add milestones, file targets, acceptance gates to all 10 ROADMAP priorities
- Each priority now has: milestones (M1-MN), concrete file target table (CREATE/MODIFY/DELETE per file), and binary acceptance gates - Created docs/plans/0001-step-to-usd-implementation.md: full execution checklist for USD pipeline (Priorities 2, 4, 5) with: - Phase 1: dual-write USD beside GLB - Phase 2: partKey + three-layer material assignment model - Phase 3: seam/sharp payload to USD mesh primvars (index-space) - Phase 4: Blender render from USD - Phase 5: frontend ThreeDViewer partKey migration - Open questions decision table - Non-regression checklist Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,326 @@
|
||||
# Implementation Plan: STEP → USD Canonical Scene Workflow
|
||||
|
||||
> Execution checklist for ROADMAP.md Priorities 2, 4, and 5.
|
||||
> RFC: `docs/rfcs/0001-step-to-usd-workflow.md`
|
||||
> Date: 2026-03-11
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [ ] Priority 1 complete (step_tasks.py decomposed, blender_render.py decomposed)
|
||||
- [ ] Decision: USD authoring library (see Open Questions below)
|
||||
- [ ] Decision: seam/sharp payload encoding (primvars vs. JSON sidecar)
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Dual-Write USD Beside GLB (Priority 2, M1–M2)
|
||||
|
||||
**Goal:** Export a valid `usd_master` alongside the existing GLB pipeline without changing any browser behavior.
|
||||
|
||||
### Task 1.1 — `export_step_to_usd.py` scaffolding
|
||||
|
||||
**File:** `render-worker/scripts/export_step_to_usd.py`
|
||||
|
||||
**What it must produce:**
|
||||
|
||||
```
|
||||
/Root — Stage root
|
||||
/Root/Assembly — Top-level assembly prim (Xform)
|
||||
/Root/Assembly/<AssemblyNode> — Per-component prim (Xform)
|
||||
/Root/Assembly/<AssemblyNode>/<PartKey> — Leaf part prim (Xform)
|
||||
/Root/Assembly/<AssemblyNode>/<PartKey>/Mesh — UsdGeomMesh
|
||||
/Root/Looks/<MaterialName> — UsdShadeMaterial (placeholder binding)
|
||||
```
|
||||
|
||||
**Required per-mesh prim attributes:**
|
||||
|
||||
| Attribute | Value source |
|
||||
|---|---|
|
||||
| `schaeffler:partKey` | `generate_part_key(xcaf_label_path)` |
|
||||
| `schaeffler:sourceName` | XCAF `TDataStd_Name` attribute |
|
||||
| `schaeffler:sourceColor` | XCAF embedded color (hex string) |
|
||||
| `schaeffler:rawMaterialName` | from `CadFile.part_materials` if available |
|
||||
| `schaeffler:tessellation:linearDeflectionMm` | CLI arg value |
|
||||
| `schaeffler:tessellation:angularDeflectionRad` | CLI arg value |
|
||||
| `primvars:schaeffler:seamEdgeVertexPairs` | OCC B-rep seam edges (index pairs in mesh-local space) |
|
||||
| `primvars:schaeffler:sharpEdgeVertexPairs` | sharp edges from `_extract_sharp_edge_pairs()` |
|
||||
|
||||
**CLI interface:**
|
||||
|
||||
```bash
|
||||
python3 export_step_to_usd.py \
|
||||
--step_path /path/to/file.stp \
|
||||
--output_path /path/to/output.usd \
|
||||
[--linear_deflection 0.03] \
|
||||
[--angular_deflection 0.05] \
|
||||
[--color_map '{"Ring": "#4C9BE8"}'] \
|
||||
[--tessellation_engine occ|gmsh]
|
||||
```
|
||||
|
||||
**Acceptance gate:** `python3 export_step_to_usd.py --step_path 81113-l_cut.stp --output_path /tmp/test.usd` →
|
||||
- File exists, parseable
|
||||
- 25 part prims with `schaeffler:partKey` attribute
|
||||
- Part count matches `export_step_to_gltf.py` output for same file
|
||||
|
||||
### Task 1.2 — `usd_master` MediaAsset type
|
||||
|
||||
**File:** `backend/app/domains/media/models.py`
|
||||
|
||||
Add `usd_master = "usd_master"` to `MediaAssetType` enum.
|
||||
|
||||
**Migration:** `backend/alembic/versions/060_usd_master_asset_type.py`
|
||||
|
||||
```sql
|
||||
ALTER TYPE mediaassettype ADD VALUE IF NOT EXISTS 'usd_master';
|
||||
```
|
||||
|
||||
**Acceptance gate:** `GET /api/media?asset_type=usd_master` returns 200 (not 422 validation error).
|
||||
|
||||
### Task 1.3 — `generate_usd_master_task` Celery task
|
||||
|
||||
**File:** `backend/app/domains/pipeline/tasks/export_glb.py`
|
||||
|
||||
New task that:
|
||||
1. Resolves STEP file path from `CadFile`
|
||||
2. Calls `export_step_to_usd.py` subprocess
|
||||
3. Stores resulting `.usd` as a `usd_master` `MediaAsset`
|
||||
4. Does NOT touch existing GLB tasks (dual-write, no removal yet)
|
||||
|
||||
**Queue:** `step_processing` (fast, < 30s — tessellation only, no Blender)
|
||||
|
||||
**Called by:** `render_step_thumbnail` task after `render_step_thumbnail` succeeds, OR triggered independently via admin action.
|
||||
|
||||
**Acceptance gate:** After triggering task for a CadFile, `GET /api/cad/{id}/media` includes an asset with `asset_type: "usd_master"`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Canonical Part Identity and Assignment Layers (Priority 2, M3–M5)
|
||||
|
||||
### Task 2.1 — `part_key_service.py`
|
||||
|
||||
**File:** `backend/app/services/part_key_service.py`
|
||||
|
||||
```python
|
||||
def generate_part_key(xcaf_label_path: str, source_name: str) -> str:
|
||||
"""Deterministic slug from XCAF path + source name.
|
||||
|
||||
Format: lowercase alphanumeric + underscores, max 64 chars.
|
||||
E.g.: 'ring_outer', 'ball_af0', 'cage_inner_ring'
|
||||
|
||||
For duplicate names: append _2, _3, etc.
|
||||
For unnamed parts: 'part_{sha256(xcaf_path)[:8]}'
|
||||
"""
|
||||
...
|
||||
|
||||
def build_scene_manifest(cad_file: CadFile, usd_asset: MediaAsset) -> dict:
|
||||
"""Read part metadata from USD or from CadFile.parsed_objects.
|
||||
|
||||
Returns dict matching SceneManifest Pydantic schema:
|
||||
{
|
||||
"cad_file_id": "...",
|
||||
"parts": [
|
||||
{
|
||||
"part_key": "ring_outer",
|
||||
"source_name": "RingOuter_AF0",
|
||||
"prim_path": "/Root/Assembly/Bearing/RingOuter",
|
||||
"effective_material": "SCHAEFFLER_010102_...",
|
||||
"assignment_provenance": "manual|auto|default",
|
||||
"is_unassigned": false
|
||||
}
|
||||
],
|
||||
"unmatched_source_rows": [...],
|
||||
"unassigned_parts": [...]
|
||||
}
|
||||
"""
|
||||
...
|
||||
```
|
||||
|
||||
### Task 2.2 — Three-layer material assignment model
|
||||
|
||||
**File:** `backend/app/domains/products/models.py`
|
||||
|
||||
Add three JSONB columns to `CadFile`:
|
||||
|
||||
```python
|
||||
source_material_assignments: dict | None # from Excel / product import; keyed by source name
|
||||
resolved_material_assignments: dict | None # auto-matched; keyed by partKey
|
||||
manual_material_overrides: dict | None # browser-authored; keyed by partKey
|
||||
```
|
||||
|
||||
**Migration:** `backend/alembic/versions/061_material_assignment_layers.py`
|
||||
|
||||
Keep existing `part_materials` column for backward compat during transition.
|
||||
|
||||
**Service helper:**
|
||||
|
||||
```python
|
||||
def get_effective_assignments(cad_file: CadFile) -> dict:
|
||||
"""Priority: manual_overrides > resolved > source color > 'unassigned'."""
|
||||
...
|
||||
```
|
||||
|
||||
### Task 2.3 — `GET /cad/{id}/scene-manifest` endpoint
|
||||
|
||||
**File:** `backend/app/api/routers/cad.py`
|
||||
|
||||
New endpoint returning `SceneManifest`. Calls `build_scene_manifest()` — reads from USD metadata if `usd_master` asset exists, otherwise reads from `CadFile.parsed_objects`.
|
||||
|
||||
**Acceptance gate:** Returns HTTP 200 with parts array; each part has `part_key`, `effective_material`, `is_unassigned`, `assignment_provenance`.
|
||||
|
||||
### Task 2.4 — Update `PUT /cad/{id}/part-materials`
|
||||
|
||||
**File:** `backend/app/api/routers/cad.py`
|
||||
|
||||
Accept `{ "part_key": "ring_outer", "material": "SCHAEFFLER_010102_..." }` body (or bulk map). Write to `manual_material_overrides` column (not the old `part_materials` column).
|
||||
|
||||
**Acceptance gate:** PUT with `partKey` → subsequent GET `/scene-manifest` shows that part's `assignment_provenance: "manual"`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Seam/Sharp Payload to USD Mesh Prims (Priority 3 integration)
|
||||
|
||||
### Task 3.1 — Port seam derivation into USD exporter
|
||||
|
||||
**File:** `render-worker/scripts/export_step_to_usd.py`
|
||||
|
||||
After tessellation (OCC or GMSH), for each mesh face:
|
||||
1. Identify seam edges from face/batch boundaries (STEPper approach: adjacent faces with different batch IDs)
|
||||
2. Identify sharp edges from `_extract_sharp_edge_pairs()` (already implemented)
|
||||
3. Convert both to **mesh-local vertex index pairs** (not world-space coordinates)
|
||||
|
||||
Write to USD mesh prim:
|
||||
|
||||
```python
|
||||
mesh_prim.GetPrimvar("schaeffler:seamEdgeVertexPairs").Set(
|
||||
Vt.Vec2iArray(seam_pairs), # [(vi0, vi1), ...]
|
||||
)
|
||||
mesh_prim.GetPrimvar("schaeffler:sharpEdgeVertexPairs").Set(
|
||||
Vt.Vec2iArray(sharp_pairs),
|
||||
)
|
||||
```
|
||||
|
||||
**Why index-space:** Survives transforms cleanly; avoids KD-tree matching workaround in `_apply_sharp_edges_from_occ()`.
|
||||
|
||||
### Task 3.2 — `import_usd.py` Blender helper
|
||||
|
||||
**File:** `render-worker/scripts/import_usd.py`
|
||||
|
||||
```python
|
||||
def import_usd_and_restore_topology(usd_path: str) -> list:
|
||||
"""Import USD stage into Blender, restore seam/sharp from primvars.
|
||||
|
||||
Returns list of imported mesh objects.
|
||||
"""
|
||||
bpy.ops.wm.usd_import(filepath=usd_path)
|
||||
for obj in bpy.context.scene.objects:
|
||||
if obj.type != 'MESH':
|
||||
continue
|
||||
# Read custom attributes set by USD importer
|
||||
seam_pairs = obj.get("schaeffler_seamEdgeVertexPairs") or []
|
||||
sharp_pairs = obj.get("schaeffler_sharpEdgeVertexPairs") or []
|
||||
_mark_seams_from_index_pairs(obj, seam_pairs)
|
||||
_mark_sharp_from_index_pairs(obj, sharp_pairs)
|
||||
...
|
||||
```
|
||||
|
||||
**Acceptance gate:** Blender log shows `[USD_IMPORT] 25 parts, 5044 seam edges restored from primvars` — no KD-tree needed.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Blender Render from USD (Priority 5)
|
||||
|
||||
### Task 4.1 — `blender_render.py` USD path
|
||||
|
||||
**File:** `render-worker/scripts/blender_render.py`
|
||||
|
||||
Add `--usd_path` argument. When provided:
|
||||
1. Call `import_usd.py` instead of `export_gltf.py` GLB import
|
||||
2. Read `schaeffler:partKey` and `schaeffler:canonicalMaterialName` per mesh object after import
|
||||
3. Apply materials by `partKey → material library name` lookup instead of object-name heuristics
|
||||
|
||||
**Migration:** Keep `--glb_path` working in parallel; switch production task to prefer `--usd_path` when `usd_master` asset exists.
|
||||
|
||||
### Task 4.2 — Backend render service update
|
||||
|
||||
**File:** `backend/app/services/render_blender.py`
|
||||
|
||||
When building `blender_cmd`, check if `usd_master` MediaAsset exists for the CadFile:
|
||||
- If yes: pass `--usd_path <usd_master_local_path>`
|
||||
- If no: fall back to `--glb_path <gltf_production_path>` (unchanged)
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — Frontend Migration (Priority 4)
|
||||
|
||||
### Task 5.1 — Scene manifest fetch on product load
|
||||
|
||||
**File:** `frontend/src/api/sceneManifest.ts`
|
||||
|
||||
```typescript
|
||||
export interface PartEntry {
|
||||
part_key: string;
|
||||
source_name: string;
|
||||
prim_path: string;
|
||||
effective_material: string | null;
|
||||
assignment_provenance: 'manual' | 'auto' | 'default';
|
||||
is_unassigned: boolean;
|
||||
}
|
||||
|
||||
export interface SceneManifest {
|
||||
cad_file_id: string;
|
||||
parts: PartEntry[];
|
||||
unmatched_source_rows: string[];
|
||||
unassigned_parts: string[];
|
||||
}
|
||||
|
||||
export async function fetchSceneManifest(cadFileId: string): Promise<SceneManifest> {
|
||||
const res = await api.get<SceneManifest>(`/cad/${cadFileId}/scene-manifest`);
|
||||
return res.data;
|
||||
}
|
||||
```
|
||||
|
||||
### Task 5.2 — ThreeDViewer partKey integration
|
||||
|
||||
**File:** `frontend/src/components/cad/ThreeDViewer.tsx`
|
||||
|
||||
- On GLB load: iterate `scene.children`, read `mesh.userData.partKey` (set by preview GLB derivation)
|
||||
- On click: identify `partKey` from `userData`, not from `mesh.name`
|
||||
- Pass `partKey` to `MaterialPanel` for display and persistence
|
||||
|
||||
### Task 5.3 — MaterialPanel reconciliation section
|
||||
|
||||
**File:** `frontend/src/components/cad/MaterialPanel.tsx`
|
||||
|
||||
Add reconciliation section showing:
|
||||
- "N unmatched Excel rows" (source names with no partKey match)
|
||||
- "M unassigned parts" (partKeys with no resolved material)
|
||||
|
||||
Clicking an unassigned part in the viewer auto-focuses it in the MaterialPanel.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions (must be decided before Phase 1 coding starts)
|
||||
|
||||
| # | Question | Options | Default recommendation |
|
||||
|---|---|---|---|
|
||||
| 1 | USD authoring library | `pxr` (full OpenUSD, heavy) / `usda` text templating (no deps) / `usd-core` pip | Start with `pxr` — pip-installable, same OCC kernel available |
|
||||
| 2 | Seam/sharp payload encoding | Custom primvars on mesh prim / separate JSON sidecar / GLB extras (current) | Index-space primvars — cleaner, survives transforms |
|
||||
| 3 | Preview GLB derivation | USD → GLB export pass / co-author from same tessellation pass | Co-author during migration (avoid round-trip loss) |
|
||||
| 4 | Single-file USD or override layers | Flat single file / canonical + overlay layers (flattened for delivery) | RFC recommends Option B (overlay layers, flatten for delivery) |
|
||||
| 5 | `pxr` install in render-worker | Add to `render-worker/Dockerfile` / use system package | `pip install usd-core` — no NVIDIA/Pixar GPU tools needed |
|
||||
|
||||
---
|
||||
|
||||
## Non-Regression Checklist
|
||||
|
||||
Before merging any Priority 2–5 work:
|
||||
|
||||
- [ ] Click a part in ThreeDViewer → selection resolves to stable `partKey`
|
||||
- [ ] Pin selection → isolate, hide, ghost all work as before
|
||||
- [ ] Unassigned parts are visually highlighted
|
||||
- [ ] Assign a Blender asset-library material name via browser → persisted by `partKey`
|
||||
- [ ] Reload page → same part still assigned
|
||||
- [ ] Subsequent Blender render uses the same assignment
|
||||
- [ ] CAD file with mismatched Excel names → system produces canonical scene, preview asset, unmatched row count
|
||||
- [ ] `geometryGltfUrl` / `productionGltfUrl` distinction no longer required by frontend (removed from API contract)
|
||||
Reference in New Issue
Block a user