Hartmut/HartOMat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: performance optimizations + part-materials validation

Browse Source 
- @timed_step decorator with wall-clock + RSS tracking (pipeline_logger)
- Blender timing laps for sharp edges and material assignment
- MeshRegistry pattern: eliminate 13 scene.traverse() calls across viewers
- Lazy material cloning (clone-on-first-write in both viewers)
- _pipeline_session context manager: 7 create_engine() → 2 in render_thumbnail
- KD-tree spatial pre-filter for sharp edge marking (bbox-based pruning)
- Batch material library append: N bpy.ops.wm.append → single bpy.data.libraries.load
- GMSH single-session batching: compound all solids into one tessellation call
- Validate part-materials save endpoints against parsed_objects (prevents bogus keys)
- ROADMAP updated with completion status

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Hartmut Nörenberg
2026-03-13 11:53:14 +01:00
parent ec667dd56a
commit 6c5873d51f
11 changed files with 612 additions and 541 deletions
Download Patch File Download Diff File Expand all files Collapse all files
plan.md
+109 -253
View File
@@ -1,295 +1,151 @@
# Plan: Extract PBR Material Properties from Blender Asset Library for 3D Viewer
> **Date:** 2026-03-13 | **Branch:** refactor/v2
# Plan: Draw Call Batching + Merge Dual STEP Parse
## Context
The 3D viewer currently shows all materials as flat colors from a hardcoded `SCHAEFFLER_COLORS` map in `MaterialPanel.tsx` (17 entries). These hex colors don't match the actual Blender materials — a "Steel-Bare" material that looks metallic and reflective in Blender renders appears as flat gray `#8a9ca8` in the viewer. The user wants visual parity: if a material is blue plastic in Blender, it should look like blue plastic in the 3D viewer too.
Two independent optimization tracks:
**Source of truth**: The Blender `.blend` asset library already contains all PBR properties (Base Color, Metallic, Roughness, Transmission, IOR) in Principled BSDF nodes for all 35 Schaeffler materials. These values are defined in `MaterialNamingSchema/generate_blend.py`.
**Track A — Draw Call Batching (Frontend):** Assemblies with 100+ parts create 100+ draw calls. Three.js issues one draw call per mesh. For large assemblies this saturates the GPU command buffer and drops frame rate below 30fps. Solution: merge meshes that share the same material into single geometries, togglable via a "Performance mode" button.
**Current flow**: `catalog_assets.py` extracts only material **names** → stored in `AssetLibrary.catalog` JSONB as `{"materials": ["name1", ...]}` → viewer uses hardcoded `SCHAEFFLER_COLORS` hex map.
**Track B — Merge Dual STEP Parse (Backend):** `extract_cad_metadata()` reads the same STEP file twice:
1. `_extract_step_objects()` — `OCC.Core.STEPCAFControl_Reader` → part names (lines 391–425)
2. `extract_mesh_edge_data()` — `OCP.STEPControl.STEPControl_Reader` → tessellates, extracts edge topology + bbox (lines 200–388)
**Target flow**: `catalog_assets.py` extracts PBR properties per material → stored in catalog JSONB → new API endpoint serves PBR map to frontend → viewers apply `MeshStandardMaterial` with correct color + roughness + metalness.
Both readers produce a `TopoDS_Shape`. The XCAF reader (`STEPCAFControl`) gives us both the labeled hierarchy AND the shape, so we can extract edge data from the same read. This eliminates ~0.5–2s of redundant STEP parsing per file.
**Important constraint for Track B:** `_extract_step_objects` runs on the `worker` container (has `OCC.Core` / pythonocc), while `extract_mesh_edge_data` has dual-import fallback (`OCP` first, then `OCC.Core`). The unified function must work with `OCC.Core` (pythonocc) since that's what the `worker` container has.
## Affected Files
| File | Change |
|------|--------|
| `render-worker/scripts/catalog_assets.py` | Extract PBR properties from Principled BSDF nodes |
| `backend/app/api/routers/asset_libraries.py` | Add public `GET /api/asset-libraries/pbr-map` endpoint |
| `frontend/src/api/assetLibraries.ts` | Add `fetchMaterialPBR()` + `MaterialPBRMap` type |
| `frontend/src/components/cad/cadUtils.ts` | Add `applyPBRToMaterial()` + `pbrColorHex()` helpers |
| `frontend/src/components/cad/ThreeDViewer.tsx` | Fetch PBR map, apply PBR props when assigning materials |
| `frontend/src/components/cad/InlineCadViewer.tsx` | Same PBR application |
| `frontend/src/components/cad/MaterialPanel.tsx` | Replace hardcoded `SCHAEFFLER_COLORS` with dynamic PBR lookup |
| File | Track | Change |
|------|-------|--------|
| `frontend/src/components/cad/useGeometryMerge.ts` | A | NEW — hook for merge/unmerge logic |
| `frontend/src/components/cad/ThreeDViewer.tsx` | A | Add Performance mode toggle + integrate hook |
| `frontend/src/components/cad/InlineCadViewer.tsx` | A | Same Performance mode toggle |
| `frontend/src/components/cad/cadUtils.ts` | A | Add `MergedGroup` type |
| `backend/app/services/step_processor.py` | B | New `extract_step_metadata()`, refactor callers |
| `backend/app/domains/pipeline/tasks/extract_metadata.py` | B | Use new unified function |
## Tasks (in order)
### [x] Task 1: Extend catalog_assets.py to extract PBR properties
---
- **File**: `render-worker/scripts/catalog_assets.py`
- **What**: After opening the .blend file, for each material with `asset_data`, find the `ShaderNodeBsdfPrincipled` node and extract:
- `base_color`: `[R, G, B]` from `inputs["Base Color"].default_value` — convert linear→sRGB via `v^(1/2.2)`
- `metallic`: float from `inputs["Metallic"].default_value`
- `roughness`: float from `inputs["Roughness"].default_value`
- `transmission`: float from `inputs["Transmission Weight"].default_value` (0.0 if absent)
- `ior`: float from `inputs["IOR"].default_value` (1.45 default)
Change output format from:
```json
{"materials": ["Mat1", "Mat2"], "node_groups": [...]}
```
to:
```json
{
"materials": [
{"name": "Mat1", "base_color": [0.76, 0.77, 0.78], "metallic": 1.0, "roughness": 0.35, "transmission": 0.0, "ior": 1.45},
...
],
"node_groups": [...]
}
```
Fallback for materials without Principled BSDF: `base_color` from `mat.diffuse_color[:3]` (already sRGB), metallic=0.0, roughness=0.5.
**Color space note**: Blender's Principled BSDF stores Base Color in **linear** space. Three.js `MeshStandardMaterial.color.setRGB()` expects **sRGB** values (it converts internally to linear for rendering). Convert in the script: `srgb = pow(linear, 1/2.2)`, rounded to 4 decimal places.
- **Acceptance gate**: Rebuilt render-worker, run catalog refresh → JSON output has PBR properties
- **Dependencies**: none
- **Risk**: Complex node graphs (textures etc.) — handled by diffuse_color fallback
### [x] Task 2: Rebuild render-worker + refresh catalog
- **File**: No code change — operational step
- **What**:
```bash
docker compose up -d --build render-worker
# Then POST /api/asset-libraries/{id}/refresh-catalog via Admin UI or curl
```
The `AssetLibrary.catalog` JSONB column is schema-free — no migration needed.
- **Acceptance gate**: Active library's catalog has materials with `base_color`, `metallic`, `roughness`
- **Dependencies**: Task 1
- **Risk**: None
### [x] Task 3: Add public API endpoint for material PBR map
- **File**: `backend/app/api/routers/asset_libraries.py`
- **What**: Add endpoint **before** the `/{lib_id}` route (to avoid path collision):
```python
@router.get("/pbr-map")
async def get_material_pbr_map(db: AsyncSession = Depends(get_db)):
"""PBR properties for all materials in the active asset library.
Public (no auth) — needed by all 3D viewers.
"""
result = await db.execute(
select(AssetLibrary).where(AssetLibrary.is_active == True).limit(1)
)
lib = result.scalar_one_or_none()
if not lib or not lib.catalog:
return {}
materials = lib.catalog.get("materials", [])
pbr_map = {}
for m in materials:
if isinstance(m, str):
continue # old format — skip
pbr_map[m["name"]] = {
"base_color": m.get("base_color", [0.5, 0.5, 0.5]),
"metallic": m.get("metallic", 0.0),
"roughness": m.get("roughness", 0.5),
"transmission": m.get("transmission", 0.0),
"ior": m.get("ior", 1.45),
}
return JSONResponse(content=pbr_map, headers={"Cache-Control": "public, max-age=3600"})
```
- **Acceptance gate**: `curl localhost:8888/api/asset-libraries/pbr-map` returns keyed PBR map
- **Dependencies**: Task 2
- **Risk**: Must be placed before `/{lib_id}` route or FastAPI will try to parse "pbr-map" as a UUID
### [x] Task 4: Add frontend API function + types
- **File**: `frontend/src/api/assetLibraries.ts`
- **What**:
1. Add types:
```typescript
export interface MaterialPBR {
base_color: [number, number, number]
metallic: number
roughness: number
transmission?: number
ior?: number
}
export type MaterialPBRMap = Record<string, MaterialPBR>
```
2. Add fetch function:
```typescript
export async function fetchMaterialPBR(): Promise<MaterialPBRMap> {
const { data } = await api.get<MaterialPBRMap>('/asset-libraries/pbr-map')
return data
}
```
3. Update `AssetLibraryCatalog.materials` type from `string[]` to `Array<string | {name: string, base_color?: number[], metallic?: number, roughness?: number}>` for backwards compat with old catalogs
- **Acceptance gate**: `npx tsc --noEmit` passes
- **Dependencies**: Task 3
- **Risk**: None
### [x] Task 5: Add PBR helpers in cadUtils.ts
### Track A — Draw Call Batching
### [ ] Task A1: Add `MergedGroup` type and merge utility to cadUtils.ts
- **File**: `frontend/src/components/cad/cadUtils.ts`
- **What**: Add two helpers:
- **What**: Add type `MergedGroup = { mergedMesh: any; sourceEntries: MeshRegistryEntry[]; materialKey: string }`. Add helper `groupRegistryByMaterial(registry: MeshRegistryEntry[], partMaterials: PartMaterialMap, pbrMap: MaterialPBRMap): Map<string, MeshRegistryEntry[]>` that groups registry entries by their resolved material name (or `__unassigned__` for parts without material).
- **Acceptance gate**: TypeScript compiles (`tsc --noEmit`). Helper is pure — no side effects, no THREE import.
- **Dependencies**: none
- **Risk**: Low
```typescript
import type { MaterialPBR } from '../../api/assetLibraries'
### [ ] Task A2: Create `useGeometryMerge` hook
- **File**: NEW `frontend/src/components/cad/useGeometryMerge.ts`
- **What**: Hook that takes `meshRegistryRef`, `partMaterials`, `pbrMap`, and `enabled` flag. When enabled:
1. Groups meshes by material key (via `groupRegistryByMaterial`)
2. For each group: calls `BufferGeometryUtils.mergeGeometries()` on all mesh geometries (with world transforms applied via `mesh.matrixWorld`)
3. Creates one new `THREE.Mesh` per group with the shared material
4. Hides original meshes (`visible = false`)
5. Adds merged meshes to the scene
6. Returns `{ mergedGroups: MergedGroup[], restore: () => void }` — `restore()` removes merged meshes, re-shows originals
/** Apply PBR material properties to a Three.js MeshStandardMaterial. */
export function applyPBRToMaterial(
mat: THREE.MeshStandardMaterial,
pbr: MaterialPBR,
): void {
mat.color.setRGB(pbr.base_color[0], pbr.base_color[1], pbr.base_color[2])
mat.metalness = pbr.metallic
mat.roughness = pbr.roughness
if (pbr.transmission && pbr.transmission > 0.1) {
mat.transparent = true
mat.opacity = 1 - pbr.transmission * 0.7
}
}
When disabled (or on cleanup): calls `restore()`.
/** Convert PBR base_color to hex string for UI swatches. */
export function pbrColorHex(pbr: MaterialPBR): string {
const [r, g, b] = pbr.base_color
return '#' + [r, g, b].map(v => Math.round(v * 255).toString(16).padStart(2, '0')).join('')
}
```
Note: `THREE` is a type-only import here — the actual THREE namespace is available at runtime in the viewer components. The helper takes the material as a parameter, so no direct THREE import needed in cadUtils.
- **Acceptance gate**: `npx tsc --noEmit` passes
- **Dependencies**: Task 4
- **Risk**: None
### [x] Task 6: Update ThreeDViewer to apply PBR materials
Important: must handle `BufferGeometryUtils` import from `three/examples/jsm/utils/BufferGeometryUtils.js`.
- **Acceptance gate**: TypeScript compiles. Hook can be called with `enabled=false` without errors.
- **Dependencies**: Task A1
- **Risk**: Medium — `mergeGeometries` requires all geometries to have same attribute layout (position, normal, uv). Some meshes may lack UVs. Must filter or skip incompatible groups.
### [ ] Task A3: Integrate Performance mode in ThreeDViewer
- **File**: `frontend/src/components/cad/ThreeDViewer.tsx`
- **What**:
1. Import `fetchMaterialPBR` and `applyPBRToMaterial` from the new modules
2. Add query:
```typescript
const { data: pbrMap = {} } = useQuery({
queryKey: ['material-pbr'],
queryFn: fetchMaterialPBR,
staleTime: 300_000,
})
```
3. Update the material-application `useEffect` (line ~567). Current code:
```typescript
if (mat && 'color' in mat) mat.color.set(previewColorForEntry(entry))
```
Replace with:
```typescript
if (mat && 'color' in mat) {
if (entry.type === 'library' && pbrMap[entry.value]) {
applyPBRToMaterial(mat as THREE.MeshStandardMaterial, pbrMap[entry.value])
} else {
mat.color.set(previewColorForEntry(entry, pbrMap))
}
}
```
4. **Important**: Clone materials before modifying. GLB loader shares material instances across meshes. Before the traverse, or inside it, ensure each mesh has its own material:
```typescript
if (mesh.material) {
mesh.material = Array.isArray(mesh.material)
? mesh.material.map(m => m.clone())
: mesh.material.clone()
}
```
Only clone once — check a flag like `mesh.userData._pbrApplied` to avoid re-cloning on re-renders.
5. Add `pbrMap` to the useEffect dependency array
- **Acceptance gate**: Steel parts look metallic/reflective. Plastic parts look matte. Colors match Blender.
- **Dependencies**: Task 5
- **Risk**: Material cloning increases memory. Acceptable for viewer scenes.
### [x] Task 7: Update InlineCadViewer with same PBR logic
1. Add `perfMode` state (boolean, default false)
2. Add toolbar button (after wireframe toggle, ~line 771): `<TBtn active={perfMode} onClick={() => setPerfMode(p => !p)} title="Performance mode — merges geometries, disables per-part hover">` with `Layers` icon from lucide-react
3. Call `useGeometryMerge({ meshRegistryRef, partMaterials: effectiveMaterials, pbrMap, enabled: perfMode, sceneRef })`
4. When `perfMode` is true: disable hover handlers (set `onPointerOver`/`onPointerOut`/`onClick` to undefined on the `<primitive>` element), hide MaterialPanel part list
5. When `perfMode` is false: restore normal interaction
6. Show draw call count in toolbar badge: `renderer.info.render.calls` (read from `gl` via `useThree`)
- **Acceptance gate**: Toggle Performance mode → `renderer.info.render.calls` drops to < 20 for 100-part assembly. Toggle back → all hover/select/material interactions work.
- **Dependencies**: Task A2
- **Risk**: Medium — must ensure merged meshes inherit correct material properties (PBR). Must not break camera fitting (merged meshes have different bounding boxes).
### [ ] Task A4: Integrate Performance mode in InlineCadViewer
- **File**: `frontend/src/components/cad/InlineCadViewer.tsx`
- **What**: Mirror Task 6:
1. Add PBR query
2. Update material-application useEffect (~line 261)
3. Clone materials before modifying
4. Add `pbrMap` to dependency array
- **What**: Same as Task A3 but for the inline viewer. Add `perfMode` toggle button to toolbar (~line 455). Integrate `useGeometryMerge` hook. Disable hover when in perf mode.
- **Acceptance gate**: Same as A3 — draw calls drop, interactions restored on toggle-off.
- **Dependencies**: Task A2
- **Risk**: Low — same pattern as A3
- **Acceptance gate**: Inline viewer (product cards) shows PBR materials
- **Dependencies**: Task 5
- **Risk**: Same as Task 6
---
### [x] Task 8: Replace SCHAEFFLER_COLORS with dynamic PBR lookup in MaterialPanel
### Track B — Merge Dual STEP Parse
- **File**: `frontend/src/components/cad/MaterialPanel.tsx`
- **What**:
1. Delete the hardcoded `SCHAEFFLER_COLORS` map (lines 12-30)
2. Update `previewColorForEntry()` signature to accept optional `pbrMap`:
```typescript
export function previewColorForEntry(
entry: PartMaterialEntry,
pbrMap?: MaterialPBRMap,
): string {
if (entry.type === 'hex') return entry.value
if (pbrMap) {
const pbr = pbrMap[entry.value]
if (pbr) return pbrColorHex(pbr)
}
return '#888888'
}
```
3. Add `pbrMap` as an optional prop to `MaterialPanelProps`
4. In the material preview swatch area, show metallic/roughness values when PBR data is available:
```tsx
{pbrEntry && (
<span className="text-[10px] text-gray-500">
M:{pbrEntry.metallic.toFixed(1)} R:{pbrEntry.roughness.toFixed(1)}
</span>
)}
```
5. Update all callers of `previewColorForEntry()` in ThreeDViewer and InlineCadViewer to pass `pbrMap`
6. In the material dropdown, show a color swatch next to each material name using PBR data
### [ ] Task B1: Create `extract_step_metadata()` unified function
- **File**: `backend/app/services/step_processor.py`
- **What**: New function (insert after line 389, before `_extract_step_objects`):
```python
@dataclass
class StepMetadata:
objects: list[str] # part names from XCAF labels
edge_data: dict # sharp_edge_pairs, suggested_smooth_angle, etc.
dimensions_mm: dict | None # bbox dimensions
bbox_center_mm: dict | None
- **Acceptance gate**: Material panel shows correct preview colors from Blender. No hardcoded `SCHAEFFLER_COLORS`.
- **Dependencies**: Tasks 6, 7
- **Risk**: Low — UI-only change
def extract_step_metadata(step_path: str) -> StepMetadata:
```
### [x] Task 9: TypeScript compilation + visual verification
Implementation approach:
1. Read STEP once with `STEPCAFControl_Reader` (same as `_extract_step_objects`)
2. Extract part names from XCAF labels (same logic as current `_extract_step_objects`)
3. Get root shape via `shape_tool.GetShape(label)` for each free label
4. Tessellate at 0.5mm deflection via `BRepMesh_IncrementalMesh`
5. Extract edge topology from the tessellated shape (same logic as current `extract_mesh_edge_data` lines 265–382, but operating on the already-loaded shape instead of re-reading)
6. Extract bbox from the same shape
7. Return `StepMetadata` dataclass
- **What**:
1. `docker compose exec frontend npx tsc --noEmit` — 0 errors
2. Open http://localhost:5173/products/{id} — verify steel parts look metallic, plastics look matte
- **Acceptance gate**: Zero type errors. Visual match with Blender appearance.
- **Dependencies**: Tasks 1-8
Must handle both `OCC.Core` (pythonocc) and `OCP` (cadquery) import paths, same as existing code.
**Keep `_extract_step_objects` and `extract_mesh_edge_data` unchanged** as fallbacks.
- **Acceptance gate**: `python3 -c "import ast; ast.parse(open('backend/app/services/step_processor.py').read())"` passes. New function returns same data as the two separate calls combined.
- **Dependencies**: none
- **Risk**: Medium — the edge extraction logic references `STEPControl_Reader`-specific APIs (`reader.TransferRoots()`, `reader.OneShape()`). With `STEPCAFControl_Reader`, the shape comes from `shape_tool.GetShape(label)` instead. The edge extraction code uses `TopTools_IndexedDataMapOfShapeListOfShape` on the root shape — this should work identically on an XCAF-sourced shape since it's the same `TopoDS_Shape` underneath. Must verify the `_using_ocp` vs `OCC.Core` static method dispatch (`_s` suffix) still works.
### [ ] Task B2: Wire `extract_step_metadata()` into `extract_cad_metadata()`
- **File**: `backend/app/services/step_processor.py`
- **What**: Modify `extract_cad_metadata()` (line 82) to:
1. Try `extract_step_metadata()` first (single read)
2. If it succeeds: use `metadata.objects` for `parsed_objects`, `metadata.edge_data` for `mesh_attributes`
3. If it fails (fallback): call `_extract_step_objects()` + `extract_mesh_edge_data()` separately (existing behavior)
4. Log which path was taken: `"[STEP] unified read: X objects, Y sharp pairs"` vs `"[STEP] fallback: separate reads"`
- **Acceptance gate**: Upload a STEP file → worker log shows single "unified read" message. `parsed_objects` and `mesh_attributes` populated correctly.
- **Dependencies**: Task B1
- **Risk**: Low — fallback preserves existing behavior
### [ ] Task B3: Also wire into `process_cad_file()` (legacy path)
- **File**: `backend/app/services/step_processor.py`
- **What**: Same change as B2 but for `process_cad_file()` (line 137) which is the legacy full-pipeline function. Try unified read first, fall back to separate reads.
- **Acceptance gate**: `process_cad_file()` still works end-to-end (upload STEP → metadata + thumbnail).
- **Dependencies**: Task B1
- **Risk**: Low
## Migration Check
**No migration required.** `AssetLibrary.catalog` is JSONB (schema-free). The new format (materials as objects instead of strings) is a data-level change only.
No migration required. All changes are code-level optimizations.
## Order Recommendation
1. Render worker script (`catalog_assets.py`) + rebuild — Tasks 1-2
2. Backend API endpoint — Task 3
3. Frontend types + helpers — Tasks 4-5
4. Viewers + MaterialPanel — Tasks 6, 7, 8 (can be parallel)
5. Final check — Task 9
Track A and Track B are fully independent — implement in parallel.
Within Track A: A1 → A2 → A3 + A4 (A3 and A4 can be parallel)
Within Track B: B1 → B2 + B3 (B2 and B3 can be parallel)
## Risks / Open Questions
1. **Color space**: Blender stores linear colors. Three.js `color.setRGB()` expects sRGB. Converting in `catalog_assets.py` with `pow(v, 1/2.2)` ensures correctness in both the hex UI preview and the Three.js renderer.
1. **BufferGeometryUtils.mergeGeometries compatibility**: All geometries in a merge group must have identical attribute sets (position, normal, uv). Meshes without UVs can't merge with UV-bearing meshes. The hook must detect this and skip incompatible groups (leave them as individual meshes).
2. **Shared materials in GLB**: Three.js GLB loader shares material instances. Must clone before modifying metalness/roughness. Check `userData._pbrApplied` flag to avoid redundant cloning.
2. **Camera fitting in Performance mode**: `CameraFit` component likely uses scene bounding box. Merged meshes may have different world-space bounds than originals if transforms aren't baked correctly. Must apply `mesh.matrixWorld` to geometry before merging.
3. **Backwards compatibility**: Old catalog format (`materials: string[]`) is handled — the API endpoint skips string entries. Frontend `AssetLibraryCatalog` type uses union.
3. **OCC.Core API differences**: pythonocc (`OCC.Core`) uses different method naming than OCP (no `_s` suffix for static methods). The unified function must handle both, same as `extract_mesh_edge_data` currently does.
4. **Complex node graphs**: Materials with textures instead of simple default values get `diffuse_color` fallback. Texture support is out of scope.
4. **Edge extraction on XCAF shape**: `extract_mesh_edge_data` calls `reader.OneShape()` which returns a single compound. From XCAF, `shape_tool.GetShape(label)` returns the shape for each free label. For multi-root STEP files (rare), we need to iterate all free labels and combine edge data. This matches the pattern already used in `export_step_to_gltf.py` (line 696–700).
5. **`previewColorForEntry` callers**: This function is exported and used in both viewers. Adding the optional `pbrMap` parameter is backwards-compatible — existing callers without it still get gray fallback.
5. **Memory**: `mergeGeometries` creates new geometry buffers. For 100 parts × 50K triangles each = 5M triangles in merged buffers + 5M in originals (hidden but not disposed). May need to dispose original geometries in Performance mode and recreate on restore. This adds complexity — defer disposal to a follow-up if memory isn't an issue.