# Plan: Extract PBR Material Properties from Blender Asset Library for 3D Viewer > **Date:** 2026-03-13 | **Branch:** refactor/v2 ## Context The 3D viewer currently shows all materials as flat colors from a hardcoded `SCHAEFFLER_COLORS` map in `MaterialPanel.tsx` (17 entries). These hex colors don't match the actual Blender materials — a "Steel-Bare" material that looks metallic and reflective in Blender renders appears as flat gray `#8a9ca8` in the viewer. The user wants visual parity: if a material is blue plastic in Blender, it should look like blue plastic in the 3D viewer too. **Source of truth**: The Blender `.blend` asset library already contains all PBR properties (Base Color, Metallic, Roughness, Transmission, IOR) in Principled BSDF nodes for all 35 Schaeffler materials. These values are defined in `MaterialNamingSchema/generate_blend.py`. **Current flow**: `catalog_assets.py` extracts only material **names** → stored in `AssetLibrary.catalog` JSONB as `{"materials": ["name1", ...]}` → viewer uses hardcoded `SCHAEFFLER_COLORS` hex map. **Target flow**: `catalog_assets.py` extracts PBR properties per material → stored in catalog JSONB → new API endpoint serves PBR map to frontend → viewers apply `MeshStandardMaterial` with correct color + roughness + metalness. ## Affected Files | File | Change | |------|--------| | `render-worker/scripts/catalog_assets.py` | Extract PBR properties from Principled BSDF nodes | | `backend/app/api/routers/asset_libraries.py` | Add public `GET /api/asset-libraries/pbr-map` endpoint | | `frontend/src/api/assetLibraries.ts` | Add `fetchMaterialPBR()` + `MaterialPBRMap` type | | `frontend/src/components/cad/cadUtils.ts` | Add `applyPBRToMaterial()` + `pbrColorHex()` helpers | | `frontend/src/components/cad/ThreeDViewer.tsx` | Fetch PBR map, apply PBR props when assigning materials | | `frontend/src/components/cad/InlineCadViewer.tsx` | Same PBR application | | `frontend/src/components/cad/MaterialPanel.tsx` | Replace hardcoded `SCHAEFFLER_COLORS` with dynamic PBR lookup | ## Tasks (in order) ### [x] Task 1: Extend catalog_assets.py to extract PBR properties - **File**: `render-worker/scripts/catalog_assets.py` - **What**: After opening the .blend file, for each material with `asset_data`, find the `ShaderNodeBsdfPrincipled` node and extract: - `base_color`: `[R, G, B]` from `inputs["Base Color"].default_value` — convert linear→sRGB via `v^(1/2.2)` - `metallic`: float from `inputs["Metallic"].default_value` - `roughness`: float from `inputs["Roughness"].default_value` - `transmission`: float from `inputs["Transmission Weight"].default_value` (0.0 if absent) - `ior`: float from `inputs["IOR"].default_value` (1.45 default) Change output format from: ```json {"materials": ["Mat1", "Mat2"], "node_groups": [...]} ``` to: ```json { "materials": [ {"name": "Mat1", "base_color": [0.76, 0.77, 0.78], "metallic": 1.0, "roughness": 0.35, "transmission": 0.0, "ior": 1.45}, ... ], "node_groups": [...] } ``` Fallback for materials without Principled BSDF: `base_color` from `mat.diffuse_color[:3]` (already sRGB), metallic=0.0, roughness=0.5. **Color space note**: Blender's Principled BSDF stores Base Color in **linear** space. Three.js `MeshStandardMaterial.color.setRGB()` expects **sRGB** values (it converts internally to linear for rendering). Convert in the script: `srgb = pow(linear, 1/2.2)`, rounded to 4 decimal places. - **Acceptance gate**: Rebuilt render-worker, run catalog refresh → JSON output has PBR properties - **Dependencies**: none - **Risk**: Complex node graphs (textures etc.) — handled by diffuse_color fallback ### [x] Task 2: Rebuild render-worker + refresh catalog - **File**: No code change — operational step - **What**: ```bash docker compose up -d --build render-worker # Then POST /api/asset-libraries/{id}/refresh-catalog via Admin UI or curl ``` The `AssetLibrary.catalog` JSONB column is schema-free — no migration needed. - **Acceptance gate**: Active library's catalog has materials with `base_color`, `metallic`, `roughness` - **Dependencies**: Task 1 - **Risk**: None ### [x] Task 3: Add public API endpoint for material PBR map - **File**: `backend/app/api/routers/asset_libraries.py` - **What**: Add endpoint **before** the `/{lib_id}` route (to avoid path collision): ```python @router.get("/pbr-map") async def get_material_pbr_map(db: AsyncSession = Depends(get_db)): """PBR properties for all materials in the active asset library. Public (no auth) — needed by all 3D viewers. """ result = await db.execute( select(AssetLibrary).where(AssetLibrary.is_active == True).limit(1) ) lib = result.scalar_one_or_none() if not lib or not lib.catalog: return {} materials = lib.catalog.get("materials", []) pbr_map = {} for m in materials: if isinstance(m, str): continue # old format — skip pbr_map[m["name"]] = { "base_color": m.get("base_color", [0.5, 0.5, 0.5]), "metallic": m.get("metallic", 0.0), "roughness": m.get("roughness", 0.5), "transmission": m.get("transmission", 0.0), "ior": m.get("ior", 1.45), } return JSONResponse(content=pbr_map, headers={"Cache-Control": "public, max-age=3600"}) ``` - **Acceptance gate**: `curl localhost:8888/api/asset-libraries/pbr-map` returns keyed PBR map - **Dependencies**: Task 2 - **Risk**: Must be placed before `/{lib_id}` route or FastAPI will try to parse "pbr-map" as a UUID ### [x] Task 4: Add frontend API function + types - **File**: `frontend/src/api/assetLibraries.ts` - **What**: 1. Add types: ```typescript export interface MaterialPBR { base_color: [number, number, number] metallic: number roughness: number transmission?: number ior?: number } export type MaterialPBRMap = Record ``` 2. Add fetch function: ```typescript export async function fetchMaterialPBR(): Promise { const { data } = await api.get('/asset-libraries/pbr-map') return data } ``` 3. Update `AssetLibraryCatalog.materials` type from `string[]` to `Array` for backwards compat with old catalogs - **Acceptance gate**: `npx tsc --noEmit` passes - **Dependencies**: Task 3 - **Risk**: None ### [x] Task 5: Add PBR helpers in cadUtils.ts - **File**: `frontend/src/components/cad/cadUtils.ts` - **What**: Add two helpers: ```typescript import type { MaterialPBR } from '../../api/assetLibraries' /** Apply PBR material properties to a Three.js MeshStandardMaterial. */ export function applyPBRToMaterial( mat: THREE.MeshStandardMaterial, pbr: MaterialPBR, ): void { mat.color.setRGB(pbr.base_color[0], pbr.base_color[1], pbr.base_color[2]) mat.metalness = pbr.metallic mat.roughness = pbr.roughness if (pbr.transmission && pbr.transmission > 0.1) { mat.transparent = true mat.opacity = 1 - pbr.transmission * 0.7 } } /** Convert PBR base_color to hex string for UI swatches. */ export function pbrColorHex(pbr: MaterialPBR): string { const [r, g, b] = pbr.base_color return '#' + [r, g, b].map(v => Math.round(v * 255).toString(16).padStart(2, '0')).join('') } ``` Note: `THREE` is a type-only import here — the actual THREE namespace is available at runtime in the viewer components. The helper takes the material as a parameter, so no direct THREE import needed in cadUtils. - **Acceptance gate**: `npx tsc --noEmit` passes - **Dependencies**: Task 4 - **Risk**: None ### [x] Task 6: Update ThreeDViewer to apply PBR materials - **File**: `frontend/src/components/cad/ThreeDViewer.tsx` - **What**: 1. Import `fetchMaterialPBR` and `applyPBRToMaterial` from the new modules 2. Add query: ```typescript const { data: pbrMap = {} } = useQuery({ queryKey: ['material-pbr'], queryFn: fetchMaterialPBR, staleTime: 300_000, }) ``` 3. Update the material-application `useEffect` (line ~567). Current code: ```typescript if (mat && 'color' in mat) mat.color.set(previewColorForEntry(entry)) ``` Replace with: ```typescript if (mat && 'color' in mat) { if (entry.type === 'library' && pbrMap[entry.value]) { applyPBRToMaterial(mat as THREE.MeshStandardMaterial, pbrMap[entry.value]) } else { mat.color.set(previewColorForEntry(entry, pbrMap)) } } ``` 4. **Important**: Clone materials before modifying. GLB loader shares material instances across meshes. Before the traverse, or inside it, ensure each mesh has its own material: ```typescript if (mesh.material) { mesh.material = Array.isArray(mesh.material) ? mesh.material.map(m => m.clone()) : mesh.material.clone() } ``` Only clone once — check a flag like `mesh.userData._pbrApplied` to avoid re-cloning on re-renders. 5. Add `pbrMap` to the useEffect dependency array - **Acceptance gate**: Steel parts look metallic/reflective. Plastic parts look matte. Colors match Blender. - **Dependencies**: Task 5 - **Risk**: Material cloning increases memory. Acceptable for viewer scenes. ### [x] Task 7: Update InlineCadViewer with same PBR logic - **File**: `frontend/src/components/cad/InlineCadViewer.tsx` - **What**: Mirror Task 6: 1. Add PBR query 2. Update material-application useEffect (~line 261) 3. Clone materials before modifying 4. Add `pbrMap` to dependency array - **Acceptance gate**: Inline viewer (product cards) shows PBR materials - **Dependencies**: Task 5 - **Risk**: Same as Task 6 ### [x] Task 8: Replace SCHAEFFLER_COLORS with dynamic PBR lookup in MaterialPanel - **File**: `frontend/src/components/cad/MaterialPanel.tsx` - **What**: 1. Delete the hardcoded `SCHAEFFLER_COLORS` map (lines 12-30) 2. Update `previewColorForEntry()` signature to accept optional `pbrMap`: ```typescript export function previewColorForEntry( entry: PartMaterialEntry, pbrMap?: MaterialPBRMap, ): string { if (entry.type === 'hex') return entry.value if (pbrMap) { const pbr = pbrMap[entry.value] if (pbr) return pbrColorHex(pbr) } return '#888888' } ``` 3. Add `pbrMap` as an optional prop to `MaterialPanelProps` 4. In the material preview swatch area, show metallic/roughness values when PBR data is available: ```tsx {pbrEntry && ( M:{pbrEntry.metallic.toFixed(1)} R:{pbrEntry.roughness.toFixed(1)} )} ``` 5. Update all callers of `previewColorForEntry()` in ThreeDViewer and InlineCadViewer to pass `pbrMap` 6. In the material dropdown, show a color swatch next to each material name using PBR data - **Acceptance gate**: Material panel shows correct preview colors from Blender. No hardcoded `SCHAEFFLER_COLORS`. - **Dependencies**: Tasks 6, 7 - **Risk**: Low — UI-only change ### [x] Task 9: TypeScript compilation + visual verification - **What**: 1. `docker compose exec frontend npx tsc --noEmit` — 0 errors 2. Open http://localhost:5173/products/{id} — verify steel parts look metallic, plastics look matte - **Acceptance gate**: Zero type errors. Visual match with Blender appearance. - **Dependencies**: Tasks 1-8 ## Migration Check **No migration required.** `AssetLibrary.catalog` is JSONB (schema-free). The new format (materials as objects instead of strings) is a data-level change only. ## Order Recommendation 1. Render worker script (`catalog_assets.py`) + rebuild — Tasks 1-2 2. Backend API endpoint — Task 3 3. Frontend types + helpers — Tasks 4-5 4. Viewers + MaterialPanel — Tasks 6, 7, 8 (can be parallel) 5. Final check — Task 9 ## Risks / Open Questions 1. **Color space**: Blender stores linear colors. Three.js `color.setRGB()` expects sRGB. Converting in `catalog_assets.py` with `pow(v, 1/2.2)` ensures correctness in both the hex UI preview and the Three.js renderer. 2. **Shared materials in GLB**: Three.js GLB loader shares material instances. Must clone before modifying metalness/roughness. Check `userData._pbrApplied` flag to avoid redundant cloning. 3. **Backwards compatibility**: Old catalog format (`materials: string[]`) is handled — the API endpoint skips string entries. Frontend `AssetLibraryCatalog` type uses union. 4. **Complex node graphs**: Materials with textures instead of simple default values get `diffuse_color` fallback. Texture support is out of scope. 5. **`previewColorForEntry` callers**: This function is exported and used in both viewers. Adding the optional `pbrMap` parameter is backwards-compatible — existing callers without it still get gray fallback.