HartOMat/plan.md

Plan: Extract PBR Material Properties from Blender Asset Library for 3D Viewer

Date: 2026-03-13 | Branch: refactor/v2

Context

The 3D viewer currently shows all materials as flat colors from a hardcoded SCHAEFFLER_COLORS map in MaterialPanel.tsx (17 entries). These hex colors don't match the actual Blender materials — a "Steel-Bare" material that looks metallic and reflective in Blender renders appears as flat gray #8a9ca8 in the viewer. The user wants visual parity: if a material is blue plastic in Blender, it should look like blue plastic in the 3D viewer too.

Source of truth: The Blender .blend asset library already contains all PBR properties (Base Color, Metallic, Roughness, Transmission, IOR) in Principled BSDF nodes for all 35 Schaeffler materials. These values are defined in MaterialNamingSchema/generate_blend.py.

Current flow: catalog_assets.py extracts only material names → stored in AssetLibrary.catalog JSONB as {"materials": ["name1", ...]} → viewer uses hardcoded SCHAEFFLER_COLORS hex map.

Target flow: catalog_assets.py extracts PBR properties per material → stored in catalog JSONB → new API endpoint serves PBR map to frontend → viewers apply MeshStandardMaterial with correct color + roughness + metalness.

Affected Files

File	Change
render-worker/scripts/catalog_assets.py	Extract PBR properties from Principled BSDF nodes
backend/app/api/routers/asset_libraries.py	Add public GET /api/asset-libraries/pbr-map endpoint
frontend/src/api/assetLibraries.ts	Add fetchMaterialPBR() + MaterialPBRMap type
frontend/src/components/cad/cadUtils.ts	Add applyPBRToMaterial() + pbrColorHex() helpers
frontend/src/components/cad/ThreeDViewer.tsx	Fetch PBR map, apply PBR props when assigning materials
frontend/src/components/cad/InlineCadViewer.tsx	Same PBR application
frontend/src/components/cad/MaterialPanel.tsx	Replace hardcoded SCHAEFFLER_COLORS with dynamic PBR lookup

Tasks (in order)

[x] Task 1: Extend catalog_assets.py to extract PBR properties

• File: render-worker/scripts/catalog_assets.py

• What: After opening the .blend file, for each material with asset_data, find the ShaderNodeBsdfPrincipled node and extract:

  • base_color: [R, G, B] from inputs["Base Color"].default_value — convert linear→sRGB via v^(1/2.2)
  • metallic: float from inputs["Metallic"].default_value
  • roughness: float from inputs["Roughness"].default_value
  • transmission: float from inputs["Transmission Weight"].default_value (0.0 if absent)
  • ior: float from inputs["IOR"].default_value (1.45 default)

  Change output format from:

  {"materials": ["Mat1", "Mat2"], "node_groups": [...]}
  

  to:

  {
    "materials": [
      {"name": "Mat1", "base_color": [0.76, 0.77, 0.78], "metallic": 1.0, "roughness": 0.35, "transmission": 0.0, "ior": 1.45},
      ...
    ],
    "node_groups": [...]
  }
  

  Fallback for materials without Principled BSDF: base_color from mat.diffuse_color[:3] (already sRGB), metallic=0.0, roughness=0.5.

  Color space note: Blender's Principled BSDF stores Base Color in linear space. Three.js MeshStandardMaterial.color.setRGB() expects sRGB values (it converts internally to linear for rendering). Convert in the script: srgb = pow(linear, 1/2.2), rounded to 4 decimal places.

• Acceptance gate: Rebuilt render-worker, run catalog refresh → JSON output has PBR properties

• Dependencies: none

• Risk: Complex node graphs (textures etc.) — handled by diffuse_color fallback

[x] Task 2: Rebuild render-worker + refresh catalog

• File: No code change — operational step

• What:

  docker compose up -d --build render-worker
  # Then POST /api/asset-libraries/{id}/refresh-catalog via Admin UI or curl
  

  The AssetLibrary.catalog JSONB column is schema-free — no migration needed.

• Acceptance gate: Active library's catalog has materials with base_color, metallic, roughness

• Dependencies: Task 1

• Risk: None

[x] Task 3: Add public API endpoint for material PBR map

• File: backend/app/api/routers/asset_libraries.py

• What: Add endpoint before the /{lib_id} route (to avoid path collision):

  @router.get("/pbr-map")
  async def get_material_pbr_map(db: AsyncSession = Depends(get_db)):
      """PBR properties for all materials in the active asset library.
      Public (no auth) — needed by all 3D viewers.
      """
      result = await db.execute(
          select(AssetLibrary).where(AssetLibrary.is_active == True).limit(1)
      )
      lib = result.scalar_one_or_none()
      if not lib or not lib.catalog:
          return {}
      materials = lib.catalog.get("materials", [])
      pbr_map = {}
      for m in materials:
          if isinstance(m, str):
              continue  # old format — skip
          pbr_map[m["name"]] = {
              "base_color": m.get("base_color", [0.5, 0.5, 0.5]),
              "metallic": m.get("metallic", 0.0),
              "roughness": m.get("roughness", 0.5),
              "transmission": m.get("transmission", 0.0),
              "ior": m.get("ior", 1.45),
          }
      return JSONResponse(content=pbr_map, headers={"Cache-Control": "public, max-age=3600"})
  

• Acceptance gate: curl localhost:8888/api/asset-libraries/pbr-map returns keyed PBR map

• Dependencies: Task 2

• Risk: Must be placed before /{lib_id} route or FastAPI will try to parse "pbr-map" as a UUID

[x] Task 4: Add frontend API function + types

• File: frontend/src/api/assetLibraries.ts

• What:

  1. Add types:

     export interface MaterialPBR {
       base_color: [number, number, number]
       metallic: number
       roughness: number
       transmission?: number
       ior?: number
     }
     export type MaterialPBRMap = Record<string, MaterialPBR>
     

  2. Add fetch function:

     export async function fetchMaterialPBR(): Promise<MaterialPBRMap> {
       const { data } = await api.get<MaterialPBRMap>('/asset-libraries/pbr-map')
       return data
     }
     

  3. Update AssetLibraryCatalog.materials type from string[] to Array<string | {name: string, base_color?: number[], metallic?: number, roughness?: number}> for backwards compat with old catalogs

• Acceptance gate: npx tsc --noEmit passes

• Dependencies: Task 3

• Risk: None

[x] Task 5: Add PBR helpers in cadUtils.ts

• File: frontend/src/components/cad/cadUtils.ts

• What: Add two helpers:

  import type { MaterialPBR } from '../../api/assetLibraries'
  
  /** Apply PBR material properties to a Three.js MeshStandardMaterial. */
  export function applyPBRToMaterial(
    mat: THREE.MeshStandardMaterial,
    pbr: MaterialPBR,
  ): void {
    mat.color.setRGB(pbr.base_color[0], pbr.base_color[1], pbr.base_color[2])
    mat.metalness = pbr.metallic
    mat.roughness = pbr.roughness
    if (pbr.transmission && pbr.transmission > 0.1) {
      mat.transparent = true
      mat.opacity = 1 - pbr.transmission * 0.7
    }
  }
  
  /** Convert PBR base_color to hex string for UI swatches. */
  export function pbrColorHex(pbr: MaterialPBR): string {
    const [r, g, b] = pbr.base_color
    return '#' + [r, g, b].map(v => Math.round(v * 255).toString(16).padStart(2, '0')).join('')
  }
  

  Note: THREE is a type-only import here — the actual THREE namespace is available at runtime in the viewer components. The helper takes the material as a parameter, so no direct THREE import needed in cadUtils.

• Acceptance gate: npx tsc --noEmit passes

• Dependencies: Task 4

• Risk: None

[x] Task 6: Update ThreeDViewer to apply PBR materials

• File: frontend/src/components/cad/ThreeDViewer.tsx

• What:

  1. Import fetchMaterialPBR and applyPBRToMaterial from the new modules
  2. Add query:

     const { data: pbrMap = {} } = useQuery({
       queryKey: ['material-pbr'],
       queryFn: fetchMaterialPBR,
       staleTime: 300_000,
     })
     

  3. Update the material-application useEffect (line ~567). Current code:

     if (mat && 'color' in mat) mat.color.set(previewColorForEntry(entry))
     

     Replace with:

     if (mat && 'color' in mat) {
       if (entry.type === 'library' && pbrMap[entry.value]) {
         applyPBRToMaterial(mat as THREE.MeshStandardMaterial, pbrMap[entry.value])
       } else {
         mat.color.set(previewColorForEntry(entry, pbrMap))
       }
     }
     

  4. Important: Clone materials before modifying. GLB loader shares material instances across meshes. Before the traverse, or inside it, ensure each mesh has its own material:

     if (mesh.material) {
       mesh.material = Array.isArray(mesh.material)
         ? mesh.material.map(m => m.clone())
         : mesh.material.clone()
     }
     

     Only clone once — check a flag like mesh.userData._pbrApplied to avoid re-cloning on re-renders.
  5. Add pbrMap to the useEffect dependency array

• Acceptance gate: Steel parts look metallic/reflective. Plastic parts look matte. Colors match Blender.

• Dependencies: Task 5

• Risk: Material cloning increases memory. Acceptable for viewer scenes.

[x] Task 7: Update InlineCadViewer with same PBR logic

• File: frontend/src/components/cad/InlineCadViewer.tsx

• What: Mirror Task 6:

  1. Add PBR query
  2. Update material-application useEffect (~line 261)
  3. Clone materials before modifying
  4. Add pbrMap to dependency array

• Acceptance gate: Inline viewer (product cards) shows PBR materials

• Dependencies: Task 5

• Risk: Same as Task 6

[x] Task 8: Replace SCHAEFFLER_COLORS with dynamic PBR lookup in MaterialPanel

• File: frontend/src/components/cad/MaterialPanel.tsx

• What:

  1. Delete the hardcoded SCHAEFFLER_COLORS map (lines 12-30)
  2. Update previewColorForEntry() signature to accept optional pbrMap:

     export function previewColorForEntry(
       entry: PartMaterialEntry,
       pbrMap?: MaterialPBRMap,
     ): string {
       if (entry.type === 'hex') return entry.value
       if (pbrMap) {
         const pbr = pbrMap[entry.value]
         if (pbr) return pbrColorHex(pbr)
       }
       return '#888888'
     }
     

  3. Add pbrMap as an optional prop to MaterialPanelProps
  4. In the material preview swatch area, show metallic/roughness values when PBR data is available:

     {pbrEntry && (
       <span className="text-[10px] text-gray-500">
         M:{pbrEntry.metallic.toFixed(1)} R:{pbrEntry.roughness.toFixed(1)}
       </span>
     )}
     

  5. Update all callers of previewColorForEntry() in ThreeDViewer and InlineCadViewer to pass pbrMap
  6. In the material dropdown, show a color swatch next to each material name using PBR data

• Acceptance gate: Material panel shows correct preview colors from Blender. No hardcoded SCHAEFFLER_COLORS.

• Dependencies: Tasks 6, 7

• Risk: Low — UI-only change

[x] Task 9: TypeScript compilation + visual verification

• What:
  1. docker compose exec frontend npx tsc --noEmit — 0 errors
  2. Open http://localhost:5173/products/{id} — verify steel parts look metallic, plastics look matte
• Acceptance gate: Zero type errors. Visual match with Blender appearance.
• Dependencies: Tasks 1-8

Migration Check

No migration required. AssetLibrary.catalog is JSONB (schema-free). The new format (materials as objects instead of strings) is a data-level change only.

Order Recommendation

1. Render worker script (catalog_assets.py) + rebuild — Tasks 1-2
2. Backend API endpoint — Task 3
3. Frontend types + helpers — Tasks 4-5
4. Viewers + MaterialPanel — Tasks 6, 7, 8 (can be parallel)
5. Final check — Task 9

Risks / Open Questions

1. Color space: Blender stores linear colors. Three.js color.setRGB() expects sRGB. Converting in catalog_assets.py with pow(v, 1/2.2) ensures correctness in both the hex UI preview and the Three.js renderer.

2. Shared materials in GLB: Three.js GLB loader shares material instances. Must clone before modifying metalness/roughness. Check userData._pbrApplied flag to avoid redundant cloning.

3. Backwards compatibility: Old catalog format (materials: string[]) is handled — the API endpoint skips string entries. Frontend AssetLibraryCatalog type uses union.

4. Complex node graphs: Materials with textures instead of simple default values get diffuse_color fallback. Texture support is out of scope.

5. previewColorForEntry callers: This function is exported and used in both viewers. Adding the optional pbrMap parameter is backwards-compatible — existing callers without it still get gray fallback.