HartOMat/plan.md

# Plan: Extract PBR Material Properties from Blender Asset Library for 3D Viewer

> **Date:** 2026-03-13 | **Branch:** refactor/v2

## Context

The 3D viewer currently shows all materials as flat colors from a hardcoded `SCHAEFFLER_COLORS` map in `MaterialPanel.tsx` (17 entries). These hex colors don't match the actual Blender materials — a "Steel-Bare" material that looks metallic and reflective in Blender renders appears as flat gray `#8a9ca8` in the viewer. The user wants visual parity: if a material is blue plastic in Blender, it should look like blue plastic in the 3D viewer too.

**Source of truth**: The Blender `.blend` asset library already contains all PBR properties (Base Color, Metallic, Roughness, Transmission, IOR) in Principled BSDF nodes for all 35 Schaeffler materials. These values are defined in `MaterialNamingSchema/generate_blend.py`.

**Current flow**: `catalog_assets.py` extracts only material **names** → stored in `AssetLibrary.catalog` JSONB as `{"materials": ["name1", ...]}` → viewer uses hardcoded `SCHAEFFLER_COLORS` hex map.

**Target flow**: `catalog_assets.py` extracts PBR properties per material → stored in catalog JSONB → new API endpoint serves PBR map to frontend → viewers apply `MeshStandardMaterial` with correct color + roughness + metalness.

## Affected Files

| File | Change |
|------|--------|
| `render-worker/scripts/catalog_assets.py` | Extract PBR properties from Principled BSDF nodes |
| `backend/app/api/routers/asset_libraries.py` | Add public `GET /api/asset-libraries/pbr-map` endpoint |
| `frontend/src/api/assetLibraries.ts` | Add `fetchMaterialPBR()` + `MaterialPBRMap` type |
| `frontend/src/components/cad/cadUtils.ts` | Add `applyPBRToMaterial()` + `pbrColorHex()` helpers |
| `frontend/src/components/cad/ThreeDViewer.tsx` | Fetch PBR map, apply PBR props when assigning materials |
| `frontend/src/components/cad/InlineCadViewer.tsx` | Same PBR application |
| `frontend/src/components/cad/MaterialPanel.tsx` | Replace hardcoded `SCHAEFFLER_COLORS` with dynamic PBR lookup |

## Tasks (in order)

### [x] Task 1: Extend catalog_assets.py to extract PBR properties

- **File**: `render-worker/scripts/catalog_assets.py`
- **What**: After opening the .blend file, for each material with `asset_data`, find the `ShaderNodeBsdfPrincipled` node and extract:
  - `base_color`: `[R, G, B]` from `inputs["Base Color"].default_value` — convert linear→sRGB via `v^(1/2.2)`
  - `metallic`: float from `inputs["Metallic"].default_value`
  - `roughness`: float from `inputs["Roughness"].default_value`
  - `transmission`: float from `inputs["Transmission Weight"].default_value` (0.0 if absent)
  - `ior`: float from `inputs["IOR"].default_value` (1.45 default)

  Change output format from:
  ```json
  {"materials": ["Mat1", "Mat2"], "node_groups": [...]}
  ```
  to:
  ```json
  {
    "materials": [
      {"name": "Mat1", "base_color": [0.76, 0.77, 0.78], "metallic": 1.0, "roughness": 0.35, "transmission": 0.0, "ior": 1.45},
      ...
    ],
    "node_groups": [...]
  }
  ```

  Fallback for materials without Principled BSDF: `base_color` from `mat.diffuse_color[:3]` (already sRGB), metallic=0.0, roughness=0.5.

  **Color space note**: Blender's Principled BSDF stores Base Color in **linear** space. Three.js `MeshStandardMaterial.color.setRGB()` expects **sRGB** values (it converts internally to linear for rendering). Convert in the script: `srgb = pow(linear, 1/2.2)`, rounded to 4 decimal places.

- **Acceptance gate**: Rebuilt render-worker, run catalog refresh → JSON output has PBR properties
- **Dependencies**: none
- **Risk**: Complex node graphs (textures etc.) — handled by diffuse_color fallback

### [x] Task 2: Rebuild render-worker + refresh catalog

- **File**: No code change — operational step
- **What**:
  ```bash
  docker compose up -d --build render-worker
  # Then POST /api/asset-libraries/{id}/refresh-catalog via Admin UI or curl
  ```
  The `AssetLibrary.catalog` JSONB column is schema-free — no migration needed.

- **Acceptance gate**: Active library's catalog has materials with `base_color`, `metallic`, `roughness`
- **Dependencies**: Task 1
- **Risk**: None

### [x] Task 3: Add public API endpoint for material PBR map

- **File**: `backend/app/api/routers/asset_libraries.py`
- **What**: Add endpoint **before** the `/{lib_id}` route (to avoid path collision):

  ```python
  @router.get("/pbr-map")
  async def get_material_pbr_map(db: AsyncSession = Depends(get_db)):
      """PBR properties for all materials in the active asset library.
      Public (no auth) — needed by all 3D viewers.
      """
      result = await db.execute(
          select(AssetLibrary).where(AssetLibrary.is_active == True).limit(1)
      )
      lib = result.scalar_one_or_none()
      if not lib or not lib.catalog:
          return {}
      materials = lib.catalog.get("materials", [])
      pbr_map = {}
      for m in materials:
          if isinstance(m, str):
              continue  # old format — skip
          pbr_map[m["name"]] = {
              "base_color": m.get("base_color", [0.5, 0.5, 0.5]),
              "metallic": m.get("metallic", 0.0),
              "roughness": m.get("roughness", 0.5),
              "transmission": m.get("transmission", 0.0),
              "ior": m.get("ior", 1.45),
          }
      return JSONResponse(content=pbr_map, headers={"Cache-Control": "public, max-age=3600"})
  ```

- **Acceptance gate**: `curl localhost:8888/api/asset-libraries/pbr-map` returns keyed PBR map
- **Dependencies**: Task 2
- **Risk**: Must be placed before `/{lib_id}` route or FastAPI will try to parse "pbr-map" as a UUID

### [x] Task 4: Add frontend API function + types

- **File**: `frontend/src/api/assetLibraries.ts`
- **What**:
  1. Add types:
     ```typescript
     export interface MaterialPBR {
       base_color: [number, number, number]
       metallic: number
       roughness: number
       transmission?: number
       ior?: number
     }
     export type MaterialPBRMap = Record<string, MaterialPBR>
     ```
  2. Add fetch function:
     ```typescript
     export async function fetchMaterialPBR(): Promise<MaterialPBRMap> {
       const { data } = await api.get<MaterialPBRMap>('/asset-libraries/pbr-map')
       return data
     }
     ```
  3. Update `AssetLibraryCatalog.materials` type from `string[]` to `Array<string | {name: string, base_color?: number[], metallic?: number, roughness?: number}>` for backwards compat with old catalogs

- **Acceptance gate**: `npx tsc --noEmit` passes
- **Dependencies**: Task 3
- **Risk**: None

### [x] Task 5: Add PBR helpers in cadUtils.ts

- **File**: `frontend/src/components/cad/cadUtils.ts`
- **What**: Add two helpers:

  ```typescript
  import type { MaterialPBR } from '../../api/assetLibraries'

  /** Apply PBR material properties to a Three.js MeshStandardMaterial. */
  export function applyPBRToMaterial(
    mat: THREE.MeshStandardMaterial,
    pbr: MaterialPBR,
  ): void {
    mat.color.setRGB(pbr.base_color[0], pbr.base_color[1], pbr.base_color[2])
    mat.metalness = pbr.metallic
    mat.roughness = pbr.roughness
    if (pbr.transmission && pbr.transmission > 0.1) {
      mat.transparent = true
      mat.opacity = 1 - pbr.transmission * 0.7
    }
  }

  /** Convert PBR base_color to hex string for UI swatches. */
  export function pbrColorHex(pbr: MaterialPBR): string {
    const [r, g, b] = pbr.base_color
    return '#' + [r, g, b].map(v => Math.round(v * 255).toString(16).padStart(2, '0')).join('')
  }
  ```

  Note: `THREE` is a type-only import here — the actual THREE namespace is available at runtime in the viewer components. The helper takes the material as a parameter, so no direct THREE import needed in cadUtils.

- **Acceptance gate**: `npx tsc --noEmit` passes
- **Dependencies**: Task 4
- **Risk**: None

### [x] Task 6: Update ThreeDViewer to apply PBR materials

- **File**: `frontend/src/components/cad/ThreeDViewer.tsx`
- **What**:
  1. Import `fetchMaterialPBR` and `applyPBRToMaterial` from the new modules
  2. Add query:
     ```typescript
     const { data: pbrMap = {} } = useQuery({
       queryKey: ['material-pbr'],
       queryFn: fetchMaterialPBR,
       staleTime: 300_000,
     })
     ```
  3. Update the material-application `useEffect` (line ~567). Current code:
     ```typescript
     if (mat && 'color' in mat) mat.color.set(previewColorForEntry(entry))
     ```
     Replace with:
     ```typescript
     if (mat && 'color' in mat) {
       if (entry.type === 'library' && pbrMap[entry.value]) {
         applyPBRToMaterial(mat as THREE.MeshStandardMaterial, pbrMap[entry.value])
       } else {
         mat.color.set(previewColorForEntry(entry, pbrMap))
       }
     }
     ```
  4. **Important**: Clone materials before modifying. GLB loader shares material instances across meshes. Before the traverse, or inside it, ensure each mesh has its own material:
     ```typescript
     if (mesh.material) {
       mesh.material = Array.isArray(mesh.material)
         ? mesh.material.map(m => m.clone())
         : mesh.material.clone()
     }
     ```
     Only clone once — check a flag like `mesh.userData._pbrApplied` to avoid re-cloning on re-renders.
  5. Add `pbrMap` to the useEffect dependency array

- **Acceptance gate**: Steel parts look metallic/reflective. Plastic parts look matte. Colors match Blender.
- **Dependencies**: Task 5
- **Risk**: Material cloning increases memory. Acceptable for viewer scenes.

### [x] Task 7: Update InlineCadViewer with same PBR logic

- **File**: `frontend/src/components/cad/InlineCadViewer.tsx`
- **What**: Mirror Task 6:
  1. Add PBR query
  2. Update material-application useEffect (~line 261)
  3. Clone materials before modifying
  4. Add `pbrMap` to dependency array

- **Acceptance gate**: Inline viewer (product cards) shows PBR materials
- **Dependencies**: Task 5
- **Risk**: Same as Task 6

### [x] Task 8: Replace SCHAEFFLER_COLORS with dynamic PBR lookup in MaterialPanel

- **File**: `frontend/src/components/cad/MaterialPanel.tsx`
- **What**:
  1. Delete the hardcoded `SCHAEFFLER_COLORS` map (lines 12-30)
  2. Update `previewColorForEntry()` signature to accept optional `pbrMap`:
     ```typescript
     export function previewColorForEntry(
       entry: PartMaterialEntry,
       pbrMap?: MaterialPBRMap,
     ): string {
       if (entry.type === 'hex') return entry.value
       if (pbrMap) {
         const pbr = pbrMap[entry.value]
         if (pbr) return pbrColorHex(pbr)
       }
       return '#888888'
     }
     ```
  3. Add `pbrMap` as an optional prop to `MaterialPanelProps`
  4. In the material preview swatch area, show metallic/roughness values when PBR data is available:
     ```tsx
     {pbrEntry && (
       <span className="text-[10px] text-gray-500">
         M:{pbrEntry.metallic.toFixed(1)} R:{pbrEntry.roughness.toFixed(1)}
       </span>
     )}
     ```
  5. Update all callers of `previewColorForEntry()` in ThreeDViewer and InlineCadViewer to pass `pbrMap`
  6. In the material dropdown, show a color swatch next to each material name using PBR data

- **Acceptance gate**: Material panel shows correct preview colors from Blender. No hardcoded `SCHAEFFLER_COLORS`.
- **Dependencies**: Tasks 6, 7
- **Risk**: Low — UI-only change

### [x] Task 9: TypeScript compilation + visual verification

- **What**:
  1. `docker compose exec frontend npx tsc --noEmit` — 0 errors
  2. Open http://localhost:5173/products/{id} — verify steel parts look metallic, plastics look matte
- **Acceptance gate**: Zero type errors. Visual match with Blender appearance.
- **Dependencies**: Tasks 1-8

## Migration Check

**No migration required.** `AssetLibrary.catalog` is JSONB (schema-free). The new format (materials as objects instead of strings) is a data-level change only.

## Order Recommendation

1. Render worker script (`catalog_assets.py`) + rebuild — Tasks 1-2
2. Backend API endpoint — Task 3
3. Frontend types + helpers — Tasks 4-5
4. Viewers + MaterialPanel — Tasks 6, 7, 8 (can be parallel)
5. Final check — Task 9

## Risks / Open Questions

1. **Color space**: Blender stores linear colors. Three.js `color.setRGB()` expects sRGB. Converting in `catalog_assets.py` with `pow(v, 1/2.2)` ensures correctness in both the hex UI preview and the Three.js renderer.

2. **Shared materials in GLB**: Three.js GLB loader shares material instances. Must clone before modifying metalness/roughness. Check `userData._pbrApplied` flag to avoid redundant cloning.

3. **Backwards compatibility**: Old catalog format (`materials: string[]`) is handled — the API endpoint skips string entries. Frontend `AssetLibraryCatalog` type uses union.

4. **Complex node graphs**: Materials with textures instead of simple default values get `diffuse_color` fallback. Texture support is out of scope.

5. **`previewColorForEntry` callers**: This function is exported and used in both viewers. Adding the optional `pbrMap` parameter is backwards-compatible — existing callers without it still get gray fallback.