Commit Graph

16 Commits

Author SHA1 Message Date
Hartmut 531994cccd fix: OCP import for BRepGProp + route metadata task to asset_pipeline
- Fixed: from OCP.BRepGProp import BRepGProp as brepgprop (was lowercase)
- Routed reextract_rich_metadata_task to asset_pipeline queue (render-worker
  has OCC/OCP installed, worker container does not)
- Backfill verified: 45/45 products updated with volume, surface area,
  part count, complexity metrics

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 19:50:12 +01:00
Hartmut cfccdd5397 feat: rich product metadata extraction from STEP files
Extract volume, surface area, part count, assembly hierarchy, and
complexity from STEP files via OCC B-rep analysis.

Backend:
- extract_rich_metadata() in step_processor.py: computes per-part volume
  (BRepGProp), surface area, triangle/vertex count, assembly depth,
  instance count, complexity score, largest part identification
- cad_metadata JSONB column on Product model (DB migration)
- Auto-populated during STEP processing (non-fatal, 10s timeout)
- Also stored in cad_files.mesh_attributes["rich_metadata"]
- Batch re-extract endpoint: POST /admin/settings/reextract-rich-metadata

AI Agent:
- search_products returns part_count, volume_cm3, complexity, largest_part
- query_database tool description documents cad_metadata schema

Frontend:
- ProductDetail page: CAD Metadata section with stat cards
  (parts, volume, surface area, complexity, triangles, assembly depth)
- Admin System Tools: "Re-extract Rich Metadata" button for backfill

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 18:49:50 +01:00
Hartmut 5a148554c0 perf: dual queue, GLB caching, WebP output, persistent BVH
Task 4: Dual render queue
- render-worker: heavy (asset_pipeline, concurrency=1) — HQ 2048x2048, animations
- render-worker-light: light (asset_pipeline_light, concurrency=2) — thumbnails, <=1024
- Thumbnails routed to light queue automatically
- Order line renders routed by resolution at dispatch time

Task 5: GLB caching (skip re-tessellation)
- Before tessellating, check if gltf_geometry MediaAsset exists for the cad_file_id
- If found, copy to expected path — render_blender.py finds it and skips tessellation
- Saves 7-11s per re-render of the same product

Task 6: WebP output format
- New 'webp' option in output_format (OutputType admin)
- Blender renders PNG intermediate, Pillow converts to WebP (quality=90, method=4)
- 50-70% smaller files with no visible quality loss
- Correct MIME type (image/webp) in MediaAsset

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 12:07:12 +01:00
Hartmut c054236d22 fix: material override pipeline — pass --material-override CLI arg to Blender scripts
The initial implementation only overrode the material_map dict in the task,
but the Blender USD primvar path bypassed it. Now:
- Added --material-override named CLI arg parsed in _blender_args.py
- Both Mode A (factory) and Mode B (template) in _blender_scene_setup.py
  override usd_material_lookup and material_map when set
- Passed through full chain: task → step_processor → render_blender → CLI → Blender
- Tested: 175-part bearing rendered with single Steel-Bare material (1/1 materials)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 14:19:21 +01:00
Hartmut b583b0d7a2 feat: per-position camera settings, material alias dialog, product delete, media browser links
- Per-render-position focal_length_mm/sensor_width_mm (DB → pipeline → Blender)
- FOV-based camera distance with min clamp fix for wide-angle lenses
- Unmapped materials blocking dialog on "Dispatch Renders" with batch alias creation
- Material check endpoint (GET /orders/{id}/check-materials)
- Batch alias endpoint (POST /materials/batch-aliases)
- Quick-map "No alias" badges on Materials page
- Full product hard-delete with storage cleanup (MinIO + disk files + orphaned CadFile)
- Delete button on ProductDetail page with confirmation
- Clickable product names in Media Browser (links to product page)
- Single-line render dispatch/retry (POST /orders/{id}/lines/{id}/dispatch-render)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 12:16:37 +01:00
Hartmut 253f11a945 feat: surface-evaluated normals, GMSH tessellation, draw call batching
USD exporter:
- Compute normals from B-Rep surface via BRepLProp_SLProps at each vertex
  UV parameter — eliminates faceting on curved surfaces (same as Stepper)
- Add GMSH Frontal-Delaunay tessellation engine (opt-in via --tessellation_engine gmsh)
  with per-solid strategy matching export_step_to_gltf.py
- Use vertex normal interpolation instead of faceVarying (6x smaller normals)
- Default engine remains OCC (GMSH has coordinate-space bug with instanced parts)

Frontend:
- Fix faceted shading in InlineCadViewer: only call computeVertexNormals()
  when geometry lacks normals, preserving smooth GLB normals from pipeline
- Add useGeometryMerge hook for draw call batching (merge by material)
- Fix unused import in cadUtils, optional props in ThreeDViewer

Backend:
- Move dataclass import to top of step_processor.py (PEP 8)
- Unified single-read STEP metadata extraction with fallback

Render worker:
- Fix USD import seam/sharp restoration: read primvars via pxr directly
  (Blender's USD importer doesn't expose custom Int2Array primvars)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 15:14:23 +01:00
Hartmut 1321ef2bd4 refactor: rename thumbnail_rendering queue to asset_pipeline
The queue handles far more than thumbnails: OCC tessellation, USD master
generation, GLB production, order line renders, and workflow renders.
asset_pipeline better reflects its role as the render-worker's primary queue.

Updated all references in: task decorators, celery_app.py, beat_tasks.py,
docker-compose.yml worker command, worker.py MONITORED_QUEUES, admin.py,
CLAUDE.md, LEARNINGS.md, Dockerfile, helpTexts.ts, test files,
and all .claude/commands/*.md skill files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 22:28:38 +01:00
Hartmut 47b5d42bb5 refactor(P1): M1 dead code removal + M3 blender_render.py split
M1 — dead code removed:
- Delete blender-renderer/ and threejs-renderer/ source files
- Remove PIL/Pillow fallback block from step_processor.py
  (_generate_thumbnail_placeholder, _finalise_image JPG path)
- Remove stl_quality param from render_blender.py, render_still_task,
  render_turntable_task (was always "low"; hardcode deflection values)
- render_turntable_task now reads scene_linear/angular_deflection from
  system_settings (consistent with export_glb.py pipeline)

M3 — blender_render.py split from 263 → 68 lines:
- _blender_args.py: parse_args() — all 25 positional + named args
- _blender_scene_setup.py: setup_scene() — MODE A/B including USD import
- _blender_render_config.py: configure_and_render() — engine + output

Post-review fixes:
- _db_engine.dispose() after settings read in render_turntable_task
- _finalise_image() fmt param removed (always PNG; PIL never installed)
- _blender_import.py committed together with new submodules to satisfy
  import_usd_file dependency

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-12 12:54:40 +01:00
Hartmut 393e4b92a7 refactor(P1): complete pipeline cleanup — M1 dead code + M3 blender split
M1 dead code removal:
- admin.py: remove VALID_STL_QUALITIES + stl_quality (7 locations)
- frontend: remove stl_quality from 6 files (api/orders.ts, api/worker.ts,
  WorkerActivity.tsx, RenderInfoModal.tsx, helpTexts.ts, mocks/handlers.ts)
- blender_render.py: delete _mark_sharp_and_seams() — dead, never called (62 lines)
- step_processor.py: delete _render_via_service() + 2 elif renderer=="threejs" branches
- renderproblems_tmp/: remove 3 orphaned debug images

M3 blender_render.py decomposition (858 → 248 lines):
- _blender_gpu.py: activate_gpu(), configure_engine()
- _blender_import.py: import_glb(), apply_rotation()
- _blender_materials.py: FAILED_MATERIAL_NAME, assign_failed_material(),
  build_mat_map_lower(), apply_material_library()
- _blender_camera.py: setup_auto_camera(), setup_auto_lights()
- _blender_scene.py: ensure_collection(), apply_smooth_batch(),
  apply_sharp_edges_from_occ(), setup_shadow_catcher()
- Entry-point: sys.path.insert for submodule discovery; arg-parse + Mode A/B orchestration only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 22:19:59 +01:00
Hartmut ca62319688 feat: sharp edge pipeline V02, tessellation presets, media cache-bust, GMSH plan
Sharp Edge Pipeline V02:
- export_step_to_gltf.py: replace BRep_Tool.Polygon3D_s (returns None in XCAF) with
  GCPnts_UniformAbscissa curve sampling at 0.3mm step — extracts 17,129 segment pairs
- Inject sharp_edge_pairs + sharp_threshold_deg into GLB extras (scenes[0].extras)
  via binary GLB JSON-chunk patching (no extra dependency)
- export_gltf.py: read schaeffler_sharp_edge_pairs from Blender scene custom props,
  apply via KD-tree to mark edges sharp=True + seam=True (OCC mm Z-up → Blender transform)
- tools/restore_sharp_marks.py: dual-pass (dihedral angle + OCC pairs), updated coordinate
  transform (X, -Z, Y) * 0.001

Tessellation:
- Admin UI: Draft / Standard / Fine preset buttons with active-state highlighting
- Default angular deflection: preview 0.5→0.1 rad, production 0.2→0.05 rad
- export_glb.py: read updated defaults from system_settings

Media / Cache:
- media/service.py: get_download_url appends ?v={file_size_bytes} cache-buster
- media/router.py: Cache-Control: no-cache for all download/thumbnail endpoints

Render pipeline:
- still_render.py / turntable_render.py: shared GPU activation + camera improvements
- render_order_line.py: global render position support
- render_thumbnail.py: updated defaults

Frontend:
- InlineCadViewer: file_size_bytes-aware URL update triggers re-fetch on regeneration
- ThreeDViewer: material panel, part selection, PBR mode improvements
- Admin.tsx: tessellation preset cards, GMSH setting dropdown
- MediaBrowser, ProductDetail, OrderDetail, Orders: various UI improvements
- New: MaterialPanel, GlobalRenderPositionsPanel, StepIndicator components
- New: renderPositions.ts API client

Plans / Docs:
- plan.md: GMSH Frontal-Delaunay tessellation plan (6 tasks)
- LEARNINGS.md: OCC Polygon3D_s None issue + GCPnts fix
- .gitignore: add backend/core (core dump from root process)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-11 14:40:36 +01:00
Hartmut 89c44b846f feat(phase5.1+6): fallback material cleanup + notification batch refactor
Phase 5.1 — MATERIAL_PALETTE removal:
- Remove MATERIAL_PALETTE + _material_to_color() from step_processor.py
- build_part_colors() now returns {part→material_name} for Blender resolver

Phase 6 — Notification Center Refactor:
- Migration 051: add channel (activity|notification|alert) to audit_log,
  add frequency (immediate|daily|never) to notification_configs
- Three notification channels: activity (per-render), notification (batch
  order summaries), alert (admin infrastructure)
- Per-render emit_notification_sync calls demoted to channel=activity
- New emit_batch_render_notification_sync(): single summary notification
  when all order lines reach terminal state ("47/50 succeeded, 3 failed")
- Beat task batch_render_notifications every 60s: safety-net for missed
  batch notifications after order completion
- GET /notifications: defaults to channel IN (notification, alert);
  accepts ?channel=activity for activity feed
- Unread count badge counts only notification+alert channels
- Notifications.tsx: three tabs (Notifications | Activity | Alerts)
- NotificationSettings.tsx: frequency dropdown per event type

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 20:20:07 +01:00
Hartmut ee6eb34b4c feat: GPU rendering + material matching + perf improvements
- GPU: fix Cycles device activation order — set compute_device_type
  BEFORE engine init, re-set AFTER open_mainfile wipes preferences
- GPU: remove _mark_sharp_and_seams edit-mode loop (redundant with
  Blender 5.0 shade_smooth_by_angle), saves ~200s/render on 175 parts
- Material: fix _AFN suffix mismatch — build AF-stripped mat_map keys
  and add prefix fallback in _apply_material_library (blender_render.py)
- Material: production GLB now uses get_material_library_path() which
  checks active AssetLibrary instead of empty legacy system setting
- Admin: RenderTemplateTable multi-select output types (M2M frontend)
- Admin: MaterialLibraryPanel replaced with link to Asset Libraries
- UX: move Toaster to top-left to avoid dispatch button overlap
- SQLAlchemy: add .unique() to all RenderTemplate M2M collection queries
- Logging: flush=True on all Blender progress prints, stdout reconfigure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 19:05:03 +01:00
Hartmut bfd58e3419 fix: media thumbnails, product dimensions, inline 3D viewer, GLB export
Bug A: Media Library thumbnails were gray because <img src> cannot send
JWT auth headers. Added useAuthBlob() hook (fetch + createObjectURL) in
MediaBrowser.tsx. Also fixed publish_asset Celery task to populate
product_id + cad_file_id on MediaAsset for thumbnail fallback resolution.

Bug B: Product dimensions now shown in Product Details card with Ruler
icon and "from CAD" label when cad_mesh_attributes.dimensions_mm exists.

Bug C: Replaced 128×128 CAD thumbnail with InlineCadViewer component.
Queries gltf_geometry MediaAssets, fetches GLB via auth fetch → blob URL
→ Three.js Canvas with OrbitControls. Falls back to thumbnail + "Load 3D
Model" button. Polling when GLB generation is in progress.

Bug D: trimesh was in [cad] optional extra but Dockerfile only installed
[dev]. Changed to pip install -e ".[dev,cad]" — trimesh now available in
backend container, GLB + Colors export works.

Also added bbox extraction (STL-first numpy parsing) in render_step_thumbnail
and admin "Re-extract CAD Metadata" bulk endpoint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-07 13:27:46 +01:00
Hartmut 382a18fd02 feat(O): UI-Vollständigkeit + v3-Workflows + OCC-Kantenanalyse
Backend:
- Phase I: notification_configs router (GET/PUT/{event}/{channel}/POST reset)
  war bereits in notifications.py — add-alias endpoint in uploads.py ergänzt
- OutputType schema: workflow_definition_id + workflow_name fields;
  PATCH unterstützt Workflow-Zuweisung; _enrich_workflow_names() batch query
- Dispatch-Integration: orders.py dispatch_renders() → dispatch_render_with_workflow()
  mit Legacy-Fallback; neues Logging
- uploads.py: POST /validations/{id}/add-alias für Material-Lücken

Pipeline:
- step_processor.py: extract_mesh_edge_data() via OCC — berechnet Dihedralwinkel
  aller Kanten, liefert suggested_smooth_angle + sharp_edge_midpoints
  Integriert in extract_cad_metadata() und process_cad_file()
- domains/rendering/tasks.py: apply_asset_library_materials_task (K3),
  export_gltf_for_order_line_task → Blender export_gltf.py (K4),
  export_blend_for_order_line_task → export_blend.py fix (K5)
- render-worker/scripts/still_render.py: _mark_sharp_and_seams() mit
  OCC midpoint KD-tree matching + UV-Seam-Markierung
- render-worker/scripts/blender_render.py: identische Funktion + mesh_attributes parsing

Frontend:
- Layout.tsx: Upload-Link in Sidebar (alle User); Asset Libraries Link (admin/PM)
- App.tsx: /asset-libraries Route
- AssetLibrary.tsx: neue Seite (Upload, Catalog-Anzeige, Refresh, Toggle, Delete)
- OutputTypeTable.tsx: Workflow-Dropdown + Legacy/Workflow Badge
- ProductDetail.tsx: Geometry-Karte (Volumen, Surface, BBox, Sharp-Winkel)
- api/outputTypes.ts + api/products.ts: neue Felder
- api/imports.ts: ImportValidation API

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 23:20:55 +01:00
Hartmut 9d1a820295 refactor(A2): replace blender-renderer HTTP service with render-worker Celery container
- Create render-worker/ with Dockerfile (Ubuntu + cadquery + Blender via host mount)
- Add render-worker/check_version.py: verifies Blender >= 5.0.1 at startup, Exit 1 on failure
- Add render-worker/scripts/: blender_render.py, still_render.py, turntable_render.py
- Create backend/app/services/render_blender.py: direct subprocess rendering
  - convert_step_to_stl() and export_per_part_stls() using cadquery
  - render_still(): STEP → STL → PNG via Blender subprocess
  - is_blender_available(): detects BLENDER_BIN env for render-worker context
- Create backend/app/domains/rendering/tasks.py: render_still_task + render_turntable_task
- Update step_processor.py: use subprocess path when BLENDER_BIN env is set (render-worker)
- Update step_tasks.py: generate_stl_cache uses direct cadquery instead of HTTP
- Remove blender-renderer and threejs-renderer from docker-compose.yml
- Replace worker-thumbnail with render-worker (Ubuntu + cadquery + Blender mount)
- Remove Docker SDK from backend Dockerfile (was only for flamenco scaling)
- Update .env.example: BLENDER_VERSION=5.0.1 documented
- Update celery_app.py: include domains.rendering.tasks in autodiscover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 15:48:46 +01:00
Hartmut bce762a783 feat: initial commit 2026-03-05 22:12:38 +01:00