Hartmut/HartOMat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
54 Commits 1 Branch 0 Tags
934728da77009e101e28001f6073fb0de6a3fa57
Commit Graph

20 Commits

Author SHA1 Message Date
Hartmut 934728da77 fix(glb): 175/175 material substitution via _AFN suffix stripping 
OCC's RWGltf_CafWriter appends _AF0/_AF1 assembly-instance suffixes to
mesh object names when a part appears multiple times in an assembly.
The material matching in export_gltf.py only stripped Blender's .001
suffix, leaving 24/175 GLB objects without materials.

Fix: strip _AFN suffixes via while loop (handles nested _AF0_AF1),
add prefix fallback (longest key wins) as last resort before no-match.

Also improve build_materials_from_excel Jaccard matching:
- Strip _AFN and numeric hash suffixes (-21227) before tokenizing
- Add prefix-based fallback (step 3) before position fallback (step 4)
- Raise threshold 0.3 → 0.35 for better precision
- Guard prefix matches to len >= 5 to prevent trivial false positives

Result: Material substitution: 175/175 mesh objects assigned (was 151/175)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 22:15:11 +01:00
Hartmut 95cfe0aa93 refactor: replace STL intermediary with OCC-native STEP→GLB pipeline 
- export_step_to_gltf.py: STEP→GLB via RWGltf_CafWriter + BRepBuilderAPI_Transform
  (mm→m pre-scaling, XCAFDoc_ShapeTool.GetComponents_s static method)
- Blender scripts (blender_render.py, still_render.py, turntable_render.py,
  export_gltf.py, export_blend.py): import GLB instead of STL, remove _scale_mm_to_m
- step_tasks.py: add generate_gltf_production_task, remove generate_stl_cache,
  replace _bbox_from_stl with _bbox_from_glb (trimesh), auto-queue geometry GLB
  after thumbnail render
- render_blender.py: replace _stl_from_cache_or_convert with _glb_from_step,
  remove convert_step_to_stl and export_per_part_stls
- domains/rendering/tasks.py: update render_turntable_task, export_gltf/blend tasks
  to use GLB instead of STL
- cad.py: remove STL download/generate endpoints, add generate-gltf-production
- admin.py: generate-missing-stls → generate-missing-geometry-glbs
- Frontend: replace STL cache UI with GLB generate buttons, remove stl_cached field

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 16:49:18 +01:00
Hartmut bfd58e3419 fix: media thumbnails, product dimensions, inline 3D viewer, GLB export 
Bug A: Media Library thumbnails were gray because <img src> cannot send
JWT auth headers. Added useAuthBlob() hook (fetch + createObjectURL) in
MediaBrowser.tsx. Also fixed publish_asset Celery task to populate
product_id + cad_file_id on MediaAsset for thumbnail fallback resolution.

Bug B: Product dimensions now shown in Product Details card with Ruler
icon and "from CAD" label when cad_mesh_attributes.dimensions_mm exists.

Bug C: Replaced 128×128 CAD thumbnail with InlineCadViewer component.
Queries gltf_geometry MediaAssets, fetches GLB via auth fetch → blob URL
→ Three.js Canvas with OrbitControls. Falls back to thumbnail + "Load 3D
Model" button. Polling when GLB generation is in progress.

Bug D: trimesh was in [cad] optional extra but Dockerfile only installed
[dev]. Changed to pip install -e ".[dev,cad]" — trimesh now available in
backend container, GLB + Colors export works.

Also added bbox extraction (STL-first numpy parsing) in render_step_thumbnail
and admin "Re-extract CAD Metadata" bulk endpoint.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 13:27:46 +01:00
Hartmut c7d74ec636 fix: use processing_status column in import-media-assets endpoint 2026-03-07 00:19:31 +01:00
Hartmut f5ca91ee02 feat: layout hamburger, media browser filters+previews, billing fixes 
- Layout: mobile hamburger menu + overlay backdrop + close button; content area always full-width
- Media browser: filter chips (default still+turntable); advanced toggle for GLB/STL; thumbnail_url previews for non-image types; video hover-play for turntable
- Backend: asset_types multi-filter, thumbnail_url in MediaAssetOut, download proxy endpoint for MinIO/local files
- Admin: "Import Existing Media" button → POST /api/admin/import-media-assets
- Billing: fix invoice create 500 (MissingGreenlet — use selectinload after commit); PDF download uses axios blob instead of bare <a href> (auth header missing); fix storage.upload() accepting str|Path
- SSE task logs: task_logs.py core + router, LiveRenderLog component
- CadPreview: fix infinite loop when no gltf_geometry assets; loading screen before ThreeDViewer render
- render-worker: add trimesh layer to Dockerfile

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-07 00:09:27 +01:00
Hartmut 382a18fd02 feat(O): UI-Vollständigkeit + v3-Workflows + OCC-Kantenanalyse 
Backend:
- Phase I: notification_configs router (GET/PUT/{event}/{channel}/POST reset)
  war bereits in notifications.py — add-alias endpoint in uploads.py ergänzt
- OutputType schema: workflow_definition_id + workflow_name fields;
  PATCH unterstützt Workflow-Zuweisung; _enrich_workflow_names() batch query
- Dispatch-Integration: orders.py dispatch_renders() → dispatch_render_with_workflow()
  mit Legacy-Fallback; neues Logging
- uploads.py: POST /validations/{id}/add-alias für Material-Lücken

Pipeline:
- step_processor.py: extract_mesh_edge_data() via OCC — berechnet Dihedralwinkel
  aller Kanten, liefert suggested_smooth_angle + sharp_edge_midpoints
  Integriert in extract_cad_metadata() und process_cad_file()
- domains/rendering/tasks.py: apply_asset_library_materials_task (K3),
  export_gltf_for_order_line_task → Blender export_gltf.py (K4),
  export_blend_for_order_line_task → export_blend.py fix (K5)
- render-worker/scripts/still_render.py: _mark_sharp_and_seams() mit
  OCC midpoint KD-tree matching + UV-Seam-Markierung
- render-worker/scripts/blender_render.py: identische Funktion + mesh_attributes parsing

Frontend:
- Layout.tsx: Upload-Link in Sidebar (alle User); Asset Libraries Link (admin/PM)
- App.tsx: /asset-libraries Route
- AssetLibrary.tsx: neue Seite (Upload, Catalog-Anzeige, Refresh, Toggle, Delete)
- OutputTypeTable.tsx: Workflow-Dropdown + Legacy/Workflow Badge
- ProductDetail.tsx: Geometry-Karte (Volumen, Surface, BBox, Sharp-Winkel)
- api/outputTypes.ts + api/products.ts: neue Felder
- api/imports.ts: ImportValidation API

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 23:20:55 +01:00
Hartmut a70cb55d01 feat(N): workflow pipeline, 3D viewer, worker management, QC tests 
- workflow_builder.py: fix broken stubs, add render_order_line_still_task
  (resolves step_path from DB instead of passing order_line_id as step_path)
- domains/rendering/tasks.py: add render_order_line_still_task,
  export_gltf_for_order_line_task, export_blend_for_order_line_task,
  generate_gltf_geometry_task (trimesh STL→GLB, no Blender needed)
- tasks/step_tasks.py: add generate_gltf_geometry_task for CadFile GLB export
- cad router: POST /{id}/generate-gltf-geometry endpoint (admin/PM)
- worker router: GET /celery-workers + POST /scale (docker compose subprocess)
- Dockerfile: pip install -e "[dev]" to enable pytest
- docker-compose.yml: docker socket + compose file mount on backend
- ThreeDViewer.tsx: mode toggle (geometry/production), wireframe, env presets,
  download buttons (GLB + .blend)
- CadPreview.tsx: load gltf_geometry/gltf_production/blend_production assets
  from MediaAsset table and pass URLs to ThreeDViewer
- ProductDetail.tsx: "View 3D" button → /cad/:id, "Generate GLB" button
- media router/service: cad_file_id filter on GET /api/media
- WorkerManagement.tsx: new page with worker status, queue depth, scale controls
- App.tsx + Layout.tsx: /workers route + sidebar link (admin/PM)
- tests: test_rendering_service.py, test_orders_service.py (backend)
- tests: WorkerActivity.test.tsx, WorkerManagement.test.tsx (frontend)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 22:56:53 +01:00
Hartmut ced64055f2 refactor(admin): remove Pillow as selectable renderer, restructure admin page 
- Backend: VALID_RENDERERS = {"blender"} only; remove pillow from renderer-status response
- Frontend: remove renderer picker (pillow/blender buttons) — Blender is the only renderer
- Blender options always visible, grouped into Render Quality / Performance / Output sections
- Maintenance buttons in 2-column grid with descriptions
- Page reorder: Pricing Summary → Users → Blender Settings → Render Templates →
  Asset Libraries → Output Types → Pricing Tiers → SMTP → Templates
- Pillow code kept internally as fallback (not exposed in UI)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 21:20:28 +01:00
Hartmut 3ac3ca1d70 fix(K): correct CATALOG_SCRIPT path + register AssetLibrary in models/__init__ 
- fix(tasks): use RENDER_SCRIPTS_DIR env var for catalog_assets.py path
  (was computing wrong path via __file__ parents → /render-worker/scripts/ which doesn't exist in container)
- fix(models): add AssetLibrary to app/models/__init__.py so alembic autogenerate discovers it
- fix(api): remove unused FileResponse import from asset_libraries.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 21:02:41 +01:00
Hartmut a18d4c23ec feat(K): Blender Asset Library + production exports (GLB + .blend) 
- feat(migration): 045_asset_libraries — new asset_libraries table (blend_file_path, catalog JSONB)
- feat(model): AssetLibrary SQLAlchemy model in domains/materials/models.py
- feat(api): POST/GET/PATCH/DELETE /api/asset-libraries + /upload-blend + /refresh-catalog endpoints
- feat(celery): refresh_asset_library_catalog task on thumbnail_rendering queue — runs Blender headless
- feat(blender): catalog_assets.py — extracts asset-marked materials + node_groups from .blend
- feat(blender): asset_library.py — apply_asset_library_materials + apply_asset_library_node_groups helpers
- feat(blender): export_gltf.py — STEP→STL→GLB production export with optional asset library
- feat(blender): export_blend.py — STEP→STL→.blend production export with pack_all()
- feat(frontend): api/assetLibraries.ts — full CRUD API client
- feat(frontend): AssetLibraryPanel in Admin.tsx — upload, refresh, expand catalog view
- docs: Blender asset_data marking requirement learning in LEARNINGS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:56:26 +01:00
Hartmut 7a1329958d feat(J): WebSocket live-events + replace polling + fix ffmpeg turntable timeout 
- fix(render): ffmpeg overlay=0:0 -> overlay=0:0:shortest=1 to prevent hang on finite PNG sequences
- feat(ws): add core/websocket.py ConnectionManager + Redis Pub/Sub subscriber loop
- feat(ws): add /api/ws WebSocket endpoint with JWT query-param auth in main.py
- feat(ws): emit render_complete/failed + cad_processing_complete events from step_tasks.py
- feat(ws): emit order_status_change events from orders router
- feat(ws): add beat_tasks.py broadcast_queue_status task (every 10s via Redis __broadcast__)
- feat(frontend): add useWebSocket hook with auto-reconnect (exponential backoff, 25s ping)
- feat(frontend): add WebSocketContext + WebSocketProvider wrapping App
- refactor(frontend): remove polling from WorkerActivity (was 5s/3s) + OrderDetail (was 5s)
- refactor(frontend): reduce polling in Layout (8s->60s) + NotificationCenter (15s->60s)
- docs: add ffmpeg shortest=1 + WebSocket JWT auth learnings to LEARNINGS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:49:34 +01:00
Hartmut d138bc4bc4 feat(activity): merge CAD + render jobs into unified timeline 
- Single chronological list sorted by updated_at (newest first)
- Type badges distinguish CAD processing (FileCode2) from Render jobs (Image)
- Render job rows now link directly to the order (/orders/:id)
- Remove separate "Render Jobs" and "CAD File Processing" sections
- Stat cards simplified to 4: CAD active/failed + Render active/failed
- Backend: add order_id to RenderJobEntry response
- Frontend: add order_id to RenderJobEntry interface, remove flamenco_job_id

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:01:03 +01:00
Hartmut ab3f9c734a fix: render pipeline + multi-tenancy bugs (B-Fix-1 through B-Fix-9) 
- Remove worker-thumbnail (no Blender, was competing on thumbnail_rendering)
- Move render_order_line_task to thumbnail_rendering queue (render-worker)
- Restore template_service.py real implementation (fix circular import shim)
- Thread tenant_id through STEP upload, Excel import, product create
- Make system tables (output_types, materials, etc.) tenant_id nullable
- Fix tenants frontend 307-redirect: use trailing slash /tenants/
- Remove Flamenco + Three.js from Admin UI (unsupported)
- Set all output_types render_backend to celery (was flamenco)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 19:34:20 +01:00
Hartmut 381f44bc8b feat: render health endpoint + test script + pipeline fixes 
- GET /api/worker/health/render: checks render-worker (thumbnail_rendering
  queue), Blender availability via active_queues inspect, queue depth,
  last render recency — returns ok/degraded/down status
- scripts/test_render_pipeline.py: integration test for full pipeline
  (--health, --sample, --full modes)
- PLAN.md: appended Render Pipeline Fixes section with all B-Fixes
- LEARNINGS.md: documented 5 new learnings (queue mismatch, circular
  import, 307 redirect, worker capability detection)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 19:34:12 +01:00
Hartmut c0ea60d984 fix: resolve open risks — invoice race condition, SMTP config, workflow seeds 
- billing/service.py: pg_advisory_xact_lock on invoice_number_seq per year
  → prevents duplicate INV-YYYY-NNNN under concurrent requests
- admin.py: SMTP settings in system_settings (smtp_host/port/user/password/
  from_address/enabled) with GET+PUT support; seed-workflows endpoint creates
  4 standard workflow definitions (still-cycles, still-eevee, turntable,
  multi-angle) idempotently
- notifications/service.py: send_email_notification_stub now sends real
  SMTP email via smtplib when smtp_enabled=true in system_settings
- Admin.tsx: SMTP settings panel (host/port/user/password/from + enable
  toggle, save button); Seed Standard Workflows maintenance button
- Upload.tsx: fix TS error — title→aria-label on Lucide icons
- Admin.tsx Settings type: add render_backend/flamenco_* fields (TS fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 18:15:45 +01:00
Hartmut f19a6ccde8 feat(F-G-H-I): STL cache, invoices, import validation, notification settings 
Phase F — STL Hash Cache:
- Migration 041: step_file_hash column on cad_files
- cache_service.py: SHA256 hash + MinIO-backed STL cache (check/store)
- render_step_thumbnail: compute+persist hash before render
- generate_stl_cache: check MinIO cache before cadquery conversion, store after

Phase G — Invoices:
- Migration 042: invoices + invoice_lines tables with RLS
- Invoice/InvoiceLine models + schemas
- billing service: generate_invoice_number (INV-YYYY-NNNN), create/list/get/delete/PDF
- WeasyPrint PDF generation; backend Dockerfile + pyproject.toml deps
- invoice_router with 6 endpoints; registered in main.py
- frontend: Billing.tsx page + api/billing.ts; route + nav link

Phase H — Import Sanity Check:
- Migration 043: import_validations table
- ImportValidation model + schemas
- run_sanity_check: material fuzzy-match (cutoff=0.8), STEP availability, duplicate detection
- validate_excel_import Celery task (queue: step_processing)
- uploads.py: create ImportValidation on /excel, fire task, expose GET /validations/{id}
- frontend: Upload.tsx polling ValidationDialog with Ampel status indicators

Phase I — Notification Settings:
- Migration 044: notification_configs table (user×event×channel toggles)
- NotificationConfig model + seeds (in_app=true, email=false)
- get/upsert/reset config endpoints on /notifications/config
- frontend: NotificationSettings.tsx page + api/notifications.ts extensions

Infrastructure:
- docker-compose.yml: add worker-thumbnail service (concurrency=1, Q=thumbnail_rendering)
- Fix Dockerfile: libgdk-pixbuf-2.0-0 (correct Debian bookworm package name)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 18:05:01 +01:00
Hartmut 9d1a820295 refactor(A2): replace blender-renderer HTTP service with render-worker Celery container 
- Create render-worker/ with Dockerfile (Ubuntu + cadquery + Blender via host mount)
- Add render-worker/check_version.py: verifies Blender >= 5.0.1 at startup, Exit 1 on failure
- Add render-worker/scripts/: blender_render.py, still_render.py, turntable_render.py
- Create backend/app/services/render_blender.py: direct subprocess rendering
  - convert_step_to_stl() and export_per_part_stls() using cadquery
  - render_still(): STEP → STL → PNG via Blender subprocess
  - is_blender_available(): detects BLENDER_BIN env for render-worker context
- Create backend/app/domains/rendering/tasks.py: render_still_task + render_turntable_task
- Update step_processor.py: use subprocess path when BLENDER_BIN env is set (render-worker)
- Update step_tasks.py: generate_stl_cache uses direct cadquery instead of HTTP
- Remove blender-renderer and threejs-renderer from docker-compose.yml
- Replace worker-thumbnail with render-worker (Ubuntu + cadquery + Blender mount)
- Remove Docker SDK from backend Dockerfile (was only for flamenco scaling)
- Update .env.example: BLENDER_VERSION=5.0.1 documented
- Update celery_app.py: include domains.rendering.tasks in autodiscover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:48:46 +01:00
Hartmut 1d6864fb64 refactor(A1): remove Flamenco, simplify render pipeline to Celery-only 
- Remove flamenco-manager and flamenco-worker from docker-compose.yml
- Delete flamenco_client.py, flamenco_tasks.py, docker_scaler.py
- Simplify render_dispatcher.py to Celery-only (removes ~300 lines)
- Remove Flamenco beat schedule from celery_app.py
- Clean admin.py: remove flamenco settings, endpoints, threejs validation
- Clean orders.py cancel-render: Celery revoke only
- Clean worker.py: remove flamenco_job_id from activity response
- Migration 032: cancel lingering flamenco jobs, remove flamenco settings
- PLAN.md: mark all decisions confirmed, status IN UMSETZUNG

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:38:37 +01:00
Hartmut 552922eb8a chore: remove build artifacts from tracking, extend .gitignore 2026-03-05 22:13:21 +01:00
Hartmut bce762a783 feat: initial commit 2026-03-05 22:12:38 +01:00