Replaced communicate() (blocking) with selectors-based line-by-line
stdout streaming — same pattern as still render. Each frame now
streams live to the frontend:
[cinematic_render] Frame 42/480 -- 55.3s elapsed (0.76 fps)
Pipeline: Blender stdout → log_callback → emit() → Redis →
LiveRenderLog poll (2s) → frontend display
Also added log_callback parameter to cinematic render task call.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fixed: from OCP.BRepGProp import BRepGProp as brepgprop (was lowercase)
- Routed reextract_rich_metadata_task to asset_pipeline queue (render-worker
has OCC/OCP installed, worker container does not)
- Backfill verified: 45/45 products updated with volume, surface area,
part count, complexity metrics
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: render-worker and render-worker-light shared the same GPU,
causing contention. Complex TRB renders went from 17s → 36s (2x slower).
Changes:
- Thumbnails back to asset_pipeline queue (not asset_pipeline_light)
- Dispatch routing always uses asset_pipeline (no queue splitting)
- render-worker-light gated behind "multi-gpu" profile — only starts with:
docker compose --profile multi-gpu up -d
- For single-GPU setups: all rendering is sequential on one worker
The dual queue approach is correct for multi-GPU machines where each
worker gets its own GPU. On single-GPU, serial execution is faster
than concurrent GPU contention.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instead of duplicating output types for every variation (WebP vs PNG,
different resolution), keep one canonical output type and override
specific fields per order line via render_overrides JSONB.
Backend:
- render_overrides JSONB column on OrderLine (DB migration)
- Render task merges overrides with output type settings (format, width,
height, samples, engine, denoiser, transparent_bg, cycles_device)
- POST /orders/{id}/batch-render-overrides endpoint for bulk override
- PatchLineBody accepts render_overrides for per-line patching
Frontend:
- Batch render overrides section on OrderDetail: output format dropdown
(PNG/JPG/WebP) + resolution dropdown (512-4096)
- Clear button to remove overrides
MCP:
- create_order tool: accepts product_ids, output_type, render_overrides,
material_override — enables "render all products as WebP" via Claude
- set_render_overrides tool: batch override on existing orders
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Task 4: Dual render queue
- render-worker: heavy (asset_pipeline, concurrency=1) — HQ 2048x2048, animations
- render-worker-light: light (asset_pipeline_light, concurrency=2) — thumbnails, <=1024
- Thumbnails routed to light queue automatically
- Order line renders routed by resolution at dispatch time
Task 5: GLB caching (skip re-tessellation)
- Before tessellating, check if gltf_geometry MediaAsset exists for the cad_file_id
- If found, copy to expected path — render_blender.py finds it and skips tessellation
- Saves 7-11s per re-render of the same product
Task 6: WebP output format
- New 'webp' option in output_format (OutputType admin)
- Blender renders PNG intermediate, Pillow converts to WebP (quality=90, method=4)
- 50-70% smaller files with no visible quality loss
- Correct MIME type (image/webp) in MediaAsset
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `material_override` nullable column on OrderLine (DB migration)
- Line override takes priority over OutputType override
- PATCH /orders/{id}/lines/{id} endpoint to update material_override
- Inline dropdown on each order line in the OrderDetail page
- Amber background when override is active
- Same output type, different material per line — no need to create a new output type
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The initial implementation only overrode the material_map dict in the task,
but the Blender USD primvar path bypassed it. Now:
- Added --material-override named CLI arg parsed in _blender_args.py
- Both Mode A (factory) and Mode B (template) in _blender_scene_setup.py
override usd_material_lookup and material_map when set
- Passed through full chain: task → step_processor → render_blender → CLI → Blender
- Tested: 175-part bearing rendered with single Steel-Bare material (1/1 materials)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `material_override` nullable column on OutputType (DB migration)
- When set, ALL product parts get rendered with this single material
- Override applies after alias resolution in render_order_line task
- Admin UI: dropdown in OutputType table to select a library material
- Display: amber badge showing active override material name
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Excel preview detects when a product already has a different STEP file linked
- Excel preview detects intra-Excel conflicts (same product, different CAD model names)
- Product STEP upload warns when replacing an existing file and shows render count
- All warnings are non-blocking (amber badges, toast warnings)
- LEARNINGS.md: all open items resolved
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Per-render-position focal_length_mm/sensor_width_mm (DB → pipeline → Blender)
- FOV-based camera distance with min clamp fix for wide-angle lenses
- Unmapped materials blocking dialog on "Dispatch Renders" with batch alias creation
- Material check endpoint (GET /orders/{id}/check-materials)
- Batch alias endpoint (POST /materials/batch-aliases)
- Quick-map "No alias" badges on Materials page
- Full product hard-delete with storage cleanup (MinIO + disk files + orphaned CadFile)
- Delete button on ProductDetail page with confirmation
- Clickable product names in Media Browser (links to product page)
- Single-line render dispatch/retry (POST /orders/{id}/lines/{id}/dispatch-render)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause 1: BRepBuilderAPI_Transform(shape, trsf, copy=True) destroys
all Poly_Triangulation data. The mm→m scaling was applied before export,
wiping the tessellation from BRepMesh_IncrementalMesh.
Fix: Remove BRepBuilderAPI_Transform entirely — RWGltf_CafWriter already
handles mm→m conversion and Z-up→Y-up rotation internally.
Root cause 2: RWGltf_CafWriter with MergeFaces=False (the default) fails
to find per-face tessellation from the XCAF component hierarchy, producing
degenerate meshes (~2 vertices per face instead of thousands).
Fix: SetMergeFaces(True) to compose face triangulations into proper
per-shape mesh buffers. Vertex count goes from 1,212 to 46,573.
Also bumps cache key version to v2 to invalidate broken cached GLBs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds early-exit checks in dispatch_order_line_render and
render_order_line_task to prevent rendering when order lines are
cancelled or the parent order is rejected/completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- @timed_step decorator with wall-clock + RSS tracking (pipeline_logger)
- Blender timing laps for sharp edges and material assignment
- MeshRegistry pattern: eliminate 13 scene.traverse() calls across viewers
- Lazy material cloning (clone-on-first-write in both viewers)
- _pipeline_session context manager: 7 create_engine() → 2 in render_thumbnail
- KD-tree spatial pre-filter for sharp edge marking (bbox-based pruning)
- Batch material library append: N bpy.ops.wm.append → single bpy.data.libraries.load
- GMSH single-session batching: compound all solids into one tessellation call
- Validate part-materials save endpoints against parsed_objects (prevents bogus keys)
- ROADMAP updated with completion status
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- export_step_to_usd.py: change stage from Y-up to Z-up, keep (X,-Z,Y)
transform — matches GLB orientation exactly (verified: bounding box match)
- export_glb.py: include canonical_material in resolved_material_assignments
DB field (was being dropped during manifest parsing)
- import_usd.py: use pxr customData read (not string primvars — Blender
ignores those, confirmed by testing)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- export_step_to_usd.py: accept --material_map CLI arg, write
schaeffler:canonicalMaterialName as customData on each Mesh prim,
fix geometry transform (strip shape Location before face exploration,
apply both face_loc and shape_loc sequentially)
- import_usd.py: after Blender USD import, use pxr to read customData
directly from the USD file — builds {part_key: material_name} lookup
(Blender ignores STRING primvars and customData, but pxr reads both)
- _blender_materials.py: add apply_material_library_direct() for exact
dict-based material assignment without name-matching heuristics
- _blender_scene_setup.py: prefer direct USD lookup, fall back to
name-matching for legacy USD files without material metadata
- export_glb.py (generate_usd_master_task): resolve material_map via
material_service.resolve_material_map() and pass to subprocess;
include material hash in cache key for invalidation
- ROADMAP.md: update P5 status, add M5-M7 milestones
Tested: 3/3 parts matched (ans_lfs120), 172/175 parts matched
(F-802007.TR4-D1-H122AG). Previous: 0/25 matched.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The queue handles far more than thumbnails: OCC tessellation, USD master
generation, GLB production, order line renders, and workflow renders.
asset_pipeline better reflects its role as the render-worker's primary queue.
Updated all references in: task decorators, celery_app.py, beat_tasks.py,
docker-compose.yml worker command, worker.py MONITORED_QUEUES, admin.py,
CLAUDE.md, LEARNINGS.md, Dockerfile, helpTexts.ts, test files,
and all .claude/commands/*.md skill files.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add per-file Redis SET NX EX 1800 locks to generate_gltf_geometry_task
and generate_usd_master_task — concurrent duplicates (e.g. double-click
of bulk action buttons) now log a warning and return immediately instead
of running two expensive OCC tessellation subprocesses on the same file
- Fix eng.dispose() called inside with Session() block in cache-hit path
of both tasks — moved to after the with block exits (Tasks 3+4 from plan)
- Add cad.updated_at = datetime.utcnow() in save_manual_material_overrides
(was missing vs parallel save_part_materials endpoint)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fine production settings (0.03mm/0.05rad) with GMSH → CharacteristicLengthMax
0.45mm → OOM kill on large assemblies even with per-solid iteration.
GMSH quality is algorithmic (conforming seams, no fan triangles) — a denser
mesh provides no extra UV-unwrap benefit. Cap GMSH tessellation at preview
settings (0.1mm/0.1rad) in the production fallback path.
Normal path (GMSH + valid geometry GLB) is unaffected — continues to reuse
_geometry.glb directly without re-tessellating.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Per-solid iteration prevents OOM on multi-part assemblies (25-part bearing:
2.3GB RAM when processing compound → ~100MB per solid with per-solid approach)
- Fix CharacteristicLengthMax multiplier 5× → 15× and cap MinimumCirclePoints
at 20 (prevents 63-pts/circle on angular_deflection=0.1rad → 231MB → 21MB)
- Geometry task timeout 120s → 600s for large assemblies
- Production task: reuse _geometry.glb when GMSH enabled (no re-tessellation)
and cache _production_geom.glb for OCC (mtime vs STEP check)
- Viewer now prefers production GLB when available (shows correct GMSH mesh)
- GMSH OpenMP multithreading (min(cpu_count,16)) for 4.4× speedup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces GMSH as an alternative to OCC BRepMesh for STEP→GLB tessellation.
GMSH produces conforming meshes that eliminate fan triangles at cylinder seam
edges — a structural limitation of OCC BRepMesh that cannot be fixed via
deflection parameters.
Changes:
- render-worker/Dockerfile: install gmsh>=4.15.0 + libglu1-mesa + libxft2
- export_step_to_gltf.py: --tessellation_engine occ|gmsh CLI arg +
_tessellate_with_gmsh() using BRep→GMSH→Poly_Triangulation write-back
- admin.py: tessellation_engine setting (SETTINGS_DEFAULTS, SettingsOut,
SettingsUpdate, validation)
- export_glb.py: pass tessellation_engine to export_step_to_gltf.py CLI in
both geometry and production GLB tasks
- Admin.tsx: radio button UI for OCC vs GMSH selection
Tested: 121 faces meshed, 0 BRepMesh fallback, 649K triangles on sample part.
Clean seam edges for UV unwrap — GMSH respects B-rep periodic face boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When re-generating a production or geometry GLB, the old approach deleted the
existing MediaAsset record and created a new one with a new UUID. Any page that
had the old download_url (/api/media/{old-id}/download) cached would then get
a 404 when trying to download, because the asset ID no longer existed in the DB.
Fix: update the existing MediaAsset record in-place (same UUID, new storage_key)
so existing download URLs remain valid after regeneration. Create a new record
only if no existing one is found.
Applies to both generate_gltf_geometry_task and generate_gltf_production_task.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Per-tenant Azure AI config stored in tenants.tenant_config JSONB
- GET/PUT /api/tenants/{id}/ai-config + POST .../test connection
- api_key never returned to frontend (has_api_key: bool pattern)
- azure_ai.py resolves creds from tenant config when ai_enabled=True
- ai_tasks.py loads tenant config and passes it to validate_thumbnail
- Admin GPU Status section: probe button + status badge + last-checked time
- Notifications: _BELL_CHANNELS filter (notification+alert only in bell)
- Tenants.tsx: per-row Azure AI Config modal with URL auto-parse helper
- Remove duplicate in-memory /gpu-probe endpoints (kept DB-backed /probe/gpu)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/workflows/pipeline-steps: returns all StepName enum values
with category (input|processing|rendering|output) + descriptions;
registered before /{workflow_id} to avoid path collision
- frontend/src/api/workflows.ts: getPipelineSteps(), PipelineStep
and PipelineStepsResponse interfaces
- WorkflowEditor: PipelineStepsPanel showing steps grouped by category
with collapsible accordion sections
- ConfigSidepanel: "Pipeline Step" select dropdown binds any node to a
StepName; selected step description shown below dropdown
- Active workflow indicator: green dot next to is_active=true entries
- Improved empty state with descriptive copy
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 5.1 — MATERIAL_PALETTE removal:
- Remove MATERIAL_PALETTE + _material_to_color() from step_processor.py
- build_part_colors() now returns {part→material_name} for Blender resolver
Phase 6 — Notification Center Refactor:
- Migration 051: add channel (activity|notification|alert) to audit_log,
add frequency (immediate|daily|never) to notification_configs
- Three notification channels: activity (per-render), notification (batch
order summaries), alert (admin infrastructure)
- Per-render emit_notification_sync calls demoted to channel=activity
- New emit_batch_render_notification_sync(): single summary notification
when all order lines reach terminal state ("47/50 succeeded, 3 failed")
- Beat task batch_render_notifications every 60s: safety-net for missed
batch notifications after order completion
- GET /notifications: defaults to channel IN (notification, alert);
accepts ?channel=activity for activity feed
- Unread count badge counts only notification+alert channels
- Notifications.tsx: three tabs (Notifications | Activity | Alerts)
- NotificationSettings.tsx: frequency dropdown per event type
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 2.3 — Fix render cancellation (real Celery task ID):
- orders.py cancel endpoints: read celery_task_id from render_job_doc
instead of synthetic "render-{line_id}" which was a no-op
- render_order_line_still_task: creates RenderJobDocument at task start,
stores self.request.id as celery_task_id, writes step-level state
(RESOLVE_STEP_PATH → BLENDER_STILL) back to order_lines.render_job_doc
Phase 3.1 — Remove Pillow overlay dead code:
- blender_render.py: deleted 55-line Pillow post-processing block
(lines 798-851, green bar + model name label)
- transparent_bg=True is always passed; the else branch was unreachable
- Removed mention from script docstring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 1 of PLAN_REFACTOR.md — all four sub-tasks implemented:
1.1 PipelineLogger (backend/app/core/pipeline_logger.py)
- Structured step_start/step_done/step_error/step_progress API
- Publishes to Python logging AND Redis SSE via log_task_event
- Context manager `pl.step("name")` for auto-timing
1.2 RenderJobDocument (backend/app/domains/rendering/job_document.py)
- Pydantic JSONB schema: state machine + per-step records + timing
- begin_step/finish_step/fail_step/skip_step helpers
- Migration 048: adds render_job_doc JSONB column to order_lines
- OrderLine model updated with render_job_doc field
1.3 TenantContextMiddleware (backend/app/core/middleware.py)
- Decodes JWT, stores tenant_id + role in request.state
- get_db updated to auto-apply RLS SET LOCAL from request.state
- Registered in main.py (runs before every request)
- JWT now embeds tenant_id claim via create_access_token()
- Login endpoint passes tenant_id to token creation
1.4 ProcessStep Registry (backend/app/core/process_steps.py)
- StepName StrEnum with all 20 pipeline step names
- Single source of truth for log prefixes, DB records, UI labels
Also adds db_utils.py with set_tenant_sync() + get_sync_session()
for use inside Celery tasks (bypass-safe RLS helper).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GPU: fix Cycles device activation order — set compute_device_type
BEFORE engine init, re-set AFTER open_mainfile wipes preferences
- GPU: remove _mark_sharp_and_seams edit-mode loop (redundant with
Blender 5.0 shade_smooth_by_angle), saves ~200s/render on 175 parts
- Material: fix _AFN suffix mismatch — build AF-stripped mat_map keys
and add prefix fallback in _apply_material_library (blender_render.py)
- Material: production GLB now uses get_material_library_path() which
checks active AssetLibrary instead of empty legacy system setting
- Admin: RenderTemplateTable multi-select output types (M2M frontend)
- Admin: MaterialLibraryPanel replaced with link to Asset Libraries
- UX: move Toaster to top-left to avoid dispatch button overlap
- SQLAlchemy: add .unique() to all RenderTemplate M2M collection queries
- Logging: flush=True on all Blender progress prints, stdout reconfigure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Bug A: Media Library thumbnails were gray because <img src> cannot send
JWT auth headers. Added useAuthBlob() hook (fetch + createObjectURL) in
MediaBrowser.tsx. Also fixed publish_asset Celery task to populate
product_id + cad_file_id on MediaAsset for thumbnail fallback resolution.
Bug B: Product dimensions now shown in Product Details card with Ruler
icon and "from CAD" label when cad_mesh_attributes.dimensions_mm exists.
Bug C: Replaced 128×128 CAD thumbnail with InlineCadViewer component.
Queries gltf_geometry MediaAssets, fetches GLB via auth fetch → blob URL
→ Three.js Canvas with OrbitControls. Falls back to thumbnail + "Load 3D
Model" button. Polling when GLB generation is in progress.
Bug D: trimesh was in [cad] optional extra but Dockerfile only installed
[dev]. Changed to pip install -e ".[dev,cad]" — trimesh now available in
backend container, GLB + Colors export works.
Also added bbox extraction (STL-first numpy parsing) in render_step_thumbnail
and admin "Re-extract CAD Metadata" bulk endpoint.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the cache_service was only used in the generate_stl_cache Celery task.
All render paths (render_still, render_turntable_to_file, render_turntable_task)
only checked for a local file and converted from scratch if missing.
Changes:
- render_blender.py: add _stl_from_cache_or_convert() helper that checks MinIO
cache before falling back to local STEP→STL conversion. Wire into render_still()
and render_turntable_to_file() (both STL conversion blocks).
- domains/rendering/tasks.py: wire MinIO cache check into render_turntable_task()
inline before convert_step_to_stl(). All errors are non-fatal (falls back to
fresh conversion).
Now a STEP file converted on one worker is available to all workers via MinIO,
avoiding redundant cadquery conversions on re-renders.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Hartmut