Task 4: Dual render queue
- render-worker: heavy (asset_pipeline, concurrency=1) — HQ 2048x2048, animations
- render-worker-light: light (asset_pipeline_light, concurrency=2) — thumbnails, <=1024
- Thumbnails routed to light queue automatically
- Order line renders routed by resolution at dispatch time
Task 5: GLB caching (skip re-tessellation)
- Before tessellating, check if gltf_geometry MediaAsset exists for the cad_file_id
- If found, copy to expected path — render_blender.py finds it and skips tessellation
- Saves 7-11s per re-render of the same product
Task 6: WebP output format
- New 'webp' option in output_format (OutputType admin)
- Blender renders PNG intermediate, Pillow converts to WebP (quality=90, method=4)
- 50-70% smaller files with no visible quality loss
- Correct MIME type (image/webp) in MediaAsset
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removed from UI (saved to DB but never read by any service):
- Max Concurrent Renders, Stall Timeout, Thumbnail Format, Product Thumbnail Priority
- Render Linear/Angular Deflection (only Scene deflections are used)
- GLB Scale Factor, Smooth Normals, GLB Material Mode, PBR Roughness, PBR Metallic
Fixed Blender status check:
- Old: called is_blender_available() in backend container (Blender not installed there)
- New: dispatches Celery task on asset_pipeline queue → runs in render-worker container
- Returns: available=true, version="Blender 5.0.1", binary path
- Status card moved to System Tools tab with refresh button
Kept active: engine, device, samples, smooth angle, tessellation, scene deflections,
3D viewer zoom limits
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
create_order and add_order_line endpoints were not passing
material_override from the request body to the OrderLine constructor.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- POST /orders/{id}/batch-material-override endpoint
- Dropdown above the lines table: "Apply to all lines…"
- Options: clear all overrides, or select a library material
- Updates all order lines in one request
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `material_override` nullable column on OrderLine (DB migration)
- Line override takes priority over OutputType override
- PATCH /orders/{id}/lines/{id} endpoint to update material_override
- Inline dropdown on each order line in the OrderDetail page
- Amber background when override is active
- Same output type, different material per line — no need to create a new output type
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The initial implementation only overrode the material_map dict in the task,
but the Blender USD primvar path bypassed it. Now:
- Added --material-override named CLI arg parsed in _blender_args.py
- Both Mode A (factory) and Mode B (template) in _blender_scene_setup.py
override usd_material_lookup and material_map when set
- Passed through full chain: task → step_processor → render_blender → CLI → Blender
- Tested: 175-part bearing rendered with single Steel-Bare material (1/1 materials)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add `material_override` nullable column on OutputType (DB migration)
- When set, ALL product parts get rendered with this single material
- Override applies after alias resolution in render_order_line task
- Admin UI: dropdown in OutputType table to select a library material
- Display: amber badge showing active override material name
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Excel preview detects when a product already has a different STEP file linked
- Excel preview detects intra-Excel conflicts (same product, different CAD model names)
- Product STEP upload warns when replacing an existing file and shows render count
- All warnings are non-blocking (amber badges, toast warnings)
- LEARNINGS.md: all open items resolved
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Per-render-position focal_length_mm/sensor_width_mm (DB → pipeline → Blender)
- FOV-based camera distance with min clamp fix for wide-angle lenses
- Unmapped materials blocking dialog on "Dispatch Renders" with batch alias creation
- Material check endpoint (GET /orders/{id}/check-materials)
- Batch alias endpoint (POST /materials/batch-aliases)
- Quick-map "No alias" badges on Materials page
- Full product hard-delete with storage cleanup (MinIO + disk files + orphaned CadFile)
- Delete button on ProductDetail page with confirmation
- Clickable product names in Media Browser (links to product page)
- Single-line render dispatch/retry (POST /orders/{id}/lines/{id}/dispatch-render)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause 1: BRepBuilderAPI_Transform(shape, trsf, copy=True) destroys
all Poly_Triangulation data. The mm→m scaling was applied before export,
wiping the tessellation from BRepMesh_IncrementalMesh.
Fix: Remove BRepBuilderAPI_Transform entirely — RWGltf_CafWriter already
handles mm→m conversion and Z-up→Y-up rotation internally.
Root cause 2: RWGltf_CafWriter with MergeFaces=False (the default) fails
to find per-face tessellation from the XCAF component hierarchy, producing
degenerate meshes (~2 vertices per face instead of thousands).
Fix: SetMergeFaces(True) to compose face triangulations into proper
per-shape mesh buffers. Vertex count goes from 1,212 to 46,573.
Also bumps cache key version to v2 to invalidate broken cached GLBs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds early-exit checks in dispatch_order_line_render and
render_order_line_task to prevent rendering when order lines are
cancelled or the parent order is rejected/completed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New endpoint POST /admin/settings/regenerate-all-canonical-scenes
queues GLB + USD master export for ALL completed CAD files, replacing
existing assets. Used after pipeline changes that affect tessellation
or normals.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- @timed_step decorator with wall-clock + RSS tracking (pipeline_logger)
- Blender timing laps for sharp edges and material assignment
- MeshRegistry pattern: eliminate 13 scene.traverse() calls across viewers
- Lazy material cloning (clone-on-first-write in both viewers)
- _pipeline_session context manager: 7 create_engine() → 2 in render_thumbnail
- KD-tree spatial pre-filter for sharp edge marking (bbox-based pruning)
- Batch material library append: N bpy.ops.wm.append → single bpy.data.libraries.load
- GMSH single-session batching: compound all solids into one tessellation call
- Validate part-materials save endpoints against parsed_objects (prevents bogus keys)
- ROADMAP updated with completion status
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract Base Color, Metallic, Roughness, Transmission, IOR from Blender
asset library materials via catalog_assets.py. Store in catalog JSON and
serve via /api/asset-libraries/pbr-map endpoint. Frontend viewers apply
PBR properties to Three.js MeshStandardMaterial using hex color strings
(avoiding Three.js ColorManagement sRGB/linear issues).
Key fixes:
- RLS bypass for material alias lookup in pbr-map endpoint
- pbrMap empty guard prevents premature grey fallback in viewers
- Cache-Control: no-cache on pbr-map requests to avoid stale data
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- export_step_to_usd.py: change stage from Y-up to Z-up, keep (X,-Z,Y)
transform — matches GLB orientation exactly (verified: bounding box match)
- export_glb.py: include canonical_material in resolved_material_assignments
DB field (was being dropped during manifest parsing)
- import_usd.py: use pxr customData read (not string primvars — Blender
ignores those, confirmed by testing)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- export_step_to_usd.py: accept --material_map CLI arg, write
schaeffler:canonicalMaterialName as customData on each Mesh prim,
fix geometry transform (strip shape Location before face exploration,
apply both face_loc and shape_loc sequentially)
- import_usd.py: after Blender USD import, use pxr to read customData
directly from the USD file — builds {part_key: material_name} lookup
(Blender ignores STRING primvars and customData, but pxr reads both)
- _blender_materials.py: add apply_material_library_direct() for exact
dict-based material assignment without name-matching heuristics
- _blender_scene_setup.py: prefer direct USD lookup, fall back to
name-matching for legacy USD files without material metadata
- export_glb.py (generate_usd_master_task): resolve material_map via
material_service.resolve_material_map() and pass to subprocess;
include material hash in cache key for invalidation
- ROADMAP.md: update P5 status, add M5-M7 milestones
Tested: 3/3 parts matched (ans_lfs120), 172/175 parts matched
(F-802007.TR4-D1-H122AG). Previous: 0/25 matched.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The queue handles far more than thumbnails: OCC tessellation, USD master
generation, GLB production, order line renders, and workflow renders.
asset_pipeline better reflects its role as the render-worker's primary queue.
Updated all references in: task decorators, celery_app.py, beat_tasks.py,
docker-compose.yml worker command, worker.py MONITORED_QUEUES, admin.py,
CLAUDE.md, LEARNINGS.md, Dockerfile, helpTexts.ts, test files,
and all .claude/commands/*.md skill files.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously the endpoint queued USD generation for ALL 295 completed CadFiles,
including 250 orphan CadFiles not linked to any product. Now filters to only
CadFiles referenced by at least one Product.cad_file_id, reducing the backfill
from ~285 to ~41 tasks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- beat_tasks.py: import app.models at module level so SQLAlchemy can
resolve relationship("Template") and relationship("User") when domain
models are imported in isolation inside task functions. Fixes all
beat tasks (batch_render_notifications, recover_stuck_cad_files) that
crashed every 60s with mapper initialization error.
- _blender_materials.py: build_mat_map_lower() now adds a slug-normalized
key variant (re.sub([^a-z0-9]+, _, kl)) for each mat_map entry. OCC
part names like 'F-802007_TR4-D1-H122AG' → slug 'f_802007_tr4_d1_h122ag'
now matches USD-imported Blender objects. Existing prefix fallback
(key.startswith(part_key)) catches AF-suffix variants.
- Orders.tsx: kanban drag-to-reject implemented. submitted/processing
cards are draggable (cursor-grab). Rejected column highlights with
red ring on drag-over. Drop opens reject reason modal via createPortal.
Confirm calls rejectOrder() mutation + invalidates orders cache.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add per-file Redis SET NX EX 1800 locks to generate_gltf_geometry_task
and generate_usd_master_task — concurrent duplicates (e.g. double-click
of bulk action buttons) now log a warning and return immediately instead
of running two expensive OCC tessellation subprocesses on the same file
- Fix eng.dispose() called inside with Session() block in cache-hit path
of both tasks — moved to after the with block exits (Tasks 3+4 from plan)
- Add cad.updated_at = datetime.utcnow() in save_manual_material_overrides
(was missing vs parallel save_part_materials endpoint)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fine production settings (0.03mm/0.05rad) with GMSH → CharacteristicLengthMax
0.45mm → OOM kill on large assemblies even with per-solid iteration.
GMSH quality is algorithmic (conforming seams, no fan triangles) — a denser
mesh provides no extra UV-unwrap benefit. Cap GMSH tessellation at preview
settings (0.1mm/0.1rad) in the production fallback path.
Normal path (GMSH + valid geometry GLB) is unaffected — continues to reuse
_geometry.glb directly without re-tessellating.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Per-solid iteration prevents OOM on multi-part assemblies (25-part bearing:
2.3GB RAM when processing compound → ~100MB per solid with per-solid approach)
- Fix CharacteristicLengthMax multiplier 5× → 15× and cap MinimumCirclePoints
at 20 (prevents 63-pts/circle on angular_deflection=0.1rad → 231MB → 21MB)
- Geometry task timeout 120s → 600s for large assemblies
- Production task: reuse _geometry.glb when GMSH enabled (no re-tessellation)
and cache _production_geom.glb for OCC (mtime vs STEP check)
- Viewer now prefers production GLB when available (shows correct GMSH mesh)
- GMSH OpenMP multithreading (min(cpu_count,16)) for 4.4× speedup
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces GMSH as an alternative to OCC BRepMesh for STEP→GLB tessellation.
GMSH produces conforming meshes that eliminate fan triangles at cylinder seam
edges — a structural limitation of OCC BRepMesh that cannot be fixed via
deflection parameters.
Changes:
- render-worker/Dockerfile: install gmsh>=4.15.0 + libglu1-mesa + libxft2
- export_step_to_gltf.py: --tessellation_engine occ|gmsh CLI arg +
_tessellate_with_gmsh() using BRep→GMSH→Poly_Triangulation write-back
- admin.py: tessellation_engine setting (SETTINGS_DEFAULTS, SettingsOut,
SettingsUpdate, validation)
- export_glb.py: pass tessellation_engine to export_step_to_gltf.py CLI in
both geometry and production GLB tasks
- Admin.tsx: radio button UI for OCC vs GMSH selection
Tested: 121 faces meshed, 0 BRepMesh fallback, 649K triangles on sample part.
Clean seam edges for UV unwrap — GMSH respects B-rep periodic face boundaries.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When re-generating a production or geometry GLB, the old approach deleted the
existing MediaAsset record and created a new one with a new UUID. Any page that
had the old download_url (/api/media/{old-id}/download) cached would then get
a 404 when trying to download, because the asset ID no longer existed in the DB.
Fix: update the existing MediaAsset record in-place (same UUID, new storage_key)
so existing download URLs remain valid after regeneration. Create a new record
only if no existing one is found.
Applies to both generate_gltf_geometry_task and generate_gltf_production_task.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Per-tenant Azure AI config stored in tenants.tenant_config JSONB
- GET/PUT /api/tenants/{id}/ai-config + POST .../test connection
- api_key never returned to frontend (has_api_key: bool pattern)
- azure_ai.py resolves creds from tenant config when ai_enabled=True
- ai_tasks.py loads tenant config and passes it to validate_thumbnail
- Admin GPU Status section: probe button + status badge + last-checked time
- Notifications: _BELL_CHANNELS filter (notification+alert only in bell)
- Tenants.tsx: per-row Azure AI Config modal with URL auto-parse helper
- Remove duplicate in-memory /gpu-probe endpoints (kept DB-backed /probe/gpu)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- gpu_probe.py: Blender script that probes OPTIX/CUDA/HIP/ONEAPI and
exits 1 on no GPU — used at startup + on-demand from Admin UI
- blender_render.py, still_render.py, turntable_render.py: emit
RENDER_DEVICE_USED: engine=CYCLES device=GPU|CPU compute_type=...
after GPU activation; exit 2 when CYCLES_DEVICE=gpu and CPU fallback
- render_blender.py: parse RENDER_DEVICE_USED token into render_log
(device_used, compute_type, gpu_fallback); handle exit code 2 as
explicit GPU strict-mode failure
- check_version.py: check_gpu() runs gpu_probe.py at container startup;
CYCLES_DEVICE=gpu aborts startup if no GPU found
- docker-compose.yml: CYCLES_DEVICE=${CYCLES_DEVICE:-auto} env var
- gpu_tasks.py: probe_gpu Celery task on thumbnail_rendering queue;
saves result to system_settings.gpu_probe_last_result; beat every 30min
- worker.py: POST /probe/gpu (trigger) + GET /probe/gpu/result (last result)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Detect OrderLines stuck in render_status='processing' for >30min,
revoke their Celery task, mark failed, update render_job_doc, and
emit a single CHANNEL_ALERT batch notification to admins.
Runs every 5min via beat schedule alongside recover_stuck_cad_files.
Section B (tenant_id in JWT) confirmed already complete from Phase 4.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/workflows/pipeline-steps: returns all StepName enum values
with category (input|processing|rendering|output) + descriptions;
registered before /{workflow_id} to avoid path collision
- frontend/src/api/workflows.ts: getPipelineSteps(), PipelineStep
and PipelineStepsResponse interfaces
- WorkflowEditor: PipelineStepsPanel showing steps grouped by category
with collapsible accordion sections
- ConfigSidepanel: "Pipeline Step" select dropdown binds any node to a
StepName; selected step description shown below dropdown
- Active workflow indicator: green dot next to is_active=true entries
- Improved empty state with descriptive copy
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 5.1 — MATERIAL_PALETTE removal:
- Remove MATERIAL_PALETTE + _material_to_color() from step_processor.py
- build_part_colors() now returns {part→material_name} for Blender resolver
Phase 6 — Notification Center Refactor:
- Migration 051: add channel (activity|notification|alert) to audit_log,
add frequency (immediate|daily|never) to notification_configs
- Three notification channels: activity (per-render), notification (batch
order summaries), alert (admin infrastructure)
- Per-render emit_notification_sync calls demoted to channel=activity
- New emit_batch_render_notification_sync(): single summary notification
when all order lines reach terminal state ("47/50 succeeded, 3 failed")
- Beat task batch_render_notifications every 60s: safety-net for missed
batch notifications after order completion
- GET /notifications: defaults to channel IN (notification, alert);
accepts ?channel=activity for activity feed
- Unread count badge counts only notification+alert channels
- Notifications.tsx: three tabs (Notifications | Activity | Alerts)
- NotificationSettings.tsx: frequency dropdown per event type
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 2.3 — Fix render cancellation (real Celery task ID):
- orders.py cancel endpoints: read celery_task_id from render_job_doc
instead of synthetic "render-{line_id}" which was a no-op
- render_order_line_still_task: creates RenderJobDocument at task start,
stores self.request.id as celery_task_id, writes step-level state
(RESOLVE_STEP_PATH → BLENDER_STILL) back to order_lines.render_job_doc
Phase 3.1 — Remove Pillow overlay dead code:
- blender_render.py: deleted 55-line Pillow post-processing block
(lines 798-851, green bar + model name label)
- transparent_bg=True is always passed; the else branch was unreachable
- Removed mention from script docstring
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Phase 1 of PLAN_REFACTOR.md — all four sub-tasks implemented:
1.1 PipelineLogger (backend/app/core/pipeline_logger.py)
- Structured step_start/step_done/step_error/step_progress API
- Publishes to Python logging AND Redis SSE via log_task_event
- Context manager `pl.step("name")` for auto-timing
1.2 RenderJobDocument (backend/app/domains/rendering/job_document.py)
- Pydantic JSONB schema: state machine + per-step records + timing
- begin_step/finish_step/fail_step/skip_step helpers
- Migration 048: adds render_job_doc JSONB column to order_lines
- OrderLine model updated with render_job_doc field
1.3 TenantContextMiddleware (backend/app/core/middleware.py)
- Decodes JWT, stores tenant_id + role in request.state
- get_db updated to auto-apply RLS SET LOCAL from request.state
- Registered in main.py (runs before every request)
- JWT now embeds tenant_id claim via create_access_token()
- Login endpoint passes tenant_id to token creation
1.4 ProcessStep Registry (backend/app/core/process_steps.py)
- StepName StrEnum with all 20 pipeline step names
- Single source of truth for log prefixes, DB records, UI labels
Also adds db_utils.py with set_tenant_sync() + get_sync_session()
for use inside Celery tasks (bypass-safe RLS helper).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GPU: fix Cycles device activation order — set compute_device_type
BEFORE engine init, re-set AFTER open_mainfile wipes preferences
- GPU: remove _mark_sharp_and_seams edit-mode loop (redundant with
Blender 5.0 shade_smooth_by_angle), saves ~200s/render on 175 parts
- Material: fix _AFN suffix mismatch — build AF-stripped mat_map keys
and add prefix fallback in _apply_material_library (blender_render.py)
- Material: production GLB now uses get_material_library_path() which
checks active AssetLibrary instead of empty legacy system setting
- Admin: RenderTemplateTable multi-select output types (M2M frontend)
- Admin: MaterialLibraryPanel replaced with link to Asset Libraries
- UX: move Toaster to top-left to avoid dispatch button overlap
- SQLAlchemy: add .unique() to all RenderTemplate M2M collection queries
- Logging: flush=True on all Blender progress prints, stdout reconfigure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OCC's RWGltf_CafWriter appends _AF0/_AF1 assembly-instance suffixes to
mesh object names when a part appears multiple times in an assembly.
The material matching in export_gltf.py only stripped Blender's .001
suffix, leaving 24/175 GLB objects without materials.
Fix: strip _AFN suffixes via while loop (handles nested _AF0_AF1),
add prefix fallback (longest key wins) as last resort before no-match.
Also improve build_materials_from_excel Jaccard matching:
- Strip _AFN and numeric hash suffixes (-21227) before tokenizing
- Add prefix-based fallback (step 3) before position fallback (step 4)
- Raise threshold 0.3 → 0.35 for better precision
- Guard prefix matches to len >= 5 to prevent trivial false positives
Result: Material substitution: 175/175 mesh objects assigned (was 151/175)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Hartmut