Commit Graph

17 Commits

Author SHA1 Message Date
Hartmut 89c44b846f feat(phase5.1+6): fallback material cleanup + notification batch refactor
Phase 5.1 — MATERIAL_PALETTE removal:
- Remove MATERIAL_PALETTE + _material_to_color() from step_processor.py
- build_part_colors() now returns {part→material_name} for Blender resolver

Phase 6 — Notification Center Refactor:
- Migration 051: add channel (activity|notification|alert) to audit_log,
  add frequency (immediate|daily|never) to notification_configs
- Three notification channels: activity (per-render), notification (batch
  order summaries), alert (admin infrastructure)
- Per-render emit_notification_sync calls demoted to channel=activity
- New emit_batch_render_notification_sync(): single summary notification
  when all order lines reach terminal state ("47/50 succeeded, 3 failed")
- Beat task batch_render_notifications every 60s: safety-net for missed
  batch notifications after order completion
- GET /notifications: defaults to channel IN (notification, alert);
  accepts ?channel=activity for activity feed
- Unread count badge counts only notification+alert channels
- Notifications.tsx: three tabs (Notifications | Activity | Alerts)
- NotificationSettings.tsx: frequency dropdown per event type

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 20:20:07 +01:00
Hartmut 9f54bc3ab1 feat(phase4+5): role hierarchy, tenant config, fallback material, dead code removal
Phase 4.1 — Role Hierarchy:
  - UserRole enum: add global_admin (platform operator) + tenant_admin
    (per-tenant admin); keep legacy 'admin' for backward compat
  - Role sets: ADMIN_ROLES, TENANT_ADMIN_ROLES, PM_ROLES, RLS_BYPASS_ROLES
  - New auth guards: require_global_admin(), require_tenant_admin_or_above(),
    require_pm_or_above(), is_admin(), is_privileged()
  - Legacy require_admin / require_admin_or_pm now check both old+new roles
  - Migration 049: ADD VALUE global_admin + tenant_admin with AUTOCOMMIT
    workaround; backfills admin → global_admin
  - Seed: new admin users created with global_admin role

Phase 4.3 — RLS bypass updated for global_admin in get_db + set_tenant_context

Phase 4.4 — Tenant Feature Flags:
  - Migration 050: tenant_config JSONB on tenants table
  - Tenant model: tenant_config field + get_config() accessor
  - Defaults: max_concurrent_renders=3, fallback_material, invoice_prefix etc.

Phase 5.1 — Fallback Material:
  - blender_render.py: replace PALETTE_LINEAR/PALETTE_HEX/_assign_palette_material
    with _assign_failed_material() → SCHAEFFLER_059999_FailedMaterial (magenta)
  - Unmatched parts now logged explicitly before rendering

Phase 5.2 — Remove EEVEE fallback:
  - render_blender.py: EEVEE→Cycles silent retry removed; hard failure on EEVEE error

Phase 5.3 — Remove Blender version check:
  - render_blender.py: deleted MIN_BLENDER_VERSION = (5, 0, 1) constant

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 19:42:10 +01:00
Hartmut ea31ed657c feat(refactor/phase1): foundation infrastructure for modular pipeline
Phase 1 of PLAN_REFACTOR.md — all four sub-tasks implemented:

1.1 PipelineLogger (backend/app/core/pipeline_logger.py)
  - Structured step_start/step_done/step_error/step_progress API
  - Publishes to Python logging AND Redis SSE via log_task_event
  - Context manager `pl.step("name")` for auto-timing

1.2 RenderJobDocument (backend/app/domains/rendering/job_document.py)
  - Pydantic JSONB schema: state machine + per-step records + timing
  - begin_step/finish_step/fail_step/skip_step helpers
  - Migration 048: adds render_job_doc JSONB column to order_lines
  - OrderLine model updated with render_job_doc field

1.3 TenantContextMiddleware (backend/app/core/middleware.py)
  - Decodes JWT, stores tenant_id + role in request.state
  - get_db updated to auto-apply RLS SET LOCAL from request.state
  - Registered in main.py (runs before every request)
  - JWT now embeds tenant_id claim via create_access_token()
  - Login endpoint passes tenant_id to token creation

1.4 ProcessStep Registry (backend/app/core/process_steps.py)
  - StepName StrEnum with all 20 pipeline step names
  - Single source of truth for log prefixes, DB records, UI labels

Also adds db_utils.py with set_tenant_sync() + get_sync_session()
for use inside Celery tasks (bypass-safe RLS helper).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 19:25:08 +01:00
Hartmut ee6eb34b4c feat: GPU rendering + material matching + perf improvements
- GPU: fix Cycles device activation order — set compute_device_type
  BEFORE engine init, re-set AFTER open_mainfile wipes preferences
- GPU: remove _mark_sharp_and_seams edit-mode loop (redundant with
  Blender 5.0 shade_smooth_by_angle), saves ~200s/render on 175 parts
- Material: fix _AFN suffix mismatch — build AF-stripped mat_map keys
  and add prefix fallback in _apply_material_library (blender_render.py)
- Material: production GLB now uses get_material_library_path() which
  checks active AssetLibrary instead of empty legacy system setting
- Admin: RenderTemplateTable multi-select output types (M2M frontend)
- Admin: MaterialLibraryPanel replaced with link to Asset Libraries
- UX: move Toaster to top-left to avoid dispatch button overlap
- SQLAlchemy: add .unique() to all RenderTemplate M2M collection queries
- Logging: flush=True on all Blender progress prints, stdout reconfigure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-08 19:05:03 +01:00
Hartmut bfc0050580 feat(L+M): configurable dashboard widget system + test framework
Phase L: Dashboard widget system
- Migration 046: dashboard_configs table (user/tenant/role fallback cascade)
- DashboardConfig model + dashboard_service with get/upsert per-user and tenant-default
- API router: GET/PUT /api/dashboard/config, GET/PUT /api/dashboard/tenant-default
- Frontend: 5 widget components (ProductionStats, QueueStatus, RecentRenders, CostOverview, WorkerStatus)
- DashboardGrid with API-backed config, DashboardCustomizeModal (checkbox selection, save/cancel)
- Dashboard.tsx: widget grid + analytics section (privileged users)
- Admin.tsx: restructured with new section order and Maintenance 2-col grid

Phase M: Test framework
- Backend: pytest-asyncio + pytest-cov + factory-boy in pyproject.toml
- conftest.py: excel_dir fixtures + async DB fixtures + mock storage/celery stubs
- Domain tests: billing_service, cache_service, notifications_service, imports_sanity
- Integration: test_api_health.py smoke test (requires running backend)
- Frontend: vitest + @testing-library/react + msw added to package.json
- vite.config.ts: test block (jsdom + globals + setupFiles)
- tsconfig.json: exclude src/__tests__ from main tsc (test runner handles its own types)
- MSW handlers for /api/auth/me, Billing.test.tsx, formatters.test.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 21:50:07 +01:00
Hartmut a18d4c23ec feat(K): Blender Asset Library + production exports (GLB + .blend)
- feat(migration): 045_asset_libraries — new asset_libraries table (blend_file_path, catalog JSONB)
- feat(model): AssetLibrary SQLAlchemy model in domains/materials/models.py
- feat(api): POST/GET/PATCH/DELETE /api/asset-libraries + /upload-blend + /refresh-catalog endpoints
- feat(celery): refresh_asset_library_catalog task on thumbnail_rendering queue — runs Blender headless
- feat(blender): catalog_assets.py — extracts asset-marked materials + node_groups from .blend
- feat(blender): asset_library.py — apply_asset_library_materials + apply_asset_library_node_groups helpers
- feat(blender): export_gltf.py — STEP→STL→GLB production export with optional asset library
- feat(blender): export_blend.py — STEP→STL→.blend production export with pack_all()
- feat(frontend): api/assetLibraries.ts — full CRUD API client
- feat(frontend): AssetLibraryPanel in Admin.tsx — upload, refresh, expand catalog view
- docs: Blender asset_data marking requirement learning in LEARNINGS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 20:56:26 +01:00
Hartmut f19a6ccde8 feat(F-G-H-I): STL cache, invoices, import validation, notification settings
Phase F — STL Hash Cache:
- Migration 041: step_file_hash column on cad_files
- cache_service.py: SHA256 hash + MinIO-backed STL cache (check/store)
- render_step_thumbnail: compute+persist hash before render
- generate_stl_cache: check MinIO cache before cadquery conversion, store after

Phase G — Invoices:
- Migration 042: invoices + invoice_lines tables with RLS
- Invoice/InvoiceLine models + schemas
- billing service: generate_invoice_number (INV-YYYY-NNNN), create/list/get/delete/PDF
- WeasyPrint PDF generation; backend Dockerfile + pyproject.toml deps
- invoice_router with 6 endpoints; registered in main.py
- frontend: Billing.tsx page + api/billing.ts; route + nav link

Phase H — Import Sanity Check:
- Migration 043: import_validations table
- ImportValidation model + schemas
- run_sanity_check: material fuzzy-match (cutoff=0.8), STEP availability, duplicate detection
- validate_excel_import Celery task (queue: step_processing)
- uploads.py: create ImportValidation on /excel, fire task, expose GET /validations/{id}
- frontend: Upload.tsx polling ValidationDialog with Ampel status indicators

Phase I — Notification Settings:
- Migration 044: notification_configs table (user×event×channel toggles)
- NotificationConfig model + seeds (in_app=true, email=false)
- get/upsert/reset config endpoints on /notifications/config
- frontend: NotificationSettings.tsx page + api/notifications.ts extensions

Infrastructure:
- docker-compose.yml: add worker-thumbnail service (concurrency=1, Q=thumbnail_rendering)
- Fix Dockerfile: libgdk-pixbuf-2.0-0 (correct Debian bookworm package name)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 18:05:01 +01:00
Hartmut 7706c514c8 fix(deploy): fix render-worker build context + migration 040 idempotency
- docker-compose.yml: change render-worker build context from ./render-worker
  to . (project root) so pyproject.toml is accessible; update dockerfile path
- render-worker/Dockerfile: update COPY paths for new build context;
  install Python 3.11 via deadsnakes PPA (Ubuntu 22.04 ships 3.10 which
  fails the >=3.11 requirement in pyproject.toml)
- 040_media_assets.py: rewrite upgrade() with raw idempotent SQL (CREATE TYPE
  inside DO $$ EXCEPTION WHEN duplicate_object $$; CREATE TABLE IF NOT EXISTS;
  CREATE INDEX IF NOT EXISTS) to handle pre-existing enum from partial runs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 17:32:42 +01:00
Hartmut c74e118b98 feat(E): add MediaAsset catalog — model, CRUD API, MediaBrowser UI
Migration 040: media_assets table with RLS (tenant_isolation + admin_bypass).
domains/media/: MediaAsset model, schemas, service, router with ZIP-download.
publish_asset Celery task in rendering/tasks.py.
core/storage.py: download_bytes() method for MinIO + LocalStorage.
frontend: MediaBrowser.tsx (grid/list, multi-select, zip-download, pagination) + api/media.ts.
Route /media (AdminRoute) + sidebar link with Image icon for admin+pm.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 17:11:17 +01:00
Hartmut 716451ff76 feat(D): OCC mesh attribute extraction + Blender smooth shading integration
Migration 039: cad_files.mesh_attributes JSONB column.
domains/products/tasks.py: extract_mesh_attributes Celery task using pythonOCC.
still_render.py + turntable_render.py: _apply_mesh_attributes() sets auto-smooth
based on curved_ratio and topology threshold from OCC analysis.
render_blender.py: passes --mesh-attributes JSON arg to Blender subprocess.
render_still_task: loads mesh_attributes from DB and passes to renderer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 17:07:55 +01:00
Hartmut 7e47e4aca7 feat(C1+C2): workflow system — WorkflowDefinition + Celery Canvas builder
Migrations 037 (workflow tables + 3 seed definitions) + 038 (output_types.workflow_definition_id).
WorkflowDefinition/Run/NodeResult SQLAlchemy models in domains/rendering/models.py.
workflow_builder.py: dispatch_workflow() with Celery Canvas for still/turntable/multi_angle.
workflow_router.py: CRUD endpoints at /api/workflows (admin/PM guards).
dispatch_service.py: dispatch_render_with_workflow() prefers workflow path when
  OutputType.workflow_definition_id is set, falls back to legacy dispatch otherwise.
main.py: registers workflows_router.
models/__init__.py: re-exports WorkflowDefinition, WorkflowRun, WorkflowNodeResult.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 17:07:21 +01:00
Hartmut 251dd703ed feat(B2): add tenant model + migrations 035/036 + RLS policies
Migration 035: tenants table with 'Schaeffler' default seed.
Migration 036: tenant_id FK on all tables, RLS policies, backfill.
New domains/tenants/ with CRUD router (admin only).
All domain models extended with tenant_id FK.
core/database.py: get_db_for_tenant with RLS context setter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 16:30:41 +01:00
Hartmut 5764118b8e feat(A5): add app_config table + typed config service
- Migration 034: creates app_config table with JSONB columns (render, storage,
  worker, notifications, billing); migrates existing system_settings values
- backend/app/core/config_service.py:
  - Typed Pydantic models: RenderConfig, WorkerConfig, StorageConfig, etc.
  - AppConfig aggregate model
  - get_app_config(db) async + get_app_config_sync() for Celery tasks
  - update_render_config() / update_worker_config() for partial updates
- system_settings table preserved for backward compat during Phase B migration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 15:51:56 +01:00
Hartmut c8ecc29d40 refactor(A3): remove threejs-renderer service, add migration 033
- Migration 033: reset threejs thumbnail_renderer setting → blender,
  remove obsolete threejs_render_size system_setting
- step_processor.py threejs code paths already fall through to Pillow
  (committed in A2); Three.js browser viewer (ThreeDViewer.tsx) remains
- threejs-renderer/ directory kept for reference but no longer built

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 15:49:16 +01:00
Hartmut 1d6864fb64 refactor(A1): remove Flamenco, simplify render pipeline to Celery-only
- Remove flamenco-manager and flamenco-worker from docker-compose.yml
- Delete flamenco_client.py, flamenco_tasks.py, docker_scaler.py
- Simplify render_dispatcher.py to Celery-only (removes ~300 lines)
- Remove Flamenco beat schedule from celery_app.py
- Clean admin.py: remove flamenco settings, endpoints, threejs validation
- Clean orders.py cancel-render: Celery revoke only
- Clean worker.py: remove flamenco_job_id from activity response
- Migration 032: cancel lingering flamenco jobs, remove flamenco settings
- PLAN.md: mark all decisions confirmed, status IN UMSETZUNG

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-06 15:38:37 +01:00
Hartmut 552922eb8a chore: remove build artifacts from tracking, extend .gitignore 2026-03-05 22:13:21 +01:00
Hartmut bce762a783 feat: initial commit 2026-03-05 22:12:38 +01:00