Hartmut/HartOMat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
29 Commits 1 Branch 0 Tags
3ac3ca1d703bd1c350ffef5f76228822e1ec9cbe
Commit Graph

21 Commits

Author SHA1 Message Date
Hartmut 3ac3ca1d70 fix(K): correct CATALOG_SCRIPT path + register AssetLibrary in models/__init__ 
- fix(tasks): use RENDER_SCRIPTS_DIR env var for catalog_assets.py path
  (was computing wrong path via __file__ parents → /render-worker/scripts/ which doesn't exist in container)
- fix(models): add AssetLibrary to app/models/__init__.py so alembic autogenerate discovers it
- fix(api): remove unused FileResponse import from asset_libraries.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 21:02:41 +01:00
Hartmut a18d4c23ec feat(K): Blender Asset Library + production exports (GLB + .blend) 
- feat(migration): 045_asset_libraries — new asset_libraries table (blend_file_path, catalog JSONB)
- feat(model): AssetLibrary SQLAlchemy model in domains/materials/models.py
- feat(api): POST/GET/PATCH/DELETE /api/asset-libraries + /upload-blend + /refresh-catalog endpoints
- feat(celery): refresh_asset_library_catalog task on thumbnail_rendering queue — runs Blender headless
- feat(blender): catalog_assets.py — extracts asset-marked materials + node_groups from .blend
- feat(blender): asset_library.py — apply_asset_library_materials + apply_asset_library_node_groups helpers
- feat(blender): export_gltf.py — STEP→STL→GLB production export with optional asset library
- feat(blender): export_blend.py — STEP→STL→.blend production export with pack_all()
- feat(frontend): api/assetLibraries.ts — full CRUD API client
- feat(frontend): AssetLibraryPanel in Admin.tsx — upload, refresh, expand catalog view
- docs: Blender asset_data marking requirement learning in LEARNINGS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:56:26 +01:00
Hartmut 7a1329958d feat(J): WebSocket live-events + replace polling + fix ffmpeg turntable timeout 
- fix(render): ffmpeg overlay=0:0 -> overlay=0:0:shortest=1 to prevent hang on finite PNG sequences
- feat(ws): add core/websocket.py ConnectionManager + Redis Pub/Sub subscriber loop
- feat(ws): add /api/ws WebSocket endpoint with JWT query-param auth in main.py
- feat(ws): emit render_complete/failed + cad_processing_complete events from step_tasks.py
- feat(ws): emit order_status_change events from orders router
- feat(ws): add beat_tasks.py broadcast_queue_status task (every 10s via Redis __broadcast__)
- feat(frontend): add useWebSocket hook with auto-reconnect (exponential backoff, 25s ping)
- feat(frontend): add WebSocketContext + WebSocketProvider wrapping App
- refactor(frontend): remove polling from WorkerActivity (was 5s/3s) + OrderDetail (was 5s)
- refactor(frontend): reduce polling in Layout (8s->60s) + NotificationCenter (15s->60s)
- docs: add ffmpeg shortest=1 + WebSocket JWT auth learnings to LEARNINGS.md

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:49:34 +01:00
Hartmut ceb0143cb6 fix: animation output types now render turntable mp4 via Celery 
- render_order_line_task: detect is_animation flag, branch to turntable
  pipeline instead of still-image pipeline
- render_blender: add render_turntable_to_file() — STL conversion,
  blender turntable_render.py for frames, ffmpeg compose to mp4
- render-worker/Dockerfile: add ffmpeg package
- Fix: Turntable Video_White was producing a .jpg still instead of .mp4

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:10:33 +01:00
Hartmut d138bc4bc4 feat(activity): merge CAD + render jobs into unified timeline 
- Single chronological list sorted by updated_at (newest first)
- Type badges distinguish CAD processing (FileCode2) from Render jobs (Image)
- Render job rows now link directly to the order (/orders/:id)
- Remove separate "Render Jobs" and "CAD File Processing" sections
- Stat cards simplified to 4: CAD active/failed + Render active/failed
- Backend: add order_id to RenderJobEntry response
- Frontend: add order_id to RenderJobEntry interface, remove flamenco_job_id

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 20:01:03 +01:00
Hartmut ab3f9c734a fix: render pipeline + multi-tenancy bugs (B-Fix-1 through B-Fix-9) 
- Remove worker-thumbnail (no Blender, was competing on thumbnail_rendering)
- Move render_order_line_task to thumbnail_rendering queue (render-worker)
- Restore template_service.py real implementation (fix circular import shim)
- Thread tenant_id through STEP upload, Excel import, product create
- Make system tables (output_types, materials, etc.) tenant_id nullable
- Fix tenants frontend 307-redirect: use trailing slash /tenants/
- Remove Flamenco + Three.js from Admin UI (unsupported)
- Set all output_types render_backend to celery (was flamenco)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 19:34:20 +01:00
Hartmut 381f44bc8b feat: render health endpoint + test script + pipeline fixes 
- GET /api/worker/health/render: checks render-worker (thumbnail_rendering
  queue), Blender availability via active_queues inspect, queue depth,
  last render recency — returns ok/degraded/down status
- scripts/test_render_pipeline.py: integration test for full pipeline
  (--health, --sample, --full modes)
- PLAN.md: appended Render Pipeline Fixes section with all B-Fixes
- LEARNINGS.md: documented 5 new learnings (queue mismatch, circular
  import, 307 redirect, worker capability detection)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 19:34:12 +01:00
Hartmut 979b0082ec fix(api): remove 307 redirect on GET /api/workflows 
Change list/create route paths from '/' to '' on workflow_router so
GET /api/workflows and POST /api/workflows respond directly without
307 redirect. The redirect was causing axios to drop the Authorization
header, making the WorkflowEditor always show an empty list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 18:26:27 +01:00
Hartmut c0ea60d984 fix: resolve open risks — invoice race condition, SMTP config, workflow seeds 
- billing/service.py: pg_advisory_xact_lock on invoice_number_seq per year
  → prevents duplicate INV-YYYY-NNNN under concurrent requests
- admin.py: SMTP settings in system_settings (smtp_host/port/user/password/
  from_address/enabled) with GET+PUT support; seed-workflows endpoint creates
  4 standard workflow definitions (still-cycles, still-eevee, turntable,
  multi-angle) idempotently
- notifications/service.py: send_email_notification_stub now sends real
  SMTP email via smtplib when smtp_enabled=true in system_settings
- Admin.tsx: SMTP settings panel (host/port/user/password/from + enable
  toggle, save button); Seed Standard Workflows maintenance button
- Upload.tsx: fix TS error — title→aria-label on Lucide icons
- Admin.tsx Settings type: add render_backend/flamenco_* fields (TS fix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 18:15:45 +01:00
Hartmut f19a6ccde8 feat(F-G-H-I): STL cache, invoices, import validation, notification settings 
Phase F — STL Hash Cache:
- Migration 041: step_file_hash column on cad_files
- cache_service.py: SHA256 hash + MinIO-backed STL cache (check/store)
- render_step_thumbnail: compute+persist hash before render
- generate_stl_cache: check MinIO cache before cadquery conversion, store after

Phase G — Invoices:
- Migration 042: invoices + invoice_lines tables with RLS
- Invoice/InvoiceLine models + schemas
- billing service: generate_invoice_number (INV-YYYY-NNNN), create/list/get/delete/PDF
- WeasyPrint PDF generation; backend Dockerfile + pyproject.toml deps
- invoice_router with 6 endpoints; registered in main.py
- frontend: Billing.tsx page + api/billing.ts; route + nav link

Phase H — Import Sanity Check:
- Migration 043: import_validations table
- ImportValidation model + schemas
- run_sanity_check: material fuzzy-match (cutoff=0.8), STEP availability, duplicate detection
- validate_excel_import Celery task (queue: step_processing)
- uploads.py: create ImportValidation on /excel, fire task, expose GET /validations/{id}
- frontend: Upload.tsx polling ValidationDialog with Ampel status indicators

Phase I — Notification Settings:
- Migration 044: notification_configs table (user×event×channel toggles)
- NotificationConfig model + seeds (in_app=true, email=false)
- get/upsert/reset config endpoints on /notifications/config
- frontend: NotificationSettings.tsx page + api/notifications.ts extensions

Infrastructure:
- docker-compose.yml: add worker-thumbnail service (concurrency=1, Q=thumbnail_rendering)
- Fix Dockerfile: libgdk-pixbuf-2.0-0 (correct Debian bookworm package name)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 18:05:01 +01:00
Hartmut c74e118b98 feat(E): add MediaAsset catalog — model, CRUD API, MediaBrowser UI 
Migration 040: media_assets table with RLS (tenant_isolation + admin_bypass).
domains/media/: MediaAsset model, schemas, service, router with ZIP-download.
publish_asset Celery task in rendering/tasks.py.
core/storage.py: download_bytes() method for MinIO + LocalStorage.
frontend: MediaBrowser.tsx (grid/list, multi-select, zip-download, pagination) + api/media.ts.
Route /media (AdminRoute) + sidebar link with Image icon for admin+pm.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 17:11:17 +01:00
Hartmut 716451ff76 feat(D): OCC mesh attribute extraction + Blender smooth shading integration 
Migration 039: cad_files.mesh_attributes JSONB column.
domains/products/tasks.py: extract_mesh_attributes Celery task using pythonOCC.
still_render.py + turntable_render.py: _apply_mesh_attributes() sets auto-smooth
based on curved_ratio and topology threshold from OCC analysis.
render_blender.py: passes --mesh-attributes JSON arg to Blender subprocess.
render_still_task: loads mesh_attributes from DB and passes to renderer.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 17:07:55 +01:00
Hartmut 7e47e4aca7 feat(C1+C2): workflow system — WorkflowDefinition + Celery Canvas builder 
Migrations 037 (workflow tables + 3 seed definitions) + 038 (output_types.workflow_definition_id).
WorkflowDefinition/Run/NodeResult SQLAlchemy models in domains/rendering/models.py.
workflow_builder.py: dispatch_workflow() with Celery Canvas for still/turntable/multi_angle.
workflow_router.py: CRUD endpoints at /api/workflows (admin/PM guards).
dispatch_service.py: dispatch_render_with_workflow() prefers workflow path when
  OutputType.workflow_definition_id is set, falls back to legacy dispatch otherwise.
main.py: registers workflows_router.
models/__init__.py: re-exports WorkflowDefinition, WorkflowRun, WorkflowNodeResult.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 17:07:21 +01:00
Hartmut 251dd703ed feat(B2): add tenant model + migrations 035/036 + RLS policies 
Migration 035: tenants table with 'Schaeffler' default seed.
Migration 036: tenant_id FK on all tables, RLS policies, backfill.
New domains/tenants/ with CRUD router (admin only).
All domain models extended with tenant_id FK.
core/database.py: get_db_for_tenant with RLS context setter.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 16:30:41 +01:00
Hartmut b87df4a3e5 refactor(B1): migrate to domain-driven project structure 
Move all models/schemas/services/routers into app/domains/.
Keep backward-compat shims in old locations for imports.
Preserves domains/rendering/tasks.py from Phase A.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 16:24:11 +01:00
Hartmut 5764118b8e feat(A5): add app_config table + typed config service 
- Migration 034: creates app_config table with JSONB columns (render, storage,
  worker, notifications, billing); migrates existing system_settings values
- backend/app/core/config_service.py:
  - Typed Pydantic models: RenderConfig, WorkerConfig, StorageConfig, etc.
  - AppConfig aggregate model
  - get_app_config(db) async + get_app_config_sync() for Celery tasks
  - update_render_config() / update_worker_config() for partial updates
- system_settings table preserved for backward compat during Phase B migration

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:51:56 +01:00
Hartmut c728358fb6 feat(A4): add MinIO service + storage abstraction (core/storage.py) 
- Add MinIO service to docker-compose.yml (port 9000 API, 9001 console)
- Add minio-data volume for persistent object storage
- Create backend/app/core/storage.py: MinIOStorage + LocalStorage abstraction
  - MinIOStorage: boto3-based, auto-creates bucket, upload/download/exists/delete/presign
  - LocalStorage: fallback for dev (UPLOAD_DIR filesystem, backward compat)
  - get_storage() singleton: auto-selects based on MINIO_URL env var
- Add MINIO_URL/USER/PASSWORD/BUCKET env vars to all service definitions
- backend/pyproject.toml: docker>=6.1.0 → boto3>=1.34.0
- Add docker-compose.worker.yml: external render-worker for remote machines
- Fix .gitignore: 'core' rule was too broad, now only matches root /core dump
- Update .env.example: MinIO connection vars documented

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:51:06 +01:00
Hartmut 9d1a820295 refactor(A2): replace blender-renderer HTTP service with render-worker Celery container 
- Create render-worker/ with Dockerfile (Ubuntu + cadquery + Blender via host mount)
- Add render-worker/check_version.py: verifies Blender >= 5.0.1 at startup, Exit 1 on failure
- Add render-worker/scripts/: blender_render.py, still_render.py, turntable_render.py
- Create backend/app/services/render_blender.py: direct subprocess rendering
  - convert_step_to_stl() and export_per_part_stls() using cadquery
  - render_still(): STEP → STL → PNG via Blender subprocess
  - is_blender_available(): detects BLENDER_BIN env for render-worker context
- Create backend/app/domains/rendering/tasks.py: render_still_task + render_turntable_task
- Update step_processor.py: use subprocess path when BLENDER_BIN env is set (render-worker)
- Update step_tasks.py: generate_stl_cache uses direct cadquery instead of HTTP
- Remove blender-renderer and threejs-renderer from docker-compose.yml
- Replace worker-thumbnail with render-worker (Ubuntu + cadquery + Blender mount)
- Remove Docker SDK from backend Dockerfile (was only for flamenco scaling)
- Update .env.example: BLENDER_VERSION=5.0.1 documented
- Update celery_app.py: include domains.rendering.tasks in autodiscover

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:48:46 +01:00
Hartmut 1d6864fb64 refactor(A1): remove Flamenco, simplify render pipeline to Celery-only 
- Remove flamenco-manager and flamenco-worker from docker-compose.yml
- Delete flamenco_client.py, flamenco_tasks.py, docker_scaler.py
- Simplify render_dispatcher.py to Celery-only (removes ~300 lines)
- Remove Flamenco beat schedule from celery_app.py
- Clean admin.py: remove flamenco settings, endpoints, threejs validation
- Clean orders.py cancel-render: Celery revoke only
- Clean worker.py: remove flamenco_job_id from activity response
- Migration 032: cancel lingering flamenco jobs, remove flamenco settings
- PLAN.md: mark all decisions confirmed, status IN UMSETZUNG

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-06 15:38:37 +01:00
Hartmut 552922eb8a chore: remove build artifacts from tracking, extend .gitignore 2026-03-05 22:13:21 +01:00
Hartmut bce762a783 feat: initial commit 2026-03-05 22:12:38 +01:00