refactor: rename thumbnail_rendering queue to asset_pipeline
The queue handles far more than thumbnails: OCC tessellation, USD master generation, GLB production, order line renders, and workflow renders. asset_pipeline better reflects its role as the render-worker's primary queue. Updated all references in: task decorators, celery_app.py, beat_tasks.py, docker-compose.yml worker command, worker.py MONITORED_QUEUES, admin.py, CLAUDE.md, LEARNINGS.md, Dockerfile, helpTexts.ts, test files, and all .claude/commands/*.md skill files. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+6
-6
@@ -15,7 +15,7 @@ Schaeffler Automat is a working Blender-based media production pipeline with:
|
||||
- 7 Docker services with GPU render-worker
|
||||
- PostgreSQL with tenant_id columns + Row Level Security (RLS) enabled but inconsistently
|
||||
applied at the application layer
|
||||
- Celery task queues with two workers (step_processing + thumbnail_rendering)
|
||||
- Celery task queues with two workers (step_processing + asset_pipeline)
|
||||
- WebSocket real-time events via Redis Pub/Sub
|
||||
- React/Vite frontend with workflow editor (ReactFlow), media browser, notifications
|
||||
|
||||
@@ -584,7 +584,7 @@ internal teams), RLS + tenant_id partitioning is sufficient.
|
||||
- Each tenant gets own PostgreSQL schema (not separate DB) with schema-based routing
|
||||
- Shared MinIO with per-tenant bucket policies
|
||||
- Separate Redis database (0-15) per tenant (max 16 tenants)
|
||||
- Celery routing: per-tenant queue prefix `{tenant_slug}.thumbnail_rendering`
|
||||
- Celery routing: per-tenant queue prefix `{tenant_slug}.asset_pipeline`
|
||||
|
||||
### 4.4 Per-Tenant Feature Flags
|
||||
|
||||
@@ -751,7 +751,7 @@ export const HELP_TEXTS: Record<string, HelpText> = {
|
||||
},
|
||||
"action.regenerate_thumbnails": {
|
||||
title: "Regenerate All Thumbnails",
|
||||
body: "Re-renders thumbnails for all STEP files using current settings. This queues all files on the thumbnail_rendering worker. Expected time: N × 30s. Only needed after changing renderer settings.",
|
||||
body: "Re-renders thumbnails for all STEP files using current settings. This queues all files on the asset_pipeline worker. Expected time: N × 30s. Only needed after changing renderer settings.",
|
||||
warning: "This will queue a large number of tasks. Only run during off-peak hours.",
|
||||
},
|
||||
// ... all settings
|
||||
@@ -889,7 +889,7 @@ rejection UI. `rejected_at` column exists but there is no rejection reason field
|
||||
### 8.1 Current Concurrency Controls
|
||||
|
||||
- `worker` (step_processing): `CELERY_WORKER_CONCURRENCY` env var, default 8
|
||||
- `render-worker` (thumbnail_rendering): hardcoded 1 (Blender serial access)
|
||||
- `render-worker` (asset_pipeline): hardcoded 1 (Blender serial access)
|
||||
- Both require Docker service restart to change concurrency
|
||||
|
||||
### 8.2 Dynamic Worker Scaling
|
||||
@@ -901,7 +901,7 @@ Use Celery's built-in `autoscale` option:
|
||||
render-worker:
|
||||
command: celery -A app.tasks.celery_app worker
|
||||
--loglevel=info
|
||||
-Q thumbnail_rendering
|
||||
-Q asset_pipeline
|
||||
--autoscale=1,1 # min=1, max=1 (single Blender concurrency)
|
||||
--concurrency=1
|
||||
```
|
||||
@@ -984,7 +984,7 @@ After Phase 2 decomposition, update `celery_app.conf.update(task_routes={...})`:
|
||||
```python
|
||||
task_routes = {
|
||||
"app.domains.pipeline.tasks.*": {"queue": "step_processing"},
|
||||
"app.domains.rendering.tasks.*": {"queue": "thumbnail_rendering"},
|
||||
"app.domains.rendering.tasks.*": {"queue": "asset_pipeline"},
|
||||
"app.domains.media.tasks.*": {"queue": "step_processing"},
|
||||
"app.tasks.ai_tasks.*": {"queue": "ai_validation"},
|
||||
"app.tasks.beat_tasks.*": {"queue": "step_processing"},
|
||||
|
||||
Reference in New Issue
Block a user