Hartmut/HartOMat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(phase8.1-8.2): dynamic worker concurrency via worker_configs

Browse Source 
- Migration 054: worker_configs table (queue_name PK, max/min_concurrency,
  enabled, updated_at); seeds step_processing(8/2), thumbnail_rendering(1/1),
  ai_validation(4/1)
- WorkerConfig SQLAlchemy model
- apply_worker_concurrency beat task: reads enabled configs, broadcasts
  pool_grow to all Celery workers every 5min
- GET/PUT /api/worker/configs (admin): list + update per-queue concurrency
- docker-compose.yml: worker uses --autoscale=${MAX_CONCURRENCY:-8},${MIN_CONCURRENCY:-2};
  render-worker uses --autoscale=1,1 --concurrency=1
- WorkerManagement.tsx: "Concurrency Settings" section with +/- steppers
  and Save button per queue

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Hartmut Nörenberg
2026-03-08 20:41:57 +01:00
parent b41e70cdad
commit 07e3d1e026
9 changed files with 344 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docker-compose.yml
+2 -1
View File
@@ -87,7 +87,7 @@ services:
build:
context: ./backend
dockerfile: Dockerfile
command: celery -A app.tasks.celery_app worker --loglevel=info -Q step_processing,ai_validation --concurrency=${CELERY_WORKER_CONCURRENCY:-8}
command: celery -A app.tasks.celery_app worker --loglevel=info -Q step_processing,ai_validation --autoscale=${MAX_CONCURRENCY:-8},${MIN_CONCURRENCY:-2} --concurrency=${MIN_CONCURRENCY:-2}
environment:
- POSTGRES_DB=${POSTGRES_DB:-schaeffler}
- POSTGRES_USER=${POSTGRES_USER:-schaeffler}
@@ -121,6 +121,7 @@ services:
dockerfile: render-worker/Dockerfile
args:
- BLENDER_VERSION=${BLENDER_VERSION:-5.0.1}
command: bash -c "python3 /check_version.py && celery -A app.tasks.celery_app worker --loglevel=info -Q thumbnail_rendering --autoscale=1,1 --concurrency=1"
environment:
- POSTGRES_DB=${POSTGRES_DB:-schaeffler}
- POSTGRES_USER=${POSTGRES_USER:-schaeffler}