refactor(P1): M1 dead code removal + M3 blender_render.py split
M1 — dead code removed: - Delete blender-renderer/ and threejs-renderer/ source files - Remove PIL/Pillow fallback block from step_processor.py (_generate_thumbnail_placeholder, _finalise_image JPG path) - Remove stl_quality param from render_blender.py, render_still_task, render_turntable_task (was always "low"; hardcode deflection values) - render_turntable_task now reads scene_linear/angular_deflection from system_settings (consistent with export_glb.py pipeline) M3 — blender_render.py split from 263 → 68 lines: - _blender_args.py: parse_args() — all 25 positional + named args - _blender_scene_setup.py: setup_scene() — MODE A/B including USD import - _blender_render_config.py: configure_and_render() — engine + output Post-review fixes: - _db_engine.dispose() after settings read in render_turntable_task - _finalise_image() fmt param removed (always PNG; PIL never installed) - _blender_import.py committed together with new submodules to satisfy import_usd_file dependency Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,183 +1,88 @@
|
||||
# Plan: Priority 1 — Pipeline Cleanup (M1 Dead Code + M3 blender_render Split)
|
||||
# Plan: P1 Remaining Cleanup — M1 Dead Code + M3 blender_render.py Split
|
||||
|
||||
## Context
|
||||
Three categories of cleanup:
|
||||
1. **M1a**: Two legacy HTTP renderer directories (`blender-renderer/`, `threejs-renderer/`) still exist in repo root despite the services being removed in Phase A.
|
||||
2. **M1b**: Dead code in backend services — PIL fallback in `step_processor.py`, `stl_quality` param (always "low") in `render_blender.py` and `domains/rendering/tasks.py`.
|
||||
3. **M3**: `render-worker/scripts/blender_render.py` is 263 lines (target < 80) — argparse, scene setup, and render config should move to submodules.
|
||||
|
||||
ROADMAP Priority 1 is "In Progress". M2 (`step_tasks.py` decomposed to `domains/pipeline/tasks/`) is **done**. Two milestones remain:
|
||||
|
||||
- **M1**: Delete dead-code directories, remove `stl_quality` from admin/frontend surface, remove dead functions
|
||||
- **M3**: Decompose `blender_render.py` (920 lines) into focused submodules
|
||||
|
||||
---
|
||||
`domains/rendering/tasks.py` is **NOT dead code** — contains 6 active Celery tasks (`render_still_task`, `render_turntable_task`, `render_order_line_still_task`, `export_gltf_for_order_line_task`, `export_blend_for_order_line_task`, `apply_asset_library_materials_task`). Only the `stl_quality` param needs removal.
|
||||
|
||||
## Affected Files
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `blender-renderer/` | DELETE directory |
|
||||
| `threejs-renderer/` | DELETE directory |
|
||||
| `renderproblems_tmp/` | DELETE directory |
|
||||
| `backend/app/api/routers/admin.py` | Remove `stl_quality` + `VALID_STL_QUALITIES` (7 locations) |
|
||||
| `frontend/src/api/orders.ts` | Remove `stl_quality?: string` |
|
||||
| `frontend/src/api/worker.ts` | Remove `stl_quality?: string` |
|
||||
| `frontend/src/pages/WorkerActivity.tsx` | Remove STL quality KV row |
|
||||
| `frontend/src/components/renders/RenderInfoModal.tsx` | Remove STL quality display row |
|
||||
| `frontend/src/help/helpTexts.ts` | Remove `setting.stl_quality` entry |
|
||||
| `backend/app/services/step_processor.py` | Remove `_render_via_service()` + dead `elif renderer == "threejs"` |
|
||||
| `render-worker/scripts/blender_render.py` | Remove `_mark_sharp_and_seams()`; thin to entry-point after submodule extraction |
|
||||
| `render-worker/scripts/_blender_gpu.py` | CREATE — `activate_gpu()` |
|
||||
| `render-worker/scripts/_blender_import.py` | CREATE — `import_glb()`, `apply_rotation()` |
|
||||
| `render-worker/scripts/_blender_materials.py` | CREATE — `build_mat_map_lower()`, `apply_material_library()`, `assign_failed_material()` |
|
||||
| `render-worker/scripts/_blender_camera.py` | CREATE — `setup_auto_camera()`, `setup_auto_lights()` |
|
||||
| `render-worker/scripts/_blender_scene.py` | CREATE — `ensure_collection()`, `apply_smooth_batch()`, `apply_sharp_edges_from_occ()`, `setup_shadow_catcher()` |
|
||||
|
||||
---
|
||||
- `blender-renderer/` — delete entire directory
|
||||
- `threejs-renderer/` — delete entire directory
|
||||
- `backend/app/services/step_processor.py` — remove PIL fallback block (~line 565)
|
||||
- `backend/app/services/render_blender.py` — remove `stl_quality` param from `_glb_from_step()`, `render_still()`, `render_turntable_to_file()`
|
||||
- `backend/app/domains/rendering/tasks.py` — remove `stl_quality` param from `render_still_task`, `render_turntable_task`
|
||||
- `render-worker/scripts/blender_render.py` — thin to < 80 lines
|
||||
- `render-worker/scripts/_blender_args.py` — new file (argument parsing)
|
||||
- `render-worker/scripts/_blender_scene_setup.py` — new file (MODE A/B scene setup)
|
||||
- `render-worker/scripts/_blender_render_config.py` — new file (engine + output config)
|
||||
|
||||
## Tasks (in order)
|
||||
|
||||
### [x] Task M1-1: Delete obsolete directories
|
||||
|
||||
- **What**: `rm -rf blender-renderer/ threejs-renderer/ renderproblems_tmp/`
|
||||
- **Acceptance gate**: `ls blender-renderer/ threejs-renderer/ renderproblems_tmp/` → all "No such file"
|
||||
### [x] Task 1: Delete legacy renderer directories
|
||||
- **File**: `blender-renderer/`, `threejs-renderer/` (repo root)
|
||||
- **What**: `git rm -rf blender-renderer/ threejs-renderer/` — removes both legacy HTTP service directories superseded by the Celery render-worker in Phase A
|
||||
- **Acceptance gate**: `ls blender-renderer/ threejs-renderer/` both return "no such file or directory"
|
||||
- **Dependencies**: none
|
||||
- **Risk**: Zero — no active source files
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M1-2: Remove stl_quality from admin.py
|
||||
|
||||
- **File**: `backend/app/api/routers/admin.py`
|
||||
- **What**: Delete all 7 references:
|
||||
1. `VALID_STL_QUALITIES = {"low", "high"}` constant
|
||||
2. `"stl_quality": "low"` from `SETTINGS_DEFAULTS`
|
||||
3. `stl_quality: str = "low"` from `SettingsOut`
|
||||
4. `stl_quality: str | None = None` from `SettingsUpdate`
|
||||
5. `stl_quality=raw["stl_quality"],` from `_settings_to_out()`
|
||||
6. `if body.stl_quality is not None and body.stl_quality not in VALID_STL_QUALITIES:` validation block
|
||||
7. `if body.stl_quality is not None: updates["stl_quality"] = body.stl_quality` update block
|
||||
- **Acceptance gate**: `grep -n "stl_quality\|VALID_STL_QUALITIES" backend/app/api/routers/admin.py` → 0 matches
|
||||
- **Dependencies**: none
|
||||
- **Risk**: Low — the DB key remains (harmless); pipeline internally still uses `gltf_*_linear_deflection`
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M1-3: Remove stl_quality from frontend
|
||||
|
||||
- **Files**:
|
||||
- `frontend/src/api/orders.ts` — remove `stl_quality?: string`
|
||||
- `frontend/src/api/worker.ts` — remove `stl_quality?: string`
|
||||
- `frontend/src/pages/WorkerActivity.tsx` — remove STL quality KV row
|
||||
- `frontend/src/components/renders/RenderInfoModal.tsx` — remove STL quality row
|
||||
- `frontend/src/help/helpTexts.ts` — remove `setting.stl_quality` entry
|
||||
- **Acceptance gate**: `grep -rn "stl_quality" frontend/src/` → 0 matches; `npx tsc --noEmit` passes
|
||||
- **Dependencies**: M1-2
|
||||
- **Risk**: Low — all uses are optional fields (`?:`)
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M1-4: Remove dead _mark_sharp_and_seams from blender_render.py
|
||||
|
||||
- **File**: `render-worker/scripts/blender_render.py`
|
||||
- **What**: Delete the `_mark_sharp_and_seams()` function (lines 196–256 approx). It is defined but never called — `_apply_sharp_edges_from_occ()` is the active implementation.
|
||||
- **Acceptance gate**: `grep -n "_mark_sharp_and_seams" render-worker/scripts/blender_render.py` → 0 matches
|
||||
- **Dependencies**: none
|
||||
- **Risk**: Zero — verifiably never called
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M1-5: Remove dead code from step_processor.py
|
||||
- **Risk**: Low — not imported by any active pipeline code
|
||||
|
||||
### [x] Task 2: Remove PIL fallback from step_processor.py
|
||||
- **File**: `backend/app/services/step_processor.py`
|
||||
- **What**: Delete `_render_via_service()` function and the `elif renderer == "threejs":` branch (which only logs a warning and falls through)
|
||||
- **Acceptance gate**: `grep -n "_render_via_service\|renderer == .threejs" backend/app/services/step_processor.py` → 0 matches
|
||||
- **Dependencies**: M1-1
|
||||
- **Risk**: Low — function is only referenced from within the dead branch
|
||||
- **What**: Find `from PIL import Image` (~line 565, inside `_generate_thumbnail()`) and the PIL thumbnail generation conditional branch. Remove the import and the branch — leave only the render-worker path.
|
||||
- **Acceptance gate**: `grep -n "PIL\|Pillow" backend/app/services/step_processor.py` returns nothing
|
||||
- **Dependencies**: none
|
||||
- **Risk**: Low — PIL path unreachable; render-worker handles all thumbnails
|
||||
|
||||
---
|
||||
### [x] Task 3: Remove stl_quality param from render_blender.py
|
||||
- **File**: `backend/app/services/render_blender.py`
|
||||
- **What**:
|
||||
- `_glb_from_step(step_path, output_dir, quality="low")` → `_glb_from_step(step_path, output_dir)` — hardcode the low-quality deflection values inline (no conditional on quality)
|
||||
- Remove `stl_quality: str = "low"` from `render_still(...)` and `render_turntable_to_file(...)`
|
||||
- Remove all internal `quality=stl_quality` pass-throughs
|
||||
- **Acceptance gate**: `grep -n "stl_quality" backend/app/services/render_blender.py` returns nothing
|
||||
- **Dependencies**: none (Task 4 updates callers)
|
||||
- **Risk**: Medium — callers in tasks.py pass `stl_quality`; update in Task 4 immediately after
|
||||
|
||||
### [x] Task M3-1: Create _blender_gpu.py
|
||||
### [x] Task 4: Remove stl_quality param from domains/rendering/tasks.py
|
||||
- **File**: `backend/app/domains/rendering/tasks.py`
|
||||
- **What**:
|
||||
- `render_still_task` (~line 48): remove `stl_quality: str = "low"` from signature and from the `render_still(...)` call
|
||||
- `render_turntable_task` (~line 152): remove `stl_quality: str = "low"` from signature. Lines ~210–228 inline OCC GLB generation reads `stl_quality` to choose deflection values — replace hardcoded quality-based values with DB settings reads (`scene_linear_deflection`, `scene_angular_deflection`). Pattern to follow: `export_glb.py` reads these settings via `sys_settings.get("scene_linear_deflection", 0.03)`.
|
||||
- **Acceptance gate**: `grep -n "stl_quality" backend/app/domains/rendering/tasks.py` returns nothing
|
||||
- **Dependencies**: Task 3
|
||||
- **Risk**: Medium — inline tessellation block must correctly read DB settings; verify key names match migration 062 output
|
||||
|
||||
- **File**: `render-worker/scripts/_blender_gpu.py` (NEW)
|
||||
- **What**: Extract `_activate_gpu()` from `blender_render.py` into a standalone module. Refactor to accept `cycles_device: str` parameter instead of reading a module-level global. Rename to `activate_gpu()`.
|
||||
- **Key signature**: `def activate_gpu(cycles_device: str = "auto") -> str | None`
|
||||
- **Acceptance gate**: `grep -c "def _activate_gpu" render-worker/scripts/blender_render.py` → 0; function callable as `from _blender_gpu import activate_gpu`
|
||||
- **Dependencies**: M1-4
|
||||
- **Risk**: Medium — must pass `sys.path` correctly so Blender Python finds the module
|
||||
### [x] Task 5: Extract _blender_args.py
|
||||
- **File**: `render-worker/scripts/blender_render.py`, new `render-worker/scripts/_blender_args.py`
|
||||
- **What**: Move the `argparse` block (lines ~44–110, ~67 lines) into `_blender_args.py` as a `parse_args()` function. `blender_render.py` calls `from _blender_args import parse_args` and uses `args = parse_args()`.
|
||||
- **Acceptance gate**: `_blender_args.py` exists with the parser; `blender_render.py` line count drops by ~60
|
||||
- **Dependencies**: none
|
||||
- **Risk**: Low — pure refactor, no logic change
|
||||
|
||||
---
|
||||
### [x] Task 6: Extract _blender_scene_setup.py
|
||||
- **File**: `render-worker/scripts/blender_render.py`, new `render-worker/scripts/_blender_scene_setup.py`
|
||||
- **What**: Move the MODE A / MODE B scene setup branches (lines ~131–214, ~84 lines) into `_blender_scene_setup.py` as `setup_scene(args, scene)` (dispatches internally to mode A or B based on `args.blend_template`). Import and call in `blender_render.py`.
|
||||
- **Acceptance gate**: `_blender_scene_setup.py` exists; `blender_render.py` line count drops by ~80
|
||||
- **Dependencies**: Task 5
|
||||
- **Risk**: Low — pure refactor; `bpy` available in Blender Python context
|
||||
|
||||
### [x] Task M3-2: Create _blender_import.py
|
||||
|
||||
- **File**: `render-worker/scripts/_blender_import.py` (NEW)
|
||||
- **What**: Extract `_import_glb()` and `_apply_rotation()` into module. Rename to `import_glb()` / `apply_rotation()`.
|
||||
- **Acceptance gate**: `grep -c "def _import_glb\|def _apply_rotation" render-worker/scripts/blender_render.py` → 0
|
||||
- **Dependencies**: M1-4
|
||||
- **Risk**: Low — no hidden globals beyond `bpy`, `math`, `Vector`
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M3-3: Create _blender_materials.py
|
||||
|
||||
- **File**: `render-worker/scripts/_blender_materials.py` (NEW)
|
||||
- **What**: Extract `_assign_failed_material()`, `_apply_material_library()`, and the `mat_map_lower` building loop. Consolidate the duplicated `mat_map_lower` logic (currently in Mode A and Mode B) into a single `build_mat_map_lower()` helper. `FAILED_MATERIAL_NAME` constant lives here.
|
||||
- **Acceptance gate**: `grep -c "def _assign_failed_material\|def _apply_material_library" render-worker/scripts/blender_render.py` → 0
|
||||
- **Dependencies**: M1-4
|
||||
- **Risk**: Medium — `_apply_material_library()` currently uses `part_names_ordered` global; must convert to parameter
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M3-4: Create _blender_camera.py
|
||||
|
||||
- **File**: `render-worker/scripts/_blender_camera.py` (NEW)
|
||||
- **What**: Extract auto-camera placement block (bounding sphere computation, isometric positioning, clip plane setup, `ELEVATION_DEG`/`AZIMUTH_DEG` constants) and `setup_auto_lights()`.
|
||||
- **Key signatures**: `def setup_auto_camera(parts, width, height) -> tuple[Vector, float]` (returns center + radius for reuse by lights); `def setup_auto_lights(bbox_center, bsphere_radius) -> None`
|
||||
- **Acceptance gate**: `grep -c "ELEVATION_DEG\|AZIMUTH_DEG\|bsphere_radius" render-worker/scripts/blender_render.py` → 0
|
||||
- **Dependencies**: M3-2
|
||||
- **Risk**: Low — camera block is self-contained
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M3-5: Create _blender_scene.py
|
||||
|
||||
- **File**: `render-worker/scripts/_blender_scene.py` (NEW)
|
||||
- **What**: Extract `_ensure_collection()`, `_apply_smooth_batch()`, `_apply_sharp_edges_from_occ()`, shadow catcher setup into module.
|
||||
- **Acceptance gate**: `grep -c "def _ensure_collection\|def _apply_smooth_batch\|def _apply_sharp_edges_from_occ" render-worker/scripts/blender_render.py` → 0
|
||||
- **Dependencies**: M1-4
|
||||
- **Risk**: Low
|
||||
|
||||
---
|
||||
|
||||
### [x] Task M3-6: Thin blender_render.py to entry-point
|
||||
|
||||
- **File**: `render-worker/scripts/blender_render.py`
|
||||
- **What**: Replace all extracted function bodies with imports from submodules. Add `sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))` before imports so Blender Python finds the submodules. Result: argument parsing + Mode A/B orchestration + timing only. Target: < 200 lines.
|
||||
- **Acceptance gate**: `wc -l render-worker/scripts/blender_render.py` → < 200; upload `81113-l_cut.stp` → thumbnail renders correctly
|
||||
- **Dependencies**: M3-1, M3-2, M3-3, M3-4, M3-5
|
||||
- **Risk**: High (integration step) — test immediately after deploy
|
||||
|
||||
---
|
||||
### [x] Task 7: Extract _blender_render_config.py and verify ≤ 80 lines
|
||||
- **File**: `render-worker/scripts/blender_render.py`, new `render-worker/scripts/_blender_render_config.py`
|
||||
- **What**: Move engine/render settings + output path logic (lines ~216–258, ~43 lines) into `_blender_render_config.py` as `configure_render(scene, args, output_path, gpu_type)`. After extraction, `blender_render.py` must be ≤ 80 lines.
|
||||
- **Acceptance gate**: `wc -l render-worker/scripts/blender_render.py` shows ≤ 80
|
||||
- **Dependencies**: Task 6
|
||||
- **Risk**: Low — pure refactor
|
||||
|
||||
## Migration Check
|
||||
|
||||
**No migration required.** `stl_quality` key stays in DB (harmless). No new columns or tables.
|
||||
|
||||
---
|
||||
No new Alembic migration required. Task 4 reads existing keys (`scene_linear_deflection`, `scene_angular_deflection`) from the `system_settings` table, already present after migration 062.
|
||||
|
||||
## Order Recommendation
|
||||
|
||||
```
|
||||
M1-1 (delete dirs) → M1-4 (dead func blender) → M1-5 (dead func step_processor)
|
||||
→ M1-2 (admin.py) → M1-3 (frontend)
|
||||
→ M3-1..M3-5 (create submodules in parallel where possible)
|
||||
→ M3-6 (thin blender_render.py — integration, highest risk, test immediately)
|
||||
```
|
||||
|
||||
Deploy after M1: `docker compose up -d --build backend`
|
||||
Deploy after M3-6: `docker compose up -d --build render-worker`
|
||||
|
||||
---
|
||||
Tasks 1 and 2 are independent — can run in parallel.
|
||||
Tasks 3 and 4 are coupled — run 3 immediately before 4.
|
||||
Tasks 5, 6, 7 are sequential — each further reduces blender_render.py line count.
|
||||
|
||||
## Risks / Open Questions
|
||||
|
||||
1. **Blender `sys.path`**: Submodule files must be at `/render-scripts/` (the volume mount path). `sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))` is the safe way to ensure this regardless of CWD.
|
||||
2. **`part_names_ordered` global**: Currently used across multiple functions in `blender_render.py`. Must be explicitly passed as a parameter to `apply_material_library()` in M3-3.
|
||||
3. **M3 scope**: M3 is a pure refactor — no behaviour change. If time is limited, M1 (dead code removal) delivers clean value on its own. M3 can be deferred to a separate session.
|
||||
- `render_turntable_task` inline tessellation: confirm exact key names are `scene_linear_deflection` / `scene_angular_deflection` (not the old `gltf_preview_*` names) by reading `export_glb.py` before Task 4.
|
||||
- After Task 7, do a smoke-test render to confirm submodule imports work inside Blender's Python interpreter.
|
||||
|
||||
Reference in New Issue
Block a user