chore: snapshot workflow migration progress
This commit is contained in:
@@ -147,11 +147,101 @@ Ergebnis:
|
||||
- Abgeschlossen: Block 6
|
||||
- Abgeschlossen: Block 7
|
||||
- Abgeschlossen: Block 8
|
||||
- In Arbeit: Block 9
|
||||
- Nächster geplanter Folgeblock: Block 9
|
||||
- Abgeschlossen: Block 9
|
||||
- Parallel in Arbeit: Block 11
|
||||
- Vorbereitet: Block 12
|
||||
- Nächster geplanter Folgeblock: Block 11
|
||||
|
||||
## Nächste Orchestrierte Batch-Wellen
|
||||
|
||||
Diese Wellen priorisieren Root-Cause-Arbeit vor weiterer UI-Politur und halten Legacy jederzeit parallel funktionsfähig.
|
||||
|
||||
### Welle P1: Vertrags- und Produktionsmodell-Schließung
|
||||
|
||||
Muss zuerst laufen:
|
||||
|
||||
- `P1-A` Node-Contract Closure
|
||||
- Backend-Registry und Schema als harte Source of Truth schließen
|
||||
- Fokus: Family-Konsistenz, param-key-Validierung, vollständige node settings contracts
|
||||
- `P1-B` Output-Type / Invocation Model Closure
|
||||
- Output Types als sauberes Workflow-Invocation-Modell abschließen
|
||||
- Fokus: artifact/family compatibility, editor/API contract clarity, sichere Erstellung neuer Output Types
|
||||
- `P1-C` Render-Template- und Asset-Library-Inputs als echte Produktionsinputs modellieren
|
||||
- Fokus: template/material-library/input contracts statt versteckter Defaults
|
||||
|
||||
Warum zuerst:
|
||||
|
||||
- diese drei Blöcke definieren die autoritativen Verträge, an denen Editor, Runtime und Golden-Gates hängen
|
||||
- weitere Runtime- und E2E-Arbeit bleibt sonst drift-anfällig
|
||||
|
||||
### Welle P2: Runtime-Parität und Graph/Legacy-Unifikation
|
||||
|
||||
Parallelisierbar nach P1-A:
|
||||
|
||||
- `P2-A` Legacy/Graph Module Unification
|
||||
- gleiche Produktionsmodule in Legacy- und Graph-Pfad verwenden
|
||||
- Fokus: template resolution, samples/defaults, dispatch parity
|
||||
- `P2-B` Canonical Graphs / Starter Blueprints / Seed Normalization
|
||||
- eine kanonische Graph-Quelle statt Drift zwischen backend blueprints, bundles und frontend starters
|
||||
- `P2-C` Run Inspection Completion
|
||||
- Preflight, dispatch, comparison und node outputs operativ debugbar machen
|
||||
|
||||
### Welle P3: CAD-/Material-Parität und Editor-Führung
|
||||
|
||||
Parallelisierbar nach P2-A:
|
||||
|
||||
- `P3-A` CAD / Material Parity
|
||||
- instance-aware part/material truth zwischen exporter, viewer und render path schließen
|
||||
- `P3-B` Editor Organization Around Modules / Families / Input Paths
|
||||
- gemeinsame Authoring-Surface weiter auf modulare Produktionspfade zuschneiden
|
||||
- `P3-C` Context Flow Simplification Follow-up
|
||||
- Kontextauswahl und Output-Type-Einstieg auf die neuen Contracts ausrichten
|
||||
|
||||
### Welle P4: Operative Freigabe und Hygiene
|
||||
|
||||
Nach P2 und P3:
|
||||
|
||||
- `P4-A` Shadow / Graph Rollout Hardening
|
||||
- pro workflow / output type steuerbar, mit klarem Rückfallpfad
|
||||
- `P4-B` Sequential Golden / Smoke / E2E Gates
|
||||
- echte Produktionsfälle mit Templates, Varianten und Output-Types sequenziell absichern
|
||||
- `P4-C` Test-Infrastruktur / Low-RAM Gates
|
||||
- reproduzierbare sequentielle Verifikation
|
||||
- `P4-D` Repo Hygiene / Generated Artifact Root Cause
|
||||
- Ownership-, Pycache- und generated-file-Ursachen bereinigen
|
||||
|
||||
## Sofort Nächste Disjunkte Arbeitsblöcke
|
||||
|
||||
Für die aktuelle nächste Ausführung werden diese drei Blöcke als kleinste sinnvolle Parallel-Batch vorbereitet:
|
||||
|
||||
- Batch `NB-1`: Node-Contract Closure
|
||||
- Status: verifiziert
|
||||
- Fokus: Registry- und Schema-Contracts als harte Source of Truth
|
||||
- Verifikation: `backend/tests/domains/test_workflow_schema.py`, `backend/tests/domains/test_workflow_node_registry.py`
|
||||
- Batch `NB-2`: Output-Type / Invocation Model Closure
|
||||
- Status: verifiziert
|
||||
- Fokus: Artifact-/Family-/Invocation-Contracts fuer Output Types
|
||||
- Verifikation: `backend/tests/domains/test_output_types_api.py`, `frontend/src/__tests__/api/outputTypes.test.ts`
|
||||
- Batch `NB-3`: CAD / Material parity root-cause closure
|
||||
- Status: verifiziert
|
||||
- Fokus: part-key-/instance-stabile Materialidentitaet zwischen Exporter, Manifest und Viewer
|
||||
- Teilfortschritt April 10, 2026: scene-manifest aliasiert jetzt auch exporter-variant keys wie `_af6` auf ihren kanonischen semantischen part key; der Viewer kann damit dieselbe autoritative Materialidentitaet konsumieren wie der Manifest-Pfad
|
||||
- Verifikation: gezielte low-RAM CAD-/Viewer-Tests nach Root-Cause-Fix
|
||||
- Abschluss April 11, 2026: live HartOMat-Export fuer `7c214057-9982-4d6e-aa87-43bfabfdb709` liefert jetzt `146` Manifest-Parts, `146` Mesh-Nodes, `146` eindeutige `partKey`s, `0` fehlende und `0` duplizierte Zuordnungen; Root Cause war die Kombination aus stale GLB cache plus nicht-atomarem OCC-Overwrite beim Re-Export
|
||||
|
||||
Merge-Reihenfolge:
|
||||
|
||||
1. `NB-1`
|
||||
2. `NB-2`
|
||||
3. `NB-3`
|
||||
4. danach erst weitere Runtime-/Editor-Folgeblöcke
|
||||
|
||||
## Letzte Verifikation
|
||||
|
||||
- `./scripts/repo_hygiene.sh`
|
||||
- Ergebnis: Dry-run listet bereinigbare Cache-/Bytecode-Artefakte plus nicht dem aktuellen Nutzer gehörende Generated Files; die Repo-Hygiene deckt jetzt auch `render-worker/scripts/__pycache__` explizit ab
|
||||
- `find . \! -user "$USER" -not -path './.git/*' -ls | sed -n '1,120p'`
|
||||
- Ergebnis: verbleibende Ownership-Reste liegen im `render-worker`-Pycache; Compose-Härtung wird nun über `PYTHONPYCACHEPREFIX=/tmp/pycache` auf die Ursache angesetzt
|
||||
- `backend/.venv/bin/pytest backend/tests/test_config_runtime_resolution.py -q`
|
||||
- Ergebnis: 3 Tests grün; Host-Runtime normalisiert Docker-Service-Aliase (`postgres`, `redis`) außerhalb von Containern nun automatisch auf `localhost`, Container-Runtime bleibt unverändert
|
||||
- `backend/.venv/bin/pytest backend/tests/domains/test_workflow_runtime_services.py -q -x`
|
||||
@@ -166,5 +256,17 @@ Ergebnis:
|
||||
- Ergebnis: 5 Tests grün; autoritative Scene-Manifest-Zuweisungen werden nun im Workflow-Renderpfad auf `part_key` und `source_name` gespiegelt, Legacy-Fallback bleibt unverändert
|
||||
- `./backend/.venv/bin/pytest backend/tests/test_part_key_service.py -q`
|
||||
- Ergebnis: 1 Test grün; part-key-basierte Manifest-Auflösung bleibt konsistent
|
||||
- `python3 scripts/compare_live_cad_parity.py --cad-id 7c214057-9982-4d6e-aa87-43bfabfdb709`
|
||||
- Ergebnis: Live-CAD-Parität grün; Manifest, ausgeliefertes GLB und Viewer-`partKey`-Grundlage stimmen für alle 146 renderbaren Teile exakt überein
|
||||
- `cd frontend && npx vitest run src/__tests__/components/workflowEditorUi.test.tsx src/__tests__/api/outputTypes.test.ts --pool forks --poolOptions.forks.singleFork=true`
|
||||
- Ergebnis: 20 Tests grün, sequenziell ausgeführt
|
||||
- `cd frontend && npm test -- src/__tests__/components/workflowEditorUi.test.tsx src/__tests__/components/workflowAuthoringGuidance.test.ts`
|
||||
- Ergebnis: 17 Tests grün; die gemeinsame Authoring-Surface bleibt nach dem jüngsten Wiring-Refactor stabil
|
||||
- `cd frontend && npm run build`
|
||||
- Ergebnis: Build grün; `/workflows` bleibt kompilierbar nach dem Authoring-Refactor
|
||||
- `./backend/.venv/bin/pytest backend/tests/test_part_key_service.py -q`
|
||||
- Ergebnis: 6 Tests grün; scene-manifest deckt jetzt neben `_2`/`_3` auch exporter-style `_af*`-Varianten ueber semantische Alias-Keys ab
|
||||
- `./backend/.venv/bin/pytest backend/tests/test_export_step_to_gltf.py -q`
|
||||
- Ergebnis: 3 Tests grün; GLB partKey stamping bleibt mit semantic-sibling-Aufloesung stabil
|
||||
- `cd frontend && npm test -- src/__tests__/components/cadUtils.test.ts`
|
||||
- Ergebnis: 11 Tests grün; Viewer-seitige part-key-Aufloesung bleibt nach dem Manifest-Alias-Fix konsistent
|
||||
|
||||
@@ -0,0 +1,252 @@
|
||||
# First Wave Execution Plan
|
||||
|
||||
## Goal
|
||||
|
||||
Translate the worker orchestration into concrete first patch slices that can be executed in parallel without breaking legacy rendering.
|
||||
|
||||
This first wave is intentionally conservative:
|
||||
|
||||
- close contract holes before broad UI refactors
|
||||
- avoid cross-cutting runtime rewrites in the same slice
|
||||
- keep all graph changes legacy-safe
|
||||
|
||||
## Current Codebase Snapshot
|
||||
|
||||
### Block A: Node Contracts
|
||||
|
||||
Already in place:
|
||||
|
||||
- node definitions already expose `family`, `module_key`, `execution_kind`, `input_contract`, `output_contract`, `artifact_roles_*`, and `legacy_source`
|
||||
- frontend already consumes node definitions and uses family-aware palette grouping
|
||||
- `glb_bbox` already gained a real editor field for `glb_path`
|
||||
|
||||
Still missing:
|
||||
|
||||
- backend workflow schema validation is still DAG-structural only
|
||||
- backend does not yet validate graph family consistency against the registry
|
||||
- backend does not yet validate node params against registry-owned field definitions
|
||||
- editor-visible nodes with weak or absent settings still need an explicit audit, especially export/CAD bridge nodes
|
||||
- status note: the canonical still-path bridge nodes have now been expanded with real settings for template override, material override/disable, auto-populate persistence controls, GLB source preference, output-save artifact expectations, and notify arming. The remaining audit emphasis is export/CAD parity rather than still-path hidden defaults.
|
||||
|
||||
### Block D: Output-Type Invocation Contracts
|
||||
|
||||
Already in place:
|
||||
|
||||
- `workflow_family`, `artifact_kind`, and `invocation_overrides` exist
|
||||
- backend already blocks mixed-family workflow links and direct family mismatches
|
||||
- frontend admin UI already exposes family, artifact kind, and workflow selection
|
||||
|
||||
Still missing:
|
||||
|
||||
- workflow selection can still communicate contract semantics more clearly
|
||||
- status note: backend now rejects workflow-family mismatches, mixed-family workflow links, and workflow/artifact mismatches against the linked workflow graph. The admin form has also been re-ordered into workflow-contract and invocation-profile sections so renderer details no longer dominate the first screenful.
|
||||
|
||||
### Block E: Editor Organization
|
||||
|
||||
Already in place:
|
||||
|
||||
- right-click canvas insertion exists
|
||||
- searchable node command menu exists
|
||||
- split of `legacy` / `bridge` / `graph` node groupings exists
|
||||
- auto-align exists
|
||||
- edge deletion exists via button, Delete key, right-click, and double-click
|
||||
- the workflow toolbar has been compressed so context, mode, authoring actions, and run actions sit in one compact canvas-adjacent strip
|
||||
|
||||
Still missing:
|
||||
|
||||
- the editor is feature-rich but still structurally busy
|
||||
- node insertion, run inspection, and inspector controls need clearer hierarchy
|
||||
- UX cleanup should follow contract cleanup so the editor does not encode unstable assumptions
|
||||
|
||||
### Block I: Rollout And Regression Gates
|
||||
|
||||
Already in place:
|
||||
|
||||
- `legacy`, `graph`, and `shadow` dispatch modes exist
|
||||
- graph mode falls back to legacy on failure
|
||||
- shadow mode keeps legacy authoritative
|
||||
- workflow comparison endpoint exists with hash, dimensions, and mean pixel delta reporting
|
||||
- there is meaningful backend coverage for shadow dispatch and comparison behavior
|
||||
- the live rollout harness now exposes `--workflow-still-smoke` and `--workflow-golden-suite` paths for canonical still and representative graph cases
|
||||
|
||||
Still missing:
|
||||
|
||||
- per-workflow and per-output-type rollout enablement is still an operational step, not yet a guided product workflow
|
||||
- smoke and golden harnesses still depend on live stack health and seeded render fixtures, so parity coverage is not yet CI-grade deterministic
|
||||
|
||||
## Parallel Patch Slices
|
||||
|
||||
### Slice A1: Registry-Backed Schema Validation
|
||||
|
||||
Owner:
|
||||
|
||||
- Block A worker
|
||||
|
||||
Files:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_schema.py`
|
||||
- `backend/tests/domains/test_workflow_schema.py`
|
||||
- `backend/tests/domains/test_workflow_node_registry.py`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- add backend validation that all nodes in a graph belong to one family unless explicitly allowed for migration
|
||||
- validate that node params only use keys declared by the node registry
|
||||
- return clear validation errors that name the offending node id, step, and param key
|
||||
|
||||
Why this slice first:
|
||||
|
||||
- it closes the largest backend contract gap without changing runtime execution
|
||||
- it gives Block D and Block E a stable source of truth to build on
|
||||
|
||||
Acceptance:
|
||||
|
||||
- a mixed-family graph without migration exemption is rejected
|
||||
- unknown node param keys are rejected
|
||||
- current canonical still graph remains valid
|
||||
|
||||
### Slice A2: Export/CAD Contract Audit
|
||||
|
||||
Owner:
|
||||
|
||||
- Block A worker
|
||||
|
||||
Files:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_node_registry.py`
|
||||
- `backend/tests/domains/test_workflow_node_registry.py`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- audit `occ_glb_export`, `export_blend`, `thumbnail_save`, and `stl_cache_generate`
|
||||
- add real field definitions only where runtime behavior genuinely supports editable inputs
|
||||
- avoid fake settings just to make the editor look complete
|
||||
|
||||
Acceptance:
|
||||
|
||||
- each audited node either has a justified field schema or an explicit no-settings contract backed by tests
|
||||
|
||||
### Slice D1: Artifact/Family Validation Tightening
|
||||
|
||||
Owner:
|
||||
|
||||
- Block D worker
|
||||
|
||||
Files:
|
||||
|
||||
- `backend/app/domains/rendering/output_type_contracts.py`
|
||||
- `backend/app/api/routers/output_types.py`
|
||||
- `backend/tests/domains/test_output_types_api.py`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- define artifact-kind compatibility rules per workflow family
|
||||
- reject impossible combinations early in create/edit APIs
|
||||
- keep legacy output types renderable if they predate strict linkage
|
||||
|
||||
Acceptance:
|
||||
|
||||
- `cad_file` output types cannot declare order-line-only artifact kinds
|
||||
- invalid create/edit payloads fail with actionable errors
|
||||
- existing valid output types still load and render
|
||||
|
||||
### Slice D2: Workflow-First Output-Type Form Cleanup
|
||||
|
||||
Owner:
|
||||
|
||||
- Block D worker with light coordination from Block E
|
||||
|
||||
Files:
|
||||
|
||||
- `frontend/src/components/admin/OutputTypeTable.tsx`
|
||||
- `frontend/src/api/outputTypes.ts`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- reorder the form to lead with family, workflow, artifact kind, then invocation overrides
|
||||
- visually separate invocation profile fields from legacy compatibility fields
|
||||
- preserve current API payload shape
|
||||
|
||||
Acceptance:
|
||||
|
||||
- a new output type can be created from top to bottom as a workflow invocation profile
|
||||
- renderer-specific detail fields no longer dominate the first screenful
|
||||
- status: completed with a four-section admin form (`Workflow Contract`, `Invocation Profile`, `Renderer Compatibility / Legacy Details`, `Catalog / Business`) while preserving the existing API payload and legacy fallback behavior
|
||||
|
||||
### Slice E1: Workflow Editor Header Simplification
|
||||
|
||||
Owner:
|
||||
|
||||
- Block E worker
|
||||
|
||||
Files:
|
||||
|
||||
- `frontend/src/components/workflows/WorkflowCanvasToolbar.tsx`
|
||||
- `frontend/src/pages/WorkflowEditor.tsx`
|
||||
- `frontend/src/__tests__/components/workflowEditorUi.test.tsx`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- compress the oversized top area
|
||||
- move non-primary metadata into secondary badges or sidebar context
|
||||
- keep node insertion, align, save, dry-run, and run controls near the canvas
|
||||
|
||||
Acceptance:
|
||||
|
||||
- above-the-fold editor space is materially reduced
|
||||
- primary actions remain visible without scrolling
|
||||
- existing right-click and edge deletion behaviors remain intact
|
||||
|
||||
### Slice I1: Golden-Case Rollout Harness
|
||||
|
||||
Owner:
|
||||
|
||||
- Block I worker
|
||||
|
||||
Files:
|
||||
|
||||
- `scripts/test_render_pipeline.py`
|
||||
- `backend/tests/domains/test_workflow_dispatch_service.py`
|
||||
- `docs/workflows/WORKFLOW_DELIVERY_CHECKLIST.md`
|
||||
|
||||
Patch scope:
|
||||
|
||||
- add a canonical non-legacy still workflow smoke path to the render test script
|
||||
- record whether the run was legacy, graph, or shadow and surface comparison output when shadow is used
|
||||
- document the rollout gate needed before enabling graph mode on real output types
|
||||
|
||||
Acceptance:
|
||||
|
||||
- one command can exercise the canonical graph still path end to end
|
||||
- the script clearly reports whether rollout conditions were met or blocked
|
||||
|
||||
## Merge Order Inside First Wave
|
||||
|
||||
1. Slice A1
|
||||
2. Slice D1
|
||||
3. Slice A2
|
||||
4. Slice D2
|
||||
5. Slice E1
|
||||
6. Slice I1
|
||||
|
||||
## Integration Gates
|
||||
|
||||
### Gate FW-1
|
||||
|
||||
- Slice A1 merged
|
||||
- schema validation errors are deterministic and test-covered
|
||||
|
||||
### Gate FW-2
|
||||
|
||||
- Slice D1 merged
|
||||
- output-type API rules align with the tightened workflow contracts
|
||||
|
||||
### Gate FW-3
|
||||
|
||||
- Slice E1 merged
|
||||
- editor remains functionally complete after toolbar simplification
|
||||
|
||||
### Gate FW-4
|
||||
|
||||
- Slice I1 merged
|
||||
- canonical graph still smoke path is runnable and documented
|
||||
@@ -0,0 +1,399 @@
|
||||
# Next Batch Orchestration
|
||||
|
||||
## Goal
|
||||
|
||||
Define the next sensible implementation batch after the export/CAD contract audit, with work split for parallel execution and an integration order that keeps the legacy workflow operational.
|
||||
|
||||
## Current Batch
|
||||
|
||||
### Batch B1: Test Infrastructure Recovery
|
||||
|
||||
Purpose:
|
||||
Restore deterministic backend test execution so workflow parity work can be validated against real DB-backed tests again.
|
||||
|
||||
Why now:
|
||||
|
||||
- current targeted DB-backed tests fail on missing tables such as `users` and `cad_files`
|
||||
- this blocks trustworthy validation for further workflow runtime work
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/tests/**`
|
||||
- `backend/app/database.py`
|
||||
- `backend/app/config.py` only if required for test bootstrapping
|
||||
|
||||
Acceptance:
|
||||
|
||||
- targeted workflow tests create their schema reliably
|
||||
- DB-backed pytest runs do not fail due to missing core tables
|
||||
- no production runtime behavior changes unless strictly required for test setup correctness
|
||||
|
||||
### Batch B2: Workflow Editor Authoring Organization
|
||||
|
||||
Purpose:
|
||||
Reduce authoring friction in `/workflows` by tightening node organization around family, module, and execution role while reclaiming canvas space.
|
||||
|
||||
Why now:
|
||||
|
||||
- the editor already has the needed primitives
|
||||
- the remaining gap is structural clarity, not missing mechanics
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `frontend/src/components/workflows/**`
|
||||
- `frontend/src/pages/WorkflowEditor.tsx`
|
||||
- `frontend/src/__tests__/components/workflowEditorUi.test.tsx`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- top-area clutter is reduced
|
||||
- node discovery is cleaner by family/module grouping
|
||||
- right-click insertion, edge deletion, align, dry-run, and run inspection still work
|
||||
|
||||
### Batch B3: Canonical Still Path Smoke-Harness Closure
|
||||
|
||||
Purpose:
|
||||
Move the non-legacy still workflow closer to a runnable, documented smoke path without weakening legacy fallback.
|
||||
|
||||
Why now:
|
||||
|
||||
- backend graph/runtime coverage is already substantial
|
||||
- the next risk is proving that the canonical still graph can be exercised as a stable rollout candidate
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/**`
|
||||
- `backend/app/domains/pipeline/tasks/**`
|
||||
- `backend/tests/domains/test_workflow_*.py`
|
||||
- `scripts/test_render_pipeline.py`
|
||||
- `docs/workflows/**` where needed
|
||||
|
||||
Acceptance:
|
||||
|
||||
- canonical still graph path has a bounded smoke route
|
||||
- graph-vs-legacy safety remains explicit
|
||||
- remaining blockers are documented as concrete runtime or fixture issues, not vague parity claims
|
||||
|
||||
## Updated Immediate Next Batch
|
||||
|
||||
The next implementation batch should now be cut along contract and root-cause boundaries instead of UI-only slices.
|
||||
|
||||
### Batch N1: Node-Contract Closure
|
||||
|
||||
Purpose:
|
||||
Make the backend node registry and workflow schema the authoritative source for graph family safety, parameter validity, and editor-visible node settings.
|
||||
|
||||
Why now:
|
||||
|
||||
- authoring UX is already good enough to build on
|
||||
- remaining parity work depends on trustworthy backend-owned contracts
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_node_registry.py`
|
||||
- `backend/app/domains/rendering/workflow_schema.py`
|
||||
- `backend/tests/domains/test_workflow_node_registry.py`
|
||||
- `backend/tests/domains/test_workflow_schema.py`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- unknown node param keys are rejected deterministically
|
||||
- family drift is blocked by schema validation
|
||||
- every production-facing node has either a justified field schema or an explicit no-settings contract
|
||||
- existing canonical still graph remains valid
|
||||
|
||||
### Batch N2: Output-Type / Invocation Model Closure
|
||||
|
||||
Purpose:
|
||||
Finish the shift from legacy renderer flags to a real workflow invocation model with explicit family, artifact, and override semantics.
|
||||
|
||||
Why now:
|
||||
|
||||
- new output types and workflow-linked variants still depend on this contract being airtight
|
||||
- this is the clean boundary between product configuration and runtime dispatch
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/output_type_contracts.py`
|
||||
- `backend/app/api/routers/output_types.py`
|
||||
- `backend/app/domains/rendering/schemas.py`
|
||||
- `backend/app/domains/rendering/models.py`
|
||||
- `frontend/src/api/outputTypes.ts`
|
||||
- `frontend/src/components/admin/OutputTypeTable.tsx`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- impossible workflow/artifact/family combinations are rejected early
|
||||
- new output types can be created top-to-bottom as invocation profiles
|
||||
- legacy-safe output types continue to render
|
||||
|
||||
### Batch N3: CAD / Material Parity Root-Cause Closure
|
||||
|
||||
Purpose:
|
||||
Close the remaining instance- and part-key-related drift between CAD exporter, GLTF viewer, preview, and downstream render consumption.
|
||||
|
||||
Why now:
|
||||
|
||||
- this is still a real production blocker, not a polish item
|
||||
- workflow parity stays superficial until geometry/material identity is stable
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/pipeline/tasks/export_glb.py`
|
||||
- `backend/app/domains/pipeline/tasks/extract_metadata.py`
|
||||
- `backend/app/services/part_key_service.py`
|
||||
- `frontend/src/components/cad/cadUtils.ts`
|
||||
- `frontend/src/components/cad/ThreeDViewer.tsx`
|
||||
- `frontend/src/components/cad/InlineCadViewer.tsx`
|
||||
|
||||
Acceptance:
|
||||
|
||||
- viewer and manifest resolve the same authoritative material identity
|
||||
- unresolved nodes are surfaced explicitly instead of silently using pseudo keys
|
||||
- legacy preview and render behavior remain intact
|
||||
|
||||
## Parallelization Rule
|
||||
|
||||
These three blocks should be prepared in parallel, but merged in order:
|
||||
|
||||
1. `N1` first, because it establishes the source of truth for the other two.
|
||||
2. `N2` second, because it builds directly on those contracts.
|
||||
3. `N3` can be investigated in parallel, but should merge after `N1` unless it proves fully isolated.
|
||||
|
||||
## Gate For The Following Batch
|
||||
|
||||
For the updated immediate next batch, the following gate applies:
|
||||
|
||||
- `N1` has deterministic backend validation and focused tests
|
||||
- `N2` preserves legacy-safe output types while tightening impossible combinations
|
||||
- `N3` proves the authoritative part/material identity chain with a focused low-RAM verification sequence
|
||||
|
||||
## Current Execution Status
|
||||
|
||||
- `N1` verified
|
||||
- focused checks green on April 10, 2026:
|
||||
- `backend/tests/domains/test_workflow_schema.py`
|
||||
- `backend/tests/domains/test_workflow_node_registry.py`
|
||||
- `N2` verified
|
||||
- focused checks green on April 10, 2026:
|
||||
- `backend/tests/domains/test_output_types_api.py`
|
||||
- `frontend/src/__tests__/api/outputTypes.test.ts`
|
||||
- `N3` in progress
|
||||
- April 10, 2026:
|
||||
- scene-manifest alias coverage expanded for exporter `_af*` suffix keys
|
||||
- order-line runtime now prefers authoritative scene-manifest assignments where manifest metadata exists
|
||||
- inline and fullscreen viewers now share the same manifest-plus-fallback merge contract
|
||||
- unresolved meshes are now surfaced explicitly in both viewers instead of silently disappearing behind pseudo keys
|
||||
- output-type authoring now consumes a backend-authored contract catalog for family/artifact/format/override constraints
|
||||
- next action: manual product-level parity check plus B04 node/module contract completion
|
||||
|
||||
## Executable Block List
|
||||
|
||||
Die naechste sinnvolle Abarbeitung ist nicht mehr nach einzelnen Features, sondern nach stabilen Vertrags- und Produktionsgrenzen geschnitten.
|
||||
|
||||
### Batch Wave W1: Identity And Contract Closure
|
||||
|
||||
- `B01` CAD manifest alias closure
|
||||
- Ziel: scene-manifest und viewer auf denselben kanonischen semantischen part keys bringen
|
||||
- Fokus: exporter-style suffixe wie `_af6`, `_af0_asm`, dedup keys, alias inheritance
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
- Gate: `backend/tests/test_part_key_service.py`, `backend/tests/test_export_step_to_gltf.py`, `frontend/src/__tests__/components/cadUtils.test.ts`
|
||||
- `B02` CAD viewer/manifest convergence
|
||||
- Ziel: unresolved parts explizit sichtbar halten, aber alle autoritativ aufloesbaren parts im Viewer korrekt materialisieren
|
||||
- Fokus: `ThreeDViewer`, `InlineCadViewer`, logical part keys, reconciliation UX
|
||||
- Parallel zu: `B03`, `B04`
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
- April 10, 2026:
|
||||
- inline und fullscreen viewer auf denselben `buildEffectiveViewerMaterials(...)` contract gezogen
|
||||
- unresolved meshes werden explizit gezaehlt und sichtbar angezeigt statt pseudo-keys zu synthetisieren
|
||||
- focused gates gruen:
|
||||
- `frontend/src/__tests__/components/cadUtils.test.ts`
|
||||
- `npm run build`
|
||||
- `./scripts/workflow_sequential_gates.sh`
|
||||
- Gate: manueller Produktcheck gegen reales CAD-Beispiel
|
||||
- `B03` Output-type authoring closure
|
||||
- Ziel: neue output types workflow-first und ohne hidden legacy assumptions anlegbar machen
|
||||
- Fokus: form-state, invocation overrides, artifact/family guards, defaults
|
||||
- Parallel zu: `B02`, `B04`
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
- April 10, 2026:
|
||||
- backend publiziert `GET /api/output-types/contract-catalog` als read-only Source of Truth fuer Family-, Artifact-, Format- und Override-Regeln
|
||||
- `frontend/src/api/outputTypes.ts` nutzt den Catalog mit lokalem Fallback statt Hardcode-Regeltabellen als primaere Truth
|
||||
- `OutputTypeTable` speist Family-, Artifact- und Rollout-Auswahl jetzt aus dem Backend-Catalog
|
||||
- focused gates gruen:
|
||||
- `backend/tests/domains/test_output_types_api.py`
|
||||
- `frontend/src/__tests__/api/outputTypes.test.ts`
|
||||
- `frontend npm run build`
|
||||
- `./scripts/workflow_sequential_gates.sh`
|
||||
- Gate: `backend/tests/domains/test_output_types_api.py`, `frontend/src/__tests__/api/outputTypes.test.ts`
|
||||
- `B04` Node/module contract completion
|
||||
- Ziel: jede produktionsrelevante node hat einen klaren backend-owned settings/input/output contract
|
||||
- Fokus: registry metadata, no-settings contracts, family-safe validation
|
||||
- Gate: `backend/tests/domains/test_workflow_schema.py`, `backend/tests/domains/test_workflow_node_registry.py`
|
||||
|
||||
### Batch Wave W2: Canonical Authoring Surface
|
||||
|
||||
- `B05` Family-scoped node organization
|
||||
- Ziel: CAD, Bridge und Graph nodes im Editor klar trennen und suchbar halten
|
||||
- Fokus: family/module grouping, right-click search, low-noise discovery
|
||||
- Abhaengigkeit: `B04`
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
- April 10, 2026:
|
||||
- raw node catalog ist jetzt family-first organisiert: family -> module -> stage -> category
|
||||
- family/module runtime chips und stage scopes bleiben sichtbar, ohne zur stage-first Navigation zurueckzufallen
|
||||
- focused gates gruen:
|
||||
- `frontend/src/__tests__/components/workflowNodeCatalog.test.ts`
|
||||
- `frontend/src/__tests__/components/workflowEditorUi.test.tsx`
|
||||
- `frontend npm run build`
|
||||
- `B06` Authoring surface simplification
|
||||
- Ziel: eine gemeinsame authoring surface fuer canvas-menu, sidebar und starter paths
|
||||
- Fokus: shared controller/model statt mehrfacher UI-Sonderlogik
|
||||
- Abhaengigkeit: `B04`
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
- April 10, 2026:
|
||||
- `workflowAuthoringSurface.ts` kapselt jetzt shared section resolution, active-section validity und insert bindings als gemeinsame Surface-Controller-Logik
|
||||
- `NodeCommandMenu` und `NodeDefinitionsPanel` nutzen denselben Controller statt paralleler lokaler Section-/Insert-State-Implementierungen
|
||||
- focused gates gruen:
|
||||
- `frontend/src/__tests__/components/workflowEditorUi.test.tsx`
|
||||
- `frontend/src/__tests__/components/workflowAuthoringGuidance.test.ts`
|
||||
- `frontend npm run build`
|
||||
- `B07` Canvas ergonomics closure
|
||||
- Ziel: edge deletion, auto-align, reduced top-area clutter, faster insertion paths sauber zusammenziehen
|
||||
- Fokus: interaction consistency statt punktuelle UX-Patches
|
||||
- Abhaengigkeit: `B06`
|
||||
- Status: in Arbeit am April 10, 2026
|
||||
- April 10, 2026:
|
||||
- `WorkflowCanvasToolbar` auf kompaktere, wiederverwendbare Action-/Field-/Badge-Bausteine gezogen
|
||||
- Top-Area auf zwei dichtere Informationsreihen reduziert: Identitaet/Status oben, Context/Hint-Rail unten
|
||||
- focused gates gruen:
|
||||
- `frontend/src/__tests__/components/workflowEditorUi.test.tsx`
|
||||
- `frontend npm run build`
|
||||
- `B08` Starter graph and module bundle normalization
|
||||
- Ziel: starter blueprints, reference bundles und seed workflows auf dieselben kanonischen module paths ziehen
|
||||
- Fokus: still graph, CAD intake graph, bundle drift verhindern
|
||||
- Parallel zu: `B07`
|
||||
|
||||
### Batch Wave W3: Runtime And Production Parity
|
||||
|
||||
- `B09` Template-aware runtime unification
|
||||
- Ziel: legacy und graph nutzen dieselbe template/material/output orchestration
|
||||
- Fokus: template resolution, samples/transparency, publish semantics
|
||||
- Abhaengigkeit: `B03`, `B04`, `B08`
|
||||
- `B10` Non-legacy still smoke closure
|
||||
- Ziel: der kanonische still graph wird als wiederholbarer smoke path belastbar
|
||||
- Fokus: preflight, dispatch, authoritative output_save, failure visibility
|
||||
- Abhaengigkeit: `B09`
|
||||
- `B11` Template parity matrix
|
||||
- Ziel: graph vs legacy mit echten render-templates, output-varianten und alpha/sample settings vergleichen
|
||||
- Fokus: echte parity-beweise statt pillow-only checks
|
||||
- Abhaengigkeit: `B09`
|
||||
- `B12` CAD intake moduleization
|
||||
- Ziel: CAD import/extract/export/bbox/material-steps als echte workflow-module verfuegbar machen
|
||||
- Fokus: node-based production fuer intake workflows
|
||||
- Abhaengigkeit: `B04`, `B08`
|
||||
|
||||
### Batch Wave W4: Operational Rollout
|
||||
|
||||
- `B13` Rollout and fallback controls
|
||||
- Ziel: graph/shadow/legacy pro workflow und pro output type sicher steuerbar halten
|
||||
- Fokus: rollout mode, immediate rollback, operator clarity
|
||||
- Abhaengigkeit: `B10`, `B11`
|
||||
- `B14` Sequential E2E gates
|
||||
- Ziel: low-RAM, reproduzierbare smoke/golden/browser gates fuer `/workflows`
|
||||
- Fokus: sequenzielle statt parallele Verifikation
|
||||
- Abhaengigkeit: `B10`, `B11`, `B13`
|
||||
- `B15` Repo hygiene root-cause closure
|
||||
- Ziel: generated artifacts, root-owned caches und compose-side effects ursachenseitig schliessen
|
||||
- Fokus: ownership, pycache, build artifacts, helper script cleanup
|
||||
- Kann parallel laufen zu: `B10` bis `B14`
|
||||
|
||||
## Recommended Immediate Parallel Batch
|
||||
|
||||
Die naechste sinnvolle Batch mit minimalen Konflikten ist:
|
||||
|
||||
1. `B02` lokal auf explizite unresolved-state-Fuehrung und viewer-level parity checks ziehen
|
||||
2. `B03` parallel als contract-catalog dedup zwischen Backend und Admin-UI bearbeiten
|
||||
3. `B15` parallel als Hygiene-Nebenstrang treiben
|
||||
|
||||
Danach:
|
||||
|
||||
1. `B04`
|
||||
2. `B05` bis `B08` als Authoring-Welle
|
||||
3. `B09` bis `B12` als Runtime-/Produktionswelle
|
||||
|
||||
## Latest Verification
|
||||
|
||||
- April 10, 2026:
|
||||
- `./scripts/workflow_sequential_gates.sh` gruen
|
||||
- backend runtime gates: `34 passed`
|
||||
- frontend workflow/editor gates: `23 passed`
|
||||
|
||||
## Executable Next Queue
|
||||
|
||||
Die naechsten 12 Blöcke werden ab jetzt als eine gemeinsame Queue gefahren. Parallel bedeutet hier: Analyse, Vorbereitung und isolierte Teilpatches koennen parallel laufen. Merge und Verifikation bleiben bewusst sequentiell.
|
||||
|
||||
### Queue Q2: Merge Order
|
||||
|
||||
1. `B04-a` Node text-contract validation
|
||||
- Ziel: unvalidierte produktionsrelevante Text-Inputs im Registry-/Schema-Layer schliessen
|
||||
- Scope: `workflow_node_registry.py`, `workflow_schema.py`, `test_workflow_schema.py`
|
||||
- Gate: neue Schema-Tests fuer UUID, absolute Pfade, float-string, hex-color, suffix-format
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
2. `B04-b` Node registry invariants
|
||||
- Ziel: defaults/fields/module_key/input-output-contracts als Registry-Invarianten pruefen
|
||||
- Scope: `test_workflow_node_registry.py`
|
||||
- Gate: registryweite Invariant-Tests gruen
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
3. `B06` Shared authoring surface
|
||||
- Ziel: gemeinsame Authoring-Schicht fuer Canvas-Menu, Sidebar und Starter-Aktionen
|
||||
- Why now: verhindert doppelte UI-Logik in `NodeCommandMenu` und `NodeDefinitionsPanel`
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
4. `B05` Family-scoped node organization
|
||||
- Ziel: modul-/family-basierte Node-Organisation auf der gemeinsamen Authoring-Schicht
|
||||
- Status: abgeschlossen am April 10, 2026
|
||||
5. `B07` Canvas ergonomics closure
|
||||
- Ziel: reduzierte Top-Area, konsistente Edge-/Insert-Interaktionen, Auto-Align sauber abschliessen
|
||||
6. `B08` Starter graph and module bundle normalization
|
||||
- Ziel: Blueprints, Bundles und New-Workflow-Einstieg auf dieselben kanonischen Pfade ziehen
|
||||
7. `B09` Template-aware runtime unification
|
||||
- Ziel: Graph und Legacy durch dieselbe Template-/Output-Orchestrierung fuehren
|
||||
8. `B10` Non-legacy still smoke closure
|
||||
- Ziel: kanonischer Still-Graph als wiederholbarer Smoke-Pfad mit klarer Fehlerflaeche
|
||||
9. `B11` Template parity matrix
|
||||
- Ziel: echte Graph-vs-Legacy-Vergleiche mit Render-Templates, Varianten und Alpha/Samples
|
||||
10. `B12` CAD intake moduleization
|
||||
- Ziel: CAD-Import/Extract/Export/BBox als echte Production-Module im Editor und in der Runtime
|
||||
11. `B13` Rollout and fallback controls
|
||||
- Ziel: Graph/Shadow/Legacy pro Workflow und Output-Type operativ steuerbar halten
|
||||
12. `B14` Sequential E2E gates
|
||||
- Ziel: Low-RAM Golden-/Smoke-/Browser-Gates fuer `/workflows`
|
||||
13. `B15` Repo hygiene root-cause closure
|
||||
- Ziel: generated artifacts, root-owned caches und compose side effects ursachenseitig schliessen
|
||||
- Parallel vorbereitbar zu `B09` bis `B14`
|
||||
|
||||
### Queue Q2: Parallel Preparation Tracks
|
||||
|
||||
- Track A: Backend contracts
|
||||
- aktiv: `B04-a`, danach `B04-b`
|
||||
- Merge-Blocker fuer fast alle Folgearbeiten
|
||||
- Track B: Frontend authoring refactor
|
||||
- Vorbereitung jetzt, Merge erst nach `B04`
|
||||
- Reihenfolge laut Analyse: `B06` -> `B05` -> `B07` -> `B08`
|
||||
- Track C: Runtime and parity
|
||||
- Investigation parallel moeglich
|
||||
- Merge-Reihenfolge: `B09` -> `B10` -> `B11` -> `B12` -> `B13` -> `B14`
|
||||
- Track D: Hygiene
|
||||
- Root-cause Sammlung und Script-Haertung parallel
|
||||
- Merge spaet, solange keine produktionskritische Blockade sichtbar wird
|
||||
|
||||
### Queue Q2: Immediate Start
|
||||
|
||||
- Aktiver Implementierungsblock: `B07`
|
||||
- Bereits abgeschlossene Merge-Slices in dieser Queue:
|
||||
- `B04-a`
|
||||
- `B04-b`
|
||||
- `B06`
|
||||
- `B05`
|
||||
- Vorbereitete Folgeblöcke:
|
||||
- `B07`
|
||||
- `B08`
|
||||
@@ -0,0 +1,245 @@
|
||||
# Node-Based Production Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
Define the target model for a reusable, node-based production system where workflow steps are backend-owned modules, the editor is schema-driven, and legacy rendering stays operational during migration.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The current workflow system already has meaningful extraction work:
|
||||
|
||||
- bridge/runtime services exist for setup, template resolution, material mapping, bbox resolution, publish, and notify
|
||||
- graph execution can already orchestrate still, turntable, and blend export flows
|
||||
- the editor already consumes backend node definitions
|
||||
|
||||
What is still missing is a clean production model.
|
||||
|
||||
Today, three different concerns are still partially collapsed into each other:
|
||||
|
||||
1. `OutputType` as user-facing commercial/render choice
|
||||
2. workflow graph as orchestration definition
|
||||
3. legacy/internal render settings as implicit execution contract
|
||||
|
||||
That makes it hard to:
|
||||
|
||||
- reuse a process step like CAD import as a true module
|
||||
- expose all node settings coherently in the editor
|
||||
- bind output types to workflows without fragile implicit assumptions
|
||||
- preserve legacy behavior while enabling graph-native production
|
||||
|
||||
## Target Model
|
||||
|
||||
### 1. Production Module
|
||||
|
||||
A production module is the canonical backend capability unit.
|
||||
|
||||
Examples:
|
||||
|
||||
- `cad.resolve_step_path`
|
||||
- `cad.extract_objects`
|
||||
- `cad.export_glb`
|
||||
- `cad.compute_bbox`
|
||||
- `materials.resolve_map`
|
||||
- `materials.auto_populate`
|
||||
- `render.resolve_template`
|
||||
- `render.blender_still`
|
||||
- `render.blender_turntable`
|
||||
- `output.publish_asset`
|
||||
- `output.notify`
|
||||
|
||||
Rules:
|
||||
|
||||
- modules are backend-owned
|
||||
- modules define typed input contract, output contract, defaults, and execution semantics
|
||||
- modules are reusable from legacy code, graph runtime, shadow mode, and tests
|
||||
- modules must not depend on editor-only UI metadata
|
||||
|
||||
### 2. Workflow Node
|
||||
|
||||
A workflow node is an orchestration wrapper around a production module.
|
||||
|
||||
It adds:
|
||||
|
||||
- node id
|
||||
- graph connectivity
|
||||
- per-instance parameter overrides
|
||||
- editor UI metadata
|
||||
- retry/failure policy
|
||||
|
||||
It must not redefine business behavior that belongs to the production module itself.
|
||||
|
||||
Implication:
|
||||
|
||||
- the node registry should evolve from "palette metadata + field definitions" to "module-backed node definitions"
|
||||
- `step` should remain the stable runtime key, but internally map to a reusable module contract
|
||||
|
||||
### 3. Workflow Family
|
||||
|
||||
Workflows must be separated into runtime families, not inferred ad hoc:
|
||||
|
||||
- `cad_file`
|
||||
- `order_line`
|
||||
|
||||
Family drives:
|
||||
|
||||
- valid entry context
|
||||
- allowed node palette
|
||||
- validation rules
|
||||
- available output contracts
|
||||
- preflight expectations
|
||||
|
||||
Mixed-family graphs may still exist temporarily for migration visibility, but must not be the target authoring model.
|
||||
|
||||
### 4. Output Type as Invocation Profile
|
||||
|
||||
`OutputType` should no longer be treated as a loose bag of renderer flags.
|
||||
|
||||
It should be the product-facing invocation profile for a workflow:
|
||||
|
||||
- commercial name and visibility
|
||||
- compatible categories
|
||||
- pricing tier binding
|
||||
- workflow family
|
||||
- linked workflow definition
|
||||
- invocation-level parameter overrides
|
||||
- output artifact contract
|
||||
|
||||
Examples of invocation-level overrides:
|
||||
|
||||
- resolution
|
||||
- samples
|
||||
- engine
|
||||
- transparency
|
||||
- animation timing
|
||||
- material override
|
||||
|
||||
Examples of artifact contract:
|
||||
|
||||
- still image
|
||||
- turntable video
|
||||
- production `.blend`
|
||||
- preview thumbnail
|
||||
- future exported package types
|
||||
|
||||
This keeps the responsibility split clean:
|
||||
|
||||
- workflow definition answers: "what steps run and in what order?"
|
||||
- output type answers: "what productized variant of that workflow do we sell and with which defaults/constraints?"
|
||||
|
||||
## Required Refactor Direction
|
||||
|
||||
### A. Formalize Node Contracts
|
||||
|
||||
Extend the node registry so each definition exposes:
|
||||
|
||||
- `family`
|
||||
- `module_key`
|
||||
- `input_contract`
|
||||
- `output_contract`
|
||||
- `param_schema`
|
||||
- `artifact_roles_produced`
|
||||
- `artifact_roles_consumed`
|
||||
- `legacy_source`
|
||||
|
||||
Current definitions already cover labels, categories, defaults, and fields. They do not yet fully express machine-usable production contracts.
|
||||
|
||||
### B. Promote Runtime Services to Module Layer
|
||||
|
||||
The extracted bridge/runtime services are the right foundation. They now need a clear module boundary so both legacy and graph runtimes call the same backend operation layer.
|
||||
|
||||
Desired shape:
|
||||
|
||||
- legacy pipeline calls module layer directly
|
||||
- graph runtime calls module layer directly or via async task adapters
|
||||
- Celery task mapping becomes transport/adaptation, not the primary execution model
|
||||
|
||||
### C. Split Graph Authoring by Family
|
||||
|
||||
The editor should author against family-scoped graphs:
|
||||
|
||||
- CAD Intake graph
|
||||
- Order Rendering graph
|
||||
|
||||
That includes:
|
||||
|
||||
- family-specific starter templates
|
||||
- family-specific node palette groups
|
||||
- validation that rejects wrong-family entry nodes early
|
||||
- cleaner organization than a single mixed library
|
||||
|
||||
### D. Reframe Output Type Creation
|
||||
|
||||
Output type creation is currently too close to legacy render settings and too far from workflow invocation.
|
||||
|
||||
Create/edit flow should become:
|
||||
|
||||
1. choose family
|
||||
2. choose or create workflow
|
||||
3. choose artifact kind
|
||||
4. set invocation overrides
|
||||
5. bind pricing/category/material constraints
|
||||
|
||||
The current `workflow_definition_id` field is directionally correct, but too weak on its own because there is no explicit invocation contract or family validation around it yet.
|
||||
|
||||
## Compatibility Rules
|
||||
|
||||
### Legacy Safety
|
||||
|
||||
- legacy dispatch remains the fallback path
|
||||
- existing output types without workflow linkage remain valid
|
||||
- graph rollout must be opt-in per output type/workflow
|
||||
|
||||
### Migration Safety
|
||||
|
||||
- old output types may continue to store render settings in their current shape
|
||||
- a compatibility adapter should map legacy render settings into invocation overrides
|
||||
- workflow definitions must remain canonical JSON DAGs
|
||||
|
||||
## Recommended Implementation Sequence
|
||||
|
||||
### Phase A: Stabilize Broken Contracts
|
||||
|
||||
- align frontend/backend `OutputType` defaults and allowed values
|
||||
- add backend validation for output-type family/workflow compatibility
|
||||
- make output type creation/editing reflect current real backend constraints
|
||||
|
||||
### Phase B: Contract-First Registry
|
||||
|
||||
- add `family`, contracts, and module metadata to node definitions
|
||||
- expose them over `/api/workflows/node-definitions`
|
||||
- move editor grouping/validation to registry-owned family metadata
|
||||
|
||||
### Phase C: Invocation Profiles
|
||||
|
||||
- extend `OutputType` into a workflow invocation profile
|
||||
- add explicit artifact kind and workflow family
|
||||
- separate invocation overrides from raw render settings
|
||||
|
||||
### Phase D: Module Unification
|
||||
|
||||
- route legacy and graph execution through the same module layer
|
||||
- keep Celery as transport where async work is needed
|
||||
- reduce duplicate logic in tasks and runtime adapters
|
||||
|
||||
### Phase E: Full Parity Authoring
|
||||
|
||||
- ship family-specific starter workflows
|
||||
- expose all module settings in editor
|
||||
- support end-to-end preflight, dispatch, run inspection, and parity verification
|
||||
|
||||
## Immediate Code Implications
|
||||
|
||||
- `workflow_node_registry.py` is the correct extension point for module contracts
|
||||
- `workflow_schema.py` will need stronger family- and contract-aware validation
|
||||
- `OutputType` needs a clearer model than raw renderer/backend defaults plus optional workflow id
|
||||
- editor UX should follow model cleanup, not lead it
|
||||
|
||||
## Decision
|
||||
|
||||
We should simplify and refactor before doing more isolated workflow-editor UX work.
|
||||
|
||||
The next implementation blocks should prioritize:
|
||||
|
||||
1. fixing the output-type/workflow contract
|
||||
2. formalizing node/module contracts
|
||||
3. only then expanding editor affordances on top of the cleaned model
|
||||
@@ -0,0 +1,52 @@
|
||||
# Template Input Audit
|
||||
|
||||
Stand: 12. April 2026
|
||||
|
||||
## Befund
|
||||
|
||||
Die Transportkette fuer `workflow_input_schema` und `template_inputs` funktioniert inzwischen end-to-end im Graph-Workflow. Der aktuelle Engpass liegt in den live hinterlegten `.blend`-Templates selbst:
|
||||
|
||||
- Alle produktiven Render-Templates haben aktuell `workflow_input_schema = []`.
|
||||
- Die live hochgeladenen `.blend`-Dateien enthalten keine erkennbaren Template-Input-Marker auf Collections, Objekten oder Worlds.
|
||||
- Damit gibt es derzeit keine realen, template-spezifischen Dropdown-/Options-Felder, die wir ehrlich in `resolve_template` exponieren koennen.
|
||||
|
||||
## Gepruefte Live-Templates
|
||||
|
||||
- `Blender_Studio_Schadowcatcher_Anim_RotOBJ`
|
||||
- `Default`
|
||||
- `Blender_Studio_Schadowcatcher_Anim`
|
||||
- `BlenderStudio_Shadowcatcher`
|
||||
- `BlenderStudio`
|
||||
|
||||
## Beobachtete Blend-Struktur
|
||||
|
||||
- `BlenderStudio`: `Collection`, `Export`, `Lighting`, `World`
|
||||
- `BlenderStudio_Shadowcatcher`: `Collection`, `Export`, `Lighting`, `Shadowcatcher`, `World`
|
||||
- `Blender_Studio_Schadowcatcher_Anim`: gleiche Struktur wie Shadowcatcher-Template
|
||||
- `Blender_Studio_Schadowcatcher_Anim_RotOBJ`: gleiche Struktur wie Shadowcatcher-Template
|
||||
- `Default`: nur `Export`, keine World, keine Marker
|
||||
|
||||
## Konsequenz
|
||||
|
||||
Der naechste saubere Schritt ist nicht ein blinder Schema-Backfill, sondern Template-Authoring:
|
||||
|
||||
1. Marker oder Scene-Property-gesteuerte Varianten in den `.blend`-Dateien anlegen.
|
||||
2. Daraus eine echte `workflow_input_schema` ableiten.
|
||||
3. Danach die Felder im Admin pflegen oder per Script backfillen.
|
||||
|
||||
## Tooling
|
||||
|
||||
Fuer diese Authoring-Arbeit gibt es jetzt ein reproduzierbares Audit-Script:
|
||||
|
||||
```bash
|
||||
python3 scripts/audit_render_templates.py --json
|
||||
python3 scripts/audit_render_templates.py --write-markdown docs/workflows/TEMPLATE_INPUT_AUDIT.generated.md
|
||||
```
|
||||
|
||||
Das Script:
|
||||
|
||||
- loggt sich gegen die lokale API ein,
|
||||
- kopiert die live verwendeten `.blend`-Dateien aus dem Backend-Container,
|
||||
- inspiziert sie mit host-Blender,
|
||||
- erkennt Marker gemaess der HartOMat-Konventionen,
|
||||
- und erzeugt daraus ggf. Schema-Vorschlaege.
|
||||
@@ -39,23 +39,29 @@ Parallel execution ownership and stage gates are defined in [`docs/workflows/WOR
|
||||
- [ ] Editor saves nodes and edges
|
||||
- [ ] Editor roundtrip preserves workflow configs
|
||||
- [ ] All node settings are editable
|
||||
- [ ] Validate, dry-run, and dispatch are available
|
||||
- [ ] Runs are visible with node-level status and logs
|
||||
- [x] Validate, dry-run, and dispatch are available
|
||||
- [x] Runs are visible with node-level status and logs
|
||||
- [ ] Editor authoring follows family-safe module contracts instead of ad hoc node metadata
|
||||
- Progress: The workflow canvas header has been compressed into a single canvas-adjacent control strip, preserving right-click node insertion, auto-align, edge deletion, preflight, dispatch, and save actions while reducing top-of-page bloat.
|
||||
- Progress: The canonical still-path bridge nodes now expose editor-visible, runtime-backed settings for template overrides, material resolution, auto-population behavior, GLB source preference, output-save artifact expectations, and notify arming. The remaining Phase 5 work is authoring hierarchy and end-to-end editor verification, not hidden backend-only params.
|
||||
|
||||
### Phase 7
|
||||
|
||||
- [x] Output-type create defaults match current backend constraints
|
||||
- [ ] Output types model workflow invocation profiles
|
||||
- [ ] Output types validate against workflow family and artifact contract
|
||||
- [ ] Admin create/edit flow is workflow-first instead of renderer-first
|
||||
- [x] Output types model workflow invocation profiles
|
||||
- [x] Output types validate against workflow family and artifact contract
|
||||
- [x] Admin create/edit flow is workflow-first instead of renderer-first
|
||||
- Progress: The admin output-type form now opens with a dedicated workflow-contract section, keeps invocation-profile inputs ahead of renderer compatibility knobs, and moves catalog/business fields into a separate closing section so legacy Blender details no longer dominate the primary authoring flow.
|
||||
- Progress: Output-type contract helpers now expose family-safe format lists, `blend` is treated as a first-class `blend_asset` contract in both frontend and backend, and the admin form no longer steers users into obviously invalid `cad_file`/video or `order_line`/model-export combinations.
|
||||
- Progress: API responses now serialize the invocation profile through the typed schema, and create/edit validation rejects mixed-family workflows plus workflow/artifact mismatches before dispatch time.
|
||||
|
||||
### Phase 6
|
||||
|
||||
- [x] Shadow mode parity execution dispatches real graph observer runs alongside authoritative legacy dispatch
|
||||
- Progress: Workflow runs now expose a comparison endpoint that resolves authoritative legacy outputs and matching shadow artifacts, including file hashes, image dimensions, and mean pixel delta for parity inspection.
|
||||
- Progress: `scripts/test_render_pipeline.py --workflow-still-smoke --execution-mode shadow` now provisions the canonical still smoke contract, runs preflight, dispatches via the real order/output-type workflow linkage, resolves the resulting workflow run, and prints the shadow comparison verdict.
|
||||
- [ ] Golden cases pass against legacy outputs
|
||||
- [x] Golden cases pass against legacy outputs
|
||||
- Progress: On April 8, 2026, the live `--workflow-golden-suite` passed end to end for `still_legacy`, `still_graph`, `still_shadow`, `turntable_graph`, and `blend_graph`. The blend export contract now completes the order line, persists the primary `.blend` result, and links the resulting `blend_production` media asset back to the workflow run.
|
||||
- [ ] Rollout can be enabled per workflow or output type
|
||||
- [ ] Rollback to legacy is immediate
|
||||
|
||||
@@ -100,12 +106,14 @@ Parallel execution ownership and stage gates are defined in [`docs/workflows/WOR
|
||||
- Invalid graphs are blocked before dispatch.
|
||||
- All node settings needed for parity are present in the editor.
|
||||
- Family-specific authoring prevents invalid `cad_file`/`order_line` graph composition.
|
||||
- Progress: Backend-owned still bridge modules now declare the settings required for parity; the next gate is browser-level confirmation that the inspector presents them cleanly enough for real authoring.
|
||||
|
||||
### QG-7: Invocation Gate
|
||||
|
||||
- Output type creation and editing use valid backend defaults.
|
||||
- Output types bind to workflows through an explicit invocation contract.
|
||||
- Legacy output types remain renderable during migration.
|
||||
- Progress: This gate is functionally green at the API layer. Remaining rollout work is operational adoption, not missing contract primitives.
|
||||
|
||||
### QG-6: Rollout Gate
|
||||
|
||||
@@ -116,11 +124,21 @@ Parallel execution ownership and stage gates are defined in [`docs/workflows/WOR
|
||||
- `python scripts/test_render_pipeline.py --workflow-still-smoke --execution-mode legacy`
|
||||
- `python scripts/test_render_pipeline.py --workflow-still-smoke --execution-mode graph`
|
||||
- `python scripts/test_render_pipeline.py --workflow-still-smoke --execution-mode shadow`
|
||||
- Sequential low-RAM gate wrapper:
|
||||
- `./scripts/workflow_sequential_gates.sh`
|
||||
- `./scripts/workflow_sequential_gates.sh --with-cad-parity`
|
||||
- `./scripts/workflow_sequential_gates.sh --with-live-shadow`
|
||||
- `./scripts/workflow_sequential_gates.sh --with-cad-parity --with-live-shadow --with-golden`
|
||||
- CAD/Viewer parity smoke for repeated-instance products:
|
||||
- `python3 scripts/compare_live_cad_parity.py --cad-id 7c214057-9982-4d6e-aa87-43bfabfdb709`
|
||||
- Progress: The repeated-instance CAD regression now has a dedicated live gate. The current reference case passes with `146` manifest parts, `146` mesh nodes, `146` unique `partKey`s, and no missing or duplicate assignments.
|
||||
- Rollout approval rule for the canonical still workflow:
|
||||
- `shadow` must finish with a successful order line and a comparison verdict of `pass`
|
||||
- `warn` or `fail` means legacy remains authoritative
|
||||
- `graph` may only be enabled on real output types after the shadow command passes cleanly
|
||||
- Progress: the canonical still smoke flow now passes live in `legacy` and `graph`; `shadow` stabilizes after a short observer-output lag and currently reports `warn` because the observer image differs slightly, so legacy remains authoritative for rollout decisions.
|
||||
- Progress: the canonical still smoke flow now passes live in `legacy`, `graph`, and `shadow`. Shadow observer output may arrive slightly later than the authoritative legacy file, but the rollout gate now treats proven de-minimis Blender drift as `pass` instead of a rollout-blocking `warn`.
|
||||
- Progress: the live smoke and golden harnesses now provision explicit `workflow_rollout_mode` values when linking non-legacy output types, so `graph` and `shadow` exercises no longer depend on whatever rollout state happened to be left behind in the database.
|
||||
- Progress: the broader golden suite is green in live graph execution, and the remaining rollout work is operational enablement per workflow/output type rather than still-shadow image drift.
|
||||
|
||||
## Definition of Done
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# Workflow Implementation Backlog
|
||||
|
||||
Execution orchestration, ownership split, and merge order are tracked in [`docs/workflows/WORKERS.md`](/home/hartmut/Documents/Copilot/schaefflerautomat/docs/workflows/WORKERS.md).
|
||||
|
||||
## Epic 1: Canonical Workflow Model
|
||||
|
||||
### Tickets
|
||||
@@ -25,6 +27,7 @@
|
||||
- `E2-T3` Add `GET /api/workflows/node-definitions`.
|
||||
- `E2-T4` Provide schema-driven defaults and editor field groups.
|
||||
- `E2-T5` Add composite bridge nodes for safe migration.
|
||||
- `E2-T6` Extend node definitions with family, module key, input contract, output contract, and artifact roles.
|
||||
|
||||
### Required Node Coverage
|
||||
|
||||
@@ -76,6 +79,11 @@
|
||||
- `E4-T4` Support retry and failure policies. `completed`
|
||||
- `E4-T5` Add execution mode switch: `legacy`, `graph`, `shadow`. `completed`
|
||||
- `E4-T6` Add hard fallback to legacy dispatch on graph failure. `completed`
|
||||
- `E4-T7` Make `output_save` graph-authoritative for still renders by disabling render-task self-publish whenever a downstream `output_save` node is present. `completed`
|
||||
- `E4-T8` Persist authoritative still output metadata back into `WorkflowNodeResult` rows and keep shadow mode non-authoritative. `completed`
|
||||
- `E4-T9` Extend runtime, dispatch, and task tests for graph-authoritative still persistence and legacy-safe notify handoff. `completed`
|
||||
- `E4-T10` Extend graph-authoritative `output_save` semantics to `export_blend`, including asset persistence and node-result updates without mutating the primary order-line render output. `completed`
|
||||
- `E4-T11` Extend graph-authoritative `output_save` semantics to `blender_turntable`, including graph/legacy-safe task argument handling, authoritative persistence, and node-result updates for video outputs. `completed`
|
||||
|
||||
## Epic 5: Editor Parity
|
||||
|
||||
@@ -87,6 +95,20 @@
|
||||
- `E5-T4` Add graph validation in the editor.
|
||||
- `E5-T5` Add dry-run and dispatch from the editor.
|
||||
- `E5-T6` Add workflow run inspection UI.
|
||||
- `E5-T7` Reorganize authoring around family-specific starter graphs and family-safe palettes.
|
||||
- Progress: The editor already supports right-click node insertion, searchable family-aware palettes, auto-align, edge deletion, dry-run/dispatch, and run inspection. The current remaining parity slice is authoring polish and validation around the canonical non-legacy still graph, not missing basic editor mechanics.
|
||||
|
||||
## Epic 7: Output-Type Invocation Profiles
|
||||
|
||||
### Tickets
|
||||
|
||||
- `E7-T1` Align frontend/backend defaults and allowed values for output-type creation. `completed`
|
||||
- `E7-T2` Define explicit workflow family on output types. `completed`
|
||||
- `E7-T3` Separate invocation overrides from legacy raw render settings. `completed`
|
||||
- `E7-T4` Add artifact-kind contract to output types. `completed`
|
||||
- `E7-T5` Validate output-type family/workflow compatibility in backend APIs. `completed`
|
||||
- `E7-T6` Redesign admin create/edit flow around workflow invocation instead of legacy renderer flags. `completed`
|
||||
- Progress: Output types now persist an explicit workflow family, artifact kind, and invocation override set; backend APIs reject family/artifact mismatches against linked workflows; serializer output exposes a typed invocation profile instead of leaking raw dict payloads.
|
||||
|
||||
## Epic 6: Rollout and Quality
|
||||
|
||||
@@ -94,6 +116,8 @@
|
||||
|
||||
- `E6-T1` Add shadow mode parity execution. `completed`
|
||||
- `E6-T2` Build output comparison tooling. `completed`
|
||||
- `E6-T3` Define golden test cases.
|
||||
- `E6-T3` Define golden test cases. `completed`
|
||||
- `E6-T3a` Harden the golden-suite harness against transient backend disconnects and 502/503/504 responses. `completed`
|
||||
- `E6-T3b` Close the primary `.blend` graph-authoritative persistence gap so graph blend exports complete the order, persist the order-line output, and publish a `blend_production` media asset. `completed`
|
||||
- `E6-T4` Roll out per workflow or output type.
|
||||
- `E6-T5` Keep legacy fallback after rollout.
|
||||
|
||||
@@ -9,6 +9,12 @@ Bring `/workflows` to full production parity with the existing legacy render pip
|
||||
- Phase 1 completed on canonical config storage, preset migration, and legacy-safe runtime extraction.
|
||||
- Phase 2 completed on backend node registry, node definitions API, and schema-driven editor palette/settings.
|
||||
- Phase 3 completed: `order_line_setup`, `resolve_template`, `material_map_resolve`, `auto_populate_materials`, `glb_bbox`, `output_save`, and `notify` are extracted behind the legacy task boundary, validated with targeted backend tests, and covered by workflow executor dispatch tests.
|
||||
- Phase 4 is partially completed: graph runtime dispatch now treats downstream `output_save` as the authoritative persistence boundary for still renders, turntable/video renders, and `.blend` exports, updates node results after persistence, and keeps shadow executions non-authoritative.
|
||||
- The canonical still bridge path now exposes runtime-backed node settings for template overrides, material resolution, auto-population rules, GLB source selection, output-save artifact filtering, and notify arming, so `/workflows` can author the real still-render contract instead of relying on hidden legacy defaults.
|
||||
- Output types now behave as explicit invocation profiles in both API and admin UI: workflow family, artifact kind, and invocation overrides are first-class, and linked workflow compatibility is enforced before dispatch.
|
||||
- Canonical reference workflows now need to be managed as at least two families, not one mixed graph:
|
||||
- `CAD Intake`
|
||||
- `Order Rendering`
|
||||
|
||||
## Non-Negotiables
|
||||
|
||||
@@ -46,6 +52,16 @@ Notes:
|
||||
- `ui` is editor-only metadata and must not change runtime semantics.
|
||||
- `edges` are mandatory for graph persistence and validation.
|
||||
|
||||
## Architecture Direction
|
||||
|
||||
- Workflow nodes are orchestration wrappers, not the canonical implementation unit.
|
||||
- The canonical implementation unit is a backend-owned production module with typed inputs, outputs, defaults, and execution semantics.
|
||||
- Workflow definitions answer "what runs"; output types answer "which productized invocation profile of that workflow is offered".
|
||||
- Workflow authoring remains split by family:
|
||||
- `cad_file`
|
||||
- `order_line`
|
||||
- The detailed target model is captured in [`docs/workflows/NODE_BASED_PRODUCTION_ARCHITECTURE.md`](/home/hartmut/Documents/Copilot/schaefflerautomat/docs/workflows/NODE_BASED_PRODUCTION_ARCHITECTURE.md).
|
||||
|
||||
## Phases
|
||||
|
||||
### Phase 1: Canonical Model and Migration Base
|
||||
@@ -81,12 +97,17 @@ Notes:
|
||||
- Support node outputs and artifact handoff across edges.
|
||||
- Keep `legacy`, `graph`, and `shadow` execution modes.
|
||||
- Current slice: graph dispatch executes extracted bridge nodes for order-line setup, template/material resolution, auto-material population, and bounding-box resolution before queueing render/export tasks.
|
||||
- Current slice completed: still-render graphs, turntable/video graphs, and `.blend` export graphs with downstream `output_save` now disable task-level self-publish, persist authoritative output/media metadata through shared runtime services, and write the result back into `WorkflowNodeResult`.
|
||||
- Next Phase 4 slice: extend the same authoritative `output_save` contract to any remaining legacy export variants and close the remaining notify/editor parity gaps.
|
||||
|
||||
### Phase 5: Workflow Editor Parity
|
||||
|
||||
- Persist and load `nodes`, `edges`, `step`, `params`, and `ui`.
|
||||
- Render all node settings dynamically from backend schemas.
|
||||
- Add validation, dry-run, dispatch, and run inspection.
|
||||
- Organize workflows and node palette by family so the editor reflects the runtime split between `cad_file` and `order_line` contexts.
|
||||
- Current state: right-click insertion, searchable family-aware palettes, auto-align, edge deletion, dry-run, dispatch, and run inspection are already in place; the remaining work is authoring clarity, node organization, and browser-verified end-to-end usability for the non-legacy still graph.
|
||||
- Do not add further editor-only UX surface before node/module contracts and output-type invocation profiles are stabilized.
|
||||
|
||||
### Phase 6: Shadow Mode and Rollout
|
||||
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
# Template Inputs For Graph Workflows
|
||||
|
||||
`resolve_template` can now expose template-specific inputs into the graph workflow without breaking legacy rendering.
|
||||
|
||||
## Where the inputs come from
|
||||
|
||||
Render templates can define a `workflow_input_schema` JSON array in Admin.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"key": "studio_variant",
|
||||
"label": "Studio Variant",
|
||||
"type": "select",
|
||||
"section": "Template Inputs",
|
||||
"default": "default",
|
||||
"options": [
|
||||
{ "value": "default", "label": "Default" },
|
||||
{ "value": "warm", "label": "Warm" }
|
||||
]
|
||||
},
|
||||
{
|
||||
"key": "camera_profile",
|
||||
"label": "Camera Profile",
|
||||
"type": "text",
|
||||
"section": "Template Inputs",
|
||||
"default": "macro"
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
These fields appear in the workflow editor on the `resolve_template` node after a template override is selected.
|
||||
|
||||
## Runtime behavior
|
||||
|
||||
At runtime the graph path resolves these values into `template_inputs` and forwards them through:
|
||||
|
||||
1. `resolve_template`
|
||||
2. workflow runtime invocation
|
||||
3. still / turntable / cinematic Blender services
|
||||
4. Blender worker CLI via `--template-inputs`
|
||||
5. template scene setup after opening the `.blend`
|
||||
|
||||
Legacy workflows continue to work if no template input schema is defined.
|
||||
|
||||
## Blender-side conventions
|
||||
|
||||
Every resolved template input is always written onto the active scene as:
|
||||
|
||||
- `template_input__<key>`
|
||||
- `hartomat_template_input__<key>`
|
||||
|
||||
Example:
|
||||
|
||||
- `template_input__studio_variant = "warm"`
|
||||
- `hartomat_template_input__camera_profile = "macro"`
|
||||
|
||||
Templates can optionally react to these values using markers on collections, objects, or worlds.
|
||||
|
||||
Supported marker styles:
|
||||
|
||||
- custom property value: `template_input=studio_variant=warm`
|
||||
- object or collection name: `template-input:studio_variant=warm`
|
||||
- object or collection name: `ti::studio_variant::warm`
|
||||
|
||||
If a marker matches the resolved input value, the target is enabled. Non-matching variants are hidden.
|
||||
|
||||
## Practical authoring pattern
|
||||
|
||||
For a lighting setup with two variants:
|
||||
|
||||
1. Put warm lights into a collection named `template-input:studio_variant=warm`
|
||||
2. Put cool lights into a collection named `template-input:studio_variant=cool`
|
||||
3. Define `studio_variant` in `workflow_input_schema`
|
||||
4. Select the template in `resolve_template`
|
||||
5. Pick the wanted variant in the node inspector
|
||||
|
||||
## Important limitation
|
||||
|
||||
The graph runtime now transports template inputs end-to-end, but existing `.blend` files only change visually if they use one of the conventions above or read the scene custom properties themselves.
|
||||
Reference in New Issue
Block a user