From 22981af1d27875d7c24dde3fcd7192fb75753983 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Hartmut=20N=C3=B6renberg?= Date: Wed, 8 Apr 2026 21:14:34 +0200 Subject: [PATCH] docs: add workflow worker orchestration --- docs/workflows/WORKERS.md | 384 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 384 insertions(+) create mode 100644 docs/workflows/WORKERS.md diff --git a/docs/workflows/WORKERS.md b/docs/workflows/WORKERS.md new file mode 100644 index 0000000..ef0de85 --- /dev/null +++ b/docs/workflows/WORKERS.md @@ -0,0 +1,384 @@ +# Workflow Worker Orchestration + +## Purpose + +This file defines the parallel work split to bring `/workflows` to full production-capable parity while keeping the legacy workflow operational at all times. + +The orchestration is intentionally contract-first: + +1. stabilize runtime and output-type contracts +2. formalize module-backed node definitions +3. organize editor authoring around those contracts +4. prove parity with dual-run and golden-case validation + +## Non-Negotiable Constraints + +- Legacy workflow dispatch must remain callable throughout the migration. +- Mixed-family graphs may exist for migration visibility, but new authoring must converge on family-safe graphs. +- No worker may revert unrelated user changes in the dirty worktree. +- `cad_file` and `order_line` concerns must be separated explicitly in contracts, editor palette organization, and output-type validation. +- A worker may only edit files in its assigned ownership scope unless the handoff section explicitly allows expansion. + +## Shared Acceptance Baseline + +Each worker block is only complete when all of the following hold: + +- targeted tests for the touched scope pass +- legacy-safe behavior is preserved or explicitly guarded with fallback +- the changed API surface is reflected in frontend/backend types where applicable +- docs and seed/default data stay aligned with the runtime behavior + +## Branching And Commit Order + +Suggested branch naming: + +- `workflow/block-a-node-contracts` +- `workflow/block-b-module-unification` +- `workflow/block-c-starter-graphs` +- `workflow/block-d-output-type-contracts` +- `workflow/block-e-editor-organization` +- `workflow/block-f-run-inspection` +- `workflow/block-g-parity-golden-cases` +- `workflow/block-h-cad-material-parity` +- `workflow/block-i-rollout-regression-gates` + +Preferred integration order: + +1. Block A +2. Block D +3. Block B +4. Block C +5. Block E +6. Block F +7. Block H +8. Block G +9. Block I + +## Worker Blocks + +### Block A: Node Contract Closure + +Purpose: +Turn the node registry into the canonical module contract source for every editor-visible graph node. + +Primary ownership: + +- `backend/app/domains/rendering/workflow_node_registry.py` +- `backend/app/domains/rendering/workflow_schema.py` +- `backend/tests/domains/test_workflow_node_registry.py` +- `backend/tests/domains/test_workflow_schema.py` +- `frontend/src/api/workflows.ts` + +Tickets: + +- audit every editor-visible node for missing or weak param schemas +- close real settings gaps for graph-authorable nodes, especially export and CAD bridge nodes +- make `family`, `module_key`, `input_contract`, `output_contract`, `artifact_roles_*`, and `legacy_source` authoritative and complete +- strengthen family-aware graph validation in the backend schema layer + +Acceptance criteria: + +- every editor-visible node has a machine-usable contract +- no node required for still-render parity is missing editable settings +- backend rejects family-invalid graphs with clear errors + +Dependencies: + +- none + +### Block B: Module Unification + +Purpose: +Route graph and legacy execution through the same reusable production-module layer wherever parity nodes already exist. + +Primary ownership: + +- `backend/app/domains/rendering/workflow_runtime_services.py` +- `backend/app/domains/rendering/workflow_executor.py` +- `backend/app/domains/rendering/workflow_graph_runtime.py` +- `backend/app/domains/rendering/tasks.py` +- `backend/app/domains/pipeline/tasks/render_order_line.py` +- `backend/tests/domains/test_workflow_graph_runtime.py` +- `backend/tests/domains/test_workflow_runtime_services.py` + +Tickets: + +- reduce duplicate behavior split across legacy task code and graph adapters +- ensure async task wrappers are transport, not business-logic owners +- close remaining drift for still, turntable, and blend export paths + +Acceptance criteria: + +- graph and legacy call the same module behavior for extracted parity nodes +- parity bug fixes land once in the shared module layer +- fallback to legacy still works on graph failure + +Dependencies: + +- Block A + +### Block C: Canonical Starter Graphs And Seeds + +Purpose: +Keep live workflow definitions, blueprints, and starter templates aligned with the corrected canonical graphs. + +Primary ownership: + +- `backend/app/domains/rendering/workflow_config_utils.py` +- `backend/app/domains/rendering/workflow_builder.py` +- `backend/app/api/routers/admin.py` +- `frontend/src/api/workflows.ts` +- seed/migration helpers and related tests + +Tickets: + +- maintain canonical `cad_file` and `order_line` starter graphs +- backfill or normalize existing seeded workflow definitions safely +- prevent drift between backend canonical graphs and frontend starter generation + +Acceptance criteria: + +- starter workflows are family-safe and match backend canonical topology +- existing seeded workflows can be repaired without manual DB surgery +- frontend new-workflow creation matches backend canonical graph shape + +Dependencies: + +- Block A + +### Block D: Output-Type Invocation Contracts + +Purpose: +Turn output types into explicit workflow invocation profiles instead of loose legacy renderer presets. + +Primary ownership: + +- `backend/app/models/output_type.py` +- `backend/app/domains/rendering/output_type_contracts.py` +- `backend/app/api/routers/output_types.py` +- `backend/alembic/versions/065_output_type_workflow_contracts.py` +- `backend/alembic/versions/066_output_type_invocation_overrides.py` +- `backend/tests/domains/test_output_types_api.py` +- `frontend/src/api/outputTypes.ts` +- `frontend/src/components/admin/OutputTypeTable.tsx` + +Tickets: + +- finalize explicit `workflow_family`, `artifact_kind`, and invocation override semantics +- validate output-type and workflow compatibility in create/edit flows +- separate productized invocation settings from raw legacy render settings +- make create/edit UX workflow-first instead of renderer-first + +Acceptance criteria: + +- new output types can be created reliably against the real backend contract +- invalid workflow-family or artifact combinations are blocked early +- legacy output types remain renderable without forced migration + +Dependencies: + +- Block A + +### Block E: Workflow Editor Organization + +Purpose: +Make the editor usable for production authoring by organizing nodes around families and modules instead of a flat mixed palette. + +Primary ownership: + +- `frontend/src/pages/WorkflowEditor.tsx` +- `frontend/src/components/workflows/` +- `frontend/src/__tests__/components/` + +Tickets: + +- organize palette and search around family, module, and execution role +- support low-friction node insertion from canvas context actions +- reduce top-of-page control bloat and move graph actions closer to the canvas +- improve graph readability with automatic alignment/layout and cleaner inspector behavior +- preserve delete/edit flows for nodes and edges + +Acceptance criteria: + +- authoring a family-safe graph is materially faster than today +- node discovery works from both sidebar and canvas-local actions +- editor UX follows the backend-owned contracts instead of bespoke per-node logic + +Dependencies: + +- Block A +- Block D for workflow-first output-type interactions that deep-link into editor usage + +### Block F: Run Inspection And Workflow Debugging + +Purpose: +Make `/workflows` operationally useful by exposing preflight, dispatch, run records, node outputs, and failure surfaces cleanly. + +Primary ownership: + +- `backend/app/domains/rendering/workflow_router.py` +- `backend/app/domains/rendering/models.py` +- `backend/app/domains/rendering/schemas.py` +- `frontend/src/api/workflows.ts` +- `frontend/src/components/workflows/` +- runtime/run inspection tests + +Tickets: + +- complete workflow run inspection UI +- expose node-level statuses, outputs, timings, and key artifacts +- make preflight and manual dispatch part of the normal debugging flow + +Acceptance criteria: + +- a failed graph run can be debugged from `/workflows` without jumping through ad hoc DB inspection +- node-level outputs are visible enough to diagnose broken contracts and missing artifacts + +Dependencies: + +- Block A +- Block B + +### Block G: Golden-Case Parity Suite + +Purpose: +Prove graph parity against legacy for representative render cases before broad rollout. + +Primary ownership: + +- `scripts/test_render_pipeline.py` +- parity fixtures, comparison tooling, and integration tests +- `docs/workflows/WORKFLOW_DELIVERY_CHECKLIST.md` + +Tickets: + +- define representative still, turntable, blend, and failure golden cases +- automate legacy-vs-graph comparison for outputs and key side effects +- capture artifacts and logs needed for regression triage + +Acceptance criteria: + +- golden cases pass for at least one real still-render graph and one legacy fallback path +- parity regressions are reproducible and attributable to specific nodes or contracts + +Dependencies: + +- Block B +- Block C +- Block F +- Block H for CAD/material-sensitive cases + +### Block H: CAD/Material Parity + +Purpose: +Restore parity for GLTF/USD preview identity, material assignment, and instance-aware part mapping so graph workflows consume trustworthy geometry state. + +Primary ownership: + +- `backend/app/api/routers/cad.py` +- `backend/app/api/routers/products.py` +- `backend/app/domains/pipeline/tasks/export_glb.py` +- `backend/app/domains/pipeline/tasks/extract_metadata.py` +- `backend/app/services/part_key_service.py` +- `frontend/src/components/cad/InlineCadViewer.tsx` +- `frontend/src/components/cad/ThreeDViewer.tsx` +- `frontend/src/components/cad/cadUtils.ts` +- CAD/material tests + +Tickets: + +- close instance-aware part identity drift in viewer and export pipeline +- restore reliable missing-material assignment and library mapping +- ensure graph nodes consume the same geometry/material truth as legacy-render preparation + +Acceptance criteria: + +- product preview and GLTF viewer show consistent material assignment on representative legacy products +- manual material repair workflows remain intact +- downstream render nodes receive stable part/material identities + +Dependencies: + +- Block A +- Block B + +### Block I: Rollout And Dual-Run Regression Gates + +Purpose: +Keep graph rollout safe by enforcing shadow-mode parity checks, fallback behavior, and rollback paths per workflow/output type. + +Primary ownership: + +- `backend/app/domains/rendering/dispatch_service.py` +- `backend/app/domains/rendering/workflow_comparison_service.py` +- `backend/tests/domains/test_workflow_dispatch_service.py` +- rollout docs and runbooks + +Tickets: + +- formalize shadow-mode regression gates +- make rollout decisions per workflow or output type +- preserve immediate rollback to legacy +- encode comparison thresholds and operational readiness checks + +Acceptance criteria: + +- shadow mode can run on representative workflows without affecting authoritative outputs +- rollout and rollback are explicit and test-backed +- graph error handling stays at or below legacy operational risk + +Dependencies: + +- Block B +- Block G + +## First-Wave Parallelization + +Run these blocks in parallel first: + +- Block A: Node contract closure +- Block D: Output-type invocation contracts +- Block E: Editor organization audit and contract-driven UX split +- Block I: Rollout and dual-run regression gate design + +Reason: + +- they have the highest leverage on all remaining work +- they define the contracts the later E2E and parity work must obey +- they are separable enough to run in parallel with minimal file overlap if ownership is respected + +## Cross-Block Handoffs + +- Block A hands canonical node/module contracts to Blocks B, C, E, and H. +- Block D hands stable output-type invocation semantics to Blocks E and I. +- Block H hands trustworthy geometry/material identity to Blocks G and I. +- Block F and G jointly define ship-readiness for `/workflows`. + +## Quality Gates By Orchestration Stage + +### Stage 1: Contract Gate + +- Blocks A and D merged +- backend and frontend contracts aligned +- new workflows and new output types validate against the same source of truth + +### Stage 2: Runtime Gate + +- Block B merged +- canonical starter graphs from Block C verified +- no regression in legacy dispatch behavior + +### Stage 3: Authoring Gate + +- Block E and Block F merged +- `/workflows` supports real authoring, preflight, dispatch, and inspection + +### Stage 4: Parity Gate + +- Block H and Block G merged +- at least one non-legacy still-render workflow passes end to end against a representative product + +### Stage 5: Rollout Gate + +- Block I merged +- shadow mode, rollback, and comparison thresholds are operationally usable