docs: add workflow worker orchestration
This commit is contained in:
@@ -0,0 +1,384 @@
|
||||
# Workflow Worker Orchestration
|
||||
|
||||
## Purpose
|
||||
|
||||
This file defines the parallel work split to bring `/workflows` to full production-capable parity while keeping the legacy workflow operational at all times.
|
||||
|
||||
The orchestration is intentionally contract-first:
|
||||
|
||||
1. stabilize runtime and output-type contracts
|
||||
2. formalize module-backed node definitions
|
||||
3. organize editor authoring around those contracts
|
||||
4. prove parity with dual-run and golden-case validation
|
||||
|
||||
## Non-Negotiable Constraints
|
||||
|
||||
- Legacy workflow dispatch must remain callable throughout the migration.
|
||||
- Mixed-family graphs may exist for migration visibility, but new authoring must converge on family-safe graphs.
|
||||
- No worker may revert unrelated user changes in the dirty worktree.
|
||||
- `cad_file` and `order_line` concerns must be separated explicitly in contracts, editor palette organization, and output-type validation.
|
||||
- A worker may only edit files in its assigned ownership scope unless the handoff section explicitly allows expansion.
|
||||
|
||||
## Shared Acceptance Baseline
|
||||
|
||||
Each worker block is only complete when all of the following hold:
|
||||
|
||||
- targeted tests for the touched scope pass
|
||||
- legacy-safe behavior is preserved or explicitly guarded with fallback
|
||||
- the changed API surface is reflected in frontend/backend types where applicable
|
||||
- docs and seed/default data stay aligned with the runtime behavior
|
||||
|
||||
## Branching And Commit Order
|
||||
|
||||
Suggested branch naming:
|
||||
|
||||
- `workflow/block-a-node-contracts`
|
||||
- `workflow/block-b-module-unification`
|
||||
- `workflow/block-c-starter-graphs`
|
||||
- `workflow/block-d-output-type-contracts`
|
||||
- `workflow/block-e-editor-organization`
|
||||
- `workflow/block-f-run-inspection`
|
||||
- `workflow/block-g-parity-golden-cases`
|
||||
- `workflow/block-h-cad-material-parity`
|
||||
- `workflow/block-i-rollout-regression-gates`
|
||||
|
||||
Preferred integration order:
|
||||
|
||||
1. Block A
|
||||
2. Block D
|
||||
3. Block B
|
||||
4. Block C
|
||||
5. Block E
|
||||
6. Block F
|
||||
7. Block H
|
||||
8. Block G
|
||||
9. Block I
|
||||
|
||||
## Worker Blocks
|
||||
|
||||
### Block A: Node Contract Closure
|
||||
|
||||
Purpose:
|
||||
Turn the node registry into the canonical module contract source for every editor-visible graph node.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_node_registry.py`
|
||||
- `backend/app/domains/rendering/workflow_schema.py`
|
||||
- `backend/tests/domains/test_workflow_node_registry.py`
|
||||
- `backend/tests/domains/test_workflow_schema.py`
|
||||
- `frontend/src/api/workflows.ts`
|
||||
|
||||
Tickets:
|
||||
|
||||
- audit every editor-visible node for missing or weak param schemas
|
||||
- close real settings gaps for graph-authorable nodes, especially export and CAD bridge nodes
|
||||
- make `family`, `module_key`, `input_contract`, `output_contract`, `artifact_roles_*`, and `legacy_source` authoritative and complete
|
||||
- strengthen family-aware graph validation in the backend schema layer
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- every editor-visible node has a machine-usable contract
|
||||
- no node required for still-render parity is missing editable settings
|
||||
- backend rejects family-invalid graphs with clear errors
|
||||
|
||||
Dependencies:
|
||||
|
||||
- none
|
||||
|
||||
### Block B: Module Unification
|
||||
|
||||
Purpose:
|
||||
Route graph and legacy execution through the same reusable production-module layer wherever parity nodes already exist.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_runtime_services.py`
|
||||
- `backend/app/domains/rendering/workflow_executor.py`
|
||||
- `backend/app/domains/rendering/workflow_graph_runtime.py`
|
||||
- `backend/app/domains/rendering/tasks.py`
|
||||
- `backend/app/domains/pipeline/tasks/render_order_line.py`
|
||||
- `backend/tests/domains/test_workflow_graph_runtime.py`
|
||||
- `backend/tests/domains/test_workflow_runtime_services.py`
|
||||
|
||||
Tickets:
|
||||
|
||||
- reduce duplicate behavior split across legacy task code and graph adapters
|
||||
- ensure async task wrappers are transport, not business-logic owners
|
||||
- close remaining drift for still, turntable, and blend export paths
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- graph and legacy call the same module behavior for extracted parity nodes
|
||||
- parity bug fixes land once in the shared module layer
|
||||
- fallback to legacy still works on graph failure
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
|
||||
### Block C: Canonical Starter Graphs And Seeds
|
||||
|
||||
Purpose:
|
||||
Keep live workflow definitions, blueprints, and starter templates aligned with the corrected canonical graphs.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_config_utils.py`
|
||||
- `backend/app/domains/rendering/workflow_builder.py`
|
||||
- `backend/app/api/routers/admin.py`
|
||||
- `frontend/src/api/workflows.ts`
|
||||
- seed/migration helpers and related tests
|
||||
|
||||
Tickets:
|
||||
|
||||
- maintain canonical `cad_file` and `order_line` starter graphs
|
||||
- backfill or normalize existing seeded workflow definitions safely
|
||||
- prevent drift between backend canonical graphs and frontend starter generation
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- starter workflows are family-safe and match backend canonical topology
|
||||
- existing seeded workflows can be repaired without manual DB surgery
|
||||
- frontend new-workflow creation matches backend canonical graph shape
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
|
||||
### Block D: Output-Type Invocation Contracts
|
||||
|
||||
Purpose:
|
||||
Turn output types into explicit workflow invocation profiles instead of loose legacy renderer presets.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/models/output_type.py`
|
||||
- `backend/app/domains/rendering/output_type_contracts.py`
|
||||
- `backend/app/api/routers/output_types.py`
|
||||
- `backend/alembic/versions/065_output_type_workflow_contracts.py`
|
||||
- `backend/alembic/versions/066_output_type_invocation_overrides.py`
|
||||
- `backend/tests/domains/test_output_types_api.py`
|
||||
- `frontend/src/api/outputTypes.ts`
|
||||
- `frontend/src/components/admin/OutputTypeTable.tsx`
|
||||
|
||||
Tickets:
|
||||
|
||||
- finalize explicit `workflow_family`, `artifact_kind`, and invocation override semantics
|
||||
- validate output-type and workflow compatibility in create/edit flows
|
||||
- separate productized invocation settings from raw legacy render settings
|
||||
- make create/edit UX workflow-first instead of renderer-first
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- new output types can be created reliably against the real backend contract
|
||||
- invalid workflow-family or artifact combinations are blocked early
|
||||
- legacy output types remain renderable without forced migration
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
|
||||
### Block E: Workflow Editor Organization
|
||||
|
||||
Purpose:
|
||||
Make the editor usable for production authoring by organizing nodes around families and modules instead of a flat mixed palette.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `frontend/src/pages/WorkflowEditor.tsx`
|
||||
- `frontend/src/components/workflows/`
|
||||
- `frontend/src/__tests__/components/`
|
||||
|
||||
Tickets:
|
||||
|
||||
- organize palette and search around family, module, and execution role
|
||||
- support low-friction node insertion from canvas context actions
|
||||
- reduce top-of-page control bloat and move graph actions closer to the canvas
|
||||
- improve graph readability with automatic alignment/layout and cleaner inspector behavior
|
||||
- preserve delete/edit flows for nodes and edges
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- authoring a family-safe graph is materially faster than today
|
||||
- node discovery works from both sidebar and canvas-local actions
|
||||
- editor UX follows the backend-owned contracts instead of bespoke per-node logic
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
- Block D for workflow-first output-type interactions that deep-link into editor usage
|
||||
|
||||
### Block F: Run Inspection And Workflow Debugging
|
||||
|
||||
Purpose:
|
||||
Make `/workflows` operationally useful by exposing preflight, dispatch, run records, node outputs, and failure surfaces cleanly.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/workflow_router.py`
|
||||
- `backend/app/domains/rendering/models.py`
|
||||
- `backend/app/domains/rendering/schemas.py`
|
||||
- `frontend/src/api/workflows.ts`
|
||||
- `frontend/src/components/workflows/`
|
||||
- runtime/run inspection tests
|
||||
|
||||
Tickets:
|
||||
|
||||
- complete workflow run inspection UI
|
||||
- expose node-level statuses, outputs, timings, and key artifacts
|
||||
- make preflight and manual dispatch part of the normal debugging flow
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- a failed graph run can be debugged from `/workflows` without jumping through ad hoc DB inspection
|
||||
- node-level outputs are visible enough to diagnose broken contracts and missing artifacts
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
- Block B
|
||||
|
||||
### Block G: Golden-Case Parity Suite
|
||||
|
||||
Purpose:
|
||||
Prove graph parity against legacy for representative render cases before broad rollout.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `scripts/test_render_pipeline.py`
|
||||
- parity fixtures, comparison tooling, and integration tests
|
||||
- `docs/workflows/WORKFLOW_DELIVERY_CHECKLIST.md`
|
||||
|
||||
Tickets:
|
||||
|
||||
- define representative still, turntable, blend, and failure golden cases
|
||||
- automate legacy-vs-graph comparison for outputs and key side effects
|
||||
- capture artifacts and logs needed for regression triage
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- golden cases pass for at least one real still-render graph and one legacy fallback path
|
||||
- parity regressions are reproducible and attributable to specific nodes or contracts
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block B
|
||||
- Block C
|
||||
- Block F
|
||||
- Block H for CAD/material-sensitive cases
|
||||
|
||||
### Block H: CAD/Material Parity
|
||||
|
||||
Purpose:
|
||||
Restore parity for GLTF/USD preview identity, material assignment, and instance-aware part mapping so graph workflows consume trustworthy geometry state.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/api/routers/cad.py`
|
||||
- `backend/app/api/routers/products.py`
|
||||
- `backend/app/domains/pipeline/tasks/export_glb.py`
|
||||
- `backend/app/domains/pipeline/tasks/extract_metadata.py`
|
||||
- `backend/app/services/part_key_service.py`
|
||||
- `frontend/src/components/cad/InlineCadViewer.tsx`
|
||||
- `frontend/src/components/cad/ThreeDViewer.tsx`
|
||||
- `frontend/src/components/cad/cadUtils.ts`
|
||||
- CAD/material tests
|
||||
|
||||
Tickets:
|
||||
|
||||
- close instance-aware part identity drift in viewer and export pipeline
|
||||
- restore reliable missing-material assignment and library mapping
|
||||
- ensure graph nodes consume the same geometry/material truth as legacy-render preparation
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- product preview and GLTF viewer show consistent material assignment on representative legacy products
|
||||
- manual material repair workflows remain intact
|
||||
- downstream render nodes receive stable part/material identities
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block A
|
||||
- Block B
|
||||
|
||||
### Block I: Rollout And Dual-Run Regression Gates
|
||||
|
||||
Purpose:
|
||||
Keep graph rollout safe by enforcing shadow-mode parity checks, fallback behavior, and rollback paths per workflow/output type.
|
||||
|
||||
Primary ownership:
|
||||
|
||||
- `backend/app/domains/rendering/dispatch_service.py`
|
||||
- `backend/app/domains/rendering/workflow_comparison_service.py`
|
||||
- `backend/tests/domains/test_workflow_dispatch_service.py`
|
||||
- rollout docs and runbooks
|
||||
|
||||
Tickets:
|
||||
|
||||
- formalize shadow-mode regression gates
|
||||
- make rollout decisions per workflow or output type
|
||||
- preserve immediate rollback to legacy
|
||||
- encode comparison thresholds and operational readiness checks
|
||||
|
||||
Acceptance criteria:
|
||||
|
||||
- shadow mode can run on representative workflows without affecting authoritative outputs
|
||||
- rollout and rollback are explicit and test-backed
|
||||
- graph error handling stays at or below legacy operational risk
|
||||
|
||||
Dependencies:
|
||||
|
||||
- Block B
|
||||
- Block G
|
||||
|
||||
## First-Wave Parallelization
|
||||
|
||||
Run these blocks in parallel first:
|
||||
|
||||
- Block A: Node contract closure
|
||||
- Block D: Output-type invocation contracts
|
||||
- Block E: Editor organization audit and contract-driven UX split
|
||||
- Block I: Rollout and dual-run regression gate design
|
||||
|
||||
Reason:
|
||||
|
||||
- they have the highest leverage on all remaining work
|
||||
- they define the contracts the later E2E and parity work must obey
|
||||
- they are separable enough to run in parallel with minimal file overlap if ownership is respected
|
||||
|
||||
## Cross-Block Handoffs
|
||||
|
||||
- Block A hands canonical node/module contracts to Blocks B, C, E, and H.
|
||||
- Block D hands stable output-type invocation semantics to Blocks E and I.
|
||||
- Block H hands trustworthy geometry/material identity to Blocks G and I.
|
||||
- Block F and G jointly define ship-readiness for `/workflows`.
|
||||
|
||||
## Quality Gates By Orchestration Stage
|
||||
|
||||
### Stage 1: Contract Gate
|
||||
|
||||
- Blocks A and D merged
|
||||
- backend and frontend contracts aligned
|
||||
- new workflows and new output types validate against the same source of truth
|
||||
|
||||
### Stage 2: Runtime Gate
|
||||
|
||||
- Block B merged
|
||||
- canonical starter graphs from Block C verified
|
||||
- no regression in legacy dispatch behavior
|
||||
|
||||
### Stage 3: Authoring Gate
|
||||
|
||||
- Block E and Block F merged
|
||||
- `/workflows` supports real authoring, preflight, dispatch, and inspection
|
||||
|
||||
### Stage 4: Parity Gate
|
||||
|
||||
- Block H and Block G merged
|
||||
- at least one non-legacy still-render workflow passes end to end against a representative product
|
||||
|
||||
### Stage 5: Rollout Gate
|
||||
|
||||
- Block I merged
|
||||
- shadow mode, rollback, and comparison thresholds are operationally usable
|
||||
Reference in New Issue
Block a user