385 lines
12 KiB
Markdown
385 lines
12 KiB
Markdown
# Workflow Worker Orchestration
|
|
|
|
## Purpose
|
|
|
|
This file defines the parallel work split to bring `/workflows` to full production-capable parity while keeping the legacy workflow operational at all times.
|
|
|
|
The orchestration is intentionally contract-first:
|
|
|
|
1. stabilize runtime and output-type contracts
|
|
2. formalize module-backed node definitions
|
|
3. organize editor authoring around those contracts
|
|
4. prove parity with dual-run and golden-case validation
|
|
|
|
## Non-Negotiable Constraints
|
|
|
|
- Legacy workflow dispatch must remain callable throughout the migration.
|
|
- Mixed-family graphs may exist for migration visibility, but new authoring must converge on family-safe graphs.
|
|
- No worker may revert unrelated user changes in the dirty worktree.
|
|
- `cad_file` and `order_line` concerns must be separated explicitly in contracts, editor palette organization, and output-type validation.
|
|
- A worker may only edit files in its assigned ownership scope unless the handoff section explicitly allows expansion.
|
|
|
|
## Shared Acceptance Baseline
|
|
|
|
Each worker block is only complete when all of the following hold:
|
|
|
|
- targeted tests for the touched scope pass
|
|
- legacy-safe behavior is preserved or explicitly guarded with fallback
|
|
- the changed API surface is reflected in frontend/backend types where applicable
|
|
- docs and seed/default data stay aligned with the runtime behavior
|
|
|
|
## Branching And Commit Order
|
|
|
|
Suggested branch naming:
|
|
|
|
- `workflow/block-a-node-contracts`
|
|
- `workflow/block-b-module-unification`
|
|
- `workflow/block-c-starter-graphs`
|
|
- `workflow/block-d-output-type-contracts`
|
|
- `workflow/block-e-editor-organization`
|
|
- `workflow/block-f-run-inspection`
|
|
- `workflow/block-g-parity-golden-cases`
|
|
- `workflow/block-h-cad-material-parity`
|
|
- `workflow/block-i-rollout-regression-gates`
|
|
|
|
Preferred integration order:
|
|
|
|
1. Block A
|
|
2. Block D
|
|
3. Block B
|
|
4. Block C
|
|
5. Block E
|
|
6. Block F
|
|
7. Block H
|
|
8. Block G
|
|
9. Block I
|
|
|
|
## Worker Blocks
|
|
|
|
### Block A: Node Contract Closure
|
|
|
|
Purpose:
|
|
Turn the node registry into the canonical module contract source for every editor-visible graph node.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/domains/rendering/workflow_node_registry.py`
|
|
- `backend/app/domains/rendering/workflow_schema.py`
|
|
- `backend/tests/domains/test_workflow_node_registry.py`
|
|
- `backend/tests/domains/test_workflow_schema.py`
|
|
- `frontend/src/api/workflows.ts`
|
|
|
|
Tickets:
|
|
|
|
- audit every editor-visible node for missing or weak param schemas
|
|
- close real settings gaps for graph-authorable nodes, especially export and CAD bridge nodes
|
|
- make `family`, `module_key`, `input_contract`, `output_contract`, `artifact_roles_*`, and `legacy_source` authoritative and complete
|
|
- strengthen family-aware graph validation in the backend schema layer
|
|
|
|
Acceptance criteria:
|
|
|
|
- every editor-visible node has a machine-usable contract
|
|
- no node required for still-render parity is missing editable settings
|
|
- backend rejects family-invalid graphs with clear errors
|
|
|
|
Dependencies:
|
|
|
|
- none
|
|
|
|
### Block B: Module Unification
|
|
|
|
Purpose:
|
|
Route graph and legacy execution through the same reusable production-module layer wherever parity nodes already exist.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/domains/rendering/workflow_runtime_services.py`
|
|
- `backend/app/domains/rendering/workflow_executor.py`
|
|
- `backend/app/domains/rendering/workflow_graph_runtime.py`
|
|
- `backend/app/domains/rendering/tasks.py`
|
|
- `backend/app/domains/pipeline/tasks/render_order_line.py`
|
|
- `backend/tests/domains/test_workflow_graph_runtime.py`
|
|
- `backend/tests/domains/test_workflow_runtime_services.py`
|
|
|
|
Tickets:
|
|
|
|
- reduce duplicate behavior split across legacy task code and graph adapters
|
|
- ensure async task wrappers are transport, not business-logic owners
|
|
- close remaining drift for still, turntable, and blend export paths
|
|
|
|
Acceptance criteria:
|
|
|
|
- graph and legacy call the same module behavior for extracted parity nodes
|
|
- parity bug fixes land once in the shared module layer
|
|
- fallback to legacy still works on graph failure
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
|
|
### Block C: Canonical Starter Graphs And Seeds
|
|
|
|
Purpose:
|
|
Keep live workflow definitions, blueprints, and starter templates aligned with the corrected canonical graphs.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/domains/rendering/workflow_config_utils.py`
|
|
- `backend/app/domains/rendering/workflow_builder.py`
|
|
- `backend/app/api/routers/admin.py`
|
|
- `frontend/src/api/workflows.ts`
|
|
- seed/migration helpers and related tests
|
|
|
|
Tickets:
|
|
|
|
- maintain canonical `cad_file` and `order_line` starter graphs
|
|
- backfill or normalize existing seeded workflow definitions safely
|
|
- prevent drift between backend canonical graphs and frontend starter generation
|
|
|
|
Acceptance criteria:
|
|
|
|
- starter workflows are family-safe and match backend canonical topology
|
|
- existing seeded workflows can be repaired without manual DB surgery
|
|
- frontend new-workflow creation matches backend canonical graph shape
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
|
|
### Block D: Output-Type Invocation Contracts
|
|
|
|
Purpose:
|
|
Turn output types into explicit workflow invocation profiles instead of loose legacy renderer presets.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/models/output_type.py`
|
|
- `backend/app/domains/rendering/output_type_contracts.py`
|
|
- `backend/app/api/routers/output_types.py`
|
|
- `backend/alembic/versions/065_output_type_workflow_contracts.py`
|
|
- `backend/alembic/versions/066_output_type_invocation_overrides.py`
|
|
- `backend/tests/domains/test_output_types_api.py`
|
|
- `frontend/src/api/outputTypes.ts`
|
|
- `frontend/src/components/admin/OutputTypeTable.tsx`
|
|
|
|
Tickets:
|
|
|
|
- finalize explicit `workflow_family`, `artifact_kind`, and invocation override semantics
|
|
- validate output-type and workflow compatibility in create/edit flows
|
|
- separate productized invocation settings from raw legacy render settings
|
|
- make create/edit UX workflow-first instead of renderer-first
|
|
|
|
Acceptance criteria:
|
|
|
|
- new output types can be created reliably against the real backend contract
|
|
- invalid workflow-family or artifact combinations are blocked early
|
|
- legacy output types remain renderable without forced migration
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
|
|
### Block E: Workflow Editor Organization
|
|
|
|
Purpose:
|
|
Make the editor usable for production authoring by organizing nodes around families and modules instead of a flat mixed palette.
|
|
|
|
Primary ownership:
|
|
|
|
- `frontend/src/pages/WorkflowEditor.tsx`
|
|
- `frontend/src/components/workflows/`
|
|
- `frontend/src/__tests__/components/`
|
|
|
|
Tickets:
|
|
|
|
- organize palette and search around family, module, and execution role
|
|
- support low-friction node insertion from canvas context actions
|
|
- reduce top-of-page control bloat and move graph actions closer to the canvas
|
|
- improve graph readability with automatic alignment/layout and cleaner inspector behavior
|
|
- preserve delete/edit flows for nodes and edges
|
|
|
|
Acceptance criteria:
|
|
|
|
- authoring a family-safe graph is materially faster than today
|
|
- node discovery works from both sidebar and canvas-local actions
|
|
- editor UX follows the backend-owned contracts instead of bespoke per-node logic
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
- Block D for workflow-first output-type interactions that deep-link into editor usage
|
|
|
|
### Block F: Run Inspection And Workflow Debugging
|
|
|
|
Purpose:
|
|
Make `/workflows` operationally useful by exposing preflight, dispatch, run records, node outputs, and failure surfaces cleanly.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/domains/rendering/workflow_router.py`
|
|
- `backend/app/domains/rendering/models.py`
|
|
- `backend/app/domains/rendering/schemas.py`
|
|
- `frontend/src/api/workflows.ts`
|
|
- `frontend/src/components/workflows/`
|
|
- runtime/run inspection tests
|
|
|
|
Tickets:
|
|
|
|
- complete workflow run inspection UI
|
|
- expose node-level statuses, outputs, timings, and key artifacts
|
|
- make preflight and manual dispatch part of the normal debugging flow
|
|
|
|
Acceptance criteria:
|
|
|
|
- a failed graph run can be debugged from `/workflows` without jumping through ad hoc DB inspection
|
|
- node-level outputs are visible enough to diagnose broken contracts and missing artifacts
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
- Block B
|
|
|
|
### Block G: Golden-Case Parity Suite
|
|
|
|
Purpose:
|
|
Prove graph parity against legacy for representative render cases before broad rollout.
|
|
|
|
Primary ownership:
|
|
|
|
- `scripts/test_render_pipeline.py`
|
|
- parity fixtures, comparison tooling, and integration tests
|
|
- `docs/workflows/WORKFLOW_DELIVERY_CHECKLIST.md`
|
|
|
|
Tickets:
|
|
|
|
- define representative still, turntable, blend, and failure golden cases
|
|
- automate legacy-vs-graph comparison for outputs and key side effects
|
|
- capture artifacts and logs needed for regression triage
|
|
|
|
Acceptance criteria:
|
|
|
|
- golden cases pass for at least one real still-render graph and one legacy fallback path
|
|
- parity regressions are reproducible and attributable to specific nodes or contracts
|
|
|
|
Dependencies:
|
|
|
|
- Block B
|
|
- Block C
|
|
- Block F
|
|
- Block H for CAD/material-sensitive cases
|
|
|
|
### Block H: CAD/Material Parity
|
|
|
|
Purpose:
|
|
Restore parity for GLTF/USD preview identity, material assignment, and instance-aware part mapping so graph workflows consume trustworthy geometry state.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/api/routers/cad.py`
|
|
- `backend/app/api/routers/products.py`
|
|
- `backend/app/domains/pipeline/tasks/export_glb.py`
|
|
- `backend/app/domains/pipeline/tasks/extract_metadata.py`
|
|
- `backend/app/services/part_key_service.py`
|
|
- `frontend/src/components/cad/InlineCadViewer.tsx`
|
|
- `frontend/src/components/cad/ThreeDViewer.tsx`
|
|
- `frontend/src/components/cad/cadUtils.ts`
|
|
- CAD/material tests
|
|
|
|
Tickets:
|
|
|
|
- close instance-aware part identity drift in viewer and export pipeline
|
|
- restore reliable missing-material assignment and library mapping
|
|
- ensure graph nodes consume the same geometry/material truth as legacy-render preparation
|
|
|
|
Acceptance criteria:
|
|
|
|
- product preview and GLTF viewer show consistent material assignment on representative legacy products
|
|
- manual material repair workflows remain intact
|
|
- downstream render nodes receive stable part/material identities
|
|
|
|
Dependencies:
|
|
|
|
- Block A
|
|
- Block B
|
|
|
|
### Block I: Rollout And Dual-Run Regression Gates
|
|
|
|
Purpose:
|
|
Keep graph rollout safe by enforcing shadow-mode parity checks, fallback behavior, and rollback paths per workflow/output type.
|
|
|
|
Primary ownership:
|
|
|
|
- `backend/app/domains/rendering/dispatch_service.py`
|
|
- `backend/app/domains/rendering/workflow_comparison_service.py`
|
|
- `backend/tests/domains/test_workflow_dispatch_service.py`
|
|
- rollout docs and runbooks
|
|
|
|
Tickets:
|
|
|
|
- formalize shadow-mode regression gates
|
|
- make rollout decisions per workflow or output type
|
|
- preserve immediate rollback to legacy
|
|
- encode comparison thresholds and operational readiness checks
|
|
|
|
Acceptance criteria:
|
|
|
|
- shadow mode can run on representative workflows without affecting authoritative outputs
|
|
- rollout and rollback are explicit and test-backed
|
|
- graph error handling stays at or below legacy operational risk
|
|
|
|
Dependencies:
|
|
|
|
- Block B
|
|
- Block G
|
|
|
|
## First-Wave Parallelization
|
|
|
|
Run these blocks in parallel first:
|
|
|
|
- Block A: Node contract closure
|
|
- Block D: Output-type invocation contracts
|
|
- Block E: Editor organization audit and contract-driven UX split
|
|
- Block I: Rollout and dual-run regression gate design
|
|
|
|
Reason:
|
|
|
|
- they have the highest leverage on all remaining work
|
|
- they define the contracts the later E2E and parity work must obey
|
|
- they are separable enough to run in parallel with minimal file overlap if ownership is respected
|
|
|
|
## Cross-Block Handoffs
|
|
|
|
- Block A hands canonical node/module contracts to Blocks B, C, E, and H.
|
|
- Block D hands stable output-type invocation semantics to Blocks E and I.
|
|
- Block H hands trustworthy geometry/material identity to Blocks G and I.
|
|
- Block F and G jointly define ship-readiness for `/workflows`.
|
|
|
|
## Quality Gates By Orchestration Stage
|
|
|
|
### Stage 1: Contract Gate
|
|
|
|
- Blocks A and D merged
|
|
- backend and frontend contracts aligned
|
|
- new workflows and new output types validate against the same source of truth
|
|
|
|
### Stage 2: Runtime Gate
|
|
|
|
- Block B merged
|
|
- canonical starter graphs from Block C verified
|
|
- no regression in legacy dispatch behavior
|
|
|
|
### Stage 3: Authoring Gate
|
|
|
|
- Block E and Block F merged
|
|
- `/workflows` supports real authoring, preflight, dispatch, and inspection
|
|
|
|
### Stage 4: Parity Gate
|
|
|
|
- Block H and Block G merged
|
|
- at least one non-legacy still-render workflow passes end to end against a representative product
|
|
|
|
### Stage 5: Rollout Gate
|
|
|
|
- Block I merged
|
|
- shadow mode, rollback, and comparison thresholds are operationally usable
|