chore: snapshot workflow migration progress
This commit is contained in:
@@ -0,0 +1,245 @@
|
||||
# Node-Based Production Architecture
|
||||
|
||||
## Purpose
|
||||
|
||||
Define the target model for a reusable, node-based production system where workflow steps are backend-owned modules, the editor is schema-driven, and legacy rendering stays operational during migration.
|
||||
|
||||
## Problem Statement
|
||||
|
||||
The current workflow system already has meaningful extraction work:
|
||||
|
||||
- bridge/runtime services exist for setup, template resolution, material mapping, bbox resolution, publish, and notify
|
||||
- graph execution can already orchestrate still, turntable, and blend export flows
|
||||
- the editor already consumes backend node definitions
|
||||
|
||||
What is still missing is a clean production model.
|
||||
|
||||
Today, three different concerns are still partially collapsed into each other:
|
||||
|
||||
1. `OutputType` as user-facing commercial/render choice
|
||||
2. workflow graph as orchestration definition
|
||||
3. legacy/internal render settings as implicit execution contract
|
||||
|
||||
That makes it hard to:
|
||||
|
||||
- reuse a process step like CAD import as a true module
|
||||
- expose all node settings coherently in the editor
|
||||
- bind output types to workflows without fragile implicit assumptions
|
||||
- preserve legacy behavior while enabling graph-native production
|
||||
|
||||
## Target Model
|
||||
|
||||
### 1. Production Module
|
||||
|
||||
A production module is the canonical backend capability unit.
|
||||
|
||||
Examples:
|
||||
|
||||
- `cad.resolve_step_path`
|
||||
- `cad.extract_objects`
|
||||
- `cad.export_glb`
|
||||
- `cad.compute_bbox`
|
||||
- `materials.resolve_map`
|
||||
- `materials.auto_populate`
|
||||
- `render.resolve_template`
|
||||
- `render.blender_still`
|
||||
- `render.blender_turntable`
|
||||
- `output.publish_asset`
|
||||
- `output.notify`
|
||||
|
||||
Rules:
|
||||
|
||||
- modules are backend-owned
|
||||
- modules define typed input contract, output contract, defaults, and execution semantics
|
||||
- modules are reusable from legacy code, graph runtime, shadow mode, and tests
|
||||
- modules must not depend on editor-only UI metadata
|
||||
|
||||
### 2. Workflow Node
|
||||
|
||||
A workflow node is an orchestration wrapper around a production module.
|
||||
|
||||
It adds:
|
||||
|
||||
- node id
|
||||
- graph connectivity
|
||||
- per-instance parameter overrides
|
||||
- editor UI metadata
|
||||
- retry/failure policy
|
||||
|
||||
It must not redefine business behavior that belongs to the production module itself.
|
||||
|
||||
Implication:
|
||||
|
||||
- the node registry should evolve from "palette metadata + field definitions" to "module-backed node definitions"
|
||||
- `step` should remain the stable runtime key, but internally map to a reusable module contract
|
||||
|
||||
### 3. Workflow Family
|
||||
|
||||
Workflows must be separated into runtime families, not inferred ad hoc:
|
||||
|
||||
- `cad_file`
|
||||
- `order_line`
|
||||
|
||||
Family drives:
|
||||
|
||||
- valid entry context
|
||||
- allowed node palette
|
||||
- validation rules
|
||||
- available output contracts
|
||||
- preflight expectations
|
||||
|
||||
Mixed-family graphs may still exist temporarily for migration visibility, but must not be the target authoring model.
|
||||
|
||||
### 4. Output Type as Invocation Profile
|
||||
|
||||
`OutputType` should no longer be treated as a loose bag of renderer flags.
|
||||
|
||||
It should be the product-facing invocation profile for a workflow:
|
||||
|
||||
- commercial name and visibility
|
||||
- compatible categories
|
||||
- pricing tier binding
|
||||
- workflow family
|
||||
- linked workflow definition
|
||||
- invocation-level parameter overrides
|
||||
- output artifact contract
|
||||
|
||||
Examples of invocation-level overrides:
|
||||
|
||||
- resolution
|
||||
- samples
|
||||
- engine
|
||||
- transparency
|
||||
- animation timing
|
||||
- material override
|
||||
|
||||
Examples of artifact contract:
|
||||
|
||||
- still image
|
||||
- turntable video
|
||||
- production `.blend`
|
||||
- preview thumbnail
|
||||
- future exported package types
|
||||
|
||||
This keeps the responsibility split clean:
|
||||
|
||||
- workflow definition answers: "what steps run and in what order?"
|
||||
- output type answers: "what productized variant of that workflow do we sell and with which defaults/constraints?"
|
||||
|
||||
## Required Refactor Direction
|
||||
|
||||
### A. Formalize Node Contracts
|
||||
|
||||
Extend the node registry so each definition exposes:
|
||||
|
||||
- `family`
|
||||
- `module_key`
|
||||
- `input_contract`
|
||||
- `output_contract`
|
||||
- `param_schema`
|
||||
- `artifact_roles_produced`
|
||||
- `artifact_roles_consumed`
|
||||
- `legacy_source`
|
||||
|
||||
Current definitions already cover labels, categories, defaults, and fields. They do not yet fully express machine-usable production contracts.
|
||||
|
||||
### B. Promote Runtime Services to Module Layer
|
||||
|
||||
The extracted bridge/runtime services are the right foundation. They now need a clear module boundary so both legacy and graph runtimes call the same backend operation layer.
|
||||
|
||||
Desired shape:
|
||||
|
||||
- legacy pipeline calls module layer directly
|
||||
- graph runtime calls module layer directly or via async task adapters
|
||||
- Celery task mapping becomes transport/adaptation, not the primary execution model
|
||||
|
||||
### C. Split Graph Authoring by Family
|
||||
|
||||
The editor should author against family-scoped graphs:
|
||||
|
||||
- CAD Intake graph
|
||||
- Order Rendering graph
|
||||
|
||||
That includes:
|
||||
|
||||
- family-specific starter templates
|
||||
- family-specific node palette groups
|
||||
- validation that rejects wrong-family entry nodes early
|
||||
- cleaner organization than a single mixed library
|
||||
|
||||
### D. Reframe Output Type Creation
|
||||
|
||||
Output type creation is currently too close to legacy render settings and too far from workflow invocation.
|
||||
|
||||
Create/edit flow should become:
|
||||
|
||||
1. choose family
|
||||
2. choose or create workflow
|
||||
3. choose artifact kind
|
||||
4. set invocation overrides
|
||||
5. bind pricing/category/material constraints
|
||||
|
||||
The current `workflow_definition_id` field is directionally correct, but too weak on its own because there is no explicit invocation contract or family validation around it yet.
|
||||
|
||||
## Compatibility Rules
|
||||
|
||||
### Legacy Safety
|
||||
|
||||
- legacy dispatch remains the fallback path
|
||||
- existing output types without workflow linkage remain valid
|
||||
- graph rollout must be opt-in per output type/workflow
|
||||
|
||||
### Migration Safety
|
||||
|
||||
- old output types may continue to store render settings in their current shape
|
||||
- a compatibility adapter should map legacy render settings into invocation overrides
|
||||
- workflow definitions must remain canonical JSON DAGs
|
||||
|
||||
## Recommended Implementation Sequence
|
||||
|
||||
### Phase A: Stabilize Broken Contracts
|
||||
|
||||
- align frontend/backend `OutputType` defaults and allowed values
|
||||
- add backend validation for output-type family/workflow compatibility
|
||||
- make output type creation/editing reflect current real backend constraints
|
||||
|
||||
### Phase B: Contract-First Registry
|
||||
|
||||
- add `family`, contracts, and module metadata to node definitions
|
||||
- expose them over `/api/workflows/node-definitions`
|
||||
- move editor grouping/validation to registry-owned family metadata
|
||||
|
||||
### Phase C: Invocation Profiles
|
||||
|
||||
- extend `OutputType` into a workflow invocation profile
|
||||
- add explicit artifact kind and workflow family
|
||||
- separate invocation overrides from raw render settings
|
||||
|
||||
### Phase D: Module Unification
|
||||
|
||||
- route legacy and graph execution through the same module layer
|
||||
- keep Celery as transport where async work is needed
|
||||
- reduce duplicate logic in tasks and runtime adapters
|
||||
|
||||
### Phase E: Full Parity Authoring
|
||||
|
||||
- ship family-specific starter workflows
|
||||
- expose all module settings in editor
|
||||
- support end-to-end preflight, dispatch, run inspection, and parity verification
|
||||
|
||||
## Immediate Code Implications
|
||||
|
||||
- `workflow_node_registry.py` is the correct extension point for module contracts
|
||||
- `workflow_schema.py` will need stronger family- and contract-aware validation
|
||||
- `OutputType` needs a clearer model than raw renderer/backend defaults plus optional workflow id
|
||||
- editor UX should follow model cleanup, not lead it
|
||||
|
||||
## Decision
|
||||
|
||||
We should simplify and refactor before doing more isolated workflow-editor UX work.
|
||||
|
||||
The next implementation blocks should prioritize:
|
||||
|
||||
1. fixing the output-type/workflow contract
|
||||
2. formalizing node/module contracts
|
||||
3. only then expanding editor affordances on top of the cleaned model
|
||||
Reference in New Issue
Block a user