docs: add workflow migration plan and checkpoint current state

2026-04-07 08:38:16 +02:00
parent 2a00abe91f
commit bc9ab5f864
5 changed files with 322 additions and 15 deletions
@@ -0,0 +1,95 @@
+# Workflow Delivery Checklist
+
+## Phase Checklist
+
+### Phase 1
+
+- [ ] Canonical workflow schema finalized
+- [ ] Frontend and backend workflow types aligned
+- [ ] Preset workflow migration helpers added
+- [ ] Tests added for legacy preset conversion
+- [ ] Legacy dispatch remains default
+
+### Phase 2
+
+- [ ] Node registry implemented
+- [ ] Node definitions API available
+- [ ] All required nodes have settings schemas
+- [ ] Editor consumes node definitions from backend
+
+### Phase 3
+
+- [ ] Missing legacy steps extracted into reusable executors
+- [ ] Extracted node behavior matches legacy services
+- [ ] Node-level tests cover success and failure paths
+
+### Phase 4
+
+- [ ] Workflow context introduced
+- [ ] Node outputs are persisted and reusable
+- [ ] Graph runtime supports legacy fallback
+- [ ] `legacy`, `graph`, and `shadow` modes exist
+
+### Phase 5
+
+- [ ] Editor saves nodes and edges
+- [ ] Editor roundtrip preserves workflow configs
+- [ ] All node settings are editable
+- [ ] Validate, dry-run, and dispatch are available
+- [ ] Runs are visible with node-level status and logs
+
+### Phase 6
+
+- [ ] Shadow mode parity checks run on real workflows
+- [ ] Golden cases pass against legacy outputs
+- [ ] Rollout can be enabled per workflow or output type
+- [ ] Rollback to legacy is immediate
+
+## Quality Gates
+
+### QG-1: Model Gate
+
+- New workflow saves only use canonical schema.
+- Backend rejects malformed configs with clear errors.
+- Existing preset workflows can be migrated without data loss.
+
+### QG-2: Node Gate
+
+- Every editor-visible node has:
+  - backend node definition
+  - validated settings schema
+  - default params
+  - executor coverage or explicit disabled status
+
+### QG-3: Legacy Safety Gate
+
+- Legacy render dispatch remains callable and unchanged in behavior.
+- Existing output types still render without workflow migration.
+- Graph failures do not block legacy renders.
+
+### QG-4: Parity Gate
+
+- Golden cases match on:
+  - render status
+  - generated output file
+  - media asset creation
+  - notifications
+  - core render log fields
+
+### QG-5: Editor Gate
+
+- Workflow configs survive save/load roundtrip without loss.
+- Invalid graphs are blocked before dispatch.
+- All node settings needed for parity are present in the editor.
+
+### QG-6: Rollout Gate
+
+- Shadow mode has been exercised on representative workflows.
+- Graph runtime error rate is at or below legacy error rate.
+- Rollout and rollback are possible per workflow or output type.
+
+## Definition of Done
+
+- `/workflows` is production-capable for authoring and running workflows.
+- Legacy functionality is available in graph form with parity coverage.
+- Legacy execution still exists as a supported fallback.
@@ -0,0 +1,99 @@
+# Workflow Implementation Backlog
+
+## Epic 1: Canonical Workflow Model
+
+### Tickets
+
+- `E1-T1` Finalize canonical workflow JSON schema with `version`, `nodes`, `edges`, and optional `ui`.
+- `E1-T2` Align frontend workflow API types with the backend schema.
+- `E1-T3` Extend backend validation for duplicate IDs, invalid edges, unknown steps, and invalid params.
+- `E1-T4` Add migration helpers for existing preset configs: `still`, `turntable`, `multi_angle`, `still_with_exports`.
+- `E1-T5` Add tests for preset-to-DAG migration.
+
+### Primary Files
+
+- [`backend/app/domains/rendering/workflow_schema.py`](/home/hartmut/Documents/Copilot/schaefflerautomat/backend/app/domains/rendering/workflow_schema.py)
+- [`frontend/src/api/workflows.ts`](/home/hartmut/Documents/Copilot/schaefflerautomat/frontend/src/api/workflows.ts)
+- [`frontend/src/pages/WorkflowEditor.tsx`](/home/hartmut/Documents/Copilot/schaefflerautomat/frontend/src/pages/WorkflowEditor.tsx)
+
+## Epic 2: Node Registry and Settings Schemas
+
+### Tickets
+
+- `E2-T1` Add a backend `NodeDefinitionRegistry`.
+- `E2-T2` Define metadata and settings schema for each workflow node.
+- `E2-T3` Add `GET /api/workflows/node-definitions`.
+- `E2-T4` Provide schema-driven defaults and editor field groups.
+- `E2-T5` Add composite bridge nodes for safe migration.
+
+### Required Node Coverage
+
+- `resolve_step_path`
+- `occ_object_extract`
+- `occ_glb_export`
+- `glb_bbox`
+- `material_map_resolve`
+- `auto_populate_materials`
+- `blender_render`
+- `threejs_render`
+- `thumbnail_save`
+- `order_line_setup`
+- `resolve_template`
+- `blender_still`
+- `blender_turntable`
+- `output_save`
+- `export_blend`
+- `stl_cache_generate`
+- `notify`
+
+## Epic 3: Legacy Step Extraction
+
+### Tickets
+
+- `E3-T1` Create a parity matrix from the legacy render pipeline.
+- `E3-T2` Extract `order_line_setup` into a reusable service/task.
+- `E3-T3` Extract `resolve_template`.
+- `E3-T4` Extract `material_map_resolve`.
+- `E3-T5` Extract `auto_populate_materials`.
+- `E3-T6` Extract `glb_bbox`.
+- `E3-T7` Extract `output_save`.
+- `E3-T8` Extract `notify`.
+- `E3-T9` Add executor tests for all extracted nodes.
+
+### Legacy Sources
+
+- [`backend/app/domains/pipeline/tasks/render_order_line.py`](/home/hartmut/Documents/Copilot/schaefflerautomat/backend/app/domains/pipeline/tasks/render_order_line.py)
+- [`backend/app/domains/rendering/tasks.py`](/home/hartmut/Documents/Copilot/schaefflerautomat/backend/app/domains/rendering/tasks.py)
+- [`backend/app/core/process_steps.py`](/home/hartmut/Documents/Copilot/schaefflerautomat/backend/app/core/process_steps.py)
+
+## Epic 4: Graph Runtime
+
+### Tickets
+
+- `E4-T1` Introduce `WorkflowContext`.
+- `E4-T2` Refactor executor to process nodes against context and node outputs.
+- `E4-T3` Persist node-level run records, logs, timings, and outputs.
+- `E4-T4` Support retry and failure policies.
+- `E4-T5` Add execution mode switch: `legacy`, `graph`, `shadow`.
+- `E4-T6` Add hard fallback to legacy dispatch on graph failure.
+
+## Epic 5: Editor Parity
+
+### Tickets
+
+- `E5-T1` Save and load `edges`.
+- `E5-T2` Replace `pipeline_step` in node UI data with canonical node `step`.
+- `E5-T3` Render node settings from backend schemas.
+- `E5-T4` Add graph validation in the editor.
+- `E5-T5` Add dry-run and dispatch from the editor.
+- `E5-T6` Add workflow run inspection UI.
+
+## Epic 6: Rollout and Quality
+
+### Tickets
+
+- `E6-T1` Add shadow mode parity execution.
+- `E6-T2` Build output comparison tooling.
+- `E6-T3` Define golden test cases.
+- `E6-T4` Roll out per workflow or output type.
+- `E6-T5` Keep legacy fallback after rollout.
@@ -0,0 +1,108 @@
+# Workflow Migration Plan
+
+## Goal
+
+Bring `/workflows` to full production parity with the existing legacy render pipeline without breaking the current legacy path at any time.
+
+## Non-Negotiables
+
+- The legacy render path remains operational throughout the migration.
+- No existing output type or order flow may regress while the graph engine is introduced.
+- Every node exposed in the workflow editor must have a backend-owned definition and a validated settings schema.
+- New workflow definitions use one canonical JSON format.
+
+## Canonical Workflow Model
+
+```json
+{
+  "version": 1,
+  "nodes": [
+    {
+      "id": "resolve_template",
+      "step": "resolve_template",
+      "params": {},
+      "ui": {
+        "type": "templateNode",
+        "position": { "x": 420, "y": 120 },
+        "label": "Resolve Template"
+      }
+    }
+  ],
+  "edges": [
+    { "from": "setup", "to": "resolve_template" }
+  ]
+}
+```
+
+Notes:
+
+- `step` and `params` are the backend-owned runtime model.
+- `ui` is editor-only metadata and must not change runtime semantics.
+- `edges` are mandatory for graph persistence and validation.
+
+## Phases
+
+### Phase 1: Canonical Model and Migration Base
+
+- Finalize canonical workflow schema.
+- Align frontend API types with backend schema.
+- Add compatibility migration from preset configs to canonical DAG configs.
+- Keep legacy dispatch as default.
+
+### Phase 2: Node Registry
+
+- Introduce a backend node registry as the single source of truth.
+- Define labels, categories, descriptions, settings schemas, defaults, input contracts, and output contracts for every node.
+- Expose node definitions over API for the editor.
+
+### Phase 3: Extract Missing Legacy Steps
+
+- Move inline legacy pipeline logic into explicit node executors/services.
+- Required parity nodes:
+  - `order_line_setup`
+  - `resolve_template`
+  - `material_map_resolve`
+  - `auto_populate_materials`
+  - `glb_bbox`
+  - `output_save`
+  - `notify`
+- Optional but planned:
+  - `threejs_render`
+
+### Phase 4: Graph Runtime
+
+- Introduce `WorkflowContext` and node-by-node execution with persistent run state.
+- Support node outputs and artifact handoff across edges.
+- Keep `legacy`, `graph`, and `shadow` execution modes.
+
+### Phase 5: Workflow Editor Parity
+
+- Persist and load `nodes`, `edges`, `step`, `params`, and `ui`.
+- Render all node settings dynamically from backend schemas.
+- Add validation, dry-run, dispatch, and run inspection.
+
+### Phase 6: Shadow Mode and Rollout
+
+- Run graph executions in parallel to the legacy flow for parity comparison.
+- Compare status, outputs, media assets, notifications, and logs.
+- Roll out per workflow or output type.
+
+## Execution Modes
+
+- `legacy`: current production path only
+- `graph`: graph runtime is authoritative
+- `shadow`: legacy remains authoritative, graph runs for comparison and observability
+
+## Deliverables
+
+- Canonical workflow schema and migration tooling
+- Backend node registry and API
+- Full editor support for all node settings
+- Graph runtime with node-level execution and tracking
+- Shadow mode parity checks
+
+## Exit Criteria
+
+- A workflow can be authored, validated, executed, and inspected entirely via `/workflows`.
+- Legacy results and graph results match for defined golden cases.
+- Legacy fallback is still available after graph rollout.
@@ -1706,7 +1706,11 @@ function AssetLibraryPanel() {
  })

  const createMut = useMutation({
-    mutationFn: () => createAssetLibrary({ name: newName, description: newDesc || undefined, blend_file: newFile! }),
+    mutationFn: () => createAssetLibrary({
+      name: newName.trim(),
+      description: newDesc.trim() || undefined,
+      blend_file: newFile!,
+    }),
    onSuccess: () => {
      toast.success('Asset library created')
      qc.invalidateQueries({ queryKey: ['asset-libraries'] })
@@ -1760,32 +1764,33 @@ function AssetLibraryPanel() {
        <div className="p-4 border-b border-border-light bg-surface-alt space-y-3">
          <div className="flex gap-3">
            <input
-              className="input flex-1"
+              className="input-base flex-1"
              placeholder="Library name"
              value={newName}
              onChange={(e) => setNewName(e.target.value)}
            />
            <input
-              className="input flex-1"
+              className="input-base flex-1"
              placeholder="Description (optional)"
              value={newDesc}
              onChange={(e) => setNewDesc(e.target.value)}
            />
          </div>
-          <div className="flex items-center gap-3">
-            <label className="btn-secondary cursor-pointer">
-              <Upload size={14} />
-              {newFile ? newFile.name : 'Choose .blend file'}
-              <input
-                type="file"
-                accept=".blend"
-                className="hidden"
-                onChange={(e) => setNewFile(e.target.files?.[0] ?? null)}
-              />
-            </label>
+          <div className="flex items-center gap-3 flex-wrap">
+            <input
+              type="file"
+              accept=".blend,application/octet-stream"
+              className="input-base min-w-[18rem] flex-1 file:mr-3 file:rounded-md file:border-0 file:bg-accent file:px-3 file:py-1.5 file:text-sm file:font-medium file:text-accent-text hover:file:bg-accent-hover"
+              onChange={(e) => setNewFile(e.target.files?.[0] ?? null)}
+            />
+            {newFile && (
+              <span className="text-xs text-content-muted font-mono">
+                {newFile.name}
+              </span>
+            )}
            <button
              className="btn-primary"
-              disabled={!newName || !newFile || createMut.isPending}
+              disabled={!newName.trim() || !newFile || createMut.isPending}
              onClick={() => createMut.mutate()}
            >
              {createMut.isPending ? 'Creating...' : 'Create'}