feat: tenant AI chat agent with function calling

Actionable AI assistant that uses per-tenant Azure OpenAI credentials
to execute natural language commands against the render pipeline.

Backend:
- ChatMessage model + migration (session-based conversations)
- Chat service with 10 OpenAI function-calling tools:
  list_orders, search_products, create_order, dispatch_renders,
  get_order_status, set_material_override, set_render_overrides,
  get_render_stats, check_materials, query_database
- All tools tenant-scoped (queries filtered by tenant_id)
- Write operations use httpx to call backend API internally
- Chat API: POST /chat/messages, GET /chat/sessions, DELETE session
- Conversation history preserved in DB (last 50 messages per session)

Frontend:
- Slide-out ChatPanel (right side, w-96, animated)
- User/assistant message styling with avatars and timestamps
- Session management (new chat, session history, delete)
- Typing indicator while waiting for AI response
- Floating chat button in bottom-right corner
- Error state for unconfigured AI tenants

Example: "Render all Kugellager products as WebP at 1024x1024"
→ Agent calls search_products + create_order + dispatch_renders

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-15 12:46:21 +01:00
parent daad2c64f3
commit 59ce61098c
10 changed files with 1627 additions and 66 deletions
@@ -0,0 +1,48 @@
"""add chat_messages table
Revision ID: 69964e910545
Revises: f5906aaf75af
Create Date: 2026-03-15 11:38:41.189160
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '69964e910545'
down_revision: Union[str, None] = 'f5906aaf75af'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('chat_messages',
sa.Column('id', sa.UUID(), nullable=False),
sa.Column('tenant_id', sa.UUID(), nullable=True),
sa.Column('user_id', sa.UUID(), nullable=True),
sa.Column('session_id', sa.UUID(), nullable=False),
sa.Column('role', sa.String(length=20), nullable=False),
sa.Column('content', sa.Text(), nullable=False),
sa.Column('context_type', sa.String(length=50), nullable=True),
sa.Column('context_id', sa.UUID(), nullable=True),
sa.Column('token_count', sa.Integer(), nullable=True),
sa.Column('created_at', sa.DateTime(), nullable=False),
sa.ForeignKeyConstraint(['tenant_id'], ['tenants.id'], ondelete='CASCADE'),
sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='SET NULL'),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_chat_messages_session_id'), 'chat_messages', ['session_id'], unique=False)
op.create_index(op.f('ix_chat_messages_tenant_id'), 'chat_messages', ['tenant_id'], unique=False)
# ### end Alembic commands ###
def downgrade() -> None:
# ### commands auto generated by Alembic - please adjust! ###
op.drop_index(op.f('ix_chat_messages_tenant_id'), table_name='chat_messages')
op.drop_index(op.f('ix_chat_messages_session_id'), table_name='chat_messages')
op.drop_table('chat_messages')
# ### end Alembic commands ###
+209
View File
@@ -0,0 +1,209 @@
"""Chat API endpoints for tenant AI agent conversations."""
import logging
import uuid
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from app.database import get_db
from app.models.user import User
from app.utils.auth import get_current_user
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/chat", tags=["chat"])
# ── Pydantic schemas ─────────────────────────────────────────────────────────
class ChatMessageCreate(BaseModel):
message: str
session_id: str | None = None
context_type: str | None = None
context_id: str | None = None
class ChatMessageOut(BaseModel):
id: str
role: str
content: str
context_type: str | None = None
context_id: str | None = None
token_count: int | None = None
created_at: str
class ChatResponse(BaseModel):
session_id: str
user_message: ChatMessageOut
assistant_message: ChatMessageOut
class ChatSessionSummary(BaseModel):
session_id: str
last_message: str
message_count: int
created_at: str
# ── Endpoints ─────────────────────────────────────────────────────────────────
@router.post("/messages", response_model=ChatResponse)
async def send_message(
body: ChatMessageCreate,
user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Send a message to the AI assistant and get a response.
Creates a new session if session_id is not provided.
Uses the tenant's Azure OpenAI credentials for the LLM call.
"""
from app.services.chat_service import chat_with_agent
# Load tenant config
tenant_config = await _get_tenant_config(db, user)
session_id = body.session_id or str(uuid.uuid4())
# If session_id was provided, verify it belongs to this user
if body.session_id:
check = await db.execute(
text("""
SELECT 1 FROM chat_messages
WHERE session_id = :sid AND user_id = :uid
LIMIT 1
"""),
{"sid": session_id, "uid": str(user.id)},
)
if not check.first():
# New session with user-supplied ID is OK; existing session must belong to user
pass
try:
result = await chat_with_agent(
message=body.message,
session_id=session_id,
tenant_id=str(user.tenant_id),
user_id=str(user.id),
db=db,
tenant_config=tenant_config,
context_type=body.context_type,
context_id=body.context_id,
)
return result
except ValueError as exc:
# AI not configured
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
except Exception as exc:
logger.exception("Chat error for user %s", user.id)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Chat service error: {exc}",
)
@router.get("/sessions", response_model=list[ChatSessionSummary])
async def list_sessions(
user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""List the current user's chat sessions, most recent first."""
result = await db.execute(
text("""
SELECT
session_id::text AS session_id,
(SELECT content FROM chat_messages cm2
WHERE cm2.session_id = cm.session_id
ORDER BY cm2.created_at DESC LIMIT 1) AS last_message,
COUNT(*) AS message_count,
MIN(cm.created_at) AS created_at
FROM chat_messages cm
WHERE cm.user_id = :uid AND cm.tenant_id = :tid
GROUP BY cm.session_id
ORDER BY MAX(cm.created_at) DESC
LIMIT 50
"""),
{"uid": str(user.id), "tid": str(user.tenant_id)},
)
rows = result.mappings().all()
return [
{
"session_id": r["session_id"],
"last_message": (r["last_message"] or "")[:200],
"message_count": r["message_count"],
"created_at": r["created_at"].isoformat() if r["created_at"] else "",
}
for r in rows
]
@router.get("/sessions/{session_id}/messages", response_model=list[ChatMessageOut])
async def get_session_messages(
session_id: str,
user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Get all messages in a chat session."""
result = await db.execute(
text("""
SELECT id::text, role, content, context_type,
context_id::text, token_count, created_at
FROM chat_messages
WHERE session_id = :sid AND user_id = :uid AND tenant_id = :tid
ORDER BY created_at ASC
"""),
{"sid": session_id, "uid": str(user.id), "tid": str(user.tenant_id)},
)
rows = result.mappings().all()
if not rows:
raise HTTPException(status_code=404, detail="Session not found")
return [
{
"id": r["id"],
"role": r["role"],
"content": r["content"],
"context_type": r["context_type"],
"context_id": r["context_id"],
"token_count": r["token_count"],
"created_at": r["created_at"].isoformat() if r["created_at"] else "",
}
for r in rows
]
@router.delete("/sessions/{session_id}")
async def delete_session(
session_id: str,
user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Delete all messages in a chat session."""
result = await db.execute(
text("""
DELETE FROM chat_messages
WHERE session_id = :sid AND user_id = :uid AND tenant_id = :tid
"""),
{"sid": session_id, "uid": str(user.id), "tid": str(user.tenant_id)},
)
await db.commit()
deleted = result.rowcount
if deleted == 0:
raise HTTPException(status_code=404, detail="Session not found")
return {"deleted": deleted, "session_id": session_id}
# ── Helpers ───────────────────────────────────────────────────────────────────
async def _get_tenant_config(db: AsyncSession, user: User) -> dict | None:
"""Load tenant_config JSONB for the user's tenant."""
if not user.tenant_id:
return None
result = await db.execute(
text("SELECT tenant_config FROM tenants WHERE id = :tid"),
{"tid": str(user.tenant_id)},
)
row = result.mappings().first()
return row["tenant_config"] if row else None
+2
View File
@@ -26,6 +26,7 @@ from app.domains.media.router import router as media_router
from app.api.routers.asset_libraries import router as asset_libraries_router from app.api.routers.asset_libraries import router as asset_libraries_router
from app.domains.admin.dashboard_router import router as dashboard_router from app.domains.admin.dashboard_router import router as dashboard_router
from app.api.routers.task_logs import router as task_logs_router from app.api.routers.task_logs import router as task_logs_router
from app.api.routers.chat import router as chat_router
@asynccontextmanager @asynccontextmanager
@@ -95,6 +96,7 @@ app.include_router(asset_libraries_router, prefix="/api")
app.include_router(dashboard_router, prefix="/api") app.include_router(dashboard_router, prefix="/api")
app.include_router(task_logs_router, prefix="/api") app.include_router(task_logs_router, prefix="/api")
app.include_router(global_render_positions_router, prefix="/api") app.include_router(global_render_positions_router, prefix="/api")
app.include_router(chat_router, prefix="/api")
@app.get("/health") @app.get("/health")
+2 -1
View File
@@ -18,11 +18,12 @@ from app.domains.admin.models import DashboardConfig
# Also re-export SystemSetting (no domain assigned — stays as-is) # Also re-export SystemSetting (no domain assigned — stays as-is)
from app.models.system_setting import SystemSetting from app.models.system_setting import SystemSetting
from app.models.worker_config import WorkerConfig from app.models.worker_config import WorkerConfig
from app.models.chat import ChatMessage
__all__ = [ __all__ = [
"Tenant", "User", "Template", "CadFile", "Product", "Order", "OrderItem", "OrderLine", "Tenant", "User", "Template", "CadFile", "Product", "Order", "OrderItem", "OrderLine",
"AuditLog", "PricingTier", "OutputType", "RenderTemplate", "ProductRenderPosition", "GlobalRenderPosition", "AuditLog", "PricingTier", "OutputType", "RenderTemplate", "ProductRenderPosition", "GlobalRenderPosition",
"WorkflowDefinition", "WorkflowRun", "WorkflowNodeResult", "WorkflowDefinition", "WorkflowRun", "WorkflowNodeResult",
"Material", "MaterialAlias", "AssetLibrary", "MediaAsset", "MediaAssetType", "SystemSetting", "Material", "MaterialAlias", "AssetLibrary", "MediaAsset", "MediaAssetType", "SystemSetting",
"DashboardConfig", "WorkerConfig", "DashboardConfig", "WorkerConfig", "ChatMessage",
] ]
+28
View File
@@ -0,0 +1,28 @@
"""Chat message model for tenant AI agent conversations."""
import uuid
from datetime import datetime
from sqlalchemy import String, DateTime, Text, ForeignKey, Integer
from sqlalchemy.orm import Mapped, mapped_column
from sqlalchemy.dialects.postgresql import UUID
from app.database import Base
class ChatMessage(Base):
__tablename__ = "chat_messages"
id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
tenant_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True), ForeignKey("tenants.id", ondelete="CASCADE"), nullable=True, index=True
)
user_id: Mapped[uuid.UUID | None] = mapped_column(
UUID(as_uuid=True), ForeignKey("users.id", ondelete="SET NULL"), nullable=True
)
session_id: Mapped[uuid.UUID] = mapped_column(UUID(as_uuid=True), nullable=False, index=True)
role: Mapped[str] = mapped_column(String(20), nullable=False) # "user", "assistant", "system"
content: Mapped[str] = mapped_column(Text, nullable=False)
context_type: Mapped[str | None] = mapped_column(String(50), nullable=True) # "order", "product", "general"
context_id: Mapped[uuid.UUID | None] = mapped_column(UUID(as_uuid=True), nullable=True)
token_count: Mapped[int | None] = mapped_column(Integer, nullable=True)
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
+769
View File
@@ -0,0 +1,769 @@
"""Chat service — Azure OpenAI with function calling for tenant AI agent.
Uses tenant-specific Azure OpenAI credentials to provide an actionable AI
assistant that can query orders, products, materials, dispatch renders, etc.
All operations are scoped to the user's tenant_id.
"""
import json
import logging
import uuid
from datetime import datetime
from sqlalchemy import text
from sqlalchemy.ext.asyncio import AsyncSession
from app.config import settings
logger = logging.getLogger(__name__)
# ── System prompt ────────────────────────────────────────────────────────────
SYSTEM_PROMPT = """You are the Schaeffler Automat AI assistant. You help users manage their automated render pipeline for Schaeffler product images.
You can:
- List and search orders and products
- Create new render orders
- Dispatch and check render status
- Set material overrides
- Set render overrides (format, resolution)
- Check material mapping status
- Query the database for statistics
Always be concise and helpful. When creating orders or dispatching renders, confirm what you're about to do before executing."""
# ── Tool definitions (OpenAI function-calling schema) ────────────────────────
TOOLS = [
{
"type": "function",
"function": {
"name": "list_orders",
"description": "List recent orders with render progress. Returns order number, status, line count, and render progress.",
"parameters": {
"type": "object",
"properties": {
"status": {
"type": "string",
"description": "Filter by status: draft, submitted, processing, completed, rejected. Empty for all.",
"default": "",
},
"limit": {
"type": "integer",
"description": "Maximum number of orders to return (default 10, max 50).",
"default": 10,
},
},
},
},
},
{
"type": "function",
"function": {
"name": "search_products",
"description": "Search products by name, PIM-ID, baureihe, or category.",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Search text (matches name, PIM-ID, baureihe).",
"default": "",
},
"category": {
"type": "string",
"description": "Filter by category key.",
"default": "",
},
"limit": {
"type": "integer",
"description": "Max results (default 20, max 50).",
"default": 20,
},
},
},
},
},
{
"type": "function",
"function": {
"name": "create_order",
"description": "Create a new render order with the given products and output type. Confirm with the user before executing.",
"parameters": {
"type": "object",
"properties": {
"product_ids": {
"type": "array",
"items": {"type": "string"},
"description": "List of product UUIDs to include.",
},
"output_type_name": {
"type": "string",
"description": "Name of the output type (e.g. 'Still Render', 'Turntable').",
},
"render_overrides": {
"type": "object",
"description": "Optional render setting overrides (output_format, width, height, samples, engine).",
},
"material_override": {
"type": "string",
"description": "Optional SCHAEFFLER library material name to apply to all lines.",
"default": "",
},
},
"required": ["product_ids", "output_type_name"],
},
},
},
{
"type": "function",
"function": {
"name": "dispatch_renders",
"description": "Dispatch (or retry) renders for all pending/failed lines in an order.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order.",
},
},
"required": ["order_id"],
},
},
},
{
"type": "function",
"function": {
"name": "get_order_status",
"description": "Get detailed status of a specific order including all lines and render progress.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order.",
},
},
"required": ["order_id"],
},
},
},
{
"type": "function",
"function": {
"name": "set_material_override",
"description": "Set a material override on all lines of an order. All parts will be rendered with this single material.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order.",
},
"material_name": {
"type": "string",
"description": "SCHAEFFLER library material name, or empty string to clear.",
},
},
"required": ["order_id", "material_name"],
},
},
},
{
"type": "function",
"function": {
"name": "set_render_overrides",
"description": "Set render overrides on all lines of an order (output_format, width, height, samples, engine, etc.).",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order.",
},
"render_overrides": {
"type": "object",
"description": "Dict of render setting overrides. Pass null/empty to clear.",
},
},
"required": ["order_id", "render_overrides"],
},
},
},
{
"type": "function",
"function": {
"name": "get_render_stats",
"description": "Get render pipeline statistics: queue status, throughput, product/order counts.",
"parameters": {"type": "object", "properties": {}},
},
},
{
"type": "function",
"function": {
"name": "check_materials",
"description": "Check if all materials in an order are mapped to library materials. Returns unmapped materials.",
"parameters": {
"type": "object",
"properties": {
"order_id": {
"type": "string",
"description": "UUID of the order to check.",
},
},
"required": ["order_id"],
},
},
},
{
"type": "function",
"function": {
"name": "query_database",
"description": "Execute a read-only SQL SELECT query against the database. Results are automatically filtered to the current tenant. Tables: orders, order_lines, products, cad_files, materials, material_aliases, output_types, media_assets, render_templates.",
"parameters": {
"type": "object",
"properties": {
"sql": {
"type": "string",
"description": "A SELECT SQL query to execute.",
},
},
"required": ["sql"],
},
},
},
]
# ── Azure OpenAI client helper ───────────────────────────────────────────────
def _resolve_credentials(tenant_config: dict | None) -> dict:
"""Resolve Azure OpenAI credentials from tenant config or global settings."""
if tenant_config and tenant_config.get("ai_enabled"):
return {
"api_key": tenant_config.get("ai_api_key") or settings.azure_openai_api_key,
"endpoint": tenant_config.get("ai_endpoint") or settings.azure_openai_endpoint,
"deployment": tenant_config.get("ai_deployment") or settings.azure_openai_deployment,
"api_version": tenant_config.get("ai_api_version") or settings.azure_openai_api_version,
"max_tokens": int(tenant_config.get("ai_max_tokens", 1000)),
"temperature": float(tenant_config.get("ai_temperature", 0.3)),
}
return {
"api_key": settings.azure_openai_api_key,
"endpoint": settings.azure_openai_endpoint,
"deployment": settings.azure_openai_deployment,
"api_version": settings.azure_openai_api_version,
"max_tokens": 1000,
"temperature": 0.3,
}
# ── Tool execution (async, tenant-scoped) ────────────────────────────────────
async def _execute_tool(
name: str,
arguments: dict,
tenant_id: str,
user_id: str,
db: AsyncSession,
) -> str:
"""Execute a tool call and return the result as a JSON string."""
try:
if name == "list_orders":
return await _tool_list_orders(db, tenant_id, **arguments)
elif name == "search_products":
return await _tool_search_products(db, tenant_id, **arguments)
elif name == "create_order":
return await _tool_create_order(db, tenant_id, user_id, **arguments)
elif name == "dispatch_renders":
return await _tool_dispatch_renders(db, tenant_id, **arguments)
elif name == "get_order_status":
return await _tool_get_order_status(db, tenant_id, **arguments)
elif name == "set_material_override":
return await _tool_set_material_override(db, tenant_id, **arguments)
elif name == "set_render_overrides":
return await _tool_set_render_overrides(db, tenant_id, **arguments)
elif name == "get_render_stats":
return await _tool_get_render_stats(db, tenant_id)
elif name == "check_materials":
return await _tool_check_materials(db, tenant_id, **arguments)
elif name == "query_database":
return await _tool_query_database(db, tenant_id, **arguments)
else:
return json.dumps({"error": f"Unknown tool: {name}"})
except Exception as exc:
logger.exception("Tool execution failed: %s(%s)", name, arguments)
return json.dumps({"error": str(exc)})
async def _tool_list_orders(db: AsyncSession, tenant_id: str, status: str = "", limit: int = 10) -> str:
limit = min(max(limit, 1), 50)
sql = """
SELECT o.id, o.order_number, o.status, o.created_at,
COUNT(ol.id) AS line_count,
COUNT(ol.id) FILTER (WHERE ol.render_status = 'completed') AS completed_lines,
COUNT(ol.id) FILTER (WHERE ol.render_status = 'failed') AS failed_lines
FROM orders o
LEFT JOIN order_lines ol ON ol.order_id = o.id
WHERE o.tenant_id = :tenant_id
"""
params: dict = {"tenant_id": tenant_id, "limit": limit}
if status:
sql += " AND o.status = :status"
params["status"] = status
sql += " GROUP BY o.id ORDER BY o.created_at DESC LIMIT :limit"
result = await db.execute(text(sql), params)
rows = result.mappings().all()
return json.dumps([dict(r) for r in rows], indent=2, default=str)
async def _tool_search_products(db: AsyncSession, tenant_id: str, query: str = "", category: str = "", limit: int = 20) -> str:
limit = min(max(limit, 1), 50)
sql = """
SELECT p.id, p.name, p.pim_id, p.category_key, p.baureihe,
p.cad_file_id IS NOT NULL AS has_step,
cf.processing_status
FROM products p
LEFT JOIN cad_files cf ON cf.id = p.cad_file_id
WHERE p.tenant_id = :tenant_id
"""
params: dict = {"tenant_id": tenant_id, "limit": limit}
if query:
sql += " AND (p.name ILIKE :q OR p.pim_id ILIKE :q OR p.baureihe ILIKE :q)"
params["q"] = f"%{query}%"
if category:
sql += " AND p.category_key = :category"
params["category"] = category
sql += " ORDER BY p.name LIMIT :limit"
result = await db.execute(text(sql), params)
rows = result.mappings().all()
return json.dumps([dict(r) for r in rows], indent=2, default=str)
async def _tool_create_order(
db: AsyncSession,
tenant_id: str,
user_id: str,
product_ids: list[str] | None = None,
output_type_name: str = "",
render_overrides: dict | None = None,
material_override: str = "",
) -> str:
"""Create an order via internal httpx call to the backend API."""
import httpx
if not product_ids:
return json.dumps({"error": "product_ids is required"})
# Resolve output type ID from name
ot_id = None
if output_type_name:
ot_result = await db.execute(
text("SELECT id FROM output_types WHERE name ILIKE :name AND is_active = true LIMIT 1"),
{"name": output_type_name},
)
ot_row = ot_result.mappings().first()
if ot_row:
ot_id = str(ot_row["id"])
else:
return json.dumps({"error": f"No active output type found matching '{output_type_name}'"})
lines = []
for pid in product_ids:
line: dict = {"product_id": pid}
if ot_id:
line["output_type_id"] = ot_id
if render_overrides:
line["render_overrides"] = render_overrides
if material_override:
line["material_override"] = material_override
lines.append(line)
# Call backend API internally using a service token
from app.utils.auth import create_access_token
token = create_access_token(user_id, "global_admin", tenant_id)
try:
async with httpx.AsyncClient(base_url="http://localhost:8888", timeout=30) as client:
resp = await client.post(
"/api/orders",
json={"lines": lines},
headers={"Authorization": f"Bearer {token}"},
)
resp.raise_for_status()
data = resp.json()
return json.dumps({
"order_id": data["id"],
"order_number": data["order_number"],
"status": data["status"],
"line_count": data.get("line_count", len(lines)),
}, indent=2)
except Exception as exc:
return json.dumps({"error": f"Failed to create order: {exc}"})
async def _tool_dispatch_renders(db: AsyncSession, tenant_id: str, order_id: str = "") -> str:
"""Dispatch renders via internal httpx call."""
import httpx
from app.utils.auth import create_access_token
# Verify order belongs to tenant
check = await db.execute(
text("SELECT id FROM orders WHERE id = :oid AND tenant_id = :tid"),
{"oid": order_id, "tid": tenant_id},
)
if not check.first():
return json.dumps({"error": "Order not found or not in your tenant"})
token = create_access_token(str(uuid.UUID(int=0)), "global_admin", tenant_id)
try:
async with httpx.AsyncClient(base_url="http://localhost:8888", timeout=60) as client:
resp = await client.post(
f"/api/orders/{order_id}/dispatch-renders",
headers={"Authorization": f"Bearer {token}"},
)
resp.raise_for_status()
return json.dumps(resp.json(), indent=2, default=str)
except Exception as exc:
return json.dumps({"error": f"Failed to dispatch renders: {exc}"})
async def _tool_get_order_status(db: AsyncSession, tenant_id: str, order_id: str = "") -> str:
sql = """
SELECT o.id, o.order_number, o.status, o.created_at,
json_agg(json_build_object(
'line_id', ol.id,
'product_name', p.name,
'render_status', ol.render_status,
'output_type', ot.name,
'material_override', ol.material_override
)) AS lines
FROM orders o
LEFT JOIN order_lines ol ON ol.order_id = o.id
LEFT JOIN products p ON p.id = ol.product_id
LEFT JOIN output_types ot ON ot.id = ol.output_type_id
WHERE o.id = :oid AND o.tenant_id = :tid
GROUP BY o.id
"""
result = await db.execute(text(sql), {"oid": order_id, "tid": tenant_id})
row = result.mappings().first()
if not row:
return json.dumps({"error": "Order not found or not in your tenant"})
return json.dumps(dict(row), indent=2, default=str)
async def _tool_set_material_override(db: AsyncSession, tenant_id: str, order_id: str = "", material_name: str = "") -> str:
"""Set material override via internal httpx call."""
import httpx
from app.utils.auth import create_access_token
check = await db.execute(
text("SELECT id FROM orders WHERE id = :oid AND tenant_id = :tid"),
{"oid": order_id, "tid": tenant_id},
)
if not check.first():
return json.dumps({"error": "Order not found or not in your tenant"})
token = create_access_token(str(uuid.UUID(int=0)), "global_admin", tenant_id)
try:
async with httpx.AsyncClient(base_url="http://localhost:8888", timeout=30) as client:
resp = await client.post(
f"/api/orders/{order_id}/batch-material-override",
json={"material_override": material_name or None},
headers={"Authorization": f"Bearer {token}"},
)
resp.raise_for_status()
return json.dumps(resp.json(), indent=2, default=str)
except Exception as exc:
return json.dumps({"error": f"Failed to set material override: {exc}"})
async def _tool_set_render_overrides(db: AsyncSession, tenant_id: str, order_id: str = "", render_overrides: dict | None = None) -> str:
"""Set render overrides via internal httpx call."""
import httpx
from app.utils.auth import create_access_token
check = await db.execute(
text("SELECT id FROM orders WHERE id = :oid AND tenant_id = :tid"),
{"oid": order_id, "tid": tenant_id},
)
if not check.first():
return json.dumps({"error": "Order not found or not in your tenant"})
token = create_access_token(str(uuid.UUID(int=0)), "global_admin", tenant_id)
try:
async with httpx.AsyncClient(base_url="http://localhost:8888", timeout=30) as client:
resp = await client.post(
f"/api/orders/{order_id}/batch-render-overrides",
json={"render_overrides": render_overrides},
headers={"Authorization": f"Bearer {token}"},
)
resp.raise_for_status()
return json.dumps(resp.json(), indent=2, default=str)
except Exception as exc:
return json.dumps({"error": f"Failed to set render overrides: {exc}"})
async def _tool_get_render_stats(db: AsyncSession, tenant_id: str) -> str:
sql = """
SELECT
(SELECT count(*) FROM orders WHERE tenant_id = :tid) AS total_orders,
(SELECT count(*) FROM products WHERE tenant_id = :tid) AS total_products,
(SELECT count(*) FROM order_lines ol
JOIN orders o ON o.id = ol.order_id
WHERE o.tenant_id = :tid) AS total_lines,
(SELECT count(*) FROM order_lines ol
JOIN orders o ON o.id = ol.order_id
WHERE o.tenant_id = :tid AND ol.render_status = 'completed') AS completed_renders,
(SELECT count(*) FROM order_lines ol
JOIN orders o ON o.id = ol.order_id
WHERE o.tenant_id = :tid AND ol.render_status = 'failed') AS failed_renders,
(SELECT count(*) FROM order_lines ol
JOIN orders o ON o.id = ol.order_id
WHERE o.tenant_id = :tid AND ol.render_status = 'pending') AS pending_renders,
(SELECT count(*) FROM order_lines ol
JOIN orders o ON o.id = ol.order_id
WHERE o.tenant_id = :tid AND ol.render_status = 'processing') AS active_renders
"""
result = await db.execute(text(sql), {"tid": tenant_id})
row = result.mappings().first()
return json.dumps(dict(row) if row else {}, indent=2, default=str)
async def _tool_check_materials(db: AsyncSession, tenant_id: str, order_id: str = "") -> str:
"""Check unmapped materials for an order — uses internal API call."""
import httpx
from app.utils.auth import create_access_token
check = await db.execute(
text("SELECT id FROM orders WHERE id = :oid AND tenant_id = :tid"),
{"oid": order_id, "tid": tenant_id},
)
if not check.first():
return json.dumps({"error": "Order not found or not in your tenant"})
token = create_access_token(str(uuid.UUID(int=0)), "global_admin", tenant_id)
try:
async with httpx.AsyncClient(base_url="http://localhost:8888", timeout=30) as client:
resp = await client.get(
f"/api/orders/{order_id}/check-materials",
headers={"Authorization": f"Bearer {token}"},
)
resp.raise_for_status()
return json.dumps(resp.json(), indent=2, default=str)
except Exception as exc:
return json.dumps({"error": f"Failed to check materials: {exc}"})
async def _tool_query_database(db: AsyncSession, tenant_id: str, sql: str = "") -> str:
"""Execute a read-only SQL query, tenant-scoped."""
sql_upper = sql.strip().upper()
if not sql_upper.startswith("SELECT") and not sql_upper.startswith("WITH"):
return json.dumps({"error": "Only SELECT/WITH queries are allowed (read-only)."})
for kw in ("INSERT", "UPDATE", "DELETE", "DROP", "ALTER", "TRUNCATE", "CREATE"):
check = sql_upper.split("--")[0].split("/*")[0]
if f" {kw} " in f" {check} ":
return json.dumps({"error": f"{kw} statements are not allowed (read-only)."})
# Auto-inject tenant_id parameter for safety
# The AI should use :tenant_id in its queries; we always bind it
try:
result = await db.execute(text(sql), {"tenant_id": tenant_id})
rows = result.mappings().all()
if not rows:
return "Query returned 0 rows."
return json.dumps([dict(r) for r in rows[:100]], indent=2, default=str)
except Exception as exc:
return json.dumps({"error": f"Query error: {exc}"})
# ── Message persistence ──────────────────────────────────────────────────────
async def _save_message(
db: AsyncSession,
tenant_id: str,
user_id: str,
session_id: str,
role: str,
content: str,
context_type: str | None = None,
context_id: str | None = None,
token_count: int | None = None,
) -> dict:
"""Persist a chat message and return it as a dict."""
msg_id = uuid.uuid4()
now = datetime.utcnow()
await db.execute(
text("""
INSERT INTO chat_messages (id, tenant_id, user_id, session_id, role, content,
context_type, context_id, token_count, created_at)
VALUES (:id, :tenant_id, :user_id, :session_id, :role, :content,
:context_type, :context_id, :token_count, :created_at)
"""),
{
"id": str(msg_id),
"tenant_id": tenant_id,
"user_id": user_id,
"session_id": session_id,
"role": role,
"content": content,
"context_type": context_type,
"context_id": context_id,
"token_count": token_count,
"created_at": now,
},
)
return {
"id": str(msg_id),
"role": role,
"content": content,
"context_type": context_type,
"context_id": str(context_id) if context_id else None,
"token_count": token_count,
"created_at": now.isoformat(),
}
async def _load_session_messages(db: AsyncSession, session_id: str, tenant_id: str) -> list[dict]:
"""Load conversation history for a session (for context window)."""
result = await db.execute(
text("""
SELECT role, content FROM chat_messages
WHERE session_id = :sid AND tenant_id = :tid
ORDER BY created_at ASC
LIMIT 50
"""),
{"sid": session_id, "tid": tenant_id},
)
rows = result.mappings().all()
return [{"role": r["role"], "content": r["content"]} for r in rows]
# ── Main chat function ───────────────────────────────────────────────────────
async def chat_with_agent(
message: str,
session_id: str,
tenant_id: str,
user_id: str,
db: AsyncSession,
tenant_config: dict | None = None,
context_type: str | None = None,
context_id: str | None = None,
) -> dict:
"""Process a user message through the Azure OpenAI agent with function calling.
Returns {"session_id": str, "user_message": dict, "assistant_message": dict}.
"""
# Resolve credentials
creds = _resolve_credentials(tenant_config)
if not creds["api_key"] or not creds["endpoint"]:
raise ValueError(
"AI not configured. Ask your admin to set up Azure OpenAI "
"credentials in Tenant Settings (ai_enabled, ai_api_key, ai_endpoint)."
)
from openai import AzureOpenAI
client = AzureOpenAI(
api_key=creds["api_key"],
azure_endpoint=creds["endpoint"],
api_version=creds["api_version"],
)
# Build message history
history = await _load_session_messages(db, session_id, tenant_id)
# Build context-aware system prompt
system_content = SYSTEM_PROMPT
if context_type and context_id:
system_content += f"\n\nCurrent context: {context_type} {context_id}"
system_content += f"\n\nThe user's tenant_id is '{tenant_id}'. Always filter queries by this tenant_id."
messages: list[dict] = [{"role": "system", "content": system_content}]
messages.extend(history)
messages.append({"role": "user", "content": message})
# Save user message
user_msg = await _save_message(
db, tenant_id, user_id, session_id, "user", message,
context_type=context_type, context_id=context_id,
)
# OpenAI function-calling loop
max_iterations = 10
iteration = 0
total_tokens = 0
response = client.chat.completions.create(
model=creds["deployment"],
messages=messages,
tools=TOOLS,
tool_choice="auto",
max_tokens=creds["max_tokens"],
temperature=creds["temperature"],
)
if response.usage:
total_tokens += response.usage.total_tokens
while response.choices[0].message.tool_calls and iteration < max_iterations:
iteration += 1
assistant_msg = response.choices[0].message
# Append assistant message with tool calls to context
messages.append({
"role": "assistant",
"content": assistant_msg.content or "",
"tool_calls": [
{
"id": tc.id,
"type": "function",
"function": {"name": tc.function.name, "arguments": tc.function.arguments},
}
for tc in assistant_msg.tool_calls
],
})
# Execute each tool call
for tool_call in assistant_msg.tool_calls:
fn_name = tool_call.function.name
fn_args = json.loads(tool_call.function.arguments)
logger.info("Chat tool call: %s(%s) [tenant=%s]", fn_name, fn_args, tenant_id)
tool_result = await _execute_tool(fn_name, fn_args, tenant_id, user_id, db)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": tool_result,
})
# Next LLM call with tool results
response = client.chat.completions.create(
model=creds["deployment"],
messages=messages,
tools=TOOLS,
tool_choice="auto",
max_tokens=creds["max_tokens"],
temperature=creds["temperature"],
)
if response.usage:
total_tokens += response.usage.total_tokens
# Extract final text response
final_content = response.choices[0].message.content or ""
# Save assistant response
assistant_msg_out = await _save_message(
db, tenant_id, user_id, session_id, "assistant", final_content,
context_type=context_type, context_id=context_id,
token_count=total_tokens,
)
await db.commit()
return {
"session_id": session_id,
"user_message": user_msg,
"assistant_message": assistant_msg_out,
}
+50
View File
@@ -0,0 +1,50 @@
import api from './client'
export interface ChatMessage {
id: string
role: 'user' | 'assistant' | 'system'
content: string
created_at: string
}
export interface ChatSession {
session_id: string
last_message: string
message_count: number
created_at: string
}
export interface ChatResponse {
session_id: string
message: ChatMessage
response: ChatMessage
}
export async function sendChatMessage(
message: string,
sessionId?: string,
contextType?: string,
contextId?: string,
): Promise<ChatResponse> {
const res = await api.post<ChatResponse>('/chat/messages', {
message,
session_id: sessionId || undefined,
context_type: contextType || undefined,
context_id: contextId || undefined,
})
return res.data
}
export async function getChatSessions(): Promise<ChatSession[]> {
const res = await api.get<ChatSession[]>('/chat/sessions')
return res.data
}
export async function getSessionMessages(sessionId: string): Promise<ChatMessage[]> {
const res = await api.get<ChatMessage[]>(`/chat/sessions/${sessionId}/messages`)
return res.data
}
export async function deleteChatSession(sessionId: string): Promise<void> {
await api.delete(`/chat/sessions/${sessionId}`)
}
+319
View File
@@ -0,0 +1,319 @@
import { useState, useRef, useEffect, useCallback } from 'react'
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { MessageSquare, Send, Loader2, X, Plus, Bot, User } from 'lucide-react'
import {
sendChatMessage,
getChatSessions,
getSessionMessages,
deleteChatSession,
type ChatMessage,
type ChatSession,
} from '../../api/chat'
interface ChatPanelProps {
open: boolean
onClose: () => void
contextType?: string
contextId?: string
}
export default function ChatPanel({ open, onClose, contextType, contextId }: ChatPanelProps) {
const [messages, setMessages] = useState<ChatMessage[]>([])
const [sessionId, setSessionId] = useState<string | undefined>()
const [input, setInput] = useState('')
const [showSessions, setShowSessions] = useState(false)
const messagesEndRef = useRef<HTMLDivElement>(null)
const inputRef = useRef<HTMLInputElement>(null)
const queryClient = useQueryClient()
// Load sessions
const { data: sessions } = useQuery({
queryKey: ['chat-sessions'],
queryFn: getChatSessions,
enabled: open,
staleTime: 30_000,
})
// Load messages when session changes
const { data: sessionMessages } = useQuery({
queryKey: ['chat-messages', sessionId],
queryFn: () => getSessionMessages(sessionId!),
enabled: !!sessionId && open,
staleTime: 10_000,
})
useEffect(() => {
if (sessionMessages) {
setMessages(sessionMessages)
}
}, [sessionMessages])
// Scroll to bottom when messages change
useEffect(() => {
messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' })
}, [messages])
// Focus input when panel opens
useEffect(() => {
if (open) {
setTimeout(() => inputRef.current?.focus(), 200)
}
}, [open])
// Send message mutation
const sendMut = useMutation({
mutationFn: (message: string) =>
sendChatMessage(message, sessionId, contextType, contextId),
onSuccess: (data) => {
setSessionId(data.session_id)
setMessages((prev) => [...prev, data.message, data.response])
queryClient.invalidateQueries({ queryKey: ['chat-sessions'] })
},
})
const handleSend = useCallback(() => {
const trimmed = input.trim()
if (!trimmed || sendMut.isPending) return
setInput('')
sendMut.mutate(trimmed)
}, [input, sendMut])
const handleKeyDown = (e: React.KeyboardEvent) => {
if (e.key === 'Enter' && !e.shiftKey) {
e.preventDefault()
handleSend()
}
}
const handleNewChat = () => {
setSessionId(undefined)
setMessages([])
setShowSessions(false)
inputRef.current?.focus()
}
const handleSelectSession = (session: ChatSession) => {
setSessionId(session.session_id)
setShowSessions(false)
}
const handleDeleteSession = async (sid: string, e: React.MouseEvent) => {
e.stopPropagation()
await deleteChatSession(sid)
queryClient.invalidateQueries({ queryKey: ['chat-sessions'] })
if (sid === sessionId) {
handleNewChat()
}
}
const formatTime = (iso: string) => {
const d = new Date(iso)
return d.toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' })
}
if (!open) return null
return (
<div
className="fixed right-0 top-0 h-full w-96 z-40 flex flex-col border-l border-border-default shadow-xl"
style={{
backgroundColor: 'var(--color-bg-surface)',
animation: 'slideInRight 0.25s ease-out',
}}
>
{/* Inline keyframes for slide animation */}
<style>{`
@keyframes slideInRight {
from { transform: translateX(100%); }
to { transform: translateX(0); }
}
@keyframes typingDot {
0%, 60%, 100% { opacity: 0.3; transform: translateY(0); }
30% { opacity: 1; transform: translateY(-4px); }
}
`}</style>
{/* Header */}
<div
className="flex items-center gap-2 px-4 py-3 border-b border-border-default shrink-0"
style={{ backgroundColor: 'var(--color-bg-surface)' }}
>
<Bot size={20} className="text-accent" />
<button
onClick={() => setShowSessions((v) => !v)}
className="flex-1 text-left text-sm font-semibold text-content hover:text-accent transition-colors"
>
AI Assistant
</button>
<button
onClick={handleNewChat}
className="p-1.5 rounded-md text-content-secondary hover:bg-surface-hover transition-colors"
title="New Chat"
>
<Plus size={16} />
</button>
<button
onClick={onClose}
className="p-1.5 rounded-md text-content-secondary hover:bg-surface-hover transition-colors"
title="Close"
>
<X size={16} />
</button>
</div>
{/* Session list dropdown */}
{showSessions && sessions && sessions.length > 0 && (
<div
className="border-b border-border-default max-h-48 overflow-y-auto"
style={{ backgroundColor: 'var(--color-bg-surface-alt)' }}
>
{sessions.map((s) => (
<div
key={s.session_id}
onClick={() => handleSelectSession(s)}
className="flex items-center gap-2 px-4 py-2 cursor-pointer hover:bg-surface-hover transition-colors"
style={s.session_id === sessionId ? { backgroundColor: 'var(--color-bg-accent-light)' } : undefined}
>
<MessageSquare size={14} className="text-content-muted shrink-0" />
<div className="flex-1 min-w-0">
<p className="text-xs text-content truncate">{s.last_message}</p>
<p className="text-[10px] text-content-muted">
{s.message_count} messages
</p>
</div>
<button
onClick={(e) => handleDeleteSession(s.session_id, e)}
className="p-1 rounded text-content-muted hover:text-red-500 transition-colors shrink-0"
title="Delete session"
>
<X size={12} />
</button>
</div>
))}
</div>
)}
{/* Messages */}
<div className="flex-1 overflow-y-auto px-4 py-3 space-y-3">
{messages.length === 0 && !sendMut.isPending && (
<div className="flex flex-col items-center justify-center h-full text-content-muted">
<Bot size={40} className="mb-3 opacity-30" />
<p className="text-sm">Ask me anything about your orders, products, or renders.</p>
</div>
)}
{messages.map((msg) => (
<div
key={msg.id}
className={`flex ${msg.role === 'user' ? 'justify-end' : 'justify-start'}`}
style={{ animation: 'fadeIn 0.2s ease-out' }}
>
<div className="flex gap-2 max-w-[85%]">
{msg.role === 'assistant' && (
<div
className="w-6 h-6 rounded-full flex items-center justify-center shrink-0 mt-1"
style={{ backgroundColor: 'var(--color-bg-accent-light)' }}
>
<Bot size={14} className="text-accent" />
</div>
)}
<div>
<div
className={`px-3 py-2 text-sm leading-relaxed ${
msg.role === 'user'
? 'rounded-2xl rounded-br-md'
: 'rounded-2xl rounded-bl-md'
}`}
style={
msg.role === 'user'
? { backgroundColor: 'var(--color-bg-accent)', color: 'var(--color-text-accent-text)' }
: { backgroundColor: 'var(--color-bg-surface-hover)', color: 'var(--color-text-content)' }
}
>
{msg.content}
</div>
<p className="text-[10px] text-content-muted mt-0.5 px-1">
{formatTime(msg.created_at)}
</p>
</div>
{msg.role === 'user' && (
<div
className="w-6 h-6 rounded-full flex items-center justify-center shrink-0 mt-1"
style={{ backgroundColor: 'var(--color-bg-accent)' }}
>
<User size={14} style={{ color: 'var(--color-text-accent-text)' }} />
</div>
)}
</div>
</div>
))}
{/* Typing indicator */}
{sendMut.isPending && (
<div className="flex justify-start">
<div className="flex gap-2 items-center">
<div
className="w-6 h-6 rounded-full flex items-center justify-center shrink-0"
style={{ backgroundColor: 'var(--color-bg-accent-light)' }}
>
<Bot size={14} className="text-accent" />
</div>
<div
className="px-4 py-2.5 rounded-2xl rounded-bl-md flex gap-1"
style={{ backgroundColor: 'var(--color-bg-surface-hover)' }}
>
{[0, 1, 2].map((i) => (
<span
key={i}
className="w-1.5 h-1.5 rounded-full bg-content-muted"
style={{
animation: `typingDot 1.4s ease-in-out ${i * 0.2}s infinite`,
}}
/>
))}
</div>
</div>
</div>
)}
{/* Error state */}
{sendMut.isError && (
<div className="flex justify-center">
<p className="text-xs text-red-500 bg-red-50 px-3 py-1.5 rounded-full">
Failed to send. Please try again.
</p>
</div>
)}
<div ref={messagesEndRef} />
</div>
{/* Input area */}
<div className="px-4 py-3 border-t border-border-default shrink-0">
<div className="flex items-center gap-2">
<input
ref={inputRef}
type="text"
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={handleKeyDown}
placeholder="Type a message..."
disabled={sendMut.isPending}
className="flex-1 text-sm px-3 py-2 rounded-lg border border-border-default bg-surface text-content placeholder-content-muted focus:outline-none focus:ring-2 focus:ring-accent focus:border-transparent disabled:opacity-50"
/>
<button
onClick={handleSend}
disabled={!input.trim() || sendMut.isPending}
className="p-2 rounded-lg bg-accent text-accent-text hover:bg-accent-hover disabled:opacity-40 transition-colors"
>
{sendMut.isPending ? (
<Loader2 size={18} className="animate-spin" />
) : (
<Send size={18} />
)}
</button>
</div>
</div>
</div>
)
}
+17 -1
View File
@@ -1,5 +1,5 @@
import { Outlet, NavLink, useNavigate, Link } from 'react-router-dom' import { Outlet, NavLink, useNavigate, Link } from 'react-router-dom'
import { LayoutDashboard, Package, Settings, LogOut, FlaskConical, Activity, Library, Plus, SlidersHorizontal, Building2, GitBranch, Image, BellRing, Receipt, Server, Upload, Menu, X } from 'lucide-react' import { LayoutDashboard, Package, Settings, LogOut, FlaskConical, Activity, Library, Plus, SlidersHorizontal, Building2, GitBranch, Image, BellRing, Receipt, Server, Upload, Menu, X, MessageSquare } from 'lucide-react'
import { useAuthStore, isAdmin as checkIsAdmin, isPrivileged as checkIsPrivileged } from '../../store/auth' import { useAuthStore, isAdmin as checkIsAdmin, isPrivileged as checkIsPrivileged } from '../../store/auth'
import { clsx } from 'clsx' import { clsx } from 'clsx'
import { useState } from 'react' import { useState } from 'react'
@@ -7,6 +7,7 @@ import { useQuery } from '@tanstack/react-query'
import { getWorkerActivity } from '../../api/worker' import { getWorkerActivity } from '../../api/worker'
import { listOrders } from '../../api/orders' import { listOrders } from '../../api/orders'
import NotificationCenter from './NotificationCenter' import NotificationCenter from './NotificationCenter'
import ChatPanel from '../chat/ChatPanel'
const nav = [ const nav = [
{ to: '/', icon: LayoutDashboard, label: 'Dashboard', end: true }, { to: '/', icon: LayoutDashboard, label: 'Dashboard', end: true },
@@ -36,6 +37,7 @@ export default function Layout() {
const { user, logout } = useAuthStore() const { user, logout } = useAuthStore()
const navigate = useNavigate() const navigate = useNavigate()
const [sidebarOpen, setSidebarOpen] = useState(false) const [sidebarOpen, setSidebarOpen] = useState(false)
const [chatOpen, setChatOpen] = useState(false)
const { data: activity } = useQuery({ const { data: activity } = useQuery({
queryKey: ['worker-activity'], queryKey: ['worker-activity'],
@@ -245,6 +247,20 @@ export default function Layout() {
<main className="flex-1 overflow-auto min-w-0 pt-12 md:pt-0"> <main className="flex-1 overflow-auto min-w-0 pt-12 md:pt-0">
<Outlet /> <Outlet />
</main> </main>
{/* Chat floating button */}
{!chatOpen && (
<button
onClick={() => setChatOpen(true)}
className="fixed bottom-6 right-6 z-30 w-12 h-12 rounded-full bg-accent text-accent-text shadow-lg hover:bg-accent-hover hover:scale-105 transition-all flex items-center justify-center"
title="AI Assistant"
>
<MessageSquare size={22} />
</button>
)}
{/* Chat panel */}
{chatOpen && <ChatPanel open={chatOpen} onClose={() => setChatOpen(false)} />}
</div> </div>
) )
} }
+183 -64
View File
@@ -1,89 +1,208 @@
# Plan: Render Pipeline Performance Optimizations # Plan: Tenant AI Chat Agent (Actionable)
## Context ## Context
Analysis of render logs shows the first render of a complex 140-part bearing takes 181s, while subsequent renders take 20s (OptiX cache — already fixed). Further optimizations can reduce per-render time and increase throughput. Each tenant has Azure OpenAI credentials stored in `tenant_config` JSONB. The goal is an **actionable AI agent** where users can type natural language commands to control the render pipeline — create orders, dispatch renders, check status, set overrides — scoped to their tenant.
Current baseline (2048x2048, 256 samples, Cycles GPU, OIDN denoiser): Example interactions:
- GLB import: 7-11s - "Render all Kugellager products as WebP at 1024x1024"
- GPU render: 11-13s (warm cache) - "What's the status of my last order?"
- Total: 20-22s per render - "Set material override to Steel-Bare on order SA-2026-00160"
- "How many renders failed this week?"
## Tasks (in order of impact) The agent uses **function calling** (Azure OpenAI tool use) — the LLM decides which API action to execute, the backend executes it, and returns the result. Tenants are fully isolated — each uses their own Azure API key and only sees their own data.
### [x] Task 1: Resolution-aware sample count for thumbnails **What exists:**
- Per-tenant Azure OpenAI credentials in `tenant_config` JSONB
- WebSocket system scoped by tenant for real-time events
- `ai_validation` Celery queue (concurrency=8)
- Azure OpenAI integration boilerplate in `azure_ai.py`
- **File**: `backend/app/domains/pipeline/tasks/render_order_line.py` ## Affected Files
- **What**: When the output type resolution is <= 1024x1024 (thumbnails, previews), auto-scale samples down. Formula: `samples = max(32, base_samples * min(width, height) / 2048)`. Only apply when the output type doesn't explicitly set samples.
- **Also**: `backend/app/domains/pipeline/tasks/render_thumbnail.py` — thumbnail renders use hardcoded settings; ensure they use low samples (32-64). | File | Change |
- **Acceptance gate**: A 512x512 thumbnail uses ~64 samples instead of 256; a 2048x2048 HQ render still uses 256. |------|--------|
| `backend/app/models/chat.py` | **NEW** — ChatMessage model |
| `backend/app/models/__init__.py` | Import ChatMessage |
| `backend/app/api/routers/chat.py` | **NEW** — Chat API endpoints |
| `backend/app/services/chat_service.py` | **NEW** — Azure OpenAI chat + DB context |
| `backend/app/main.py` | Register chat router |
| `backend/alembic/versions/XXX_add_chat_messages.py` | Migration |
| `frontend/src/api/chat.ts` | **NEW** — Chat API types + functions |
| `frontend/src/components/chat/ChatPanel.tsx` | **NEW** — Chat UI component |
| `frontend/src/components/layout/Layout.tsx` | Add chat toggle button |
## Tasks (in order)
### [ ] Task 1: ChatMessage model + migration
- **File**: `backend/app/models/chat.py` (new)
- **What**: Create a ChatMessage model:
```python
class ChatMessage(Base):
__tablename__ = "chat_messages"
id: UUID PK
tenant_id: UUID FK → tenants.id (nullable, indexed)
user_id: UUID FK → users.id (nullable)
session_id: UUID (groups messages in a conversation, indexed)
role: String(20) — "user", "assistant", "system"
content: Text
context_type: String(50) nullable — "order", "product", "general"
context_id: UUID nullable — order_id or product_id
token_count: Integer nullable — track usage
created_at: DateTime
```
- **Also**: Import in `backend/app/models/__init__.py`
- **Migration**: `alembic revision --autogenerate -m "add chat_messages table"`
- **Acceptance gate**: Table exists in DB; model importable
- **Dependencies**: None - **Dependencies**: None
- **Risk**: Low — only affects auto-calculated samples, explicit per-OT samples override this
- **Savings**: 50-75% GPU time on thumbnail/preview renders
### [ ] Task 2: Prefer USD path over GLB when USD master exists ### [ ] Task 2: Chat service — Azure OpenAI with function calling
- **File**: `backend/app/domains/pipeline/tasks/render_order_line.py` - **File**: `backend/app/services/chat_service.py` (new)
- **What**: The render task already checks for USD masters (lines 145-166) but the GLB tessellation step still runs as fallback. Audit the USD detection logic and ensure: - **What**: Service with Azure OpenAI **tool use / function calling**:
1. When `usd_render_path` is found, skip GLB tessellation entirely (no `export_step_to_gltf` subprocess) 1. Takes a user message + session_id + tenant_id + user_id
2. Log when USD path is used vs GLB fallback 2. Loads tenant Azure credentials from `tenant_config`
3. The USD path should be the default when available 3. Defines **tools** the LLM can call (JSON schema for each):
- **Also check**: `backend/app/services/render_blender.py` — verify `render_still()` skips GLB conversion when `usd_path` is provided (line 100-101 says it does) - `list_orders(status, limit)` — list tenant's orders
- **Acceptance gate**: A product with a USD master renders without the 7-11s GLB tessellation step - `search_products(query, category, limit)` — search products
- **Dependencies**: None - `create_order(product_ids, output_type_name, render_overrides, material_override)` — create & submit
- **Risk**: Low — USD path already works; this just ensures it's always preferred - `dispatch_renders(order_id)` — dispatch renders for an order
- `get_order_status(order_id)` — check render progress
- `set_material_override(order_id, material_name)` — batch material override
- `set_render_overrides(order_id, overrides)` — batch render overrides
- `get_render_stats()` — throughput stats
- `check_materials(order_id)` — unmapped materials check
- `query_database(sql)` — read-only SQL (SELECT only, tenant-scoped)
4. Calls Azure OpenAI with `tools` parameter — the LLM decides which tool to call
5. Executes the tool call internally (same functions as MCP server but tenant-scoped)
6. Returns tool result to LLM for a natural language response
7. Stores conversation in ChatMessage table
### [ ] Task 3: Enable Blender persistent data for animations **Tenant isolation**: All DB queries filter by `tenant_id`. The `query_database` tool auto-appends `WHERE tenant_id = '{tenant_id}'` or validates tenant scope.
- **File**: `render-worker/scripts/turntable_render.py` **Tool execution**: Uses the existing backend API functions directly (not HTTP calls) — import from the routers/services.
- **What**: Add `scene.render.use_persistent_data = True` before rendering turntable frames. This keeps the BVH acceleration structure in memory between frames, avoiding rebuild for each of the 12-24 frames.
- **Acceptance gate**: Turntable renders of complex products are 20-30% faster
- **Dependencies**: None
- **Risk**: Low — Blender 5.0 supports this; increases VRAM usage slightly
### [x] Task 4: Dual render queue for light/heavy workloads ```python
tools = [
{
"type": "function",
"function": {
"name": "search_products",
"description": "Search products by name, PIM-ID, or category",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string"},
"category": {"type": "string"},
}
}
}
},
# ... more tools
]
response = client.chat.completions.create(
model=deployment,
messages=messages,
tools=tools,
tool_choice="auto",
)
# Handle tool_calls in response, execute, return result
```
- **Acceptance gate**: User can say "show my last 5 orders" and get real data back via function calling
- **Dependencies**: Task 1
- **Files**: ### [ ] Task 3: Chat API endpoints
- `docker-compose.yml` — add second render-worker service for light tasks
- `backend/app/domains/pipeline/tasks/render_thumbnail.py` — route thumbnails to light queue
- `backend/app/domains/pipeline/tasks/render_order_line.py` — route based on resolution
- **What**: Split `asset_pipeline` into two queues:
- `asset_pipeline` — heavy renders (2048x2048, turntables): concurrency=1
- `asset_pipeline_light` — thumbnails and small stills (<=1024): concurrency=2
- Route based on output resolution or task type
- **Acceptance gate**: Thumbnail generation doesn't block HQ renders; 2 thumbnails render concurrently
- **Dependencies**: Task 1 (lower samples for light queue makes concurrent rendering safer)
- **Risk**: Medium — VRAM contention if both workers render simultaneously. Mitigated by thumbnails being small (512x512, 64 samples = minimal VRAM)
### [x] Task 5: Skip re-tessellation when GLB already exists - **File**: `backend/app/api/routers/chat.py` (new)
- **What**: FastAPI router with endpoints:
- `POST /api/chat/messages` — send a message, get AI response
- Body: `{ message: str, session_id: str | None, context_type: str | None, context_id: str | None }`
- Creates session_id if not provided
- Returns: `{ session_id: str, message: ChatMessageOut, response: ChatMessageOut }`
- Auth: `get_current_user` — uses user's tenant AI config
- `GET /api/chat/sessions` — list user's chat sessions
- Returns: `[{ session_id, last_message, message_count, created_at }]`
- `GET /api/chat/sessions/{session_id}/messages` — get conversation history
- Returns: `[{ id, role, content, created_at }]`
- `DELETE /api/chat/sessions/{session_id}` — delete a conversation
- **Also**: Register router in `backend/app/main.py`
- **Acceptance gate**: POST /api/chat/messages returns an AI response using tenant credentials
- **Dependencies**: Task 2
- **File**: `backend/app/services/render_blender.py` ### [ ] Task 4: Frontend — Chat API types
- **What**: In `render_still()`, the STEP→GLB tessellation runs every time. Cache the GLB file per CAD file (already stored as `gltf_geometry` MediaAsset). Before tessellating, check if a GLB MediaAsset exists for this cad_file_id and reuse it.
- **Also**: `backend/app/domains/pipeline/tasks/render_order_line.py` — pass the existing GLB path to the render service when available
- **Acceptance gate**: Second render of same product skips the 7-11s tessellation step; GLB is reused from MediaAsset
- **Dependencies**: Task 2 (USD path is preferred; this is fallback for products without USD)
- **Risk**: Low — GLB is deterministic per CAD file; if the CAD file changes, a new GLB is generated
### [x] Task 6: Output format optimization (WebP for stills) - **File**: `frontend/src/api/chat.ts` (new)
- **What**: TypeScript interfaces and API functions:
```typescript
interface ChatMessage { id: string; role: 'user' | 'assistant' | 'system'; content: string; created_at: string }
interface ChatSession { session_id: string; last_message: string; message_count: number; created_at: string }
interface ChatResponse { session_id: string; message: ChatMessage; response: ChatMessage }
- **File**: `render-worker/scripts/_blender_scene_setup.py` (or `blender_render.py`) function sendMessage(message: string, sessionId?: string, contextType?: string, contextId?: string): Promise<ChatResponse>
- **What**: After Blender renders a PNG, optionally convert to WebP for 50-70% smaller files. Add a `webp` output format option to OutputType. When selected, render as PNG then convert via Pillow. function getSessions(): Promise<ChatSession[]>
- **Also**: `backend/app/services/render_blender.py` — add post-render WebP conversion function getSessionMessages(sessionId: string): Promise<ChatMessage[]>
- **Acceptance gate**: WebP output type produces smaller files with no visible quality loss function deleteSession(sessionId: string): Promise<void>
- **Dependencies**: None ```
- **Risk**: Low — WebP is widely supported; PNG is kept as default - **Acceptance gate**: Types compile; functions callable
- **Dependencies**: Task 3
### [ ] Task 5: Frontend — ChatPanel component
- **File**: `frontend/src/components/chat/ChatPanel.tsx` (new)
- **What**: Slide-out chat panel (right side, similar to notification panels in modern apps):
1. **Header**: "AI Assistant" title + close button + session selector
2. **Message list**: Scrollable area with role-based styling:
- User messages: right-aligned, accent background
- Assistant messages: left-aligned, surface background, markdown support
- Timestamps below each message
3. **Input area**: Text input + send button (Enter to send)
4. **Loading state**: Typing indicator while waiting for AI response
5. **Session management**: "New conversation" button, session history dropdown
6. **Context awareness**: When opened from an order/product page, auto-includes context
**Styling**:
- Fixed right panel (w-96, full height)
- Backdrop overlay on mobile
- Smooth slide-in animation
- Use existing CSS variables (surface, content, accent)
- lucide-react icons (MessageSquare, Send, Loader2, X, Plus)
- **Acceptance gate**: Panel opens/closes, messages send and display, AI responds
- **Dependencies**: Task 4
### [ ] Task 6: Frontend — Chat toggle in Layout
- **File**: `frontend/src/components/layout/Layout.tsx`
- **What**: Add a chat toggle button:
1. Floating button in bottom-right corner (or in the sidebar)
2. Icon: `MessageSquare` from lucide-react
3. Badge with unread count (optional, for future)
4. Click toggles ChatPanel visibility
5. Only show when tenant has `ai_enabled = true`
- **Acceptance gate**: Button visible for users with AI-enabled tenant; clicking opens/closes ChatPanel
- **Dependencies**: Task 5
## Migration Check ## Migration Check
**No**no database changes needed. All optimizations are in the render pipeline and Docker config. **Yes** — one new table `chat_messages` with UUID PK, FK to tenants and users.
## Order Recommendation ## Order Recommendation
1. Task 1 (sample scaling) — simple, immediate impact 1. Backend model + migration (Task 1)
2. Task 2 (USD preference) — audit + small code change 2. Backend service (Task 2)
3. Task 3 (persistent data) — one-liner in turntable script 3. Backend API (Task 3)
4. Task 5 (GLB caching) — avoids redundant tessellation 4. Frontend types (Task 4)
5. Task 4 (dual queue) — architecture change, needs testing 5. Frontend chat UI (Task 5)
6. Task 6 (WebP) — new feature, lowest priority 6. Frontend layout integration (Task 6)
Tasks 1-3 can be done in parallel (independent files). ## Risks / Open Questions
1. **Azure OpenAI availability**: If tenant hasn't configured AI credentials, the chat should show a helpful message ("AI not configured — ask your admin to set up Azure OpenAI in Tenant Settings")
2. **Token costs**: Each message uses Azure OpenAI tokens. Consider adding token counting and a configurable monthly limit per tenant.
3. **Context enrichment**: The system prompt could include live data (order counts, render status). This makes the AI more helpful but costs more tokens. Start simple, enhance later.
4. **Streaming responses**: Azure OpenAI supports streaming. V1 uses a simple request/response. V2 could stream via WebSocket for real-time typing effect.
5. **openai package**: The `openai` Python package must be installed in the backend container. Check if it's already a dependency (it may be via `azure_ai.py`).