HartOMat/plan.md

Plan: Tenant AI Chat Agent (Actionable)

Context

Each tenant has Azure OpenAI credentials stored in tenant_config JSONB. The goal is an actionable AI agent where users can type natural language commands to control the render pipeline — create orders, dispatch renders, check status, set overrides — scoped to their tenant.

Example interactions:

• "Render all Kugellager products as WebP at 1024x1024"
• "What's the status of my last order?"
• "Set material override to Steel-Bare on order SA-2026-00160"
• "How many renders failed this week?"

The agent uses function calling (Azure OpenAI tool use) — the LLM decides which API action to execute, the backend executes it, and returns the result. Tenants are fully isolated — each uses their own Azure API key and only sees their own data.

What exists:

• Per-tenant Azure OpenAI credentials in tenant_config JSONB
• WebSocket system scoped by tenant for real-time events
• ai_validation Celery queue (concurrency=8)
• Azure OpenAI integration boilerplate in azure_ai.py

Affected Files

File	Change
backend/app/models/chat.py	NEW — ChatMessage model
backend/app/models/__init__.py	Import ChatMessage
backend/app/api/routers/chat.py	NEW — Chat API endpoints
backend/app/services/chat_service.py	NEW — Azure OpenAI chat + DB context
backend/app/main.py	Register chat router
backend/alembic/versions/XXX_add_chat_messages.py	Migration
frontend/src/api/chat.ts	NEW — Chat API types + functions
frontend/src/components/chat/ChatPanel.tsx	NEW — Chat UI component
frontend/src/components/layout/Layout.tsx	Add chat toggle button

Tasks (in order)

[ ] Task 1: ChatMessage model + migration

• File: backend/app/models/chat.py (new)
• What: Create a ChatMessage model:

  class ChatMessage(Base):
      __tablename__ = "chat_messages"
      id: UUID PK
      tenant_id: UUID FK → tenants.id (nullable, indexed)
      user_id: UUID FK → users.id (nullable)
      session_id: UUID (groups messages in a conversation, indexed)
      role: String(20) — "user", "assistant", "system"
      content: Text
      context_type: String(50) nullable — "order", "product", "general"
      context_id: UUID nullable — order_id or product_id
      token_count: Integer nullable — track usage
      created_at: DateTime
  

• Also: Import in backend/app/models/__init__.py
• Migration: alembic revision --autogenerate -m "add chat_messages table"
• Acceptance gate: Table exists in DB; model importable
• Dependencies: None

[ ] Task 2: Chat service — Azure OpenAI with function calling

• File: backend/app/services/chat_service.py (new)

• What: Service with Azure OpenAI tool use / function calling:

  1. Takes a user message + session_id + tenant_id + user_id
  2. Loads tenant Azure credentials from tenant_config
  3. Defines tools the LLM can call (JSON schema for each):
     • list_orders(status, limit) — list tenant's orders
     • search_products(query, category, limit) — search products
     • create_order(product_ids, output_type_name, render_overrides, material_override) — create & submit
     • dispatch_renders(order_id) — dispatch renders for an order
     • get_order_status(order_id) — check render progress
     • set_material_override(order_id, material_name) — batch material override
     • set_render_overrides(order_id, overrides) — batch render overrides
     • get_render_stats() — throughput stats
     • check_materials(order_id) — unmapped materials check
     • query_database(sql) — read-only SQL (SELECT only, tenant-scoped)
  4. Calls Azure OpenAI with tools parameter — the LLM decides which tool to call
  5. Executes the tool call internally (same functions as MCP server but tenant-scoped)
  6. Returns tool result to LLM for a natural language response
  7. Stores conversation in ChatMessage table

  Tenant isolation: All DB queries filter by tenant_id. The query_database tool auto-appends WHERE tenant_id = '{tenant_id}' or validates tenant scope.

  Tool execution: Uses the existing backend API functions directly (not HTTP calls) — import from the routers/services.

  tools = [
      {
          "type": "function",
          "function": {
              "name": "search_products",
              "description": "Search products by name, PIM-ID, or category",
              "parameters": {
                  "type": "object",
                  "properties": {
                      "query": {"type": "string"},
                      "category": {"type": "string"},
                  }
              }
          }
      },
      # ... more tools
  ]
  response = client.chat.completions.create(
      model=deployment,
      messages=messages,
      tools=tools,
      tool_choice="auto",
  )
  # Handle tool_calls in response, execute, return result
  

• Acceptance gate: User can say "show my last 5 orders" and get real data back via function calling

• Dependencies: Task 1

[ ] Task 3: Chat API endpoints

• File: backend/app/api/routers/chat.py (new)
• What: FastAPI router with endpoints:
  • POST /api/chat/messages — send a message, get AI response
    • Body: { message: str, session_id: str | None, context_type: str | None, context_id: str | None }
    • Creates session_id if not provided
    • Returns: { session_id: str, message: ChatMessageOut, response: ChatMessageOut }
    • Auth: get_current_user — uses user's tenant AI config
  • GET /api/chat/sessions — list user's chat sessions
    • Returns: [{ session_id, last_message, message_count, created_at }]
  • GET /api/chat/sessions/{session_id}/messages — get conversation history
    • Returns: [{ id, role, content, created_at }]
  • DELETE /api/chat/sessions/{session_id} — delete a conversation
• Also: Register router in backend/app/main.py
• Acceptance gate: POST /api/chat/messages returns an AI response using tenant credentials
• Dependencies: Task 2

[ ] Task 4: Frontend — Chat API types

• File: frontend/src/api/chat.ts (new)
• What: TypeScript interfaces and API functions:

  interface ChatMessage { id: string; role: 'user' | 'assistant' | 'system'; content: string; created_at: string }
  interface ChatSession { session_id: string; last_message: string; message_count: number; created_at: string }
  interface ChatResponse { session_id: string; message: ChatMessage; response: ChatMessage }
  
  function sendMessage(message: string, sessionId?: string, contextType?: string, contextId?: string): Promise<ChatResponse>
  function getSessions(): Promise<ChatSession[]>
  function getSessionMessages(sessionId: string): Promise<ChatMessage[]>
  function deleteSession(sessionId: string): Promise<void>
  

• Acceptance gate: Types compile; functions callable
• Dependencies: Task 3

[ ] Task 5: Frontend — ChatPanel component

• File: frontend/src/components/chat/ChatPanel.tsx (new)

• What: Slide-out chat panel (right side, similar to notification panels in modern apps):

  1. Header: "AI Assistant" title + close button + session selector
  2. Message list: Scrollable area with role-based styling:
     • User messages: right-aligned, accent background
     • Assistant messages: left-aligned, surface background, markdown support
     • Timestamps below each message
  3. Input area: Text input + send button (Enter to send)
  4. Loading state: Typing indicator while waiting for AI response
  5. Session management: "New conversation" button, session history dropdown
  6. Context awareness: When opened from an order/product page, auto-includes context

  Styling:

  • Fixed right panel (w-96, full height)
  • Backdrop overlay on mobile
  • Smooth slide-in animation
  • Use existing CSS variables (surface, content, accent)
  • lucide-react icons (MessageSquare, Send, Loader2, X, Plus)

• Acceptance gate: Panel opens/closes, messages send and display, AI responds

• Dependencies: Task 4

[ ] Task 6: Frontend — Chat toggle in Layout

• File: frontend/src/components/layout/Layout.tsx
• What: Add a chat toggle button:
  1. Floating button in bottom-right corner (or in the sidebar)
  2. Icon: MessageSquare from lucide-react
  3. Badge with unread count (optional, for future)
  4. Click toggles ChatPanel visibility
  5. Only show when tenant has ai_enabled = true
• Acceptance gate: Button visible for users with AI-enabled tenant; clicking opens/closes ChatPanel
• Dependencies: Task 5

Migration Check

Yes — one new table chat_messages with UUID PK, FK to tenants and users.

Order Recommendation

1. Backend model + migration (Task 1)
2. Backend service (Task 2)
3. Backend API (Task 3)
4. Frontend types (Task 4)
5. Frontend chat UI (Task 5)
6. Frontend layout integration (Task 6)

Risks / Open Questions

1. Azure OpenAI availability: If tenant hasn't configured AI credentials, the chat should show a helpful message ("AI not configured — ask your admin to set up Azure OpenAI in Tenant Settings")

2. Token costs: Each message uses Azure OpenAI tokens. Consider adding token counting and a configurable monthly limit per tenant.

3. Context enrichment: The system prompt could include live data (order counts, render status). This makes the AI more helpful but costs more tokens. Start simple, enhance later.

4. Streaming responses: Azure OpenAI supports streaming. V1 uses a simple request/response. V2 could stream via WebSocket for real-time typing effect.

5. openai package: The openai Python package must be installed in the backend container. Check if it's already a dependency (it may be via azure_ai.py).