bakery-ia/AI_INSIGHTS_DEMO_SETUP_GUIDE.md

# AI Insights Demo Setup Guide

## Overview
This guide explains how to populate demo JSON files to generate AI insights across different services during demo sessions.

## Architecture Summary

```
Demo Session Creation
  ↓
Clone Base Tenant Data (from JSON files)
  ↓
Populate Database with 90 days of history
  ↓
Trigger ML Models in Services
  ↓
Post AI Insights to AI Insights Service
  ↓
Display in Frontend
```

## Key Files to Populate

### 1. **03-inventory.json** - Stock Movements (CRITICAL for AI Insights)
**Location**: `/shared/demo/fixtures/professional/03-inventory.json`

**What to Add**: `stock_movements` array with 90 days of historical data

```json
{
  "stock_movements": [
    {
      "id": "uuid",
      "tenant_id": "a1b2c3d4-e5f6-47a8-b9c0-d1e2f3a4b5c6",
      "ingredient_id": "10000000-0000-0000-0000-000000000001",
      "stock_id": null,
      "movement_type": "PRODUCTION_USE",  // or "PURCHASE"
      "quantity": 45.23,
      "unit_cost": 0.85,
      "total_cost": 38.45,
      "quantity_before": null,
      "quantity_after": null,  // Set to 0.0 for stockout events!
      "movement_date": "BASE_TS - 7d",
      "reason_code": "production_consumption",
      "notes": "Daily production usage",
      "created_at": "BASE_TS - 7d",
      "created_by": "c1a2b3c4-d5e6-47a8-b9c0-d1e2f3a4b5c6"
    }
  ]
}
```

**Why This Matters**:
- **Safety Stock Optimizer** needs 90 days of `PRODUCTION_USE` movements to calculate:
  - Average daily consumption
  - Demand variability
  - Optimal reorder points
  - Cost savings from optimized safety stock levels
- **Stockout events** (quantity_after = 0.0) trigger critical insights
- **Purchase patterns** help identify supplier reliability

**AI Insights Generated**:
- `"Safety stock optimization: Reduce Harina T55 from 200kg to 145kg, save €1,200/year"`
- `"Detected 3 stockouts in 90 days for Levadura Fresca - increase safety stock by 25%"`
- `"Inventory carrying cost opportunity: €850/year savings across 5 ingredients"`

---

### 2. **06-production.json** - Worker Assignments (CRITICAL for Yield Predictions)
**Location**: `/shared/demo/fixtures/professional/06-production.json`

**What to Add**: Worker IDs to `batches` array + actual duration

```json
{
  "batches": [
    {
      "id": "40000000-0000-0000-0000-000000000001",
      "product_id": "20000000-0000-0000-0000-000000000001",
      "status": "COMPLETED",
      "yield_percentage": 96.5,
      "staff_assigned": [
        "50000000-0000-0000-0000-000000000001"  // Juan Panadero (expert)
      ],
      "actual_start_time": "BASE_TS - 6d 7h",
      "planned_duration_minutes": 180,
      "actual_duration_minutes": 175.5,
      "completed_at": "BASE_TS - 6d 4h"
    }
  ]
}
```

**Why This Matters**:
- **Yield Predictor** correlates worker skill levels with yield performance
- Needs historical batches with:
  - `staff_assigned` (worker IDs)
  - `yield_percentage`
  - `actual_duration_minutes`
- Worker skill levels defined in `generate_ai_insights_data.py`:
  - María García (Owner): 0.98 - Expert
  - Juan Panadero (Baker): 0.95 - Very skilled
  - Isabel Producción: 0.90 - Experienced
  - Carlos Almacén: 0.78 - Learning

**AI Insights Generated**:
- `"Batch #4502 predicted yield: 94.2% (±2.1%) - assign expert worker for 98% yield"`
- `"Waste reduction opportunity: Training junior staff could save €2,400/year"`
- `"Optimal staffing: Schedule María for croissants (complex), Carlos for baguettes (standard)"`

---

### 3. **09-sales.json** - Sales History (For Demand Forecasting)
**Location**: `/shared/demo/fixtures/professional/09-sales.json`

**What's Already There**: Daily sales records with variability

```json
{
  "sales_data": [
    {
      "id": "SALES-202501-2287",
      "tenant_id": "a1b2c3d4-e5f6-47a8-b9c0-d1e2f3a4b5c6",
      "product_id": "20000000-0000-0000-0000-000000000001",
      "quantity": 51.11,
      "unit_price": 6.92,
      "total_amount": 335.29,
      "sales_date": "BASE_TS - 7d 4h",
      "sales_channel": "online",
      "payment_method": "cash",
      "customer_id": "50000000-0000-0000-0000-000000000001"
    }
  ]
}
```

**AI Insights Generated**:
- `"Demand trending up 15% for Croissants - increase next week's production by 12 units"`
- `"Weekend sales 40% lower - reduce Saturday production to avoid waste"`
- `"Seasonal pattern detected: Baguette demand peaks Mondays (+25%)"`

---

### 4. **07-procurement.json** - Purchase Orders (For Supplier Performance)
**Location**: `/shared/demo/fixtures/professional/07-procurement.json`

**What's Already There**: Purchase orders with delivery tracking

```json
{
  "purchase_orders": [
    {
      "id": "50000000-0000-0000-0000-0000000000c1",
      "po_number": "PO-LATE-0001",
      "supplier_id": "40000000-0000-0000-0000-000000000001",
      "status": "confirmed",
      "required_delivery_date": "BASE_TS - 4h",
      "estimated_delivery_date": "BASE_TS - 4h",
      "notes": "⚠️ EDGE CASE: Delivery should have arrived 4 hours ago",
      "reasoning_data": {
        "type": "low_stock_detection",
        "metadata": {
          "delivery_delayed": true,
          "delay_hours": 4
        }
      }
    }
  ]
}
```

**AI Insights Generated**:
- `"Supplier 'Harinas del Norte' late on 3/10 deliveries - consider backup supplier"`
- `"Price trend: Mantequilla up 8% in 60 days - consider bulk purchase now"`
- `"Procurement optimization: Consolidate 3 orders to Lácteos Gipuzkoa, save €45 shipping"`

---

### 5. **11-orchestrator.json** - Orchestration Metadata
**Location**: `/shared/demo/fixtures/professional/11-orchestrator.json`

**What's Already There**: Last orchestration run results

```json
{
  "orchestration_run": {
    "id": "90000000-0000-0000-0000-000000000001",
    "status": "completed",
    "run_type": "daily",
    "started_at": "BASE_TS - 1d 16h",
    "completed_at": "BASE_TS - 1d 15h45m"
  },
  "orchestration_results": {
    "production_batches_created": 18,
    "purchase_orders_created": 6,
    "ai_insights_posted": 5  // ← Number of AI insights generated
  },
  "ai_insights": {
    "yield_improvement_suggestions": 2,
    "waste_reduction_opportunities": 1,
    "demand_forecasting_updates": 3,
    "procurement_optimization": 2,
    "production_scheduling": 1
  }
}
```

**Purpose**: Shows orchestration metadata, NOT the insights themselves (those are in ai_insights service)

---

## How AI Insights Are Generated

### Step 1: Demo Session Creation
When a user creates a demo session:

```bash
POST /api/demo/sessions
{
  "demo_account_type": "professional"
}
```

### Step 2: Data Cloning (Automatic)
The `CloneOrchestrator` clones base tenant data from JSON files:
- Copies inventory products, recipes, suppliers, etc.
- **Crucially**: Loads 90 days of stock movements
- Loads production batches with worker assignments
- Loads sales history

**File**: `/services/demo_session/app/services/clone_orchestrator.py`

### Step 3: AI Model Execution (After Data Clone)
Each service runs its ML models:

#### **Inventory Service**
```python
# File: /services/inventory/app/ml/safety_stock_insights_orchestrator.py
async def generate_portfolio_summary(tenant_id: str):
    # Analyze 90 days of stock movements
    # Calculate optimal safety stock levels
    # Generate insights with cost impact
    insights = await ai_insights_client.create_insights_bulk(tenant_id, insights_list)
```

**Triggers**: After inventory data is cloned
**Publishes Event**: `ai_safety_stock_optimization`

#### **Production Service**
```python
# File: /services/production/app/ml/yield_insights_orchestrator.py
async def generate_yield_predictions(tenant_id: str):
    # Analyze historical batches + worker performance
    # Predict yield for upcoming batches
    # Identify waste reduction opportunities
    insights = await ai_insights_client.create_insights_bulk(tenant_id, insights_list)
```

**Triggers**: After production batches are cloned
**Publishes Event**: `ai_yield_prediction`

#### **Forecasting Service**
```python
# File: /services/forecasting/app/ml/demand_insights_orchestrator.py
async def generate_demand_insights(tenant_id: str):
    # Analyze sales history
    # Detect trends, seasonality
    # Recommend production adjustments
    insights = await ai_insights_client.create_insights_bulk(tenant_id, insights_list)
```

**Triggers**: After forecasts are generated
**Publishes Event**: `ai_demand_forecast`

#### **Procurement Service**
```python
# File: /services/procurement/app/ml/price_insights_orchestrator.py
async def generate_price_insights(tenant_id: str):
    # Analyze purchase order history
    # Detect price trends
    # Recommend bulk buying opportunities
    insights = await ai_insights_client.create_insights_bulk(tenant_id, insights_list)
```

**Triggers**: After purchase orders are cloned
**Publishes Event**: `ai_price_forecast`

### Step 4: AI Insights Storage
All insights are posted to:
```
POST /api/ai-insights/tenants/{tenant_id}/insights
```

Stored in `ai_insights` service database with:
- Priority (low, medium, high, critical)
- Confidence score (0-100)
- Impact metrics (cost savings, waste reduction, etc.)
- Recommendation actions
- Expiration (default 7 days)

### Step 5: Frontend Display
User sees insights in:
- **AI Insights Page**: `/app/analytics/ai-insights`
- **Dashboard Widget**: Summary of actionable insights
- **Service-specific pages**: Contextual insights (e.g., production page shows yield predictions)

---

## Running the Generator Script

### Automated Approach (Recommended)
Run the provided script to populate **03-inventory.json** and **06-production.json**:

```bash
cd /Users/urtzialfaro/Documents/bakery-ia
python shared/demo/fixtures/professional/generate_ai_insights_data.py
```

**What it does**:
1. Generates **~800-900 stock movements** (90 days × 10 ingredients):
   - Daily PRODUCTION_USE movements with variability
   - Bi-weekly PURCHASE deliveries
   - 5-8 stockout events (quantity_after = 0.0)

2. Adds **worker assignments** to production batches:
   - Assigns workers based on yield performance
   - Adds actual_duration_minutes
   - Correlates high yields with expert workers

3. **Output**:
   ```
   ✅ AI INSIGHTS DATA GENERATION COMPLETE

   📊 DATA ADDED:
      • Stock movements (PRODUCTION_USE): 720 records (90 days)
      • Stock movements (PURCHASE): 60 deliveries
      • Stockout events: 6
      • Worker assignments: 245 batches

   🎯 AI INSIGHTS READINESS:
      ✓ Safety Stock Optimizer: READY (90 days demand data)
      ✓ Yield Predictor: READY (worker data added)
      ✓ Sustainability Metrics: READY (existing waste data)
   ```

---

## Manual Data Population (Alternative)

If you need custom data, manually add to JSON files:

### For Safety Stock Insights
Add to `03-inventory.json`:
```json
{
  "stock_movements": [
    // 90 days of daily consumption for each ingredient
    {
      "movement_type": "PRODUCTION_USE",
      "ingredient_id": "10000000-0000-0000-0000-000000000001",
      "quantity": 45.0,  // Average daily usage
      "movement_date": "BASE_TS - 1d"
    },
    {
      "movement_type": "PRODUCTION_USE",
      "ingredient_id": "10000000-0000-0000-0000-000000000001",
      "quantity": 52.3,  // Variability is key!
      "movement_date": "BASE_TS - 2d"
    },
    // ... repeat for 90 days

    // Add stockout events (triggers critical insights)
    {
      "movement_type": "PRODUCTION_USE",
      "ingredient_id": "10000000-0000-0000-0000-000000000001",
      "quantity": 48.0,
      "quantity_before": 45.0,
      "quantity_after": 0.0,  // STOCKOUT!
      "movement_date": "BASE_TS - 15d",
      "notes": "STOCKOUT - Ran out during production"
    }
  ]
}
```

### For Yield Prediction Insights
Add to `06-production.json`:
```json
{
  "batches": [
    {
      "id": "batch-uuid",
      "product_id": "20000000-0000-0000-0000-000000000001",
      "status": "COMPLETED",
      "yield_percentage": 96.5,  // High yield
      "staff_assigned": [
        "50000000-0000-0000-0000-000000000001"  // Expert worker (Juan)
      ],
      "actual_duration_minutes": 175.5,
      "planned_duration_minutes": 180
    },
    {
      "id": "batch-uuid-2",
      "product_id": "20000000-0000-0000-0000-000000000001",
      "status": "COMPLETED",
      "yield_percentage": 88.2,  // Lower yield
      "staff_assigned": [
        "50000000-0000-0000-0000-000000000005"  // Junior worker (Carlos)
      ],
      "actual_duration_minutes": 195.0,
      "planned_duration_minutes": 180
    }
  ]
}
```

---

## Verifying AI Insights Generation

### 1. Check Demo Session Logs
After creating a demo session, check service logs:

```bash
# Inventory service (safety stock insights)
docker logs bakery-inventory-service | grep "ai_safety_stock"

# Production service (yield insights)
docker logs bakery-production-service | grep "ai_yield"

# Forecasting service (demand insights)
docker logs bakery-forecasting-service | grep "ai_demand"

# Procurement service (price insights)
docker logs bakery-procurement-service | grep "ai_price"
```

### 2. Query AI Insights API
```bash
curl -X GET "http://localhost:8000/api/ai-insights/tenants/{tenant_id}/insights" \
  -H "Authorization: Bearer {token}"
```

**Expected Response**:
```json
{
  "items": [
    {
      "id": "insight-uuid",
      "type": "optimization",
      "category": "inventory",
      "priority": "medium",
      "confidence": 88.5,
      "title": "Safety stock optimization opportunity for Harina T55",
      "description": "Reduce safety stock from 200kg to 145kg based on 90-day demand analysis",
      "impact_type": "cost_savings",
      "impact_value": 1200.0,
      "impact_unit": "EUR/year",
      "is_actionable": true,
      "recommendation_actions": [
        "Update reorder point to 145kg",
        "Adjust automatic procurement rules"
      ],
      "status": "new",
      "detected_at": "2025-01-16T10:30:00Z"
    }
  ],
  "total": 5,
  "page": 1
}
```

### 3. Check Frontend
Navigate to: `http://localhost:3000/app/analytics/ai-insights`

Should see:
- **Statistics**: Total insights, actionable count, average confidence
- **Insight Cards**: Categorized by type (inventory, production, procurement, forecasting)
- **Action Buttons**: Apply, Dismiss, Acknowledge

---

## Troubleshooting

### No Insights Generated

**Problem**: AI Insights page shows 0 insights after demo session creation

**Solutions**:
1. **Check stock movements count**:
   ```bash
   # Should have ~800+ movements
   cat shared/demo/fixtures/professional/03-inventory.json | jq '.stock_movements | length'
   ```
   If < 100, run `generate_ai_insights_data.py`

2. **Check worker assignments**:
   ```bash
   # Should have ~200+ batches with staff_assigned
   cat shared/demo/fixtures/professional/06-production.json | jq '[.batches[] | select(.staff_assigned != null)] | length'
   ```
   If 0, run `generate_ai_insights_data.py`

3. **Check service logs for errors**:
   ```bash
   docker logs bakery-ai-insights-service --tail 100
   ```

4. **Verify ML models are enabled**:
   Check `.env` files for:
   ```
   AI_INSIGHTS_ENABLED=true
   ML_MODELS_ENABLED=true
   ```

### Insights Not Showing in Frontend

**Problem**: API returns insights but frontend shows empty

**Solutions**:
1. **Check tenant_id mismatch**:
   - Frontend uses virtual_tenant_id from demo session
   - Insights should be created with same virtual_tenant_id

2. **Check filters**:
   - Frontend may filter by status, priority, category
   - Try "Show All" filter

3. **Check browser console**:
   ```javascript
   // In browser dev tools
   localStorage.getItem('demo_session')
   // Should show virtual_tenant_id
   ```

### Low Confidence Scores

**Problem**: Insights generated but confidence < 50%

**Causes**:
- Insufficient historical data (< 60 days)
- High variability in data (inconsistent patterns)
- Missing worker assignments for yield predictions

**Solutions**:
- Ensure 90 days of stock movements
- Add more consistent patterns (reduce random variability)
- Verify all batches have `staff_assigned` and `yield_percentage`

---

## Summary Checklist

Before creating a demo session, verify:

- [ ] `03-inventory.json` has 800+ stock movements (90 days)
- [ ] Stock movements include PRODUCTION_USE and PURCHASE types
- [ ] 5-8 stockout events present (quantity_after = 0.0)
- [ ] `06-production.json` batches have `staff_assigned` arrays
- [ ] Batches have `yield_percentage` and `actual_duration_minutes`
- [ ] `09-sales.json` has daily sales for 30+ days
- [ ] `07-procurement.json` has purchase orders with delivery dates
- [ ] All JSON files are valid (no syntax errors)

**Quick Validation**:
```bash
cd /Users/urtzialfaro/Documents/bakery-ia
python -c "
import json
with open('shared/demo/fixtures/professional/03-inventory.json') as f:
    data = json.load(f)
    movements = len(data.get('stock_movements', []))
    stockouts = sum(1 for m in data['stock_movements'] if m.get('quantity_after') == 0.0)
    print(f'✓ Stock movements: {movements}')
    print(f'✓ Stockout events: {stockouts}')

with open('shared/demo/fixtures/professional/06-production.json') as f:
    data = json.load(f)
    batches_with_workers = sum(1 for b in data['batches'] if b.get('staff_assigned'))
    print(f'✓ Batches with workers: {batches_with_workers}')
"
```

**Expected Output**:
```
✓ Stock movements: 842
✓ Stockout events: 6
✓ Batches with workers: 247
```

---

## Next Steps

1. **Run generator script** (if not already done):
   ```bash
   python shared/demo/fixtures/professional/generate_ai_insights_data.py
   ```

2. **Create demo session**:
   ```bash
   curl -X POST http://localhost:8000/api/demo/sessions \
     -H "Content-Type: application/json" \
     -d '{"demo_account_type": "professional"}'
   ```

3. **Wait for cloning** (~40 seconds)

4. **Navigate to AI Insights**:
   `http://localhost:3000/app/analytics/ai-insights`

5. **Verify insights** (should see 5-10 insights across categories)

6. **Test actions**:
   - Click "Apply" on an insight
   - Check if recommendation is executed
   - Provide feedback on outcome

---

## Additional Resources

- **AI Insights Service**: `/services/ai_insights/README.md`
- **ML Models Documentation**: `/services/*/app/ml/README.md`
- **Demo Session Flow**: `/services/demo_session/README.md`
- **Frontend Integration**: `/frontend/src/pages/app/analytics/ai-insights/README.md`

For questions or issues, check service logs:
```bash
docker-compose logs -f ai-insights-service inventory-service production-service forecasting-service procurement-service
```