Add comprehensive documentation and final improvements

Documentation Added:
- AI_INSIGHTS_DEMO_SETUP_GUIDE.md: Complete setup guide for demo sessions
- AI_INSIGHTS_DATA_FLOW.md: Architecture and data flow diagrams
- AI_INSIGHTS_QUICK_START.md: Quick reference guide
- DEMO_SESSION_ANALYSIS_REPORT.md: Detailed analysis of demo session d67eaae4
- ROOT_CAUSE_ANALYSIS_AND_FIXES.md: Complete analysis of 8 issues (6 fixed, 2 analyzed)
- COMPLETE_FIX_SUMMARY.md: Executive summary of all fixes
- FIX_MISSING_INSIGHTS.md: Forecasting and procurement fix guide
- FINAL_STATUS_SUMMARY.md: Status overview
- verify_fixes.sh: Automated verification script
- enhance_procurement_data.py: Procurement data enhancement script

Service Improvements:
- Demo session cleanup worker: Use proper settings for Redis configuration with TLS/auth
- Procurement service: Add Redis initialization with proper error handling and cleanup
- Production fixture: Remove duplicate worker assignments (cleaned 56 duplicates)
- Orchestrator fixture: Add purchase order metadata for better tracking

Impact:
- Complete documentation for troubleshooting and setup
- Improved Redis connection handling across services
- Clean production data without duplicates
- Better error handling and logging

🤖 Generated with Claude Code (https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
Urtzi Alfaro
2025-12-16 11:32:45 +01:00
parent 4418ff0876
commit 9f3b39bd28
14 changed files with 3982 additions and 60 deletions

354
AI_INSIGHTS_DATA_FLOW.md Normal file
View File

@@ -0,0 +1,354 @@
# AI Insights Data Flow Diagram
## Quick Reference: JSON Files → AI Insights
```
┌─────────────────────────────────────────────────────────────────┐
│ DEMO SESSION CREATION │
│ │
│ POST /api/demo/sessions {"demo_account_type": "professional"} │
└────────────────────────────┬────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ CLONE ORCHESTRATOR (Auto-triggered) │
│ │
│ Loads JSON files from: │
│ shared/demo/fixtures/professional/*.json │
│ │
│ Clones data to virtual_tenant_id │
└────────────────┬────────────────────┬────────────────┬──────────┘
│ │ │
▼ ▼ ▼
┌────────────────────┐ ┌─────────────────┐ ┌──────────────┐
│ 03-inventory.json │ │ 06-production │ │ 09-sales.json│
│ │ │ .json │ │ │
│ • stock_movements │ │ │ │ • sales_data │
│ (90 days) │ │ • batches with │ │ (30+ days) │
│ • PRODUCTION_USE │ │ staff_assigned│ │ │
│ • PURCHASE │ │ • yield_% │ │ │
│ • Stockouts (!) │ │ • duration │ │ │
└─────────┬──────────┘ └────────┬────────┘ └──────┬───────┘
│ │ │
│ │ │
▼ ▼ ▼
┌────────────────────┐ ┌─────────────────┐ ┌──────────────┐
│ Inventory Service │ │ Production │ │ Forecasting │
│ │ │ Service │ │ Service │
│ ML Model: │ │ │ │ │
│ Safety Stock │ │ ML Model: │ │ ML Model: │
│ Optimizer │ │ Yield Predictor │ │ Demand │
│ │ │ │ │ Analyzer │
└─────────┬──────────┘ └────────┬────────┘ └──────┬───────┘
│ │ │
│ Analyzes 90 days │ Correlates │ Detects
│ consumption │ worker skills │ trends &
│ patterns │ with yields │ seasonality
│ │ │
▼ ▼ ▼
┌────────────────────────────────────────────────────────────┐
│ Each Service Posts Insights via AIInsightsClient │
│ │
│ POST /api/ai-insights/tenants/{virtual_tenant_id}/insights│
└────────────────────────────┬───────────────────────────────┘
┌───────────────────────────────────────┐
│ AI Insights Service │
│ │
│ Database Tables: │
│ • ai_insights │
│ • insight_feedback │
│ • insight_correlations │
│ │
│ Stores: │
│ • Title, description │
│ • Priority, confidence │
│ • Impact metrics (€/year) │
│ • Recommendation actions │
│ • Expires in 7 days │
└───────────────┬───────────────────────┘
┌───────────────────────────────────────┐
│ RabbitMQ Events Published │
│ │
│ • ai_safety_stock_optimization │
│ • ai_yield_prediction │
│ • ai_demand_forecast │
│ • ai_price_forecast │
│ • ai_supplier_performance │
└───────────────┬───────────────────────┘
┌───────────────────────────────────────┐
│ Frontend Consumes │
│ │
│ GET /api/ai-insights/tenants/{id}/ │
│ insights?filters... │
│ │
│ Displays: │
│ • AIInsightsPage.tsx │
│ • AIInsightsWidget.tsx (dashboard) │
│ • Service-specific widgets │
└───────────────────────────────────────┘
```
## Data Requirements by AI Model
### 1. Safety Stock Optimizer
**Requires**: 90 days of stock movements
```json
// 03-inventory.json
{
"stock_movements": [
// Daily consumption (PRODUCTION_USE)
{ "movement_type": "PRODUCTION_USE", "quantity": 45.0, "movement_date": "BASE_TS - 1d" },
{ "movement_type": "PRODUCTION_USE", "quantity": 52.3, "movement_date": "BASE_TS - 2d" },
// ... repeat 90 days per ingredient
// Stockout events (critical!)
{ "movement_type": "PRODUCTION_USE", "quantity_after": 0.0, "movement_date": "BASE_TS - 15d" }
]
}
```
**Generates**:
- Optimal reorder points
- Cost savings from reduced safety stock
- Stockout risk alerts
---
### 2. Yield Predictor
**Requires**: Historical batches with worker data
```json
// 06-production.json
{
"batches": [
{
"yield_percentage": 96.5,
"staff_assigned": ["50000000-0000-0000-0000-000000000001"], // Expert worker
"actual_duration_minutes": 175.5
},
{
"yield_percentage": 88.2,
"staff_assigned": ["50000000-0000-0000-0000-000000000005"], // Junior worker
"actual_duration_minutes": 195.0
}
]
}
```
**Generates**:
- Yield predictions for upcoming batches
- Worker-product performance correlations
- Waste reduction opportunities
---
### 3. Demand Analyzer
**Requires**: Sales history (30+ days)
```json
// 09-sales.json
{
"sales_data": [
{ "product_id": "...", "quantity": 51.11, "sales_date": "BASE_TS - 1d" },
{ "product_id": "...", "quantity": 48.29, "sales_date": "BASE_TS - 2d" }
// ... repeat 30+ days
]
}
```
**Generates**:
- Trend analysis (up/down)
- Seasonal patterns
- Production recommendations
---
### 4. Price Forecaster
**Requires**: Purchase order history
```json
// 07-procurement.json
{
"purchase_orders": [
{
"supplier_id": "...",
"items": [{ "unit_price": 0.85, "ordered_quantity": 500 }],
"order_date": "BASE_TS - 7d"
},
{
"supplier_id": "...",
"items": [{ "unit_price": 0.92, "ordered_quantity": 500 }], // Price increased!
"order_date": "BASE_TS - 1d"
}
]
}
```
**Generates**:
- Price trend analysis
- Bulk buying opportunities
- Supplier cost comparisons
---
### 5. Supplier Performance Analyzer
**Requires**: Purchase orders with delivery tracking
```json
// 07-procurement.json
{
"purchase_orders": [
{
"supplier_id": "40000000-0000-0000-0000-000000000001",
"required_delivery_date": "BASE_TS - 4h",
"estimated_delivery_date": "BASE_TS - 4h",
"status": "confirmed", // Still not delivered = LATE
"reasoning_data": {
"metadata": {
"delivery_delayed": true,
"delay_hours": 4
}
}
}
]
}
```
**Generates**:
- Supplier reliability scores
- Delivery performance alerts
- Risk management recommendations
---
## Insight Types Generated
| Service | Category | Priority | Example Title |
|---------|----------|----------|---------------|
| Inventory | inventory | medium | "Safety stock optimization for Harina T55: Reduce from 200kg to 145kg, save €1,200/year" |
| Inventory | inventory | critical | "Stockout risk: Levadura Fresca below critical level (3 events in 90 days)" |
| Production | production | medium | "Yield prediction: Batch #4502 expected 94.2% yield - assign expert worker for 98%" |
| Production | production | high | "Waste reduction: Training junior staff on croissants could save €2,400/year" |
| Forecasting | forecasting | medium | "Demand trending up 15% for Croissants - increase production by 12 units next week" |
| Forecasting | forecasting | low | "Weekend sales 40% lower - optimize Saturday production to reduce waste" |
| Procurement | procurement | high | "Price alert: Mantequilla up 8% in 60 days - consider bulk purchase now" |
| Procurement | procurement | medium | "Supplier performance: Harinas del Norte late on 3/10 deliveries - consider backup" |
---
## Testing Checklist
Run this before creating a demo session:
```bash
cd /Users/urtzialfaro/Documents/bakery-ia
# 1. Generate AI insights data
python shared/demo/fixtures/professional/generate_ai_insights_data.py
# 2. Verify data counts
python -c "
import json
# Check inventory
with open('shared/demo/fixtures/professional/03-inventory.json') as f:
inv = json.load(f)
movements = len(inv.get('stock_movements', []))
stockouts = sum(1 for m in inv['stock_movements'] if m.get('quantity_after') == 0.0)
print(f'✓ Stock movements: {movements} (need 800+)')
print(f'✓ Stockout events: {stockouts} (need 5+)')
# Check production
with open('shared/demo/fixtures/professional/06-production.json') as f:
prod = json.load(f)
batches_with_workers = sum(1 for b in prod['batches'] if b.get('staff_assigned'))
batches_with_yield = sum(1 for b in prod['batches'] if b.get('yield_percentage'))
print(f'✓ Batches with workers: {batches_with_workers} (need 200+)')
print(f'✓ Batches with yield: {batches_with_yield} (need 200+)')
# Check sales
with open('shared/demo/fixtures/professional/09-sales.json') as f:
sales = json.load(f)
sales_count = len(sales.get('sales_data', []))
print(f'✓ Sales records: {sales_count} (need 30+)')
# Check procurement
with open('shared/demo/fixtures/professional/07-procurement.json') as f:
proc = json.load(f)
po_count = len(proc.get('purchase_orders', []))
delayed = sum(1 for po in proc['purchase_orders'] if po.get('reasoning_data', {}).get('metadata', {}).get('delivery_delayed'))
print(f'✓ Purchase orders: {po_count} (need 5+)')
print(f'✓ Delayed deliveries: {delayed} (need 1+)')
"
# 3. Validate JSON syntax
for file in shared/demo/fixtures/professional/*.json; do
echo "Checking $file..."
python -m json.tool "$file" > /dev/null && echo " ✓ Valid" || echo " ✗ INVALID JSON"
done
```
**Expected Output**:
```
✓ Stock movements: 842 (need 800+)
✓ Stockout events: 6 (need 5+)
✓ Batches with workers: 247 (need 200+)
✓ Batches with yield: 312 (need 200+)
✓ Sales records: 44 (need 30+)
✓ Purchase orders: 8 (need 5+)
✓ Delayed deliveries: 2 (need 1+)
Checking shared/demo/fixtures/professional/01-tenant.json...
✓ Valid
Checking shared/demo/fixtures/professional/02-auth.json...
✓ Valid
...
```
---
## Troubleshooting Quick Guide
| Problem | Cause | Solution |
|---------|-------|----------|
| No insights generated | Missing stock movements | Run `generate_ai_insights_data.py` |
| Low confidence scores | < 60 days of data | Ensure 90 days of movements |
| No yield predictions | Missing staff_assigned | Run generator script |
| No supplier insights | No delayed deliveries | Check 07-procurement.json for delayed POs |
| Insights not in frontend | Tenant ID mismatch | Verify virtual_tenant_id matches |
| DB errors during cloning | JSON syntax error | Validate all JSON files |
---
## Files Modified by Generator
When you run `generate_ai_insights_data.py`, these files are updated:
1. **03-inventory.json**:
- Adds ~842 stock movements
- Includes 5-8 stockout events
- Spans 90 days of history
2. **06-production.json**:
- Adds `staff_assigned` to ~247 batches
- Adds `actual_duration_minutes`
- Correlates workers with yields
**Backup your files first** (optional):
```bash
cp shared/demo/fixtures/professional/03-inventory.json shared/demo/fixtures/professional/03-inventory.json.backup
cp shared/demo/fixtures/professional/06-production.json shared/demo/fixtures/professional/06-production.json.backup
```
To restore:
```bash
cp shared/demo/fixtures/professional/03-inventory.json.backup shared/demo/fixtures/professional/03-inventory.json
cp shared/demo/fixtures/professional/06-production.json.backup shared/demo/fixtures/professional/06-production.json
```