Add comprehensive documentation and final improvements
Documentation Added: - AI_INSIGHTS_DEMO_SETUP_GUIDE.md: Complete setup guide for demo sessions - AI_INSIGHTS_DATA_FLOW.md: Architecture and data flow diagrams - AI_INSIGHTS_QUICK_START.md: Quick reference guide - DEMO_SESSION_ANALYSIS_REPORT.md: Detailed analysis of demo session d67eaae4 - ROOT_CAUSE_ANALYSIS_AND_FIXES.md: Complete analysis of 8 issues (6 fixed, 2 analyzed) - COMPLETE_FIX_SUMMARY.md: Executive summary of all fixes - FIX_MISSING_INSIGHTS.md: Forecasting and procurement fix guide - FINAL_STATUS_SUMMARY.md: Status overview - verify_fixes.sh: Automated verification script - enhance_procurement_data.py: Procurement data enhancement script Service Improvements: - Demo session cleanup worker: Use proper settings for Redis configuration with TLS/auth - Procurement service: Add Redis initialization with proper error handling and cleanup - Production fixture: Remove duplicate worker assignments (cleaned 56 duplicates) - Orchestrator fixture: Add purchase order metadata for better tracking Impact: - Complete documentation for troubleshooting and setup - Improved Redis connection handling across services - Clean production data without duplicates - Better error handling and logging 🤖 Generated with Claude Code (https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit is contained in:
354
AI_INSIGHTS_DATA_FLOW.md
Normal file
354
AI_INSIGHTS_DATA_FLOW.md
Normal file
@@ -0,0 +1,354 @@
|
||||
# AI Insights Data Flow Diagram
|
||||
|
||||
## Quick Reference: JSON Files → AI Insights
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ DEMO SESSION CREATION │
|
||||
│ │
|
||||
│ POST /api/demo/sessions {"demo_account_type": "professional"} │
|
||||
└────────────────────────────┬────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────────┐
|
||||
│ CLONE ORCHESTRATOR (Auto-triggered) │
|
||||
│ │
|
||||
│ Loads JSON files from: │
|
||||
│ shared/demo/fixtures/professional/*.json │
|
||||
│ │
|
||||
│ Clones data to virtual_tenant_id │
|
||||
└────────────────┬────────────────────┬────────────────┬──────────┘
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────────────┐ ┌─────────────────┐ ┌──────────────┐
|
||||
│ 03-inventory.json │ │ 06-production │ │ 09-sales.json│
|
||||
│ │ │ .json │ │ │
|
||||
│ • stock_movements │ │ │ │ • sales_data │
|
||||
│ (90 days) │ │ • batches with │ │ (30+ days) │
|
||||
│ • PRODUCTION_USE │ │ staff_assigned│ │ │
|
||||
│ • PURCHASE │ │ • yield_% │ │ │
|
||||
│ • Stockouts (!) │ │ • duration │ │ │
|
||||
└─────────┬──────────┘ └────────┬────────┘ └──────┬───────┘
|
||||
│ │ │
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────────────┐ ┌─────────────────┐ ┌──────────────┐
|
||||
│ Inventory Service │ │ Production │ │ Forecasting │
|
||||
│ │ │ Service │ │ Service │
|
||||
│ ML Model: │ │ │ │ │
|
||||
│ Safety Stock │ │ ML Model: │ │ ML Model: │
|
||||
│ Optimizer │ │ Yield Predictor │ │ Demand │
|
||||
│ │ │ │ │ Analyzer │
|
||||
└─────────┬──────────┘ └────────┬────────┘ └──────┬───────┘
|
||||
│ │ │
|
||||
│ Analyzes 90 days │ Correlates │ Detects
|
||||
│ consumption │ worker skills │ trends &
|
||||
│ patterns │ with yields │ seasonality
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ Each Service Posts Insights via AIInsightsClient │
|
||||
│ │
|
||||
│ POST /api/ai-insights/tenants/{virtual_tenant_id}/insights│
|
||||
└────────────────────────────┬───────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────────────┐
|
||||
│ AI Insights Service │
|
||||
│ │
|
||||
│ Database Tables: │
|
||||
│ • ai_insights │
|
||||
│ • insight_feedback │
|
||||
│ • insight_correlations │
|
||||
│ │
|
||||
│ Stores: │
|
||||
│ • Title, description │
|
||||
│ • Priority, confidence │
|
||||
│ • Impact metrics (€/year) │
|
||||
│ • Recommendation actions │
|
||||
│ • Expires in 7 days │
|
||||
└───────────────┬───────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────────────┐
|
||||
│ RabbitMQ Events Published │
|
||||
│ │
|
||||
│ • ai_safety_stock_optimization │
|
||||
│ • ai_yield_prediction │
|
||||
│ • ai_demand_forecast │
|
||||
│ • ai_price_forecast │
|
||||
│ • ai_supplier_performance │
|
||||
└───────────────┬───────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────────────┐
|
||||
│ Frontend Consumes │
|
||||
│ │
|
||||
│ GET /api/ai-insights/tenants/{id}/ │
|
||||
│ insights?filters... │
|
||||
│ │
|
||||
│ Displays: │
|
||||
│ • AIInsightsPage.tsx │
|
||||
│ • AIInsightsWidget.tsx (dashboard) │
|
||||
│ • Service-specific widgets │
|
||||
└───────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Data Requirements by AI Model
|
||||
|
||||
### 1. Safety Stock Optimizer
|
||||
**Requires**: 90 days of stock movements
|
||||
|
||||
```json
|
||||
// 03-inventory.json
|
||||
{
|
||||
"stock_movements": [
|
||||
// Daily consumption (PRODUCTION_USE)
|
||||
{ "movement_type": "PRODUCTION_USE", "quantity": 45.0, "movement_date": "BASE_TS - 1d" },
|
||||
{ "movement_type": "PRODUCTION_USE", "quantity": 52.3, "movement_date": "BASE_TS - 2d" },
|
||||
// ... repeat 90 days per ingredient
|
||||
|
||||
// Stockout events (critical!)
|
||||
{ "movement_type": "PRODUCTION_USE", "quantity_after": 0.0, "movement_date": "BASE_TS - 15d" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Generates**:
|
||||
- Optimal reorder points
|
||||
- Cost savings from reduced safety stock
|
||||
- Stockout risk alerts
|
||||
|
||||
---
|
||||
|
||||
### 2. Yield Predictor
|
||||
**Requires**: Historical batches with worker data
|
||||
|
||||
```json
|
||||
// 06-production.json
|
||||
{
|
||||
"batches": [
|
||||
{
|
||||
"yield_percentage": 96.5,
|
||||
"staff_assigned": ["50000000-0000-0000-0000-000000000001"], // Expert worker
|
||||
"actual_duration_minutes": 175.5
|
||||
},
|
||||
{
|
||||
"yield_percentage": 88.2,
|
||||
"staff_assigned": ["50000000-0000-0000-0000-000000000005"], // Junior worker
|
||||
"actual_duration_minutes": 195.0
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Generates**:
|
||||
- Yield predictions for upcoming batches
|
||||
- Worker-product performance correlations
|
||||
- Waste reduction opportunities
|
||||
|
||||
---
|
||||
|
||||
### 3. Demand Analyzer
|
||||
**Requires**: Sales history (30+ days)
|
||||
|
||||
```json
|
||||
// 09-sales.json
|
||||
{
|
||||
"sales_data": [
|
||||
{ "product_id": "...", "quantity": 51.11, "sales_date": "BASE_TS - 1d" },
|
||||
{ "product_id": "...", "quantity": 48.29, "sales_date": "BASE_TS - 2d" }
|
||||
// ... repeat 30+ days
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Generates**:
|
||||
- Trend analysis (up/down)
|
||||
- Seasonal patterns
|
||||
- Production recommendations
|
||||
|
||||
---
|
||||
|
||||
### 4. Price Forecaster
|
||||
**Requires**: Purchase order history
|
||||
|
||||
```json
|
||||
// 07-procurement.json
|
||||
{
|
||||
"purchase_orders": [
|
||||
{
|
||||
"supplier_id": "...",
|
||||
"items": [{ "unit_price": 0.85, "ordered_quantity": 500 }],
|
||||
"order_date": "BASE_TS - 7d"
|
||||
},
|
||||
{
|
||||
"supplier_id": "...",
|
||||
"items": [{ "unit_price": 0.92, "ordered_quantity": 500 }], // Price increased!
|
||||
"order_date": "BASE_TS - 1d"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Generates**:
|
||||
- Price trend analysis
|
||||
- Bulk buying opportunities
|
||||
- Supplier cost comparisons
|
||||
|
||||
---
|
||||
|
||||
### 5. Supplier Performance Analyzer
|
||||
**Requires**: Purchase orders with delivery tracking
|
||||
|
||||
```json
|
||||
// 07-procurement.json
|
||||
{
|
||||
"purchase_orders": [
|
||||
{
|
||||
"supplier_id": "40000000-0000-0000-0000-000000000001",
|
||||
"required_delivery_date": "BASE_TS - 4h",
|
||||
"estimated_delivery_date": "BASE_TS - 4h",
|
||||
"status": "confirmed", // Still not delivered = LATE
|
||||
"reasoning_data": {
|
||||
"metadata": {
|
||||
"delivery_delayed": true,
|
||||
"delay_hours": 4
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**Generates**:
|
||||
- Supplier reliability scores
|
||||
- Delivery performance alerts
|
||||
- Risk management recommendations
|
||||
|
||||
---
|
||||
|
||||
## Insight Types Generated
|
||||
|
||||
| Service | Category | Priority | Example Title |
|
||||
|---------|----------|----------|---------------|
|
||||
| Inventory | inventory | medium | "Safety stock optimization for Harina T55: Reduce from 200kg to 145kg, save €1,200/year" |
|
||||
| Inventory | inventory | critical | "Stockout risk: Levadura Fresca below critical level (3 events in 90 days)" |
|
||||
| Production | production | medium | "Yield prediction: Batch #4502 expected 94.2% yield - assign expert worker for 98%" |
|
||||
| Production | production | high | "Waste reduction: Training junior staff on croissants could save €2,400/year" |
|
||||
| Forecasting | forecasting | medium | "Demand trending up 15% for Croissants - increase production by 12 units next week" |
|
||||
| Forecasting | forecasting | low | "Weekend sales 40% lower - optimize Saturday production to reduce waste" |
|
||||
| Procurement | procurement | high | "Price alert: Mantequilla up 8% in 60 days - consider bulk purchase now" |
|
||||
| Procurement | procurement | medium | "Supplier performance: Harinas del Norte late on 3/10 deliveries - consider backup" |
|
||||
|
||||
---
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
Run this before creating a demo session:
|
||||
|
||||
```bash
|
||||
cd /Users/urtzialfaro/Documents/bakery-ia
|
||||
|
||||
# 1. Generate AI insights data
|
||||
python shared/demo/fixtures/professional/generate_ai_insights_data.py
|
||||
|
||||
# 2. Verify data counts
|
||||
python -c "
|
||||
import json
|
||||
|
||||
# Check inventory
|
||||
with open('shared/demo/fixtures/professional/03-inventory.json') as f:
|
||||
inv = json.load(f)
|
||||
movements = len(inv.get('stock_movements', []))
|
||||
stockouts = sum(1 for m in inv['stock_movements'] if m.get('quantity_after') == 0.0)
|
||||
print(f'✓ Stock movements: {movements} (need 800+)')
|
||||
print(f'✓ Stockout events: {stockouts} (need 5+)')
|
||||
|
||||
# Check production
|
||||
with open('shared/demo/fixtures/professional/06-production.json') as f:
|
||||
prod = json.load(f)
|
||||
batches_with_workers = sum(1 for b in prod['batches'] if b.get('staff_assigned'))
|
||||
batches_with_yield = sum(1 for b in prod['batches'] if b.get('yield_percentage'))
|
||||
print(f'✓ Batches with workers: {batches_with_workers} (need 200+)')
|
||||
print(f'✓ Batches with yield: {batches_with_yield} (need 200+)')
|
||||
|
||||
# Check sales
|
||||
with open('shared/demo/fixtures/professional/09-sales.json') as f:
|
||||
sales = json.load(f)
|
||||
sales_count = len(sales.get('sales_data', []))
|
||||
print(f'✓ Sales records: {sales_count} (need 30+)')
|
||||
|
||||
# Check procurement
|
||||
with open('shared/demo/fixtures/professional/07-procurement.json') as f:
|
||||
proc = json.load(f)
|
||||
po_count = len(proc.get('purchase_orders', []))
|
||||
delayed = sum(1 for po in proc['purchase_orders'] if po.get('reasoning_data', {}).get('metadata', {}).get('delivery_delayed'))
|
||||
print(f'✓ Purchase orders: {po_count} (need 5+)')
|
||||
print(f'✓ Delayed deliveries: {delayed} (need 1+)')
|
||||
"
|
||||
|
||||
# 3. Validate JSON syntax
|
||||
for file in shared/demo/fixtures/professional/*.json; do
|
||||
echo "Checking $file..."
|
||||
python -m json.tool "$file" > /dev/null && echo " ✓ Valid" || echo " ✗ INVALID JSON"
|
||||
done
|
||||
```
|
||||
|
||||
**Expected Output**:
|
||||
```
|
||||
✓ Stock movements: 842 (need 800+)
|
||||
✓ Stockout events: 6 (need 5+)
|
||||
✓ Batches with workers: 247 (need 200+)
|
||||
✓ Batches with yield: 312 (need 200+)
|
||||
✓ Sales records: 44 (need 30+)
|
||||
✓ Purchase orders: 8 (need 5+)
|
||||
✓ Delayed deliveries: 2 (need 1+)
|
||||
|
||||
Checking shared/demo/fixtures/professional/01-tenant.json...
|
||||
✓ Valid
|
||||
Checking shared/demo/fixtures/professional/02-auth.json...
|
||||
✓ Valid
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting Quick Guide
|
||||
|
||||
| Problem | Cause | Solution |
|
||||
|---------|-------|----------|
|
||||
| No insights generated | Missing stock movements | Run `generate_ai_insights_data.py` |
|
||||
| Low confidence scores | < 60 days of data | Ensure 90 days of movements |
|
||||
| No yield predictions | Missing staff_assigned | Run generator script |
|
||||
| No supplier insights | No delayed deliveries | Check 07-procurement.json for delayed POs |
|
||||
| Insights not in frontend | Tenant ID mismatch | Verify virtual_tenant_id matches |
|
||||
| DB errors during cloning | JSON syntax error | Validate all JSON files |
|
||||
|
||||
---
|
||||
|
||||
## Files Modified by Generator
|
||||
|
||||
When you run `generate_ai_insights_data.py`, these files are updated:
|
||||
|
||||
1. **03-inventory.json**:
|
||||
- Adds ~842 stock movements
|
||||
- Includes 5-8 stockout events
|
||||
- Spans 90 days of history
|
||||
|
||||
2. **06-production.json**:
|
||||
- Adds `staff_assigned` to ~247 batches
|
||||
- Adds `actual_duration_minutes`
|
||||
- Correlates workers with yields
|
||||
|
||||
**Backup your files first** (optional):
|
||||
```bash
|
||||
cp shared/demo/fixtures/professional/03-inventory.json shared/demo/fixtures/professional/03-inventory.json.backup
|
||||
cp shared/demo/fixtures/professional/06-production.json shared/demo/fixtures/professional/06-production.json.backup
|
||||
```
|
||||
|
||||
To restore:
|
||||
```bash
|
||||
cp shared/demo/fixtures/professional/03-inventory.json.backup shared/demo/fixtures/professional/03-inventory.json
|
||||
cp shared/demo/fixtures/professional/06-production.json.backup shared/demo/fixtures/professional/06-production.json
|
||||
```
|
||||
Reference in New Issue
Block a user