Files
bakery-ia/docs/08-api-reference/ai-insights-api.md
2025-11-05 13:34:56 +01:00

1019 lines
24 KiB
Markdown

# Technical Documentation - Bakery IA AI Insights Platform
## Table of Contents
1. [API Reference](#api-reference)
2. [Deployment Guide](#deployment-guide)
3. [Implementation Details](#implementation-details)
4. [Dynamic Rules Engine](#dynamic-rules-engine)
5. [Database Management](#database-management)
6. [Configuration](#configuration)
7. [Troubleshooting](#troubleshooting)
---
## API Reference
### Base URL
```
http://ai-insights-service:8000/api/v1/ai-insights
```
### Authentication
All endpoints require either:
- JWT token in `Authorization: Bearer <token>` header
- Service token in `X-Service-Token` header
- Demo session ID in `X-Demo-Session-Id` header
### Tenant Context
All endpoints include tenant ID in the path:
```
/api/v1/ai-insights/tenants/{tenant_id}/...
```
---
### Insights Endpoints
#### Create Insight
**POST** `/tenants/{tenant_id}/insights`
Creates a new AI insight.
**Request Body:**
```json
{
"type": "prediction", // required: prediction, recommendation, alert, optimization
"priority": "high", // required: critical, high, medium, low
"category": "forecasting", // required: forecasting, inventory, production, procurement, etc.
"title": "Weekend Demand Surge", // required: max 255 chars
"description": "Detailed explanation...", // optional: text
"confidence": 87, // required: 0-100
"metrics_json": { // optional: JSONB object
"product_id": "croissant",
"predicted_demand": 130,
"increase_percentage": 30
},
"impact_type": "revenue_increase", // optional: revenue_increase, cost_reduction, etc.
"impact_value": 450.00, // optional: decimal
"impact_unit": "euros", // optional: string
"actionable": true, // optional: boolean, default true
"recommendation_actions": [ // optional: array of actions
{
"service": "production",
"action": "increase_production",
"parameters": "{\"product_id\": \"croissant\", \"quantity\": 30}"
}
],
"source_service": "forecasting", // required: originating service
"source_data_id": "forecast_001", // optional: reference ID
"valid_from": "2025-11-03T00:00:00Z", // optional: ISO 8601
"valid_until": "2025-11-05T23:59:59Z" // optional: ISO 8601
}
```
**Response:** `201 Created`
```json
{
"id": "uuid",
"tenant_id": "uuid",
"type": "prediction",
"priority": "high",
"category": "forecasting",
"title": "Weekend Demand Surge",
"description": "Detailed explanation...",
"confidence": 87,
"metrics_json": {...},
"impact_type": "revenue_increase",
"impact_value": 450.00,
"impact_unit": "euros",
"status": "new",
"actionable": true,
"recommendation_actions": [...],
"source_service": "forecasting",
"source_data_id": "forecast_001",
"valid_from": "2025-11-03T00:00:00Z",
"valid_until": "2025-11-05T23:59:59Z",
"created_at": "2025-11-03T10:30:00Z",
"updated_at": "2025-11-03T10:30:00Z"
}
```
---
#### List Insights
**GET** `/tenants/{tenant_id}/insights`
Retrieves paginated list of insights with optional filters.
**Query Parameters:**
- `skip` (int, default=0): Pagination offset
- `limit` (int, default=100, max=1000): Results per page
- `priority` (string): Filter by priority (critical, high, medium, low)
- `category` (string): Filter by category
- `status` (string): Filter by status (new, acknowledged, in_progress, applied, dismissed)
- `actionable_only` (boolean): Only actionable insights
- `min_confidence` (int, 0-100): Minimum confidence score
**Response:** `200 OK`
```json
{
"items": [
{
"id": "uuid",
"title": "...",
// ... full insight object
}
],
"total": 42,
"skip": 0,
"limit": 100
}
```
---
#### Get Single Insight
**GET** `/tenants/{tenant_id}/insights/{insight_id}`
**Response:** `200 OK`
```json
{
"id": "uuid",
// ... full insight object
}
```
**Errors:**
- `404 Not Found`: Insight doesn't exist
- `403 Forbidden`: Tenant mismatch
---
#### Update Insight
**PATCH** `/tenants/{tenant_id}/insights/{insight_id}`
Updates specific fields of an insight.
**Request Body:**
```json
{
"status": "acknowledged", // new, acknowledged, in_progress, applied, dismissed
"priority": "critical", // optional: upgrade/downgrade priority
"notes": "Additional info" // optional: any field that's updatable
}
```
**Response:** `200 OK`
```json
{
// updated insight object
}
```
---
#### Delete Insight (Soft Delete)
**DELETE** `/tenants/{tenant_id}/insights/{insight_id}`
Marks insight as deleted (soft delete).
**Response:** `204 No Content`
---
#### Get Orchestration-Ready Insights
**GET** `/tenants/{tenant_id}/insights/orchestration-ready`
Retrieves insights grouped by category, ready for orchestration.
**Query Parameters:**
- `target_date` (ISO 8601): Target execution date
- `min_confidence` (int, default=70): Minimum confidence threshold
**Response:** `200 OK`
```json
{
"forecast_adjustments": [
{
"id": "uuid",
"title": "...",
"confidence": 87,
"recommendation_actions": [...]
}
],
"procurement_recommendations": [...],
"production_optimizations": [...],
"supplier_alerts": [...],
"price_opportunities": [...]
}
```
---
### Feedback Endpoints
#### Record Feedback
**POST** `/tenants/{tenant_id}/insights/{insight_id}/feedback`
Records actual outcome and compares with prediction.
**Request Body:**
```json
{
"action_taken": "increased_production",
"success": true,
"result_data": {
"planned_increase": 30,
"actual_increase": 28,
"revenue_impact": 420.00
},
"expected_impact_value": 450.00,
"actual_impact_value": 420.00,
"variance_percentage": -6.67,
"accuracy_score": 93.3,
"notes": "Slightly lower than predicted due to supply constraints"
}
```
**Response:** `200 OK`
```json
{
"id": "uuid",
"insight_id": "uuid",
"action_taken": "increased_production",
"success": true,
"result_data": {...},
"expected_impact_value": 450.00,
"actual_impact_value": 420.00,
"variance_percentage": -6.67,
"accuracy_score": 93.3,
"notes": "...",
"created_at": "2025-11-03T18:00:00Z"
}
```
**Side Effects:**
- Automatically updates insight status to "applied"
- Triggers FeedbackLearningSystem analysis
- May trigger model retraining if performance degrades
---
### Metrics Endpoints
#### Get Summary Metrics
**GET** `/tenants/{tenant_id}/insights/metrics/summary`
Retrieves aggregate metrics for all insights.
**Response:** `200 OK`
```json
{
"total_insights": 147,
"actionable_insights": 98,
"average_confidence": 82.5,
"critical_priority_count": 12,
"high_priority_count": 45,
"medium_priority_count": 67,
"low_priority_count": 23,
"by_category": {
"forecasting": 42,
"inventory": 35,
"production": 28,
"procurement": 22,
"customer": 20
},
"by_status": {
"new": 56,
"acknowledged": 28,
"in_progress": 15,
"applied": 42,
"dismissed": 6
}
}
```
---
## Deployment Guide
### Prerequisites
- **Kubernetes Cluster:** 1.24+
- **Docker:** 20.10+
- **Kind:** 0.20+ (for local development)
- **kubectl:** 1.24+
- **Tilt:** 0.30+ (optional, for development)
### Local Development Setup
#### 1. Start Kubernetes Cluster
```bash
# Create Kind cluster
kind create cluster --name bakery-ia-local --config infrastructure/kind-config.yaml
# Verify cluster
kubectl cluster-info
```
#### 2. Deploy Infrastructure
```bash
# Create namespace
kubectl create namespace bakery-ia
# Deploy databases
kubectl apply -f infrastructure/kubernetes/base/components/databases/
# Wait for databases to be ready
kubectl wait --for=condition=ready pod -l app=postgresql-main -n bakery-ia --timeout=300s
kubectl wait --for=condition=ready pod -l app=postgresql-ai-insights -n bakery-ia --timeout=300s
kubectl wait --for=condition=ready pod -l app=redis -n bakery-ia --timeout=300s
```
#### 3. Deploy Services
```bash
# Deploy all services
kubectl apply -f infrastructure/kubernetes/base/
# Watch deployment
kubectl get pods -n bakery-ia -w
```
#### 4. Run Database Migrations
```bash
# AI Insights Service migration
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
python -m alembic upgrade head
# Other services...
for service in orders inventory production suppliers; do
kubectl exec -n bakery-ia deployment/${service}-service -- \
python -m alembic upgrade head
done
```
#### 5. Verify Deployment
```bash
# Check all pods are running
kubectl get pods -n bakery-ia
# Check services
kubectl get svc -n bakery-ia
# Test AI Insights Service health
kubectl port-forward -n bakery-ia svc/ai-insights-service 8000:8000 &
curl http://localhost:8000/health
```
---
### Production Deployment
#### Environment Configuration
Create environment-specific configurations:
```yaml
# infrastructure/kubernetes/overlays/production/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base
replicas:
- name: ai-insights-service
count: 3
- name: orchestration-service
count: 2
configMapGenerator:
- name: ai-insights-config
env: production.env
secretGenerator:
- name: database-secrets
envs:
- secrets.env
images:
- name: bakery/ai-insights-service
newTag: v1.0.0
```
#### Deploy to Production
```bash
# Apply with kustomize
kubectl apply -k infrastructure/kubernetes/overlays/production/
# Rolling update
kubectl set image deployment/ai-insights-service \
ai-insights-service=bakery/ai-insights-service:v1.0.1 \
-n bakery-ia
# Monitor rollout
kubectl rollout status deployment/ai-insights-service -n bakery-ia
```
---
### Database Management
#### Create AI Insights Database
```bash
# Connect to PostgreSQL
kubectl exec -it -n bakery-ia postgresql-ai-insights-0 -- psql -U postgres
# Create database
CREATE DATABASE ai_insights_db;
# Create user
CREATE USER ai_insights_user WITH PASSWORD 'secure_password';
GRANT ALL PRIVILEGES ON DATABASE ai_insights_db TO ai_insights_user;
# Enable UUID extension
\c ai_insights_db
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
```
#### Run Migrations
```bash
# Check current version
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
python -m alembic current
# Upgrade to latest
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
python -m alembic upgrade head
# Downgrade one version
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
python -m alembic downgrade -1
# Show migration history
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
python -m alembic history
```
#### Backup and Restore
```bash
# Backup
kubectl exec -n bakery-ia postgresql-ai-insights-0 -- \
pg_dump -U postgres ai_insights_db > backup-$(date +%Y%m%d).sql
# Restore
kubectl exec -i -n bakery-ia postgresql-ai-insights-0 -- \
psql -U postgres ai_insights_db < backup-20251103.sql
```
---
## Implementation Details
### Service Structure
```
services/ai_insights/
├── app/
│ ├── __init__.py
│ ├── main.py # FastAPI application
│ ├── core/
│ │ ├── config.py # Configuration
│ │ ├── database.py # Database connection
│ │ └── security.py # Auth utilities
│ ├── models/
│ │ ├── ai_insight.py # SQLAlchemy models
│ │ └── feedback.py
│ ├── schemas/
│ │ ├── insight.py # Pydantic schemas
│ │ └── feedback.py
│ ├── api/
│ │ ├── insights.py # Insight endpoints
│ │ ├── feedback.py # Feedback endpoints
│ │ └── metrics.py # Metrics endpoints
│ ├── services/
│ │ ├── insight_service.py # Business logic
│ │ └── feedback_service.py
│ ├── repositories/
│ │ ├── insight_repository.py # Data access
│ │ └── feedback_repository.py
│ └── ml/
│ └── feedback_learning_system.py # Learning system
├── tests/
│ ├── unit/
│ ├── integration/
│ └── conftest.py
├── migrations/
│ └── versions/ # Alembic migrations
├── Dockerfile
├── requirements.txt
└── alembic.ini
```
### Key Components
#### FastAPI Application
```python
# app/main.py
from fastapi import FastAPI
from app.api import insights, feedback, metrics
from app.core.database import engine
from app.models import Base
app = FastAPI(
title="AI Insights Service",
version="1.0.0",
description="Centralized AI insights management"
)
# Include routers
app.include_router(insights.router, prefix="/api/v1/ai-insights", tags=["insights"])
app.include_router(feedback.router, prefix="/api/v1/ai-insights", tags=["feedback"])
app.include_router(metrics.router, prefix="/api/v1/ai-insights", tags=["metrics"])
@app.on_event("startup")
async def startup():
# Initialize database
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
@app.get("/health")
async def health_check():
return {"status": "healthy", "service": "ai-insights"}
```
#### Repository Pattern
```python
# app/repositories/insight_repository.py
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy import select, and_
from app.models import AIInsight
class InsightRepository:
def __init__(self, session: AsyncSession):
self.session = session
async def create(self, insight_data: dict) -> AIInsight:
insight = AIInsight(**insight_data)
self.session.add(insight)
await self.session.commit()
await self.session.refresh(insight)
return insight
async def get_by_id(self, tenant_id: str, insight_id: str) -> AIInsight:
query = select(AIInsight).where(
and_(
AIInsight.tenant_id == tenant_id,
AIInsight.id == insight_id,
AIInsight.deleted_at.is_(None)
)
)
result = await self.session.execute(query)
return result.scalar_one_or_none()
async def list_insights(
self,
tenant_id: str,
skip: int = 0,
limit: int = 100,
**filters
) -> tuple[list[AIInsight], int]:
# Build query with filters
query = select(AIInsight).where(
and_(
AIInsight.tenant_id == tenant_id,
AIInsight.deleted_at.is_(None)
)
)
# Apply filters
if priority := filters.get('priority'):
query = query.where(AIInsight.priority == priority)
if category := filters.get('category'):
query = query.where(AIInsight.category == category)
if min_confidence := filters.get('min_confidence'):
query = query.where(AIInsight.confidence >= min_confidence)
# Get total count
count_query = select(func.count()).select_from(query.subquery())
total = await self.session.execute(count_query)
total = total.scalar()
# Apply pagination
query = query.offset(skip).limit(limit)
result = await self.session.execute(query)
return result.scalars().all(), total
```
---
## Dynamic Rules Engine
The Dynamic Rules Engine adapts business rules based on historical patterns.
### Architecture
```
Historical Data
Pattern Detector (analyzes trends, seasonality, anomalies)
Rules Orchestrator (adapts thresholds and parameters)
Rule Evaluation (applies adapted rules to current data)
Insights Generated
```
### Rule Types
1. **Demand Threshold Rules**
- High demand alert: demand > adaptive_threshold
- Low demand alert: demand < adaptive_threshold
- Threshold adapts based on historical mean and variance
2. **Volatility Rules**
- Triggered when coefficient of variation > threshold
- Warns of unpredictable demand patterns
3. **Trend Rules**
- Upward trend: sustained increase over N periods
- Downward trend: sustained decrease over N periods
4. **Seasonal Rules**
- Detects recurring patterns (weekly, monthly)
- Adjusts baselines for seasonal effects
5. **Anomaly Rules**
- Statistical outliers (> 3 standard deviations)
- Sudden changes (> X% from baseline)
### Usage Example
```python
from app.ml.dynamic_rules_engine import DynamicRulesEngine
# Initialize engine
engine = DynamicRulesEngine(tenant_id=tenant_id)
# Train on historical data
historical_data = pd.DataFrame({
'date': [...],
'product_id': [...],
'quantity': [...]
})
engine.train(historical_data)
# Generate insights for current data
current_data = pd.DataFrame({
'product_id': ['croissant'],
'current_demand': [130],
'date': ['2025-11-03']
})
insights = await engine.generate_insights(current_data)
# Store insights in AI Insights Service
for insight in insights:
await insight_service.create_insight(tenant_id, insight)
```
### Configuration
```python
# services/forecasting/app/core/config.py
class RulesEngineSettings(BaseSettings):
# Thresholds
HIGH_DEMAND_THRESHOLD: float = 1.2 # 20% above baseline
LOW_DEMAND_THRESHOLD: float = 0.8 # 20% below baseline
VOLATILITY_THRESHOLD: float = 0.3 # CV > 30%
# Pattern detection
SEASONALITY_PERIODS: list[int] = [7, 30] # Weekly, monthly
TREND_WINDOW: int = 14 # Days to detect trends
ANOMALY_SIGMA: float = 3.0 # Standard deviations
# Adaptation
ADAPTATION_RATE: float = 0.1 # How quickly to adapt thresholds
MIN_SAMPLES: int = 30 # Minimum data points for adaptation
CONFIDENCE_DECAY: float = 0.95 # Confidence decay over time
```
---
## Configuration
### Environment Variables
```bash
# Database
AI_INSIGHTS_DATABASE_URL=postgresql+asyncpg://user:pass@host:5432/ai_insights_db
DATABASE_POOL_SIZE=20
DATABASE_MAX_OVERFLOW=10
# Redis
REDIS_URL=redis://redis:6379/0
# Service URLs
FORECASTING_SERVICE_URL=http://forecasting-service:8000
PRODUCTION_SERVICE_URL=http://production-service:8000
INVENTORY_SERVICE_URL=http://inventory-service:8000
PROCUREMENT_SERVICE_URL=http://procurement-service:8000
ORCHESTRATION_SERVICE_URL=http://orchestration-service:8000
# Authentication
JWT_SECRET_KEY=your-secret-key-here
JWT_ALGORITHM=HS256
JWT_EXPIRATION_MINUTES=60
# Logging
LOG_LEVEL=INFO
LOG_FORMAT=json
# ML Configuration
MIN_CONFIDENCE_THRESHOLD=70
RETRAINING_ACCURACY_THRESHOLD=0.75
FEEDBACK_SAMPLE_SIZE=100
```
### Kubernetes ConfigMap
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: ai-insights-config
namespace: bakery-ia
data:
LOG_LEVEL: "INFO"
MIN_CONFIDENCE_THRESHOLD: "70"
FORECASTING_SERVICE_URL: "http://forecasting-service:8000"
PRODUCTION_SERVICE_URL: "http://production-service:8000"
INVENTORY_SERVICE_URL: "http://inventory-service:8000"
```
### Kubernetes Secrets
```yaml
apiVersion: v1
kind: Secret
metadata:
name: database-secrets
namespace: bakery-ia
type: Opaque
stringData:
AI_INSIGHTS_DATABASE_URL: "postgresql+asyncpg://user:pass@postgresql-ai-insights:5432/ai_insights_db"
REDIS_URL: "redis://redis:6379/0"
JWT_SECRET_KEY: "your-secure-secret-key"
```
---
## Troubleshooting
### Common Issues
#### 1. Service Not Starting
```bash
# Check pod logs
kubectl logs -n bakery-ia deployment/ai-insights-service --tail=100
# Check pod events
kubectl describe pod -n bakery-ia <pod-name>
# Common causes:
# - Database connection failure
# - Missing environment variables
# - Port conflicts
```
#### 2. Database Connection Errors
```bash
# Test database connectivity
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- \
python -c "from app.core.database import engine; import asyncio; asyncio.run(engine.connect())"
# Check database pod status
kubectl get pods -n bakery-ia -l app=postgresql-ai-insights
# Verify database URL
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
env | grep DATABASE_URL
```
#### 3. High Memory Usage
```bash
# Check resource usage
kubectl top pods -n bakery-ia
# Increase limits
kubectl set resources deployment/ai-insights-service \
--limits=memory=2Gi \
-n bakery-ia
# Enable query result streaming for large datasets
# (already implemented in repository pattern)
```
#### 4. Slow API Responses
```bash
# Check database query performance
kubectl exec -it -n bakery-ia postgresql-ai-insights-0 -- \
psql -U postgres -d ai_insights_db -c "
SELECT query, calls, mean_exec_time, total_exec_time
FROM pg_stat_statements
ORDER BY total_exec_time DESC
LIMIT 10;
"
# Add missing indexes if needed
# Check slow query log
kubectl logs -n bakery-ia -l app=postgresql-ai-insights | grep "duration"
```
#### 5. Insight Creation Failures
```bash
# Check validation errors
kubectl logs -n bakery-ia deployment/ai-insights-service | grep -i error
# Common issues:
# - Invalid confidence score (must be 0-100)
# - Missing required fields
# - Invalid tenant ID
# - Database constraint violations
```
### Debugging Commands
```bash
# Interactive shell in pod
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- /bin/bash
# Python REPL with app context
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- \
python -c "from app.core.database import engine; import asyncio; # your code"
# Check API health
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
curl http://localhost:8000/health
# View recent logs with timestamps
kubectl logs -n bakery-ia deployment/ai-insights-service \
--since=1h \
--timestamps
# Follow logs in real-time
kubectl logs -n bakery-ia deployment/ai-insights-service -f
```
---
## Performance Optimization
### Database Optimization
```sql
-- Create covering indexes
CREATE INDEX idx_insights_tenant_priority_confidence
ON ai_insights(tenant_id, priority, confidence)
WHERE deleted_at IS NULL;
-- Vacuum regularly
VACUUM ANALYZE ai_insights;
-- Check index usage
SELECT schemaname, tablename, indexname, idx_scan
FROM pg_stat_user_indexes
WHERE schemaname = 'public'
ORDER BY idx_scan;
```
### Redis Caching
```python
# Cache frequently accessed insights
from app.core.cache import redis_client
async def get_insight_cached(tenant_id: str, insight_id: str):
# Check cache
cache_key = f"insight:{tenant_id}:{insight_id}"
cached = await redis_client.get(cache_key)
if cached:
return json.loads(cached)
# Fetch from database
insight = await repository.get_by_id(tenant_id, insight_id)
# Cache for 5 minutes
await redis_client.setex(
cache_key,
300,
json.dumps(insight.dict())
)
return insight
```
### Batch Operations
```python
# Bulk insert insights
async def create_insights_batch(tenant_id: str, insights_data: list[dict]):
async with session.begin():
insights = [AIInsight(**data) for data in insights_data]
session.add_all(insights)
await session.flush()
return insights
```
---
## Monitoring and Observability
### Health Checks
```python
@app.get("/health")
async def health_check():
return {
"status": "healthy",
"service": "ai-insights",
"version": "1.0.0",
"timestamp": datetime.utcnow().isoformat()
}
@app.get("/health/detailed")
async def detailed_health_check():
# Check database
try:
async with engine.connect() as conn:
await conn.execute(text("SELECT 1"))
db_status = "healthy"
except Exception as e:
db_status = f"unhealthy: {str(e)}"
# Check Redis
try:
await redis_client.ping()
redis_status = "healthy"
except Exception as e:
redis_status = f"unhealthy: {str(e)}"
return {
"status": "healthy" if db_status == "healthy" and redis_status == "healthy" else "unhealthy",
"components": {
"database": db_status,
"redis": redis_status
}
}
```
### Metrics Endpoint
```python
from prometheus_client import Counter, Histogram, generate_latest
insight_created = Counter('insights_created_total', 'Total insights created')
insight_applied = Counter('insights_applied_total', 'Total insights applied')
api_latency = Histogram('api_request_duration_seconds', 'API request latency')
@app.get("/metrics")
async def metrics():
return Response(generate_latest(), media_type="text/plain")
```
---
*For comprehensive testing procedures, validation steps, and test cases, refer to TESTING_GUIDE.md.*