1019 lines
24 KiB
Markdown
1019 lines
24 KiB
Markdown
|
|
# Technical Documentation - Bakery IA AI Insights Platform
|
||
|
|
|
||
|
|
## Table of Contents
|
||
|
|
|
||
|
|
1. [API Reference](#api-reference)
|
||
|
|
2. [Deployment Guide](#deployment-guide)
|
||
|
|
3. [Implementation Details](#implementation-details)
|
||
|
|
4. [Dynamic Rules Engine](#dynamic-rules-engine)
|
||
|
|
5. [Database Management](#database-management)
|
||
|
|
6. [Configuration](#configuration)
|
||
|
|
7. [Troubleshooting](#troubleshooting)
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## API Reference
|
||
|
|
|
||
|
|
### Base URL
|
||
|
|
|
||
|
|
```
|
||
|
|
http://ai-insights-service:8000/api/v1/ai-insights
|
||
|
|
```
|
||
|
|
|
||
|
|
### Authentication
|
||
|
|
|
||
|
|
All endpoints require either:
|
||
|
|
- JWT token in `Authorization: Bearer <token>` header
|
||
|
|
- Service token in `X-Service-Token` header
|
||
|
|
- Demo session ID in `X-Demo-Session-Id` header
|
||
|
|
|
||
|
|
### Tenant Context
|
||
|
|
|
||
|
|
All endpoints include tenant ID in the path:
|
||
|
|
```
|
||
|
|
/api/v1/ai-insights/tenants/{tenant_id}/...
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
### Insights Endpoints
|
||
|
|
|
||
|
|
#### Create Insight
|
||
|
|
|
||
|
|
**POST** `/tenants/{tenant_id}/insights`
|
||
|
|
|
||
|
|
Creates a new AI insight.
|
||
|
|
|
||
|
|
**Request Body:**
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"type": "prediction", // required: prediction, recommendation, alert, optimization
|
||
|
|
"priority": "high", // required: critical, high, medium, low
|
||
|
|
"category": "forecasting", // required: forecasting, inventory, production, procurement, etc.
|
||
|
|
"title": "Weekend Demand Surge", // required: max 255 chars
|
||
|
|
"description": "Detailed explanation...", // optional: text
|
||
|
|
"confidence": 87, // required: 0-100
|
||
|
|
"metrics_json": { // optional: JSONB object
|
||
|
|
"product_id": "croissant",
|
||
|
|
"predicted_demand": 130,
|
||
|
|
"increase_percentage": 30
|
||
|
|
},
|
||
|
|
"impact_type": "revenue_increase", // optional: revenue_increase, cost_reduction, etc.
|
||
|
|
"impact_value": 450.00, // optional: decimal
|
||
|
|
"impact_unit": "euros", // optional: string
|
||
|
|
"actionable": true, // optional: boolean, default true
|
||
|
|
"recommendation_actions": [ // optional: array of actions
|
||
|
|
{
|
||
|
|
"service": "production",
|
||
|
|
"action": "increase_production",
|
||
|
|
"parameters": "{\"product_id\": \"croissant\", \"quantity\": 30}"
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"source_service": "forecasting", // required: originating service
|
||
|
|
"source_data_id": "forecast_001", // optional: reference ID
|
||
|
|
"valid_from": "2025-11-03T00:00:00Z", // optional: ISO 8601
|
||
|
|
"valid_until": "2025-11-05T23:59:59Z" // optional: ISO 8601
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Response:** `201 Created`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"id": "uuid",
|
||
|
|
"tenant_id": "uuid",
|
||
|
|
"type": "prediction",
|
||
|
|
"priority": "high",
|
||
|
|
"category": "forecasting",
|
||
|
|
"title": "Weekend Demand Surge",
|
||
|
|
"description": "Detailed explanation...",
|
||
|
|
"confidence": 87,
|
||
|
|
"metrics_json": {...},
|
||
|
|
"impact_type": "revenue_increase",
|
||
|
|
"impact_value": 450.00,
|
||
|
|
"impact_unit": "euros",
|
||
|
|
"status": "new",
|
||
|
|
"actionable": true,
|
||
|
|
"recommendation_actions": [...],
|
||
|
|
"source_service": "forecasting",
|
||
|
|
"source_data_id": "forecast_001",
|
||
|
|
"valid_from": "2025-11-03T00:00:00Z",
|
||
|
|
"valid_until": "2025-11-05T23:59:59Z",
|
||
|
|
"created_at": "2025-11-03T10:30:00Z",
|
||
|
|
"updated_at": "2025-11-03T10:30:00Z"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
#### List Insights
|
||
|
|
|
||
|
|
**GET** `/tenants/{tenant_id}/insights`
|
||
|
|
|
||
|
|
Retrieves paginated list of insights with optional filters.
|
||
|
|
|
||
|
|
**Query Parameters:**
|
||
|
|
- `skip` (int, default=0): Pagination offset
|
||
|
|
- `limit` (int, default=100, max=1000): Results per page
|
||
|
|
- `priority` (string): Filter by priority (critical, high, medium, low)
|
||
|
|
- `category` (string): Filter by category
|
||
|
|
- `status` (string): Filter by status (new, acknowledged, in_progress, applied, dismissed)
|
||
|
|
- `actionable_only` (boolean): Only actionable insights
|
||
|
|
- `min_confidence` (int, 0-100): Minimum confidence score
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"items": [
|
||
|
|
{
|
||
|
|
"id": "uuid",
|
||
|
|
"title": "...",
|
||
|
|
// ... full insight object
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"total": 42,
|
||
|
|
"skip": 0,
|
||
|
|
"limit": 100
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
#### Get Single Insight
|
||
|
|
|
||
|
|
**GET** `/tenants/{tenant_id}/insights/{insight_id}`
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"id": "uuid",
|
||
|
|
// ... full insight object
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Errors:**
|
||
|
|
- `404 Not Found`: Insight doesn't exist
|
||
|
|
- `403 Forbidden`: Tenant mismatch
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
#### Update Insight
|
||
|
|
|
||
|
|
**PATCH** `/tenants/{tenant_id}/insights/{insight_id}`
|
||
|
|
|
||
|
|
Updates specific fields of an insight.
|
||
|
|
|
||
|
|
**Request Body:**
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"status": "acknowledged", // new, acknowledged, in_progress, applied, dismissed
|
||
|
|
"priority": "critical", // optional: upgrade/downgrade priority
|
||
|
|
"notes": "Additional info" // optional: any field that's updatable
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
// updated insight object
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
#### Delete Insight (Soft Delete)
|
||
|
|
|
||
|
|
**DELETE** `/tenants/{tenant_id}/insights/{insight_id}`
|
||
|
|
|
||
|
|
Marks insight as deleted (soft delete).
|
||
|
|
|
||
|
|
**Response:** `204 No Content`
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
#### Get Orchestration-Ready Insights
|
||
|
|
|
||
|
|
**GET** `/tenants/{tenant_id}/insights/orchestration-ready`
|
||
|
|
|
||
|
|
Retrieves insights grouped by category, ready for orchestration.
|
||
|
|
|
||
|
|
**Query Parameters:**
|
||
|
|
- `target_date` (ISO 8601): Target execution date
|
||
|
|
- `min_confidence` (int, default=70): Minimum confidence threshold
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"forecast_adjustments": [
|
||
|
|
{
|
||
|
|
"id": "uuid",
|
||
|
|
"title": "...",
|
||
|
|
"confidence": 87,
|
||
|
|
"recommendation_actions": [...]
|
||
|
|
}
|
||
|
|
],
|
||
|
|
"procurement_recommendations": [...],
|
||
|
|
"production_optimizations": [...],
|
||
|
|
"supplier_alerts": [...],
|
||
|
|
"price_opportunities": [...]
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
### Feedback Endpoints
|
||
|
|
|
||
|
|
#### Record Feedback
|
||
|
|
|
||
|
|
**POST** `/tenants/{tenant_id}/insights/{insight_id}/feedback`
|
||
|
|
|
||
|
|
Records actual outcome and compares with prediction.
|
||
|
|
|
||
|
|
**Request Body:**
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"action_taken": "increased_production",
|
||
|
|
"success": true,
|
||
|
|
"result_data": {
|
||
|
|
"planned_increase": 30,
|
||
|
|
"actual_increase": 28,
|
||
|
|
"revenue_impact": 420.00
|
||
|
|
},
|
||
|
|
"expected_impact_value": 450.00,
|
||
|
|
"actual_impact_value": 420.00,
|
||
|
|
"variance_percentage": -6.67,
|
||
|
|
"accuracy_score": 93.3,
|
||
|
|
"notes": "Slightly lower than predicted due to supply constraints"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"id": "uuid",
|
||
|
|
"insight_id": "uuid",
|
||
|
|
"action_taken": "increased_production",
|
||
|
|
"success": true,
|
||
|
|
"result_data": {...},
|
||
|
|
"expected_impact_value": 450.00,
|
||
|
|
"actual_impact_value": 420.00,
|
||
|
|
"variance_percentage": -6.67,
|
||
|
|
"accuracy_score": 93.3,
|
||
|
|
"notes": "...",
|
||
|
|
"created_at": "2025-11-03T18:00:00Z"
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
**Side Effects:**
|
||
|
|
- Automatically updates insight status to "applied"
|
||
|
|
- Triggers FeedbackLearningSystem analysis
|
||
|
|
- May trigger model retraining if performance degrades
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
### Metrics Endpoints
|
||
|
|
|
||
|
|
#### Get Summary Metrics
|
||
|
|
|
||
|
|
**GET** `/tenants/{tenant_id}/insights/metrics/summary`
|
||
|
|
|
||
|
|
Retrieves aggregate metrics for all insights.
|
||
|
|
|
||
|
|
**Response:** `200 OK`
|
||
|
|
```json
|
||
|
|
{
|
||
|
|
"total_insights": 147,
|
||
|
|
"actionable_insights": 98,
|
||
|
|
"average_confidence": 82.5,
|
||
|
|
"critical_priority_count": 12,
|
||
|
|
"high_priority_count": 45,
|
||
|
|
"medium_priority_count": 67,
|
||
|
|
"low_priority_count": 23,
|
||
|
|
"by_category": {
|
||
|
|
"forecasting": 42,
|
||
|
|
"inventory": 35,
|
||
|
|
"production": 28,
|
||
|
|
"procurement": 22,
|
||
|
|
"customer": 20
|
||
|
|
},
|
||
|
|
"by_status": {
|
||
|
|
"new": 56,
|
||
|
|
"acknowledged": 28,
|
||
|
|
"in_progress": 15,
|
||
|
|
"applied": 42,
|
||
|
|
"dismissed": 6
|
||
|
|
}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Deployment Guide
|
||
|
|
|
||
|
|
### Prerequisites
|
||
|
|
|
||
|
|
- **Kubernetes Cluster:** 1.24+
|
||
|
|
- **Docker:** 20.10+
|
||
|
|
- **Kind:** 0.20+ (for local development)
|
||
|
|
- **kubectl:** 1.24+
|
||
|
|
- **Tilt:** 0.30+ (optional, for development)
|
||
|
|
|
||
|
|
### Local Development Setup
|
||
|
|
|
||
|
|
#### 1. Start Kubernetes Cluster
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Create Kind cluster
|
||
|
|
kind create cluster --name bakery-ia-local --config infrastructure/kind-config.yaml
|
||
|
|
|
||
|
|
# Verify cluster
|
||
|
|
kubectl cluster-info
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 2. Deploy Infrastructure
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Create namespace
|
||
|
|
kubectl create namespace bakery-ia
|
||
|
|
|
||
|
|
# Deploy databases
|
||
|
|
kubectl apply -f infrastructure/kubernetes/base/components/databases/
|
||
|
|
|
||
|
|
# Wait for databases to be ready
|
||
|
|
kubectl wait --for=condition=ready pod -l app=postgresql-main -n bakery-ia --timeout=300s
|
||
|
|
kubectl wait --for=condition=ready pod -l app=postgresql-ai-insights -n bakery-ia --timeout=300s
|
||
|
|
kubectl wait --for=condition=ready pod -l app=redis -n bakery-ia --timeout=300s
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 3. Deploy Services
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Deploy all services
|
||
|
|
kubectl apply -f infrastructure/kubernetes/base/
|
||
|
|
|
||
|
|
# Watch deployment
|
||
|
|
kubectl get pods -n bakery-ia -w
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 4. Run Database Migrations
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# AI Insights Service migration
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -m alembic upgrade head
|
||
|
|
|
||
|
|
# Other services...
|
||
|
|
for service in orders inventory production suppliers; do
|
||
|
|
kubectl exec -n bakery-ia deployment/${service}-service -- \
|
||
|
|
python -m alembic upgrade head
|
||
|
|
done
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 5. Verify Deployment
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check all pods are running
|
||
|
|
kubectl get pods -n bakery-ia
|
||
|
|
|
||
|
|
# Check services
|
||
|
|
kubectl get svc -n bakery-ia
|
||
|
|
|
||
|
|
# Test AI Insights Service health
|
||
|
|
kubectl port-forward -n bakery-ia svc/ai-insights-service 8000:8000 &
|
||
|
|
curl http://localhost:8000/health
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
### Production Deployment
|
||
|
|
|
||
|
|
#### Environment Configuration
|
||
|
|
|
||
|
|
Create environment-specific configurations:
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
# infrastructure/kubernetes/overlays/production/kustomization.yaml
|
||
|
|
apiVersion: kustomize.config.k8s.io/v1beta1
|
||
|
|
kind: Kustomization
|
||
|
|
|
||
|
|
resources:
|
||
|
|
- ../../base
|
||
|
|
|
||
|
|
replicas:
|
||
|
|
- name: ai-insights-service
|
||
|
|
count: 3
|
||
|
|
- name: orchestration-service
|
||
|
|
count: 2
|
||
|
|
|
||
|
|
configMapGenerator:
|
||
|
|
- name: ai-insights-config
|
||
|
|
env: production.env
|
||
|
|
|
||
|
|
secretGenerator:
|
||
|
|
- name: database-secrets
|
||
|
|
envs:
|
||
|
|
- secrets.env
|
||
|
|
|
||
|
|
images:
|
||
|
|
- name: bakery/ai-insights-service
|
||
|
|
newTag: v1.0.0
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Deploy to Production
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Apply with kustomize
|
||
|
|
kubectl apply -k infrastructure/kubernetes/overlays/production/
|
||
|
|
|
||
|
|
# Rolling update
|
||
|
|
kubectl set image deployment/ai-insights-service \
|
||
|
|
ai-insights-service=bakery/ai-insights-service:v1.0.1 \
|
||
|
|
-n bakery-ia
|
||
|
|
|
||
|
|
# Monitor rollout
|
||
|
|
kubectl rollout status deployment/ai-insights-service -n bakery-ia
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
### Database Management
|
||
|
|
|
||
|
|
#### Create AI Insights Database
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Connect to PostgreSQL
|
||
|
|
kubectl exec -it -n bakery-ia postgresql-ai-insights-0 -- psql -U postgres
|
||
|
|
|
||
|
|
# Create database
|
||
|
|
CREATE DATABASE ai_insights_db;
|
||
|
|
|
||
|
|
# Create user
|
||
|
|
CREATE USER ai_insights_user WITH PASSWORD 'secure_password';
|
||
|
|
GRANT ALL PRIVILEGES ON DATABASE ai_insights_db TO ai_insights_user;
|
||
|
|
|
||
|
|
# Enable UUID extension
|
||
|
|
\c ai_insights_db
|
||
|
|
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Run Migrations
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check current version
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -m alembic current
|
||
|
|
|
||
|
|
# Upgrade to latest
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -m alembic upgrade head
|
||
|
|
|
||
|
|
# Downgrade one version
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -m alembic downgrade -1
|
||
|
|
|
||
|
|
# Show migration history
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -m alembic history
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Backup and Restore
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Backup
|
||
|
|
kubectl exec -n bakery-ia postgresql-ai-insights-0 -- \
|
||
|
|
pg_dump -U postgres ai_insights_db > backup-$(date +%Y%m%d).sql
|
||
|
|
|
||
|
|
# Restore
|
||
|
|
kubectl exec -i -n bakery-ia postgresql-ai-insights-0 -- \
|
||
|
|
psql -U postgres ai_insights_db < backup-20251103.sql
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Implementation Details
|
||
|
|
|
||
|
|
### Service Structure
|
||
|
|
|
||
|
|
```
|
||
|
|
services/ai_insights/
|
||
|
|
├── app/
|
||
|
|
│ ├── __init__.py
|
||
|
|
│ ├── main.py # FastAPI application
|
||
|
|
│ ├── core/
|
||
|
|
│ │ ├── config.py # Configuration
|
||
|
|
│ │ ├── database.py # Database connection
|
||
|
|
│ │ └── security.py # Auth utilities
|
||
|
|
│ ├── models/
|
||
|
|
│ │ ├── ai_insight.py # SQLAlchemy models
|
||
|
|
│ │ └── feedback.py
|
||
|
|
│ ├── schemas/
|
||
|
|
│ │ ├── insight.py # Pydantic schemas
|
||
|
|
│ │ └── feedback.py
|
||
|
|
│ ├── api/
|
||
|
|
│ │ ├── insights.py # Insight endpoints
|
||
|
|
│ │ ├── feedback.py # Feedback endpoints
|
||
|
|
│ │ └── metrics.py # Metrics endpoints
|
||
|
|
│ ├── services/
|
||
|
|
│ │ ├── insight_service.py # Business logic
|
||
|
|
│ │ └── feedback_service.py
|
||
|
|
│ ├── repositories/
|
||
|
|
│ │ ├── insight_repository.py # Data access
|
||
|
|
│ │ └── feedback_repository.py
|
||
|
|
│ └── ml/
|
||
|
|
│ └── feedback_learning_system.py # Learning system
|
||
|
|
├── tests/
|
||
|
|
│ ├── unit/
|
||
|
|
│ ├── integration/
|
||
|
|
│ └── conftest.py
|
||
|
|
├── migrations/
|
||
|
|
│ └── versions/ # Alembic migrations
|
||
|
|
├── Dockerfile
|
||
|
|
├── requirements.txt
|
||
|
|
└── alembic.ini
|
||
|
|
```
|
||
|
|
|
||
|
|
### Key Components
|
||
|
|
|
||
|
|
#### FastAPI Application
|
||
|
|
|
||
|
|
```python
|
||
|
|
# app/main.py
|
||
|
|
from fastapi import FastAPI
|
||
|
|
from app.api import insights, feedback, metrics
|
||
|
|
from app.core.database import engine
|
||
|
|
from app.models import Base
|
||
|
|
|
||
|
|
app = FastAPI(
|
||
|
|
title="AI Insights Service",
|
||
|
|
version="1.0.0",
|
||
|
|
description="Centralized AI insights management"
|
||
|
|
)
|
||
|
|
|
||
|
|
# Include routers
|
||
|
|
app.include_router(insights.router, prefix="/api/v1/ai-insights", tags=["insights"])
|
||
|
|
app.include_router(feedback.router, prefix="/api/v1/ai-insights", tags=["feedback"])
|
||
|
|
app.include_router(metrics.router, prefix="/api/v1/ai-insights", tags=["metrics"])
|
||
|
|
|
||
|
|
@app.on_event("startup")
|
||
|
|
async def startup():
|
||
|
|
# Initialize database
|
||
|
|
async with engine.begin() as conn:
|
||
|
|
await conn.run_sync(Base.metadata.create_all)
|
||
|
|
|
||
|
|
@app.get("/health")
|
||
|
|
async def health_check():
|
||
|
|
return {"status": "healthy", "service": "ai-insights"}
|
||
|
|
```
|
||
|
|
|
||
|
|
#### Repository Pattern
|
||
|
|
|
||
|
|
```python
|
||
|
|
# app/repositories/insight_repository.py
|
||
|
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||
|
|
from sqlalchemy import select, and_
|
||
|
|
from app.models import AIInsight
|
||
|
|
|
||
|
|
class InsightRepository:
|
||
|
|
def __init__(self, session: AsyncSession):
|
||
|
|
self.session = session
|
||
|
|
|
||
|
|
async def create(self, insight_data: dict) -> AIInsight:
|
||
|
|
insight = AIInsight(**insight_data)
|
||
|
|
self.session.add(insight)
|
||
|
|
await self.session.commit()
|
||
|
|
await self.session.refresh(insight)
|
||
|
|
return insight
|
||
|
|
|
||
|
|
async def get_by_id(self, tenant_id: str, insight_id: str) -> AIInsight:
|
||
|
|
query = select(AIInsight).where(
|
||
|
|
and_(
|
||
|
|
AIInsight.tenant_id == tenant_id,
|
||
|
|
AIInsight.id == insight_id,
|
||
|
|
AIInsight.deleted_at.is_(None)
|
||
|
|
)
|
||
|
|
)
|
||
|
|
result = await self.session.execute(query)
|
||
|
|
return result.scalar_one_or_none()
|
||
|
|
|
||
|
|
async def list_insights(
|
||
|
|
self,
|
||
|
|
tenant_id: str,
|
||
|
|
skip: int = 0,
|
||
|
|
limit: int = 100,
|
||
|
|
**filters
|
||
|
|
) -> tuple[list[AIInsight], int]:
|
||
|
|
# Build query with filters
|
||
|
|
query = select(AIInsight).where(
|
||
|
|
and_(
|
||
|
|
AIInsight.tenant_id == tenant_id,
|
||
|
|
AIInsight.deleted_at.is_(None)
|
||
|
|
)
|
||
|
|
)
|
||
|
|
|
||
|
|
# Apply filters
|
||
|
|
if priority := filters.get('priority'):
|
||
|
|
query = query.where(AIInsight.priority == priority)
|
||
|
|
if category := filters.get('category'):
|
||
|
|
query = query.where(AIInsight.category == category)
|
||
|
|
if min_confidence := filters.get('min_confidence'):
|
||
|
|
query = query.where(AIInsight.confidence >= min_confidence)
|
||
|
|
|
||
|
|
# Get total count
|
||
|
|
count_query = select(func.count()).select_from(query.subquery())
|
||
|
|
total = await self.session.execute(count_query)
|
||
|
|
total = total.scalar()
|
||
|
|
|
||
|
|
# Apply pagination
|
||
|
|
query = query.offset(skip).limit(limit)
|
||
|
|
result = await self.session.execute(query)
|
||
|
|
|
||
|
|
return result.scalars().all(), total
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Dynamic Rules Engine
|
||
|
|
|
||
|
|
The Dynamic Rules Engine adapts business rules based on historical patterns.
|
||
|
|
|
||
|
|
### Architecture
|
||
|
|
|
||
|
|
```
|
||
|
|
Historical Data
|
||
|
|
↓
|
||
|
|
Pattern Detector (analyzes trends, seasonality, anomalies)
|
||
|
|
↓
|
||
|
|
Rules Orchestrator (adapts thresholds and parameters)
|
||
|
|
↓
|
||
|
|
Rule Evaluation (applies adapted rules to current data)
|
||
|
|
↓
|
||
|
|
Insights Generated
|
||
|
|
```
|
||
|
|
|
||
|
|
### Rule Types
|
||
|
|
|
||
|
|
1. **Demand Threshold Rules**
|
||
|
|
- High demand alert: demand > adaptive_threshold
|
||
|
|
- Low demand alert: demand < adaptive_threshold
|
||
|
|
- Threshold adapts based on historical mean and variance
|
||
|
|
|
||
|
|
2. **Volatility Rules**
|
||
|
|
- Triggered when coefficient of variation > threshold
|
||
|
|
- Warns of unpredictable demand patterns
|
||
|
|
|
||
|
|
3. **Trend Rules**
|
||
|
|
- Upward trend: sustained increase over N periods
|
||
|
|
- Downward trend: sustained decrease over N periods
|
||
|
|
|
||
|
|
4. **Seasonal Rules**
|
||
|
|
- Detects recurring patterns (weekly, monthly)
|
||
|
|
- Adjusts baselines for seasonal effects
|
||
|
|
|
||
|
|
5. **Anomaly Rules**
|
||
|
|
- Statistical outliers (> 3 standard deviations)
|
||
|
|
- Sudden changes (> X% from baseline)
|
||
|
|
|
||
|
|
### Usage Example
|
||
|
|
|
||
|
|
```python
|
||
|
|
from app.ml.dynamic_rules_engine import DynamicRulesEngine
|
||
|
|
|
||
|
|
# Initialize engine
|
||
|
|
engine = DynamicRulesEngine(tenant_id=tenant_id)
|
||
|
|
|
||
|
|
# Train on historical data
|
||
|
|
historical_data = pd.DataFrame({
|
||
|
|
'date': [...],
|
||
|
|
'product_id': [...],
|
||
|
|
'quantity': [...]
|
||
|
|
})
|
||
|
|
|
||
|
|
engine.train(historical_data)
|
||
|
|
|
||
|
|
# Generate insights for current data
|
||
|
|
current_data = pd.DataFrame({
|
||
|
|
'product_id': ['croissant'],
|
||
|
|
'current_demand': [130],
|
||
|
|
'date': ['2025-11-03']
|
||
|
|
})
|
||
|
|
|
||
|
|
insights = await engine.generate_insights(current_data)
|
||
|
|
|
||
|
|
# Store insights in AI Insights Service
|
||
|
|
for insight in insights:
|
||
|
|
await insight_service.create_insight(tenant_id, insight)
|
||
|
|
```
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
|
||
|
|
```python
|
||
|
|
# services/forecasting/app/core/config.py
|
||
|
|
class RulesEngineSettings(BaseSettings):
|
||
|
|
# Thresholds
|
||
|
|
HIGH_DEMAND_THRESHOLD: float = 1.2 # 20% above baseline
|
||
|
|
LOW_DEMAND_THRESHOLD: float = 0.8 # 20% below baseline
|
||
|
|
VOLATILITY_THRESHOLD: float = 0.3 # CV > 30%
|
||
|
|
|
||
|
|
# Pattern detection
|
||
|
|
SEASONALITY_PERIODS: list[int] = [7, 30] # Weekly, monthly
|
||
|
|
TREND_WINDOW: int = 14 # Days to detect trends
|
||
|
|
ANOMALY_SIGMA: float = 3.0 # Standard deviations
|
||
|
|
|
||
|
|
# Adaptation
|
||
|
|
ADAPTATION_RATE: float = 0.1 # How quickly to adapt thresholds
|
||
|
|
MIN_SAMPLES: int = 30 # Minimum data points for adaptation
|
||
|
|
CONFIDENCE_DECAY: float = 0.95 # Confidence decay over time
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
### Environment Variables
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Database
|
||
|
|
AI_INSIGHTS_DATABASE_URL=postgresql+asyncpg://user:pass@host:5432/ai_insights_db
|
||
|
|
DATABASE_POOL_SIZE=20
|
||
|
|
DATABASE_MAX_OVERFLOW=10
|
||
|
|
|
||
|
|
# Redis
|
||
|
|
REDIS_URL=redis://redis:6379/0
|
||
|
|
|
||
|
|
# Service URLs
|
||
|
|
FORECASTING_SERVICE_URL=http://forecasting-service:8000
|
||
|
|
PRODUCTION_SERVICE_URL=http://production-service:8000
|
||
|
|
INVENTORY_SERVICE_URL=http://inventory-service:8000
|
||
|
|
PROCUREMENT_SERVICE_URL=http://procurement-service:8000
|
||
|
|
ORCHESTRATION_SERVICE_URL=http://orchestration-service:8000
|
||
|
|
|
||
|
|
# Authentication
|
||
|
|
JWT_SECRET_KEY=your-secret-key-here
|
||
|
|
JWT_ALGORITHM=HS256
|
||
|
|
JWT_EXPIRATION_MINUTES=60
|
||
|
|
|
||
|
|
# Logging
|
||
|
|
LOG_LEVEL=INFO
|
||
|
|
LOG_FORMAT=json
|
||
|
|
|
||
|
|
# ML Configuration
|
||
|
|
MIN_CONFIDENCE_THRESHOLD=70
|
||
|
|
RETRAINING_ACCURACY_THRESHOLD=0.75
|
||
|
|
FEEDBACK_SAMPLE_SIZE=100
|
||
|
|
```
|
||
|
|
|
||
|
|
### Kubernetes ConfigMap
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
apiVersion: v1
|
||
|
|
kind: ConfigMap
|
||
|
|
metadata:
|
||
|
|
name: ai-insights-config
|
||
|
|
namespace: bakery-ia
|
||
|
|
data:
|
||
|
|
LOG_LEVEL: "INFO"
|
||
|
|
MIN_CONFIDENCE_THRESHOLD: "70"
|
||
|
|
FORECASTING_SERVICE_URL: "http://forecasting-service:8000"
|
||
|
|
PRODUCTION_SERVICE_URL: "http://production-service:8000"
|
||
|
|
INVENTORY_SERVICE_URL: "http://inventory-service:8000"
|
||
|
|
```
|
||
|
|
|
||
|
|
### Kubernetes Secrets
|
||
|
|
|
||
|
|
```yaml
|
||
|
|
apiVersion: v1
|
||
|
|
kind: Secret
|
||
|
|
metadata:
|
||
|
|
name: database-secrets
|
||
|
|
namespace: bakery-ia
|
||
|
|
type: Opaque
|
||
|
|
stringData:
|
||
|
|
AI_INSIGHTS_DATABASE_URL: "postgresql+asyncpg://user:pass@postgresql-ai-insights:5432/ai_insights_db"
|
||
|
|
REDIS_URL: "redis://redis:6379/0"
|
||
|
|
JWT_SECRET_KEY: "your-secure-secret-key"
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Troubleshooting
|
||
|
|
|
||
|
|
### Common Issues
|
||
|
|
|
||
|
|
#### 1. Service Not Starting
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check pod logs
|
||
|
|
kubectl logs -n bakery-ia deployment/ai-insights-service --tail=100
|
||
|
|
|
||
|
|
# Check pod events
|
||
|
|
kubectl describe pod -n bakery-ia <pod-name>
|
||
|
|
|
||
|
|
# Common causes:
|
||
|
|
# - Database connection failure
|
||
|
|
# - Missing environment variables
|
||
|
|
# - Port conflicts
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 2. Database Connection Errors
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Test database connectivity
|
||
|
|
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -c "from app.core.database import engine; import asyncio; asyncio.run(engine.connect())"
|
||
|
|
|
||
|
|
# Check database pod status
|
||
|
|
kubectl get pods -n bakery-ia -l app=postgresql-ai-insights
|
||
|
|
|
||
|
|
# Verify database URL
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
env | grep DATABASE_URL
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 3. High Memory Usage
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check resource usage
|
||
|
|
kubectl top pods -n bakery-ia
|
||
|
|
|
||
|
|
# Increase limits
|
||
|
|
kubectl set resources deployment/ai-insights-service \
|
||
|
|
--limits=memory=2Gi \
|
||
|
|
-n bakery-ia
|
||
|
|
|
||
|
|
# Enable query result streaming for large datasets
|
||
|
|
# (already implemented in repository pattern)
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 4. Slow API Responses
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check database query performance
|
||
|
|
kubectl exec -it -n bakery-ia postgresql-ai-insights-0 -- \
|
||
|
|
psql -U postgres -d ai_insights_db -c "
|
||
|
|
SELECT query, calls, mean_exec_time, total_exec_time
|
||
|
|
FROM pg_stat_statements
|
||
|
|
ORDER BY total_exec_time DESC
|
||
|
|
LIMIT 10;
|
||
|
|
"
|
||
|
|
|
||
|
|
# Add missing indexes if needed
|
||
|
|
# Check slow query log
|
||
|
|
kubectl logs -n bakery-ia -l app=postgresql-ai-insights | grep "duration"
|
||
|
|
```
|
||
|
|
|
||
|
|
#### 5. Insight Creation Failures
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Check validation errors
|
||
|
|
kubectl logs -n bakery-ia deployment/ai-insights-service | grep -i error
|
||
|
|
|
||
|
|
# Common issues:
|
||
|
|
# - Invalid confidence score (must be 0-100)
|
||
|
|
# - Missing required fields
|
||
|
|
# - Invalid tenant ID
|
||
|
|
# - Database constraint violations
|
||
|
|
```
|
||
|
|
|
||
|
|
### Debugging Commands
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Interactive shell in pod
|
||
|
|
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- /bin/bash
|
||
|
|
|
||
|
|
# Python REPL with app context
|
||
|
|
kubectl exec -it -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
python -c "from app.core.database import engine; import asyncio; # your code"
|
||
|
|
|
||
|
|
# Check API health
|
||
|
|
kubectl exec -n bakery-ia deployment/ai-insights-service -- \
|
||
|
|
curl http://localhost:8000/health
|
||
|
|
|
||
|
|
# View recent logs with timestamps
|
||
|
|
kubectl logs -n bakery-ia deployment/ai-insights-service \
|
||
|
|
--since=1h \
|
||
|
|
--timestamps
|
||
|
|
|
||
|
|
# Follow logs in real-time
|
||
|
|
kubectl logs -n bakery-ia deployment/ai-insights-service -f
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Performance Optimization
|
||
|
|
|
||
|
|
### Database Optimization
|
||
|
|
|
||
|
|
```sql
|
||
|
|
-- Create covering indexes
|
||
|
|
CREATE INDEX idx_insights_tenant_priority_confidence
|
||
|
|
ON ai_insights(tenant_id, priority, confidence)
|
||
|
|
WHERE deleted_at IS NULL;
|
||
|
|
|
||
|
|
-- Vacuum regularly
|
||
|
|
VACUUM ANALYZE ai_insights;
|
||
|
|
|
||
|
|
-- Check index usage
|
||
|
|
SELECT schemaname, tablename, indexname, idx_scan
|
||
|
|
FROM pg_stat_user_indexes
|
||
|
|
WHERE schemaname = 'public'
|
||
|
|
ORDER BY idx_scan;
|
||
|
|
```
|
||
|
|
|
||
|
|
### Redis Caching
|
||
|
|
|
||
|
|
```python
|
||
|
|
# Cache frequently accessed insights
|
||
|
|
from app.core.cache import redis_client
|
||
|
|
|
||
|
|
async def get_insight_cached(tenant_id: str, insight_id: str):
|
||
|
|
# Check cache
|
||
|
|
cache_key = f"insight:{tenant_id}:{insight_id}"
|
||
|
|
cached = await redis_client.get(cache_key)
|
||
|
|
|
||
|
|
if cached:
|
||
|
|
return json.loads(cached)
|
||
|
|
|
||
|
|
# Fetch from database
|
||
|
|
insight = await repository.get_by_id(tenant_id, insight_id)
|
||
|
|
|
||
|
|
# Cache for 5 minutes
|
||
|
|
await redis_client.setex(
|
||
|
|
cache_key,
|
||
|
|
300,
|
||
|
|
json.dumps(insight.dict())
|
||
|
|
)
|
||
|
|
|
||
|
|
return insight
|
||
|
|
```
|
||
|
|
|
||
|
|
### Batch Operations
|
||
|
|
|
||
|
|
```python
|
||
|
|
# Bulk insert insights
|
||
|
|
async def create_insights_batch(tenant_id: str, insights_data: list[dict]):
|
||
|
|
async with session.begin():
|
||
|
|
insights = [AIInsight(**data) for data in insights_data]
|
||
|
|
session.add_all(insights)
|
||
|
|
await session.flush()
|
||
|
|
return insights
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
## Monitoring and Observability
|
||
|
|
|
||
|
|
### Health Checks
|
||
|
|
|
||
|
|
```python
|
||
|
|
@app.get("/health")
|
||
|
|
async def health_check():
|
||
|
|
return {
|
||
|
|
"status": "healthy",
|
||
|
|
"service": "ai-insights",
|
||
|
|
"version": "1.0.0",
|
||
|
|
"timestamp": datetime.utcnow().isoformat()
|
||
|
|
}
|
||
|
|
|
||
|
|
@app.get("/health/detailed")
|
||
|
|
async def detailed_health_check():
|
||
|
|
# Check database
|
||
|
|
try:
|
||
|
|
async with engine.connect() as conn:
|
||
|
|
await conn.execute(text("SELECT 1"))
|
||
|
|
db_status = "healthy"
|
||
|
|
except Exception as e:
|
||
|
|
db_status = f"unhealthy: {str(e)}"
|
||
|
|
|
||
|
|
# Check Redis
|
||
|
|
try:
|
||
|
|
await redis_client.ping()
|
||
|
|
redis_status = "healthy"
|
||
|
|
except Exception as e:
|
||
|
|
redis_status = f"unhealthy: {str(e)}"
|
||
|
|
|
||
|
|
return {
|
||
|
|
"status": "healthy" if db_status == "healthy" and redis_status == "healthy" else "unhealthy",
|
||
|
|
"components": {
|
||
|
|
"database": db_status,
|
||
|
|
"redis": redis_status
|
||
|
|
}
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
### Metrics Endpoint
|
||
|
|
|
||
|
|
```python
|
||
|
|
from prometheus_client import Counter, Histogram, generate_latest
|
||
|
|
|
||
|
|
insight_created = Counter('insights_created_total', 'Total insights created')
|
||
|
|
insight_applied = Counter('insights_applied_total', 'Total insights applied')
|
||
|
|
api_latency = Histogram('api_request_duration_seconds', 'API request latency')
|
||
|
|
|
||
|
|
@app.get("/metrics")
|
||
|
|
async def metrics():
|
||
|
|
return Response(generate_latest(), media_type="text/plain")
|
||
|
|
```
|
||
|
|
|
||
|
|
---
|
||
|
|
|
||
|
|
*For comprehensive testing procedures, validation steps, and test cases, refer to TESTING_GUIDE.md.*
|