bakery-admin/bakery-ia
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix Purchase Order modal and reorganize documentation

Browse Source 
Frontend Changes:
- Fix runtime error: Remove undefined handleModify reference from ActionQueueCard in DashboardPage
- Migrate PurchaseOrderDetailsModal to use correct PurchaseOrderItem type from purchase_orders service
- Fix item display: Parse unit_price as string (Decimal) instead of number
- Use correct field names: item_notes instead of notes
- Remove deprecated PurchaseOrder types from suppliers.ts to prevent type conflicts
- Update CreatePurchaseOrderModal to use unified types
- Clean up API exports: Remove old PO hooks re-exported from suppliers
- Add comprehensive translations for PO modal (en, es, eu)

Documentation Reorganization:
- Move WhatsApp implementation docs to docs/03-features/notifications/whatsapp/
- Move forecast validation docs to docs/03-features/forecasting/
- Move specification docs to docs/03-features/specifications/
- Move deployment docs (Colima, K8s, VPS sizing) to docs/05-deployment/
- Archive completed implementation summaries to docs/archive/implementation-summaries/
- Delete obsolete FRONTEND_CHANGES_NEEDED.md
- Standardize filenames to lowercase with hyphens

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Urtzi Alfaro
2025-11-18 11:59:23 +01:00
parent 5c45164c8e
commit 3c3d3ce042
32 changed files with 654 additions and 874 deletions
Download Patch File Download Diff File Expand all files Collapse all files

582
docs/03-features/forecasting/validation-implementation.md Normal file
View File

@@ -0,0 +1,582 @@
# Forecast Validation & Continuous Improvement Implementation Summary
**Date**: November 18, 2025
**Status**: ✅ Complete
**Services Modified**: Forecasting, Orchestrator
---
## Overview
Successfully implemented a comprehensive 3-phase validation and continuous improvement system for the Forecasting Service. The system automatically validates forecast accuracy, handles late-arriving sales data, monitors performance trends, and triggers model retraining when needed.
---
## Phase 1: Daily Forecast Validation ✅
### Objective
Implement daily automated validation of forecasts against actual sales data.
### Components Created
#### 1. Database Schema
**New Table**: `validation_runs`
- Tracks each validation execution
- Stores comprehensive accuracy metrics (MAPE, MAE, RMSE, R², Accuracy %)
- Records product and location performance breakdowns
- Links to orchestration runs
- **Migration**: `00002_add_validation_runs_table.py`
#### 2. Core Services
**ValidationService** ([services/forecasting/app/services/validation_service.py](services/forecasting/app/services/validation_service.py))
- `validate_date_range()` - Validates any date range
- `validate_yesterday()` - Daily validation convenience method
- `_fetch_forecasts_with_sales()` - Matches forecasts with sales data via Sales Service
- `_calculate_and_store_metrics()` - Computes all accuracy metrics
**SalesClient** ([services/forecasting/app/services/sales_client.py](services/forecasting/app/services/sales_client.py))
- Wrapper around shared Sales Service client
- Fetches sales data with pagination support
- Handles errors gracefully (returns empty list to allow validation to continue)
#### 3. API Endpoints
**Validation Router** ([services/forecasting/app/api/validation.py](services/forecasting/app/api/validation.py))
- `POST /validation/validate-date-range` - Validate specific date range
- `POST /validation/validate-yesterday` - Validate yesterday's forecasts
- `GET /validation/runs` - List validation runs with filtering
- `GET /validation/runs/{run_id}` - Get detailed validation run results
- `GET /validation/performance-trends` - Get accuracy trends over time
#### 4. Scheduled Jobs
**Daily Validation Job** ([services/forecasting/app/jobs/daily_validation.py](services/forecasting/app/jobs/daily_validation.py))
- `daily_validation_job()` - Called by orchestrator after forecast generation
- `validate_date_range_job()` - For backfilling specific date ranges
#### 5. Orchestrator Integration
**Forecast Client Update** ([shared/clients/forecast_client.py](shared/clients/forecast_client.py))
- Updated `validate_forecasts()` method to call new validation endpoint
- Transforms response to match orchestrator's expected format
- Integrated into orchestrator's daily saga as **Step 5**
### Key Metrics Calculated
- **MAE** (Mean Absolute Error) - Average absolute difference
- **MAPE** (Mean Absolute Percentage Error) - Average percentage error
- **RMSE** (Root Mean Squared Error) - Penalizes large errors
- **R²** (R-squared) - Goodness of fit (0-1 scale)
- **Accuracy %** - 100 - MAPE
### Health Status Thresholds
- **Healthy**: MAPE ≤ 20%
- **Warning**: 20% < MAPE ≤ 30%
- **Critical**: MAPE > 30%
---
## Phase 2: Historical Data Integration ✅
### Objective
Handle late-arriving sales data and backfill validation for historical forecasts.
### Components Created
#### 1. Database Schema
**New Table**: `sales_data_updates`
- Tracks late-arriving sales data
- Records update source (import, manual, pos_sync)
- Links to validation runs
- Tracks validation status (pending, in_progress, completed, failed)
- **Migration**: `00003_add_sales_data_updates_table.py`
#### 2. Core Services
**HistoricalValidationService** ([services/forecasting/app/services/historical_validation_service.py](services/forecasting/app/services/historical_validation_service.py))
- `detect_validation_gaps()` - Finds dates with forecasts but no validation
- `backfill_validation()` - Validates historical date ranges
- `auto_backfill_gaps()` - Automatic gap detection and processing
- `register_sales_data_update()` - Registers late data uploads and triggers validation
- `get_pending_validations()` - Retrieves pending validation queue
#### 3. API Endpoints
**Historical Validation Router** ([services/forecasting/app/api/historical_validation.py](services/forecasting/app/api/historical_validation.py))
- `POST /validation/detect-gaps` - Detect validation gaps (lookback 90 days)
- `POST /validation/backfill` - Manual backfill for specific date range
- `POST /validation/auto-backfill` - Auto detect and backfill gaps (max 10)
- `POST /validation/register-sales-update` - Register late data upload
- `GET /validation/pending` - Get pending validations
**Webhook Router** ([services/forecasting/app/api/webhooks.py](services/forecasting/app/api/webhooks.py))
- `POST /webhooks/sales-import-completed` - Sales import notification
- `POST /webhooks/pos-sync-completed` - POS sync notification
- `GET /webhooks/health` - Webhook health check
#### 4. Event Listeners
**Sales Data Listener** ([services/forecasting/app/jobs/sales_data_listener.py](services/forecasting/app/jobs/sales_data_listener.py))
- `handle_sales_import_completion()` - Processes CSV/Excel import events
- `handle_pos_sync_completion()` - Processes POS synchronization events
- `process_pending_validations()` - Retry mechanism for failed validations
#### 5. Automated Jobs
**Auto Backfill Job** ([services/forecasting/app/jobs/auto_backfill_job.py](services/forecasting/app/jobs/auto_backfill_job.py))
- `auto_backfill_all_tenants()` - Multi-tenant gap processing
- `process_all_pending_validations()` - Multi-tenant pending processing
- `daily_validation_maintenance_job()` - Combined maintenance workflow
- `run_validation_maintenance_for_tenant()` - Single tenant convenience function
### Integration Points
1. **Sales Service** → Calls webhook after imports/sync
2. **Forecasting Service** → Detects gaps, validates historical forecasts
3. **Event System** → Webhook-based notifications for real-time processing
### Gap Detection Logic
```python
# Find dates with forecasts
forecast_dates = {f.forecast_date for f in forecasts}
# Find dates already validated
validated_dates = {v.validation_date_start for v in validation_runs}
# Find gaps
gap_dates = forecast_dates - validated_dates
# Group consecutive dates into ranges
gaps = group_consecutive_dates(gap_dates)
```
---
## Phase 3: Model Improvement Loop ✅
### Objective
Monitor performance trends and automatically trigger model retraining when accuracy degrades.
### Components Created
#### 1. Core Services
**PerformanceMonitoringService** ([services/forecasting/app/services/performance_monitoring_service.py](services/forecasting/app/services/performance_monitoring_service.py))
- `get_accuracy_summary()` - 30-day rolling accuracy metrics
- `detect_performance_degradation()` - Trend analysis (first half vs second half)
- `_identify_poor_performers()` - Products with MAPE > 30%
- `check_model_age()` - Identifies outdated models
- `generate_performance_report()` - Comprehensive report with recommendations
**RetrainingTriggerService** ([services/forecasting/app/services/retraining_trigger_service.py](services/forecasting/app/services/retraining_trigger_service.py))
- `evaluate_and_trigger_retraining()` - Main evaluation loop
- `_trigger_product_retraining()` - Triggers retraining via Training Service
- `trigger_bulk_retraining()` - Multi-product retraining
- `check_and_trigger_scheduled_retraining()` - Age-based retraining
- `get_retraining_recommendations()` - Recommendations without auto-trigger
#### 2. API Endpoints
**Performance Monitoring Router** ([services/forecasting/app/api/performance_monitoring.py](services/forecasting/app/api/performance_monitoring.py))
- `GET /monitoring/accuracy-summary` - 30-day accuracy metrics
- `GET /monitoring/degradation-analysis` - Performance degradation check
- `GET /monitoring/model-age` - Check model age vs threshold
- `POST /monitoring/performance-report` - Comprehensive report generation
- `GET /monitoring/health` - Quick health status for dashboards
**Retraining Router** ([services/forecasting/app/api/retraining.py](services/forecasting/app/api/retraining.py))
- `POST /retraining/evaluate` - Evaluate and optionally trigger retraining
- `POST /retraining/trigger-product` - Trigger single product retraining
- `POST /retraining/trigger-bulk` - Trigger multi-product retraining
- `GET /retraining/recommendations` - Get retraining recommendations
- `POST /retraining/check-scheduled` - Check for age-based retraining
### Performance Thresholds
```python
MAPE_WARNING_THRESHOLD = 20.0 # Warning if MAPE > 20%
MAPE_CRITICAL_THRESHOLD = 30.0 # Critical if MAPE > 30%
MAPE_TREND_THRESHOLD = 5.0 # Alert if MAPE increases > 5%
MIN_SAMPLES_FOR_ALERT = 5 # Minimum validations before alerting
TREND_LOOKBACK_DAYS = 30 # Days to analyze for trends
```
### Degradation Detection
- Splits validation runs into first half and second half
- Compares average MAPE between periods
- Severity levels:
- **None**: MAPE change ≤ 5%
- **Medium**: 5% < MAPE change ≤ 10%
- **High**: MAPE change > 10%
### Automatic Retraining Triggers
1. **Poor Performance**: MAPE > 30% for any product
2. **Degradation**: MAPE increased > 5% over 30 days
3. **Age-Based**: Model not updated in 30+ days
4. **Manual**: Triggered via API by admin/owner
### Training Service Integration
- Calls Training Service API to trigger retraining
- Passes `tenant_id`, `inventory_product_id`, `reason`, `priority`
- Tracks training job ID for monitoring
- Returns status: triggered/failed/no_response
---
## Files Modified
### New Files Created (35 files)
#### Models (2)
1. `services/forecasting/app/models/validation_run.py`
2. `services/forecasting/app/models/sales_data_update.py`
#### Services (5)
1. `services/forecasting/app/services/validation_service.py`
2. `services/forecasting/app/services/sales_client.py`
3. `services/forecasting/app/services/historical_validation_service.py`
4. `services/forecasting/app/services/performance_monitoring_service.py`
5. `services/forecasting/app/services/retraining_trigger_service.py`
#### API Endpoints (5)
1. `services/forecasting/app/api/validation.py`
2. `services/forecasting/app/api/historical_validation.py`
3. `services/forecasting/app/api/webhooks.py`
4. `services/forecasting/app/api/performance_monitoring.py`
5. `services/forecasting/app/api/retraining.py`
#### Jobs (3)
1. `services/forecasting/app/jobs/daily_validation.py`
2. `services/forecasting/app/jobs/sales_data_listener.py`
3. `services/forecasting/app/jobs/auto_backfill_job.py`
#### Database Migrations (2)
1. `services/forecasting/migrations/versions/20251117_add_validation_runs_table.py` (00002)
2. `services/forecasting/migrations/versions/20251117_add_sales_data_updates_table.py` (00003)
### Existing Files Modified (5)
1. **services/forecasting/app/models/__init__.py**
- Added ValidationRun and SalesDataUpdate imports
2. **services/forecasting/app/api/__init__.py**
- Added validation, historical_validation, webhooks, performance_monitoring, retraining router imports
3. **services/forecasting/app/main.py**
- Registered all new routers
- Updated expected_migration_version to "00003"
- Added validation_runs and sales_data_updates to expected_tables
4. **services/forecasting/README.md**
- Added comprehensive validation system documentation (350+ lines)
- Documented all 3 phases with architecture, APIs, thresholds, jobs
- Added integration guides and troubleshooting
5. **services/orchestrator/README.md**
- Added "Forecast Validation Integration" section (150+ lines)
- Documented Step 5 integration in daily workflow
- Added monitoring dashboard metrics
6. **services/forecasting/app/repositories/performance_metric_repository.py**
- Added `bulk_create_metrics()` for efficient bulk insertion
- Added `get_metrics_by_date_range()` for querying specific periods
7. **shared/clients/forecast_client.py**
- Updated `validate_forecasts()` method to call new validation endpoint
- Transformed response to match orchestrator's expected format
---
## Database Schema Changes
### New Tables
#### validation_runs
```sql
CREATE TABLE validation_runs (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL,
validation_date_start DATE NOT NULL,
validation_date_end DATE NOT NULL,
status VARCHAR(50) DEFAULT 'pending',
started_at TIMESTAMP NOT NULL,
completed_at TIMESTAMP,
orchestration_run_id UUID,
-- Metrics
total_forecasts_evaluated INTEGER DEFAULT 0,
forecasts_with_actuals INTEGER DEFAULT 0,
overall_mape FLOAT,
overall_mae FLOAT,
overall_rmse FLOAT,
overall_r_squared FLOAT,
overall_accuracy_percentage FLOAT,
-- Breakdowns
products_evaluated INTEGER DEFAULT 0,
locations_evaluated INTEGER DEFAULT 0,
product_performance JSONB,
location_performance JSONB,
error_message TEXT,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX ix_validation_runs_tenant_created ON validation_runs(tenant_id, started_at);
CREATE INDEX ix_validation_runs_status ON validation_runs(status, started_at);
CREATE INDEX ix_validation_runs_orchestration ON validation_runs(orchestration_run_id);
```
#### sales_data_updates
```sql
CREATE TABLE sales_data_updates (
id UUID PRIMARY KEY,
tenant_id UUID NOT NULL,
update_date_start DATE NOT NULL,
update_date_end DATE NOT NULL,
records_affected INTEGER NOT NULL,
update_source VARCHAR(50) NOT NULL,
import_job_id VARCHAR(255),
validation_status VARCHAR(50) DEFAULT 'pending',
validation_triggered_at TIMESTAMP,
validation_completed_at TIMESTAMP,
validation_run_id UUID REFERENCES validation_runs(id),
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX ix_sales_updates_tenant ON sales_data_updates(tenant_id);
CREATE INDEX ix_sales_updates_dates ON sales_data_updates(update_date_start, update_date_end);
CREATE INDEX ix_sales_updates_status ON sales_data_updates(validation_status);
```
---
## API Endpoints Summary
### Validation (5 endpoints)
- `POST /api/v1/forecasting/{tenant_id}/validation/validate-date-range`
- `POST /api/v1/forecasting/{tenant_id}/validation/validate-yesterday`
- `GET /api/v1/forecasting/{tenant_id}/validation/runs`
- `GET /api/v1/forecasting/{tenant_id}/validation/runs/{run_id}`
- `GET /api/v1/forecasting/{tenant_id}/validation/performance-trends`
### Historical Validation (5 endpoints)
- `POST /api/v1/forecasting/{tenant_id}/validation/detect-gaps`
- `POST /api/v1/forecasting/{tenant_id}/validation/backfill`
- `POST /api/v1/forecasting/{tenant_id}/validation/auto-backfill`
- `POST /api/v1/forecasting/{tenant_id}/validation/register-sales-update`
- `GET /api/v1/forecasting/{tenant_id}/validation/pending`
### Webhooks (3 endpoints)
- `POST /api/v1/forecasting/{tenant_id}/webhooks/sales-import-completed`
- `POST /api/v1/forecasting/{tenant_id}/webhooks/pos-sync-completed`
- `GET /api/v1/forecasting/{tenant_id}/webhooks/health`
### Performance Monitoring (5 endpoints)
- `GET /api/v1/forecasting/{tenant_id}/monitoring/accuracy-summary`
- `GET /api/v1/forecasting/{tenant_id}/monitoring/degradation-analysis`
- `GET /api/v1/forecasting/{tenant_id}/monitoring/model-age`
- `POST /api/v1/forecasting/{tenant_id}/monitoring/performance-report`
- `GET /api/v1/forecasting/{tenant_id}/monitoring/health`
### Retraining (5 endpoints)
- `POST /api/v1/forecasting/{tenant_id}/retraining/evaluate`
- `POST /api/v1/forecasting/{tenant_id}/retraining/trigger-product`
- `POST /api/v1/forecasting/{tenant_id}/retraining/trigger-bulk`
- `GET /api/v1/forecasting/{tenant_id}/retraining/recommendations`
- `POST /api/v1/forecasting/{tenant_id}/retraining/check-scheduled`
**Total**: 23 new API endpoints
---
## Scheduled Jobs
### Daily Jobs
1. **Daily Validation** (8:00 AM after orchestrator)
- Validates yesterday's forecasts vs actual sales
- Stores validation results
- Identifies poor performers
2. **Daily Maintenance** (6:00 AM)
- Processes pending validations (retry failures)
- Auto-backfills detected gaps (90-day lookback)
### Weekly Jobs
1. **Retraining Evaluation** (Sunday night)
- Analyzes 30-day performance
- Triggers retraining for products with MAPE > 30%
- Triggers retraining for degraded performance
---
## Business Impact
### Before Implementation
- ❌ No systematic forecast validation
- ❌ No visibility into model accuracy
- ❌ Late sales data ignored
- ❌ Manual model retraining decisions
- ❌ No tracking of forecast quality over time
- ❌ Trust in forecasts based on intuition
### After Implementation
- ✅ **Daily accuracy tracking** with MAPE, MAE, RMSE metrics
- ✅ **100% validation coverage** (no gaps in historical data)
- ✅ **Automatic backfill** when late data arrives
- ✅ **Performance monitoring** with trend analysis
- ✅ **Automatic retraining** when MAPE > 30%
- ✅ **Product-level insights** for optimization
- ✅ **Complete audit trail** of forecast performance
### Expected Results
**After 1 Month:**
- 100% of forecasts validated daily
- Baseline accuracy metrics established
- Poor performers identified
**After 3 Months:**
- 10-15% accuracy improvement from automatic retraining
- MAPE reduced from 25% → 15% average
- Better inventory decisions from trusted forecasts
- Reduced waste from accurate predictions
**After 6 Months:**
- Continuous improvement cycle established
- Optimal accuracy for each product category
- Predictable performance metrics
- Full trust in forecast-driven decisions
### ROI Impact
- **Waste Reduction**: Additional 5-10% from improved accuracy
- **Trust Building**: Validated metrics increase user confidence
- **Time Savings**: Zero manual validation work
- **Model Quality**: Continuous improvement vs. static models
- **Competitive Advantage**: Industry-leading forecast accuracy tracking
---
## Technical Implementation Details
### Error Handling
- All services use try/except with structured logging
- Graceful degradation (validation continues if some forecasts fail)
- Retry mechanism for failed validations
- Transaction safety with rollback on errors
### Performance Optimizations
- Bulk insertion for validation metrics
- Pagination for large datasets
- Efficient gap detection with set operations
- Indexed queries for fast lookups
- Async/await throughout for concurrency
### Security
- Role-based access control (@require_user_role)
- Tenant isolation (all queries scoped to tenant_id)
- Input validation with Pydantic schemas
- SQL injection prevention (parameterized queries)
- Audit logging for all operations
### Testing Considerations
- Unit tests needed for all services
- Integration tests for workflow flows
- Performance tests for bulk operations
- End-to-end tests for orchestrator integration
---
## Integration with Existing Services
### Forecasting Service
- ✅ New validation workflow integrated
- ✅ Performance monitoring added
- ✅ Retraining triggers implemented
- ✅ Webhook endpoints for external integration
### Orchestrator Service
- ✅ Step 5 added to daily saga
- ✅ Calls forecast_client.validate_forecasts()
- ✅ Logs validation results
- ✅ Handles validation failures gracefully
### Sales Service
- 🔄 **TODO**: Add webhook calls after imports/sync
- 🔄 **TODO**: Notify Forecasting Service of data updates
### Training Service
- ✅ Receives retraining triggers from Forecasting Service
- ✅ Returns training job ID for tracking
- ✅ Handles priority-based scheduling
---
## Deployment Checklist
### Database
- ✅ Run migration 00002 (validation_runs table)
- ✅ Run migration 00003 (sales_data_updates table)
- ✅ Verify indexes created
- ✅ Test migration rollback
### Configuration
- ⏳ Set MAPE thresholds (if customization needed)
- ⏳ Configure scheduled job times
- ⏳ Set up webhook endpoints in Sales Service
- ⏳ Configure Training Service client
### Monitoring
- ⏳ Add validation metrics to Grafana dashboards
- ⏳ Set up alerts for critical MAPE thresholds
- ⏳ Monitor validation job execution times
- ⏳ Track retraining trigger frequency
### Documentation
- ✅ Forecasting Service README updated
- ✅ Orchestrator Service README updated
- ✅ API documentation complete
- ⏳ User-facing documentation (how to interpret metrics)
---
## Known Limitations & Future Enhancements
### Current Limitations
1. Model age tracking incomplete (needs Training Service data)
2. Retraining status tracking not implemented
3. No UI dashboard for validation metrics
4. No email/SMS alerts for critical performance
5. No A/B testing framework for model comparison
### Planned Enhancements
1. **Performance Alerts** - Email/SMS when MAPE > 30%
2. **Model Versioning** - Track which model version generated each forecast
3. **A/B Testing** - Compare old vs new models
4. **Explainability** - SHAP values to explain forecast drivers
5. **Forecasting Confidence** - Confidence intervals for each prediction
6. **Multi-Region Support** - Different thresholds per region
7. **Custom Thresholds** - Per-tenant or per-product customization
---
## Conclusion
The Forecast Validation & Continuous Improvement system is now **fully implemented** across all 3 phases:
✅ **Phase 1**: Daily forecast validation with comprehensive metrics
✅ **Phase 2**: Historical data integration with gap detection and backfill
✅ **Phase 3**: Performance monitoring and automatic retraining
This implementation provides a complete closed-loop system where forecasts are:
1. Generated daily by the orchestrator
2. Validated automatically the next day
3. Monitored for performance trends
4. Improved through automatic retraining
The system is production-ready and provides significant business value through improved forecast accuracy, reduced waste, and increased trust in AI-driven decisions.
---
**Implementation Date**: November 18, 2025
**Implementation Status**: ✅ Complete
**Code Quality**: Production-ready
**Documentation**: Complete
**Testing Status**: ⏳ Pending
**Deployment Status**: ⏳ Ready for deployment
---
© 2025 Bakery-IA. All rights reserved.

402
docs/03-features/notifications/whatsapp/implementation-summary.md Normal file
View File

@@ -0,0 +1,402 @@
# WhatsApp Shared Account Implementation - Summary
## What Was Implemented
A **simplified WhatsApp notification system** using a **shared master account** model, perfect for your 10-bakery pilot program. This eliminates the need for non-technical bakery owners to configure Meta credentials.
---
## Key Changes Made
### ✅ Backend Changes
1. **Tenant Settings Model** - Removed per-tenant credentials, added display phone number
- File: [tenant_settings.py](services/tenant/app/models/tenant_settings.py)
- File: [tenant_settings.py](services/tenant/app/schemas/tenant_settings.py)
2. **Notification Service** - Always uses shared master credentials with tenant-specific phone numbers
- File: [whatsapp_business_service.py](services/notification/app/services/whatsapp_business_service.py)
3. **Phone Number Management API** - New admin endpoints for assigning phone numbers
- File: [whatsapp_admin.py](services/tenant/app/api/whatsapp_admin.py)
- Registered in: [main.py](services/tenant/app/main.py)
### ✅ Frontend Changes
4. **Simplified Settings UI** - Removed credential inputs, shows assigned phone number only
- File: [NotificationSettingsCard.tsx](frontend/src/pages/app/database/ajustes/cards/NotificationSettingsCard.tsx)
- Types: [settings.ts](frontend/src/api/types/settings.ts)
5. **Admin Interface** - New page for assigning phone numbers to tenants
- File: [WhatsAppAdminPage.tsx](frontend/src/pages/app/admin/WhatsAppAdminPage.tsx)
### ✅ Documentation
6. **Comprehensive Guides**
- [WHATSAPP_SHARED_ACCOUNT_GUIDE.md](WHATSAPP_SHARED_ACCOUNT_GUIDE.md) - Full implementation details
- [WHATSAPP_MASTER_ACCOUNT_SETUP.md](WHATSAPP_MASTER_ACCOUNT_SETUP.md) - Step-by-step setup
---
## Quick Start (For You - Platform Admin)
### Step 1: Set Up Master WhatsApp Account (One-Time)
Follow the detailed guide: [WHATSAPP_MASTER_ACCOUNT_SETUP.md](WHATSAPP_MASTER_ACCOUNT_SETUP.md)
**Summary:**
1. Create Meta Business Account
2. Add WhatsApp product
3. Verify business (1-3 days wait)
4. Add 10 phone numbers
5. Create message templates
6. Get credentials (WABA ID, Access Token, Phone Number IDs)
**Time:** 2-3 hours + verification wait
### Step 2: Configure Environment Variables
Edit `services/notification/.env`:
```bash
WHATSAPP_BUSINESS_ACCOUNT_ID=your-waba-id-here
WHATSAPP_ACCESS_TOKEN=your-access-token-here
WHATSAPP_PHONE_NUMBER_ID=default-phone-id-here
WHATSAPP_API_VERSION=v18.0
ENABLE_WHATSAPP_NOTIFICATIONS=true
WHATSAPP_WEBHOOK_VERIFY_TOKEN=your-secret-token-here
```
### Step 3: Restart Services
```bash
docker-compose restart notification-service tenant-service
```
### Step 4: Assign Phone Numbers to Bakeries
**Option A: Via Admin UI (Recommended)**
1. Open: `http://localhost:5173/app/admin/whatsapp`
2. For each bakery:
- Select phone number from dropdown
- Click assign
**Option B: Via API**
```bash
curl -X POST http://localhost:8001/api/v1/admin/whatsapp/tenants/{tenant_id}/assign-phone \
-H "Content-Type: application/json" \
-d '{
"phone_number_id": "123456789012345",
"display_phone_number": "+34 612 345 678"
}'
```
### Step 5: Test
1. Login as a bakery owner
2. Go to Settings → Notifications
3. Toggle WhatsApp ON
4. Verify phone number is displayed
5. Create a test purchase order
6. Supplier should receive WhatsApp message!
---
## For Bakery Owners (What They Need to Do)
### Before:
❌ Navigate Meta Business Suite
❌ Create WhatsApp Business Account
❌ Get 3 different credential IDs
❌ Copy/paste into settings
**Time:** 1-2 hours, high error rate
### After:
✅ Go to Settings → Notifications
✅ Toggle WhatsApp ON
✅ Done!
**Time:** 30 seconds
**No configuration needed - phone number is already assigned by you (admin)!**
---
## Architecture Overview
```
┌─────────────────────────────────────────────┐
│ Master WhatsApp Business Account │
│ - Admin manages centrally │
│ - Single set of credentials │
│ - 10 phone numbers (one per bakery) │
└─────────────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
Phone #1 Phone #2 Phone #3
+34 612 +34 612 +34 612
345 678 345 679 345 680
│ │ │
Bakery A Bakery B Bakery C
```
---
## API Endpoints Created
### Admin Endpoints (New)
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/v1/admin/whatsapp/phone-numbers` | List available phone numbers |
| GET | `/api/v1/admin/whatsapp/tenants` | List tenants with WhatsApp status |
| POST | `/api/v1/admin/whatsapp/tenants/{id}/assign-phone` | Assign phone to tenant |
| DELETE | `/api/v1/admin/whatsapp/tenants/{id}/unassign-phone` | Unassign phone from tenant |
### Test Commands
```bash
# View available phone numbers
curl http://localhost:8001/api/v1/admin/whatsapp/phone-numbers | jq
# View tenant WhatsApp status
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | jq
# Assign phone to tenant
curl -X POST http://localhost:8001/api/v1/admin/whatsapp/tenants/{tenant_id}/assign-phone \
-H "Content-Type: application/json" \
-d '{"phone_number_id": "XXX", "display_phone_number": "+34 612 345 678"}'
```
---
## Database Changes
### Tenant Settings Schema
**Before:**
```json
{
"notification_settings": {
"whatsapp_enabled": false,
"whatsapp_phone_number_id": "",
"whatsapp_access_token": "", // REMOVED
"whatsapp_business_account_id": "", // REMOVED
"whatsapp_api_version": "v18.0", // REMOVED
"whatsapp_default_language": "es"
}
}
```
**After:**
```json
{
"notification_settings": {
"whatsapp_enabled": false,
"whatsapp_phone_number_id": "", // Phone from shared account
"whatsapp_display_phone_number": "", // NEW: Display format
"whatsapp_default_language": "es"
}
}
```
**Migration:** No SQL migration needed (JSONB is schema-less). Existing data will work with defaults.
---
## Cost Estimate
### WhatsApp Messaging Costs (Spain)
- **Per conversation:** €0.0319 - €0.0699
- **Conversation window:** 24 hours
- **User-initiated:** Free
### Monthly Estimate (10 Bakeries)
```
5 POs per bakery per day × 10 bakeries × 30 days = 1,500 messages/month
1,500 × €0.05 (avg) = €75/month
```
### Setup Cost Savings
**Old Model (Per-Tenant):**
- 10 bakeries × 1.5 hours × €50/hr = **€750 in setup time**
**New Model (Shared Account):**
- Admin: 2 hours setup (one time)
- Per bakery: 5 minutes × 10 = **€0 in bakery time**
**Savings:** €750 in bakery owner time + reduced support tickets
---
## Monitoring & Maintenance
### Check Quality Rating (Weekly)
```bash
curl -X GET "https://graph.facebook.com/v18.0/{PHONE_NUMBER_ID}" \
-H "Authorization: Bearer {ACCESS_TOKEN}" \
| jq '.quality_rating'
```
**Quality Ratings:**
- **GREEN** ✅ - All good
- **YELLOW** ⚠️ - Review messaging patterns
- **RED** ❌ - Fix immediately
### View Message Logs
```bash
# Docker logs
docker logs -f notification-service | grep whatsapp
# Database query
SELECT tenant_id, recipient_phone, status, created_at, error_message
FROM whatsapp_messages
WHERE created_at > NOW() - INTERVAL '24 hours'
ORDER BY created_at DESC;
```
### Rotate Access Token (Every 60 Days)
1. Generate new token in Meta Business Manager
2. Update `WHATSAPP_ACCESS_TOKEN` in `.env`
3. Restart notification service
4. Revoke old token
---
## Troubleshooting
### Bakery doesn't receive WhatsApp messages
**Checklist:**
1. ✅ WhatsApp enabled in tenant settings?
2. ✅ Phone number assigned to tenant?
3. ✅ Master credentials in environment variables?
4. ✅ Template approved by Meta?
5. ✅ Recipient phone in E.164 format (+34612345678)?
**Check logs:**
```bash
docker logs -f notification-service | grep -i "whatsapp\|error"
```
### Phone assignment fails: "Already assigned"
Find which tenant has it:
```bash
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | \
jq '.[] | select(.phone_number_id == "YOUR_PHONE_ID")'
```
Unassign first:
```bash
curl -X DELETE http://localhost:8001/api/v1/admin/whatsapp/tenants/{tenant_id}/unassign-phone
```
### "WhatsApp master account not configured"
Ensure environment variables are set:
```bash
docker exec notification-service env | grep WHATSAPP
```
Should show all variables (WABA ID, Access Token, Phone Number ID).
---
## Next Steps
### Immediate (Before Pilot)
- [ ] Complete master account setup (follow [WHATSAPP_MASTER_ACCOUNT_SETUP.md](WHATSAPP_MASTER_ACCOUNT_SETUP.md))
- [ ] Assign phone numbers to all 10 pilot bakeries
- [ ] Send email to bakeries: "WhatsApp notifications are ready - just toggle ON in settings"
- [ ] Test with 2-3 bakeries first
- [ ] Monitor for first week
### Short-term (During Pilot)
- [ ] Collect bakery feedback
- [ ] Monitor quality rating daily
- [ ] Track message costs
- [ ] Document common support questions
### Long-term (After Pilot)
- [ ] Consider WhatsApp Embedded Signup for self-service (if scaling beyond 10)
- [ ] Create additional templates (inventory alerts, production alerts)
- [ ] Implement rich media messages (images, documents)
- [ ] Add interactive buttons (approve/reject PO via WhatsApp)
---
## Files Modified/Created
### Backend
**Modified:**
- `services/tenant/app/models/tenant_settings.py`
- `services/tenant/app/schemas/tenant_settings.py`
- `services/notification/app/services/whatsapp_business_service.py`
- `services/tenant/app/main.py`
**Created:**
- `services/tenant/app/api/whatsapp_admin.py`
### Frontend
**Modified:**
- `frontend/src/pages/app/database/ajustes/cards/NotificationSettingsCard.tsx`
- `frontend/src/api/types/settings.ts`
**Created:**
- `frontend/src/pages/app/admin/WhatsAppAdminPage.tsx`
### Documentation
**Created:**
- `WHATSAPP_SHARED_ACCOUNT_GUIDE.md` - Full implementation guide
- `WHATSAPP_MASTER_ACCOUNT_SETUP.md` - Admin setup instructions
- `WHATSAPP_IMPLEMENTATION_SUMMARY.md` - This file
---
## Support
**Questions?**
- Technical implementation: Review [WHATSAPP_SHARED_ACCOUNT_GUIDE.md](WHATSAPP_SHARED_ACCOUNT_GUIDE.md)
- Setup help: Follow [WHATSAPP_MASTER_ACCOUNT_SETUP.md](WHATSAPP_MASTER_ACCOUNT_SETUP.md)
- Meta documentation: https://developers.facebook.com/docs/whatsapp
**Common Issues:**
- Most problems are due to missing/incorrect environment variables
- Check logs: `docker logs -f notification-service`
- Verify Meta credentials haven't expired
- Ensure templates are APPROVED (not PENDING)
---
## Summary
✅ **Zero configuration** for bakery users
✅ **5-minute setup** per bakery (admin)
✅ **€750 saved** in setup costs
✅ **Lower support burden**
✅ **Perfect for 10-bakery pilot**
✅ **Can scale** to 120 bakeries with same model
**Next:** Set up your master WhatsApp account following [WHATSAPP_MASTER_ACCOUNT_SETUP.md](WHATSAPP_MASTER_ACCOUNT_SETUP.md)
---
**Implementation Date:** 2025-01-17
**Status:** ✅ Complete and Ready for Pilot
**Estimated Setup Time:** 2-3 hours (one-time)
**Per-Bakery Time:** 5 minutes

691
docs/03-features/notifications/whatsapp/master-account-setup.md Normal file
View File

@@ -0,0 +1,691 @@
# WhatsApp Master Account Setup Guide
**Quick Setup Guide for Platform Admin**
This guide walks you through setting up the Master WhatsApp Business Account for the bakery-ia pilot program.
---
## Prerequisites
- [ ] Meta/Facebook Business account
- [ ] Business verification documents (tax ID, business registration)
- [ ] 10 phone numbers for pilot bakeries
- [ ] Credit card for WhatsApp Business API billing
**Time Required:** 2-3 hours (including verification wait time)
---
## Step 1: Create Meta Business Account
### 1.1 Create Business Manager
1. Go to [Meta Business Suite](https://business.facebook.com)
2. Click **Create Account**
3. Enter business details:
- Business Name: "Bakery Platform" (or your company name)
- Your Name
- Business Email
4. Click **Submit**
### 1.2 Verify Your Business
Meta requires business verification for WhatsApp API access:
1. In Business Settings → **Security Center**
2. Click **Start Verification**
3. Choose verification method:
- **Business Documents** (Recommended)
- Upload tax registration document
- Upload business license or registration
- **Domain Verification**
- Add DNS TXT record to your domain
- **Phone Verification**
- Receive call/SMS to business phone
4. Wait for verification (typically 1-3 business days)
**Status Check:**
```
Business Settings → Security Center → Verification Status
```
---
## Step 2: Add WhatsApp Product
### 2.1 Enable WhatsApp
1. In Business Manager, go to **Settings**
2. Click **Accounts** → **WhatsApp Accounts**
3. Click **Add** → **Create a new WhatsApp Business Account**
4. Fill in details:
- Display Name: "Bakery Platform"
- Category: Food & Beverage
- Description: "Bakery management notifications"
5. Click **Create**
### 2.2 Configure WhatsApp Business Account
1. After creation, note your **WhatsApp Business Account ID (WABA ID)**
- Found in: WhatsApp Manager → Settings → Business Info
- Format: `987654321098765` (15 digits)
- **Save this:** You'll need it for environment variables
---
## Step 3: Add Phone Numbers
### 3.1 Add Your First Phone Number
**Option A: Use Your Own Phone Number** (Recommended for testing)
1. In WhatsApp Manager → **Phone Numbers**
2. Click **Add Phone Number**
3. Enter phone number in E.164 format: `+34612345678`
4. Choose verification method:
- **SMS** (easiest)
- **Voice call**
5. Enter verification code
6. Note the **Phone Number ID**:
- Format: `123456789012345` (15 digits)
- **Save this:** Default phone number for environment variables
**Option B: Use Meta-Provided Free Number**
1. In WhatsApp Manager → **Phone Numbers**
2. Click **Get a free phone number**
3. Choose country: Spain (+34)
4. Meta assigns a number in format: `+1555XXXXXXX`
5. Note: Free numbers have limitations:
- Can't be ported to other accounts
- Limited to 1,000 conversations/day
- Good for pilot, not production
### 3.2 Add Additional Phone Numbers (For Pilot Bakeries)
Repeat the process to add 10 phone numbers total (one per bakery).
**Tips:**
- Use virtual phone number services (Twilio, Plivo, etc.)
- Cost: ~€5-10/month per number
- Alternative: Request Meta phone numbers (via support ticket)
**Request Phone Number Limit Increase:**
If you need more than 2 phone numbers:
1. Open support ticket at [WhatsApp Business Support](https://business.whatsapp.com/support)
2. Request: "Increase phone number limit to 10 for pilot program"
3. Provide business justification
4. Wait 1-2 days for approval
---
## Step 4: Create System User & Access Token
### 4.1 Create System User
**Why:** System Users provide permanent access tokens (don't expire every 60 days).
1. In Business Settings → **Users** → **System Users**
2. Click **Add**
3. Enter details:
- Name: "WhatsApp API Service"
- Role: **Admin**
4. Click **Create System User**
### 4.2 Generate Access Token
1. Select the system user you just created
2. Click **Add Assets**
3. Choose **WhatsApp Accounts**
4. Select your WhatsApp Business Account
5. Grant permissions:
- ✅ Manage WhatsApp Business Account
- ✅ Manage WhatsApp Business Messaging
- ✅ Read WhatsApp Business Insights
6. Click **Generate New Token**
7. Select token permissions:
- ✅ `whatsapp_business_management`
- ✅ `whatsapp_business_messaging`
8. Click **Generate Token**
9. **IMPORTANT:** Copy the token immediately
- Format: `EAAxxxxxxxxxxxxxxxxxxxxxxxx` (long string)
- **Save this securely:** You can't view it again!
**Token Security:**
```bash
# Good: Use environment variable
WHATSAPP_ACCESS_TOKEN=EAAxxxxxxxxxxxxx
# Bad: Hardcode in source code
# token = "EAAxxxxxxxxxxxxx" # DON'T DO THIS!
```
---
## Step 5: Create Message Templates
WhatsApp requires pre-approved templates for business-initiated messages.
### 5.1 Create PO Notification Template
1. In WhatsApp Manager → **Message Templates**
2. Click **Create Template**
3. Fill in template details:
```
Template Name: po_notification
Category: UTILITY
Languages: Spanish (es)
Message Body:
Hola {{1}}, has recibido una nueva orden de compra {{2}} por un total de {{3}}.
Parameters:
1. Supplier Name (text)
2. PO Number (text)
3. Total Amount (text)
Example:
Hola Juan García, has recibido una nueva orden de compra PO-12345 por un total de €250.50.
```
4. Click **Submit for Approval**
**Approval Time:**
- Typical: 15 minutes to 2 hours
- Complex templates: Up to 24 hours
- If rejected: Review feedback and resubmit
### 5.2 Check Template Status
**Via UI:**
```
WhatsApp Manager → Message Templates → Filter by Status
```
**Via API:**
```bash
curl "https://graph.facebook.com/v18.0/{WABA_ID}/message_templates?fields=name,status,language" \
-H "Authorization: Bearer {ACCESS_TOKEN}" | jq
```
**Template Statuses:**
- `PENDING` - Under review
- `APPROVED` - Ready to use
- `REJECTED` - Review feedback and fix
- `DISABLED` - Paused due to quality issues
### 5.3 Create Additional Templates (Optional)
For inventory alerts, production alerts, etc.:
```
Template Name: low_stock_alert
Category: UTILITY
Language: Spanish (es)
Message:
⚠️ Alerta: El ingrediente {{1}} tiene stock bajo.
Cantidad actual: {{2}} {{3}}.
Punto de reorden: {{4}} {{5}}.
```
---
## Step 6: Configure Webhooks (For Status Updates)
### 6.1 Create Webhook Endpoint
Webhooks notify you of message delivery status, read receipts, etc.
**Your webhook endpoint:**
```
https://your-domain.com/api/v1/whatsapp/webhook
```
**Implemented in:** `services/notification/app/api/whatsapp_webhooks.py`
### 6.2 Register Webhook with Meta
1. In WhatsApp Manager → **Configuration**
2. Click **Edit** next to Webhook
3. Enter details:
```
Callback URL: https://your-domain.com/api/v1/whatsapp/webhook
Verify Token: random-secret-token-here
```
4. Click **Verify and Save**
**Meta will send GET request to verify:**
```
GET /api/v1/whatsapp/webhook?hub.verify_token=YOUR_TOKEN&hub.challenge=XXXXX
```
**Your endpoint must respond with:** `hub.challenge` value
### 6.3 Subscribe to Webhook Events
Select events to receive:
- ✅ `messages` - Incoming messages (for replies)
- ✅ `message_status` - Delivery, read receipts
- ✅ `message_echoes` - Sent message confirmations
**Environment Variable:**
```bash
WHATSAPP_WEBHOOK_VERIFY_TOKEN=random-secret-token-here
```
---
## Step 7: Configure Environment Variables
### 7.1 Collect All Credentials
You should now have:
1. ✅ **WhatsApp Business Account ID (WABA ID)**
- Example: `987654321098765`
- Where: WhatsApp Manager → Settings → Business Info
2. ✅ **Access Token**
- Example: `EAAxxxxxxxxxxxxxxxxxxxxxxxx`
- Where: System User token you generated
3. ✅ **Phone Number ID** (default/fallback)
- Example: `123456789012345`
- Where: WhatsApp Manager → Phone Numbers
4. ✅ **Webhook Verify Token** (you chose this)
- Example: `my-secret-webhook-token-12345`
### 7.2 Update Notification Service Environment
**File:** `services/notification/.env`
```bash
# ================================================================
# WhatsApp Business Cloud API Configuration
# ================================================================
# Master WhatsApp Business Account ID (15 digits)
WHATSAPP_BUSINESS_ACCOUNT_ID=987654321098765
# System User Access Token (starts with EAA)
WHATSAPP_ACCESS_TOKEN=EAAxxxxxxxxxxxxxxxxxxxxxxxx
# Default Phone Number ID (15 digits) - fallback if tenant has none assigned
WHATSAPP_PHONE_NUMBER_ID=123456789012345
# WhatsApp Cloud API Version
WHATSAPP_API_VERSION=v18.0
# Enable/disable WhatsApp notifications globally
ENABLE_WHATSAPP_NOTIFICATIONS=true
# Webhook verification token (random secret you chose)
WHATSAPP_WEBHOOK_VERIFY_TOKEN=my-secret-webhook-token-12345
```
### 7.3 Restart Services
```bash
# Docker Compose
docker-compose restart notification-service
# Kubernetes
kubectl rollout restart deployment/notification-service
# Or rebuild
docker-compose up -d --build notification-service
```
---
## Step 8: Verify Setup
### 8.1 Test API Connectivity
**Check if credentials work:**
```bash
curl -X GET "https://graph.facebook.com/v18.0/{PHONE_NUMBER_ID}" \
-H "Authorization: Bearer {ACCESS_TOKEN}" \
| jq
```
**Expected Response:**
```json
{
"verified_name": "Bakery Platform",
"display_phone_number": "+34 612 345 678",
"quality_rating": "GREEN",
"id": "123456789012345"
}
```
**If error:**
```json
{
"error": {
"message": "Invalid OAuth access token",
"type": "OAuthException",
"code": 190
}
}
```
→ Check your access token
### 8.2 Test Sending a Message
**Via API:**
```bash
curl -X POST "https://graph.facebook.com/v18.0/{PHONE_NUMBER_ID}/messages" \
-H "Authorization: Bearer {ACCESS_TOKEN}" \
-H "Content-Type: application/json" \
-d '{
"messaging_product": "whatsapp",
"to": "+34612345678",
"type": "template",
"template": {
"name": "po_notification",
"language": {
"code": "es"
},
"components": [
{
"type": "body",
"parameters": [
{"type": "text", "text": "Juan García"},
{"type": "text", "text": "PO-12345"},
{"type": "text", "text": "€250.50"}
]
}
]
}
}'
```
**Expected Response:**
```json
{
"messaging_product": "whatsapp",
"contacts": [
{
"input": "+34612345678",
"wa_id": "34612345678"
}
],
"messages": [
{
"id": "wamid.XXXxxxXXXxxxXXX"
}
]
}
```
**Check WhatsApp on recipient's phone!**
### 8.3 Test via Notification Service
**Trigger PO notification:**
```bash
curl -X POST http://localhost:8002/api/v1/whatsapp/send \
-H "Content-Type: application/json" \
-d '{
"tenant_id": "uuid-here",
"recipient_phone": "+34612345678",
"recipient_name": "Juan García",
"message_type": "template",
"template": {
"template_name": "po_notification",
"language": "es",
"components": [
{
"type": "body",
"parameters": [
{"type": "text", "text": "Juan García"},
{"type": "text", "text": "PO-TEST-001"},
{"type": "text", "text": "€150.00"}
]
}
]
}
}'
```
**Check logs:**
```bash
docker logs -f notification-service | grep whatsapp
```
**Expected log output:**
```
[INFO] Using shared WhatsApp account tenant_id=xxx phone_number_id=123456789012345
[INFO] WhatsApp template message sent successfully message_id=xxx whatsapp_message_id=wamid.XXX
```
---
## Step 9: Assign Phone Numbers to Tenants
Now that the master account is configured, assign phone numbers to each bakery.
### 9.1 Access Admin Interface
1. Open: `http://localhost:5173/app/admin/whatsapp`
2. You should see:
- **Available Phone Numbers:** List of your 10 numbers
- **Bakery Tenants:** List of all bakeries
### 9.2 Assign Each Bakery
For each of the 10 pilot bakeries:
1. Find tenant in the list
2. Click dropdown: **Assign phone number...**
3. Select a phone number
4. Verify green checkmark appears
**Example:**
```
Panadería San Juan → +34 612 345 678
Panadería Goiko → +34 612 345 679
Bakery Artesano → +34 612 345 680
... (7 more)
```
### 9.3 Verify Assignments
```bash
# Check all assignments
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | jq
# Should show each tenant with assigned phone
```
---
## Step 10: Monitor & Maintain
### 10.1 Monitor Quality Rating
WhatsApp penalizes low-quality messaging. Check your quality rating weekly:
```bash
curl -X GET "https://graph.facebook.com/v18.0/{PHONE_NUMBER_ID}" \
-H "Authorization: Bearer {ACCESS_TOKEN}" \
| jq '.quality_rating'
```
**Quality Ratings:**
- **GREEN** ✅ - All good, no restrictions
- **YELLOW** ⚠️ - Warning, review messaging patterns
- **RED** ❌ - Restricted, fix issues immediately
**Common Issues Leading to Low Quality:**
- High block rate (users blocking your number)
- Sending to invalid phone numbers
- Template violations (sending promotional content in UTILITY templates)
- User reports (spam complaints)
### 10.2 Check Message Costs
```bash
# View billing in Meta Business Manager
Business Settings → Payments → WhatsApp Business API
```
**Cost per Conversation (Spain):**
- Business-initiated: €0.0319 - €0.0699
- User-initiated: Free (24hr window)
**Monthly Estimate (10 Bakeries):**
- 5 POs per day per bakery = 50 messages/day
- 50 × 30 days = 1,500 messages/month
- 1,500 × €0.05 = **~€75/month**
### 10.3 Rotate Access Token (Every 60 Days)
Even though system user tokens are "permanent," rotate for security:
1. Generate new token (Step 4.2)
2. Update environment variable
3. Restart notification service
4. Revoke old token
**Set reminder:** Calendar alert every 60 days
---
## Troubleshooting
### Issue: Business verification stuck
**Solution:**
- Check Business Manager → Security Center
- Common reasons:
- Documents unclear/incomplete
- Business name mismatch with documents
- Banned domain/business
- Contact Meta Business Support if > 5 days
### Issue: Phone number verification fails
**Error:** "This phone number is already registered with WhatsApp"
**Solution:**
- Number is used for personal WhatsApp
- You must use a different number OR
- Delete personal WhatsApp account (this is permanent!)
### Issue: Template rejected
**Common Rejection Reasons:**
1. **Contains promotional content in UTILITY template**
- Fix: Remove words like "offer," "sale," "discount"
- Use MARKETING category instead
2. **Missing variable indicators**
- Fix: Ensure {{1}}, {{2}}, {{3}} are clearly marked
- Provide good example values
3. **Unclear purpose**
- Fix: Add context in template description
- Explain use case clearly
**Resubmit:** Edit template and click "Submit for Review" again
### Issue: "Invalid OAuth access token"
**Solutions:**
1. Token expired → Generate new one (Step 4.2)
2. Wrong token → Copy correct token from System User
3. Token doesn't have permissions → Regenerate with correct scopes
### Issue: Webhook verification fails
**Error:** "The URL couldn't be validated. Callback verification failed"
**Checklist:**
- [ ] Endpoint is publicly accessible (not localhost)
- [ ] Returns `200 OK` status
- [ ] Returns the `hub.challenge` value exactly
- [ ] HTTPS enabled (not HTTP)
- [ ] Verify token matches environment variable
**Test webhook manually:**
```bash
curl "https://your-domain.com/api/v1/whatsapp/webhook?hub.verify_token=YOUR_TOKEN&hub.challenge=12345"
# Should return: 12345
```
---
## Checklist: You're Done When...
- [ ] Meta Business Account created and verified
- [ ] WhatsApp Business Account created (WABA ID saved)
- [ ] 10 phone numbers added and verified
- [ ] System User created
- [ ] Access Token generated and saved securely
- [ ] Message template `po_notification` approved
- [ ] Webhook configured and verified
- [ ] Environment variables set in `.env`
- [ ] Notification service restarted
- [ ] Test message sent successfully
- [ ] All 10 bakeries assigned phone numbers
- [ ] Quality rating is GREEN
- [ ] Billing configured in Meta Business Manager
**Estimated Total Time:** 2-3 hours (plus 1-3 days for business verification)
---
## Next Steps
1. **Inform Bakeries:**
- Send email: "WhatsApp notifications are now available"
- Instruct them to toggle WhatsApp ON in settings
- No configuration needed on their end!
2. **Monitor First Week:**
- Check quality rating daily
- Review message logs for errors
- Gather bakery feedback
3. **Scale Beyond Pilot:**
- Request phone number limit increase (up to 120)
- Consider WhatsApp Embedded Signup for self-service
- Evaluate tiered pricing (Standard vs. Enterprise)
---
## Support Resources
**Meta Documentation:**
- WhatsApp Cloud API: https://developers.facebook.com/docs/whatsapp/cloud-api
- Getting Started Guide: https://developers.facebook.com/docs/whatsapp/cloud-api/get-started
- Template Guidelines: https://developers.facebook.com/docs/whatsapp/message-templates/guidelines
**Meta Support:**
- Business Support: https://business.whatsapp.com/support
- Developer Community: https://developers.facebook.com/community/
**Internal:**
- Full Implementation Guide: `WHATSAPP_SHARED_ACCOUNT_GUIDE.md`
- Admin Interface: `http://localhost:5173/app/admin/whatsapp`
- API Documentation: `http://localhost:8001/docs#/whatsapp-admin`
---
**Document Version:** 1.0
**Last Updated:** 2025-01-17
**Author:** Platform Engineering Team
**Estimated Setup Time:** 2-3 hours
**Difficulty:** Intermediate

327
docs/03-features/notifications/whatsapp/multi-tenant-implementation.md Normal file
View File

@@ -0,0 +1,327 @@
# Multi-Tenant WhatsApp Configuration - Implementation Summary
## Overview
This implementation allows each bakery (tenant) to configure their own WhatsApp Business credentials in the settings UI, enabling them to send notifications to suppliers using their own WhatsApp Business phone number.
## ✅ COMPLETED WORK
### Phase 1: Backend - Tenant Service ✅
#### 1. Database Schema
**File**: `services/tenant/app/models/tenant_settings.py`
- Added `notification_settings` JSON column to store WhatsApp and email configuration
- Includes fields: `whatsapp_enabled`, `whatsapp_phone_number_id`, `whatsapp_access_token`, `whatsapp_business_account_id`, etc.
#### 2. Pydantic Schemas
**File**: `services/tenant/app/schemas/tenant_settings.py`
- Created `NotificationSettings` schema with validation
- Added validators for required fields when WhatsApp is enabled
#### 3. Service Layer
**File**: `services/tenant/app/services/tenant_settings_service.py`
- Added "notification" category support
- Mapped notification category to `notification_settings` column
#### 4. Database Migration
**File**: `services/tenant/migrations/versions/002_add_notification_settings.py`
- Created migration to add `notification_settings` column with default values
- All existing tenants get default settings automatically
### Phase 2: Backend - Notification Service ✅
#### 1. Tenant Service Client
**File**: `shared/clients/tenant_client.py`
- Added `get_notification_settings(tenant_id)` method
- Fetches notification settings via HTTP from Tenant Service
#### 2. WhatsApp Business Service
**File**: `services/notification/app/services/whatsapp_business_service.py`
**Changes:**
- Modified `__init__` to accept `tenant_client` parameter
- Renamed global config to `global_access_token`, `global_phone_number_id`, etc.
- Added `_get_whatsapp_credentials(tenant_id)` method:
- Fetches tenant notification settings
- Checks if `whatsapp_enabled` is True
- Returns tenant credentials if configured
- Falls back to global config if not configured or incomplete
- Updated `send_message()` to call `_get_whatsapp_credentials()` for each message
- Modified `_send_template_message()` and `_send_text_message()` to accept credentials as parameters
- Updated `health_check()` to use global credentials
#### 3. WhatsApp Service Wrapper
**File**: `services/notification/app/services/whatsapp_service.py`
- Modified `__init__` to accept `tenant_client` parameter
- Passes `tenant_client` to `WhatsAppBusinessService`
#### 4. Service Initialization
**File**: `services/notification/app/main.py`
- Added import for `TenantServiceClient`
- Initialize `TenantServiceClient` in `on_startup()`
- Pass `tenant_client` to `WhatsAppService` initialization
### Phase 3: Frontend - TypeScript Types ✅
#### 1. Settings Types
**File**: `frontend/src/api/types/settings.ts`
- Created `NotificationSettings` interface
- Added to `TenantSettings` interface
- Added to `TenantSettingsUpdate` interface
- Added 'notification' to `SettingsCategory` type
### Phase 4: Frontend - Component ✅
#### 1. Notification Settings Card
**File**: `frontend/src/pages/app/database/ajustes/cards/NotificationSettingsCard.tsx`
- Complete UI component with sections for:
- WhatsApp Configuration (credentials, API version, language)
- Email Configuration (from address, name, reply-to)
- Notification Preferences (PO, inventory, production, forecast alerts)
- Channel selection (email/WhatsApp) for each notification type
- Includes helpful setup instructions for WhatsApp Business
- Responsive design with proper styling
### Phase 5: Frontend - Translations ✅
#### 1. Spanish Translations
**Files**:
- `frontend/src/locales/es/ajustes.json` - notification section added
- `frontend/src/locales/es/settings.json` - "notifications" tab added
#### 2. Basque Translations
**Files**:
- `frontend/src/locales/eu/ajustes.json` - notification section added
- `frontend/src/locales/eu/settings.json` - "notifications" tab added
**Translation Keys Added:**
- `notification.title`
- `notification.whatsapp_config`
- `notification.whatsapp_enabled`
- `notification.whatsapp_phone_number_id` (+ `_help`)
- `notification.whatsapp_access_token` (+ `_help`)
- `notification.whatsapp_business_account_id` (+ `_help`)
- `notification.whatsapp_api_version`
- `notification.whatsapp_default_language`
- `notification.whatsapp_setup_note/step1/step2/step3`
- `notification.email_config`
- `notification.email_enabled`
- `notification.email_from_address/name/reply_to`
- `notification.preferences`
- `notification.enable_po_notifications/inventory_alerts/production_alerts/forecast_alert s`
- `bakery.tabs.notifications`
## 📋 REMAINING WORK
### Frontend - BakerySettingsPage Integration
**File**: `frontend/src/pages/app/settings/bakery/BakerySettingsPage.tsx`
**Changes needed** (see `FRONTEND_CHANGES_NEEDED.md` for detailed instructions):
1. Add `Bell` icon to imports
2. Import `NotificationSettings` type
3. Import `NotificationSettingsCard` component
4. Add `notificationSettings` state variable
5. Load notification settings in useEffect
6. Add notifications tab trigger
7. Add notifications tab content
8. Update `handleSaveOperationalSettings` validation
9. Add `notification_settings` to mutation
10. Update `handleDiscard` function
11. Update floating save button condition
**Estimated time**: 15 minutes
## 🔄 How It Works
### Message Flow
1. **PO Event Triggered**: When a purchase order is approved, an event is published to RabbitMQ
2. **Event Consumed**: Notification service receives the event with `tenant_id` and supplier information
3. **Credentials Lookup**:
- `WhatsAppBusinessService._get_whatsapp_credentials(tenant_id)` is called
- Fetches notification settings from Tenant Service via HTTP
- Checks if `whatsapp_enabled` is `True`
- If tenant has WhatsApp enabled AND credentials configured → uses tenant credentials
- Otherwise → falls back to global environment variable credentials
4. **Message Sent**: Uses resolved credentials to send message via Meta WhatsApp API
5. **Logging**: Logs which credentials were used (tenant-specific or global)
### Configuration Levels
**Global (Fallback)**:
- Environment variables: `WHATSAPP_ACCESS_TOKEN`, `WHATSAPP_PHONE_NUMBER_ID`, etc.
- Used when tenant settings are not configured or WhatsApp is disabled
- Configured at deployment time
**Per-Tenant (Primary)**:
- Stored in `tenant_settings.notification_settings` JSON column
- Configured through UI in Bakery Settings → Notifications tab
- Each tenant can have their own WhatsApp Business credentials
- Takes precedence over global config when enabled and configured
### Backward Compatibility
✅ Existing code continues to work without changes
✅ PO event consumer already passes `tenant_id` - no changes needed
✅ Falls back gracefully to global config if tenant settings not configured
✅ Migration adds default settings to existing tenants automatically
## 📊 Testing Checklist
### Backend Testing
- [ ] Run tenant service migration: `cd services/tenant && alembic upgrade head`
- [ ] Verify `notification_settings` column exists in `tenant_settings` table
- [ ] Test API endpoint: `GET /api/v1/tenants/{tenant_id}/settings/notification`
- [ ] Test API endpoint: `PUT /api/v1/tenants/{tenant_id}/settings/notification`
- [ ] Verify notification service starts successfully with tenant_client
- [ ] Send test WhatsApp message with tenant credentials
- [ ] Send test WhatsApp message without tenant credentials (fallback)
- [ ] Check logs for "Using tenant-specific WhatsApp credentials" message
- [ ] Check logs for "Using global WhatsApp credentials" message
### Frontend Testing
- [ ] Apply BakerySettingsPage changes
- [ ] Navigate to Settings → Bakery Settings
- [ ] Verify "Notifications" tab appears
- [ ] Click Notifications tab
- [ ] Verify NotificationSettingsCard renders correctly
- [ ] Toggle "Enable WhatsApp" checkbox
- [ ] Verify credential fields appear/disappear
- [ ] Fill in WhatsApp credentials
- [ ] Verify helper text appears correctly
- [ ] Verify setup instructions appear
- [ ] Toggle notification preferences
- [ ] Verify channel checkboxes (Email/WhatsApp)
- [ ] WhatsApp channel checkbox should be disabled when WhatsApp not enabled
- [ ] Click Save button
- [ ] Verify success toast appears
- [ ] Refresh page and verify settings persist
- [ ] Test in both Spanish and Basque languages
### Integration Testing
- [ ] Configure tenant WhatsApp credentials via UI
- [ ] Create a purchase order for a supplier with phone number
- [ ] Approve the purchase order
- [ ] Verify WhatsApp message is sent using tenant credentials
- [ ] Check logs confirm tenant credentials were used
- [ ] Disable tenant WhatsApp in UI
- [ ] Approve another purchase order
- [ ] Verify message uses global credentials (fallback)
- [ ] Re-enable tenant WhatsApp
- [ ] Remove credentials (leave fields empty)
- [ ] Verify fallback to global credentials
## 🔒 Security Considerations
### Current Implementation
- ✅ Credentials stored in database (PostgreSQL JSONB)
- ✅ Access controlled by tenant isolation
- ✅ Only admin/owner roles can modify settings
- ✅ HTTPS required for API communication
- ✅ Password input type for access token field
### Future Enhancements (Recommended)
- [ ] Implement field-level encryption for `whatsapp_access_token`
- [ ] Add audit logging for credential changes
- [ ] Implement credential rotation mechanism
- [ ] Add "Test Connection" button to verify credentials
- [ ] Rate limiting on settings updates
- [ ] Alert on failed message sends
## 📚 Documentation
### Existing Documentation
- ✅ `services/notification/WHATSAPP_SETUP_GUIDE.md` - WhatsApp Business setup guide
- ✅ `services/notification/WHATSAPP_TEMPLATE_EXAMPLE.md` - Template creation guide
- ✅ `services/notification/WHATSAPP_QUICK_REFERENCE.md` - Quick reference
- ✅ `services/notification/MULTI_TENANT_WHATSAPP_IMPLEMENTATION.md` - Implementation details
### Documentation Updates Needed
- [ ] Update `WHATSAPP_SETUP_GUIDE.md` with per-tenant configuration instructions
- [ ] Add screenshots of UI settings page
- [ ] Document fallback behavior
- [ ] Add troubleshooting section for tenant-specific credentials
- [ ] Update API documentation with new tenant settings endpoint
## 🚀 Deployment Steps
### 1. Backend Deployment
```bash
# 1. Deploy tenant service changes
cd services/tenant
alembic upgrade head
kubectl apply -f kubernetes/tenant-deployment.yaml
# 2. Deploy notification service changes
cd services/notification
kubectl apply -f kubernetes/notification-deployment.yaml
# 3. Verify services are running
kubectl get pods -n bakery-ia
kubectl logs -f deployment/tenant-service -n bakery-ia
kubectl logs -f deployment/notification-service -n bakery-ia
```
### 2. Frontend Deployment
```bash
# 1. Apply BakerySettingsPage changes (see FRONTEND_CHANGES_NEEDED.md)
# 2. Build frontend
cd frontend
npm run build
# 3. Deploy
kubectl apply -f kubernetes/frontend-deployment.yaml
```
### 3. Verification
```bash
# Check database
psql -d tenant_db -c "SELECT tenant_id, notification_settings->>'whatsapp_enabled' FROM tenant_settings;"
# Check logs
kubectl logs -f deployment/notification-service -n bakery-ia | grep -i whatsapp
# Test message send
curl -X POST http://localhost:8000/api/v1/test-whatsapp \
-H "Content-Type: application/json" \
-d '{"tenant_id": "xxx", "phone": "+34612345678"}'
```
## 📞 Support
For questions or issues:
- Check logs: `kubectl logs deployment/notification-service -n bakery-ia`
- Review documentation in `services/notification/`
- Verify credentials in Meta Business Suite
- Test with global credentials first, then tenant credentials
## ✅ Success Criteria
Implementation is complete when:
- ✅ Backend can fetch tenant notification settings
- ✅ Backend uses tenant credentials when configured
- ✅ Backend falls back to global credentials when needed
- ✅ UI displays notification settings tab
- ✅ Users can configure WhatsApp credentials
- ✅ Settings save and persist correctly
- ✅ Messages sent using tenant-specific credentials
- ✅ Logs confirm credential selection
- ✅ All translations work in Spanish and Basque
- ✅ Backward compatibility maintained
---
**Implementation Status**: 95% Complete (Frontend integration remaining)
**Last Updated**: 2025-11-13

750
docs/03-features/notifications/whatsapp/shared-account-guide.md Normal file
View File

@@ -0,0 +1,750 @@
# WhatsApp Shared Account Model - Implementation Guide
## Overview
This guide documents the **Shared WhatsApp Business Account** implementation for the bakery-ia pilot program. This model simplifies WhatsApp setup by using a single master WhatsApp Business Account with phone numbers assigned to each bakery tenant.
---
## Architecture
### Shared Account Model
```
┌─────────────────────────────────────────────┐
│ Master WhatsApp Business Account (WABA) │
│ - Centrally managed by platform admin │
│ - Single set of credentials │
│ - Multiple phone numbers (up to 120) │
└─────────────────────────────────────────────┘
│
┌─────────────┼─────────────┐
│ │ │
Phone #1 Phone #2 Phone #3
Bakery A Bakery B Bakery C
```
### Key Benefits
✅ **Zero configuration for bakery users** - No Meta navigation required
✅ **5-minute setup** - Admin assigns phone number via UI
✅ **Lower support burden** - Centralized management
✅ **Predictable costs** - One WABA subscription
✅ **Perfect for pilot** - Quick deployment for 10 bakeries
---
## User Experience
### For Bakery Owners (Non-Technical Users)
**Before (Manual Setup):**
- Navigate Meta Business Suite ❌
- Create WhatsApp Business Account ❌
- Create message templates ❌
- Get credentials (3 different IDs) ❌
- Copy/paste into settings ❌
- **Time:** 1-2 hours, high error rate
**After (Shared Account):**
- Toggle WhatsApp ON ✓
- See assigned phone number ✓
- **Time:** 30 seconds, zero configuration
### For Platform Admin
**Admin Workflow:**
1. Access WhatsApp Admin page (`/app/admin/whatsapp`)
2. View list of tenants
3. Select tenant
4. Assign phone number from dropdown
5. Done!
---
## Technical Implementation
### Backend Changes
#### 1. Tenant Settings Model
**File:** `services/tenant/app/models/tenant_settings.py`
**Changed:**
```python
# OLD (Per-Tenant Credentials)
notification_settings = {
"whatsapp_enabled": False,
"whatsapp_phone_number_id": "",
"whatsapp_access_token": "", # REMOVED
"whatsapp_business_account_id": "", # REMOVED
"whatsapp_api_version": "v18.0", # REMOVED
"whatsapp_default_language": "es"
}
# NEW (Shared Account)
notification_settings = {
"whatsapp_enabled": False,
"whatsapp_phone_number_id": "", # Phone # from shared account
"whatsapp_display_phone_number": "", # Display format "+34 612 345 678"
"whatsapp_default_language": "es"
}
```
#### 2. WhatsApp Business Service
**File:** `services/notification/app/services/whatsapp_business_service.py`
**Changed `_get_whatsapp_credentials()` method:**
```python
async def _get_whatsapp_credentials(self, tenant_id: str) -> Dict[str, str]:
"""
Uses global master account credentials with tenant-specific phone number
"""
# Always use global master account
access_token = self.global_access_token
business_account_id = self.global_business_account_id
phone_number_id = self.global_phone_number_id # Default
# Fetch tenant's assigned phone number
if self.tenant_client:
notification_settings = await self.tenant_client.get_notification_settings(tenant_id)
if notification_settings and notification_settings.get('whatsapp_enabled'):
tenant_phone_id = notification_settings.get('whatsapp_phone_number_id', '')
if tenant_phone_id:
phone_number_id = tenant_phone_id # Use tenant's phone
return {
'access_token': access_token,
'phone_number_id': phone_number_id,
'business_account_id': business_account_id
}
```
**Key Change:** Always uses global credentials, but selects the phone number based on tenant assignment.
#### 3. Phone Number Management API
**New File:** `services/tenant/app/api/whatsapp_admin.py`
**Endpoints:**
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/v1/admin/whatsapp/phone-numbers` | List available phone numbers from master WABA |
| GET | `/api/v1/admin/whatsapp/tenants` | List all tenants with WhatsApp status |
| POST | `/api/v1/admin/whatsapp/tenants/{id}/assign-phone` | Assign phone to tenant |
| DELETE | `/api/v1/admin/whatsapp/tenants/{id}/unassign-phone` | Remove phone assignment |
**Example: Assign Phone Number**
```bash
curl -X POST http://localhost:8001/api/v1/admin/whatsapp/tenants/{tenant_id}/assign-phone \
-H "Content-Type: application/json" \
-d '{
"phone_number_id": "123456789012345",
"display_phone_number": "+34 612 345 678"
}'
```
**Response:**
```json
{
"success": true,
"message": "Phone number +34 612 345 678 assigned to tenant 'Panadería San Juan'",
"tenant_id": "uuid-here",
"phone_number_id": "123456789012345",
"display_phone_number": "+34 612 345 678"
}
```
### Frontend Changes
#### 1. Simplified Notification Settings Card
**File:** `frontend/src/pages/app/database/ajustes/cards/NotificationSettingsCard.tsx`
**Removed:**
- Access Token input field
- Business Account ID input field
- Phone Number ID input field
- API Version selector
- Setup wizard instructions
**Added:**
- Display-only phone number (green badge if configured)
- "Contact support" message if not configured
- Language selector only
**UI Before/After:**
```
BEFORE:
┌────────────────────────────────────────┐
│ WhatsApp Business API Configuration │
│ │
│ Phone Number ID: [____________] │
│ Access Token: [____________] │
│ Business Acct: [____________] │
│ API Version: [v18.0 ▼] │
│ Language: [Español ▼] │
│ │
│ ℹ️ Setup Instructions: │
│ 1. Create WhatsApp Business... │
│ 2. Create templates... │
│ 3. Get credentials... │
└────────────────────────────────────────┘
AFTER:
┌────────────────────────────────────────┐
│ WhatsApp Configuration │
│ │
│ ✅ WhatsApp Configured │
│ Phone: +34 612 345 678 │
│ │
│ Language: [Español ▼] │
│ │
│ ℹ️ WhatsApp Notifications Included │
│ WhatsApp messaging is included │
│ in your subscription. │
└────────────────────────────────────────┘
```
#### 2. Admin Interface
**New File:** `frontend/src/pages/app/admin/WhatsAppAdminPage.tsx`
**Features:**
- Lists all available phone numbers from master WABA
- Shows phone number quality rating (GREEN/YELLOW/RED)
- Lists all tenants with WhatsApp status
- Dropdown to assign phone numbers
- One-click unassign button
- Real-time status updates
**Screenshot Mockup:**
```
┌──────────────────────────────────────────────────────────────┐
│ WhatsApp Admin Management │
│ Assign WhatsApp phone numbers to bakery tenants │
├──────────────────────────────────────────────────────────────┤
│ 📞 Available Phone Numbers (3) │
├──────────────────────────────────────────────────────────────┤
│ +34 612 345 678 Bakery Platform [GREEN] │
│ +34 612 345 679 Bakery Support [GREEN] │
│ +34 612 345 680 Bakery Notifications [YELLOW] │
└──────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────┐
│ 👥 Bakery Tenants (10) │
├──────────────────────────────────────────────────────────────┤
│ Panadería San Juan ✅ Active │
│ Phone: +34 612 345 678 [Unassign] │
├──────────────────────────────────────────────────────────────┤
│ Panadería Goiko ⚠️ Not Configured │
│ No phone number assigned [Assign phone number... ▼] │
└──────────────────────────────────────────────────────────────┘
```
---
## Setup Instructions
### Step 1: Create Master WhatsApp Business Account (One-Time)
**Prerequisites:**
- Meta/Facebook Business account
- Verified business
- Phone number(s) to register
**Instructions:**
1. **Create WhatsApp Business Account**
- Go to [Meta Business Suite](https://business.facebook.com)
- Add WhatsApp product
- Complete business verification (1-3 days)
2. **Add Phone Numbers**
- Add at least 10 phone numbers (one per pilot bakery)
- Verify each phone number
- Note: You can request up to 120 phone numbers per WABA
3. **Create Message Templates**
- Create `po_notification` template:
```
Category: UTILITY
Language: Spanish (es)
Message: "Hola {{1}}, has recibido una nueva orden de compra {{2}} por un total de {{3}}."
```
- Submit for approval (15 min - 24 hours)
4. **Get Master Credentials**
- Business Account ID: From WhatsApp Manager settings
- Access Token: Create System User or use temporary token
- Phone Number ID: Listed in phone numbers section
### Step 2: Configure Environment Variables
**File:** `services/notification/.env`
```bash
# Master WhatsApp Business Account Credentials
WHATSAPP_BUSINESS_ACCOUNT_ID=987654321098765
WHATSAPP_ACCESS_TOKEN=EAAxxxxxxxxxxxxxxxxxxxxxxxxxx
WHATSAPP_PHONE_NUMBER_ID=123456789012345 # Default/fallback phone
WHATSAPP_API_VERSION=v18.0
ENABLE_WHATSAPP_NOTIFICATIONS=true
WHATSAPP_WEBHOOK_VERIFY_TOKEN=random-secret-token-here
```
**Security Notes:**
- Store `WHATSAPP_ACCESS_TOKEN` securely (use secrets manager in production)
- Rotate token every 60 days
- Use System User token (not temporary token) for production
### Step 3: Assign Phone Numbers to Tenants
**Via Admin UI:**
1. Access admin page: `http://localhost:5173/app/admin/whatsapp`
2. See list of tenants
3. For each tenant:
- Select phone number from dropdown
- Click assign
- Verify green checkmark appears
**Via API:**
```bash
# Assign phone to tenant
curl -X POST http://localhost:8001/api/v1/admin/whatsapp/tenants/{tenant_id}/assign-phone \
-H "Content-Type: application/json" \
-d '{
"phone_number_id": "123456789012345",
"display_phone_number": "+34 612 345 678"
}'
```
### Step 4: Test Notifications
**Enable WhatsApp for a Tenant:**
1. Login as bakery owner
2. Go to Settings → Notifications
3. Toggle WhatsApp ON
4. Verify phone number is displayed
5. Save settings
**Trigger Test Notification:**
```bash
# Create a purchase order (will trigger WhatsApp notification)
curl -X POST http://localhost:8003/api/v1/orders/purchase-orders \
-H "Content-Type: application/json" \
-H "X-Tenant-ID: {tenant_id}" \
-d '{
"supplier_id": "uuid",
"items": [...]
}'
```
**Verify:**
- Check notification service logs: `docker logs -f notification-service`
- Supplier should receive WhatsApp message from assigned phone number
- Message status tracked in `whatsapp_messages` table
---
## Monitoring & Operations
### Check Phone Number Usage
```bash
# List all tenants with assigned phone numbers
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | jq
```
### View WhatsApp Message Logs
```sql
-- In notification database
SELECT
tenant_id,
recipient_phone,
template_name,
status,
created_at,
error_message
FROM whatsapp_messages
WHERE created_at > NOW() - INTERVAL '24 hours'
ORDER BY created_at DESC;
```
### Monitor Meta Rate Limits
WhatsApp Cloud API has the following limits:
| Metric | Limit |
|--------|-------|
| Messages per second | 80 |
| Messages per day (verified) | 100,000 |
| Messages per day (unverified) | 1,000 |
| Conversations per 24h | Unlimited (pay per conversation) |
**Check Quality Rating:**
```bash
curl -X GET "https://graph.facebook.com/v18.0/{PHONE_NUMBER_ID}" \
-H "Authorization: Bearer {ACCESS_TOKEN}" \
| jq '.quality_rating'
```
**Quality Ratings:**
- **GREEN** - No issues, full limits
- **YELLOW** - Warning, limits may be reduced
- **RED** - Quality issues, severely restricted
---
## Migration from Per-Tenant to Shared Account
If you have existing tenants with their own credentials:
### Automatic Migration Script
```python
# services/tenant/scripts/migrate_to_shared_account.py
"""
Migrate existing tenant WhatsApp credentials to shared account model
"""
import asyncio
from sqlalchemy import select
from app.core.database import database_manager
from app.models.tenant_settings import TenantSettings
async def migrate():
async with database_manager.get_session() as session:
# Get all tenant settings
result = await session.execute(select(TenantSettings))
all_settings = result.scalars().all()
for settings in all_settings:
notification_settings = settings.notification_settings
# If tenant has old credentials, preserve phone number ID
if notification_settings.get('whatsapp_access_token'):
phone_id = notification_settings.get('whatsapp_phone_number_id', '')
# Update to new schema
notification_settings['whatsapp_phone_number_id'] = phone_id
notification_settings['whatsapp_display_phone_number'] = '' # Admin will set
# Remove old fields
notification_settings.pop('whatsapp_access_token', None)
notification_settings.pop('whatsapp_business_account_id', None)
notification_settings.pop('whatsapp_api_version', None)
settings.notification_settings = notification_settings
print(f"Migrated tenant: {settings.tenant_id}")
await session.commit()
print("Migration complete!")
if __name__ == "__main__":
asyncio.run(migrate())
```
---
## Troubleshooting
### Issue: Tenant doesn't receive WhatsApp messages
**Checklist:**
1. ✅ WhatsApp enabled in tenant settings?
2. ✅ Phone number assigned to tenant?
3. ✅ Master credentials configured in environment?
4. ✅ Template approved by Meta?
5. ✅ Recipient phone number in E.164 format (+34612345678)?
**Check Logs:**
```bash
# Notification service logs
docker logs -f notification-service | grep whatsapp
# Look for:
# - "Using tenant-assigned WhatsApp phone number"
# - "WhatsApp template message sent successfully"
# - Any error messages
```
### Issue: Phone number assignment fails
**Error:** "Phone number already assigned to another tenant"
**Solution:**
```bash
# Find which tenant has the phone number
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | \
jq '.[] | select(.phone_number_id == "123456789012345")'
# Unassign from old tenant first
curl -X DELETE http://localhost:8001/api/v1/admin/whatsapp/tenants/{old_tenant_id}/unassign-phone
```
### Issue: "WhatsApp master account not configured"
**Solution:**
Ensure environment variables are set:
```bash
# Check if variables exist
docker exec notification-service env | grep WHATSAPP
# Should show:
# WHATSAPP_BUSINESS_ACCOUNT_ID=...
# WHATSAPP_ACCESS_TOKEN=...
# WHATSAPP_PHONE_NUMBER_ID=...
```
### Issue: Template not found
**Error:** "Template po_notification not found"
**Solution:**
1. Create template in Meta Business Manager
2. Wait for approval (check status):
```bash
curl -X GET "https://graph.facebook.com/v18.0/{WABA_ID}/message_templates" \
-H "Authorization: Bearer {TOKEN}" \
| jq '.data[] | select(.name == "po_notification")'
```
3. Ensure template language matches tenant's `whatsapp_default_language`
---
## Cost Analysis
### WhatsApp Business API Pricing (as of 2024)
**Meta Pricing:**
- **Business-initiated conversations:** €0.0319 - €0.0699 per conversation (Spain)
- **User-initiated conversations:** Free (24-hour window)
- **Conversation window:** 24 hours
**Monthly Cost Estimate (10 Bakeries):**
- Assume 5 PO notifications per bakery per day
- 5 × 10 bakeries × 30 days = 1,500 messages/month
- Cost: 1,500 × €0.05 = **€75/month**
**Shared Account vs. Individual Accounts:**
| Model | Setup Time | Monthly Cost | Support Burden |
|-------|------------|--------------|----------------|
| Individual Accounts | 1-2 hrs/bakery | €75 total | High |
| Shared Account | 5 min/bakery | €75 total | Low |
**Savings:** Time savings = 10 hrs × €50/hr = **€500 in setup cost**
---
## Future Enhancements
### Option 1: Template Management API
Automate template creation for new tenants:
```python
async def create_po_template(waba_id: str, access_token: str):
"""Programmatically create PO notification template"""
url = f"https://graph.facebook.com/v18.0/{waba_id}/message_templates"
payload = {
"name": "po_notification",
"language": "es",
"category": "UTILITY",
"components": [{
"type": "BODY",
"text": "Hola {{1}}, has recibido una nueva orden de compra {{2}} por un total de {{3}}."
}]
}
response = await httpx.post(url, headers={"Authorization": f"Bearer {access_token}"}, json=payload)
return response.json()
```
### Option 2: WhatsApp Embedded Signup
For scaling beyond pilot:
- Apply for Meta Business Solution Provider program
- Implement OAuth-style signup flow
- Users click "Connect WhatsApp" → auto-configured
- Estimated implementation: 2-4 weeks
### Option 3: Tiered Pricing
```
Basic Tier (Free):
- Email notifications only
Standard Tier (€29/month):
- Shared WhatsApp account
- Pre-approved templates
- Up to 500 messages/month
Enterprise Tier (€99/month):
- Own WhatsApp Business Account
- Custom templates
- Unlimited messages
- White-label phone number
```
---
## Security & Compliance
### Data Privacy
**GDPR Compliance:**
- WhatsApp messages contain supplier contact info (phone numbers)
- Ensure GDPR consent for sending notifications
- Provide opt-out mechanism
- Data retention: Messages stored for 90 days (configurable)
**Encryption:**
- WhatsApp messages: End-to-end encrypted by Meta
- Access tokens: Stored in environment variables (use secrets manager in production)
- Database: Encrypt `notification_settings` JSON column
### Access Control
**Admin Access:**
- Only platform admins can assign/unassign phone numbers
- Implement role-based access control (RBAC)
- Audit log for phone number assignments
```python
# Example: Add admin check
@router.post("/admin/whatsapp/tenants/{tenant_id}/assign-phone")
async def assign_phone(tenant_id: UUID, current_user = Depends(require_admin_role)):
# Only admins can access
pass
```
---
## Support & Contacts
**Meta Support:**
- WhatsApp Business API Support: https://business.whatsapp.com/support
- Developer Docs: https://developers.facebook.com/docs/whatsapp
**Platform Admin:**
- Email: admin@bakery-platform.com
- Phone number assignment requests
- Template approval assistance
**Bakery Owner Help:**
- Settings → Notifications → Toggle WhatsApp ON
- If phone number not showing: Contact support
- Language preferences can be changed anytime
---
## Appendix
### A. Database Schema Changes
**Migration Script:**
```sql
-- Add new field, remove old fields
-- services/tenant/migrations/versions/00002_shared_whatsapp_account.py
ALTER TABLE tenant_settings
-- The notification_settings JSONB column now has:
-- + whatsapp_display_phone_number (new)
-- - whatsapp_access_token (removed)
-- - whatsapp_business_account_id (removed)
-- - whatsapp_api_version (removed)
;
-- No ALTER TABLE needed (JSONB is schema-less)
-- Migration handled by application code
```
### B. API Reference
**Phone Number Info Schema:**
```typescript
interface WhatsAppPhoneNumberInfo {
id: string; // Meta Phone Number ID
display_phone_number: string; // E.164 format: +34612345678
verified_name: string; // Business name verified by Meta
quality_rating: string; // GREEN, YELLOW, RED
}
```
**Tenant WhatsApp Status Schema:**
```typescript
interface TenantWhatsAppStatus {
tenant_id: string;
tenant_name: string;
whatsapp_enabled: boolean;
phone_number_id: string | null;
display_phone_number: string | null;
}
```
### C. Environment Variables Reference
```bash
# Notification Service (services/notification/.env)
WHATSAPP_BUSINESS_ACCOUNT_ID= # Meta WABA ID
WHATSAPP_ACCESS_TOKEN= # Meta System User Token
WHATSAPP_PHONE_NUMBER_ID= # Default phone (fallback)
WHATSAPP_API_VERSION=v18.0 # Meta API version
ENABLE_WHATSAPP_NOTIFICATIONS=true
WHATSAPP_WEBHOOK_VERIFY_TOKEN= # Random secret for webhook verification
```
### D. Useful Commands
```bash
# View all available phone numbers
curl http://localhost:8001/api/v1/admin/whatsapp/phone-numbers | jq
# View tenant WhatsApp status
curl http://localhost:8001/api/v1/admin/whatsapp/tenants | jq
# Assign phone to tenant
curl -X POST http://localhost:8001/api/v1/admin/whatsapp/tenants/{id}/assign-phone \
-H "Content-Type: application/json" \
-d '{"phone_number_id": "XXX", "display_phone_number": "+34 612 345 678"}'
# Unassign phone from tenant
curl -X DELETE http://localhost:8001/api/v1/admin/whatsapp/tenants/{id}/unassign-phone
# Test WhatsApp connectivity
curl -X GET "https://graph.facebook.com/v18.0/{PHONE_ID}" \
-H "Authorization: Bearer {TOKEN}"
# Check message template status
curl "https://graph.facebook.com/v18.0/{WABA_ID}/message_templates?fields=name,status,language" \
-H "Authorization: Bearer {TOKEN}" | jq
```
---
**Document Version:** 1.0
**Last Updated:** 2025-01-17
**Author:** Platform Engineering Team
**Status:** Production Ready for Pilot

585
docs/03-features/specifications/poi-detection-system.md Normal file
View File

@@ -0,0 +1,585 @@
# POI Detection System - Implementation Documentation
## Overview
The POI (Point of Interest) Detection System is a comprehensive location-based feature engineering solution for bakery demand forecasting. It automatically detects nearby points of interest (schools, offices, transport hubs, competitors, etc.) and generates ML features that improve prediction accuracy for location-specific demand patterns.
## System Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Bakery SaaS Platform │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ External Data Service (POI MODULE) │ │
│ ├──────────────────────────────────────────────────────────┤ │
│ │ POI Detection Service → Overpass API (OpenStreetMap) │ │
│ │ POI Feature Selector → Relevance Filtering │ │
│ │ Competitor Analyzer → Competitive Pressure Modeling │ │
│ │ POI Cache Service → Redis (90-day TTL) │ │
│ │ TenantPOIContext → PostgreSQL Storage │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ │ POI Features │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Training Service (ENHANCED) │ │
│ ├──────────────────────────────────────────────────────────┤ │
│ │ Training Data Orchestrator → Fetches POI Features │ │
│ │ Data Processor → Merges POI Features into Training Data │ │
│ │ Prophet + XGBoost Trainer → Uses POI as Regressors │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │ │
│ │ Trained Models │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Forecasting Service (ENHANCED) │ │
│ ├──────────────────────────────────────────────────────────┤ │
│ │ POI Feature Service → Fetches POI Features │ │
│ │ Prediction Engine → Uses Same POI Features as Training │ │
│ └──────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
## Implementation Status
### ✅ Phase 1: Core POI Detection Infrastructure (COMPLETED)
**Files Created:**
- `services/external/app/models/poi_context.py` - POI context data model
- `services/external/app/core/poi_config.py` - POI categories and configuration
- `services/external/app/services/poi_detection_service.py` - POI detection via Overpass API
- `services/external/app/services/poi_feature_selector.py` - Feature relevance filtering
- `services/external/app/services/competitor_analyzer.py` - Competitive pressure analysis
- `services/external/app/cache/poi_cache_service.py` - Redis caching layer
- `services/external/app/repositories/poi_context_repository.py` - Data access layer
- `services/external/app/api/poi_context.py` - REST API endpoints
- `services/external/app/core/redis_client.py` - Redis client accessor
- `services/external/migrations/versions/20251110_1554_add_poi_context.py` - Database migration
**Files Modified:**
- `services/external/app/main.py` - Added POI router and table
- `services/external/requirements.txt` - Added overpy dependency
**Key Features:**
- 9 POI categories: schools, offices, gyms/sports, residential, tourism, competitors, transport hubs, coworking, retail
- Research-based search radii (400m-1000m) per category
- Multi-tier feature engineering:
- Tier 1: Proximity-weighted scores (PRIMARY)
- Tier 2: Distance band counts (0-100m, 100-300m, 300-500m, 500-1000m)
- Tier 3: Distance to nearest POI
- Tier 4: Binary flags
- Feature relevance thresholds to filter low-signal features
- Competitive pressure modeling with market classification
- 90-day Redis cache with 180-day refresh cycle
- Complete REST API for detection, retrieval, refresh, deletion
### ✅ Phase 2: ML Training Pipeline Integration (COMPLETED)
**Files Created:**
- `services/training/app/ml/poi_feature_integrator.py` - POI feature integration for training
**Files Modified:**
- `services/training/app/services/training_orchestrator.py`:
- Added `poi_features` to `TrainingDataSet`
- Added `POIFeatureIntegrator` initialization
- Modified `_collect_external_data` to fetch POI features concurrently
- Added `_collect_poi_features` method
- Updated `TrainingDataSet` creation to include POI features
- `services/training/app/ml/data_processor.py`:
- Added `poi_features` parameter to `prepare_training_data`
- Added `_add_poi_features` method
- Integrated POI features into training data preparation flow
- Added `poi_features` parameter to `prepare_prediction_features`
- Added POI features to prediction feature generation
- `services/training/app/ml/trainer.py`:
- Updated training calls to pass `poi_features` from `training_dataset`
- Updated test data preparation to include POI features
**Key Features:**
- Automatic POI feature fetching during training data preparation
- POI features added as static columns (broadcast to all dates)
- Concurrent fetching with weather and traffic data
- Graceful fallback if POI service unavailable
- Feature consistency between training and testing
### ✅ Phase 3: Forecasting Service Integration (COMPLETED)
**Files Created:**
- `services/forecasting/app/services/poi_feature_service.py` - POI feature service for forecasting
**Files Modified:**
- `services/forecasting/app/ml/predictor.py`:
- Added `POIFeatureService` initialization
- Modified `_prepare_prophet_dataframe` to fetch POI features
- Ensured feature parity between training and prediction
**Key Features:**
- POI features fetched from External service for each prediction
- Same POI features used in both training and prediction (consistency)
- Automatic feature retrieval based on tenant_id
- Graceful handling of missing POI context
### ✅ Phase 4: Frontend POI Visualization (COMPLETED)
**Status:** Complete frontend implementation with geocoding and visualization
**Files Created:**
- `frontend/src/types/poi.ts` - Complete TypeScript type definitions with POI_CATEGORY_METADATA
- `frontend/src/services/api/poiContextApi.ts` - API client for POI operations
- `frontend/src/services/api/geocodingApi.ts` - Geocoding API client (Nominatim)
- `frontend/src/hooks/usePOIContext.ts` - React hook for POI state management
- `frontend/src/hooks/useAddressAutocomplete.ts` - Address autocomplete hook with debouncing
- `frontend/src/components/ui/AddressAutocomplete.tsx` - Reusable address input component
- `frontend/src/components/domain/settings/POIMap.tsx` - Interactive Leaflet map with POI markers
- `frontend/src/components/domain/settings/POISummaryCard.tsx` - POI summary statistics card
- `frontend/src/components/domain/settings/POICategoryAccordion.tsx` - Expandable category details
- `frontend/src/components/domain/settings/POIContextView.tsx` - Main POI management view
- `frontend/src/components/domain/onboarding/steps/POIDetectionStep.tsx` - Onboarding wizard step
**Key Features:**
- Address autocomplete with real-time suggestions (Nominatim API)
- Interactive map with color-coded POI markers by category
- Distance rings visualization (100m, 300m, 500m)
- Detailed category analysis with distance distribution
- Automatic POI detection during onboarding
- POI refresh functionality with competitive insights
- Full TypeScript type safety
- Map with bakery marker at center
- Color-coded POI markers by category
- Distance rings (100m, 300m, 500m)
- Expandable category accordions with details
- Refresh button for manual POI re-detection
- Integration into Settings page and Onboarding wizard
### ✅ Phase 5: Background Refresh Jobs & Geocoding (COMPLETED)
**Status:** Complete implementation of periodic POI refresh and address geocoding
**Files Created (Background Jobs):**
- `services/external/app/models/poi_refresh_job.py` - POI refresh job data model
- `services/external/app/services/poi_refresh_service.py` - POI refresh job management service
- `services/external/app/services/poi_scheduler.py` - Background scheduler for periodic refresh
- `services/external/app/api/poi_refresh_jobs.py` - REST API for job management
- `services/external/migrations/versions/20251110_1801_df9709132952_add_poi_refresh_jobs_table.py` - Database migration
**Files Created (Geocoding):**
- `services/external/app/services/nominatim_service.py` - Nominatim geocoding service
- `services/external/app/api/geocoding.py` - Geocoding REST API endpoints
**Files Modified:**
- `services/external/app/main.py` - Integrated scheduler startup/shutdown, added routers
- `services/external/app/api/poi_context.py` - Auto-schedules refresh job after POI detection
**Key Features - Background Refresh:**
- **Automatic 6-month refresh cycle**: Jobs scheduled 180 days after initial POI detection
- **Hourly scheduler**: Checks for pending jobs every hour and executes them
- **Change detection**: Analyzes differences between old and new POI results
- **Retry logic**: Up to 3 attempts with 1-hour retry delay
- **Concurrent execution**: Configurable max concurrent jobs (default: 5)
- **Job tracking**: Complete audit trail with status, timestamps, results, errors
- **Manual triggers**: API endpoints for immediate job execution
- **Auto-scheduling**: Next refresh automatically scheduled on completion
**Key Features - Geocoding:**
- **Address autocomplete**: Real-time suggestions from Nominatim API
- **Forward geocoding**: Convert address to coordinates
- **Reverse geocoding**: Convert coordinates to address
- **Rate limiting**: Respects 1 req/sec for public Nominatim API
- **Production ready**: Easy switch to self-hosted Nominatim instance
- **Country filtering**: Default to Spain (configurable)
**Background Job API Endpoints:**
- `POST /api/v1/poi-refresh-jobs/schedule` - Schedule a refresh job
- `GET /api/v1/poi-refresh-jobs/{job_id}` - Get job details
- `GET /api/v1/poi-refresh-jobs/tenant/{tenant_id}` - Get tenant's jobs
- `POST /api/v1/poi-refresh-jobs/{job_id}/execute` - Manually execute job
- `GET /api/v1/poi-refresh-jobs/pending` - Get pending jobs
- `POST /api/v1/poi-refresh-jobs/process-pending` - Process all pending jobs
- `POST /api/v1/poi-refresh-jobs/trigger-scheduler` - Trigger immediate scheduler check
- `GET /api/v1/poi-refresh-jobs/scheduler/status` - Get scheduler status
**Geocoding API Endpoints:**
- `GET /api/v1/geocoding/search?q={query}` - Address search/autocomplete
- `GET /api/v1/geocoding/geocode?address={address}` - Forward geocoding
- `GET /api/v1/geocoding/reverse?lat={lat}&lon={lon}` - Reverse geocoding
- `GET /api/v1/geocoding/validate?lat={lat}&lon={lon}` - Coordinate validation
- `GET /api/v1/geocoding/health` - Service health check
**Scheduler Lifecycle:**
- **Startup**: Scheduler automatically starts with External service
- **Runtime**: Runs in background, checking every 3600 seconds (1 hour)
- **Shutdown**: Gracefully stops when service shuts down
- **Immediate check**: Can be triggered via API for testing/debugging
## POI Categories & Configuration
### Detected Categories
| Category | OSM Query | Search Radius | Weight | Impact |
|----------|-----------|---------------|--------|--------|
| **Schools** | `amenity~"school\|kindergarten\|university"` | 500m | 1.5 | Morning drop-off rush |
| **Offices** | `office` | 800m | 1.3 | Weekday lunch demand |
| **Gyms/Sports** | `leisure~"fitness_centre\|sports_centre"` | 600m | 0.8 | Morning/evening activity |
| **Residential** | `building~"residential\|apartments"` | 400m | 1.0 | Base demand |
| **Tourism** | `tourism~"attraction\|museum\|hotel"` | 1000m | 1.2 | Tourist foot traffic |
| **Competitors** | `shop~"bakery\|pastry"` | 1000m | -0.5 | Competition pressure |
| **Transport Hubs** | `railway~"station\|subway_entrance"` | 800m | 1.4 | Commuter traffic |
| **Coworking** | `amenity="coworking_space"` | 600m | 1.1 | Flexible workers |
| **Retail** | `shop` | 500m | 0.9 | General foot traffic |
### Feature Relevance Thresholds
Features are only included in ML models if they pass relevance criteria:
**Example - Schools:**
- `min_proximity_score`: 0.5 (moderate proximity required)
- `max_distance_to_nearest_m`: 500 (must be within 500m)
- `min_count`: 1 (at least 1 school)
If a bakery has no schools within 500m → school features NOT added (prevents noise)
## Feature Engineering Strategy
### Hybrid Multi-Tier Approach
**Research Basis:** Academic studies (2023-2024) show single-method approaches underperform
**Tier 1: Proximity-Weighted Scores (PRIMARY)**
```python
proximity_score = Σ(1 / (1 + distance_km)) for each POI
weighted_proximity_score = proximity_score × category.weight
```
**Example:**
- Bakery 200m from 5 schools: score = 5 × (1/1.2) = 4.17
- Bakery 100m from 1 school: score = 1 × (1/1.1) = 0.91
- First bakery has higher school impact despite further distance!
**Tier 2: Distance Band Counts**
```python
count_0_100m = count(POIs within 100m)
count_100_300m = count(POIs within 100-300m)
count_300_500m = count(POIs within 300-500m)
count_500_1000m = count(POIs within 500-1000m)
```
**Tier 3: Distance to Nearest**
```python
distance_to_nearest_m = min(distances)
```
**Tier 4: Binary Flags**
```python
has_within_100m = any(distance <= 100m)
has_within_300m = any(distance <= 300m)
has_within_500m = any(distance <= 500m)
```
### Competitive Pressure Modeling
Special treatment for competitor bakeries:
**Zones:**
- **Direct** (<100m): -1.0 multiplier per competitor (strong negative)
- **Nearby** (100-500m): -0.5 multiplier (moderate negative)
- **Market** (500-1000m):
- If 5+ bakeries → +0.3 (bakery district = destination area)
- If 2-4 bakeries → -0.2 (competitive market)
## API Endpoints
### POST `/api/v1/poi-context/{tenant_id}/detect`
Detect POIs for a tenant's bakery location.
**Query Parameters:**
- `latitude` (float, required): Bakery latitude
- `longitude` (float, required): Bakery longitude
- `force_refresh` (bool, optional): Force re-detection, skip cache
**Response:**
```json
{
"status": "success",
"source": "detection", // or "cache"
"poi_context": {
"id": "uuid",
"tenant_id": "uuid",
"location": {"latitude": 40.4168, "longitude": -3.7038},
"total_pois_detected": 42,
"high_impact_categories": ["schools", "transport_hubs"],
"ml_features": {
"poi_schools_proximity_score": 3.45,
"poi_schools_count_0_100m": 2,
"poi_schools_distance_to_nearest_m": 85.0,
// ... 81+ more features
}
},
"feature_selection": {
"relevant_categories": ["schools", "transport_hubs", "offices"],
"relevance_report": [...]
},
"competitor_analysis": {
"competitive_pressure_score": -1.5,
"direct_competitors_count": 1,
"competitive_zone": "high_competition",
"market_type": "competitive_market"
},
"competitive_insights": [
"⚠️ High competition: 1 direct competitor(s) within 100m. Focus on differentiation and quality."
]
}
```
### GET `/api/v1/poi-context/{tenant_id}`
Retrieve stored POI context for a tenant.
**Response:**
```json
{
"poi_context": {...},
"is_stale": false,
"needs_refresh": false
}
```
### POST `/api/v1/poi-context/{tenant_id}/refresh`
Refresh POI context (re-detect POIs).
### DELETE `/api/v1/poi-context/{tenant_id}`
Delete POI context for a tenant.
### GET `/api/v1/poi-context/{tenant_id}/feature-importance`
Get feature importance summary.
### GET `/api/v1/poi-context/{tenant_id}/competitor-analysis`
Get detailed competitor analysis.
### GET `/api/v1/poi-context/health`
Check POI detection service health (Overpass API accessibility).
### GET `/api/v1/poi-context/cache/stats`
Get cache statistics (key count, memory usage).
## Database Schema
### Table: `tenant_poi_contexts`
```sql
CREATE TABLE tenant_poi_contexts (
id UUID PRIMARY KEY,
tenant_id UUID UNIQUE NOT NULL,
-- Location
latitude FLOAT NOT NULL,
longitude FLOAT NOT NULL,
-- POI Detection Data
poi_detection_results JSONB NOT NULL DEFAULT '{}',
ml_features JSONB NOT NULL DEFAULT '{}',
total_pois_detected INTEGER DEFAULT 0,
high_impact_categories JSONB DEFAULT '[]',
relevant_categories JSONB DEFAULT '[]',
-- Detection Metadata
detection_timestamp TIMESTAMP WITH TIME ZONE NOT NULL,
detection_source VARCHAR(50) DEFAULT 'overpass_api',
detection_status VARCHAR(20) DEFAULT 'completed',
detection_error VARCHAR(500),
-- Refresh Strategy
next_refresh_date TIMESTAMP WITH TIME ZONE,
refresh_interval_days INTEGER DEFAULT 180,
last_refreshed_at TIMESTAMP WITH TIME ZONE,
-- Timestamps
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE INDEX idx_tenant_poi_location ON tenant_poi_contexts (latitude, longitude);
CREATE INDEX idx_tenant_poi_refresh ON tenant_poi_contexts (next_refresh_date);
CREATE INDEX idx_tenant_poi_status ON tenant_poi_contexts (detection_status);
```
## ML Model Integration
### Training Pipeline
POI features are automatically fetched and integrated during training:
```python
# TrainingDataOrchestrator fetches POI features
poi_features = await poi_feature_integrator.fetch_poi_features(
tenant_id=tenant_id,
latitude=lat,
longitude=lon
)
# Features added to TrainingDataSet
training_dataset = TrainingDataSet(
sales_data=filtered_sales,
weather_data=weather_data,
traffic_data=traffic_data,
poi_features=poi_features, # NEW
...
)
# Data processor merges POI features into training data
daily_sales = self._add_poi_features(daily_sales, poi_features)
# Prophet model uses POI features as regressors
for feature_name in poi_features.keys():
model.add_regressor(feature_name, mode='additive')
```
### Forecasting Pipeline
POI features are fetched and used for predictions:
```python
# POI Feature Service retrieves features
poi_features = await poi_feature_service.get_poi_features(tenant_id)
# Features added to prediction dataframe
df = await data_processor.prepare_prediction_features(
future_dates=future_dates,
weather_forecast=weather_df,
poi_features=poi_features, # SAME features as training
...
)
# Prophet generates forecast with POI features
forecast = model.predict(df)
```
### Feature Consistency
**Critical:** POI features MUST be identical in training and prediction!
- Training: POI features fetched from External service
- Prediction: POI features fetched from External service (same tenant)
- Features are static (location-based, don't vary by date)
- Stored in `TenantPOIContext` ensures consistency
## Performance Optimizations
### Caching Strategy
**Redis Cache:**
- TTL: 90 days
- Cache key: Rounded coordinates (4 decimals ≈ 10m precision)
- Allows reuse for bakeries in close proximity
- Reduces Overpass API load
**Database Storage:**
- POI context stored in PostgreSQL
- Refresh cycle: 180 days (6 months)
- Background job refreshes stale contexts
### API Rate Limiting
**Overpass API:**
- Public endpoint: Rate limited
- Retry logic: 3 attempts with 2-second delay
- Timeout: 30 seconds per query
- Concurrent queries: All POI categories fetched in parallel
**Recommendation:** Self-host Overpass API instance for production
## Testing & Validation
### Model Performance Impact
Expected improvements with POI features:
- MAPE improvement: 5-10% for bakeries with significant POI presence
- Accuracy maintained: For bakeries with no relevant POIs (features filtered out)
- Feature count: 81+ POI features per bakery (if all categories relevant)
### A/B Testing
Compare models with and without POI features:
```python
# Model A: Without POI features
model_a = train_model(sales, weather, traffic)
# Model B: With POI features
model_b = train_model(sales, weather, traffic, poi_features)
# Compare MAPE, MAE, R² score
```
## Troubleshooting
### Common Issues
**1. No POI context found**
- **Cause:** POI detection not run during onboarding
- **Fix:** Call `/api/v1/poi-context/{tenant_id}/detect` endpoint
**2. Overpass API timeout**
- **Cause:** API overload or network issues
- **Fix:** Retry mechanism handles this automatically; check health endpoint
**3. POI features not in model**
- **Cause:** Feature relevance thresholds filter out low-signal features
- **Fix:** Expected behavior; check relevance report
**4. Feature count mismatch between training and prediction**
- **Cause:** POI context refreshed between training and prediction
- **Fix:** Models store feature manifest; prediction uses same features
## Future Enhancements
1. **Neighborhood Clustering**
- Group bakeries by neighborhood type (business district, residential, tourist)
- Reduce from 81+ individual features to 4-5 cluster features
- Enable transfer learning across similar neighborhoods
2. **Automated POI Verification**
- User confirmation of auto-detected POIs
- Manual addition/removal of POIs
3. **Temporal POI Features**
- School session times (morning vs. afternoon)
- Office hours variations (hybrid work)
- Event-based POIs (concerts, sports matches)
4. **Multi-City Support**
- City-specific POI weights
- Regional calendar integration (school holidays vary by region)
5. **POI Change Detection**
- Monitor for new POIs (e.g., new school opens)
- Automatic re-training when significant POI changes detected
## References
### Academic Research
1. "Gravity models for potential spatial healthcare access measurement" (2023)
2. "What determines travel time and distance decay in spatial interaction" (2024)
3. "Location Profiling for Retail-Site Recommendation Using Machine Learning" (2024)
4. "Predicting ride-hailing passenger demand: A POI-based adaptive clustering" (2024)
### Technical Documentation
- Overpass API: https://wiki.openstreetmap.org/wiki/Overpass_API
- OpenStreetMap Tags: https://wiki.openstreetmap.org/wiki/Map_features
- Facebook Prophet: https://facebook.github.io/prophet/
## License & Attribution
POI data from OpenStreetMap contributors (© OpenStreetMap contributors)
Licensed under Open Database License (ODbL)

2144
docs/03-features/specifications/wizard-flow-specification.md Normal file
View File

File diff suppressed because it is too large Load Diff