docs: Add delivery tracking service documentation to orchestrator README

This commit is contained in:
Urtzi Alfaro
2025-11-26 07:06:14 +01:00
parent 21651b396e
commit 945b9a3464

View File

@@ -903,4 +903,318 @@ New metrics available for dashboards:
--- ---
## Delivery Tracking Service
### Overview
The Delivery Tracking Service provides **proactive monitoring** of expected deliveries with time-based alert generation. Unlike reactive event-driven alerts, this service periodically checks delivery windows against current time to generate predictive and overdue notifications.
**Key Capabilities**:
- Proactive "arriving soon" alerts (T-2 hours before delivery)
- Overdue delivery detection (30 min after window)
- Incomplete receipt reminders (2 hours after window)
- Integration with Procurement Service for PO delivery schedules
- Automatic alert resolution when deliveries are received
### Cronjob Configuration
```yaml
# infrastructure/kubernetes/base/cronjobs/delivery-tracking-cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: delivery-tracking-cronjob
spec:
schedule: "30 * * * *" # Hourly at minute 30
concurrencyPolicy: Forbid
successfulJobsHistoryLimit: 3
failedJobsHistoryLimit: 3
jobTemplate:
spec:
activeDeadlineSeconds: 1800 # 30 minutes timeout
template:
spec:
containers:
- name: delivery-tracking
image: orchestrator-service:latest
command: ["python3", "-m", "app.services.delivery_tracking_service"]
resources:
requests:
memory: "128Mi"
cpu: "50m"
limits:
memory: "256Mi"
cpu: "100m"
```
**Schedule Rationale**: Hourly checks provide timely alerts without excessive polling. The :30 offset avoids collision with priority recalculation cronjob (:15).
### Delivery Alert Lifecycle
```
Purchase Order Approved (t=0)
System publishes DELIVERY_SCHEDULED (informational event)
[Time passes - no alerts]
T-2 hours before expected delivery time
CronJob detects: now >= (expected_delivery - 2 hours)
Generate DELIVERY_ARRIVING_SOON alert
- Priority: 70 (important)
- Class: action_needed
- Action Queue: Yes
- Smart Action: Open StockReceiptModal in create mode
[Delivery window arrives]
Expected delivery time + 30 minutes (grace period)
CronJob detects: now >= (delivery_window_end + 30 min)
Generate DELIVERY_OVERDUE alert
- Priority: 95 (critical)
- Class: critical
- Escalation: Time-sensitive
- Smart Action: Contact supplier + Open receipt modal
Expected delivery time + 2 hours
CronJob detects: still no stock receipt
Generate STOCK_RECEIPT_INCOMPLETE alert
- Priority: 80 (important)
- Class: action_needed
- Smart Action: Open existing receipt in edit mode
```
**Auto-Resolution**: All delivery alerts are automatically resolved when:
- Stock receipt is confirmed (`onConfirm` in StockReceiptModal)
- Event `delivery.received` is published
- Alert Processor marks alerts as `resolved` with reason: "Delivery received"
### Service Methods
#### `check_expected_deliveries()` - Main Entry Point
```python
async def check_expected_deliveries(tenant_id: str) -> None:
"""
Hourly job to check all purchase orders with expected deliveries.
Queries Procurement Service for POs with:
- status: approved or sent
- expected_delivery_date: within next 48 hours or past due
For each PO, checks:
1. Arriving soon? (T-2h) → _send_arriving_soon_alert()
2. Overdue? (T+30m) → _send_overdue_alert()
3. Receipt incomplete? (T+2h) → _send_receipt_incomplete_alert()
"""
```
#### `_send_arriving_soon_alert(po: PurchaseOrder)` - Proactive Warning
```python
async def _send_arriving_soon_alert(po: PurchaseOrder) -> None:
"""
Generates alert 2 hours before expected delivery.
Alert Details:
- event_type: DELIVERY_ARRIVING_SOON
- priority_score: 70 (important)
- alert_class: action_needed
- domain: supply_chain
- smart_action: open_stock_receipt_modal (create mode)
Context Enrichment:
- PO ID, supplier name, expected items count
- Delivery window (start/end times)
- Preparation checklist (clear receiving area, verify items)
"""
```
#### `_send_overdue_alert(po: PurchaseOrder)` - Critical Escalation
```python
async def _send_overdue_alert(po: PurchaseOrder) -> None:
"""
Generates critical alert 30 minutes after delivery window.
Alert Details:
- event_type: DELIVERY_OVERDUE
- priority_score: 95 (critical)
- alert_class: critical
- domain: supply_chain
- smart_actions: [contact_supplier, open_receipt_modal]
Business Impact:
- Production delays if ingredients missing
- Spoilage risk if perishables delayed
- Customer order fulfillment risk
Suggested Actions:
1. Contact supplier immediately
2. Check for delivery rescheduling
3. Activate backup supplier if needed
4. Adjust production plan if ingredients critical
"""
```
#### `_send_receipt_incomplete_alert(po: PurchaseOrder)` - Reminder
```python
async def _send_receipt_incomplete_alert(po: PurchaseOrder) -> None:
"""
Generates reminder 2 hours after delivery window if no receipt.
Alert Details:
- event_type: STOCK_RECEIPT_INCOMPLETE
- priority_score: 80 (important)
- alert_class: action_needed
- domain: inventory
- smart_action: open_stock_receipt_modal (edit mode if draft exists)
Checks:
- Stock receipts table for PO ID
- If draft exists → Edit mode with pre-filled data
- If no draft → Create mode
HACCP Compliance Note:
- Food safety requires timely receipt documentation
- Expiration date tracking depends on receipt
- Incomplete receipts block lot tracking
"""
```
### Integration with Alert System
**Publishing Flow**:
```python
# services/orchestrator/app/services/delivery_tracking_service.py
from shared.clients.alerts_client import AlertsClient
alerts_client = AlertsClient(service_name="orchestrator")
await alerts_client.publish_alert(
tenant_id=tenant_id,
event_type="DELIVERY_OVERDUE",
entity_type="purchase_order",
entity_id=po.id,
severity="critical",
priority_score=95,
context={
"po_number": po.po_number,
"supplier_name": po.supplier.name,
"expected_delivery": po.expected_delivery_date.isoformat(),
"delay_minutes": delay_in_minutes,
"items_count": len(po.line_items)
}
)
```
**Alert Processing**:
1. Delivery Tracking Service → RabbitMQ (supply_chain.alerts exchange)
2. Alert Processor consumes message
3. Full enrichment pipeline (Tier 1 - ALERTS)
4. Smart action handler assigned (open_stock_receipt_modal)
5. Store in PostgreSQL with priority_score
6. Publish to Redis Pub/Sub → Gateway SSE
7. Frontend `useSupplyChainNotifications()` hook receives alert
8. UnifiedActionQueueCard displays in "Urgent" section
9. User clicks → StockReceiptModal opens with PO context
### Architecture Decision: Why CronJob Over Event System?
**Question**: Could we replace this cronjob with scheduled events?
**Answer**: ❌ No - CronJob is the right tool for this job.
#### Comparison Matrix
| Feature | Event System | CronJob | Best Choice |
|---------|--------------|---------|-------------|
| Time-based alerts | ❌ Requires complex scheduling | ✅ Natural fit | **CronJob** |
| Predictive alerts | ❌ Must schedule at PO creation | ✅ Dynamic checks | **CronJob** |
| Delivery window changes | ❌ Need to reschedule events | ✅ Adapts automatically | **CronJob** |
| System restarts | ❌ Lose scheduled events | ✅ Persistent schedule | **CronJob** |
| Complexity | ❌ High (event scheduler needed) | ✅ Low (periodic check) | **CronJob** |
| Maintenance | ❌ Many scheduled events | ✅ Single job | **CronJob** |
**Event System Challenges**:
- Would need to schedule 3 events per PO at approval time:
1. "arriving_soon" event at (delivery_time - 2h)
2. "overdue" event at (delivery_time + 30m)
3. "incomplete" event at (delivery_time + 2h)
- Requires persistent event scheduler (like Celery Beat)
- Rescheduling when delivery dates change is complex
- System restarts would lose in-memory scheduled events
- Essentially rebuilding cron functionality
**CronJob Advantages**:
- ✅ Simple periodic check against current time
- ✅ Adapts to delivery date changes automatically
- ✅ No state management for scheduled events
- ✅ Easy to adjust alert timing thresholds
- ✅ Built-in Kubernetes scheduling and monitoring
- ✅ Resource-efficient (runs 1 minute every hour)
**Verdict**: Periodic polling is more maintainable than scheduled events for time-based conditions.
### Monitoring & Observability
**Metrics Tracked**:
- `delivery_tracking_job_duration_seconds` - Execution time
- `delivery_alerts_generated_total{type}` - Counter by alert type
- `deliveries_checked_total` - Total POs scanned
- `delivery_tracking_errors_total` - Failure rate
**Logs**:
```
[2025-11-26 14:30:02] INFO: Delivery tracking job started for tenant abc123
[2025-11-26 14:30:03] INFO: Found 12 purchase orders with upcoming deliveries
[2025-11-26 14:30:03] INFO: Generated DELIVERY_ARRIVING_SOON for PO-2025-043 (delivery in 1h 45m)
[2025-11-26 14:30:03] WARNING: Generated DELIVERY_OVERDUE for PO-2025-041 (45 minutes late)
[2025-11-26 14:30:04] INFO: Delivery tracking job completed in 2.3s
```
**Alerting** (for Ops team):
- Job fails 3 times consecutively → Page on-call engineer
- Job duration > 5 minutes → Warning (performance degradation)
- Zero deliveries checked for 24 hours → Warning (data issue)
### Testing
**Unit Tests**:
```python
# tests/services/test_delivery_tracking_service.py
async def test_arriving_soon_alert_generated():
# Given: PO with delivery in 1 hour 55 minutes
po = create_test_po(expected_delivery=now() + timedelta(hours=1, minutes=55))
# When: Check deliveries
await delivery_tracking_service.check_expected_deliveries(tenant_id)
# Then: DELIVERY_ARRIVING_SOON alert generated
assert_alert_published("DELIVERY_ARRIVING_SOON", po.id)
```
**Integration Tests**:
- Test full flow from cronjob → alert → frontend SSE
- Verify alert auto-resolution on stock receipt confirmation
- Test grace period boundaries (exactly 30 minutes)
### Performance Characteristics
**Typical Execution**:
- Query Procurement Service: 50-100ms
- Filter POs by time windows: 5-10ms
- Generate alerts (avg 3 per run): 150-300ms
- Total: **200-400ms per tenant**
**Scaling**:
- Single-tenant deployment: Trivial (<1s per hour)
- Multi-tenant (100 tenants): ~40s per run (well under 30min timeout)
- Multi-tenant (1000+ tenants): Consider tenant sharding across multiple cronjobs
---
**Copyright © 2025 Bakery-IA. All rights reserved.** **Copyright © 2025 Bakery-IA. All rights reserved.**