docs: Add delivery tracking service documentation to orchestrator README
This commit is contained in:
@@ -903,4 +903,318 @@ New metrics available for dashboards:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Delivery Tracking Service
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
The Delivery Tracking Service provides **proactive monitoring** of expected deliveries with time-based alert generation. Unlike reactive event-driven alerts, this service periodically checks delivery windows against current time to generate predictive and overdue notifications.
|
||||||
|
|
||||||
|
**Key Capabilities**:
|
||||||
|
- Proactive "arriving soon" alerts (T-2 hours before delivery)
|
||||||
|
- Overdue delivery detection (30 min after window)
|
||||||
|
- Incomplete receipt reminders (2 hours after window)
|
||||||
|
- Integration with Procurement Service for PO delivery schedules
|
||||||
|
- Automatic alert resolution when deliveries are received
|
||||||
|
|
||||||
|
### Cronjob Configuration
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# infrastructure/kubernetes/base/cronjobs/delivery-tracking-cronjob.yaml
|
||||||
|
apiVersion: batch/v1
|
||||||
|
kind: CronJob
|
||||||
|
metadata:
|
||||||
|
name: delivery-tracking-cronjob
|
||||||
|
spec:
|
||||||
|
schedule: "30 * * * *" # Hourly at minute 30
|
||||||
|
concurrencyPolicy: Forbid
|
||||||
|
successfulJobsHistoryLimit: 3
|
||||||
|
failedJobsHistoryLimit: 3
|
||||||
|
jobTemplate:
|
||||||
|
spec:
|
||||||
|
activeDeadlineSeconds: 1800 # 30 minutes timeout
|
||||||
|
template:
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: delivery-tracking
|
||||||
|
image: orchestrator-service:latest
|
||||||
|
command: ["python3", "-m", "app.services.delivery_tracking_service"]
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
memory: "128Mi"
|
||||||
|
cpu: "50m"
|
||||||
|
limits:
|
||||||
|
memory: "256Mi"
|
||||||
|
cpu: "100m"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Schedule Rationale**: Hourly checks provide timely alerts without excessive polling. The :30 offset avoids collision with priority recalculation cronjob (:15).
|
||||||
|
|
||||||
|
### Delivery Alert Lifecycle
|
||||||
|
|
||||||
|
```
|
||||||
|
Purchase Order Approved (t=0)
|
||||||
|
↓
|
||||||
|
System publishes DELIVERY_SCHEDULED (informational event)
|
||||||
|
↓
|
||||||
|
[Time passes - no alerts]
|
||||||
|
↓
|
||||||
|
T-2 hours before expected delivery time
|
||||||
|
↓
|
||||||
|
CronJob detects: now >= (expected_delivery - 2 hours)
|
||||||
|
↓
|
||||||
|
Generate DELIVERY_ARRIVING_SOON alert
|
||||||
|
- Priority: 70 (important)
|
||||||
|
- Class: action_needed
|
||||||
|
- Action Queue: Yes
|
||||||
|
- Smart Action: Open StockReceiptModal in create mode
|
||||||
|
↓
|
||||||
|
[Delivery window arrives]
|
||||||
|
↓
|
||||||
|
Expected delivery time + 30 minutes (grace period)
|
||||||
|
↓
|
||||||
|
CronJob detects: now >= (delivery_window_end + 30 min)
|
||||||
|
↓
|
||||||
|
Generate DELIVERY_OVERDUE alert
|
||||||
|
- Priority: 95 (critical)
|
||||||
|
- Class: critical
|
||||||
|
- Escalation: Time-sensitive
|
||||||
|
- Smart Action: Contact supplier + Open receipt modal
|
||||||
|
↓
|
||||||
|
Expected delivery time + 2 hours
|
||||||
|
↓
|
||||||
|
CronJob detects: still no stock receipt
|
||||||
|
↓
|
||||||
|
Generate STOCK_RECEIPT_INCOMPLETE alert
|
||||||
|
- Priority: 80 (important)
|
||||||
|
- Class: action_needed
|
||||||
|
- Smart Action: Open existing receipt in edit mode
|
||||||
|
```
|
||||||
|
|
||||||
|
**Auto-Resolution**: All delivery alerts are automatically resolved when:
|
||||||
|
- Stock receipt is confirmed (`onConfirm` in StockReceiptModal)
|
||||||
|
- Event `delivery.received` is published
|
||||||
|
- Alert Processor marks alerts as `resolved` with reason: "Delivery received"
|
||||||
|
|
||||||
|
### Service Methods
|
||||||
|
|
||||||
|
#### `check_expected_deliveries()` - Main Entry Point
|
||||||
|
```python
|
||||||
|
async def check_expected_deliveries(tenant_id: str) -> None:
|
||||||
|
"""
|
||||||
|
Hourly job to check all purchase orders with expected deliveries.
|
||||||
|
|
||||||
|
Queries Procurement Service for POs with:
|
||||||
|
- status: approved or sent
|
||||||
|
- expected_delivery_date: within next 48 hours or past due
|
||||||
|
|
||||||
|
For each PO, checks:
|
||||||
|
1. Arriving soon? (T-2h) → _send_arriving_soon_alert()
|
||||||
|
2. Overdue? (T+30m) → _send_overdue_alert()
|
||||||
|
3. Receipt incomplete? (T+2h) → _send_receipt_incomplete_alert()
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `_send_arriving_soon_alert(po: PurchaseOrder)` - Proactive Warning
|
||||||
|
```python
|
||||||
|
async def _send_arriving_soon_alert(po: PurchaseOrder) -> None:
|
||||||
|
"""
|
||||||
|
Generates alert 2 hours before expected delivery.
|
||||||
|
|
||||||
|
Alert Details:
|
||||||
|
- event_type: DELIVERY_ARRIVING_SOON
|
||||||
|
- priority_score: 70 (important)
|
||||||
|
- alert_class: action_needed
|
||||||
|
- domain: supply_chain
|
||||||
|
- smart_action: open_stock_receipt_modal (create mode)
|
||||||
|
|
||||||
|
Context Enrichment:
|
||||||
|
- PO ID, supplier name, expected items count
|
||||||
|
- Delivery window (start/end times)
|
||||||
|
- Preparation checklist (clear receiving area, verify items)
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `_send_overdue_alert(po: PurchaseOrder)` - Critical Escalation
|
||||||
|
```python
|
||||||
|
async def _send_overdue_alert(po: PurchaseOrder) -> None:
|
||||||
|
"""
|
||||||
|
Generates critical alert 30 minutes after delivery window.
|
||||||
|
|
||||||
|
Alert Details:
|
||||||
|
- event_type: DELIVERY_OVERDUE
|
||||||
|
- priority_score: 95 (critical)
|
||||||
|
- alert_class: critical
|
||||||
|
- domain: supply_chain
|
||||||
|
- smart_actions: [contact_supplier, open_receipt_modal]
|
||||||
|
|
||||||
|
Business Impact:
|
||||||
|
- Production delays if ingredients missing
|
||||||
|
- Spoilage risk if perishables delayed
|
||||||
|
- Customer order fulfillment risk
|
||||||
|
|
||||||
|
Suggested Actions:
|
||||||
|
1. Contact supplier immediately
|
||||||
|
2. Check for delivery rescheduling
|
||||||
|
3. Activate backup supplier if needed
|
||||||
|
4. Adjust production plan if ingredients critical
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
#### `_send_receipt_incomplete_alert(po: PurchaseOrder)` - Reminder
|
||||||
|
```python
|
||||||
|
async def _send_receipt_incomplete_alert(po: PurchaseOrder) -> None:
|
||||||
|
"""
|
||||||
|
Generates reminder 2 hours after delivery window if no receipt.
|
||||||
|
|
||||||
|
Alert Details:
|
||||||
|
- event_type: STOCK_RECEIPT_INCOMPLETE
|
||||||
|
- priority_score: 80 (important)
|
||||||
|
- alert_class: action_needed
|
||||||
|
- domain: inventory
|
||||||
|
- smart_action: open_stock_receipt_modal (edit mode if draft exists)
|
||||||
|
|
||||||
|
Checks:
|
||||||
|
- Stock receipts table for PO ID
|
||||||
|
- If draft exists → Edit mode with pre-filled data
|
||||||
|
- If no draft → Create mode
|
||||||
|
|
||||||
|
HACCP Compliance Note:
|
||||||
|
- Food safety requires timely receipt documentation
|
||||||
|
- Expiration date tracking depends on receipt
|
||||||
|
- Incomplete receipts block lot tracking
|
||||||
|
"""
|
||||||
|
```
|
||||||
|
|
||||||
|
### Integration with Alert System
|
||||||
|
|
||||||
|
**Publishing Flow**:
|
||||||
|
```python
|
||||||
|
# services/orchestrator/app/services/delivery_tracking_service.py
|
||||||
|
from shared.clients.alerts_client import AlertsClient
|
||||||
|
|
||||||
|
alerts_client = AlertsClient(service_name="orchestrator")
|
||||||
|
|
||||||
|
await alerts_client.publish_alert(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
event_type="DELIVERY_OVERDUE",
|
||||||
|
entity_type="purchase_order",
|
||||||
|
entity_id=po.id,
|
||||||
|
severity="critical",
|
||||||
|
priority_score=95,
|
||||||
|
context={
|
||||||
|
"po_number": po.po_number,
|
||||||
|
"supplier_name": po.supplier.name,
|
||||||
|
"expected_delivery": po.expected_delivery_date.isoformat(),
|
||||||
|
"delay_minutes": delay_in_minutes,
|
||||||
|
"items_count": len(po.line_items)
|
||||||
|
}
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Alert Processing**:
|
||||||
|
1. Delivery Tracking Service → RabbitMQ (supply_chain.alerts exchange)
|
||||||
|
2. Alert Processor consumes message
|
||||||
|
3. Full enrichment pipeline (Tier 1 - ALERTS)
|
||||||
|
4. Smart action handler assigned (open_stock_receipt_modal)
|
||||||
|
5. Store in PostgreSQL with priority_score
|
||||||
|
6. Publish to Redis Pub/Sub → Gateway SSE
|
||||||
|
7. Frontend `useSupplyChainNotifications()` hook receives alert
|
||||||
|
8. UnifiedActionQueueCard displays in "Urgent" section
|
||||||
|
9. User clicks → StockReceiptModal opens with PO context
|
||||||
|
|
||||||
|
### Architecture Decision: Why CronJob Over Event System?
|
||||||
|
|
||||||
|
**Question**: Could we replace this cronjob with scheduled events?
|
||||||
|
|
||||||
|
**Answer**: ❌ No - CronJob is the right tool for this job.
|
||||||
|
|
||||||
|
#### Comparison Matrix
|
||||||
|
|
||||||
|
| Feature | Event System | CronJob | Best Choice |
|
||||||
|
|---------|--------------|---------|-------------|
|
||||||
|
| Time-based alerts | ❌ Requires complex scheduling | ✅ Natural fit | **CronJob** |
|
||||||
|
| Predictive alerts | ❌ Must schedule at PO creation | ✅ Dynamic checks | **CronJob** |
|
||||||
|
| Delivery window changes | ❌ Need to reschedule events | ✅ Adapts automatically | **CronJob** |
|
||||||
|
| System restarts | ❌ Lose scheduled events | ✅ Persistent schedule | **CronJob** |
|
||||||
|
| Complexity | ❌ High (event scheduler needed) | ✅ Low (periodic check) | **CronJob** |
|
||||||
|
| Maintenance | ❌ Many scheduled events | ✅ Single job | **CronJob** |
|
||||||
|
|
||||||
|
**Event System Challenges**:
|
||||||
|
- Would need to schedule 3 events per PO at approval time:
|
||||||
|
1. "arriving_soon" event at (delivery_time - 2h)
|
||||||
|
2. "overdue" event at (delivery_time + 30m)
|
||||||
|
3. "incomplete" event at (delivery_time + 2h)
|
||||||
|
- Requires persistent event scheduler (like Celery Beat)
|
||||||
|
- Rescheduling when delivery dates change is complex
|
||||||
|
- System restarts would lose in-memory scheduled events
|
||||||
|
- Essentially rebuilding cron functionality
|
||||||
|
|
||||||
|
**CronJob Advantages**:
|
||||||
|
- ✅ Simple periodic check against current time
|
||||||
|
- ✅ Adapts to delivery date changes automatically
|
||||||
|
- ✅ No state management for scheduled events
|
||||||
|
- ✅ Easy to adjust alert timing thresholds
|
||||||
|
- ✅ Built-in Kubernetes scheduling and monitoring
|
||||||
|
- ✅ Resource-efficient (runs 1 minute every hour)
|
||||||
|
|
||||||
|
**Verdict**: Periodic polling is more maintainable than scheduled events for time-based conditions.
|
||||||
|
|
||||||
|
### Monitoring & Observability
|
||||||
|
|
||||||
|
**Metrics Tracked**:
|
||||||
|
- `delivery_tracking_job_duration_seconds` - Execution time
|
||||||
|
- `delivery_alerts_generated_total{type}` - Counter by alert type
|
||||||
|
- `deliveries_checked_total` - Total POs scanned
|
||||||
|
- `delivery_tracking_errors_total` - Failure rate
|
||||||
|
|
||||||
|
**Logs**:
|
||||||
|
```
|
||||||
|
[2025-11-26 14:30:02] INFO: Delivery tracking job started for tenant abc123
|
||||||
|
[2025-11-26 14:30:03] INFO: Found 12 purchase orders with upcoming deliveries
|
||||||
|
[2025-11-26 14:30:03] INFO: Generated DELIVERY_ARRIVING_SOON for PO-2025-043 (delivery in 1h 45m)
|
||||||
|
[2025-11-26 14:30:03] WARNING: Generated DELIVERY_OVERDUE for PO-2025-041 (45 minutes late)
|
||||||
|
[2025-11-26 14:30:04] INFO: Delivery tracking job completed in 2.3s
|
||||||
|
```
|
||||||
|
|
||||||
|
**Alerting** (for Ops team):
|
||||||
|
- Job fails 3 times consecutively → Page on-call engineer
|
||||||
|
- Job duration > 5 minutes → Warning (performance degradation)
|
||||||
|
- Zero deliveries checked for 24 hours → Warning (data issue)
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
**Unit Tests**:
|
||||||
|
```python
|
||||||
|
# tests/services/test_delivery_tracking_service.py
|
||||||
|
async def test_arriving_soon_alert_generated():
|
||||||
|
# Given: PO with delivery in 1 hour 55 minutes
|
||||||
|
po = create_test_po(expected_delivery=now() + timedelta(hours=1, minutes=55))
|
||||||
|
|
||||||
|
# When: Check deliveries
|
||||||
|
await delivery_tracking_service.check_expected_deliveries(tenant_id)
|
||||||
|
|
||||||
|
# Then: DELIVERY_ARRIVING_SOON alert generated
|
||||||
|
assert_alert_published("DELIVERY_ARRIVING_SOON", po.id)
|
||||||
|
```
|
||||||
|
|
||||||
|
**Integration Tests**:
|
||||||
|
- Test full flow from cronjob → alert → frontend SSE
|
||||||
|
- Verify alert auto-resolution on stock receipt confirmation
|
||||||
|
- Test grace period boundaries (exactly 30 minutes)
|
||||||
|
|
||||||
|
### Performance Characteristics
|
||||||
|
|
||||||
|
**Typical Execution**:
|
||||||
|
- Query Procurement Service: 50-100ms
|
||||||
|
- Filter POs by time windows: 5-10ms
|
||||||
|
- Generate alerts (avg 3 per run): 150-300ms
|
||||||
|
- Total: **200-400ms per tenant**
|
||||||
|
|
||||||
|
**Scaling**:
|
||||||
|
- Single-tenant deployment: Trivial (<1s per hour)
|
||||||
|
- Multi-tenant (100 tenants): ~40s per run (well under 30min timeout)
|
||||||
|
- Multi-tenant (1000+ tenants): Consider tenant sharding across multiple cronjobs
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
**Copyright © 2025 Bakery-IA. All rights reserved.**
|
**Copyright © 2025 Bakery-IA. All rights reserved.**
|
||||||
|
|||||||
Reference in New Issue
Block a user