Improve the frontend 4
This commit is contained in:
Urtzi Alfaro
470
docs/COMPLETION_CHECKLIST.md
Normal file
470
docs/COMPLETION_CHECKLIST.md
Normal file
@@ -0,0 +1,470 @@
|
||||
# Completion Checklist - Tenant & User Deletion System
|
||||
|
||||
**Current Status:** 75% Complete
|
||||
**Time to 100%:** ~4 hours implementation + 2 days testing
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Complete Remaining Services (1.5 hours)
|
||||
|
||||
### POS Service (30 minutes)
|
||||
|
||||
- [ ] Create `services/pos/app/services/tenant_deletion_service.py`
|
||||
- [ ] Copy template from QUICK_START_REMAINING_SERVICES.md
|
||||
- [ ] Import models: POSConfiguration, POSTransaction, POSSession
|
||||
- [ ] Implement `get_tenant_data_preview()`
|
||||
- [ ] Implement `delete_tenant_data()` with correct order:
|
||||
- [ ] 1. POSTransaction
|
||||
- [ ] 2. POSSession
|
||||
- [ ] 3. POSConfiguration
|
||||
|
||||
- [ ] Add endpoints to `services/pos/app/api/{router}.py`
|
||||
- [ ] DELETE /tenant/{tenant_id}
|
||||
- [ ] GET /tenant/{tenant_id}/deletion-preview
|
||||
|
||||
- [ ] Test manually:
|
||||
```bash
|
||||
curl -X GET "http://localhost:8000/api/v1/pos/tenant/{id}/deletion-preview"
|
||||
curl -X DELETE "http://localhost:8000/api/v1/pos/tenant/{id}"
|
||||
```
|
||||
|
||||
### External Service (30 minutes)
|
||||
|
||||
- [ ] Create `services/external/app/services/tenant_deletion_service.py`
|
||||
- [ ] Copy template
|
||||
- [ ] Import models: ExternalDataCache, APIKeyUsage
|
||||
- [ ] Implement `get_tenant_data_preview()`
|
||||
- [ ] Implement `delete_tenant_data()` with order:
|
||||
- [ ] 1. APIKeyUsage
|
||||
- [ ] 2. ExternalDataCache
|
||||
|
||||
- [ ] Add endpoints to `services/external/app/api/{router}.py`
|
||||
- [ ] DELETE /tenant/{tenant_id}
|
||||
- [ ] GET /tenant/{tenant_id}/deletion-preview
|
||||
|
||||
- [ ] Test manually
|
||||
|
||||
### Alert Processor Service (30 minutes)
|
||||
|
||||
- [ ] Create `services/alert_processor/app/services/tenant_deletion_service.py`
|
||||
- [ ] Copy template
|
||||
- [ ] Import models: Alert, AlertRule, AlertHistory
|
||||
- [ ] Implement `get_tenant_data_preview()`
|
||||
- [ ] Implement `delete_tenant_data()` with order:
|
||||
- [ ] 1. AlertHistory
|
||||
- [ ] 2. Alert
|
||||
- [ ] 3. AlertRule
|
||||
|
||||
- [ ] Add endpoints to `services/alert_processor/app/api/{router}.py`
|
||||
- [ ] DELETE /tenant/{tenant_id}
|
||||
- [ ] GET /tenant/{tenant_id}/deletion-preview
|
||||
|
||||
- [ ] Test manually
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Refactor Existing Services (2.5 hours)
|
||||
|
||||
### Forecasting Service (45 minutes)
|
||||
|
||||
- [ ] Review existing deletion logic in forecasting service
|
||||
- [ ] Create new `services/forecasting/app/services/tenant_deletion_service.py`
|
||||
- [ ] Extend BaseTenantDataDeletionService
|
||||
- [ ] Move existing logic into standard pattern
|
||||
- [ ] Import models: Forecast, PredictionBatch, etc.
|
||||
|
||||
- [ ] Update endpoints to use new pattern
|
||||
- [ ] Replace existing DELETE logic
|
||||
- [ ] Add deletion-preview endpoint
|
||||
|
||||
- [ ] Test both endpoints
|
||||
|
||||
### Training Service (45 minutes)
|
||||
|
||||
- [ ] Review existing deletion logic
|
||||
- [ ] Create new `services/training/app/services/tenant_deletion_service.py`
|
||||
- [ ] Extend BaseTenantDataDeletionService
|
||||
- [ ] Move existing logic into standard pattern
|
||||
- [ ] Import models: TrainingJob, TrainedModel, ModelArtifact
|
||||
|
||||
- [ ] Update endpoints to use new pattern
|
||||
|
||||
- [ ] Test both endpoints
|
||||
|
||||
### Notification Service (45 minutes)
|
||||
|
||||
- [ ] Review existing deletion logic
|
||||
- [ ] Create new `services/notification/app/services/tenant_deletion_service.py`
|
||||
- [ ] Extend BaseTenantDataDeletionService
|
||||
- [ ] Move existing logic into standard pattern
|
||||
- [ ] Import models: Notification, NotificationPreference, etc.
|
||||
|
||||
- [ ] Update endpoints to use new pattern
|
||||
|
||||
- [ ] Test both endpoints
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Integration (2 hours)
|
||||
|
||||
### Update Auth Service
|
||||
|
||||
- [ ] Open `services/auth/app/services/admin_delete.py`
|
||||
|
||||
- [ ] Import DeletionOrchestrator:
|
||||
```python
|
||||
from app.services.deletion_orchestrator import DeletionOrchestrator
|
||||
```
|
||||
|
||||
- [ ] Update `_delete_tenant_data()` method:
|
||||
```python
|
||||
async def _delete_tenant_data(self, tenant_id: str):
|
||||
orchestrator = DeletionOrchestrator(auth_token=self.get_service_token())
|
||||
job = await orchestrator.orchestrate_tenant_deletion(
|
||||
tenant_id=tenant_id,
|
||||
tenant_name=tenant_info.get("name"),
|
||||
initiated_by=self.requesting_user_id
|
||||
)
|
||||
return job.to_dict()
|
||||
```
|
||||
|
||||
- [ ] Remove old manual service calls
|
||||
|
||||
- [ ] Test complete user deletion flow
|
||||
|
||||
### Verify Service URLs
|
||||
|
||||
- [ ] Check orchestrator SERVICE_DELETION_ENDPOINTS
|
||||
- [ ] Update URLs for your environment:
|
||||
- [ ] Development: localhost ports
|
||||
- [ ] Staging: service names
|
||||
- [ ] Production: service names
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Testing (2 days)
|
||||
|
||||
### Unit Tests (Day 1)
|
||||
|
||||
- [ ] Test TenantDataDeletionResult
|
||||
```python
|
||||
def test_deletion_result_creation():
|
||||
result = TenantDataDeletionResult("tenant-123", "test-service")
|
||||
assert result.tenant_id == "tenant-123"
|
||||
assert result.success == True
|
||||
```
|
||||
|
||||
- [ ] Test BaseTenantDataDeletionService
|
||||
```python
|
||||
async def test_safe_delete_handles_errors():
|
||||
# Test error handling
|
||||
```
|
||||
|
||||
- [ ] Test each service deletion class
|
||||
```python
|
||||
async def test_orders_deletion():
|
||||
# Create test data
|
||||
# Call delete_tenant_data()
|
||||
# Verify data deleted
|
||||
```
|
||||
|
||||
- [ ] Test DeletionOrchestrator
|
||||
```python
|
||||
async def test_orchestrator_parallel_execution():
|
||||
# Mock service responses
|
||||
# Verify all called
|
||||
```
|
||||
|
||||
- [ ] Test DeletionJob tracking
|
||||
```python
|
||||
def test_job_status_tracking():
|
||||
# Create job
|
||||
# Check status transitions
|
||||
```
|
||||
|
||||
### Integration Tests (Day 1-2)
|
||||
|
||||
- [ ] Test tenant deletion endpoint
|
||||
```python
|
||||
async def test_delete_tenant_endpoint():
|
||||
response = await client.delete(f"/api/v1/tenants/{tenant_id}")
|
||||
assert response.status_code == 200
|
||||
```
|
||||
|
||||
- [ ] Test service-to-service calls
|
||||
```python
|
||||
async def test_orders_deletion_via_orchestrator():
|
||||
# Create tenant with orders
|
||||
# Delete tenant
|
||||
# Verify orders deleted
|
||||
```
|
||||
|
||||
- [ ] Test CASCADE deletes
|
||||
```python
|
||||
async def test_cascade_deletes_children():
|
||||
# Create parent with children
|
||||
# Delete parent
|
||||
# Verify children also deleted
|
||||
```
|
||||
|
||||
- [ ] Test error handling
|
||||
```python
|
||||
async def test_partial_failure_handling():
|
||||
# Mock one service failure
|
||||
# Verify job shows failure
|
||||
# Verify other services succeeded
|
||||
```
|
||||
|
||||
### E2E Tests (Day 2)
|
||||
|
||||
- [ ] Test complete tenant deletion
|
||||
```python
|
||||
async def test_complete_tenant_deletion():
|
||||
# Create tenant with data in all services
|
||||
# Delete tenant
|
||||
# Verify all data deleted
|
||||
# Check deletion job status
|
||||
```
|
||||
|
||||
- [ ] Test complete user deletion
|
||||
```python
|
||||
async def test_user_deletion_with_owned_tenants():
|
||||
# Create user with owned tenants
|
||||
# Create other admins
|
||||
# Delete user
|
||||
# Verify ownership transferred
|
||||
# Verify user data deleted
|
||||
```
|
||||
|
||||
- [ ] Test owner deletion with tenant deletion
|
||||
```python
|
||||
async def test_owner_deletion_no_other_admins():
|
||||
# Create user with tenant (no other admins)
|
||||
# Delete user
|
||||
# Verify tenant deleted
|
||||
# Verify all cascade deletes
|
||||
```
|
||||
|
||||
### Manual Testing (Throughout)
|
||||
|
||||
- [ ] Test with small dataset (<100 records)
|
||||
- [ ] Test with medium dataset (1,000 records)
|
||||
- [ ] Test with large dataset (10,000+ records)
|
||||
- [ ] Measure performance
|
||||
- [ ] Verify database queries are efficient
|
||||
- [ ] Check logs for errors
|
||||
- [ ] Verify audit trail
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Database Persistence (1 day)
|
||||
|
||||
### Create Migration
|
||||
|
||||
- [ ] Create deletion_jobs table:
|
||||
```sql
|
||||
CREATE TABLE deletion_jobs (
|
||||
id UUID PRIMARY KEY,
|
||||
tenant_id UUID NOT NULL,
|
||||
tenant_name VARCHAR(255),
|
||||
initiated_by UUID,
|
||||
status VARCHAR(50) NOT NULL,
|
||||
service_results JSONB,
|
||||
total_items_deleted INTEGER DEFAULT 0,
|
||||
started_at TIMESTAMP WITH TIME ZONE,
|
||||
completed_at TIMESTAMP WITH TIME ZONE,
|
||||
error_log TEXT[],
|
||||
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
|
||||
);
|
||||
|
||||
CREATE INDEX idx_deletion_jobs_tenant ON deletion_jobs(tenant_id);
|
||||
CREATE INDEX idx_deletion_jobs_status ON deletion_jobs(status);
|
||||
CREATE INDEX idx_deletion_jobs_initiated ON deletion_jobs(initiated_by);
|
||||
```
|
||||
|
||||
- [ ] Run migration in dev
|
||||
- [ ] Run migration in staging
|
||||
|
||||
### Update Orchestrator
|
||||
|
||||
- [ ] Add database session to DeletionOrchestrator
|
||||
- [ ] Save job to database in orchestrate_tenant_deletion()
|
||||
- [ ] Update job status in database
|
||||
- [ ] Query jobs from database in get_job_status()
|
||||
- [ ] Query jobs from database in list_jobs()
|
||||
|
||||
### Add Job API Endpoints
|
||||
|
||||
- [ ] Create `services/auth/app/api/deletion_jobs.py`
|
||||
```python
|
||||
@router.get("/deletion-jobs/{job_id}")
|
||||
async def get_job_status(job_id: str):
|
||||
# Query from database
|
||||
|
||||
@router.get("/deletion-jobs")
|
||||
async def list_deletion_jobs(
|
||||
tenant_id: Optional[str] = None,
|
||||
status: Optional[str] = None,
|
||||
limit: int = 100
|
||||
):
|
||||
# Query from database with filters
|
||||
```
|
||||
|
||||
- [ ] Test job status endpoints
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Production Prep (2 days)
|
||||
|
||||
### Performance Testing
|
||||
|
||||
- [ ] Create test dataset with 100K records
|
||||
- [ ] Run deletion and measure time
|
||||
- [ ] Identify bottlenecks
|
||||
- [ ] Optimize slow queries
|
||||
- [ ] Add batch processing if needed
|
||||
- [ ] Re-test and verify improvement
|
||||
|
||||
### Monitoring Setup
|
||||
|
||||
- [ ] Add Prometheus metrics:
|
||||
```python
|
||||
deletion_duration_seconds = Histogram(...)
|
||||
deletion_items_deleted = Counter(...)
|
||||
deletion_errors_total = Counter(...)
|
||||
deletion_jobs_status = Gauge(...)
|
||||
```
|
||||
|
||||
- [ ] Create Grafana dashboard:
|
||||
- [ ] Active deletions gauge
|
||||
- [ ] Deletion rate graph
|
||||
- [ ] Error rate graph
|
||||
- [ ] Average duration graph
|
||||
- [ ] Items deleted by service
|
||||
|
||||
- [ ] Configure alerts:
|
||||
- [ ] Alert if deletion >5 minutes
|
||||
- [ ] Alert if >10% error rate
|
||||
- [ ] Alert if service timeouts
|
||||
|
||||
### Documentation Updates
|
||||
|
||||
- [ ] Update API documentation
|
||||
- [ ] Create operations runbook
|
||||
- [ ] Document rollback procedures
|
||||
- [ ] Create troubleshooting guide
|
||||
|
||||
### Rollout Plan
|
||||
|
||||
- [ ] Deploy to dev environment
|
||||
- [ ] Run full test suite
|
||||
- [ ] Deploy to staging
|
||||
- [ ] Run smoke tests
|
||||
- [ ] Deploy to production with feature flag
|
||||
- [ ] Monitor for 24 hours
|
||||
- [ ] Enable for all tenants
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Optional Enhancements (Future)
|
||||
|
||||
### Soft Delete (2 days)
|
||||
|
||||
- [ ] Add deleted_at column to tenants table
|
||||
- [ ] Implement 30-day retention
|
||||
- [ ] Add restoration endpoint
|
||||
- [ ] Add cleanup job for expired deletions
|
||||
- [ ] Update queries to filter deleted tenants
|
||||
|
||||
### Advanced Features (1 week)
|
||||
|
||||
- [ ] WebSocket progress updates
|
||||
- [ ] Email notifications on completion
|
||||
- [ ] Deletion reports (PDF download)
|
||||
- [ ] Scheduled deletions
|
||||
- [ ] Deletion preview aggregation
|
||||
|
||||
---
|
||||
|
||||
## Sign-Off Checklist
|
||||
|
||||
### Code Quality
|
||||
|
||||
- [ ] All services implemented
|
||||
- [ ] All endpoints tested
|
||||
- [ ] No compiler warnings
|
||||
- [ ] Code reviewed
|
||||
- [ ] Documentation complete
|
||||
|
||||
### Testing
|
||||
|
||||
- [ ] Unit tests passing (>80% coverage)
|
||||
- [ ] Integration tests passing
|
||||
- [ ] E2E tests passing
|
||||
- [ ] Performance tests passing
|
||||
- [ ] Manual testing complete
|
||||
|
||||
### Production Readiness
|
||||
|
||||
- [ ] Monitoring configured
|
||||
- [ ] Alerts configured
|
||||
- [ ] Logging verified
|
||||
- [ ] Rollback plan documented
|
||||
- [ ] Runbook created
|
||||
|
||||
### Security & Compliance
|
||||
|
||||
- [ ] Authorization verified
|
||||
- [ ] Audit logging enabled
|
||||
- [ ] GDPR compliance verified
|
||||
- [ ] Data retention policy documented
|
||||
- [ ] Security review completed
|
||||
|
||||
---
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### Files to Create (3 new services):
|
||||
1. `services/pos/app/services/tenant_deletion_service.py`
|
||||
2. `services/external/app/services/tenant_deletion_service.py`
|
||||
3. `services/alert_processor/app/services/tenant_deletion_service.py`
|
||||
|
||||
### Files to Modify (3 refactored services):
|
||||
1. `services/forecasting/app/services/tenant_deletion_service.py`
|
||||
2. `services/training/app/services/tenant_deletion_service.py`
|
||||
3. `services/notification/app/services/tenant_deletion_service.py`
|
||||
|
||||
### Files to Update (integration):
|
||||
1. `services/auth/app/services/admin_delete.py`
|
||||
|
||||
### Tests to Write (~50 tests):
|
||||
- 10 unit tests (base classes)
|
||||
- 24 service-specific tests (2 per service × 12 services)
|
||||
- 10 integration tests
|
||||
- 6 E2E tests
|
||||
|
||||
### Time Estimate:
|
||||
- Implementation: 4 hours
|
||||
- Testing: 2 days
|
||||
- Deployment: 2 days
|
||||
- **Total: ~5 days**
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
✅ All 12 services have deletion logic
|
||||
✅ All deletion endpoints working
|
||||
✅ Orchestrator coordinating successfully
|
||||
✅ Job tracking persisted to database
|
||||
✅ All tests passing
|
||||
✅ Performance acceptable (<5 min for large tenants)
|
||||
✅ Monitoring in place
|
||||
✅ Documentation complete
|
||||
✅ Production deployment successful
|
||||
|
||||
---
|
||||
|
||||
**Keep this checklist handy and mark items as you complete them!**
|
||||
|
||||
**Remember:** Templates and examples are in QUICK_START_REMAINING_SERVICES.md
|
||||
Reference in New Issue
Block a user