bakery-ia/docs/MIGRATION-CHECKLIST.md

# Production Migration Quick Checklist

This is a condensed checklist for migrating from local dev (Kind + Colima) to production (MicroK8s on Clouding.io VPS).

## Pre-Migration (Do this BEFORE deployment)

### 1. VPS Setup
- [ ] VPS provisioned (Ubuntu 20.04+, 8GB+ RAM, 4+ CPU cores, 100GB+ disk)
- [ ] SSH access configured
- [ ] Domain name registered
- [ ] DNS records configured (A records pointing to VPS IP)

### 2. MicroK8s Installation
```bash
# Install MicroK8s
sudo snap install microk8s --classic --channel=1.28/stable
sudo usermod -a -G microk8s $USER
newgrp microk8s

# Enable required addons
microk8s enable dns hostpath-storage ingress cert-manager metrics-server rbac

# Setup kubectl alias
echo "alias kubectl='microk8s kubectl'" >> ~/.bashrc
source ~/.bashrc
```

### 3. Firewall Configuration
```bash
sudo ufw allow 22/tcp 80/tcp 443/tcp
sudo ufw enable
```

### 4. Configuration Updates

#### Update Domain Names
Edit `infrastructure/kubernetes/overlays/prod/prod-ingress.yaml`:
- [ ] Replace `bakery.yourdomain.com` with your actual domain
- [ ] Replace `api.yourdomain.com` with your actual API domain
- [ ] Replace `monitoring.yourdomain.com` with your actual monitoring domain
- [ ] Update CORS origins with your domains
- [ ] Update cert-manager email address

#### Update Production Secrets
Edit `infrastructure/kubernetes/base/secrets.yaml`:
- [ ] Generate strong passwords: `openssl rand -base64 32`
- [ ] Update all database passwords
- [ ] Update JWT secrets
- [ ] Update API keys
- [ ] **NEVER commit real secrets to git!**

#### Configure Container Registry
Choose one option:

**Option A: Docker Hub (Recommended)**
- [ ] Create Docker Hub account
- [ ] Login: `docker login`
- [ ] Update image names in `infrastructure/kubernetes/overlays/prod/kustomization.yaml`

**Option B: MicroK8s Registry**
- [ ] Enable registry: `microk8s enable registry`
- [ ] Configure insecure registry in `/etc/docker/daemon.json`

### 5. DNS Configuration
Point your domains to VPS IP:
```
Type    Host                Value           TTL
A       bakery              YOUR_VPS_IP     300
A       api                 YOUR_VPS_IP     300
A       monitoring          YOUR_VPS_IP     300
```

- [ ] DNS records configured
- [ ] Wait for DNS propagation (test with `nslookup bakery.yourdomain.com`)

## Deployment Phase

### 6. Build and Push Images

**Using provided script:**
```bash
# Build all images
docker-compose build

# Tag for your registry (Docker Hub example)
./scripts/tag-images.sh YOUR_DOCKERHUB_USERNAME

# Push to registry
./scripts/push-images.sh YOUR_DOCKERHUB_USERNAME
```

**Manual:**
- [ ] Build all Docker images
- [ ] Tag with registry prefix
- [ ] Push to container registry

### 7. Deploy to MicroK8s

**Using provided script (on VPS):**
```bash
# Copy deployment script to VPS
scp scripts/deploy-production.sh user@YOUR_VPS_IP:~/

# SSH to VPS
ssh user@YOUR_VPS_IP

# Clone your repository (or copy kubernetes manifests)
git clone YOUR_REPO_URL
cd bakery_ia

# Run deployment script
./deploy-production.sh
```

**Manual deployment:**
```bash
# On VPS
kubectl apply -k infrastructure/kubernetes/overlays/prod
kubectl get pods -n bakery-ia -w
```

### 8. Verify Deployment

- [ ] All pods running: `kubectl get pods -n bakery-ia`
- [ ] Services created: `kubectl get svc -n bakery-ia`
- [ ] Ingress configured: `kubectl get ingress -n bakery-ia`
- [ ] PVCs bound: `kubectl get pvc -n bakery-ia`
- [ ] Certificates issued: `kubectl get certificate -n bakery-ia`

### 9. Test Application

- [ ] Frontend accessible: `curl -k https://bakery.yourdomain.com`
- [ ] API responding: `curl -k https://api.yourdomain.com/health`
- [ ] SSL certificate valid (Let's Encrypt)
- [ ] Login functionality works
- [ ] Database connections working
- [ ] All microservices healthy

### 10. Setup Monitoring & Backups

**Monitoring:**
- [ ] Prometheus accessible
- [ ] Grafana accessible (if enabled)
- [ ] Set up alerts

**Backups:**
```bash
# Copy backup script to VPS
scp scripts/backup-databases.sh user@YOUR_VPS_IP:~/

# Setup daily backups
crontab -e
# Add: 0 2 * * * ~/backup-databases.sh
```

- [ ] Backup script configured
- [ ] Test backup restoration
- [ ] Set up off-site backup storage

## Post-Deployment

### 11. Security Hardening
- [ ] Change all default passwords
- [ ] Review and update secrets regularly
- [ ] Enable pod security policies
- [ ] Configure network policies
- [ ] Set up monitoring and alerting
- [ ] Review firewall rules
- [ ] Enable audit logging

### 12. Performance Tuning
- [ ] Monitor resource usage: `kubectl top pods -n bakery-ia`
- [ ] Adjust resource limits if needed
- [ ] Configure HPA (Horizontal Pod Autoscaling)
- [ ] Optimize database settings
- [ ] Set up CDN for frontend (optional)

### 13. Documentation
- [ ] Document custom configurations
- [ ] Create runbooks for common operations
- [ ] Document recovery procedures
- [ ] Update team wiki/documentation

## Key Differences from Local Dev

| Aspect | Local (Kind) | Production (MicroK8s) |
|--------|--------------|----------------------|
| Ingress | Custom NGINX | MicroK8s ingress addon |
| Storage Class | `standard` | `microk8s-hostpath` |
| Image Pull | `Never` (local) | `Always` (from registry) |
| SSL Certs | Self-signed | Let's Encrypt |
| Domains | localhost | Real domains |
| Replicas | 1 per service | 2-3 per service |
| Resources | Minimal | Production-grade |
| Secrets | Dev secrets | Production secrets |

## Troubleshooting Quick Reference

### Pods Not Starting
```bash
kubectl describe pod POD_NAME -n bakery-ia
kubectl logs POD_NAME -n bakery-ia
```

### Ingress Not Working
```bash
kubectl describe ingress bakery-ingress-prod -n bakery-ia
kubectl logs -n ingress -l app.kubernetes.io/name=ingress-nginx
sudo netstat -tlnp | grep -E '(80|443)'
```

### SSL Certificate Issues
```bash
kubectl describe certificate bakery-ia-prod-tls-cert -n bakery-ia
kubectl logs -n cert-manager deployment/cert-manager
kubectl get challenges -n bakery-ia
```

### Database Connection Errors
```bash
kubectl get pods -n bakery-ia -l app.kubernetes.io/component=database
kubectl logs -n bakery-ia deployment/auth-db
kubectl exec -n bakery-ia deployment/auth-service -- nc -zv auth-db 5432
```

## Rollback Procedure

If deployment fails:
```bash
# Rollback specific deployment
kubectl rollout undo deployment/DEPLOYMENT_NAME -n bakery-ia

# Check rollout history
kubectl rollout history deployment/DEPLOYMENT_NAME -n bakery-ia

# Rollback to specific revision
kubectl rollout undo deployment/DEPLOYMENT_NAME --to-revision=2 -n bakery-ia
```

## Important Commands

```bash
# View all resources
kubectl get all -n bakery-ia

# Check logs
kubectl logs -f deployment/gateway -n bakery-ia

# Check events
kubectl get events -n bakery-ia --sort-by='.lastTimestamp'

# Resource usage
kubectl top nodes
kubectl top pods -n bakery-ia

# Scale deployment
kubectl scale deployment/gateway --replicas=5 -n bakery-ia

# Restart deployment
kubectl rollout restart deployment/gateway -n bakery-ia

# Execute in pod
kubectl exec -it deployment/gateway -n bakery-ia -- /bin/bash
```

## Success Criteria

Deployment is successful when:
- [ ] All pods are in Running state
- [ ] Application accessible via HTTPS
- [ ] SSL certificate is valid and auto-renewing
- [ ] Database migrations completed
- [ ] All health checks passing
- [ ] Monitoring and alerts configured
- [ ] Backups running successfully
- [ ] Team can access and operate the system
- [ ] Performance meets requirements
- [ ] No critical security issues

## Support Resources

- **Full Migration Guide:** See `docs/K8S-MIGRATION-GUIDE.md`
- **MicroK8s Docs:** https://microk8s.io/docs
- **Kubernetes Docs:** https://kubernetes.io/docs
- **Cert-Manager Docs:** https://cert-manager.io/docs

---

**Note:** This is a condensed checklist. Refer to the full migration guide for detailed explanations and troubleshooting.