23b8523b36
This commit adds complete documentation and tooling for migrating from local development (Kind/Colima on macOS) to production deployment (MicroK8s on Ubuntu VPS at Clouding.io). Documentation added: - K8S-MIGRATION-GUIDE.md: Comprehensive step-by-step migration guide covering all phases from VPS setup to post-deployment operations - MIGRATION-CHECKLIST.md: Quick reference checklist for migration tasks - MIGRATION-SUMMARY.md: High-level overview and key changes summary Configuration updates: - Added storage-patch.yaml for MicroK8s storage class compatibility (changes from 'standard' to 'microk8s-hostpath') - Updated prod/kustomization.yaml to include storage patch Helper scripts: - deploy-production.sh: Interactive deployment script with validation - tag-and-push-images.sh: Automated image tagging and registry push - backup-databases.sh: Database backup script for production Key differences addressed: - Ingress: MicroK8s addon vs custom NGINX - Storage: MicroK8s hostpath vs Kind standard storage - Registry: Container registry configuration for production - SSL: Let's Encrypt production certificates - Domains: Real domain configuration vs localhost - Resources: Production-grade resource limits and scaling The migration guide covers: - VPS setup and MicroK8s installation - Configuration adaptations required - Container registry setup options - SSL certificate configuration - Monitoring and backup setup - Troubleshooting common issues - Security hardening checklist - Rollback procedures All existing Kubernetes manifests remain unchanged and compatible.
290 lines
7.7 KiB
Markdown
290 lines
7.7 KiB
Markdown
# Production Migration Quick Checklist
|
|
|
|
This is a condensed checklist for migrating from local dev (Kind + Colima) to production (MicroK8s on Clouding.io VPS).
|
|
|
|
## Pre-Migration (Do this BEFORE deployment)
|
|
|
|
### 1. VPS Setup
|
|
- [ ] VPS provisioned (Ubuntu 20.04+, 8GB+ RAM, 4+ CPU cores, 100GB+ disk)
|
|
- [ ] SSH access configured
|
|
- [ ] Domain name registered
|
|
- [ ] DNS records configured (A records pointing to VPS IP)
|
|
|
|
### 2. MicroK8s Installation
|
|
```bash
|
|
# Install MicroK8s
|
|
sudo snap install microk8s --classic --channel=1.28/stable
|
|
sudo usermod -a -G microk8s $USER
|
|
newgrp microk8s
|
|
|
|
# Enable required addons
|
|
microk8s enable dns hostpath-storage ingress cert-manager metrics-server rbac
|
|
|
|
# Setup kubectl alias
|
|
echo "alias kubectl='microk8s kubectl'" >> ~/.bashrc
|
|
source ~/.bashrc
|
|
```
|
|
|
|
### 3. Firewall Configuration
|
|
```bash
|
|
sudo ufw allow 22/tcp 80/tcp 443/tcp
|
|
sudo ufw enable
|
|
```
|
|
|
|
### 4. Configuration Updates
|
|
|
|
#### Update Domain Names
|
|
Edit `infrastructure/kubernetes/overlays/prod/prod-ingress.yaml`:
|
|
- [ ] Replace `bakery.yourdomain.com` with your actual domain
|
|
- [ ] Replace `api.yourdomain.com` with your actual API domain
|
|
- [ ] Replace `monitoring.yourdomain.com` with your actual monitoring domain
|
|
- [ ] Update CORS origins with your domains
|
|
- [ ] Update cert-manager email address
|
|
|
|
#### Update Production Secrets
|
|
Edit `infrastructure/kubernetes/base/secrets.yaml`:
|
|
- [ ] Generate strong passwords: `openssl rand -base64 32`
|
|
- [ ] Update all database passwords
|
|
- [ ] Update JWT secrets
|
|
- [ ] Update API keys
|
|
- [ ] **NEVER commit real secrets to git!**
|
|
|
|
#### Configure Container Registry
|
|
Choose one option:
|
|
|
|
**Option A: Docker Hub (Recommended)**
|
|
- [ ] Create Docker Hub account
|
|
- [ ] Login: `docker login`
|
|
- [ ] Update image names in `infrastructure/kubernetes/overlays/prod/kustomization.yaml`
|
|
|
|
**Option B: MicroK8s Registry**
|
|
- [ ] Enable registry: `microk8s enable registry`
|
|
- [ ] Configure insecure registry in `/etc/docker/daemon.json`
|
|
|
|
### 5. DNS Configuration
|
|
Point your domains to VPS IP:
|
|
```
|
|
Type Host Value TTL
|
|
A bakery YOUR_VPS_IP 300
|
|
A api YOUR_VPS_IP 300
|
|
A monitoring YOUR_VPS_IP 300
|
|
```
|
|
|
|
- [ ] DNS records configured
|
|
- [ ] Wait for DNS propagation (test with `nslookup bakery.yourdomain.com`)
|
|
|
|
## Deployment Phase
|
|
|
|
### 6. Build and Push Images
|
|
|
|
**Using provided script:**
|
|
```bash
|
|
# Build all images
|
|
docker-compose build
|
|
|
|
# Tag for your registry (Docker Hub example)
|
|
./scripts/tag-images.sh YOUR_DOCKERHUB_USERNAME
|
|
|
|
# Push to registry
|
|
./scripts/push-images.sh YOUR_DOCKERHUB_USERNAME
|
|
```
|
|
|
|
**Manual:**
|
|
- [ ] Build all Docker images
|
|
- [ ] Tag with registry prefix
|
|
- [ ] Push to container registry
|
|
|
|
### 7. Deploy to MicroK8s
|
|
|
|
**Using provided script (on VPS):**
|
|
```bash
|
|
# Copy deployment script to VPS
|
|
scp scripts/deploy-production.sh user@YOUR_VPS_IP:~/
|
|
|
|
# SSH to VPS
|
|
ssh user@YOUR_VPS_IP
|
|
|
|
# Clone your repository (or copy kubernetes manifests)
|
|
git clone YOUR_REPO_URL
|
|
cd bakery_ia
|
|
|
|
# Run deployment script
|
|
./deploy-production.sh
|
|
```
|
|
|
|
**Manual deployment:**
|
|
```bash
|
|
# On VPS
|
|
kubectl apply -k infrastructure/kubernetes/overlays/prod
|
|
kubectl get pods -n bakery-ia -w
|
|
```
|
|
|
|
### 8. Verify Deployment
|
|
|
|
- [ ] All pods running: `kubectl get pods -n bakery-ia`
|
|
- [ ] Services created: `kubectl get svc -n bakery-ia`
|
|
- [ ] Ingress configured: `kubectl get ingress -n bakery-ia`
|
|
- [ ] PVCs bound: `kubectl get pvc -n bakery-ia`
|
|
- [ ] Certificates issued: `kubectl get certificate -n bakery-ia`
|
|
|
|
### 9. Test Application
|
|
|
|
- [ ] Frontend accessible: `curl -k https://bakery.yourdomain.com`
|
|
- [ ] API responding: `curl -k https://api.yourdomain.com/health`
|
|
- [ ] SSL certificate valid (Let's Encrypt)
|
|
- [ ] Login functionality works
|
|
- [ ] Database connections working
|
|
- [ ] All microservices healthy
|
|
|
|
### 10. Setup Monitoring & Backups
|
|
|
|
**Monitoring:**
|
|
- [ ] Prometheus accessible
|
|
- [ ] Grafana accessible (if enabled)
|
|
- [ ] Set up alerts
|
|
|
|
**Backups:**
|
|
```bash
|
|
# Copy backup script to VPS
|
|
scp scripts/backup-databases.sh user@YOUR_VPS_IP:~/
|
|
|
|
# Setup daily backups
|
|
crontab -e
|
|
# Add: 0 2 * * * ~/backup-databases.sh
|
|
```
|
|
|
|
- [ ] Backup script configured
|
|
- [ ] Test backup restoration
|
|
- [ ] Set up off-site backup storage
|
|
|
|
## Post-Deployment
|
|
|
|
### 11. Security Hardening
|
|
- [ ] Change all default passwords
|
|
- [ ] Review and update secrets regularly
|
|
- [ ] Enable pod security policies
|
|
- [ ] Configure network policies
|
|
- [ ] Set up monitoring and alerting
|
|
- [ ] Review firewall rules
|
|
- [ ] Enable audit logging
|
|
|
|
### 12. Performance Tuning
|
|
- [ ] Monitor resource usage: `kubectl top pods -n bakery-ia`
|
|
- [ ] Adjust resource limits if needed
|
|
- [ ] Configure HPA (Horizontal Pod Autoscaling)
|
|
- [ ] Optimize database settings
|
|
- [ ] Set up CDN for frontend (optional)
|
|
|
|
### 13. Documentation
|
|
- [ ] Document custom configurations
|
|
- [ ] Create runbooks for common operations
|
|
- [ ] Document recovery procedures
|
|
- [ ] Update team wiki/documentation
|
|
|
|
## Key Differences from Local Dev
|
|
|
|
| Aspect | Local (Kind) | Production (MicroK8s) |
|
|
|--------|--------------|----------------------|
|
|
| Ingress | Custom NGINX | MicroK8s ingress addon |
|
|
| Storage Class | `standard` | `microk8s-hostpath` |
|
|
| Image Pull | `Never` (local) | `Always` (from registry) |
|
|
| SSL Certs | Self-signed | Let's Encrypt |
|
|
| Domains | localhost | Real domains |
|
|
| Replicas | 1 per service | 2-3 per service |
|
|
| Resources | Minimal | Production-grade |
|
|
| Secrets | Dev secrets | Production secrets |
|
|
|
|
## Troubleshooting Quick Reference
|
|
|
|
### Pods Not Starting
|
|
```bash
|
|
kubectl describe pod POD_NAME -n bakery-ia
|
|
kubectl logs POD_NAME -n bakery-ia
|
|
```
|
|
|
|
### Ingress Not Working
|
|
```bash
|
|
kubectl describe ingress bakery-ingress-prod -n bakery-ia
|
|
kubectl logs -n ingress -l app.kubernetes.io/name=ingress-nginx
|
|
sudo netstat -tlnp | grep -E '(80|443)'
|
|
```
|
|
|
|
### SSL Certificate Issues
|
|
```bash
|
|
kubectl describe certificate bakery-ia-prod-tls-cert -n bakery-ia
|
|
kubectl logs -n cert-manager deployment/cert-manager
|
|
kubectl get challenges -n bakery-ia
|
|
```
|
|
|
|
### Database Connection Errors
|
|
```bash
|
|
kubectl get pods -n bakery-ia -l app.kubernetes.io/component=database
|
|
kubectl logs -n bakery-ia deployment/auth-db
|
|
kubectl exec -n bakery-ia deployment/auth-service -- nc -zv auth-db 5432
|
|
```
|
|
|
|
## Rollback Procedure
|
|
|
|
If deployment fails:
|
|
```bash
|
|
# Rollback specific deployment
|
|
kubectl rollout undo deployment/DEPLOYMENT_NAME -n bakery-ia
|
|
|
|
# Check rollout history
|
|
kubectl rollout history deployment/DEPLOYMENT_NAME -n bakery-ia
|
|
|
|
# Rollback to specific revision
|
|
kubectl rollout undo deployment/DEPLOYMENT_NAME --to-revision=2 -n bakery-ia
|
|
```
|
|
|
|
## Important Commands
|
|
|
|
```bash
|
|
# View all resources
|
|
kubectl get all -n bakery-ia
|
|
|
|
# Check logs
|
|
kubectl logs -f deployment/gateway -n bakery-ia
|
|
|
|
# Check events
|
|
kubectl get events -n bakery-ia --sort-by='.lastTimestamp'
|
|
|
|
# Resource usage
|
|
kubectl top nodes
|
|
kubectl top pods -n bakery-ia
|
|
|
|
# Scale deployment
|
|
kubectl scale deployment/gateway --replicas=5 -n bakery-ia
|
|
|
|
# Restart deployment
|
|
kubectl rollout restart deployment/gateway -n bakery-ia
|
|
|
|
# Execute in pod
|
|
kubectl exec -it deployment/gateway -n bakery-ia -- /bin/bash
|
|
```
|
|
|
|
## Success Criteria
|
|
|
|
Deployment is successful when:
|
|
- [ ] All pods are in Running state
|
|
- [ ] Application accessible via HTTPS
|
|
- [ ] SSL certificate is valid and auto-renewing
|
|
- [ ] Database migrations completed
|
|
- [ ] All health checks passing
|
|
- [ ] Monitoring and alerts configured
|
|
- [ ] Backups running successfully
|
|
- [ ] Team can access and operate the system
|
|
- [ ] Performance meets requirements
|
|
- [ ] No critical security issues
|
|
|
|
## Support Resources
|
|
|
|
- **Full Migration Guide:** See `docs/K8S-MIGRATION-GUIDE.md`
|
|
- **MicroK8s Docs:** https://microk8s.io/docs
|
|
- **Kubernetes Docs:** https://kubernetes.io/docs
|
|
- **Cert-Manager Docs:** https://cert-manager.io/docs
|
|
|
|
---
|
|
|
|
**Note:** This is a condensed checklist. Refer to the full migration guide for detailed explanations and troubleshooting.
|