Add new infra architecture
This commit is contained in:
Urtzi Alfaro
@@ -886,7 +886,7 @@ microk8s kubectl apply -f infrastructure/ci-cd/tekton/pipelines/
|
||||
microk8s kubectl apply -f infrastructure/ci-cd/tekton/triggers/
|
||||
|
||||
# Apply Flux configurations
|
||||
microk8s kubectl apply -f infrastructure/ci-cd/flux/
|
||||
microk8s kubectl apply -k infrastructure/ci-cd/flux/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -1,120 +0,0 @@
|
||||
# Docker Maintenance Guide for Local Development
|
||||
|
||||
## The Problem
|
||||
|
||||
When developing with Tilt and local Kubernetes (Kind), Docker accumulates:
|
||||
- **Multiple image versions** from each code change (Tilt rebuilds)
|
||||
- **Unused volumes** from previous cluster runs
|
||||
- **Build cache** that grows over time
|
||||
|
||||
This quickly fills up disk space, causing pods to fail with "No space left on device" errors.
|
||||
|
||||
## Quick Fix (When You Hit Disk Issues)
|
||||
|
||||
```bash
|
||||
# Clean up all unused Docker resources
|
||||
docker system prune -a --volumes -f
|
||||
```
|
||||
|
||||
This removes:
|
||||
- All unused images
|
||||
- All unused volumes
|
||||
- All build cache
|
||||
|
||||
**Expected recovery**: 60-100GB
|
||||
|
||||
## Regular Maintenance
|
||||
|
||||
### Option 1: Use the Cleanup Script (Recommended)
|
||||
|
||||
Run the maintenance script weekly:
|
||||
|
||||
```bash
|
||||
./scripts/cleanup-docker.sh
|
||||
```
|
||||
|
||||
Or run it automatically without confirmation:
|
||||
|
||||
```bash
|
||||
./scripts/cleanup-docker.sh --auto
|
||||
```
|
||||
|
||||
### Option 2: Manual Commands
|
||||
|
||||
```bash
|
||||
# Remove images older than 24 hours
|
||||
docker image prune -af --filter "until=24h"
|
||||
|
||||
# Remove unused volumes
|
||||
docker volume prune -f
|
||||
|
||||
# Remove build cache
|
||||
docker builder prune -af
|
||||
```
|
||||
|
||||
### Option 3: Set Up Automated Cleanup
|
||||
|
||||
Add to your crontab (run every Sunday at 2 AM):
|
||||
|
||||
```bash
|
||||
crontab -e
|
||||
# Add this line:
|
||||
0 2 * * 0 /Users/urtzialfaro/Documents/bakery-ia/scripts/cleanup-docker.sh --auto >> /tmp/docker-cleanup.log 2>&1
|
||||
```
|
||||
|
||||
## Monitoring Disk Usage
|
||||
|
||||
### Check Docker disk usage:
|
||||
```bash
|
||||
docker system df
|
||||
```
|
||||
|
||||
### Check Kind node disk usage:
|
||||
```bash
|
||||
docker exec bakery-ia-local-control-plane df -h /var
|
||||
```
|
||||
|
||||
### Alert thresholds:
|
||||
- **< 70%**: Healthy ✅
|
||||
- **70-85%**: Consider cleanup soon ⚠️
|
||||
- **> 85%**: Run cleanup immediately 🚨
|
||||
- **> 95%**: Critical - pods will fail ❌
|
||||
|
||||
## Prevention Tips
|
||||
|
||||
1. **Run cleanup weekly** to prevent accumulation
|
||||
2. **Monitor disk usage** before long dev sessions
|
||||
3. **Delete old Kind clusters** when switching projects:
|
||||
```bash
|
||||
kind delete cluster --name bakery-ia-local
|
||||
```
|
||||
4. **Increase Docker disk allocation** in Docker Desktop settings if you frequently rebuild many services
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Pods in CrashLoopBackOff after disk issues:
|
||||
|
||||
1. Run cleanup (see Quick Fix above)
|
||||
2. Restart failed pods:
|
||||
```bash
|
||||
kubectl get pods -n bakery-ia | grep -E "(CrashLoopBackOff|Error)" | awk '{print $1}' | xargs kubectl delete pod -n bakery-ia
|
||||
```
|
||||
|
||||
### Cleanup didn't free enough space:
|
||||
|
||||
If still above 90% after cleanup:
|
||||
|
||||
```bash
|
||||
# Nuclear option - rebuild everything
|
||||
kind delete cluster --name bakery-ia-local
|
||||
docker system prune -a --volumes -f
|
||||
# Then recreate cluster with your setup scripts
|
||||
```
|
||||
|
||||
## What Happened Today (2026-01-12)
|
||||
|
||||
- **Issue**: Disk was 100% full (113GB/113GB), causing database pods to crash
|
||||
- **Root cause**: 122 unused Docker images + 16 unused volumes + 6GB build cache
|
||||
- **Solution**: Ran `docker system prune -a --volumes -f`
|
||||
- **Result**: Freed 89GB, disk now at 22% usage (24GB/113GB)
|
||||
- **All services recovered successfully**
|
||||
@@ -1,413 +0,0 @@
|
||||
# Infrastructure Reorganization Proposal for Bakery-IA
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This document presents a comprehensive analysis of the current infrastructure organization and proposes a restructured layout that improves maintainability, scalability, and operational efficiency. The proposal is based on a detailed examination of the existing 177 files across 31 directories in the infrastructure folder.
|
||||
|
||||
## Current Infrastructure Analysis
|
||||
|
||||
### Current Structure Overview
|
||||
|
||||
```
|
||||
infrastructure/
|
||||
├── ci-cd/ # 18 files - CI/CD pipeline components
|
||||
├── helm/ # 8 files - Helm charts and scripts
|
||||
├── kubernetes/ # 103 files - Kubernetes manifests and configs
|
||||
├── signoz/ # 11 files - Monitoring dashboards and scripts
|
||||
└── tls/ # 37 files - TLS certificates and generation scripts
|
||||
```
|
||||
|
||||
### Key Findings
|
||||
|
||||
1. **Kubernetes Base Components (103 files)**: The most complex area with:
|
||||
- 20+ service deployments across 15+ microservices
|
||||
- 20+ database configurations (PostgreSQL, RabbitMQ, MinIO)
|
||||
- 19 migration jobs for different services
|
||||
- Infrastructure components (gateway, monitoring, etc.)
|
||||
|
||||
2. **CI/CD Pipeline (18 files)**:
|
||||
- Tekton tasks and pipelines for GitOps workflow
|
||||
- Flux CD configuration for continuous delivery
|
||||
- Gitea configuration for Git repository management
|
||||
|
||||
3. **Monitoring (11 files)**:
|
||||
- SigNoz dashboards for comprehensive observability
|
||||
- Import scripts for dashboard management
|
||||
|
||||
4. **TLS Certificates (37 files)**:
|
||||
- CA certificates and generation scripts
|
||||
- Service-specific certificates (PostgreSQL, Redis, MinIO)
|
||||
- Certificate signing requests and configurations
|
||||
|
||||
### Strengths of Current Organization
|
||||
|
||||
1. **Logical Grouping**: Components are generally well-grouped by function
|
||||
2. **Base/Overlay Pattern**: Kubernetes uses proper base/overlay structure
|
||||
3. **Comprehensive Monitoring**: SigNoz dashboards cover all major aspects
|
||||
4. **Security Focus**: Dedicated TLS certificate management
|
||||
|
||||
### Challenges Identified
|
||||
|
||||
1. **Complexity in Kubernetes Base**: 103 files make navigation difficult
|
||||
2. **Mixed Component Types**: Services, databases, and infrastructure mixed together
|
||||
3. **Limited Environment Separation**: Only dev/prod overlays, no staging
|
||||
4. **Script Scattering**: Automation scripts spread across directories
|
||||
5. **Documentation Gaps**: Some components lack clear documentation
|
||||
|
||||
## Proposed Infrastructure Organization
|
||||
|
||||
### High-Level Structure
|
||||
|
||||
```
|
||||
infrastructure/
|
||||
├── environments/ # Environment-specific configurations
|
||||
├── platform/ # Platform-level infrastructure
|
||||
├── services/ # Application services and microservices
|
||||
├── monitoring/ # Observability and monitoring
|
||||
├── cicd/ # CI/CD pipeline components
|
||||
├── security/ # Security configurations and certificates
|
||||
├── scripts/ # Automation and utility scripts
|
||||
├── docs/ # Infrastructure documentation
|
||||
└── README.md # Top-level infrastructure guide
|
||||
```
|
||||
|
||||
### Detailed Structure Proposal
|
||||
|
||||
```
|
||||
infrastructure/
|
||||
├── environments/ # Environment-specific configurations
|
||||
│ ├── dev/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ │ ├── base/
|
||||
│ │ │ │ ├── namespace.yaml
|
||||
│ │ │ │ ├── configmap.yaml
|
||||
│ │ │ │ ├── secrets.yaml
|
||||
│ │ │ │ └── ingress-https.yaml
|
||||
│ │ │ ├── components/
|
||||
│ │ │ │ ├── databases/
|
||||
│ │ │ │ ├── infrastructure/
|
||||
│ │ │ │ ├── microservices/
|
||||
│ │ │ │ └── cert-manager/
|
||||
│ │ │ ├── configs/
|
||||
│ │ │ ├── cronjobs/
|
||||
│ │ │ ├── jobs/
|
||||
│ │ │ └── migrations/
|
||||
│ │ ├── kustomization.yaml
|
||||
│ │ └── values/
|
||||
│ ├── staging/ # New staging environment
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── values/
|
||||
│ └── prod/
|
||||
│ ├── k8s-manifests/
|
||||
│ ├── terraform/ # Production-specific IaC
|
||||
│ └── values/
|
||||
├── platform/ # Platform-level infrastructure
|
||||
│ ├── cluster/
|
||||
│ │ ├── eks/ # AWS EKS configuration
|
||||
│ │ │ ├── terraform/
|
||||
│ │ │ └── manifests/
|
||||
│ │ └── kind/ # Local development cluster
|
||||
│ │ ├── config.yaml
|
||||
│ │ └── manifests/
|
||||
│ ├── networking/
|
||||
│ │ ├── dns/
|
||||
│ │ ├── load-balancers/
|
||||
│ │ └── ingress/
|
||||
│ │ ├── nginx/
|
||||
│ │ └── cert-manager/
|
||||
│ ├── security/
|
||||
│ │ ├── rbac/
|
||||
│ │ ├── network-policies/
|
||||
│ │ └── tls/
|
||||
│ │ ├── ca/
|
||||
│ │ ├── postgres/
|
||||
│ │ ├── redis/
|
||||
│ │ └── minio/
|
||||
│ └── storage/
|
||||
│ ├── postgres/
|
||||
│ ├── redis/
|
||||
│ └── minio/
|
||||
├── services/ # Application services
|
||||
│ ├── databases/
|
||||
│ │ ├── postgres/
|
||||
│ │ │ ├── k8s-manifests/
|
||||
│ │ │ ├── backups/
|
||||
│ │ │ ├── monitoring/
|
||||
│ │ │ └── maintenance/
|
||||
│ │ ├── redis/
|
||||
│ │ │ ├── configs/
|
||||
│ │ │ └── monitoring/
|
||||
│ │ └── minio/
|
||||
│ │ ├── buckets/
|
||||
│ │ └── policies/
|
||||
│ ├── api-gateway/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ └── microservices/
|
||||
│ ├── auth/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── tenant/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── training/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── forecasting/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── sales/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── external/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── notification/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── inventory/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── recipes/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── suppliers/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── pos/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── orders/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── production/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── procurement/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── orchestrator/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── alert-processor/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── ai-insights/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ ├── demo-session/
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── configs/
|
||||
│ └── frontend/
|
||||
│ ├── k8s-manifests/
|
||||
│ └── configs/
|
||||
├── monitoring/ # Observability stack
|
||||
│ ├── signoz/
|
||||
│ │ ├── manifests/
|
||||
│ │ ├── dashboards/
|
||||
│ │ │ ├── alert-management.json
|
||||
│ │ │ ├── api-performance.json
|
||||
│ │ │ ├── application-performance.json
|
||||
│ │ │ ├── database-performance.json
|
||||
│ │ │ ├── error-tracking.json
|
||||
│ │ │ ├── index.json
|
||||
│ │ │ ├── infrastructure-monitoring.json
|
||||
│ │ │ ├── log-analysis.json
|
||||
│ │ │ ├── system-health.json
|
||||
│ │ │ └── user-activity.json
|
||||
│ │ ├── values-dev.yaml
|
||||
│ │ ├── values-prod.yaml
|
||||
│ │ ├── deploy-signoz.sh
|
||||
│ │ ├── verify-signoz.sh
|
||||
│ │ └── generate-test-traffic.sh
|
||||
│ └── opentelemetry/
|
||||
│ ├── collector/
|
||||
│ └── agent/
|
||||
├── cicd/ # CI/CD pipeline
|
||||
│ ├── gitea/
|
||||
│ │ ├── values.yaml
|
||||
│ │ └── ingress.yaml
|
||||
│ ├── tekton/
|
||||
│ │ ├── tasks/
|
||||
│ │ │ ├── git-clone.yaml
|
||||
│ │ │ ├── detect-changes.yaml
|
||||
│ │ │ ├── kaniko-build.yaml
|
||||
│ │ │ └── update-gitops.yaml
|
||||
│ │ ├── pipelines/
|
||||
│ │ └── triggers/
|
||||
│ └── flux/
|
||||
│ ├── git-repository.yaml
|
||||
│ └── kustomization.yaml
|
||||
├── security/ # Security configurations
|
||||
│ ├── policies/
|
||||
│ │ ├── network-policies.yaml
|
||||
│ │ ├── pod-security.yaml
|
||||
│ │ └── rbac.yaml
|
||||
│ ├── certificates/
|
||||
│ │ ├── ca/
|
||||
│ │ ├── services/
|
||||
│ │ └── rotation-scripts/
|
||||
│ ├── scanning/
|
||||
│ │ ├── trivy/
|
||||
│ │ └── policies/
|
||||
│ └── compliance/
|
||||
│ ├── cis-benchmarks/
|
||||
│ └── audit-scripts/
|
||||
├── scripts/ # Automation scripts
|
||||
│ ├── setup/
|
||||
│ │ ├── generate-certificates.sh
|
||||
│ │ ├── generate-minio-certificates.sh
|
||||
│ │ └── setup-dockerhub-secrets.sh
|
||||
│ ├── deployment/
|
||||
│ │ ├── deploy-signoz.sh
|
||||
│ │ └── verify-signoz.sh
|
||||
│ ├── maintenance/
|
||||
│ │ ├── regenerate_migrations_k8s.sh
|
||||
│ │ └── kubernetes_restart.sh
|
||||
│ └── verification/
|
||||
│ └── verify-registry.sh
|
||||
├── docs/ # Infrastructure documentation
|
||||
│ ├── architecture/
|
||||
│ │ ├── diagrams/
|
||||
│ │ └── decisions/
|
||||
│ ├── operations/
|
||||
│ │ ├── runbooks/
|
||||
│ │ └── troubleshooting/
|
||||
│ ├── onboarding/
|
||||
│ └── reference/
|
||||
│ ├── api/
|
||||
│ └── configurations/
|
||||
└── README.md
|
||||
```
|
||||
|
||||
## Migration Strategy
|
||||
|
||||
### Phase 1: Preparation and Planning
|
||||
|
||||
1. **Inventory Analysis**: Complete detailed inventory of all current files
|
||||
2. **Dependency Mapping**: Identify dependencies between components
|
||||
3. **Impact Assessment**: Determine which components can be moved safely
|
||||
4. **Backup Strategy**: Ensure all files are backed up before migration
|
||||
|
||||
### Phase 2: Non-Critical Components
|
||||
|
||||
1. **Documentation**: Move and update all documentation files
|
||||
2. **Scripts**: Organize automation scripts into new structure
|
||||
3. **Monitoring**: Migrate SigNoz dashboards and configurations
|
||||
4. **CI/CD**: Reorganize pipeline components
|
||||
|
||||
### Phase 3: Environment-Specific Components
|
||||
|
||||
1. **Create Environment Structure**: Set up dev/staging/prod directories
|
||||
2. **Migrate Kubernetes Manifests**: Move base components to appropriate locations
|
||||
3. **Update References**: Ensure all cross-references are corrected
|
||||
4. **Environment Validation**: Test each environment separately
|
||||
|
||||
### Phase 4: Service Components
|
||||
|
||||
1. **Database Migration**: Move database configurations to services/databases
|
||||
2. **Microservice Organization**: Group microservices by domain
|
||||
3. **Infrastructure Components**: Move gateway and other infrastructure
|
||||
4. **Service Validation**: Test each service in isolation
|
||||
|
||||
### Phase 5: Finalization
|
||||
|
||||
1. **Integration Testing**: Test complete infrastructure workflow
|
||||
2. **Documentation Update**: Finalize all documentation
|
||||
3. **Team Training**: Conduct training on new structure
|
||||
4. **Cleanup**: Remove old structure and temporary files
|
||||
|
||||
## Benefits of Proposed Structure
|
||||
|
||||
### 1. Improved Navigation
|
||||
- **Clear Hierarchy**: Logical grouping by function and environment
|
||||
- **Consistent Patterns**: Standardized structure across all components
|
||||
- **Reduced Cognitive Load**: Easier to find specific components
|
||||
|
||||
### 2. Enhanced Maintainability
|
||||
- **Environment Isolation**: Clear separation of dev/staging/prod
|
||||
- **Component Grouping**: Related components grouped together
|
||||
- **Standardized Structure**: Consistent patterns across services
|
||||
|
||||
### 3. Better Scalability
|
||||
- **Modular Design**: Easy to add new services or environments
|
||||
- **Domain Separation**: Services organized by business domain
|
||||
- **Infrastructure Independence**: Platform components separate from services
|
||||
|
||||
### 4. Improved Security
|
||||
- **Centralized Security**: All security configurations in one place
|
||||
- **Environment-Specific Policies**: Tailored security for each environment
|
||||
- **Better Secret Management**: Clear structure for sensitive data
|
||||
|
||||
### 5. Enhanced Observability
|
||||
- **Comprehensive Monitoring**: All observability tools grouped
|
||||
- **Standardized Dashboards**: Consistent monitoring across services
|
||||
- **Centralized Logging**: Better log management structure
|
||||
|
||||
## Implementation Considerations
|
||||
|
||||
### Tools and Technologies
|
||||
- **Terraform**: For infrastructure as code (IaC)
|
||||
- **Kustomize**: For Kubernetes manifest management
|
||||
- **Helm**: For complex application deployments
|
||||
- **SOPS/Sealed Secrets**: For secret management
|
||||
- **Trivy**: For vulnerability scanning
|
||||
|
||||
### Team Adaptation
|
||||
- **Training Plan**: Develop comprehensive training materials
|
||||
- **Migration Guide**: Create step-by-step migration documentation
|
||||
- **Support Period**: Provide dedicated support during transition
|
||||
- **Feedback Mechanism**: Establish channels for team feedback
|
||||
|
||||
### Risk Mitigation
|
||||
- **Phased Approach**: Implement changes incrementally
|
||||
- **Rollback Plan**: Develop comprehensive rollback procedures
|
||||
- **Testing Strategy**: Implement thorough testing at each phase
|
||||
- **Monitoring**: Enhanced monitoring during migration period
|
||||
|
||||
## Expected Outcomes
|
||||
|
||||
1. **Reduced Time-to-Find**: 40-60% reduction in time spent locating files
|
||||
2. **Improved Deployment Speed**: 25-35% faster deployment cycles
|
||||
3. **Enhanced Collaboration**: Better team coordination and understanding
|
||||
4. **Reduced Errors**: 30-50% reduction in configuration errors
|
||||
5. **Better Scalability**: Easier to add new services and features
|
||||
|
||||
## Conclusion
|
||||
|
||||
The proposed infrastructure reorganization represents a significant improvement over the current structure. By implementing a clear, logical hierarchy with proper separation of concerns, the new organization will:
|
||||
|
||||
- **Improve operational efficiency** through better navigation and maintainability
|
||||
- **Enhance security** with centralized security management
|
||||
- **Support growth** with a scalable, modular design
|
||||
- **Reduce errors** through standardized patterns and structures
|
||||
- **Facilitate collaboration** with intuitive organization
|
||||
|
||||
The key to successful implementation is a phased approach with thorough testing and team involvement at each stage. With proper planning and execution, this reorganization will provide long-term benefits for the Bakery-IA project's infrastructure management.
|
||||
|
||||
## Appendix: File Migration Mapping
|
||||
|
||||
### Current → Proposed Mapping
|
||||
|
||||
**Kubernetes Components:**
|
||||
- `infrastructure/kubernetes/base/components/*` → `infrastructure/services/microservices/*/`
|
||||
- `infrastructure/kubernetes/base/components/databases/*` → `infrastructure/services/databases/*/`
|
||||
- `infrastructure/kubernetes/base/migrations/*` → `infrastructure/services/microservices/*/migrations/`
|
||||
- `infrastructure/kubernetes/base/configs/*` → `infrastructure/environments/*/values/`
|
||||
|
||||
**CI/CD Components:**
|
||||
- `infrastructure/ci-cd/*` → `infrastructure/cicd/*/`
|
||||
|
||||
**Monitoring Components:**
|
||||
- `infrastructure/signoz/*` → `infrastructure/monitoring/signoz/*/`
|
||||
- `infrastructure/helm/*` → `infrastructure/monitoring/signoz/*/` (signoz-related)
|
||||
|
||||
**Security Components:**
|
||||
- `infrastructure/tls/*` → `infrastructure/security/certificates/*/`
|
||||
|
||||
**Scripts:**
|
||||
- `infrastructure/kubernetes/*.sh` → `infrastructure/scripts/*/`
|
||||
- `infrastructure/helm/*.sh` → `infrastructure/scripts/deployment/*/`
|
||||
- `infrastructure/tls/*.sh` → `infrastructure/scripts/setup/*/`
|
||||
|
||||
This mapping provides a clear path for migrating each component to its new location while maintaining functionality and relationships between components.
|
||||
@@ -81,7 +81,7 @@ For production deployment on clouding.io with Kubernetes:
|
||||
3. Configure production-specific values
|
||||
4. Deploy using the production kustomization:
|
||||
```bash
|
||||
kubectl apply -k infrastructure/kubernetes/environments/production/
|
||||
kubectl apply -k infrastructure/environments/prod/k8s-manifests
|
||||
```
|
||||
|
||||
## 🤝 Contributing
|
||||
|
||||
335
Tiltfile
335
Tiltfile
@@ -17,6 +17,43 @@
|
||||
# =============================================================================
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# PREPULL BASE IMAGES STEP - CRITICAL FIRST STEP
|
||||
# =============================================================================
|
||||
|
||||
# Run the prepull script first - if this fails, don't continue
|
||||
local_resource(
|
||||
'prepull-base-images',
|
||||
cmd='''#!/usr/bin/env bash
|
||||
echo "=========================================="
|
||||
echo "PREPULLING BASE IMAGES - CRITICAL STEP"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Run the prepull script
|
||||
if ./scripts/prepull-base-images.sh; then
|
||||
echo ""
|
||||
echo "✓ Base images prepull completed successfully"
|
||||
echo "=========================================="
|
||||
echo "CONTINUING WITH TILT SETUP..."
|
||||
echo "=========================================="
|
||||
exit 0
|
||||
else
|
||||
echo ""
|
||||
echo "❌ Base images prepull FAILED - stopping Tilt execution"
|
||||
echo "This usually happens due to Docker Hub rate limits"
|
||||
echo "Please try again later or configure Docker Hub credentials"
|
||||
echo "=========================================="
|
||||
# Exit with error code to prevent further execution
|
||||
exit 1
|
||||
fi
|
||||
''',
|
||||
labels=['00-prepull'],
|
||||
auto_init=True,
|
||||
allow_parallel=False
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# TILT CONFIGURATION
|
||||
# =============================================================================
|
||||
@@ -191,132 +228,68 @@ Monitoring:
|
||||
Applying security configurations...
|
||||
""")
|
||||
|
||||
# Create Docker Hub secret for image pulls (if credentials are available)
|
||||
local_resource(
|
||||
'dockerhub-secret',
|
||||
cmd='''
|
||||
echo "Setting up Docker Hub image pull secret..."
|
||||
|
||||
# Check if Docker Hub credentials are available
|
||||
if [ -n "$DOCKERHUB_USERNAME" ] && [ -n "$DOCKERHUB_PASSWORD" ]; then
|
||||
echo " Found DOCKERHUB_USERNAME and DOCKERHUB_PASSWORD environment variables"
|
||||
./infrastructure/kubernetes/create-dockerhub-secret.sh
|
||||
elif [ -f "$HOME/.docker/config.json" ]; then
|
||||
echo " Attempting to use Docker CLI credentials..."
|
||||
./infrastructure/kubernetes/create-dockerhub-secret.sh
|
||||
else
|
||||
echo " Docker Hub credentials not found"
|
||||
echo " To enable automatic Docker Hub authentication:"
|
||||
echo " 1. Run 'docker login', OR"
|
||||
echo " 2. Set environment variables:"
|
||||
echo " export DOCKERHUB_USERNAME='your-username'"
|
||||
echo " export DOCKERHUB_PASSWORD='your-password-or-token'"
|
||||
echo ""
|
||||
echo " Continuing without Docker Hub authentication..."
|
||||
echo " (This is OK for local development using local registry)"
|
||||
fi
|
||||
''',
|
||||
labels=['00-security'],
|
||||
auto_init=True
|
||||
)
|
||||
|
||||
# Apply security configurations before loading main manifests
|
||||
local_resource(
|
||||
'security-setup',
|
||||
cmd='''
|
||||
echo "Applying security secrets and configurations..."
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets.yaml
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets/postgres-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets/redis-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/kubernetes/base/configs/postgres-init-config.yaml
|
||||
kubectl apply -f infrastructure/kubernetes/base/configmaps/postgres-logging-config.yaml
|
||||
|
||||
# First, ensure all required namespaces exist
|
||||
echo "Creating namespaces..."
|
||||
kubectl apply -f infrastructure/namespaces/bakery-ia.yaml
|
||||
kubectl apply -f infrastructure/namespaces/tekton-pipelines.yaml
|
||||
kubectl apply -f infrastructure/namespaces/flux-system.yaml
|
||||
|
||||
# Wait for namespaces to be ready
|
||||
echo "Waiting for namespaces to be ready..."
|
||||
for ns in bakery-ia tekton-pipelines flux-system; do
|
||||
until kubectl get namespace $ns 2>/dev/null; do
|
||||
echo "Waiting for namespace $ns to be created..."
|
||||
sleep 2
|
||||
done
|
||||
echo "Namespace $ns is available"
|
||||
done
|
||||
|
||||
# Apply common secrets and configs
|
||||
kubectl apply -f infrastructure/environments/common/configs/configmap.yaml
|
||||
kubectl apply -f infrastructure/environments/common/configs/secrets.yaml
|
||||
|
||||
# Apply database secrets and configs
|
||||
kubectl apply -f infrastructure/platform/storage/postgres/secrets/postgres-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/platform/storage/postgres/configs/postgres-init-config.yaml
|
||||
kubectl apply -f infrastructure/platform/storage/postgres/configs/postgres-logging-config.yaml
|
||||
|
||||
# Apply Redis secrets
|
||||
kubectl apply -f infrastructure/platform/storage/redis/secrets/redis-tls-secret.yaml
|
||||
|
||||
# Apply MinIO secrets and configs
|
||||
kubectl apply -f infrastructure/platform/storage/minio/minio-secrets.yaml
|
||||
kubectl apply -f infrastructure/platform/storage/minio/secrets/minio-tls-secret.yaml
|
||||
|
||||
# Apply Mail/SMTP secrets
|
||||
kubectl apply -f infrastructure/platform/mail/mailu/mailu-secrets.yaml
|
||||
|
||||
# Apply CI/CD secrets
|
||||
kubectl apply -f infrastructure/cicd/tekton/secrets/secrets.yaml
|
||||
|
||||
echo "Security configurations applied"
|
||||
''',
|
||||
resource_deps=['dockerhub-secret'],
|
||||
resource_deps=['prepull-base-images'], # Removed dockerhub-secret dependency
|
||||
labels=['00-security'],
|
||||
auto_init=True
|
||||
)
|
||||
|
||||
# Verify TLS certificates are mounted correctly
|
||||
local_resource(
|
||||
'verify-tls',
|
||||
cmd='''
|
||||
echo "Verifying TLS configuration..."
|
||||
sleep 5 # Wait for pods to be ready
|
||||
|
||||
# Check if auth-db pod exists and has TLS certs
|
||||
AUTH_POD=$(kubectl get pods -n bakery-ia -l app.kubernetes.io/name=auth-db -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$AUTH_POD" ]; then
|
||||
echo " Checking PostgreSQL TLS certificates..."
|
||||
kubectl exec -n bakery-ia "$AUTH_POD" -- ls -la /tls/ 2>/dev/null && \
|
||||
echo " PostgreSQL TLS certificates mounted" || \
|
||||
echo " PostgreSQL TLS certificates not found (pods may still be starting)"
|
||||
fi
|
||||
|
||||
# Check if redis pod exists and has TLS certs
|
||||
REDIS_POD=$(kubectl get pods -n bakery-ia -l app.kubernetes.io/name=redis -o jsonpath='{.items[0].metadata.name}' 2>/dev/null || echo "")
|
||||
|
||||
if [ -n "$REDIS_POD" ]; then
|
||||
echo " Checking Redis TLS certificates..."
|
||||
kubectl exec -n bakery-ia "$REDIS_POD" -- ls -la /tls/ 2>/dev/null && \
|
||||
echo " Redis TLS certificates mounted" || \
|
||||
echo " Redis TLS certificates not found (pods may still be starting)"
|
||||
fi
|
||||
|
||||
echo "TLS verification complete"
|
||||
''',
|
||||
resource_deps=['auth-db', 'redis'],
|
||||
auto_init=True,
|
||||
labels=['00-security']
|
||||
)
|
||||
|
||||
# Verify PVCs are bound
|
||||
local_resource(
|
||||
'verify-pvcs',
|
||||
cmd='''
|
||||
echo "Verifying PersistentVolumeClaims..."
|
||||
kubectl get pvc -n bakery-ia | grep -E "NAME|db-pvc" || echo " PVCs not yet bound"
|
||||
PVC_COUNT=$(kubectl get pvc -n bakery-ia -o json | jq '.items | length')
|
||||
echo " Found $PVC_COUNT PVCs"
|
||||
echo "PVC verification complete"
|
||||
''',
|
||||
resource_deps=['auth-db'],
|
||||
auto_init=True,
|
||||
labels=['00-security']
|
||||
)
|
||||
|
||||
# Install and verify cert-manager
|
||||
local_resource(
|
||||
'cert-manager-install',
|
||||
cmd='''
|
||||
echo "Installing cert-manager..."
|
||||
|
||||
# Check if cert-manager CRDs already exist
|
||||
if kubectl get crd certificates.cert-manager.io >/dev/null 2>&1; then
|
||||
echo " cert-manager CRDs already installed"
|
||||
else
|
||||
echo " Installing cert-manager v1.13.2..."
|
||||
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.2/cert-manager.yaml
|
||||
|
||||
echo " Waiting for cert-manager to be ready..."
|
||||
kubectl wait --for=condition=available --timeout=120s deployment/cert-manager -n cert-manager
|
||||
kubectl wait --for=condition=available --timeout=120s deployment/cert-manager-webhook -n cert-manager
|
||||
|
||||
echo " cert-manager installed and ready"
|
||||
fi
|
||||
|
||||
echo "cert-manager verification complete"
|
||||
''',
|
||||
labels=['00-security'],
|
||||
auto_init=True
|
||||
)
|
||||
|
||||
# =============================================================================
|
||||
# LOAD KUBERNETES MANIFESTS
|
||||
# =============================================================================
|
||||
|
||||
k8s_yaml(kustomize('infrastructure/kubernetes/overlays/dev'))
|
||||
# Load the main kustomize overlay for the dev environment
|
||||
k8s_yaml(kustomize('infrastructure/environments/dev/k8s-manifests'))
|
||||
|
||||
# =============================================================================
|
||||
# DOCKER BUILD HELPERS
|
||||
@@ -509,6 +482,9 @@ k8s_resource('nominatim', labels=['01-infrastructure'])
|
||||
k8s_resource('minio', resource_deps=['security-setup'], labels=['01-infrastructure'])
|
||||
k8s_resource('minio-bucket-init', resource_deps=['minio'], labels=['01-infrastructure'])
|
||||
|
||||
# Mail Infrastructure (Mailu)
|
||||
k8s_resource('mailu-front', resource_deps=['security-setup'], labels=['01-infrastructure'])
|
||||
|
||||
# =============================================================================
|
||||
# MONITORING RESOURCES - SigNoz (Unified Observability)
|
||||
# =============================================================================
|
||||
@@ -520,15 +496,6 @@ local_resource(
|
||||
echo "Deploying SigNoz Monitoring Stack..."
|
||||
echo ""
|
||||
|
||||
# Ensure Docker Hub secret exists in bakery-ia namespace
|
||||
echo "Ensuring Docker Hub secret exists in bakery-ia namespace..."
|
||||
if ! kubectl get secret dockerhub-creds -n bakery-ia &>/dev/null; then
|
||||
echo " Docker Hub secret not found, attempting to create..."
|
||||
./infrastructure/kubernetes/create-dockerhub-secret.sh || echo " Continuing without Docker Hub authentication..."
|
||||
else
|
||||
echo " Docker Hub secret exists"
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# Check if SigNoz is already deployed
|
||||
if helm list -n bakery-ia | grep -q signoz; then
|
||||
@@ -544,7 +511,7 @@ local_resource(
|
||||
# Install SigNoz with custom values in the bakery-ia namespace
|
||||
helm upgrade --install signoz signoz/signoz \
|
||||
-n bakery-ia \
|
||||
-f infrastructure/helm/signoz-values-dev.yaml \
|
||||
-f infrastructure/monitoring/signoz/signoz-values-dev.yaml \
|
||||
--timeout 10m \
|
||||
--wait
|
||||
|
||||
@@ -568,43 +535,6 @@ local_resource(
|
||||
auto_init=False,
|
||||
)
|
||||
|
||||
# Track SigNoz pods in Tilt UI using workload tracking
|
||||
# These will automatically discover pods once SigNoz is deployed
|
||||
local_resource(
|
||||
'signoz-status',
|
||||
cmd='''
|
||||
echo "SigNoz Status Check"
|
||||
echo ""
|
||||
|
||||
# Check pod status
|
||||
echo "Current SigNoz pods:"
|
||||
kubectl get pods -n bakery-ia -l app.kubernetes.io/instance=signoz -o wide 2>/dev/null || echo "No pods found"
|
||||
|
||||
echo ""
|
||||
echo "SigNoz Services:"
|
||||
kubectl get svc -n bakery-ia -l app.kubernetes.io/instance=signoz 2>/dev/null || echo "No services found"
|
||||
|
||||
# Check if all pods are ready
|
||||
TOTAL_PODS=$(kubectl get pods -n bakery-ia -l app.kubernetes.io/instance=signoz --no-headers 2>/dev/null | wc -l | tr -d ' ')
|
||||
READY_PODS=$(kubectl get pods -n bakery-ia -l app.kubernetes.io/instance=signoz --field-selector=status.phase=Running --no-headers 2>/dev/null | wc -l | tr -d ' ')
|
||||
|
||||
if [ "$TOTAL_PODS" -gt 0 ]; then
|
||||
echo ""
|
||||
echo "Pod Status: $READY_PODS/$TOTAL_PODS ready"
|
||||
|
||||
if [ "$READY_PODS" -eq "$TOTAL_PODS" ]; then
|
||||
echo "All SigNoz pods are running!"
|
||||
echo ""
|
||||
echo "Access SigNoz at: https://monitoring.bakery-ia.local"
|
||||
echo "Credentials: admin / admin"
|
||||
else
|
||||
echo "Waiting for pods to become ready..."
|
||||
fi
|
||||
fi
|
||||
''',
|
||||
labels=['05-monitoring'],
|
||||
auto_init=False,
|
||||
)
|
||||
|
||||
# Optional exporters (in monitoring namespace) - DISABLED since using SigNoz
|
||||
# k8s_resource('node-exporter', labels=['05-monitoring'])
|
||||
@@ -774,6 +704,98 @@ watch_settings(
|
||||
]
|
||||
)
|
||||
|
||||
# =============================================================================
|
||||
# CI/CD INFRASTRUCTURE - MANUAL TRIGGERS
|
||||
# =============================================================================
|
||||
|
||||
# Tekton Pipelines - Manual trigger for local development
|
||||
local_resource(
|
||||
'tekton-pipelines',
|
||||
cmd='''
|
||||
echo "Setting up Tekton Pipelines for CI/CD..."
|
||||
echo ""
|
||||
|
||||
# Check if Tekton CRDs are already installed
|
||||
if kubectl get crd pipelines.tekton.dev >/dev/null 2>&1; then
|
||||
echo " Tekton CRDs already installed"
|
||||
else
|
||||
echo " Installing Tekton v0.57.0..."
|
||||
kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml
|
||||
|
||||
echo " Waiting for Tekton to be ready..."
|
||||
kubectl wait --for=condition=available --timeout=180s deployment/tekton-pipelines-controller -n tekton-pipelines
|
||||
kubectl wait --for=condition=available --timeout=180s deployment/tekton-pipelines-webhook -n tekton-pipelines
|
||||
|
||||
echo " Tekton installed and ready"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Applying Tekton configurations..."
|
||||
kubectl apply -f infrastructure/cicd/tekton/kustomization.yaml
|
||||
kubectl apply -f infrastructure/cicd/tekton/rbac/
|
||||
kubectl apply -f infrastructure/cicd/tekton/tasks/
|
||||
kubectl apply -f infrastructure/cicd/tekton/pipelines/
|
||||
|
||||
echo ""
|
||||
echo "Tekton setup complete!"
|
||||
echo "To check status: kubectl get pods -n tekton-pipelines"
|
||||
''',
|
||||
labels=['99-cicd'],
|
||||
auto_init=False, # Manual trigger only
|
||||
)
|
||||
|
||||
# Flux CD - Manual trigger for GitOps
|
||||
local_resource(
|
||||
'flux-cd',
|
||||
cmd='''
|
||||
echo "Setting up Flux CD for GitOps..."
|
||||
echo ""
|
||||
|
||||
# Check if Flux CRDs are already installed
|
||||
if kubectl get crd gitrepositories.source.toolkit.fluxcd.io >/dev/null 2>&1; then
|
||||
echo " Flux CRDs already installed"
|
||||
else
|
||||
echo " Installing Flux v2.2.3..."
|
||||
curl -sL https://fluxcd.io/install.sh | sudo bash
|
||||
flux install --version=latest
|
||||
|
||||
echo " Flux installed and ready"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Applying Flux configurations..."
|
||||
kubectl apply -f infrastructure/cicd/flux/
|
||||
|
||||
echo ""
|
||||
echo "Flux setup complete!"
|
||||
echo "To check status: flux check"
|
||||
''',
|
||||
labels=['99-cicd'],
|
||||
auto_init=False, # Manual trigger only
|
||||
)
|
||||
|
||||
# Gitea - Manual trigger for local Git server
|
||||
local_resource(
|
||||
'gitea',
|
||||
cmd='''
|
||||
echo "Setting up Gitea for local Git server..."
|
||||
echo ""
|
||||
|
||||
# Apply Gitea configurations
|
||||
kubectl create namespace gitea || true
|
||||
kubectl apply -f infrastructure/cicd/gitea/
|
||||
|
||||
echo ""
|
||||
echo "Gitea setup complete!"
|
||||
echo "Access Gitea at: http://gitea.local (add to /etc/hosts)"
|
||||
echo "Default credentials: admin/admin123 (change after first login)"
|
||||
echo "To check status: kubectl get pods -n gitea"
|
||||
''',
|
||||
labels=['99-cicd'],
|
||||
auto_init=False, # Manual trigger only
|
||||
)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# STARTUP SUMMARY
|
||||
# =============================================================================
|
||||
@@ -804,11 +826,16 @@ Access your application:
|
||||
|
||||
SigNoz (Unified Observability):
|
||||
Deploy via Tilt: Trigger 'signoz-deployment' resource
|
||||
Manual deploy: ./infrastructure/helm/deploy-signoz.sh dev
|
||||
Manual deploy: ./infrastructure/monitoring/signoz/deploy-signoz.sh dev
|
||||
Access (if deployed): https://monitoring.bakery-ia.local
|
||||
Username: admin
|
||||
Password: admin
|
||||
|
||||
CI/CD Infrastructure (Manual Triggers):
|
||||
Tekton: Trigger 'tekton-pipelines' resource
|
||||
Flux: Trigger 'flux-cd' resource
|
||||
Gitea: Trigger 'gitea' resource
|
||||
|
||||
Verify security:
|
||||
kubectl get pvc -n bakery-ia
|
||||
kubectl get secrets -n bakery-ia | grep tls
|
||||
|
||||
@@ -685,7 +685,7 @@ kubectl scale deployment auth-service -n bakery-ia --replicas=2
|
||||
# 2. Install MicroK8s (follow pilot launch guide)
|
||||
# 3. Copy latest backup to new VPS
|
||||
# 4. Deploy infrastructure and databases
|
||||
kubectl apply -k infrastructure/kubernetes/overlays/prod
|
||||
kubectl apply -k infrastructure/environments/prod/k8s-manifests
|
||||
|
||||
# 5. Wait for databases to be ready
|
||||
kubectl wait --for=condition=ready pod -l app.kubernetes.io/component=database -n bakery-ia
|
||||
@@ -699,7 +699,7 @@ for backup in /backups/latest/*.sql; do
|
||||
done
|
||||
|
||||
# 7. Deploy services
|
||||
kubectl apply -k infrastructure/kubernetes/overlays/prod
|
||||
kubectl apply -k infrastructure/environments/prod/k8s-manifests
|
||||
|
||||
# 8. Update DNS to point to new VPS
|
||||
# 9. Verify all services healthy
|
||||
@@ -830,12 +830,12 @@ nproc
|
||||
kubectl scale deployment orders-service -n bakery-ia --replicas=5
|
||||
|
||||
# Or update in kustomization for persistence
|
||||
# Edit: infrastructure/kubernetes/overlays/prod/kustomization.yaml
|
||||
# Edit: infrastructure/environments/prod/k8s-manifests/kustomization.yaml
|
||||
replicas:
|
||||
- name: orders-service
|
||||
count: 5
|
||||
|
||||
kubectl apply -k infrastructure/kubernetes/overlays/prod
|
||||
kubectl apply -k infrastructure/environments/prod/k8s-manifests
|
||||
```
|
||||
|
||||
### Auto-Scaling (HPA)
|
||||
@@ -976,7 +976,7 @@ resources:
|
||||
memory: "1Gi" # Increased from 512Mi
|
||||
|
||||
# 4. Redeploy
|
||||
kubectl apply -k infrastructure/kubernetes/overlays/prod
|
||||
kubectl apply -k infrastructure/environments/prod/k8s-manifests
|
||||
```
|
||||
|
||||
#### Incident: Certificate Expired
|
||||
|
||||
@@ -324,12 +324,12 @@ log_line_prefix = '%t [%p]: [%l-1] user=%u,db=%d,app=%a,client=%h '
|
||||
**Renewal Process:**
|
||||
```bash
|
||||
# 1. Regenerate certificates (90 days before expiry)
|
||||
cd infrastructure/tls && ./generate-certificates.sh
|
||||
cd infrastructure/security/certificates && ./generate-certificates.sh
|
||||
|
||||
# 2. Update Kubernetes secrets
|
||||
kubectl delete secret postgres-tls redis-tls -n bakery-ia
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets/postgres-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets/redis-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/environments/dev/k8s-manifests/base/secrets/postgres-tls-secret.yaml
|
||||
kubectl apply -f infrastructure/environments/dev/k8s-manifests/base/secrets/redis-tls-secret.yaml
|
||||
|
||||
# 3. Restart database pods (automatic)
|
||||
kubectl rollout restart deployment -l app.kubernetes.io/component=database -n bakery-ia
|
||||
@@ -351,7 +351,7 @@ kubectl rollout restart deployment -l app.kubernetes.io/component=database -n ba
|
||||
./scripts/update-k8s-secrets.sh
|
||||
|
||||
# 4. Apply secrets
|
||||
kubectl apply -f infrastructure/kubernetes/base/secrets.yaml
|
||||
kubectl apply -f infrastructure/environments/common/configs/secrets.yaml
|
||||
|
||||
# 5. Restart databases and services
|
||||
kubectl rollout restart deployment -n bakery-ia
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Add this stage at the top of each service Dockerfile
|
||||
FROM python:3.11-slim AS shared
|
||||
FROM localhost:5000/python_3.11-slim AS shared
|
||||
WORKDIR /shared
|
||||
COPY shared/ /shared/
|
||||
|
||||
# Then your main service stage
|
||||
FROM python:3.11-slim
|
||||
FROM localhost:5000/python_3.11-slim
|
||||
|
||||
# Create non-root user for security
|
||||
RUN groupadd -r appgroup && useradd -r -g appgroup appuser
|
||||
|
||||
119
infrastructure/NAMESPACES.md
Normal file
119
infrastructure/NAMESPACES.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Bakery-IA Namespace Management
|
||||
|
||||
## Overview
|
||||
|
||||
This document explains the namespace strategy for the Bakery-IA platform and how to properly manage namespaces during deployment.
|
||||
|
||||
## Namespace Architecture
|
||||
|
||||
The Bakery-IA platform uses the following namespaces:
|
||||
|
||||
### Core Namespaces
|
||||
|
||||
1. **`bakery-ia`** - Main application namespace
|
||||
- Contains all microservices, databases, and application components
|
||||
- Defined in: `infrastructure/namespaces/bakery-ia.yaml`
|
||||
|
||||
2. **`tekton-pipelines`** - CI/CD pipeline namespace
|
||||
- Contains Tekton pipeline resources, tasks, and triggers
|
||||
- Defined in: `infrastructure/namespaces/tekton-pipelines.yaml`
|
||||
|
||||
3. **`flux-system`** - GitOps namespace
|
||||
- Contains Flux CD components for GitOps deployments
|
||||
- Defined in: `infrastructure/namespaces/flux-system.yaml`
|
||||
|
||||
### Infrastructure Namespaces
|
||||
|
||||
Additional namespaces may be created for:
|
||||
- Monitoring components
|
||||
- Logging components
|
||||
- Security components
|
||||
|
||||
## Deployment Order
|
||||
|
||||
**CRITICAL**: Namespaces must be created BEFORE any resources that depend on them.
|
||||
|
||||
### Correct Deployment Sequence
|
||||
|
||||
```bash
|
||||
# 1. Create namespaces first
|
||||
kubectl apply -f infrastructure/namespaces/
|
||||
|
||||
# 2. Apply common configurations (depends on bakery-ia namespace)
|
||||
kubectl apply -f infrastructure/environments/common/configs/
|
||||
|
||||
# 3. Apply platform components
|
||||
kubectl apply -f infrastructure/platform/
|
||||
|
||||
# 4. Apply CI/CD components (depends on tekton-pipelines and flux-system)
|
||||
kubectl apply -f infrastructure/cicd/
|
||||
|
||||
# 5. Apply monitoring components
|
||||
kubectl apply -f infrastructure/monitoring/
|
||||
```
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
### Issue: "namespace not found" errors
|
||||
|
||||
**Symptoms**: Errors like:
|
||||
```
|
||||
Error from server (NotFound): error when creating "path/to/resource.yaml": namespaces "[namespace-name]" not found
|
||||
```
|
||||
|
||||
**Solutions**:
|
||||
|
||||
1. **Ensure namespaces are created first** - Use the deployment script that applies namespaces before other resources
|
||||
|
||||
2. **Check for templating issues** - If you see names like `[redacted secret rabbitmq-secrets:RABBITMQ_USER]-ia`, there may be environment variable substitution happening incorrectly
|
||||
|
||||
3. **Verify namespace YAML files** - Ensure the namespace files exist and are properly formatted
|
||||
|
||||
### Issue: Resource conflicts across namespaces
|
||||
|
||||
**Solution**: Use proper namespace isolation and RBAC policies to prevent cross-namespace conflicts.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Namespace Isolation**: Keep resources properly isolated by namespace
|
||||
2. **RBAC**: Use namespace-specific RBAC roles and bindings
|
||||
3. **Resource Quotas**: Apply resource quotas per namespace
|
||||
4. **Network Policies**: Use network policies to control cross-namespace communication
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Verify namespaces exist
|
||||
|
||||
```bash
|
||||
kubectl get namespaces
|
||||
```
|
||||
|
||||
### Check namespace labels
|
||||
|
||||
```bash
|
||||
kubectl get namespace bakery-ia --show-labels
|
||||
```
|
||||
|
||||
### View namespace events
|
||||
|
||||
```bash
|
||||
kubectl describe namespace bakery-ia
|
||||
```
|
||||
|
||||
## Migration from Old Structure
|
||||
|
||||
If you're migrating from the old structure where namespaces were scattered across different directories:
|
||||
|
||||
1. **Remove old namespace files** from:
|
||||
- `infrastructure/environments/common/configs/namespace.yaml`
|
||||
- `infrastructure/cicd/flux/namespace.yaml`
|
||||
|
||||
2. **Update kustomization files** to reference the centralized namespace files
|
||||
|
||||
3. **Use the new deployment script** that follows the correct order
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Add namespace lifecycle management
|
||||
- Implement namespace cleanup scripts
|
||||
- Add namespace validation checks to CI/CD pipelines
|
||||
57
infrastructure/README.md
Normal file
57
infrastructure/README.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# Bakery-IA Infrastructure
|
||||
|
||||
This directory contains all infrastructure-as-code for the Bakery-IA project, organized according to best practices for maintainability and scalability.
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
infrastructure/
|
||||
├── environments/ # Environment-specific configurations
|
||||
│ ├── dev/ # Development environment
|
||||
│ │ ├── k8s-manifests/ # Kubernetes manifests for dev
|
||||
│ │ └── values/ # Environment-specific values
|
||||
│ ├── staging/ # Staging environment
|
||||
│ │ ├── k8s-manifests/
|
||||
│ │ └── values/
|
||||
│ └── prod/ # Production environment
|
||||
│ ├── k8s-manifests/
|
||||
│ ├── terraform/ # Production-specific IaC
|
||||
│ └── values/
|
||||
├── platform/ # Platform-level infrastructure
|
||||
│ ├── cluster/ # Cluster configuration (EKS, Kind)
|
||||
│ ├── networking/ # Network configuration
|
||||
│ ├── security/ # Security policies and TLS
|
||||
│ └── storage/ # Storage configuration
|
||||
├── services/ # Application services
|
||||
│ ├── databases/ # Database configurations
|
||||
│ ├── api-gateway/ # API gateway configuration
|
||||
│ └── microservices/ # Individual microservice configs
|
||||
├── monitoring/ # Observability stack
|
||||
│ └── signoz/ # SigNoz configuration
|
||||
├── cicd/ # CI/CD pipeline components
|
||||
├── security/ # Security configurations
|
||||
├── scripts/ # Automation scripts
|
||||
└── docs/ # Infrastructure documentation
|
||||
```
|
||||
|
||||
## Environments
|
||||
|
||||
Each environment (dev, staging, prod) has its own configuration with appropriate isolation and security settings.
|
||||
|
||||
## Services
|
||||
|
||||
Services are organized by business domain with clear separation between databases, microservices, and infrastructure components.
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. **Local Development**: Use `tilt up` to start the development environment
|
||||
2. **Deployment**: Use `skaffold run` to deploy to your target environment
|
||||
3. **CI/CD**: Tekton pipelines manage automated deployments
|
||||
|
||||
## Security
|
||||
|
||||
Security configurations are centralized in the `security/` directory with:
|
||||
- TLS certificates and rotation scripts
|
||||
- Network policies
|
||||
- RBAC configurations
|
||||
- Compliance checks
|
||||
@@ -1,27 +0,0 @@
|
||||
# Flux Kustomization for Bakery-IA Production Deployment
|
||||
# This resource tells Flux how to deploy the application
|
||||
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: bakery-ia-prod
|
||||
namespace: flux-system
|
||||
spec:
|
||||
interval: 5m
|
||||
path: ./infrastructure/kubernetes/overlays/prod
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: bakery-ia
|
||||
targetNamespace: bakery-ia
|
||||
timeout: 5m
|
||||
retryInterval: 1m
|
||||
healthChecks:
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: auth-service
|
||||
namespace: bakery-ia
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: gateway
|
||||
namespace: bakery-ia
|
||||
@@ -1,25 +0,0 @@
|
||||
# Gitea Ingress configuration for Bakery-IA CI/CD
|
||||
# This provides external access to Gitea within the cluster
|
||||
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: gitea-ingress
|
||||
namespace: gitea
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: "0"
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
||||
spec:
|
||||
rules:
|
||||
- host: gitea.bakery-ia.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: gitea-http
|
||||
port:
|
||||
number: 3000
|
||||
@@ -1,38 +0,0 @@
|
||||
# Gitea Helm values configuration for Bakery-IA CI/CD
|
||||
# This configuration sets up Gitea with registry support and appropriate storage
|
||||
|
||||
service:
|
||||
type: ClusterIP
|
||||
httpPort: 3000
|
||||
sshPort: 2222
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 50Gi
|
||||
storageClass: "microk8s-hostpath"
|
||||
|
||||
gitea:
|
||||
config:
|
||||
server:
|
||||
DOMAIN: gitea.bakery-ia.local
|
||||
SSH_DOMAIN: gitea.bakery-ia.local
|
||||
ROOT_URL: http://gitea.bakery-ia.local
|
||||
repository:
|
||||
ENABLE_PUSH_CREATE_USER: true
|
||||
ENABLE_PUSH_CREATE_ORG: true
|
||||
registry:
|
||||
ENABLED: true
|
||||
|
||||
postgresql:
|
||||
enabled: true
|
||||
persistence:
|
||||
size: 20Gi
|
||||
|
||||
# Resource configuration for production environment
|
||||
resources:
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 1Gi
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
@@ -1,70 +0,0 @@
|
||||
# OpenTelemetry Collector for Bakery-IA CI/CD Monitoring
|
||||
# This collects metrics and traces from Tekton pipelines
|
||||
|
||||
apiVersion: opentelemetry.io/v1alpha1
|
||||
kind: OpenTelemetryCollector
|
||||
metadata:
|
||||
name: tekton-otel
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
config: |
|
||||
receivers:
|
||||
otlp:
|
||||
protocols:
|
||||
grpc:
|
||||
endpoint: 0.0.0.0:4317
|
||||
http:
|
||||
endpoint: 0.0.0.0:4318
|
||||
prometheus:
|
||||
config:
|
||||
scrape_configs:
|
||||
- job_name: 'tekton-pipelines'
|
||||
scrape_interval: 30s
|
||||
static_configs:
|
||||
- targets: ['tekton-pipelines-controller.tekton-pipelines.svc.cluster.local:9090']
|
||||
|
||||
processors:
|
||||
batch:
|
||||
timeout: 5s
|
||||
send_batch_size: 1000
|
||||
memory_limiter:
|
||||
check_interval: 2s
|
||||
limit_percentage: 75
|
||||
spike_limit_percentage: 20
|
||||
|
||||
exporters:
|
||||
otlp:
|
||||
endpoint: "signoz-otel-collector.monitoring.svc.cluster.local:4317"
|
||||
tls:
|
||||
insecure: true
|
||||
retry_on_failure:
|
||||
enabled: true
|
||||
initial_interval: 5s
|
||||
max_interval: 30s
|
||||
max_elapsed_time: 300s
|
||||
logging:
|
||||
logLevel: debug
|
||||
|
||||
service:
|
||||
pipelines:
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
processors: [memory_limiter, batch]
|
||||
exporters: [otlp, logging]
|
||||
metrics:
|
||||
receivers: [otlp, prometheus]
|
||||
processors: [memory_limiter, batch]
|
||||
exporters: [otlp, logging]
|
||||
telemetry:
|
||||
logs:
|
||||
level: "info"
|
||||
encoding: "json"
|
||||
|
||||
mode: deployment
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
requests:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
@@ -1,83 +0,0 @@
|
||||
# Main CI Pipeline for Bakery-IA
|
||||
# This pipeline orchestrates the build, test, and deploy process
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Pipeline
|
||||
metadata:
|
||||
name: bakery-ia-ci
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
- name: docker-credentials
|
||||
params:
|
||||
- name: git-url
|
||||
type: string
|
||||
description: Repository URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision/commit hash
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
default: "gitea.bakery-ia.local:5000"
|
||||
tasks:
|
||||
- name: fetch-source
|
||||
taskRef:
|
||||
name: git-clone
|
||||
workspaces:
|
||||
- name: output
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: url
|
||||
value: $(params.git-url)
|
||||
- name: revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
- name: detect-changes
|
||||
runAfter: [fetch-source]
|
||||
taskRef:
|
||||
name: detect-changed-services
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
|
||||
- name: build-and-push
|
||||
runAfter: [detect-changes]
|
||||
taskRef:
|
||||
name: kaniko-build
|
||||
when:
|
||||
- input: "$(tasks.detect-changes.results.changed-services)"
|
||||
operator: notin
|
||||
values: ["none"]
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
- name: docker-credentials
|
||||
workspace: docker-credentials
|
||||
params:
|
||||
- name: services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
- name: update-gitops-manifests
|
||||
runAfter: [build-and-push]
|
||||
taskRef:
|
||||
name: update-gitops
|
||||
when:
|
||||
- input: "$(tasks.detect-changes.results.changed-services)"
|
||||
operator: notin
|
||||
values: ["none"]
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
@@ -1,64 +0,0 @@
|
||||
# Tekton Detect Changed Services Task for Bakery-IA CI/CD
|
||||
# This task identifies which services have changed in the repository
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: detect-changed-services
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
results:
|
||||
- name: changed-services
|
||||
description: Comma-separated list of changed services
|
||||
steps:
|
||||
- name: detect
|
||||
image: alpine/git
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
cd $(workspaces.source.path)
|
||||
|
||||
echo "Detecting changed files..."
|
||||
# Get list of changed files compared to previous commit
|
||||
CHANGED_FILES=$(git diff --name-only HEAD~1 HEAD 2>/dev/null || git diff --name-only HEAD)
|
||||
|
||||
echo "Changed files: $CHANGED_FILES"
|
||||
|
||||
# Map files to services
|
||||
CHANGED_SERVICES=()
|
||||
for file in $CHANGED_FILES; do
|
||||
if [[ $file == services/* ]]; then
|
||||
SERVICE=$(echo $file | cut -d'/' -f2)
|
||||
# Only add unique service names
|
||||
if [[ ! " ${CHANGED_SERVICES[@]} " =~ " ${SERVICE} " ]]; then
|
||||
CHANGED_SERVICES+=("$SERVICE")
|
||||
fi
|
||||
elif [[ $file == frontend/* ]]; then
|
||||
CHANGED_SERVICES+=("frontend")
|
||||
break
|
||||
elif [[ $file == gateway/* ]]; then
|
||||
CHANGED_SERVICES+=("gateway")
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# If no specific services changed, check for infrastructure changes
|
||||
if [ ${#CHANGED_SERVICES[@]} -eq 0 ]; then
|
||||
for file in $CHANGED_FILES; do
|
||||
if [[ $file == infrastructure/* ]]; then
|
||||
CHANGED_SERVICES+=("infrastructure")
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Output result
|
||||
if [ ${#CHANGED_SERVICES[@]} -eq 0 ]; then
|
||||
echo "No service changes detected"
|
||||
echo "none" | tee $(results.changed-services.path)
|
||||
else
|
||||
echo "Detected changes in services: ${CHANGED_SERVICES[@]}"
|
||||
echo $(printf "%s," "${CHANGED_SERVICES[@]}" | sed 's/,$//') | tee $(results.changed-services.path)
|
||||
fi
|
||||
@@ -1,31 +0,0 @@
|
||||
# Tekton Git Clone Task for Bakery-IA CI/CD
|
||||
# This task clones the source code repository
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: git-clone
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
workspaces:
|
||||
- name: output
|
||||
params:
|
||||
- name: url
|
||||
type: string
|
||||
description: Repository URL to clone
|
||||
- name: revision
|
||||
type: string
|
||||
description: Git revision to checkout
|
||||
default: "main"
|
||||
steps:
|
||||
- name: clone
|
||||
image: alpine/git
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
echo "Cloning repository: $(params.url)"
|
||||
git clone $(params.url) $(workspaces.output.path)
|
||||
cd $(workspaces.output.path)
|
||||
echo "Checking out revision: $(params.revision)"
|
||||
git checkout $(params.revision)
|
||||
echo "Repository cloned successfully"
|
||||
@@ -1,40 +0,0 @@
|
||||
# Tekton Kaniko Build Task for Bakery-IA CI/CD
|
||||
# This task builds and pushes container images using Kaniko
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: kaniko-build
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
- name: docker-credentials
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to build
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
default: "gitea.bakery-ia.local:5000"
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision for image tag
|
||||
default: "latest"
|
||||
steps:
|
||||
- name: build-and-push
|
||||
image: gcr.io/kaniko-project/executor:v1.9.0
|
||||
args:
|
||||
- --dockerfile=$(workspaces.source.path)/services/$(params.services)/Dockerfile
|
||||
- --context=$(workspaces.source.path)
|
||||
- --destination=$(params.registry)/bakery/$(params.services):$(params.git-revision)
|
||||
- --verbosity=info
|
||||
volumeMounts:
|
||||
- name: docker-config
|
||||
mountPath: /kaniko/.docker
|
||||
securityContext:
|
||||
runAsUser: 0
|
||||
volumes:
|
||||
- name: docker-config
|
||||
emptyDir: {}
|
||||
@@ -1,66 +0,0 @@
|
||||
# Tekton Update GitOps Manifests Task for Bakery-IA CI/CD
|
||||
# This task updates Kubernetes manifests with new image tags
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: update-gitops
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to update
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision for image tag
|
||||
steps:
|
||||
- name: update-manifests
|
||||
image: bitnami/kubectl
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
cd $(workspaces.source.path)
|
||||
|
||||
echo "Updating GitOps manifests for services: $(params.services)"
|
||||
|
||||
# Split services by comma
|
||||
IFS=',' read -ra SERVICES <<< "$(params.services)"
|
||||
|
||||
for service in "${SERVICES[@]}"; do
|
||||
echo "Processing service: $service"
|
||||
|
||||
# Find and update Kubernetes manifests
|
||||
if [ "$service" = "frontend" ]; then
|
||||
# Update frontend deployment
|
||||
if [ -f "infrastructure/kubernetes/overlays/prod/frontend-deployment.yaml" ]; then
|
||||
sed -i "s|image:.*|image: $(params.registry)/bakery/frontend:$(params.git-revision)|g" \
|
||||
"infrastructure/kubernetes/overlays/prod/frontend-deployment.yaml"
|
||||
fi
|
||||
elif [ "$service" = "gateway" ]; then
|
||||
# Update gateway deployment
|
||||
if [ -f "infrastructure/kubernetes/overlays/prod/gateway-deployment.yaml" ]; then
|
||||
sed -i "s|image:.*|image: $(params.registry)/bakery/gateway:$(params.git-revision)|g" \
|
||||
"infrastructure/kubernetes/overlays/prod/gateway-deployment.yaml"
|
||||
fi
|
||||
else
|
||||
# Update service deployment
|
||||
DEPLOYMENT_FILE="infrastructure/kubernetes/overlays/prod/${service}-deployment.yaml"
|
||||
if [ -f "$DEPLOYMENT_FILE" ]; then
|
||||
sed -i "s|image:.*|image: $(params.registry)/bakery/${service}:$(params.git-revision)|g" \
|
||||
"$DEPLOYMENT_FILE"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Commit changes
|
||||
git config --global user.name "bakery-ia-ci"
|
||||
git config --global user.email "ci@bakery-ia.local"
|
||||
git add .
|
||||
git commit -m "CI: Update image tags for $(params.services) to $(params.git-revision)"
|
||||
git push origin HEAD
|
||||
@@ -1,26 +0,0 @@
|
||||
# Tekton EventListener for Bakery-IA CI/CD
|
||||
# This listener receives webhook events and triggers pipelines
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1alpha1
|
||||
kind: EventListener
|
||||
metadata:
|
||||
name: bakery-ia-listener
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
serviceAccountName: tekton-triggers-sa
|
||||
triggers:
|
||||
- name: bakery-ia-gitea-trigger
|
||||
bindings:
|
||||
- ref: bakery-ia-trigger-binding
|
||||
template:
|
||||
ref: bakery-ia-trigger-template
|
||||
interceptors:
|
||||
- ref:
|
||||
name: "gitlab"
|
||||
params:
|
||||
- name: "secretRef"
|
||||
value:
|
||||
secretName: gitea-webhook-secret
|
||||
secretKey: secretToken
|
||||
- name: "eventTypes"
|
||||
value: ["push"]
|
||||
@@ -1,14 +0,0 @@
|
||||
# GitLab/Gitea Webhook Interceptor for Tekton Triggers
|
||||
# This interceptor validates and processes Gitea webhook events
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1alpha1
|
||||
kind: ClusterInterceptor
|
||||
metadata:
|
||||
name: gitlab
|
||||
spec:
|
||||
clientConfig:
|
||||
service:
|
||||
name: tekton-triggers-core-interceptors
|
||||
namespace: tekton-pipelines
|
||||
path: "/v1/webhook/gitlab"
|
||||
port: 8443
|
||||
@@ -1,16 +0,0 @@
|
||||
# Tekton TriggerBinding for Bakery-IA CI/CD
|
||||
# This binding extracts parameters from Gitea webhook events
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1alpha1
|
||||
kind: TriggerBinding
|
||||
metadata:
|
||||
name: bakery-ia-trigger-binding
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
params:
|
||||
- name: git-repo-url
|
||||
value: $(body.repository.clone_url)
|
||||
- name: git-revision
|
||||
value: $(body.head_commit.id)
|
||||
- name: git-repo-name
|
||||
value: $(body.repository.name)
|
||||
@@ -1,43 +0,0 @@
|
||||
# Tekton TriggerTemplate for Bakery-IA CI/CD
|
||||
# This template defines how PipelineRuns are created when triggers fire
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1alpha1
|
||||
kind: TriggerTemplate
|
||||
metadata:
|
||||
name: bakery-ia-trigger-template
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
params:
|
||||
- name: git-repo-url
|
||||
description: The git repository URL
|
||||
- name: git-revision
|
||||
description: The git revision/commit hash
|
||||
- name: git-repo-name
|
||||
description: The git repository name
|
||||
default: "bakery-ia"
|
||||
resourcetemplates:
|
||||
- apiVersion: tekton.dev/v1beta1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
generateName: bakery-ia-ci-run-$(params.git-repo-name)-
|
||||
spec:
|
||||
pipelineRef:
|
||||
name: bakery-ia-ci
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
- name: docker-credentials
|
||||
secret:
|
||||
secretName: gitea-registry-credentials
|
||||
params:
|
||||
- name: git-url
|
||||
value: $(params.git-repo-url)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
- name: registry
|
||||
value: "gitea.bakery-ia.local:5000"
|
||||
@@ -19,8 +19,7 @@ graph TD
|
||||
```
|
||||
infrastructure/ci-cd/
|
||||
├── gitea/ # Gitea configuration (Git server + registry)
|
||||
│ ├── values.yaml # Helm values for Gitea
|
||||
│ └── ingress.yaml # Ingress configuration
|
||||
│ └── values.yaml # Helm values for Gitea (ingress now in main config)
|
||||
├── tekton/ # Tekton CI/CD pipeline configuration
|
||||
│ ├── tasks/ # Individual pipeline tasks
|
||||
│ │ ├── git-clone.yaml
|
||||
@@ -59,8 +58,8 @@ infrastructure/ci-cd/
|
||||
-n gitea \
|
||||
-f infrastructure/ci-cd/gitea/values.yaml
|
||||
|
||||
# Apply ingress
|
||||
microk8s kubectl apply -f infrastructure/ci-cd/gitea/ingress.yaml
|
||||
# Note: Gitea ingress is now included in the main ingress configuration
|
||||
# No separate ingress needs to be applied
|
||||
```
|
||||
|
||||
2. **Deploy Tekton**:
|
||||
@@ -85,8 +84,8 @@ infrastructure/ci-cd/
|
||||
# Verify Flux installation
|
||||
microk8s kubectl get pods -n flux-system
|
||||
|
||||
# Apply Flux configurations
|
||||
microk8s kubectl apply -f infrastructure/ci-cd/flux/
|
||||
# Apply Flux configurations using kustomize
|
||||
microk8s kubectl apply -k infrastructure/ci-cd/flux/
|
||||
```
|
||||
|
||||
### Phase 2: Configuration
|
||||
76
infrastructure/cicd/flux/flux-kustomization.yaml
Normal file
76
infrastructure/cicd/flux/flux-kustomization.yaml
Normal file
@@ -0,0 +1,76 @@
|
||||
# Flux Kustomization for Bakery-IA Production Deployment
|
||||
# This resource tells Flux how to deploy the application
|
||||
#
|
||||
# Prerequisites:
|
||||
# 1. Flux CD must be installed: flux install
|
||||
# 2. GitRepository 'bakery-ia' must be created and ready
|
||||
# 3. Secret 'gitea-credentials' must exist in flux-system namespace
|
||||
|
||||
apiVersion: kustomize.toolkit.fluxcd.io/v1
|
||||
kind: Kustomization
|
||||
metadata:
|
||||
name: bakery-ia-prod
|
||||
namespace: flux-system
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia
|
||||
app.kubernetes.io/component: flux
|
||||
spec:
|
||||
# Wait for GitRepository to be ready before reconciling
|
||||
dependsOn: []
|
||||
interval: 5m
|
||||
path: ./infrastructure/environments/prod
|
||||
prune: true
|
||||
sourceRef:
|
||||
kind: GitRepository
|
||||
name: bakery-ia
|
||||
targetNamespace: bakery-ia
|
||||
timeout: 10m
|
||||
retryInterval: 1m
|
||||
wait: true
|
||||
# Health checks for critical services
|
||||
healthChecks:
|
||||
# Core Infrastructure
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: gateway
|
||||
namespace: bakery-ia
|
||||
# Authentication & Authorization
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: auth-service
|
||||
namespace: bakery-ia
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: tenant-service
|
||||
namespace: bakery-ia
|
||||
# Core Business Services
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: inventory-service
|
||||
namespace: bakery-ia
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: orders-service
|
||||
namespace: bakery-ia
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: pos-service
|
||||
namespace: bakery-ia
|
||||
# Data Services
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: forecasting-service
|
||||
namespace: bakery-ia
|
||||
- apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
name: notification-service
|
||||
namespace: bakery-ia
|
||||
# Post-build variable substitution
|
||||
postBuild:
|
||||
substituteFrom:
|
||||
- kind: ConfigMap
|
||||
name: bakery-ia-config
|
||||
optional: true
|
||||
- kind: Secret
|
||||
name: bakery-ia-secrets
|
||||
optional: true
|
||||
25
infrastructure/cicd/flux/kustomization.yaml
Normal file
25
infrastructure/cicd/flux/kustomization.yaml
Normal file
@@ -0,0 +1,25 @@
|
||||
# Kustomize build configuration for Flux resources
|
||||
# This file is used to build and apply the Flux resources
|
||||
#
|
||||
# IMPORTANT: Apply resources in this order:
|
||||
# 1. Install Flux CD first: flux install
|
||||
# 2. Apply this kustomization: kubectl apply -k infrastructure/cicd/flux/
|
||||
#
|
||||
# The GitRepository must be ready before the Flux Kustomization can reconcile.
|
||||
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
# Resources to apply in order (namespace and secrets first, then sources, then kustomizations)
|
||||
resources:
|
||||
- namespace.yaml
|
||||
- git-repository.yaml
|
||||
- flux-kustomization.yaml
|
||||
|
||||
# Common labels for all resources
|
||||
commonLabels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: flux
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
|
||||
# Note: Do NOT set namespace here as resources already have explicit namespaces
|
||||
15
infrastructure/cicd/flux/namespace.yaml
Normal file
15
infrastructure/cicd/flux/namespace.yaml
Normal file
@@ -0,0 +1,15 @@
|
||||
# Flux System Namespace
|
||||
# This namespace is required for Flux CD components
|
||||
# It should be created before any Flux resources are applied
|
||||
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: flux-system
|
||||
labels:
|
||||
app.kubernetes.io/name: flux
|
||||
app.kubernetes.io/component: system
|
||||
kubernetes.io/metadata.name: flux-system
|
||||
pod-security.kubernetes.io/enforce: restricted
|
||||
pod-security.kubernetes.io/audit: restricted
|
||||
pod-security.kubernetes.io/warn: restricted
|
||||
44
infrastructure/cicd/gitea/ingress.yaml.disabled
Normal file
44
infrastructure/cicd/gitea/ingress.yaml.disabled
Normal file
@@ -0,0 +1,44 @@
|
||||
# Gitea Ingress Configuration
|
||||
# Routes external traffic to Gitea service for web UI and Git HTTP access
|
||||
#
|
||||
# Prerequisites:
|
||||
# - Gitea must be deployed in the 'gitea' namespace
|
||||
# - Ingress controller must be installed (nginx, traefik, etc.)
|
||||
# - For HTTPS: cert-manager with a ClusterIssuer named 'letsencrypt-prod' or 'local-ca-issuer'
|
||||
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: gitea-ingress
|
||||
namespace: gitea
|
||||
labels:
|
||||
app.kubernetes.io/name: gitea
|
||||
app.kubernetes.io/component: ingress
|
||||
app.kubernetes.io/part-of: bakery-ia-cicd
|
||||
annotations:
|
||||
# For nginx ingress controller
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
||||
# For traefik ingress controller
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: web,websecure
|
||||
# For TLS with cert-manager (uncomment for HTTPS)
|
||||
# cert-manager.io/cluster-issuer: "local-ca-issuer"
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
# Uncomment for HTTPS
|
||||
# tls:
|
||||
# - hosts:
|
||||
# - gitea.bakery-ia.local
|
||||
# secretName: gitea-tls
|
||||
rules:
|
||||
- host: gitea.bakery-ia.local
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: gitea-http
|
||||
port:
|
||||
number: 3000
|
||||
48
infrastructure/cicd/gitea/setup-admin-secret.sh
Executable file
48
infrastructure/cicd/gitea/setup-admin-secret.sh
Executable file
@@ -0,0 +1,48 @@
|
||||
#!/bin/bash
|
||||
# Setup Gitea Admin Secret
|
||||
#
|
||||
# This script creates the Kubernetes secret required for Gitea admin credentials.
|
||||
# Run this BEFORE installing Gitea with Helm.
|
||||
#
|
||||
# Usage:
|
||||
# ./setup-admin-secret.sh [password]
|
||||
#
|
||||
# If password is not provided, a random one will be generated.
|
||||
|
||||
set -e
|
||||
|
||||
KUBECTL="kubectl"
|
||||
NAMESPACE="gitea"
|
||||
|
||||
# Check if running in microk8s
|
||||
if command -v microk8s &> /dev/null; then
|
||||
KUBECTL="microk8s kubectl"
|
||||
fi
|
||||
|
||||
# Get or generate password
|
||||
if [ -n "$1" ]; then
|
||||
ADMIN_PASSWORD="$1"
|
||||
else
|
||||
ADMIN_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=' | head -c 20)
|
||||
echo "Generated admin password: $ADMIN_PASSWORD"
|
||||
fi
|
||||
|
||||
# Create namespace if it doesn't exist
|
||||
$KUBECTL create namespace "$NAMESPACE" --dry-run=client -o yaml | $KUBECTL apply -f -
|
||||
|
||||
# Create the secret
|
||||
$KUBECTL create secret generic gitea-admin-secret \
|
||||
--namespace "$NAMESPACE" \
|
||||
--from-literal=username=bakery-admin \
|
||||
--from-literal=password="$ADMIN_PASSWORD" \
|
||||
--dry-run=client -o yaml | $KUBECTL apply -f -
|
||||
|
||||
echo ""
|
||||
echo "Gitea admin secret created successfully!"
|
||||
echo ""
|
||||
echo "Admin credentials:"
|
||||
echo " Username: bakery-admin"
|
||||
echo " Password: $ADMIN_PASSWORD"
|
||||
echo ""
|
||||
echo "Now install Gitea with:"
|
||||
echo " helm install gitea gitea/gitea -n gitea -f infrastructure/cicd/gitea/values.yaml"
|
||||
83
infrastructure/cicd/gitea/values.yaml
Normal file
83
infrastructure/cicd/gitea/values.yaml
Normal file
@@ -0,0 +1,83 @@
|
||||
# Gitea Helm values configuration for Bakery-IA CI/CD
|
||||
# This configuration sets up Gitea with registry support and appropriate storage
|
||||
#
|
||||
# Installation:
|
||||
# helm repo add gitea https://dl.gitea.io/charts
|
||||
# kubectl create namespace gitea
|
||||
# helm install gitea gitea/gitea -n gitea -f infrastructure/cicd/gitea/values.yaml
|
||||
#
|
||||
# NOTE: The namespace is determined by the -n flag during helm install, not in this file.
|
||||
|
||||
service:
|
||||
http:
|
||||
type: ClusterIP
|
||||
port: 3000
|
||||
ssh:
|
||||
type: ClusterIP
|
||||
port: 2222
|
||||
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 10Gi
|
||||
# Use standard storage class (works with Kind's default provisioner)
|
||||
# For microk8s: storageClass: "microk8s-hostpath"
|
||||
# For Kind: leave empty or use "standard"
|
||||
storageClass: ""
|
||||
|
||||
gitea:
|
||||
admin:
|
||||
username: bakery-admin
|
||||
# IMPORTANT: Override this with --set gitea.admin.password=<secure-password>
|
||||
# or use existingSecret
|
||||
password: ""
|
||||
email: admin@bakery-ia.local
|
||||
existingSecret: gitea-admin-secret
|
||||
config:
|
||||
server:
|
||||
DOMAIN: gitea.bakery-ia.local
|
||||
SSH_DOMAIN: gitea.bakery-ia.local
|
||||
# Use HTTP internally; TLS termination happens at ingress
|
||||
ROOT_URL: http://gitea.bakery-ia.local
|
||||
HTTP_PORT: 3000
|
||||
# For external HTTPS access via ingress, set:
|
||||
# ROOT_URL: https://gitea.bakery-ia.local
|
||||
repository:
|
||||
ENABLE_PUSH_CREATE_USER: true
|
||||
ENABLE_PUSH_CREATE_ORG: true
|
||||
packages:
|
||||
ENABLED: true
|
||||
webhook:
|
||||
ALLOWED_HOST_LIST: "*"
|
||||
# Allow internal cluster URLs for Tekton EventListener
|
||||
SKIP_TLS_VERIFY: true
|
||||
service:
|
||||
DISABLE_REGISTRATION: false
|
||||
REQUIRE_SIGNIN_VIEW: false
|
||||
|
||||
# Use embedded SQLite for simpler local development
|
||||
# For production, enable postgresql
|
||||
postgresql:
|
||||
enabled: false
|
||||
|
||||
# Use embedded in-memory cache for local dev
|
||||
redis-cluster:
|
||||
enabled: false
|
||||
|
||||
# Resource configuration for local development
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
|
||||
# Init containers timeout
|
||||
initContainers:
|
||||
resources:
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
10
infrastructure/cicd/kustomization.yaml
Normal file
10
infrastructure/cicd/kustomization.yaml
Normal file
@@ -0,0 +1,10 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- tekton/
|
||||
- ../../namespaces/flux-system.yaml
|
||||
|
||||
# Gitea is managed via Helm, so we don't include it directly here
|
||||
# The Gitea Helm chart is deployed separately and referenced in the ingress
|
||||
# Flux configuration is a Flux Kustomization resource, not a kustomize config
|
||||
222
infrastructure/cicd/tekton/cleanup/cleanup.yaml
Normal file
222
infrastructure/cicd/tekton/cleanup/cleanup.yaml
Normal file
@@ -0,0 +1,222 @@
|
||||
# Workspace and PipelineRun Cleanup for Bakery-IA CI/CD
|
||||
# This CronJob cleans up old PipelineRuns and PVCs to prevent storage exhaustion
|
||||
|
||||
---
|
||||
# ServiceAccount for cleanup job
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tekton-cleanup-sa
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
|
||||
---
|
||||
# ClusterRole for cleanup operations
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: tekton-cleanup-role
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
rules:
|
||||
- apiGroups: ["tekton.dev"]
|
||||
resources: ["pipelineruns", "taskruns"]
|
||||
verbs: ["get", "list", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["persistentvolumeclaims"]
|
||||
verbs: ["get", "list", "delete"]
|
||||
- apiGroups: [""]
|
||||
resources: ["pods"]
|
||||
verbs: ["get", "list", "delete"]
|
||||
|
||||
---
|
||||
# ClusterRoleBinding for cleanup
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tekton-cleanup-binding
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tekton-cleanup-sa
|
||||
namespace: tekton-pipelines
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: tekton-cleanup-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
---
|
||||
# CronJob to clean up old PipelineRuns
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: tekton-pipelinerun-cleanup
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
spec:
|
||||
# Run every 6 hours
|
||||
schedule: "0 */6 * * *"
|
||||
concurrencyPolicy: Forbid
|
||||
successfulJobsHistoryLimit: 3
|
||||
failedJobsHistoryLimit: 3
|
||||
jobTemplate:
|
||||
spec:
|
||||
ttlSecondsAfterFinished: 3600
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
spec:
|
||||
serviceAccountName: tekton-cleanup-sa
|
||||
restartPolicy: OnFailure
|
||||
containers:
|
||||
- name: cleanup
|
||||
image: bitnami/kubectl:latest
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "============================================"
|
||||
echo "Tekton Cleanup Job"
|
||||
echo "Timestamp: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
|
||||
echo "============================================"
|
||||
|
||||
# Configuration
|
||||
NAMESPACE="tekton-pipelines"
|
||||
MAX_AGE_HOURS=24
|
||||
KEEP_RECENT=10
|
||||
|
||||
echo ""
|
||||
echo "Configuration:"
|
||||
echo " Namespace: $NAMESPACE"
|
||||
echo " Max Age: ${MAX_AGE_HOURS} hours"
|
||||
echo " Keep Recent: $KEEP_RECENT"
|
||||
echo ""
|
||||
|
||||
# Get current timestamp
|
||||
CURRENT_TIME=$(date +%s)
|
||||
|
||||
# Clean up completed PipelineRuns older than MAX_AGE_HOURS
|
||||
echo "Cleaning up old PipelineRuns..."
|
||||
|
||||
# Get all completed PipelineRuns
|
||||
COMPLETED_RUNS=$(kubectl get pipelineruns -n "$NAMESPACE" \
|
||||
--no-headers \
|
||||
-o custom-columns=NAME:.metadata.name,STATUS:.status.conditions[0].reason,AGE:.metadata.creationTimestamp \
|
||||
2>/dev/null | grep -E "Succeeded|Failed" || true)
|
||||
|
||||
DELETED_COUNT=0
|
||||
|
||||
echo "$COMPLETED_RUNS" | while read -r line; do
|
||||
if [ -z "$line" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
RUN_NAME=$(echo "$line" | awk '{print $1}')
|
||||
RUN_TIME=$(echo "$line" | awk '{print $3}')
|
||||
|
||||
if [ -z "$RUN_NAME" ] || [ -z "$RUN_TIME" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Convert timestamp to seconds
|
||||
RUN_TIMESTAMP=$(date -d "$RUN_TIME" +%s 2>/dev/null || echo "0")
|
||||
|
||||
if [ "$RUN_TIMESTAMP" = "0" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Calculate age in hours
|
||||
AGE_SECONDS=$((CURRENT_TIME - RUN_TIMESTAMP))
|
||||
AGE_HOURS=$((AGE_SECONDS / 3600))
|
||||
|
||||
if [ "$AGE_HOURS" -gt "$MAX_AGE_HOURS" ]; then
|
||||
echo "Deleting PipelineRun: $RUN_NAME (age: ${AGE_HOURS}h)"
|
||||
kubectl delete pipelinerun "$RUN_NAME" -n "$NAMESPACE" --ignore-not-found=true
|
||||
DELETED_COUNT=$((DELETED_COUNT + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
echo "Deleted $DELETED_COUNT old PipelineRuns"
|
||||
|
||||
# Clean up orphaned PVCs (PVCs without associated PipelineRuns)
|
||||
echo ""
|
||||
echo "Cleaning up orphaned PVCs..."
|
||||
|
||||
ORPHANED_PVCS=$(kubectl get pvc -n "$NAMESPACE" \
|
||||
-l tekton.dev/pipelineRun \
|
||||
--no-headers \
|
||||
-o custom-columns=NAME:.metadata.name,PIPELINERUN:.metadata.labels.tekton\\.dev/pipelineRun \
|
||||
2>/dev/null || true)
|
||||
|
||||
echo "$ORPHANED_PVCS" | while read -r line; do
|
||||
if [ -z "$line" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
PVC_NAME=$(echo "$line" | awk '{print $1}')
|
||||
PR_NAME=$(echo "$line" | awk '{print $2}')
|
||||
|
||||
if [ -z "$PVC_NAME" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Check if associated PipelineRun exists
|
||||
if ! kubectl get pipelinerun "$PR_NAME" -n "$NAMESPACE" > /dev/null 2>&1; then
|
||||
echo "Deleting orphaned PVC: $PVC_NAME (PipelineRun $PR_NAME not found)"
|
||||
kubectl delete pvc "$PVC_NAME" -n "$NAMESPACE" --ignore-not-found=true
|
||||
fi
|
||||
done
|
||||
|
||||
# Clean up completed/failed pods older than 1 hour
|
||||
echo ""
|
||||
echo "Cleaning up old completed pods..."
|
||||
|
||||
kubectl delete pods -n "$NAMESPACE" \
|
||||
--field-selector=status.phase=Succeeded \
|
||||
--ignore-not-found=true 2>/dev/null || true
|
||||
|
||||
kubectl delete pods -n "$NAMESPACE" \
|
||||
--field-selector=status.phase=Failed \
|
||||
--ignore-not-found=true 2>/dev/null || true
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "Cleanup complete"
|
||||
echo "============================================"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 256Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
|
||||
---
|
||||
# ConfigMap for cleanup configuration
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: cleanup-config
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: cleanup
|
||||
data:
|
||||
# Maximum age of completed PipelineRuns to keep (in hours)
|
||||
MAX_AGE_HOURS: "24"
|
||||
# Number of recent PipelineRuns to keep regardless of age
|
||||
KEEP_RECENT: "10"
|
||||
# Cleanup schedule (cron format)
|
||||
CLEANUP_SCHEDULE: "0 */6 * * *"
|
||||
5
infrastructure/cicd/tekton/cleanup/kustomization.yaml
Normal file
5
infrastructure/cicd/tekton/cleanup/kustomization.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- cleanup.yaml
|
||||
5
infrastructure/cicd/tekton/configs/kustomization.yaml
Normal file
5
infrastructure/cicd/tekton/configs/kustomization.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- pipeline-config.yaml
|
||||
41
infrastructure/cicd/tekton/configs/pipeline-config.yaml
Normal file
41
infrastructure/cicd/tekton/configs/pipeline-config.yaml
Normal file
@@ -0,0 +1,41 @@
|
||||
# CI/CD Pipeline Configuration for Bakery-IA
|
||||
# This ConfigMap contains configurable values for the CI/CD pipeline
|
||||
#
|
||||
# IMPORTANT: When changing REGISTRY_URL, also update:
|
||||
# - infrastructure/cicd/tekton/triggers/trigger-template.yaml (registry-url default)
|
||||
# - infrastructure/cicd/tekton/secrets/secrets.yaml (registry credentials)
|
||||
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: pipeline-config
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: config
|
||||
data:
|
||||
# Container Registry Configuration
|
||||
# Change this to your actual registry URL
|
||||
# Also update trigger-template.yaml and secrets when changing this!
|
||||
REGISTRY_URL: "gitea.bakery-ia.local:5000"
|
||||
|
||||
# Git Configuration
|
||||
GIT_BRANCH: "main"
|
||||
GIT_USER_NAME: "bakery-ia-ci"
|
||||
GIT_USER_EMAIL: "ci@bakery-ia.local"
|
||||
|
||||
# Build Configuration
|
||||
BUILD_CACHE_TTL: "24h"
|
||||
BUILD_VERBOSITY: "info"
|
||||
|
||||
# Test Configuration
|
||||
SKIP_TESTS: "false"
|
||||
SKIP_LINT: "false"
|
||||
|
||||
# Deployment Configuration
|
||||
DEPLOY_NAMESPACE: "bakery-ia"
|
||||
FLUX_NAMESPACE: "flux-system"
|
||||
|
||||
# Workspace Configuration
|
||||
WORKSPACE_SIZE: "5Gi"
|
||||
WORKSPACE_STORAGE_CLASS: "standard"
|
||||
11
infrastructure/cicd/tekton/kustomization.yaml
Normal file
11
infrastructure/cicd/tekton/kustomization.yaml
Normal file
@@ -0,0 +1,11 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- rbac/
|
||||
- secrets/
|
||||
- configs/
|
||||
- tasks/
|
||||
- triggers/
|
||||
- pipelines/
|
||||
- cleanup/
|
||||
149
infrastructure/cicd/tekton/pipelines/ci-pipeline.yaml
Normal file
149
infrastructure/cicd/tekton/pipelines/ci-pipeline.yaml
Normal file
@@ -0,0 +1,149 @@
|
||||
# Main CI Pipeline for Bakery-IA
|
||||
# This pipeline orchestrates the build, test, and deploy process
|
||||
# Includes: fetch -> detect changes -> test -> build -> update gitops
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Pipeline
|
||||
metadata:
|
||||
name: bakery-ia-ci
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: pipeline
|
||||
spec:
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
description: Shared workspace for source code
|
||||
- name: docker-credentials
|
||||
description: Docker registry credentials
|
||||
- name: git-credentials
|
||||
description: Git credentials for pushing GitOps updates
|
||||
optional: true
|
||||
params:
|
||||
- name: git-url
|
||||
type: string
|
||||
description: Repository URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision/commit hash
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: git-branch
|
||||
type: string
|
||||
description: Target branch for GitOps updates
|
||||
default: "main"
|
||||
- name: skip-tests
|
||||
type: string
|
||||
description: Skip tests if "true"
|
||||
default: "false"
|
||||
- name: dry-run
|
||||
type: string
|
||||
description: Dry run mode - don't push changes
|
||||
default: "false"
|
||||
|
||||
tasks:
|
||||
# Stage 1: Fetch source code
|
||||
- name: fetch-source
|
||||
taskRef:
|
||||
name: git-clone
|
||||
workspaces:
|
||||
- name: output
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: url
|
||||
value: $(params.git-url)
|
||||
- name: revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
# Stage 2: Detect which services changed
|
||||
- name: detect-changes
|
||||
runAfter: [fetch-source]
|
||||
taskRef:
|
||||
name: detect-changed-services
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
|
||||
# Stage 3: Run tests on changed services
|
||||
- name: run-tests
|
||||
runAfter: [detect-changes]
|
||||
taskRef:
|
||||
name: run-tests
|
||||
when:
|
||||
- input: "$(tasks.detect-changes.results.changed-services)"
|
||||
operator: notin
|
||||
values: ["none", "infrastructure"]
|
||||
- input: "$(params.skip-tests)"
|
||||
operator: notin
|
||||
values: ["true"]
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: skip-tests
|
||||
value: $(params.skip-tests)
|
||||
|
||||
# Stage 4: Build and push container images
|
||||
- name: build-and-push
|
||||
runAfter: [run-tests]
|
||||
taskRef:
|
||||
name: kaniko-build
|
||||
when:
|
||||
- input: "$(tasks.detect-changes.results.changed-services)"
|
||||
operator: notin
|
||||
values: ["none", "infrastructure"]
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
- name: docker-credentials
|
||||
workspace: docker-credentials
|
||||
params:
|
||||
- name: services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
# Stage 5: Update GitOps manifests
|
||||
- name: update-gitops-manifests
|
||||
runAfter: [build-and-push]
|
||||
taskRef:
|
||||
name: update-gitops
|
||||
when:
|
||||
- input: "$(tasks.detect-changes.results.changed-services)"
|
||||
operator: notin
|
||||
values: ["none", "infrastructure"]
|
||||
- input: "$(tasks.build-and-push.results.build-status)"
|
||||
operator: in
|
||||
values: ["success", "partial"]
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
- name: git-credentials
|
||||
workspace: git-credentials
|
||||
params:
|
||||
- name: services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
- name: git-branch
|
||||
value: $(params.git-branch)
|
||||
- name: dry-run
|
||||
value: $(params.dry-run)
|
||||
|
||||
# Final tasks that run regardless of pipeline success/failure
|
||||
finally:
|
||||
- name: pipeline-summary
|
||||
taskRef:
|
||||
name: pipeline-summary
|
||||
params:
|
||||
- name: changed-services
|
||||
value: $(tasks.detect-changes.results.changed-services)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
6
infrastructure/cicd/tekton/pipelines/kustomization.yaml
Normal file
6
infrastructure/cicd/tekton/pipelines/kustomization.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- ci-pipeline.yaml
|
||||
- prod-deploy-pipeline.yaml
|
||||
118
infrastructure/cicd/tekton/pipelines/prod-deploy-pipeline.yaml
Normal file
118
infrastructure/cicd/tekton/pipelines/prod-deploy-pipeline.yaml
Normal file
@@ -0,0 +1,118 @@
|
||||
# Production Deployment Pipeline for Bakery-IA
|
||||
# This pipeline handles production deployments with manual approval gate
|
||||
# It should be triggered after the CI pipeline succeeds
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Pipeline
|
||||
metadata:
|
||||
name: bakery-ia-prod-deploy
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: pipeline
|
||||
app.kubernetes.io/environment: production
|
||||
spec:
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
description: Shared workspace for source code
|
||||
- name: git-credentials
|
||||
description: Git credentials for pushing GitOps updates
|
||||
optional: true
|
||||
params:
|
||||
- name: git-url
|
||||
type: string
|
||||
description: Repository URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision/commit hash to deploy
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to deploy
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: approver
|
||||
type: string
|
||||
description: Name of the person who approved this deployment
|
||||
default: "automated"
|
||||
- name: approval-ticket
|
||||
type: string
|
||||
description: Ticket/issue number for deployment approval
|
||||
default: "N/A"
|
||||
|
||||
tasks:
|
||||
# Stage 1: Fetch source code
|
||||
- name: fetch-source
|
||||
taskRef:
|
||||
name: git-clone
|
||||
workspaces:
|
||||
- name: output
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: url
|
||||
value: $(params.git-url)
|
||||
- name: revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
# Stage 2: Verify images exist in registry
|
||||
- name: verify-images
|
||||
runAfter: [fetch-source]
|
||||
taskRef:
|
||||
name: verify-images
|
||||
params:
|
||||
- name: services
|
||||
value: $(params.services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
|
||||
# Stage 3: Pre-deployment validation
|
||||
- name: pre-deploy-validation
|
||||
runAfter: [verify-images]
|
||||
taskRef:
|
||||
name: pre-deploy-validation
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
params:
|
||||
- name: services
|
||||
value: $(params.services)
|
||||
- name: environment
|
||||
value: "production"
|
||||
|
||||
# Stage 4: Update production manifests
|
||||
- name: update-prod-manifests
|
||||
runAfter: [pre-deploy-validation]
|
||||
taskRef:
|
||||
name: update-gitops
|
||||
workspaces:
|
||||
- name: source
|
||||
workspace: shared-workspace
|
||||
- name: git-credentials
|
||||
workspace: git-credentials
|
||||
params:
|
||||
- name: services
|
||||
value: $(params.services)
|
||||
- name: registry
|
||||
value: $(params.registry)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
- name: git-branch
|
||||
value: "main"
|
||||
- name: dry-run
|
||||
value: "false"
|
||||
|
||||
finally:
|
||||
- name: deployment-summary
|
||||
taskRef:
|
||||
name: prod-deployment-summary
|
||||
params:
|
||||
- name: services
|
||||
value: $(params.services)
|
||||
- name: git-revision
|
||||
value: $(params.git-revision)
|
||||
- name: approver
|
||||
value: $(params.approver)
|
||||
- name: approval-ticket
|
||||
value: $(params.approval-ticket)
|
||||
6
infrastructure/cicd/tekton/rbac/kustomization.yaml
Normal file
6
infrastructure/cicd/tekton/rbac/kustomization.yaml
Normal file
@@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- rbac.yaml
|
||||
- resource-quota.yaml
|
||||
159
infrastructure/cicd/tekton/rbac/rbac.yaml
Normal file
159
infrastructure/cicd/tekton/rbac/rbac.yaml
Normal file
@@ -0,0 +1,159 @@
|
||||
# Tekton RBAC Configuration for Bakery-IA CI/CD
|
||||
# This file defines ServiceAccounts, Roles, and RoleBindings for Tekton
|
||||
|
||||
---
|
||||
# ServiceAccount for Tekton Triggers EventListener
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tekton-triggers-sa
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
|
||||
---
|
||||
# ServiceAccount for Pipeline execution
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: tekton-pipeline-sa
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: pipeline
|
||||
|
||||
---
|
||||
# ClusterRole for Tekton Triggers to create PipelineRuns
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: tekton-triggers-role
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
rules:
|
||||
# Ability to create PipelineRuns from triggers
|
||||
- apiGroups: ["tekton.dev"]
|
||||
resources: ["pipelineruns", "taskruns"]
|
||||
verbs: ["create", "get", "list", "watch"]
|
||||
# Ability to read pipelines and tasks
|
||||
- apiGroups: ["tekton.dev"]
|
||||
resources: ["pipelines", "tasks", "clustertasks"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Ability to manage PVCs for workspaces
|
||||
- apiGroups: [""]
|
||||
resources: ["persistentvolumeclaims"]
|
||||
verbs: ["create", "get", "list", "watch", "delete"]
|
||||
# Ability to read secrets for credentials
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Ability to read configmaps
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Ability to manage events for logging
|
||||
- apiGroups: [""]
|
||||
resources: ["events"]
|
||||
verbs: ["create", "patch"]
|
||||
|
||||
---
|
||||
# ClusterRoleBinding for Tekton Triggers
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tekton-triggers-binding
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tekton-triggers-sa
|
||||
namespace: tekton-pipelines
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: tekton-triggers-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
---
|
||||
# ClusterRole for Pipeline execution (needed for git operations and deployments)
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRole
|
||||
metadata:
|
||||
name: tekton-pipeline-role
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: pipeline
|
||||
rules:
|
||||
# Ability to read/update deployments for GitOps
|
||||
- apiGroups: ["apps"]
|
||||
resources: ["deployments"]
|
||||
verbs: ["get", "list", "watch", "patch", "update"]
|
||||
# Ability to read secrets for credentials
|
||||
- apiGroups: [""]
|
||||
resources: ["secrets"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Ability to read configmaps
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
# Ability to manage pods for build operations
|
||||
- apiGroups: [""]
|
||||
resources: ["pods", "pods/log"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
|
||||
---
|
||||
# ClusterRoleBinding for Pipeline execution
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: tekton-pipeline-binding
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: pipeline
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tekton-pipeline-sa
|
||||
namespace: tekton-pipelines
|
||||
roleRef:
|
||||
kind: ClusterRole
|
||||
name: tekton-pipeline-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
|
||||
---
|
||||
# Role for EventListener to access triggers resources
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: Role
|
||||
metadata:
|
||||
name: tekton-triggers-eventlistener-role
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
rules:
|
||||
- apiGroups: ["triggers.tekton.dev"]
|
||||
resources: ["eventlisteners", "triggerbindings", "triggertemplates", "triggers", "interceptors"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
- apiGroups: [""]
|
||||
resources: ["configmaps", "secrets"]
|
||||
verbs: ["get", "list", "watch"]
|
||||
|
||||
---
|
||||
# RoleBinding for EventListener
|
||||
apiVersion: rbac.authorization.k8s.io/v1
|
||||
kind: RoleBinding
|
||||
metadata:
|
||||
name: tekton-triggers-eventlistener-binding
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
subjects:
|
||||
- kind: ServiceAccount
|
||||
name: tekton-triggers-sa
|
||||
namespace: tekton-pipelines
|
||||
roleRef:
|
||||
kind: Role
|
||||
name: tekton-triggers-eventlistener-role
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
64
infrastructure/cicd/tekton/rbac/resource-quota.yaml
Normal file
64
infrastructure/cicd/tekton/rbac/resource-quota.yaml
Normal file
@@ -0,0 +1,64 @@
|
||||
# ResourceQuota for Tekton Pipelines Namespace
|
||||
# Prevents resource exhaustion from runaway pipeline runs
|
||||
#
|
||||
# This quota limits:
|
||||
# - Total CPU and memory that can be requested/used
|
||||
# - Number of concurrent pods
|
||||
# - Number of PVCs for workspaces
|
||||
|
||||
apiVersion: v1
|
||||
kind: ResourceQuota
|
||||
metadata:
|
||||
name: tekton-pipelines-quota
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: quota
|
||||
spec:
|
||||
hard:
|
||||
# Limit total CPU
|
||||
requests.cpu: "8"
|
||||
limits.cpu: "16"
|
||||
# Limit total memory
|
||||
requests.memory: "16Gi"
|
||||
limits.memory: "32Gi"
|
||||
# Limit number of pods (controls concurrent pipeline tasks)
|
||||
pods: "20"
|
||||
# Limit PVCs (controls workspace storage)
|
||||
persistentvolumeclaims: "10"
|
||||
# Limit storage
|
||||
requests.storage: "50Gi"
|
||||
|
||||
---
|
||||
# LimitRange to set defaults and limits for individual pods
|
||||
# Ensures every pod has resource requests/limits
|
||||
apiVersion: v1
|
||||
kind: LimitRange
|
||||
metadata:
|
||||
name: tekton-pipelines-limits
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: quota
|
||||
spec:
|
||||
limits:
|
||||
# Default limits for containers
|
||||
- type: Container
|
||||
default:
|
||||
cpu: "1"
|
||||
memory: "1Gi"
|
||||
defaultRequest:
|
||||
cpu: "100m"
|
||||
memory: "256Mi"
|
||||
max:
|
||||
cpu: "4"
|
||||
memory: "8Gi"
|
||||
min:
|
||||
cpu: "50m"
|
||||
memory: "64Mi"
|
||||
# Limits for PVCs
|
||||
- type: PersistentVolumeClaim
|
||||
max:
|
||||
storage: "10Gi"
|
||||
min:
|
||||
storage: "1Gi"
|
||||
4
infrastructure/cicd/tekton/secrets/.gitignore
vendored
Normal file
4
infrastructure/cicd/tekton/secrets/.gitignore
vendored
Normal file
@@ -0,0 +1,4 @@
|
||||
# Ignore generated secrets
|
||||
.webhook-secret
|
||||
*-actual.yaml
|
||||
sealed-secrets.yaml
|
||||
167
infrastructure/cicd/tekton/secrets/generate-secrets.sh
Executable file
167
infrastructure/cicd/tekton/secrets/generate-secrets.sh
Executable file
@@ -0,0 +1,167 @@
|
||||
#!/bin/bash
|
||||
# Generate CI/CD Secrets for Bakery-IA
|
||||
#
|
||||
# This script creates Kubernetes secrets required for the CI/CD pipeline.
|
||||
# Run this script once during initial setup.
|
||||
#
|
||||
# Usage:
|
||||
# ./generate-secrets.sh [options]
|
||||
#
|
||||
# Options:
|
||||
# --registry-url Container registry URL (default: gitea.bakery-ia.local:5000)
|
||||
# --gitea-user Gitea username (will prompt if not provided)
|
||||
# --gitea-password Gitea password (will prompt if not provided)
|
||||
# --dry-run Print commands without executing
|
||||
|
||||
set -e
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# Default values
|
||||
REGISTRY_URL="${REGISTRY_URL:-gitea.bakery-ia.local:5000}"
|
||||
DRY_RUN=false
|
||||
KUBECTL="kubectl"
|
||||
|
||||
# Check if running in microk8s
|
||||
if command -v microk8s &> /dev/null; then
|
||||
KUBECTL="microk8s kubectl"
|
||||
fi
|
||||
|
||||
# Parse arguments
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
--registry-url)
|
||||
REGISTRY_URL="$2"
|
||||
shift 2
|
||||
;;
|
||||
--gitea-user)
|
||||
GITEA_USERNAME="$2"
|
||||
shift 2
|
||||
;;
|
||||
--gitea-password)
|
||||
GITEA_PASSWORD="$2"
|
||||
shift 2
|
||||
;;
|
||||
--dry-run)
|
||||
DRY_RUN=true
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}Unknown option: $1${NC}"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo "=========================================="
|
||||
echo " Bakery-IA CI/CD Secrets Generator"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
|
||||
# Prompt for credentials if not provided
|
||||
if [ -z "$GITEA_USERNAME" ]; then
|
||||
read -p "Enter Gitea username: " GITEA_USERNAME
|
||||
fi
|
||||
|
||||
if [ -z "$GITEA_PASSWORD" ]; then
|
||||
read -s -p "Enter Gitea password: " GITEA_PASSWORD
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# Generate webhook secret
|
||||
WEBHOOK_SECRET=$(openssl rand -hex 32)
|
||||
|
||||
echo ""
|
||||
echo -e "${YELLOW}Configuration:${NC}"
|
||||
echo " Registry URL: $REGISTRY_URL"
|
||||
echo " Gitea User: $GITEA_USERNAME"
|
||||
echo " Webhook Secret: ${WEBHOOK_SECRET:0:8}..."
|
||||
echo ""
|
||||
|
||||
# Function to create secret
|
||||
create_secret() {
|
||||
local cmd="$1"
|
||||
if [ "$DRY_RUN" = true ]; then
|
||||
echo -e "${YELLOW}[DRY-RUN]${NC} $cmd"
|
||||
else
|
||||
eval "$cmd"
|
||||
fi
|
||||
}
|
||||
|
||||
# Ensure namespaces exist
|
||||
echo -e "${GREEN}Creating namespaces if they don't exist...${NC}"
|
||||
create_secret "$KUBECTL create namespace tekton-pipelines --dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
create_secret "$KUBECTL create namespace flux-system --dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}Creating secrets...${NC}"
|
||||
|
||||
# 1. Webhook Secret
|
||||
echo " Creating gitea-webhook-secret..."
|
||||
create_secret "$KUBECTL create secret generic gitea-webhook-secret \
|
||||
--namespace tekton-pipelines \
|
||||
--from-literal=secretToken='$WEBHOOK_SECRET' \
|
||||
--dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
|
||||
# 2. Registry Credentials (docker-registry type)
|
||||
echo " Creating gitea-registry-credentials..."
|
||||
create_secret "$KUBECTL create secret docker-registry gitea-registry-credentials \
|
||||
--namespace tekton-pipelines \
|
||||
--docker-server='$REGISTRY_URL' \
|
||||
--docker-username='$GITEA_USERNAME' \
|
||||
--docker-password='$GITEA_PASSWORD' \
|
||||
--dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
|
||||
# 3. Git Credentials for Tekton
|
||||
echo " Creating gitea-git-credentials..."
|
||||
create_secret "$KUBECTL create secret generic gitea-git-credentials \
|
||||
--namespace tekton-pipelines \
|
||||
--from-literal=username='$GITEA_USERNAME' \
|
||||
--from-literal=password='$GITEA_PASSWORD' \
|
||||
--dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
|
||||
# 4. Flux Git Credentials
|
||||
echo " Creating gitea-credentials for Flux..."
|
||||
create_secret "$KUBECTL create secret generic gitea-credentials \
|
||||
--namespace flux-system \
|
||||
--from-literal=username='$GITEA_USERNAME' \
|
||||
--from-literal=password='$GITEA_PASSWORD' \
|
||||
--dry-run=client -o yaml | $KUBECTL apply -f -"
|
||||
|
||||
# Label all secrets
|
||||
echo ""
|
||||
echo -e "${GREEN}Adding labels to secrets...${NC}"
|
||||
for ns in tekton-pipelines flux-system; do
|
||||
for secret in gitea-webhook-secret gitea-registry-credentials gitea-git-credentials gitea-credentials; do
|
||||
if $KUBECTL get secret "$secret" -n "$ns" &> /dev/null; then
|
||||
create_secret "$KUBECTL label secret $secret -n $ns app.kubernetes.io/name=bakery-ia-cicd --overwrite 2>/dev/null || true"
|
||||
fi
|
||||
done
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=========================================="
|
||||
echo -e "${GREEN}Secrets created successfully!${NC}"
|
||||
echo "=========================================="
|
||||
echo ""
|
||||
echo -e "${YELLOW}IMPORTANT:${NC} Save this webhook secret for Gitea webhook configuration:"
|
||||
echo ""
|
||||
echo " Webhook Secret: $WEBHOOK_SECRET"
|
||||
echo ""
|
||||
echo "Configure this in Gitea:"
|
||||
echo " 1. Go to Repository Settings > Webhooks"
|
||||
echo " 2. Add webhook with URL: http://el-bakery-ia-listener.tekton-pipelines.svc.cluster.local:8080"
|
||||
echo " 3. Set Secret to the webhook secret above"
|
||||
echo " 4. Select events: Push"
|
||||
echo ""
|
||||
|
||||
# Save webhook secret to a file for reference (gitignored)
|
||||
if [ "$DRY_RUN" = false ]; then
|
||||
echo "$WEBHOOK_SECRET" > "$(dirname "$0")/.webhook-secret"
|
||||
chmod 600 "$(dirname "$0")/.webhook-secret"
|
||||
echo "Webhook secret saved to .webhook-secret (gitignored)"
|
||||
fi
|
||||
19
infrastructure/cicd/tekton/secrets/kustomization.yaml
Normal file
19
infrastructure/cicd/tekton/secrets/kustomization.yaml
Normal file
@@ -0,0 +1,19 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- secrets.yaml
|
||||
|
||||
# Note: In production, use sealed-secrets or external-secrets-operator
|
||||
# to manage secrets securely. The secrets.yaml file contains placeholder
|
||||
# values that must be replaced before deployment.
|
||||
#
|
||||
# Example using sealed-secrets:
|
||||
# 1. Install sealed-secrets controller
|
||||
# 2. Create SealedSecret resources instead of plain Secrets
|
||||
# 3. Commit the SealedSecret manifests to Git (safe to commit)
|
||||
#
|
||||
# Example using external-secrets-operator:
|
||||
# 1. Install external-secrets-operator
|
||||
# 2. Configure a SecretStore (AWS Secrets Manager, HashiCorp Vault, etc.)
|
||||
# 3. Create ExternalSecret resources that reference the SecretStore
|
||||
79
infrastructure/cicd/tekton/secrets/secrets-template.yaml
Normal file
79
infrastructure/cicd/tekton/secrets/secrets-template.yaml
Normal file
@@ -0,0 +1,79 @@
|
||||
# CI/CD Secrets Template for Tekton Pipelines
|
||||
#
|
||||
# DO NOT commit this file with actual credentials!
|
||||
# Use the generate-secrets.sh script to create secrets safely.
|
||||
#
|
||||
# For production, use one of these approaches:
|
||||
# 1. Sealed Secrets: kubeseal < secrets.yaml > sealed-secrets.yaml
|
||||
# 2. External Secrets Operator: Configure with your secret store
|
||||
# 3. Manual creation: kubectl create secret ... (see generate-secrets.sh)
|
||||
|
||||
---
|
||||
# Secret for Gitea webhook validation
|
||||
# Used by EventListener to validate incoming webhooks
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-webhook-secret
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Generate with: openssl rand -hex 32
|
||||
secretToken: "${WEBHOOK_SECRET_TOKEN}"
|
||||
|
||||
---
|
||||
# Secret for Gitea container registry credentials
|
||||
# Used by Kaniko to push images to Gitea registry
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-registry-credentials
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: build
|
||||
type: kubernetes.io/dockerconfigjson
|
||||
stringData:
|
||||
.dockerconfigjson: |
|
||||
{
|
||||
"auths": {
|
||||
"${REGISTRY_URL}": {
|
||||
"username": "${GITEA_USERNAME}",
|
||||
"password": "${GITEA_PASSWORD}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
---
|
||||
# Secret for Git credentials (used by pipeline to push GitOps updates)
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-git-credentials
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: gitops
|
||||
type: Opaque
|
||||
stringData:
|
||||
username: "${GITEA_USERNAME}"
|
||||
password: "${GITEA_PASSWORD}"
|
||||
|
||||
---
|
||||
# Secret for Flux GitRepository access
|
||||
# Used by Flux to pull from Gitea repository
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-credentials
|
||||
namespace: flux-system
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: flux
|
||||
type: Opaque
|
||||
stringData:
|
||||
username: "${GITEA_USERNAME}"
|
||||
password: "${GITEA_PASSWORD}"
|
||||
98
infrastructure/cicd/tekton/secrets/secrets.yaml
Normal file
98
infrastructure/cicd/tekton/secrets/secrets.yaml
Normal file
@@ -0,0 +1,98 @@
|
||||
# CI/CD Secrets for Tekton Pipelines
|
||||
#
|
||||
# WARNING: This file contains EXAMPLE values only!
|
||||
# DO NOT use these values in production.
|
||||
#
|
||||
# To create actual secrets, use ONE of these methods:
|
||||
#
|
||||
# Method 1 (Recommended): Use the generate-secrets.sh script
|
||||
# ./generate-secrets.sh --gitea-user <username> --gitea-password <password>
|
||||
#
|
||||
# Method 2: Create secrets manually with kubectl
|
||||
# kubectl create secret generic gitea-webhook-secret \
|
||||
# --namespace tekton-pipelines \
|
||||
# --from-literal=secretToken="$(openssl rand -hex 32)"
|
||||
#
|
||||
# Method 3: Use Sealed Secrets for GitOps
|
||||
# kubeseal < secrets-template.yaml > sealed-secrets.yaml
|
||||
#
|
||||
# Method 4: Use External Secrets Operator
|
||||
# Configure ESO to pull from your secret store (Vault, AWS SM, etc.)
|
||||
|
||||
---
|
||||
# Example Secret for Gitea webhook validation
|
||||
# Used by EventListener to validate incoming webhooks
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-webhook-secret
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
annotations:
|
||||
note: "EXAMPLE - Replace with actual secret using generate-secrets.sh"
|
||||
type: Opaque
|
||||
stringData:
|
||||
# Generate with: openssl rand -hex 32
|
||||
secretToken: "example-webhook-token-do-not-use-in-production"
|
||||
|
||||
---
|
||||
# Example Secret for Gitea container registry credentials
|
||||
# Used by Kaniko to push images to Gitea registry
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-registry-credentials
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: build
|
||||
annotations:
|
||||
note: "EXAMPLE - Replace with actual secret using generate-secrets.sh"
|
||||
type: kubernetes.io/dockerconfigjson
|
||||
stringData:
|
||||
.dockerconfigjson: |
|
||||
{
|
||||
"auths": {
|
||||
"gitea.bakery-ia.local:5000": {
|
||||
"username": "example-user",
|
||||
"password": "example-password"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
---
|
||||
# Example Secret for Git credentials (used by pipeline to push GitOps updates)
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-git-credentials
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: gitops
|
||||
annotations:
|
||||
note: "EXAMPLE - Replace with actual secret using generate-secrets.sh"
|
||||
type: Opaque
|
||||
stringData:
|
||||
username: "example-user"
|
||||
password: "example-password"
|
||||
|
||||
---
|
||||
# Example Secret for Flux GitRepository access
|
||||
# Used by Flux to pull from Gitea repository
|
||||
apiVersion: v1
|
||||
kind: Secret
|
||||
metadata:
|
||||
name: gitea-credentials
|
||||
namespace: flux-system
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: flux
|
||||
annotations:
|
||||
note: "EXAMPLE - Replace with actual secret using generate-secrets.sh"
|
||||
type: Opaque
|
||||
stringData:
|
||||
username: "example-user"
|
||||
password: "example-password"
|
||||
154
infrastructure/cicd/tekton/tasks/detect-changes.yaml
Normal file
154
infrastructure/cicd/tekton/tasks/detect-changes.yaml
Normal file
@@ -0,0 +1,154 @@
|
||||
# Tekton Detect Changed Services Task for Bakery-IA CI/CD
|
||||
# This task identifies which services have changed in the repository
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: detect-changed-services
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: detect
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
description: Source code workspace
|
||||
params:
|
||||
- name: base-ref
|
||||
type: string
|
||||
description: Base reference for comparison (default HEAD~1)
|
||||
default: "HEAD~1"
|
||||
results:
|
||||
- name: changed-services
|
||||
description: Comma-separated list of changed services
|
||||
- name: changed-files-count
|
||||
description: Number of files changed
|
||||
steps:
|
||||
- name: detect
|
||||
image: alpine/git:2.43.0
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
BASE_REF="$(params.base-ref)"
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
echo "============================================"
|
||||
echo "Detect Changed Services"
|
||||
echo "============================================"
|
||||
echo "Base ref: $BASE_REF"
|
||||
echo "============================================"
|
||||
|
||||
# Get list of changed files compared to base reference
|
||||
echo ""
|
||||
echo "Detecting changed files..."
|
||||
|
||||
# Try to get diff, fall back to listing all files if this is the first commit
|
||||
CHANGED_FILES=$(git diff --name-only "$BASE_REF" HEAD 2>/dev/null || git ls-tree -r HEAD --name-only)
|
||||
|
||||
FILE_COUNT=$(echo "$CHANGED_FILES" | grep -c "." || echo "0")
|
||||
echo "Found $FILE_COUNT changed files"
|
||||
echo "$FILE_COUNT" > $(results.changed-files-count.path)
|
||||
|
||||
if [ "$FILE_COUNT" = "0" ]; then
|
||||
echo "No files changed"
|
||||
echo "none" > $(results.changed-services.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Changed files:"
|
||||
echo "$CHANGED_FILES" | head -20
|
||||
if [ "$FILE_COUNT" -gt 20 ]; then
|
||||
echo "... and $((FILE_COUNT - 20)) more files"
|
||||
fi
|
||||
|
||||
# Map files to services using simple shell (no bash arrays)
|
||||
echo ""
|
||||
echo "Mapping files to services..."
|
||||
|
||||
CHANGED_SERVICES=""
|
||||
|
||||
# Process each file
|
||||
echo "$CHANGED_FILES" | while read -r file; do
|
||||
if [ -z "$file" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
# Check services directory
|
||||
if echo "$file" | grep -q "^services/"; then
|
||||
SERVICE=$(echo "$file" | cut -d'/' -f2)
|
||||
if [ -n "$SERVICE" ] && ! echo "$CHANGED_SERVICES" | grep -q "$SERVICE"; then
|
||||
if [ -z "$CHANGED_SERVICES" ]; then
|
||||
CHANGED_SERVICES="$SERVICE"
|
||||
else
|
||||
CHANGED_SERVICES="$CHANGED_SERVICES,$SERVICE"
|
||||
fi
|
||||
echo "$CHANGED_SERVICES" > /tmp/services.txt
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check frontend
|
||||
if echo "$file" | grep -q "^frontend/"; then
|
||||
if ! echo "$CHANGED_SERVICES" | grep -q "frontend"; then
|
||||
if [ -z "$CHANGED_SERVICES" ]; then
|
||||
CHANGED_SERVICES="frontend"
|
||||
else
|
||||
CHANGED_SERVICES="$CHANGED_SERVICES,frontend"
|
||||
fi
|
||||
echo "$CHANGED_SERVICES" > /tmp/services.txt
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check gateway
|
||||
if echo "$file" | grep -q "^gateway/"; then
|
||||
if ! echo "$CHANGED_SERVICES" | grep -q "gateway"; then
|
||||
if [ -z "$CHANGED_SERVICES" ]; then
|
||||
CHANGED_SERVICES="gateway"
|
||||
else
|
||||
CHANGED_SERVICES="$CHANGED_SERVICES,gateway"
|
||||
fi
|
||||
echo "$CHANGED_SERVICES" > /tmp/services.txt
|
||||
fi
|
||||
fi
|
||||
|
||||
# Check infrastructure
|
||||
if echo "$file" | grep -q "^infrastructure/"; then
|
||||
if ! echo "$CHANGED_SERVICES" | grep -q "infrastructure"; then
|
||||
if [ -z "$CHANGED_SERVICES" ]; then
|
||||
CHANGED_SERVICES="infrastructure"
|
||||
else
|
||||
CHANGED_SERVICES="$CHANGED_SERVICES,infrastructure"
|
||||
fi
|
||||
echo "$CHANGED_SERVICES" > /tmp/services.txt
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
# Read the accumulated services
|
||||
if [ -f /tmp/services.txt ]; then
|
||||
CHANGED_SERVICES=$(cat /tmp/services.txt)
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
|
||||
# Output result
|
||||
if [ -z "$CHANGED_SERVICES" ]; then
|
||||
echo "No service changes detected"
|
||||
echo "none" > $(results.changed-services.path)
|
||||
else
|
||||
echo "Detected changes in services: $CHANGED_SERVICES"
|
||||
echo "$CHANGED_SERVICES" > $(results.changed-services.path)
|
||||
fi
|
||||
|
||||
echo "============================================"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
95
infrastructure/cicd/tekton/tasks/git-clone.yaml
Normal file
95
infrastructure/cicd/tekton/tasks/git-clone.yaml
Normal file
@@ -0,0 +1,95 @@
|
||||
# Tekton Git Clone Task for Bakery-IA CI/CD
|
||||
# This task clones the source code repository
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: git-clone
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: source
|
||||
spec:
|
||||
workspaces:
|
||||
- name: output
|
||||
description: Workspace to clone the repository into
|
||||
params:
|
||||
- name: url
|
||||
type: string
|
||||
description: Repository URL to clone
|
||||
- name: revision
|
||||
type: string
|
||||
description: Git revision to checkout
|
||||
default: "main"
|
||||
- name: depth
|
||||
type: string
|
||||
description: Git clone depth (0 for full history)
|
||||
default: "1"
|
||||
results:
|
||||
- name: commit-sha
|
||||
description: The commit SHA that was checked out
|
||||
- name: commit-message
|
||||
description: The commit message
|
||||
steps:
|
||||
- name: clone
|
||||
image: alpine/git:2.43.0
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
URL="$(params.url)"
|
||||
REVISION="$(params.revision)"
|
||||
DEPTH="$(params.depth)"
|
||||
OUTPUT_PATH="$(workspaces.output.path)"
|
||||
|
||||
echo "============================================"
|
||||
echo "Git Clone Task"
|
||||
echo "============================================"
|
||||
echo "URL: $URL"
|
||||
echo "Revision: $REVISION"
|
||||
echo "Depth: $DEPTH"
|
||||
echo "============================================"
|
||||
|
||||
# Clone with depth for faster checkout
|
||||
if [ "$DEPTH" = "0" ]; then
|
||||
echo "Cloning full repository..."
|
||||
git clone "$URL" "$OUTPUT_PATH"
|
||||
else
|
||||
echo "Cloning with depth $DEPTH..."
|
||||
git clone --depth "$DEPTH" "$URL" "$OUTPUT_PATH"
|
||||
fi
|
||||
|
||||
cd "$OUTPUT_PATH"
|
||||
|
||||
# Fetch the specific revision if needed
|
||||
if [ "$REVISION" != "main" ] && [ "$REVISION" != "master" ]; then
|
||||
echo "Fetching revision: $REVISION"
|
||||
git fetch --depth 1 origin "$REVISION" 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Checkout the revision
|
||||
echo "Checking out: $REVISION"
|
||||
git checkout "$REVISION" 2>/dev/null || git checkout "origin/$REVISION"
|
||||
|
||||
# Get commit info
|
||||
COMMIT_SHA=$(git rev-parse HEAD)
|
||||
COMMIT_MSG=$(git log -1 --pretty=format:"%s")
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "Clone Complete"
|
||||
echo "============================================"
|
||||
echo "Commit: $COMMIT_SHA"
|
||||
echo "Message: $COMMIT_MSG"
|
||||
echo "============================================"
|
||||
|
||||
# Write results
|
||||
echo -n "$COMMIT_SHA" > $(results.commit-sha.path)
|
||||
echo -n "$COMMIT_MSG" > $(results.commit-message.path)
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
200
infrastructure/cicd/tekton/tasks/kaniko-build.yaml
Normal file
200
infrastructure/cicd/tekton/tasks/kaniko-build.yaml
Normal file
@@ -0,0 +1,200 @@
|
||||
# Tekton Kaniko Build Task for Bakery-IA CI/CD
|
||||
# This task builds and pushes container images using Kaniko
|
||||
# Supports building multiple services from a comma-separated list
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: kaniko-build
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: build
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
description: Source code workspace
|
||||
- name: docker-credentials
|
||||
description: Docker registry credentials
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to build
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision for image tag
|
||||
default: "latest"
|
||||
results:
|
||||
- name: built-images
|
||||
description: List of successfully built images
|
||||
- name: build-status
|
||||
description: Overall build status (success/failure)
|
||||
steps:
|
||||
# Step 1: Setup docker credentials
|
||||
- name: setup-docker-config
|
||||
image: alpine:3.18
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
echo "Setting up Docker credentials..."
|
||||
mkdir -p /kaniko/.docker
|
||||
|
||||
# Check if credentials secret is mounted
|
||||
if [ -f "$(workspaces.docker-credentials.path)/config.json" ]; then
|
||||
cp "$(workspaces.docker-credentials.path)/config.json" /kaniko/.docker/config.json
|
||||
echo "Docker config copied from secret"
|
||||
elif [ -f "$(workspaces.docker-credentials.path)/.dockerconfigjson" ]; then
|
||||
cp "$(workspaces.docker-credentials.path)/.dockerconfigjson" /kaniko/.docker/config.json
|
||||
echo "Docker config copied from .dockerconfigjson"
|
||||
else
|
||||
echo "Warning: No docker credentials found, builds may fail for private registries"
|
||||
echo '{}' > /kaniko/.docker/config.json
|
||||
fi
|
||||
volumeMounts:
|
||||
- name: docker-config
|
||||
mountPath: /kaniko/.docker
|
||||
resources:
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 64Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 32Mi
|
||||
|
||||
# Step 2: Build each service iteratively
|
||||
- name: build-services
|
||||
image: gcr.io/kaniko-project/executor:v1.23.0
|
||||
script: |
|
||||
#!/busybox/sh
|
||||
set -e
|
||||
|
||||
SERVICES="$(params.services)"
|
||||
REGISTRY="$(params.registry)"
|
||||
REVISION="$(params.git-revision)"
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
BUILT_IMAGES=""
|
||||
FAILED_SERVICES=""
|
||||
|
||||
echo "============================================"
|
||||
echo "Starting build for services: $SERVICES"
|
||||
echo "Registry: $REGISTRY"
|
||||
echo "Tag: $REVISION"
|
||||
echo "============================================"
|
||||
|
||||
# Skip if no services to build
|
||||
if [ "$SERVICES" = "none" ] || [ -z "$SERVICES" ]; then
|
||||
echo "No services to build, skipping..."
|
||||
echo "none" > $(results.built-images.path)
|
||||
echo "skipped" > $(results.build-status.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Convert comma-separated list to space-separated
|
||||
SERVICES_LIST=$(echo "$SERVICES" | tr ',' ' ')
|
||||
|
||||
for SERVICE in $SERVICES_LIST; do
|
||||
# Trim whitespace
|
||||
SERVICE=$(echo "$SERVICE" | tr -d ' ')
|
||||
|
||||
# Skip infrastructure changes (not buildable)
|
||||
if [ "$SERVICE" = "infrastructure" ]; then
|
||||
echo "Skipping infrastructure (not a buildable service)"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--------------------------------------------"
|
||||
echo "Building service: $SERVICE"
|
||||
echo "--------------------------------------------"
|
||||
|
||||
# Determine Dockerfile path based on service type
|
||||
if [ "$SERVICE" = "frontend" ]; then
|
||||
DOCKERFILE_PATH="$SOURCE_PATH/frontend/Dockerfile"
|
||||
CONTEXT_PATH="$SOURCE_PATH/frontend"
|
||||
elif [ "$SERVICE" = "gateway" ]; then
|
||||
DOCKERFILE_PATH="$SOURCE_PATH/gateway/Dockerfile"
|
||||
CONTEXT_PATH="$SOURCE_PATH/gateway"
|
||||
else
|
||||
DOCKERFILE_PATH="$SOURCE_PATH/services/$SERVICE/Dockerfile"
|
||||
CONTEXT_PATH="$SOURCE_PATH"
|
||||
fi
|
||||
|
||||
# Check if Dockerfile exists
|
||||
if [ ! -f "$DOCKERFILE_PATH" ]; then
|
||||
echo "Warning: Dockerfile not found at $DOCKERFILE_PATH, skipping $SERVICE"
|
||||
FAILED_SERVICES="$FAILED_SERVICES $SERVICE"
|
||||
continue
|
||||
fi
|
||||
|
||||
IMAGE_NAME="$REGISTRY/bakery/$SERVICE:$REVISION"
|
||||
IMAGE_NAME_LATEST="$REGISTRY/bakery/$SERVICE:latest"
|
||||
|
||||
echo "Dockerfile: $DOCKERFILE_PATH"
|
||||
echo "Context: $CONTEXT_PATH"
|
||||
echo "Image: $IMAGE_NAME"
|
||||
|
||||
# Run Kaniko build
|
||||
/kaniko/executor \
|
||||
--dockerfile="$DOCKERFILE_PATH" \
|
||||
--context="$CONTEXT_PATH" \
|
||||
--destination="$IMAGE_NAME" \
|
||||
--destination="$IMAGE_NAME_LATEST" \
|
||||
--cache=true \
|
||||
--cache-ttl=24h \
|
||||
--verbosity=info \
|
||||
--snapshot-mode=redo \
|
||||
--use-new-run
|
||||
|
||||
BUILD_EXIT_CODE=$?
|
||||
|
||||
if [ $BUILD_EXIT_CODE -eq 0 ]; then
|
||||
echo "Successfully built and pushed: $IMAGE_NAME"
|
||||
if [ -z "$BUILT_IMAGES" ]; then
|
||||
BUILT_IMAGES="$IMAGE_NAME"
|
||||
else
|
||||
BUILT_IMAGES="$BUILT_IMAGES,$IMAGE_NAME"
|
||||
fi
|
||||
else
|
||||
echo "Failed to build: $SERVICE (exit code: $BUILD_EXIT_CODE)"
|
||||
FAILED_SERVICES="$FAILED_SERVICES $SERVICE"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "Build Summary"
|
||||
echo "============================================"
|
||||
echo "Built images: $BUILT_IMAGES"
|
||||
echo "Failed services: $FAILED_SERVICES"
|
||||
|
||||
# Write results
|
||||
if [ -z "$BUILT_IMAGES" ]; then
|
||||
echo "none" > $(results.built-images.path)
|
||||
else
|
||||
echo "$BUILT_IMAGES" > $(results.built-images.path)
|
||||
fi
|
||||
|
||||
if [ -n "$FAILED_SERVICES" ]; then
|
||||
echo "partial" > $(results.build-status.path)
|
||||
echo "Warning: Some services failed to build: $FAILED_SERVICES"
|
||||
else
|
||||
echo "success" > $(results.build-status.path)
|
||||
fi
|
||||
volumeMounts:
|
||||
- name: docker-config
|
||||
mountPath: /kaniko/.docker
|
||||
securityContext:
|
||||
runAsUser: 0
|
||||
resources:
|
||||
limits:
|
||||
cpu: 2000m
|
||||
memory: 4Gi
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 1Gi
|
||||
volumes:
|
||||
- name: docker-config
|
||||
emptyDir: {}
|
||||
14
infrastructure/cicd/tekton/tasks/kustomization.yaml
Normal file
14
infrastructure/cicd/tekton/tasks/kustomization.yaml
Normal file
@@ -0,0 +1,14 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- git-clone.yaml
|
||||
- detect-changes.yaml
|
||||
- run-tests.yaml
|
||||
- kaniko-build.yaml
|
||||
- update-gitops.yaml
|
||||
- pipeline-summary.yaml
|
||||
# Production deployment tasks
|
||||
- verify-images.yaml
|
||||
- pre-deploy-validation.yaml
|
||||
- prod-deployment-summary.yaml
|
||||
62
infrastructure/cicd/tekton/tasks/pipeline-summary.yaml
Normal file
62
infrastructure/cicd/tekton/tasks/pipeline-summary.yaml
Normal file
@@ -0,0 +1,62 @@
|
||||
# Tekton Pipeline Summary Task for Bakery-IA CI/CD
|
||||
# This task runs at the end of the pipeline and provides a summary
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: pipeline-summary
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: summary
|
||||
spec:
|
||||
params:
|
||||
- name: changed-services
|
||||
type: string
|
||||
description: List of changed services
|
||||
default: "none"
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision that was built
|
||||
default: "unknown"
|
||||
steps:
|
||||
- name: summary
|
||||
image: alpine:3.18
|
||||
script: |
|
||||
#!/bin/sh
|
||||
|
||||
SERVICES="$(params.changed-services)"
|
||||
REVISION="$(params.git-revision)"
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo " Pipeline Execution Summary"
|
||||
echo "============================================"
|
||||
echo ""
|
||||
echo "Git Revision: $REVISION"
|
||||
echo "Changed Services: $SERVICES"
|
||||
echo ""
|
||||
echo "Timestamp: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo ""
|
||||
|
||||
if [ "$SERVICES" = "none" ] || [ -z "$SERVICES" ]; then
|
||||
echo "No services were changed in this commit."
|
||||
echo "Pipeline completed without building any images."
|
||||
else
|
||||
echo "The following services were processed:"
|
||||
echo "$SERVICES" | tr ',' '\n' | while read service; do
|
||||
echo " - $service"
|
||||
done
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 64Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 32Mi
|
||||
76
infrastructure/cicd/tekton/tasks/pre-deploy-validation.yaml
Normal file
76
infrastructure/cicd/tekton/tasks/pre-deploy-validation.yaml
Normal file
@@ -0,0 +1,76 @@
|
||||
# Task for pre-deployment validation
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: pre-deploy-validation
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: validation
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
description: Source code workspace
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to validate
|
||||
- name: environment
|
||||
type: string
|
||||
description: Target environment (staging/production)
|
||||
default: "production"
|
||||
results:
|
||||
- name: validation-status
|
||||
description: Status of validation (passed/failed)
|
||||
steps:
|
||||
- name: validate
|
||||
image: registry.k8s.io/kustomize/kustomize:v5.3.0
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
SERVICES="$(params.services)"
|
||||
ENVIRONMENT="$(params.environment)"
|
||||
|
||||
echo "============================================"
|
||||
echo "Pre-Deployment Validation"
|
||||
echo "============================================"
|
||||
echo "Environment: $ENVIRONMENT"
|
||||
echo "Services: $SERVICES"
|
||||
echo "============================================"
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
# Validate kustomization can be built
|
||||
KUSTOMIZE_DIR="infrastructure/environments/$ENVIRONMENT"
|
||||
|
||||
if [ -d "$KUSTOMIZE_DIR" ]; then
|
||||
echo ""
|
||||
echo "Validating kustomization..."
|
||||
if kustomize build "$KUSTOMIZE_DIR" > /dev/null 2>&1; then
|
||||
echo " ✓ Kustomization is valid"
|
||||
else
|
||||
echo " ✗ Kustomization validation failed"
|
||||
echo "failed" > $(results.validation-status.path)
|
||||
exit 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# Additional validation checks can be added here
|
||||
# - Schema validation
|
||||
# - Policy checks (OPA/Gatekeeper)
|
||||
# - Security scanning
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "All validations passed"
|
||||
echo "============================================"
|
||||
echo "passed" > $(results.validation-status.path)
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 256Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 128Mi
|
||||
@@ -0,0 +1,57 @@
|
||||
# Task for production deployment summary
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: prod-deployment-summary
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: summary
|
||||
spec:
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: List of deployed services
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision that was deployed
|
||||
- name: approver
|
||||
type: string
|
||||
description: Name of the approver
|
||||
- name: approval-ticket
|
||||
type: string
|
||||
description: Approval ticket number
|
||||
steps:
|
||||
- name: summary
|
||||
image: alpine:3.18
|
||||
script: |
|
||||
#!/bin/sh
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo " Production Deployment Summary"
|
||||
echo "============================================"
|
||||
echo ""
|
||||
echo "Git Revision: $(params.git-revision)"
|
||||
echo "Services: $(params.services)"
|
||||
echo "Approved By: $(params.approver)"
|
||||
echo "Approval Ticket: $(params.approval-ticket)"
|
||||
echo "Timestamp: $(date -u +"%Y-%m-%dT%H:%M:%SZ")"
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo ""
|
||||
echo "Deployment to production initiated."
|
||||
echo "Flux CD will reconcile the changes."
|
||||
echo ""
|
||||
echo "Monitor deployment status with:"
|
||||
echo " kubectl get kustomization -n flux-system"
|
||||
echo " kubectl get pods -n bakery-ia"
|
||||
echo ""
|
||||
echo "============================================"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 64Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 32Mi
|
||||
205
infrastructure/cicd/tekton/tasks/run-tests.yaml
Normal file
205
infrastructure/cicd/tekton/tasks/run-tests.yaml
Normal file
@@ -0,0 +1,205 @@
|
||||
# Tekton Test Task for Bakery-IA CI/CD
|
||||
# This task runs unit tests and linting for changed services
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: run-tests
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: test
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
description: Source code workspace
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to test
|
||||
- name: skip-lint
|
||||
type: string
|
||||
description: Skip linting if "true"
|
||||
default: "false"
|
||||
- name: skip-tests
|
||||
type: string
|
||||
description: Skip tests if "true"
|
||||
default: "false"
|
||||
results:
|
||||
- name: test-status
|
||||
description: Overall test status (passed/failed/skipped)
|
||||
- name: tested-services
|
||||
description: List of services that were tested
|
||||
- name: failed-services
|
||||
description: List of services that failed tests
|
||||
steps:
|
||||
- name: run-tests
|
||||
image: python:3.11-slim
|
||||
script: |
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
SERVICES="$(params.services)"
|
||||
SKIP_LINT="$(params.skip-lint)"
|
||||
SKIP_TESTS="$(params.skip-tests)"
|
||||
|
||||
TESTED_SERVICES=""
|
||||
FAILED_SERVICES=""
|
||||
OVERALL_STATUS="passed"
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
echo "============================================"
|
||||
echo "Running Tests"
|
||||
echo "============================================"
|
||||
echo "Services: $SERVICES"
|
||||
echo "Skip Lint: $SKIP_LINT"
|
||||
echo "Skip Tests: $SKIP_TESTS"
|
||||
echo "============================================"
|
||||
|
||||
# Skip if no services to test
|
||||
if [ "$SERVICES" = "none" ] || [ -z "$SERVICES" ]; then
|
||||
echo "No services to test, skipping..."
|
||||
echo "skipped" > $(results.test-status.path)
|
||||
echo "none" > $(results.tested-services.path)
|
||||
echo "none" > $(results.failed-services.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Install common test dependencies
|
||||
echo ""
|
||||
echo "Installing test dependencies..."
|
||||
pip install --quiet pytest pytest-cov pytest-asyncio ruff mypy 2>/dev/null || true
|
||||
|
||||
# Convert comma-separated list to space-separated
|
||||
SERVICES_LIST=$(echo "$SERVICES" | tr ',' ' ')
|
||||
|
||||
for SERVICE in $SERVICES_LIST; do
|
||||
# Trim whitespace
|
||||
SERVICE=$(echo "$SERVICE" | tr -d ' ')
|
||||
|
||||
# Skip infrastructure changes
|
||||
if [ "$SERVICE" = "infrastructure" ]; then
|
||||
echo "Skipping infrastructure (not testable)"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "--------------------------------------------"
|
||||
echo "Testing service: $SERVICE"
|
||||
echo "--------------------------------------------"
|
||||
|
||||
# Determine service path
|
||||
if [ "$SERVICE" = "frontend" ]; then
|
||||
SERVICE_PATH="$SOURCE_PATH/frontend"
|
||||
elif [ "$SERVICE" = "gateway" ]; then
|
||||
SERVICE_PATH="$SOURCE_PATH/gateway"
|
||||
else
|
||||
SERVICE_PATH="$SOURCE_PATH/services/$SERVICE"
|
||||
fi
|
||||
|
||||
# Check if service exists
|
||||
if [ ! -d "$SERVICE_PATH" ]; then
|
||||
echo "Warning: Service directory not found: $SERVICE_PATH"
|
||||
continue
|
||||
fi
|
||||
|
||||
cd "$SERVICE_PATH"
|
||||
SERVICE_FAILED=false
|
||||
|
||||
# Install service-specific dependencies if requirements.txt exists
|
||||
if [ -f "requirements.txt" ]; then
|
||||
echo "Installing service dependencies..."
|
||||
pip install --quiet -r requirements.txt 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# Run linting (ruff)
|
||||
if [ "$SKIP_LINT" != "true" ]; then
|
||||
echo ""
|
||||
echo "Running linter (ruff)..."
|
||||
if [ -d "app" ]; then
|
||||
ruff check app/ --output-format=text 2>&1 || {
|
||||
echo "Linting failed for $SERVICE"
|
||||
SERVICE_FAILED=true
|
||||
}
|
||||
fi
|
||||
fi
|
||||
|
||||
# Run tests
|
||||
if [ "$SKIP_TESTS" != "true" ]; then
|
||||
echo ""
|
||||
echo "Running tests (pytest)..."
|
||||
if [ -d "tests" ]; then
|
||||
pytest tests/ -v --tb=short 2>&1 || {
|
||||
echo "Tests failed for $SERVICE"
|
||||
SERVICE_FAILED=true
|
||||
}
|
||||
elif [ -d "app/tests" ]; then
|
||||
pytest app/tests/ -v --tb=short 2>&1 || {
|
||||
echo "Tests failed for $SERVICE"
|
||||
SERVICE_FAILED=true
|
||||
}
|
||||
else
|
||||
echo "No tests directory found, skipping tests"
|
||||
fi
|
||||
fi
|
||||
|
||||
# Track results
|
||||
if [ -z "$TESTED_SERVICES" ]; then
|
||||
TESTED_SERVICES="$SERVICE"
|
||||
else
|
||||
TESTED_SERVICES="$TESTED_SERVICES,$SERVICE"
|
||||
fi
|
||||
|
||||
if [ "$SERVICE_FAILED" = true ]; then
|
||||
OVERALL_STATUS="failed"
|
||||
if [ -z "$FAILED_SERVICES" ]; then
|
||||
FAILED_SERVICES="$SERVICE"
|
||||
else
|
||||
FAILED_SERVICES="$FAILED_SERVICES,$SERVICE"
|
||||
fi
|
||||
fi
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "Test Summary"
|
||||
echo "============================================"
|
||||
echo "Tested services: $TESTED_SERVICES"
|
||||
echo "Failed services: $FAILED_SERVICES"
|
||||
echo "Overall status: $OVERALL_STATUS"
|
||||
|
||||
# Write results
|
||||
echo "$OVERALL_STATUS" > $(results.test-status.path)
|
||||
|
||||
if [ -z "$TESTED_SERVICES" ]; then
|
||||
echo "none" > $(results.tested-services.path)
|
||||
else
|
||||
echo "$TESTED_SERVICES" > $(results.tested-services.path)
|
||||
fi
|
||||
|
||||
if [ -z "$FAILED_SERVICES" ]; then
|
||||
echo "none" > $(results.failed-services.path)
|
||||
else
|
||||
echo "$FAILED_SERVICES" > $(results.failed-services.path)
|
||||
fi
|
||||
|
||||
# Exit with error if tests failed
|
||||
if [ "$OVERALL_STATUS" = "failed" ]; then
|
||||
echo ""
|
||||
echo "ERROR: Some tests failed!"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "All tests passed!"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 2Gi
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 1Gi
|
||||
302
infrastructure/cicd/tekton/tasks/update-gitops.yaml
Normal file
302
infrastructure/cicd/tekton/tasks/update-gitops.yaml
Normal file
@@ -0,0 +1,302 @@
|
||||
# Tekton Update GitOps Manifests Task for Bakery-IA CI/CD
|
||||
# This task updates Kubernetes manifests with new image tags using Kustomize
|
||||
# It uses a safer approach than sed for updating image references
|
||||
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: update-gitops
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: gitops
|
||||
spec:
|
||||
workspaces:
|
||||
- name: source
|
||||
description: Source code workspace with Git repository
|
||||
- name: git-credentials
|
||||
description: Git credentials for pushing changes
|
||||
optional: true
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to update
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision for image tag
|
||||
- name: git-branch
|
||||
type: string
|
||||
description: Target branch for GitOps updates
|
||||
default: "main"
|
||||
- name: dry-run
|
||||
type: string
|
||||
description: If "true", only show what would be changed without committing
|
||||
default: "false"
|
||||
results:
|
||||
- name: updated-services
|
||||
description: List of services that were updated
|
||||
- name: commit-sha
|
||||
description: Git commit SHA of the update (empty if dry-run)
|
||||
steps:
|
||||
- name: update-manifests
|
||||
# Use alpine with curl to install kustomize
|
||||
image: alpine:3.19
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
# Install kustomize
|
||||
echo "Installing kustomize..."
|
||||
wget -q "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" -O - | sh
|
||||
mv kustomize /usr/local/bin/
|
||||
echo "Kustomize version: $(kustomize version)"
|
||||
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
SERVICES="$(params.services)"
|
||||
REGISTRY="$(params.registry)"
|
||||
REVISION="$(params.git-revision)"
|
||||
DRY_RUN="$(params.dry-run)"
|
||||
UPDATED_SERVICES=""
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
echo "============================================"
|
||||
echo "GitOps Manifest Update"
|
||||
echo "============================================"
|
||||
echo "Services: $SERVICES"
|
||||
echo "Registry: $REGISTRY"
|
||||
echo "Revision: $REVISION"
|
||||
echo "Dry Run: $DRY_RUN"
|
||||
echo "============================================"
|
||||
|
||||
# Skip if no services to update
|
||||
if [ "$SERVICES" = "none" ] || [ -z "$SERVICES" ]; then
|
||||
echo "No services to update, skipping..."
|
||||
echo "none" > $(results.updated-services.path)
|
||||
echo "" > $(results.commit-sha.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Define the kustomization directory
|
||||
KUSTOMIZE_DIR="infrastructure/environments/prod"
|
||||
|
||||
# Check if kustomization.yaml exists, create if not
|
||||
if [ ! -f "$KUSTOMIZE_DIR/kustomization.yaml" ]; then
|
||||
echo "Creating kustomization.yaml in $KUSTOMIZE_DIR"
|
||||
mkdir -p "$KUSTOMIZE_DIR"
|
||||
printf '%s\n' \
|
||||
"apiVersion: kustomize.config.k8s.io/v1beta1" \
|
||||
"kind: Kustomization" \
|
||||
"" \
|
||||
"resources:" \
|
||||
" - ../base" \
|
||||
"" \
|
||||
"images: []" \
|
||||
> "$KUSTOMIZE_DIR/kustomization.yaml"
|
||||
fi
|
||||
|
||||
# Convert comma-separated list to space-separated
|
||||
SERVICES_LIST=$(echo "$SERVICES" | tr ',' ' ')
|
||||
|
||||
# Build the images section for kustomization
|
||||
echo ""
|
||||
echo "Updating image references..."
|
||||
|
||||
for SERVICE in $SERVICES_LIST; do
|
||||
# Trim whitespace
|
||||
SERVICE=$(echo "$SERVICE" | tr -d ' ')
|
||||
|
||||
# Skip infrastructure changes
|
||||
if [ "$SERVICE" = "infrastructure" ]; then
|
||||
echo "Skipping infrastructure (not a deployable service)"
|
||||
continue
|
||||
fi
|
||||
|
||||
echo "Processing: $SERVICE"
|
||||
|
||||
# Determine the image name based on service
|
||||
NEW_IMAGE="$REGISTRY/bakery/$SERVICE:$REVISION"
|
||||
|
||||
# Use kustomize to set the image
|
||||
# This is safer than sed as it understands the YAML structure
|
||||
cd "$SOURCE_PATH/$KUSTOMIZE_DIR"
|
||||
|
||||
# Check if this service has a deployment
|
||||
SERVICE_DEPLOYMENT=""
|
||||
if [ "$SERVICE" = "frontend" ]; then
|
||||
SERVICE_DEPLOYMENT="frontend"
|
||||
elif [ "$SERVICE" = "gateway" ]; then
|
||||
SERVICE_DEPLOYMENT="gateway"
|
||||
else
|
||||
SERVICE_DEPLOYMENT="$SERVICE-service"
|
||||
fi
|
||||
|
||||
# Update the kustomization with the new image
|
||||
# Using kustomize edit to safely modify the file
|
||||
kustomize edit set image "bakery/$SERVICE=$NEW_IMAGE" 2>/dev/null || \
|
||||
kustomize edit set image "$SERVICE=$NEW_IMAGE" 2>/dev/null || \
|
||||
echo "Note: Could not set image via kustomize edit, will use alternative method"
|
||||
|
||||
# Track updated services
|
||||
if [ -z "$UPDATED_SERVICES" ]; then
|
||||
UPDATED_SERVICES="$SERVICE"
|
||||
else
|
||||
UPDATED_SERVICES="$UPDATED_SERVICES,$SERVICE"
|
||||
fi
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
done
|
||||
|
||||
# Alternative: Update images in kustomization.yaml directly if kustomize edit didn't work
|
||||
# This creates/updates an images section in the kustomization
|
||||
echo ""
|
||||
echo "Ensuring image overrides in kustomization.yaml..."
|
||||
|
||||
# Create a patch file for image updates
|
||||
IMAGES_FILE="$KUSTOMIZE_DIR/images.yaml"
|
||||
printf '%s\n' \
|
||||
"# Auto-generated by CI/CD pipeline" \
|
||||
"# Commit: $REVISION" \
|
||||
"# Updated: $(date -u +"%Y-%m-%dT%H:%M:%SZ")" \
|
||||
"images:" \
|
||||
> "$IMAGES_FILE"
|
||||
|
||||
for SERVICE in $SERVICES_LIST; do
|
||||
SERVICE=$(echo "$SERVICE" | tr -d ' ')
|
||||
if [ "$SERVICE" != "infrastructure" ]; then
|
||||
printf '%s\n' \
|
||||
" - name: bakery/$SERVICE" \
|
||||
" newName: $REGISTRY/bakery/$SERVICE" \
|
||||
" newTag: \"$REVISION\"" \
|
||||
>> "$IMAGES_FILE"
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "Generated images.yaml:"
|
||||
cat "$IMAGES_FILE"
|
||||
|
||||
# Validate the kustomization
|
||||
echo ""
|
||||
echo "Validating kustomization..."
|
||||
cd "$SOURCE_PATH/$KUSTOMIZE_DIR"
|
||||
if kustomize build . > /dev/null 2>&1; then
|
||||
echo "Kustomization is valid"
|
||||
else
|
||||
echo "Warning: Kustomization validation failed, but continuing..."
|
||||
fi
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
# Write results
|
||||
echo "$UPDATED_SERVICES" > $(results.updated-services.path)
|
||||
|
||||
if [ "$DRY_RUN" = "true" ]; then
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "DRY RUN - Changes not committed"
|
||||
echo "============================================"
|
||||
echo "Would update services: $UPDATED_SERVICES"
|
||||
git diff --stat || true
|
||||
echo "" > $(results.commit-sha.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Committing changes..."
|
||||
resources:
|
||||
limits:
|
||||
cpu: 500m
|
||||
memory: 512Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
|
||||
- name: commit-and-push
|
||||
image: alpine/git:2.43.0
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
SOURCE_PATH="$(workspaces.source.path)"
|
||||
SERVICES="$(params.services)"
|
||||
REVISION="$(params.git-revision)"
|
||||
BRANCH="$(params.git-branch)"
|
||||
DRY_RUN="$(params.dry-run)"
|
||||
|
||||
cd "$SOURCE_PATH"
|
||||
|
||||
if [ "$DRY_RUN" = "true" ]; then
|
||||
echo "Dry run mode - skipping commit"
|
||||
echo "" > $(results.commit-sha.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
if [ "$SERVICES" = "none" ] || [ -z "$SERVICES" ]; then
|
||||
echo "No services to commit"
|
||||
echo "" > $(results.commit-sha.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Check if there are changes to commit
|
||||
if git diff --quiet && git diff --cached --quiet; then
|
||||
echo "No changes to commit"
|
||||
echo "" > $(results.commit-sha.path)
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Configure git
|
||||
git config --global user.name "bakery-ia-ci"
|
||||
git config --global user.email "ci@bakery-ia.local"
|
||||
git config --global --add safe.directory "$SOURCE_PATH"
|
||||
|
||||
# Setup git credentials if provided
|
||||
if [ -d "$(workspaces.git-credentials.path)" ]; then
|
||||
if [ -f "$(workspaces.git-credentials.path)/username" ] && [ -f "$(workspaces.git-credentials.path)/password" ]; then
|
||||
GIT_USER=$(cat "$(workspaces.git-credentials.path)/username")
|
||||
GIT_PASS=$(cat "$(workspaces.git-credentials.path)/password")
|
||||
|
||||
# Get the remote URL and inject credentials
|
||||
REMOTE_URL=$(git remote get-url origin)
|
||||
# Handle both http and https
|
||||
if echo "$REMOTE_URL" | grep -q "^http"; then
|
||||
REMOTE_URL=$(echo "$REMOTE_URL" | sed "s|://|://$GIT_USER:$GIT_PASS@|")
|
||||
git remote set-url origin "$REMOTE_URL"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# Stage changes
|
||||
git add -A
|
||||
|
||||
# Create commit with detailed message
|
||||
COMMIT_MSG=$(printf 'ci: Update image tags to %s\n\nServices updated: %s\n\nThis commit was automatically generated by the CI/CD pipeline.\nPipeline run triggered by commit: %s' "$REVISION" "$SERVICES" "$REVISION")
|
||||
|
||||
git commit -m "$COMMIT_MSG"
|
||||
|
||||
# Get the commit SHA
|
||||
COMMIT_SHA=$(git rev-parse HEAD)
|
||||
echo "$COMMIT_SHA" > $(results.commit-sha.path)
|
||||
|
||||
echo "Created commit: $COMMIT_SHA"
|
||||
|
||||
# Push changes
|
||||
echo "Pushing to origin/$BRANCH..."
|
||||
git push origin HEAD:"$BRANCH"
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
echo "GitOps Update Complete"
|
||||
echo "============================================"
|
||||
echo "Commit: $COMMIT_SHA"
|
||||
echo "Branch: $BRANCH"
|
||||
echo "Services: $SERVICES"
|
||||
resources:
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 64Mi
|
||||
91
infrastructure/cicd/tekton/tasks/verify-images.yaml
Normal file
91
infrastructure/cicd/tekton/tasks/verify-images.yaml
Normal file
@@ -0,0 +1,91 @@
|
||||
# Task to verify images exist in the registry before deploying
|
||||
apiVersion: tekton.dev/v1beta1
|
||||
kind: Task
|
||||
metadata:
|
||||
name: verify-images
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: validation
|
||||
spec:
|
||||
params:
|
||||
- name: services
|
||||
type: string
|
||||
description: Comma-separated list of services to verify
|
||||
- name: registry
|
||||
type: string
|
||||
description: Container registry URL
|
||||
- name: git-revision
|
||||
type: string
|
||||
description: Git revision/tag to verify
|
||||
results:
|
||||
- name: verification-status
|
||||
description: Status of image verification (success/failed)
|
||||
- name: missing-images
|
||||
description: List of images that were not found
|
||||
steps:
|
||||
- name: verify
|
||||
image: gcr.io/go-containerregistry/crane:latest
|
||||
script: |
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
SERVICES="$(params.services)"
|
||||
REGISTRY="$(params.registry)"
|
||||
REVISION="$(params.git-revision)"
|
||||
MISSING=""
|
||||
|
||||
echo "============================================"
|
||||
echo "Verifying Images in Registry"
|
||||
echo "============================================"
|
||||
echo "Registry: $REGISTRY"
|
||||
echo "Revision: $REVISION"
|
||||
echo "Services: $SERVICES"
|
||||
echo "============================================"
|
||||
|
||||
# Convert comma-separated list to space-separated
|
||||
SERVICES_LIST=$(echo "$SERVICES" | tr ',' ' ')
|
||||
|
||||
for SERVICE in $SERVICES_LIST; do
|
||||
SERVICE=$(echo "$SERVICE" | tr -d ' ')
|
||||
|
||||
if [ "$SERVICE" = "infrastructure" ]; then
|
||||
continue
|
||||
fi
|
||||
|
||||
IMAGE="$REGISTRY/bakery/$SERVICE:$REVISION"
|
||||
echo ""
|
||||
echo "Checking: $IMAGE"
|
||||
|
||||
if crane manifest "$IMAGE" > /dev/null 2>&1; then
|
||||
echo " ✓ Found"
|
||||
else
|
||||
echo " ✗ NOT FOUND"
|
||||
if [ -z "$MISSING" ]; then
|
||||
MISSING="$SERVICE"
|
||||
else
|
||||
MISSING="$MISSING,$SERVICE"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "============================================"
|
||||
|
||||
if [ -n "$MISSING" ]; then
|
||||
echo "ERROR: Missing images: $MISSING"
|
||||
echo "failed" > $(results.verification-status.path)
|
||||
echo "$MISSING" > $(results.missing-images.path)
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "All images verified successfully"
|
||||
echo "success" > $(results.verification-status.path)
|
||||
echo "none" > $(results.missing-images.path)
|
||||
resources:
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 128Mi
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 64Mi
|
||||
35
infrastructure/cicd/tekton/triggers/event-listener.yaml
Normal file
35
infrastructure/cicd/tekton/triggers/event-listener.yaml
Normal file
@@ -0,0 +1,35 @@
|
||||
# Tekton EventListener for Bakery-IA CI/CD
|
||||
# This listener receives webhook events and triggers pipelines
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: EventListener
|
||||
metadata:
|
||||
name: bakery-ia-listener
|
||||
namespace: tekton-pipelines
|
||||
spec:
|
||||
serviceAccountName: tekton-triggers-sa
|
||||
triggers:
|
||||
- name: bakery-ia-gitea-trigger
|
||||
bindings:
|
||||
- ref: bakery-ia-trigger-binding
|
||||
template:
|
||||
ref: bakery-ia-trigger-template
|
||||
# Using CEL interceptor for local development (no TLS/CA bundle required)
|
||||
# The CEL interceptor is built-in and doesn't need external services
|
||||
interceptors:
|
||||
- name: "filter-push-events"
|
||||
ref:
|
||||
name: "cel"
|
||||
params:
|
||||
# Filter for push events from Gitea or GitHub
|
||||
- name: "filter"
|
||||
value: "header.match('X-Gitea-Event', 'push') || header.match('X-GitHub-Event', 'push')"
|
||||
# Add overlays to standardize the payload
|
||||
- name: "overlays"
|
||||
value:
|
||||
- key: "git_url"
|
||||
expression: "body.repository.clone_url"
|
||||
- key: "git_revision"
|
||||
expression: "body.after"
|
||||
- key: "git_branch"
|
||||
expression: "body.ref.split('/')[2]"
|
||||
9
infrastructure/cicd/tekton/triggers/kustomization.yaml
Normal file
9
infrastructure/cicd/tekton/triggers/kustomization.yaml
Normal file
@@ -0,0 +1,9 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
# NOTE: gitlab-interceptor.yaml removed - uses built-in Tekton Triggers interceptor
|
||||
# The gitlab ClusterInterceptor is provided by Tekton Triggers installation
|
||||
- event-listener.yaml
|
||||
- trigger-template.yaml
|
||||
- trigger-binding.yaml
|
||||
31
infrastructure/cicd/tekton/triggers/trigger-binding.yaml
Normal file
31
infrastructure/cicd/tekton/triggers/trigger-binding.yaml
Normal file
@@ -0,0 +1,31 @@
|
||||
# Tekton TriggerBinding for Bakery-IA CI/CD
|
||||
# This binding extracts parameters from Gitea webhook events
|
||||
#
|
||||
# Note: We use CEL overlay extensions for consistent field access
|
||||
# The EventListener's CEL interceptor creates these extensions:
|
||||
# - extensions.git_url: Repository clone URL
|
||||
# - extensions.git_revision: Commit SHA (from body.after)
|
||||
# - extensions.git_branch: Branch name (extracted from ref)
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: TriggerBinding
|
||||
metadata:
|
||||
name: bakery-ia-trigger-binding
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
spec:
|
||||
params:
|
||||
# Use CEL overlay extensions for consistent access across Git providers
|
||||
- name: git-repo-url
|
||||
value: $(extensions.git_url)
|
||||
- name: git-revision
|
||||
value: $(extensions.git_revision)
|
||||
- name: git-branch
|
||||
value: $(extensions.git_branch)
|
||||
# Direct body access for fields not in overlays
|
||||
- name: git-repo-name
|
||||
value: $(body.repository.name)
|
||||
- name: git-repo-full-name
|
||||
value: $(body.repository.full_name)
|
||||
86
infrastructure/cicd/tekton/triggers/trigger-template.yaml
Normal file
86
infrastructure/cicd/tekton/triggers/trigger-template.yaml
Normal file
@@ -0,0 +1,86 @@
|
||||
# Tekton TriggerTemplate for Bakery-IA CI/CD
|
||||
# This template defines how PipelineRuns are created when triggers fire
|
||||
#
|
||||
# Registry URL Configuration:
|
||||
# The registry URL is configured via the 'registry' parameter.
|
||||
# Default value should match pipeline-config ConfigMap's REGISTRY_URL.
|
||||
# To change the registry, update BOTH:
|
||||
# 1. This template's default value
|
||||
# 2. The pipeline-config ConfigMap
|
||||
|
||||
apiVersion: triggers.tekton.dev/v1beta1
|
||||
kind: TriggerTemplate
|
||||
metadata:
|
||||
name: bakery-ia-trigger-template
|
||||
namespace: tekton-pipelines
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
app.kubernetes.io/component: triggers
|
||||
spec:
|
||||
params:
|
||||
- name: git-repo-url
|
||||
description: The git repository URL
|
||||
- name: git-revision
|
||||
description: The git revision/commit hash
|
||||
- name: git-branch
|
||||
description: The git branch name
|
||||
default: "main"
|
||||
- name: git-repo-name
|
||||
description: The git repository name
|
||||
default: "bakery-ia"
|
||||
- name: git-repo-full-name
|
||||
description: The full repository name (org/repo)
|
||||
default: "bakery/bakery-ia"
|
||||
# Registry URL - keep in sync with pipeline-config ConfigMap
|
||||
- name: registry-url
|
||||
description: Container registry URL
|
||||
default: "gitea.bakery-ia.local:5000"
|
||||
resourcetemplates:
|
||||
- apiVersion: tekton.dev/v1beta1
|
||||
kind: PipelineRun
|
||||
metadata:
|
||||
generateName: bakery-ia-ci-run-
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia-cicd
|
||||
tekton.dev/pipeline: bakery-ia-ci
|
||||
triggers.tekton.dev/trigger: bakery-ia-gitea-trigger
|
||||
annotations:
|
||||
# Track the source commit
|
||||
bakery-ia.io/git-revision: $(tt.params.git-revision)
|
||||
bakery-ia.io/git-branch: $(tt.params.git-branch)
|
||||
spec:
|
||||
pipelineRef:
|
||||
name: bakery-ia-ci
|
||||
serviceAccountName: tekton-pipeline-sa
|
||||
workspaces:
|
||||
- name: shared-workspace
|
||||
volumeClaimTemplate:
|
||||
spec:
|
||||
accessModes: ["ReadWriteOnce"]
|
||||
resources:
|
||||
requests:
|
||||
storage: 5Gi
|
||||
- name: docker-credentials
|
||||
secret:
|
||||
secretName: gitea-registry-credentials
|
||||
- name: git-credentials
|
||||
secret:
|
||||
secretName: gitea-git-credentials
|
||||
params:
|
||||
- name: git-url
|
||||
value: $(tt.params.git-repo-url)
|
||||
- name: git-revision
|
||||
value: $(tt.params.git-revision)
|
||||
- name: git-branch
|
||||
value: $(tt.params.git-branch)
|
||||
# Use template parameter for registry URL
|
||||
- name: registry
|
||||
value: $(tt.params.registry-url)
|
||||
- name: skip-tests
|
||||
value: "false"
|
||||
- name: dry-run
|
||||
value: "false"
|
||||
# Timeout for the entire pipeline run
|
||||
timeouts:
|
||||
pipeline: "1h0m0s"
|
||||
tasks: "45m0s"
|
||||
@@ -7,7 +7,6 @@ metadata:
|
||||
app.kubernetes.io/name: bakery-ia
|
||||
app.kubernetes.io/component: config
|
||||
data:
|
||||
# ================================================================
|
||||
# ENVIRONMENT & BUILD SETTINGS
|
||||
# ================================================================
|
||||
ENVIRONMENT: "development"
|
||||
@@ -31,7 +30,7 @@ data:
|
||||
BUILD_DATE: "2024-01-20T10:00:00Z"
|
||||
VCS_REF: "latest"
|
||||
IMAGE_TAG: "latest"
|
||||
DOMAIN: "bakery.yourdomain.com"
|
||||
DOMAIN: "bakewise.ai"
|
||||
AUTO_RELOAD: "false"
|
||||
PROFILING_ENABLED: "false"
|
||||
MOCK_EXTERNAL_APIS: "false"
|
||||
@@ -177,13 +176,13 @@ data:
|
||||
# ================================================================
|
||||
# EMAIL CONFIGURATION
|
||||
# ================================================================
|
||||
SMTP_HOST: "smtp.gmail.com"
|
||||
SMTP_HOST: "email-smtp.bakery-ia.svc.cluster.local"
|
||||
SMTP_PORT: "587"
|
||||
SMTP_TLS: "true"
|
||||
SMTP_SSL: "false"
|
||||
DEFAULT_FROM_EMAIL: "noreply@bakeryforecast.es"
|
||||
DEFAULT_FROM_EMAIL: "noreply@bakewise.ai"
|
||||
DEFAULT_FROM_NAME: "Bakery-Forecast"
|
||||
EMAIL_FROM_ADDRESS: "alerts@bakery.local"
|
||||
EMAIL_FROM_ADDRESS: "alerts@bakewise.ai"
|
||||
EMAIL_FROM_NAME: "Bakery Alert System"
|
||||
|
||||
# ================================================================
|
||||
@@ -444,6 +443,13 @@ data:
|
||||
SIGNOZ_ENDPOINT: "http://signoz.bakery-ia.svc.cluster.local:8080"
|
||||
SIGNOZ_FRONTEND_URL: "https://monitoring.bakery-ia.local"
|
||||
|
||||
# ================================================================
|
||||
# DISTRIBUTION & ROUTING OPTIMIZATION SETTINGS
|
||||
# ================================================================
|
||||
VRP_TIME_LIMIT_SECONDS: "30"
|
||||
VRP_DEFAULT_VEHICLE_CAPACITY_KG: "1000"
|
||||
VRP_AVERAGE_SPEED_KMH: "30"
|
||||
|
||||
# ================================================================
|
||||
# REPLENISHMENT PLANNING SETTINGS
|
||||
# ================================================================
|
||||
@@ -0,0 +1,6 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
resources:
|
||||
- configmap.yaml
|
||||
- secrets.yaml
|
||||
@@ -160,8 +160,12 @@ metadata:
|
||||
app.kubernetes.io/component: notifications
|
||||
type: Opaque
|
||||
data:
|
||||
SMTP_USER: eW91ci1lbWFpbEBnbWFpbC5jb20= # your-email@gmail.com
|
||||
SMTP_PASSWORD: eW91ci1hcHAtc3BlY2lmaWMtcGFzc3dvcmQ= # your-app-specific-password
|
||||
# SMTP credentials for internal Mailu server
|
||||
# These are used by notification-service to send emails via mailu-smtp
|
||||
SMTP_USER: cG9zdG1hc3RlckBiYWtld2lzZS5haQ== # postmaster@bakewise.ai
|
||||
SMTP_PASSWORD: VzJYS2tSdUxpT25ZS2RCWVFTQXJvbjFpeWtFU1M1b2I= # W2XKkRuLiOnYKdBYQSAron1iykESS5ob
|
||||
# Dovecot admin password for IMAP management
|
||||
DOVEADM_PASSWORD: WnZhMzNoaVBJc2ZtV3RxUlBWV29taTRYZ2xLTlZPcHY= # Zva33hiPIsfmWtqRPVWomi4XglKNVOpv
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
159
infrastructure/environments/dev/k8s-manifests/kustomization.yaml
Normal file
159
infrastructure/environments/dev/k8s-manifests/kustomization.yaml
Normal file
@@ -0,0 +1,159 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: bakery-ia-dev
|
||||
|
||||
# NOTE: Do NOT set a global namespace here.
|
||||
# Each resource already has its namespace explicitly defined.
|
||||
# A global namespace would incorrectly transform cluster-scoped resources
|
||||
# like cert-manager namespaces.
|
||||
|
||||
resources:
|
||||
- ../../../environments/common/configs
|
||||
- ../../../platform/infrastructure
|
||||
- ../../../platform/cert-manager
|
||||
- ../../../platform/networking/ingress/overlays/dev
|
||||
- ../../../platform/storage
|
||||
- ../../../platform/mail/mailu
|
||||
- ../../../services/databases
|
||||
- ../../../services/microservices
|
||||
# NOTE: cicd is NOT included here - it's deployed manually via Tilt triggers
|
||||
# Run 'tilt trigger tekton-install' followed by 'tilt trigger tekton-pipelines-deploy'
|
||||
# - ../../../cicd
|
||||
- dev-certificate.yaml
|
||||
|
||||
|
||||
|
||||
# Dev-specific patches
|
||||
patches:
|
||||
- target:
|
||||
kind: ConfigMap
|
||||
name: bakery-config
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /data/ENVIRONMENT
|
||||
value: "development"
|
||||
- op: replace
|
||||
path: /data/DEBUG
|
||||
value: "true"
|
||||
# Suspend nominatim in dev to save resources
|
||||
- target:
|
||||
kind: StatefulSet
|
||||
name: nominatim
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/replicas
|
||||
value: 0
|
||||
# Suspend nominatim-init job in dev (not needed when nominatim is scaled to 0)
|
||||
- target:
|
||||
kind: Job
|
||||
name: nominatim-init
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/suspend
|
||||
value: true
|
||||
# Mailu TLS: Use self-signed dev certificate
|
||||
- target:
|
||||
kind: Deployment
|
||||
name: mailu-front
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/volumes/1/secret/secretName
|
||||
value: "bakery-dev-tls-cert"
|
||||
# Mailu Config: Update for dev environment
|
||||
- target:
|
||||
kind: ConfigMap
|
||||
name: mailu-config
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /data/DOMAIN
|
||||
value: "bakery-ia.local"
|
||||
- op: replace
|
||||
path: /data/HOSTNAMES
|
||||
value: "mail.bakery-ia.local"
|
||||
- op: replace
|
||||
path: /data/RELAY_LOGIN
|
||||
value: "postmaster@bakery-ia.local"
|
||||
- op: replace
|
||||
path: /data/WEBMAIL_ADMIN
|
||||
value: "admin@bakery-ia.local"
|
||||
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
environment: development
|
||||
tier: local
|
||||
|
||||
# Dev image overrides - use local registry to avoid Docker Hub rate limits
|
||||
# IMPORTANT: All image names must be lowercase (Docker requirement)
|
||||
# The prepull-base-images.sh script converts names to lowercase when pushing to local registry
|
||||
images:
|
||||
# Database images
|
||||
- name: postgres
|
||||
newName: localhost:5000/postgres_17-alpine
|
||||
newTag: latest
|
||||
- name: redis
|
||||
newName: localhost:5000/redis_7.4-alpine
|
||||
newTag: latest
|
||||
- name: rabbitmq
|
||||
newName: localhost:5000/rabbitmq_4.1-management-alpine
|
||||
newTag: latest
|
||||
# Utility images
|
||||
- name: busybox
|
||||
newName: localhost:5000/busybox_1.36
|
||||
newTag: latest
|
||||
- name: curlimages/curl
|
||||
newName: localhost:5000/curlimages_curl_latest
|
||||
newTag: latest
|
||||
- name: bitnami/kubectl
|
||||
newName: localhost:5000/bitnami_kubectl_latest
|
||||
newTag: latest
|
||||
# Alpine variants
|
||||
- name: alpine
|
||||
newName: localhost:5000/alpine_3.19
|
||||
newTag: latest
|
||||
- name: alpine/git
|
||||
newName: localhost:5000/alpine_git_2.43.0
|
||||
newTag: latest
|
||||
# CI/CD images (cached locally for consistency)
|
||||
- name: gcr.io/kaniko-project/executor
|
||||
newName: localhost:5000/gcr.io_kaniko-project_executor_v1.23.0
|
||||
newTag: latest
|
||||
- name: gcr.io/go-containerregistry/crane
|
||||
newName: localhost:5000/gcr.io_go-containerregistry_crane_latest
|
||||
newTag: latest
|
||||
- name: registry.k8s.io/kustomize/kustomize
|
||||
newName: localhost:5000/registry.k8s.io_kustomize_kustomize_v5.3.0
|
||||
newTag: latest
|
||||
# Storage images (lowercase - RELEASE becomes release)
|
||||
- name: minio/minio
|
||||
newName: localhost:5000/minio_minio_release.2024-11-07t00-52-20z
|
||||
newTag: latest
|
||||
- name: minio/mc
|
||||
newName: localhost:5000/minio_mc_release.2024-11-17t19-35-25z
|
||||
newTag: latest
|
||||
# Geocoding
|
||||
- name: mediagis/nominatim
|
||||
newName: localhost:5000/mediagis_nominatim_4.4
|
||||
newTag: latest
|
||||
# Python base image
|
||||
- name: python
|
||||
newName: localhost:5000/python_3.11-slim
|
||||
newTag: latest
|
||||
# Mail server (Mailu)
|
||||
- name: ghcr.io/mailu/nginx
|
||||
newName: localhost:5000/ghcr.io_mailu_nginx_2024.06
|
||||
newTag: latest
|
||||
- name: ghcr.io/mailu/admin
|
||||
newName: localhost:5000/ghcr.io_mailu_admin_2024.06
|
||||
newTag: latest
|
||||
- name: ghcr.io/mailu/postfix
|
||||
newName: localhost:5000/ghcr.io_mailu_postfix_2024.06
|
||||
newTag: latest
|
||||
- name: ghcr.io/mailu/dovecot
|
||||
newName: localhost:5000/ghcr.io_mailu_dovecot_2024.06
|
||||
newTag: latest
|
||||
- name: ghcr.io/mailu/rspamd
|
||||
newName: localhost:5000/ghcr.io_mailu_rspamd_2024.06
|
||||
newTag: latest
|
||||
@@ -4,18 +4,28 @@ kind: Kustomization
|
||||
metadata:
|
||||
name: bakery-ia-prod
|
||||
|
||||
namespace: bakery-ia
|
||||
# NOTE: Do NOT set a global namespace here.
|
||||
# Each resource already has its namespace explicitly defined.
|
||||
# A global namespace would incorrectly transform cluster-scoped resources
|
||||
# like flux-system and cert-manager namespaces.
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
- prod-ingress.yaml
|
||||
- ../../../environments/common/configs
|
||||
- ../../../platform/infrastructure
|
||||
- ../../../platform/cert-manager
|
||||
- ../../../platform/networking/ingress/overlays/prod
|
||||
- ../../../platform/storage
|
||||
- ../../../platform/mail/mailu
|
||||
- ../../../services/databases
|
||||
- ../../../services/microservices
|
||||
- ../../../cicd
|
||||
- prod-certificate.yaml
|
||||
|
||||
|
||||
# SigNoz is managed via Helm deployment (see infrastructure/helm/deploy-signoz.sh)
|
||||
# Monitoring is handled by SigNoz (no separate monitoring components needed)
|
||||
# SigNoz paths are now included in the main ingress (ingress-https.yaml)
|
||||
|
||||
patchesStrategicMerge:
|
||||
- storage-patch.yaml
|
||||
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
@@ -159,8 +169,17 @@ patches:
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "500m"
|
||||
# Mailu TLS: Use Let's Encrypt production certificate
|
||||
- target:
|
||||
kind: Deployment
|
||||
name: mailu-front
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/volumes/1/secret/secretName
|
||||
value: "bakery-ia-prod-tls-cert"
|
||||
|
||||
images:
|
||||
# Application services
|
||||
- name: bakery/auth-service
|
||||
newTag: latest
|
||||
- name: bakery/tenant-service
|
||||
@@ -193,6 +212,58 @@ images:
|
||||
newTag: latest
|
||||
- name: bakery/dashboard
|
||||
newTag: latest
|
||||
# =============================================================================
|
||||
# Production Base Images - mapped to production registry
|
||||
# TODO: Update PROD_REGISTRY_URL to your production registry (e.g., ghcr.io/your-org)
|
||||
# =============================================================================
|
||||
# Database images (using canonical Docker Hub - no rate limits in prod with auth)
|
||||
- name: postgres
|
||||
newTag: 17-alpine
|
||||
- name: redis
|
||||
newTag: 7.4-alpine
|
||||
- name: rabbitmq
|
||||
newTag: 4.1-management-alpine
|
||||
# Utility images
|
||||
- name: busybox
|
||||
newTag: "1.36"
|
||||
- name: curlimages/curl
|
||||
newTag: latest
|
||||
- name: bitnami/kubectl
|
||||
newTag: latest
|
||||
# Alpine variants
|
||||
- name: alpine
|
||||
newTag: "3.19"
|
||||
- name: alpine/git
|
||||
newTag: 2.43.0
|
||||
# CI/CD images (GCR/registry.k8s.io - no rate limits)
|
||||
- name: gcr.io/kaniko-project/executor
|
||||
newTag: v1.23.0
|
||||
- name: gcr.io/go-containerregistry/crane
|
||||
newTag: latest
|
||||
- name: registry.k8s.io/kustomize/kustomize
|
||||
newTag: v5.3.0
|
||||
# Storage images
|
||||
- name: minio/minio
|
||||
newTag: RELEASE.2024-11-07T00-52-20Z
|
||||
- name: minio/mc
|
||||
newTag: RELEASE.2024-11-17T19-35-25Z
|
||||
# Geocoding
|
||||
- name: mediagis/nominatim
|
||||
newTag: "4.4"
|
||||
# Python base image
|
||||
- name: python
|
||||
newTag: 3.11-slim
|
||||
# Mail server (Mailu) - using canonical GHCR names
|
||||
- name: ghcr.io/mailu/nginx
|
||||
newTag: "2024.06"
|
||||
- name: ghcr.io/mailu/admin
|
||||
newTag: "2024.06"
|
||||
- name: ghcr.io/mailu/postfix
|
||||
newTag: "2024.06"
|
||||
- name: ghcr.io/mailu/dovecot
|
||||
newTag: "2024.06"
|
||||
- name: ghcr.io/mailu/rspamd
|
||||
newTag: "2024.06"
|
||||
|
||||
replicas:
|
||||
- name: auth-service
|
||||
@@ -0,0 +1,48 @@
|
||||
apiVersion: cert-manager.io/v1
|
||||
kind: Certificate
|
||||
metadata:
|
||||
name: bakery-ia-prod-tls-cert
|
||||
namespace: bakery-ia
|
||||
spec:
|
||||
# Let's Encrypt certificate for production
|
||||
secretName: bakery-ia-prod-tls-cert
|
||||
|
||||
# Certificate duration and renewal
|
||||
duration: 2160h # 90 days (Let's Encrypt default)
|
||||
renewBefore: 360h # 15 days before expiry
|
||||
|
||||
# Subject configuration
|
||||
subject:
|
||||
organizations:
|
||||
- Bakery IA
|
||||
|
||||
# Common name
|
||||
commonName: bakewise.ai
|
||||
|
||||
# DNS names this certificate is valid for
|
||||
dnsNames:
|
||||
- bakewise.ai
|
||||
- www.bakewise.ai
|
||||
- mail.bakewise.ai
|
||||
- monitoring.bakewise.ai
|
||||
- gitea.bakewise.ai
|
||||
- api.bakewise.ai
|
||||
|
||||
# Use Let's Encrypt production issuer
|
||||
issuerRef:
|
||||
name: letsencrypt-production
|
||||
kind: ClusterIssuer
|
||||
group: cert-manager.io
|
||||
|
||||
# Private key configuration
|
||||
privateKey:
|
||||
algorithm: RSA
|
||||
encoding: PKCS1
|
||||
size: 2048
|
||||
|
||||
# Usages
|
||||
usages:
|
||||
- server auth
|
||||
- client auth
|
||||
- digital signature
|
||||
- key encipherment
|
||||
@@ -1,299 +0,0 @@
|
||||
# Bakery IA Kubernetes Configuration
|
||||
|
||||
This directory contains Kubernetes manifests for deploying the Bakery IA platform in local development and production environments with HTTPS support using cert-manager and NGINX ingress.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Deploy the entire platform with these 4 commands:
|
||||
|
||||
```bash
|
||||
# 1. Start Colima with adequate resources
|
||||
colima start --cpu 6 --memory 12 --disk 120 --runtime docker --profile k8s-local
|
||||
|
||||
# 2. Create Kind cluster with permanent localhost access
|
||||
kind create cluster --config kind-config.yaml
|
||||
|
||||
# 3. Install NGINX Ingress Controller
|
||||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
|
||||
kubectl wait --namespace ingress-nginx --for=condition=ready pod --selector=app.kubernetes.io/component=controller --timeout=300s
|
||||
|
||||
# 4. Deploy with Tilt
|
||||
tilt up
|
||||
|
||||
# 🎉 Access at: http://localhost (or see Tilt for individual service ports)
|
||||
```
|
||||
|
||||
> **Note**: The kind-config.yaml already configures port mappings (30080→80, 30443→443) for localhost access, so no additional service patching is needed. The NGINX Ingress for Kind uses NodePort by default on those exact ports.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Install the following tools on macOS:
|
||||
|
||||
```bash
|
||||
# Install via Homebrew
|
||||
brew install colima kind kubectl skaffold
|
||||
|
||||
# Verify installations
|
||||
colima version && kind version && kubectl version --client && skaffold version
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
infrastructure/kubernetes/
|
||||
├── base/ # Base Kubernetes resources
|
||||
│ ├── namespace.yaml # Namespace definition
|
||||
│ ├── configmap.yaml # Shared configuration
|
||||
│ ├── secrets.yaml # Base64 encoded secrets
|
||||
│ ├── ingress-https.yaml # HTTPS ingress rules
|
||||
│ ├── kustomization.yaml # Base kustomization
|
||||
│ └── components/ # Individual component manifests
|
||||
│ ├── cert-manager/ # Certificate management
|
||||
│ ├── auth/ # Authentication service
|
||||
│ ├── tenant/ # Tenant management
|
||||
│ ├── training/ # ML training service
|
||||
│ ├── forecasting/ # Demand forecasting
|
||||
│ ├── sales/ # Sales management
|
||||
│ ├── external/ # External API service
|
||||
│ ├── notification/ # Notification service
|
||||
│ ├── inventory/ # Inventory management
|
||||
│ ├── recipes/ # Recipe management
|
||||
│ ├── suppliers/ # Supplier management
|
||||
│ ├── pos/ # Point of sale
|
||||
│ ├── orders/ # Order management
|
||||
│ ├── production/ # Production planning
|
||||
│ ├── alert-processor/ # Alert processing
|
||||
│ ├── frontend/ # React frontend
|
||||
│ ├── databases/ # Database deployments
|
||||
│ └── infrastructure/ # Gateway & monitoring
|
||||
└── overlays/
|
||||
└── dev/ # Development environment
|
||||
├── kustomization.yaml # Dev-specific configuration
|
||||
└── dev-patches.yaml # Development patches
|
||||
```
|
||||
|
||||
## Access URLs
|
||||
|
||||
### Primary Access (Standard Web Ports)
|
||||
- **Frontend**: https://localhost
|
||||
- **API Gateway**: https://localhost/api
|
||||
|
||||
### Named Host Access (Optional)
|
||||
Add to `/etc/hosts` for named access:
|
||||
```bash
|
||||
echo "127.0.0.1 bakery-ia.local" | sudo tee -a /etc/hosts
|
||||
echo "127.0.0.1 api.bakery-ia.local" | sudo tee -a /etc/hosts
|
||||
echo "127.0.0.1 monitoring.bakery-ia.local" | sudo tee -a /etc/hosts
|
||||
```
|
||||
|
||||
Then access via:
|
||||
- **Frontend**: https://bakery-ia.local
|
||||
- **API**: https://api.bakery-ia.local
|
||||
- **Monitoring**: https://monitoring.bakery-ia.local
|
||||
|
||||
### Direct Service Access (Development)
|
||||
- **Frontend**: http://localhost:3000
|
||||
- **Gateway**: http://localhost:8000
|
||||
|
||||
## Development Workflow
|
||||
|
||||
### Start Development Environment
|
||||
```bash
|
||||
# Start development mode with hot-reload using Tilt
|
||||
tilt up
|
||||
|
||||
# Or start in background
|
||||
tilt up --stream
|
||||
```
|
||||
|
||||
### Key Features
|
||||
- ✅ **Hot-reload development** - Automatic rebuilds on code changes
|
||||
- ✅ **Permanent localhost access** - No port forwarding needed
|
||||
- ✅ **HTTPS by default** - Local CA certificates for secure development
|
||||
- ✅ **Microservices architecture** - All services deployed together
|
||||
- ✅ **Database management** - PostgreSQL, Redis, and RabbitMQ included
|
||||
|
||||
### Monitor and Debug
|
||||
```bash
|
||||
# Check all resources
|
||||
kubectl get all -n bakery-ia
|
||||
|
||||
# View logs
|
||||
kubectl logs -n bakery-ia deployment/auth-service -f
|
||||
|
||||
# Check ingress status
|
||||
kubectl get ingress -n bakery-ia
|
||||
|
||||
# Debug certificate issues
|
||||
kubectl describe certificate bakery-ia-tls-cert -n bakery-ia
|
||||
```
|
||||
|
||||
## Certificate Management
|
||||
|
||||
The platform uses cert-manager for automatic HTTPS certificate generation:
|
||||
|
||||
- **Local CA**: For development (default)
|
||||
- **Let's Encrypt Staging**: For testing
|
||||
- **Let's Encrypt Production**: For production deployments
|
||||
|
||||
### Trust Local Certificates
|
||||
```bash
|
||||
# Export CA certificate
|
||||
kubectl get secret local-ca-key-pair -n cert-manager -o jsonpath='{.data.tls\.crt}' | base64 -d > bakery-ia-ca.crt
|
||||
|
||||
# Trust in macOS
|
||||
open bakery-ia-ca.crt
|
||||
# In Keychain Access, set "bakery-ia-local-ca" to "Always Trust"
|
||||
```
|
||||
|
||||
## Configuration Management
|
||||
|
||||
### Secrets
|
||||
Base64-encoded secrets are stored in `base/secrets.yaml`. For production:
|
||||
- Use external secret management (HashiCorp Vault, AWS Secrets Manager)
|
||||
- Never commit real secrets to version control
|
||||
|
||||
```bash
|
||||
# Encode secrets
|
||||
echo -n "your-secret-value" | base64
|
||||
|
||||
# Decode secrets
|
||||
echo "eW91ci1zZWNyZXQtdmFsdWU=" | base64 -d
|
||||
```
|
||||
|
||||
### Environment Configuration
|
||||
Development-specific settings are in `overlays/dev/`:
|
||||
- **Resource limits**: Reduced for local development
|
||||
- **Image pull policy**: Never (for local images)
|
||||
- **Debug settings**: Enabled
|
||||
- **CORS**: Configured for localhost
|
||||
|
||||
## Scaling and Resource Management
|
||||
|
||||
### Scale Services
|
||||
```bash
|
||||
# Scale individual service
|
||||
kubectl scale -n bakery-ia deployment/auth-service --replicas=3
|
||||
|
||||
# Or update kustomization.yaml replicas section
|
||||
```
|
||||
|
||||
### Resource Configuration
|
||||
Development environment uses minimal resources:
|
||||
- **Databases**: 64Mi-256Mi memory, 25m-200m CPU
|
||||
- **Services**: 64Mi-256Mi memory, 25m-200m CPU
|
||||
- **Training Service**: 256Mi-1Gi memory (ML workloads)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
1. **Images not found**
|
||||
```bash
|
||||
# Build images with Skaffold
|
||||
skaffold build --profile=dev
|
||||
```
|
||||
|
||||
2. **Database corruption after restart**
|
||||
```bash
|
||||
# Delete corrupted PVC and restart
|
||||
kubectl delete pod -n bakery-ia -l app.kubernetes.io/name=inventory-db
|
||||
kubectl delete pvc -n bakery-ia inventory-db-pvc
|
||||
```
|
||||
|
||||
3. **HTTPS certificate not issued**
|
||||
```bash
|
||||
# Check cert-manager logs
|
||||
kubectl logs -n cert-manager deployment/cert-manager
|
||||
kubectl describe certificate bakery-ia-tls-cert -n bakery-ia
|
||||
```
|
||||
|
||||
4. **Port conflicts**
|
||||
```bash
|
||||
# Check what's using ports 80/443
|
||||
sudo lsof -i :80 -i :443
|
||||
```
|
||||
|
||||
### Debug Commands
|
||||
```bash
|
||||
# Get cluster events
|
||||
kubectl get events -n bakery-ia --sort-by='.firstTimestamp'
|
||||
|
||||
# Resource usage
|
||||
kubectl top pods -n bakery-ia
|
||||
kubectl top nodes
|
||||
|
||||
# Execute in pod
|
||||
kubectl exec -n bakery-ia -it <pod-name> -- bash
|
||||
```
|
||||
|
||||
## Cleanup
|
||||
|
||||
### Quick Cleanup
|
||||
```bash
|
||||
# Stop Skaffold (Ctrl+C or)
|
||||
skaffold delete --profile=dev
|
||||
```
|
||||
|
||||
### Complete Cleanup
|
||||
```bash
|
||||
# Delete everything
|
||||
kubectl delete namespace bakery-ia
|
||||
kind delete cluster --name bakery-ia-local
|
||||
colima stop --profile k8s-local
|
||||
```
|
||||
|
||||
### Restart Sequence
|
||||
```bash
|
||||
# Post-restart startup (or use kubernetes_restart.sh script)
|
||||
colima start --cpu 6 --memory 12 --disk 120 --runtime docker --profile k8s-local
|
||||
kind create cluster --config kind-config.yaml
|
||||
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml
|
||||
kubectl wait --namespace ingress-nginx --for=condition=ready pod --selector=app.kubernetes.io/component=controller --timeout=300s
|
||||
tilt up
|
||||
```
|
||||
|
||||
## Production Deployment
|
||||
|
||||
### Production URLs
|
||||
|
||||
The production environment uses the following domains:
|
||||
|
||||
- **Main Application**: https://bakewise.ai
|
||||
- Frontend application and all public pages
|
||||
- API endpoints: https://bakewise.ai/api/v1/...
|
||||
|
||||
- **Monitoring Stack**: https://monitoring.bakewise.ai
|
||||
- Grafana: https://monitoring.bakewise.ai/grafana
|
||||
- Prometheus: https://monitoring.bakewise.ai/prometheus
|
||||
- Jaeger: https://monitoring.bakewise.ai/jaeger
|
||||
- AlertManager: https://monitoring.bakewise.ai/alertmanager
|
||||
|
||||
### Production Configuration
|
||||
|
||||
The production overlay (`overlays/prod/`) includes:
|
||||
- **Domain Configuration**: bakewise.ai with Let's Encrypt certificates
|
||||
- **High Availability**: Multi-replica deployments (2-3 replicas per service)
|
||||
- **Enhanced Security**: Rate limiting, CORS restrictions, security headers
|
||||
- **Monitoring**: Full observability stack with Prometheus, Grafana, Jaeger
|
||||
|
||||
### Production Considerations
|
||||
|
||||
For production deployment:
|
||||
|
||||
- **Security**: Implement RBAC, network policies, pod security standards
|
||||
- **Monitoring**: Deploy Prometheus, Grafana, and alerting
|
||||
- **Backup**: Database backup strategies
|
||||
- **High Availability**: Multi-replica deployments with anti-affinity
|
||||
- **External Secrets**: Use managed secret services
|
||||
- **TLS**: Production Let's Encrypt certificates
|
||||
- **CI/CD**: Automated deployment pipelines
|
||||
- **DNS**: Configure DNS A/CNAME records pointing to your cluster's load balancer
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. Add comprehensive monitoring and logging
|
||||
2. Implement automated testing
|
||||
3. Set up CI/CD pipelines
|
||||
4. Add health checks and metrics endpoints
|
||||
5. Implement proper backup strategies
|
||||
@@ -1,14 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: cert-manager
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: ServiceAccount
|
||||
metadata:
|
||||
name: cert-manager-webhook
|
||||
namespace: cert-manager
|
||||
---
|
||||
# Cert-manager installation using Helm repository
|
||||
# This will be installed via kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.2/cert-manager.yaml
|
||||
# The actual installation will be done via command line, this file documents the resources
|
||||
@@ -1,78 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: distribution-service-config
|
||||
data:
|
||||
# Service settings
|
||||
SERVICE_NAME: "distribution-service"
|
||||
APP_NAME: "Bakery Distribution Service"
|
||||
DESCRIPTION: "Distribution service for enterprise tier bakery management"
|
||||
VERSION: "1.0.0"
|
||||
|
||||
# Database settings
|
||||
DB_POOL_SIZE: "10"
|
||||
DB_MAX_OVERFLOW: "20"
|
||||
DB_POOL_TIMEOUT: "30"
|
||||
DB_POOL_RECYCLE: "3600"
|
||||
DB_POOL_PRE_PING: "true"
|
||||
DB_ECHO: "false"
|
||||
|
||||
# Redis settings
|
||||
REDIS_DB: "7" # Use separate database for distribution service
|
||||
REDIS_MAX_CONNECTIONS: "50"
|
||||
REDIS_RETRY_ON_TIMEOUT: "true"
|
||||
REDIS_SOCKET_KEEPALIVE: "true"
|
||||
|
||||
# RabbitMQ settings
|
||||
RABBITMQ_EXCHANGE: "bakery_events"
|
||||
RABBITMQ_QUEUE_PREFIX: "distribution"
|
||||
RABBITMQ_RETRY_ATTEMPTS: "3"
|
||||
RABBITMQ_RETRY_DELAY: "5"
|
||||
|
||||
# Authentication settings
|
||||
JWT_ALGORITHM: "HS256"
|
||||
JWT_ACCESS_TOKEN_EXPIRE_MINUTES: "30"
|
||||
JWT_REFRESH_TOKEN_EXPIRE_DAYS: "7"
|
||||
ENABLE_SERVICE_AUTH: "true"
|
||||
|
||||
# HTTP client settings
|
||||
HTTP_TIMEOUT: "30"
|
||||
HTTP_RETRIES: "3"
|
||||
HTTP_RETRY_DELAY: "1.0"
|
||||
|
||||
# CORS settings
|
||||
CORS_ORIGINS: "http://localhost:3000,http://localhost:3001"
|
||||
CORS_ALLOW_CREDENTIALS: "true"
|
||||
CORS_ALLOW_METHODS: "GET,POST,PUT,DELETE,PATCH,OPTIONS"
|
||||
CORS_ALLOW_HEADERS: "*"
|
||||
|
||||
# Rate limiting
|
||||
RATE_LIMIT_ENABLED: "true"
|
||||
RATE_LIMIT_REQUESTS: "100"
|
||||
RATE_LIMIT_WINDOW: "60"
|
||||
RATE_LIMIT_BURST: "10"
|
||||
|
||||
# Monitoring and observability
|
||||
LOG_LEVEL: "INFO"
|
||||
PROMETHEUS_ENABLED: "true"
|
||||
PROMETHEUS_PORT: "9090"
|
||||
JAEGER_ENABLED: "false"
|
||||
JAEGER_AGENT_HOST: "jaeger-agent"
|
||||
JAEGER_AGENT_PORT: "6831"
|
||||
|
||||
# Health check settings
|
||||
HEALTH_CHECK_TIMEOUT: "30"
|
||||
HEALTH_CHECK_INTERVAL: "30"
|
||||
|
||||
# Business rules
|
||||
MAX_FORECAST_DAYS: "30"
|
||||
MIN_HISTORICAL_DAYS: "60"
|
||||
CONFIDENCE_THRESHOLD: "0.8"
|
||||
|
||||
# Routing optimization settings
|
||||
VRP_TIME_LIMIT_SECONDS: "30"
|
||||
VRP_DEFAULT_VEHICLE_CAPACITY_KG: "1000"
|
||||
VRP_AVERAGE_SPEED_KMH: "30"
|
||||
|
||||
# Service-specific settings
|
||||
DISTRIBUTION_SERVICE_URL: "http://distribution-service:8000"
|
||||
@@ -1,134 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: distribution-service
|
||||
labels:
|
||||
app: distribution-service
|
||||
tier: backend
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: distribution-service
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: distribution-service
|
||||
tier: backend
|
||||
spec:
|
||||
imagePullSecrets:
|
||||
- name: dockerhub-creds
|
||||
containers:
|
||||
- name: distribution-service
|
||||
image: bakery/distribution-service:latest
|
||||
imagePullPolicy: Always
|
||||
ports:
|
||||
- containerPort: 8000
|
||||
name: http
|
||||
env:
|
||||
- name: DATABASE_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: database-secret
|
||||
key: url
|
||||
- name: REDIS_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-secret
|
||||
key: url
|
||||
- name: RABBITMQ_URL
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: rabbitmq-secret
|
||||
key: url
|
||||
- name: JWT_SECRET_KEY
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: auth-secret
|
||||
key: jwt-secret
|
||||
- name: ENVIRONMENT
|
||||
value: "production"
|
||||
- name: LOG_LEVEL
|
||||
value: "INFO"
|
||||
- name: DB_POOL_SIZE
|
||||
value: "10"
|
||||
- name: DB_MAX_OVERFLOW
|
||||
value: "20"
|
||||
- name: REDIS_MAX_CONNECTIONS
|
||||
value: "50"
|
||||
- name: HTTP_TIMEOUT
|
||||
value: "30"
|
||||
- name: HTTP_RETRIES
|
||||
value: "3"
|
||||
# OpenTelemetry Configuration
|
||||
- name: OTEL_COLLECTOR_ENDPOINT
|
||||
value: "http://signoz-otel-collector.bakery-ia.svc.cluster.local:4318"
|
||||
- name: OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: OTEL_EXPORTER_OTLP_ENDPOINT
|
||||
- name: OTEL_SERVICE_NAME
|
||||
value: "distribution-service"
|
||||
- name: ENABLE_TRACING
|
||||
value: "true"
|
||||
# Logging Configuration
|
||||
- name: OTEL_LOGS_EXPORTER
|
||||
value: "otlp"
|
||||
- name: OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED
|
||||
value: "true"
|
||||
# Metrics Configuration
|
||||
- name: ENABLE_OTEL_METRICS
|
||||
value: "true"
|
||||
- name: ENABLE_SYSTEM_METRICS
|
||||
value: "true"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
timeoutSeconds: 5
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health
|
||||
port: 8000
|
||||
initialDelaySeconds: 5
|
||||
periodSeconds: 5
|
||||
timeoutSeconds: 3
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "250m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
allowPrivilegeEscalation: false
|
||||
readOnlyRootFilesystem: false
|
||||
capabilities:
|
||||
drop:
|
||||
- ALL
|
||||
securityContext:
|
||||
runAsNonRoot: true
|
||||
runAsUser: 1000
|
||||
fsGroup: 2000
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: distribution-service
|
||||
labels:
|
||||
app: distribution-service
|
||||
tier: backend
|
||||
spec:
|
||||
selector:
|
||||
app.kubernetes.io/name: distribution-service
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 8000
|
||||
targetPort: 8000
|
||||
name: http
|
||||
type: ClusterIP
|
||||
@@ -1,151 +0,0 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: {{SERVICE_NAME}}-service
|
||||
namespace: bakery-ia
|
||||
labels:
|
||||
app.kubernetes.io/name: {{SERVICE_NAME}}-service
|
||||
app.kubernetes.io/component: microservice
|
||||
app.kubernetes.io/part-of: bakery-ia
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/name: {{SERVICE_NAME}}-service
|
||||
app.kubernetes.io/component: microservice
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app.kubernetes.io/name: {{SERVICE_NAME}}-service
|
||||
app.kubernetes.io/component: microservice
|
||||
spec:
|
||||
containers:
|
||||
- name: {{SERVICE_NAME}}-service
|
||||
image: bakery/{{SERVICE_NAME}}-service:latest
|
||||
ports:
|
||||
- containerPort: 8000
|
||||
name: http
|
||||
env:
|
||||
- name: ENVIRONMENT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: ENVIRONMENT
|
||||
- name: DEBUG
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: DEBUG
|
||||
- name: LOG_LEVEL
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: LOG_LEVEL
|
||||
- name: {{SERVICE_NAME_UPPER}}_DB_HOST
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: {{SERVICE_NAME_UPPER}}_DB_HOST
|
||||
- name: {{SERVICE_NAME_UPPER}}_DB_PORT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: DB_PORT
|
||||
- name: {{SERVICE_NAME_UPPER}}_DB_NAME
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: {{SERVICE_NAME_UPPER}}_DB_NAME
|
||||
- name: {{SERVICE_NAME_UPPER}}_DB_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: database-secrets
|
||||
key: {{SERVICE_NAME_UPPER}}_DB_USER
|
||||
- name: {{SERVICE_NAME_UPPER}}_DB_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: database-secrets
|
||||
key: {{SERVICE_NAME_UPPER}}_DB_PASSWORD
|
||||
- name: REDIS_HOST
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: REDIS_HOST
|
||||
- name: REDIS_PORT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: REDIS_PORT
|
||||
- name: REDIS_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: redis-secrets
|
||||
key: REDIS_PASSWORD
|
||||
- name: RABBITMQ_HOST
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: RABBITMQ_HOST
|
||||
- name: RABBITMQ_PORT
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: RABBITMQ_PORT
|
||||
- name: RABBITMQ_USER
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: rabbitmq-secrets
|
||||
key: RABBITMQ_USER
|
||||
- name: RABBITMQ_PASSWORD
|
||||
valueFrom:
|
||||
secretKeyRef:
|
||||
name: rabbitmq-secrets
|
||||
key: RABBITMQ_PASSWORD
|
||||
- name: AUTH_SERVICE_URL
|
||||
valueFrom:
|
||||
configMapKeyRef:
|
||||
name: bakery-config
|
||||
key: AUTH_SERVICE_URL
|
||||
resources:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /health/live
|
||||
port: 8000
|
||||
initialDelaySeconds: 30
|
||||
timeoutSeconds: 5
|
||||
periodSeconds: 10
|
||||
failureThreshold: 3
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /health/ready
|
||||
port: 8000
|
||||
initialDelaySeconds: 15
|
||||
timeoutSeconds: 3
|
||||
periodSeconds: 5
|
||||
failureThreshold: 5
|
||||
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: {{SERVICE_NAME}}-service
|
||||
namespace: bakery-ia
|
||||
labels:
|
||||
app.kubernetes.io/name: {{SERVICE_NAME}}-service
|
||||
app.kubernetes.io/component: microservice
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 8000
|
||||
targetPort: 8000
|
||||
protocol: TCP
|
||||
name: http
|
||||
selector:
|
||||
app.kubernetes.io/name: {{SERVICE_NAME}}-service
|
||||
app.kubernetes.io/component: microservice
|
||||
@@ -1,69 +0,0 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: bakery-ingress-https
|
||||
namespace: bakery-ia
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia
|
||||
app.kubernetes.io/component: ingress
|
||||
annotations:
|
||||
# Nginx ingress controller annotations
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
|
||||
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
||||
# SSE and WebSocket configuration for long-lived connections
|
||||
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
||||
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"
|
||||
nginx.ingress.kubernetes.io/upstream-keepalive-timeout: "3600"
|
||||
# WebSocket upgrade support
|
||||
nginx.ingress.kubernetes.io/websocket-services: "gateway-service"
|
||||
# CORS configuration for HTTPS
|
||||
nginx.ingress.kubernetes.io/enable-cors: "true"
|
||||
nginx.ingress.kubernetes.io/cors-allow-origin: "https://your-domain.com" # To be overridden in overlays
|
||||
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS, PATCH"
|
||||
nginx.ingress.kubernetes.io/cors-allow-headers: "Content-Type, Authorization, X-Requested-With, Accept, Origin, Cache-Control"
|
||||
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
|
||||
# Cert-manager annotations for automatic certificate issuance
|
||||
# Using issuer appropriate for environment
|
||||
cert-manager.io/cluster-issuer: "letsencrypt-prod" # To be overridden in dev overlay
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- your-domain.com # To be overridden in overlays
|
||||
secretName: bakery-tls-cert # To be overridden in overlays
|
||||
rules:
|
||||
- host: your-domain.com # To be overridden in overlays
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: frontend-service
|
||||
port:
|
||||
number: 3000
|
||||
- path: /api
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: gateway-service
|
||||
port:
|
||||
number: 8000
|
||||
- host: api.your-domain.com # To be overridden in overlays
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: gateway-service
|
||||
port:
|
||||
number: 8000
|
||||
# Note: SigNoz monitoring is deployed via Helm in the 'signoz' namespace
|
||||
# SigNoz creates its own Ingress via Helm chart configuration
|
||||
# Access at: https://monitoring.your-domain.com/ (configured in signoz-values.yaml)
|
||||
# SignOz ingress is managed separately - no need to configure here
|
||||
@@ -1,181 +0,0 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: bakery-ia-base
|
||||
|
||||
resources:
|
||||
# Base configuration
|
||||
- namespace.yaml
|
||||
- configmap.yaml
|
||||
- secrets.yaml
|
||||
- ingress-https.yaml
|
||||
|
||||
# TLS configuration
|
||||
- configmaps/postgres-logging-config.yaml
|
||||
- secrets/postgres-tls-secret.yaml
|
||||
- secrets/redis-tls-secret.yaml
|
||||
|
||||
# Additional configs
|
||||
- configs/postgres-init-config.yaml
|
||||
|
||||
# MinIO Storage (with TLS)
|
||||
- components/minio/minio-secrets.yaml
|
||||
- secrets/minio-tls-secret.yaml
|
||||
- components/minio/minio-pvc.yaml
|
||||
- components/minio/minio-deployment.yaml
|
||||
- jobs/minio-bucket-init-job.yaml
|
||||
|
||||
# Migration jobs
|
||||
- migrations/auth-migration-job.yaml
|
||||
- migrations/tenant-migration-job.yaml
|
||||
# Note: tenant-seed-pilot-coupon-job.yaml removed - pilot coupon is now seeded
|
||||
# automatically during tenant-service startup (see app/jobs/startup_seeder.py)
|
||||
- migrations/training-migration-job.yaml
|
||||
- migrations/forecasting-migration-job.yaml
|
||||
- migrations/sales-migration-job.yaml
|
||||
- migrations/external-migration-job.yaml
|
||||
- migrations/notification-migration-job.yaml
|
||||
- migrations/inventory-migration-job.yaml
|
||||
- migrations/recipes-migration-job.yaml
|
||||
- migrations/suppliers-migration-job.yaml
|
||||
- migrations/pos-migration-job.yaml
|
||||
- migrations/orders-migration-job.yaml
|
||||
- migrations/production-migration-job.yaml
|
||||
- migrations/alert-processor-migration-job.yaml
|
||||
- migrations/demo-session-migration-job.yaml
|
||||
- migrations/procurement-migration-job.yaml
|
||||
- migrations/orchestrator-migration-job.yaml
|
||||
- migrations/ai-insights-migration-job.yaml
|
||||
- migrations/distribution-migration-job.yaml
|
||||
- migrations/demo-seed-rbac.yaml
|
||||
|
||||
# External data initialization job (v2.0)
|
||||
- jobs/external-data-init-job.yaml
|
||||
|
||||
# CronJobs
|
||||
- cronjobs/demo-cleanup-cronjob.yaml
|
||||
- cronjobs/external-data-rotation-cronjob.yaml
|
||||
|
||||
# Infrastructure components
|
||||
- components/databases/redis.yaml
|
||||
- components/databases/rabbitmq.yaml
|
||||
- components/infrastructure/gateway-service.yaml
|
||||
|
||||
# Distribution service
|
||||
- components/distribution/distribution-deployment.yaml
|
||||
- components/distribution/distribution-configmap.yaml
|
||||
|
||||
# Nominatim geocoding service
|
||||
- components/nominatim/nominatim.yaml
|
||||
- jobs/nominatim-init-job.yaml
|
||||
|
||||
# Cert manager cluster issuers
|
||||
- components/cert-manager/cluster-issuer-staging.yaml
|
||||
- components/cert-manager/local-ca-issuer.yaml
|
||||
|
||||
# Database services
|
||||
- components/databases/auth-db.yaml
|
||||
- components/databases/tenant-db.yaml
|
||||
- components/databases/training-db.yaml
|
||||
- components/databases/forecasting-db.yaml
|
||||
- components/databases/sales-db.yaml
|
||||
- components/databases/external-db.yaml
|
||||
- components/databases/notification-db.yaml
|
||||
- components/databases/inventory-db.yaml
|
||||
- components/databases/recipes-db.yaml
|
||||
- components/databases/suppliers-db.yaml
|
||||
- components/databases/pos-db.yaml
|
||||
- components/databases/orders-db.yaml
|
||||
- components/databases/production-db.yaml
|
||||
- components/databases/procurement-db.yaml
|
||||
- components/databases/orchestrator-db.yaml
|
||||
- components/databases/alert-processor-db.yaml
|
||||
- components/databases/ai-insights-db.yaml
|
||||
- components/databases/distribution-db.yaml
|
||||
|
||||
# Demo session components
|
||||
- components/demo-session/database.yaml
|
||||
- components/demo-session/rbac.yaml
|
||||
- components/demo-session/service.yaml
|
||||
- components/demo-session/deployment.yaml
|
||||
|
||||
# Demo cleanup worker (background job processor)
|
||||
- deployments/demo-cleanup-worker.yaml
|
||||
|
||||
# Microservices
|
||||
- components/auth/auth-service.yaml
|
||||
- components/tenant/tenant-service.yaml
|
||||
- components/training/training-service.yaml
|
||||
- components/forecasting/forecasting-service.yaml
|
||||
- components/sales/sales-service.yaml
|
||||
- components/external/external-service.yaml
|
||||
- components/notification/notification-service.yaml
|
||||
- components/inventory/inventory-service.yaml
|
||||
- components/recipes/recipes-service.yaml
|
||||
- components/suppliers/suppliers-service.yaml
|
||||
- components/pos/pos-service.yaml
|
||||
- components/orders/orders-service.yaml
|
||||
- components/production/production-service.yaml
|
||||
- components/procurement/procurement-service.yaml
|
||||
- components/orchestrator/orchestrator-service.yaml
|
||||
- components/alert-processor/alert-processor.yaml
|
||||
- components/ai-insights/ai-insights-service.yaml
|
||||
|
||||
# Frontend
|
||||
- components/frontend/frontend-service.yaml
|
||||
|
||||
# HorizontalPodAutoscalers (for production autoscaling)
|
||||
- components/hpa/orders-hpa.yaml
|
||||
- components/hpa/forecasting-hpa.yaml
|
||||
- components/hpa/notification-hpa.yaml
|
||||
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
app.kubernetes.io/part-of: bakery-ia
|
||||
app.kubernetes.io/managed-by: kustomize
|
||||
|
||||
images:
|
||||
- name: bakery/auth-service
|
||||
newTag: latest
|
||||
- name: bakery/tenant-service
|
||||
newTag: latest
|
||||
- name: bakery/training-service
|
||||
newTag: latest
|
||||
- name: bakery/forecasting-service
|
||||
newTag: latest
|
||||
- name: bakery/sales-service
|
||||
newTag: latest
|
||||
- name: bakery/external-service
|
||||
newTag: latest
|
||||
- name: bakery/notification-service
|
||||
newTag: latest
|
||||
- name: bakery/inventory-service
|
||||
newTag: latest
|
||||
- name: bakery/recipes-service
|
||||
newTag: latest
|
||||
- name: bakery/suppliers-service
|
||||
newTag: latest
|
||||
- name: bakery/pos-service
|
||||
newTag: latest
|
||||
- name: bakery/orders-service
|
||||
newTag: latest
|
||||
- name: bakery/production-service
|
||||
newTag: latest
|
||||
- name: bakery/procurement-service
|
||||
newTag: latest
|
||||
- name: bakery/orchestrator-service
|
||||
newTag: latest
|
||||
- name: bakery/alert-processor
|
||||
newTag: latest
|
||||
- name: bakery/ai-insights-service
|
||||
newTag: latest
|
||||
- name: bakery/demo-session-service
|
||||
newTag: latest
|
||||
- name: bakery/gateway
|
||||
newTag: latest
|
||||
- name: bakery/dashboard
|
||||
newTag: latest
|
||||
- name: bakery/distribution-service
|
||||
newTag: latest
|
||||
@@ -1,11 +0,0 @@
|
||||
apiVersion: apiserver.config.k8s.io/v1
|
||||
kind: EncryptionConfiguration
|
||||
resources:
|
||||
- resources:
|
||||
- secrets
|
||||
providers:
|
||||
- aescbc:
|
||||
keys:
|
||||
- name: key1
|
||||
secret: 2eAEevJmGb+y0bPzYhc4qCpqUa3r5M5Kduch1b4olHE=
|
||||
- identity: {}
|
||||
@@ -1,699 +0,0 @@
|
||||
apiVersion: kustomize.config.k8s.io/v1beta1
|
||||
kind: Kustomization
|
||||
|
||||
metadata:
|
||||
name: bakery-ia-dev
|
||||
|
||||
# Note: Removed global namespace to prevent monitoring namespace conflict
|
||||
# All base resources already have namespace: bakery-ia defined
|
||||
|
||||
resources:
|
||||
- ../../base
|
||||
- dev-ingress.yaml
|
||||
# SigNoz is managed via Helm deployment (see Tiltfile signoz-deploy)
|
||||
# Monitoring is handled by SigNoz (no separate monitoring components needed)
|
||||
# Dev-Prod Parity: Enable HTTPS with self-signed certificates
|
||||
- dev-certificate.yaml
|
||||
# SigNoz paths are now included in the main ingress (ingress-https.yaml)
|
||||
|
||||
# Exclude nominatim from dev to save resources
|
||||
# Using scale to 0 for StatefulSet to prevent pod creation
|
||||
patches:
|
||||
# Override specific ConfigMap values for development
|
||||
- target:
|
||||
kind: ConfigMap
|
||||
name: bakery-config
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /data/ENVIRONMENT
|
||||
value: "development"
|
||||
- op: replace
|
||||
path: /data/DEBUG
|
||||
value: "true"
|
||||
- op: replace
|
||||
path: /data/LOG_LEVEL
|
||||
value: "DEBUG"
|
||||
- op: replace
|
||||
path: /data/AUTO_RELOAD
|
||||
value: "true"
|
||||
- op: replace
|
||||
path: /data/PROFILING_ENABLED
|
||||
value: "true"
|
||||
- op: replace
|
||||
path: /data/MOCK_EXTERNAL_APIS
|
||||
value: "false"
|
||||
- op: replace
|
||||
path: /data/TESTING
|
||||
value: "false"
|
||||
- op: replace
|
||||
path: /data/DOMAIN
|
||||
value: "localhost"
|
||||
- op: replace
|
||||
path: /data/API_DOCS_ENABLED
|
||||
value: "true"
|
||||
- op: replace
|
||||
path: /data/CORS_ORIGINS
|
||||
value: "http://frontend-service:3000,http://localhost:3000,http://localhost:3001,http://localhost,http://127.0.0.1:3000,http://127.0.0.1:3001,http://bakery-ia.local,https://localhost,https://127.0.0.1"
|
||||
- op: replace
|
||||
path: /data/VITE_ENVIRONMENT
|
||||
value: "development"
|
||||
- op: replace
|
||||
path: /data/VITE_API_URL
|
||||
value: "/api"
|
||||
- op: replace
|
||||
path: /data/STRIPE_PUBLISHABLE_KEY
|
||||
value: "pk_test_your_stripe_publishable_key_here"
|
||||
- op: replace
|
||||
path: /data/SQUARE_ENVIRONMENT
|
||||
value: "sandbox"
|
||||
- op: replace
|
||||
path: /data/TOAST_ENVIRONMENT
|
||||
value: "sandbox"
|
||||
- op: replace
|
||||
path: /data/LIGHTSPEED_ENVIRONMENT
|
||||
value: "sandbox"
|
||||
- op: replace
|
||||
path: /data/RATE_LIMIT_ENABLED
|
||||
value: "true" # Changed from false for dev-prod parity
|
||||
- op: add
|
||||
path: /data/RATE_LIMIT_PER_MINUTE
|
||||
value: "1000" # High limit for development (prod: 60)
|
||||
- op: replace
|
||||
path: /data/DB_FORCE_RECREATE
|
||||
value: "false"
|
||||
- op: add
|
||||
path: /data/DEVELOPMENT_MODE
|
||||
value: "true"
|
||||
- op: add
|
||||
path: /data/DEBUG_LOGGING
|
||||
value: "true"
|
||||
- op: add
|
||||
path: /data/SKIP_MIGRATION_VERSION_CHECK
|
||||
value: "false"
|
||||
- target:
|
||||
kind: StatefulSet
|
||||
name: nominatim
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/replicas
|
||||
value: 0
|
||||
# Suspend nominatim-init job in dev (not needed when nominatim is scaled to 0)
|
||||
- target:
|
||||
kind: Job
|
||||
name: nominatim-init
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/suspend
|
||||
value: true
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: auth-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: redis
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: rabbitmq
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "300m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: auth-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "50m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: frontend
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/imagePullPolicy
|
||||
value: Never
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "512Mi"
|
||||
cpu: "200m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "1000m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: gateway
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/imagePullPolicy
|
||||
value: Never
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: alert-processor
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
# Database patches
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: external-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: forecasting-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: inventory-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: notification-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: orders-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: pos-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: production-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: recipes-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: sales-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: suppliers-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: tenant-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: training-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: ai-insights-db
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
# Service patches
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: external-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: forecasting-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: inventory-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: notification-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: orders-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: pos-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: production-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: recipes-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: sales-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: suppliers-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: tenant-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "64Mi"
|
||||
cpu: "25m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "200m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: training-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "256Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "1Gi"
|
||||
cpu: "500m"
|
||||
- target:
|
||||
group: apps
|
||||
version: v1
|
||||
kind: Deployment
|
||||
name: ai-insights-service
|
||||
patch: |-
|
||||
- op: replace
|
||||
path: /spec/template/spec/containers/0/resources
|
||||
value:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "50m"
|
||||
limits:
|
||||
memory: "512Mi"
|
||||
cpu: "300m"
|
||||
|
||||
secretGenerator:
|
||||
- name: dev-secrets
|
||||
literals:
|
||||
- DEV_MODE=true
|
||||
|
||||
labels:
|
||||
- includeSelectors: true
|
||||
pairs:
|
||||
environment: development
|
||||
tier: local
|
||||
|
||||
images:
|
||||
- name: bakery/auth-service
|
||||
newTag: dev
|
||||
- name: bakery/tenant-service
|
||||
newTag: dev
|
||||
- name: bakery/training-service
|
||||
newTag: dev
|
||||
- name: bakery/forecasting-service
|
||||
newTag: dev
|
||||
- name: bakery/sales-service
|
||||
newTag: dev
|
||||
- name: bakery/external-service
|
||||
newTag: dev
|
||||
- name: bakery/notification-service
|
||||
newTag: dev
|
||||
- name: bakery/inventory-service
|
||||
newTag: dev
|
||||
- name: bakery/recipes-service
|
||||
newTag: dev
|
||||
- name: bakery/suppliers-service
|
||||
newTag: dev
|
||||
- name: bakery/pos-service
|
||||
newTag: dev
|
||||
- name: bakery/orders-service
|
||||
newTag: dev
|
||||
- name: bakery/production-service
|
||||
newTag: dev
|
||||
- name: bakery/alert-processor
|
||||
newTag: dev
|
||||
- name: bakery/ai-insights-service
|
||||
newTag: dev
|
||||
- name: bakery/demo-session-service
|
||||
newTag: dev
|
||||
- name: bakery/gateway
|
||||
newTag: dev
|
||||
- name: bakery/dashboard
|
||||
newTag: dev
|
||||
|
||||
replicas:
|
||||
# Dev-Prod Parity: Run 2 replicas of critical services
|
||||
# This helps catch load balancing, session management, and race condition issues
|
||||
- name: auth-service
|
||||
count: 2 # Increased from 1 for dev-prod parity
|
||||
- name: tenant-service
|
||||
count: 1
|
||||
- name: training-service
|
||||
count: 2 # Safe with MinIO storage
|
||||
- name: forecasting-service
|
||||
count: 1
|
||||
- name: sales-service
|
||||
count: 1
|
||||
- name: external-service
|
||||
count: 1
|
||||
- name: notification-service
|
||||
count: 1
|
||||
- name: inventory-service
|
||||
count: 1
|
||||
- name: recipes-service
|
||||
count: 1
|
||||
- name: suppliers-service
|
||||
count: 1
|
||||
- name: pos-service
|
||||
count: 1
|
||||
- name: orders-service
|
||||
count: 1
|
||||
- name: production-service
|
||||
count: 1
|
||||
- name: alert-processor
|
||||
count: 1
|
||||
- name: ai-insights-service
|
||||
count: 1
|
||||
- name: demo-session-service
|
||||
count: 1
|
||||
- name: gateway
|
||||
count: 2 # Increased from 1 for dev-prod parity
|
||||
- name: frontend
|
||||
count: 1
|
||||
@@ -1,74 +0,0 @@
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: bakery-ingress-prod
|
||||
labels:
|
||||
app.kubernetes.io/name: bakery-ia
|
||||
app.kubernetes.io/component: ingress
|
||||
annotations:
|
||||
# Nginx ingress controller annotations
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
|
||||
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
|
||||
nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
|
||||
nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
|
||||
|
||||
# CORS configuration for production
|
||||
nginx.ingress.kubernetes.io/enable-cors: "true"
|
||||
nginx.ingress.kubernetes.io/cors-allow-origin: "https://bakewise.ai"
|
||||
nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS, PATCH"
|
||||
nginx.ingress.kubernetes.io/cors-allow-headers: "Content-Type, Authorization, X-Requested-With, Accept, Origin"
|
||||
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
|
||||
|
||||
# Security headers
|
||||
nginx.ingress.kubernetes.io/configuration-snippet: |
|
||||
more_set_headers "X-Frame-Options: DENY";
|
||||
more_set_headers "X-Content-Type-Options: nosniff";
|
||||
more_set_headers "X-XSS-Protection: 1; mode=block";
|
||||
more_set_headers "Referrer-Policy: strict-origin-when-cross-origin";
|
||||
|
||||
# Rate limiting
|
||||
nginx.ingress.kubernetes.io/limit-rps: "100"
|
||||
nginx.ingress.kubernetes.io/limit-connections: "50"
|
||||
|
||||
# Cert-manager annotations for automatic certificate issuance
|
||||
cert-manager.io/cluster-issuer: "letsencrypt-production"
|
||||
cert-manager.io/acme-challenge-type: http01
|
||||
|
||||
spec:
|
||||
ingressClassName: nginx
|
||||
tls:
|
||||
- hosts:
|
||||
- bakewise.ai
|
||||
- monitoring.bakewise.ai
|
||||
secretName: bakery-ia-prod-tls-cert
|
||||
rules:
|
||||
- host: bakewise.ai
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: frontend-service
|
||||
port:
|
||||
number: 3000
|
||||
- path: /api/v1
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: gateway-service
|
||||
port:
|
||||
number: 8000
|
||||
# SigNoz Monitoring on subdomain (deployed via Helm in bakery-ia namespace)
|
||||
- host: monitoring.bakewise.ai
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: signoz
|
||||
port:
|
||||
number: 8080
|
||||
@@ -1,79 +0,0 @@
|
||||
# SigNoz Helm Chart Values - Customized for Bakery IA
|
||||
# https://github.com/SigNoz/charts
|
||||
|
||||
# Global settings
|
||||
global:
|
||||
storageClass: "standard"
|
||||
|
||||
# Frontend configuration
|
||||
frontend:
|
||||
service:
|
||||
type: ClusterIP
|
||||
port: 3301
|
||||
ingress:
|
||||
enabled: true
|
||||
hosts:
|
||||
- host: localhost
|
||||
paths:
|
||||
- path: /signoz
|
||||
pathType: Prefix
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /$2
|
||||
|
||||
# Query Service configuration
|
||||
queryService:
|
||||
replicaCount: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
limits:
|
||||
cpu: 200m
|
||||
memory: 512Mi
|
||||
|
||||
# AlertManager configuration
|
||||
alertmanager:
|
||||
replicaCount: 1
|
||||
resources:
|
||||
requests:
|
||||
cpu: 50m
|
||||
memory: 128Mi
|
||||
limits:
|
||||
cpu: 100m
|
||||
memory: 256Mi
|
||||
|
||||
# ClickHouse configuration
|
||||
clickhouse:
|
||||
persistence:
|
||||
enabled: true
|
||||
size: 10Gi
|
||||
resources:
|
||||
requests:
|
||||
cpu: 500m
|
||||
memory: 1Gi
|
||||
limits:
|
||||
cpu: 1000m
|
||||
memory: 2Gi
|
||||
|
||||
# OpenTelemetry Collector configuration
|
||||
otelCollector:
|
||||
enabled: true
|
||||
config:
|
||||
exporters:
|
||||
otlp:
|
||||
endpoint: "signoz-query-service:8080"
|
||||
service:
|
||||
pipelines:
|
||||
traces:
|
||||
receivers: [otlp]
|
||||
exporters: [otlp]
|
||||
metrics:
|
||||
receivers: [otlp]
|
||||
exporters: [otlp]
|
||||
logs:
|
||||
receivers: [otlp]
|
||||
exporters: [otlp]
|
||||
|
||||
# Resource optimization for development
|
||||
# These can be increased for production
|
||||
development: true
|
||||
@@ -349,18 +349,20 @@ podDisruptionBudget:
|
||||
## Monitoring and Alerting
|
||||
|
||||
### Email Alerts (Production)
|
||||
Configure SMTP in production values:
|
||||
Configure SMTP in production values (using Mailu with Mailgun relay):
|
||||
```yaml
|
||||
signoz:
|
||||
env:
|
||||
signoz_smtp_enabled: "true"
|
||||
signoz_smtp_host: "smtp.gmail.com"
|
||||
signoz_smtp_host: "mailu-smtp.bakery-ia.svc.cluster.local"
|
||||
signoz_smtp_port: "587"
|
||||
signoz_smtp_from: "alerts@bakewise.ai"
|
||||
signoz_smtp_username: "alerts@bakewise.ai"
|
||||
# Set via secret: signoz_smtp_password
|
||||
```
|
||||
|
||||
**Note**: Signoz now uses the internal Mailu SMTP service, which relays to Mailgun for better deliverability and centralized email management.
|
||||
|
||||
### Slack Alerts (Production)
|
||||
Configure webhook in Alertmanager:
|
||||
```yaml
|
||||
@@ -373,6 +375,69 @@ alertmanager:
|
||||
channel: '#alerts-critical'
|
||||
```
|
||||
|
||||
### Mailgun Integration for Alert Emails
|
||||
|
||||
Signoz has been configured to use Mailgun for sending alert emails through the Mailu SMTP service. This provides:
|
||||
|
||||
**Benefits:**
|
||||
- Better email deliverability through Mailgun's infrastructure
|
||||
- Centralized email management via Mailu
|
||||
- Improved tracking and analytics for alert emails
|
||||
- Compliance with email sending best practices
|
||||
|
||||
**Architecture:**
|
||||
```
|
||||
Signoz Alertmanager → Mailu SMTP → Mailgun Relay → Recipients
|
||||
```
|
||||
|
||||
**Configuration Requirements:**
|
||||
|
||||
1. **Mailu Configuration** (`infrastructure/platform/mail/mailu/mailu-configmap.yaml`):
|
||||
```yaml
|
||||
RELAYHOST: "smtp.mailgun.org:587"
|
||||
RELAY_LOGIN: "postmaster@bakewise.ai"
|
||||
```
|
||||
|
||||
2. **Mailu Secrets** (`infrastructure/platform/mail/mailu/mailu-secrets.yaml`):
|
||||
```yaml
|
||||
RELAY_PASSWORD: "<mailgun-api-key>" # Base64 encoded Mailgun API key
|
||||
```
|
||||
|
||||
3. **DNS Configuration** (required for Mailgun):
|
||||
```
|
||||
# MX record
|
||||
bakewise.ai. IN MX 10 mail.bakewise.ai.
|
||||
|
||||
# SPF record (authorize Mailgun)
|
||||
bakewise.ai. IN TXT "v=spf1 include:mailgun.org ~all"
|
||||
|
||||
# DKIM record (provided by Mailgun)
|
||||
m1._domainkey.bakewise.ai. IN TXT "v=DKIM1; k=rsa; p=<mailgun-public-key>"
|
||||
|
||||
# DMARC record
|
||||
_dmarc.bakewise.ai. IN TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc@bakewise.ai"
|
||||
```
|
||||
|
||||
4. **Signoz SMTP Configuration** (already configured in `signoz-values-prod.yaml`):
|
||||
```yaml
|
||||
signoz_smtp_host: "mailu-smtp.bakery-ia.svc.cluster.local"
|
||||
signoz_smtp_port: "587"
|
||||
signoz_smtp_from: "alerts@bakewise.ai"
|
||||
```
|
||||
|
||||
**Testing the Integration:**
|
||||
|
||||
1. Trigger a test alert from Signoz UI
|
||||
2. Check Mailu logs: `kubectl logs -f mailu-smtp-<pod-id> -n bakery-ia`
|
||||
3. Check Mailgun dashboard for delivery status
|
||||
4. Verify email receipt in destination inbox
|
||||
|
||||
**Troubleshooting:**
|
||||
|
||||
- **SMTP Authentication Failed**: Verify Mailu credentials and Mailgun API key
|
||||
- **Email Delivery Delays**: Check Mailu queue with `kubectl exec -it mailu-smtp-<pod-id> -n bakery-ia -- mailq`
|
||||
- **SPF/DKIM Issues**: Verify DNS records and Mailgun domain verification
|
||||
|
||||
### Self-Monitoring
|
||||
SigNoz monitors itself:
|
||||
```yaml
|
||||
@@ -71,9 +71,9 @@ signoz:
|
||||
# Only enable if you have a stable OpAMP backend server
|
||||
signoz_opamp_server_enabled: "false"
|
||||
# signoz_opamp_server_endpoint: "0.0.0.0:4320"
|
||||
# SMTP configuration for email alerts
|
||||
# SMTP configuration for email alerts - now using Mailu as SMTP server
|
||||
signoz_smtp_enabled: "true"
|
||||
signoz_smtp_host: "smtp.gmail.com"
|
||||
signoz_smtp_host: "email-smtp.bakery-ia.svc.cluster.local"
|
||||
signoz_smtp_port: "587"
|
||||
signoz_smtp_from: "alerts@bakewise.ai"
|
||||
signoz_smtp_username: "alerts@bakewise.ai"
|
||||
@@ -136,7 +136,7 @@ alertmanager:
|
||||
config:
|
||||
global:
|
||||
resolve_timeout: 5m
|
||||
smtp_smarthost: 'smtp.gmail.com:587'
|
||||
smtp_smarthost: 'email-smtp.bakery-ia.svc.cluster.local:587'
|
||||
smtp_from: 'alerts@bakewise.ai'
|
||||
smtp_auth_username: 'alerts@bakewise.ai'
|
||||
smtp_auth_password: '${SMTP_PASSWORD}'
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user