bakery-ia/docs/DATABASE_INITIALIZATION.md

Database Initialization System

This document explains the automatic database initialization system for the Bakery-IA microservices architecture.

Overview

The system handles two main scenarios:

1. Production/First-time Deployment: Automatically creates tables from SQLAlchemy models and sets up Alembic version tracking
2. Development Workflow: Provides easy reset capabilities to start with a clean slate

Key Features

• ✅ Automatic Table Creation: Creates tables from SQLAlchemy models when database is empty
• ✅ Alembic Integration: Properly manages migration versions and history
• ✅ Development Reset: Easy clean-slate restart for development
• ✅ Production Ready: Safe for production deployments
• ✅ All 14 Services: Works across all microservices

How It Works

1. Automatic Detection

The system automatically detects the database state:

• Empty Database: Creates tables from models and initializes Alembic
• Existing Database with Alembic: Runs pending migrations
• Existing Database without Alembic: Initializes Alembic on existing schema
• Force Recreate Mode: Drops everything and recreates (development only)

2. Integration Points

Service Startup

# In your service main.py
class AuthService(StandardFastAPIService):
    # Migration verification happens automatically during startup
    pass

Kubernetes Migration Jobs

# Enhanced migration jobs handle automatic table creation
containers:
- name: migrate
  image: bakery/auth-service:${IMAGE_TAG}
  command: ["python", "/app/scripts/run_migrations.py", "auth"]

Environment Variables

# Control behavior via environment variables
DB_FORCE_RECREATE=true    # Force recreate tables (development)
DEVELOPMENT_MODE=true     # Enable development features

Usage Scenarios

1. First-Time Production Deployment

What happens:

1. Migration job detects empty database
2. Creates all tables from SQLAlchemy models
3. Stamps Alembic with the latest migration version
4. Service starts and verifies migration state

No manual intervention required!

2. Development - Clean Slate Reset

Option A: Using the Development Script

# Reset specific service
./scripts/dev-reset-database.sh --service auth

# Reset all services
./scripts/dev-reset-database.sh --all

# Reset with auto-confirmation
./scripts/dev-reset-database.sh --service auth --yes

Option B: Using the Workflow Script

# Clean start with dev profile
./scripts/dev-workflow.sh clean --profile dev

# Reset specific service and restart
./scripts/dev-workflow.sh reset --service auth

Option C: Manual Environment Variable

# Set force recreate mode
kubectl patch configmap development-config -n bakery-ia \
  --patch='{"data":{"DB_FORCE_RECREATE":"true"}}'

# Run migration job
kubectl apply -f infrastructure/kubernetes/base/migrations/auth-migration-job.yaml

3. Regular Development Workflow

# Start development environment
./scripts/dev-workflow.sh start --profile minimal

# Check status
./scripts/dev-workflow.sh status

# View logs for specific service
./scripts/dev-workflow.sh logs --service auth

# Run migrations only
./scripts/dev-workflow.sh migrate --service auth

Configuration

Skaffold Profiles

The system supports different deployment profiles:

# skaffold.yaml profiles
profiles:
  - name: minimal          # Only auth and inventory
  - name: full            # All services + infrastructure
  - name: single          # Template for single service
  - name: dev             # Full development environment

Environment Variables

Variable	Description	Default
DB_FORCE_RECREATE	Force recreate tables	false
DEVELOPMENT_MODE	Enable development features	false
DEBUG_LOGGING	Enable debug logging	false
SKIP_MIGRATION_VERSION_CHECK	Skip version verification	false

Service Configuration

Each service automatically detects its configuration:

• Models Module: services.{service}.app.models
• Alembic Config: services/{service}/alembic.ini
• Migration Scripts: services/{service}/migrations/versions/

Development Workflows

Quick Start

# 1. Start minimal environment
./scripts/dev-workflow.sh start --profile minimal

# 2. Reset specific service when needed
./scripts/dev-workflow.sh reset --service auth

# 3. Clean restart when you want fresh start
./scripts/dev-workflow.sh clean --profile dev

Database Reset Workflows

Scenario 1: "I want to reset auth service only"

./scripts/dev-reset-database.sh --service auth

Scenario 2: "I want to start completely fresh"

./scripts/dev-reset-database.sh --all
# or
./scripts/dev-workflow.sh clean --profile dev

Scenario 3: "I want to reset and restart in one command"

./scripts/dev-workflow.sh reset --service auth

Technical Details

Database Initialization Manager

The core logic is in shared/database/init_manager.py:

# Main initialization method
async def initialize_database(self) -> Dict[str, Any]:
    # Check current database state
    db_state = await self._check_database_state()

    # Handle different scenarios
    if self.force_recreate:
        result = await self._handle_force_recreate()
    elif db_state["is_empty"]:
        result = await self._handle_first_time_deployment()
    # ... etc

Migration Job Enhancement

Migration jobs now use the enhanced runner:

containers:
- name: migrate
  command: ["python", "/app/scripts/run_migrations.py", "auth"]
  env:
  - name: AUTH_DATABASE_URL
    valueFrom:
      secretKeyRef:
        name: database-secrets
        key: AUTH_DATABASE_URL
  - name: DB_FORCE_RECREATE
    valueFrom:
      configMapKeyRef:
        name: development-config
        key: DB_FORCE_RECREATE

Service Integration

Services automatically handle table initialization during startup:

async def _handle_database_tables(self):
    # Check if we're in force recreate mode
    force_recreate = os.getenv("DB_FORCE_RECREATE", "false").lower() == "true"

    # Initialize database with automatic table creation
    result = await initialize_service_database(
        database_manager=self.database_manager,
        service_name=self.service_name,
        force_recreate=force_recreate
    )

Troubleshooting

Common Issues

1. Migration Job Fails

# Check job logs
kubectl logs -l job-name=auth-migration -n bakery-ia

# Check database connectivity
kubectl exec auth-db-pod -n bakery-ia -- pg_isready

2. Service Won't Start

# Check service logs
kubectl logs -l app.kubernetes.io/name=auth -n bakery-ia

# Check database state
./scripts/dev-workflow.sh status

3. Tables Not Created

# Force recreate mode
./scripts/dev-reset-database.sh --service auth --yes

# Check migration job status
kubectl get jobs -n bakery-ia

Debugging Commands

# Check all components
./scripts/dev-workflow.sh status

# View specific service logs
./scripts/dev-workflow.sh logs --service auth

# Check migration jobs
kubectl get jobs -l app.kubernetes.io/component=migration -n bakery-ia

# Check ConfigMaps
kubectl get configmaps -n bakery-ia

# View database pods
kubectl get pods -l app.kubernetes.io/component=database -n bakery-ia

Benefits

1. Zero Manual Setup: Tables are created automatically on first deployment
2. Development Friendly: Easy reset capabilities for clean development
3. Production Safe: Handles existing databases gracefully
4. Alembic Compatible: Maintains proper migration history and versioning
5. Service Agnostic: Works identically across all 14 microservices
6. Kubernetes Native: Integrates seamlessly with Kubernetes workflows

Migration from TODO State

If you have existing services with TODO migrations:

1. Keep existing models: Your SQLAlchemy models are the source of truth
2. Deploy normally: The system will create tables from models automatically
3. Alembic versions: Will be stamped with the latest migration version
4. No data loss: Existing data is preserved in production deployments

The system eliminates the need to manually fill in TODO migration files while maintaining proper Alembic version tracking.