bakery-admin/bakery-ia
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b420af32c5be2baa596445a6efc9ed623428c911
bakery-ia/TIMEZONE_AWARE_DATETIME_FIX.md

8.2 KiB
Raw Blame History

Timezone-Aware Datetime Fix

Date: 2025-10-09 Status: ✅ RESOLVED

Problem

Error in forecasting service logs:

[error] Failed to get cached prediction
error=can't compare offset-naive and offset-aware datetimes

Root Cause

The forecasting service database uses DateTime(timezone=True) for all timestamp columns, which means they store timezone-aware datetime objects. However, the code was using datetime.utcnow() throughout, which returns timezone-naive datetime objects.

When comparing these two types (e.g., checking if cache has expired), Python raises:

TypeError: can't compare offset-naive and offset-aware datetimes

Database Schema

All datetime columns in forecasting service models use DateTime(timezone=True):

# From app/models/predictions.py
class PredictionCache(Base):
    forecast_date = Column(DateTime(timezone=True), nullable=False)
    created_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
    expires_at = Column(DateTime(timezone=True), nullable=False)  # ← Compared with datetime.utcnow()
    # ... other columns

class ModelPerformanceMetric(Base):
    evaluation_date = Column(DateTime(timezone=True), nullable=False)
    created_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
    # ... other columns

# From app/models/forecasts.py
class Forecast(Base):
    forecast_date = Column(DateTime(timezone=True), nullable=False, index=True)
    created_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))

class PredictionBatch(Base):
    requested_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
    completed_at = Column(DateTime(timezone=True))

Solution

Replaced all datetime.utcnow() calls with datetime.now(timezone.utc) throughout the forecasting service.

Before (BROKEN):

# Returns timezone-naive datetime
cache_entry.expires_at < datetime.utcnow()  # ❌ TypeError!

After (WORKING):

# Returns timezone-aware datetime
cache_entry.expires_at < datetime.now(timezone.utc)  # ✅ Works!

Files Fixed

1. Import statements updated

Added timezone to imports in all affected files:

from datetime import datetime, timedelta, timezone

2. All datetime.utcnow() replaced

Fixed in 9 files across the forecasting service:

  1. services/forecasting/app/repositories/prediction_cache_repository.py

    • Line 53: Cache expiration time calculation
    • Line 105: Cache expiry check (the main error)
    • Line 175: Cleanup expired cache entries
    • Line 212: Cache statistics query

  2. services/forecasting/app/repositories/prediction_batch_repository.py

    • Lines 84, 113, 143, 184: Batch completion timestamps
    • Line 273: Recent activity queries
    • Line 318: Cleanup old batches
    • Line 357: Batch progress calculations

  3. services/forecasting/app/repositories/forecast_repository.py

    • Lines 162, 241: Forecast accuracy and trend analysis date ranges

  4. services/forecasting/app/repositories/performance_metric_repository.py

    • Line 101: Performance trends date range calculation

  5. services/forecasting/app/repositories/base.py

    • Lines 116, 118: Recent records queries
    • Lines 124, 159, 161: Cleanup and statistics

  6. services/forecasting/app/services/forecasting_service.py

    • Lines 292, 365, 393, 409, 447, 553: Processing time calculations and timestamps

  7. services/forecasting/app/api/forecasting_operations.py

    • Line 274: API response timestamps

  8. services/forecasting/app/api/scenario_operations.py

    • Lines 68, 134, 163: Scenario simulation timestamps

  9. services/forecasting/app/services/messaging.py

    • Message timestamps

Verification

# Before fix
$ grep -r "datetime\.utcnow()" services/forecasting/app --include="*.py" | wc -l
20

# After fix
$ grep -r "datetime\.utcnow()" services/forecasting/app --include="*.py" | wc -l
0

Why This Matters

Timezone-Naive (datetime.utcnow())

>>> datetime.utcnow()
datetime.datetime(2025, 10, 9, 9, 10, 37, 123456)  # No timezone info

Timezone-Aware (datetime.now(timezone.utc))

>>> datetime.now(timezone.utc)
datetime.datetime(2025, 10, 9, 9, 10, 37, 123456, tzinfo=datetime.timezone.utc)  # Has timezone

When PostgreSQL stores DateTime(timezone=True) columns, it stores them as timezone-aware. Comparing these with timezone-naive datetimes fails.

Impact

This fix resolves:

  • ✅ Cache expiration checks
  • ✅ Batch status updates
  • ✅ Performance metric queries
  • ✅ Forecast analytics date ranges
  • ✅ Cleanup operations
  • ✅ Recent activity queries

Best Practice

Always use timezone-aware datetimes with PostgreSQL DateTime(timezone=True) columns:

# ✅ GOOD
created_at = Column(DateTime(timezone=True), default=lambda: datetime.now(timezone.utc))
expires_at = datetime.now(timezone.utc) + timedelta(hours=24)
if record.created_at < datetime.now(timezone.utc):
    ...

# ❌ BAD
created_at = Column(DateTime(timezone=True), default=datetime.utcnow)  # No timezone!
expires_at = datetime.utcnow() + timedelta(hours=24)  # Naive!
if record.created_at < datetime.utcnow():  # TypeError!
    ...

Additional Issue Found and Fixed

Local Import Shadowing

After the initial fix, a new error appeared:

[error] Multi-day forecast generation failed
error=cannot access local variable 'timezone' where it is not associated with a value

Cause: In forecasting_service.py line 428, there was a local import inside a conditional block that shadowed the module-level import:

# Module level (line 9)
from datetime import datetime, date, timedelta, timezone

# Inside function (line 428) - PROBLEM
if day_offset > 0:
    from datetime import timedelta, timezone  # ← Creates LOCAL variable
    current_date = current_date + timedelta(days=day_offset)

# Later in same function (line 447)
processing_time = (datetime.now(timezone.utc) - start_time)  # ← Error! timezone not accessible

When Python sees the local import on line 428, it creates a local variable timezone that only exists within that if block. When line 447 tries to use timezone.utc, Python looks for the local variable but can't find it (it's out of scope), resulting in: "cannot access local variable 'timezone' where it is not associated with a value".

Fix: Removed the redundant local import since timezone is already imported at module level:

# Before (BROKEN)
if day_offset > 0:
    from datetime import timedelta, timezone
    current_date = current_date + timedelta(days=day_offset)

# After (WORKING)
if day_offset > 0:
    current_date = current_date + timedelta(days=day_offset)

File: services/forecasting/app/services/forecasting_service.py

Deployment

# Restart forecasting service to apply changes
kubectl -n bakery-ia rollout restart deployment forecasting-service

# Monitor for errors
kubectl -n bakery-ia logs -f deployment/forecasting-service | grep -E "(can't compare|cannot access)"

Related Issues

This same issue may exist in other services. Search for:

# Find services using timezone-aware columns
grep -r "DateTime(timezone=True)" services/*/app/models --include="*.py"

# Find services using datetime.utcnow()
grep -r "datetime\.utcnow()" services/*/app --include="*.py"

References