feat(traffic-calls): Add Traffic Calls API and Database Seeding

[iacapuca]

🚦 Traffic Calls API - CTTU Recife Data Integration

This PR introduces a new traffic-calls service to the Atlas monorepo, providing API access to traffic incident data from CTTU (Autarquia de Trânsito e Transporte Urbano do Recife) covering 2016-2024.

---

📋 Summary

• ✅ New API service for traffic incident data management
• ✅ Comprehensive database schema with hybrid storage approach
• ✅ Automated CSV data seeding with S3 support
• ✅ Successfully imported ALL 20,239 records (fixed data validation issues)
• ✅ Full integration with Atlas monorepo infrastructure

---

🎯 Key Features

1. New Traffic Calls App

• Created using create-atlas-app CLI tool
• Port: 3020
• Database: atlas (shared database)
• Framework: Hono + Zod OpenAPI

2. Database Schema Design

Implemented a hybrid approach balancing performance with flexibility:

Indexed Columns (for fast queries):

• datetime - Timestamp of incident
• nature - Incident classification (COM VÍTIMA, VÍTIMA FATAL, SEM VÍTIMA)
• street_name - Main street address
• neighborhood - Neighborhood (bairro)
• total_victims, injured_victims, fatal_victims - Victim counts
• coordinates - Future PostGIS support

JSONB Fields (for flexible data):

• crash_data - Crash type, description, vehicle details
• environmental_data - Weather, traffic lights, signage, road conditions
• metadata - Original ID, protocol, status, verification info

3. Data Seeding Infrastructure

• CSV Upload: 5.3MB file uploaded to DigitalOcean Spaces
• Batch Processing: 100 records per batch for optimal performance
• Data Validation: Automatic date/time validation with fallback handling
• Results:
  • ✅ 20,239 records successfully imported
  • ✅ 0 records skipped
  • ✅ 0 errors

Data Quality Fixes:

Fixed two data validation issues that were causing 6,384 records to be skipped:

1. Malformed Date Fields: Some records had full ISO datetime in date field (e.g., 2023-01-01T00:00:00)
2. Non-Zero-Padded Time: Time values with single-digit hours (e.g., 6:05:00 instead of 06:05:00)

Solution: Added datetime normalization and extraction logic to handle these edge cases.

4. S3 Configuration

• Fixed S3 endpoint URLs in manifest to use bucket-specific endpoints
• Added fallback mechanism to load from local files if S3 fails
• Proper error tracking and reporting

5. Integration with Master Seed

Traffic Calls now appears in the seed summary alongside:

• 🚴 Cyclist Counts
• 👤 Cyclist Profiles
• 🚗 Traffic Deaths
• 🚦 Traffic Calls (NEW!)

---

📁 Files Changed

New Files:

• apps/traffic-calls/ - Complete API service
• packages/database/src/schemas/traffic-calls/ - Database schema
• packages/database/src/seed-traffic-calls.ts - Seeding script
• packages/database/src/migrations/0005_striped_agent_zero.sql - Migration
• packages/database/seed-data/traffic-calls/ - CSV data

Modified Files:

• packages/database/src/seed.ts - Added traffic-calls to master seed
• packages/database/seed-data/manifest.json - Added dataset metadata with correct S3 endpoints

---

🧪 Testing

• ✅ Unit tests with Vitest
• ✅ Database mocking for health checks
• ✅ OpenAPI spec generation configured
• ✅ Biome linting: All 183 files pass CI checks

---

📚 Documentation

• Comprehensive DATA_FORMAT.md with:
  • CSV structure analysis (43 columns)
  • Database schema rationale
  • JSONB field structures
  • Common query patterns
  • Example data transformations

---

🐳 Docker Support

• Dockerfile configured for deployment
• Health check endpoints
• Environment variable configuration
• Ready for Portainer deployment

---

🚀 Usage

Run the API:

pnpm --filter @atlas/traffic-calls dev

Seed the database:

# All seeds
pnpm --filter @atlas/database db:seed

# Traffic calls only
pnpm --filter @atlas/database db:seed --only=calls

# With S3 support
SEED_DATA_USE_S3=true pnpm --filter @atlas/database db:seed

Run tests:

pnpm --filter @atlas/traffic-calls test

---

📊 Data Source

• Provider: CTTU Recife
• Period: 2016-2024
• Records: 20,239 traffic incidents
• Format: CSV
• Storage: DigitalOcean Spaces + Local seed data

---

🔗 Related

• Part of the Atlas data integration initiative
• Complements existing traffic-deaths service

---

✅ Checklist

• Code follows project conventions
• Tests added and passing
• Documentation updated
• Database migration generated
• Seed data uploaded to S3
• OpenAPI spec generation configured
• Docker configuration added
• Integration with master seed completed
• S3 endpoint configuration fixed
• Data validation issues resolved
• All 20,239 records successfully imported
• Biome linting passed