feature(examples): Add Aftership connector

[fivertran-karunveluru]

AfterShip Connector

Created: 2025-10-31

Business Owner: Shipping & Logistics Operations Team

Technical Owner: Data Engineering Team

Last Updated: 2025-10-31

Business Context

• Data Source: AfterShip API for package tracking, courier information, and shipment analytics
• Business Criticality: High - supports customer service, delivery optimization, and logistics analytics
• Data Consumers: Customer service teams, logistics coordinators, operations managers, supply chain analysts, executive leadership
• Business SLAs: Tracking data must be fresh within 1 hour for customer service, 4 hours for analytics reporting
• Compliance Requirements: GDPR compliance for customer tracking data, shipping carrier data agreements
• Budget Constraints: AfterShip API access based on subscription tier, rate limits vary by plan

Technical Context

• API Documentation: https://docs.aftership.com/
• Authentication Method: API key authentication via aftership-api-key header
• Rate Limits: Varies by AfterShip plan, typically 1000-10000 requests/hour depending on subscription tier
• Data Volume:
  • Trackings: 100-100,000+ tracking records per month
  • Couriers: 50-500+ supported carriers
  • Checkpoints: 2-20+ checkpoint events per tracking
  • Active trackings: 5,000-500,000+ active shipments at any time
• Data Velocity: Tracking updates occur in real-time as packages move, courier data updated monthly, checkpoint data updated hourly
• Data Quality: Structured JSON with consistent schema, some fields may be null for incomplete tracking records
• Network Considerations: HTTPS only, RESTful API with standard reliability, global CDN infrastructure

Operational Context

• Deployment Environment: Development (sandbox), staging, and production environments
• Monitoring Requirements: Alert on >2% error rate, >1 hour sync time, tracking data discrepancies
• Maintenance Windows: Weekends for non-critical updates, immediate deployment for tracking-critical fixes
• Team Structure: Data Engineering team, Logistics Operations, Customer Service, Operations managers
• Escalation Path: Data Engineer → Team Lead → Logistics Director → VP Operations

API-Specific Details

• Base Endpoint: https://api.aftership.com/tracking/2025-07
• Authentication: API key in aftership-api-key header (API key authentication)
• Pagination: page and limit parameters (max 200 per page, default 100)
• Date Format: ISO 8601 (e.g., 2024-01-15T10:30:00Z)
• Response Format: JSON with nested objects and arrays, including checkpoint arrays
• Key Endpoints:
  • /trackings - List and retrieve package tracking information
  • /couriers - List supported shipping carriers and their capabilities
  • /trackings/{tracking_number} - Get specific tracking details and checkpoints
  • Webhook endpoints for real-time tracking updates (future enhancement)

Data Schema Overview

• trackings: Package tracking information including status, checkpoints, delivery details, and shipment metadata
• couriers: Shipping carrier information including capabilities, contact details, and configuration options

Data Replication Expectations

• Initial Sync: Last 30-90 days of tracking data for historical baseline (configurable via initial_sync_days)
• Incremental Sync: Data since last successful sync timestamp (last_updated_at field)
• Sync Frequency:
  • Production: Every 1 hour for tracking data updates, daily for courier data
  • Development: Every 4 hours for tracking data, daily for couriers
• Data Retention: 2-5 years of historical tracking data for analytics and compliance
• Backfill Capability: Full historical data available based on AfterShip account retention policies
• Data Consistency: Near real-time with 1-hour maximum lag for customer service operations

Operational Requirements

• Uptime SLA: 99.5% availability during business hours (tracking status critical for customer service)
• Performance SLA:
  • Initial sync: <2 hours for 30 days of tracking data
  • Incremental sync: <30 minutes for hourly updates
• Error Handling:
  • Automatic retry with exponential backoff
  • Dead letter queue for failed tracking records
  • Alert on consecutive sync failures during peak shipping periods
• Monitoring:
  • API response times and error rates
  • Tracking count trends and anomaly detection
  • Checkpoint data completeness validation
  • Courier sync status and coverage
• Security:
  • API keys rotated every 90 days
  • Access logs maintained for 1 year (compliance)
  • Customer PII handling per privacy regulations

Rate Limiting Strategy

• Starter Plan: 1,000 requests/hour, 10,000 requests/day
• Professional Plan: 5,000 requests/hour, 50,000 requests/day
• Enterprise Plan: 10,000+ requests/hour, 100,000+ requests/day (custom limits)
• Recommended: Implement exponential backoff with jitter for 429 responses, respect Retry-After header
• Error Handling: 429 status code indicates rate limit exceeded, respect Retry-After header when provided
• Monitoring: Track rate limit utilization and plan for subscription upgrades based on volume growth

Data Quality Considerations

• Required Fields: id, tracking_number, slug, tag, last_updated_at
• Optional Fields: customer_name, order_id, expected_delivery, checkpoints, custom_fields
• Data Validation:
  • Tracking IDs must be unique
  • Tracking numbers must match carrier format when available
  • Courier slugs must match supported carrier list
  • Timestamps must be valid ISO 8601 format
  • Checkpoint arrays must be valid JSON
• Data Completeness:
  • Trackings: 95%+ have basic tracking information
  • Checkpoints: 80%+ have at least one checkpoint event
  • Couriers: 100% have basic carrier information
• Duplicate Handling: Primary key constraints prevent duplicate tracking records (by id)

Integration Points

• Fivetran Destinations: Snowflake, BigQuery, Redshift, PostgreSQL
• Downstream Systems:
  • Customer service platforms (for tracking inquiries)
  • Logistics management systems
  • Analytics and BI platforms
  • E-commerce platforms (for order tracking)
  • Warehouse management systems
  • Supply chain visibility tools
• Data Dependencies: None - standalone package tracking data source
• External Dependencies: AfterShip API availability, shipping carrier API responses

Disaster Recovery

• Backup Strategy: Daily snapshots of all tracking and courier tables
• Recovery Time Objective: 2 hours for full data recovery
• Recovery Point Objective: 1 hour maximum data loss for active tracking data
• Failover: Automatic failover to backup API keys
• Testing: Monthly disaster recovery drills with operations team validation

Compliance & Security

• Data Classification: Customer tracking data - moderately sensitive, courier data - public information
• Retention Policy: 2-5 years for tracking data (analytics and compliance), 1 year for operational data
• Access Controls: Strict role-based access with principle of least privilege
• Audit Trail: All data access logged and monitored for compliance audits
• Encryption: Data encrypted in transit and at rest with enterprise-grade security
• Privacy: GDPR compliance for EU customer data, CCPA compliance for CA customer data

Performance Optimization

• Parallel Processing: Multiple API calls for different couriers and tracking statuses
• Caching: Courier data cached for 24 hours (rarely changes)
• Indexing: Tracking ID, tracking number, order ID, courier slug, and timestamp columns indexed
• Partitioning: Tracking data partitioned by date and courier for efficient querying
• Compression: Historical tracking data compressed for storage efficiency
• Streaming: Memory-efficient generator-based processing prevents data accumulation

Troubleshooting Guide

• Common Issues:
  • Rate limit exceeded: Reduce sync frequency or upgrade AfterShip plan
  • API key expired/invalid: Verify API key in AfterShip admin portal and permissions
  • Missing tracking data: Check tracking number format and carrier slug compatibility
  • Timeout errors: Increase timeout values or reduce batch size (max_records_per_page)
  • Checkpoint data missing: Verify tracking is active and carrier supports detailed tracking
  • Pagination errors: Check page/limit parameters and total_pages metadata
• Debug Mode: Enable detailed logging via enable_debug_logging configuration for tracking and API troubleshooting
• Support Contacts:
  • Technical: Data Engineering team
  • Business: Logistics Operations team
  • Vendor: AfterShip support (for API and account issues)
  • Compliance: Legal/Compliance team (for privacy and regulatory issues)

Checklist

Some tips and links to help validate your PR:

• Tested the connector with fivetran debug command.
• Added/Updated example specific README.md file, refer here for template.
• Followed Python Coding Standards, refer here

[github-actions]

🧹 Python Code Quality Check

✅ No issues found in Python Files.

🔍 See how this check works

This comment is auto-updated with every commit.