AsterixDB Pump

Description:

This tool retrieves data from a source AsterixDB, and ingest into a destination AsterixDB. For now it is builtin supporting twitter.ds_tweet.

Assumptions:

You have already have twitter.ds_tweet setup on both source and destination AsterixDB. Also TweetFeed should be enabled on the destination side.

Requirements:

[email protected]+

Install dependencies:

pip install -r requirements.txt

Config

modify config/pump_conf.ini:

change source and destination db ip
change your start_time and end_time
indicate your running mode: incremental or one-time
specify your sql

Run:

python3 asterixdb_pump.py