diff --git a/.gitattributes b/.gitattributes deleted file mode 100644 index e69de29bb..000000000 diff --git a/docs/quick-start.md b/docs/quick-start.md index 28ecf7b8c..73c9e8fd6 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -36,14 +36,14 @@ The second command will download and run a prebuilt AsterixDB docker container f After it finishes, you should see the messages as shown in the following screenshot: ![docker][docker] -**Step 4**: Run the following command to ingest sample tweets (about 324K) and US population data into AsterixDB. +**Step 4**: Run the following command to ingest sample tweets (about 47K) and US population data into AsterixDB. ``` ~/cloudberry> ./script/ingestAllTwitterToLocalCluster.sh ``` -This step is downloading about 70MB of data, and it may take 5 minutes, again, depending on your network speed. You should see the messages as shown in the following screenshot: +When it finishes you should see the messages as shown in the following screenshot: ![ingestion][ingestion] **Step 5**: Compile and run the Cloudberry server. @@ -80,9 +80,16 @@ The instructions above assume that we use an AsterixDB instance in a Docker cont **Step 8**: Follow the instructions on the [AsterixDB Installation Guide](https://ci.apache.org/projects/asterixdb/index.html) to install an AsterixDB cluster. Select your preferred installation option. -**Step 9**: Ingest twitter data. +**Step 9**: Ingest twitter data to AsterixDB -**Step 10**: Change the Cloudberry middleware configuration to connect to this new AsterixDB cluster. You can modify the AsterixDB hostname in the configuration file `neo/conf/application.conf` and change the `asterixdb.url` value to the AsterixDB hostname. +You need to give the RESTFul API link of the AsterixDB cluster and one of its NC names to the ingestion script as following: + +``` +~/cloudberry> ./script/ingestAllTwitterToLocalCluster.sh http://YourAsterixDBServerIP:19002/aql ONE_OF_NC_NAMES +``` + +**Step 10**: Change the Cloudberry middleware configuration to connect to this new AsterixDB cluster. +You can modify the AsterixDB hostname in the configuration file `neo/conf/application.conf` by changing the `asterixdb.url` value. ``` asterixdb.url = "http://YourAsterixDBHostName:19002/query/service" @@ -90,7 +97,7 @@ asterixdb.url = "http://YourAsterixDBHostName:19002/query/service" ## Build your own application -For more information about Cloudberry, please read its [documentation](/documentation). +TwitterMap is one example of how to use Cloudberry. To develop your own application, please find more information in [documentation](/documentation). [architecture]: /img/quick-start-architecture.png {: width="800px"} diff --git a/script/.gitattributes b/script/.gitattributes deleted file mode 100644 index e69de29bb..000000000 diff --git a/script/dockerClean.sh b/script/dockerClean.sh index a493aa905..e32d1552e 100755 --- a/script/dockerClean.sh +++ b/script/dockerClean.sh @@ -1,6 +1,6 @@ #!/usr/bin/env bash #clean up the existing images -docker stop -f cc nc1 +docker stop cc nc1 docker rm -f cc nc1 docker volume rm dbstore # remove the local image to fetch the newest remote version diff --git a/script/fileFeed.sh b/script/fileFeed.sh index 96e095b83..881372981 100755 --- a/script/fileFeed.sh +++ b/script/fileFeed.sh @@ -1,3 +1,7 @@ #!/usr/bin/env bash +link=${1-"localhost"} +host=$(basename $(dirname $link)) +host=${host%%:*} +port=${2-"10001"} sbt "project noah" --error "run-main edu.uci.ics.cloudberry.noah.feed.FileFeedDriver \ --u localhost -p ${1:-10001}" +-u $host -p $port" diff --git a/script/ingestAllTwitterToLocalCluster.sh b/script/ingestAllTwitterToLocalCluster.sh index 2de18152a..6d150644f 100755 --- a/script/ingestAllTwitterToLocalCluster.sh +++ b/script/ingestAllTwitterToLocalCluster.sh @@ -18,10 +18,11 @@ #=============================================================================== host=${1:-'http://localhost:19002/aql'} +nc=${2:-"nc1"} echo "Ingesting sample tweets..." -./script/ingestTwitterToLocalCluster.sh $host +./script/ingestTwitterToLocalCluster.sh $host $nc echo "Ingesting population data..." -./script/ingestPopulationToLocalCluster.sh $host +./script/ingestPopulationToLocalCluster.sh $host $nc echo "Data ingestion completed!" \ No newline at end of file diff --git a/script/ingestPopulationToLocalCluster.sh b/script/ingestPopulationToLocalCluster.sh index 17c0b7df5..cc2fa6727 100755 --- a/script/ingestPopulationToLocalCluster.sh +++ b/script/ingestPopulationToLocalCluster.sh @@ -20,8 +20,9 @@ set -o nounset # Treat unset variables as an error host=${1:-'http://localhost:19002/aql'} +nc=${2:-"nc1"} # ddl to register the twitter dataset -cat <<'EOF' | curl -XPOST --data-binary @- $host +cat <