The Simple Data Pipe is an app that loads data from cloud data sources into Cloudant, IBM's managed NoSQL database. Out of the box you can load data from salesforce.com and stripe.com. You can load data from other cloud data sources, such as Reddit using [custom connectors] (https://developer.ibm.com/clouddataservices/simple-data-pipe-connectors/).
To use the Simple Data Pipe, grab the code on Github, and then deploy it to IBM Bluemix, where it runs. It lives at a URL in Bluemix, and comes complete with an AngularJS UI for connecting, scheduling, and reporting.
To follow our full tutorial please visit : https://developer.ibm.com/clouddataservices/simple-data-pipe/
The fastest way to deploy this application to Bluemix is to click the Deploy to Bluemix button below. If you prefer instead to deploy manually to Bluemix then read the entirety of this section.
If you:
- don't have a Bluemix account, you'll be prompted to sign up. Create your account, verify your email address, then return here and click the the Deploy to Bluemix button again.
- have trouble deploying, make sure you haven't already reached your Bluemix memory and services quotas. You get 10 services with your account's free trial period. If you have questions about working in Bluemix, find answers in the Bluemix Docs.
Complete these steps first if you have not already:
- Install the Cloud Foundry command line interface.
- Follow the instructions at the above link to connect to Bluemix.
- Follow the instructions at the above link to log in to Bluemix.
Create a Cloudant service within Bluemix if one has not already been created:
$ cf create-service cloudantNoSQLDB Shared sdp-cloudant-service
Create a Single Sign On (SSO) service within Bluemix if one has not already been created:
$ cf create-service SingleSignOn standard pipes-sso-service
To deploy manually to Bluemix, simply:
$ git clone https://github.com/ibm-cds-labs/simple-data-pipe.git
$ cd simple-data-pipe
$ cf push
The Simple Data Pipe comes with a couple of data source connectors pre-installed. You can deploy additional connectors or build your own. Visit Connector Central to learn more.
Note: You may notice that Bluemix assigns a URL to your application containing a random word. This is defined in the manifest.yml file where the random-route key set to the value of true. This ensures that multiple people deploying this application to Bluemix do not run into naming collisions. To specify your own route, remove the random-route line from the manifest.yml file and add a host key with the unique value you would like to use for the host name.
This web application includes code to track deployments to IBM Bluemix and other Cloud Foundry platforms. The following information is sent to a Deployment Tracker service on each deployment:
- Application Name (
application_name) - Space ID (
space_id) - Application Version (
application_version) - Application URIs (
application_uris)
This data is collected from the VCAP_APPLICATION environment variable in IBM Bluemix and other Cloud Foundry platforms. This data is used by IBM to track metrics around deployments of sample applications to IBM Bluemix to measure the usefulness of our examples, so that we can continuously improve the content we offer to you. Only deployments of sample applications that include code to ping the Deployment Tracker service will be tracked.
Deployment tracking can be disabled by removing the following line from server.js:
require("cf-deployment-tracker-client").track();
Once that line is removed, you may also uninstall the cf-deployment-tracker-client npm package.
Copyright [2015-2016] [IBM Cloud Data Services]
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.