The IBMStreams/streamsx.dps GitHub repository is home to the Streams toolkit named DPS (Distributed Process Store). This Streams toolkit allows a simple way for the SPL, C++ and Java operators belonging to a single or multiple applications to share the application specific state information via an external K/V store. It does this via a collection of APIs that can be called from any part of the SPL, C++ and Java operator code.
Following are the external NoSQL K/V stores that can be configured to work with the DPS toolkit for the purpose of sharing application state in a distributed manner.
- Memcached
- Redis [version 2.x that doesn't have a built-in cluster feature]
- Cassandra
- IBM Cloudant
- HBase
- Mongo
- Couchbase
- Redis-Cluster [New cluster feature is available in Redis version 3 and above]
This toolkit evolved from the early research work done at the IBM.T.J.Watson Research Center, Yorktown Heights, New York. The links below highlight the origins of this asset.
For an API description see the GitHub pages for this toolkit. The documentation is also available locally in the doc folder, after building the toolkit. For configuration hints and other information, see the Wiki pages
Other things to consider when working with this toolkit:
To build the toolkit perform the following steps:
- Clone the repository
git clone https://github.com/IBMStreams/streamsx.dps.git - Build the toolkit
make clean all
Prerequisuites
you need to have the following rpm packages installed on the build machine:
curl
curl-devel
lua
lua-devel
openldap-devel
openssl-devel
cyrus-sasl
cyrus-sasl-devel