Issue 3563: Docker Compose Config changes and doc update (pravega#3565)

* Exposes the Controller REST API in Docker Compose deployments.
* Fixes the BookKeeper Container failures.
* Updates Docker Compose deployment documentation to provide detailed information about how to use it.

Signed-off-by: Ravi Sharda <[email protected]>

docker/compose/Readme.md

@@ -7,16 +7,52 @@ You may obtain a copy of the License at
 
     http://www.apache.org/licenses/LICENSE-2.0
 -->
-# docker-compose
+# Containerized Deployment Using Docker Compose
 
-Docker compose can be used to quickly spin up a Pravega cluster to use for testing or development. Unlike 
-`pravega-standalone`, a compose cluster will use a real standalone HDFS, Zookeeper, BookKeeper, and will run the 
-Segment Store and the Controller in separate processes.
+Docker Compose can be used to quickly spin up a Pravega cluster to use for testing or development. Unlike
+`pravega-standalone`, a Compose cluster will use a real standalone HDFS, Zookeeper and BookKeeper, and will also run
+Segment Store and Controller in separate processes.
 
 ## Running
 
-The IP or hostname must be provided as a `HOST_IP` environment variable. Otherwise, things should just work.
+1. Ensure that Docker and Docker Compose are installed in the host machine.
 
-`HOST_IP=1.2.3.4 docker-compose up`
+2. Clone Pravega repository to fetch the code.
+
+   ```
+   $ git clone https://github.com/pravega/pravega.git
+   ```
+
+3. Navigate to the directory containing Docker Compose configuration `.yml` files.
+
+   ```
+   $ cd /path/to/pravega/docker/compose
+   ```
+
+4. Add HOST_IP as an environment variable with the value as the IP address of the host.
+
+   ```
+   $ export HOST_IP=<HOST_IP>
+   ```
+
+5. Run the following command to start a deployment comprising of multiple Docker containers, as specified in the
+   `docker-compose.yml` file.
+
+   ```
+   $ docker-compose up -d
+   ```
+
+   To use one of the other files in the directory, use the `-f` option to specify the file.
+
+   ```
+   $ docker-compose up -d -f docker-compose-nfs.yml
+   ```
+
+6. Verify that the deployment is up and running.
+
+   ```
+   $ docker-compose ps
+   ```
+
+Clients can then connect to the Controller at `<HOST_IP>:9090`.
 
-Clients can then connect to the Controller at `${HOST_IP}:9090`.

docker/compose/docker-compose-extendeds3.yml

@@ -18,6 +18,7 @@ services:
     image: pravega/bookkeeper
     ports:
       - "3181:3181"
+    restart: always
     environment:
       ZK_URL: zookeeper:2181
       bookiePort: 3181
@@ -28,6 +29,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3182:3182"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3182
@@ -38,6 +40,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3183:3183"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3183
@@ -53,9 +56,9 @@ services:
     environment:
       WAIT_FOR: zookeeper:2181
       ZK_URL: zookeeper:2181
+      REST_SERVER_PORT: 10080
       JAVA_OPTS: |
         -Dcontroller.service.port=9090
-        -Dcontroller.service.restPort=10080
         -Dconfig.controller.metricenableCSVReporter=false
         -Xmx512m
         -XX:OnError="kill -9 p%"

docker/compose/docker-compose-nfs-mount.yml

@@ -18,6 +18,7 @@ services:
     image: pravega/bookkeeper
     ports:
       - "3181:3181"
+    restart: always
     environment:
       ZK_URL: zookeeper:2181
       bookiePort: 3181
@@ -28,6 +29,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3182:3182"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3182
@@ -38,6 +40,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3183:3183"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3183
@@ -53,6 +56,7 @@ services:
     environment:
       WAIT_FOR: zookeeper:2181
       ZK_URL: zookeeper:2181
+      REST_SERVER_PORT: 10080
       JAVA_OPTS: |
         -Dcontroller.service.port=9090
         -Dcontroller.service.restPort=10080

docker/compose/docker-compose-nfs.yml

@@ -25,6 +25,7 @@ services:
     image: pravega/bookkeeper
     ports:
       - "3181:3181"
+    restart: always
     environment:
       ZK_URL: zookeeper:2181
       bookiePort: 3181
@@ -35,6 +36,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3182:3182"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3182
@@ -45,6 +47,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3183:3183"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3183
@@ -60,9 +63,9 @@ services:
     environment:
       WAIT_FOR: zookeeper:2181
       ZK_URL: zookeeper:2181
+      REST_SERVER_PORT: 10080
       JAVA_OPTS: |
         -Dcontroller.service.port=9090
-        -Dcontroller.service.restPort=10080
         -Dconfig.controller.metricenableCSVReporter=false
         -Xmx512m
         -XX:OnError="kill -9 p%"

docker/compose/docker-compose.yml

@@ -32,6 +32,7 @@ services:
     image: pravega/bookkeeper
     ports:
       - "3181:3181"
+    restart: always
     environment:
       ZK_URL: zookeeper:2181
       bookiePort: 3181
@@ -42,6 +43,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3182:3182"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3182
@@ -52,6 +54,7 @@ services:
       image: pravega/bookkeeper
       ports:
         - "3183:3183"
+      restart: always
       environment:
         ZK_URL: zookeeper:2181
         bookiePort: 3183
@@ -67,9 +70,9 @@ services:
     environment:
       WAIT_FOR: zookeeper:2181
       ZK_URL: zookeeper:2181
+      REST_SERVER_PORT: 10080
       JAVA_OPTS: |
         -Dcontroller.service.port=9090
-        -Dcontroller.service.restPort=10080
         -Dconfig.controller.metricenableCSVReporter=false
         -Xmx512m
         -XX:OnError="kill -9 p%"

documentation/src/docs/deployment/run-local.md

@@ -60,23 +60,62 @@ docker run -it -e HOST_IP=<ip> -p 9090:9090 -p 12345:12345 pravega/pravega:lates
 
 ## Docker Compose (Distributed Mode)
 
-Unlike other options for running locally, the docker compose option runs a full deployment of Pravega
+Unlike other options for running locally, the Docker Compose option runs a full deployment of Pravega
 in distributed mode. It contains containers for running Zookeeper, Bookkeeper and HDFS. Hence Pravega operates as if it would in production. This is the easiest way to get started with the standalone option but requires additional resources.
 
-**Prerequisite:** Docker `1.12` or later.
+1. Ensure that your host machine meets the following prerequisites:
 
-Download the [docker-compose.yml](https://github.com/pravega/pravega/tree/master/docker/compose/docker-compose.yml) from github. For example:
+   * It has Docker `1.12` or later installed.
+   * It has Docker Compose installed.
 
-```
-wget https://raw.githubusercontent.com/pravega/pravega/master/docker/compose/docker-compose.yml
-```
+2. Download the [docker-compose.yml](https://github.com/pravega/pravega/tree/master/docker/compose/docker-compose.yml) file from [Pravega](https://github.com/pravega/pravega) GitHub repository.
+
+   ```
+   $ wget https://raw.githubusercontent.com/pravega/pravega/master/docker/compose/docker-compose.yml
+   ```
 
-We need to set the IP address of our local machine as the value of `HOST_IP` in the following command.
+   Alternatively, clone the Pravega repository to fetch the code.
+
+   ```
+   $ git clone https://github.com/pravega/pravega.git
+   ```
+
+3. Navigate to the directory containing Docker Compose configuration `.yml` files.
+
+   ```
+   $ cd /path/to/pravega/docker/compose
+   ```
+
+4. Add HOST_IP as an environment variable with the value as the IP address of the host.
+
+   ```
+   $ export HOST_IP=<HOST_IP>
+   ```
+
+5. Run the following command to start a deployment comprising of multiple Docker containers, as specified in the
+   `docker-compose.yml` file.
+
+   ```
+   $ docker-compose up -d
+   ```
+
+   To use one of the other files in the directory, use the `-f` option to specify the file.
+
+   ```
+   $ docker-compose up -d -f docker-compose-nfs.yml
+   ```
+
+6. Verify that the deployment is up and running.
+
+   ```
+   $ docker-compose ps
+   ```
+
+Clients can then connect to the Controller at `<HOST_IP>:9090`.
+
+To access the Pravega Controller `REST` API, invoke it using a URL of the form `http://<HOST_IP>:10080/v1/scopes` (where
+`/scopes` is one of the many endpoints that the API supports).
 
-```
-HOST_IP=1.2.3.4 docker-compose up
-```
-Clients can then connect to the controller at `${HOST_IP}:9090`.
 
 ## Running Pravega in Standalone Mode with SSL/TLS Enabled