Skip to content

Commit

Permalink
Merge pull request #1206 from RumbleDB/Version1.20
Browse files Browse the repository at this point in the history
Version1.20
  • Loading branch information
ghislainfourny authored Nov 7, 2022
2 parents 5aa2c6a + 894fb80 commit 38e07ca
Show file tree
Hide file tree
Showing 10 changed files with 32 additions and 32 deletions.
4 changes: 2 additions & 2 deletions docs/Docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Known issue

On occasion, the docker version of RumbleDB used to throw a Kryo NoSuchMethodError on some systems. This should be fixed with version 1.19.0, let us know if this is not the case.
On occasion, the docker version of RumbleDB used to throw a Kryo NoSuchMethodError on some systems. This should be fixed with version 1.20.0, let us know if this is not the case.

You can upgrade to the newest version with

Expand All @@ -29,7 +29,7 @@ The RumbleDB shell appears:
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.19.0 "Tipuana Tipu" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.20.0 "Honeylocust" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/


Expand Down
14 changes: 7 additions & 7 deletions docs/Getting started.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,17 @@ Do make sure it is not Java 17, which will not work.

RumbleDB is just a download and no installation is required.

In order to run RumbleDB, you simply need to download rumbledb-1.19.0-standalone.jar from the [download page](https://github.com/RumbleDB/rumble/releases) and put it in a directory of your choice, for example, right besides your data.
In order to run RumbleDB, you simply need to download rumbledb-1.20.0-standalone.jar from the [download page](https://github.com/RumbleDB/rumble/releases) and put it in a directory of your choice, for example, right besides your data.

Make sure to use the corresponding jar name accordingly in all our instructions in lieu of rumbledb.jar.

You can test that it works with:

java -jar rumbledb-1.19.0-standalone.jar run -q '1+1'
java -jar rumbledb-1.20.0-standalone.jar run -q '1+1'

or launch a JSONiq shell with:

java -jar rumbledb-1.19.0-standalone.jar repl
java -jar rumbledb-1.20.0-standalone.jar repl

If you run out of memory, you can set allocate more memory to Java with an additional Java parameter, e.g., -Xmx10g

Expand Down Expand Up @@ -113,11 +113,11 @@ Like Spark, RumbleDB is just a download and no installation is required.

In order to run RumbleDB, you simply need to download one of the small .jar files from the [download page](https://github.com/RumbleDB/rumble/releases) and put it in a directory of your choice, for example, right besides your data.

If you use Spark 3.0+, use rumbledb-1.19.0-for-spark-3.0.jar.
If you use Spark 3.0+, use rumbledb-1.20.0-for-spark-3.0.jar.

If you use Spark 3.1+, use rumbledb-1.19.0-for-spark-3.1.jar.
If you use Spark 3.1+, use rumbledb-1.20.0-for-spark-3.1.jar.

If you use Spark 3.2+, use rumbledb-1.19.0-for-spark-3.2.jar.
If you use Spark 3.2+, use rumbledb-1.20.0-for-spark-3.2.jar.

These jars do not embed Spark, since you chose to set it up separately. They will work with your Spark installation with the spark-submit command.

Expand Down Expand Up @@ -162,7 +162,7 @@ The RumbleDB shell appears:
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.19.0 "Tipuana Tipu" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.20.0 "Honeylocust" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/


Expand Down
8 changes: 4 additions & 4 deletions docs/HTTPServer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

RumbleDB can be run as an HTTP server that listens for queries. In order to do so, you can use the --server and --port parameters:

spark-submit rumbledb-1.19.0.jar serve -p 8001
spark-submit rumbledb-1.20.0.jar serve -p 8001

This command will not return until you force it to (Ctrl+C on Linux and Mac). This is because the server has to run permanently to listen to incoming requests.

Expand Down Expand Up @@ -69,19 +69,19 @@ Then there are two options
- Connect to the master with SSH with an extra parameter for securely tunneling the HTTP connection (for example `-L 8001:localhost:8001` or any port of your choosing)
- Download the RumbleDB jar to the master node

wget https://github.com/RumbleDB/rumble/releases/download/v1.19.0/rumbledb-1.19.0.jar
wget https://github.com/RumbleDB/rumble/releases/download/v1.20.0/rumbledb-1.20.0.jar

- Launch the HTTP server on the master node (it will be accessible under `http://localhost:8001/jsoniq`).

spark-submit rumbledb-1.19.0.jar serve -p 8001
spark-submit rumbledb-1.20.0.jar serve -p 8001

- And then use Jupyter notebooks in the same way you would do it locally (it magically works because of the tunneling)

### With the EC2 hostname

There is also another way that does not need any tunnelling: you can specify the hostname of your EC2 machine (copied over from the EC2 dashboard) with the --host parameter. For example, with the placeholder <ec2-hostname>:

spark-submit rumbledb-1.19.0.jar serve -p 8001 -h <ec2-hostname>
spark-submit rumbledb-1.20.0.jar serve -p 8001 -h <ec2-hostname>

You also need to make sure in your EMR security group that the chosen port (e.g., 8001) is accessible from the machine in which you run your Jupyter notebook. Then, you can point your Jupyter notebook on this machine to `http://<ec2-hostname>:8001/jsoniq`.

Expand Down
4 changes: 2 additions & 2 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ After successful completion, you can check the `target` directory, which should

The most straightforward to test if the above steps were successful is to run the RumbleDB shell locally, like so:

$ spark-submit target/rumbledb-1.19.0.jar repl
$ spark-submit target/rumbledb-1.20.0.jar repl

The RumbleDB shell should start:

Expand All @@ -73,7 +73,7 @@ The RumbleDB shell should start:
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.19.0 "Tipuana Tipu" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.20.0 "Honeylocust" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/

Master: local[2]
Expand Down
2 changes: 1 addition & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
site_name: RumbleDB 1.19 "Tipuana Tipu" beta
site_name: RumbleDB 1.20 "Honeylocust" beta
nav:
- '1. Documentation home': 'index.md'
- '2. Getting started': 'Getting started.md'
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@

<groupId>com.github.rumbledb</groupId>
<artifactId>rumbledb</artifactId>
<version>1.19.0</version>
<version>1.20.0</version>
<packaging>jar</packaging>
<name>RumbleDB</name>
<description>A JSONiq engine to query large-scale JSON datasets stored on HDFS. Spark under the hood.</description>
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/assets/banner.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
____ __ __ ____ ____
/ __ \__ ______ ___ / /_ / /__ / __ \/ __ )
/ /_/ / / / / __ `__ \/ __ \/ / _ \/ / / / __ | The distributed JSONiq engine
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.19.0 "Tipuana Tipu" beta
/ _, _/ /_/ / / / / / / /_/ / / __/ /_/ / /_/ / 1.20.0 "Honeylocust" beta
/_/ |_|\__,_/_/ /_/ /_/_.___/_/\___/_____/_____/

24 changes: 12 additions & 12 deletions src/main/resources/assets/defaultscreen.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,46 +17,46 @@ for directly running a query from an input file or (with -q) provided directly o

It is the default mode.

spark-submit rumbledb-1.19.0.jar run my-query.jq
spark-submit rumbledb-1.19.0.jar run -q '1+1'
spark-submit rumbledb-1.20.0.jar run my-query.jq
spark-submit rumbledb-1.20.0.jar run -q '1+1'

You can specify an output path with -o like so:
spark-submit rumbledb-1.19.0.jar run -q '1+1' -o my-output.txt
spark-submit rumbledb-1.20.0.jar run -q '1+1' -o my-output.txt

**** serve ****
for running as an HTTP server listening on the specified port (-p) and host (-h).

spark-submit rumbledb-1.19.0.jar serve -p 9090
spark-submit rumbledb-1.20.0.jar serve -p 9090

RumbleDB also supports Apache Livy for use in Jupyter notebooks, which may be
even more convenient if you are using a cluster.

**** repl ****
for shell mode.

spark-submit rumbledb-1.19.0.jar repl
spark-submit rumbledb-1.20.0.jar repl


**** resource use configuration ****

For a local use, you can control the number of cores, as well as allocated
memory, with:
spark-submit --master local[*] rumbledb-1.19.0.jar repl
spark-submit --master local[*] rumbledb-1.19.0.jar repl
spark-submit --master local[2] rumbledb-1.19.0.jar repl
spark-submit --master local[*] --driver-memory 10G rumbledb-1.19.0.jar repl
spark-submit --master local[*] rumbledb-1.20.0.jar repl
spark-submit --master local[*] rumbledb-1.20.0.jar repl
spark-submit --master local[2] rumbledb-1.20.0.jar repl
spark-submit --master local[*] --driver-memory 10G rumbledb-1.20.0.jar repl

You can use RumbleDB remotely with:
spark-submit --master yarn rumbledb-1.19.0.jar repl
spark-submit --master yarn rumbledb-1.20.0.jar repl

(Although for clusters provided as a service, --master yarn is often implicit
and unnecessary).

For remote use (e.g., logged in on the Spark cluster with ssh), you can set the
number of executors, cores and memory, you can use:
spark-submit --executor-cores 3 --executor-memory 5G rumbledb-1.19.0.jar repl
spark-submit --executor-cores 3 --executor-memory 5G rumbledb-1.20.0.jar repl

For remote use, you can also use other file system paths such as S3, HDFS, etc:
spark-submit rumbledb-1.19.0.jar run hdfs://server:port/my-query.jq -o hdfs://server:port/my-output.json
spark-submit rumbledb-1.20.0.jar run hdfs://server:port/my-query.jq -o hdfs://server:port/my-output.json

More documentation on available CLI parameters is available on https://www.rumbledb.org/
2 changes: 1 addition & 1 deletion src/main/resources/assets/jsound-validator.html
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ <h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif
</span>
</div>
<div>
<button id="submit" style="margin: 0 auto; padding: 10px;">Validate with RumbleDB 1.19.0</button>
<button id="submit" style="margin: 0 auto; padding: 10px;">Validate with RumbleDB 1.20.0</button>
</div>

<h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif; color: #336699;">Results</h2>
Expand Down
2 changes: 1 addition & 1 deletion src/main/resources/assets/public.html
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ <h2 style="font-family: liberation sans, Myriad, Bitstream Vera Sans, sans-serif
</span>
</div>
<div>
<button id="submit" style="margin: 20px; padding: 10px;">Evaluate with RumbleDB 1.19.0</button>
<button id="submit" style="margin: 20px; padding: 10px;">Evaluate with RumbleDB 1.20.0</button>
</div>

</div>
Expand Down

0 comments on commit 38e07ca

Please sign in to comment.