TESTING.md

Testing JanusGraph

Audience of this Document

This page is written for developers familiar with Java, JanusGraph, and Maven who want information on how to run JanusGraph's test suite.

Overview

JanusGraph runs all tests using JUnit. To compile, package, and run the default test suite for JanusGraph, use the standard mvn clean install command.

JanusGraph has a specialty tests, disabled by default, intended to generate basic performance metrics or stress its cache structures under memory pressure. The next section describes how JanusGraph's tests are internally categorized and the Maven options that enabled/disable test categories.

Continuous Integration

JanusGraph runs continuous integration via Travis; see the dashboard for current status.

Travis sends emails on test failures and status transitions (to/from failure) to janusgraph-ci@googlegroups.com mailing list.

JUnit Test Categories

All of JanusGraph's tests are written for JUnit. JanusGraph's JUnit tests are annotated with the following JUnit Categories:

Category Name	Maven Property	Default	Comment
MemoryTests	test.skip.mem	true (disabled)	Tests intended to exert memory pressure
PerformanceTests	test.skip.perf	true (disabled)	Tests written as simple speed tests using JUnitBenchmarks
OrderedKeyStoreTests	test.skip.ordered	false (enabled)	Tests written for a storage backend that stores data in key order
UnorderedKeyStoreTests	test.skip.unordered	false (enabled)	Tests written for a storage backend that doesn't store data in key order
(No category)	test.skip.default	false (enabled)	Tests without any Category annotations

Category Name above is a Java interface defined in the package org.janusgraph.testcategory. These interfaces appear as arguments to the JUnit @Category(...) annotation, e.g. @Category({MemoryTests.class}).

Maven Property above is a boolean-valued pom.xml property that skips the associated test category when true and executes the associated test category when false. The default values defined in pom.xml can be overridden on the command-line in the ordinary Maven way, e.g. mvn -Dtest.skip.mem=false test.

Implementation Note. The Maven property naming pattern "test.skip.x=boolean" is needlessly verbose, a cardinal sin for command line options. A more concise alternative would be "test.x" with the boolean sense negated. However, this complicates the pom.xml configuration for the Surefire plugin since it precludes direct use of the Surefire plugin's <skip> configuration tag, as in <skip>${test.skip.perf}</skip>. There doesn't seem to be a straightforward way to negate a boolean or otherwise make this easy, at least without resorting to profiles or a custom plugin, though I might be missing something. Also, the mold is arguably already set by Surefire's "maven.test.skip" property, though that has slightly different interpretation semantics than the properties above.

Running a Single Test via Maven

The standard maven-surefire-plugin option applies for most tests:

mvn test -Dtest=full.or.partial.classname#methodname

However, MemoryTests and PerformanceTests are disabled by default regardless of whatever -Dtest=... option might be specified. When running a single MemoryTest or PerformanceTest, specify -Dtest.mem=true or -Dtest.perf=true as appropriate for the test in question.

Here's a concrete example.

# Executes no tests because the MemoryTests category is disabled by default
mvn test -Dtest=BerkeleyJEGraphPerformanceMemoryTest
# Executes the specified test
mvn test -Dtest=BerkeleyJEGraphPerformanceMemoryTest -Dtest.skip.mem=false

Running Solr Tests with External Server

Solr tests can be run using an external Solr server using Docker with Compose.

# optionally update the solr image version in the compose file prior to starting
docker-compose -f janusgraph-solr/src/test/resources/docker-compose.yml up

Monitor container logs and wait until the setup container exits (message will be resources_janusgraph_solr_test_setup_1 exited with code 0). Once setup is complete run janusgraph-solr tests with the index.search.solr.zookeeper-url property.

mvn clean install -pl janusgraph-solr -Dindex.search.solr.zookeeper-url=localhost:2181

Running Thrift/CQL tests with ScyllaDB (requires Docker)

To run Thrift or CQL tests with the ScyllaDB backend, ensure Docker is running and run tests under the scylladb-test Maven profile.

mvn clean install -pl janusgraph-cql -Pscylladb-test

Running Elasticsearch tests against managed Docker container

To run Elasicsearch tests using an embedded Elasticsearch Docker container, use the es-docker profile

mvn clean install -pl janusgraph-es -Pes-docker

Running Hadoop tests with Cassandra-3 using CQL record reader (requires Docker)

To run Hadoop tests with Cassandra-3 using the CQL record reader, start a Cassandra-3 Docker container and run tests with -DskipCassandra3=false. Note core HBase and Cassandra-2 Hadoop tests must be skipped when an external Cassandra instance is running.

docker run --name jg-cassandra -p 9160:9160 -p 9042:9042 -e CASSANDRA_START_RPC=true -d cassandra:3.10
mvn clean install -pl :janusgraph-hadoop-2 -DskipHBase -DskipCassandra -DskipCassandra3=false