README.md

These integration tests verify the correctness and consistency of mapbox-gl-js and mapbox-gl-native rendering.

Organization

Tests are contained in a directory tree, generally organized by style specification property: background-color, line-width, etc., with a second level of directories below that for individual tests. For example, the test for specifying a literal circle-radius value lives in test/integration/render-tests/circle-radius/literal/.

Within a leaf directory is a style.json file (e.g. circle-radius/literal/style.json), which contains the minimal style needed for the given test case. The style can specify the map size, center, bearing, and pitch, and additional test metadata (e.g. output image dimensions).

The expected output for a given test case is in expected.png, e.g. circle-radius/literal/expected.png.

Supporting files -- glyphs, sprites, and tiles -- live in their own respective subdirectories at the top level. The test harness sets up the environment such that requests for these resources are directed to the correct location.

The contents of vector tile fixtures can be read using the vt2geojson tool (see below).

Running tests

To run the entire integration test suite (both render or query tests), from within the mapbox-gl-js directory run the command:

yarn run test-suite

To run only the render/query tests:

yarn run test-render

or

yarn run test-query-node

To run only the expression tests:

yarn run test-expressions

Running specific tests

To run a subset of tests or an individual test, you can pass a specific subdirectory to the test-render script. For example, to run all the tests for a given property, e.g. circle-radius:

$ yarn run test-render circle-radius
...
* passed circle-radius/antimeridian
* passed circle-radius/default
* passed circle-radius/function
* passed circle-radius/literal
* passed circle-radius/property-function
* passed circle-radius/zoom-and-property-function
6 passed (100.0%)
Results at: ./test/integration/render-tests/index.html
Done in 2.71s.

Or to run a single test:

$ yarn run test-render circle-radius/literal
...
* passed circle-radius/literal
1 passed (100.0%)
Results at: ./test/integration/render-tests/index.html
Done in 2.32s.

Viewing test results

During a test run, the test harness will use GL-JS to create an actual.png image from the given style.json, and will then use pixelmatch to compare that image to expected.png, generating a diff.png highlighting the mismatching pixels (if any) in red.

After the test(s) have run, you can view the results graphically by opening the index.html file generated by the harness:

open ./test/integration/render-tests/index.html

or

open ./test/integration/query-tests/index.html

Running tests in the browser

Query tests can be run in the browser, the server for serving up the test page and test fixtures starts when you run

yarn run start

OR

yarn run start-debug

If you want to run only the test server run:

yarn run watch-query

Then open the following url in the browser of your choice to start running the tests.

http://localhost:7357/

Running specific tests

A filter can be specified by using the filter query param in the url. E.g, adding

?filter=circle-pitch

to the end of the url will only run the tests that contain circle-pitch in the name.

Build Notifications

The terminal window can be very noisy with both the build and the test servers running in the same session. So the server uses platform notifications to inform when the build has finished. If this behaviour is annoying, it can be disabled by setting the following env-var

DISABLE_BUILD_NOTIFICATIONS=true

Writing new tests

Note: Expected results are always generated with the js implementation. This is merely for consistency and does not imply that in the event of a rendering discrepancy, the js implementation is always correct.

To add a new render test:

1. Create a new directory test/integration/render-tests/<property-name>/<new-test-name>

2. Create a new style.json file within that directory, specifying the map to load. Feel free to copy & modify one of the existing style.json files from the render-tests subdirectories. In this file, you can add additional information to describe the test and expected outcomes using the description metadata field.

3. Generate an expected.png image from the given style by running the new test with the UPDATE flag enabled:

   $ UPDATE=1 yarn run test-render <property-name>/<new-test-name>
   

   The test will appear to fail, but you'll now see a new expected.png in the test directory.

4. Manually inspect expected.png to verify it looks as expected, and optionally run the test again without the update flag (yarn run test-render <property-name>/<new-test-name>) to watch it pass (enjoy that dopamine kick!)

5. Commit the new style.json and expected.png 🚀

Reading Vector Tile Fixtures

Install vt2geojson, a command line utility which turns vector tiles into geojson, and harp, a simple file server.

npm install -g vt2geojson harp

Start a static file server

harp server .

Read the contents of an entire vector tile

vt2geojson -z 14 -y 8803 -x 5374 http://localhost:9000/tiles/14-8803-5374.mvt

Read the contents of a particular layer in a vector tile

vt2geojson --layer poi_label -z 14 -y 8803 -x 5374 http://localhost:9000/tiles/14-8803-5374.mvt