Skip to content

Latest commit

 

History

History
191 lines (166 loc) · 9.77 KB

glsl-fuzz-reduce.md

File metadata and controls

191 lines (166 loc) · 9.77 KB

glsl-reduce manual for reducing fuzzed shaders generated by glsl-generate

For instructions on how to run glsl-reduce as a stand-alone tool, see the glsl-reduce manual for stand-alone reducing.

Synopsis

glsl-reduce SHADER_JOB --reduction-kind KIND [--preserve-semantics] [--error-string ERROR] [other options]

See below for common examples.

Description

glsl-reduce takes a shader job SHADER_JOB (a .json file) as an argument as well as further arguments or options to specify the interestingness test. glsl-reduce will try to simplify the given shader to a smaller, simpler shader that is still deemed "interesting".

SHADER_JOB should be a .json shader job file that represents all shaders and metadata needed to render an image. If the shader job is named foo.json, glsl-reduce will look for corresponding shaders alongside this file named foo.frag, foo.vert and foo.comp, of which there must be at least one, and will apply reduction to all such shaders that are present. On termination, the reduced shader job will be named foo_reduced_final.json (with associated shader files).

KIND specifies the "kind of reduction" (i.e. the type of interestingness test), the most common being:

  • ABOVE_THRESHOLD: a shader job is interesting if it produces an image that is sufficiently different from a reference image. This reduction kind is normally used with --preserve-semantics, otherwise the reducer will change the semantics of the shaders and the reference image will no longer be valid.
  • NO_IMAGE: a shader job is interesting if it does not produce an image and, optionally, the run log from running the shader job includes the regular expression ERROR.
    • This reduction kind is typically used without --preserve-semantics, so the reducer can change the semantics of the shader; we are usually reducing a crash or incorrect shader compilation error, so changing the semantics of the shaders is fine, as they should still compile and run. Note that the reducer always produces valid shaders that should compile.
    • This reduction kind is often used with --skip-render, which causes remote workers to skip rendering the shader. This can be much faster. In cases where the interesting error occurs during shader compilation, there is no need to render the shader to check if the error still occurs so --skip-render should be used. However, if the error occurs while rendering the shader, or as a result of rendering the shader, --skip-render should not be used.

Note that unless --preserve-semantics is passed, changes made by the reducer can cause a shader to exhibit very high or infinite runtime. For example, the reducer might remove statements from a loop that prevent the loop from terminating.

Given a shader job shader-job.json, the following are common reduction commands:

# Wrong image reduction.
glsl-reduce shader-job.json --preserve-semantics --reduction-kind ABOVE_THRESHOLD --reference reference.info.json

# A "no image" reduction for a crash that occurs during shader compilation.
# --skip-render should be used to speed up the interestingness test,
# since the error occurs during shader compilation; successful shader 
# compilation will be regarded as uninteresting. 
glsl-reduce shader-job.json --reduction-kind NO_IMAGE --error-string "Fatal signal 11" --skip-render

# A "no image" reduction for a crash that occurs during rendering.
# --skip-render must NOT be used since the error occurs during rendering;
# we need to render to reproduce the crash.
glsl-reduce shader-job.json --reduction-kind NO_IMAGE --error-string "Fatal signal 11"

# Custom interestingness test:
glsl-reduce shader-job.json interestingness_test.sh

Note regarding reference.info.json above: the reducer will look for reference.png alongside this file and use it as the reference image.

For the custom interestingness test, see the glsl-reduce manual for stand-alone reducing. Note that the custom interestingness test can still be used on shader jobs produced by glsl-generate.

Options:

usage: glsl-reduce [-h] [--reduction-kind REDUCTION_KIND] [--output OUTPUT]
                   [--preserve-semantics] [--max-steps MAX_STEPS]
                   [--verbose] [--seed SEED] [--timeout TIMEOUT]
                   [--metric METRIC] [--reference REFERENCE]
                   [--threshold THRESHOLD] [--retry-limit RETRY_LIMIT]
                   [--skip-render] [--error-string ERROR_STRING]
                   [--server SERVER] [--worker WORKER] [--stop-on-error]
                   [--swiftshader] [--continue-previous-reduction]
                   shader_job [interestingness_test]

Reduce GLSL shaders, driven by a  criterion  of  interest. The tool takes a
"shader job" as input, which is a  json  file (e.g. NAME.json) and a set of
files with the samename  in  the  same  directory.  NAME.json is a metadata
file that can  just  contain  "{}".  There  should  also  be  some graphics
shaders (NAME.frag and/or NAME.vert) or a compute shader (NAME.comp).

positional arguments:
  shader_job             Path  of  shader   job   to   be   reduced.   E.g.
                         /path/to/shaderjob.json 
  interestingness_test   Path to an  executable  shell  script  that should
                         decide whether a shader  job is interesting.  Only
                         allowed (and then  also  required) when performing
                         a custom reduction, which is the default.

optional arguments:
  -h, --help             show this help message and exit
  --reduction-kind REDUCTION_KIND
                         Kind of reduction to be performed.  Options are:
                         CUSTOM              Reduces  based   on   a  user-
                         supplied interestingness test.
                         NO_IMAGE                 Reduces    while    image
                         generation fails to produce an image.
                         NOT_IDENTICAL         Reduces    while    produced
                         image is not identical to reference.
                         IDENTICAL             Reduces    while    produced
                         image is identical to reference.
                         BELOW_THRESHOLD       Reduces    while   histogram
                         difference between  produced  image  and reference
                         is below a threshold.
                         ABOVE_THRESHOLD       Reduces    while   histogram
                         difference between  produced  image  and reference
                         is above a threshold.
                         VALIDATOR_ERROR       Reduces    while   validator
                         gives a particular error
                         ALWAYS_REDUCE        Always  reduces  (useful  for
                         testing)
                          (default: CUSTOM)
  --output OUTPUT        Directory  to  which  reduction  intermediate  and
                         final results will be written. (default: .)
  --preserve-semantics   Only  perform   semantics-preserving   reductions.
                         (default: false)
  --max-steps MAX_STEPS  The maximum  number  of  reduction  steps  to take
                         before giving up and  outputting the final reduced
                         file. (default: 2000)
  --verbose              Emit   detailed   information   related   to   the
                         reduction process. (default: false)
  --seed SEED            Seed with which  to  initialize  the random number
                         generator  that  is  used   to  control  reduction
                         decisions.
  --timeout TIMEOUT      Time   in    seconds    after    which    checking
                         interestingness  of  a  shader   job  is  aborted.
                         (default: 30)
  --metric METRIC        Metric used for image comparison.  Options are:
                            HISTOGRAM_CHISQR
                            PSNR
                          (default: HISTOGRAM_CHISQR)
  --reference REFERENCE  Path to reference  .info.json  result  (with image
                         result) for comparison.
  --threshold THRESHOLD  Threshold used  for  image  comparison.  (default:
                         100.0)
  --retry-limit RETRY_LIMIT
                         When getting an image  via  the server, the number
                         of times the  server  should  allow  the worker to
                         retry a shader before  assuming the shader crashes
                         the worker and marking it as SKIPPED. (default: 2)
  --skip-render          Don't render (just compile)  the  shader on remote
                         workers. Useful  when  reducing  compile  or  link
                         errors. (default: false)
  --error-string ERROR_STRING
                         For  NO_IMAGE  reductions,  a   shader  is  deemed
                         interesting if this  string  is  found  in the run
                         log. E.g. "Signal 11".
  --server SERVER        For non-CUSTOM  reductions,  shaders  will  be run
                         via a worker connected to this server URL.
  --worker WORKER        For non-CUSTOM reductions, shaders  will be run on
                         this worker. Use with --server.
  --stop-on-error        Quit if  something  goes  wrong  during reduction;
                         useful for testing. (default: false)
  --swiftshader          Use swiftshader for rendering. (default: false)
  --continue-previous-reduction
                         Carry on from where  a  previous reduction attempt
                         left off.  Requires  the  temporary  files written
                         by the previous reduction  to be intact, including
                         the  presence  of   a  REDUCTION_INCOMPLETE  file.
                         (default: false)