You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit fixes the remaining issues open about Daffodil's Layers
extension/plugins. Partial progress was in commit ff9ba4. This
finishes that work.
- DAFFODIL-2845 - Layer Improvements (umbrella ticket)
- DAFFODIL-2844 - LayerNotEnoughDataException ...
- DAFFODIL-2825 - Define Supported Layering API
- DAFFODIL-2841 - Layers do not support created SDE or PE/UE Errors
- DAFFODIL-2843 - How to create PE/UE from a layer
The layer API is all defined in Java clases and interfaces now. One
of the built-in layers (Gzip) is written in Java now.
There is only one layer property now: dfdlx:layer which takes a QName of
the layer class.
Layer classes are derived from
org.apache.daffodil.runtime1.layers.api.Layer
A layer a name and target namespace, and hence a QName is required to
identify it. This namespace is OWNED by the layer. All DFDL variables
defined in that namespace are either used to pass parameters to the
layer code, or receive results (such as a checksum) back from the
layer code. This is enforced. A layer that has no DFDL variables does
not have to define a DFDL schema that defines the layer's target
namespace, but any layer that uses DFDL variables *must* define a
schema with the layer's namespace as its target namespace, with the
variables declared in it (using dfdl:defineVariable).
There is also an abstract base class for defining checksum layers
called,
org.apache.daffodil.runtime1.layers.api.ChecksumLayer
All the layer properties about layer length kinds etc. are gone.
dfdlx:layerLengthKind='boundaryMark' and dfdlx:boundaryMark properties
are replaced by a layer named 'boundaryMark'.
dfdlx:layerLengthKind='explicit' and dfdlx:layerLength properties are
replaced by a layer named 'fixedLength'.
dfdlx:layerLengthKind='implicit' is replaced by just not using one of
the two above layers.
dfdlx:layerTransform is replaced by just dfdlx:layer and is now a
QName. Layers no longer have both transformers and length-limiters
mixed together.
There are two ways to do explicit layer limiting, that differ in an
important way.
If you use 'implicit' layer length (no restriction layer), and
constrain the layer length by a surrounding element of specified
length, then the layer length is controlled when parsing, but NOT
controlled when unparsing.
If you use the layer "fixedLength", that dictates the parse and
unparse length to be the value of the layerLength variable. Similarly
if you use a checksum layer built with the ChecksumLayer base class,
the length is controlled for both parsing and unparsing as it works
like the fixedLength layer.
Layer Variables:
Each variable is either a parameter passed to a special setter named
setLayerVariableParameters, or is a return variable which is populated
from a special getter. The special setter args and getters taken
together must use up all the DFDL variables defined in the layer's
namespace. (See the javadoc for the Layer base class.)
A special getter example might be:
int getLayerVariableResult_checksum() { .... }
This corresponds to, and returns the value that populates, the DFDL
variable named 'checksum' in the layer's namespace. That DFDL variable
must have a DFDL type corresponding to the int type (xs:unsignedShort or
xs:int).
Layers could have multiple result variables, but we have no actual
examples of more than one value being returned. (Returning multiple
variables is tested. We just have no real use cases for it at this
time.)
Layer Exception Handling:
Added setProcessingErrorException(...) method to Layer
This allows the layer to specify that if the layer throws specific
exceptions or runtime exceptions that they are converted into processing
errors. This eliminates most need for layers to contain try-catches.
Additional Fixes in this Commit:
Fixed gzip layer which was not constructing on Java 17 or 21 due to
static initializer differences of some sort.
Removed restriction that a layer can only have a single term inside it.
Added close() to InputSourceDataInputStream, as we weren't closing these.
They now implement java.io.Closeable so can use try-with-resources for auto
closing in Java.
There are a large number of simplifications in the "layering" of the various
layer-related implementation classes.
LayerRegistry is gone, merged into LayerFactory, which is gone, merged into
LayerDriver.
In the examples/tests for layering: Changed check digit bad digit style to
construct an always-invalid element instead of an assertion failure.
About Testing:
The testing is quite extensive now. There are tests for every way that
a user can goof up the definition of a layer class, and there are
tests for processingError, runtimeSchemaDefinitionError, and throwing
an Exception from every place a user-defined Layer could cause these.
Parse errors cause backtracking in all sensible cases. Nothing aborts.
Protecting every place that layer code is called from any sort of
error/throw requires quite a number of try/catch blocks. This may have
performance implications.
Add some fuzz testing around gzip layer
GZip layer converts IOExceptions to PE explicitly.
Fuzz tests show that gzip does a good job converting
various bad data issues into IOException, so
really that's the only one the GzipLayer has to deal with.
Changed boundaryMarkLayer and gzip layer and AIS (test) layer
to use setProcessingErrorException to handle IOExceptions.
(and the test SimpleBombOutLayer.scala used to drive negative tests.)
Added bug-related comments to other files about bugs discovered
while fuzz-testing of other non-daffodil layer schemas.
DEPRECATION/COMPATABILITY
This new Layer API is 100% incompatible with schemas or layer code
from Daffodil 3.6.0 and all prior versions of Daffodil. The layer
feature was just an experimental feature in Daffodil, so we reserved
the right to change it.
However, it is our hope that the new Layer API introduced by this commit
will prove to be stable and supportable indefinitely.
DAFFODIL-2845, DAFFODIL-2844, DAFFODIL-2825, DAFFODIL-2841, DAFFODIL-2843
0 commit comments