-
Notifications
You must be signed in to change notification settings - Fork 12
Transformation spec proposal for NGFF
Right-handed coordinate system (x→, y↓, z↗), positive rotation angles rotate clock-wise (↻). Index according to axis index (x:0, y:1, z:2, ...).
Transformations are specified as forward transformations, i.e. a transformation transforms a vector from source into target space, a sequence of transformations is applied from first to last element. Rendering transformed images or volumes typically goes through the inverse of a transformation and some transformations cannot be inverted easily. If the sole purpose of a transformation is to be inverted (as in for rendering an image or volume), then its inverse can be explicitly defined as inverse_of
an otherwise undefined transformation.
Each transformation has a String:type
property from the below list and defines how other fields are interpreted. More transformations and their fields will be added to the list as they become available.
type | fields | description |
---|---|---|
identity |
identity transformation, is the default transformation and is typically not explicitly defined | |
sequence |
Transformation[]:transformations |
sequence of transformations, an empty sequence is the identity
|
translation |
one of:number[]:translation String :path
|
translation vector, stored either as a number[] (translation ) or as binary data at a location in this container (path ). If both are present, path is preferred. length of vector defines number of dimensions |
scale |
one of:number[]:scale String :path
|
scale vector, stored either as a number[] (scale ) or as binary data at a location in this container (path ). If both are present, path is preferred. length of vector defines number of dimensions |
affine |
one of:number[]:affine String :path
|
affine transformation matrix defined as list consisting of n sets of n + 1 scalar numbers, stored either as a number[] (affine ) or as binary data at a location in this container (path ). If both are present, path is preferred. n is number of dimensions |
d_field |
String:url String:path(null) boolean:interleaved(true)
|
deformation / displacement field storing one offset for each grid coordinate, url points to the data container (e.g. "/field.hdf5" or "/field.tif" and path points to the dataset if the container can hold multiple datasets or is empty for single dataset containers |
p_field |
String:url String:path(null) boolean:interleaved(true)
|
position field storing one absolute position for each grid coordinate, url points to the data container (e.g. "/field.hdf5" or "/field.tif" and path points to the dataset if the container can hold multiple datasets or is empty for single dataset containers |
inverse_of |
Transformation:transformation |
inverse of a transformation |
translation
{
"type" : "translation",
"translation" : [-1, 2.2, 0]
}
inverse_of / scale
{
"type" : "inverse_of",
"transformation" : {
"type" : "scale",
"scale" : [1.1, 2.2, 3.3]
}
}
affine (2d)
{
"type" : "affine",
"affine" : [2.0, 0.1, 10, -0.1, 2.0, -10]
}
affine (2d) path
{
"type" : "affine",
"path" : "/a/volume/affine_parameters"
}
where path
in this container must contain a 1d dataset of length 6.
affine (3d)
{
"type" : "affine",
"affine" : [2.0, 0.1, -0.01, 10.0, -0.1, 2.0, 0.01, -10.0, 0.0, 0.0, 1.0, 5.0]
}
sequence
{
"type" : "sequence",
"transforms" : [
{
"type" : "translation",
"translation" : [ -1, 2.2, 0 ]
},
{
"type" : "scale",
"scale" : [ 1.2, 1, 2.2 ]
}
]
}
displacement / deformation field hdf5
{
"type" : "d_field",
"url" : "/path/to/my/JRC2018F_FCWB.h5",
"path" : "/0/dfield"
}
displacement / deformation field nrrd
{
"type" : "d_field",
"url" : "file:///path/to/my/JRC2018F_FCWB.nrrd,
}
This section is preliminary.
Axes may also be tagged with a type that describes the kind of domain (time, space, wavelength, amino-acid residue).
This should be stored in an axes
field in a datasets attributes. The value of this field must be an array of
length to the dataset's dimensionality. Each element of the array should be an axis object as described below.
See here for a longer list of specifications used for brainstorming.
"axes": {
"labels" : ["x", "y", "c", "z", "t" ],
"types" : ["space", "space", "channel", "space", "time" ],
"units" : [ "µm", "µm", "pixel", "µm", "ms" ]
}
See https://github.com/ome/ngff/issues/35
Example:
"axes": [
{
"label": "x",
"type": "space",
"unit": "µm"
},
{
"label": "y",
"type": "space",
"unit": "µm"
},
{
"label": "c",
"type": "channels",
"unit": null
},
{
"label": "z",
"type": "space",
"unit": "µm"
},
{
"label": "t",
"type": "time",
"unit": "ms"
}
Each axis object has the following fields:
-
label
:string
- an arbitrary string label / name for the axis
-
type
:string
- the type of domain of the axis. e.g. (
space
,time
,channel
)
- the type of domain of the axis. e.g. (
-
unit
:string
- the unit of the domain, or one of (
pixel
,px
, ornull
) if arbitrary or not applicable. e.g. (mm
,microns
,seconds
)
- the unit of the domain, or one of (
This proposal combines the descriptions of axes and transformations. A benefit of this is that it more clearly describes which dimensions can be "mixed" by transformations (see the alternative below). In proposals A and B, the assumption is that dimensions of the same type can be "mixed".
To be precise, "mixed" means it is possible to interpolate between those dimensions. It also suggests that a flag should indicate whether interpolation is allowed within an axis, i.e., whether its domain is continuous or discrete (see the "Axis types and Interpolation" section below).
The example below shows an example that motivates the inputIndexes
and outputIndexes
fields. ImageJ's
ImagePlus
stores dimensions in the XYCZ
order, but if these dimensions are stored as XYZC
order
in the storage container, then a permutation is necessary: a feature that these fields enable.
{ "axes": [
{
"type":"spatial",
"labels":["X","Y","Z"],
"inputIndexes":[0,1,2],
"outputIndexes":[0,1,3],
"transform": {
"type":"affine",
"affine":[2.0, 0.0, 0.0, 10.0,
0.0, 0.5, 0.0 -5.0,
0.0, 0.0, 1.1 0.1 ]
},
"unit":"nm"
},
{
"type":"channel",
"discrete" : true,
"labels":["C"],
"inputIndexes":[3],
"outputIndexes":[2],
"transform": { "type":"identity" }
}
]
}
The type
, and label
fields are as above, though here the label
field is a String[]
, and gives a name to
the dimension in the output "world" coordinates.
-
transform
(Optional) : an object describing a transformation object- if not provided, assume the
identity
transformation.
- if not provided, assume the
-
inputIndexes
(Optional) :- if not provided, assume
[ M+1, M+2, ..., M+N-1]
, whereM
is the largest integer in theinputIndex
fields for all axes in the list prior to this one, andN
is the number of dimensions for this axes (length oflabels
). See the "Inferring indexes" section below for examples.
- if not provided, assume
-
outputIndexes
(Optional) :- if not provided, assume is identical to
inputIndexes
- if not provided, assume is identical to
Example 1
[
{
"type":"spatial",
"labels":["X","Y"],
"unit":"nm"
},
{
"type":"channel",
"labels":["C"],
}
]
implies
[
{
"type":"spatial",
"labels":["X","Y"],
"unit":"nm"
"inputIndexes":[0,1],
"outputIndexes":[0,1],
},
{
"type":"channel",
"labels":["C"],
"inputIndexes":[2],
"outputIndexes":[2],
}
]
Example 2
[
{
"type":"channel",
"labels":["C"],
},
{
"type":"space",
"labels":["X","Y"],
"unit":"nm"
}
]
implies
[
{
"type":"channel",
"labels":["C"],
"inputIndexes":[0],
"outputIndexes":[0],
},
{
"type":"space",
"labels":["X","Y"],
"unit":"nm"
"inputIndexes":[1,2],
"outputIndexes":[1,2],
}
]
Specifying the type (continuous / discrete) of the domain indicates which axes may be interpolated, which may be jointly interpolated, and how that interpolation should occur.
- An axis may be interpolated if it's domain is not discrete.
- example: "angle" axes should be circularly interpolated
- Two axes may be jointly interpolated ("mixed") if they
- The
type
s the both axes are equal. - They're part of an axis group (as
X
andY
above), which implies they share atype
.
- The
Proposal C associates a transformation with every axis (or group of axes). This proposal associates
transformations with input and output axes. A convenient (necessary?) addition to make this clear
is to give identities to the "raw" or "data" axis dimensions. In this example we name these dim_$i
,
as are the defaults for xarray.
In the examples below, we assume the data are stored in XYZC
order and are permuted to XYCZ
order in
order to be imported into ImageJ (similar to the Proposal C examples).
{ "transforms" : [
{
"type" : "affine",
"affine" : [2.0, 0.0, 0.0, 10.0,
0.0, 0.5, 0.0 -5.0,
0.0, 0.0, 1.1 0.1 ],
"inputs" : [
{
"label" : "dim_0",
"type" : "data",
"index" : 0,
"unit" : null
},
{
"label" : "dim_1",
"type" : "data",
"index" : 1,
"unit" : null
},
{
"label" : "dim_2",
"type" : "data",
"index" : 2,
"unit" : null
}
],
"outputs" : [
{
"label" : "x",
"type" : "space",
"unit" : "um"
},
{
"label" : "y",
"type" : "space",
"unit" : "um"
},
{
"label" : "z",
"type" : "space",
"unit" : "um"
}
]
},
{
"type":"identity",
"inputs" : [
{
"label" : "dim_3",
"type" : "data",
"index" : 3,
"unit" : null
}
],
"outputs" : [
{
"label" : "c",
"type" : "channel",
"unit" : null
}
]
}
]
}
Consider labeling the data axes outside the list of transformations, like this:
"dataAxes" : [
{ "label" : "i", },
{ "label" : "j", },
{ "label" : "k", },
{ "label" : "l", }
],
"outputAxes" : [
{
"label" : "x",
"type" : "space",
"unit" : "um"
},
{
"label" : "y",
"type" : "space",
"unit" : "um"
},
{
"label" : "c",
"type" : "channel",
"unit" : "none"
},
{
"label" : "z",
"type" : "space",
"unit" : "um"
}
],
"transforms" : [
{
"type" : "affine",
"affine" : [2.0, 0.0, 0.0, 10.0,
0.0, 0.5, 0.0 -5.0,
0.0, 0.0, 1.1 0.1 ],
"inputs" : [ "i", "j", "l" ],
"outputs" : [ "x", "y", "z" ]
},
{
"type" : "identity",
"inputs" : [ "k" ],
"outputs" : [ "c" ]
}
]
Or we could forget about the "dataAxes" and go with indexes of the input like this:
"transforms" : [
{
"type" : "affine",
"affine" : [2.0, 0.0, 0.0, 10.0,
0.0, 0.5, 0.0 -5.0,
0.0, 0.0, 1.1 0.1 ],
"inputIndexes" : [ 0, 1, 2 ],
"outputs" : [ "x", "y", "z" ]
},
{
"type" : "identity",
"inputIndexes" : [ 3 ],
"outputs" : [ "c" ]
}
]
where the types of the axes are inferred.
How should we handle cases in which transforms are applied to non-sequential axes,
XYCZ
, for example.
"transforms" : [
{
"type" : "affine",
"affine" : [2.0, 0.0, 0.0, 10.0,
0.0, 0.5, 0.0 -5.0,
0.0, 0.0, 1.1 0.1 ],
"inputIndexes" : [ 0, 1, 3 ],
"outputs" : [ "x", "y", "z" ]
},
{
"type" : "identity",
"inputIndexes" : [ 2 ],
"outputs" : [ "c" ]
}
]
The default output axes should be the same as the input axes, specifically XYCZ
.
Consider allowing overriding the defaults with an outputIndexes
attribute.
For example, this specification:
"transforms" : [
{
"type" : "identity",
"inputIndexes" : [ 0, 1, 3 ],
"outputIndexes" : [ 2, 1, 0 ],
"outputs" : [ "x", "y", "z" ]
},
{
"type" : "identity",
"inputIndexes" : [ 2 ],
"outputIndexes" : [ 3 ],
"outputs" : [ "c" ]
}
]
converts XYCZ
to ZYXC
, since it's index mapping is
0 -> 2
1 -> 1
3 -> 0
2 -> 3
units
are optional. If not provided, assume they are arbitrary "pixel" or discrete units.
Consider allowing types to be inferred from labels:
label | type |
---|---|
x , y , z , X , Y , Z
|
space |
c , C
|
channel |
t , T
|
time |
v , V
|
vector |
m , M
|
matrix |
theta , rho , phi
|
angle |
As a result, this axis specification:
"axes": {
"labels" : ["x", "y", "c" ],
}
implies:
"axes": {
"labels" : ["x", "y", "c", "z", "t" ],
"types" : ["space", "space", "channel" ],
"units" : [ "pixel", "pixel", "pixel" ]
}
What should assumptions be if no axes
are specified? Here are some ideas:
2D datasets imply:
"axes": {
"labels" : ["x", "y"],
"types" : ["space", "space" ],
"units" : [ "pixel", "pixel" ]
}
3D datasets imply:
"axes": {
"labels" : ["x", "y", "z"],
"types" : ["space", "space", "space" ],
"units" : [ "pixel", "pixel", "pixel" ]
}
4D datasets imply:
"axes": {
"labels" : ["x", "y", "z", "c" ],
"types" : ["space", "space", "space", "channel" ],
"units" : [ "pixel", "pixel", "pixel" ]
}
5D datasets imply:
"axes": {
"labels" : ["x", "y", "z", "c", "t" ],
"types" : ["space", "space", "space", "channel", "time" ],
"units" : [ "pixel", "pixel", "pixel", "pixel" ]
}