Fixes to make NDMeta slicing work, especially as part of an NDCollect…

…ion.

docs/explaining_ndcube/metadata.rst

@@ -54,7 +54,7 @@ Types of Axis-aware Metadata: Axis-aligned vs. Grid-aligned
 
 There are two types of axis-aware metadata: axis-aligned and grid-aligned.
 Axis-aligned metadata associates a scalar or string with an array axis.
-It can also assign an array of scalars or strings to mutliple array axes, so long as there is one value per associated axis.
+It can also assign an array of scalars or strings to multiple array axes, so long as there is one value per associated axis.
 For example, the data produced by a scanning slit spectrograph is associated with real world values.
 But each axis also corresponds to features of the instrument: dispersion (spectral), pixels along the slit (spatial), position of the slit in the rastering sequence (spatial and short timescales), and the raster number (longer timescales).
 The axis-aligned metadata concept allows us to avoid ambiguity by assigning each axis with a label (e.g. ``("dispersion", "slit", "slit step", "raster")``).
@@ -102,14 +102,14 @@ To do this, we provide another `~collections.abc.Mapping`, e.g. a `dict`, with t
 
   >>> key_comments = {"name": "Each planet in the solar system has a name."}
   >>> meta = NDMeta(raw_meta, key_comments=key_comments)
-  
+
 We can now access the comments by indexing the `~ndcube.NDMeta.key_comments` property:
 
 .. code-block:: python
 
   >>> meta.key_comments["name"]
   "Each planet in the solar system has a name."
-  
+
 Now let's discuss how to initialize how to `~ndcube.NDMeta` with axis-aware metadata.
 (Here, we will specifically consider grid-aligned metadata.  Axis-aligned metadata is assigned in the same way.  But see the :ref:`assigning_axis_aligned_metadata` section for more details.)
 Similar to ``key_comments``, we assign metadata to axes by providing a `~collections.abc.Mapping`, e.g. a `dict`, via its ``axes`` kwarg.
@@ -135,13 +135,13 @@ It is easy to see which axes a piece of metadata corresponds to by indexing the
   (0,)
   >>> meta.axes["pixel response"]
   (0, 2)
-  
+
 Finally, it is possible to attach the shape of the associated data to the `~ndcube.NDMeta` instance via the ``data_shape`` kwarg:
 
 .. code-block:: python
 
   >>> meta = NDMeta(raw_meta, axes=axes, key_comments=key_comments, data_shape=(5, 1, 2))
-  
+
 Or by directly setting the ``~ndcube.NDMeta.data_shape`` property after instantiation:
 
 .. code-block:: python
@@ -170,7 +170,7 @@ Let's use this method to add a voltage that varies with time, i.e. the first dat
   >>> meta.add("voltage", u.Quantity([1.]*5, unit=u.V), key_comment="detector bias voltage can vary with time and pixel column.", axes=(0,))
   >>> meta["voltage"]
   <Quantity [1., 1., 1., 1., 1.] V>
-  
+
 If you try to add metadata with a pre-existing key, `~ndcube.NDMeta.add` will error.
 To replace the value, comment, or axes values of pre-existing metadata, set the ``overwrite`` kwarg to ``True``.
 
@@ -179,15 +179,15 @@ To replace the value, comment, or axes values of pre-existing metadata, set the
   >>> meta.add("voltage", u.Quantity([-300.]*5, unit=u.V), comment="detector bias voltage", axes=(0,), overwrite=True)
   >>> meta["voltage"]
   <Quantity [-300., -300., -300., -300., -300.] V>
-  
+
 Unwanted metadata can be removing by employing the `del` operator.
 
 .. code-block:: python
 
   >>> del meta["voltage"]
   >>> meta.get("voltage", "deleted")
   "deleted"
-  
+
 Note that the `del` operator also removes associated comments and axes.
 
 .. code-block:: python
@@ -213,7 +213,7 @@ If no axis-aware metadata is present, `~ndcube.NDMeta.data_shape` is empty:
   >>> meta = NDMeta(raw_meta)
   >>> meta.data_shape
   array([], dtype=int64)
-  
+
 If we now add the ``"pixel response"`` metadata that we used, earlier the `~ndcube.NDMeta.data_shape` will be updated.
 
 .. code-block:: python
@@ -233,7 +233,7 @@ For example, if we add a 1-D ``"exposure time"`` and associate it with the 1st a
 .. code-block:: python
 
   >>> meta.add("exposure time", [1.9, 2.1, 5, 2, 2] * u.s, axes=0)
-  
+
 Moreover, if we now directly set the `~ndcube.NDMeta.data_shape` via ``meta.data_shape = new_shape``, we cannot change the length of axes already associated with grid-aligned metadata, without first removing or altering that metadata.
 However, these restrictions do not apply if we want to change the shape of the 2nd axis, or add new metadata to it, because its length is ``0``, and hence considered undefined.
 
@@ -254,7 +254,7 @@ To provide axis-aligned metadata, i.e. where each axis has a single value (see :
 .. code-block:: python
 
   >>> meta.add("axis name", np.array(["a", "b", "c", "d"]), axes=(0, 1, 2, 3))
-  
+
 Note that the length of ``"axis name"`` is the same as the number of its associated axes.
 Also note that we have now indicated that there is 4th axis.
 ``meta.data_shape`` has therefore been automatically updated accordingly.
@@ -282,9 +282,9 @@ It stores the metadata that was originally passed to the `~ndcube.NDMeta` constr
   >>> meta
   ???
   >>> meta.original_meta
-  
+
 Note that, ``meta.original_meta`` does not contain ``"exclamation"``, but still contains ``"name"``.
-This is because these were added and removed after initialzation.
+This is because these were added and removed after initialization.
 Also note that the type of the original metadata object is maintained.
 
 The `~ndcube.NDMeta.original_shape` property is a useful reference back to the original metadata, even after it has been altered via a complex sequence of operations.

docs/explaining_ndcube/slicing.rst

@@ -444,7 +444,7 @@ Therefore, slicing is achieved by applying Python's slicing API to `~ndcube.NDMe
   >>> sliced_meta = meta.slice[0, 1:3]
   >>> sliced_meta.data_shape
   array([2, 5])
-  
+
 Note that by applying the slice item ``[0, 1:3]`` to ``meta``, the shape of the ``sliced_meta`` has been altered accordingly.
 The first axis has been sliced away, the second has been truncated to a length of 2, and the third remains unchanged.
 The shape of ``"pixel response"`` has been altered:
@@ -453,21 +453,21 @@ The shape of ``"pixel response"`` has been altered:
 
   >>> sliced_meta["pixel response"].shape
   (2, 5)
-  
+
 while ``"exposure time"`` has been reduced to a scalar:
 
 .. code-block:: python
 
   >>> sliced_meta["exposure time"]
   <Quantity 2. s>
-  
+
 Moreover, because the first axis has been sliced away, ``"exposure time"`` is no longer associated with a data array axis, and so is no longer present in the ``axes`` property:
 
 .. code-block:: python
 
   >>> list(sliced_meta.axes.keys())
   ["pixel response"]
-  
+
 Finally, note that axis-agnostic metadata is unaltered by the slicing process.
 
 .. code-block:: python
@@ -494,7 +494,7 @@ To demonstrate this, let's reinstantiate the same metadata object as in the abov
   (4, 4, 5)
   >>> my_cube.meta.data_shape
   array([4, 4, 5])
-  
+
 Now let's apply the same slice item to the cube as we applied to ``meta`` in the above section.
 Note that shape of the resultant `~ndcube.NDCube` and its associated `~ndcube.NDMeta` object now have the same new shape consistent with the slice item.
 
@@ -505,7 +505,7 @@ Note that shape of the resultant `~ndcube.NDCube` and its associated `~ndcube.ND
   (2, 5)
   >>> sliced_cube.meta.data_shape
   array([2, 5])
-  
+
 Furthermore, the metadata's values, axis-awareness, etc., have also been altered in line with the slice item.
 In fact, ``sliced_cube.meta`` is equivalent to ``sliced_meta`` from the previous section, because we have applied the same slice item to two equivalent `~ndcube.NDMeta` objects.
 

ndcube/conftest.py

@@ -16,7 +16,7 @@
 from astropy.wcs import WCS
 
 from ndcube import ExtraCoords, GlobalCoords, NDCube, NDCubeSequence, NDMeta
-from . import helpers
+from ndcube.tests import helpers
 
 # Force MPL to use non-gui backends for testing.
 try:

ndcube/ndcollection.py

@@ -1,3 +1,5 @@
+import copy
+import numbers
 import textwrap
 import collections.abc
 
@@ -152,6 +154,7 @@ def __getitem__(self, item):
                 new_data = [self[_item] for _item in item]
                 new_keys = item
                 new_aligned_axes = tuple([self.aligned_axes[item_] for item_ in item])
+                new_meta = copy.deepcopy(self.meta)
 
             # Else, the item is assumed to be a typical slicing item.
             # Slice each cube in collection using information in this item.
@@ -169,7 +172,33 @@ def __getitem__(self, item):
                 # Therefore the collection keys remain unchanged.
                 new_keys = list(self.keys())
                 # Slice meta if sliceable
-                new_meta = self.meta.slice[item] if self.meta.__ndcube_can_slice else copy.deepcopy(self.meta)
+                if hasattr(self.meta, "__ndcube_can_slice__") and self.meta.__ndcube_can_slice__:
+                    # Convert negative indices to positive indices as they are not supported by NDMeta.slice
+                    sanitized_item = copy.deepcopy(item)
+                    aligned_shape = self.aligned_dimensions
+                    if isinstance(item, numbers.Integral):
+                        if item < 0:
+                            sanitized_item = int(self.aligned_dimensions[0] + item)
+                    elif isinstance(item, slice):
+                        if (item.start is not None and item.start < 0) or (item.stop is not None and item.stop < 0):
+                            new_start = aligned_shape[0] + item.start if item.start < 0 else item.start
+                            new_stop = aligned_shape[0] + item.stop if item.stop < 0 else item.stop
+                            sanitized_item = slice(new_start, new_stop)
+                    else:
+                        sanitized_item = list(sanitized_item)
+                        for i, ax_it in enumerate(item):
+                            if isinstance(ax_it, numbers.Integral) and ax_it < 0:
+                                sanitized_item[i] = aligned_shape[i] + ax_it
+                            elif isinstance(ax_it, slice):
+                                if (ax_it.start is not None and ax_it.start < 0) or (ax_it.stop is not None and ax_it.stop < 0):
+                                    new_start = aligned_shape[i] + ax_it.start if ax_it.start < 0 else ax_it.start
+                                    new_stop = aligned_shape[i] + ax_it.stop if ax_it.stop < 0 else ax_it.stop
+                                    sanitized_item[i] = slice(new_start, new_stop)
+                        sanitized_item = tuple(sanitized_item)
+                    # Use sanitized item to slice meta.
+                    new_meta = self.meta.slice[sanitized_item]
+                else:
+                    new_meta = copy.deepcopy(self.meta)
 
             return self.__class__(list(zip(new_keys, new_data)), aligned_axes=new_aligned_axes,
                                   meta=new_meta, sanitize_inputs=False)

ndcube/ndcube_sequence.py

@@ -123,7 +123,7 @@ def __getitem__(self, item):
         if isinstance(item, numbers.Integral):
             return self.data[item]
         # Determine whether meta attribute should be sliced.
-        new_meta = self.meta.slice[item] if self.meta.__ndcube_can_slice else copy.deepcopy(self.meta)
+        new_meta = self.meta.slice[item] if (hasattr(self.meta, "__ndcube_can_slice__") and self.meta.__ndcube_can_slice__) else copy.deepcopy(self.meta)
         # Create an empty sequence in which to place the sliced cubes.
         result = type(self)([], meta=new_meta, common_axis=self._common_axis)
         if isinstance(item, slice):

ndcube/tests/helpers.py

@@ -13,6 +13,7 @@
 from numpy.testing import assert_equal
 
 import astropy
+import astropy.units as u
 from astropy.wcs.wcsapi import BaseHighLevelWCS
 from astropy.wcs.wcsapi.fitswcs import SlicedFITSWCS
 from astropy.wcs.wcsapi.low_level_api import BaseLowLevelWCS
@@ -196,8 +197,15 @@ def assert_collections_equal(collection1, collection2, check_data=True):
         else:
             raise TypeError(f"Unsupported Type in NDCollection: {type(cube1)}")
 
+def ndmeta_et0_pr01(shape):
+    return NDMeta({"salutation": "hello",
+                   "exposure time": u.Quantity([2.] * shape[0], unit=u.s),
+                   "pixel response": (100 * np.ones((shape[0], shape[1]), dtype=float)) * u.percent},
+                   axes={"exposure time": 0, "pixel response": (0, 1)}, data_shape=shape)
+
+
 def ndmeta_et0_pr02(shape):
     return NDMeta({"salutation": "hello",
-                   "exposure time": u.Quantity([2.] * shape[0], unit=u.s)
+                   "exposure time": u.Quantity([2.] * shape[0], unit=u.s),
                    "pixel response": (100 * np.ones((shape[0], shape[2]), dtype=float)) * u.percent},
-                   axes={"exposure time": 0, "pixel response": (0, 2)})
+                   axes={"exposure time": 0, "pixel response": (0, 2)}, data_shape=shape)

ndcube/tests/test_ndcollection.py

@@ -37,7 +37,7 @@
 # Define collections
 aligned_axes = ((1, 2), (2, 0), (1, 2))
 keys = ("cube0", "cube1", "cube2")
-cube_coll_meta = helpers.ndmeta_et0_pr02((4, 5))
+cube_coll_meta = helpers.ndmeta_et0_pr01((4, 5))
 cube_collection = NDCollection([("cube0", cube0), ("cube1", cube1), ("cube2", cube2)], aligned_axes, meta=cube_coll_meta)
 unaligned_collection = NDCollection([("cube0", cube0), ("cube1", cube1), ("cube2", cube2)], aligned_axes=None)
 seq_collection = NDCollection([("seq0", sequence02), ("seq1", sequence20)], aligned_axes="all")
@@ -50,23 +50,23 @@
 
     (slice(1, 3), cube_collection, NDCollection(
         [("cube0", cube0[:, 1:3]), ("cube1", cube1[:, :, 1:3]), ("cube2", cube2[:, 1:3])],
-        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[1:3]))),
+        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[1:3])),
 
     (slice(-3, -1), cube_collection, NDCollection(
         [("cube0", cube0[:, -3:-1]), ("cube1", cube1[:, :, -3:-1]), ("cube2", cube2[:, -3:-1])],
-        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[-3:-1]))),
+        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[1:3])),
 
     ((slice(None), slice(1, 2)), cube_collection, NDCollection(
         [("cube0", cube0[:, :, 1:2]), ("cube1", cube1[1:2]), ("cube2", cube2[:, :, 1:2])],
-        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[:, 1:2]))),
+        aligned_axes=aligned_axes, meta=cube_coll_meta.slice[:, 1:2])),
 
     ((slice(2, 4), slice(-3, -1)), cube_collection, NDCollection(
         [("cube0", cube0[:, 2:4, -3:-1]), ("cube1", cube1[-3:-1, :, 2:4]),
-            ("cube2", cube2[:, 2:4, -3:-1])], aligned_axes=aligned_axes, meta=cube_coll_meta.slice[2:4, -3:-1]))),
+            ("cube2", cube2[:, 2:4, -3:-1])], aligned_axes=aligned_axes, meta=cube_coll_meta.slice[2:4, 2:4])),
 
     ((0, 0), cube_collection, NDCollection(
         [("cube0", cube0[:, 0, 0]), ("cube1", cube1[0, :, 0]), ("cube2", cube2[:, 0, 0])],
-        aligned_axes=None, meta=cube_coll_meta.slice[0, 0]))),
+        aligned_axes=None, meta=cube_coll_meta.slice[0, 0])),
 
     (("cube0", "cube2"), cube_collection, NDCollection(
         [("cube0", cube0), ("cube2", cube2)], aligned_axes=(aligned_axes[0], aligned_axes[2]), meta=cube_coll_meta)),

ndcube/tests/test_ndcube.py

@@ -1,5 +1,5 @@
-import copy
 import re
+import copy
 from inspect import signature
 from textwrap import dedent
 

ndcube/tests/test_ndcubesequence.py

@@ -203,5 +203,5 @@ def test_slice_meta(ndcubesequence_4c_ln_lt_l_cax1):
     expected_meta = NDMeta({"salutation": "hello",
                             "exposure time": u.Quantity([2] * 4, unit=u.s),
                             "pixel response": u.Quantity([100] * 4, unit=u.percent)},
-                           axes={"exposure time": 0, "pixel response": 0})
+                           axes={"exposure time": 0, "pixel response": 0}, data_shape=(4, 2, 4))
     helpers.assert_metas_equal(sliced_seq.meta, expected_meta)