Fix/tensor indexing bug #936

[pawel-wolff]

Description

This PR fixes the issue #936.

The bug was caused by the recent upgrade (#932) of the earthkit tensor indexing support from BASIC to OUTER (see https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#indexing-examples).

With the OUTER indexing support, xarray can ask the earthkit backend to sub-select a tensor axis using:

• a single integer
• a slice
• an array of integers or a boolean mask array

The last kind of indexer turned out to be problematic. More specifically, if a field part v of a tensor is a 2D lat-lon array and one wants to subselect it using a pair of indexers of the third kind (e.g. index=([9, 10, 11], [0, 1]), see:

earthkit-data/tests/xr_engine/test_xr_engine_core.py

Line 226 in 8442708

da[:, 0, :, 2, [9, 10, 11], [0, 1]],

), then the numpy-style indexing v[index] (

earthkit-data/src/earthkit/data/core/field.py

Line 488 in 5d2b2af

v = v[index]

)
does not work, because it applies the so-called "advanced indexing" (https://docs.xarray.dev/en/stable/internals/how-to-add-new-backend.html#indexing-examples).

In order to solve this problem, an outer indexing is applied explicitly to sub-select a field in the following methods:

• Field.to_numpy():

  earthkit-data/src/earthkit/data/core/field.py

  Line 488 in 8442708

  v = outer_indexing(v, index)

• Field.to_arrray():

  earthkit-data/src/earthkit/data/core/field.py

  Line 534 in 8442708

  v = outer_indexing(v, index)

• Field.data():

  earthkit-data/src/earthkit/data/core/field.py

  Line 618 in 8442708

  v = outer_indexing(v, index)

Note that in the case of similar methods:

• NumpyNDArrayWrapper.to_numpy()
• XarrayDataArrayData.to_numpy()

the indexing for a field is still v[index], so the usual numpy-style indexing applies (which might be an advanced indexing, depending on index).

Several tests has been added or extended.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[sandorkertesz]

@pawel-wolff, thank you for this great improvement.