refactor implementation of scale/offset, unsigned conversion, and convert missing #1163
Conversation
haileyajohnson
force-pushed
the
scale-offset
branch
from
f62711b
to
36c0790
Compare
|
I'm currently investigating a performance issue in TDS serving up an NDVI dataset encoded with add_offset and scale_factor. I suspect some sort of memory leak or at best superfluous memory usage. I haven't looked through this PR yet, but do you suspect it might address those issues? Is it in a state where I could try it out myself and see if the problem is better or worse? |
| Array data = v.read(); | ||
| assertThat((int[]) data.copyTo1DJavaArray()).isEqualTo(expected); | ||
| } | ||
| } |
There was a problem hiding this comment.
One test I'd be curious to see is if I take values that are from -1 to 1 and encode them as unsigned 8 bit integers (so, scale_factor = 0.008 and add_offset = -1), will the decode process give me back an array of floats rather than doubles. It isn't immediately clear to me that it would do that given that internally the scale and offset are stored as doubles.
There's more changes to follow this PR that should fix some of the performance issues, I'd hold off on testing for a bit. |
haileyajohnson
force-pushed
the
scale-offset
branch
from
36c0790
to
dd11979
Compare
haileyajohnson
force-pushed
the
scale-offset
branch
from
bd4124c
to
3f412b3
Compare
…df-java into scale-offset
haileyajohnson
force-pushed
the
scale-offset
branch
from
72f1815
to
c2c9b74
Compare
haileyajohnson
force-pushed
the
scale-offset
branch
from
c4c7ea1
to
5eabf3a
Compare
cdm/core/src/main/java/ucar/nc2/dataset/EnhanceScaleMissingUnsigned.java
Show resolved
Hide resolved
Co-authored-by: Tara Drwenski <[email protected]>
Co-authored-by: Tara Drwenski <[email protected]>
Co-authored-by: Tara Drwenski <[email protected]>
…df-java into scale-offset
|
This should be g2g now :) |
No description provided.