Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor implementation of scale/offset, unsigned conversion, and convert missing #1163

Merged
merged 29 commits into from
Jul 6, 2023
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
212ee25
add default values for scale and offset params
Oct 3, 2022
acf1ac4
refactor scale, missing, and unsigned implementations
Dec 23, 2022
a2d9fe2
add tests
Mar 27, 2023
dd11979
fix bugs
Mar 28, 2023
bd4124c
apply fill value without using enhance modes
Apr 28, 2023
6e9e909
add default values for scale and offset params
Oct 3, 2022
382db30
refactor scale, missing, and unsigned implementations
Dec 23, 2022
65291f3
add tests
Mar 27, 2023
884444a
fix bugs
Mar 28, 2023
3f412b3
apply fill value without using enhance modes
Apr 28, 2023
c83b700
Merge branch 'scale-offset' of https://github.com/haileyajohnson/netc…
Apr 28, 2023
c2c9b74
onlyy convertMissing for floats and doubles
May 2, 2023
4ed4c83
fix test
May 8, 2023
4f033f0
fix application of convertUnsigned
May 9, 2023
29ede73
fix misplaces hasMissing flag
May 9, 2023
5eabf3a
Merge branch 'maint-5.x' into scale-offset
May 9, 2023
4112976
fix typo and formatting
May 10, 2023
6f595ab
preserve orgDataType if no orgVar
May 10, 2023
8424040
Merge branch 'maint-5.x' into scale-offset
Jun 28, 2023
bc4cde0
Merge branch 'maint-5.x' into scale-offset
Jun 30, 2023
4552be7
propagate enhanceMode to coordinate vars
Jun 30, 2023
9dc7aab
incorporate PR feedback
Jul 3, 2023
aa203e7
add rounding back in
Jul 3, 2023
abbfbf5
Update cdm/core/src/main/java/ucar/nc2/dataset/VariableDS.java
Jul 6, 2023
1e44cb3
Update cdm/core/src/main/java/ucar/nc2/dataset/VariableDS.java
Jul 6, 2023
8627be3
Update cdm/core/src/main/java/ucar/nc2/filter/ConvertMissing.java
Jul 6, 2023
5a56969
PR feedback and ignore DAP4 remote tests
Jul 6, 2023
a141b29
Merge branch 'scale-offset' of https://github.com/haileyajohnson/netc…
Jul 6, 2023
940cc63
revert refactor
Jul 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions cdm-test/src/test/java/ucar/nc2/TestSequence.java
Original file line number Diff line number Diff line change
Expand Up @@ -36,16 +36,12 @@
import org.junit.Assert;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ucar.ma2.*;
import ucar.nc2.dataset.NetcdfDataset;
import ucar.nc2.dataset.NetcdfDatasets;
import ucar.unidata.util.test.category.NeedsCdmUnitTest;
import ucar.unidata.util.test.TestDir;
import java.io.File;
import java.io.IOException;
import java.lang.invoke.MethodHandles;
import java.util.Arrays;
import java.util.List;

Expand All @@ -56,7 +52,6 @@
* @since Nov 10, 2009
*/
public class TestSequence {
private static final Logger logger = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());

@Test
@Category(NeedsCdmUnitTest.class)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ucar.nc2.NetcdfFile;
import ucar.nc2.constants.FeatureType;
import ucar.nc2.dataset.NetcdfDatasets;
Expand All @@ -16,7 +14,6 @@
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import java.lang.invoke.MethodHandles;
import java.util.ArrayList;
import java.util.List;

Expand All @@ -28,7 +25,6 @@
*/
@RunWith(Parameterized.class)
public class TestCfDocDsgExamples {
private static final Logger logger = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
private static final String cfDocDsgExamplesDir = TestDir.cdmLocalFromTestDataDir + "cfDocDsgExamples/";

private static List<Object[]> getPointDatasets() {
Expand Down
4 changes: 2 additions & 2 deletions cdm/core/src/main/java/ucar/nc2/NetcdfFiles.java
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ public static NetcdfFile open(String location, int buffer_size, ucar.nc2.util.Ca
* @param buffer_size RandomAccessFile buffer size, if <= 0, use default size
* @param cancelTask allow task to be cancelled; may be null.
* @param iospMessage special iosp tweaking (sent before open is called), may be null
* @return NetcdfFile object, or null if cant find IOServiceProver
* @return NetcdfFile object, or null if can't find IOServiceProver
* @throws IOException if error
*/
public static NetcdfFile open(String location, int buffer_size, ucar.nc2.util.CancelTask cancelTask,
Expand All @@ -290,7 +290,7 @@ public static NetcdfFile open(String location, int buffer_size, ucar.nc2.util.Ca
* @param iospMessage special iosp tweaking (sent before open is called), may be null
* @return NetcdfFile object, or null if cant find IOServiceProver
* @throws IOException if read error
* @throws ClassNotFoundException cannat find iospClassName in thye class path
* @throws ClassNotFoundException cannot find iospClassName in the class path
* @throws InstantiationException if class cannot be instantiated
* @throws IllegalAccessException if class is not accessible
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
* <li>Values will be {@link DataType#widenNumber widened}, which effectively reinterprets signed data as unsigned
* data.</li>
* <li>To accommodate the unsigned conversion, the variable's data type will be changed to the
* {@link EnhanceScaleMissingUnsignedImpl#nextLarger(DataType) next larger type}.</li>
* {@link ucar.nc2.filter.FilterHelpers#nextLarger(DataType) next larger type}.</li>
haileyajohnson marked this conversation as resolved.
Show resolved Hide resolved
* </ol>
*
* <h3>Implementation rules for scale/offset</h3>
Expand Down Expand Up @@ -117,7 +117,9 @@
*
* @author caron
* @author cwardgar
* @deprecated use implementations in Filter package
*/
@Deprecated
public interface EnhanceScaleMissingUnsigned extends IsMissingEvaluator {
/** true if Variable data will be converted using scale and offset */
boolean hasScaleOffset();
Expand Down
Loading