Skip to content

Commit

Permalink
Fix SIDD 3.0 validation error with J2K compression (#741)
Browse files Browse the repository at this point in the history
* copy six.sidd to six.sidd30

* six.sidd30 project and unittests

* sidd -> sidd30

* more sidd -> sidd30

* more sidd -> sidd30

* begin removing SIDD 3.0 support from six.sidd; it is now (or will be) just in six.sidd30

* remove more SIDD 3.0 unittest code from six.sidd, it's not just in six.sidd30

* remove last? traces of SIDD3.0 from six.sidd

* tweaks from feature/ISM

* latest from coda-oss and nitro

* Squashed 'externals/coda-oss/' changes from 1582c6ac9..aabc5818e

aabc5818e remove more C++11 work-arounds (#642)
b04ccca17 remove more C++11 work-arounds (#641)
e460970a7 increase GCC warnings (#640)
91284249b provide ComplexView iterator support (#639)
453b7ec68 try to get std::make_unique working with old Intel compiler (#638)
5736cfd4d is_trivially_copyable for old Intel compiler (#637)
7c2aea38f use H5 SDK naming conventions (#636)
fc3008b8d use 'modern' C pre-processor
b44709a95 added ComplexSpansView utility (#635)
986307c9c Merge branch 'master'
e1b0fa349 commit (but not build) HDF5 source code (#634)
3fd5ddae5 Merge branch 'main' into develop/remove-cpp11-workarounds
37d7ec76e restore C++14 workarounds for older compilers (#632)
244dfdb3d Merge branch 'develop/main' into develop/remove-cpp11-workarounds
714ca4ed7 restore C++14 workarounds for older compilers
640f2247c Delete hdf5-hdf5-1_13_2.tar.gz
eb365cbfb Merge branch 'master' into develop/remove-cpp11-workarounds
57b938eeb don't need __INTEL_COMPILER work-around with C++14
65779f3b1 build C code with -std=gnu99
94dc9280f C++14 work-arounds for an old version of the Intel compiler
3744d05e2 slam-in "main"
82b2fd63b restore some C++14 work-arounds for old compilers that only have partial C++14 support
94f6950a2 Merge branch 'main' into develop/remove-cpp11-workarounds
1ac97fe48 add HDF5 source code (not built) (#612)
8ab982820 Squashed commit of the following:
31781a14a remove C++17 work-arounds
c15f6930c Revert "<filesystem> is in C++17"
0fe38c16f restore recent buld.py changes
0c75411ee <filesystem> is in C++17
bbbcd562a remove C++11 work-arounds
c1e15e9d9 Merge branch 'main' into main-cpp17
bea818fb0 must use C++17 in this branch
91653a096 another place where C++17 is set
7d555f2be back to C++17
b1969d996 merge main
0792c77bf Merge branch 'main' into develop/remove-cpp11-workarounds
e49f273c1 Merge branch 'main' into develop/remove-cpp11-workarounds
240b61044 allow sys::Mutex references, instead of pointers, to be passed to sys::ConditionVar
9500d6f4b Merge branch 'main' into develop/remove-cpp11-workarounds
f1529942f std::make_unique instead of new
be9b6a71c Squashed commit of the following:
62a2ec4e1 Revert "remove our GSL work-around for C++11"
1db082c21 Revert "try the current "main" of GSL"
44b70edde try the current "main" of GSL
4edbf6bad Revert ""There is no warning 4814""
bcd7458be "There is no warning 4814"
c185bf3f1 starting working on MutexCpp11
313bdd1d9 AtomicCounter* files have been removed
c1c448051 remove our GSL work-around for C++11
37ae1193e remove our mem::SharedPtr, it's now always std::shared_ptr
71f3c1299 remove no-longer-used NSPR code
8e77cb210 removed no-longer-used Irix and Solaris files
fabef6966 use std::atomic for all AtomicCounter implementations
ace5b6d5a TEST_FAIL -> TEST_FAIL_MSG
30076c1d1 turns out there was already a TEST_FAIL macro
d73ee16fb need "::" to find global test
ba01c48df remove C++11 workarounds for C++14 features
cbd4a31ae Squashed commit of the following:
b21c203b8 Squashed commit of the following:
5b542c60f Squashed commit of the following:
cb0cf94ff ByteStream.cpp is now warning-free
f5daa37fc cast to DWORD; "const" correctness
bfaf644ff #pragma-away compiler warnings from standard headers
72d202dbe Merge branch 'master' into develop/reduce-compiler-warnings
b076580cd trying to fix GCC compiler warning
1687dc7fc Merge branch 'master' into develop/reduce-compiler-warnings
2c5db86db Merge branch 'master' into develop/reduce-compiler-warnings
6f76c2e3e Update CMakeLists.txt
c3ca4a538 do in-line initialization
8e6733618 remove G++ compiler warning
a0eff51cf Merge branch 'master' into develop/reduce-compiler-warnings
af19d88a6 Merge branch 'master' into develop/reduce-compiler-warnings
ba31cbaa9 use "void*" to avoid reinterpret_cast<>
08da1f8a1 Merge branch 'master' into develop/reduce-compiler-warnings
d1d453a2e Merge branch 'master' into develop/reduce-compiler-warnings
f5074a996 CODA_OSS_NDEBUG is extra gunk and not used; omit for now.
22de48858 don't even bother compiling our own versions of localtime_s() and gmtime_s() if we're not going to use them (compiler warning about "unused static funtion.")
0c92eba58 tweak NDEBUG/CODA_OSS_NDEBUG; update comments
544604775 Don't need to expose our own versions of localtime_s() and gmtime_s()
a050796c1 Merge branch 'develop/reduce-compiler-warnings' of github.com:mdaus/coda-oss into develop/reduce-compiler-warnings
fed604617 older compilers want std::string() as argument to std::logic_error()
cf2c502a6 do more debug & release builds on GitHub
d4aa99256 build some debug targets on Gitlab
368245258 tweak CODA_OSS_DEBUG again for G++
24ba64f71 tweak how CODA_OSS_DEBUG is set
9680e8f0e add check for _GLIBCXX_DEBUG
943218f83 Merge branch 'master' into develop/reduce-compiler-warnings
01c68e705 %#@*@%)*%@ fix RowCol::cast() again! :-(
b8472b457 oops ... gsl::narrow<> should be gsl::narrow_cast<> so that int/double conversion doesn't throw
109255ca8 "final" breaks SWIG; wrap RowCol use of gsl::narrow
e51e4c2ad no need to create local just to return
e546f2510 Merge branch 'master' into develop/reduce-compiler-warnings
d1391f989 use template for to_std_string() overloading
566942680 fix std::char8_t when building with C++20
cf604b1ab fix C++17 build errors
48a0fbedb get rid of coda_oss namespace; use CODA_OSS_cpp17 instead
9706a4336 remove code-analysis diagnostics
50592e4f8 G++ complains about an unneeded call to std::move()
9c7643c3f Merge branch 'master' into develop/reduce-compiler-warnings
ecfe1f8a7 more "noexcept"
b7ac50fb9 Merge branch 'master' into develop/reduce-compiler-warnings
eaecf46ed tweak CODA_OSS_cplusplus from "main"
5773b1ec1 trying to get C++17 WAF build working on Windows
a048a00a1 unique names for waf & CMake
7a5a0d5b2 do a better job of setting CODA_OSS_DEBUG
4355aae92 Merge branch 'master' into develop/reduce-compiler-warnings
1fd8cede2 cleanup for VS 2019 16.10
4dffa1394 explicitly =delete copy/assignment
df8f8920a Merge branch 'master' into develop/reduce-compiler-warnings
42b4e99b6 remove code-analysis diagnostics
70ce322c9 reduce compiler warnings/code-analysis diagnostics when building six
20eac2f49 Merge branch 'master' into develop/reduce-compiler-warnings
ff3ded56d get rid of more compiler warnings/code-analysis diagnostics
669dd51ff Merge branch 'master' into develop/reduce-compiler-warnings
623b65ab9 Merge branch 'master' into develop/reduce-compiler-warnings
11c75df53 fix release build errors
66f048b92 turn on more compiler warnings where possible
eb615e315 consolidate calls to mem::make::unique
a6f5eb690 reduce use of explicit new
84d5e1b01 use mem::make::unique instead of explicit new
019d726ea = default for default constructors
86ab5d8f3 use std::unique_ptr and "const noexcept"
b071972d7 another batch of =default for default constructors
98276da94 more =default for default .ctor()s
ceb7bc596 do default constructors with =default
7bed51750 more code-analysis cleanup
f9c04b117 "noexcept" code-analysis warnings
6586ae390 need gsl::make_span from gsl/span_ext
722089774 remove gsl/mem dependency cycle
d741a0761 use gsl::narrow<> instead of static_cast
99eb4b36e get rid of signed/unsigned mismatch compiler warning
814450522 remove more code-analysis diagnostics
c8441b309 Merge branch 'master' into develop/reduce-compiler-warnings
e6d70a9ff last of "const" from code-analysis
b8445814a mark a few more things as "const" per code-analysis
d18544ec7 don't need to create temporary std::strings
ee7faa979 still more code-analysis cleanup
5cd6f864d cleanup more code-analysis diagnostics
492fb0a86 one more compiler warning about implicitly deleted copy/assignment
b8f782c88 remove compiler warnings about implicitly deleted copy/assignment/move
14d1d1cb8 #pragma-away warnings from GSL
86c40ea6c assert() to prevent some code-analysis diagnostics
7204517a9 make copy/assignment explicit
3ce824079 reduce code-analysis diagnostics
01e367a56 Merge branch 'master' into develop/reduce-compiler-warnings
03f3ddde9 proper printf format
877d7dc93 Merge branch 'master' into develop/reduce-compiler-warnings
5a4a3d575 code-analysis doesn't like C-style casts
a3fdae416 code-analysis doesn't like NULL
6309baedc Merge branch 'master' into develop/reduce-compiler-warnings
d1194a2fe Merge branch 'master' into develop/reduce-compiler-warnings
535b4757d Merge branch 'develop/reduce-compiler-warnings' of github.com:mdaus/coda-oss into develop/reduce-compiler-warnings
a60b95d60 "constexpr" can be hard to get right with older C++11 compilers; don't even try
c07b5303e Merge branch 'master' into develop/reduce-compiler-warnings
04306bbee need at least VS2017 to use the real GSL
796ad820b Merge branch 'master' into develop/reduce-compiler-warnings
16c240cd6 Merge branch 'master' into develop/reduce-compiler-warnings
e07e89f0c turn on "warnings as errors" for CMAKE builds
5c019f891 complex "constexpr" functions not available with our version of g++
4ab074a58 fix a few more compiler warnings ... nearing the end! :-)
dea7ea41b And still more compiler warnings ... almost done 🤞🏻
fbe3bbb20 remove assorted compiler warnings
e60b7fd3c Merge branch 'master' into develop/reduce-compiler-warnings
e9df212c6 Arguments should be "const&" to prevent C4866
94e75cbed Merge branch 'master' into develop/reduce-compiler-warnings
d3130414a Merge branch 'master' into develop/reduce-compiler-warnings
9a5e71f31 Merge branch 'master' into develop/reduce-compiler-warnings
bd27c0232 Merge branch 'master' into develop/reduce-compiler-warnings
c053362d2 remove still more compiler warnings
9f133b85a remove some more compiler warnings
0883c729c Merge branch 'master' into develop/reduce-compiler-warnings
f4b766880 avoid comparision between signed/unsigned to prevent compiler warning
1e6564a6e eliminate more compiler warnings
85a7b8ad0 remove a few compiler warnings
09c82049f Merge branch 'master' into develop/reduce-compiler-warnings
31f1662fa Merge branch 'master' into develop/reduce-compiler-warnings
a313562a3 Merge branch 'feature/fix-compiler-warnings' into develop/reduce-compiler-warnings
8547cdd24 eliminiate GCC warnings from Filesystem.h
b5044b2a2 fix several compiler warnings/code-analysis diagnostics
9d2f90b8c GetEnvironmentVariable() and getenv() aren't quite the same
6faa3666f remove more compiler warnings

git-subtree-dir: externals/coda-oss
git-subtree-split: aabc5818e651be550c0fcc53be6f94902c61905e

* Squashed 'externals/nitro/' changes from e1a64da1f..24ff32863

24ff32863 -j 6 seems to work for SIX, how about NITRO?
713ee8538 -j 6 seems to work for SIX, how about NITRO?
4ccdf5b37 latest from coda-oss (#527)
c71acc492 latest from coda-oss (#526)
08cbb96f9 build with older C++14 compilers (#524)

git-subtree-dir: externals/nitro
git-subtree-split: 24ff328636e3acbf1745e49df45b787403ecb113

* latest from coda-oss and nitro

* use common code from six.sidd

* consistent convertDerivedClassificationToXML() API

* use common code from six.sidd

* compile SIDD30 Utilities.cpp

* get six.sidd30 unittests working

* remove files duplicated between six.sidd and six.sidd30

* build with CMake

* SIDD_V3.0.0 directory in "schema"

* easy access to test XML files from Visual Studio

* ValidatorXerces can be moved, but not copied

* fix SIDD version bug when generating fake data

* set things up for different schema paths for SIDD 3.0 (not yet implemented)

* more detailed output from schema validation failure

* SIDD_V3.0.0

* SIDD 3.0 schemas are in the SIDD_V3.0.0 directory

* no ISM-v201609 in this branch

* IonoParameters/F2Height is optional

* remove six.sidd30

* remove SIDD_V3.0.0

* slam in 'main'

* ISMVersion enum for 201609 and 13

* need to be able to set the ISM version in more places

* start rippling changes for ISMVersion

* add a "current" to make it easy for developers to switch between V201609 and v13

* change the ISM namespace

* restore SIDD-3.0_ISM-v13 to a separate directory

* Create sidd300_ISM-v13.xml

* wrap the xml::lite::validator in our own class

* pass a XmlLiteValidator instance

* xml.lite changes from coda-oss

* despite various checks for a NULL logger, it really can't be NULL

* use xml::lite::Validator directly

* move ISMVersion to the a sidd300 namespace to make it clear it's part of SIDD 3.0

* got a scoping issue somewhere :-(

* unittest for ISM-v13

* hack to read ISM-v13 SIDD from unittest code; WIP

* don't need make_ utility, at least right now

* log_errors_and_throw() utility

* prepare for validating a schema-path at a time

* latest xml.lite tweaks from coda-oss

* "new" code does XML validation one path at a time

* adjust schema paths if ISMVersion is set

* automatically prepend the ISM schema path

* remove ISMVersion from DerivedXMLControl

* allow the hard-coded "SIDD_V3.0.0_ISM-" name to be changed

* first pass at finding the corresponding XSD for a XML

* add a "hook" so that we can intercept validate() in SIDD code

* provide standard validateXMLImpl() implementation

* Revert "first pass at finding the corresponding XSD for a XML"

This reverts commit 6bb5937.

* move schema finding code to six.sidd

* new code only needs to work with std::vector<path>

* If we found the right XSD, use that directory first.

* Revert "allow the hard-coded "SIDD_V3.0.0_ISM-" name to be changed"

This reverts commit 8055430.

* always validate one schema-path at a time

* try not to open so many XSDs

* getEnvIfSet()

* look for SICommonTypes_V1.0.xsd to find the schema directory

* "hook" for validating created XML

* find_SIDD_schema_V_files() utility

* put the hard-code SIDD schema name in one place

* copy ISM-v201609 files to separate directory

* There is now a separate directory for each SIDD 3.0 ISM

* latest from coda-oss

* Squashed 'externals/coda-oss/' changes from e87c32b4d..ff4f820ed

ff4f820ed xml.lite tweaks to support SIDD 3.0 ISM (#672)
b1de8c0e5 std::byte should be a unique type (#671)
c05bf9a02 allow enums to be byte-swapped too
1f9fd88d6 remove spurious
47684c45b byteSwap now uses byte buffers (#670)
cbc659db2 add swapBytes() utility from SIX (#669)
891481b64 simplify byte-swapping (#668)
540ae763e more byteSwap() tweaks
0774c03c4 threaded byteSwap() (#667)
d156370d3 swapping a single-byte value makes no sense
c120e3255 be sure parameter is used to avoid compiler warning
e85ec9331 --output-on-failure for CTest (#666)
e80376197 turn off "there is no warning number" warning
c5f0a5d15 A C-string may not be NULL-terminated (#665)
0c5eb29ae use platform-specific routines for byteSwap() (#664)
0b7d581fa remove transform_async() (#663)
f6489b6be Merge branch 'main' into feature/xml.lite_tweaks
836c426a2 use function-pointers so that isConsoleOutput is only checked once
56e3c45b1 move depthPrint() functionality into non-member function in preparation for future changes
ddcd26d97 Merge branch 'main' into feature/xml.lite_tweaks
69cc0e506 use the more rigorous create_and_check_datatype()
9efb87558 Merge branch 'main' into feature/hdf5
4d2f2f417 more HighFive unittests (#662)
14191a844 HighFive::create_datatype() goes from C++ to HighFive
98583473f utility routines to read string attributes
1fa75ce81 use the C API to read a string attribute
71e7b69f5 still can't figure out how to read a string attribute :-(
e96f37a69 test reading the file attributes
a25244519 getAttribute() unittest
8f12a3000 getDataType() unittest
857ff0af3 HighFive utility routines (#661)
1d687db57 writeDataSet() utility overload
106aa6894 sigh ... H5Easy::dump() fill fails on Windows/WAF :-(
2641b60b2 Merge branch 'main' into feature/hdf5
35c19e7e8 change actions to @V3 (#660)
212bbd3a3 works on local machine, but not build server ... ?
7125118b0 dump of 1D vector doesn't work :-(
c704db435 sigh ... WAF build still failing :-(
01aae4616 does dump() of a 1D vector work?
76a53c813 comment-out H5 writing :-(
0f0e19aff test_highfive_dump()
cb8f73795 trying to get highfive_dump() unittest working w/Windows-WAF
6584a264a does test_highfive_create() work?
cde6147ce tweak HighFive wrappers (#659)
316566854 Revert "trying HighFive "write" unittests again"
a9ec24ca4 trying HighFive "write" unittests again
585ad49a5 tweak names of utility routines
4c91a4d97 make it easier to read a std::vector and std::vecotr<std::vector>
0217ffa26 readDataSet() now works for 1D data
95e8973f2 trying to get hdf5::lite::load() working
4d294611d "const" correctness
5e6305c3f fix load_complex()
4a134dc5f start work on utility routines to read complex data from HDF5
9d76a7f41 Merge branch 'main' into feature/hdf5
8f9667a24 whitepsace
ebd3fc99e Merge branch 'main' into feature/hdf5
40091b069 comment-out writing tests for now ... need to figure our WAF bulid failure
935aa3459 be sure the dataset has real data
114b9bf33 update release notes
bd9c0b26c tweak HighFive utility routines
acda1ef57 turn off diagnostics around expected failures
22a748840 readDataSet() utility routine for HighFive
bd88a8c25 HighFive writeDataSet() utility to work with our SpanRC
6142f5b33 use HighFive routines to write a HDF5 file
5bbf1abaf Use HiveFive routines to get info about the file
84fbc8378 duplicate unittests with H5Easy
f1f054c03 Merge branch 'main' into feature/hdf5
9b63ca470 fix directory names
f6f826689 fix directory names
7aeb82c33 Merge branch 'main' into feature/hdf5
d028baaeb hook up HighFive header-only library (#653)
3083b0a31 Revert "HighFive 2.6.2"
246985a7f Revert ""build" HighFive HDF5 library"
a8b75a586 Revert "turn off HighFive Boost support"
ec68d5f83 Revert "Add HighFive unittests"
f1f85b9e7 Revert "get test_high_five_base more-or-less compiling"
5ea634ee1 Revert "more work on getting HighFive unittests to build"
ecc45433c more work on getting HighFive unittests to build
bb194788a get test_high_five_base more-or-less compiling
d42bde000 Add HighFive unittests
ddc86bb32 turn off HighFive Boost support
b255122d4 "build" HighFive HDF5 library
396cc3ef2 HighFive 2.6.2
5e5f9d9c0 Merge branch 'main' into feature/hdf5
ee938b4a5 changes from SIX
bb764df90 Merge branch 'main' into feature/xml.lite_tweaks
de2a24380 make derived classes 'final' if possible
14e19bcd2 Change xml lite function to virtual (#645)
8f42ac8e9 Merge branch 'main' into feature/xml.lite_tweaks
18ad90645 hdf5Write unittest
3462e1179 createFile() and writeFile() overloads
ecee81d53 fix typos
197eecfa6 sketch-out hdf5::lite::writeFile()
bd2311795 use SpanRC for writeFile(), not yet implemented
ea9af7510 simple SpanRC to hold a 2D-size and pointer
027c19ee8 createFile() unittest
1f9d07ecb hook up createFile()
8c7e4473f start hooking up HDF5 writing
146e0bea3 Merge branch 'main' into feature/hdf5
88ca9fcb7 Merge branch 'main' into feature/hdf5
42b604b46 Squashed commit of the following:
10ee602c2 Merge branch 'main' into feature/hdf5
67aa42b69 restore changes from "main"
8bbfcbfbf unittests can be simplified to match fewer "view" classes
126bb802e Merge branch 'main' into feature/hdf5
3f8ba7a42 again, don't need a class just to convert from std::vector<> to std::span<>
24c2b489c Squashed commit of the following:
2703c119d Squashed commit of the following:
9d5228a2b don't need an entire class just to convert a std::vector<> into std::span<>
51bc931dc Merge branch 'main' into feature/hdf5
a84f25816 Squashed commit of the following:
c4d2ed696 add missing #include guards, fix type in existing #include guard
d541525a0 use a single ComplexViewConstIterator for all views
86e6a459f CODA_OSS_disable_warning causes GCC errors :-(
5d4b9c2cb only need an custom iterator for ComplexSpansView
d9f0fb128 hook up iterators
b9329e4db initial pass at a ComplexViewConstIterator
635238873 remove compiler warning about unused "constexpr" variables
b39f6096f use the casing from H5 to make copy/pasting code slightly easier
0887b13eb Merge branch 'main' into feature/hdf5
bd07df1ca Consistent casing for Dataset, Datatype, Dataspace
7acd30ee2 tweak hdf5.lite dependencies
38ab914df Jupyter notebook for creating H5 files
95a040e0b _small.h5 is now (correctly) FLOAT32
107e7c487 make a simple values() member function to avoid template magic
e1feca919 use TEST_SPECIFIC_EXCEPTION macro instead of try/catch
738333688 readDatasetT() now throws for the wrong buffer type
6b2cc2529 Merge branch 'feature/hdf5' of github.com:mdaus/coda-oss into feature/hdf5
310f8fd3d can't get template magic right for copy_axis()
86b306d59 stepping through copy ctors in the debugger is annoying
f243e92d6 trying to make wrong type of buffer fail
2b10d9652 read in new sample file
a28e59d8c help the compiler with type deduction
49bf5e9bc nested_complex_float_data_small.h5
e029325fc utility routines to "deconstruct" and array of std::complex
dede3bd39 Merge branch 'main' into feature/hdf5
904b1ef5e tweak class names, make_() and copy() utility routines
8237b9efb make it harder to pass the wrong types to ComplexViews
4d9aeda2c ComplexArrayView and ComplexParallelView utility classes
f5e367dfa test std::span<const std::complex<float>>
a4a2844f2 read in the nested "i" and "r" data
115615265 sample file has subgroups
8e1b7869a Merge branch 'feature/hdf5' of github.com:mdaus/coda-oss into feature/hdf5
9f4232a1d update sample H5 file
8c55db73a walk through HDF5 sub-groups
7775ed9c4 Update 123_barfoo_catdog_cx.h5
677975d7c Matlab code to create sample H5 file
a0e7dfe07 Update test_hdf5info.cpp
0b67e1602 pass __FILE__ and __LINE__ from calling site for a more accurate exception message
86a677321 skeleton for more sample data
85f79b099 Merge branch 'main' into feature/hdf5
18088e942 Merge branch 'main' into feature/hdf5
3a1d17692 Merge branch 'main' into feature/hdf5
1755c69d7 Merge branch 'main' into feature/hdf5
9ad015432 No more "11" suffix on exception names
c20d96251 Squashed commit of the following:
c88cee999 other values to be filled-in
bca4a4ecd incorporation NamedObject from HDF5 docs
61fa68f72 groupInfo()
460e7d766 datasetInfo()
14eb9b764 start filling in DatasetInfo
afe5f1c3a start to fill in DatasetInfo
77a968c72 start filling in GroupInfo
d81bcdfd9 openGroup() to open groups (loc)
a0cd29469 comment-out "dataset" unittest for now
86e006024 begin filling in FileInfo
366dda6ab a return_type_of utility is needed to deduce the return type
e21928263 explicitly pass return type to template
4937ccd11 template to reduce boilerplate when calling try_catch_H5Exceptions
b3b5ebde7 use new exception utility routines
ea1c03ef0 put exception handling/conversion in a utility routine
819a99d39 utility routine for exception handling
6f34eea97 put utilities in a separate file for easier reuse
fcbde4f24 break utility routines into smaller pieces for easier reuse
52358ea8a WIN32 no longer automatically defined?
5a4286472 Revert "build HDF5 with C89"
680e599e9 build HDF5 with C89
a87a07121 Merge branch 'main' into feature/hdf5
8447c1a90 Revert "sym-links instead of copying files"
db3b5e12b Merge branch 'main' into feature/xml.lite_tweaks
fb60b5696 Merge branch 'main' into feature/hdf5
5110a5cc8 Comments about _u and _q
1a937d32c Merge branch 'main' into feature/xml.lite_tweaks
fa06f04d7 get ready for hdf5.lite enhancdements
b040c7c43 sym-links instead of copying files
aa431bb47 use _u for xml::lite::Uri
3d0c6d58c fix case-sensitive #include filename
93dcd0e52 operator() for getElementByName()
75a93af85 more operator[] overloads to make attribute management easier
4ab8216f8 user-defined string literals to remove some noise around xml::lite::QName
f82f0b0fc Merge branch 'main' into feature/xml.lite_tweaks
ae30e3644 Merge branch 'feature/xml.lite_tweaks' of github.com:mdaus/coda-oss into feature/xml.lite_tweaks
ffdd9beb0 simplify attribute creation
9bf5414f5 simplify attribute creation
82d7a4e95 SWIG gets confused about namespaces
7a61d0741 fix bug on Element ctor uncovered by unittest
fdd7e58c1 QName is also in the xerces namespace which confuses SWIG bindings
a325b7053 operator+=() overload for addChild
daf30e6c0 Merge branch 'feature/xml.lite_tweaks' of github.com:mdaus/coda-oss into feature/xml.lite_tweaks
b887d2b47 provide overloads for Element& rather than creating new "reference" types
1fa6bba38 rename test_xmleasy.cpp
7c8c9e0f1 += overload
850da6f63 overload for std::string
4547fc5a7 use UIT-8 strings for characterData
4723462a3 convenient addChild() overloads
e48720753 copy over ElementReference from xml.easy
a4ca30a0d Merge branch 'main' into feature/xml.lite_tweaks
6ae9f0b71 Revert "check-in of new xml.easy (to move code between computers)"
f7466a6d7 Revert "simple routines for single element"
a5490230d Revert "make some operators simplier ways of calling functions"
c9a25630a Revert "get document creation working"
8af8710b0 Revert "free functions instead of member functions"
16c3847cb Revert "ElementReference distinct from Element"
7d68e156f Revert "ElementMutableReference"
00eb2a282 Merge branch 'main' into feature/xml.lite_tweaks
a42969c1f ElementMutableReference
a20ae9355 ElementReference distinct from Element
14eeeea0b free functions instead of member functions
4aae014b3 get document creation working
883569269 make some operators simplier ways of calling functions
053bd1212 simple routines for single element
8bf701a2e check-in of new xml.easy (to move code between computers)
41f959051 unittests for creating XML documents from scratch
9752d50ae Merge branch 'main' into feature/xml.lite_tweaks
1531d5709 by default, don't validate strings passed to Uri()
46d13d4bf Merge branch 'master' into feature/xml.lite_tweaks
39b547d32 remove more vestiges of Expat and LibXML
ec8274d52 remove LibXML and Expat as they're no longer used/supported.
20eeefeef Merge branch 'master' into feature/xml.lite_tweaks
95074b9b1 update for newer Intel compiler
7024f71e1 Merge branch 'master' into feature/xml.lite_tweaks
57b1cbc83 Merge branch 'master' into feature/xml.lite_tweaks
4b67561c3 remove validate() overload that nobody is using
fa15f1e5d Squashed commit of the following:
1484a9090 test the new validate() API
470da70fb hookup StringStreamT routines
2cddf2504 begin hooking up validate() overloads
1b5d910f3 overload validate() for UTF-8 and Windows-1252
03309b8c9 Squashed commit of the following:
b72c6c5bf older compiler doesn't like our make_unique
af8f00307 validate UTF-8 XML on Linux
211188613 unit-test for LEGACY XML validation
3c1169d2b Squashed commit of the following:
3afff19ca std::filesystem::path for FileInputStreamOS
908d452f8 WIP: validate all of our sample XML files
00f9bb16b validate against a XML schema
243d8c356 Merge branch 'master' into feature/xml.lite_tweaks
2815d707d fix to work with SWIG bindings. :-(
460862132 trying (again) to remove vestiages of old code
e3c83a858 Revert "new code should use UTF-8"
811207c92 new code should use UTF-8
0ffd835f9 Squashed commit of the following:
1e7e03ded Merge branch 'master' into feature/xml.lite_tweaks
c1d806aff Merge branch 'master' into feature/xml.lite_tweaks
850d3c811 str::strip() that can be easier to use than str::trim()
580ba9c8c explicitly =delete move
2b39831a8 Squashed commit of the following:
39eebdc23 Merge branch 'master' into feature/xml.lite_tweaks
9adf86cba force calling new UTF-8 write() routines
ea61b6204 Merge branch 'master' into feature/xml.lite_tweaks
8a34583fa overload to take schemaPaths as filesystem::path
8671b442f parse XML embedded in a binary file
ec4a902f1 updates from xerces.lite
80dc4d963 updates from xerces.lite
549766d6c Attributes::contains() no longer catches an exception
8a645ceac need "sys/" when building in other environments
36af08269 super-simple URI validation
78ef28a3e SWIG bindings are a PITA! :-(
e9cba8491 SWIG needs help with Uri
8a8d8dc07 another routines used by pre-build SWIG bindings
818e1ec5d pre-build SWIG bindings use getElementByTagName() member function
067cac5d8 old compiler gets confused on unadorned QName
ba92c0ae7 more use of Uri and QName
446c7d17a use QName in new code
d6f8b0c83 more direct use of QName
90fff1c73 use xml::lite::QName instead of tuple<string, string>
646cbb5ed more direct use of QName and Uri
ba589ea3b make QName more robust
bab0ee8b5 createElement() -> addNewElement()
e3a145747 grab changes from six-library
32285e95c Merge branch 'master' into feature/xml.lite_tweaks
9f79f0bf6 Merge branch 'master' into feature/xml.lite_tweaks
a12bbc32c make it easier to create new Elements with a value
fc9967f98 make it easy for callers to addChild() keep a reference to the Element
4627766b7 be sure test_xmlparser works in "externals" of other projects
bf2276396 "private" is part of the name-mangling
fad92bcc8 making sure copy-ctor is implemented
f90fdcead consolidate common XML test code
9fc53f2d5 use str:: utility for casting
6da6f794b still trying to find the right macro for SWIG
0c1b86c56 still trying to fix SWIG
fdc6fc9bd trying to fix SWIG build error
7835e8c27 SWIG needs copy-ctor
585695942 disable copy/assignment for Element, it's probably almost always wrong
391fed613 fix double-delete caused by copying
61790fe69 retry parsing XML with Windows-1252 if first parse() fails
63cffac59 change string_encoding to match coda-oss style of PascalCase
010479bbe read an XML file we know is wrongly encoded as Windows-1252
9a0505062 more references instead of pointers
2d44b6951 Reading Windows-1252 w/o "encoding" fails
63dc7b076 read Windows-1252 too
c9434c9cb test as UIT-8 too
f310ccf0c get reading from UTF-8 XML working on Windows
1fa39c2be get testReadUtf8XmlFile working on Linux
1a83cd815 sys::Path is too much trouble right now
ed60aa22c unit-test to read XML from a file
a9336db7c Squashed commit of the following:
0825beb0d Merge branch 'master' into feature/xml.lite_tweaks
c618489be Merge branch 'master' into feature/xml.lite_tweaks
e8e4b8fe1 determine string_encoding based on platform
1f43bcfc2 create a new Element by using the platform to determine "characterData" encoding
961bef66b Merge branch 'master' into feature/xml.lite_tweaks
e9798a5cb fix static_assert()
6f7772874 Merge branch 'master' into feature/xml.lite_tweaks
b98d4f5a9 Merge branch 'master' into feature/xml.lite_tweaks
1b5abba2a The (old) version of SWIG we're using doesn't like certain C++11 features.
53bdeabaf Merge branch 'master' into feature/xml.lite_tweaks
60cf8ae80 "" doesn't work with decltype() in older C++
97e72477a reduce getValue() overloads by making "key" a template argument
5e6373e55 reduce code duplication
f9e7cfeee provide castValue<T> instead of getValue(T&)
cbd0bd8f2 castValue<T> throws instead of returning a bool like getValue(T&)
87c7514fc Merge branch 'master' into feature/xml.lite_tweaks
10cc61223 make getElement*() consistent for zero or >1 results
f5b137e3c Merge branch 'master' into feature/xml.lite_tweaks
1765efc62 allow clients to specify toType() and toString() for getValue() and setValue()
df8b746e1 allow clients to specify their own toType/toString routines
66702726a Merge branch 'master' into feature/xml.lite_tweaks
6956311f1 Merge branch 'master' into feature/xml.lite_tweaks
d505f3593 Merge branch 'master' into feature/xml.lite_tweaks
fbd106115 catch a BadCastException and return false from getValue()
3a78377b5 use a template to reduce duplicated code
0ad4b8606 Merge branch 'master' into feature/xml.lite_tweaks
a848aa3a2 get & set the characer data as a type
f3ee1ee12 utility routines to set an attribute value
595227683 templates to get an attribute value convert to a specific type
06639227b miised a change in last commit
1aa458ef8 add getValue() overloads that return true/false rather than throwing
faa6d3075 added getElementByTagName() overloads as that's a very common use-case

git-subtree-dir: externals/coda-oss
git-subtree-split: ff4f820ed8c406d43e3768508ca06d1a334b831f

* Squashed 'externals/nitro/' changes from c8ecbe9ae..2fd7a0bfa

2fd7a0bfa latest from coda-oss (#552)
0eecce004 invoke() utility to reduce code duplication (#550)
59fb02fe9 latest from coda-oss (#551)
9fbf2b7b8 Fill out adapter free block which is used for nitf decompression (#549)
089ba0b5b latest from coda-oss
3b52f0025 latest from coda-oss (#547)
90c6263e2 latest from coda-oss (#544)
90d513ac5 latest from coda-oss (#543)

git-subtree-dir: externals/nitro
git-subtree-split: 2fd7a0bfa708a93a4bd17e44dbb4eba400b424d2

* toXMLImplValidate() isn't needed (at least not right now)

* simplify validation code by converting to std::filesystem::path rather than using templates

* No longer need ScoppedISMVersion to read ISM-v13 XML

* comments to indicate the attributes for which we're looking

* be sure SIDD XML is really SIDD

* latest coda-oss from xml.lite branch

* use std::vector::insert

* create a SIDD Version enum rather than using strings

* expose ISMVersion in fromXML() and toXML()

* don't need old ISM with SIDD 3.0

* use six::sidd::Version rather than a string for creating fake data

* require ISMVersion for SIDD 3.0.0

* more use of six::sidd::Version

* save ISMVersion for SIDD 3.0.0

* adjust desVersion for ISMVersion

* use the ISMVersion from the environment, if any

* still more use of sidd::Version

* still more use of sidd::Version instead of strVersion

* latest from CODA-OSS and NITRO

* latest from CODA-OSS

* Squashed 'externals/coda-oss/' changes from ff4f820ed..d1244a080

d1244a080 don't need our own make_unique in C++14 (#680)
aeec0131c assert()s for mem::CopyablePtr (#679)
72b0ebd60 add types::complex_short (#678)
932130a58 patch to build other projects
c00c1f203 coda-oss release 2023-06-06 (#677)
ef54bbcd5 remove more compiler warnings (#676)
dadfc5ce6 distinguish between byte-swapping a buffer and single value (#674)
90187f6cd more xml.lite tweaks for SIDD-3.0/ISM (#675)
eb9960772 stronger type-checking for byteSwap() (#673)

git-subtree-dir: externals/coda-oss
git-subtree-split: d1244a080457bb87e03879f743e3b2467145da08

* Squashed 'externals/nitro/' changes from 2fd7a0bfa..05dae18a3

05dae18a3 patch to build other projects
8f974e995 NITRO-2.11.3 (#554)
870aa6afd update to coda-oss 2023-06-05 (#553)

git-subtree-dir: externals/nitro
git-subtree-split: 05dae18a3a9c5ad70e1ce814c21ca8ec278829be

* latest from CODA-OSS

* latest from CODA-OSS

* xml.lite tweaks from CODA-OSS speed things up

* allow schema paths to be found/validated separate from XML validation

* XercesContext can't be "static" (yet)

* only initialize Xerces for multiple ValidatorXerces instances

* use sys::convertPaths() utility

* remove unneeded ValidatorXerces() overloads

* don't expose findValidSchemaPaths() right now; no actual use-case

* account for xml.lite changes in CODA-OSS

* latest from CODA-OSS

* `positiveInteger` -> `nonNegativeInteger` in SIDD 3.0

* convertJ2KLayerInfoToXML() routine so the `index` attribute can be written differently for SIDD 2.0 vs. 3.0

* get rid of convertJ2KLayerInfoToXML(), it's just noise

* convertCompressionToXML() already existed, make it virtual

* already had convertJ2KToXML(), make it `virtual`

* validate the value of the `index` attribute

* fix compiler warning

* update sample SIDD XML with <Compression>
  • Loading branch information
JDanielSmith committed Mar 14, 2024
1 parent ff922e6 commit d178aad
Show file tree
Hide file tree
Showing 10 changed files with 77 additions and 33 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -1442,7 +1442,7 @@
</xs:annotation>
</xs:element>
</xs:sequence>
<xs:attribute name="index" type="xs:positiveInteger" use="required"/>
<xs:attribute name="index" type="xs:nonNegativeInteger" use="required"/>
</xs:complexType>
<xs:complexType name="DigitalElevationDataType">
<xs:annotation>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1442,7 +1442,7 @@
</xs:annotation>
</xs:element>
</xs:sequence>
<xs:attribute name="index" type="xs:positiveInteger" use="required"/>
<xs:attribute name="index" type="xs:nonNegativeInteger" use="required"/>
</xs:complexType>
<xs:complexType name="DigitalElevationDataType">
<xs:annotation>
Expand Down
6 changes: 3 additions & 3 deletions six/modules/c++/six.sidd/include/six/sidd/Compression.h
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ struct J2KCompression
{
Layer();

double bitRate;
double bitRate = 0.0;
};

J2KCompression();

size_t numWaveletLevels;
size_t numBands;
size_t numWaveletLevels = 0;
size_t numBands = 0;

std::vector<Layer> layerInfo;
};
Expand Down
6 changes: 2 additions & 4 deletions six/modules/c++/six.sidd/include/six/sidd/DerivedXMLParser.h
Original file line number Diff line number Diff line change
Expand Up @@ -194,10 +194,8 @@ struct DerivedXMLParser : public six::XMLParser
XMLElem parent = nullptr) const = 0;
XMLElem convertAnnotationToXML(const Annotation *a,
XMLElem parent = nullptr) const;
XMLElem convertCompressionToXML(const Compression *c,
XMLElem parent = nullptr) const;
void convertJ2KToXML(const J2KCompression* c, XMLElem& parent) const;
XMLElem convertSFAGeometryToXML(const SFAGeometry *g,
public: virtual void convertJ2KToXML(const J2KCompression&, xml::lite::Element&) const {}
protected: XMLElem convertSFAGeometryToXML(const SFAGeometry *g,
XMLElem parent = nullptr) const;
XMLElem convertGeographicCoordinateSystemToXML(
const SFAGeographicCoordinateSystem* geographicCoordinateSystem,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,6 @@ struct DerivedXMLParser200 final : public DerivedXMLParser
virtual void parseCompressionFromXML(const xml::lite::Element* compressionElem,
Compression& compression) const;

virtual XMLElem convertCompressionToXML(const Compression& compression,
XMLElem parent = nullptr) const;

virtual XMLElem convertDisplayToXML(const Display& display,
XMLElem parent = nullptr) const override;

Expand Down Expand Up @@ -141,8 +138,7 @@ struct DerivedXMLParser200 final : public DerivedXMLParser
static xml::lite::Element& convertFilterToXML(const DerivedXMLParser&,
const std::string& name, const Filter&, xml::lite::Element& parent);

static void convertJ2KToXML(const DerivedXMLParser&,
const J2KCompression&, xml::lite::Element& parent);
void convertJ2KToXML(const J2KCompression&, xml::lite::Element&) const override;

void parseJ2KCompression(const xml::lite::Element* j2kElem,
J2KCompression& j2k) const;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ struct DerivedXMLParser300 final : public DerivedXMLParser
void parseBankFromXML(const xml::lite::Element* bankElem, Filter::Bank&) const;
void parseFilterFromXML(const xml::lite::Element& filterELem, Filter& filter) const;
void parseJ2KCompression(const xml::lite::Element& j2kElem, J2KCompression& j2k) const;
void convertJ2KToXML(const J2KCompression&, xml::lite::Element&) const override;
void parseGeoDataFromXML(const xml::lite::Element& elem, GeoDataBase&) const;
void parseDigitalElevationDataFromXML(const xml::lite::Element& elem, DigitalElevationData&) const;
void parseProductGenerationOptionsFromXML(const xml::lite::Element& optionsElem, ProductGenerationOptions&) const;
Expand Down
7 changes: 2 additions & 5 deletions six/modules/c++/six.sidd/source/Compression.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,11 @@ namespace six
{
namespace sidd
{
J2KCompression::Layer::Layer() :
bitRate(0.0)
J2KCompression::Layer::Layer()
{
}

J2KCompression::J2KCompression() :
numWaveletLevels(0),
numBands(0)
J2KCompression::J2KCompression()
{
}
}
Expand Down
20 changes: 7 additions & 13 deletions six/modules/c++/six.sidd/source/DerivedXMLParser200.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ xml::lite::Document* DerivedXMLParser200::toXML(const DerivedData* derived) cons
// optional
if (derived->compression.get())
{
convertCompressionToXML(*derived->compression, root);
convertCompressionToXML(*this, *(derived->compression), *root);
}
// optional
if (derived->digitalElevationData.get())
Expand Down Expand Up @@ -1482,32 +1482,26 @@ xml::lite::Element& DerivedXMLParser200::convertFilterToXML(const DerivedXMLPars
return filterElem;
}

XMLElem DerivedXMLParser200::convertCompressionToXML(
const Compression& compression,
XMLElem parent) const
{
assert(parent != nullptr);
return &convertCompressionToXML(*this, compression, *parent);
}
xml::lite::Element& DerivedXMLParser200::convertCompressionToXML(const DerivedXMLParser& parser,
const Compression& compression, xml::lite::Element& parent)
{
auto& compressionElem = parser.newElement("Compression", parent);
auto& j2kElem = parser.newElement("J2K", compressionElem);
auto& originalElem = parser.newElement("Original", j2kElem);
convertJ2KToXML(parser, compression.original, originalElem);
parser.convertJ2KToXML(compression.original, originalElem);

if (compression.parsed.get())
{
auto& parsedElem = parser.newElement("Parsed", j2kElem);
convertJ2KToXML(parser, *compression.parsed, parsedElem);
parser.convertJ2KToXML(*(compression.parsed), parsedElem);
}
return compressionElem;
}

void DerivedXMLParser200::convertJ2KToXML(const DerivedXMLParser& parser,
const J2KCompression& j2k, xml::lite::Element& parent)
void DerivedXMLParser200::convertJ2KToXML(const J2KCompression& j2k, xml::lite::Element& parent) const
{
auto& parser = *this;

parser.createInt("NumWaveletLevels", j2k.numWaveletLevels, parent);
parser.createInt("NumBands", j2k.numBands, parent);

Expand All @@ -1518,7 +1512,7 @@ void DerivedXMLParser200::convertJ2KToXML(const DerivedXMLParser& parser,
for (size_t ii = 0; ii < numLayers; ++ii)
{
auto& layerElem = parser.newElement("Layer", layerInfoElem);
parser.setAttribute(layerElem, "index", ii + 1);
parser.setAttribute(layerElem, "index", ii + 1); // `positiveInteger` in SIDD 2.0; changed to `nonNegativeInteger` in SIDD 3.0
parser.createDouble("Bitrate", j2k.layerInfo[ii].bitRate, layerElem);
}
}
Expand Down
35 changes: 34 additions & 1 deletion six/modules/c++/six.sidd/source/DerivedXMLParser300.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@

#include <sstream>
#include <stdexcept>
#include <tuple>

#include <gsl/gsl.h>

Expand Down Expand Up @@ -222,7 +223,7 @@ xml::lite::Document* DerivedXMLParser300::toXML(const DerivedData* derived) cons
// optional
if (derived->compression.get())
{
DerivedXMLParser200::convertCompressionToXML(*this, *derived->compression, *root);
DerivedXMLParser200::convertCompressionToXML(*this, *(derived->compression), *root);
}
// optional
if (derived->digitalElevationData.get())
Expand Down Expand Up @@ -335,10 +336,42 @@ void DerivedXMLParser300::parseJ2KCompression(const xml::lite::Element& j2kElem,

for (size_t ii = 0; ii < layerElems.size(); ++ii)
{
// In SIDD 3.0, the `index` attribute type changed from `positiveInteger` to `nonNegativeInteger`
// (matching C-style indexing). Since we had a problem with this, use the opportunity to
// validate the `index` value.
const auto& attributes = layerElems[ii]->getAttributes();
std::string strIndex;
if (attributes.getValue("index", strIndex))
{
// The schema says this is required, but we might not be validating.
const size_t index = std::stoi(strIndex);
std::ignore = index; // compiler warning
assert(ii == index); // again, we might not be validating the XML
}

parseDouble(getFirstAndOnly(layerElems[ii], "Bitrate"), j2k.layerInfo[ii].bitRate);
}
}

void DerivedXMLParser300::convertJ2KToXML(const J2KCompression& j2k, xml::lite::Element& parent) const
{
auto& parser = *this;

parser.createInt("NumWaveletLevels", j2k.numWaveletLevels, parent);
parser.createInt("NumBands", j2k.numBands, parent);

const auto numLayers = j2k.layerInfo.size();
auto& layerInfoElem = parser.newElement("LayerInfo", parent);
parser.setAttribute(layerInfoElem, "numLayers", numLayers);

for (size_t ii = 0; ii < numLayers; ++ii)
{
auto& layerElem = parser.newElement("Layer", layerInfoElem);
parser.setAttribute(layerElem, "index", ii); // `ii + 1` in SIDD 2.0, `positiveInteger` is now `nonNegativeInteger`
parser.createDouble("Bitrate", j2k.layerInfo[ii].bitRate, layerElem);
}
}

void DerivedXMLParser300::parseDisplayFromXML(const xml::lite::Element& displayElem,
Display& display) const
{
Expand Down
25 changes: 25 additions & 0 deletions six/modules/c++/six.sidd/tests/sample_xml/sidd300.xml
Original file line number Diff line number Diff line change
Expand Up @@ -504,6 +504,31 @@
<si:Coef exponent1="0" exponent2="0">0</si:Coef>
</si:GammaZeroSFPoly>
</Radiometric>
<Compression>
<J2K>
<Original>
<NumWaveletLevels>1</NumWaveletLevels>
<NumBands>2</NumBands>
<LayerInfo numLayers="1">
<Layer index="0">
<Bitrate>3.14</Bitrate>
</Layer>
</LayerInfo>
</Original>
<Parsed>
<NumWaveletLevels>3</NumWaveletLevels>
<NumBands>4</NumBands>
<LayerInfo numLayers="2">
<Layer index="0">
<Bitrate>3.1415</Bitrate>
</Layer>
<Layer index="1">
<Bitrate>3.1415926</Bitrate>
</Layer>
</LayerInfo>
</Parsed>
</J2K>
</Compression>
<DigitalElevationData>
<GeographicCoordinates>
<LongitudeDensity>1</LongitudeDensity>
Expand Down

0 comments on commit d178aad

Please sign in to comment.