netCDF::exceptions::NcHdfErr after calling getVar too often

[DimitriSotnik]

Hi everybody,
working here on an ns2 simulator, which is using netCDF 4.7.3 (tried also 4.8.1) / netcdf-cxx4-4.3.1, and we encounter a limitation on reading our HDF5 files. The database is called on every transmission. So the amount is quite large. After calling it around 570000 times during the run netCDF crashes with following exception:

terminate called after throwing an instance of 'netCDF::exceptions::NcHdfErr'
what(): NetCDF: HDF error
file: ncVar.cpp line:1537

Heres a small c++ example which can reproduce the behaviour. i tried different nc Files, getting the same result.

#include

#include <netcdf.h>
#include <ncFile.h>
#include <ncVar.h>

std::string path_;
netCDF::NcFile *netcdf_db;
netCDF::NcVar netcdf_LocMat;
netCDF::NcVar netcdf_ValueMat;

bool openNetCdfDb();

void closeNetCdfDb();

int main() {

srand (0);
int64_t count = 0;
path_ = (char *) ("../FILE.nc");


while (1) {
	if (openNetCdfDb()) {

			std::cout << "count " << count++ << std::endl;
			netcdf_LocMat = netcdf_db->getVar("LocMat");
		netcdf_ValueMat = netcdf_db->getVar("ValueMat");

			double x, y,result;

			for (uint i = 0; i < 9; i++) {
				netcdf_LocMat.getVar({0, i}, &x);
				netcdf_LocMat.getVar({1, i}, &y);
				for (uint j = 0; j <= 70; j++) {
					netcdf_ValueMat.getVar({i, j}, &result);
					std::cout << "x " << x << " y " << y << " value " << result << std::endl;
				}
			}

	}
	closeNetCdfDb();
}
return 0;

}

bool openNetCdfDb() {
if (netcdf_db == NULL) {

	std::cout << " openNetCdfDb(): Opening file " << path_ << std::endl;
	netcdf_db = new netCDF::NcFile(path_, netCDF::NcFile::read);
	if (netcdf_db != NULL) {

		std::cout << "opened" << std::endl;
		return true;
	}
	return false;
}

return true;

}

void closeNetCdfDb() {
if (netcdf_db != NULL) {
delete netcdf_db;
netcdf_db = NULL;
std::cout << "closed" << std::endl;
}
}

[WardF]

Thanks, I'll take a look and see what's going on. What version of HDF5 are you working with?

[DimitriSotnik]

Thank you for your help.

We are using HDF5 1.8.13

[edwardhartnett]

Shouldn't this be in the netcdf-cxx project?

[MetalKnight]

Hi @WardF @edwardhartnett is there any update on this issue?
we have opened the issue also on NetCDFC++ as requested but in our opinion it is not a matter of C++, which is simply wrapping over the C functions and throwing an exception due to the HDF error.
This issue has been also found in a public project that uses NetCDF4 for its databases
MetalKnight/woss-ns3#43

Thanks again

Regards

[MetalKnight]

Update:

• upgrading to HDF5 1.14.2 and to NetCDF-C 4.9.2 fixed the HDF error