Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

howto run the test suite ? #38

Open
picca opened this issue Dec 20, 2021 · 19 comments
Open

howto run the test suite ? #38

picca opened this issue Dec 20, 2021 · 19 comments

Comments

@picca
Copy link

picca commented Dec 20, 2021

Hello, I try to run the test suite but I got this error message

make[1]: Entering directory '/<<PKGBUILDDIR>>'
cd TestSuite && ./testsuite.pl -a
No such file or directory at ../psimaglite/TestSuite/TestSuiteGlobals.pm line 249.

When I am looking at the 249 line I find this

#Runs a single test
sub testSuite
{
	my ($testNum)=@_;
	#$tempNum -= 100 if($testNum >= 100);
	my $procFile = $TestSuiteGlobals::inputsDir."processing$testNum.txt";
	my $procLib = $TestSuiteGlobals::inputsDir."processingLibrary.txt";
	
	if(-r $procLib) {
		if(-r "$procFile") {
			print "*******START OF TEST $testNum*******\n";
			$TestSuiteGlobals::specFile = getSpecFile($procFile);
			my @analyses = extractAnalyses($procFile) ;
			(@analyses) ? (processing(@analyses, $procLib)) : (print "Test $testNum does not includes any processing analyses.\n");
			print "*******END OF TEST ".$testNum."*******\n";
		} else {
			die "Could not find $procFile: $!";
		}
	} else {
		die "$!";
	}
}

So it expect the processingLibrary.txt in the inputs directory.

But I can not find this file in the inputs directory.

What is wrong here ?

thanks
@g1257
Copy link
Owner

g1257 commented Dec 20, 2021 via email

@picca
Copy link
Author

picca commented Dec 20, 2021 via email

@g1257
Copy link
Owner

g1257 commented Dec 20, 2021 via email

@picca
Copy link
Author

picca commented Dec 20, 2021 via email

@picca
Copy link
Author

picca commented Dec 20, 2021 via email

@g1257
Copy link
Owner

g1257 commented Dec 20, 2021 via email

@picca
Copy link
Author

picca commented Dec 22, 2021 via email

@g1257
Copy link
Owner

g1257 commented Dec 23, 2021 via email

@picca
Copy link
Author

picca commented Jan 6, 2022 via email

@picca
Copy link
Author

picca commented Jul 28, 2023

what do you think about this

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1026864

Source: dmrgpp
Version: 6.02-3
Severity: serious
User: [email protected]
Usertags: flaky timeout

Dear maintainer(s),

I looked at the results of the autopkgtest of your package. I noticed 
that it regularly fails on amd64 because it times out after 2 hours and 
47 minutes. Apparently this only happens on our most powerful worker: 
ci-worker13, but not all runs there time out. I'm wondering if this 
might be a race condition that the test only experiences with lots of 
CPUs (64) or lots of memory (256GB) (and depends on the load of the system).

Because the unstable-to-testing migration software now blocks on
regressions in testing, flaky tests, i.e. tests that flip between
passing and failing without changes to the list of installed packages,
are causing people unrelated to your package to spend time on these
tests.

Don't hesitate to reach out if you need help and some more information
from our infrastructure.

Paul

It mades the packages excluded from Debian stable...

@picca
Copy link
Author

picca commented Aug 1, 2023

In fact the tests timeout in our infrastructure afer 10000s. The load on the computer is high and makes the dmrgpp test run in more than this timeout. Is it possible to reduce the number of test to reduce this duration without reducing too much the coverage of the tests ?

tanks for your advice.

I have the feeling that I can reduce this with -n but I do not know what is the right range to achieve thie quite good coverage.

@picca
Copy link
Author

picca commented Aug 3, 2023

In fatc this is not about the load of the computer. Whe run on a computer with more than 100 cores, the code becomes really slow. Is there a way to limit the number of core used by dmrgpp ?

@picca
Copy link
Author

picca commented Aug 3, 2023

I mean globally in the test suite

@g1257
Copy link
Owner

g1257 commented Aug 3, 2023 via email

@utkarsh2102
Copy link

Oh, sweet. This is great. Thanks, @g1257 (and @picca for driving this!).

@picca
Copy link
Author

picca commented Aug 30, 2023

Hello, with the new 6.06 upload into Debian I now have a more reliable test suite. thanks

I limits the number of core used by the openmp part via this snipset

# Limit the number of threads in OpenMP
OMP_NUM_THREADS=$(test $(nproc) -gt 4 && echo 4 || echo $(nproc))
export OMP_NUM_THREADS
echo "Limiting the number of OpenMP thread via OMP_NUM_THREADS=$OMP_NUM_THREADS"

Now I have a bunch of missing oracles files, is it normal ?

I use the master/main branch of https://code.ornl.gov/gonzalo_3/OraclesDmrg.git

which I embed in the Debian package (this way no need to have an internet connection during the tests)

Cheers

3264s |5502|: No tests/runForinput5502.cout found
3264s |5502|: MaxEnergyDiff = NEW ENERGIES UNDEFINED   .NGST.
3264s |5502|: MaxRAM: UNDEFINED 939244 kB
3264s |5502|: UserTime: UNDEFINED 0.796
3264s |5502|: Version: UNDEFINED 5.55 24c483dc956d80ca9b0d349b8ee589ab065affee
3264s |5502| has 1 dmrg lines
3264s |5502| ignoring dmrg label in postCi mode
3264s |5502| has 1 CollectBrakets lines
3264s ./postCi.pl: Could not open tests/CollectBrakets5502_0.txt : No such file or directory
3264s ./postCi.pl: kompare tests/CollectBrakets5502_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5502_0.txt
3264s WARNING: Braket n. of labels differs 0 != 6
3264s Warning, Braket with label <gs|c'|P4> not in working dir
3264s Warning, Braket with label <gs|c'|P0> not in working dir
3264s Warning, Braket with label <gs|c'|P3> not in working dir
3264s Warning, Braket with label <gs|c'|P2> not in working dir
3264s Warning, Braket with label <gs|c'|P1> not in working dir
3264s Warning, Braket with label <gs|c'|P5> not in working dir
3264s -----------------------------------------------
3264s |5503|: No tests/runForinput5503.cout found
3264s |5503|: MaxEnergyDiff = NEW ENERGIES UNDEFINED   .NGST.
3264s |5503|: MaxRAM: UNDEFINED 939160 kB
3264s |5503|: UserTime: UNDEFINED 0.399
3264s |5503|: Version: UNDEFINED 5.55 24c483dc956d80ca9b0d349b8ee589ab065affee
3264s |5503| has 1 dmrg lines
3264s |5503| ignoring dmrg label in postCi mode
3264s |5503| has 1 CollectBrakets lines
3264s ./postCi.pl: Could not open tests/CollectBrakets5503_0.txt : No such file or directory
3264s ./postCi.pl: kompare tests/CollectBrakets5503_0.txt /tmp/autopkgtest-lxc.79d7q79s/downtmp/autopkgtest_tmp/OraclesDmrg/oracles/tests/CollectBrakets5503_0.txt
3264s WARNING: Braket n. of labels differs 0 != 2
3264s Warning, Braket with label <gs|A|P2> not in working dir
3264s Warning, Braket with label <gs|A|P3> not in working dir

@g1257
Copy link
Owner

g1257 commented Nov 28, 2023 via email

@picca
Copy link
Author

picca commented Nov 29, 2023 via email

@g1257
Copy link
Owner

g1257 commented Nov 29, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@g1257 @picca @utkarsh2102