FOPI plotting produces error

[NGoetz]

The FOPI target fails during plotting:

[100%] Built target FOPI_pions_sims
Analyzing data for FOPI_pions.
Plotting data for FOPI_pions.
3.0
Traceback (most recent call last):
  File "/lustre/hyihp/ngoetz/smash-analysis/test/FOPI_pions/plot.py", line 42, in <module>
    smash_data = np.loadtxt(data_file, unpack=True)
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 1148, in loadtxt
    for x in read_data(_loadtxt_chunksize):
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 995, in read_data
    raise ValueError("Wrong number of columns at line %d"
ValueError: Wrong number of columns at line 3
make[3]: *** [test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/build.make:74: test/FOPI_pions/FOPI_pions.pdf] Error 1
make[2]: *** [CMakeFiles/Makefile2:4919: test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:4926: test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/rule] Error 2
make: *** [Makefile:2217: FOPI_pions_plots] Error 2

I assume that it could be related to the binary reader, though I am not sure. I assign this to @Hendrik1704 for now, but if you find that it is not related, you can also assign someone else.

[Hendrik1704]

Did you use the develop branch for the analysis? I had a look at the implementation of the new column with the strangeness in the format version 9 again and it looks ok.

I just went to the point in the analysis causing the error:

smash-analysis/test/FOPI_pions/plot.py

Lines 41 to 42 in 11823db

	data_file = os.getcwd() + "/pion_yields.txt"
	smash_data = np.loadtxt(data_file, unpack=True)

The pion_yields.txt file used there seems to be produced by the yields.py script. In this script the binary reader is used, but I don't see how this can cause an error for the read in of pion_yields.txt with the wrong number of columns.
Can you maybe give the content of the pion_yields.txt file here? Just to check, if the file contains: energy, Npi+, Npi0, Npi-, Nevent, Ntestparticles, ptpi+, ptpi0, ptpi-, erorrs.

[NGoetz]

The file looks like this:

3.0
#version SMASH-3.1rc-3-gc02aec6e6 format 9 analysis SMASH-analysis-3.0-3-g11823db
0.4 29.92 0.5135890744298767 50.28999999999999 0.6862715839000525 75.39 0.7348324896728422 100 20 0.1490352523291481 0.0014978252999565363 0.1497638835889731 0.0011162455280036927 0.15110275684980323 0.0009651001104260252
0.5 58.94 0.5828907411870736 91.9 0.9474580558931185 134.43999999999994 1.1014517509160164 100 20 0.16018016766192159 0.001114676989380034 0.1596423914801718 0.0009622938979028352 0.16107010246014944 0.00074775944031733
0.6 94.21000000000001 0.9954741015559809 141.21999999999997 1.0651106074303294 203.66000000000005 1.294137519428256 100 20 0.16883397862303456 0.0009910079369182846 0.16772714565926825 0.0008284729458712584 0.16909026636501862 0.0006099353247725015
0.7 133.81 1.1302717113217249 195.61 1.2904384189612894 272.8399999999999 1.417332820205727 100 20 0.1756713658278267 0.000915515379546336 0.17469328348731725 0.0007716942931732351 0.17574156263458282 0.0005922863552648695
0.8 172.07999999999996 1.1748784592200985 253.30000000000007 1.3821005825199624 341.15000000000003 1.2370847941344958 100 20 0.18289504699727074 0.0007783538007393761 0.18245256236106105 0.0006037746432513933 0.1821568786726539 0.0005488274129234326
0.9 218.26999999999995 1.4735276849736612 308.4100000000001 1.7607445050663126 410.84999999999997 1.6569032207187844 100 20 0.18925977435478603 0.0006804693279419694 0.18900857785520708 0.0006209197320552451 0.18892443938047163 0.0005514209220627661
1.0 263.77000000000027 1.4191230041860712 366.3200000000001 1.8868968699904614 480.7799999999999 1.6813642849144432 100 20 0.1937195821675367 0.0007205731580340784 0.19383590694346828 0.0005722100365934549 0.19303143623844288 0.0005985473803091436
1.1 314.78000000000014 1.479528999582658 424.51000000000016 1.9578639669628004 547.6300000000001 1.61818298149998 100 20 0.20044119734558224 0.000652409097729008 0.19885578882610438 0.0006126017139448641 0.19797440621850956 0.0005551136368668634
1.2 361.68999999999994 1.6229226037546514 488.3899999999999 2.1341804662741763 615.0099999999998 1.8855374605537583 100 20 0.2054680492097961 0.0006427096703619368 0.20308004484831294 0.0006386579803008518 0.2034439004808124 0.0005561641482213311
1.3 411.82 1.7536904809658698 543.5699999999999 2.283726132990786 680.3599999999999 2.1133304655367224 100 20 0.21057375765266925 0.0006282561663121394 0.20864193296190944 0.0005773689145434261 0.20753871374723443 0.0005017946877165187
1.4 459.2600000000001 1.724371939535837 597.68 2.1355991754970858 742.0600000000002 2.146682665101311 100 20 0.21354035249894562 0.000644390088812411 0.21236068893305557 0.0006123563172167805 0.2114251799027013 0.0004903498187374154
1.5 510.95000000000016 1.8211648508966511 656.37 2.3643247055198056 805.5300000000004 2.0813969182530103 100 20 0.2168523628792753 0.0006158928422928895 0.21708312457988724 0.0005161236059113513 0.21429998496108174 0.000531962470388285

I wonder where the 3.0 comes from? maybe that is the issue

[Hendrik1704]

The 3.0 really looks wrong. I just checked the output file in a LibreOffice table and the columns really look like they should according to yields.py:

<title></title>
<meta name="generator" content="LibreOffice 7.3.7.2 (Linux)"/>
<style type="text/css">
	body,div,table,thead,tbody,tfoot,tr,th,td,p { font-family:"Liberation Sans"; font-size:x-small }
	a.comment-indicator:hover + comment { background:#ffd; position:absolute; display:block; border:1px solid black; padding:0.5em;  } 
	a.comment-indicator { background:red; display:inline-block; border:1px solid black; width:0.5em; height:0.5em;  } 
	comment { display:none;  } 
</style>

e	N211	N211err	N111	N111err	N-211	N-211err	nevent	ntestp	N211pt_avg	N211pt_avg_err	N111pt_avg	N111pt_avg_err	N-211pt_avg	N-211pt_avg_err

Is the 3.0 really part of the output file?

[Hendrik1704]

I looked at the yields.py file for a while now and the only 3.0 in the script is in an assert(a.avg == 3.0), which can not cause the number to appear in the output file. This could only fail if the Average class is implemented in a wrong way and then the script would raise an error.
There is also no other active print statement before line 78, where the #version ... is printed, so the number should in principle not be there...

[NGoetz]

I tested it again, removing the corrupted file and executing the plotting target:

Analyzing data for FOPI_pions.
Plotting data for FOPI_pions.
3.0
Traceback (most recent call last):
  File "/lustre/hyihp/ngoetz/smash-analysis/test/FOPI_pions/plot.py", line 42, in <module>
    smash_data = np.loadtxt(data_file, unpack=True)
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 1148, in loadtxt
    for x in read_data(_loadtxt_chunksize):
  File "/usr/lib/python3/dist-packages/numpy/lib/npyio.py", line 995, in read_data
    raise ValueError("Wrong number of columns at line %d"
ValueError: Wrong number of columns at line 3
make[3]: *** [test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/build.make:74: test/FOPI_pions/FOPI_pions.pdf] Error 1
make[2]: *** [CMakeFiles/Makefile2:4919: test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:4926: test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/rule] Error 2
make: *** [Makefile:303: test/FOPI_pions/CMakeFiles/FOPI_pions_plots.dir/rule] Error 2

The 3.0 is both printed out to the user and to the file.
Maybe it prints the 3.0 instead of failing when the asset fails? But how could the implementation of Average suddenly break?

[NGoetz]

Could you check if the 3.0 comes from the assert itself?

[Hendrik1704]

I did a check in an external python script with

a = 3.0
assert(a == 3.0)

This does not print a 3.0 in the terminal. There is no output at all. Only if the line is not true, then it would print:

Traceback (most recent call last):
  File "~/assert.py", line 2, in <module>
    assert(a == 3.0)
AssertionError

So this should not be the origin of the 3.0.

[NGoetz]

This is very odd. Probably the best thing is to build a minimal example creating this error.

[NGoetz]

The 3.0 comes from version.py. I gonna fix this.