Empty values and column-data-storage refactor

[JorisGoosen]

Empty values should be stored in the data and per column.

To do this properly it required a lot of changes to the underlying code.
Ill make sure the documentation for the classes at least resembles what is going now ;)

Possible problems:
[X] importing from 0.17.3 and earlier (converting from emptyValuesMap in metadata.json)
Ignore: importing from 0.18.2 (broken emptyvalues (i might just ignore it))
[X] importing from 0.18.3 (breaking up emptyvalues into per workspace and column)
[X]] also making sure that we can still import scalar columns with a non-scalar value in them that is not convertible to a double (like in the testcsv tsv ods etc)
Should probably be a separate PR: [] Add locale support (should still be done but testing will take time anyway)
[] importing a nominal/ordinal now as scalar means getting the ints we wrote to the _ints column. We should probably try to use the "value" shown in the label-editor instead! :o This will help with jasp-stats/jasp-issues#2530 or jasp-stats/jasp-issues#2325 ) (But most of that will probably be separate)

[JorisGoosen]

[JorisGoosen]

I started a bunch of builds under emptyValuesRefactor in case anybode feels like testing it out.

I think this is probably ready now?

[JorisGoosen]

Its maybe not completely done, but close enough that you guys could start reviewing it.
And Ill also ask people to test it once the nightlies are build because im sure there are some things ive missed...

[JorisGoosen]

macos arm
macos x86

Aah, windows doesnt work it seems...

[shun2wang]

the build test failed maybe also failing on windows.

[RensDofferhoff]

• Press new data
• add any string in an empty column
• No Type is detected
• Use this column in descriptives and the engine crashes

[JorisGoosen]

• Press new data

  • add any string in an empty column

  • No Type is detected

  • Use this column in descriptives and the engine crashes

Oops!

Edit: shouldve set columntype with the default values...
Nightlies are building with this fix for anyone curious

[JorisGoosen]

Aah moving labels also doesnt work anymore for columns where all values are convertible to double

[boutinb]

Some issues:

• The empty values are empty for a column, though the workspace has some empty values
• In Edit mode, if I change a value, and press the tab key, the change is lost
• With a nominal column, if I set a new value, it does not appear in the Label Editor.
• I have tried to add a computed column with just contNormal + 1: the engine crashes

Nice feature to see the labels for a scale column (if they are not the same as the value). It could be also nice to see the value of nominal or ordinal column, if the label and the value is not the same.

[shun2wang]

FileOperation menu has some overlap with the ▶ , especially for language-specific strings lengths.

[JorisGoosen]

Ive also added a new feature to computed columns, see: https://github.com/jasp-stats/INTERNAL-jasp/issues/2487

[boutinb]

When reading a JASP file from 0.18.0 where for example 0 was set as an empty value, JASP displays the values that had 0 as empty values in the data view, but does not display the original value (0) in grey, and it does not set in the workspace that 0 is an empty value.

Sorry bad manipulation. It works: the 0 is set as empty value in the workspace. Only minor thing is that the if 0 is unset, the cells are not set back to 0: the original value (0) is lost.

[boutinb]

Sorry bad manipulation. It works: the 0 is set as empty value in the workspace. Only minor thing is that the if 0 is unset, the cells are not set back to 0: the original value (0) is lost.

I have built it with the last commits, and now if I load a JASP file from 0.18.0, the workspace empty values are empty!

[JorisGoosen]

I only made an effort to support pre-0.18.0 versions of JASP and specifically 0.18.3.
This because there were some quite terrible bugs in those earlier 0.18.* versions and because we released them in such short order I think we can safely ignore them.