Feature request: Add support for types

[pali]

Hello!

In some cases it is needed to distinguish in YAML between number stored as string literals (number in single or double quotes) and number stored as number (without quotes).

Similar thing is needed also in JSON and e.g. Cpanel::JSON::XS encoder/decoder provides additional argument for passing types (when encoding) or getting types (when decoding). Documentation is written at: https://metacpan.org/pod/Cpanel::JSON::XS::Type

JSON example of usage:

use Cpanel::JSON::XS;
use Cpanel::JSON::XS::Type;
my $json_string = '{"key1":{"key2":[10,"10",10.6]},"key3":"10.5"}';
my $perl_struct = decode_json($json_string, 0, my $type_spec);
# $perl_struct is { key1 => { key2 => [ 10, 10, 10.6 ] }, key3 => 10.5 }
# $type_spec is { key1 => { key2 => [ JSON_TYPE_INT, JSON_TYPE_STRING, JSON_TYPE_FLOAT ] }, key3 => JSON_TYPE_STRING }

use Cpanel::JSON::XS;
use Cpanel::JSON::XS::Type;
encode_json([10, 10, 10.25], [JSON_TYPE_STRING, JSON_TYPE_INT, JSON_TYPE_STRING]);
# '["10",10,"10.25"]'

Via YAML::PP I can load e.g. following example

use YAML::PP qw(Load);
my $perl_struct = Load(<<EOF);
array:
 - "10000000000000000000"
 - 10000000000000000000
EOF

And I would like to get information what are types of the key array (like in above JSON example). So e.g. that first element is string and second element is a number.

It is possible to provide some kind of support for types when loading and dumping YAMLs via YAML::PP?

[perlpunk]

Currently not.

The logic YAML::PP is using, is to check the flags of the input scalar.

• has an IOK flag -> dump as int
• has a NOK flag -> dump as float (and add .0 if necessary)
• only has POK flag => dump as string

I think this is the default logic used by Cpanel::JSON::XS.
Here are some examples:
https://perlpunk.github.io/perl5-numbers-and-json-yaml/

So in most cases this is the right logic, and the user has to take care to not use a scalar that is supposed to be a string as a number before dumping.

For checking the types of the loaded data, you would have to use the B module (and B::SVp_IOK etc.) yourself.

What you describe sounds like a schema.
I would like to support a schema at some point, e.g. JSONSchema, for loading and possibly dumping.
That would allow more things additionally to just ensuring int/float/string.

[perlpunk]

It could make sense to add the kind of functionality you describe as a pre/prost-processing step, independently from loading/dumping JSON/YAML.

So before dumping it would pre-walk the data and convert the scalars if necessary.
And after loading it could walk the data and return the types.

[pali]

Hello! Looking at internal (IOK, NOK, POK) flags is problematic as Perl interpreter and also XS modules may add these flags to scalar at any time. And the worse is that this internal behavior is changing between Perl versions, so looking at POK vs IOK is Perl version specific. Moreover scalars may have more flags, e.g both IOK and POK. This is infamous problem of more JSON encoders and also reason why Cpanel::JSON::XS was extended to support these types -- to provide API for stable JSON encoding and decoding. @choroba has talk at YAPC about this problem, see: http://e-choroba.eu/18-yapc

[perlpunk]

I know. I saw the talk :)
(I created the above linked table at the Toolchain Summit 2018, but never got around to post it somewhere.)
I'm just saying that YAML::PP uses the same default logic as Cpanel::JSON::XS, which seems to be the right thing in most cases.

Implementing schema support would help with this.
Implementing just this functionality is maybe not worth it if I want to add Schema support anyway.