Implement the NBT name redesign

[BenWoodworth]

See #29

Tasks:

• Update dependencies
  • Kotlin 2.0
  • kotlinx-serialization
  • okio
• Refactor parameterized testing
  • Replace Kotest's property testing with Parameterize
  • Tidy up test suite / test data
• Port over recent changes from v0.11
  • LGPL
  • "master" branch renamed to "main"
  • Shared NbtFormatBuilder interface
  • parameterizedNbtFormat() testing function
  • Builtin polymorphic serializer support
  • ...
• General
  • Split up list-like deprecation interfaces
    • The NbtList/Arrays all sharing another interface causes issues with type inference. e.g. listOf(NbtList, NbtByteArray) is inferred as List<Any> instead of List<NbtTag>
  • Implement Back Nbt*Arrays with Lists, and encode arrays as NbtList by default, and require @NbtArray #34, fixing the problem with elements whose serializers delegate to Byte/Int/LongArrays (see here)
    • Remove fast-path optimization with array types for now, as it's making it more difficult to iterate on the implementation. (See b8d52f4 which implemented checks against descriptors instead of checking that the serializer is an instance of *ArraySerializer
• Gradually implement new design, ideally keeping all tests valid between each change
  • Rename @NbtNamed to @NbtName, retaining the existing NBT names on test classes that have them
  • Refactor encoder/decoder-verifying testing so it will better cover named/unnamed and nullabe/non-nullable NBT formats
  • Add the NbtNamed<T> class next, that way the later changes have something to migrate to when moving away from the single-entry-root-compound name representation
    • Add convenience properties for NbtNamed<NbtTag> to match those on NbtTag, so that dealing with named tags is more seamless when the name is unimportant. E.g. namedTag.nbtInt
  • Add encodeTo/decodeFromNamedNbtTag() functions, so tests have a way to check against the serialized representation
  • Refactor general NBT serialization checks so they can be shared between the encoder and decoder
  • Add NbtName.Dynamic that disables root name validation for a type (and later allows name to be serialized dynamically with encoder/decoder functions)
    • Add checks that require @NbtName.Dynamic when delegating to another dynamic serializer
    • For consistency during the design change, all descriptors that have Dynamic will be considered to have an NbtName ("" by default). As opposed to requiring @NbtName or not being named until the name is dynamically encoded, since all types will eventually be implicitly NbtNamed anyway)
  • Make @NbtName work on any serializable type
    • A lot of the codebase assumes it's only used on conpounds/classes, so a lot of (related/unrelated) tests fail when broadening them to any type. Will do with care and comb through the code to check everything over.
  • Make the new NbtNamed class serializable. Done after implementing it as an encode/decode representation because those will be necessary for testing its serialization
  • Change buildNbtCompound(name) and buildNbtList(name) functions to return NbtNamed
  • Change @NbtName to only nest under the name for the root value
    • Since the name will eventually only apply to the root value
  • Remove the single-entry-root-compound name representation when encoding to NbtTags, binary, and snbt
    • @NbtName docs, which says it instructs the serialization to nest within an NBT compound so it's named
    • JavaNetwork docs: "Serializes unnamed NbtTags for use in network packets"
      • can remove mention of "unnamed", since there is no longer a difference in behavior with this design. (It was special because it didn't use an outer nbt compound for the internal representation, but now that's the way they all work)
    • Readme examples
  • Make all serializable types have an NBT name, defaulting to an empty name if not marked with @NbtName
  • lenientNbtName configuration option, and name validation
    • Apply to empty-named JavaNetwork, which gracefully discards the name currently
  • Replace NbtVariant with specialized NbtFormat implementations (JavaNbt, JavaNetworkNbt(val protocolVersion), etc.)
    • With some variants being named, and with some having nullable roots, it doesn't make sense for all the variants to share the Nbt class. Requiring that they all have the same encodeToNbtTag() is forcing an abstraction. In reality, they are all (slightly) different formats, so should each be its own NbtFormat implementation (with all the same binary serialization methods, but different in-memory NbtTag serialization methods)
    • Additional in-memory NbtFormat providing encodeTo/decodeFromNbtTag with all NBT capabilities (named root, nullable root, special floating point values, etc) that fully captures how a class would serialize to NBT, not just how a specific limited variant would be serialized. This is useful for testing/experimenting by library users, but also necessary for NbtTransformingSerializer so serializer's using it aren't unnecessarily limited by a specific NBT format.
  • Allow serializing null as the root tag in NBT formats that support it (Java network)

    • May be a TAG_END (0)

    • discardTag() might need to accept TAG_End
    • Add nullable NbtTag?. convenience properties, maybe including .nbtTag and .nbtEnd
  • De-merge CompositeNbtEncoder/Decoder?
    • decodeElementNbtTagType() later could have slightly different semantics between the base decoder and the composite decoder, and may be better as a separate element-specific function.
  • Expose NbtTagType
    • Make NbtTag.type public (the member property and the extension property)
    • Add decodeNbtTagType() function to NbtDecoder (and change decodeNotNullMark() to use it)
  • Remove @NbtName entirely?
    • Not sure it has a compelling use case, since encode/decodeNbtName are straightforward to use, and its validation functionality mixed with @NbtName.Dynamic adds complexity when that could be left entirely to a simple KSerializer implementation (especially now that @KeepGeneratedSerializer is a thing, letting you tack an extra line of NBT name code onto the generated serialization process)
  • ...
  • Code cleanup
  • Documentation (in readme, doc comments, etc.)
    • Text
    • Code examples

[Ayfri]

Hey, any update ?

[BenWoodworth]

@Ayfri

Hey, any update ?

Sorry for the late reply! I'm recently in between jobs, so I have some more free time and this is one of the things I want to try to wrap up and put a nice bow on to finally get these changes out there.

Last time I was working on it I kept running into issues representing the tag types in the SerialDescriptors. I started writing up an issue for kotlinx.serialization that I'm hoping hits at the heart of some of the inconsistencies I'm facing.

That, and I also want to put out a more straightforward PR that will make introspecting serial descriptors much better, since especially with NBT and the need to encode the tag type ahead of time, not being able to introspect descriptors as well as I'd like means I have to jump through some hoops, e.g. defer encoding until a serializer begins encoding a value or some child elements (with TAG_End or TAG_List/arrays), and I lose out on validation and consistency checks because of that. If that makes sense!

I do think I can get a "good enough" implementation without that in the meantime though. After opening the KXS issue and after the PR, you should start seeing new commits here while I've got the free time during my job search :)

[Ayfri]

@Ayfri

Hey, any update ?

Sorry for the late reply! I'm recently in between jobs, so I have some more free time and this is one of the things I want to try to wrap up and put a nice bow on to finally get these changes out there.

Last time I was working on it I kept running into issues representing the tag types in the SerialDescriptors. I started writing up an issue for kotlinx.serialization that I'm hoping hits at the heart of some of the inconsistencies I'm facing.

That, and I also want to put out a more straightforward PR that will make introspecting serial descriptors much better, since especially with NBT and the need to encode the tag type ahead of time, not being able to introspect descriptors as well as I'd like means I have to jump through some hoops, e.g. defer encoding until a serializer begins encoding a value or some child elements (with TAG_End or TAG_List/arrays), and I lose out on validation and consistency checks because of that. If that makes sense!

I do think I can get a "good enough" implementation without that in the meantime though. After opening the KXS issue and after the PR, you should start seeing new commits here while I've got the free time during my job search :)

Oh I didn't know that, sorry to rush you, I hope you'll have a stable situation soon!
I see why it's that difficult and hope you'll find a good way to implement this!