Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove json_max type #103

Merged
merged 3 commits into from
Jan 26, 2022
Merged

Remove json_max type #103

merged 3 commits into from
Jan 26, 2022

Conversation

Leonidas-from-XIV
Copy link
Member

@Leonidas-from-XIV Leonidas-from-XIV commented Jul 24, 2020

While removing json I came across json_max and wondered what that is for. So when I deleted it, I figured out it works this way:

  • There is a top-level Yojson module which has a type t, that contains all the variants.
  • That module has another module, Yojson.Pretty which has the prettyprinter for the "biggest" type.
  • All the Yojson.{Basic,Safe,Raw} variants upcast their t type to this biggest type and and then use the functions from Yojson.Pretty.
  • (Amusingly, it is therefore possible to get a `String in a variant that does not define STRING on the CPPO level, since it always uses `String instead of `Stringlit, though this is mostly a weird behaviour and probably not a bug)

This PR changes it in a way that every of the variants has its own Pretty module which is specialized to only pretty-print the specific variant.

I've made this a separate PR since I would like to know your input:

  • The code is in my opinion somewhat easier to understand since all the different variants (Basic, Safe, Raw use their own Pretty module.
  • Potentially faster, since the pattern match is smaller, but I'd be surprised if it makes a nontrivial difference
  • The generated code is larger since Pretty exists 4 times now instead of once.

This is separated into its separate PR since I don't want to press it on the one that is important for the 2.0.0 release.

(Note this PR is branched off #100)

Copy link
Collaborator

@NathanReb NathanReb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing the json_max type seems sound but I'm unsure why you made those changes to Pretty or to be more specific how they are related to the removal of a type alias. I might be missing something!

@Leonidas-from-XIV
Copy link
Member Author

I was also confused why removing this alias would ripple through so much code, until I understood what it is used for. I'll try to explain what is going on.

The current code has one Pretty module, Yojson.Pretty. This module has functions to pretty-print a very specific type — json_max, which includes all the possible variants (note that it includes mutually exclusive variants like `Int and `IntLit, so no parser ever generates values of json_max type).

Then each of the Basic/Safe/Raw has its own write2.ml which takes a {Basic,Safe,Raw}.t and uses Yojson.Pretty to pretty print it. Since these specialized types are subsets of json_max they get upcast (which is safe) to json_max so they can use the functions in the top-level Pretty module.

Removing the json_max name means:

  1. Either I have to write out the full type by hand for the upcast which is very tedious
  2. Or I have to have a Pretty module that actually matches the type t and not a superset of it.
  3. Maybe some other solutions, but I couldn't think of anything on the spot, input welcome

I chose to do number 2, thus I made CPPO generate Pretty modules that would operate on the respective {Basic,Safe,Raw}.t types, so I had to add all the #ifdefs and now each of the modules has its own specialized Pretty module that only pretty-prints the exact subset of t.

Having gone through all this I am not sure it was worth it. It certainly is more code and the printer looks worse. On the other hand, I think it is now slightly more understandable, since the Basic/Safe/Raw variants do not share a mysterious Pretty module under the hood.

Hence this PR is somewhat more on the "well, I'll ask for input from others" side, since I am not sure it is a win or not.

@c-cube
Copy link
Member

c-cube commented Jul 31, 2020 via email

@Leonidas-from-XIV
Copy link
Member Author

Haven't looked at the PR, but if pretty was just a private module, thanks to dune, it would be an implementation detail so there's no downside in keeping it, I think?

The json_max type is public so I thought I might remove it for 2.0.0 so I started pulling the string and the whole thing came with it.

But I can try to solve it in a different way, where the Pretty that works on json_max is in its own submodule and can be accessed without needing the json_max alias. Though personally I don't really like how Pretty just has to support everything.

I also dislike CPPO, for the same reason. I was thinking whether replacing the whole misery with functors or just using composition of functions that operate on polymorphic variants would work better. But reducing the need for this duplicated code and code generation (#105) would be even better.

Copy link
Collaborator

@panglesd panglesd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this can be merged, in the interest of release 2.0 without the json_all type.

I don't think that moving the Pretty module inside the different variants is necessary, but I'm ok with the change. I agree it seems more natural than having one module for all variants, even if it also adds some cppo.

Anyway, I think that after the release we should work on finding a way to remove the cppo, to be able to use ocamlformat to format the code, and get Merlin to work again.

lib/pretty.ml Outdated Show resolved Hide resolved
lib/pretty.ml Outdated Show resolved Hide resolved
@@ -132,7 +132,6 @@ end
#define VARIANT
#include "type.ml"
#include "monomorphic.mli"
type json_max = t
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that if we want to remove the json_max alias from the public API without making too much changes to the code, only removing this line is sufficient. It would stay defined in the implementation but hidden from the public. That would be enough for releasing 2.0, and we can decide about the cppo madness later.
Am I wrong?

Copy link
Member Author

@Leonidas-from-XIV Leonidas-from-XIV Jan 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would agree. Removing it from the public API should free us from compatibility constraints and then we can implement the rest in any other way going forward.

I've incorporated your suggestions and rebased it on current master so the changelog and CI should succeed now.

Leonidas-from-XIV and others added 3 commits January 26, 2022 13:33
The `json_max` type is the type which includes all polymorphic variants.
This is done so the `Pretty` module can be reused through all the
variants, but it is somewhat obscure how that works since it requires
upcasting. This commit changes it to generate different `Pretty`
modules, each specialized to the exact selected variant.
Copy link
Collaborator

@panglesd panglesd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Leonidas-from-XIV Leonidas-from-XIV merged commit c35769a into master Jan 26, 2022
@Leonidas-from-XIV Leonidas-from-XIV deleted the remove-json-max-type branch January 26, 2022 14:15
panglesd added a commit to panglesd/opam-repository that referenced this pull request Jun 2, 2022
CHANGES:

### Removed

- Removed dependency on easy-format and removed `pretty_format` from
  `Yojson`, `Yojson.Basic`, `Yojson.Safe` and `Yojson.Raw`. (@c-cube, ocaml-community/yojson#90)
- Removed dependency on `biniou`, simplifying the chain of dependencies. This
  changes some APIs:
  * `Bi_outbuf.t` in signatures is replaced with `Buffer.t`
  * `to_outbuf` becomes `to_buffer` and `stream_to_outbuf` becomes
    `stream_to_buffer`
  (@Leonidas-from-XIV, ocaml-community/yojson#74, and @gasche, ocaml-community/yojson#132)
- Removed `yojson-biniou` library
- Removed deprecated `json` type aliasing type `t` which has been available
  since 1.6.0 (@Leonidas-from-XIV, ocaml-community/yojson#100).
- Removed `json_max` type (@Leonidas-from-XIV, ocaml-community/yojson#103)
- Removed constraint that the "root" value being rendered (via either
  `pretty_print` or `to_string`) must be an object or array. (@cemerick, ocaml-community/yojson#121)
- Removed `validate_json` as it only made sense if the type was called `json`.
  (@Leonidas-from-XIV, ocaml-community/yojson#137)

### Add

- Add an opam package `yojson-bench` to deal with benchmarks dependency
  (@tmcgilchrist, ocaml-community/yojson#117)
- Add a benchmark to judge the respective performance of providing a buffer vs
  letting Yojson create an internal (ocaml-community/yojson#134, @Leonidas-from-XIV)
- Add an optional `suf` keyword argument was added to functions that write
  serialized JSON, thus allowing NDJSON output. Most functions default to not
  adding any suffix except for `to_file` (ocaml-community/yojson#124, @panglesd) and functions
  writing sequences of values where the default is `\n` (ocaml-community/yojson#135,
  @Leonidas-from-XIV)

### Change

- The `stream_from_*` and `stream_to_*` functions now use a `Seq.t` instead of a
  `Stream.t`, and they are renamed into `seq_from_*` and `seq_to_*` (@gasche, ocaml-community/yojson#131).

### Fix

- Avoid copying unnecessarily large amounts of strings when parsing (ocaml-community/yojson#85, ocaml-community/yojson#108,
  @Leonidas-from-XIV)
- Fix `stream_to_file` (ocaml-community/yojson#133, @tcoopman and @gasche)
panglesd added a commit to panglesd/opam-repository that referenced this pull request Jun 3, 2022
CHANGES:

### Removed

- Removed dependency on easy-format and removed `pretty_format` from
  `Yojson`, `Yojson.Basic`, `Yojson.Safe` and `Yojson.Raw`. (@c-cube, ocaml-community/yojson#90)
- Removed dependency on `biniou`, simplifying the chain of dependencies. This
  changes some APIs:
  * `Bi_outbuf.t` in signatures is replaced with `Buffer.t`
  * `to_outbuf` becomes `to_buffer` and `stream_to_outbuf` becomes
    `stream_to_buffer`
  (@Leonidas-from-XIV, ocaml-community/yojson#74, and @gasche, ocaml-community/yojson#132)
- Removed `yojson-biniou` library
- Removed deprecated `json` type aliasing type `t` which has been available
  since 1.6.0 (@Leonidas-from-XIV, ocaml-community/yojson#100).
- Removed `json_max` type (@Leonidas-from-XIV, ocaml-community/yojson#103)
- Removed constraint that the "root" value being rendered (via either
  `pretty_print` or `to_string`) must be an object or array. (@cemerick, ocaml-community/yojson#121)
- Removed `validate_json` as it only made sense if the type was called `json`.
  (@Leonidas-from-XIV, ocaml-community/yojson#137)

### Add

- Add an opam package `yojson-bench` to deal with benchmarks dependency
  (@tmcgilchrist, ocaml-community/yojson#117)
- Add a benchmark to judge the respective performance of providing a buffer vs
  letting Yojson create an internal (ocaml-community/yojson#134, @Leonidas-from-XIV)
- Add an optional `suf` keyword argument was added to functions that write
  serialized JSON, thus allowing NDJSON output. Most functions default to not
  adding any suffix except for `to_file` (ocaml-community/yojson#124, @panglesd) and functions
  writing sequences of values where the default is `\n` (ocaml-community/yojson#135,
  @Leonidas-from-XIV)

### Change

- The `stream_from_*` and `stream_to_*` functions now use a `Seq.t` instead of a
  `Stream.t`, and they are renamed into `seq_from_*` and `seq_to_*` (@gasche, ocaml-community/yojson#131).

### Fix

- Avoid copying unnecessarily large amounts of strings when parsing (ocaml-community/yojson#85, ocaml-community/yojson#108,
  @Leonidas-from-XIV)
- Fix `stream_to_file` (ocaml-community/yojson#133, @tcoopman and @gasche)
Leonidas-from-XIV pushed a commit to panglesd/opam-repository that referenced this pull request Jun 9, 2022
CHANGES:

### Removed

- Removed dependency on easy-format and removed `pretty_format` from
  `Yojson`, `Yojson.Basic`, `Yojson.Safe` and `Yojson.Raw`. (@c-cube, ocaml-community/yojson#90)
- Removed dependency on `biniou`, simplifying the chain of dependencies. This
  changes some APIs:
  * `Bi_outbuf.t` in signatures is replaced with `Buffer.t`
  * `to_outbuf` becomes `to_buffer` and `stream_to_outbuf` becomes
    `stream_to_buffer`
  (@Leonidas-from-XIV, ocaml-community/yojson#74, and @gasche, ocaml-community/yojson#132)
- Removed `yojson-biniou` library
- Removed deprecated `json` type aliasing type `t` which has been available
  since 1.6.0 (@Leonidas-from-XIV, ocaml-community/yojson#100).
- Removed `json_max` type (@Leonidas-from-XIV, ocaml-community/yojson#103)
- Removed constraint that the "root" value being rendered (via either
  `pretty_print` or `to_string`) must be an object or array. (@cemerick, ocaml-community/yojson#121)
- Removed `validate_json` as it only made sense if the type was called `json`.
  (@Leonidas-from-XIV, ocaml-community/yojson#137)

### Add

- Add an opam package `yojson-bench` to deal with benchmarks dependency
  (@tmcgilchrist, ocaml-community/yojson#117)
- Add a benchmark to judge the respective performance of providing a buffer vs
  letting Yojson create an internal (ocaml-community/yojson#134, @Leonidas-from-XIV)
- Add an optional `suf` keyword argument was added to functions that write
  serialized JSON, thus allowing NDJSON output. Most functions default to not
  adding any suffix except for `to_file` (ocaml-community/yojson#124, @panglesd) and functions
  writing sequences of values where the default is `\n` (ocaml-community/yojson#135,
  @Leonidas-from-XIV)

### Change

- The `stream_from_*` and `stream_to_*` functions now use a `Seq.t` instead of a
  `Stream.t`, and they are renamed into `seq_from_*` and `seq_to_*` (@gasche, ocaml-community/yojson#131).

### Fix

- Avoid copying unnecessarily large amounts of strings when parsing (ocaml-community/yojson#85, ocaml-community/yojson#108,
  @Leonidas-from-XIV)
- Fix `stream_to_file` (ocaml-community/yojson#133, @tcoopman and @gasche)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants