Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make serde classes public #386

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

jameskyle
Copy link

Description of changes:

This PR lifts the private restrictions on the SERDE classes. It also reorganizes the code to a single object/class per file for easier code navigation and discovery.

No functional or dependency changes were made.

My personal motivation for these changes is the ability to have an entirely configuration driven validation class where checks and metrics are defined in a json, yaml, or hocon configuration file.

However, I see it's come up a couple of times. Like the PR for making the AnalysisResult SERDE public and the Extending MetricsRepository Issue #232.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@twollnik
Copy link
Contributor

Hi, thanks for introducing these changes. We have no overall concerns. Do you have an idea for further test cases for example to validate your use case of having custom configurations? If nothing comes to mind, we are happy to merge as is, or we can do this as a follow-up task.

@jameskyle
Copy link
Author

jameskyle commented Oct 19, 2021

I still think this is valuable, but in my own code I ended up using Lift's native scala json library. I'd actually recommend migrating this code over to that library for a few reasons:

  • Avoids the necessity to move types into/out of native java objects.
  • Does not require an explicit serializer. Field names map to method names for case classes, for example.
  • much, much less boiler plate.

For example, here's a serialized histogram

      {
        "column": "state",
        "maxDetailBins": 1000,
        "where": "iso_country_code = 'US'"
      }

And here's how you deserialize it

import net.liftweb.json.{DefaultFormats, JValue, parse}
import com.amazon.deequ.analyzers.Histogram

val check = parse(jsonString).extract[Histogram]

I didn't dive in and go this route because the current Serde design is effected to get the cleanest implementation. So it would take a bit more thought to migrate.

@buddiex
Copy link

buddiex commented May 25, 2022

Any plans to merge this?
@jameskyle @twollnik

@jameskyle
Copy link
Author

Any plans to merge this? @jameskyle @twollnik

I actually took a different approach. writing a serde that did not require juggling java to scala types.

If this is still seen as valuable, I can merge it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants