Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

serialize more schema information beyond just the schema's qualified identifier #1

Open
jrevels opened this issue Jun 13, 2021 · 0 comments

Comments

@jrevels
Copy link
Member

jrevels commented Jun 13, 2021

Legolas upon release will already include Legolas.LEGOLAS_SCHEMA_QUALIFIED_METADATA_KEY, Legolas.schema_qualified_string, etc. to facilitate automated schema discovery for downstream consumers. This approach still expects consumers to have the requisite schema definitions preloaded if they want to do anything with this information, however.

We could serialize more schema information; at the farthest end of this spectrum, we could serialize a schema's full Julia definitions. In that case, I'd fear people would start writing consumption code that automatically eval'd such code, which could be a pretty significant attack vector (though maybe it's not any worse than folks using .jls files or JLSO.jl?). Also this still, in the most general sense, would require the consumer to eval this information into an appropriately compatible Julia environment.

A nice middle ground might be to serialize the schema's expected column names/types, which is already the only thing that Legolas.validate really checks anyway.

Note that the reason I'm not including this kind of thing in Legolas's initial release is that I'd like to see whether practical usage ends up demonstrating a clear need for it; I'm not sure it will. I'm also not sure how this would interact with some of the versioning tips given to schema authors here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant