Import R data frame attributes as metadata #93
Conversation
Use the `metadata` function recently added to DataAPI and DataFrames to import R `data.frame` attributes and set them as `DataFrame` metadata. R stores per-column attributes in vector objects, while DataFrames.jl stores them in the `DataFrame` object, as there is no generic mechanism to attach metadata to an `AbstractVector` object. The `row.names` attribute is skipped as it is not appropriate to store it as global metadata given that it will get out of sync after subsetting rows. We could provide a way to turn row names into a column instead. Also add methods to check equality between two `DictoVec` objects as these are useful for tests (haven commonly sets named numeric vectors to store value labels so this case deserves testing).
|
Thanks! I'll try to review it in the next few days. |
|
@alyst - we still need to decide on the best API in JuliaData/DataAPI.jl#48 before finalizing this PR, so if you have any thoughts on this please comment there. Thank you! |
| ZonedDateTime(2022, 05, 26, 22, 5, tz"UTC")], | ||
| :default), | ||
| "comment" => ("This is a data frame", :note), | ||
| "row.names" => ([missing, -6], :default))) |
There was a problem hiding this comment.
@alyst Any idea what might be going on here? Is this due to the fact that R stores default row names as an AltRep object?
There was a problem hiding this comment.
It seems some conversion from R to Julia is missing, but it is not in the scope of this PR?
There was a problem hiding this comment.
@alyst Do you think we should tackle this here?
There was a problem hiding this comment.
No idea yet :( I think the choice between the standard and alternative representation is dynamic and depends on e.g. the length of the vector.
I agree that it would better to address this in a separate PR + add tests that explicit "row.names" attributes are handled correctly (at least that their presence doesn't break the whole ".rda" import).
There was a problem hiding this comment.
OK. TBH I'm not sure I'd be able to fix that but I can try to have a look.
There was a problem hiding this comment.
| "row.names" => ([missing, -6], :default))) | |
| "row.names" => ([missing, -6], :default))) # FIXME check the conversion |
There was a problem hiding this comment.
OK. TBH I'm not sure I'd be able to fix that but I can try to have a look.
I'll try to have a look within 1-2 weeks.
Co-authored-by: Alexey Stukalov <[email protected]>
Co-authored-by: Bogumił Kamiński <[email protected]>
Codecov ReportBase: 85.75% // Head: 86.64% // Increases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## main #93 +/- ##
==========================================
+ Coverage 85.75% 86.64% +0.88%
==========================================
Files 14 14
Lines 646 689 +43
==========================================
+ Hits 554 597 +43
Misses 92 92
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
|
Ah, DataFrames 1.4 requires Julia 1.6, so we have to drop support for older Julia versions here too. But the convention is to bump the minor release when doing that, so that we would still be able to tag a bugfix release on 1.0 if necessary. Maybe take the occasion to tag RData 1.0? |
|
I think tagging 1.0.0 is a good idea. |
|
Thanks! See JuliaRegistries/General#69824 |
Use the
metadatafunction that are going to be added to DataAPI and DataFrames (JuliaData/DataFrames.jl#3055) to import Rdata.frameattributes and set them asDataFramemetadata.R stores per-column attributes in vector objects, while DataFrames.jl stores them in the
DataFrameobject, as there is no generic mechanism to attach metadata to anAbstractVectorobject.The
row.namesattribute is skipped as it is not appropriate to store it as global metadata given that it will get out of sync after subsetting rows. We could provide a way to turn row names into a column instead.Also add methods to check equality between two
DictoVecobject as these are useful for tests (haven commonly sets named numeric vectors to store value labels so this case deserves testing).(Test pass locally when using DataAPI and DataFrame branches.)