Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

load("bin") broken by drop of DataType mutable field #408

Open
mrufsvold opened this issue Feb 22, 2022 · 4 comments · May be fixed by #409
Open

load("bin") broken by drop of DataType mutable field #408

mrufsvold opened this issue Feb 22, 2022 · 4 comments · May be fixed by #409

Comments

@mrufsvold
Copy link

I am trying to load a dataset from a bin directory for some out-of-core processing. However, when I run bintbl = load("bin"), I get:

ERROR: LoadError: type DataType has no field mutable
Stacktrace:
  [1] getproperty
    @ .\Base.jl:37 [inlined]
  [2] _deser(io::Serialization.Serializer{IOStream}, t::Type)
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:243
  [3] deserialize(io::Serialization.Serializer{IOStream}, DT::Type{JuliaDB.DIndexedTable{NamedTuple{(:datasetId, Symbol("id.patient"), Symbol("id.practice"), :V1, :V2, :V3, :V4, :V5), Tuple{String, Int64, Int64, Float64, Int64, Int64, Float64, String}}, Tuple{Int64}}})
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:239
  [4] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:870
  [5] deserialize(s::Serialization.Serializer{IOStream})
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801
  [6] handle_deserialize(s::Serialization.Serializer{IOStream}, b::Int32)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:907
  [7] deserialize
    @ C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:801 [inlined]    
  [8] deserialize(s::IOStream)
    @ Serialization C:\Users\mrufsvold\AppData\Local\Programs\Julia-1.7.0\share\julia\stdlib\v1.7\Serialization\src\Serialization.jl:788
  [9] #198
    @ C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:176 [inlined]
 [10] open(f::JuliaDB.var"#198#199", args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base .\io.jl:330
 [11] open
    @ .\io.jl:328 [inlined]
 [12] load(f::String; procs::Vector{Int64})
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:175
 [13] load(f::String)
    @ JuliaDB C:\Users\mrufsvold\.julia\packages\JuliaDB\7cG1k\src\io.jl:174
 [14] top-level scope
    @ c:\Users\mrufsvold\Projects\acic-data-challenge\src\modules\juliadb_utils.jl:9

I believe this is related to this issue noting that in Julia 1.7, DataType dropped the field mutable which broke BSON.jl.

I'm not the most advanced Julia user, so I could just be making an error myself. So I apologize if this is just me!

@jpsamaroo
Copy link
Collaborator

You should be able to change t.mutable to ismutabletype(t) at

t.mutable && Serialization.deserialize_cycle(io, x)
.

@mrufsvold
Copy link
Author

I was able to fix this in 1.7 by doing exactly what you suggested.

I know the fix is microscopic, but I'm happy to submit a PR with this.

I also added

# Define check mutability function based on Julia version
_ver1_5orgreater = all(parse.(Ref(Int), split(string(VERSION), ".")) .>= [1,5,0])
if _ver1_5orgreater
    checkmutability(t) = Base.ismutable(t) 
else
    checkmutability(t) = t.mutable
end

So that it will still run in older versions of Julia.

Could leave that in or cut it. Not sure how much backwards compatibility is a priority.

@jpsamaroo
Copy link
Collaborator

A PR would be much appreciated!

1.6 is the new LTS, so I would potentially just set the Julia compat bound in the Project.toml to 1.6. Also, a better way to compare versions is probably VERSION >= v"1.5".

@mrufsvold
Copy link
Author

Oh, yeah, that's a way better comparison!

I'll get the PR in shortly. Thanks for your help!

@mrufsvold mrufsvold linked a pull request Feb 22, 2022 that will close this issue
@jpsamaroo jpsamaroo linked a pull request Feb 22, 2022 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants