Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework tests to only perform a single fetch data from IANA servers #445

Open
omus opened this issue Aug 22, 2023 · 1 comment
Open

Rework tests to only perform a single fetch data from IANA servers #445

omus opened this issue Aug 22, 2023 · 1 comment

Comments

@omus
Copy link
Member

omus commented Aug 22, 2023

The current TimeZones.jl setup fetches a specific version of the tzdata from the IANA services for running tests. When running on CI this data is always fetched at least once. It was noticed in #441 that we perform a second fetch from the IANA services in tzdata/build.jl which hasn't been problematic but causes unnecessary load on their servers.

We may also want to look into using GHA caching to avoid downloading from their servers entirely.

@omus
Copy link
Member Author

omus commented Aug 23, 2023

Definitely should use some GHA caching:

[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
ERROR: LoadError: InitError: RequestError: Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds while requesting https://data.iana.org/time-zones/releases/tzdata2016j.tar.gz

https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554201#step:6:131

Looks like the tzdata_version check may also be an isssue:

[ Info: Downloading Windows to POSIX timezone ID XML version: release-43-1
[ Info: Compiling Windows time zone name translation
[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
[ Info: Decompressing tzdata 2016j region data
[ Info: Compiling tzdata 2016j region data
tzdata_version: Error During Test at D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:57
  Got exception outside of a @test
  RequestError: HTTP/1.1 200 OK (Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds) while requesting https://data.iana.org/time-zones/releases/
  Stacktrace:
    [1] (::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool})(easy::Downloads.Curl.Easy)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:388
    [2] with_handle(f::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, handle::Downloads.Curl.Easy)
      @ Downloads.Curl C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Curl\Curl.jl:90
    [3] #8
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:329 [inlined]
    [4] arg_write(f::Downloads.var"#8#17"{Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
    [5] #7
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:328 [inlined]
    [6] arg_read
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:76 [inlined]
    [7] request(url::String; input::Nothing, output::IOBuffer, method::Nothing, headers::Vector{Pair{String, String}}, timeout::Float64, progress::Nothing, verbose::Bool, debug::Nothing, throw::Bool, downloader::Nothing)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:327
    [8] request
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:295 [inlined]
    [9] #3
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:233 [inlined]
   [10] arg_write(f::Downloads.var"#3#4"{Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Nothing, String}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
   [11] #download#2
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:232 [inlined]
   [12] download
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:221 [inlined]
   [13] tzdata_versions()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:56
   [14] tzdata_latest_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:81
   [15] tzdata_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\version.jl:92
   [16] (::var"#16#19")()
      @ Main D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:69
   [17] withenv(f::var"#16#19", keyvals::Pair{String, String})
      @ Base .\env.jl:197
   [18] macro expansion
      @ D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:68 [inlined]
   [19] macro expansion
...

https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554082#step:6:137

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant