dependency error, platform mismatch and other errors when renv.config.pak.enabled = TRUE #2051
Comments
mihem
changed the title
|
Here's the code remotes <- c(MASS = "[email protected]", Matrix = "[email protected]", R6 = "[email protected]",
RColorBrewer = "[email protected]", cli = "[email protected]", colorspace = "[email protected]",
fansi = "[email protected]", farver = "[email protected]", ggplot2 = "[email protected]",
glue = "[email protected]", gtable = "[email protected]", isoband = "[email protected]",
labeling = "[email protected]", lattice = "[email protected]", lifecycle = "[email protected]",
magrittr = "[email protected]", mgcv = "[email protected]", munsell = "[email protected]",
nlme = "[email protected]", pillar = "[email protected]", pkgconfig = "[email protected]",
rlang = "[email protected]", scales = "[email protected]", tibble = "[email protected]",
utf8 = "[email protected]", vctrs = "[email protected]", viridisLite = "[email protected]",
withr = "[email protected]")
pak::pkg_install(remotes)For what it's worth, I wasn't able to reproduce locally on macOS: > options(renv.config.pak.enabled = TRUE); renv::restore(lockfile = "renv.lock", library = tempdir())
The following package(s) will be installed:
- pak [0.7.2]
These packages will be installed into "/private/var/folders/9n/3nxsgkrj0rlfx196kzdttpww0000gn/T/Rtmp9VmNNJ".
Do you want to proceed? [Y/n]: y
# Installing packages --------------------------------------------------------
- Installing pak ... OK [copied from cache in 0.36s]
Successfully installed 1 package in 0.37 seconds.
✔ Loading metadata database ... done
→ Will install 23 packages.
→ Will update 1 package.
→ Will download 1 CRAN package (6.17 MB), cached: 20 (21.42 MB).
→ Will download 3 packages with unknown size.
+ cli 3.6.3 🔧
+ colorspace 2.1-1 🔧
+ fansi 1.0.6 🔧
+ farver 2.1.2 🔧
+ ggplot2 3.5.1
+ glue 1.7.0 👷🏾♀️🔧 ⬇
+ gtable 0.3.5 👷🏾♀️ ⬇
+ isoband 0.2.7 🔧
+ labeling 0.4.3
+ lifecycle 1.0.4
+ magrittr 2.0.3 🔧
+ Matrix 1.7-0 → 1.7-1 🔧 ⬇ (6.17 MB)
+ munsell 0.5.1
+ pillar 1.9.0
+ pkgconfig 2.0.3
+ R6 2.5.1
+ RColorBrewer 1.1-3
+ rlang 1.1.4 🔧
+ scales 1.3.0 🔧
+ tibble 3.2.1 🔧
+ utf8 1.2.4 🔧
+ vctrs 0.6.5 🔧
+ viridisLite 0.4.2
+ withr 3.0.1 👷🏻 ⬇ |
|
@kevinushey Thanks for the fast response. What about reproducing using the above Dockerfile in combination with the renv.lock? Then you have a fresh ubuntu environment and we don't have any problems reproducing issues across machines? And at the end the Thanks!! |
|
Sorry, but I'm still not able to reproduce. I tried using a modified version of your Dockerfile: # get tidyverse docker from rocker
FROM rocker/r-ver:4.4.2
# system libraries
# Try to only install system libraries you actually need
# Package Manager is a good resource to help discover system deps
RUN apt-get update --yes \
&& apt-get upgrade --yes
# install R packages required
RUN R -e 'install.packages("https://packagemanager.posit.co/cran/latest/src/contrib/Archive/renv/renv_1.0.10.tar.gz")'
RUN echo "options(Ncpus = 6, renv.config.pak.enabled = TRUE)" | tee /usr/local/lib/R/etc/Rprofile.site
# Copy renv files
WORKDIR /testproject
COPY renv.lock renv.lock
# Restore the R environment
RUN R -e "renv::restore(lockfile = 'renv.lock')"and it all built successfully. |
|
@kevinushey thank you for trying to reproduce this issue with my dockerfile. Wow, I always thought that docker solves the "this does not work on my machine" problem. I used your Dockerfile and still get the same error. Do you have any idea how I can make this more reproducible for you? docker_renv_debug-2024-12-06_21.38.37.mp4 |
|
That is incredibly odd! Just to rule it out, are we definitely using the same lockfile? I have: For what it's worth, here's the session info in the container; I assume it must be the same in your case... this is captured using And this is the full output I see when invoking |
|
Also, |
|
@kevinushey . Thanks for digging into this. Agree, very strange. Yes,
To be on the safe side, I tried to avoid all caching when running your dockerfile docker logSo the same error as before. and adding renv diagnosticsI think this is the same as your diagnostics so really do not understand where the bug is. |
|
@kevinushey To further debug this, i created a github repo: https://github.com/mihem/docker_renv_pak_debug To make it more reproducible, I created a github workflow to build the docker image via GitHub Actions. I am happy that I can reproduce this not only on my local machine. I tried to add mac-os to ubuntu-latest, but there seems to be a licensing problem with mac os and docker. So I cannot reproduce your successful building. But I hope this helps you to further debug this issue. |
|
One possibility is that it works on my machine because I'm using an aarch64 machine, not an x86_64 one... I'll see if I can reproduce later on an x86_64 machine next. (That could also affect what packages PPM would report as being available, I suspect?) |
|
Ah, I missed that details in rev diagnostics, so yes maybe you are right, it's because of arm64. However, this did not work. It's only available to Team or Enterprise users. But since x86_64 is still the default, I think this should not be too hard to find some machine to debug this? Excited what you'll find out. Also happy to help with debuggin on my machine. |
|
You can run most x86_64 images on Apple Silicon Macs by explicitly requesting that architecture with the
|
|
Thanks -- I was able to reproduce the issue on my x86_64 machine. I suspect the issue here arises because And indeed, I can reproduce the same error with: As a workaround, you can request that In this scenario, |
|
@kevinushey great that you could reproduce the problem. I agree this workaround sucessfully resolves the dependencies. However, my use case is that i have a project with lots of packages in renv.lock which take ~60min to download and ~90min to builld (using PPM). I would like to reduce that time as much as possible. One idea that you recommended to speed up @remlapmot Thanks, did not know that. But does that also mean that I can just use on my x86 machine using this dockerfile https://github.com/mihem/docker_renv_pak_debug . But this failed with If that would work, this could be a better workaround that not using binaries, I think. |
|
I don't really see what building an arm64 image would benefit you (unless your target machine is actually arm). I believe that you can build arm64 images under x86_64 Docker. However I guess there are lots of edge cases where that emulation fails and it will be probably be very slow. It's most reliably done through a call to But RSPM/PPPM doesn't have Linux ARM binary packages, so all the packages would be built from source in the ARM image. If you want the fastest Linux build possible I'd stick to amd64 and use a combination of package versions that you know that pak can resolve. In my experience the easiest way to choose such a version combination is to take all packages from a specific date on CRAN by setting pak to use a PPPM snapshot date. See https://pak.r-lib.org/reference/get-started.html?q=pak::repo_add#time-travel-using-rspm, so e.g., call |
|
@remlapmot Thanks! Well since kevin reported that this worked with arm64, while i did not work with x86, I had hoped that i could just specify the platform and build it on my x86 machine. Same error when using Yes, I want the fasted Linux build possible. But I want to restore my historic academic project with the exact versions that are given in renv. Otherwise I am worried that this is not compatible with my script or even lead to different results. And since the results are already published, I would like to keep everything as it is. Just freeze it (as fails with This only works with dates till January 2024 or so. |
|
Okay sorry, I was wrong about the last problem, I could resolve this. So |
|
For what it's worth, I only mentioned arm64 because I was working on my M1 macOS machine, with an aarch64 Linux virtual machine therein. It seems like Ultimately, this is either an issue in |
|
Isn't this r-lib/pak#724? |
|
It is, but I wasn't sure if |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have a large project with many packages. When I want to build a Docker image as explained here https://rstudio.github.io/renv/articles/docker.html this is painfully slow. Only downloading takes over an hour, which is not due to a slow internet connection, the r docker image that this image is build on is probably larger than all packages combined and is downloaded in a minute or so.
I`ve read that pak considerably speeds up this process and pak is available for renv since some time.
However, I get lots of errors when I set
renv.config.pak.enabled = TRUE.I tried this with a minimal reproducible example, where I only used
ggplot2.When using ggplot2 and dependencies with my currently installed version this fails:
renv.lock
Dockerfile
log
I can solve this problem by disabiling pak, but I would like to use pak because it's supposed to be much faster.
I can also solve this problem by updating all packages in my local environment by
pak::pak("ggplot2")andrenv::snapshotwhich does the followingThis is no problem in this toy example. But in my real project, I have a historical scientific project. I would like to build a docker images for this exact enviroment, so using the exact same versions recorded in renv, to ensure reproducibility.
@kevinushey could you maybe help me here? I've found similar issues, especially here #1829. But i didn't find an answer to that problem.
Thanks!
The text was updated successfully, but these errors were encountered: