Canot get ximmer to run on Ubuntu 16.04

[chapplec]

I have been trying (and failing) to run ximmer on an Ubuntu 16.04 LTS server for a few days now. I couldn't get the two CNV callers I tried (connifer and exomedepth) to work natively because of various issues so I turned to the docker image. I built with:

 cd ./docker && docker build -t ximmer . 

And launch with:

docker run -e XIMMER_REF=/genomes/hg19/hg19.noalt.fa -v /devel/genomes:/genomes \ 
    -v /devel/data01/chapplec/cnv/ximmer:/data  \
    -v /external.data:/external.data ximmer ximmer \
    -c /data/exome1/exome1.config  -nosim -v -o /data/output.docker 

This fails with:

    Error in library(VariantAnnotation) : 
      there is no package called 'VariantAnnotation'
    Calls: source -> withVisible -> eval -> eval -> library
    Execution halted

I assume this means I need to install VariantAnnotation in the docker image, so I ran:

docker container run ximmer R --vanilla -e "source('http://bioconductor.org/biocLite.R'); biocLite('VariantAnnotation')"

However, this fails with:

ERROR: dependencies 'AnnotationDbi', 'GenomicFeatures' are not available for package 'VariantAnnotation'
* removing '/usr/local/lib/R/site-library/VariantAnnotation'


The downloaded source packages are in
        '/tmp/RtmpXrG0Gi/downloaded_packages'
Old packages: 'KernSmooth', 'boot', 'class', 'codetools', 'foreign', 'lattice',
  'mgcv', 'nnet', 'rpart', 'spatial', 'survival'
Warning messages:
1: In install.packages(pkgs = pkgs, lib = lib, repos = repos, ...) :
  installation of package 'AnnotationDbi' had non-zero exit status
2: In install.packages(pkgs = pkgs, lib = lib, repos = repos, ...) :
  installation of package 'GenomicFeatures' had non-zero exit status
3: In install.packages(pkgs = pkgs, lib = lib, repos = repos, ...) :
  installation of package 'VariantAnnotation' had non-zero exit status

I thought I'd try installing AnnotationDbi manually, but the R version in the docker image is 2.02, quite old, so that fails with:

> install.packages('AnnotationDbi',repos='https://cloud.r-project.org/')
Installing package into '/usr/local/lib/R/site-library'
(as 'lib' is unspecified)
Warning: unable to access index for repository https://cloud.r-project.org/src/contrib
Warning message:
package 'AnnotationDbi' is not available (for R version 3.0.2) 

---

So, questions:

1. What am I doing wrong? How can I get ximmer to run and analyze my data?
2. Why is the R version so old? I have no experience with docker, but I see in the docker file I used to build the image, that it runs apt-get install -y r-base, so why is it bringing such an old version? Is the apt in the docker image configured to use the Ubuntu 14.04 repos? Can that be changed?

[ssadedin]

Hmm, I am guessing that the version of R that is default with Ubuntu 14.0.4 is actually that old, and perhaps what has happened is the libraries we depend on are no longer supported with that version of R.

I've created a branch where I've added instructions to the docker build to get R from the CRAN respositories rather than default Ubuntu repos. Could you give it a try? eg:

git pull
git checkout update-docker-r-version

Then run the docker build as normal and see if that helps.

NB: You can see some alternate install instructions here which get everything running by manually installing from scratch on Ubuntu 16.0.4:

http://ssadedin.github.io/ximmer/example.html

This may help if you encounter other issues.

[chapplec]

Thank you for the quick reply! Unfortunately, this still doesn't work. The command

docker container run ximmer R --vanilla -e "source('http://bioconductor.org/biocLite.R'); biocLite('VariantAnnotation')"
Now fails with:

* removing '/usr/local/lib/R/site-library/GenomicFeatures'
ERROR: dependency 'GenomicFeatures' is not available for package 'VariantAnnotation'
* removing '/usr/local/lib/R/site-library/VariantAnnotation'

And installing without docker, following the instructions of the link you posted fails with:

          ./bin/ximmer: line 37: /devel/data01/chapplec/cnv/ximmer/eval/tools/bpipe/0.9.9.5/bin/bpipe: No such file or directory

And, after sed -i 's/0.9.9.5/0.9.9.6/' bin/ximmer to fix the version given in the file, I get:

No tools are currently configured in the tool database. Please check your installation for the tool database configuration

Thank you again for your help, but I think I will need to forgo ximmer and go back to trying to get individual tools to run manually.

[ssadedin]

Sorry to hear that - thanks for mentioning the bpipe version part, leaves me wondering how my local docker install worked. I will make a completely clean setup so I can get this working - hope you might be willing to check back in a few days once it's ironed out, as I definitely want a clean, working docker install.

[ssadedin]

Just wanted to follow up: I've done a complete revamp of the Docker container, as there were a couple of other issues. If you're up for another try it would be great to get your feedback. As a note, the instructions for building the docker image have also changed in that you now should run directly from the Ximmer main directory rather than inside the docker folder. ie:

git clone https://github.com/ssadedin/ximmer.git
cd ximmer
docker build -t ximmer . 

[chapplec]

Thanks for taking the time! I will try and test it in the next few days and let you know.

[chapplec]

I finally found the time to test this again. I tried running with:

docker run -e XIMMER_REF=/genomes/hg19/hg19.noalt.fa -v /devel/genomes:/genomes \ 
    -v /devel/data01/chapplec/cnv/ximmer:/data  \
    -v /external.data:/external.data ximmer ximmer \
    -c /data/exome1/exome1.config  -nosim -v -o /data/output.docker 

It now fails with:

groovy: JAVA_HOME not set and cannot find javac to deduce location, please set JAVA_HOME.

I tried setting JAVA_HOME on launch:

docker run -e XIMMER_REF=/genomes/hg19/hg19.noalt.fa -v /devel/genomes:/genomes \ 
    -v /devel/data01/chapplec/cnv/ximmer:/data  \
    -e JAVA_HOME=/java/ -v /usr/lib/jvm/java-8-openjdk-amd64/:/java
    -v /external.data:/external.data ximmer ximmer \
    -c /data/exome1/exome1.config  -nosim -v -o /data/output.docker 

But that made no difference. Neither did running export JAVA_HOME= /usr/lib/jvm/java-8-openjdk-amd64/ in my shell session before running the docker command.

I see that the log output has this line, however:

unset JAVA_HOME; export PATH="/data/output.docker/bin:$PATH";
So it looks like you're unsetting JAVA_HOME, so it makes sense it then complains that it is unset :)

[bopohdr]

I am having similar/same problem.
Running with :
docker run

-e XIMMER_REF=/reference/hg38.fa 
-v /Users/dmitrijsrots/Downloads/NGS_Afib/reference:/reference 
-v /Users/dmitrijsrots/Downloads/NGS_Afib/data:/data 
ximmer ximmer 
-c /data/config.groovy 
-o /data/output.docker

fails with:
`

====================================================================================================
|                              Starting Pipeline at 2018-09-30 15:34                               |
====================================================================================================

============================================ Stage init ============================================

================================== Stage create_analysable_target ==================================
groovy: JAVA_HOME not set and cannot find javac to deduce location, please set JAVA_HOME.
ERROR: stage create_analysable_target failed: Command in stage create_analysable_target failed with exit status = 1 : 


unset JAVA_HOME; export PATH="/data/output.docker/bin:$PATH";
unset GROOVY_HOME`

I have tried to set JAVA_HOME path in the dockerfile (as stated in docker recommendation for ENV).
As well as tried to make path and move JAVA_HOME to place, which is stated in dockerfile:
ENV JAVA_HOME="/usr/lib/jvm/java-8-openjdk-amd64/jre"
I also tried with deleted dockerfile in ximmer folder.

Maybe ximmer is not referring to the dockerfile ????

The problem occuring in mac OS and in Windows 10 using build in terminals or docker's CLI.