Having issues with one of the RUN in the docker file

[howardxu520]

Hi Brian, I'm having issues with RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt in the docker file. Now I'm just using a copy of deseq2/1.39's files and only added one line in conda-packages.txt file, bioconductor-ensdb.hsapiens.v79.

But when I try to build it, I always encounter issues with the error below:

(base) shuhaoxu@Shuhaos-MBP 1.39.3 % docker build -t deseq2/1.39.3 .
[+] Building 617.1s (10/12)                                                                          
 => [internal] load .dockerignore                                                               0.0s
 => => transferring context: 2B                                                                 0.0s
 => [internal] load build definition from Dockerfile                                            0.0s
 => => transferring dockerfile: 1.62kB                                                          0.0s
 => [internal] load metadata for docker.io/jupyter/datascience-notebook:hub-2.2.2               1.1s
 => [auth] jupyter/datascience-notebook:pull token for registry-1.docker.io                     0.0s
 => [1/7] FROM docker.io/jupyter/datascience-notebook:hub-2.2.2@sha256:f5984966ed1024e70369860  0.0s
 => [internal] load build context                                                               0.0s
 => => transferring context: 825B                                                               0.0s
 => CACHED [2/7] COPY apt-packages.txt /tmp/apt-packages.txt                                    0.0s
 => [3/7] COPY conda-packages.txt /tmp/conda-packages.txt                                       0.0s
 => [4/7] RUN apt-get update && xargs apt-get install -y < /tmp/apt-packages.txt              231.7s
 => ERROR [5/7] RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.t  384.7s 
------                                                                                               
 > [5/7] RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt:             
------                                                                                               
Dockerfile:26                                                                                        
--------------------                                                                                 
  24 |     
  25 |     # Install additional R packages from conda-forge using mamba
  26 | >>> RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt
  27 |     
  28 |     RUN Rscript -e 'install.packages("devtools", repos = "https://cran.us.r-project.org")'
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt" did not complete successfully: exit code: 137

I also tried to just switch using mamda to conda , which did not work neither.

[byee4]

You might be running out of memory. See if you can increase your Docker desktop to use 8G if your computer can handle it?

[howardxu520]

OK, Thank you!

[howardxu520]

Hi Brian, I tried increasing my CPU to 8, memory to 8GB, Swap to 3GB, Vitual disk limit to 136GB. But I still failed the build.

(base) shuhaoxu@Shuhaos-MBP 1.39.3 % docker build -t deseq2/1.39.3 .
[+] Building 1554.3s (10/12)                                                    
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => [internal] load build definition from Dockerfile                       0.0s
 => => transferring dockerfile: 1.62kB                                     0.0s
 => [internal] load metadata for docker.io/jupyter/datascience-notebook:h  1.2s
 => [auth] jupyter/datascience-notebook:pull token for registry-1.docker.  0.0s
 => [1/7] FROM docker.io/jupyter/datascience-notebook:hub-2.2.2@sha256:f5  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 259B                                          0.0s
 => CACHED [2/7] COPY apt-packages.txt /tmp/apt-packages.txt               0.0s
 => CACHED [3/7] COPY conda-packages.txt /tmp/conda-packages.txt           0.0s
 => CACHED [4/7] RUN apt-get update && xargs apt-get install -y < /tmp/ap  0.0s
 => ERROR [5/7] RUN mamba install -y -c bioconda -c conda-forge --file  1552.9s
------
 > [5/7] RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt:
------
Dockerfile:26
--------------------
  24 |     
  25 |     # Install additional R packages from conda-forge using mamba
  26 | >>> RUN mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt
  27 |     
  28 |     RUN Rscript -e 'install.packages("devtools", repos = "https://cran.us.r-project.org")'
--------------------
ERROR: failed to solve: process "/bin/bash -o pipefail -c mamba install -y -c bioconda -c conda-forge --file /tmp/conda-packages.txt" did not complete successfully: exit code: 137

However, it does last longer, from around 300 seconds to 1552.9 seconds. But I think It will take a total of over 5000 seconds to build. It's still far away from finishing building.

I have those packages on my computer. So we are not in rush to update deseq2. I also want to take this as a chance for me to practice using dock container which can definitely help me to be more prepared for TSCC.

[howardxu520]

Hi Brian, I created a new folder in my branch called Visual. There are two folders inside, 1.0.0 and 1.0.1. Each one is a different way I try to build an image that contains ensdb.hsapiens.v79. The apt-packages.txt is the same one from DESeq2/1.39. I believe apt-packages.txt ran successfully. I stored the error messages in two files and uploaded here. 1.0.1's error is new to me and fails by not being able to install package bioconda::bioconductor-genomeinfodbdata-1.2.9-r42hdfd78af_0. I think this is the case you said sometimes the environment doesn't work out for some reasons.

error-1.0.0.txt
error-1.0.1.txt

[howardxu520]

I have another attempt in folder 1.0.2. Here is the error message. It's nice to see these different error messages so more I will know what to do with them.

error-1.0.2.txt