Tukukkk

I am building a docker image that uses conda (conda 4.5.10 & miniconda 4.5.4 are installed on the base image) to create an environment and install some packages. I am having a problem whereby after the conda create this subsequent docker command fails:

RUN source /opt/conda/bin/activate /opt/conda/envs/python2 && 

    pip install --no-cache-dir --upgrade pip && 

    pip install --no-cache-dir --upgrade -r /requirements_py2_pip.txt && 

    rm /requirements_py2_pip.txt

Error is:

Step 12/17 : RUN source /opt/conda/bin/activate /opt/conda/envs/python2 && pip install --no-cache-dir --upgrade pip && pip install --no-cache-dir --upgrade -r /requirements_py2_pip.txt && rm /requirements_py2_pip.txt

---> Running in e828112b4ae0 .
Could not find platform independent libraries

Could not find platform dependent libraries

Consider setting $PYTHONHOME to [:]

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007fe6d4851700 (most recent call first):

The command '/bin/bash -c source /opt/conda/bin/activate /opt/conda/envs/python2 && pip install --no-cache-dir --upgrade pip && pip install --no-cache-dir --upgrade -r /requirements_py2_pip.txt && rm /requirements_py2_pip.txt' returned a non-zero code: 134

The fact that it fails intermittently really bugged me, if it failed every time I wouldn't mind so much but it fails approximately 50% of the time. I googled around and found this SO thread: ImportError: No module named 'encodings' that suggested it may be a problem with the conda environments. Hence, I added this docker command prior to the one that was failing intermittently:

RUN conda env list

Annoyingly that command fails intermittently too. When it succeeds it returns:

Step 12/18 : RUN conda env list

---> Running in 05eaa6f726a9

# conda environments:

#

base * /opt/conda

python3 /opt/conda/envs/python3

python3.6 /opt/conda/envs/python3.6

When it fails it returns:

Step 12/18 : RUN conda env list

---> Running in 20247acc3824

Could not find platform independent libraries

Could not find platform dependent libraries

Consider setting $PYTHONHOME to [:]

Fatal Python error: Py_Initialize: Unable to get the locale encoding

ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f7a5f5ef700 (most recent call first):

The command '/bin/bash -c conda env list' returned a non-zero code: 139

Which as you will see is very similar to the previous error. This suggested to me that there was a problem with the conda environment which is created using this command:

# Create a Python 2.x environment using conda including at least the ipython kernel

# and the kernda utility. Add any additional packages you want available for use

# in a Python 2 notebook to the first line here (e.g., pandas, matplotlib, etc.)

RUN conda create --quiet --yes -p $CONDA_DIR/envs/python2 python=2.7 --file requirements_py2_conda.txt && 

    source /opt/conda/bin/activate /opt/conda/envs/python2 && 

    conda remove --quiet --yes --force qt pyqt && 

    conda clean -tipsy && 

    rm -rf $CONDA_DIR/envs/python2/share/jupyter/lab/staging && 

    rm -rf /usr/local/share/.cache /tmp/* /opt/conda/pkgs/* /opt/conda/envs/python2/pkgs/* 

    /requirements_py2_conda.txt /opt/conda/envs/python3/pkgs/*

I compared the output of that command in both a succeeded and a failed build. Both install the same packages, they just happen to install them in a different order.

By the way, requirements_py2_conda.txt contains:

beautifulsoup4=4.6.*  

bokeh=0.13.*  

bz2file=0.98  

cloudpickle=0.5.*  

colour=0.1.*  

configparser=3.5.*  

cython=0.28.*  

dill=0.2.*  

fastparquet=0.1.*  

future=0.16.*  

gensim=3.4.*  

graphviz=2.40.*  

h5py=2.8.*  

hdf5=1.10.*  

imageio=2.3.*  

ipykernel=4.8.*  

ipython=5.8.*  

ipywidgets=7.4.*  

keras=2.2.*  

lxml=4.2.*  

matplotlib=2.2.*  

mysqlclient=1.3.*  

mpld3=0.3  

nltk=3.3.*  

nose=1.3.*  

numba=0.39.*  

numexpr=2.6.*  

numpy=1.15.*  

pandas=0.23.*  

pathlib2=2.3.*  

patsy=0.5.*  

pexpect=4.6.*  

pivottablejs=0.9.*  

protobuf=3.*  

pyemd=0.5.*  

pymc3=3.5  

pyparsing=2.2.*  

pystan=2.17.*  

pytest=3.7.*  

python=2.7.*  

pylint  

py-xgboost=0.72.*  

pyyaml=3.13  

requests=2.19.*  

scandir=1.*  

scikit-image=0.14.*  

scikit-learn=0.19.*  

scipy=1.1.*  

seaborn=0.9.*  

sh=1.12.*  

simplegeneric=0.8.*  

singledispatch=3.4.0.*  

six=1.11.*  

sortedcontainers=2.0.*  

sqlalchemy=1.2.*  

SQLAlchemy=1.2.*  

statsmodels=0.9.*  

subprocess32=3.5.*  

sympy=1.2.*  

tabulate=0.8.*  

tensorflow=1.10.*  

texlive-core=20180414  

theano=1.0.*  

widgetsnbextension=3.4.*  

xlrd=1.1.*  

The error message suggests

Consider setting $PYTHONHOME to [:]

In order to see if that was the issue I added:

echo $PYTHONHOME

This resulted in the same output (i.e. empty PYTHONHOME) both when the build succeeds and when it fails:

Step 12/20 : RUN echo $PYTHONHOME

---> Running in 59ce6f2776c7

Removing intermediate container 59ce6f2776c7

---> cee1ad9f695e

I'm now stumped. I don't know what to do next in order to investigate and I'm not familiar with conda (I didn't write the Dockerfile, but these intermittent failures are blocking me hence I'm investigating). Any suggestions of things I could do to try and uncover the problem would be welcomed.