This posting is motivated by cases where you are stuck using an older server OS (CentOS 5.8 in this case) and want to install an up-to-date python 2.7 along with a raft of necessary packages. What is the best way?
distribute? pip? setuptools? virtualenv? What to make of all this?
Bottom line: use pip. To install pip, you either install it from source, or else install virtualenv (which includes pip). Installing from source allows you to manage packages with pip globally on your system (and not in a virtual environment), then you have to install pip from source. But to install pip yourself, you first need to install setuptools. All of these installs should be done using the target python interpreter. Got that?
I wrote the following notes after digesting the helpful hints given at a variety of places, such as
these two gist pages. Note that although there are many tutorials that still guide you to install virtualenv with easy_install (such as
here or
here), we're actually
not supposed to do it this way any longer. My notes that follow will not make use of virtualenv, but in some situations this might be desirable -- this is just for a system-wide fresh installation of Python 2.7 and pip to install packages. (
Here is a nice overview of setting up virtualenv if you want to go that route.)
First you want to be sure your CentOS machine is ready to built something like Python:
yum groupinstall "Development tools"
yum install zlib-devel bzip2-devel openssl-devel ncurses-devel
Then you will fetch and unzip Python:
wget http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
tar xf Python-2.7.5.tar.bz2
cd Python-2.7.5
Do the usual "configure, make, make install" waltz:
./configure --prefix=/opt/python2.7.5 --with-threads --enable-shared
makemake install
At this point, it seems like you're done but not quite.
$ /opt/python2.7.5/bin/python
/opt/python2.7.5/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory
/opt/python2.7.5/
The trick is to tell ldconfig where to find the new python shared libraries:
echo "/opt/python2.7.5/lib/" > /etc/ld.so.conf.d/opt-python2.7.5.confldconfig
Now we're cooking with gas:
$ /opt/python2.7.5/bin/python
Python 2.7.5 (default, Aug 21 2013, 09:50:18)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Get setuptools from the
setuptools project page. You have to click on the version you want and then to find the download link, scroll to the very bottom of the page ("In the basement, down a long hallway behind a door marked 'danger-TIGER!'")
Aside: I am having bad luck these days trying to wget from https servers -- something about SSL certificates and my corporate proxy and the old version of wget that is on CentOS 5.8.
# wget "https://pypi.python.org/packages/source/s/setuptools/setuptools-1.0.tar.gz"
--2013-08-21 11:49:47-- https://pypi.python.org/packages/source/s/setuptools/setuptools-1.0.tar.gz
Resolving proxy.company.com...
Connecting to proxy.company.com:8080... connected.
ERROR: certificate common name `*.a.ssl.proxy.net' doesn't match requested host name `pypi.python.org'.
To connect to pypi.python.org insecurely, use `--no-check-certificate'.
Unable to establish SSL connection.
So curl seems to work better. Maybe because it is not checking SSL certs?
# curl -O "https://pypi.python.org/packages/source/s/setuptools/setuptools-1.0.tar.gz"
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 663k 100 663k 0 0 229k 0 0:00:02 0:00:02 --:--:-- 359k
Once you have downloaded and untar'ed setuptools, you want to install it. But it seems the setuptools
instructions appear flawed:
# /opt/python2.7.5/bin/python setup.py --prefix=/opt/setuptools
error: option --prefix not recognized
So just do the usual "python setup.py build and install" two-step. Be sure you are calling the desired target python version.
/opt/python2.7.5/bin/python setup.py build
/opt/python2.7.5/bin/python setup.py install
Now you can install pip:
curl -O https://pypi.python.org/packages/source/p/pip/pip-1.4.1.tar.gz
tar xvzf pip-1.4.1.tar.gz
cd pip-1.4.1
/opt/python2.7.5/bin/python setup.py build
/opt/python2.7.5/bin/python setup.py install
And now...
now... NOW! Praise the Almighty, you are ready to use pip to install packages!!!
/opt/python2.7.5/bin/pip install numpy # install this before biopython
/opt/python2.7.5/bin/pip install scipy
/opt/python2.7.5/bin/pip install biopython
/opt/python2.7.5/bin/pip install pysam
/colorblind-gray.png)
/ggplot2_scale_hue_colors.png)