Installing python modules

Sometimes it is useful to install your own python modules rather than asking Catherine and Greg to add them to the sector image: either you want to play with the code, you want to track a rapidly developing project more closely, there are some issues with adding it to the image or just for the hell of it.

If the modules are written in pure python (and so requires no compilation), then doing this is quite easy. The general approach is to:

• obtain the source code, either via checking out from a repository or by downloading a tarball. The Python Cheeseshop has a great many useful packages. Make sure you download the tarball rather than the egg, as we will not be using the easy_install mechanism (which isn't installed on the workstations).
• add the relevant path of the source code to your PYTHONPATH (this is often the directory which a script called __init__.py, and is usually not the main directory a tarball extracts itself to, or is a single file, as in the case of using the environment modules in python). If you're lucky, the documentation will describe how to install a development version: follow them in that case!
• add any useful scripts to your PATH.

It is somewhat inconvenient to having to keep extending your PATH and PYTHONPATH environment variables (plus, it makes it much easier to completely barf them). I use the following setup (which models how ubuntu seems to work on my desktop at home):

• ~/local/src/python: I place all the python source code here, in whatever directory structure each module likes.
• ~/local/lib/python/site-packages: I link the python module (the path which needs to be added to PYTHONPATH) here.

Then my .bashrc contains the following:

test -d ~/local/lib/python/site-packages && export PYTHONPATH=~/local/lib/python/site-packages:${PATH}

which adds ~/local/lib/python/site-packages to my PYTHONPATH (if the directory exists). This means I no longer have to worry about adding a new module to my PYTHONPATH.

A few examples illustrate this.

IPython

IPython is a vastly improved shell to the standard interactive python interpreter. A version is on the workstations and may of the servers, but it is being developed at a very rapid rate (and no packages are keeping up). Some really useful features are being added on a frequent basis. You can download it using bzr (which will be on the 10.3 image) or the tarballs from launchpad. Their developer notes give some useful information (but the notes on IPython and IPython1 are slightly outdated---a huge effort has recently merged the divergent codes together). I did the following:

• Extracted the tarball to ~/local/src/python/ipython
• Linked the module to site-packages:

ln -s ~/local/python/src/ipython/IPython ~/local/lib/python/site-packages/IPython

I didn't need to add the ipython startup script to my PATH: the existing one picked up my local IPython module instead of the older one installed on the image without any problems.

Sphinx

Sphinx is a tool to make beautiful documentation, both in html and in pdf (via LaTeX) formats, from reStructuredText, a light-weight, intuitive markup language. This has the advantage that the documentation source is easy-to-read and write (unlike html and LaTeX), so is good for working in a terminal, yet also you have all the advantages of web and pdf documents if you need them. Sphinx is used for producing documentation by python itself, IPython, matplotlib and the Alavi group, amongst others. As it is written in python, it is also very easy to add extensions (including the ability to use LaTeX to produce equations). Quite simply, I am a fan. ;-)

Sphinx is a little bit more involved to install, as it has three dependencies (docutils, Pygments and Jinja 1.2). I did the following:

• Downloaded the tarballs for docutils, Pygments and Jinja 1.2. They're available from the Python cheeseshop or from the project websites. I have also put them on chimaera: ~jss43/python/.
• Extracted them in ~/local/src/python.
• I downloaded Sphinx via svn (a tarball also exists, but I was working remotely, so svn was marginally easier!):

svn co http://svn.python.org/projects/doctools/trunk ~/local/src/python/sphinx

• Linked the relevant files and folders to site-packages:

ln -s ~/local/python/sphinx/sphinx ~/local/lib/python/site-packages/spinx
ln -s ~/local/python/docutils/docutils ~/local/lib/python/site-packages/
ln -s ~/local/python/Jinja-1.2/jinja ~/local/lib/python/site-packages/jinja
ln -s ~/local/python/Pygments-0.10/pygments ~/local/lib/python/site-packages/pygments

• docutils has a few standalone modules that also need to be added:

for module in ~/local/python/docutils/extras/*py; do ln -s $module ~/local/lib/python/site-packages/`basename $module`; done

The last step is to add the relevant sphinx scripts to the PATH. The necessary files are sphinx-build.py and sphinx-quickstart.py. Helpfully, sphinx expects them to be available as sphinx-build and sphinx-quickstart. I did:

ln -s ~/local/python/sphinx/sphinx/sphinx-quickstart.py ~/bin/sphinx-quickstart && ln -s ~/local/python/sphinx/sphinx/sphinx-build.py ~/bin/sphinx-build

as ~/bin exists on my path.

To initialise a documentation project, run sphinx-quickstart and answer the questions. For more information, see the Sphinx documentation.