Python software on Archer

For running Python on work nodes on Archer, there are python-compute (native) and anaconda-compute modules available. Users are discouraged from using anaconda-compute because it is not optimised for running on Archer. There are some preinstalled packages for python-compute and they begin with pc-. For installation of additional packages, virtual environments are encouraged.

For more info see: http://www.archer.ac.uk/documentation/user-guide/python.php

When compiling Python software (like PySCF) Archer by default builds all libraries as static libraries. This leads to errors like:

 ImportError Cannot import name ...

or

 File "crystal.py", line 3, in <module>
   import pyscf.pbc
 File "/work/e507/e507/ap837/code/pyscf/__init__.py", line 118, in <module>
   from pyscf import gto
 File "/work/e507/e507/ap837/code/pyscf/pyscf/__init__.py", line 118, in <module>
   from pyscf import gto
 File "/work/e507/e507/ap837/code/pyscf/gto/__init__.py", line 7, in <module>
   from pyscf.gto import mole
 File "/work/e507/e507/ap837/code/pyscf/gto/mole.py", line 16, in <module>
   from pyscf import lib
 File "/work/e507/e507/ap837/code/pyscf/lib/__init__.py", line 11, in <module>
   from pyscf.lib import numpy_helper
 File "/work/e507/e507/ap837/code/pyscf/lib/numpy_helper.py", line 17, in <module>
   _np_helper = misc.load_library('libnp_helper')
 File "/work/e507/e507/ap837/code/pyscf/lib/misc.py", line 41, in load_library
   return numpy.ctypeslib.load_library(libname, _loaderpath)
 File "/work/y07/y07/cse/numpy/1.9.2-libsci/lib/python2.7/site-packages/numpy/ctypeslib.py", line 128, in load_library
   raise OSError("no file with expected extension")
 OSError: no file with expected extension

and other errors.

To prevent dynamic libraries from becoming static you must:

 export CRAYPE_LINK_TYPE=dynamic

For more information see: http://www.archer.ac.uk/documentation/user-guide/development.php#sec-4.6

When using numpy, the following error may occur:

 Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so

Tha solution is to do:

 export LD_PRELOAD=/opt/intel/mkl/lib/intel64/libmkl_core.so:/opt/intel/mkl/lib/intel64/libmkl_sequential.so

as found here http://debugjournal.tumblr.com/post/98401758462/intel-mkl-dynamic-link-library-error.