Compiling Wales Group code using CMake

CMake (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory.

Although the majority of the below refers to compiling GMIN, the same procedure works for OPTIM and PATHSAMPLE

Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with nagfor or gfortran. This is nothing to do with our code - it's a CHARMM issue.

Acquiring the Wales Group source code

You can find the source code for all of our programs in a tarball here. This tarball does not contain any of the CHARMM35/36 or AMBER9/12 source code or interfaces for licensing reasons. If you want to use our codes with AMBER or CHARMM and have a valid license, please get in touch with Professor Wales and we can provide the interfaces to you. Attempting to build A9GMIN without these interfaces being present will fail.

This tutorial assumes that you have untarred the source code in your home directory, creating a directory ~/wales. If you have it elsewhere, modify the paths below appropriately.

Preparing to compile

Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:

cmake --version

You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with ifort, you would make a directory here:

mkdir -p GMIN/builds/ifort
cd GMIN/builds/ifort

You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded.

Compiling using the ccmake GUI interface to set options

One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. svn/GMIN/builds/gfortran). We specify the Fortran Compiler by setting the $FC environment variable (in this case the GNU Fortran compiler, ifort), and then running cmake, passing it the location of the GMIN source directory:

FC=ifort cmake ../../source

If you run ls, you will see some cmake files have been generated:

csw34@sinister:~/wales/GMIN/builds/ifort> ls
CMakeCache.txt  CMakeFiles  cmake_install.cmake  Makefile  modules

You can now run ccmake in your build directory to open the GUI:

ccmake .

To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile GMIN with AMBER9 (A9GMIN) for example, you would need to toggle the WITH_AMBER option ON. We are not going that here so leave it set to OFF. Once you are done changing these options, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.

Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options

You can now compile GMIN in parallel as follows:

make -j8

The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have a GMIN binary in your build directory - congratulations! You can move this into anywhere in your $PATH to make running it simple.

Compiling by setting options on the command line

If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to cmake on the command line, bypassing the need for ccmake. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:

FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8

where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.

Compiling with MPI

To compile with MPI support add the following flags when running cmake on the command line:

FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes

Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland pgi compiler. Make sure you have matching fortran and mpi compilers loaded (in this case pgi and mpi-pgi).

You can of course set these flags in ccmake if you prefer.

Advanced mode - changing compiler flags with ccmake

Although initially the ccmake GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to make when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do profiling.

As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.

Note that these changes only apply in the build directory in which you make them.

Debugging runtime problems using gdb or valgrind

If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like gdb or valgrind. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the CMAKE_BUILD_TYPE in ccmake to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:

FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source
make -j8

You can then run the binary through gdb or valgrind as follows:

gdb A9GMIN

or

valgrind A9GMIN

I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)

Debugging compilation problems

There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the VERBOSE option enabled. This will dump a lot of potentially useful output:

VERBOSE=1 make

If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.

Extra command line build examples

The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ccmake as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your svn repository is set up in /home/CRSID/svn - make the appropriate modifications if you have it elsewhere.

GMIN

GMIN (plain GMIN) using gfortran:

mkdir -p ~/wales/GMIN/builds/gfortran
cd !$
FC=gfortran cmake ../../source
make -j8

A12GMIN (GMIN with AMBER12) using ifort:

mkdir -p ~/wales/GMIN/builds/ifort_amber12
cd !$
FC=ifort cmake -DWITH_AMBER12=1 ../../source
make -j8

C35GMIN (GMIN with CHARMM 35) using pgi:

mkdir -p ~/wales/GMIN/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8

CUDAGMIN (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:

module load cuda/5.5
mkdir -p ~/wales/GMIN/builds/ifort_cuda
cd !$
FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source
make -j8

This will only work on machines with specific NVIDIA GPUs, for example the workstations hammerstein, quartz and rojaws when using CUDA 5.5. More info can be found on the page concerning Using GMIN with GPUs.

OPTIM

OPTIM (plain OPTIM) using gfortran:

mkdir -p ~/wales/OPTIM/builds/gfortran
cd !$
FC=gfortran cmake -DWITH_AMBER=1 ../../source
make -j8

A9OPTIM (OPTIM with AMBER9) using ifort:

mkdir -p ~/wales/OPTIM/builds/ifort_amber
cd !$
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8

C35OPTIM (OPTIM with CHARMM 35) using pgi:

mkdir -p ~/wales/OPTIM/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8

PATHSAMPLE

There are very few options for PATHSAMPLE as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called PATHSAMPLE.

Using gfortran:

mkdir -p ~/wales/PATHSAMPLE/builds/gfortran
cd !$
FC=gfortran cmake ../../source
make -j8

Using nagfor (the NAG fortran compiler - very strict!):

mkdir -p ~/wales/PATHSAMPLE/builds/nagfor
cd !$
FC=nagfor cmake ../../source
make -j8