View source for Compiling Wales Group code using CMake

[http://www.cmake.org/ CMake] (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory. 

Although the majority of the below refers to compiling [[GMIN]], the same procedure works for [[OPTIM]] and [[PATHSAMPLE]] 

Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with ''nagfor'' or ''gfortran''. This is nothing to do with our code - it's a CHARMM issue.

==Acquiring the Wales Group source code==
You can find the source code for all of our programs in a tarball [http://www-wales.ch.cam.ac.uk/svn.tar.bz2 here]. This tarball does not contain any of the CHARMM35/36 or AMBER9/12 source code or interfaces for licensing reasons. If you want to use our codes with AMBER or CHARMM '''and have a valid license''', please get in touch with Professor Wales and we can provide the interfaces to you. Attempting to build ''A9GMIN'' without these interfaces being present will fail.

This tutorial assumes that you have untarred the source code in your home directory, creating a directory ~/wales. If you have it elsewhere, modify the paths below appropriately.

==Preparing to compile==
Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:
<pre>
cmake --version
</pre>

You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with gfortran, you would make a directory here:
<pre>
mkdir -p ~/wales/GMIN/builds/gfortran
cd ~/wales/GMIN/builds/gfortran
</pre>
You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded.

==Compiling using the ccmake GUI interface to set options==

One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. svn/GMIN/builds/gfortran). We specify the '''F'''ortran '''C'''ompiler by setting the '''$FC''' environment variable (in this case the GNU Fortran compiler, ifort), and then running ''cmake'', passing it the location of the [[GMIN]] source directory:
<pre>
FC=gfortran cmake ../../source
</pre>

If you run ''ls'', you will see some cmake files have been generated:
<pre>
csw34@sinister:~/wales/GMIN/builds/gfortran> ls
CMakeCache.txt  CMakeFiles  cmake_install.cmake  Makefile  modules
</pre>

You can now run ''ccmake'' in your build directory to open the GUI:
<pre>
ccmake .
</pre>

To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile [[GMIN]] with AMBER9 (A9GMIN) for example, you would need to toggle the ''WITH_AMBER'' option ''ON''. We are not going that here so leave it set to ''OFF''. Once you are done changing these options, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.

'''Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options'''

You can now compile GMIN in parallel as follows:
<pre>
make -j8
</pre>

The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have a ''GMIN'' binary in your build directory - congratulations! You can move this into anywhere in your ''$PATH'' to make running it simple.

==Compiling by setting options on the command line==
If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to ''cmake'' on the command line, bypassing the need for ''ccmake''. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:
<pre>
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>
where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.

==Compiling with MPI==
To compile with MPI support add the following flags when running cmake on the command line:
<pre>
FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes
</pre>
Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland ''pgi'' compiler. Make sure you have matching fortran and mpi compilers loaded (in this case ''pgi and ''mpi-pgi''). 

You can of course set these flags in ''ccmake'' if you prefer.

==Advanced mode - changing compiler flags with ccmake==

Although initially the ''ccmake'' GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to ''make'' when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do  profiling. 

As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.

Note that these changes only apply in the build directory in which you make them.

==Debugging runtime problems using gdb or valgrind==
If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like [http://www.gnu.org/software/gdb/ gdb] or [http://valgrind.org/ valgrind]. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the ''CMAKE_BUILD_TYPE'' in ''ccmake'' to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:
<pre>
FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source
make -j8
</pre>

You can then run the binary ''through'' gdb or valgrind as follows:
<pre>
gdb A9GMIN
</pre>
or
<pre>
valgrind A9GMIN
</pre>

I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)

==Debugging compilation problems==
There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the ''VERBOSE'' option enabled. This will dump a lot of potentially useful output:
<pre>
VERBOSE=1 make
</pre>

If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.

==Extra command line build examples==
The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ''ccmake'' as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your svn repository is set up in ''/home/CRSID/svn'' - make the appropriate modifications if you have it elsewhere.

===GMIN===

'''GMIN''' (plain GMIN) using gfortran:
<pre>
mkdir -p ~/wales/GMIN/builds/gfortran
cd !$
FC=gfortran cmake ../../source
make -j8
</pre>

'''A12GMIN''' (GMIN with AMBER12) using ifort:
<pre>
mkdir -p ~/wales/GMIN/builds/ifort_amber12
cd !$
FC=ifort cmake -DWITH_AMBER12=1 ../../source
make -j8
</pre>

'''C35GMIN''' (GMIN with CHARMM 35) using pgi:
<pre>
mkdir -p ~/wales/GMIN/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAGMIN''' (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/wales/GMIN/builds/ifort_cuda
cd !$
FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs when using CUDA 5.5.

===OPTIM===

'''OPTIM''' (plain OPTIM) using gfortran:
<pre>
mkdir -p ~/wales/OPTIM/builds/gfortran
cd !$
FC=gfortran cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>

'''A9OPTIM''' (OPTIM with AMBER9) using ifort:
<pre>
mkdir -p ~/wales/OPTIM/builds/ifort_amber
cd !$
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>

'''C35OPTIM''' (OPTIM with CHARMM 35) using pgi:
<pre>
mkdir -p ~/wales/OPTIM/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

===PATHSAMPLE===
There are very few options for [[PATHSAMPLE]] as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called ''PATHSAMPLE''.

Using gfortran:
<pre>
mkdir -p ~/wales/PATHSAMPLE/builds/gfortran
cd !$
FC=gfortran cmake ../../source
make -j8
</pre>

Using nagfor (the NAG fortran compiler - very strict!):
<pre>
mkdir -p ~/wales/PATHSAMPLE/builds/nagfor
cd !$
FC=nagfor cmake ../../source
make -j8
</pre>