Compiling Wales Group code using CMake
CMake (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory.
Although everything below refers to compiling GMIN with the Intel ifort compiler and AMBER9 - the exact same procedure works for OPTIM and PATHSAMPLE and other compilers such as gfortran.
Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with nagfor or gfortran. This is nothing to do with our code - it's a CHARMM issue.
Preparing to compile
Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:
cmake --version
You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with ifort, you would make a directory here:
mkdir -p GMIN/builds/ifort cd GMIN/builds/ifort
You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded.
Compiling using the ccmake GUI interface to set options
One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. svn/GMIN/builds/ifort). We specify the Fortran Compiler by setting the $FC environment variable (in this case the Intel Fortran compiler, ifort), and then running cmake, passing it the location of the GMIN source directory:
FC=ifort cmake ../../source
If you run ls, you will see some cmake files have been generated:
csw34@sinister:~/wales/GMIN/builds/ifort> ls CMakeCache.txt CMakeFiles cmake_install.cmake Makefile modules
You can now run ccmake in your build directory to open the GUI:
ccmake .
To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile GMIN with AMBER9 (A9GMIN), we need to toggle the WITH_AMBER option ON. Once you have done this, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.
Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options
You can now compile A9GMIN in parallel as follows:
make -j8
The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have an A9GMIN binary in your build directory - congratulations!
Linking Fortran executable A9GMIN [100%] Built target A9GMIN --------------------------------------------------------------------------------------------- 15:23:45 csw34@sinister:~/wales/GMIN/builds/ifort> ls A9GMIN cmake_install.cmake libcudadummylib.a libmylapack.a NAB AMBER display_version.f90 libdummylib.a Makefile nab_binaries_built CMakeCache.txt GMIN libgminlib.a modules porfuncs.f90 CMakeFiles libamber12dummylib.a libmyblas.a n
Plain GMIN is also built at the same time should you need it. You can move this into anywhere in your $PATH to make running it simple.
Compiling by setting options on the command line
If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to cmake on the command line, bypassing the need for ccmake. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:
FC=ifort cmake -DWITH_AMBER=1 ../../source make -j8
where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.
Compiling with MPI
To compile with MPI support add the following flags when running cmake on the command line:
FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes
Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland pgi compiler. Make sure you have matching fortran and mpi compilers loaded (in this case pgi and mpi-pgi).
You can of course set these flags in ccmake if you prefer.
Advanced mode - changing compiler flags with ccmake
Although initially the ccmake GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to make when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do profiling.
As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.
Note that these changes only apply in the build directory in which you make them.
Debugging runtime problems using gdb or valgrind
If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like gdb or valgrind. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the CMAKE_BUILD_TYPE in ccmake to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:
FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source make -j8
You can then run the binary through gdb or valgrind as follows:
gdb A9GMIN
or
valgrind A9GMIN
I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)
Debugging compilation problems
There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the VERBOSE option enabled. This will dump a lot of potentially useful output:
VERBOSE=1 make
If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.
Extra command line build examples
The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ccmake as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your svn repository is set up in /home/CRSID/svn - make the appropriate modifications if you have it elsewhere.
GMIN
GMIN (plain GMIN) using gfortran:
mkdir -p ~/wales/GMIN/builds/gfortran cd !$ FC=gfortran cmake ../../source make -j8
A12GMIN (GMIN with AMBER12) using ifort:
mkdir -p ~/wales/GMIN/builds/ifort_amber12 cd !$ FC=ifort cmake -DWITH_AMBER12=1 ../../source make -j8
C35GMIN (GMIN with CHARMM 35) using pgi:
mkdir -p ~/wales/GMIN/builds/pgi_charmm35 cd !$ FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source make -j8
CUDAGMIN (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:
module load cuda/5.5 mkdir -p ~/wales/GMIN/builds/ifort_cuda cd !$ FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source make -j8
This will only work on machines with specific NVIDIA GPUs, for example the workstations hammerstein, quartz and rojaws when using CUDA 5.5. More info can be found on the page concerning Using GMIN with GPUs.
OPTIM
A9OPTIM (OPTIM with AMBER9) using ifort:
mkdir -p ~/wales/OPTIM/builds/ifort_amber cd !$ FC=ifort cmake -DWITH_AMBER=1 ../../source make -j8
C35OPTIM (OPTIM with CHARMM 35) using pgi:
mkdir -p ~/wales/OPTIM/builds/pgi_charmm35 cd !$ FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source make -j8
PATHSAMPLE
There are very few options for PATHSAMPLE as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called PATHSAMPLE.
Using nagfor (the NAG fortran compiler - very strict!):
mkdir -p ~/wales/PATHSAMPLE/builds/nagfor cd !$ FC=nagfor cmake ../../source make -j8
Using gfortran (available on almost every Linux machine - no license required):
mkdir -p ~/wales/PATHSAMPLE/builds/gfortran cd !$ FC=gfortran cmake ../../source make -j8