Docswiki - User contributions [en]

VMD

2026-04-12T07:53:40Z

Dw34:

VMD is a molecular visualization program installed on all workstations and clusters. The official documentation can be found [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html here] with some tutorials [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html here]. Like gnuplot however, the wealth of options means that often it takes a long time to find the one command you need to use so below you will find some useful basic settings/info for using VMD. For producing graphics for publication, [[Pymol]] is probably a better option as it has a built in ray-tracing routine but for general visualization, VMD is much quicker.

It is possible to load most files using command line flags, making loading many frames into different topology files easy. The -f flag indicates that all subsequent files (until the next -f flag or the end) should be loaded into a single molecule. There are also flags for selecting different file types (default is .pdb), most commonly parm7 for topology files generated by Amber and rst7 for restart files generated by Amber. mdcrd files are denoted -crd and periodic mdcrd files -crdbox.

e.g.
vmd -f first_mol.pdb \
-f -parm7 second_mol.prmtop -rst7 second_mol.rst \
-f -parm7 third_mol.prmtop -crdbox third_mol_1st_frames.crd -crdbox third_mol_2nd_frames

== Rendering Molecules with a Transparent Background ==

Dumping selected frames from a movie or a single image can be achieved using the render option in the vmd gui.
Choosing the povray option should produce a vmdscene.pov file. This pov file can be converted to a png with a
transparent background using povray if you must have a white background in vmd:

povray +W829 +H771 -Ivmdscene.pov -Ovmdscene.pov.tga -Otest +Q11 +J +A +FN +UA

The +W and +H values must be taken from the vmdscene.pov file; they should be given in a comment statement at the top.

== Movie Making Tips ==
To load all frames in one go, select the file type in the "Determine file type" box, and then the button "load all at once"
will not be greyed out, so you can select it.

vmd movie making seems not work properly with step sizes different from one. The last frame is repeated many times. Instead, the
frames can be selected using sed:

Try extracting frames first with sed:

sed -e '1~66087,+62939d' path.xyz > temp

1,+62939d deletes lines 1 to 62940, deleting 20 frames

The ~66087 repeats the action every 21 frames. The counter operates on the original line numbers.

This example is for a
system with 3145 atoms, so each frame is 3147 lines with the xyz header.

sed -e '1~Y,+Xd' path.xyz > temp

1,+Xd deletes lines 1 to X+1, so to select every nth frame for frames of length m you need

X=n*m-1

and Y=(n+1)*m

m is the number of atoms plus two. If the total number of frames is not a multiple of n
then some frames will be lost at the end. Padding the start and finish of the xyz file with copies of the
initial and final minimum will make the movie pause at these end points.

To make the movie pause at the start and finish, just duplicate these end points sufficiently. If there are
slow portions around local minima try adjusting the energy difference parameter on the PATH line in the OPTIM odata file for intial
generation of path.xyz.

== Tutorials ==
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* making movies from a '.pdb' file containing multiple structures. ''This is dealt with in the OPTIM section as part of the tutorial on making a movie of a path''
* [[visualising normal modes using VMD and OPTIM]]

VMD

2026-04-12T07:45:39Z

Dw34:

VMD is a molecular visualization program installed on all workstations and clusters. The official documentation can be found [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html here] with some tutorials [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html here]. Like gnuplot however, the wealth of options means that often it takes a long time to find the one command you need to use so below you will find some useful basic settings/info for using VMD. For producing graphics for publication, [[Pymol]] is probably a better option as it has a built in ray-tracing routine but for general visualization, VMD is much quicker.

It is possible to load most files using command line flags, making loading many frames into different topology files easy. The -f flag indicates that all subsequent files (until the next -f flag or the end) should be loaded into a single molecule. There are also flags for selecting different file types (default is .pdb), most commonly parm7 for topology files generated by Amber and rst7 for restart files generated by Amber. mdcrd files are denoted -crd and periodic mdcrd files -crdbox.

e.g.
vmd -f first_mol.pdb \
-f -parm7 second_mol.prmtop -rst7 second_mol.rst \
-f -parm7 third_mol.prmtop -crdbox third_mol_1st_frames.crd -crdbox third_mol_2nd_frames

== Rendering Molecules with a Transparent Background ==

Dumping selected frames from a movie or a single image can be achieved using the render option in the vmd gui.
Choosing the povray option should produce a vmdscene.pov file. This pov file can be converted to a png with a
transparent background using povray if you must have a white background in vmd:

povray +W829 +H771 -Ivmdscene.pov -Ovmdscene.pov.tga -Otest +Q11 +J +A +FN +UA

The +W and +H values must be taken from the vmdscene.pov file; they should be given in a comment statement at the top.

== Movie Making Tips ==
To load all frames in one go, select the file type in the "Determine file type" box, and then the button "load all at once"
will not be greyed out, so you can select it.

vmd movie making seems not work properly with step sizes different from one. The last frame is repeated many times. Instead, the
frames can be selected using sed:

Try extracting frames first with sed:

sed -e '1~66087,+62939d' path.xyz > temp

1,+62939d deletes lines 1 to 62940, deleting 20 frames

The ~66087 repeats the action every 21 frames. The counter operates on the original line numbers.

This example is for a
system with 3145 atoms, so each frame is 3147 lines with the xyz header.

sed -e '1~Y,+Xd' path.xyz > temp

1,+Xd deletes lines 1 to X+1, so to select every nth frame for frames of length m you need

X=n*m-1

and Y=(n+1)*m

m is the number of atoms plus two. The total number of frames needs to be a multiple of n.
If necessary, pad the original file with some extra copies of the last frame.

To make the movie pause at the start and finish, just duplicate these end points sufficiently. If there are
slow portions around local minima try adjusting the energy difference parameter on the PATH line in the OPTIM odata file for intial
generation of path.xyz.

== Tutorials ==
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* making movies from a '.pdb' file containing multiple structures. ''This is dealt with in the OPTIM section as part of the tutorial on making a movie of a path''
* [[visualising normal modes using VMD and OPTIM]]

Building tleap

2026-04-07T14:30:29Z

Dw34:

To start from the tarball :

bunzip2 AmberTools20.tar.bz2
tar -xvf AmberTools20.tar
cd amber20_src

export AMBERHOME=`pwd`

If there are missing packages you may need an install:

apt-get install csh flex gfortran g++ xorg-dev zlib1g-dev libbz2-dev patch python-tk python-matplotlib

To set up the environment variables use
source /home/wales/ambertools2/amber20_src/amber.sh
whichcould go in ~/.bashrc

in ~/ambertools2/amber20_src/AmberTools/src/leap

make
make install

now tleap and teleap are in

~/ambertools2/amber20_src/bin

which tleap

~/ambertools2/amber20_src/bin/tleap

now it should be possible to do

tleap -f leap.in

Building tleap

2026-04-07T14:28:11Z

Dw34: Created page with "source /home/wales/ambertools2/amber20_src/amber.sh in ~/ambertools2/amber20_src/AmberTools/src/leap make make install now tleap and teleap are in ~/ambertools2/amber20_src/bin which tleap ~/ambertools2/amber20_src/bin/tleap now it should be possible to do tleap -f leap.in"

source /home/wales/ambertools2/amber20_src/amber.sh

in ~/ambertools2/amber20_src/AmberTools/src/leap

make
make install

now tleap and teleap are in

~/ambertools2/amber20_src/bin

which tleap

~/ambertools2/amber20_src/bin/tleap

now it should be possible to do

tleap -f leap.in

AMBER

2026-04-07T14:22:58Z

Dw34:

'''[[Notes on AMBER 12 interface]]'''

[[Image:Amberpic.jpg|thumb|"The bugs have magically disappeared!"|200px|right]]
[http://amber.scripps.edu/ "AMBER"] (Assisted Model Building with Energy Refinement) refers to two things: a set of molecular mechanical force fields for the simulation of biomolecules (which are in the public domain, and are used in a variety of simulation programs); and a package of molecular simulation programs which includes source code and demos. We mainly use the MM forcefields interfaced with other group software i.e. [[GMIN]] or [[OPTIM]]. The included programmes such as ''sander'' and ''antechamber'' are however, extremely useful in some circumstances! The full user manual for AMBER9 can be found in pdf format [http://amber.scripps.edu/doc9/amber9.pdf here].

As of July 2009, the SVN repository also contains AMBER Tools, the stand alone suite of programs that generate AMBER input files and allow you to analyse output. You can find a manual within the repository. Look in AMBERTOOLS/doc.

== Tutorials ==

* [[Using AMBER 14 on the GPU and compute clusters]]
* [http://amber.scripps.edu/tutorials/ Ross Walker's AMBER9 tutorials] - recommended reading for '''ANYONE''' using AMBER!
* [[Generating parameters using AMBER's built in General Forcefield (gaff)]]
* [[Generating parameters using RESP charges from GAMESS-US]]
* [[Building tleap]]
* [[Simple scripts for LEaP to create topology and coordinate files]]
* [[Preparing an AMBER topology file for a protein system]] - step by step guide
* [[Setting up]] - step by step guide to prepare and then symmetrise a simple (protein-only) system
* [[Preparing an AMBER topology file for a protein plus ligand system]] - step by step guide
* [[Symmetrising AMBER topology files]] - step by step guide for symmetrising a complex protein+ligand system
* [[Producing a PDB from a coordinates and topology file]] - using ''amdpdb''
* [[Running GMIN with MD move steps AMBER]]
* [[Evaluating different components of AMBER energy function with SANDER]]
* [[Running MD with AMBER]]
* [[Running MD on GPUS with pmemd_cuda]]
* [[REMD with AMBER]]
* [[Performing a hydrogen-bond analysis]]

Compiling Wales Group codes using cmake

2025-01-17T21:27:59Z

Dw34: /* Compiling with MPI */

[http://www.cmake.org/ CMake] (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory. It also enables us to use the [[Jenkins CI]] 'build bot' system to automatically compile and test the code on a nightly basis - helping us catch troublesome commits before they affect other users.

Although everything below refers to compiling [[GMIN]] with the Intel ''ifort'' compiler and AMBER9 - the exact same procedure works for [[OPTIM]] and [[PATHSAMPLE]].

Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with ''nagfor'' or ''gfortran''. This is nothing to do with our code - it's a CHARMM issue. You can get an idea for what should work by looking at the automated [[Jenkins CI]] builds.

==Preparing to compile==
Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:
<pre>
cmake --version
</pre>

The clusters have a module for cmake 3.0 (cmake 3.6.2 on Nest), which you can load using the following command:
<pre>
module load cmake/3.0.0
</pre>

You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with ifort, you would make a directory here:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort
cd ~/softwarewales/GMIN/builds/ifort
</pre>
You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded. This is important as the different clusters and workstations may have different default versions loaded, some of which might not work properly. You can check the compiler version currently loaded using the same '--version' flag we used for ''cmake'' above:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ifort --version
ifort (IFORT) 12.1.3 20120212
Copyright (C) 1985-2012 Intel Corporation. All rights reserved.
</pre>

To load a different compiler, you can use the ''module load'' or ''module swap'' commands. A list of all available modules can be accessed using:
<pre>
module av
</pre>
If you are having problems compiling, one of the first things to check is whether it works with a different version of the compiler!

'''Note: When compiling GMIN, if you are getting the error that there is no implicit type for ERFC in ewald.f90, try using a newer version of your compiler. This should be the built-in complementary error function.'''

==Compiling using the ccmake GUI interface to set options==
[[Image:Ccmake.png|thumb|ccmake set up to compile A9GMIN|200px|right]]

One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. softwarewales/GMIN/builds/ifort). We specify the '''F'''ortran '''C'''ompiler by setting the '''$FC''' environment variable (in this case the Intel Fortran compiler, ifort), and then run ''cmake'' (on the command line), passing it the relative location of the [[GMIN]] source directory:
<pre>
FC=ifort cmake ../../source
</pre>

If you run ''ls'', you will see some cmake files have been generated:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
CMakeCache.txt CMakeFiles cmake_install.cmake Makefile modules
</pre>

You can now run ''ccmake'' to open the GUI:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ccmake .
</pre>

To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile [[GMIN]] with AMBER9 (A9GMIN), we need to toggle the ''WITH_AMBER'' option ''ON''. Once you have done this, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.

'''Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options'''

You can now compile A9GMIN in parallel as follows:
<pre>
make -j8
</pre>

The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have an A9GMIN binary in your build directory - congratulations!
<pre>
Linking Fortran executable A9GMIN
[100%] Built target A9GMIN

--------------------------------------------------------------------------------------------- 15:23:45

csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
A9GMIN cmake_install.cmake libcudadummylib.a libmylapack.a NAB
AMBER display_version.f90 libdummylib.a Makefile nab_binaries_built
CMakeCache.txt GMIN libgminlib.a modules porfuncs.f90
CMakeFiles libamber12dummylib.a libmyblas.a n
</pre>
Plain [[GMIN]] is also built at the same time should you need it. You can move this into your ~/bin directory if you like, or anywhere else in your ''$PATH'' to make running it simple.

'''Note: If you want to use OPTIM with the new C++ implementation of the NEB routine, you will need to obtain the source code for that separately. See [https://wikis.ch.cam.ac.uk/wales/wiki/index.php/OPTIM here] for instructions.'''

==Compiling by setting options on the command line==
If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to ''cmake'' on the command line, bypassing the need for ''ccmake''. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:
<pre>
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>
where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.

'''Note: Sometimes you may get error''' (for example, Fatal Error: Can't open module file 'someModule.mod' for reading at (1): No such file or directory) when following this procedure. In that case there are three things you could try: make sure you are building in a new directory, if that does not help run `make VERBOSE=1` instead of `make -j8` or simply switch to using ccmake.

==Compiling with MPI==
To compile with MPI support add the following flags when running cmake on the command line for ifort:
<pre>
FC=mpiifort cmake ~/softwarewales/GMIN/source/ -DWITH_MPI=yes
</pre>
On nest this command line can be used to build A20GMIN for BHPT runs with AMBER20 when WITH_AMBER20
is employed via ccmake or -DWITH_AMBER20=yes. Modules that work are:
<pre>
module load gcc/7.5.0 cmake/3.23.2 ifort/64/2020/4/304 mpi/intel/2023.1.0
</pre>

The corresponding gfortran build on nest requires:
<pre>
FC=mpifort CC=mpicc CXX=mpicxx cmake ~/softwarewales/GMIN/source/ -DBLAS_LIBRARIES=/lib64/libopenblas.so.0 DCOMPILER_SWITCH=gfortran
</pre>
and modules:
<pre>
module load gcc/7.5.0 cmake/3.23.2 mpi/openmpi/gnu7/4.1.0
</pre>
The above examples with ifort and gfortran should work with MYBLAS and MYLAPACK turned off, and the system blas library
is significantly faster.

An older pgi build used:
<pre>
FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes
</pre>
Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland ''pgi'' compiler. Make sure you have the correct modules loaded (in this case ''pgi'' and ''mpi-pgi''), and that the particular mpi you want (in this case ''mpi-pgi'') is listed before any other mpi's loaded (so that it has the highest priority). The modules can be loaded by typing:
<pre>
module load pgi/64/
module load mpi/openmpi/pgi/64/1.6.3
</pre>

and you can check which modules are loaded and in which order/priority by the ''module list'' command. You may need to ''module unload <name>'' any other mpi's that are higher up in the list than the one you want. You can of course set the COMPILER_SWITCH and WITH_MPI flags in ''ccmake'' if you prefer.

Note: It has been observed that pgi/64/15.1 leads to compilation errors, and for now, it is best to use pgi/64/14.9

==Advanced mode - changing compiler flags with ccmake==
[[Image:Ccmakeadvanced.png|thumb|ccmake advanced mode|200px|right]]

Although initially the ''ccmake'' GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to ''make'' when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do profiling.

As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.

Note that these changes only apply in the build directory in which you make them.

==Debugging runtime problems using gdb or valgrind==
If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like [http://www.gnu.org/software/gdb/ gdb] or [http://valgrind.org/ valgrind]. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the ''CMAKE_BUILD_TYPE'' in ''ccmake'' to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:
<pre>
FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source
make -j8
</pre>

You can then run the binary ''through'' gdb or valgrind as follows:
<pre>
gdb A9GMIN
</pre>
or
<pre>
valgrind A9GMIN
</pre>

I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)

==Debugging compilation problems==
There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the ''VERBOSE'' option enabled. This will dump a lot of potentially useful output:
<pre>
VERBOSE=1 make
</pre>

One possible gotcha: all .f and .f90 files in the relevant source directories will be compiled and added to a library. This is quite different from the old Makefile way of doing things, where source files were explicitly specified for compilation (via their corresponding .o file). So, if you are testing something by for instance copying code.f90 to code.myhack.f90 and code.orig.f90, then slightly editing a line or two of code.myhack.f90 and copying it back to code.f90 for use, this will probably cause linking problems due to multiply-defined subroutines (from all three files). The solution, if you must have alternative versions of the same file hanging round, is to differentiate the filenames by a suffix AFTER the .f[90] .

Another occasional issue is the unexplained compiler bug - a problem with the version of the compiler you happen to be using. You can can an idea for which compiler versions we expect to work by checking the Jenkins build-bot output, as described in the 'Seeing console output' section of the [[Jenkins CI]] page. If you are using a different version of the compiler in question, consider swapping to the version Jenkins is using with 'module swap'.

If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.

To build the executables with the QUIP interface, it may be necessary to do make clean in the QUIP directory.

==Extra command line build examples==
The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ''ccmake'' as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your git repository is set up in ''/home/CRSID/softwarewales'' - make the appropriate modifications if you have it elsewhere.

===GMIN===
'''A12GMIN''' (GMIN with AMBER12) using ifort:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort_amber12
cd !$
FC=ifort cmake -DWITH_AMBER12=1 ../../source
make -j8
</pre>

'''C35GMIN''' (GMIN with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAGMIN''' (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/GMIN/builds/ifort_cuda
cd !$
FC=ifort cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN with GPUs]] page.

'''DFTBGMIN''' (GMIN with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/GMIN/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/GMIN/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===OPTIM===
'''A9OPTIM''' (OPTIM with AMBER9) using ifort:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/ifort_amber
cd !$
FC=ifort cmake -DWITH_AMBER9=1 ../../source
make -j8
</pre>

'''C35OPTIM''' (OPTIM with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAOPTIM''' (OPTIM leveraging GPU via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/OPTIM/builds/ifort_cuda5.5
cd !$
FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN and OPTIM with GPUs]] page.

'''DFTBOPTIM''' (OPTIM with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/OPTIM/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/OPTIM/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===PATHSAMPLE===
There are very few options for [[PATHSAMPLE]] as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called ''PATHSAMPLE''.

Using nagfor (the NAG fortran compiler - check you have the module loaded - very strict!):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/nagfor
cd !$
FC=nagfor cmake ../../source
make -j8
</pre>

Using pgi (much more generous with coding slips/non-standard uses):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/pgi
cd !$
FC=pgf90 cmake ../../source
make -j8
</pre>

==Configuring defaults - for developers==

Fortran compilers and their corresponding default settings are all controlled by the file $SVN/CMakeModules/FindFORTRANCOMPILER.cmake ($SVN is your svn root directory). In particular, we may wish to edit the flags used for each set of compilers and build type. These are contained in the following block:

<pre>
if(NOT COMPILER_FLAGS_WERE_SET)
message("Setting initial values for compiler flags")
if(COMPILER_SWITCH MATCHES "pgi")
set (CMAKE_Fortran_FLAGS "-Mextend" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3 -Munroll -Mnoframe" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Mdwarf3 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-Mfree" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "gfortran")
set (CMAKE_Fortran_FLAGS "-ffixed-line-length-200 -ffree-line-length-0" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fimplicit-none -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "${CMAKE_Fortran_FLAGS_DEBUG} -fimplicit-none" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-ffree-form" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "nag")
set (CMAKE_Fortran_FLAGS "-132 -kind=byte -maxcontin=3000" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-mismatch_all -O4" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-C=all -mtrace=all -gline -g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING) # js850> is this ever used?
elseif(COMPILER_SWITCH MATCHES "ifort")
set (CMAKE_Fortran_FLAGS "-132 -heap-arrays -assume byterecl" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# Warnings about temporary argument creation and edit descriptor widths are disabled with the final flags.
set (CMAKE_Fortran_FLAGS_DEBUG "-C -g -traceback -debug full -check all,noarg_temp_created -diag-disable 8290,8291" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-debug all -check all -implicitnone -warn unused -fp-stack-check -ftrapuv -check pointers -check bounds" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING)
else()
message(FATAL_ERROR "unknown comiler switch: ${COMPILER_SWITCH}")
endif()
SET(COMPILER_FLAGS_WERE_SET yes CACHE TYPE INTERNAL)
endif(NOT COMPILER_FLAGS_WERE_SET)
</pre>

The main if/elseif blocks correspond to compiler switches. Inside these, there are the default flags for each of our build types (release, debug and debug_slow), which are configured using ccmake. These can be edited, if we wish to change the default behaviour (e.g. a recent addition of -check all,noarg_temp_created -diag-disable 8290,8291 to disable annoying warning messages for ifort).

Compiling Wales Group codes using cmake

2025-01-17T21:26:08Z

Dw34:

[http://www.cmake.org/ CMake] (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory. It also enables us to use the [[Jenkins CI]] 'build bot' system to automatically compile and test the code on a nightly basis - helping us catch troublesome commits before they affect other users.

Although everything below refers to compiling [[GMIN]] with the Intel ''ifort'' compiler and AMBER9 - the exact same procedure works for [[OPTIM]] and [[PATHSAMPLE]].

Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with ''nagfor'' or ''gfortran''. This is nothing to do with our code - it's a CHARMM issue. You can get an idea for what should work by looking at the automated [[Jenkins CI]] builds.

==Preparing to compile==
Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:
<pre>
cmake --version
</pre>

The clusters have a module for cmake 3.0 (cmake 3.6.2 on Nest), which you can load using the following command:
<pre>
module load cmake/3.0.0
</pre>

You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with ifort, you would make a directory here:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort
cd ~/softwarewales/GMIN/builds/ifort
</pre>
You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded. This is important as the different clusters and workstations may have different default versions loaded, some of which might not work properly. You can check the compiler version currently loaded using the same '--version' flag we used for ''cmake'' above:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ifort --version
ifort (IFORT) 12.1.3 20120212
Copyright (C) 1985-2012 Intel Corporation. All rights reserved.
</pre>

To load a different compiler, you can use the ''module load'' or ''module swap'' commands. A list of all available modules can be accessed using:
<pre>
module av
</pre>
If you are having problems compiling, one of the first things to check is whether it works with a different version of the compiler!

'''Note: When compiling GMIN, if you are getting the error that there is no implicit type for ERFC in ewald.f90, try using a newer version of your compiler. This should be the built-in complementary error function.'''

==Compiling using the ccmake GUI interface to set options==
[[Image:Ccmake.png|thumb|ccmake set up to compile A9GMIN|200px|right]]

One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. softwarewales/GMIN/builds/ifort). We specify the '''F'''ortran '''C'''ompiler by setting the '''$FC''' environment variable (in this case the Intel Fortran compiler, ifort), and then run ''cmake'' (on the command line), passing it the relative location of the [[GMIN]] source directory:
<pre>
FC=ifort cmake ../../source
</pre>

If you run ''ls'', you will see some cmake files have been generated:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
CMakeCache.txt CMakeFiles cmake_install.cmake Makefile modules
</pre>

You can now run ''ccmake'' to open the GUI:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ccmake .
</pre>

To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile [[GMIN]] with AMBER9 (A9GMIN), we need to toggle the ''WITH_AMBER'' option ''ON''. Once you have done this, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.

'''Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options'''

You can now compile A9GMIN in parallel as follows:
<pre>
make -j8
</pre>

The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have an A9GMIN binary in your build directory - congratulations!
<pre>
Linking Fortran executable A9GMIN
[100%] Built target A9GMIN

--------------------------------------------------------------------------------------------- 15:23:45

csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
A9GMIN cmake_install.cmake libcudadummylib.a libmylapack.a NAB
AMBER display_version.f90 libdummylib.a Makefile nab_binaries_built
CMakeCache.txt GMIN libgminlib.a modules porfuncs.f90
CMakeFiles libamber12dummylib.a libmyblas.a n
</pre>
Plain [[GMIN]] is also built at the same time should you need it. You can move this into your ~/bin directory if you like, or anywhere else in your ''$PATH'' to make running it simple.

'''Note: If you want to use OPTIM with the new C++ implementation of the NEB routine, you will need to obtain the source code for that separately. See [https://wikis.ch.cam.ac.uk/wales/wiki/index.php/OPTIM here] for instructions.'''

==Compiling by setting options on the command line==
If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to ''cmake'' on the command line, bypassing the need for ''ccmake''. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:
<pre>
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>
where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.

'''Note: Sometimes you may get error''' (for example, Fatal Error: Can't open module file 'someModule.mod' for reading at (1): No such file or directory) when following this procedure. In that case there are three things you could try: make sure you are building in a new directory, if that does not help run `make VERBOSE=1` instead of `make -j8` or simply switch to using ccmake.

==Compiling with MPI==
To compile with MPI support add the following flags when running cmake on the command line for ifort:
<pre>
FC=mpiifort cmake ~/softwarewales/GMIN/source/ -DWITH_MPI=yes
</pre>
On nest this command line can be used to build A20GMIN for BHPT runs with AMBER20 when WITH_AMBER20
is employed via ccmake or -DWITH_AMBER20=yes. Modules that work are:
<pre>
module load gcc/7.5.0 cmake/3.23.2 ifort/64/2020/4/304 mpi/intel/2023.1.0
</pre>

The corresponding gfortran build on nest requires:
<pre>
FC=mpifort CC=mpicc CXX=mpicxx cmake ~/softwarewales/GMIN/source/ -DBLAS_LIBRARIES=/lib64/libopenblas.so.0 DCOMPILER_SWITCH=gfortran
</pre>
and modules:
<pre>
module load gcc/7.5.0 cmake/3.23.2 mpi/openmpi/gnu7/4.1.0
</pre>

An older pgi build used:
<pre>
FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes
</pre>
Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland ''pgi'' compiler. Make sure you have the correct modules loaded (in this case ''pgi'' and ''mpi-pgi''), and that the particular mpi you want (in this case ''mpi-pgi'') is listed before any other mpi's loaded (so that it has the highest priority). The modules can be loaded by typing:
<pre>
module load pgi/64/
module load mpi/openmpi/pgi/64/1.6.3
</pre>

and you can check which modules are loaded and in which order/priority by the ''module list'' command. You may need to ''module unload <name>'' any other mpi's that are higher up in the list than the one you want. You can of course set the COMPILER_SWITCH and WITH_MPI flags in ''ccmake'' if you prefer.

Note: It has been observed that pgi/64/15.1 leads to compilation errors, and for now, it is best to use pgi/64/14.9

==Advanced mode - changing compiler flags with ccmake==
[[Image:Ccmakeadvanced.png|thumb|ccmake advanced mode|200px|right]]

Although initially the ''ccmake'' GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to ''make'' when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do profiling.

As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.

Note that these changes only apply in the build directory in which you make them.

==Debugging runtime problems using gdb or valgrind==
If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like [http://www.gnu.org/software/gdb/ gdb] or [http://valgrind.org/ valgrind]. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the ''CMAKE_BUILD_TYPE'' in ''ccmake'' to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:
<pre>
FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source
make -j8
</pre>

You can then run the binary ''through'' gdb or valgrind as follows:
<pre>
gdb A9GMIN
</pre>
or
<pre>
valgrind A9GMIN
</pre>

I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)

==Debugging compilation problems==
There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the ''VERBOSE'' option enabled. This will dump a lot of potentially useful output:
<pre>
VERBOSE=1 make
</pre>

One possible gotcha: all .f and .f90 files in the relevant source directories will be compiled and added to a library. This is quite different from the old Makefile way of doing things, where source files were explicitly specified for compilation (via their corresponding .o file). So, if you are testing something by for instance copying code.f90 to code.myhack.f90 and code.orig.f90, then slightly editing a line or two of code.myhack.f90 and copying it back to code.f90 for use, this will probably cause linking problems due to multiply-defined subroutines (from all three files). The solution, if you must have alternative versions of the same file hanging round, is to differentiate the filenames by a suffix AFTER the .f[90] .

Another occasional issue is the unexplained compiler bug - a problem with the version of the compiler you happen to be using. You can can an idea for which compiler versions we expect to work by checking the Jenkins build-bot output, as described in the 'Seeing console output' section of the [[Jenkins CI]] page. If you are using a different version of the compiler in question, consider swapping to the version Jenkins is using with 'module swap'.

If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.

To build the executables with the QUIP interface, it may be necessary to do make clean in the QUIP directory.

==Extra command line build examples==
The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ''ccmake'' as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your git repository is set up in ''/home/CRSID/softwarewales'' - make the appropriate modifications if you have it elsewhere.

===GMIN===
'''A12GMIN''' (GMIN with AMBER12) using ifort:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort_amber12
cd !$
FC=ifort cmake -DWITH_AMBER12=1 ../../source
make -j8
</pre>

'''C35GMIN''' (GMIN with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAGMIN''' (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/GMIN/builds/ifort_cuda
cd !$
FC=ifort cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN with GPUs]] page.

'''DFTBGMIN''' (GMIN with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/GMIN/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/GMIN/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===OPTIM===
'''A9OPTIM''' (OPTIM with AMBER9) using ifort:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/ifort_amber
cd !$
FC=ifort cmake -DWITH_AMBER9=1 ../../source
make -j8
</pre>

'''C35OPTIM''' (OPTIM with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAOPTIM''' (OPTIM leveraging GPU via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/OPTIM/builds/ifort_cuda5.5
cd !$
FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN and OPTIM with GPUs]] page.

'''DFTBOPTIM''' (OPTIM with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/OPTIM/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/OPTIM/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===PATHSAMPLE===
There are very few options for [[PATHSAMPLE]] as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called ''PATHSAMPLE''.

Using nagfor (the NAG fortran compiler - check you have the module loaded - very strict!):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/nagfor
cd !$
FC=nagfor cmake ../../source
make -j8
</pre>

Using pgi (much more generous with coding slips/non-standard uses):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/pgi
cd !$
FC=pgf90 cmake ../../source
make -j8
</pre>

==Configuring defaults - for developers==

Fortran compilers and their corresponding default settings are all controlled by the file $SVN/CMakeModules/FindFORTRANCOMPILER.cmake ($SVN is your svn root directory). In particular, we may wish to edit the flags used for each set of compilers and build type. These are contained in the following block:

<pre>
if(NOT COMPILER_FLAGS_WERE_SET)
message("Setting initial values for compiler flags")
if(COMPILER_SWITCH MATCHES "pgi")
set (CMAKE_Fortran_FLAGS "-Mextend" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3 -Munroll -Mnoframe" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Mdwarf3 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-Mfree" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "gfortran")
set (CMAKE_Fortran_FLAGS "-ffixed-line-length-200 -ffree-line-length-0" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fimplicit-none -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "${CMAKE_Fortran_FLAGS_DEBUG} -fimplicit-none" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-ffree-form" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "nag")
set (CMAKE_Fortran_FLAGS "-132 -kind=byte -maxcontin=3000" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-mismatch_all -O4" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-C=all -mtrace=all -gline -g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING) # js850> is this ever used?
elseif(COMPILER_SWITCH MATCHES "ifort")
set (CMAKE_Fortran_FLAGS "-132 -heap-arrays -assume byterecl" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# Warnings about temporary argument creation and edit descriptor widths are disabled with the final flags.
set (CMAKE_Fortran_FLAGS_DEBUG "-C -g -traceback -debug full -check all,noarg_temp_created -diag-disable 8290,8291" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-debug all -check all -implicitnone -warn unused -fp-stack-check -ftrapuv -check pointers -check bounds" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING)
else()
message(FATAL_ERROR "unknown comiler switch: ${COMPILER_SWITCH}")
endif()
SET(COMPILER_FLAGS_WERE_SET yes CACHE TYPE INTERNAL)
endif(NOT COMPILER_FLAGS_WERE_SET)
</pre>

The main if/elseif blocks correspond to compiler switches. Inside these, there are the default flags for each of our build types (release, debug and debug_slow), which are configured using ccmake. These can be edited, if we wish to change the default behaviour (e.g. a recent addition of -check all,noarg_temp_created -diag-disable 8290,8291 to disable annoying warning messages for ifort).

Git Workflow

2024-07-22T20:24:21Z

Dw34:

Group software and papers in the process of being written are stored on the University's GitLab repositories [https://gitlab.developers.cam.ac.uk/]. You should be able to log in via Raven, but someone with privileges will need to add you to the two projects. This page describes a typical workflow for retrieving, modifying and updating the repositories. It is not, however, a comprehensive guide to Git. For that, consult your favourite web search engine.

==Setting up SSH access==

To smoothly access Gitlab without having to type your user name and password the whole time, set up an SSH key. In a terminal on your desktop, type

$ ssh-keygen -t ed25519 -C "GitLab"

If it complains about overwriting, then you have already done this step and you probably don't want to overwrite. It will ask you where to save the key: the default location should be fine. Now you are prompted for a passphrase. You can press ENTER to leave it blank, although that does mean that anyone who breaks into your computer can get access to GitLab with no further effort. You now have a file (default location is ~/.ssh/id_ed25519.pub) that contains your public key. Copy the entire contents of this file.

On the GitLab website, click on your user icon in the top right and select 'Settings' and then 'SSH Keys' from the left menu. Paste the contents of your public key file into the box. Put something useful in the title, like the name of your desktop (yes, you should probably do this for each machine you want to access GitLab from, rather than copying keys between machines). Optionally, you can insert an expiry date, such as the date your funding runs out. Click the 'Add key' button.

To test that you now have access, in a terminal type

$ ssh -T git@gitlab.developers.cam.ac.uk

After accepting the RSA identity, you should see a welcome message and then the connection will close.

Some users have reported problems with using ed25519. RSA is available as an alternative. Generate an RSA key with

$ ssh-keygen -t rsa -C "GitLab"

which will be saved by default at ~/.ssh/id_rsa.pub. Follow the same steps as above to add your public key to Gitlab.

==Installing git LFS==

Our software repository (git@gitlab.developers.cam.ac.uk:ch/wales/softwarewales.git) uses git Large File Storage (LFS) to manage some of the larger files. You must have the git LFS addon installed and initialised before cloning the software repository. If you do not, the clone will appear to succeed, but you will be missing some files. If you are using a department managed workstation, or any cluster other than rogue or nest, you will need to load a newer version of git:

$ module load git/2.0.0

Replace 2.0.0 with whatever the newest available version of git is. Or, on your personal Ubuntu machine, run

$ sudo apt-get install git-lfs

to install the necessary package. Whatever computer you are using, then run

$ git lfs install

to inform git about the new LFS addon. This command only needs to be run once on each machine you intend to clone the software repository on. If you get the error message

Error: failed to call git rev-parse --git-dir: exit status 128 : fatal: .git

then try

$ git lfs install --skip-repo

instead. After you have cloned the repository, you should inspect the LFS files to make sure the clone worked correctly. You can see a list of the files in .gitattributes in the repository root directory. A good file to check might be THESES/PHD/ChrisWhittlestonPhD.pdf. If this file is a PDF of several megabytes, then the LFS succeeded. If it is a small plaintext file containing a URL, the LFS clone did not succeed.

If you have already cloned the repository before installing the LFS addon, you will need to clone again (a pull will not suffice).

==Initial Checkout==

You need to fetch the repository. In Git terms, this is called cloning. You should only need to carry out this step once for each repository. In your favourite web browser, navigate to the project page on Gitlab, eg. [https://gitlab.developers.cam.ac.uk/ch/wales/softwarewales software]. Spot the blue button on the right labelled 'Clone' and click on it. Copy the link under 'Clone with SSH' to the clipboard (don't use the 'clone with HTTPS' link, or you will have to type your username and password every time). In a terminal, choose a suitable location, like your home directory and change to there. Now type

$ git clone git@gitlab.developers.cam.ac.uk:ch/wales/softwarewales.git

replacing the address with what you just copied. Git will download the repository. Once it has finished, check that you now have lots of new directories with the contents of the repository.

You should also tell Git your name and email address. Git will record these in the commit logs so other users will know who to complain to when a commit breaks everything. Run

$ git config --global user.name "An Other"
$ git config --global user.email "ao123@cam.ac.uk"

replacing the name and email address in quotes as appropriate.

==Submodules==

Our software repository has become quite large as external potentials have been added. Submodules offer a way to compartmentalise the repository and speed up clones and updates. Self-contained third-party potentials are good candidates for splitting off into submodules. We will use GDML (Gradient-Descent Machine Learning) as an example. On an initial checkout, you will see a GDML/ directory in the root of the repository, but the directory will be empty. Most users do not need GDML, so will not care or even be particularly aware that GDML has not be cloned.

Let us suppose that you actually need GDML. You can tell git that it is required by running

$ git submodule init GDML

Then the next time you run

$ git submodule update

the contents of GDML will be checked out. If GDML is subsequently updated, run the update command again to get the newest version. If at some later point you decide that you have finished with GDML and no longer need it checked out, run

$ git submodule deinit GDML

and the GDML directory will be emptied. If the submodule you are interested in has its own submodules, add the --recursive flag to the commands. If you decide that you want all the submodules run

$ git submodule update --init --recursive

and all submodules will be checked out. This command is not recommended and you must have a good reason why you need all the submodules before you consider running it.

===Creating a new submodule===

If you are creating an interface to a new large external potential, it may be appropriate to add the external potential as a submodule. It is appropriate if the potential is quite large (more than a few MB), is being placed in the root of the repository, and is not likely to require much in the way of changes after the interface is set up. Instead of adding the external potential to the softwarewales repository, create a new repository under Wales Group on Gitlab and place the files there. Within softwarewales run

$ git submodule add git@gitlab.developers.cam.ac.uk:ch/wales/my_new_repository.git

where my_new_repository is replaced with whatever you named the new repository. You could also got the appropriate URL by going onto Gitlab for the new repository, clicking the 'Clone' button and copying. This command makes the necessary changes to the .gitmodules file, which will then need to be committed and pushed.

Turning an existing subdirectory into a submodule is also possible, but is slightly more complicated and is considered an advanced topic. Google is your friend. Do not mess with the Gitlab repository until you are satisfied you have made the correct adjustments locally.

==Basic Workflow==

Details for specific cases are below, but first, we mention the most important commands that you'll be running all the time. Imagine you've just arrived in the morning and it's time to start working on myfile.f90. The first command to run is

$ git pull

This command contacts the remote repository on GitLab and fetches any commits that people may have made. Run this command frequently, and at least before every commit you make. One notable difference from updating in svn is that git will not merge other people's changes with files you have changed since your last commit. If other people have changed files you are working on, the pull will fail with an informative message. In this case, run

$ git stash

which sets aside your local changes. Try the pull again, which should now succeed. Then run

$ git stash pop

to reapply your local changes to the updated repository. The merge will usually happen automatically, but sometimes you will need to resolve conflicts yourself.

Now you edit myfile.90 and want to commit your changes. Run

$ git add myfile.f90

Now the file is, in Git terminology, staged for commit. You haven't committed anything yet. You can add other files to the staging area too. Once your commit is ready, run

$ git commit -m "Informative message."

Replace 'Informative message' with a brief message describing what changes are in your commit. At this point, you have updated your local repository and entered a commit in the permanent record. However, the commit hasn't gone to GitLab yet. To send it to GitLab (called the remote by Git), run

$ git push

You can send multiple commits at once. This workflow should encourage you to commit often. Maybe you write a new function. Put in a commit. Then you add some stuff to keywords.f90 for the new functionality. Do another commit. Next you find a bug and fix it. Do another commit.

Note that the remote can function as the backup of your work. Therefore you should probably push any new commits at least as often as the end of each day.

==Checking out an older version of the master branch==

There are all sorts of useful git tools for checking how individual files and branches have changed.
See the documentation for git diff for example. To check out the master branch current at a given date you can use:

git checkout `git rev-list -1 --before="Feb 11 2024" master`

==Writing a Paper==

Writing a paper is slightly simpler than editing the group code (Discuss...) because we aren't worrying about multiple branches. Each paper is a separate repository. To start a new paper, go to Gitlab and create a new repository by clicking the blue 'New Project' button on your home screen. Create a blank project and make sure the project URL indicates it is under ch/wales rather than in your user space. Checkout the new repository and start writing the paper in the blank directory. Each session of editing should involve

# git pull
# make some edits
# git pull
# git add all the edited files
# git commit with a helpful message
# git push

Simples. All authors will be editing the same branch (the master branch), so you'll see other authors' updates straight away. This approach keeps things easy, but if two authors are working at exactly the same time, there may be some merging to do. Reduce the amount of merging by committing, pulling and pushing often.

Do not add intermediate LaTeX files (.aux, .log, etc.) to the repository. Do not add your .dvi/.ps/.pdf documents either (except perhaps for proofs created by the journal). When it comes to revisions and resubmissions, do not create a new subdirectory for the new version. Git keeps the whole history so it is always possible to revert to a previous version.

You will also want to checkout the [https://gitlab.developers.cam.ac.uk/ch/wales/bib bibliography repository] but you only need one copy of this, not one for each paper. A suitable directory structure might be to have a papers/ directory under your home directory that contains a directory for the bibliography repository and a directory for each paper you are currently working on.

==Working on the group code==

You've just been talking to David and you've come up with an exciting new feature to add to GMIN. It's going to take several days of coding, during which you'll want to back up your work on the remote, but you don't want to interfere with other people using GMIN. The solution is to create a new branch. A branch is your own version controlled copy of the code that you can edit at will without messing GMIN up for anyone else. All development should occur on branches. To create a new branch, run

$ git checkout -b exciting_feature

This command both creates the branch and switches your working copy to it. Initially, your new branch is the same as the master branch you cut it from. However, the branch does not yet exist on GitLab. To create it, run

$ git push --set-upstream origin exciting_feature

Now go ahead and edit files, making commits and pushing them to GitLab frequently.

When your feature is complete and you have checked it works and that you haven't broken anything else, it's time to get it into the master branch. Several steps are required. Firstly, it's quite likely that other people have changed master since you cut off your branch. You need to test that your changes function with the new changes to master, so first you need to merge in master:

$ git checkout master
$ git pull
$ git checkout exciting_feature
$ git merge master
$ git push

The pull commands makes sure that your local copy of the repository is up to date. The merge command merges changes that have been made to the master branch to your branch. It creates new commits, that you then push to the remote of your branch.

Most users do not have permission to edit the master branch. To get your new feature in, you have to create a merge request. Go to the project page on GitLab. From the drop down menu of branches, select exciting_feature. Click the blue 'Create merge request' button in the top right. You now have a page in which you can give your merge request a title and description. You should assign the request to yourself and anyone else who has worked on this branch. Choose the person who is going to review for you. Make sure you tick the boxes to delete the feature branch after the request is accepted. These options help keep the remote repository and history clean. You can edit the commit message for the one commit that will be created: by default it will be the name of your branch. The person you select as reviewer will get a notification and a copy of your changes. They will look through your changes to make sure they follow the group coding standards ([https://wikis.ch.cam.ac.uk/wales/wiki/index.php/Wales_Group_Fortran_conventions_for_group_software here]) and that you haven't broken anything. If there are any issues, they may request that you make some changes, which you can then commit to the exciting_feature branch. The pull request will be updated and the reviewer will get a notification. However, the reviewer will not be doing extensive testing, so it remains your responsibility to follow the coding standards and make sure everything works. You should make sure your changes compile with nagfor before submitting the merge request, as that is the most particular compiler. Once your code has passed the review, the reviewer will click the Merge button and your branch will be merged into master. Just 'Approve' from the reviewer won't usually be enough because you probably don't have permission to write to Master and hence cannot merge your new branch into master yourself. Once it has been merged, you can clean up your repository with the following commands

$ git checkout master
$ git pull
$ git branch -d exciting_feature

These commands switch your working copy back to master, update your local copy of the repository, and delete the branch you made. You might like to check that your new feature is in the master branch files before deleting your branch. If something has gone wrong and you delete your branch before the changes are in master, it is possible to recover, as the commits won't actually be deleted from the remote for a few weeks, but the recovery is an advanced topic that is best avoided.

===Large Files===

If any new file you are adding is large (>10MB), it should be stored on git LFS, rather than as a normal file. Fortunately, this is easy to do for new files. If you have already committed a large file and would now like to change it, you have a very complicated process ahead. The best instructions the author could find when doing this in the initial repository migration were here https://stackoverflow.com/questions/60995429/gradually-converting-a-repo-to-use-git-lfs-for-certain-files-one-file-at-a-time in the question, with the caveat that the bfg utility does not work and it was necessary to use git filter-branch as described here https://dalibornasevic.com/posts/2-permanently-remove-files-and-folders-from-a-git-repository instead. Note this process will delete the history of your existing file and it will only appear from the most recent commit. It may be possible to adjust this with a git rebase, but the author has not investigated.

Anyway, if you haven't yet committed your large file, you simply need to run

$ git lfs track "<path-to-file>"

which informs git that this file is to be uploaded using LFS. This command edits the file .gitattributes, which will also need to be added to your commit. If you make a mistake, you can edit .gitattributes manually. You can see some examples as well as the current list of files that are uploaded with LFS in .gitattributes. As you can see from inspecting the file, it is also possible to use wildcards ('*') to specify multiple files at once. Be careful with your rules though: the rule is applied over all files, so if you add a rule for 'myfile.f90' and somewhere else in the repository there is another file with the same name, it will now also be uploaded with LFS. This can be useful for specifying, for example, all .mp4 files in the whole repository, with "*.mp4". However, if you really want just a specific file, specify the path from the repository root, for example "OPTIM/source/myfile.f90".

==Useful commands to know==

$ git status

At any point, this command will show you what branch you are on, what files you have modified and staged and your local position compared to the remote. Use it often.

$ git branch

Display a list of all the current branches.

$ git diff

Show the differences between your working copy and the last commit, for all files. Add a file name to show only the differences for a specific file.

$ git log

Display the commit history. Add a file or directory name afterwards to only show the commits that affected that file, or any file in the directory.

$ git reset HEAD myfile.f90

Unstage myfile.f90 that you accidentally staged for the next commit, but actually don't want to commit just yet. The working copy of the file is not altered.

$ git checkout -- myfile.f90

Revert myfile.f90 that you've completely messed up to what it was at the last commit. Changes to your working copy are lost.

$ git reset --hard

Throw away all working and staged changes, reverting the current state to the last commit.

$ git reset --hard 909a3cac63ae8782b258ebb8c27af361b555bff6

Throw away all working and staged changes, reverting the current state to that of the commit specified. The long hex number is a commit hash. It is not human readable, but you can copy the relevant one from the commit log.

$ git clean -f

Throw away all untracked files. They will be deleted. Run with -n rather than -f to see which files would be deleted, but without actually doing anything.

$ git fetch -p && for branch in $(git branch -vv | grep ': gone]' | awk '{print $1}'); do git branch -D $branch; done

Delete all local branches that do not exist on GitLab. This command is useful to periodically clean up local branches after they have been merged and deleted on GitLab. Warning: do not run this command if you've created a new local branch and not yet pushed it to GitLab.

Allowing read access to your directories

2024-03-16T16:03:49Z

Dw34: Created page with "Some of the clusters have permissions set on home directories that do not allow group read access. To change the access recursively use: chmod -R g+r /home/<your crsid> To m..."

Some of the clusters have permissions set on home directories that do not allow group read access.
To change the access recursively use:

chmod -R g+r /home/<your crsid>

To make all the subfolders accessible, you also need to add the executable bit. To do this for directories only use

find /home/<your crsid> -type d -exec chmod g+x {}

Compiling Wales Group codes using cmake

2022-10-31T20:38:48Z

Dw34: /* Debugging compilation problems */

[http://www.cmake.org/ CMake] (Cross-platform Make) provides a simple, platform independent way for us to compile and test the group codebase. Dependencies are handled automatically, compilation can proceed in parallel to avoid long waits while testing changes and builds are done entirely outside of the source directory. It also enables us to use the [[Jenkins CI]] 'build bot' system to automatically compile and test the code on a nightly basis - helping us catch troublesome commits before they affect other users.

Although everything below refers to compiling [[GMIN]] with the Intel ''ifort'' compiler and AMBER9 - the exact same procedure works for [[OPTIM]] and [[PATHSAMPLE]].

Note that not every option for our codes is expected to actually compile with every compiler, for example, anything using CHARMM35/36 will not compile with ''nagfor'' or ''gfortran''. This is nothing to do with our code - it's a CHARMM issue. You can get an idea for what should work by looking at the automated [[Jenkins CI]] builds.

==Preparing to compile==
Before you get started, you need to ensure that the machine you are planning to compile on has cmake 2.8 or higher installed. You can check the current version like so:
<pre>
cmake --version
</pre>

The clusters have a module for cmake 3.0, which you can load using the following command:
<pre>
module load cmake/3.0.0
</pre>

You also need to create a directory to build the code in. We suggest that you create a directory for the compiler you are using within the program directory, under a subdirectory called 'builds' - for example for compiling GMIN with ifort, you would make a directory here:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort
cd ~/softwarewales/GMIN/builds/ifort
</pre>
You can call these directories whatever you like - but make sure it is clear to you what they contain! You might also want to check which version of the compiler you have loaded. This is important as the different clusters and workstations may have different default versions loaded, some of which might not work properly. You can check the compiler version currently loaded using the same '--version' flag we used for ''cmake'' above:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ifort --version
ifort (IFORT) 12.1.3 20120212
Copyright (C) 1985-2012 Intel Corporation. All rights reserved.
</pre>

To load a different compiler, you can use the ''module load'' or ''module swap'' commands. A list of all available modules can be accessed using:
<pre>
module av
</pre>
If you are having problems compiling, one of the first things to check is whether it works with a different version of the compiler!

'''Note: When compiling GMIN, if you are getting the error that there is no implicit type for ERFC in ewald.f90, try using a newer version of your compiler. This should be the built-in complementary error function.'''

==Compiling using the ccmake GUI interface to set options==
[[Image:Ccmake.png|thumb|ccmake set up to compile A9GMIN|200px|right]]

One advantage using cmake has over make is that we can use the simple ccmake GUI. This interface lets us set options like compiling with AMBER9, or CHARMM35, toggle between 'Release' and 'Debug' builds (see below) - and examine and alter the flags being uses for the compilation if we wish. Before we can run ccmake, we need to specify the compiler and run cmake in our build directory (e.g. softwarewales/GMIN/builds/ifort). We specify the '''F'''ortran '''C'''ompiler by setting the '''$FC''' environment variable (in this case the Intel Fortran compiler, ifort), and then run ''cmake'' (on the command line), passing it the relative location of the [[GMIN]] source directory:
<pre>
FC=ifort cmake ../../source
</pre>

If you run ''ls'', you will see some cmake files have been generated:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
CMakeCache.txt CMakeFiles cmake_install.cmake Makefile modules
</pre>

You can now run ''ccmake'' to open the GUI:
<pre>
csw34@sinister:~/softwarewales/GMIN/builds/ifort> ccmake .
</pre>

To navigate between options, use the arrow keys. Options can be toggled by pressing Return. To compile [[GMIN]] with AMBER9 (A9GMIN), we need to toggle the ''WITH_AMBER'' option ''ON''. Once you have done this, you need to configure and generate appropriate cmake info. This is done by pressing 'c' to configure, 'e' to exit and then 'g' to generate.

'''Note: for some builds (CHARMM with DFTB and CUDAGMIN), you might need to configure, exit and generate twice to set all necessary options'''

You can now compile A9GMIN in parallel as follows:
<pre>
make -j8
</pre>

The '-j8' flag here tells make to use up to 8 'threads' when building. For optimal performance, you should keep this slightly greater than the number of cores (CPUs) the node you are working on has. If all goes well, you should now have an A9GMIN binary in your build directory - congratulations!
<pre>
Linking Fortran executable A9GMIN
[100%] Built target A9GMIN

--------------------------------------------------------------------------------------------- 15:23:45

csw34@sinister:~/softwarewales/GMIN/builds/ifort> ls
A9GMIN cmake_install.cmake libcudadummylib.a libmylapack.a NAB
AMBER display_version.f90 libdummylib.a Makefile nab_binaries_built
CMakeCache.txt GMIN libgminlib.a modules porfuncs.f90
CMakeFiles libamber12dummylib.a libmyblas.a n
</pre>
Plain [[GMIN]] is also built at the same time should you need it. You can move this into your ~/bin directory if you like, or anywhere else in your ''$PATH'' to make running it simple.

'''Note: If you want to use OPTIM with the new C++ implementation of the NEB routine, you will need to obtain the source code for that separately. See [https://wikis.ch.cam.ac.uk/wales/wiki/index.php/OPTIM here] for instructions.'''

==Compiling by setting options on the command line==
If you know the options you'd like to set already (you can see them all in ccmake), you can save some time by passing them directly to ''cmake'' on the command line, bypassing the need for ''ccmake''. For example, to compile A9GMIN (GMIN with the AMBER9 interface) using the Intel ifort compiler, you would run the following commands:
<pre>
FC=ifort cmake -DWITH_AMBER=1 ../../source
make -j8
</pre>
where '../../source' is the relative location of the GMIN source directory. You can find some more examples of compiling from the command line below.

'''Note: Sometimes you may get error''' (for example, Fatal Error: Can't open module file 'someModule.mod' for reading at (1): No such file or directory) when following this procedure. In that case there are three things you could try: make sure you are building in a new directory, if that does not help run `make VERBOSE=1` instead of `make -j8` or simply switch to using ccmake.

==Compiling with MPI==
To compile with MPI support add the following flags when running cmake on the command line:
<pre>
FC=mpif90 CC=mpicc cmake ../source -DCOMPILER_SWITCH=pgi -DWITH_MPI=yes
</pre>
Here -DCOMPILER_SWITCH=pgi assumes you're using the Portland ''pgi'' compiler. Make sure you have the correct modules loaded (in this case ''pgi'' and ''mpi-pgi''), and that the particular mpi you want (in this case ''mpi-pgi'') is listed before any other mpi's loaded (so that it has the highest priority). The modules can be loaded by typing:
<pre>
module load pgi/64/
module load mpi/openmpi/pgi/64/1.6.3
</pre>

and you can check which modules are loaded and in which order/priority by the ''module list'' command. You may need to ''module unload <name>'' any other mpi's that are higher up in the list than the one you want. You can of course set the COMPILER_SWITCH and WITH_MPI flags in ''ccmake'' if you prefer.

Note: It has been observed that pgi/64/15.1 leads to compilation errors, and for now, it is best to use pgi/64/14.9

==Advanced mode - changing compiler flags with ccmake==
[[Image:Ccmakeadvanced.png|thumb|ccmake advanced mode|200px|right]]

Although initially the ''ccmake'' GUI looks very simple, there is a lot going on under the hood. By pressing 't' you can enter 'Advanced mode' which will show you all of the hidden options, for example the compiler flags that are being passed to ''make'' when you compile the code. You can also make changes to the flags here, for example if you would like to add '-p' to do profiling.

As with changing the build type, you simply select the field you'd like to change using the arrow keys, press Return, make your changes and press Return again to save them. When you subsequently configure and generate as above, those altered flag will be used for the subsequent compilation.

Note that these changes only apply in the build directory in which you make them.

==Debugging runtime problems using gdb or valgrind==
If you are getting a segmentation fault, crash or other unexpected behaviour, you might want to run your job through a debugger like [http://www.gnu.org/software/gdb/ gdb] or [http://valgrind.org/ valgrind]. In order to maximise your chances of getting useful output, you should build a 'Debug' version of the program you are having trouble. To do this, you can either change the ''CMAKE_BUILD_TYPE'' in ''ccmake'' to 'Debug' (press Return, change 'Release' to 'Debug' and press Return again), or on the command line like so for GMIN with AMBER 9 using the Intel ifort compiler:
<pre>
FC=ifort cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_AMBER=1 ../../source
make -j8
</pre>

You can then run the binary ''through'' gdb or valgrind as follows:
<pre>
gdb A9GMIN
</pre>
or
<pre>
valgrind A9GMIN
</pre>

I won't cover debugging with these tools here as it's a science in itself! Do some Googling and ask for help as needed :)

==Debugging compilation problems==
There are many ways to try and track down why your code is not compiling. Before you start changing compilers, building a 'Debug' version or changing machines, you might want to try running make again with the ''VERBOSE'' option enabled. This will dump a lot of potentially useful output:
<pre>
VERBOSE=1 make
</pre>

One possible gotcha: all .f and .f90 files in the relevant source directories will be compiled and added to a library. This is quite different from the old Makefile way of doing things, where source files were explicitly specified for compilation (via their corresponding .o file). So, if you are testing something by for instance copying code.f90 to code.myhack.f90 and code.orig.f90, then slightly editing a line or two of code.myhack.f90 and copying it back to code.f90 for use, this will probably cause linking problems due to multiply-defined subroutines (from all three files). The solution, if you must have alternative versions of the same file hanging round, is to differentiate the filenames by a suffix AFTER the .f[90] .

Another occasional issue is the unexplained compiler bug - a problem with the version of the compiler you happen to be using. You can can an idea for which compiler versions we expect to work by checking the Jenkins build-bot output, as described in the 'Seeing console output' section of the [[Jenkins CI]] page. If you are using a different version of the compiler in question, consider swapping to the version Jenkins is using with 'module swap'.

If the error message you are getting doesn't make sense to you after some Googling, go and ask someone - we all have these problems. Things you can try first include trying a different compiler version, or an entirely different compiler e.g. pgi rather than ifort for example. You should bear in mind that as mentioned above, not all versions of each code will compile with every compiler. Make sure you're not trying to build something that isn't expected to work.

To build the executables with the QUIP interface, it may be necessary to do make clean in the QUIP directory.

==Extra command line build examples==
The below commands are absolutely not an exhaustive list, but should give you an idea of what is possible. You can use ''ccmake'' as described above to discover which variables (e.g. WITH_AMBER) can be manipulated on the command line like this. All of these examples assume your git repository is set up in ''/home/CRSID/softwarewales'' - make the appropriate modifications if you have it elsewhere.

===GMIN===
'''A12GMIN''' (GMIN with AMBER12) using ifort:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/ifort_amber12
cd !$
FC=ifort cmake -DWITH_AMBER12=1 ../../source
make -j8
</pre>

'''C35GMIN''' (GMIN with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/GMIN/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAGMIN''' (GMIN leveraging GPU minimisation via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/GMIN/builds/ifort_cuda
cd !$
FC=ifort cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN with GPUs]] page.

'''DFTBGMIN''' (GMIN with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/GMIN/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/GMIN/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===OPTIM===
'''A9OPTIM''' (OPTIM with AMBER9) using ifort:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/ifort_amber
cd !$
FC=ifort cmake -DWITH_AMBER9=1 ../../source
make -j8
</pre>

'''C35OPTIM''' (OPTIM with CHARMM 35) using pgi:
<pre>
mkdir -p ~/softwarewales/OPTIM/builds/pgi_charmm35
cd !$
FC=pgf90 cmake -DWITH_CHARMM35=1 ../../source
make -j8
</pre>

'''CUDAOPTIM''' (OPTIM leveraging GPU via the AMBER 12 interface) using ifort:
<pre>
module load cuda/5.5
mkdir -p ~/softwarewales/OPTIM/builds/ifort_cuda5.5
cd !$
FC=ifort CC=icc cmake -DWITH_CUDA=1 ../../source
make -j8
</pre>
This will only work on machines with specific NVIDIA GPUs, for example when submitting jobs on the pat cluster. There is some additional information on the [[Using GMIN and OPTIM with GPUs]] page.

'''DFTBOPTIM''' (OPTIM with DFTBP) using Intel on nest:
<pre>
module load mkl/64/2022/0/0 cmake/3.23.2 ifort/64/2020/4/304
mkdir -p ~/softwarewales/OPTIM/builds/ifort_dftbp
cd !$
FC=ifort CC=icc CXX=icc cmake ../../source -DWITH_DFTBP=yes
make -j8
</pre>

or using GCC on nest:
<pre>
module load cmake/3.23.2 gcc/12.2.0
mkdir -p ~/softwarewales/OPTIM/builds/gfortran_dftbp
cd !$
cmake ../../source -DWITH_DFTBP=yes -DWITH_MYBLAS=no -DBLAS_LIBRARIES=/usr/lib64/libopenblas.so.0
make -j8
</pre>

===PATHSAMPLE===
There are very few options for [[PATHSAMPLE]] as we don't need to worry about interfacing with a particular potential. As a result, every binary is simply called ''PATHSAMPLE''.

Using nagfor (the NAG fortran compiler - check you have the module loaded - very strict!):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/nagfor
cd !$
FC=nagfor cmake ../../source
make -j8
</pre>

Using pgi (much more generous with coding slips/non-standard uses):
<pre>
mkdir -p ~/softwarewales/PATHSAMPLE/builds/pgi
cd !$
FC=pgf90 cmake ../../source
make -j8
</pre>

==Configuring defaults - for developers==

Fortran compilers and their corresponding default settings are all controlled by the file $SVN/CMakeModules/FindFORTRANCOMPILER.cmake ($SVN is your svn root directory). In particular, we may wish to edit the flags used for each set of compilers and build type. These are contained in the following block:

<pre>
if(NOT COMPILER_FLAGS_WERE_SET)
message("Setting initial values for compiler flags")
if(COMPILER_SWITCH MATCHES "pgi")
set (CMAKE_Fortran_FLAGS "-Mextend" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3 -Munroll -Mnoframe" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-Mextend -C -g -gopt -Mbounds -Mchkfpstk -Mchkptr -Mchkstk -Mcoff -Mdwarf1 -Mdwarf2 -Mdwarf3 -Melf -Mpgicoff -traceback" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-Mfree" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "gfortran")
set (CMAKE_Fortran_FLAGS "-ffixed-line-length-200 -ffree-line-length-0" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fimplicit-none -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -fbounds-check -Wuninitialized -O -ftrapv -fno-automatic" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "${CMAKE_Fortran_FLAGS_DEBUG} -fimplicit-none" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-ffree-form" CACHE TYPE STRING)
elseif(COMPILER_SWITCH MATCHES "nag")
set (CMAKE_Fortran_FLAGS "-132 -kind=byte -maxcontin=3000" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-mismatch_all -O4" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG "-g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-C=all -mtrace=all -gline -g -mismatch_all -ieee=stop" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING) # js850> is this ever used?
elseif(COMPILER_SWITCH MATCHES "ifort")
set (CMAKE_Fortran_FLAGS "-132 -heap-arrays -assume byterecl" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_RELEASE "-O3" CACHE TYPE STRING FORCE)
# Warnings about temporary argument creation and edit descriptor widths are disabled with the final flags.
set (CMAKE_Fortran_FLAGS_DEBUG "-C -g -traceback -debug full -check all,noarg_temp_created -diag-disable 8290,8291" CACHE TYPE STRING FORCE)
set (CMAKE_Fortran_FLAGS_DEBUG_SLOW "-debug all -check all -implicitnone -warn unused -fp-stack-check -ftrapuv -check pointers -check bounds" CACHE TYPE STRING FORCE)
set (FORTRAN_FREEFORM_FLAG "-free" CACHE TYPE STRING)
else()
message(FATAL_ERROR "unknown comiler switch: ${COMPILER_SWITCH}")
endif()
SET(COMPILER_FLAGS_WERE_SET yes CACHE TYPE INTERNAL)
endif(NOT COMPILER_FLAGS_WERE_SET)
</pre>

The main if/elseif blocks correspond to compiler switches. Inside these, there are the default flags for each of our build types (release, debug and debug_slow), which are configured using ccmake. These can be edited, if we wish to change the default behaviour (e.g. a recent addition of -check all,noarg_temp_created -diag-disable 8290,8291 to disable annoying warning messages for ifort).

VMD

2021-10-23T07:33:28Z

Dw34:

VMD is a molecular visualization program installed on all workstations and clusters. The official documentation can be found [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html here] with some tutorials [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html here]. Like gnuplot however, the wealth of options means that often it takes a long time to find the one command you need to use so below you will find some useful basic settings/info for using VMD. For producing graphics for publication, [[Pymol]] is probably a better option as it has a built in ray-tracing routine but for general visualization, VMD is much quicker.

It is possible to load most files using command line flags, making loading many frames into different topology files easy. The -f flag indicates that all subsequent files (until the next -f flag or the end) should be loaded into a single molecule. There are also flags for selecting different file types (default is .pdb), most commonly parm7 for topology files generated by Amber and rst7 for restart files generated by Amber. mdcrd files are denoted -crd and periodic mdcrd files -crdbox.

e.g.
vmd -f first_mol.pdb \
-f -parm7 second_mol.prmtop -rst7 second_mol.rst \
-f -parm7 third_mol.prmtop -crdbox third_mol_1st_frames.crd -crdbox third_mol_2nd_frames

== Rendering Molecules with a Transparent Background ==

Dumping selected frames from a movie or a single image can be achieved using the render option in the vmd gui.
Choosing the povray option should produce a vmdscene.pov file. This pov file can be converted to a png with a
transparent background using povray if you must have a white background in vmd:

povray +W829 +H771 -Ivmdscene.pov -Ovmdscene.pov.tga -Otest +Q11 +J +A +FN +UA

The +W and +H values must be taken from the vmdscene.pov file; they should be given in a comment statement at the top.

== Movie Making Tips ==
To load all frames in one go, select the file type in the "Determine file type" box, and then the button "load all at once"
will not be greyed out, so you can select it.

vmd movie making seems not work properly with step sizes different from one. The last frame is repeated many times. Instead, the
frames can be selected using sed:

Try extracting frames first with sed:

sed -e '1~66087,+62939d' path.xyz > temp

1,+62939d deletes lines 1 to 62940, deleting 20 frames

The ~66087 repeats the action every 21 frames. The counter operates on the original line numbers.

This example is for a
system with 3145 atoms, so each frame is 3147 lines with the xyz header.

sed -e '1~Y,+Xd' path.xyz > temp

1,+Xd deletes lines 1 to X+1, so to select every nth frame for frames of length m you need

X=n*m-1

and Y=(n+1)*m

m is the number of atoms plus two.

To make the movie pause at the start and finish, just duplicate these end points sufficiently. If there are
slow portions around local minima try adjusting the energy difference parameter on the PATH line in the OPTIM odata file for intial
generation of path.xyz.

== Tutorials ==
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* making movies from a '.pdb' file containing multiple structures. ''This is dealt with in the OPTIM section as part of the tutorial on making a movie of a path''
* [[visualising normal modes using VMD and OPTIM]]

VMD

2021-08-10T08:53:30Z

Dw34:

VMD is a molecular visualization program installed on all workstations and clusters. The official documentation can be found [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html here] with some tutorials [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html here]. Like gnuplot however, the wealth of options means that often it takes a long time to find the one command you need to use so below you will find some useful basic settings/info for using VMD. For producing graphics for publication, [[Pymol]] is probably a better option as it has a built in ray-tracing routine but for general visualization, VMD is much quicker.

It is possible to load most files using command line flags, making loading many frames into different topology files easy. The -f flag indicates that all subsequent files (until the next -f flag or the end) should be loaded into a single molecule. There are also flags for selecting different file types (default is .pdb), most commonly parm7 for topology files generated by Amber and rst7 for restart files generated by Amber. mdcrd files are denoted -crd and periodic mdcrd files -crdbox.

e.g.
vmd -f first_mol.pdb \
-f -parm7 second_mol.prmtop -rst7 second_mol.rst \
-f -parm7 third_mol.prmtop -crdbox third_mol_1st_frames.crd -crdbox third_mol_2nd_frames

== Movie Making Tips ==
To load all frames in one go, select the file type in the "Determine file type" box, and then the button "load all at once"
will not be greyed out, so you can select it.

vmd movie making seems not work properly with step sizes different from one. The last frame is repeated many times. Instead, the
frames can be selected using sed:

Try extracting frames first with sed:

sed -e '1~66087,+62939d' path.xyz > temp

1,+62939d deletes lines 1 to 62940, deleting 20 frames

The ~66087 repeats the action every 21 frames. The counter operates on the original line numbers.

This example is for a
system with 3145 atoms, so each frame is 3147 lines with the xyz header.

sed -e '1~Y,+Xd' path.xyz > temp

1,+Xd deletes lines 1 to X+1, so to select every nth frame for frames of length m you need

X=n*m-1

and Y=(n+1)*m

m is the number of atoms plus two.

To make the movie pause at the start and finish, just duplicate these end points sufficiently. If there are
slow portions around local minima try adjusting the energy difference parameter on the PATH line in the OPTIM odata file for intial
generation of path.xyz.

== Tutorials ==
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* making movies from a '.pdb' file containing multiple structures. ''This is dealt with in the OPTIM section as part of the tutorial on making a movie of a path''
* [[visualising normal modes using VMD and OPTIM]]

VMD

2021-08-10T08:42:27Z

Dw34:

VMD is a molecular visualization program installed on all workstations and clusters. The official documentation can be found [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html here] with some tutorials [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html here]. Like gnuplot however, the wealth of options means that often it takes a long time to find the one command you need to use so below you will find some useful basic settings/info for using VMD. For producing graphics for publication, [[Pymol]] is probably a better option as it has a built in ray-tracing routine but for general visualization, VMD is much quicker.

It is possible to load most files using command line flags, making loading many frames into different topology files easy. The -f flag indicates that all subsequent files (until the next -f flag or the end) should be loaded into a single molecule. There are also flags for selecting different file types (default is .pdb), most commonly parm7 for topology files generated by Amber and rst7 for restart files generated by Amber. mdcrd files are denoted -crd and periodic mdcrd files -crdbox.

e.g.
vmd -f first_mol.pdb \
-f -parm7 second_mol.prmtop -rst7 second_mol.rst \
-f -parm7 third_mol.prmtop -crdbox third_mol_1st_frames.crd -crdbox third_mol_2nd_frames

== Movie Making Tips ==
To load all frames in one go, select the file type in the "Determine file type" box, and then the button "load all at once"
will not be greyed out, so you can select it.

vmd movie making seems not work properly with step sizes different from one. The last frame is repeated many times. Instead, the
frames can be selected using sed:

Try extracting frames first with sed:
sed -e '1~66087,+62939d' path.xyz > temp
1,+62939d deletes lines 1 to 62940, deleting 20 frames
The ~66087 repeats the action every 21 frames. The counter operates on the original line numbers. This example is for a
system with 3145 atoms, so each frame is 3147 lines with the xyz header.

sed -e '1~Y,+Xd' path.xyz > temp
1,+Xd deletes lines 1 to X+1, so to select every nth frame for frames of length m you need X=n*m-1
and Y=(n+1)*m
m is the number of atoms plus two.

== Tutorials ==
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* making movies from a '.pdb' file containing multiple structures. ''This is dealt with in the OPTIM section as part of the tutorial on making a movie of a path''
* [[visualising normal modes using VMD and OPTIM]]

Beginner's guide to working in Wales group

2020-04-13T18:16:23Z

Dw34: /* How to access and add papers to the group bib? */

===How to access and add papers to the group bib===
*You will need svn to work, which should be transparent if you are logged in to a computer in the department. Whenever asked for a password, enter your admitto password.
*In the directory where you wish to handle paper reading/writing, create a working copy of the bib directory.
For example,
<pre>
cd ~
svn checkout https://svn.ch.cam.ac.uk/svn/wales/groups/djwpapers/bib
cd bib
svn update
</pre>
* To add a bib entry, make the desired changes and then save them. Make sure that the entry does not
exist already. Duplicate entries with different bibtex mnemonics cause erroneous entries in the bibliography,
which are tedious to correct if they are only detected at the proof stage. The group files are an important resource, which can save a great deal of time in paper and thesis writing.

All the other group papers in progress can be found in the djwpapers directory, but be warned that it is rather large because there is a backlog of papers that have appeared, but have yet to be moved to David's archive. It is probably best to check out individual directories that you are interested in.

The bib entry for a paper can usually be obtained by googling the paper first, find the link 'cite this' somewhere, choose the 'BibTeX' option when asked to select citation manager/file format, and then download it (or directly open it) to see the bib entry. For naming the keys, please follow the group practice as given in https://wikis.ch.cam.ac.uk/wales/wiki/index.php/Wales_Group_Conventions_when_using_LaTex
*Commit your changes with a descriptive message so that others know what the commit corresponds to.
<pre>
svn commit -m "Your message here"
</pre>

For further details visit http://subversion.apache.org/quick-start

Beginner's guide to working in Wales group

2020-04-13T18:13:29Z

Dw34: /* How to access and add papers to group bib repository */

===How to access and add papers to the group bib?===
*You will need svn to work, which should be transparent if you are logged in to a computer in the department. Whenever asked for a password, enter your admitto password.
*In the directory where you wish to handle paper reading/writing, create a working copy of the bib directory.
For example,
<pre>
cd ~
svn checkout https://svn.ch.cam.ac.uk/svn/wales/groups/djwpapers/bib
cd bib
svn update
</pre>
* To add a bib entry, make the desired changes and then save them. Make sure that the entry does not
exist already. Duplicate entries with different bibtex mnemonics cause erroneous entries in the bibliography,
which are tedious to correct if they are only detected at the proof stage. The group files are an important resource, which can save a great deal of time in paper and thesis writing.

The bib entry for a paper can usually be obtained by googling the paper first, find the link 'cite this' somewhere, choose the 'BibTeX' option when asked to select citation manager/file format, and then download it (or directly open it) to see the bib entry. For naming the keys, please follow the group practice as given in https://wikis.ch.cam.ac.uk/wales/wiki/index.php/Wales_Group_Conventions_when_using_LaTex
*Commit your changes with a descriptive message so that others know what the commit corresponds to.
<pre>
svn commit -m "Your message here"
</pre>

For further details visit http://subversion.apache.org/quick-start

Comprehensive Contents Page

2020-01-29T14:48:48Z

Dw34:

This page is designed to organise all of the pages on this wiki, as well as provide other useful links. Note that some pages may appear under more than one heading.

== Getting Started ==
[[Wales Group]] provides good step-by-step instructions. Relevant pages are:

=== Acquiring and compiling the group software ===
* [[SVN setup]]
* [[Wales Group Version control]] - to keep the code standardised.
* Theory Sector [http://wwmm.ch.cam.ac.uk/wikis/cuc3/index.php/SVN_Page SVN Page] - some useful general information on SVN commands.
* [[Compiling Wales Group codes using cmake]] - CMake (Cross-platform Make) allows us to compile and test the group codebase regardless of platform. This page provides crucial information how to compile using cmake.
* [[ElaborateDiff]]

=== Maintaining code health ===
* [[Jenkins CI]] - explains Jenkins, which we use to download our code and compile each of our targets with each of the compilers every night.
* https://wales-jenkins.ch.cam.ac.uk/ - log for our Jenkins tests.
* [[Branching and Merging]]
* [[Cmake interface building]]
* [[Installing python modules]]
* [[Revamping the modules system]]

=== Collaborators without access to the SVN repository ===
For licensing reasons, some code cannot be included in the Wales Group public tarball.
* http://www-wales.ch.cam.ac.uk/svn.tar.bz2 - Wales group public tarball. Includes [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]].
If a collaborator has a [[CHARMM]] or [[AMBER]] licence, we do maintain separate tarballs which include the [[CHARMM]], [[AMBER]] and [[CHARMM]]+[[AMBER]] source and interfaces. These are not linked anywhere on the website and require a username ('''wales''') and password ('''group''') to download:

* [http://www-wales.ch.cam.ac.uk/CHARMM/svn.CHARMM.tar.bz2 CHARMM]
* [http://www-wales.ch.cam.ac.uk/AMBER/svn.AMBER.tar.bz2 AMBER]
* [http://www-wales.ch.cam.ac.uk/both/svn.both.tar.bz2 AMBER+CHARMM]

=== Running on Windows ===
Not particularly recommended.
* [[Running Wales Group software on Windows 7]]

== Wales Group Programs ==

=== Programs ===
* [[GMIN]]: A program for finding global minima and calculating thermodynamic properties from basin-sampling.
* [[OPTIM]]: A program for optimizing geometries and calculating reaction pathways.
* [[PATHSAMPLE]]: A driver for OPTIM to create stationary point databases using discrete path sampling and perform kinetic analysis.
* [[Pele]]: Python energy landscape explorer. A pythonic rewrite of some core functionality of GMIN, OPTIM, and PATHSAMPLE. Can be very useful for visualizing your system and for rapidly implementing and testing new ideas.

=== Curated Examples ===
* https://github.com/wales-group/examples - set of tutorials detailing how to use GMIN, OPTIM and PATHSAMPLE. Essential for beginners.
* http://www-wales.ch.cam.ac.uk/VM/Wales_Group_VM.ova - Pre-prepared teaching virtual machine. This contains the code and examples.
* https://www.virtualbox.org/wiki/Downloads - This is required if using the VM above.
* https://github.com/wales-group/examples.git - Alternatively, you can run the examples on your own machine. To get hold of the relevant files:
<pre>
git clone https://github.com/wales-group/examples.git
</pre>

== Useful Notes on Wales Group Programs and Subroutines ==
=== [[GMIN]] ===
* [[Adding a model to GMIN]] - rough outline of the subroutines that need to be changed to add a new model to GMIN
* [[Compiling Wales Group codes using cmake | Compiling GMIN using cmake ]]
* [[Selecting search parameters for GMIN]]
* [[Global optimization of biomolecules using CHARMM]]
* [[Global optimization of biomolecules using AMBER9]]
* [[Global optimization of biomolecules using AMBER9 with Structural Restraints]]
* [[Calculating binding free energy using the FSA method]]
* [[Restarting a GMIN run from a dump file]]
* [[Using the implicit membrane model IMM1]]
* [[Running a Go model with the AMHGMIN]]
* [[Running a G\=o model with the AMHGMIN]]
* [[Ligand binding-mode searches with HBONDMATRIX]]
* [[Compiling and using GMIN with QUIP]]
* [[Using GMIN and OPTIM with GPUs]]
* [[Using GMIN to generate endpoints]]
* [[Using GMIN to generate endpoints (CHARMM)]]
* [[Generating a GMIN Eclipse project]]
* [[Mutational BH steps]]
* [[Biomolecules in the energy landscape framework]]
* [[DMAGMIN setup]]
* [[Keywords]]
* [[PYGMIN & DMACRYS]]
* [[Rotamer moves in AMBER]]
* [[Python interface for GMIN/OPTIM]]

==== Scripts ====
* [[makerestart]]: A bash script to automatically set up a GMIN restart run
* [[progress]] A bash script to tell you the % completion of a GMIN job and give an estimated time remaining

==== Useful info for coding GMIN ====
* [[Program flow]] - contains information about what the various files in GMIN do and what order they're called.
* [[amberinterface]]

==== Projects ====
* [[GMIN MOVES module]]
* [[GMIN SANITY module]]
* [[GMIN TESTS module]]
* [[CAMSHIFT]]

=== [[OPTIM]] ===
* [[Adding a model to OPTIM]] - rough outline of the subrounties that need to be changed to add a new model to OPTIM
* [[Adding partially finished OPTIM stationary points to a PATHSAMPLE database]]
* [[perm-pdb.py]]: A python program that creates a ''perm.allow'' file for use with [[OPTIM]] and [[PATHSAMPLE]].
* [[visualising normal modes using VMD and OPTIM]]
* [[Compiling Wales Group codes using cmake | Compiling OPTIM using cmake ]]
* [[OPTIM/Q-Chem Tutorial]]
* [[OPTIM and PY ellipsoids tutorial]]
* [[OPTIM output files]]
* [[Minimizing a structure using OPTIM and AMBER9]]
* [[Minimizing a structure using OPTIM and CHARMM]]
* [[Creating movies (.mpg) of paths using OPTIM]]
* [[Performing a normal mode analysis of a biomolecule using OPTIM (AMBER and CHARMM)]]
* [[Debugging odd transition states in OPTIM]]
* [[Connecting two minima with a pathway]] - step by step
* [[Compiling and using OPTIM with QUIP]]
* [[Running an Gaussian03 interfaced OPTIM job]]
* [[The effect of calculating less than the maximum number of eigenvalues using ENDHESS n]]
* [[Biomolecules in the energy landscape framework]]
* [[BLJ60 example setup]]
* [[Finding an initial path with OPTIM and starting up PATHSAMPLE]]
* [[Finding an initial path with OPTIM and starting up PATHSAMPLE (CHARMM)]]
* [[Python interface for GMIN/OPTIM]]
* [[Thomson problem in OPTIM]]
* [[Instanton tunneling and classical rate calculations with OPTIM]]
* [[Loading OPTIM's min.data.info files into PATHSAMPLE]]
* [[common setup problem : No Frequency Warning]]

=== [[PATHSAMPLE]] ===
* [[Adding a model to PATHSAMPLE]] - rough outline of the subrounties that need to be changed to add a new model to PATHSAMPLE
* [[Alternatively, making the initial path with PATHSAMPLE itself]]
* [[Alternatively, making the initial path with PATHSAMPLE itself (CHARMM)]]
* [[perm-pdb.py]]: A python program that creates a ''perm.allow'' file for use with [[OPTIM]] and [[PATHSAMPLE]].
* [[dijkstra_test.py]]: A python script to test whether the information in pairlist and ts.data connects the A and B set. (If not, PATHSAMPLE will not work without actually exiting.)
* [[Compiling Wales Group codes using cmake | Compiling PATHSAMPLE using cmake ]]
* [[IMPORTANT: Using PATHSAMPLE safely on sinister]]
* [[Adding a model for PATHSAMPLE]]
* [[List of output files for PATHSAMPLE]]
* [[Using BHINTERP to find minima between two end points]]
* [[Finding an initial path between two end points (minima)]]
* [[Adding partially finished OPTIM stationary points to a PATHSAMPLE database]]
* [[Optimising a path]]
* [[Fine tuning UNTRAP]] - ensuring that it picks sensible minima
* [[Calculating rate constants (GT and fastest path)]]
* [[Calculating rate constants (SGT, DGT, and SDGT)]]
* [[Identifying the k fastest paths between endpoints using KSHORTESTPATHS]]
* [[Removing minima and transition states from the database]]
* [[Relaxing existing minima with new potential and creating new database]]
* [[Relaxing existing transition states with new potential and creating new database]]
* [[If things go wrong...]]
* [[If you lost file min.data, but still you have points.min]]
* [[path.info file is not read, causes PATHSAMPLE to die]]
* [[BLJ60 example setup]]
* [[When PATHSAMPLE finds a connected path, but using DIJKSTRA 0 fails to find the connected path]]
* [[Biomolecules in PATHSAMPLE]]
* [[Biomolecules in the energy landscape framework]]
* [[Expanding the kinetic transition network with PATHSAMPLE]]
* [[Expanding the kinetic transition network with PATHSAMPLE (CHARMM)]]
* [[Finding an initial path with OPTIM and starting up PATHSAMPLE]]
* [[Finding an initial path with OPTIM and starting up PATHSAMPLE (CHARMM)]]
* [[Pathsampling short paths]]
* [[Pathsampling short paths (CHARMM)]]
* [[Loading OPTIM's min.data.info files into PATHSAMPLE]]
* [[Connecting Sub-databases]]

=== [[Notes on MINPERMDIST | MINPERMDIST]] ===

=== [[Quasi-continuous interpolation for biomolecules | QCI]] ===

== Non-Group Software ==

=== [[AMBER]] ===
Molecular dynamics simulation program and associated force fields.
* [http://ambermd.org/ AMBER]
* [http://ambermd.org/tutorials/ AMBER tutorials] - recommended reading for '''ANYONE''' using AMBER!
* [[Notes on AMBER 12 interface]]
* [[Using AMBER 14 on the GPU and compute clusters]]
* [[Generating parameters using AMBER's built in General Forcefield (gaff)]]
* [[Generating parameters using RESP charges from GAMESS-US]]
* [[Simple scripts for LEaP to create topology and coordinate files]]
* [[Preparing an AMBER topology file for a protein system]] - step by step guide
* [[Setting up]] - step by step guide to prepare and then symmetrise a simple (protein-only) system
* [[Using Molfacture to edit molecules and add hydrogens]]
* [[Preparing an AMBER topology file for a protein plus ligand system]] - step by step guide
* [[Symmetrising AMBER topology files]] - step by step guide for symmetrising a complex protein+ligand system
* [[Producing a PDB from a coordinates and topology file]] - using ''amdpdb''
* [[Running GMIN with MD move steps AMBER]]
* [[Performing a normal mode analysis of a biomolecule using OPTIM (AMBER and CHARMM)]]
* [[Evaluating different components of AMBER energy function with SANDER]]
* [[Mutational BH steps]]
* [[REMD with AMBER]]
* [[Performing a hydrogen-bond analysis]]
* [[Alternatively, making the initial path with PATHSAMPLE itself]]
* [[Biomolecules in the energy landscape framework]]
* [[perm-prmtop.py]] - A python program that converts an AMBER9 topology file into one with a symmetrised potential with respect to exchange (updated for AMBER12 and ff14SB).
* [[Rotamer moves in AMBER]]

=== [[aux2bib]] ===
To generate a bib file containing only the entries cited in a given .tex file from a larger bib or multiple bib files.
* [https://ctan.org/pkg/bibtools Get script here]

=== [[CamCasp]] ===
Cambridge package for Calculation of Anisotropic Site Properties
From Anthony Stone's website: 'CamCASP is a collection of scripts and programs written by Dr Alston Misquitta and myself for the calculation ab initio of distributed multipoles, polarizabilities, dispersion coefficients and repulsion parameters for individual molecules, and interaction energies between pairs of molecules using SAPT(DFT).'
* [http://www-stone.ch.cam.ac.uk/programs.html CamCASP home]
* [[CamCASP/Programming]]
* [[CamCASP/Programming/5/example1]]
* [[CamCASP/Notes]]
* [[CamCASP/Bugs]]
* [[CamCASP/ToDo/diskIO]]
* [[CamCASP/ToDo/Memory]]
* [[CamCASP/CodeExamples/DirectAccess]]

=== [[CPMD]] ===
Implementation of DFT for ''ab-initio'' molecular dynamics.
* [http://www.cpmd.org/ Home Page]
* [[CPMDInput]]

=== [[CHARMM]] ===
Molecular dynamics simulation program and associated force fields.
* [https://www.charmm.org/charmm/?CFID=65f7b3aa-8037-452a-bcd1-7583dd83a087&CFTOKEN=0 CHARMM]
* [[Generating pdb, crd and psf for a peptide sequence]]
* [[Converting between '.crd' and '.pdb']]
* [[Calculating energy of a conformation]]
* [[Calculating molecular properties]]
* [[Calculating order parameters]]
* [[CAMSHIFT]]
* [[Setting up (CHARMM)]] - step by step guide to prepare and then symmetrise a simple (protein-only) system
* [[If you need to change the number of atoms (e.g. making a united-atom charmm19 .crd file, or if atoms are missing)]]
* [[Performing a normal mode analysis of a biomolecule using OPTIM (AMBER and CHARMM)]]
* [[Minimizing a structure using OPTIM and CHARMM]]
* [[Alternatively, making the initial path with PATHSAMPLE itself (CHARMM)]]
* [[Expanding the kinetic transition network with PATHSAMPLE (CHARMM)]]
* [[Finding an initial path with OPTIM and starting up PATHSAMPLE (CHARMM)]]
* [[Pathsampling short paths (CHARMM)]]

=== [[disconnectionDPS]] ===
Produces disconnectivity graphs from min.data and ts.data files. This is included in the Wales group public tarball.
* [[Constructing Free Energy Disconnectivity Graphs]]

=== [[DMACRYS]] ===
Package which models crystals of rigid molecules.
* [http://www.chem.ucl.ac.uk/cposs/dmacrys/index.html Home Page]
* [[DMACRYS interface]]
* [[DMAGMIN setup]]
* [[PYGMIN & DMACRYS]]

=== [[GAMESS]] ===
General ''ab initio'' quantum chemistry package.
* [https://www.msg.chem.iastate.edu/gamess/ GAMESS]

=== [[Gaussian]] ===
General purpose package for computational chemistry calculations.
* [[Running an Gaussian03 interfaced OPTIM job]]

=== [[gnuplot]] ===
Open source graphing program.
* [http://www.gnuplot.info/ gnuplot]
* [[Plotting a quick histogram in gnuplot using the raw data]]
* [[Plotting data in real time]]
* [[Linear and non-linear regression in gnuplot]]

=== [[GROMACS]] ===
Molecular dynamics package.
* [[Installing GROMACS on Clust]]
* [http://www.mdtutorials.com/gmx/ External tutorials]
* [http://www.gromacs.org/Documentation/Tutorials More external tutorials]

=== [[HiRE-RNA]] ===
High-res course-grained energy model for RNA.
* [https://pubs.acs.org/doi/10.1021/jp102497y Explanatory Paper]

=== [[latex2html]] ===
Script which converts latex documents into HTML pages.
* [https://www.latex2html.org/ Get script here]

=== [[MMTSB-toolset]] ===
Group of perl scripts which can be used to setup and run energy minimization, structural analysis and MD with CHARMM or AMBER.
* [http://feig.bch.msu.edu/mmtsb/Main_Page Documentation]
* [http://www.mmtsb.org/workshops/mmtsb-ctbp_2006/Tutorials/WorkshopTutorials_2006.html External tutorials]
* [[Installing and setting up the MMTSB toolset]]
* [[REX (Replica EXchange MD) with the MMTSB-toolset]]

=== [[Simulations using OPEP | OPEP]] ===
OPEP is a coarse-grained force field providing a potential for proteins and RNA.
* [http://opep.galaxy.ibpc.fr/ OPEP file generator here]
* [[Biomolecules in the energy landscape framework]]

=== [[pgprof]] ===
Profiler for portland-compiled codes
* [[Portland compiler fails trying to allocate an unexpectedly large amount of memory: issue with large arrays]]

=== [[Pymol]] ===
Molecular visualisation program.
* [https://pymol.org/2/ PyMOL]
* [https://pymolwiki.org/index.php/Main_Page PyMOL Community Wiki]
* [[loading AMBER prmtop and inpcrd files into Pymol]]
* [[producing sexy ray-traced images]]
* [[advanced colouring]]
* [[Installing python modules]]
* [[PYGMIN & DMACRYS]]
* [[path2pdb.py]] - A python program to convert ''path.info'' to ''path_all.pdb'' - you can easy visualize your path in VMD :)
* [[extractedmin2pdb.py]]: A python program to convert ''exctractedmin'' to PDB format
=== [[VASP]] ===
OPTIM has an interface to VASP, which is installed on CSD3. In collaboration with Bora Karasulu the interface has been updated to use VASP format POSCAR input files for both single- and double-ended optimisations and path searches. The OPTIM odata file requires a line like

VASP 'mpirun -ppn 16 -np 16 /home/bk393/APPS/vasp.5.4.4/with-VTST/bin/vasp_std > vasp.out'

POSCAR files can be visualised using ase, the Atomic Simulation Environment, which can be accessed on volkhan via

module load anaconda/python3/5.3.0

pip install ase --user

ase-gui POSCAR1.vasp &

which assumes that ~/.input/bin is in your $PATH environment variable.

=== [[VMD]] ===
Molecular visualisation program.
* [http://www.ks.uiuc.edu/Research/vmd/current/ug/ug.html Documentation]
* [http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/index.html External tutorials]
* [[using VMD to display and manipulate '.pdb' files]]
* [[loading coordinate files into VMD with the help of an AMBER topology file]] e.g. to visualise the results of a GMIN run using AMBER9
* [[visualising normal modes using VMD and OPTIM]]
* [[path2pdb.py]]: A python program to convert ''path.info'' to ''path_all.pdb'' - you can easy visualize your path in VMD :)
* [[path2xyz.py]]: A python program to convert ''path.info'' to ''path_all.xyz''
* [[extractedmin2pdb.py]]: A python program to convert ''exctractedmin'' to PDB format
* [[Useful .vmdrc file]]
* [[plotGMINms.tcl]]: a tcl script for plotting ellipsoids in VMD.
* [[VMD script to annotate each frame of a trajectory]]

=== [[xfig]] ===
Open source vector graphics editor
* [https://ctan.org/tex-archive/support/epstopdf/ Convert eps to pdf]

=== [[Xmakemol]] ===
Program for visualising atomic and molecular systems.
* [https://www.nongnu.org/xmakemol/ XMakemol]

=== [[xmgrace]] ===
2D plotting tool.
* [http://exciting-code.org/xmgrace-quickstart Xmgrace]

== Theoretical/Mathematical Notes ==

* [[Density of states and thermodynamics from energy distributions at different temperatures]]
* [[Ellipsoid.model]]
* [[Ellipsoid.model.xyz]]
* [[Ellipsoid.xyz]]
* [[Gencoords]]
* [[GenCoords]]
* [[GenCoords Models]]
* [[Rotamer moves in AMBER]]
* [[Thomson problem in OPTIM]]

=== Angle-axis notes ===

* [[Angle-axis framework]]
* [[Computing normal modes in angle-axis]]

=== Rigid Bodies ===

* [[Automatic Rigid Body Grouping]]
* [[Rigid body input files for proteins using genrigid-input.py]]
* [[Local Rigid Body Framework]]
* [[Local rigid body in OPTIM]]

== Useful Scripts ==
* [[perm-prmtop.py]]: A python program that converts an AMBER9 topology file into one with a symmetrised potential with respect to exchange (updated for AMBER12 and ff14SB).
* [[perm-pdb.py]]: A python program that creates a ''perm.allow'' file for use with [[OPTIM]] and [[PATHSAMPLE]].
* [[path2pdb.py]]: A python program to convert ''path.info'' to ''path_all.pdb'' - you can easy visualize your path in VMD :)
* [[path2xyz.py]]: A python program to convert ''path.info'' to ''path_all.xyz''
* [[dijkstra_test.py]]: A python script to test whether the information in pairlist and ts.data connects the A and B set. (If not, PATHSAMPLE will not work without actually exiting.)
* [[extractedmin2pdb.py]]: A python program to convert ''exctractedmin'' to PDB format
* [[colourdiscon.py]]: A python program for sorting input for disconnectivity graphs
* [[pdb_to_movie.py]]: A python program to create an AMH movieseg file from a PDB file
* [[makerestart]]: A bash script to automatically set up a GMIN restart run
* [[progress]] A bash script to tell you the % completion of a GMIN job and give an estimated time remaining
* [[recommended bash aliases]]
* [[David's .inputrc file]]
* [[Useful .vmdrc file]]
* [[Density of states and thermodynamics from energy distributions at different temperatures]]
* [[GenCoords]]: A fortran program to generate coarse grain building blocks and initial coords using a set of geometric models.
* [[plotGMINms.tcl]]: a tcl script for plotting ellipsoids in VMD.
See also the SCRIPTS/ directory in the SVN repository!
* [[Computing CHARMM FF energy using GMIN, MMTSB and CHARMM]] - Computes the Charmm FF energy of the same structure. Useful for cross-validating force field settings in GMIN data file, CHARMM input file and MMTSB options.
* [[Automatic Rigid Body Grouping]]
* [[ElaborateDiff]]
* [[Parameter-scanning script]]
* [[Pdb to movie.py]]
* [[VMD script to annotate each frame of a trajectory]]

== Useful links ==
* [http://www.ch.cam.ac.uk/computing/theory-compute-clusters The Theory Compute Clusters support page]. Contains useful cluster specific information, including example job submission scripts.

* A useful website which contains AMBER (GAFF) and OPLS parameters for small molecules. http://virtualchemistry.org/gmld.php . This could save us lot of time while trying to derive parameters on our own. If you are lucky, the molecule of your interest may already be there in the existing database. The topology files are in GROMACS format but possibly can be converted into AMBER parameter files. (script anyone ?)

* The moving-domain QM/MM method developed by Victor Batista's group http://gascon.chem.uconn.edu/software. This approach can be used in the derivation of charges for large proteins and nucleic acids, where a full-fledged ONIOM based calculation is comptutationally prohibitive. It has been applied to systems like the Gramicidin ion channel and Photosystem II.

== Miscellaneous ==
* [[Animated GIF on the group website]]
* [[Backup strategy]]
* [[Chain crossing]]
* [[Computer Office services]]
* [[Computing values only once]]
* [[Decoding heat capacity curves]]
* [[Differences from Clust]]
* [[Fixing thunderbird links]]
* [[If you need to change the number of atoms (e.g. making a united-atom charmm19 .crd file, or if atoms are missing)]]
* [[Intel Trace Analyzer and Collector]]
* [[LDAP plans]]
* [[Lapack compilation]]
* [[Mek-quake Queueing system]]
* [[Mek-quake initial setup notes]]
* [[New mek-quake]]
* [[Maui compilation]]
* [[Torque and Maui]]
* [[Mercurial]]
* [[Migrating to the new SVN server]]
* [[NECI Parallelization]]
* [[Optimization tricks]]
* [[Other IT stuff]]
* [[Porfuncs Documentation]]
* [[Progress]]
* [[Proposed changes to backup and archiving]]
* [[Rama upgrade]]
* [[Remastering Knoppix]]
* [[See unpacked nodes]]
* [[Tardis scheduling policy]]
* [[Zippo Sicortex machine]]

== Useful linux stuff ==

===Basics===
* [[basic linux commands everyone should know!]]
* [[piping and redirecting output from one command or file to another]] - how to save yourself hours!
* [[bash loop tricks]]
* [[bash history searching]]

===Remote access===
* [[setting up aliases to quickly log you in to a different machine]]
* [[transfering files to and from your workstation]] -using ''scp'' or ''rsync''
* [[using 'ssh-keygen' to automatically log you into clusters from your workstation]] (no more typing in your password!)
* [[mounting sharedscratch locally]]

===Find and replace===
* [[short 'sed' examples]]
* [[quick guide to awk]]
* [[short 'awk' examples]]

===File manipulation===
* [[sorting a file by multiple columns]]
* [[using tar and gzip to compress/uncompress files | using tar and bzip2 to compress/uncompress files]]
* [[conversion between different data file formats]] -'almost one-line' scripts
* [[conversion between different image file formats]] - the ''convert'' command
* [[removing an excessive number of files from a directory - when 'rm' just isn't enough]]

===Cluster queues===
* [[submitting jobs, interactively or to a cluster queue system]]
* [[identifying job on a node]] - if you need to kill only one of few running jobs
* [[a guide to using SLURM to run PATHSAMPLE]]
* [[a guide to using SLURM to run GPU jobs on pat]]

===Miscellaneous/uncategorised===
* [[installing packages on your managed CUC3 workstation]]
* [[running programs in the background]] - so you can use your shell for other things at the same time
* [[finding bugs in latex documents that will not compile]]
* [[printing files from the command line using 'lpr']]
* [[uploading non image files to the wiki]]

== Compiler Flags ==

* [[Compiler Flags]]
* [[Blacklisting Compilers]]
* [[Lapack compilation]]
* [[Pdb to movie.py]]
* [[Portland compiler fails trying to allocate an unexpectedly large amount of memory: issue with large arrays]]

== SuSE ==

* [[Upgrading destiny]]
* [[Upgrading sword]]
* [[SuSE 10.1 workstation image]]
* [[SuSE 10.2 workstation image]]
* [[SuSE 10.3 workstation image]]
* [[SuSE 11.1]]

--[[User:adk44|adk44]] 17.00, 9 May 2019 (BST)

Wales Group Version control

2020-01-06T10:42:49Z

Dw34:

As of 16th July 2008, all the group code is under version control. [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]], along with the [[CHARMM]] and [[AMBER]] code used with them have been added to the repository, along with the official documentation. Changes to the documentation in the repository are automatically applied to the version on the website every night (at midnight) and the RSS feeds below are updated every ten minutes. The next step is to set up the nightly compilation and test suite for each code.

There are instructions on how to set up your SVN details and obtain the group code on the [[SVN setup]] page.

==RSS feeds==

RSS feeds (including diffs of files that have been changed) are generated every 10 minutes from the SVN log files any uploaded to the Wales group web server [http://www-wales.ch.cam.ac.uk/rss/ here]. There is currently one feed per top level directory in the repository:

* [http://www-wales.ch.cam.ac.uk/rss/trunk_log.xml trunk_log.xml] - contains the logs/diffs for every change to the code
* [http://www-wales.ch.cam.ac.uk/rss/gmin_log.xml gmin_log.xml] - only contains the logs/diffs when code in the GMIN directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/pathsample_log.xml pathsample_log.xml] - only contains the logs/diffs when code in the PATHSAMPLE directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/optim_log.xml optim_log.xml] - only contains the logs/diffs when code in the OPTIM directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/charmm_log.xml charmm_log.xml] - only contains the logs/diffs when code in the CHARMM31 directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/amber_log.xml amber_log.xml] - only contains the logs/diffs when code in the AMBER directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/nab_log.xml nab_log.xml] - only contains the logs/diffs when code in the NAB directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/disconnect_log.xml disconnect_log.xml] - only contains the logs/diffs when code in the DISCONNECT directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/doc_log.xml doc_log.xml] - only contains the logs/diffs when code in the DOC directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/scripts_log.xml scripts_log.xml] - only contains the logs/diffs when code in the SCRIPTS directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/ambertools_log.xml ambertools_log.xml] - only contains the logs/diffs when code in the AMBERTOOLS directory is changed

Using Firefox to look at the feeds is ok, but I suggest you set up a different RSS reader, such as [http://www.google.com/reader Google Reader] to keep them all in one place. Note that Google Reader has a delay in updating the feeds, so you won't see new logs up to an hour after they are created and are visible with Firefox.

==Access restrictions==
Raven authentication has been re-enabled for the diffs only. You can view the feeds without logging in (as this allows email clients to be used), but if you are outside the department, you will need to log in via Raven to access the diffs linked from the feed. Also, the diffs directory is now excluded from Google and other bot searches. This should hopefully prevent chunks of code ending up on Google.

==Usage tips==
What follows are a few suggestions to make using SVN a joyful experience. Please add any others you think are important! In general you should:

* please be careful with svn commands. svn is a very powerful tool, and it is possible to revert all changes made over a long period, which is probably not what you want. Please consult before you revert changes made by other people.
* make sure you followed the setup instructions on the [[SVN setup]] page. The procedure includes setting up a template file to ensure that all logs are in the same basic format
* run 'svn update' on a regular basis to ensure you have the latest bug fixes. It is also important to update regularly to detect any bugs that may have appeared due to changes elsewhere in the code. Please report any changes in behaviour that might indicate bugs.
* '''always''' do an 'svn update' before you do 'svn commit'
* don't forget 'svn add <filename>' to schedule a new file for addition at the next commit!
* get into the habit of running 'svn diff' before committing. And looking closely at the output... Optionally, run 'svn diff | grep Index' to see a summary list of the files that you have changed. Try not to commit extra things you didn't mean to.
* commit your changes regularly. If you wait a two months before commit your changes, you're likely to have to resolve more conflicts in parts of the code that has been changed by others in the meantime.
* if you get a conflict (C flag) when running 'svn update', find out who introduced the change and talk to them about it to make sure you can introduce your code without damaging theirs.
* consider making a development branch (more on this on the [[SVN setup]] page) if your hacking is going to involve a lot of potentially disruptive changes over a significant period of time. Talk to DJW and current developers about this first though...

* to obtain a previous version of one file. In the svn directory where that file lives use svn -r with the revision number, the file name, and pipe the output to a target. For example, to obtain revision 36240 of commons.f90 use
svn -r 36240 cat commons.f90 > commons.f90.36240
* to see a log of svn commits use svn log in the appropriate directory. The log contains the changes entered by the user who made the commit. Please fill in relevant fields for the log when you make changes.

==Papers==
We have a repository for preparing papers too: see [[Papers_in_preparation]] for information.

Wales Group Version control

2020-01-03T10:35:07Z

Dw34: /* Usage tips */

As of 16th July 2008, all the group code is under version control. [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]], along with the [[CHARMM]] and [[AMBER]] code used with them have been added to the repository, along with the official documentation. Changes to the documentation in the repository are automatically applied to the version on the website every night (at midnight) and the RSS feeds below are updated every ten minutes. The next step is to set up the nightly compilation and test suite for each code.

There are instructions on how to set up your SVN details and obtain the group code on the [[SVN setup]] page.

==RSS feeds==

RSS feeds (including diffs of files that have been changed) are generated every 10 minutes from the SVN log files any uploaded to the Wales group web server [http://www-wales.ch.cam.ac.uk/rss/ here]. There is currently one feed per top level directory in the repository:

* [http://www-wales.ch.cam.ac.uk/rss/trunk_log.xml trunk_log.xml] - contains the logs/diffs for every change to the code
* [http://www-wales.ch.cam.ac.uk/rss/gmin_log.xml gmin_log.xml] - only contains the logs/diffs when code in the GMIN directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/pathsample_log.xml pathsample_log.xml] - only contains the logs/diffs when code in the PATHSAMPLE directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/optim_log.xml optim_log.xml] - only contains the logs/diffs when code in the OPTIM directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/charmm_log.xml charmm_log.xml] - only contains the logs/diffs when code in the CHARMM31 directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/amber_log.xml amber_log.xml] - only contains the logs/diffs when code in the AMBER directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/nab_log.xml nab_log.xml] - only contains the logs/diffs when code in the NAB directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/disconnect_log.xml disconnect_log.xml] - only contains the logs/diffs when code in the DISCONNECT directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/doc_log.xml doc_log.xml] - only contains the logs/diffs when code in the DOC directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/scripts_log.xml scripts_log.xml] - only contains the logs/diffs when code in the SCRIPTS directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/ambertools_log.xml ambertools_log.xml] - only contains the logs/diffs when code in the AMBERTOOLS directory is changed

Using Firefox to look at the feeds is ok, but I suggest you set up a different RSS reader, such as [http://www.google.com/reader Google Reader] to keep them all in one place. Note that Google Reader has a delay in updating the feeds, so you won't see new logs up to an hour after they are created and are visible with Firefox.

==Access restrictions==
Raven authentication has been re-enabled for the diffs only. You can view the feeds without logging in (as this allows email clients to be used), but if you are outside the department, you will need to log in via Raven to access the diffs linked from the feed. Also, the diffs directory is now excluded from Google and other bot searches. This should hopefully prevent chunks of code ending up on Google.

==Usage tips==
What follows are a few suggestions to make using SVN a joyful experience. Please add any others you think are important! In general you should:

* please be careful with svn commands. svn is a very powerful tool, and it is possible to revert all changes made over a long period, which is probably not what you want
* make sure you followed the setup instructions on the [[SVN setup]] page. The procedure includes setting up a template file to ensure that all logs are in the same basic format
* run 'svn update' on a regular basis to ensure you have the latest bug fixes. It is also important to update regularly to detect any bugs that may have appeared due to changes elsewhere in the code. Please report any changes in behaviour that might indicate bugs.
* '''always''' do an 'svn update' before you do 'svn commit'
* don't forget 'svn add <filename>' to schedule a new file for addition at the next commit!
* get into the habit of running 'svn diff' before committing. And looking closely at the output... Optionally, run 'svn diff | grep Index' to see a summary list of the files that you have changed. Try not to commit extra things you didn't mean to.
* commit your changes regularly. If you wait a two months before commit your changes, you're likely to have to resolve more conflicts in parts of the code that has been changed by others in the meantime.
* if you get a conflict (C flag) when running 'svn update', find out who introduced the change and talk to them about it to make sure you can introduce your code without damaging theirs.
* consider making a development branch (more on this on the [[SVN setup]] page) if your hacking is going to involve a lot of potentially disruptive changes over a significant period of time. Talk to DJW and current developers about this first though...

* to obtain a previous version of one file. In the svn directory where that file lives use svn -r with the revision number, the file name, and pipe the output to a target. For example, to obtain revision 36240 of commons.f90 use
svn -r 36240 cat commons.f90 > commons.f90.36240
* to see a log of svn commits use svn log in the appropriate directory. The log contains the changes entered by the user who made the commit. Please fill in relevant fields for the log when you make changes.

==Papers==
We have a repository for preparing papers too: see [[Papers_in_preparation]] for information.

Wales Group Version control

2020-01-03T10:32:54Z

Dw34: /* Usage tips */

As of 16th July 2008, all the group code is under version control. [[GMIN]], [[OPTIM]] and [[PATHSAMPLE]], along with the [[CHARMM]] and [[AMBER]] code used with them have been added to the repository, along with the official documentation. Changes to the documentation in the repository are automatically applied to the version on the website every night (at midnight) and the RSS feeds below are updated every ten minutes. The next step is to set up the nightly compilation and test suite for each code.

There are instructions on how to set up your SVN details and obtain the group code on the [[SVN setup]] page.

==RSS feeds==

RSS feeds (including diffs of files that have been changed) are generated every 10 minutes from the SVN log files any uploaded to the Wales group web server [http://www-wales.ch.cam.ac.uk/rss/ here]. There is currently one feed per top level directory in the repository:

* [http://www-wales.ch.cam.ac.uk/rss/trunk_log.xml trunk_log.xml] - contains the logs/diffs for every change to the code
* [http://www-wales.ch.cam.ac.uk/rss/gmin_log.xml gmin_log.xml] - only contains the logs/diffs when code in the GMIN directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/pathsample_log.xml pathsample_log.xml] - only contains the logs/diffs when code in the PATHSAMPLE directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/optim_log.xml optim_log.xml] - only contains the logs/diffs when code in the OPTIM directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/charmm_log.xml charmm_log.xml] - only contains the logs/diffs when code in the CHARMM31 directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/amber_log.xml amber_log.xml] - only contains the logs/diffs when code in the AMBER directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/nab_log.xml nab_log.xml] - only contains the logs/diffs when code in the NAB directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/disconnect_log.xml disconnect_log.xml] - only contains the logs/diffs when code in the DISCONNECT directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/doc_log.xml doc_log.xml] - only contains the logs/diffs when code in the DOC directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/scripts_log.xml scripts_log.xml] - only contains the logs/diffs when code in the SCRIPTS directory is changed
* [http://www-wales.ch.cam.ac.uk/rss/ambertools_log.xml ambertools_log.xml] - only contains the logs/diffs when code in the AMBERTOOLS directory is changed

Using Firefox to look at the feeds is ok, but I suggest you set up a different RSS reader, such as [http://www.google.com/reader Google Reader] to keep them all in one place. Note that Google Reader has a delay in updating the feeds, so you won't see new logs up to an hour after they are created and are visible with Firefox.

==Access restrictions==
Raven authentication has been re-enabled for the diffs only. You can view the feeds without logging in (as this allows email clients to be used), but if you are outside the department, you will need to log in via Raven to access the diffs linked from the feed. Also, the diffs directory is now excluded from Google and other bot searches. This should hopefully prevent chunks of code ending up on Google.

==Usage tips==
What follows are a few suggestions to make using SVN a joyful experience. Please add any others you think are important! In general you should:

* please be careful with svn commands. svn is a very powerful tool, and it is possible to revert all changes made over a long period, which is probably not what you want
* make sure you followed the setup instructions on the [[SVN setup]] page. The procedure includes setting up a template file to ensure that all logs are in the same basic format
* run 'svn update' on a regular basis to ensure you have the latest bug fixes
* '''always''' do an 'svn update' before you do 'svn commit'
* don't forget 'svn add <filename>' to schedule a new file for addition at the next commit!
* get into the habit of running 'svn diff' before committing. And looking closely at the output... Optionally, run 'svn diff | grep Index' to see a summary list of the files that you have changed. Try not to commit extra things you didn't mean to.
* commit your changes regularly. If you wait a two months before commit your changes, you're likely to have to resolve more conflicts in parts of the code that has been changed by others in the meantime.
* if you get a conflict (C flag) when running 'svn update', find out who introduced the change and talk to them about it to make sure you can introduce your code without damaging theirs.
* consider making a development branch (more on this on the [[SVN setup]] page) if your hacking is going to involve a lot of potentially disruptive changes over a significant period of time. Talk to DJW and current developers about this first though...

* to obtain a previous version of one file. In the svn directory where that file lives use svn -r with the revision number, the file name, and pipe the output to a target. For example, to obtain revision 36240 of commons.f90 use
svn -r 36240 cat commons.f90 > commons.f90.36240
* to see a log of svn commits use svn log in the appropriate directory. The log contains the changes entered by the user who made the commit. Please fill in relevant fields for the log when you make changes.

==Papers==
We have a repository for preparing papers too: see [[Papers_in_preparation]] for information.

Common setup problem : No Frequency Warning

2019-07-18T15:17:41Z

Dw34:

If you use NOFRQS in the PATHDATA file then, make sure to use NOFRQS in the ODATA.CONNECT file.

For example if the PATHDATA file looks like following:

<pre>
SLURM
CYCLES 100
ADDPATH path.info.first
TEMPERATURE 0.592
PLANCK 9.536D-14
EXEC /home/ab2480/svn/OPTIM/builds/AMBER/A12OPTIM
NATOMS 68
COPYFILES perm.allow min.in coords.prmtop coords.inpcrd
COPYOPTIM
DIRECTION BA
CONNECTREGION 1 2 200
CONNECTIONS 1
ITOL 10.0D0
NOFRQS
GEOMDIFFTOL 0.3D0
EDIFFTOL 1.0D-4
PERMDIST
AMBER12
</pre>
ODATA.CONNECT file should be :
<pre>
REOPTIMISEENDPOINTS
UPDATES 20 4 20 10 4
NEWCONNECT 20 3 10.0 40.0 50 2.0 0.001
NEWNEB 40 400 0.1
NEBK 10.0
NOCISTRANS
DIJKSTRA EXP
DUMPALLPATHS
CHECKCHIRALITY
EDIFFTOL 1.0D-4
MAXERISE 1.0D-4 1.0D-2
GEOMDIFFTOL 0.1D0
BFGSTS 1000 20 200 0.01 100
BFGSMIN 1.0D-6
NOHESS
NOFRQS
PERMDIST
MAXSTEP 0.2
TRAD 0.5
MAXMAX 1.0
BFGSCONV 1.0D-6
PUSHOPT 0.2 0.001 100
STEPS 1000
BFGSSTEPS 60000
MAXBFGS 0.2
AMBER12 start
</pre>
Note that it is '''essential''' to have AMBER12 start at the end. PATHSAMPLE creates start.<pid> and finish.<pid> files, which will be copied to start and finish when OPTIM runs locally.

Connecting Sub-databases

2019-05-17T10:42:19Z

Dw34: /* Executive Summary */

== Definitions ==

For the purposes of this tutorial, I am defining sub-databases to be sets of connected minima and transition states within a larger database.

== Context and Motivation ==

In databases containing many thousands of minima and TSs, it is unlikely that these will all be connected to one another. This is particularly the case when the database has been grown using such methods as '''ADDPATH''' and '''MERGEDB'''. Instead, the database is more likely to consist of many sub-databases of varying size. Therefore, when constructing a disconnectivity graph, which cannot plot more than one set of connected minima (i.e. more than one sub-database) at a time, a lot of data present in the min.data, points.min, points.ts and ts.data files is ignored. The sub-database that the disconnectivity graph plots depends on the numerical argument to the keyword '''CONNECTMIN''' in the dinfo file. These numerical arguments correspond to minima, as listed in the min.data file. For example, an argument of 12 corresponds to line 12 of the min.data file. Therefore, only this minimum plus any others it is connected to, are plotted in the disconnectivity graph.

The question, therefore, is how to efficiently connect minima already present in the min.data file. It would be particularly important to connect sub-databases with a lot of minima in them (it would probably be a waste of time to connect all those sub-databases with only 2 minima in them, for example, as by doing so you’re not collecting much more information).

Another consideration is that we want the connection attempts between sub-databases to be efficient. We want to try to connect sub-databases that are closer to one another (or, more specifically, sub-databases which have at least one minimum which is close in chemical space to a minimum in another sub-database). This consideration is especially important for large systems (such as large proteins with cofactors) as trying to connect minima far apart in space can be very slow or even break down due to memory issues.

=== Systems for which this approach might be particularly useful ===

This methodology might be particularly useful for cases where you have a protein with a cofactor and various sites within a pocket that you think the cofactor can attach to. It provides an efficient method to connect these sites within the pocket, having already sampled each.

== Step 1: Using disconnectionDPS to determine the breakdown of sub-databases within your database ==

=== Requirements ===

A folder containing the files min.data, points.min, points.ts, ts.data, dinfo, the script '''find_connections.sh''' (to be found in the svn at '''~svn/SCRIPTS/DISCONNECT''') and the binary [[disconnectionDPS]], plus any other auxiliary files you may need.

=== Method ===

In dinfo, you need to use the keyword '''PRINTCONNECTED'''. An example dinfo file that I've used is:

<pre>
! REQUIRED KEYWORDS

DELTA 0.25
FIRST -15120.0
LEVELS 800
MINIMA min.data
TS ts.data

! OPTIONAL KEYWORDS

NCONNMIN 0
CONNECTMIN 1
LABELFORMAT F8.1
PRINTCONNECTED
</pre>

'''PRINTCONNECTED''' ensures that a file called '''connected''' is written, which lists all of the minima plotted in the disconnectivity graph (i.e. all of the minima present in the sub-database considered). In the example above, because the argument to '''CONNECTMIN''' is 1, this means that minimum 1, and all those minima to which 1 is connected, gets plotted.

This gives us information on only one sub-database present in the file. To find out about all of them, the script '''find_connections.sh''' is used.

This script cycles through the min.data file, executing a [[disconnectionDPS]] command for every iteration of the argument to '''CONNECTMIN'''. The '''connected''' file produced is then renamed '''connected_*''' where * is the argument to '''CONNECTMIN''' when that [[disconnectionDPS]] command was executed. For a min.data file with 17603 lines (and therefore 17603 minima) for example, the argument to '''CONNECTMIN''' therefore ranges from CONNECTMIN 1 to CONNECTMIN 17603. If a minimum is already present in a previous '''connected_*''' file then that argument is skipped. For example, if a [[disconnectionDPS]] execution when the argument to '''CONNECTMIN''' was set to '''CONNECTMIN 1''' gave a sub-database with minima 1 and 2 (i.e. the minima on lines 1 and 2 in min.data) in it, then a [[disconnectionDPS]] attempt using '''CONNECTMIN 2''' will not be attempted as minimum 2 is already assigned to the sub-database described in '''connected_1'''. The next iteration will be using '''CONNECTMIN 3'''.

This script cycles until all the minima in min.data have been considered.

Another feature of '''find_connections.sh''' is that, when '''connected_*''' files exceed a set number of minima (I think 10 is sensible) then they get copied to a corresponding '''relevant_connected_*''' file, eg if '''connected_3''' has 50 minima then it exceeds 10 and so the information in this file is copied to another one called '''relevant_connected_3'''. This is a piece of book-keeping which allows the user to identify more easily larger sub-databases (and so ones that s/he is more likely to want to connect to one another).

A few notes on use: to use this script, it is sensible to copy the min.data, points.min, points.ts and ts.data files of the database you are interested in to another folder. The only other files you need are the script itself, the relevant binary and dinfo (plus perhaps some case-specific auxiliary files). It should be ensured that before executing the binary, the argument to '''CONNECTMIN''' in dinfo is 1. Also, '''PRINTCONNECTED''' must be included as a keyword.

== Step 2: '''RETAINSP''' and '''CONNECTUNC LOWESTTEST''' ==

So, we now have a list of files '''relevant_connected_*''' corresponding to sub-databases which we would like to connect.

Remember, though, we want to connect them efficiently!

Before attempting any connections then, it is probably advised to get a flavour of the distances separating these sub-databases from one another (or, at least, the shortest distance possible between any two minima of all of the sub-databases).

To do this, we need to limit the min.data file (and ts.data) in a sub-folder so that only those minima corresponding to the two sub-databases we are interested in are considered. We can use the keyword found in [[PATHSAMPLE]], '''RETAINSP''', for this purpose. By using an adapted version of '''CONNECTUNC''' with a new argument called '''LOWESTTEST''', we can identify sensible connections to make, without actually attempting the connection.

This approach works as long as min.A and min.B both correspond to minima in the AB set (this is accounted for in the script I’ve written, '''connect_sub_databases.sh'''). What this does is find the unconnected minima (i.e. those in the set which is not the AB set) of lowest energy. It then loops through all the minima in the AB set, printing the distance between each pair of minima without actually attempting the connection. A further loop operates so that all unconnected minima are considered too.

Once all minima are considered, the loop is abruptly exited by a STOP statement.

Using grep (don't worry about executing these commands yourself as they are all contained in the script '''connect_sub_databases.sh'''):

<pre>
grep "connectlowest> Distance: " pathsample_connectunc_test.out > distances
sed -e "s/^/$dirname /g" distances > distances_tmp
</pre>

we are able to build up a list of all the proposed connections made by '''CONNECTLOWESTTEST''' between the two chosen sub-databases. This information then gets concatenated into an overall file called distances_tot in the folder where the script was originally launched. Eventually, once all pairs of sub-databases are considered, we should have a massive file listing all of the potential connections between all of the minima in all of the sub-databases, along with the distances separating them. An example of a few lines from such a file is as follows:

<pre>
00003_00303 359 4550 connectlowest> Distance: 42.42963734
00003_00303 341 147 connectlowest> Distance: 39.39663225
00003_02150 2280 1932 connectlowest> Distance: 75.54181654
</pre>

The first column lists the two sub-databases which were considered. Another nice feature of the '''CONNECTUNC LOWESTTEST''' keyword and argument is that, alongside the distance, the specific minima (DMIN1 and DMIN2) from the two sub-databases being considered are listed (highlighted in red below):

00003_00303 359 4550 connectlowest> Distance: 42.42963734

359, therefore, is a minimum which belongs to sub-database 00003 (i.e. the sub-database described by the file '''relevant_connected_00003''') and 4550 a minimum which belongs to sub-database 00303.

== Step 3: Organising Calculations to Attempt ==

Clearly, a pair of minima separated by 39.397 is a more feasible calculation to make than one separated by 42.430 or 75.542. The script we have ('''connect_sub_databases''') therefore reorganises distances_tot to list the proposed connections between pairs from shortest distance to longest. This new reorganised file we give the rather unimaginative name of lowest_to_highest_distances_tot.

The rest of the script is concerned with connecting all of the sub-databases in as efficient a way, using as few steps, as possible. This is probably best illustrated by an example:

I have 15 sub-databases I wish to connect. The minima comprising each can be found in:

<pre>
relevant_connected_00003
relevant_connected_00164
relevant_connected_00303
relevant_connected_02150
relevant_connected_06061
relevant_connected_06274
relevant_connected_06610
relevant_connected_06913
relevant_connected_07339
relevant_connected_09000
relevant_connected_09969
relevant_connected_10040
relevant_connected_12405
relevant_connected_14191
relevant_connected_14775
</pre>

Here are the first ten lines of lowest_to_highest_distances_tot. Those coloured green are connections attempted, whilst those coloured red were skipped over because they turn out to be superfluous (why attempt line 5, for example, when line 4 is attempting to connect the same two sub-databases?):

00003_02150 2657 2663 connectlowest> Distance: 0.39352080 
09000_12405 3003 3033 connectlowest> Distance: 0.84958725 
09000_10040 1228 1251 connectlowest> Distance: 1.01130262 
09000_14191 3176 3209 connectlowest> Distance: 1.07183817 
09000_14191 3194 3209 connectlowest> Distance: 1.81036433 
09000_14191 3193 3187 connectlowest> Distance: 1.88481550 
09000_14775 3450 3457 connectlowest> Distance: 2.41249957 
09000_14191 3203 3187 connectlowest> Distance: 2.42913148 
09000_14191 3177 3209 connectlowest> Distance: 2.45715932 
00003_02150 2572 2663 connectlowest> Distance: 2.82747537 

Using these principles, the sub-databases were therefore connected as follows.

The first line of lowest_to_highest_distances_tot:

00003_02150 2657 2663 connectlowest> Distance: 0.39352080

[[Image:first step.png|230px|center]]

After next line:

09000_12405 3003 3033 connectlowest> Distance: 0.84958725

[[Image:second step.png|250px|center]]

09000_10040 1228 1251 connectlowest> Distance: 1.01130262

[[Image:third step.png|400px|center]]

09000_14191 3176 3209 connectlowest> Distance: 1.07183817

[[Image:fourth step.png|400px|center]]

09000_14775 3450 3457 connectlowest> Distance: 2.41249957

[[Image:fifth step.png|420px|center]]

00003_00164 134 159 connectlowest> Distance: 5.00815137

[[Image:sixth step.png|420px|center]]

06061_06913 402 296 connectlowest> Distance: 5.01723232

[[Image:seventh step.png|420px|center]]

00003_07339 3893 3899 connectlowest> Distance: 5.68186344

[[Image:eighth step.png|450px|center]]

06610_07339 135 137 connectlowest> Distance: 7.04874883

[[Image:ninth step.png|450px|center]]

00003_00303 670 811 connectlowest> Distance: 24.67395896

[[Image:tenth step.png|450px|center]]

06061_06274 257 459 connectlowest> Distance: 31.59473639

[[Image:eleventh step.png|450px|center]]

09000_09969 1317 1946 connectlowest> Distance: 40.39286979

[[Image:twelfth step.png|450px|center]]

00003_06061 4149 4481 connectlowest> Distance: 44.36489142

[[Image:thirteenth step.png|450px|center]]

00003_09000 7369 2632 connectlowest> Distance: 71.98718590

[[Image:fourteenth step.png|700px|center]]

Connection attempts are therefore chosen using as small distances as possible. It should also be possible to connect these 15 sub-databases using 14 connections. Some [[OPTIM]] runs for whatever reason may not work (although this is unlikely) in which case the user can manually look through the lowest_to_highest_distances_tot file to find the next most suitable pair of minima from the two sub-databases in question to connect.

The diagrams above list 14 connections which are recommended to be made. What the script therefore does is to create 14 folders for these connections to be attempted. These are named as 00003_02150, 09000_12405 etc.

Within each sub-folder, a [[PATHSAMPLE]] calculation using '''RETAINSP''' is first of all performed. This ensures that only the minima pertaining to the two sub-databases we are trying to connect are included (eg 00003 and 02150 in the case of the sub-folder 00003_02150) and that the numbering scheme in min.data is consistent with when '''CONNECTUNC LOWESTTEST''' was used. Once this is done, the pathdata file is altered so that rather than '''RETAINSP''' we now wish to use the keywords '''CONNECTPAIRS connectfile''' and '''CYCLES 1'''.

What '''CONNECTPAIRS''' does is to chose specific minima from min.data for connection attempts. This requires an argument which specifies a file to be read in order to list the minima we want to connect. Typically, this argument is '''connectfile''', and therefore we require a file called '''connectfile'''. This file lists all of the minima we wish to connect according to their line number in min.data. As shown above (and copied immediately below) for the connection we wish to make between the sub-databases 00003 and 02150, the minima we are interested in are 2657 and 2663.

00003_02150 2657 2663 connectlowest> Distance: 0.39352080

Therefore, '''connectfile''', upon opening, should appear simply as:

2657 2663

Thus, an OPTIM job gets launched, which tries to connect minima 2657 (in the 00003 sub-database) and 2663 (in the 02150 sub-database). Because these minima are separated by a really short distance of 0.394, it will hopefully (this is not foolproof!) be a simple connection.

In this example, following the use of the script, we should end up with 14 folders, each with a connected path between two sub-databases.

== How to execute steps 2 and 3? ==

Steps 2 and 3 above clearly involves a number of steps. A script has been written which does all of this (i.e. chooses which connections to make and then attempts them) for you. This script is called '''connect_sub_databases.sh''' and can be found in the svn at '''~/svn/SCRIPTS/PATHSAMPLE/connecting_sub_databases'''. An annotated version of this script is also available.

The following files are required in the folder from which you launch this script (assuming you are using the [[AMBER]] interface):

coords.inpcrd, coord.mdcrd, coords.prmtop, min.in, min.A, min.B, min.data, odata.connect, pathdata, perm.allow, points.min, points.ts, ts.data, untrap_sub_script, relevant_connected_*

Where relevant_connected_* are all of the sub-databases found in step 1. This could be any number of files. Note that untrap_sub_script just happens to be the name of the sub-script I used for my calculations. You can either rename your sub-script to untrap_sub_script or alter '''connect_sub_databases.sh''' before launching your calculations. It is essential that your pathdata file is correctly formatted before launching the script too. Two lines

RETAINSP
! CYCLES 1

must be included. An example of a pathdata file I used is as follows:

EXEC /home/adk44/bin/CUDAOPTIM_ppt_final_210918
CPUS 1
NATOMS 5430
SEED 1
DIRECTION AB
CONNECTIONS 1
TEMPERATURE 0.592
PLANCK 9.536D-14

PERMDIST
ETOL 8D-4
GEOMDIFFTOL 0.2D0
ITOL 0.1D0
NOINVERSION
NOFRQS

RETAINSP
! CYCLES 1

AMBER12

Make sure that this pathdata points towards a valid binary and that the number of atoms is consistent with the system you are examining.

== Step 4: Merging the sub-databases ==

Following the use of the script, we should end up with x number of folders, each with a connected path between two sub-databases. The total number of folders will encompass the total number of connections which needed to be made in order to connect all of the sub-databases we were interested in.

Assuming all of the connections were successfully made, all we need to do now is to merge these new connected databases together. This can be achieved using the '''MERGEDB''' keyword in [[PATHSAMPLE]], as is outlined [http://www-wales.ch.cam.ac.uk/PATHSAMPLE.2.1.doc/node5.html here].

== Summary for those who don't like screeds of writing ==

It is possible that your [[PATHSAMPLE]] database contains many sub-databases not necessarily connected to one another. Therefore, a lot of information is lost when you try to construct disconnectivity graphs.

To retrieve this information, we should therefore connect these sub-databases. This connection can be done efficiently by first identifying which minima contained in respective sub-databases are closest to each other, and then trying to connect these first.

To first identify the sub-databases present in your database, launch the script '''find_connections.sh''' using '''disconnectionDPS''' with the '''PRINTCONNECTED''' keyword included and the argument to '''CONNECTMIN''' set to '''1'''. This produces a list of '''relevant_connected_*''' files, each one representing a sub-database which lists the minima present in it.

Following this step, the script '''connect_sub_databases.sh''' is used. It determines the distances between all of the possible minima in all of the possible sub-databases. It then prioritises calculations in order to connect the maximum number of sub-databases in the minimum possible number of steps, with those of shortest distance being attempted first.

A number of folders are created, each providing a connected path between two sub-databases. The data from these folders can be merged into the original using the [[PATHSAMPLE]] keyword '''MERGEDB''', thus connecting all of the previously unconnected sub-databases.

This methodology is particularly useful for cases where you have a protein with a cofactor and various sites within a pocket that you think the cofactor can attach to. It provides an efficient method to connect these sites within the pocket, having already sampled each.

--adk44 14.30, 16 May 2019 (BST)

Connecting Sub-databases

2019-05-17T10:40:57Z

Dw34:

== Definitions ==

For the purposes of this tutorial, I am defining sub-databases to be sets of connected minima and transition states within a larger database.

== Context and Motivation ==

In databases containing many thousands of minima and TSs, it is unlikely that these will all be connected to one another. This is particularly the case when the database has been grown using such methods as '''ADDPATH''' and '''MERGEDB'''. Instead, the database is more likely to consist of many sub-databases of varying size. Therefore, when constructing a disconnectivity graph, which cannot plot more than one set of connected minima (i.e. more than one sub-database) at a time, a lot of data present in the min.data, points.min, points.ts and ts.data files is ignored. The sub-database that the disconnectivity graph plots depends on the numerical argument to the keyword '''CONNECTMIN''' in the dinfo file. These numerical arguments correspond to minima, as listed in the min.data file. For example, an argument of 12 corresponds to line 12 of the min.data file. Therefore, only this minimum plus any others it is connected to, are plotted in the disconnectivity graph.

The question, therefore, is how to efficiently connect minima already present in the min.data file. It would be particularly important to connect sub-databases with a lot of minima in them (it would probably be a waste of time to connect all those sub-databases with only 2 minima in them, for example, as by doing so you’re not collecting much more information).

Another consideration is that we want the connection attempts between sub-databases to be efficient. We want to try to connect sub-databases that are closer to one another (or, more specifically, sub-databases which have at least one minimum which is close in chemical space to a minimum in another sub-database). This consideration is especially important for large systems (such as large proteins with cofactors) as trying to connect minima far apart in space can be very slow or even break down due to memory issues.

=== Systems for which this approach might be particularly useful ===

This methodology might be particularly useful for cases where you have a protein with a cofactor and various sites within a pocket that you think the cofactor can attach to. It provides an efficient method to connect these sites within the pocket, having already sampled each.

== Step 1: Using disconnectionDPS to determine the breakdown of sub-databases within your database ==

=== Requirements ===

A folder containing the files min.data, points.min, points.ts, ts.data, dinfo, the script '''find_connections.sh''' (to be found in the svn at '''~svn/SCRIPTS/DISCONNECT''') and the binary [[disconnectionDPS]], plus any other auxiliary files you may need.

=== Method ===

In dinfo, you need to use the keyword '''PRINTCONNECTED'''. An example dinfo file that I've used is:

<pre>
! REQUIRED KEYWORDS

DELTA 0.25
FIRST -15120.0
LEVELS 800
MINIMA min.data
TS ts.data

! OPTIONAL KEYWORDS

NCONNMIN 0
CONNECTMIN 1
LABELFORMAT F8.1
PRINTCONNECTED
</pre>

'''PRINTCONNECTED''' ensures that a file called '''connected''' is written, which lists all of the minima plotted in the disconnectivity graph (i.e. all of the minima present in the sub-database considered). In the example above, because the argument to '''CONNECTMIN''' is 1, this means that minimum 1, and all those minima to which 1 is connected, gets plotted.

This gives us information on only one sub-database present in the file. To find out about all of them, the script '''find_connections.sh''' is used.

This script cycles through the min.data file, executing a [[disconnectionDPS]] command for every iteration of the argument to '''CONNECTMIN'''. The '''connected''' file produced is then renamed '''connected_*''' where * is the argument to '''CONNECTMIN''' when that [[disconnectionDPS]] command was executed. For a min.data file with 17603 lines (and therefore 17603 minima) for example, the argument to '''CONNECTMIN''' therefore ranges from CONNECTMIN 1 to CONNECTMIN 17603. If a minimum is already present in a previous '''connected_*''' file then that argument is skipped. For example, if a [[disconnectionDPS]] execution when the argument to '''CONNECTMIN''' was set to '''CONNECTMIN 1''' gave a sub-database with minima 1 and 2 (i.e. the minima on lines 1 and 2 in min.data) in it, then a [[disconnectionDPS]] attempt using '''CONNECTMIN 2''' will not be attempted as minimum 2 is already assigned to the sub-database described in '''connected_1'''. The next iteration will be using '''CONNECTMIN 3'''.

This script cycles until all the minima in min.data have been considered.

Another feature of '''find_connections.sh''' is that, when '''connected_*''' files exceed a set number of minima (I think 10 is sensible) then they get copied to a corresponding '''relevant_connected_*''' file, eg if '''connected_3''' has 50 minima then it exceeds 10 and so the information in this file is copied to another one called '''relevant_connected_3'''. This is a piece of book-keeping which allows the user to identify more easily larger sub-databases (and so ones that s/he is more likely to want to connect to one another).

A few notes on use: to use this script, it is sensible to copy the min.data, points.min, points.ts and ts.data files of the database you are interested in to another folder. The only other files you need are the script itself, the relevant binary and dinfo (plus perhaps some case-specific auxiliary files). It should be ensured that before executing the binary, the argument to '''CONNECTMIN''' in dinfo is 1. Also, '''PRINTCONNECTED''' must be included as a keyword.

== Step 2: '''RETAINSP''' and '''CONNECTUNC LOWESTTEST''' ==

So, we now have a list of files '''relevant_connected_*''' corresponding to sub-databases which we would like to connect.

Remember, though, we want to connect them efficiently!

Before attempting any connections then, it is probably advised to get a flavour of the distances separating these sub-databases from one another (or, at least, the shortest distance possible between any two minima of all of the sub-databases).

To do this, we need to limit the min.data file (and ts.data) in a sub-folder so that only those minima corresponding to the two sub-databases we are interested in are considered. We can use the keyword found in [[PATHSAMPLE]], '''RETAINSP''', for this purpose. By using an adapted version of '''CONNECTUNC''' with a new argument called '''LOWESTTEST''', we can identify sensible connections to make, without actually attempting the connection.

This approach works as long as min.A and min.B both correspond to minima in the AB set (this is accounted for in the script I’ve written, '''connect_sub_databases.sh'''). What this does is find the unconnected minima (i.e. those in the set which is not the AB set) of lowest energy. It then loops through all the minima in the AB set, printing the distance between each pair of minima without actually attempting the connection. A further loop operates so that all unconnected minima are considered too.

Once all minima are considered, the loop is abruptly exited by a STOP statement.

Using grep (don't worry about executing these commands yourself as they are all contained in the script '''connect_sub_databases.sh'''):

<pre>
grep "connectlowest> Distance: " pathsample_connectunc_test.out > distances
sed -e "s/^/$dirname /g" distances > distances_tmp
</pre>

we are able to build up a list of all the proposed connections made by '''CONNECTLOWESTTEST''' between the two chosen sub-databases. This information then gets concatenated into an overall file called distances_tot in the folder where the script was originally launched. Eventually, once all pairs of sub-databases are considered, we should have a massive file listing all of the potential connections between all of the minima in all of the sub-databases, along with the distances separating them. An example of a few lines from such a file is as follows:

<pre>
00003_00303 359 4550 connectlowest> Distance: 42.42963734
00003_00303 341 147 connectlowest> Distance: 39.39663225
00003_02150 2280 1932 connectlowest> Distance: 75.54181654
</pre>

The first column lists the two sub-databases which were considered. Another nice feature of the '''CONNECTUNC LOWESTTEST''' keyword and argument is that, alongside the distance, the specific minima (DMIN1 and DMIN2) from the two sub-databases being considered are listed (highlighted in red below):

00003_00303 359 4550 connectlowest> Distance: 42.42963734

359, therefore, is a minimum which belongs to sub-database 00003 (i.e. the sub-database described by the file '''relevant_connected_00003''') and 4550 a minimum which belongs to sub-database 00303.

== Step 3: Organising Calculations to Attempt ==

Clearly, a pair of minima separated by 39.397 is a more feasible calculation to make than one separated by 42.430 or 75.542. The script we have ('''connect_sub_databases''') therefore reorganises distances_tot to list the proposed connections between pairs from shortest distance to longest. This new reorganised file we give the rather unimaginative name of lowest_to_highest_distances_tot.

The rest of the script is concerned with connecting all of the sub-databases in as efficient a way, using as few steps, as possible. This is probably best illustrated by an example:

I have 15 sub-databases I wish to connect. The minima comprising each can be found in:

<pre>
relevant_connected_00003
relevant_connected_00164
relevant_connected_00303
relevant_connected_02150
relevant_connected_06061
relevant_connected_06274
relevant_connected_06610
relevant_connected_06913
relevant_connected_07339
relevant_connected_09000
relevant_connected_09969
relevant_connected_10040
relevant_connected_12405
relevant_connected_14191
relevant_connected_14775
</pre>

Here are the first ten lines of lowest_to_highest_distances_tot. Those coloured green are connections attempted, whilst those coloured red were skipped over because they turn out to be superfluous (why attempt line 5, for example, when line 4 is attempting to connect the same two sub-databases?):

00003_02150 2657 2663 connectlowest> Distance: 0.39352080 
09000_12405 3003 3033 connectlowest> Distance: 0.84958725 
09000_10040 1228 1251 connectlowest> Distance: 1.01130262 
09000_14191 3176 3209 connectlowest> Distance: 1.07183817 
09000_14191 3194 3209 connectlowest> Distance: 1.81036433 
09000_14191 3193 3187 connectlowest> Distance: 1.88481550 
09000_14775 3450 3457 connectlowest> Distance: 2.41249957 
09000_14191 3203 3187 connectlowest> Distance: 2.42913148 
09000_14191 3177 3209 connectlowest> Distance: 2.45715932 
00003_02150 2572 2663 connectlowest> Distance: 2.82747537 

Using these principles, the sub-databases were therefore connected as follows.

The first line of lowest_to_highest_distances_tot:

00003_02150 2657 2663 connectlowest> Distance: 0.39352080

[[Image:first step.png|230px|center]]

After next line:

09000_12405 3003 3033 connectlowest> Distance: 0.84958725

[[Image:second step.png|250px|center]]

09000_10040 1228 1251 connectlowest> Distance: 1.01130262

[[Image:third step.png|400px|center]]

09000_14191 3176 3209 connectlowest> Distance: 1.07183817

[[Image:fourth step.png|400px|center]]

09000_14775 3450 3457 connectlowest> Distance: 2.41249957

[[Image:fifth step.png|420px|center]]

00003_00164 134 159 connectlowest> Distance: 5.00815137

[[Image:sixth step.png|420px|center]]

06061_06913 402 296 connectlowest> Distance: 5.01723232

[[Image:seventh step.png|420px|center]]

00003_07339 3893 3899 connectlowest> Distance: 5.68186344

[[Image:eighth step.png|450px|center]]

06610_07339 135 137 connectlowest> Distance: 7.04874883

[[Image:ninth step.png|450px|center]]

00003_00303 670 811 connectlowest> Distance: 24.67395896

[[Image:tenth step.png|450px|center]]

06061_06274 257 459 connectlowest> Distance: 31.59473639

[[Image:eleventh step.png|450px|center]]

09000_09969 1317 1946 connectlowest> Distance: 40.39286979

[[Image:twelfth step.png|450px|center]]

00003_06061 4149 4481 connectlowest> Distance: 44.36489142

[[Image:thirteenth step.png|450px|center]]

00003_09000 7369 2632 connectlowest> Distance: 71.98718590

[[Image:fourteenth step.png|700px|center]]

Connection attempts are therefore chosen using as small distances as possible. It should also be possible to connect these 15 sub-databases using 14 connections. Some [[OPTIM]] runs for whatever reason may not work (although this is unlikely) in which case the user can manually look through the lowest_to_highest_distances_tot file to find the next most suitable pair of minima from the two sub-databases in question to connect.

The diagrams above list 14 connections which are recommended to be made. What the script therefore does is to create 14 folders for these connections to be attempted. These are named as 00003_02150, 09000_12405 etc.

Within each sub-folder, a [[PATHSAMPLE]] calculation using '''RETAINSP''' is first of all performed. This ensures that only the minima pertaining to the two sub-databases we are trying to connect are included (eg 00003 and 02150 in the case of the sub-folder 00003_02150) and that the numbering scheme in min.data is consistent with when '''CONNECTUNC LOWESTTEST''' was used. Once this is done, the pathdata file is altered so that rather than '''RETAINSP''' we now wish to use the keywords '''CONNECTPAIRS connectfile''' and '''CYCLES 1'''.

What '''CONNECTPAIRS''' does is to chose specific minima from min.data for connection attempts. This requires an argument which specifies a file to be read in order to list the minima we want to connect. Typically, this argument is '''connectfile''', and therefore we require a file called '''connectfile'''. This file lists all of the minima we wish to connect according to their line number in min.data. As shown above (and copied immediately below) for the connection we wish to make between the sub-databases 00003 and 02150, the minima we are interested in are 2657 and 2663.

00003_02150 2657 2663 connectlowest> Distance: 0.39352080

Therefore, '''connectfile''', upon opening, should appear simply as:

2657 2663

Thus, an OPTIM job gets launched, which tries to connect minima 2657 (in the 00003 sub-database) and 2663 (in the 02150 sub-database). Because these minima are separated by a really short distance of 0.394, it will hopefully (this is not foolproof!) be a simple connection.

In this example, following the use of the script, we should end up with 14 folders, each with a connected path between two sub-databases.

== How to execute steps 2 and 3? ==

Steps 2 and 3 above clearly involves a number of steps. A script has been written which does all of this (i.e. chooses which connections to make and then attempts them) for you. This script is called '''connect_sub_databases.sh''' and can be found in the svn at '''~/svn/SCRIPTS/PATHSAMPLE/connecting_sub_databases'''. An annotated version of this script is also available.

The following files are required in the folder from which you launch this script (assuming you are using the [[AMBER]] interface):

coords.inpcrd, coord.mdcrd, coords.prmtop, min.in, min.A, min.B, min.data, odata.connect, pathdata, perm.allow, points.min, points.ts, ts.data, untrap_sub_script, relevant_connected_*

Where relevant_connected_* are all of the sub-databases found in step 1. This could be any number of files. Note that untrap_sub_script just happens to be the name of the sub-script I used for my calculations. You can either rename your sub-script to untrap_sub_script or alter '''connect_sub_databases.sh''' before launching your calculations. It is essential that your pathdata file is correctly formatted before launching the script too. Two lines

RETAINSP
! CYCLES 1

must be included. An example of a pathdata file I used is as follows:

EXEC /home/adk44/bin/CUDAOPTIM_ppt_final_210918
CPUS 1
NATOMS 5430
SEED 1
DIRECTION AB
CONNECTIONS 1
TEMPERATURE 0.592
PLANCK 9.536D-14

PERMDIST
ETOL 8D-4
GEOMDIFFTOL 0.2D0
ITOL 0.1D0
NOINVERSION
NOFRQS

RETAINSP
! CYCLES 1

AMBER12

Make sure that this pathdata points towards a valid binary and that the number of atoms is consistent with the system you are examining.

== Step 4: Merging the sub-databases ==

Following the use of the script, we should end up with x number of folders, each with a connected path between two sub-databases. The total number of folders will encompass the total number of connections which needed to be made in order to connect all of the sub-databases we were interested in.

Assuming all of the connections were successfully made, all we need to do now is to merge these new connected databases together. This can be achieved using the '''MERGEDB''' keyword in [[PATHSAMPLE]], as is outlined [http://www-wales.ch.cam.ac.uk/PATHSAMPLE.2.1.doc/node5.html here].

== Summary for those who don't like screeds of writing ==

It is possible that your [[PATHSAMPLE]] database contains many sub-databases not necessarily connected to one another. Therefore, a lot of information is lost when you try to construct disconnectivity graphs.

To retrieve this information, we should therefore connect these sub-databases. This can be done efficiently by first identifying which minima contained in respective sub-databases are closest to each other, and then trying to connect these first.

To at first identify the sub-databases present in your database, launch the script '''find_connections.sh''' using '''disconnectionDPS''' with the '''PRINTCONNECTED''' keyword included and the argument to '''CONNECTMIN''' set to '''1'''. This produces a list of '''relevant_connected_*''' files, each one representing a sub-database which lists the minima present in it.

Following this step, the script '''connect_sub_databases.sh''' is launched. It determines the distances between all of the possible minima in all of the possible sub-databases. It then prioritises calculations in order to connect the maximum number of sub-databases in the minimum possible number of steps, with those of shortest distance being attempted first.

The outcome to the launch of this latter script is that a number of folders are created, each providing a connected path between two sub-databases. The data from these folders can be merged into the original using the [[PATHSAMPLE]] keyword '''MERGEDB''', thus connecting all of the previously unconnected sub-databases.

This methodology is particularly useful for cases where you have a protein with a cofactor and various sites within a pocket that you think the cofactor can attach to. It provides an efficient method to connect these sites within the pocket, having already sampled each.

--adk44 14.30, 16 May 2019 (BST)

Main Page

2019-05-10T15:54:19Z

Dw34:

<big>'''Welcome to the Wales group software wiki'''</big>

For info on compiling our code from the source tarball, see the [[Compiling Wales Group code using CMake | cmake]] page.

== Comprehensive Contents Page ==
Please click [[Comprehensive Contents Page | here]] for a comprehensive, organised list of all of the pages comprising this wiki + some other useful links.

= Group Software =
All of our software is freely available under the [http://www.gnu.org/licenses/gpl.html GPL]. However, there are cases when we interface to commercial codes such as [http://ambermd.org/ AMBER] and [http://www.charmm.org/ CHARMM]. These files are absent from the source tarball. If you do have a license, please contact Professor Wales for access to a full version. We work on three separate programs:

*[[GMIN]]: A program for finding global minima and calculating thermodynamic properties from basin-sampling.
GMIN employs the basin-hopping algorithm described by Wales and Doye (''J. Phys. Chem. A, 101, 5111, 1997''[http://pubs.acs.org/doi/abs/10.1021/jp970984n]) to locate global minima on a potential energy surface. Many potentials are included. The latest version also includes an implementation of basin-sampling as described in T.V. Bogdan, D.J. Wales and F. Calvo (''J. Chem. Phys., 124, 044102, 2006''[http://www-wales.ch.cam.ac.uk/pdf/JCP.124.044102.2006.pdf]).

*[[OPTIM]]: A program for optimizing geometries and calculating reaction pathways
The geometry optimization scheme in OPTIM is based on eigenvector-following and was originally built from the optimizer in the ACES package written by Prof. John F. Stanton. OPTIM has analytic first and second derivatives coded for dozens of empirical potentials, and can also treat systems involving periodic boundary conditions and solve general optimization problems such as least squares fits.

*[[PATHSAMPLE]]: A driver for OPTIM to create stationary point databases using discrete path sampling and perform kinetic analysis.

= Helpful Software =
We have developed many scripts within the group to use in conjunction with our software - all of which are provided here. We also use other programs which are linked below:

*[[DisconnectionDPS]]

= Contact details =

If you have something to add to this wiki, or would like to contribute code, please get in touch with Professor Wales.

Main Page

2019-05-09T13:08:35Z

Dw34:

<big>'''Welcome to the Wales group software wiki!'''</big>

For info on compiling our code from the source tarball, see the [[Compiling Wales Group code using CMake | cmake]] page.

= Group Software =
All of our software is freely available under the [http://www.gnu.org/licenses/gpl.html GPL]. However, there are cases when we interface to commercial codes such as [http://ambermd.org/ AMBER] and [http://www.charmm.org/ CHARMM]. These files are absent from the source tarball. If you do have a license, please contact Professor Wales for access to a full version. We work on three separate programs:

*[[GMIN]]: A program for finding global minima and calculating thermodynamic properties from basin-sampling.
GMIN employs the basin-hopping algorithm described by Wales and Doye (''J. Phys. Chem. A, 101, 5111, 1997''[http://pubs.acs.org/doi/abs/10.1021/jp970984n]) to locate global minima on a potential energy surface. Many potentials are included. The latest version also includes an implementation of basin-sampling as described in T.V. Bogdan, D.J. Wales and F. Calvo (''J. Chem. Phys., 124, 044102, 2006''[http://www-wales.ch.cam.ac.uk/pdf/JCP.124.044102.2006.pdf]).

*[[OPTIM]]: A program for optimizing geometries and calculating reaction pathways
The geometry optimization scheme in OPTIM is based on eigenvector-following and was originally built from the optimizer in the ACES package written by Prof. John F. Stanton. OPTIM has analytic first and second derivatives coded for dozens of empirical potentials, and can also treat systems involving periodic boundary conditions and solve general optimization problems such as least squares fits.

*[[PATHSAMPLE]]: A driver for OPTIM to create stationary point databases using discrete path sampling and perform kinetic analysis.

= Helpful Software =
We have developed many scripts within the group to use in conjunction with our software - all of which are provided here. We also use other programs which are linked below:

*[[DisconnectionDPS]]

= Contact details =

If you have something to add to this wiki, or would like to contribute code, please get in touch with Professor Wales.