NECI-testcode

This is a new page to post info on how to migrate the test code between machines, as was done between discovery and scepter recently. I will be adding to this as and when I have spare time. --js615 12:52, 12 October 2009 (BST)

Setting up the testcode

1. Check out the components

This script written by JSS sets the correct permissions for the CLEAN directory. The 2775 sets the directory to have rwx permissions to the user and group and rx permissions for everyone else. The setgid bit is set so that anything created within the CLEAN directory inherits the group ownership of the CLEAN directory rather than the default group of the user (which is normally a group containing just that user). Please set umask 2 in your ~/.bashrc before changing files in the CLEAN directory so that the group retains write permissions on files modified by you within the CLEAN directory.

 #!/bin/bash -x
 umask 0002
 SVN=https://wwmm.ch.cam.ac.uk/svn2/groups/alavi
 mkdir CLEAN
 chgrp alavi CLEAN
 chmod 2775 CLEAN
 cd CLEAN
 svn co $SVN/NECI/trunk NECI
 svn co $SVN/CPMD/branches/QMC/trunk CPMD
 git clone scepter:~ajwt3/NECI.git NECI

Currently, svn and git are supported by the testcode.

CPMD then needs to be told where NECI is:

 js615@scepter:~/CLEAN/CPMD> cat .compileconf 
 NECIsrc=/home/js615/CLEAN/NECI

The testcode also needs to be told the locations of and version control systems used for CPMD and NECI. Do this by copying userconfig-template and setting the obvious variables to your personal settings. The automated compilation and testing script, checktest.py, needs to know the location of the test suite. This is defined in main_testcode/tools/checktest_config file residing in the test suite (the testcode directory).

testcode options can also be specified within checktest_config on a compiler-level basis. One of the more useful options is to specify how many cores the parallel tests should use. Note that new benchmarks need to be produced if the number of cores changes as certain FCIMC options are given on a per-core basis and so changing the number of cores used actually changes the calculation. See the testcode suite documentation for more details of the possible options.

Add the pseudopotential library to your ~/.bashrc as well.

 export PP_LIBRARY_PATH=~/CLEAN/CPMD/PP_LIBRARY

2. Compiling the libraries

Certain compilers require certain libraries to be available when compiling CPMD and NECI. Which libraries are required can be investigated in the file checktest_config.

 js615@scepter:~/CLEAN/testcode/main_testcode/tools> cat checktest_config 

This contains blocks like this.

 [PC-PGI]
 modules = pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
 job_types = neci
 job_categories = NECI

Although not all versions are used, it is convenient to compile lapack and fftw across a range of compilers so that they are accessible to the test code. JSS has written a set of scripts to do this. Extract scepter:~jss43/setup_libraries.tar.gz and run setup_libraries.sh. This installs libraries to ~/local/lib: you can adjust this location in the scripts if you want. It also sets up private environment modules for your own use.

It's important to note that the current version of fftw (3.2.2) doesn't compile with pathscale. (setup_libraries.tar.gz includes fftw 3.2 for this reason.)

See Lapack compilation for more information on compiling lapack.

3. Setting up the libraries as modules

The command

 module load use.own

allows use of local libraries via environment modules in the folder ~/privatemodules. See Environment modules for more information. The test code requires each library to be set up in this manner.

The procedure to set up a module is as follows:

 js615@scepter:~/privatemodules> mkdir -p fftw3.2/64
 js615@scepter:~/privatemodules> vi fftw3.2/64/pathscale

And then use the following input:

 #%Module1.0#####################################################################
 ##
 ## 
 ##
 set root /home/js615/local/lib/fftw-3.2
 set version 3.2.2
 module-whatis "Loads version $version of the fftw library, compiled in 64-bit."
 prepend-path LIBRARY_PATH $root/lib
 prepend-path LD_LIBRARY_PATH $root/lib
 prepend-path MANPATH $root/manpages/man/

Change the root to reflect where you've actually got this version of fftw.

The following statement should be added to the end of modules compiled with pathscale:

 if { [ llength [ array get env PSC_GENFLAGS ] ] } {
       set mypscflags  "-L $root/lib $env(PSC_GENFLAGS)"
 } else {
       set mypscflags "-L $root/lib"
 }
 setenv PSC_GENFLAGS $mypscflags

(JSS: note that this causes serious issues if multiple pathscale libraries are being used. Avoid.)

Now the same module files need to be made for lapack. These require a slightly different header:

 js615@scepter:~/privatemodules/lapack3.1.1/32> cat ifort
 #%Module1.0#####################################################################
 ##
 ## 
 ##
 set root /home/js615/local/lib/lapack-3.1.1/32/ifort
 prepend-path LIBRARY_PATH $root/lib
 prepend-path LIBRARY_PATH $root/include
 prepend-path LD_LIBRARY_PATH $root/lib
 prepend-path LD_LIBRARY_PATH $root/include
 prepend-path MANPATH $root/manpages/man/

Note that in the current set-up there is slightly more information in these files, including (proc ModulesHelp and module-whatis). These have been ommitted here for clarity. People who are curious can go to the directory above and 'cat' the current file.

There is a more elegant solution to creating multiple modules for different compilers, using .base files (as done in setup_libraries---see http://www-theor.ch.cam.ac.uk/IT/software/modules.html). Furthermore, there is a better way to set PSC_GENFLAGS for pathscale. The build above is sufficient for the test code. To see these designs, go to:

 scepter:/home/jss43/privatemodules

and also look at Environment modules.

4. Link the libraries to the test code.

The test code has a file called checktest_config which tells it which modules to load for each compiler. This is currently at:

 js615@scepter:~/CLEAN/testcode/main_testcode/tools/checktest_config 

These need to match the directory structure of your privatemodules set-up. To test this, take a block of checktest_config:

 [PC-PGI]
 modules = pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
 job_types = neci
 job_categories = NECI

and try to load everything on the "modules =" line (a "module purge" might be needed to avoid conflicts). E.g.

 js615@scepter:~/CLEAN/testcode/main_testcode/tools> module load pgi/32/8.0/5 use.own fftw3.2.2/32/pgi

The test code should now work.

5. Setting up cron (for nightly testing)

Use the command:

 crontab -e

from anywhere to set up cron. The following setup is used at the moment for this file:

 PP_LIBRARY_PATH=/home/js615/CLEAN/CPMD/PP_LIBRARY
 30   01  * * *  umask 2 && /home/js615/CLEAN/testcode/main_testcode/tools/checktest.py 

Re-stating the pseudopotential library path and umask 2 are required since cron doesn't source your ~/.bashrc. The following line runs checktest.py with the default settings at 1.30am.

6. Setting permissions

There are two ways of getting the correct permissions, but the neater version written by JSS only works under certain non-default conditions (using ACLs, as are present on discovery).

 #!/bin/bash
 for file in `find . -type f`; do
       chmod 0000 $file
       chm d ug=rw $file
       i=`file $file | grep script | wc -l`
       if [ $i = "1" ]; then
               chmod 777 $file
       fi
       if [ -d $file ]; then
               chmod 2775 $file
       fi
 done