Difference between revisions of "NECI-testcode"

From CUC3
Jump to navigation Jump to search
import>Js615
import>Jss43
Line 1: Line 1:
−
This is a new page to post info on how to migrate the test code between machines, as was done between discovery and scepter recently. I will be adding to this as and when I have spare time. --[[User:js615|js615]] 12:52, 12 October 2009 (BST)
 +
This is a new page to post info on how to migrate the test code between
  +
machines, as was done between discovery and scepter recently. I will be adding
  +
to this as and when I have spare time. --[[User:js615|js615]] 12:52, 12 October 2009 (BST)
   
 
==Setting up the testcode==
  
==Setting up the testcode==
Line 5: Line 7:
 
1. Check out the components
  
1. Check out the components
   
−
This script written by JSS sets the correct permissions for the CLEAN directory, including the "2" sticky. To take advantage of these permissions, all group members need to add umask 2 to their .bashrc files.
 +
This script written by JSS sets the correct permissions for the CLEAN
  +
directory. The 2775 sets the directory to have rwx permissions to the user and
  +
group and rx permissions for everyone else. The setgid bit is set so that
  +
anything created within the CLEAN directory inherits the group ownership of the
  +
CLEAN directory rather than the default group of the user (which is normally a
  +
group containing just that user). Please set umask 2 in your ~/.bashrc before
  +
changing files in the CLEAN directory so that the group retains write
  +
permissions on files modified by you within the CLEAN directory.
   
 
#!/bin/bash -x
  
#!/bin/bash -x
Line 15: Line 24:
 
svn co $SVN/NECI/trunk NECI
  
svn co $SVN/NECI/trunk NECI
 
svn co $SVN/CPMD/branches/QMC/trunk CPMD
  
svn co $SVN/CPMD/branches/QMC/trunk CPMD
  +
git clone scepter:~ajwt3/NECI.git NECI
−
svn co $SVN/testcode testcode
  
   
−
Although this example uses svn, other version control software can be used. Currently, svn and git are supported by the testcode.
 +
Currently, svn and git are supported by the testcode.
   
 
CPMD then needs to be told where NECI is:
  
CPMD then needs to be told where NECI is:
Line 24: Line 33:
 
NECIsrc=/home/js615/CLEAN/NECI
  
NECIsrc=/home/js615/CLEAN/NECI
   
−
The testcode also needs to be told where NECI and CPMD are, and what version control they are under.
+
The testcode also needs to be told the locations of and version control
  +
systems used for CPMD and NECI. Do this by copying userconfig-template and
−
 
  +
setting the obvious variables to your personal settings. The automated
−
js615@scepter:~/CLEAN/testcode> cp userconfig-template userconfig
  
  +
compilation and testing script, checktest.py, needs to know the location of the
  +
test suite. This is defined in main_testcode/tools/checktest_config file
  +
residing in the test suite (the testcode directory).
  +
  +
testcode options can also be specified within checktest_config on a
  +
compiler-level basis. One of the more useful options is to specify how many
  +
cores the parallel tests should use. Note that new benchmarks need to be
  +
produced if the number of cores changes as certain FCIMC options are given on a
  +
per-core basis and so changing the number of cores used actually changes the
  +
calculation. See the testcode suite documentation for more details of the
  +
possible options.
   
  ⚫
Add the pseudopotential library to your ~/.bashrc as well.
−
Now edit userconfig to your personal settings. Also, the testcode needs to know where the testcode is. This is defined in the [user] block of:
  
   
  ⚫
export PP_LIBRARY_PATH=~/CLEAN/CPMD/PP_LIBRARY
−
js615@scepter:~/CLEAN/testcode/main_testcode/tools/checktest_config
 
−  
−
In the same file, specify how many cores the tests should use. Parallel jobs will need to be re-benchmarked if this changes, as in the migration from discovery to scepter.
  
−  
⚫
Add the pseudopotential library to your .bashrc as well.
  
−  
⚫
PP_LIBRARY_PATH=~/CLEAN/CPMD/PP_LIBRARY
  
−
export PP_LIBRARY_PATH
  
   
 
2. Compiling the libraries
  
2. Compiling the libraries
   
−
Certain compilers require certain libraries to be available when compiling CPMD and NECI. Which libraries are required can be investigated in the file checktest_config.
 +
Certain compilers require certain libraries to be available when compiling CPMD
  +
and NECI. Which libraries are required can be investigated in the file
  +
checktest_config.
   
 
js615@scepter:~/CLEAN/testcode/main_testcode/tools> cat checktest_config
 
js615@scepter:~/CLEAN/testcode/main_testcode/tools> cat checktest_config
Line 52: Line 67:
 
job_categories = NECI
  
job_categories = NECI
   
−
Although not all versions are used, it is convenient to compile lapack and fftw across a range of compilers so that they are accessible to the test code.
 +
Although not all versions are used, it is convenient to compile lapack and fftw
  +
across a range of compilers so that they are accessible to the test code. JSS
  +
has written a set of scripts to do this. Extract
  +
scepter:~jss43/setup_libraries.tar.gz and run setup_libraries.sh. This installs
  +
libraries to ~/local/lib: you can adjust this location in the scripts if you want.
  +
It also sets up private environment modules for your own use.
   
  ⚫
It's important to note that the current version of fftw (3.2.2) doesn't compile
−
Download the latest fftw and lapack into ~/local/lib. There are then four scripts (written by JSS) used to compile these with different compilers. Make edits to the versions as appropriate.
  
  +
with pathscale. (setup_libraries.tar.gz includes fftw 3.2 for this reason.)
   
  ⚫
See [[Lapack compilation]] for more information on compiling lapack.
−
[[setup_lapack.sh]]
  
−  
−
[[setup_fftw.sh]]
  
−  
−
[[compile_fftw.sh]]
  
−  
−
[[compile_lapack.sh]]
  
−  
⚫
It's important to note that the current version of fftw (3.2.2) doesn't compile with pathscale.
  
−  
−
Also, to use lapack as a library, different files need to be present in the lapack directories:
  
−  
−
ln -s lapack_LINUX.a liblapack.a
  
−  
⚫
See [[Lapack compilation]] for more information.
  
   
 
3. Setting up the libraries as modules
  
3. Setting up the libraries as modules
  +
−  
 
The command
  
The command
   
Line 108: Line 115:
 
}
  
}
 
setenv PSC_GENFLAGS $mypscflags
  
setenv PSC_GENFLAGS $mypscflags
  +
  +
(JSS: note that this causes serious issues if multiple pathscale libraries are being used. Avoid.)
   
 
Now the same module files need to be made for lapack. These require a slightly different header:
  
Now the same module files need to be made for lapack. These require a slightly different header:
Line 123: Line 132:
 
prepend-path MANPATH $root/manpages/man/
  
prepend-path MANPATH $root/manpages/man/
   
−
Note that in the current set-up there is slightly more information in these files, including (proc ModulesHelp and module-whatis). These have been ommitted here for clarity. People who are curious can go to the directory above and 'cat' the current file.
 +
Note that in the current set-up there is slightly more information in these
  +
files, including (proc ModulesHelp and module-whatis). These have been ommitted
  +
here for clarity. People who are curious can go to the directory above and
  +
'cat' the current file.
   
−
There is a more elegant solution to creating multiple modules for different compilers, using .base files. Furthermore, there is a better way to set PSC_GENFLAGS for pathscale. The build above is sufficient for the test code. To see these designs, go to:
 +
There is a more elegant solution to creating multiple modules for different
  +
compilers, using .base files (as done in setup_libraries---see http://www-theor.ch.cam.ac.uk/IT/software/modules.html). Furthermore, there is a better way to set
  +
PSC_GENFLAGS for pathscale. The build above is sufficient for the test code. To
  +
see these designs, go to:
   
 
scepter:/home/jss43/privatemodules
  
scepter:/home/jss43/privatemodules
Line 132: Line 147:
   
 
4. Link the libraries to the test code.
  
4. Link the libraries to the test code.
  +
−  
−
The test code has a file called checktest_config which tells it which modules to load for each compiler. This is currently at:
 +
The test code has a file called checktest_config which tells it which modules
  +
to load for each compiler. This is currently at:
   
 
js615@scepter:~/CLEAN/testcode/main_testcode/tools/checktest_config
 
js615@scepter:~/CLEAN/testcode/main_testcode/tools/checktest_config
   
−
These need to match the directory structure of your privatemodules set-up. To test this, take a block of checktest_config:
 +
These need to match the directory structure of your privatemodules set-up. To
  +
test this, take a block of checktest_config:
   
 
[PC-PGI]
  
[PC-PGI]
Line 144: Line 161:
 
job_categories = NECI
  
job_categories = NECI
   
−
and try to load everything on the "modules =" line (a "module purge" might be needed to avoid conflicts). E.g.
 +
and try to load everything on the "modules =" line (a "module purge" might be
  +
needed to avoid conflicts). E.g.
   
 
js615@scepter:~/CLEAN/testcode/main_testcode/tools> module load pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
  
js615@scepter:~/CLEAN/testcode/main_testcode/tools> module load pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
Line 156: Line 174:
 
crontab -e
  
crontab -e
   
−
from anywhere to set up cron. The following setup is used at the moment for this file:
 +
from anywhere to set up cron. The following setup is used at the moment for
  +
this file:
 
 
 
PP_LIBRARY_PATH=/home/js615/CLEAN/CPMD/PP_LIBRARY
  
PP_LIBRARY_PATH=/home/js615/CLEAN/CPMD/PP_LIBRARY
 
30 01 * * * umask 2 && /home/js615/CLEAN/testcode/main_testcode/tools/checktest.py
 
30 01 * * * umask 2 && /home/js615/CLEAN/testcode/main_testcode/tools/checktest.py
   
−
Re-stating the pseudopotential library path and umask 2 are required since cron doesn't source your .bashrc. The following line runs checktest.py with the default settings at 1.30am.
 +
Re-stating the pseudopotential library path and umask 2 are required since cron
  +
doesn't source your ~/.bashrc. The following line runs checktest.py with the
  +
default settings at 1.30am.

Revision as of 13:47, 13 October 2009

This is a new page to post info on how to migrate the test code between machines, as was done between discovery and scepter recently. I will be adding to this as and when I have spare time. --js615 12:52, 12 October 2009 (BST)

Setting up the testcode

1. Check out the components

This script written by JSS sets the correct permissions for the CLEAN directory. The 2775 sets the directory to have rwx permissions to the user and group and rx permissions for everyone else. The setgid bit is set so that anything created within the CLEAN directory inherits the group ownership of the CLEAN directory rather than the default group of the user (which is normally a group containing just that user). Please set umask 2 in your ~/.bashrc before changing files in the CLEAN directory so that the group retains write permissions on files modified by you within the CLEAN directory. 

 #!/bin/bash -x
 SVN=https://wwmm.ch.cam.ac.uk/svn2/groups/alavi
 mkdir CLEAN
 chgrp alavi CLEAN
 chmod 2775 CLEAN
 cd CLEAN
 svn co $SVN/NECI/trunk NECI
 svn co $SVN/CPMD/branches/QMC/trunk CPMD
 git clone scepter:~ajwt3/NECI.git NECI

Currently, svn and git are supported by the testcode.

CPMD then needs to be told where NECI is: 

 js615@scepter:~/CLEAN/CPMD> cat .compileconf 
 NECIsrc=/home/js615/CLEAN/NECI

The testcode also needs to be told the locations of and version control systems used for CPMD and NECI. Do this by copying userconfig-template and setting the obvious variables to your personal settings. The automated compilation and testing script, checktest.py, needs to know the location of the test suite. This is defined in main_testcode/tools/checktest_config file residing in the test suite (the testcode directory).

testcode options can also be specified within checktest_config on a compiler-level basis. One of the more useful options is to specify how many cores the parallel tests should use. Note that new benchmarks need to be produced if the number of cores changes as certain FCIMC options are given on a per-core basis and so changing the number of cores used actually changes the calculation. See the testcode suite documentation for more details of the possible options.

Add the pseudopotential library to your ~/.bashrc as well. 

 export PP_LIBRARY_PATH=~/CLEAN/CPMD/PP_LIBRARY

2. Compiling the libraries

Certain compilers require certain libraries to be available when compiling CPMD and NECI. Which libraries are required can be investigated in the file checktest_config. 

 js615@scepter:~/CLEAN/testcode/main_testcode/tools> cat checktest_config

This contains blocks like this. 

 [PC-PGI]
 modules = pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
 job_types = neci
 job_categories = NECI

Although not all versions are used, it is convenient to compile lapack and fftw across a range of compilers so that they are accessible to the test code. JSS has written a set of scripts to do this. Extract scepter:~jss43/setup_libraries.tar.gz and run setup_libraries.sh. This installs libraries to ~/local/lib: you can adjust this location in the scripts if you want. It also sets up private environment modules for your own use.

It's important to note that the current version of fftw (3.2.2) doesn't compile with pathscale. (setup_libraries.tar.gz includes fftw 3.2 for this reason.)

See Lapack compilation for more information on compiling lapack.

3. Setting up the libraries as modules

The command 

 module load use.own

allows use of local libraries via environment modules in the folder ~/privatemodules. See Environment modules for more information. The test code requires each library to be set up in this manner.

The procedure to set up a module is as follows: 

 js615@scepter:~/privatemodules> mkdir -p fftw3.2/64
 js615@scepter:~/privatemodules> vi fftw3.2/64/pathscale

And then use the following input: 

 #%Module1.0#####################################################################
 ##
 ## 
 ##
 set root /home/js615/local/lib/fftw-3.2
 set version 3.2.2
 module-whatis "Loads version $version of the fftw library, compiled in 64-bit."
 prepend-path LIBRARY_PATH $root/lib
 prepend-path LD_LIBRARY_PATH $root/lib
 prepend-path MANPATH $root/manpages/man/

Change the root to reflect where you've actually got this version of fftw.

The following statement should be added to the end of modules compiled with pathscale: 

 if { [ llength [ array get env PSC_GENFLAGS ] ] } {
       set mypscflags  "-L $root/lib $env(PSC_GENFLAGS)"
 } else {
       set mypscflags "-L $root/lib"
 }
 setenv PSC_GENFLAGS $mypscflags

(JSS: note that this causes serious issues if multiple pathscale libraries are being used. Avoid.)

Now the same module files need to be made for lapack. These require a slightly different header: 

 js615@scepter:~/privatemodules/lapack3.1.1/32> cat ifort
 #%Module1.0#####################################################################
 ##
 ## 
 ##
 set root /home/js615/local/lib/lapack-3.1.1/32/ifort
 prepend-path LIBRARY_PATH $root/lib
 prepend-path LIBRARY_PATH $root/include
 prepend-path LD_LIBRARY_PATH $root/lib
 prepend-path LD_LIBRARY_PATH $root/include
 prepend-path MANPATH $root/manpages/man/

Note that in the current set-up there is slightly more information in these files, including (proc ModulesHelp and module-whatis). These have been ommitted here for clarity. People who are curious can go to the directory above and 'cat' the current file.

There is a more elegant solution to creating multiple modules for different compilers, using .base files (as done in setup_libraries---see http://www-theor.ch.cam.ac.uk/IT/software/modules.html). Furthermore, there is a better way to set PSC_GENFLAGS for pathscale. The build above is sufficient for the test code. To see these designs, go to: 

 scepter:/home/jss43/privatemodules

and also look at Environment modules.

4. Link the libraries to the test code.

The test code has a file called checktest_config which tells it which modules to load for each compiler. This is currently at: 

 js615@scepter:~/CLEAN/testcode/main_testcode/tools/checktest_config

These need to match the directory structure of your privatemodules set-up. To test this, take a block of checktest_config: 

 [PC-PGI]
 modules = pgi/32/8.0/5 use.own fftw3.2.2/32/pgi
 job_types = neci
 job_categories = NECI

and try to load everything on the "modules =" line (a "module purge" might be needed to avoid conflicts). E.g. 

 js615@scepter:~/CLEAN/testcode/main_testcode/tools> module load pgi/32/8.0/5 use.own fftw3.2.2/32/pgi

The test code should now work.

5. Setting up cron (for nightly testing)

Use the command: 

 crontab -e

from anywhere to set up cron. The following setup is used at the moment for this file: 

 PP_LIBRARY_PATH=/home/js615/CLEAN/CPMD/PP_LIBRARY
 30   01  * * *  umask 2 && /home/js615/CLEAN/testcode/main_testcode/tools/checktest.py

Re-stating the pseudopotential library path and umask 2 are required since cron doesn't source your ~/.bashrc. The following line runs checktest.py with the default settings at 1.30am.

Retrieved from "http://wikis.ch.cam.ac.uk/cuc3/wiki/index.php?title=NECI-testcode&oldid=4009"