Difference between revisions of "Useful Makefiles"

From CUC3
Jump to navigation Jump to search
import>Cen1001
 
import>Cen1001
 
(51 intermediate revisions by 3 users not shown)
Line 1: Line 1:
−
This page is a place to stick Makefiles for common codes that you've got working on the local compute servers
 +
This page is a place to stick Makefiles or other config files for common codes that you've got working on the local compute servers
  +
  +
== Tardis ==
  +
  +
=== cp2k with intel compilers ===
  +
  +
Setting the FORT_C_NAME variable to 'intel' helps cp2k's build system select the right compiler.
  +
  +
You need the mkl, mpi/mvapich/intel, blacs/mvapich/intel64, and scalapack/intel modules loaded
  +
  +
# The following settings worked for:
  +
# - AMD64 Opteron cluster
  +
# - SUSE Linux 10.0 (x86_64)
  +
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 9.1.037
  +
# - Intel(R) Cluster Math Kernel Library v7.2 for Linux
  +
# - MVAPICH
  +
# - BLACS and ScaLAPACK compiled for Intel
  +
#
  +
PERL = perl
  +
CC = cc
  +
CPP = cpp
  +
FC = mpif90 -FR
  +
LD = mpif90
  +
AR = ar -r
  +
DFLAGS = -D__INTEL -D__FFTSG\
  +
-D__parallel -D__BLACS -D__SCALAPACK\
  +
-Dfftwnd_f77=fftwnd_f77_\
  +
-Dfftwnd_f77_one=fftwnd_f77_one_\
  +
-Dfftw3d_f77_create_plan=fftw3d_f77_create_plan_\
  +
-Dfftw2d_f77_create_plan=fftw2d_f77_create_plan_\
  +
-Dfftwnd_f77_destroy_plan=fftwnd_f77_destroy_plan_\
  +
-Dfftw_f77_create_plan=fftw_f77_create_plan_\
  +
-Dfftw_f77=fftw_f77_\
  +
-Dfftw_f77_destroy_plan=fftw_f77_destroy_plan_
  +
CPPFLAGS = -traditional -C $(DFLAGS) -P
  +
FCFLAGS = $(DFLAGS) -O2
  +
MKLPATH = /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t
  +
LDFLAGS = $(FCFLAGS) -i-static
  +
LIBS = \
  +
-L/usr/local/Cluster-Apps/scalapack/intel/lib64 -lscalapack \
  +
$(MKLPATH)/libmkl_lapack.a \
  +
-L/usr/local/Cluster-Apps/blacs/mvapich/intel/lib64 -lblacsF77init -lblacs \
  +
$(MKLPATH)/libmkl_em64t.a \
  +
$(MKLPATH)/libguide.a\
  +
-lpthread
  +
  +
OBJECTS_ARCHITECTURE = machine_intel.o
  +
  +
# -D__FFTW\
  +
  +
  +
=== Jochen's CPMD with Portland compilers and MVAPICH ===
  +
  +
You need the mpi/mvapich/pgi module but ACML comes in automatically with PGI.
  +
  +
#----------------------------------------------------------------------------
  +
# Makefile for cpmd.x (plane wave electronic calculation)
  +
# Configuration: PGI-AMD64-MPI
  +
# Creation of Makefile: Dec 5 2006
  +
# on Linux tardis 2.6.15.1-clustervision-128_cvos #1 SMP Mon Sep 25 12:05:46 CEST 2006 x86_64 x86_64 x86_64 GNU/Linux
  +
# Author: jb376
  +
#----------------------------------------------------------------------------
  +
#
  +
SHELL = /bin/sh
  +
#
  +
#--------------- Default Configuration for PGI-AMD64-MPI ---------------
  +
SRC = .
  +
DEST = .
  +
BIN = .
  +
#QMMM_FLAGS = -D__QMECHCOUPL
  +
#QMMM_LIBS = -L. -lmm
  +
FFLAGS = -r8 -pc=64 -Msignextend
  +
#LFLAGS = -Bstatic -L. -latlas_x86-64 $(QMMM_LIBS)
  +
#LFLAGS = -Bstatic -L. -latlas_x86_64 $(QMMM_LIBS)
  +
LFLAGS = -lacml $(QMMM_LIBS)
  +
CFLAGS =
  +
CPP = /lib/cpp -P -C -traditional
  +
#CPPFLAGS = -D__Linux -D__PGI -DLAPACK -DFFT_DEFAULT -DPOINTER8 -D__pgf90 \
  +
# -DPARALLEL -DMP_LIBRARY=__MPI -DMYRINET
  +
CPPFLAGS = -D__Linux -D__PGI -DLAPACK -DFFT_DEFAULT -DPOINTER8 -D__pgf90 \
  +
-DPARALLEL -DMP_LIBRARY=__MPI
  +
NOOPT_FLAG =
  +
CC = mpicc -O2 -Wall -m64
  +
FC = mpif77 -c -fastsse -tp k8-64
  +
LD = mpif77 -fastsse -tp k8-64
  +
AR =
  +
#----------------------------------------------------------------------------
  +
  +
I had problems compiling CPMD v3.11.1 using mpif77: the compiler complained about some valid Fortran statements (e.g. append and cycle). Using mpif90 instead resolved this.--[[User:jss43|james]] 11:56, 8 August 2007 (BST)
  +
  +
The recent upgrade to tardis has changed how some modules work. In particular, mpicc now points to pgcc rather than gcc if the portland environment module is loaded. This is, Catherine and I think, the sane approach. The above CC options cause make to barf, as pgcc uses different flags to gcc. Change "-O2 -Wall -m64" to "-O2 -Minform=inform -pc=64" to give pgcc the equivalent options.--[[User:jss43|james]] 19:37, 7 March 2008 (GMT)
  +
  +
===NAMD2 with Intel compilers===
  +
  +
This one requires the openmpi/intel64 module
  +
  +
====Step 1: charm++====
  +
  +
./build charm++ mpi-linux-amd64 icc ifort
  +
  +
my src/arch/mpi-linux-amd64/conv-mach.sh
  +
  +
CMK_REAL_COMPILER=`mpiCC -show 2>/dev/null | cut -d' ' -f1 `
  +
case "$CMK_REAL_COMPILER" in
  +
g++) CMK_AMD64="-m64 -fPIC" ;;
  +
esac
  +
  +
CMK_CPP_CHARM="/lib/cpp -P"
  +
CMK_CPP_C="mpicc -E"
  +
CMK_CC="mpicc $CMK_AMD64 "
  +
CMK_CXX="mpiCC $CMK_AMD64 "
  +
CMK_CXXPP="mpiCC -E $CMK_AMD64 "
  +
  +
#CMK_SYSLIBS="-lmpich"
  +
# -lmpich is not needed as we replace 'icc' with 'mpicc' in the cc-icc.sh file
  +
CMK_SYSLIBS=" "
  +
CMK_LIBS="-lckqt $CMK_SYSLIBS "
  +
CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"
  +
  +
CMK_NATIVE_CC="gcc $CMK_AMD64 "
  +
CMK_NATIVE_LD="gcc $CMK_AMD64 "
  +
CMK_NATIVE_CXX="g++ $CMK_AMD64 "
  +
CMK_NATIVE_LDXX="g++ $CMK_AMD64 "
  +
CMK_NATIVE_LIBS=""
  +
  +
# fortran compiler
  +
CMK_CF77="f77"
  +
CMK_CF90="f90"
  +
CMK_F90LIBS=" "
  +
CMK_F77LIBS=" "
  +
CMK_MOD_NAME_ALLCAPS=1
  +
CMK_MOD_EXT="mod"
  +
CMK_F90_USE_MODDIR=1
  +
CMK_F90_MODINC="-p"
  +
  +
CMK_QT='generic64'
  +
CMK_RANLIB="ranlib"
  +
  +
and src/arch/common/cc-icc.sh
  +
  +
# Changed all the C/C++ compilers and linkers to the mpi compiler version s/icc/mpicc/ s/icpc/mpiCC/
  +
CMK_CPP_C='mpicc -E '
  +
CMK_CC="mpicc -fpic "
  +
CMK_CXX="mpiCC -fpic "
  +
CMK_CXXPP='mpiCC -E '
  +
CMK_LD='mpicc -i_dynamic '
  +
CMK_LDXX='mpiCC -i_dynamic '
  +
CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"
  +
# The F90 needed changing to ifort and -fPIC adding
  +
CMK_CF90='ifort -auto -fPIC '
  +
CMK_CF90_FIXED="$CMK_CF90 -132 -FI "
  +
CMK_NATIVE_F90="$CMK_CF90"
  +
CMK_NATIVE_CC="$CMK_CC"
  +
CMK_NATIVE_CXX="$CMK_CXX"
  +
CMK_NATIVE_LD="$CMK_LD"
  +
CMK_NATIVE_LDXX="$CMK_LDXX"
  +
# I removed a bunch of bogus -L options pointing to an ancient and nonexistent ifc installation
  +
CMK_F90LIBS='-lintrins -lIEPCF90 -lPEPCF90 -lF90 -lintrins -limf '
  +
CMK_MOD_NAME_ALLCAPS=1
  +
CMK_MOD_EXT="mod"
  +
CMK_F90_USE_MODDIR=""
  +
  +
Since tardis's IB stack was updated I can't get the charm++ built with Intel compilers to pass all the tests anymore. One built with gcc seems to do fine though. Load the environment/64-bit/openmpi/gnu64 module. Unpack fresh charm++ source and cd to it.
  +
  +
Edit src/arch/mpi-linux-amd64/conv-mach.sh and change the
  +
CMK_SYSLIBS="-lmpich"
  +
to
  +
CMK_SYSLIBS=" "
  +
then
  +
./build charm++ mpi-linux-amd64 --no-shared -O -DCMK_OPTIMIZE
  +
  +
The best test (according to the NAMD people) is to cd tests/charm++/megatest; build and run that one. This does pass.
  +
  +
====Step 2: NAMD====
  +
  +
Load the environment/64-bit/openmpi/intel64 module
  +
  +
./config tcl fftw Linux-amd64-MPI-icc
  +
cd Linux-amd64-MPI-icc
  +
  +
my arch/Linux-amd64.tcl:
  +
  +
TCLDIR=/usr
  +
TCLINCL=-I$(TCLDIR)/include
  +
TCLLIB=-L$(TCLDIR)/lib -ltcl8.4 -ldl
  +
TCLFLAGS=-DNAMD_TCL -DUSE_NON_CONST
  +
TCL=$(TCLINCL) $(TCLFLAGS)
  +
  +
  +
my arch/Linux-amd64.fftw:
  +
  +
FFTDIR=/usr/local/fftw2/intel/64/2.1.5
  +
FFTINCL=-I$(FFTDIR)/include -I$(HOME)/fftw/include
  +
FFTLIB=-L$(FFTDIR)/lib -L$(HOME)/fftw/lib -lsrfftw -lsfftw
  +
FFTFLAGS=-DNAMD_FFTW
  +
FFT=$(FFTINCL) $(FFTFLAGS)
  +
  +
my Make.charmm
  +
  +
CHARMBASE = /usr/local/charm++/charm-5.9-openmpi-gcc
  +
  +
my Linux-amd64-MPI-icc.arch which owes a great deal to Jochen's below
  +
  +
NAMD_ARCH = Linux-amd64
  +
CHARMARCH = mpi-linux-amd64
  +
FLOATOPTS = -ip -fno-rtti
  +
CXX = mpiCC
  +
CXXOPTS = -i-static -static-libcxa -O2 -unroll $(FLOATOPTS)
  +
CXXNOALIASOPTS = -O2 -unroll -fno-alias $(FLOATOPTS)
  +
CC = mpicc
  +
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)
  +
  +
now
  +
  +
make
  +
  +
Test it by doing an interactive qsub and
  +
  +
mpirun ./namd2 src/alanin
  +
  +
Jochen's Linux-amd64-MPI-icc.arch (Vastly improves performance over the defaults):
  +
  +
NAMD_ARCH = Linux-amd64
  +
# If using the gcc compiled charm++ you want to uncomment the line immediately below this
  +
# CHARMARCH = mpi-linux-amd64
  +
# and comment out the one below this - CEN
  +
CHARMARCH = mpi-linux-amd64-icc
  +
FLOATOPTS = -fno-rtti
  +
CXX = /usr/local/Cluster-Apps/ofed/1.0/mpi/intel/mvapich-0.9.7-mlx2.1.0/bin/mpicxx
  +
# This is a little odd as -tpp6 is a Portland option - CEN
  +
CXXOPTS = -tpp6 -pc64 -i-static -static-libcxa -O2 -unroll $(FLOATOPTS)
  +
CXXNOALIASOPTS = -O2 -unroll -fno-alias $(FLOATOPTS)
  +
CC = /usr/local/Cluster-Apps/ofed/1.0/mpi/intel/mvapich-0.9.7-mlx2.1.0/bin/mpicc
  +
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)
  +
  +
=== FFTW 2.1.5 with MVAPICH ===
  +
  +
This is not really well-documented elsewhere. FFTW 2.1.5 is used in (e.g.) CPMD as an alternative to the default FFT engine supplied. It is trivial to compile in serial (FFTW 3 is even easier, but sadly the parallel version is in alpha and incompatible with the widely-used FFTW2). On tardis, in fftw-2.1.5 directory formed by extracting the tarball:
  +
  +
# env CC=mpicc F77=mpif90 ./configure --prefix=`pwd` --enable-mpi --enable-sse
  +
# make
  +
# make install
  +
  +
Having to set the c compiler for parallel compilation is not mentioned in the docs...
  +
This should also work with openMPI.
  +
  +
=== BLACS and scaLAPACK with openMPI ===
  +
  +
BLACS and scaLAPACK are the parallel equivalents of the BLAS and LAPACK libraries. Note that they require the BLAS and LAPACK libraries.
  +
  +
I didn't have any luck using the BLACS and scaLAPACK provided on tardis (either the intel modules or via MKL), so compiled my own. The BLACS and scaLAPACK documentation on using openMPI is woeful. Fortunately the openMPI people are lovely and tell us how to do it in their [http://www.open-mpi.org/faq/?category=mpi-apps#blacs FAQ].
  +
  +
I did the following with the the openmpi/intel package.
  +
  +
For BLACS:
  +
  +
* Download [http://www.netlib.org/blacs/mpiblacs.tgz mpiblacs.tgz] and [http://www.netlib.org/blacs/mpiblacs-patch03.tg mpiblacs-patch03.tgz] and extract. Extract the patch second, to apply it.
  +
* copy the relevant template make include file to the BLACS home directory:
  +
cp BMAKES/BMake.MPI-LINUX BMake.inc
  +
* Edit Bmake.inc according to the [http://www.open-mpi.org/faq/?category=mpi-apps#blacs openMPI FAQ] (except I used mpif90 rather than mpif77).
  +
* Set INTFACE=-DADD_ (this is crucial if you ever want to link it to something!).
  +
* compile the libraries:
  +
make mpi
  +
* compile the tests:
  +
make tester
  +
  +
The libraries reside in BLACS/LIB/*.a and the test executables in BLACS/TESTING/EXE/x*. The tests need to be run using mpirun. All the tests pass (note that the final test is an abort call, which gives a stack trace---this is the correct behaviour and is not an error).
  +
  +
For scaLAPACK:
  +
  +
* Download [http://www.netlib.org/scalapack/scalapack.tgz scalapack.tgz] and extract.
  +
* Copy the SLmake.inc.example to SLmake.inc and edit according to the [http://www.open-mpi.org/faq/?category=mpi-apps#scalapack openMPI FAQ] (again, I used mpif90 rather than mpif77).
  +
* Specify the locations of your BLACS, BLAS and LAPACK libraries.
  +
* Change -Df77IsF2C to -DAdd_ in CDEFS.
  +
* compile
  +
make
  +
* You can also make tests:
  +
make exe
  +
  +
The library is libscalapack.a in the scaLAPACK home directory and the tests are in the TESTING subdirectory. The tests either pass (or warn about input values) apart from xcqr (which tests the single-precision complex QR factorisation routines: not something I worry about).
  +
  +
=== GROMACS ===
  +
  +
With PGI and OpenMPI
  +
  +
Download the source from http://www.gromacs.org/
  +
  +
$ module li
  +
Currently Loaded Modulefiles:
  +
1) pgi/64/7.1/6 6) compilers/64
  +
2) icc/64/10.0/026(default) 7) ofed/64/1.3
  +
3) ifort/64/10.0/026(default) 8) mpi/openmpi/64/pgi71/1.2.5
  +
4) idb/64/10.0/031 9) package/64/openmpi/pgi
  +
5) pathscale/64/3.0(default) 10) fftw/64/pgi/3.1.1
  +
  +
$ export CC=pgcc # this seems to be the magic
  +
$ export CXX=pgCC
  +
$ ./configure --disable-float --prefix=/home/cen1001/gromacs --enable-mpi
  +
$ make
  +
$ make install
  +
  +
== mek-quake ==
  +
=== GAMESS-US ===
  +
  +
This will probably work on clust and nimbus too as they are very similar machines.
  +
  +
module add pgi64/7.1-6
  +
  +
You need to edit the scripts comp, compall, lked, and ddi/compddi. The TARGET in all of these should be linux64 and the fortran compiler (FORTRAN) set to pgf77 within the appropriate section. Leave the C compiler (CCOMP) as gcc. It will automatically link in the Portland copy of ACML so you will get a fast blas library. We did not need to change any other options.
  +
  +
In the DDI compilation set COMM to sockets.
  +
  +
=== CP2K ===
  +
  +
The most recent version of cp2k can be obtained through anonymous CVS access. Typing the following will create a cp2k directory and download the source code (if asked for a password just press enter):
  +
  +
touch $HOME/.cvspass
  +
cvs -d:pserver:anonymous@cvs.cp2k.berlios.de:/cvsroot/cp2k login
  +
cvs -z3 -d:pserver:anonymous@cvs.cp2k.berlios.de:/cvsroot/cp2k co cp2k
  +
  +
To update the source to its latest version just type:
  +
  +
cd cp2k
  +
cvs update -dAP
  +
  +
To get the most out of the cp2k program it is advisable to also install the FFTW and LIBINT libraries for respectively the fast fourier transform handling and the efficient evaluation of two-body integrals over Gaussian functions. The latest version of FFTW (fftw-3.2.1) can be get from anonymous ftp to fftw.org
  +
  +
ftp fftw.org
  +
  +
If you are prompted for a user name or password just type "anonymous" in both cases and then type:
  +
  +
cd pub/fftw
  +
mget fftw-3.2.1.tar.gz
  +
bye
  +
  +
Untarring the file using
  +
  +
tar -xvf fftw-3.2.1.tar.gz
  +
  +
will create a directory called fftw-3.2.1. Change to this directory
  +
  +
cd fftw-3.2.1
  +
  +
and compile the libraries.
  +
  +
./configure --prefix=/home/el316
  +
make
  +
make install
  +
  +
The process will create a lib directory in /home/el316 which contains the file libfftw.a which will be needed for future linking. By changing the prefix path above you can ofcourse regulate yourself where the library will be build.
  +
  +
To install LIBINT you will first have to retrieve the source file (libint-1.1.4.tar.gz) from the following URL:
  +
  +
http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html
  +
  +
Untar and make the source code:
  +
  +
tar -xvf libint-1.1.4.tar.gz
  +
cd libint-1.1.4
  +
./configure --prefix=/home/el316
  +
make
  +
make install
  +
  +
This should create two libraries called libderiv.a and libint.a.
  +
  +
Before starting with the compilation of the cp2k code let me first list the modules which where loaded during my compilation:
  +
  +
modules
  +
icc/64/10.1/018
  +
ifort/64/10.1/018
  +
mpi/openmpi/64/intel/1.3
  +
mkl/64/10.1/0.015
  +
  +
Obviously it is a intel build using openmpi (if parallel). Before we can actually compile cp2k we still have to perform two actions.
  +
First, if we want to use the LIBINT libraries which are written in C, a small wrapper has to be compiled to use them with CP2K.
  +
Just go to the libint_tools directory of cp2k
  +
  +
cd cp2k/tools/hfx_tools/libint_tools
  +
  +
and type:
  +
  +
icc -O3 -c libint_cpp_wrapper.cpp -I/home/el316/include
  +
  +
The include folder above should have been created when you compiled the LIBINT code. You should now have created the object file libint_cpp_wrapper.o.
  +
  +
For simplicity I usually at this point make two copies of the cp2k folder. One called cp2koptim (for the serial build which however will not be used alot) and cp2koptim2 (for the parallel build). Before we can compile the cp2k code first a working architecture file should be created. The following two files should work on mek-quake for respectively a serial and parallel build:
  +
  +
# Serial build / Mek-quake
  +
# The following settings worked for:
  +
# - AMD64 Opteron / Mek-quake
  +
# - SUSE Linux Enterprise Server 10.3 (x86_64)
  +
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 10.1.018
  +
# - Intel MKL 10.1.0.015
  +
# - fftw-3.2.1 (ftp.fftw.org)
  +
# - libint-1.1.4 (http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html)
  +
#
  +
PERL = perl
  +
CC = cc
  +
CPP = cpp
  +
FC = ifort -FR
  +
LD = ifort
  +
AR = ar -r
  +
DFLAGS = -D__INTEL -D__FFTSG -D__FFT3W -D__HAS_NO_ISO_C_BINDING -D__LIBINT
  +
MKLPATH = /usr/local/shared/intel/mkl/10.1.0.015/lib/em64t
  +
INTLIB = /usr/local/shared/intel/fce/10.1.015/lib
  +
INCPATH = /home/el316/include
  +
LIBPATH = /home/el316/lib
  +
CP2KPATH = /home/el316/cp2koptim
  +
COMFLAGS = $(DFLAGS) -I$(MKLPATH) -I$(INCPATH)
  +
CPPFLAGS = $(COMFLAGS) -C -traditional -P
  +
FCFLAGS = $(COMFLAGS) -O2 -heap-arrays 64 -unroll -pc64 -fpp -free
  +
LDFLAGS = $(FCFLAGS) -L/$(LIBPATH) -L/$(INTLIB)
  +
LIBS = $(MKLPATH)/libmkl_lapack.a \
  +
$(MKLPATH)/libmkl_em64t.a \
  +
$(MKLPATH)/libguide.a \
  +
$(LIBPATH)/libfftw3.a \
  +
$(CP2KPATH)/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o \
  +
$(LIBPATH)/libderiv.a \
  +
$(LIBPATH)/libint.a \
  +
-lpthread \
  +
-lstdc++
  +
  +
OBJECTS_ARCHITECTURE = machine_intel.o
  +
  +
# Parallel build / Mek-quake
  +
# The following settings worked for:
  +
# - AMD64 Opteron /Mek-quake
  +
# - SUSE Linux Enterprise Server 10.3 (x86_64)
  +
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 10.1.018
  +
# - Intel MKL 10.1.0.015
  +
# - OpenMPI/Intel 1.3
  +
# - fftw-3.2.1 (ftp.fftw.org)
  +
# - libint-1.1.4 (http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html)
  +
#
  +
PERL = perl
  +
CC = cc
  +
CPP = cpp
  +
FC = mpif90 -FR
  +
LD = mpif90
  +
AR = ar -r
  +
DFLAGS = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -D__SCALAPACK \
  +
-D__HAS_NO_ISO_C_BINDING -D__LIBINT
  +
MKLPATH = /usr/local/shared/intel/mkl/10.1.0.015/lib/em64t
  +
INTLIB = /usr/local/shared/intel/fce/10.1.015/lib
  +
INCPATH = /home/el316/include
  +
LIBPATH = /home/el316/lib
  +
CP2KPATH = /home/el316/cp2koptim2
  +
COMFLAGS = $(DFLAGS) -I$(MKLPATH) -I$(INCPATH)
  +
CPPFLAGS = $(COMFLAGS) -C -traditional -P
  +
FCFLAGS = $(COMFLAGS) -O2 -heap-arrays 64 -unroll -pc64 -fpp -free
  +
LDFLAGS = $(FCFLAGS) -L/$(LIBPATH) -L/$(INTLIB)
  +
LIBS = $(MKLPATH)/libmkl_scalapack_lp64.a \
  +
$(MKLPATH)/libmkl_lapack.a \
  +
$(MKLPATH)/libmkl_blacs_openmpi_lp64.a \
  +
$(MKLPATH)/libmkl_em64t.a \
  +
$(MKLPATH)/libguide.a \
  +
$(LIBPATH)/libfftw3.a \
  +
$(CP2KPATH)/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o \
  +
$(LIBPATH)/libderiv.a \
  +
$(LIBPATH)/libint.a \
  +
-lpthread \
  +
-lstdc++
  +
  +
OBJECTS_ARCHITECTURE = machine_intel.o
  +
  +
The above architecture files should be copied to the "/arch" directory of respectively the cp2koptim and cp2koptim2 folder. Following the naming scheme of the arch files already present in this folder I have chosen to call them "Linux-x86-64-intel.optim" and "Linux-x86-64-intel.optim2" for repectively the serial and parallel arch-file. Both files can also be found in the my public folder (/home/el316/public) on mek-quake. For the final step of the actual compilation you should go to the "/makefiles" folder of the cp2k directory and type:
  +
  +
make VERSION=optim ARCH=Linux-x86-64-intel > compilations.inf
  +
make VERSION=optim2 ARCH=Linux-x86-64-intel > compilationp.inf
  +
  +
If everything went well, this should create the working executables "cp2k.optim" or "cp2k.optim2" in the "/exe/Linux-x86-64-intel" folder of cp2k. Note that redirecting the output to the files "compilations.inf" and "compilationp.inf" makes sure that only the warnings and error messages are shown on screen. Some warnings during compilation time are perfectly normal. In case of a first time compilation of the code, it is advisable to leave out the optimizations and replace the FCFLAGS variable in the architecture file by: "$(COMFLAGS) -O0 -heap-arrays 64". The -heap-arrays 64 option should always be there, otherwise you will almost certainly encounter segmentation faults during exectution of the code.
  +
  +
====CP2K and OPTIM====
  +
  +
The parallel CP2K code can now also be used together with the OPTIM program. For this a little program (cp2kdriver.f90) has to be compiled, which will make it possible for the OPTIM code to call the CP2K program. The program can be found in my public folder on mek-quake (/home/el316/public). Just copy the cp2kdriver.f90 program to your home folder and type as one line with of course all el316's substituted by your userid:
  +
  +
mpif90 -o /home/el316/cp2koptim2/exe/Linux-x86-64-intel/cp2kdriver cp2kdriver.f90 -FR -O0 -heap-arrays 64
  +
-L/home/el316/cp2koptim2/lib/Linux-x86-64-intel/optim2/ -lcp2k_lib -lcp2k_base_lib -lcp2k_fft_lib
  +
-L/usr/local/shared/intel/mkl/10.1.0.015/lib/em64t -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_em64t -lguide -L/home/el316/lib/
  +
-lfftw3 /home/el316/cp2koptim2/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o -lderiv -lint -lstdc++ -pthread
  +
  +
It should create a working executable called cp2kdriver in the same folder as the CP2K executable "cp2k.optim2". If all compilations above were successful, you should now be able to make combined OPTIM/CP2K runs, by specifying the "CP2K" keyword in the odata file (see OPTIM documentation for its exact use).
  +
  +
== Workstations ==
  +
  +
=== Graphviz ===
  +
  +
The graphviz package contains both the dot language needed and several parsers, which produce graphs in different formats (using some rather clever maths...).
  +
  +
It is not installed in the sector, but is easy to compile (all of its dependencies are already present).
  +
  +
I downloaded the tarball from [http://www.graphviz.org/Download_source.php graphviz's download page] and compiled using:
  +
./configure --prefix=$HOME/local && make && make install
  +
It takes a while(!) to compile and this installs the executables to $HOME/local/bin. Various other files are installed to the sub-directories of $HOME/local, including manpages. You can extend your MANPATH environment variable to view these easily (or use the appropriate option with ''man'').
  +
  +
Please note that due to a bug in libtool (fixed upstream) it is vital that the prefix path does *not* end with a /. If you do, it results in an error in the make install step. To fix this, run a make clean and then run configure again with the correct path. (That was fun to figure out!)
  +
  +
=== makedepf90 ===
  +
  +
[http://personal.inet.fi/private/erikedelmann/makedepf90/ makedepf90] is a very useful program which analyses a set of fortran files and produces the dependency list of modules and include files for use in a makefile. Neatly, you can add a target to your makefile to automatically produce the dependencies (see the manpage for an example).
  +
  +
I have a directory ${HOME}/local where I put utilities I compile myself. I did the following:
  +
  +
jss43@discovery:~/local/src$ tar xvzf makedepf90-2.8.8.tar.gz
  +
$ cd makedepf90-2.8.8/
  +
$./configure --prefix=${HOME}/local --mandir=${HOME}/local/share/man
  +
$ make && make install
  +
  +
This compiles it. I used a non-default location for the man page as I already had some local manpages in the ${HOME}/local/share/man directory.
  +
  +
You then need to add it to your path. If you use the same layout as me, then add to your .bashrc:
  +
  +
export PATH=${PATH}:${HOME}/local/bin
  +
export MANPATH=${MANPATH}:${HOME}/local/share/man
  +
  +
You can now run makedepf90 and see the manpage from the command line.
  +
  +
It is quite easy to write a simple script which uses makedepf90 to generate a makefile for different compilers/optimisation levels as required and is suited for a wide variety of projects (ie is rather independent of the codebase).
  +
  +
=== gromacs on Ubuntu workstation ===
  +
  +
The gcc on the Ubuntu machines needs an extra flag to build a working gromacs 4.0.5.
  +
  +
CFLAGS="-U_FORTIFY_SOURCE" ./configure --enable-float --enable-threads --prefix=/path/to/whereever
  +
make
  +
make install

Latest revision as of 09:53, 9 July 2010

This page is a place to stick Makefiles or other config files for common codes that you've got working on the local compute servers

Tardis

cp2k with intel compilers

Setting the FORT_C_NAME variable to 'intel' helps cp2k's build system select the right compiler.

You need the mkl, mpi/mvapich/intel, blacs/mvapich/intel64, and scalapack/intel modules loaded 

# The following settings worked for:
# - AMD64 Opteron cluster
# - SUSE Linux 10.0 (x86_64)
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 9.1.037
# - Intel(R) Cluster Math Kernel Library v7.2 for Linux
# - MVAPICH
# - BLACS and ScaLAPACK compiled for Intel
#
PERL     = perl
CC       = cc
CPP      = cpp
FC       = mpif90 -FR
LD       = mpif90
AR       = ar -r
DFLAGS   = -D__INTEL -D__FFTSG\
          -D__parallel -D__BLACS -D__SCALAPACK\
          -Dfftwnd_f77=fftwnd_f77_\
          -Dfftwnd_f77_one=fftwnd_f77_one_\
          -Dfftw3d_f77_create_plan=fftw3d_f77_create_plan_\
          -Dfftw2d_f77_create_plan=fftw2d_f77_create_plan_\
          -Dfftwnd_f77_destroy_plan=fftwnd_f77_destroy_plan_\
          -Dfftw_f77_create_plan=fftw_f77_create_plan_\
          -Dfftw_f77=fftw_f77_\
          -Dfftw_f77_destroy_plan=fftw_f77_destroy_plan_
CPPFLAGS = -traditional -C $(DFLAGS) -P
FCFLAGS  = $(DFLAGS) -O2
MKLPATH  = /usr/local/Cluster-Apps/intel/mkl/8.0/lib/em64t
LDFLAGS  = $(FCFLAGS) -i-static
LIBS     = \
          -L/usr/local/Cluster-Apps/scalapack/intel/lib64 -lscalapack \
          $(MKLPATH)/libmkl_lapack.a \
          -L/usr/local/Cluster-Apps/blacs/mvapich/intel/lib64 -lblacsF77init -lblacs \
          $(MKLPATH)/libmkl_em64t.a \
          $(MKLPATH)/libguide.a\
          -lpthread

OBJECTS_ARCHITECTURE = machine_intel.o

#           -D__FFTW\


Jochen's CPMD with Portland compilers and MVAPICH

You need the mpi/mvapich/pgi module but ACML comes in automatically with PGI. 

#----------------------------------------------------------------------------
# Makefile for cpmd.x (plane wave electronic calculation)
# Configuration: PGI-AMD64-MPI
# Creation of Makefile: Dec  5 2006
# on Linux tardis 2.6.15.1-clustervision-128_cvos #1 SMP Mon Sep 25 12:05:46 CEST 2006   x86_64 x86_64 x86_64 GNU/Linux
# Author: jb376
#----------------------------------------------------------------------------
#
SHELL = /bin/sh
#
#--------------- Default Configuration for PGI-AMD64-MPI ---------------
SRC  = .
DEST = .
BIN  = .
#QMMM_FLAGS = -D__QMECHCOUPL
#QMMM_LIBS  = -L. -lmm
FFLAGS = -r8 -pc=64 -Msignextend  
#LFLAGS = -Bstatic -L. -latlas_x86-64 $(QMMM_LIBS)
#LFLAGS = -Bstatic -L. -latlas_x86_64 $(QMMM_LIBS)
LFLAGS = -lacml $(QMMM_LIBS)
CFLAGS = 
CPP = /lib/cpp -P -C -traditional
#CPPFLAGS = -D__Linux -D__PGI -DLAPACK -DFFT_DEFAULT -DPOINTER8 -D__pgf90 \
#               -DPARALLEL -DMP_LIBRARY=__MPI -DMYRINET  
CPPFLAGS = -D__Linux -D__PGI -DLAPACK -DFFT_DEFAULT -DPOINTER8 -D__pgf90 \
              -DPARALLEL -DMP_LIBRARY=__MPI 
NOOPT_FLAG = 
CC = mpicc -O2 -Wall -m64
FC = mpif77 -c -fastsse -tp k8-64
LD = mpif77 -fastsse -tp k8-64
AR = 
#----------------------------------------------------------------------------

I had problems compiling CPMD v3.11.1 using mpif77: the compiler complained about some valid Fortran statements (e.g. append and cycle). Using mpif90 instead resolved this.--james 11:56, 8 August 2007 (BST)

The recent upgrade to tardis has changed how some modules work. In particular, mpicc now points to pgcc rather than gcc if the portland environment module is loaded. This is, Catherine and I think, the sane approach. The above CC options cause make to barf, as pgcc uses different flags to gcc. Change "-O2 -Wall -m64" to "-O2 -Minform=inform -pc=64" to give pgcc the equivalent options.--james 19:37, 7 March 2008 (GMT)

NAMD2 with Intel compilers

This one requires the openmpi/intel64 module

Step 1: charm++

./build charm++ mpi-linux-amd64 icc ifort

my src/arch/mpi-linux-amd64/conv-mach.sh 

CMK_REAL_COMPILER=`mpiCC -show 2>/dev/null | cut -d' ' -f1 `
case "$CMK_REAL_COMPILER" in
 g++) CMK_AMD64="-m64 -fPIC" ;;
esac

CMK_CPP_CHARM="/lib/cpp -P"
CMK_CPP_C="mpicc -E"
CMK_CC="mpicc $CMK_AMD64 "
CMK_CXX="mpiCC $CMK_AMD64 "
CMK_CXXPP="mpiCC -E $CMK_AMD64 "

#CMK_SYSLIBS="-lmpich" 
# -lmpich is not needed as we replace 'icc' with 'mpicc' in the cc-icc.sh file
CMK_SYSLIBS=" "
CMK_LIBS="-lckqt $CMK_SYSLIBS "
CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"

CMK_NATIVE_CC="gcc $CMK_AMD64 "
CMK_NATIVE_LD="gcc $CMK_AMD64 "
CMK_NATIVE_CXX="g++ $CMK_AMD64 "
CMK_NATIVE_LDXX="g++ $CMK_AMD64 "
CMK_NATIVE_LIBS=""

# fortran compiler 
CMK_CF77="f77"
CMK_CF90="f90"
CMK_F90LIBS=" "
CMK_F77LIBS=" "
CMK_MOD_NAME_ALLCAPS=1
CMK_MOD_EXT="mod"
CMK_F90_USE_MODDIR=1
CMK_F90_MODINC="-p"

CMK_QT='generic64'
CMK_RANLIB="ranlib"

and src/arch/common/cc-icc.sh 

# Changed all the C/C++ compilers and linkers to the mpi compiler version s/icc/mpicc/ s/icpc/mpiCC/
CMK_CPP_C='mpicc -E '
CMK_CC="mpicc -fpic "
CMK_CXX="mpiCC -fpic "
CMK_CXXPP='mpiCC -E '
CMK_LD='mpicc -i_dynamic '
CMK_LDXX='mpiCC -i_dynamic '
CMK_LD_LIBRARY_PATH="-Wl,-rpath,$CHARMLIBSO/"
# The F90 needed changing to ifort and -fPIC adding
CMK_CF90='ifort -auto -fPIC '
CMK_CF90_FIXED="$CMK_CF90 -132 -FI "
CMK_NATIVE_F90="$CMK_CF90"
CMK_NATIVE_CC="$CMK_CC"
CMK_NATIVE_CXX="$CMK_CXX"
CMK_NATIVE_LD="$CMK_LD"
CMK_NATIVE_LDXX="$CMK_LDXX"
# I removed a bunch of bogus -L options pointing to an ancient and nonexistent ifc installation
CMK_F90LIBS='-lintrins -lIEPCF90 -lPEPCF90 -lF90 -lintrins -limf  '
CMK_MOD_NAME_ALLCAPS=1
CMK_MOD_EXT="mod"
CMK_F90_USE_MODDIR=""

Since tardis's IB stack was updated I can't get the charm++ built with Intel compilers to pass all the tests anymore. One built with gcc seems to do fine though. Load the environment/64-bit/openmpi/gnu64 module. Unpack fresh charm++ source and cd to it.

Edit src/arch/mpi-linux-amd64/conv-mach.sh and change the 

 CMK_SYSLIBS="-lmpich"

to 

CMK_SYSLIBS=" "

then 

./build charm++ mpi-linux-amd64 --no-shared -O -DCMK_OPTIMIZE

The best test (according to the NAMD people) is to cd tests/charm++/megatest; build and run that one. This does pass.

Step 2: NAMD

Load the environment/64-bit/openmpi/intel64 module 

./config tcl fftw Linux-amd64-MPI-icc
cd Linux-amd64-MPI-icc

my arch/Linux-amd64.tcl: 

TCLDIR=/usr
TCLINCL=-I$(TCLDIR)/include 
TCLLIB=-L$(TCLDIR)/lib -ltcl8.4 -ldl
TCLFLAGS=-DNAMD_TCL -DUSE_NON_CONST 
TCL=$(TCLINCL) $(TCLFLAGS)


my arch/Linux-amd64.fftw: 

FFTDIR=/usr/local/fftw2/intel/64/2.1.5
FFTINCL=-I$(FFTDIR)/include -I$(HOME)/fftw/include
FFTLIB=-L$(FFTDIR)/lib -L$(HOME)/fftw/lib -lsrfftw -lsfftw
FFTFLAGS=-DNAMD_FFTW
FFT=$(FFTINCL) $(FFTFLAGS)

my Make.charmm 

CHARMBASE = /usr/local/charm++/charm-5.9-openmpi-gcc

my Linux-amd64-MPI-icc.arch which owes a great deal to Jochen's below 

NAMD_ARCH = Linux-amd64
CHARMARCH = mpi-linux-amd64
FLOATOPTS = -ip -fno-rtti
CXX = mpiCC
CXXOPTS =  -i-static -static-libcxa -O2 -unroll $(FLOATOPTS)
CXXNOALIASOPTS = -O2 -unroll -fno-alias $(FLOATOPTS)
CC = mpicc
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)

now 

make

Test it by doing an interactive qsub and 

mpirun ./namd2 src/alanin

Jochen's Linux-amd64-MPI-icc.arch (Vastly improves performance over the defaults): 

NAMD_ARCH = Linux-amd64
# If using the gcc compiled charm++ you want to uncomment the line immediately below this
# CHARMARCH = mpi-linux-amd64
# and comment out the one below this - CEN
CHARMARCH = mpi-linux-amd64-icc
FLOATOPTS = -fno-rtti
CXX = /usr/local/Cluster-Apps/ofed/1.0/mpi/intel/mvapich-0.9.7-mlx2.1.0/bin/mpicxx
# This is a little odd as -tpp6 is a Portland option - CEN
CXXOPTS = -tpp6 -pc64 -i-static -static-libcxa -O2 -unroll $(FLOATOPTS)
CXXNOALIASOPTS = -O2 -unroll -fno-alias $(FLOATOPTS)
CC = /usr/local/Cluster-Apps/ofed/1.0/mpi/intel/mvapich-0.9.7-mlx2.1.0/bin/mpicc
COPTS = -i-static -static-libcxa -O2 $(FLOATOPTS)

FFTW 2.1.5 with MVAPICH

This is not really well-documented elsewhere. FFTW 2.1.5 is used in (e.g.) CPMD as an alternative to the default FFT engine supplied. It is trivial to compile in serial (FFTW 3 is even easier, but sadly the parallel version is in alpha and incompatible with the widely-used FFTW2). On tardis, in fftw-2.1.5 directory formed by extracting the tarball:

  1. env CC=mpicc F77=mpif90 ./configure --prefix=`pwd` --enable-mpi --enable-sse
  2. make
  3. make install

Having to set the c compiler for parallel compilation is not mentioned in the docs... This should also work with openMPI.

BLACS and scaLAPACK with openMPI

BLACS and scaLAPACK are the parallel equivalents of the BLAS and LAPACK libraries. Note that they require the BLAS and LAPACK libraries.

I didn't have any luck using the BLACS and scaLAPACK provided on tardis (either the intel modules or via MKL), so compiled my own. The BLACS and scaLAPACK documentation on using openMPI is woeful. Fortunately the openMPI people are lovely and tell us how to do it in their FAQ.

I did the following with the the openmpi/intel package.

For BLACS:

  • Download mpiblacs.tgz and mpiblacs-patch03.tgz and extract. Extract the patch second, to apply it.
  • copy the relevant template make include file to the BLACS home directory:
cp BMAKES/BMake.MPI-LINUX BMake.inc
  • Edit Bmake.inc according to the openMPI FAQ (except I used mpif90 rather than mpif77).
  • Set INTFACE=-DADD_ (this is crucial if you ever want to link it to something!).
  • compile the libraries:
make mpi
  • compile the tests:
make tester

The libraries reside in BLACS/LIB/*.a and the test executables in BLACS/TESTING/EXE/x*. The tests need to be run using mpirun. All the tests pass (note that the final test is an abort call, which gives a stack trace---this is the correct behaviour and is not an error).

For scaLAPACK:

  • Download scalapack.tgz and extract.
  • Copy the SLmake.inc.example to SLmake.inc and edit according to the openMPI FAQ (again, I used mpif90 rather than mpif77).
  • Specify the locations of your BLACS, BLAS and LAPACK libraries.
  • Change -Df77IsF2C to -DAdd_ in CDEFS.
  • compile
make
  • You can also make tests:
make exe

The library is libscalapack.a in the scaLAPACK home directory and the tests are in the TESTING subdirectory. The tests either pass (or warn about input values) apart from xcqr (which tests the single-precision complex QR factorisation routines: not something I worry about).

GROMACS

With PGI and OpenMPI

Download the source from http://www.gromacs.org/ 

$ module li
Currently Loaded Modulefiles:
 1) pgi/64/7.1/6                 6) compilers/64
 2) icc/64/10.0/026(default)     7) ofed/64/1.3
 3) ifort/64/10.0/026(default)   8) mpi/openmpi/64/pgi71/1.2.5
 4) idb/64/10.0/031              9) package/64/openmpi/pgi
 5) pathscale/64/3.0(default)   10) fftw/64/pgi/3.1.1

$ export CC=pgcc # this seems to be the magic
$ export CXX=pgCC
$ ./configure --disable-float --prefix=/home/cen1001/gromacs --enable-mpi
$ make
$ make install

mek-quake

GAMESS-US

This will probably work on clust and nimbus too as they are very similar machines. 

module add pgi64/7.1-6

You need to edit the scripts comp, compall, lked, and ddi/compddi. The TARGET in all of these should be linux64 and the fortran compiler (FORTRAN) set to pgf77 within the appropriate section. Leave the C compiler (CCOMP) as gcc. It will automatically link in the Portland copy of ACML so you will get a fast blas library. We did not need to change any other options.

In the DDI compilation set COMM to sockets.

CP2K

The most recent version of cp2k can be obtained through anonymous CVS access. Typing the following will create a cp2k directory and download the source code (if asked for a password just press enter): 

touch $HOME/.cvspass
cvs -d:pserver:anonymous@cvs.cp2k.berlios.de:/cvsroot/cp2k login 
cvs -z3 -d:pserver:anonymous@cvs.cp2k.berlios.de:/cvsroot/cp2k co cp2k

To update the source to its latest version just type: 

cd cp2k
cvs update -dAP

To get the most out of the cp2k program it is advisable to also install the FFTW and LIBINT libraries for respectively the fast fourier transform handling and the efficient evaluation of two-body integrals over Gaussian functions. The latest version of FFTW (fftw-3.2.1) can be get from anonymous ftp to fftw.org 

ftp fftw.org

If you are prompted for a user name or password just type "anonymous" in both cases and then type: 

cd pub/fftw
mget fftw-3.2.1.tar.gz
bye

Untarring the file using 

tar -xvf fftw-3.2.1.tar.gz

will create a directory called fftw-3.2.1. Change to this directory 

cd fftw-3.2.1

and compile the libraries. 

./configure --prefix=/home/el316
make
make install

The process will create a lib directory in /home/el316 which contains the file libfftw.a which will be needed for future linking. By changing the prefix path above you can ofcourse regulate yourself where the library will be build.

To install LIBINT you will first have to retrieve the source file (libint-1.1.4.tar.gz) from the following URL: 

http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html

Untar and make the source code: 

tar -xvf libint-1.1.4.tar.gz
cd libint-1.1.4
./configure --prefix=/home/el316
make
make install

This should create two libraries called libderiv.a and libint.a.

Before starting with the compilation of the cp2k code let me first list the modules which where loaded during my compilation: 

modules
icc/64/10.1/018
ifort/64/10.1/018
mpi/openmpi/64/intel/1.3
mkl/64/10.1/0.015

Obviously it is a intel build using openmpi (if parallel). Before we can actually compile cp2k we still have to perform two actions. First, if we want to use the LIBINT libraries which are written in C, a small wrapper has to be compiled to use them with CP2K. Just go to the libint_tools directory of cp2k 

cd cp2k/tools/hfx_tools/libint_tools

and type: 

icc -O3 -c libint_cpp_wrapper.cpp -I/home/el316/include

The include folder above should have been created when you compiled the LIBINT code. You should now have created the object file libint_cpp_wrapper.o.

For simplicity I usually at this point make two copies of the cp2k folder. One called cp2koptim (for the serial build which however will not be used alot) and cp2koptim2 (for the parallel build). Before we can compile the cp2k code first a working architecture file should be created. The following two files should work on mek-quake for respectively a serial and parallel build: 

# Serial build / Mek-quake
# The following settings worked for:
# - AMD64 Opteron / Mek-quake
# - SUSE Linux Enterprise Server 10.3 (x86_64)
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 10.1.018
# - Intel MKL 10.1.0.015
# - fftw-3.2.1 (ftp.fftw.org)
# - libint-1.1.4 (http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html)
#
PERL     = perl
CC       = cc
CPP      = cpp
FC       = ifort -FR
LD       = ifort
AR       = ar -r
DFLAGS   = -D__INTEL -D__FFTSG -D__FFT3W -D__HAS_NO_ISO_C_BINDING -D__LIBINT
MKLPATH  = /usr/local/shared/intel/mkl/10.1.0.015/lib/em64t
INTLIB   = /usr/local/shared/intel/fce/10.1.015/lib
INCPATH  = /home/el316/include
LIBPATH  = /home/el316/lib
CP2KPATH = /home/el316/cp2koptim
COMFLAGS = $(DFLAGS) -I$(MKLPATH) -I$(INCPATH)
CPPFLAGS = $(COMFLAGS) -C -traditional -P
FCFLAGS  = $(COMFLAGS) -O2 -heap-arrays 64 -unroll -pc64 -fpp -free
LDFLAGS  = $(FCFLAGS) -L/$(LIBPATH) -L/$(INTLIB)
LIBS     = $(MKLPATH)/libmkl_lapack.a \
           $(MKLPATH)/libmkl_em64t.a \
           $(MKLPATH)/libguide.a \
           $(LIBPATH)/libfftw3.a \
           $(CP2KPATH)/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o \
           $(LIBPATH)/libderiv.a \
           $(LIBPATH)/libint.a \
           -lpthread \
           -lstdc++

OBJECTS_ARCHITECTURE = machine_intel.o

# Parallel build / Mek-quake
# The following settings worked for:
# - AMD64 Opteron /Mek-quake
# - SUSE Linux Enterprise Server 10.3 (x86_64)
# - Intel(R) Fortran Compiler for Intel(R) EM64T-based applications, Version 10.1.018
# - Intel MKL 10.1.0.015
# - OpenMPI/Intel 1.3
# - fftw-3.2.1 (ftp.fftw.org)
# - libint-1.1.4 (http://www.files.chem.vt.edu/chem-dept/valeev/software/libint/download.html)
#
PERL     = perl
CC       = cc
CPP      = cpp
FC       = mpif90 -FR
LD       = mpif90
AR       = ar -r
DFLAGS   = -D__INTEL -D__FFTSG -D__FFTW3 -D__parallel -D__BLACS -D__SCALAPACK \
           -D__HAS_NO_ISO_C_BINDING -D__LIBINT
MKLPATH  = /usr/local/shared/intel/mkl/10.1.0.015/lib/em64t
INTLIB   = /usr/local/shared/intel/fce/10.1.015/lib
INCPATH  = /home/el316/include
LIBPATH  = /home/el316/lib
CP2KPATH = /home/el316/cp2koptim2
COMFLAGS = $(DFLAGS) -I$(MKLPATH) -I$(INCPATH)
CPPFLAGS = $(COMFLAGS) -C -traditional -P
FCFLAGS  = $(COMFLAGS) -O2 -heap-arrays 64 -unroll -pc64 -fpp -free
LDFLAGS  = $(FCFLAGS) -L/$(LIBPATH) -L/$(INTLIB)
LIBS     = $(MKLPATH)/libmkl_scalapack_lp64.a \
           $(MKLPATH)/libmkl_lapack.a \
           $(MKLPATH)/libmkl_blacs_openmpi_lp64.a \
           $(MKLPATH)/libmkl_em64t.a \
           $(MKLPATH)/libguide.a \
           $(LIBPATH)/libfftw3.a \
           $(CP2KPATH)/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o \
           $(LIBPATH)/libderiv.a \
           $(LIBPATH)/libint.a \
           -lpthread \
           -lstdc++ 

OBJECTS_ARCHITECTURE = machine_intel.o

The above architecture files should be copied to the "/arch" directory of respectively the cp2koptim and cp2koptim2 folder. Following the naming scheme of the arch files already present in this folder I have chosen to call them "Linux-x86-64-intel.optim" and "Linux-x86-64-intel.optim2" for repectively the serial and parallel arch-file. Both files can also be found in the my public folder (/home/el316/public) on mek-quake. For the final step of the actual compilation you should go to the "/makefiles" folder of the cp2k directory and type: 

make VERSION=optim ARCH=Linux-x86-64-intel > compilations.inf
make VERSION=optim2 ARCH=Linux-x86-64-intel > compilationp.inf

If everything went well, this should create the working executables "cp2k.optim" or "cp2k.optim2" in the "/exe/Linux-x86-64-intel" folder of cp2k. Note that redirecting the output to the files "compilations.inf" and "compilationp.inf" makes sure that only the warnings and error messages are shown on screen. Some warnings during compilation time are perfectly normal. In case of a first time compilation of the code, it is advisable to leave out the optimizations and replace the FCFLAGS variable in the architecture file by: "$(COMFLAGS) -O0 -heap-arrays 64". The -heap-arrays 64 option should always be there, otherwise you will almost certainly encounter segmentation faults during exectution of the code.

CP2K and OPTIM

The parallel CP2K code can now also be used together with the OPTIM program. For this a little program (cp2kdriver.f90) has to be compiled, which will make it possible for the OPTIM code to call the CP2K program. The program can be found in my public folder on mek-quake (/home/el316/public). Just copy the cp2kdriver.f90 program to your home folder and type as one line with of course all el316's substituted by your userid: 

mpif90 -o /home/el316/cp2koptim2/exe/Linux-x86-64-intel/cp2kdriver cp2kdriver.f90 -FR -O0 -heap-arrays 64 
-L/home/el316/cp2koptim2/lib/Linux-x86-64-intel/optim2/ -lcp2k_lib -lcp2k_base_lib -lcp2k_fft_lib 
-L/usr/local/shared/intel/mkl/10.1.0.015/lib/em64t -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_em64t -lguide -L/home/el316/lib/ 
-lfftw3 /home/el316/cp2koptim2/tools/hfx_tools/libint_tools/libint_cpp_wrapper.o -lderiv -lint -lstdc++ -pthread

It should create a working executable called cp2kdriver in the same folder as the CP2K executable "cp2k.optim2". If all compilations above were successful, you should now be able to make combined OPTIM/CP2K runs, by specifying the "CP2K" keyword in the odata file (see OPTIM documentation for its exact use).

Workstations

Graphviz

The graphviz package contains both the dot language needed and several parsers, which produce graphs in different formats (using some rather clever maths...).

It is not installed in the sector, but is easy to compile (all of its dependencies are already present).

I downloaded the tarball from graphviz's download page and compiled using: 

./configure --prefix=$HOME/local && make && make install

It takes a while(!) to compile and this installs the executables to $HOME/local/bin. Various other files are installed to the sub-directories of $HOME/local, including manpages. You can extend your MANPATH environment variable to view these easily (or use the appropriate option with man).

Please note that due to a bug in libtool (fixed upstream) it is vital that the prefix path does *not* end with a /. If you do, it results in an error in the make install step. To fix this, run a make clean and then run configure again with the correct path. (That was fun to figure out!)

makedepf90

makedepf90 is a very useful program which analyses a set of fortran files and produces the dependency list of modules and include files for use in a makefile. Neatly, you can add a target to your makefile to automatically produce the dependencies (see the manpage for an example).

I have a directory ${HOME}/local where I put utilities I compile myself. I did the following: 

jss43@discovery:~/local/src$ tar xvzf makedepf90-2.8.8.tar.gz
$ cd makedepf90-2.8.8/
$./configure --prefix=${HOME}/local --mandir=${HOME}/local/share/man
$ make && make install

This compiles it. I used a non-default location for the man page as I already had some local manpages in the ${HOME}/local/share/man directory.

You then need to add it to your path. If you use the same layout as me, then add to your .bashrc: 

export PATH=${PATH}:${HOME}/local/bin
export MANPATH=${MANPATH}:${HOME}/local/share/man

You can now run makedepf90 and see the manpage from the command line.

It is quite easy to write a simple script which uses makedepf90 to generate a makefile for different compilers/optimisation levels as required and is suited for a wide variety of projects (ie is rather independent of the codebase).

gromacs on Ubuntu workstation

The gcc on the Ubuntu machines needs an extra flag to build a working gromacs 4.0.5. 

CFLAGS="-U_FORTIFY_SOURCE" ./configure --enable-float --enable-threads --prefix=/path/to/whereever
make
make install
Retrieved from "http://wikis.ch.cam.ac.uk/cuc3/wiki/index.php?title=Useful_Makefiles&oldid=1762"