Useful PBS scripts - Revision history

import>Mm695 at 12:40, 19 March 2008

2008-03-19T12:40:07Z

← Older revision		Revision as of 12:40, 19 March 2008
Line 86:		Line 86:
	mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}		mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}



	for node in $nodes		for node in $nodes
	do		do

import>Mm695 at 12:39, 19 March 2008

2008-03-19T12:39:38Z

← Older revision		Revision as of 12:39, 19 March 2008
Line 52:		Line 52:
	== CPMD runscript if several nodes are needed ==		== CPMD runscript if several nodes are needed ==

	# PBS -q s32		#PBS -q s32
	# PBS -l walltime=18:00:00		#PBS -l walltime=18:00:00
	# PBS -l nodes=8:ppn=4		#PBS -l nodes=8:ppn=4

	HERE=$PBS_O_WORKDIR # the directory from which the script was submitted		HERE=$PBS_O_WORKDIR # the directory from which the script was submitted
Line 86:		Line 86:
	mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}		mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}


	This copying back (see below) does not work properly, I'm happy for any suggestions. For the moment I leave out all rsh and run it directly from HERE, is just as good and less complicated --Martina

	for node in $nodes		for node in $nodes
	do		do
	rsh $node 'mv ${SCRATCH}/* ${HERE}'		rsh $node "mv ${SCRATCH}/* ${HERE}"
	rsh $node 'rm -f ${SCRATCH}/*' # See note below		rsh $node "rm -f ${SCRATCH}/*" # See note below
	rsh $node 'rmdir /scratch/mm695/$file' # about post-job tidying.		rsh $node "rmdir /scratch/mm695/$file" # about post-job tidying.
	done		done

import>Mm695 at 09:44, 19 March 2008

2008-03-19T09:44:52Z

← Older revision		Revision as of 09:44, 19 March 2008
Line 86:		Line 86:
	mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}		mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}


			This copying back (see below) does not work properly, I'm happy for any suggestions. For the moment I leave out all rsh and run it directly from HERE, is just as good and less complicated --Martina

	for node in $nodes		for node in $nodes
	do		do

import>Jss43: /* CPMD runscript if several nodes are needed */

2008-03-17T21:07:31Z

CPMD runscript if several nodes are needed

← Older revision		Revision as of 21:07, 17 March 2008
Line 96:		Line 96:


	I've had problems in the past in with large CPMD RESTART files not being correctly copied back (worse than failure: they get corrupted or are only partially copied with no error message). This causes many "interesting" issues when I attempted to use the RESTART files for new calculations. For this reason I prefer not to do post-job tidying until I've checked things are copied back correctly. Instead, I periodically (rather, have a script to) tidy up the scratch space on nodes.		I've had problems in the past in with large CPMD RESTART files not being correctly copied back (worse than failure: they get corrupted or are only partially copied with no error message). This causes many "interesting" issues when I attempted to use the RESTART files for new calculations. For this reason I prefer not to do post-job tidying until I've checked things are copied back correctly. Instead, I periodically (rather, have a [[James_Spencer#Cleaning_up_scratch\|script]] to) tidy up the scratch space on nodes.
	--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]		--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]

import>Jss43: /* CPMD runscript if several nodes are needed */

2008-03-17T16:50:55Z

CPMD runscript if several nodes are needed

← Older revision		Revision as of 16:50, 17 March 2008
Line 56:		Line 56:
	# PBS -l nodes=8:ppn=4		# PBS -l nodes=8:ppn=4

			HERE=$PBS_O_WORKDIR # the directory from which the script was submitted
	HERE=/home/mm695/whatever
	file=dho2498_singlePoint		file=dho2498_singlePoint
	inpfile=${file}.inp		inpfile=${file}.inp
	outfile=${file}.out		outfile=${file}.out
	SCRATCH=/scratch/mm695/$file		#SCRATCH=/scratch/mm695/$file
			SCRATCH=/scratch/mm695/job-$PBS_JOBID # generate a unique directory in scratch.
	nodes=`cat $PBS_NODEFILE \| uniq`		nodes=`cat $PBS_NODEFILE \| uniq`
	for node in $nodes		for node in $nodes
	do		do
	rsh $node "rm -f $SCRATCH/*"		rsh $node "rm -f $SCRATCH/*" # Not needed if using
	rsh $node "rmdir $SCRATCH"		rsh $node "rmdir $SCRATCH" # a unique scratch directory
	rsh $node "mkdir $SCRATCH"		rsh $node "mkdir $SCRATCH" # for each job.
	rsh $node "cp ${HERE}/gromos* $SCRATCH"		rsh $node "cp ${HERE}/gromos* $SCRATCH"
	rsh $node "cp ${HERE}/geom_end_of_sim.crd $SCRATCH"		rsh $node "cp ${HERE}/geom_end_of_sim.crd $SCRATCH"
	rsh $node "cp ${HERE}/RESTART $SCRATCH"		rsh $node "cp ${HERE}/RESTART $SCRATCH"
	rsh $node "cp ${HERE}/${inpfile} $SCRATCH"		rsh $node "cp ${HERE}/${inpfile} $SCRATCH"
	done		done
	exe=/home/mm695/SOURCE/cpmd.x		exe=/home/mm695/SOURCE/cpmd.x
	pp=/home/mm695/pseudopot		pp=/home/mm695/pseudopot
Line 83:		Line 84:
	echo		echo

	mvapichwrapper $exe $inpfile $pp > ${~~HERE~~}/${outfile}		mvapichwrapper $exe $inpfile $pp > ${SCRATCH}/${outfile}

	for node in $nodes		for node in $nodes
	do		do
	rsh $node 'mv ${SCRATCH}/* ${HERE}'		rsh $node 'mv ${SCRATCH}/* ${HERE}'
	rsh $node 'rm -f ${SCRATCH}/*'		rsh $node 'rm -f ${SCRATCH}/*' # See note below
	rsh $node 'rmdir /scratch/mm695/$file'		rsh $node 'rmdir /scratch/mm695/$file' # about post-job tidying.
	done		done

Line 96:		Line 97:

	I've had problems in the past in with large CPMD RESTART files not being correctly copied back (worse than failure: they get corrupted or are only partially copied with no error message). This causes many "interesting" issues when I attempted to use the RESTART files for new calculations. For this reason I prefer not to do post-job tidying until I've checked things are copied back correctly. Instead, I periodically (rather, have a script to) tidy up the scratch space on nodes.		I've had problems in the past in with large CPMD RESTART files not being correctly copied back (worse than failure: they get corrupted or are only partially copied with no error message). This causes many "interesting" issues when I attempted to use the RESTART files for new calculations. For this reason I prefer not to do post-job tidying until I've checked things are copied back correctly. Instead, I periodically (rather, have a script to) tidy up the scratch space on nodes.
	I find writing to the home disk on tardis can be incredibly expensive: you might as well write output to scratch and then copy it back.
	wd=$PBS_O_WORKDIR is my friend, and saves much submit script editing.

	--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]		--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]

import>Cen1001: /* CPMD runscript if several nodes are needed */

2008-03-13T18:47:04Z

CPMD runscript if several nodes are needed

← Older revision		Revision as of 18:47, 13 March 2008
Line 100:		Line 100:

	--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]		--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]

			Ahem, yes, the bit with the cat was mine. I usually want to send PBS_NODEFILE through a rather more complex transformation than this one, and in those cases the idiom with cat is more legible. You are right that it adds nothing here! uniq $PBS_NODEFILE would be fine.
			--[[User:cen1001\|Catherine]] 18:47, 13 March 2008 (GMT)

import>Jss43: /* CPMD runscript if several nodes are needed */

2008-03-13T17:47:52Z

CPMD runscript if several nodes are needed

← Older revision		Revision as of 17:47, 13 March 2008
Line 93:		Line 93:

	qstat -f $PBS_JOBID		qstat -f $PBS_JOBID


			I've had problems in the past in with large CPMD RESTART files not being correctly copied back (worse than failure: they get corrupted or are only partially copied with no error message). This causes many "interesting" issues when I attempted to use the RESTART files for new calculations. For this reason I prefer not to do post-job tidying until I've checked things are copied back correctly. Instead, I periodically (rather, have a script to) tidy up the scratch space on nodes.
			I find writing to the home disk on tardis can be incredibly expensive: you might as well write output to scratch and then copy it back.
			wd=$PBS_O_WORKDIR is my friend, and saves much submit script editing.

			--[[User:jss43\|james]] 17:47, 13 March 2008 (GMT) [People who are kind to cats leave them to laze in the sun until they're needed...]

import>Mm695 at 17:42, 12 March 2008

2008-03-12T17:42:00Z

← Older revision		Revision as of 17:42, 12 March 2008
Line 53:		Line 53:

	# PBS -q s32		# PBS -q s32

	# PBS -l walltime=18:00:00		# PBS -l walltime=18:00:00

	# PBS -l nodes=8:ppn=4		# PBS -l nodes=8:ppn=4


	HERE=/home/mm695/whatever		HERE=/home/mm695/whatever

	file=dho2498_singlePoint		file=dho2498_singlePoint

	inpfile=${file}.inp		inpfile=${file}.inp

	outfile=${file}.out		outfile=${file}.out

	SCRATCH=/scratch/mm695/$file		SCRATCH=/scratch/mm695/$file

	nodes=`cat $PBS_NODEFILE \| uniq`		nodes=`cat $PBS_NODEFILE \| uniq`

	for node in $nodes		for node in $nodes
	do		do
Line 80:		Line 72:
	rsh $node "cp ${HERE}/${inpfile} $SCRATCH"		rsh $node "cp ${HERE}/${inpfile} $SCRATCH"
	done		done

	exe=/home/mm695/SOURCE/cpmd.x		exe=/home/mm695/SOURCE/cpmd.x

	pp=/home/mm695/pseudopot		pp=/home/mm695/pseudopot

	cd $SCRATCH		cd $SCRATCH


	# Write out some helpful info to the output file		# Write out some helpful info to the output file

	echo "Starting job $PBS_JOBID"		echo "Starting job $PBS_JOBID"
	echo		echo
Line 94:		Line 82:
	cat $PBS_NODEFILE		cat $PBS_NODEFILE
	echo		echo


	mvapichwrapper $exe $inpfile $pp > ${HERE}/${outfile}		mvapichwrapper $exe $inpfile $pp > ${HERE}/${outfile}


	for node in $nodes		for node in $nodes
	do		do
Line 103:		Line 91:
	rsh $node 'rmdir /scratch/mm695/$file'		rsh $node 'rmdir /scratch/mm695/$file'
	done		done


	qstat -f $PBS_JOBID		qstat -f $PBS_JOBID

import>Mm695 at 17:37, 12 March 2008

2008-03-12T17:37:41Z

← Older revision		Revision as of 17:37, 12 March 2008
Line 48:		Line 48:

	# get our PBS stats		# get our PBS stats
			qstat -f $PBS_JOBID

			== CPMD runscript if several nodes are needed ==

			# PBS -q s32

			# PBS -l walltime=18:00:00

			# PBS -l nodes=8:ppn=4

			HERE=/home/mm695/whatever

			file=dho2498_singlePoint

			inpfile=${file}.inp

			outfile=${file}.out

			SCRATCH=/scratch/mm695/$file

			nodes=`cat $PBS_NODEFILE \| uniq`

			for node in $nodes
			do
			rsh $node "rm -f $SCRATCH/*"
			rsh $node "rmdir $SCRATCH"
			rsh $node "mkdir $SCRATCH"
			rsh $node "cp ${HERE}/gromos* $SCRATCH"
			rsh $node "cp ${HERE}/geom_end_of_sim.crd $SCRATCH"
			rsh $node "cp ${HERE}/RESTART $SCRATCH"
			rsh $node "cp ${HERE}/${inpfile} $SCRATCH"
			done

			exe=/home/mm695/SOURCE/cpmd.x

			pp=/home/mm695/pseudopot

			cd $SCRATCH

			# Write out some helpful info to the output file

			echo "Starting job $PBS_JOBID"
			echo
			echo "PBS assigned me this node:"
			cat $PBS_NODEFILE
			echo

			mvapichwrapper $exe $inpfile $pp > ${HERE}/${outfile}

			for node in $nodes
			do
			rsh $node 'mv ${SCRATCH}/* ${HERE}'
			rsh $node 'rm -f ${SCRATCH}/*'
			rsh $node 'rmdir /scratch/mm695/$file'
			done

	qstat -f $PBS_JOBID		qstat -f $PBS_JOBID

import>Cen1001 at 15:22, 12 March 2008

2008-03-12T15:22:20Z

← Older revision		Revision as of 15:22, 12 March 2008
Line 1:		Line 1:
	If you have put some effort into writing a PBS job script for a particular type of job, please consider adding it here.		If you have put some effort into writing a PBS job script for a particular type of job, please consider adding it here.

			== Job script with signal handler ==

	# This is an example PBS job script that can carry out an action to clean up		# This is an example PBS job script that can carry out an action to clean up