Getting Started with cerebro - Revision history

Cak49 at 19:18, 27 July 2021

2021-07-27T19:18:57Z

← Older revision		Revision as of 19:18, 27 July 2021
Line 47:		Line 47:
	source /home/maf63/code/qcsetup-general.bash		source /home/maf63/code/qcsetup-general.bash
	source ~/.slurmrc</pre>		source ~/.slurmrc</pre>
	to your <code>.bashrc</code> file, then do <code>source ~/.bashrc</code>. Doing <code>which qchem</code> should give <code>/sharedscratch/~~cbh31~~/~~code/qchem5/qc521~~/bin/qchem</code>.		to your <code>.bashrc</code> file, then do <code>source ~/.bashrc</code>. Doing <code>which qchem</code> should give <code>/sharedscratch/maf63/qchem-general/bin/qchem</code>.

	Once you have Q-Chem set up, you'll be able to submit jobs. Here is a generic submit file for Q-Chem on cerebro:		Once you have Q-Chem set up, you'll be able to submit jobs. Here is a generic submit file for Q-Chem on cerebro:

Cak49 at 18:27, 27 July 2021

2021-07-27T18:27:36Z

@@ Line 41: / Line 41: @@
 =Using Q-Chem=
-If you have not used Q-Chem on cerebro before, first check if you have access with <code>which qchem</code>.
+If you have not used Q-Chem on cerebro before, first check whether you have access with <code>which qchem</code>. If that does not return anything, you'll need to add a few lines to your <code>.bashrc</code> file. For Q-Chem 5.3, add
 Once you have Q-Chem set up, you'll be able to submit jobs. Here is a generic submit file for Q-Chem on cerebro:
@@ Line 80: / Line 86: @@
 <pre>sbatch $old $test submit.qcmagic</pre>
-You can include multiple commands in a submit file.
+You can include multiple commands in a submit file, e.g.:

Cak49: Added information about getting started with cerebro

2021-07-25T00:03:06Z

Added information about getting started with cerebro

New page

=Overview=
cerebro is a compute cluster. Unlike on the workstations, jobs are not run directly from the command line (e.g. just doing <code>qchem file.in file.out</code>). Instead, jobs are sent to a queue which is managed by <code>SLURM</code>. To submit a job, you need to write a submit file with information about your job then submit it to the queue using the <code>sbatch</code> command. Jobs create a <code>slurm-<jobid>.out</code> file which contains their terminal output. If you think something might have gone wrong with a job (e.g. something crashed, it ran out of time, etc...) the <code>slurm</code> file is usually a good place to start looking for the issue.

General information about cerebro is available on the department website [https://www.ch.cam.ac.uk/computing/cerebro-user-notes here].

=Helpful Commands=
==<code>sbatch</code>==
<code>sbatch</code> is used to submit a job to the queue, e.g.
<pre>sbatch $old $long submit_file</pre>

You can specify whether a job runs on the old nodes (12 CPUs max) or the new nodes (16 CPUs max) using <code>$old</code> and <code>$new</code>, respectively.

You can also specify the partition on which you would like the job to run using one of four options:
{| class="wikitable"
! style="width: 8ex;" |'''Partition''' || style="width: 14ex;" | '''sbatch Option''' || style="width: 10ex;" | '''Time Limit'''|| style="width: 45ex;" | '''Other Notes'''
|-
| TEST || $test || 4 hours || highest priority partition - jobs should run right away
|-
| LONG || $long || 48 hours || default partition
|-
| XLONG || $xlong || 7 days || 96 core limit
|-
| XXLONG || $xxlong || 30 days || 56 core limit
|}
More information about SLURM on cerebro is available on the department website [https://www.ch.cam.ac.uk/computing/slurm-usage here] and [https://www.ch.cam.ac.uk/computing/cerebro-queues-and-parallel-job-configuration here], and general information about SLURM can be found [https://slurm.schedmd.com/overview.html here].

==<code>squeue</code>==
To see all the jobs in the queue, do
<pre>squeue</pre>

To see all the jobs that you have queued or running, do
<pre>squeue -u <your-crsid></pre>

==<code>scancel</code>==
To cancel a job, do
<pre>scancel <JOBID></pre>
'''Make sure you have the right job ID!'''

More information on queuing is available [https://wikis.ch.cam.ac.uk/thom/wiki/index.php/Queuing here].

=Using Q-Chem=
If you have not used Q-Chem on cerebro before, first check if you have access with <code>which qchem</code>.

Once you have Q-Chem set up, you'll be able to submit jobs. Here is a generic submit file for Q-Chem on cerebro:
<pre>#!/bin/bash

# Set default outfile if not defined
outfile=${outfile:-qchem.out}
scratch=${scratch:-qchem.scratch}
export OMP_NUM_THREADS=$SLURM_CPUS_ON_NODE

# Run the calculation
echo "Using Q-Chem: " $QC
rm -rf $QCSCRATCH/$scratch
cp -r $scratch $QCSCRATCH/$scratch
qchem -nt $SLURM_CPUS_ON_NODE -save $infile $outfile $scratch

# Recover the scratch directory (optional)
# cp -r $QCSCRATCH/$scratch/* $scratch/</pre>

To use this submit file, copy it to <code>submit.qchem</code>, then do
# <code>export infile=<your_qchem_input_file></code>
# <code>export outfile=<name_of_qchem_output_file></code>
# <code>export scratch=<name_of_a_scratch_directory></code>
# <code>sbatch <your options> submit.qchem</pre></code>

=Using QCMagic=
If you have not used QCMagic on cerebro before, you will need to install it first - instructions are available [https://wikis.ch.cam.ac.uk/thom/wiki/index.php/QCMagic#Installation here].

Here is a submit file template for a QCMagic job:
<pre>#!/bin/bash

export OMP_NUM_THREADS=$SLURM_CPUS_ON_NODE

# Run the calculation
runscanSurface.py -p 12 -L --read-minima=1 etc... </pre>

If you copy this information to a file called <code>submit.qcmagic</code>, you can then submit it with
<pre>sbatch $old $test submit.qcmagic</pre>

You can include multiple commands in a submit file.