View source for Quickstart/Batch Jobs

==What is a batch job?==
Having been introduced to the Slurm scheduler in [[Quickstart/Slurm|Slurm]] and then one of the ways of using Viper with Interactive sessions in [[Quickstart/Interactive|Interactive]] the next way of making use of an HPC system like Viper is by submitting jobs to run automatically without interaction when the resource becomes available. We call these '''batch jobs'''.

In order to run a batch job, we need to provide Slurm with information about what we want to do. We do this via a job submission script, which is sort of like a recipe for the job. 

==What is a Slurm script?==
The submission script is a text file that provides information to Slurm about the task you are running so that it can be allocated to the appropriate resource, and sets up the environment so the task can run. A minimal submission script has three main components:

* A set of directives that provides Slurm with some high-level information such as what resource is required, a job name, how long the task should run for and where to log any output that would normally be shown on the screen.
* Information about how the job environment should be set up, for example, what application [[Quickstart/Using Modules|using modules]] should be loaded.
* The actual command(s) that need to be run.

==Slurm Scripts==
For ease of use please give your .job files a descriptive name. While you can specify the directory, it is easier to create and submit the script from the working directory.
===#!/bin/bash===
All Slurm scripts must start with ''#!/bin/bash'' (called a shebang).
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
</pre>
===#SBATCH===
The following #SBATCH directives are essential to a Slurm script, there are other directives you can include which can be found in [[FurtherTopics/Advanced Batch Jobs|further topics]].

Please note these are case sensitive.
====#SBATCH -J ==== 
''<nowiki>#SBATCH</nowiki> -J jobname'' tells Slurm the name of your job - this is what you will see when you run [[Quickstart/Slurm #squeue| squeue]].
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
</pre>

====#SBATCH -n====
''<nowiki>#SBATCH</nowiki> -n Number'' tells Slurm how many cores you would like to use.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
</pre>
====#SBATCH -o====
''<nowiki>#SBATCH</nowiki> -o %N.%j.out'' tells Slurm where you would like standard output to be written. %N is the node number and %j is the assigned job number. Though this can be named anything it must end in .out.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
</pre>
====#SBATCH -e====
''<nowiki>#SBATCH</nowiki> -e %N.%j.err'' tells Slurm where you would like standard error to be written. %N is the node number and %j is the assigned job number. Though this can be named anything it must end in .err.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
</pre>
====#SBATCH -p====
''<nowiki>#SBATCH</nowiki> -p Partition'' tells Slurm what partition you would like to use: compute/highmem/gpu.  Please note [[FurtherTopics/Advanced Batch Jobs #highmem|highmem]] and [[FurtherTopics/Advanced Batch Jobs#gpu|gpu]] jobs have additional #SBATCH requirements. 
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
</pre>

====#SBATCH -t====
''<nowiki>#SBATCH</nowiki> -t=HH:MM:SS'' tells Slurm how long to run the job for the job will end automatically when finished or it will be terminated if it takes longer than the specified time.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
#SBATCH -t=02:30:00
</pre>

====#SBATCH --exclusive====
For demanding tasks such as [[Applications/Matlab| Matlab]] please use exclusive nodes. If a job or session hasn't requested an appropriate resource, this multithreading can cause contention for CPU resources and negatively impact other users. This can be done by using ''#SBATCH -N 1'' and '' #SBATCH --exclusive''.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -N 1
#SBATCH --exclusive
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
#SBATCH -t=02:30:00
</pre>

===Environment Setup===
The next part of the script sets up the environment in which the job will run in.
====module purge====
''module purge'' unloads any previously loaded modules, it is good practice to start with this section
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
#SBATCH -t=02:30:00

module purge
</pre>
====module add modulename====
Just like you would in an interactive session add any modules you need.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
#SBATCH -t=02:30:00

module purge
module add python/anaconda/202111/3.9
</pre>

===Commands===
This section is where you tell Slurm what you would like it to run - mostly the same as you would in an interactive session. There are some [[#Example Batch Jobs| examples]] at the bottom of this page of basic batch jobs with what commands you may run or you can check the modules pages on [[Modules| Modules Available]] for the specifics of what commands to use if you are not sure.
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J helloWorld
#SBATCH -n 4
#SBATCH -o %N.%j.out
#SBATCH -e %N.%j.err
#SBATCH -p compute
#SBATCH -t=02:30:00

module purge
module add python/anaconda/202111/3.9

python helloWorld.py
</pre>
The above batch script is now complete; this will use a compute node with 4 cores for a maximum of 2 hours 30 minutes and will load the anaconda module before running the python file helloWorld.py.

==How to submit a batch job to slurm==
Submitting a job is easy! Just run sbatch jobscript.job.
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch PythonTest.job
Submitted batch job 289522
</pre>
You will now be able to see the status of your batch job by running [[Quickstart/Slurm #squeue -u username | squeue -u your_username]].

==Example Batch Jobs==
===R===
R should be used on an exclusive node: this is done by the #SBATCH directives ''#SBATCH -N 1'' and ''#SBATCH --exclusive''
<pre class="mw-collapsible mw-collapsed" style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J My_R_job         # Job name, you can change it to whatever you want
#SBATCH -N 1                # Number of nodes 
#SBATCH -o %N.%j.out        # Standard output will be written here
#SBATCH -e %N.%j.err        # Standard error will be written here
#SBATCH -p compute          # Slurm partition, where you want the job to be queued 
#SBATCH -t=01:00:00         # Max time job runs for 1 hour
#SBATCH --exclusive         # Run on one node without any other users

 
module purge
module add R/4.0.2
 
R CMD BATCH Random.R output.data     #The output from the R interpreter will appear in the file output.data
</pre>

===Matlab===
Matlab requires an exclusive node: this is done by the #SBATCH directives ''#SBATCH -N 1'' and ''#SBATCH --exclusive''
<pre class="mw-collapsible mw-collapsed" style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J MATLAB           # Job name, you can change it to whatever you want
#SBATCH -N 1                # Number of nodes (for Matlab should be always one)
#SBATCH -o %N.%j.out        # Standard output will be written here
#SBATCH -e %N.%j.err        # Standard error will be written here
#SBATCH -p compute          # Slurm partition, where you want the job to be queued
#SBATCH -t=01:00:00         # Max time job runs for
#SBATCH --exclusive         # Run on one node without any other users
 
module purge
module add matlab/2016a
 
matlab -nodisplay -nojvm -nodesktop -nosplash -r my_matlab_m_file
</pre>

===Python Virtual Environment===
<pre class="mw-collapsible mw-collapsed" style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J matrixMulti      # Job name, you can change it to whatever you want
#SBATCH -n 1                # Number of cores = 1
#SBATCH -o %N.%j.out        # Standard output will be written here
#SBATCH -e %N.%j.err        # Standard error will be written here
#SBATCH -p compute          # Slurm partition, where you want the job to be queued 
#SBATCH -t=01:00:00         # Max time job runs for 1 hour
#SBATCH --exclusive         # Run on one node without any other users

module purge
module add python/anaconda/202111/3.9

source activate /home/<user>/.conda/envs/numpyenv
python matrixMultiplication.py
</pre>

===Openfoam===
Openfoam should be used on an exclusive node: this is done by the #SBATCH directives ''#SBATCH -N 1'' and ''#SBATCH --exclusive''
<pre class="mw-collapsible mw-collapsed" style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J openfoamExample  # Job name, you can change it to whatever you want
#SBATCH -N 1                # Number of nodes 
#SBATCH -o %N.%j.out        # Standard output will be written here
#SBATCH -e %N.%j.err        # Standard error will be written here
#SBATCH -p compute          # Slurm partition, where you want the job to be queued
#SBATCH -t=01:00:00         # Max time job runs for
#SBATCH --exclusive         # Run on one node without any other users

module add openfoam/4.0

export I_MPI_DEBUG=5
export I_MPI_FABRICS=shm:tmi
export I_MPI_FALLBACK=no

mpirun -n 28 -cpu_bind=cores interFoam -parallel

</pre>

==More detailed scripts==
For more information on batch scripts visit: [[FurtherTopics/Advanced Batch Jobs| Advanced Batch Jobs]]. This includes information on using high memory and GPU nodes, resource management, how to use multiple nodes, and node reservations




[[Main Page #Quickstart|Back]]   /   [[Quickstart/Data Management| Next (Data Management)]]