Difference between revisions of "Programming/Python"
m (Pysdlb moved page Programming/Python to Python without leaving a redirect) |
m |
||
| Line 7: | Line 7: | ||
When programming with Python in a HPC environment you will need to change the first line from: | When programming with Python in a HPC environment you will need to change the first line from: | ||
| − | <pre style="background-color: #C8C8C8; color: black; border: 2px solid | + | <pre style="background-color: #C8C8C8; color: black; border: 2px solid #C8C8C8; font-family: monospace, sans-serif;"> |
#!/usr/bin/python | #!/usr/bin/python | ||
</pre> | </pre> | ||
| Line 13: | Line 13: | ||
To | To | ||
| − | <pre style="background-color: #C8C8C8; color: black; border: 2px solid | + | <pre style="background-color: #C8C8C8; color: black; border: 2px solid #C8C8C8; font-family: monospace, sans-serif;"> |
#!/usr/bin/env python | #!/usr/bin/env python | ||
</pre> | </pre> | ||
| Line 19: | Line 19: | ||
==== Python example ==== | ==== Python example ==== | ||
| − | <pre style="background-color: #C8C8C8; color: black; border: 2px solid | + | <pre style="background-color: #C8C8C8; color: black; border: 2px solid #C8C8C8; font-family: monospace, sans-serif;"> |
#!/usr/bin/env python | #!/usr/bin/env python | ||
| Line 72: | Line 72: | ||
== Usage Examples == | == Usage Examples == | ||
| − | == Batch example == | + | === Batch example === |
| Line 81: | Line 81: | ||
#SBATCH -N 1 | #SBATCH -N 1 | ||
#SBATCH --ntasks-per-node 20 | #SBATCH --ntasks-per-node 20 | ||
| − | |||
#SBATCH -o %N.%j.%a.out | #SBATCH -o %N.%j.%a.out | ||
#SBATCH -e %N.%j.%a.err | #SBATCH -e %N.%j.%a.err | ||
| Line 91: | Line 90: | ||
module purge | module purge | ||
module add anaconda/4.0 | module add anaconda/4.0 | ||
| − | module | + | module add openmpi/gcc/1.10.2 |
export I_MPI_DEBUG=5 | export I_MPI_DEBUG=5 | ||
Revision as of 17:08, 7 February 2017
Contents
Programming Details
Python is a widely used high-level programming language used for general-purpose programming.
An interpreted language, Python has a design philosophy which emphasizes code readability (notably using whitespace indentation to delimit code blocks rather than curly braces or keywords), and a syntax which allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or JAVA
When programming with Python in a HPC environment you will need to change the first line from:
#!/usr/bin/python
To
#!/usr/bin/env python
Python example
#!/usr/bin/env python
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
if rank == 0:
data = {'key1' : [7, 2.72, 2+3j],
'key2' : ( 'abc', 'xyz')}
else:
data = None
data = comm.bcast(data, root=0)
if rank != 0:
print ("data is %s and %d" % (data,rank))
else:
print ("I am master\n")
Note due to the limited of the Python interpreter (GIL) there is no point to use threads for CPU intensive tasks in Python. You need either multiprocessing or use C extensions that release GIL during computations e.g., some of numpy functions, example.
Modules Available
The following modules are available:
Python provided by the Python Software foundation
- module add python/2.7.11
- module add python/3.5.1
Anaconda python Anaconda is the open data science platform powered by Continuum
- module add python/anaconda/4.0/2.7
- module add python/anaconda/4.0/3.5
- module add python/anaconda/4.1.1/2.7
Compilation
Python is byte compiled at runtime by typing for example
[username@login01 ~]$ python myPython.py
Usage Examples
Batch example
#!/bin/bash #SBATCH -J compute-single-node #SBATCH -N 1 #SBATCH --ntasks-per-node 20 #SBATCH -o %N.%j.%a.out #SBATCH -e %N.%j.%a.err #SBATCH -p compute #SBATCH --exclusive echo $SLURM_JOB_NODELIST module purge module add anaconda/4.0 module add openmpi/gcc/1.10.2 export I_MPI_DEBUG=5 export I_MPI_FABRICS=shm:tmi export I_MPI_FALLBACK=no mpirun python broadcast.py
[username@login01 ~]$ sbatch python-demo.job Submitted batch job 289572