Difference between revisions of "Programming/Python"

From HPC
Jump to: navigation , search
m (Pysdlb moved page Programming/Python to Python without leaving a redirect)
m
Line 43: Line 43:
  
 
<strong>Note</strong> due to the limited of the Python interpreter (GIL) there is no point to use threads for CPU intensive tasks in Python. You need either multiprocessing or use C extensions that release GIL during computations e.g., some of numpy functions, example.
 
<strong>Note</strong> due to the limited of the Python interpreter (GIL) there is no point to use threads for CPU intensive tasks in Python. You need either multiprocessing or use C extensions that release GIL during computations e.g., some of numpy functions, example.
 +
 +
==== Modules Available ====
 +
 +
The following modules are available:
 +
 +
* module load python/2.7.11
 +
* module load python/3.5.1
 +
 +
* module load python/anaconda/4.0/2.7
 +
* module load python/anaconda/4.0/3.5
 +
* module load python/anaconda/4.1.1/2.7
 +
 +
==== Compilation ====
 +
 +
Python is byte compiled at runtime by typing for example
 +
 +
<pre style="background-color: #C8C8C8; color: black; border: 2px solid black; font-family: monospace, sans-serif;">
 +
 +
python myPython.py
 +
 +
</pre>
 +
 +
  
 
== Usage Examples ==
 
== Usage Examples ==

Revision as of 14:25, 31 January 2017

Programming Details

Python is a widely used high-level programming language used for general-purpose programming.

An interpreted language, Python has a design philosophy which emphasizes code readability (notably using whitespace indentation to delimit code blocks rather than curly braces or keywords), and a syntax which allows programmers to express concepts in fewer lines of code than possible in languages such as C++ or Java. scale.

When programming with Python in a HPC environment you will need to change the first line from:

 #!/usr/bin/python

To

 #!/usr/bin/env python

Python example


#!/usr/bin/env python

from mpi4py import MPI

comm = MPI.COMM_WORLD
rank = comm.Get_rank()

if rank == 0:
   data = {'key1' : [7, 2.72, 2+3j],
           'key2' : ( 'abc', 'xyz')}
else:
   data = None

data = comm.bcast(data, root=0)

if rank != 0:
        print ("data is %s and %d" % (data,rank))
else:
        print ("I am master\n")

Note due to the limited of the Python interpreter (GIL) there is no point to use threads for CPU intensive tasks in Python. You need either multiprocessing or use C extensions that release GIL during computations e.g., some of numpy functions, example.

Modules Available

The following modules are available:

  • module load python/2.7.11
  • module load python/3.5.1
  • module load python/anaconda/4.0/2.7
  • module load python/anaconda/4.0/3.5
  • module load python/anaconda/4.1.1/2.7

Compilation

Python is byte compiled at runtime by typing for example


python myPython.py


Usage Examples

Batch example

 
#!/bin/bash
#SBATCH -J compute-single-node
#SBATCH -N 1
#SBATCH --ntasks-per-node 20
#SBATCH -D /home/pysdlb/PYTHON
#SBATCH -o %N.%j.%a.out
#SBATCH -e %N.%j.%a.err
#SBATCH -p compute
#SBATCH --exclusive

echo $SLURM_JOB_NODELIST

module purge
module load anaconda/4.0
module load openmpi/gcc/1.10.2

export I_MPI_DEBUG=5
export I_MPI_FABRICS=shm:tmi
export I_MPI_FALLBACK=no

mpirun python broadcast.py


[username@login01 ~]$ sbatch python-demo.job
Submitted batch job 289572