Programming/OpenMP - Revision history

Pysdlb at 10:37, 9 December 2022

2022-12-09T10:37:39Z

Pysdlb: /* Next Steps */

2022-11-16T11:09:40Z

‎Next Steps

Pysdlb: /* Navigation */

2022-11-16T11:09:27Z

‎Navigation

Pysdlb: /* Next Steps */

2022-11-16T11:09:16Z

‎Next Steps

Pysdlb at 12:20, 2 November 2022

2022-11-02T12:20:05Z

Pysdlb at 12:19, 2 November 2022

2022-11-02T12:19:24Z

Pysdlb at 12:18, 2 November 2022

2022-11-02T12:18:59Z

Pysdlb at 12:18, 2 November 2022

2022-11-02T12:18:04Z

Pysdlb at 14:50, 9 June 2022

2022-06-09T14:50:57Z

Pysdlb at 09:07, 24 May 2021

2021-05-24T09:07:32Z

@@ Line 5: / Line 5: @@
 The programming model for shared memory is based on the notion of threads:
-* Threads are like processes, except that threads can share memory with each other (as well as have private memory)
+* Threads are like processes, except that threads can share the memory with each other (as well as have private memory)
 * Shared data can be accessed by all threads
 * Private data can only be accessed by the owning thread
@@ Line 40: / Line 40: @@
 ==Programming model==
-Within the idea of the shared memory model, we use the idea of '''threads''' which can share memory with all the other threads. These also have the following characteristics:
+Within the idea of the shared memory model, we use the idea of '''threads''' which can share the memory with all the other threads. These also have the following characteristics:
 * Private data can only be accessed by the thread owning it
@@ Line 386: / Line 386: @@
 For C
-[username@login01 ~]$  module add gcc/8.2.0
+[username@login01 ~]$  module add gcc/10.2.0
 [username@login01 ~]$  gcc -o test2 -fopenmp test2.c
 For Fortran
-[username@login01 ~]$  module add gcc/8.2.0
+[username@login01 ~]$  module add gcc/10.2.0
 [username@login01 ~]$  gfortran -o test2 -fopenmp test2.c
@@ Line 399: / Line 399: @@
 The following modules are available for OpenMP:
-* module add gcc/8.2.0 (GNU compiler)
+* module add gcc/10.2.0 (GNU compiler)
 * module add intel/2018 (Intel compiler)
 {|
@@ Line 427: / Line 427: @@
 module purge
-module add gcc/8.2.0
+module add gcc/10.2.0
 export I_MPI_DEBUG=5

@@ Line 524: / Line 524: @@
 * [http://fortranwiki.org/fortran/show/OpenMP http://fortranwiki.org/fortran/show/OpenMP]
 * [https://www.youtube.com/watch?v=nE-xN4Bf8XI&list=PLLX-Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG 'Youtube - Intel Training videos']
 * [https://en.wikipedia.org/wiki/Open_MPI https://en.wikipedia.org/wiki/Open_MPI]

← Older revision		Revision as of 11:09, 16 November 2022
Line 533:		Line 533:

	{{Librariespagenav}}		{{Librariespagenav}}
−
−	~~==Navigation==~~
−
−	* [[Main_Page\|Home]]
−	* [[Applications\|Application support]]
−	* [[General\|General]]
−	~~* [[Programming\|Programming support]] *~~

@@ Line 524: / Line 524: @@
 * [http://fortranwiki.org/fortran/show/OpenMP http://fortranwiki.org/fortran/show/OpenMP]
 * [https://www.youtube.com/watch?v=nE-xN4Bf8XI&list=PLLX-Q6B8xqZ8n8bwjGdzBJ25X2utwnoEG 'Youtube - Intel Training videos']
 * [https://en.wikipedia.org/wiki/Open_MPI https://en.wikipedia.org/wiki/Open_MPI]
@@ Line 532: / Line 531: @@
 * [[programming/Fortran|Fortran Programming]]
 * [[programming/Python|Python Programming]]
 ==Navigation==

@@ Line 449: / Line 449: @@
 This programming model runs on one node but can be programmed as a hybrid model as well with MPI.
-An application built with the hybrid model of parallel programming can run on a computer cluster using both OpenMP and Message Passing Interface (MPI), such that OpenMP is used for parallelism within a (multi-core) node while MPI is used for parallelism between nodes. There have also been efforts to run OpenMP on software distributed shared memory systems.
+An application built with the hybrid model of parallel programming can run on a computer cluster using both OpenMP and Message Passing Interface (MPI), such that OpenMP is used for parallelism within a (multi-core) node while MPI is used for parallelism between nodes. There have also been efforts to run OpenMP on software-distributed shared memory systems.
 Four possible performance reasons for mixed OpenMP/MPI codes:
@@ Line 459: / Line 459: @@
-==openMP pragmas==
+===openMP pragmas===
 {| class="wikitable"

← Older revision		Revision as of 12:19, 2 November 2022
Line 516:		Line 516:


−	=== Next Steps ===	+	== Next Steps ==

@@ Line 40: / Line 40: @@
 ==Programming model==
-Within the idea of the shared memory model, we use the idea of '''threads''' which can share memory by all the other threads. These also have the following characteristics:
+Within the idea of the shared memory model, we use the idea of '''threads''' which can share memory with all the other threads. These also have the following characteristics:
 * Private data can only be accessed by the thread owning it
-* Each thread can run simultaneously to other threads but also asynchronously, so we need to be careful of race conditions.
+* Each thread can run simultaneously with other threads but also asynchronously, so we need to be careful of race conditions.
 * Usually we have one thread per processing core, although there may be hardware support for more (e.g. hyper-threading)
@@ Line 52: / Line 52: @@
 Although this makes for a very flexible system we must be very careful about the actions on shared variables that occur in the correct order.
-* e.g. If we access a variable to read on thread 1 before thread 2 has written to it we will cause a program crash, likewise is updates to shared variables are accessed by different threads at the same time, one of the updates may get overwritten.
+* e.g. If we access a variable to read on thread 1 before thread 2 has written to it we will cause a program crash, likewise if updates to shared variables are accessed by different threads at the same time, one of the updates may get overwritten.
 To prevent this happen we must either use variables that are independent of the different threads (ie different parts of an array) or perform some sort of synchronization within the code so different threads get to the same point at the same time.
@@ Line 85: / Line 85: @@
 </pre>
-Lets look at the extra components to make this a parallel threaded program
+Let us look at the extra components to make this a parallel-threaded program
 * We have an openMP include file '''(#include <omp.h>''')
@@ Line 106: / Line 106: @@
 </pre>
-* Not very coherent but remember the threads all executed at different times, that is asynchronously and this is why we must be very careful about communicating between different threads of the same program.
+* Not very coherent but remember the threads all executed at different times, that is asynchronous and this is why we must be very careful about communicating between different threads of the same program.
 ==Second threaded program==
@@ Line 372: / Line 372: @@
 * If you have large private data structures, it is possible to run out of stack space.
 ** the size of the thread stack apart from the master thread can be controlled by the OMP_STACKSIZE environment variable.
-* Writing code that works without OpenMP too. The macro _OPENMP is defined if code is compiled with the OpenMP switch.
+* Writing code that works without OpenMP too. The macro _OPENMP is defined if the code is compiled with the OpenMP switch.
 * Be aware The overhead of executing a parallel region is typically in the tens of microseconds range
 ** depends on the compiler, hardware, no. of threads
@@ Line 526: / Line 526: @@
-=== General ===
+=== Next Steps ===

@@ Line 40: / Line 40: @@
 ==Programming model==
-Within the idea of the shared memory model we use the idea of '''threads''' which can share memory by all the other threads. These also have the following characteristics:
+Within the idea of the shared memory model, we use the idea of '''threads''' which can share memory by all the other threads. These also have the following characteristics:
 * Private data can only be accessed by the thread owning it
-* Each thread can run simultaneous to other threads but also asynchronously, so we need to be careful of race conditions.
+* Each thread can run simultaneously to other threads but also asynchronously, so we need to be careful of race conditions.
 * Usually we have one thread per processing core, although there may be hardware support for more (e.g. hyper-threading)
 ==Thread Synchronization==
-As previously mention threads execute asynchronous;y, which means each thread proceeds through program instruction independently of other threads.
+As previously mention threads execute asynchronously, which means each thread proceeds through program instruction independently of other threads.
-Although this makes for a very flexible system we must be very careful about the actions on shared variables occur in the correct order.
+Although this makes for a very flexible system we must be very careful about the actions on shared variables that occur in the correct order.
 * e.g. If we access a variable to read on thread 1 before thread 2 has written to it we will cause a program crash, likewise is updates to shared variables are accessed by different threads at the same time, one of the updates may get overwritten.
@@ Line 110: / Line 110: @@
 ==Second threaded program==
-Although the previous program is threaded it does not represent a real world example
+Although the previous program is threaded it does not represent a real-world example
 <pre>
@@ Line 129: / Line 129: @@
 Here each thread executes the same code independently, the only thing different is the omp thread ID is also passed to each call of pooh(ID,A).
-All threads again wait of the end of the closing parallel region brace to finish before proceeding (i.e. a synchronization barrier).
+All threads again wait for the end of the closing parallel region brace to finish before proceeding (i.e. a synchronization barrier).
-In this program we always expect 4 threads to be given to us by the underlying operating system. Unfortunately this may not happen and we will allocated what the scheduler is prepared to offer. This could cause us serious program difficulties if we rely on a fixed number of threads every time.
+In this program we always expect 4 threads to be given to us by the underlying operating system. Unfortunately, this may not happen and we will be allocated what the scheduler is prepared to offer. This could cause us serious program difficulties if we rely on a fixed number of threads every time.
 We must call the  openMP library (at runtime) how many threads we actually got, this is done with the following code:
@@ Line 161: / Line 161: @@
 ==Parallel loops==
-Loops are the main source of parallelism in many applications. If the iterations of a loop are independent (can be done in any order) then we can share out the iterations between different threads. openMP has a native call to do this efficiently:
+Loops are the main source of parallelism in many applications. If the iterations of a loop are independent (can be done in any order) then we can share out the iterations between different threads. OpenMP has a native call to do this efficiently:
@@ Line 175: / Line 175: @@
 </pre>
-This is a much neater way and allows the compiler to perform optimizations automatically (unless otherwise stated). The variable loop is made 'private' by each thread by default. Also all threads have to wait at the end of the parallel loop before proceeding past the end of this region.
+This is a much neater way and allows the compiler to perform optimizations automatically (unless otherwise stated). The variable loop is made 'private' by each thread by default. Also, all threads have to wait at the end of the parallel loop before proceeding past the end of this region.
 One OpenMP shortcut is to put the '''parallel pragma''' in the '''parallel for''' part, which just makes the code more readable.
@@ Line 191: / Line 191: @@
 </pre>
-There is an side effect of threading is called '''False sharing''' which can cause poor scaling. If independent data elements happen to sit on the same cache line, each update will cause the cache lines to “slosh back and forth” between threads hence each time the variable have to be re-loaded from main memory.
+There is a side effect of threading is called '''False sharing''' which can cause poor scaling. If independent data elements happen to sit on the same cache line, each update will cause the cache lines to “slosh back and forth” between threads hence each time the variable has to be re-loaded from the main memory.
 One solution is to pad arrays so elements are on distinct cache lines, another method is to allow single threads to operate on a cache line completely. A better way is to re-write the program so this effect is avoiding, an example is given below:
@@ Line 228: / Line 228: @@
 * Here we create a scalar local to each thread to accumulate partial sums
 * No array, no false sharing
-* Within the parallel region we create a '''#pragma omp''' critical region to allow on one thread at a time to enter to perform the summation of the result of pi.
+* Within the parallel region we create a '''#pragma omp''' critical region to allow one thread at a time to enter to perform the summation of the result of pi.
-This solution will scale much better than using an array based design. Again because the above code of often seen in loops, openMP provides a reduction tool as well:
+This solution will scale much better than using an array-based design. Again because the above code of often seen in loops, openMP provides a reduction tool as well:
 <pre>
@@ Line 277: / Line 277: @@
 * Avoid false sharing possibilities, this will cause the greatest limitation to scaling
-* Does your program justify the overhead of threads, if you're parallelising out only a small '''for..loop''' the overhead cannot be justified and will increase processing time.
+* Does your program justify the overhead of threads, if you're parallelizing out only a small '''for..loop''' the overhead cannot be justified and will increase processing time.
 * Best choice of schedule might change with the system
-* Minimize synchronisation, use nowait where practical
+* Minimize synchronization, use nowait where practical
 * Locality, must systems are NUMA, modify your loop nest, or change loop order to get better cache behaviour
@@ Line 369: / Line 369: @@
 * Mistyping the sentinel (e.g. !OMP or #pragma opm ) typically raises no error message.
-* Don’t forget that private variables are uninitialised on entry to parallel regions.
+* Don’t forget that private variables are uninitialized on entry to parallel regions.
 * If you have large private data structures, it is possible to run out of stack space.
-** the size of thread stack apart from the master thread can be controlled by the OMP_STACKSIZE environment variable.
+** the size of the thread stack apart from the master thread can be controlled by the OMP_STACKSIZE environment variable.
 * Writing code that works without OpenMP too. The macro _OPENMP is defined if code is compiled with the OpenMP switch.
 * Be aware The overhead of executing a parallel region is typically in the tens of microseconds range
-** depends on compiler, hardware, no. of threads
+** depends on the compiler, hardware, no. of threads
 * Tuning the chunk size for static or dynamic schedules can be tricky because the optimal chunk size can depend quite strongly on the number of threads.
 * Make sure your timer actually does measure wall clock time. Do use '''omp_get_wtime()'''
@@ Line 447: / Line 447: @@
 ==== Hybrid OpenMP/MPI codes ====
-This programming model runs on one node but can be programmed to as a hybrid model as well with MPI.
+This programming model runs on one node but can be programmed as a hybrid model as well with MPI.
 An application built with the hybrid model of parallel programming can run on a computer cluster using both OpenMP and Message Passing Interface (MPI), such that OpenMP is used for parallelism within a (multi-core) node while MPI is used for parallelism between nodes. There have also been efforts to run OpenMP on software distributed shared memory systems.
@@ Line 488: / Line 488: @@
 | #pragma omp for
 #pragma omp parallel for
-| worksharing, parallel loops, loop carried dependencies
+| work sharing, parallel loops, loop carried dependencies
 |-
 | reduction(op:list)