Training/Matlab-Viper-plugin - Revision history

Pysdlb at 09:36, 21 April 2023

2023-04-21T09:36:10Z

Pysdlb at 10:51, 23 March 2023

2023-03-23T10:51:02Z

Pysdlb at 10:50, 23 March 2023

2023-03-23T10:50:21Z

Pysdlb at 14:02, 17 November 2022

2022-11-17T14:02:27Z

Pysdlb: /* Further Information */

2022-11-16T10:20:45Z

‎Further Information

Pysdlb: /* Navigation */

2022-11-16T10:20:35Z

‎Navigation

Pysdlb at 13:21, 13 October 2022

2022-10-13T13:21:52Z

Pysdlb at 12:51, 3 May 2022

2022-05-03T12:51:05Z

Pysdlb at 13:06, 24 May 2021

2021-05-24T13:06:46Z

Pysdlb at 15:27, 22 July 2020

2020-07-22T15:27:40Z

@@ Line 7: / Line 7: @@
 ==Introduction==
-There is a feature in Matlab that enables you to submit jobs to clusters directly from your Matlab client (normal desktop Matlab). we were able to develop a Matlab-Viper plug-in that utilizes this feature and integrates Matlab with viper to enable all our Matlab users to submit some Matlab jobs to viper easily without the need to know much about HPC or Linux. Everything could be done from the normal desktop Matlab version that many users are already using.
+There is a feature in Matlab that enables you to submit jobs to clusters directly from your Matlab client (normal desktop Matlab). we were able to develop a Matlab-Viper plug-in that utilizes this feature and integrates Matlab with Viper to enable all our Matlab users to submit some Matlab jobs to Viper easily without the need to know much about HPC or Linux. Everything could be done from the normal desktop Matlab version that many users are already using.
-This guide should explain more about how to use viper through the Matlab viper plug-in. In general, the toolbox that could utilize such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorials that you could use and try on our HPC.
+This guide should explain more about how to use Viper through the Matlab Viper plug-in. In general, the toolbox that could utilize such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorials that you could use and try on our HPC.
-*If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ support portal] we will help you to install it and configure it.
+*If you don't have the Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ support portal] we will help you to install it and configure it.
-*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact our [https://support.hull.ac.uk/ support portal].
+*This is only available for Matlab 2017a, we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact our [https://support.hull.ac.uk/ support portal].
 * This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on the Matlab distributed computing server through this plug-in.
@@ Line 30: / Line 30: @@
 ===Run Matlab Job===
-Using a parallel toolbox, you can run your Matlab code on viper using 2 modes:
+Using a parallel toolbox, you can run your Matlab code on Viper using 2 modes:
 ====Batch job====
@@ Line 36: / Line 36: @@
 * This is the main way to utilize the Matlab-viper-plugin. This submits your job to Viper queuing system (SLURM) and sends back the results to your client once everything is done.
 * Very useful for a time-consuming job that needs hours or even days to finish. you will submit your job and you don't need to keep your Matlab running or even the whole computer could be turned off as all your simulations will be running on Viper.
-* If you want to know more about the job life cycles and different stages https://uk.mathworks.com/help/distcomp/how-parallel-computing-products-run-a-job.html#f3-11033
+* If you want to know more about the job life cycles and different stages https://uk.mathworks.com/help/distcomp/how-parallel-computing-products-run-a-job.html
 *'''Create Matlab script'''
 *:This can be either a normal serial script or a parallel one that uses functions like parfor. if you have an existing script make sure that the following is done before running it:
@@ Line 79: / Line 80: @@
 ====Parallel pool====
 * Another mode of operation that is easier is if you are testing a piece of code or running a small job that you expect to finish in a few minutes.
-* This requires a running connection to be in place between your computer and viper. at any point you shut down your Matlab/Wifi or even your laptop everything will be lost and you will need to rerun your code/simulation.
+* This requires a running connection to be in place between your computer and Viper. at any point you shut down your Matlab/Wifi or even your laptop everything will be lost and you will need to rerun your code/simulation.
 == Further Information ==
@@ Line 89: / Line 90: @@
-[[Main Page]]   /   [[FurtherTopics/FurtherTopics #Matlab| Further Topics]]
+[[Main Page]]   /   [[FurtherTopics/FurtherTopics#Matlab| Further Topics]]
 -->

@@ Line 11: / Line 11: @@
 *If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ support portal] we will help you to install it and configure it.
-*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk].
+*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact our [https://support.hull.ac.uk/ support portal].
 * This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on the Matlab distributed computing server through this plug-in.

@@ Line 8: / Line 8: @@
 ==Introduction==
 There is a feature in Matlab that enables you to submit jobs to clusters directly from your Matlab client (normal desktop Matlab). we were able to develop a Matlab-Viper plug-in that utilizes this feature and integrates Matlab with viper to enable all our Matlab users to submit some Matlab jobs to viper easily without the need to know much about HPC or Linux. Everything could be done from the normal desktop Matlab version that many users are already using.
-This guide should explain more about how to use viper through the Matlab viper plug-in. In general, the toolbox that could utilize such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorials that you could use and try on viper.
+This guide should explain more about how to use viper through the Matlab viper plug-in. In general, the toolbox that could utilize such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorials that you could use and try on our HPC.
-*If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ Topdesk] we will help you to install it and configure it.
+*If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ support portal] we will help you to install it and configure it.
 *This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk].
-* This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on Matlab distributed computing server through this plug-in.
+* This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on the Matlab distributed computing server through this plug-in.
 ==Matlab to Viper==

← Older revision		Revision as of 14:02, 17 November 2022
Line 1:		Line 1:
		+	<!--
	__TOC__		__TOC__

Line 89:		Line 90:

	[[Main Page]] / [[FurtherTopics/FurtherTopics #Matlab\| Further Topics]]		[[Main Page]] / [[FurtherTopics/FurtherTopics #Matlab\| Further Topics]]
		+	-->

@@ Line 84: / Line 84: @@
 * [https://matlabacademy.mathworks.com/ Matlab online courses]
 * [https://uk.mathworks.com/help/distcomp/batch.html/ Batch command documentation]
 [[Main Page]]   /   [[FurtherTopics/FurtherTopics #Matlab| Further Topics]]

← Older revision		Revision as of 10:20, 16 November 2022
Line 86:		Line 86:


−	~~==Navigation==~~	+	[[Main Page]] / [[FurtherTopics/FurtherTopics #Matlab\| Further Topics]]
−
−	* [[~~Main_Page\|Home]]~~
−	* [[Applications\|Application support]]
−	* [[General\|General]] *
−	* [[~~Programming~~\|~~Programming support~~]]

@@ Line 10: / Line 10: @@
 *If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ Topdesk] we will help you to install it and configure it.
-*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk] .
+*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk].
 * This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on Matlab distributed computing server through this plug-in.
@@ Line 21: / Line 21: @@
 #: The client could run on your laptop or desktop computer. This is where you will run Matlab commands and submit your jobs.
 #'''Worker:'''
-#: The worker is an instance of Matlab distributed computing environment that runs on Viper in order to perform a calculation for one of the clients. you could submit batch jobs from your clients that run some simulations in parallel using functions like ''parfor'', you could also run normal Matlab code on it.
+#: The worker is an instance of a Matlab distributed computing environment that runs on Viper in order to perform a calculation for one of the clients. you could submit batch jobs from your clients that run some simulations in parallel using functions like ''parfor'', you could also run normal Matlab code on it.
 ==== limitations ====
@@ Line 39: / Line 39: @@
 *:This can be either a normal serial script or a parallel one that uses functions like parfor. if you have an existing script make sure that the following is done before running it:
 ** Remove any parpool commands if any
-** Modify the file paths if any is used in your script. consider that all your files will move to the cluster and run *:there so if your script contains a filepath that exisits only on your client it will not be able to find it there.
+** Modify the file paths if any is used in your script. consider that all your files will move to the cluster and run *: there so if your script contains a file path that exists only on your client it will not be able to find it there.
 *:example:
 *:use
@@ Line 46: / Line 46: @@
 <pre style="background-color: #f5f5dc; color: black; font-family: monospace, sans-serif;"> load ('/INPUT_DIRECTORY_NAME/INPUT_FILE_NAME')
 </pre>
-*'''Batch command''': This command begins an automated process that connects to the cluster, submits a job to the scheduler, and initializes MDCS. it will ask you to enter your user name and password. you must also make sure that you are using on the campus network or while using VPN
+*'''Batch command''': This command begins an automated process that connects to the cluster, submits a job to the scheduler, and initializes MDCS. it will ask you to enter your username and password. you must also make sure that you are using the campus network or while using VPN
 *For example if you submit a script to Viper with one input file:
 <pre style="background-color: #f5f5dc; color: black; font-family: monospace, sans-serif;"> viper = parcluster()
@@ Line 52: / Line 52: @@
 </pre>
 :*Specify the name of your script without the “m” prefix. For instance use test, and not test.m
-:*The argument pool specified the number of workers you need for execution of the script. Note that the MATLAB adds one worker to the number of workers you specified in pool. So for instance if you specify 9 workers in the pool, then MATLAB will use 10 workers.
+:*The argument pool specified the number of workers you need for the execution of the script. Note that MATLAB adds one worker to the number of workers you specified in the pool. So for instance, if you specify 9 workers in the pool, then MATLAB will use 10 workers.
-:*The 4th Argument 'AttachedFiles' highlight that we are going to attach input file/files to this batch job. you can send multiple files by including them in braces. For Example<pre style="background-color: #f5f5dc; color: black; font-family: monospace, sans-serif;"> job = viper.batch('test', 'pool', 9, 'AttachedFiles', {input1, input2})</pre>
+:*The 4th Argument 'AttachedFiles' highlights that we are going to attach input file(s) to this batch job. you can send multiple files by including them in braces. For Example<pre style="background-color: #f5f5dc; color: black; font-family: monospace, sans-serif;"> job = viper.batch('test', 'pool', 9, 'AttachedFiles', {input1, input2})</pre>
-*When you finish working with this job and get all your outputs back. permanently delete its data and remove its reference from the workspace:
+*When you finish working on this job and get all your outputs back. permanently delete its data and remove its reference from the workspace:
 <pre style="background-color: #f5f5dc; color: black; font-family: monospace, sans-serif;">
 delete(job)
@@ Line 60: / Line 60: @@
 </pre>
 ==Useful commands==
-below is some useful commands and its detailed offical documentation
+below are some useful commands and their detailed official documentation
 {| class="wikitable"
 | style="width:25%" | <Strong>Training Area</Strong>
@@ Line 77: / Line 77: @@
 ====Parallel pool====
-* Another mode of operation that is easier if you are testing a piece of code or running a small job that you expect to finish in a few minutes.
+* Another mode of operation that is easier is if you are testing a piece of code or running a small job that you expect to finish in a few minutes.
 * This requires a running connection to be in place between your computer and viper. at any point you shut down your Matlab/Wifi or even your laptop everything will be lost and you will need to rerun your code/simulation.

@@ Line 1: / Line 1: @@
-=Matlab viper plugin=
+__TOC__
-please note this page is still under development.
 ==Introduction==
-There is a feature in Matlab that enables you to submit jobs to clusters directly from your Matlab client (normal desktop Matlab). we were able to develop Matlab-Viper plug-in that utilise this feature and integrate Matlab with viper to enable all our Matlab users to submit some Matlab jobs to viper easily without the need to know much about HPC or Linux. Everything could be done from the normal desktop Matlab version that many users are already using.
+There is a feature in Matlab that enables you to submit jobs to clusters directly from your Matlab client (normal desktop Matlab). we were able to develop a Matlab-Viper plug-in that utilizes this feature and integrates Matlab with viper to enable all our Matlab users to submit some Matlab jobs to viper easily without the need to know much about HPC or Linux. Everything could be done from the normal desktop Matlab version that many users are already using.
-This guide should explain more on how to use viper through the Matlab viper plug-in. In-general the toolbox that could utilise such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorial that you could use and try on viper.
+This guide should explain more on how to use viper through the Matlab viper plug-in. In general, the toolbox that could utilize such integration is the Matlab parallel computing toolbox. we will try to explain the main concepts here and point you to some MathWorks tutorials that you could use and try on viper.
 *If you don't have Matlab-Viper plug-in installed and tested on your Matlab contact [https://support.hull.ac.uk/ Topdesk] we will help you to install it and configure it.
-*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk] .
+*This is only available for Matlab 2017a , we will try to support it for all the upcoming versions starting from 2017a. if you have a special use case that requires the usage of an older version you could check the possibility of creating this plug-in for it contact [https://support.hull.ac.uk/ Topdesk] .
 * This link https://uk.mathworks.com/products/ineligible_programs.html  shows a list of Ineligible programs that couldn't run on Matlab distributed computing server through this plug-in.
@@ Line 15: / Line 19: @@
 ==== This plugin works by connecting 2 components together:====
 #'''Client:'''
-#:The client could run on your laptop or desktop computer. This is where you will run Matlab commands and submit your jobs.
+#: The client could run on your laptop or desktop computer. This is where you will run Matlab commands and submit your jobs.
 #'''Worker:'''
-#:The worker is and instance of Matlab distributed computing environment that runs on Viper in order to perform a calculation for one of the clients. you could submit batch jobs from your clients that runs some simulations in parallel using functions like ''parfor'', you could also run normal Matlab code on it.
+#: The worker is an instance of Matlab distributed computing environment that runs on Viper in order to perform a calculation for one of the clients. you could submit batch jobs from your clients that run some simulations in parallel using functions like ''parfor'', you could also run normal Matlab code on it.
 ==== limitations ====
-submitting a job to Viper has a a lot of overhead. if you are running a small computing task it might take even longer to run on the cluster as there are a lot of time will be lost in communication and transferring your data to Viper then in sometimes your job could be queued and start running in a later time. however in many other cases it will show you a speed-up and as a minimum you will be able to free up your client from doing this computation.
 ===Run Matlab Job===
-using parallel tool box, you can run your Matlab code on viper using 2 modes:
 ====Batch job====
-*This is the main way to utilize the Matlab-viper-plugin. This submit your job to Viper queuing system (SLURM) and send back the results to your client once everything is done.
+* This is the main way to utilize the Matlab-viper-plugin. This submits your job to Viper queuing system (SLURM) and sends back the results to your client once everything is done.
-*Very useful for time consuming job that need hours or even days to finish. you will submit your job and you don't need to keep your Matlab running or even the whole computer could be turned off as all your simulation will be running on Viper.
+* Very useful for a time-consuming job that needs hours or even days to finish. you will submit your job and you don't need to keep your Matlab running or even the whole computer could be turned off as all your simulation will be running on Viper.
-* if you want to know more about the job life cycles and different stages https://uk.mathworks.com/help/distcomp/how-parallel-computing-products-run-a-job.html#f3-11033
+* If you want to know more about the job life cycles and different stages https://uk.mathworks.com/help/distcomp/how-parallel-computing-products-run-a-job.html#f3-11033
 *'''Create Matlab script'''
-*:this can be either a normal serial script or a parallel one that uses functions like parfor. if you have an existing script make sure that the following is done before running it:
+*:This can be either a normal serial script or a parallel one that uses functions like parfor. if you have an existing script make sure that the following is done before running it:
-** remove any parpool commands if any
+** Remove any parpool commands if any
-**modify the file paths if any is used in your script. consider that all your files will move to the cluster and run *:there so if your script contains a filepath that exisits only on your client it will not be able to find it there.
+** Modify the file paths if any is used in your script. consider that all your files will move to the cluster and run *:there so if your script contains a filepath that exisits only on your client it will not be able to find it there.
 *:example:
 *:use
@@ Line 62: / Line 69: @@
 |-
 | [[https://uk.mathworks.com/help/distcomp/job-monitor.html/XQuartz| Job Monitor]]
-| enable you to monitor all your jobs and its status
+| enable you to monitor all your jobs and their status
 |-
 | [[https://uk.mathworks.com/help/distcomp/index.html| Parallel Computing Toolbox]]
@@ Line 70: / Line 77: @@
 ====Parallel pool====
-* Another mode of operation that is easier if you are testing piece of code or running a small job that you expect to finish in few minutes.
+* Another mode of operation that is easier if you are testing a piece of code or running a small job that you expect to finish in few minutes.
-* This requires a running connection to be in place between your computer and viper. at any point you shutdown your Matlab/Wifi or even your laptop everything will be lost and you will need to rerun your code/simulation.
+* This requires a running connection to be in place between your computer and viper. at any point you shut down your Matlab/Wifi or even your laptop everything will be lost and you will need to rerun your code/simulation.
 == Further Information ==