HPC - User contributions [en-gb]

Applications/Python

2018-03-02T16:02:01Z

561978: 561978 moved page Python to Applications/Python without leaving a redirect

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a YAML File''' ===

To create a virtual environment from a YAML file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the YAML called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

=== '''Exporting a Virtual Environment in Anaconda to a YAML File''' ===

To export a virtual environment to a YAML file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T16:00:59Z

561978: /* Exporting a Virtual Environment in Anaconda to a YAML File */

Applications/Python

2018-03-02T16:00:38Z

561978: /* Creation of a Virtual Environment in Anaconda Using a YAML File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a YAML File''' ===

To create a virtual environment from a YAML file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the YAML called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

=== '''Exporting a Virtual Environment in Anaconda to a YAML File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T16:00:19Z

561978: /* Creation of a Virtual Environment in Anaconda Using a YAML File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a YAML File''' ===

To create a virtual environment from a YAML file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

=== '''Exporting a Virtual Environment in Anaconda to a YAML File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:59:50Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a YAML File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

=== '''Exporting a Virtual Environment in Anaconda to a YAML File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:59:40Z

561978: /* Creation of a Virtual Environment in Anaconda Using a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a YAML File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:58:44Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)'''Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:58:29Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
'''2.)''' 2. Export your active virtual environment using the following command
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env export > env.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:56:45Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.)''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:56:30Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

'''1.''' Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:56:16Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

*1.* Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:55:52Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

# Activate the Virtual environment you wish to export
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:54:55Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

# Activate the Virtual environment you wish to export

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:53:36Z

561978: /* Exporting a Virtual Environment in Anaconda to a Yaml File' */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== '''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T15:53:23Z

561978: /* Creation of a Virtual Environment in Anaconda Using a Yaml File */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>
=== ''Exporting a Virtual Environment in Anaconda to a Yaml File''' ===

To export a virtual environment to a yaml file so that you or a third party can replicate your environment using Anaconda can be done using the following steps:

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:26:11Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

== Virtual Environments ==
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:25:19Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

==='''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''===

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of Virtual Environment Using Anaconda on Viper''' ===

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

=== '''Creation of a Virtual Environment in Anaconda Using a Yaml File''' ===

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:22:36Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This above command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file called "myenv.yml".

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:19:01Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:18:46Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

{{big|'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''}}

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:18:04Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

{{big|'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper''}}'

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:16:31Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:11:21Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

name: ytenv
channels:
- defaults
dependencies:
- ca-certificates=2017.08.26=h1d4fec5_0
- certifi=2018.1.18=py27_0
- intel-openmp=2018.0.0=hc7b2577_8
- libedit=3.1=heed3624_0
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=h7cc24e2_2
- libgfortran-ng=7.2.0=h9f7466a_2
- libstdcxx-ng=7.2.0=h7a57d05_2
- mkl=2018.0.1=h19d6760_4
- ncurses=6.0=h9df7e31_2
- numpy=1.14.0=py27h3dfced4_1
- openssl=1.0.2n=hb7f436b_0
- pip=9.0.1=py27ha730c48_4
- python=2.7.14=h1571d57_29
- readline=7.0=ha6073c6_4
- setuptools=38.4.0=py27_0
- sqlite=3.22.0=h1bed415_0
- tk=8.6.7=hc745277_3
- wheel=0.30.0=py27h2bc6bb2_1
- zlib=1.2.11=ha838bed_2
- pip:
- backports.functools-lru-cache==1.5
- backports.shutil-get-terminal-size==1.0.0
- cycler==0.10.0
- decorator==4.2.1
- enum34==1.1.6
- h5py==2.7.1
- ipython==5.5.0
- ipython-genutils==0.2.0
- matplotlib==2.1.2
- mpmath==1.0.0
- pathlib2==2.3.0
- pexpect==4.4.0
- pickleshare==0.7.4
- prompt-toolkit==1.0.15
- ptyprocess==0.5.2
- pygments==2.2.0
- pyparsing==2.2.0
- python-dateutil==2.6.1
- pytz==2018.3
- scandir==1.7
- simplegeneric==0.8.1
- six==1.11.0
- subprocess32==3.2.7
- sympy==1.1.1
- traitlets==4.3.2
- wcwidth==0.1.7
- yt==3.4.1

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:10:43Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

This command is creating a virtual environment from the yaml called myenv.yml. Below is a copy of the mark-up contained within the file.

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:10:19Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda env create -f myenv.yml
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:09:35Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

To create a virtual environment from a yaml file you would issue the following command:

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:09:16Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''
To create a virtual environment from a yaml file you would issue the following command:

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:08:52Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:08:36Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''
'''Creation of a Virtual Environment in Anaconda Using a Yaml File'''

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:06:59Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:06:09Z

561978: /* Virtual Environments */

__TOC__
=== Application Details ===
* Description: Python is a high-level interpreted programming language for general-purpose programming, supported by a large number of libraries for many tasks
* Versions: Python 2.7.11 and 3.5.1
* Module names: python/2.7.11 python/3.5.1
* License: Free to use - [https://en.wikipedia.org/wiki/Python_Software_Foundation_License Python Software Foundation License]

== Usage Examples ==

Python is provided by the [[applications/Anaconda|Anaconda package]] too

=== Interactive ===

Interactive with command line:

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ module add python/anaconda/4.0/2.7
[username@c170 ~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>

</pre>

Interactive with command line with IPython:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@c170 ~]$ ipython
Python 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.

In [1]:
</pre>

=== Batch Submission ===
<pre style="background-color: #C8C8C8; color: black; font-family: monospace, sans-serif;">
#!/bin/bash
#SBATCH -J PythonTest # Job name, you can change it to whatever you want
#SBATCH -N 1 # Number of nodes
#SBATCH --ntasks-per-node 1 # Number of cores per node
#SBATCH -o %N.%j.out # Standard output will be written here
#SBATCH -e %N.%j.err # Standard error will be written here
#SBATCH -p compute # Slurm partition, where you want the job to be queued

module purge
module add python/3.5.1

python PythonTest.py
</pre>

This is then submitted as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[username@login01 ~]$ sbatch Pythontest.job
Submitted batch job 289522
</pre>

=== Virtual Environments ===
A Python Virtual Environment allows users to create a custom environment/s in which they can have the packages, and versions of said packages that they require without having to have elevated user privileges on the system.

There are many benefits to using a virtual environment on a system. Here is a short non-exhaustive list of a few of the key benefits:

* Control over packages and package versions
* An arbitrary number of virtual environments can be created for different tasks
* Reproducibility - A user can replicate a python environment on any system, so as to be able to run or re-run a particular task or job under the same conditions

'''Creation of Virtual Environment Using Python 2.7.11 or 3.5.1 on Viper'''

To create a virtual environment you would issue the following command:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ virtualenv foo
</pre>
This creates a directory containing your virtual environment named '''foo''' . To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source foo/bin/activate
</pre>
If the environment starts correctly you should see the name of your environment in round brackets in front of the command prompt. For this particular example it should appear as follows:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(foo) [user@c001 ~ ]$
</pre>
To exit the virtual environment use the key combination '''ctrl + d'''

'''Creation of Virtual Environment Using Anaconda on Viper'''

To create a virtual environment using anaconda 4.1.1 with python version 2.7 on Viper, you would use the conda create command as follows:

'''IMPORTANT NOTE: By default the virtual environment does not use the python system packages. However because of the way viper is configured it will see python system packages because of the PYTHONPATH environment variable. So it is advised if you would like a clean environment (meaning no system packages being included) to set this environment variable to empty as follows: ''export PYTHONPATH=''

<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ conda create –n newenv pip numpy
</pre>
The above command creates a new virtual environment called newenv and installs pip and numpy within this environment.

To activate this virtual environment you would issue the following command:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
[user@c001 ~ ]$ source activate newenv
</pre>
On successful activation of this virtual environment you should the name of your environment in front of your login prompt like so:
<pre style="background-color: #000000; color: white; border: 2px solid black; font-family: monospace, sans-serif;">
(newenv) [user@c001 ~ ]$
</pre>

== Further Information ==
* [https://www.hpc.hull.ac.uk/forum/viewforum.php?f=15 UoH Python forum]
* [[Programming/Python|Wiki: Python Programming]]
* [[Main Page#General_Support|Wiki: General Support]]
* [[Main Page|Wiki: Home]]

{|
|style="width:5%; border-width: 0" | [[File:icon_home.png]]
|style="width:95%; border-width: 0" |
* [[Main_Page|Home]]
* [[Applications|Application support]]
* [[General|General]]
* [[Training|Training]]
* [[Programming|Programming support]]
|-
|}

Applications/Python

2018-03-02T14:05:23Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T14:04:59Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T14:04:11Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T14:03:47Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T14:03:15Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T13:56:38Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T13:54:59Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T13:54:04Z

561978: /* Virtual Environments */

Applications/Python

2018-03-02T13:50:56Z

561978: 561978 moved page Applications/Python to Python without leaving a redirect

Applications/Python

2018-02-27T16:58:57Z

561978: 561978 moved page Python to Applications/Python without leaving a redirect

Applications/Python

2018-02-27T16:55:14Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:54:00Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:51:36Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:48:52Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:48:37Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:47:28Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:47:05Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:42:54Z

561978: /* Virtual Environments */

Applications/Python

2018-02-27T16:42:21Z

561978: /* Virtual Environments */