Virtualenv

If you need python libraries installed that we don’t have, or that are newer than the versions we currently have installed, you should be able to install everything you need on your own using a virtual environment.

Here are the steps needed to set up your own python virtualenv:

All of the steps below assume your CS username is csmajor1 (i.e., when you execute the commands below, please change csmajor1 to your username).

make the virtual environment

Here’s how to make a virtual environment, called my_project, that uses python 3:

export PYTHONPATH=
export WORKON_HOME=~/Envs
source /usr/local/bin/virtualenvwrapper.sh
mkvirtualenv -p /usr/bin/python3 my_project

At this point you have made and are in the virtual environment. Your prompt should have changed to include (my_project) at the beginning.

install software with pip

Use pip to install any packages you need! Use pip list to see what’s installed. Here are some examples:

(my_project) pip install numpy
(my_project) pip install scipy
(my_project) pip install scikit-learn
(my_project) pip list 
Package          Version
---------------- -------
numpy           1.12.1     
scikit-learn    0.18.1     
scipy           0.19.0     
(my_project) python
>>> import sklearn
>>> sklearn.__version__
'0.18.1'

At this point your virtualenv is ready to use. If you want, start using it and run your programs in it.

when you are done, run deactivate to end the virtualenv

If you want to get back out of your virtualenv, you can deactivate it or just close/exit the terminal window.

(my_project) deactivate
$

when you want to work on it again, use workon

To get back into your virtualenv:

$ export PYTHONPATH=
$ export WORKON_HOME=~/Envs
$ source /usr/local/bin/virtualenvwrapper.sh
$ workon my_project
(my_project)

I sometimes put the above four commands in a file, so I just have to source that one file to get into my virtual environment (I also added some CUDA environment variables, in case you’re using the GPUs):

$ cat goproj
export CUDA_HOME=/usr/local/cuda-11.2
export LD_LIBRARY_PATH=/usr/local/cuda-11.2/lib64
export PYTHONPATH=
export WORKON_HOME=/scratch/csmajor1/Envs
source /usr/local/bin/virtualenvwrapper.sh
workon my_project

$ source goproj
(my_project)

Note: if you are using cuda, we might have different versions available. Some versions of tensorflow only work with specific cuda versions. Check the docs. You’re welcome to change the above “exports” to use another cuda version (use ls -d /usr/local/cuda* to see what’s available).

sharing with partners

If you want to share your virtualenv with a lab partner, you just need to allow read-execute access to the Envs directory and any files below that. All that’s stored in there is the software you installed (e.g., with pip), so it’s OK to open up this directory for others to access (but don’t put your lab work in the Envs directory).

Here’s a command to allow read and execute permission for everyone on the /scratch/csmajor1/Envs directory, and any files and directories below that:

$ chmod -R a+rX /scratch/csmajor1/Envs

The R is for recursive, and the a+rX says grant all users read and execute access.

And if you made a goproj file to source, and want to share that with partners, do this:

$ chmod 644 goproj

Additional Docs

Here are a few pages with additional info on using python virtual environments:

Back to SwatCS Help Docs