= Creating and Using a Virtual Python Environment =

[[TOC]]

== What We Are Trying To Do ==

Your computer's system Python is used by your computer's operating system for many things including installing your OS itself and installing additional software on your OS. It is not good to pollute that Python with your own packages, and Python packages which your OS uses might interfere with your own packages.

Also, your computer's Python might not even be the right version for packages you want to use.

You may need many different versions of Python on your computer for doing different things.

You may also need different versions of Python packages for doing different things.

You may also find you have installed a Python package you don't like for some reason. You need a way to get rid of Python packages you don't want.

You want to do two things:

 1. Use the right version of Python for your job.
 1. Isolate the Python packages used by your applications from other applications.

A virtual Python environment will give you a separate Python for each application. That Python will be exactly the right version for your application. And that Python will have exactly the right version of packages installed for your application. You will be able to change the versions of packages used by your application without disturbing any of your other applications.

== Some Terminology If You Get Lost ==
{{{
#!html
<dl>
<dt><strong>module</strong></dt><dd>A file containing Python code. It may be executed directly by the Python interpreter, or imported into another module if found in a directory listed in <code>sys.path</code>.</dd>
<dt><strong>package</strong></dt><dd>A hierarchy of directories containing Python modules. It may be imported by a Python module if found in a directory listed in <code>sys.path</code>.</dd>
<dt><strong>site-packages</strong></dt><dd>A subdirectory in your Python's <code>lib</code> directory which contains your installed third party Python packages. Something is seriously wrong if this subdirectory is not listed in <code>sys.path</code>.</dd>
<dt><strong>ez_setup.py</strong></dt><dd>A Python module which installs the <code>setuptools</code> package and the <code>easy_install</code> executable.</dd>
<dt><strong>easy_install</strong></dt><dd>An executable in your Python's <code>bin</code> directory which can automatically install Python packages.</dd>
<dt><strong>setuptools</strong></dt><dd>A Python package which facilitates creating and distributing third party Python packages. <code>setuptools</code> is a third party package that is not yet a part of the Python Standard Library of packages.</dd>
<dt><strong>distutils</strong></dt><dd>A Python package which is the predecessor of <code>setuptools</code> still in wide use. <code>distutils</code> is part of the Python Standard Library of packages.</dd>
<dt><strong>setup.py</strong></dt><dd>A Python module which, when placed at the top level of a Python package, tells <code>setuptools</code> or <code>distutils</code> how to create a distributable form of the package. It normally consists of one function call (<code>setup()</code>) with numerous arguments. A <code>distutils</code> distributed package can also be installed by unzipping it and running the <code>setup.py</code> module in the unzipped package directory. <code>setup.py</code> is defined by the package author.</dd>
<dt><strong><a href="http://pypi.python.org/">Pypi</a></strong></dt><dd>The Python Package Index (aka "Cheeseshop"). A master repository of third party Python packages. easy_install downloads packages from Pypi by default. Pypi contains packages in both "egg" form created by <code>setuptools<code> as well as packages created by <code>distutils</code>.</dd>
<dt><strong>egg</strong></dt><dd>A zipped file created by setuptools which contains a distributable form of a Python package.</dd>
<dt><strong>virtualenv</strong><dt><dd>An executable in your Python's <code>bin</code> directory which can create virtual Python environment for an application. This environment will have its own isolated <code>bin</code> directory with virtual <code>python</code> and <code>easy_install</code> executables, as well as its own <code>lib/site-packages</code> subdirectory where the virtual <code>easy_install</code> will install private instances of third party Python packages.</dd>
</dl>
}}}

== There are three places to start: ==
 * If you don't yet have a Python compiled separately from your system Python,
 * If you have a Python instance separate from your system Python, but it doesn't yet have the setuptools package installed, or
 * If you already have a separated compiled Python with the setuptools package installed.

=== If you don't yet have a Python compiled separately from your system Python ===

 * XXX
 * Proceed to:

=== If you have a Python instance separate from your system Python, but it doesn't yet have the setuptools module installed ===
 * Get the latest version of [http://peak.telecommunity.com/dist/ez_setup.py ez_setup.py] (Right click to save as python file)
 * Run ez_setup.py with your separately compiled Python:
  {{{/usr/local/python2.6/bin/python ez_setup.py}}}
 * Your separately compiled Python's {{{bin}}} directory will now contain a new executable called {{{easy_install}}}
 * Proceed to:

=== If you already have a separately compiled Python with the setuptools module installed ===
 * Install {{{virtualenv}}} with {{{easy_install}}}:
  {{{/usr/local/python2.6/bin/easy_install virtualenv}}}
 * Your separately compiled Python's {{{bin}}} directory will now contain a new executable called {{{virtualenv}}}

== Creating a Virtual Python Environment ==
 * Use the {{{virtualenv}}} executable in your separately compiled Python to create a new virtual Python environment:
  {{{/usr/local/python2.6/bin/virtualenv /opt/env/my-virt-py-env}}}
 * Your new virtual Python environment at {{{/opt/env/my-virt-py-env}}} now contains its own {{{bin/python}}} executable, {{{bin/easy_install}}} executable, and {{{lib/site-packages}}} subdirectory

== Using a Virtual Python Environment ==
 * Install the packages required for your application in your virtual Python environment with {{{easy_install}}}:
  {{{/opt/env/my-virt-py-env/bin/easy_install Paste 2>&1 | tee /opt/env/my-virt-env/install-paste.stdout}}}
 * You may need to manually install {{{distutil}}}-created (non-egg) packages the old way with the package's {{{setup.py}}}:
  {{{/opt/env/my-virt-py-env/bin/python /opt/env/my-virt-py-env/numpy/setup.py install 2>&1 | tee /opt/env/my-virt-py-env/numpy/install.stdout}}}
 * Run your application using your virtual Python environment's {{{python}}} executable:
  {{{/opt/env/my-virt-py-env/bin/python /opt/env/my-virt-py-env/my-application/my-script.py}}}

== Best Practices ==
 * Download and untar {{{distutils}}}-created (non-egg) package into the virtual Python environment (see {{{numpy}}} example above)
 * Run applications using fully qualified paths to the executable and all arguments (see {{{my-script.py}}} example above)
 * Develop your application in your virtual Python environment directory
 * While developing, you could change directories ({{{cd}}}) into your virtual Python environment directory to avoid having to fully qualify the paths to executables and arguments (i.e., just use {{{bin/python}}} and {{{bin/easy_install}}}
 * Ignore the {{{activate}}} script supplied in your virtual Python environment's {{{bin}}} directory. No good comes of it. Use fully qualified paths.
 * Do not set the environment variable {{{PYTHONPATH}}}. Only evil comes of it. Use fully qualified paths.

== Resources ==
 * [http://pypi.python.org/pypi/virtualenv virtualenv documentation] on Pypi
 * A [http://trizpug.org/Members/cbc/DistUtils.pdf short presentation] on {{{distutils}}}
 * A [http://trizpug.org/Members/cbc/PythonEggs.pdf short presentation] on {{{setuptools}}}
 * A [http://www.packtpub.com/expert-python-programming/book really good book] about software engineering with Python written by the developer who is adding Pypi mirroring capability to {{{setuptools}}}. Thoroughly Covers eggs, setuptools, and virtualenv.

== To Do ==
 * How to compile Python
 * Fix location of site-packages in lib path
 * Explore the effects of OS user and permissions when running virtualenv
 * How to cron an application in a virtual Python environment
 * And other best practices
 * Anatomy of a path file
 * Anatomy of an egg
 * Using {{{easy_install}}} to install from somewhere other than Pypi
 * Better Resources
 * Intro to Paste
 * Intro to Buildout
 * Creating a Python package
 * Creating a Python egg
 * Submitting a Python package to Pypi
 * Creating a Pypi mirror