Python - Packages (Archive, Distribution)

> Procedural Languages > Python > Python - (Shipping|Packaging)

1 - About

Code Shipping - Archive (Distribution) in Python

Packages (or better Distribution package) are the result of the packaging of a project that may contains one or more (regular) package

Not to confound with regular package which are parents of modules. One distribution package may contains several regular package.

A package is a compressed tarball file that contains:

  • system-level libraries,
  • Python or other modules,
  • executable programs and other components.
Advertising

3 - Format

3.1 - Source

3.2 - Binary

3.2.1 - Python

Wheel and Egg are both binary packaging formats created from compiled python file.

  • The Egg format was introduced by setuptools in 2004
  • The Wheel format was introduced by PEP 427 in 2012.

Wheel is currently considered the standard for built and binary packaging for Python.

3.2.2 - Exe

4 - Management

4.1 - Install

Doc

First update the tools

python -m pip install --upgrade pip setuptools wheel
Advertising

4.1.1 - In a binary format

You can install your package:

pip install package_name
pip install "requirement-specifiers"
# Example
pip install "pyspark == 2.2.2"
easy_install package

4.1.2 - From a tarball / source

cd /rootOfProject
python setup.py install
# or
setup.py install
  • from a branch with pip
pip install https://github.com/myName/myPackage/tarball/master 
pip install https://github.com/myName/myPackage/tarball/X.X.X
  • by downloading the source
curl --location --output myPackage-X.X.X.tar.gz https://github.com/myName/myPackage/tarball/X.X.X
tar xvfz myPackage-X.X.X.tar.gz
cd myPackage
# as script
python myScript.py args
# as package
pip install .
  • From version control
pip install -e git+https://git.repo/some_pkg.git#egg=SomeProject          # from git
pip install -e hg+https://hg.repo/some_pkg.git#egg=SomeProject            # from mercurial
pip install -e svn+svn://svn.repo/some_pkg/trunk/#egg=SomeProject         # from svn
pip install -e git+https://git.repo/[email protected]#egg=SomeProject  # from a branch

where: VCS:

  • are detected using url prefixes: “git+”, “hg+”, “bzr+”, “svn+”.
  • command are required in the path: git, hg, svn, or bzr.
Advertising

4.2 - Installation Location

Install location

4.2.1 - Global

default global site-packages directory

(for instance /usr/lib/python2.7/site-packages)

GLOBALE_HOME/pythonX.X/site-packages/

This is the pip default

Example: install global of myPackage

pip install myPackage

4.2.2 - User

The user location :

  • Default to ~/.local/, or %APPDATA%\Python on Windows. (See the Python documentation for site.USER_BASE for full details.)
  • or via the env PYTHONUSERBASE

The package are then installed at USER_HOME/pythonX.X/site-packages/

To install in a user location, use the --user option of pip

Example:

export PYTHONUSERBASE=$HOME
pip install --user virtualenv

4.2.3 - Virtual environment

When working in a virtual environment, the package will be installed in it. See Python - virtualenv (Python Environment)

4.3 - Update

See the –update option of pip

Example: upgrade the setuptools and wheel package

python3 -m pip install --user --upgrade setuptools wheel

4.4 - Packaging

Packaging task management, see setup.py cli

5 - Reference / Specification

6 - Documentation / Reference