Modules and Packaging

At some point, you will want to organize and distribute your code library for the whole world to share, preferably on PyPI so that it is pip installable.


In Pythoh, any .py file is a module in that it can be imported. Because the interpreter runs the entrie file when a moudle is imported, it is traditional to use a guard to ignore code that should only run when the file is executed as a script.

When this file is imported with `import foo`,
only `useful_func1()` and `useful_func()` are loaded,
and the test code `assert ...` is ignored. However,
when we run as a script `python`, then
the two assert statements are run.
Most commonly, the code under `if __naem__ == '__main__':`
consists of simple examples or test cases for the functions
defined in the moule.

def useful_func1():

def useful_fucn2():

if __name__ == '__main__':
    assert(useful_func1() is None)
    assert(useful_fucn2() is None)

Organization of files in a module

When the number of files you write grow large, you will probably want to orgnize them into their own directory structure. To make a folder a module, you just need to include a file named in the folder. This file can be empty. For example, here is a module named pkg with sub-modules sub1 and sub2.

./pkg:      sub1        sub2


import as foo

How to import a module at the same level

Within a package, we need to use absolute path names for importing other modules in the same directory. This prevents confusion as to whether you want to import a system moudle with the same name. For example, imports functions from

! cat pkg/sub1/
from pkg.sub1.sub1_stuff import g1, g2

def g3():
    return 'g3 uses %s, %s' % (g1(), g2())
from pkg.sub1.more_sub1_stuff import g3

'g3 uses g1, g2'

How to import a moudle at a different level

Again, just use absolute paths. For example, in the sub2 directory uses functions from in the sub1 directory:

! cat pkg/sub2/
from pkg.sub1.sub1_stuff import g1, g2

def h1():
    return g1()

def h2():
    return g1() + g2()
from pkg.sub2.sub2_stuff import h2


Distributing your package

Suppose we want to distribute our code as a library (for example, on PyPI so that it cnn be installed with pip). Let’s create an sta663 library containing the pkg package and some other files:

  • some information about the library
  • a standalone module
  • a script (intended for use as python
! ls -R sta663
pkg//       tests//

sta663/pkg:      sub1//        sub2//



! cat sta663/
import as foo
from pkg.sub1.more_sub1_stuff import g3
from pkg.sub2.sub2_stuff import h2

print foo.f1()
print g3()
print h2()

Using distutils

All we need to do is to write a file.

! cat sta663/
from distutils.core import setup

setup(name = "sta663",
      version = "1.0",
      author='Cliburn Chan',
      py_modules = ['sta663'],
      packages = ['pkg', 'pkg/sub1', 'pkg/sub2'],
      scripts = ['']

Build a source archive for distribution


cd sta663
python sdist
cd -
running sdist
running check
writing manifest file 'MANIFEST'
creating sta663-1.0
creating sta663-1.0/pkg
creating sta663-1.0/pkg/sub1
creating sta663-1.0/pkg/sub2
making hard links in sta663-1.0...
hard linking README.txt -> sta663-1.0
hard linking -> sta663-1.0
hard linking -> sta663-1.0
hard linking -> sta663-1.0
hard linking pkg/ -> sta663-1.0/pkg
hard linking pkg/ -> sta663-1.0/pkg
hard linking pkg/sub1/ -> sta663-1.0/pkg/sub1
hard linking pkg/sub1/ -> sta663-1.0/pkg/sub1
hard linking pkg/sub1/ -> sta663-1.0/pkg/sub1
hard linking pkg/sub2/ -> sta663-1.0/pkg/sub2
hard linking pkg/sub2/ -> sta663-1.0/pkg/sub2
creating dist
Creating tar archive
removing 'sta663-1.0' (and everything under it)
warning: sdist: manifest template '' does not exist (using default file list)
! ls -R sta663
MANIFEST      dist//
README.txt    pkg//       tests//


sta663/pkg:      sub1//        sub2//





You can now distribute sta663-1.0.tar.gz to somebody else for installation in the usual way.


cp sta663/dist/sta663-1.0.tar.gz /tmp
cd /tmp
tar xzf sta663-1.0.tar.gz
cd sta663-1.0
python install
running install
running build
running build_py
running build_scripts
running install_lib
running install_scripts
changing mode of /Users/cliburn/anaconda/bin/ to 755
running install_egg_info
Writing /Users/cliburn/anaconda/lib/python2.7/site-packages/sta663-1.0-py2.7.egg-info

Distributing to PyPI

Just enter python register and respond to the prompts to register as a new user.