I hate Python! – importing modules from subdirectories (PYTHONPATH)

So, you have your python application, and you want to make it tidy. You want to keep all you modules in proper subdirectories (e.g. database manipulation, views, etc.). So, here you are with your subdirectories and the code.

If you are looking for a sample code, go straight here: . If you are looking for some sort of explanation, go on with reading.

Step one – everything at the same level

pythonapp/
├── module_root.py
└── simple.py

The code is really simple and looks like this:

''' simple.py '''
from module_root import *

print 'Hello, I will call module_root'
fun_root()

and for the module_root it is equally simple:

''' module_root.py '''
def fun_root():
  print 'Hello from module_root'

Step two – make it runnable from any location (using PATH, chmod +x and python location)

Now, everything is just fine when we call the application from the same directory where the file is. But what will happen if we change directory to some other location?

> cd ~
> python simple.py
python: can't open file 'simple.py': [Errno 2] No such file or directory

What we can do is to specify the file location explicitly

> python pythonapp/simple.py
Hello, I will call module_root
Hello from module_root

Now, it works fine, but we don’t want to specify the full path all the time.

First, let’s modify code a little bit

#!/usr/bin/python
''' simple.py '''
from module_root import *

print 'Hello, I will call module_root'
fun_root()

> chmod +x ~/pythonapp/simple.py
> export PATH=~/pythonapp:$PATH
> simple.py
Hello, I will call module_root
Hello from module_root

Step three – We want to split codes across different sub-directories

That’s cool. So far, so good. However, there is one more issue. We don’t want to keep all the sources at the same level. We want to put them inside subdirectories. Like this:

pythonapp/
├── module
│   └── module_sub.py
├── module_root.py
└── simple.py

And, of course, we want to use module_sub as well. That’s simple! Let’s modify the code a little bit

#!/usr/bin/python
''' simple.py '''
from module_root import *
from module.module_sub import *

print 'Hello, I will call module_root'
fun_root()

print 'I will call module_sub'
fun_sub()

and, the code for the module_sub.py looks like this

''' module_sub.py '''
def fun_sub():
  print 'Hello from submodule'

Let’s roll

> simple.py
Traceback (most recent call last):
  File '.../pythonapp/simple.py', line 3, in 
    from module.module_sub import *
ImportError: No module named module.module_sub

Surprise, surprise. It doesn’t work. Well, it should. At least that’s what you expect when you are coming from Java world where everything below CLASSPATH dir is treated as packages.

Step four – We want to mark directory as module

Not in here. In python, you have to mark directory as “module” by doing it explicitly. You have to create __init__.py file insde directory. Like this:

cd pythonapp/module
touch __init__.py

And your structure should resemble something like this:

pythonapp/
├── module
│   ├── __init__.py
│   ├── module_sub.py
├── module_root.py
└── simple.py

And now …

> simple.py
Hello, I will call module_root
Hello from module_root
I will call module_sub
Hello from submodule

You are happy to GO! You can find sample code below.