10

This isn't the first time I am cringing over imports in Python. But I guess this one is an interesting use case, so I thought to ask it here to get a much better insight. The structure of my project is as follows:

sample_project
   - src
        - __init__.py
        - module1
           - __init__.py
           -  utils.py
        - module2
           - __init__.py 
           - models.py
        - app.py

The module1 imports methods from module2 and app imports method from all the other. Also, when you run the app it needs to create a folder called logs outside of src folder. There are now to ways to run the app:

  1. From inside src folder flask run app
  2. From outside of src folder flask run src.app

To make sure that I don't get import errors because of the change of the top level module where the app is started, I do this:

import sys
sys.path.append("..")

Is there any better solution to this problem?

enterML
  • 2,110
  • 4
  • 26
  • 38

4 Answers4

13

The pythonic solution for the import problem is to not add sys.path (or indirectly PYTHONPATH) hacks to any file that could potentially serve as top-level script (incl. unit tests), since this is what makes your code base difficult to change and maintain. Assume you have to reorganize your project structure or rename folders.

Instead this is what editable installs are made for. They can be achieved in 2 ways:

  1. pip install --editable <path> (requires a basic setup.py)
  2. conda develop <path> (requires the conda-build package)

Either way will add a symlink into your site-packages folder and make your local project behave as if it was fully installed while at the same time you can continue editing.

Always remember: KEEP THINGS EASY TO CHANGE

Peter
  • 10,959
  • 2
  • 30
  • 47
2

After doing research (here1, here2, here3, here4, here5, here6), I come up with the better solution at this time. This is in each python file, you can add its current path before import. The example code below:

import os
import sys
if os.path.dirname(os.path.abspath(__file__)) not in sys.path:
    sys.path.append(os.path.dirname(os.path.abspath(__file__)))
M.Vu
  • 397
  • 2
  • 9
0

Take a look at the Python import system documentation and at the PYTHONPATH environment variable.

When your code does import X.Y, what the Python runtime does is look in each folder listed in your PYTHONPATH for a package X (a package simply being a folder containing an __init__.py file) containing a Y package.

Most of the time, appending to sys.path is a poor solution. It is better to take care of what your PYTHONPATH is set to : check that it contains your root directory (which contains your top-level packages) and nothing else (except site-packages which i). Then, from wherever you run your commands, it will work the same (at least for imports, os.cwd is another problem).

Depending of the way to run your Python scripts, . may be the only relevant paths in it, so that it depends on your current directory, requiring to append .. if you run it from inside one of your top-level package.

And maybe you should not run your scripts from a directory that is not the root of your project ?

TL;DR : a good PYTHONPATH makes for way less import errors.

Lenormju
  • 4,078
  • 2
  • 8
  • 22
-1

Wraps the flask command into a small script in the sample_project directory and set PYTHONPATH according to your project:

#!/bin/env bash

# Assuming script is sample_project
path=`dirname ${BASH_SOURCE[0]}`
full_path=`realpath "$p"`
export PYTHONPATH=$full_path/src:$PYTHONPATH

flask run app

You can also switch current directory to a working directory.

But it is best to package your project using setuptools and install it (possibly in developpement mode), in user space according to PYTHONUSERBASE or in virtual environment.

Balaïtous
  • 826
  • 6
  • 9