preface

In the process of using Python in our daily projects, we often need to solve the import problems of various modules, but also often encounter the reference path can not find, cross-import modules and other problems, so THIS article aims to describe the Python module import mechanism and we should follow the module import specification in our daily large projects

Python Module import

In daily programming, in order to reuse the written code logic, we will package these codes into modules, which can be directly imported and reused when needed, so as to improve our development efficiency. A module can define functions, classes, variables, and can contain executable code. Modules come from three sources: ①Python built-in modules (standard library); ② Third-party module; ③ Custom module;

Import the principle

Module imports are typically used in file headersimportImport a module executes the imported module once, and then establishes a relationship with the imported module’s namespace in the local namespace. It creates a new variable in the local namespace. The variable name is the name of the imported module, pointing to the imported module’s namespace. So this imported module is equivalent to a variable,Therefore, importing the same module more than once will only be performed the first time(Subsequent imports will determine that this module variable already exists, so do not execute)

Pathfinding mechanism

Modules. When Python is started, it will be loaded into memory. When we import a new module, sys.modules will automatically record the module. Python modules find paths by:

  1. Check whether the module exists in all paths in sys.path. If yes, create a new space to load the module.
  2. Check sys.modules to see if there is a built-in package or installed third-party package, if there is a new space to load the module;

So for modules we wrote ourselves, if packaged and published to PyPi, it can be installed directly with PIP Install and loaded in memory at startup, which can be viewed through sys.modules, while for modules that only need to be reused in this project, We added its path to sys.path in the reuse code, which can also reference this module.

Absolute path import

All module imports start at the “root node”. The location of the root node is determined by the path in sys.path. If you want the program to run everywhere, you need to manually modify sys.path

import sys,os
BASE_DIR = os.path.dirname(os.path.abspath(__file__))The absolute path to the project root directory

sys.path.append(BASE_DIR)

import A, B Import package A and package B
Copy the code

Relative path import

Just care about the module location relative to your current directory. Cannot execute directly inside the package (error). Modules in the package are positioned correctly relative to each other regardless of where the root node is.

#from. Import b2 # This type of import will report errors and can only be imported if the package is executed directly within the package.
import b2# right
b2.print_b2()
Copy the code

Common Problems with Python module import

  • When a package name is individually imported, all submodules contained in the package are not imported

Solution: The imported package also contains the imports of other packages. In this case, you need to import all modules under the package in the __init__.py file of each package, so that the top layer can directly reference the classes and methods of the bottom layer package

__init__ file

When a folder has __init__.py, it means that the folder is a package with multiple modules that can be imported into other code through the same package. The __init__.py file is used to organize packages, facilitate the management of references between modules, and control the import behavior of packages.

The file can be empty (empty, you can’t do anything with import) and just exist, acting as a tag.

In python3, the import package does not report an error even if the __init__.py file is not in the package, whereas in python2, the file must be in the package otherwise the import package will report an error

__all__ variable

All is an important variable that specifies which modules will be imported into the current scope when this package is imported *. Modules that are not in the __all__ list are not referenced by other programs. It is possible to override __all__, such as all= [‘ current package module 1 name ‘, ‘module 1 name’], if this is written, it will be imported by module names in the list

__name__ variable

When running directly inside the package, the package’s __name__ == ‘main’, but when importing the package externally, it can pass

if __name__ == '__main__':
Copy the code

To avoid implementing logic for package internal debugging

Cycle import

Can not import name XXX, can not import name XXX, can not import name XXX, can not import name XXX

# a.py
print('from a.py')
from b import x

y = 'a'
Copy the code
# b.py
print('from b.py')
from a import y

x = 'b'
Copy the code

Let’s analyze how this error occurs:

  1. Look for the symbol “module B” in sys.modules;
  2. If the symbol “module b” exists, get the module object corresponding to the symbol “module B”;

Gets the object corresponding to the symbol “x” from __dict__ of. If “x” does not exist, ImportError: cannot import name ‘x’ is raised. If the symbol “module b” does not exist, a new module object is created. However, the __dict__ of the new Module object is empty. Then execute the statement in the module B. py file, padding __dict__.

From a to B import y from b import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y from B import y We didn’t get to x=’b’ when we introduced b

The solution

  1. Deferred import, in which the import statement is written in a method/function and its scope is limited locally;
  2. Import from x import y to import x.y;
  3. In fact, the root cause of cyclic reference problem is unreasonable program design, each package should be used by the upper module to import, and should not be in the package and package between the various mutual import, so should change the code layout, can merge or separate competing resources;

Import specifications for Python modules in large projects

  1. Separate modules and put modules of the same category in the same directory to form a categorically distinct directory architecture. For example:

2. Each module directory must write an __init__.py file, which can also define __all__ to limit the scope of import; 3. You can define BASE_DIR in the source root directory, specify the root directory path, start py file can use the absolute path to import each module, add all necessary modules to sys.path; 4. Between services (for example, model needs to introduce common’s module method), modules can be referenced through a relative path; 5. Avoid circular import during program design, callers (service files) can be used as the upper third party to introduce the required modules, so that the mutual import of each module can be reduced.

reference

  1. www.cnblogs.com/Zzbj/p/9607…
  2. Cloud.tencent.com/developer/a…
  3. Blog.csdn.net/weixin_3825…