• How to Set up a Perfect Python Project
  • Written by Brendan Maginnis
  • Translator: Ya-ya
  • Proofreader: HelloGitHub- Cut slightly cold

When starting a new Python project, it’s easy to jump in and start coding. Taking a little time to choose the right library can save a lot of time and make coding a much happier experience later on.

In an ideal world, the relationships of all developers would be interdependent and related (collaborative development), the code would be perfectly formatted, there would be no low-level errors, and the tests would cover all the code. In addition, all of this will be guaranteed every time you commit. (Uniform code style, type detection, high test coverage, automatic detection)

In this article, I’ll show you how to build a project that does these things. You can follow the steps or skip right to the project generation section using the Cookiecutter (veteran). First, let’s create a new project directory:

mkdir best_practices
cd best_practices
Copy the code

Pipx installs the Python tripartite library command line tool

Pipx is a command-line tool that can be used to quickly install Python’s tripartite libraries. We’ll use it to install pipenv and cookiecutter. Install pipx with the following command:

python3 -m pip install --user pipx
python3 -m pipx ensurepath
Copy the code

Use Pipenv for dependency management

Pipenv automatically creates and manages virtualEnv (virtual environment) for your project and adds/removes packages from Pipfile when installing/uninstalling them. It also generates the all-important pipfile.lock to ensure dependency reliability.

Knowing that you and your teammates are using the same version of the library greatly increases the confidence and enjoyment of programming. Pipenv is a good solution to the problem of using the same library with different versions. Pipenv has gained a lot of attention and recognition in the past period of time, and you can use it safely. The installation command is as follows:

pipx install pipenv
Copy the code

Use black and isort for code formatting

Black can format our code:

Black is an uncompromising Library for formatting Python code. By using it, you give up the details of manually formatting your code. In return, Black brings speed, certainty, and the hassle of tweaking Python code styles, freeing up more energy and time to focus on more important things.

No matter what project you’re reading, the code formatted with black looks pretty much the same. After a while format becomes less of an issue, so you can focus more on content.

Black makes code inspection faster by reducing code variability.

Isort sorts our imports section:

Isort sorts the Python package parts (imports) that you import, so you don’t have to sort your imports manually. It can sort imports alphabetically and automatically split them into parts.

Install them using Pipenv so that they do not mess up the deployment (you can specify that they are installed only in the development environment) :

pipenv install black isort --dev
Copy the code

Black and isort are not compatible with the default option, so we will have isort follow the principles of Black. Create a setup.cfg file and add the following configuration:

[isort]
multi_line_output=3
include_trailing_comma=True
force_grid_wrap=0
use_parentheses=True
line_length=88
Copy the code

We can run these tools using the following commands:

pipenv run black
pipenv run isort
Copy the code

Flake8 ensures code style

Flake8 ensures that the code follows the standard Python code specifications defined in PEP8. Install using pipenV:

pipenv install flake8 --dev
Copy the code

Like isort, it requires some configuration to work well with Black. Add these configurations to setup.cfg:

[flake8]
ignore = E203, E266, E501, W503
max-line-length = 88
max-complexity = 18
select = B,C,E,F,W,T4
Copy the code

Now we can run Flake8 with the command: pipenv run Flake8.

Use mypy for static type checking

Mypy is Python’s optional static type checker, designed to combine the best of dynamic (or “duck”) typing with static typing. Mypy combines the expressiveness and convenience of Python with compile-time type checking for a powerful type system, running them using any Python VM with virtually no runtime overhead.

Using types in Python takes a little getting used to, but the benefits are huge. As follows:

  • Static typing makes programs easier to understand and maintain
  • Static typing can help you catch errors earlier and reduce testing and debugging time
  • Static typing can help you find hard-to-find errors before your code goes into production
pipenv install mypy --dev
Copy the code

By default, Mypy recursively checks for type comments for all imported packages, and an error is reported when the library does not contain these comments. We need to configure mypy to run only on our code and ignore import errors without type comments. We assume that our code is in the best_practices package configured below. Add this to setup.cfg:

[mypy]
files=best_practices,test
ignore_missing_imports=true
Copy the code

Now we can run mypy:

pipenv run mypy
Copy the code

This is a useful cheat sheet.

Test with PyTest and Pytest-coV

Writing tests with PyTest is very easy, and removing the drag of writing tests means you can write more tests quickly!

pipenv install pytest pytest-cov --dev
Copy the code

Here’s a simple example from the PyTest website:

# content of test_sample.py
def inc(x) :
    return x + 1

def test_answer() :
    assert inc(3) = =5
Copy the code

To execute it:

$ pipenv run pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item

test_sample.py F                                                     [100%]

================================= FAILURES =================================
_______________________________ test_answer ________________________________

    def test_answer():
>       assert inc(3) == 5E assert 4 == 5 E + where 4 = inc(3) test_sample.py:6: AssertionError = = = = = = = = = = = = = = = = = = = = = = = = = 1 failed in 0.12 seconds = = = = = = = = = = = = = = = = = = = = = = = = =Copy the code

All of our test code is in the test directory, so please add this directory to setup.cfg:

[tool:pytest]
testpaths=test
Copy the code

If you also want to see test coverage. Create a new file,.coveragerc, specifying that only coverage statistics for our project code are returned. For the example best_practices project, set as follows:

[run]
source = best_practices

[report]
exclude_lines =
    # Have to re-enable the standard pragma
    pragma: no cover

    # Don't complain about missing debug-only code:
    def __repr__
    if self\.debug

    # Don't complain if tests don't hit defensive assertion code:
    raise AssertionError
    raise NotImplementedError

    # Don't complain if non-runnable code isn't run:
    if 0:
    if __name__ == .__main__.:
Copy the code

Now we can run the tests and look at coverage.

pipenv run pytest --cov --cov-fail-under=100
Copy the code

— CoV-fail-Under =100 specifies that the project will fail if its test coverage is less than 100%.

The pre – commit Git hooks

Git hooks let you run scripts whenever you want to commit or push. This enabled us to automatically run all the checks and tests on every commit/push. Pre-commit makes it easy to configure these hooks.

Git hook scripts are useful for identifying simple problems before submitting code for review. We will run hooks on every commit to automatically point out problems in the code, such as missing semicolons, trailing whitespace, and debug statements. By pointing out these issues before code review, code reviewers can focus on the changed code content and not waste time dealing with trivial style issues.

Here, we configure all of the above tools to execute when we commit Python code changes (Git commit), and then run PyTest Coverage only when we push (because the test is at the last step). Create a new file.pre-commit-config.yaml with the following configuration:

repos:
  - repo: local
    hooks:
      - id: isort
        name: isort
        stages: [commit]
        language: system
        entry: pipenv run isort
        types: [python]

      - id: black
        name: black
        stages: [commit]
        language: system
        entry: pipenv run black
        types: [python]

      - id: flake8
        name: flake8
        stages: [commit]
        language: system
        entry: pipenv run flake8
        types: [python]
        exclude: setup.py

      - id: mypy
        name: mypy
        stages: [commit]
        language: system
        entry: pipenv run mypy
        types: [python]
        pass_filenames: false

      - id: pytest
        name: pytest
        stages: [commit]
        language: system
        entry: pipenv run pytest
        types: [python]

      - id: pytest-cov
        name: pytest
        stages: [push]
        language: system
        entry: pipenv run pytest --cov --cov-fail-under=100
        types: [python]
        pass_filenames: false
Copy the code

If you need to skip these hooks, run git commit –no-verify or git push –no-verify

Generate the project using the Cookiecutter

Now that we know what is in the ideal project, we can convert it to a template so that we can generate a new project containing these libraries and configurations using a single command:

pipx run cookiecutter gh:sourcery-ai/python-best-practices-cookiecutter
Copy the code

Fill in the project name and warehouse name and a new project will be generated for you.

To complete the setup, perform the following steps:

# Enter project directory
cd <repo_name>

# Initialise git repo
git init

# Install dependencies
pipenv install --dev

# Setup pre-commit and pre-push hooks
pipenv run pre-commit install -t pre-commit
pipenv run pre-commit install -t pre-push
Copy the code

The template project contains a very simple Python file and tests to try out the above tools. Once you have written the code and are comfortable with it, you can perform your first Git commit, and all the hooks will run.

Integration into the editor

Although it is exciting to know that the project’s code is always at the highest level at the time of submission. But it’s still a pain to find out there’s a problem after the code has all been changed (when it’s committed). So it’s much better to expose problems in real time.

When you save the file, take a moment to make sure the code editor runs these commands. Having immediate feedback means you can quickly fix any minor issues introduced while the code is still fresh in your mind.

I personally use some excellent Vim plug-ins to accomplish this task:

  • Ale runs Flake8 in real time and runs black, isort, and mypy when the file is saved
  • Projectionist integrated VIM-Test runs PyTest on file preservation

Welcome those who love technology and open source to join HG’s translation yipu series, please leave a comment and tell us.