If you’re like me, you occasionally write a useful Python utility applet that you’d like to share with your colleagues. The best way to do this is to make a Python package: it’s easy to install and avoids copying.
You might think that creating a package is a hassle. That’s not true anymore. I’ll explain with this step-by-step guide. There are only three main steps (and a series of optional steps) to follow, with a few GitHub links.
1. Initialization
We will create PodSearch – a utility that searches for podcasts in iTunes. Let’s create a directory and a virtual environment:
$ mkdir podsearch
$ cd podsearch
$ python3 -m venv env
$ . env/bin/activate
Copy the codeDefine a minimum package structure:
.├ ─.├ ─ podsearch ├─.├ ─.├ ─Copy the code"""Let's find some podcasts!"""
__version__ = "0.1.0 from"
def search(name, count=5) :"""Search podcast by name."""
raise NotImplementedError()
Copy the code2. The test package
Creating a package in Python used to be a cumbersome task. Fortunately, there are now a great flit small program can simplify all operations (https://flit.readthedocs.io/en/latest/). Let’s install it:
pip install flit
Copy the codeCreate package description:
$ flit init
Module name [podsearch]:
Author [Anton Zhiyanov]:
Author email [[email protected]]:
Home page [https://github.com/nalgeon/podsearch-py]:
Choose a license (see http://choosealicense.com/ for more info)
1. MIT - simple and permissive
2. Apache - explicitly grants patent rights
3. GPL - ensures that code based on this is shared with the same terms
4. Skip - choose a license later
Enter 14 - [1] :1
Written pyproject.toml; edit that file to add optional extra info.
Copy the codepyproject.toml
Flit has created the PyProject.toml – project metadata file. It already has everything you need to publish packages to the common repository -PyPI.
Register TestPyPi (test repository) and PyPI (primary repository). They are completely separate, so you will need two accounts.
Set access to the repository in ~/.pypirc:
[distutils]
index-servers =
pypi
pypitest
[pypi]
username: nalgeon # replace with your PyPI username
[pypitest]
repository: https://test.pypi.org/legacy/
username: nalgeon # replace with your TestPyPI username
Copy the codeAnd publish the package to the test repository:
$ flit publish --repository pypitest
Found 4 files tracked in git
...
Package is at https://test.pypi.org/project/podsearch/
Copy the codeFinished! The package is available on TestPyPi.
3. Expose the software package
Let’s improve the code so that it can actually search for podcasts:
#... SEARCH_URL ="https://itunes.apple.com/search"
@dataclass
class Podcast:
"""Podcast metadata."""
id: str
name: str
author: str
url: str
feed: Optional[str] = None
category: Optional[str] = None
image: Optional[str] = None
def search(name: str, limit: int = 5) -> List[Podcast]:
"""Search podcast by name."""
params = {"term": name, "limit": limit, "media": "podcast"}
response = _get(url=SEARCH_URL, params=params)
return _parse(response)
Copy the codeAnd publish to the main repository -PyPI. Perform this step only if your package contains useful code. Do not distribute invalid packages and stubs.
flit publish
Copy the codeRelease complete! It’s time to share it with colleagues. To make the package easy to use, I recommend you follow these steps.
A. Readme file and changelog changelog
No one likes to document. However, people are unlikely to want to install your package without documentation, so we need to add readme.md and Changelog.md.
README.mdCHANGELOG.md
Add README to PyProject.toml so that PyPI displays it on the package page:
description-file = "README.md"
Copy the codeAlso specify the lowest supported Python version:
requires-python = "> = 3.7"
Copy the codeUpdate the version in __init__.py and publish the package via flit publish:
B. Linters and tests
Let’s consider formatting (Black), test coverage, code quality (Flake8, PyLint, McCabe), and static analysis (MypY). We’re going to tox everything.
$ pip install black coverage flake8 mccabe mypy pylint pytest tox
Copy the codeCreate tox configuration in tox.ini:
[tox]
isolated_build = True
envlist = py37,py38,py39
[testenv]
deps =
black
coverage
flake8
mccabe
mypy
pylint
pytest
commands =
black podsearch
flake8 podsearch
pylint podsearch
mypy podsearch
coverage erase
coverage run --include=podsearch/* -m pytest -ra coverage report -mCopy the codetox.ini
And run all checks:
$ tox -e py39
...
py39 run-test: commands[0] | black podsearch All done! . py39 run-test: commands[2] | pylint podsearch
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)... py39 run-test: commands[6] | coverage report -m
TOTAL 100%... py39: commands succeeded congratulations :)Copy the codeLinters passed their tests, they passed their tests, with 100 percent coverage.
C. the clouds build
Every solid open source project does cloud testing after every commit, so we will do the same. A nice side effect is the nice badge in the readme file.
Let’s build the project using GitHub Actions, use Codecov to check test coverage, and use Code Climate to check Code quality.
You will have to register Codecov and Code Climate (both with GitHub login support) and enable the package repository in your Settings.
After that, add the GitHub Actions build configuration to. GitHub/workflows/build.yml:
#... jobs: build: runs-on: ubuntu-latest strategy: matrix: python-version: [3.7.3.8.3.9]
env:
USING_COVERAGE: "3.9"
steps:
- name: Checkout sources
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: $
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install black coverage flake8 flit mccabe mypy pylint pytest tox tox-gh-actions
- name: Run tox
run: |
python -m tox
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v1
if: contains(env.USING_COVERAGE, matrix.python-version)
with:
fail_ci_if_error: true
Copy the codebuild.yml
Just like we did before, GitHub is tox tested. The tox-gh-Actions package and the USING_COVERAGE setting ensure that tox uses the same Python version of GitHub Actions required by Strategy.matrix.
The final step is to send the test coverage to Codecov. Code Climate does not require a separate step – it automatically detects repository changes.
Now, submit in a minute, push and enjoy the results. And let everyone enjoy adding badges to readme.md too:
[![PyPI Version][pypi-image]][pypi-url] [![Build Status][build-image]][build-url] [![Code Coverage][coverage-image]][coverage-url] [![Code Quality][quality-image]][quality-url] ... <! -- Badges --> [pypi-image]: https://img.shields.io/pypi/v/podsearch
[pypi-url]: https://pypi.org/project/podsearch/
[build-image]: https://github.com/nalgeon/podsearch-py/actions/workflows/build.yml/badge.svg
[build-url]: https://github.com/nalgeon/podsearch-py/actions/workflows/build.yml
[coverage-image]: https://codecov.io/gh/nalgeon/podsearch-py/branch/main/graph/badge.svg
[coverage-url]: https://codecov.io/gh/nalgeon/podsearch-py
[quality-image]: https://api.codeclimate.com/v1/badges/3130fa0ba3b7993fbf0a/maintainability
[quality-url]: https://codeclimate.com/github/nalgeon/podsearch-py
Copy the codeIsn’t that cool?
D. Task automation
Tox is good, but not very convenient for development. Running a single command (e.g. Pylint, Coverage, etc.) is faster. But they are verbose, so we automate some meaningless operations.
Let’s create short aliases for frequent operations on makefiles:
.DEFAULT_GOAL := help
.PHONY: coverage deps help lint push test
coverage: ## Run tests with coverage
coverage erase
coverage run --include=podsearch/* -m pytest -ra coverage report -m deps: ## Install dependencies pip install black coverage flake8 mccabe mypy pylint pytest tox lint: ## Lint and static-check flake8 podsearch pylint podsearch mypy podsearch push: ## Push code with tags git push && git push --tags test: ## Run tests pytest -raCopy the codeMakefile
This is our mission:
$ make help
Usage: make [task]
task help
------ ----
coverage Run tests with coverage
deps Install dependencies
lint Lint and static-check
push Push code with tags
test Run tests
help Show help message
Copy the codeTo make the code more concise, replace the original build.yml step with a make call:
- name: Install dependencies
run: |
make deps
- name: Run tox
run: |
make tox
Copy the codeE. cloud
GitHub has the ability to run Flit Publish for us. Let’s create a separate workflow:
name: publish
on:
release:
types: [created]
jobs:
publish:
runs-on: ubuntu-latest
steps:
- name: Checkout sources
uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: "3.9"
- name: Install dependencies
run: |
make deps
- name: Publish to PyPi
env:
FLIT_USERNAME: ${{ secrets.PYPI_USERNAME }}
FLIT_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
make publish
Copy the codepublish.yml
PYPI_USERNAME and PYPI_PASSWORD are set in repository Settings (Settings > Secrets > New Repository secret). Use your PyPi username and password, or even better -API token.
GitHub now automatically publishes the package once a new version is created.
Your package is ready! It has everything anyone could dream of: clean code, clean documentation, testing, and cloud builds. It’s time to tell your colleagues and friends.
These Settings will make your Python package AWESOME:
pyproject.tomltox.iniMakefilebuild.ymlpublish.yml
Read more
5 minutes to get started with pyTest test framework
5 minutes to master Python random hill-climbing algorithm \
5 minutes to quickly master Adam optimization algorithm \
Special recommendation \
Click below to read the article and join the community