Cc /posts/2019-…

Photo source network.

As Docker has grown in popularity, it has become the first choice for deployment projects. Today, I will share with you how to deploy Django stack projects using Docker.

The Django stack we are talking about here is python3.6, Django2.2, Redis, mysql, celery, gunicorn and nginx. In a real production project, where these components are distributed across different machines in a cluster, Nginx, Redis, and Mysql might have separate teams or departments responsible for them. The deployment architecture and container choreography involved are more complex and will not be explored in this article. This article focuses on how to orchestrate these components using Docker-compose, which is suitable for deployment in a test environment or your personal Sideproject.

Docker and Docker-compose are composed for docker, docker-compose, docker-compose, docker-compose, docker-compose, docker-compose, docker-compose, docker-compose

  • Docker
  • Docker official documentation
  • Docker Compose documents

Here’s how to deploy it.

Project organization Structure

First of all, let’s take a look at the organizational structure of our project, which is as follows:

├ ─ ─ LICENSE ├ ─ ─ the README. Md ├ ─ ─ compose │ ├ ─ ─ celery │ │ ├ ─ ─ Dockerfile │ │ ├ ─ ─ celery - beat. Sh │ │ └ ─ ─ celery. Sh │ ├ ─ ─ │ ├─ ├─ ├─ web │ ├─ ├─ en.gz │ ├─ en.gz │ ├─ en.gz │ ├─ en.gz │ ├─ en.gz │ ├─ en.gz │ ├─ en.gz │ ├─ en.gz └ ─ ─ gunicorn. Sh ├ ─ ─ docker - compose. Yml ├ ─ ─ docker_django_demo │ ├ ─ ─ just set py │ ├ ─ ─ celery. Py │ ├ ─ ─ Settings. Py │ ├ ─ ─ Urls.py │ ├── env.tpl │ ├─ manage.py │ ├─ requirementsCopy the code

In addition to the Django project files, the compose configuration file directory and the docker-comemage. yml configuration file have been added.

  • composeThe directory stores the dockerfile files and startup scripts of each component.
  • docker-compose.ymlIs the orchestration configuration file for Docker-compose.

Write the Dockerfile and start the initialization script

In Docker-compose, there are two ways to start a container, one is to start the container directly with a public image, the other is to write our own Dockerfile. As we are installing additional toolkits and initializing associated configurations, web and celery components we use custom Dockerfile mode.

Compose /Dockerfile for the web container:

FROM python:3.6
ENV PYTHONUNBUFFERED 1

RUN mkdir /code
WORKDIR /code

COPY ./requirements.txt /code/
RUNpip install --no-cache-dir -r requirements.txt \ && rm -rf requirements.txt
COPY . /code/
COPY ./compose/web/*.sh /code/
RUN sed -i 's/\r//' gunicorn.sh \
    && chmod +x gunicorn.sh \
    && sed -i 's/\r//' entrypoint.sh \
    && chmod +x entrypoint.sh

ENTRYPOINT ["/bin/bash"."entrypoint.sh"]
Copy the code

Other files for the Web container:

  • compose/web/entrypoint.shA startup script for the Web container that performs some initialization or detection logic.
  • compose/web/gunicorn.confGunicorn configuration file.
  • compose/web/gunicorn.shGunicorn startup script.

Celery Dockerfile:

FROM python:3.6
ENV PYTHONUNBUFFERED 1

RUN mkdir /code
WORKDIR /code

COPY ./requirements.txt /code/
COPY ./compose/celery/*.sh /code/
RUNpip install --no-cache-dir -r requirements.txt \ && rm -rf requirements.txt && sh init_env.sh
COPY . /code/
COPY ./compose/celery/*.sh /code/
RUN sed -i 's/\r//' celery.sh \
    && chmod +x celery.sh \
    && sed -i 's/\r//' celery-beat.sh \
    && chmod +x celery-beat.sh
Copy the code

Celery other documents:

  • compose/celery/celery.shCelery startup script.
  • compose/celery/celery-beat.shCelery -beat startup script.

Write the Compose startup configuration file

Docker-compose configuration is as follows:

Version: '2' services: redis: image: redis ports: - "6379:6379" db: restart: always image: mysql:5.7.19 # command: --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci volumes: - ./compose/mysql/:/etc/mysql/conf.d - ./db:/var/lib/mysql # for test ports: - "127.0.0.1:3307:3306" # (HOST:CONTAINER) env_file: -. Env web: # restart: always build: context:.dockerfile: ./compose/web/Dockerfile command: sh gunicorn.sh # ["/bin/bash", "gunicorn.sh"] ports: - "8080:8002" # (HOST:CONTAINER) volumes: - ./logs:/var/logs/ - ./collect_static:/code/collect_static - ./static:/code/static - ./templates:/code/templates - /uploads:/code/uploads env_file:. Env depends_on: -redis - db nginx: restart: always image: nginx:1.13.0 volumes: - ./compose/nginx:/etc/nginx/conf.d/ - ./staticfiles:/code/staticfiles - ./logs:/var/log/nginx ports: - "80:80" # (HOST:CONTAINER) depends_on: - web celery: build: context: . dockerfile: ./compose/celery/Dockerfile command: sh celery.sh volumes: - ./logs:/var/logs/ - ./uploads:/code/uploads depends_on: - redis - db env_file: .env celery-beat: build: context: . dockerfile: ./compose/celery/Dockerfile command: sh celery-beat.sh volumes: - ./logs:/var/logs/ depends_on: - redis - db env_file: .envCopy the code

Here we use the same image Dockerfile, start two containers to run worker and beat processes separately according to the principle of one image one process.

Compile testing

After writing the configuration file, compile the image test run:

docker-compose build 
docker-compose up  # foreground run
docker-compose up -d  Can run in background after no error
Copy the code

Docker-compose ps can now see the docker-compose container:

$ docker-compose ps Name Command State Ports -------------------------------------------------------------------------------------------------- dockerdjangodemo_celery-beat_1 sh celery-beat.sh Up dockerdjangodemo_celery_1 sh celery.sh Up dockerdjangodemo_db_1 Sh mysqld Up 127.0.0.1:3307->3306/ TCP dockerdjangodemo_nginx_1 nginx -g daemon off; Up 0.0.0.0:80->80/ TCP dockerdjangodemo_redis_1 docker-entrypoint.sh redis... Up 0.0.0.0:6379->6379/ TCP dockerdjangodemo_web_1 /bin/bash entrypoint.sh sh... The Up 0.0.0.0:8080 - > 8002 / TCPCopy the code

You can adjust the port mapping based on the actual situation.

The problem

Here are a few things to be aware of during the build process.

Mysql coding problem

Mysql image provided by Docker, the default encoding is latin1, when saving Chinese will display garbled characters. The official provides a way to change the encoding mode, after launching the script to specify the encoding format, documentation can be seen here. The mysql container version 5.7.19 can be directly followed by the parameter –character-set-server= UTf8mb4 after command in docker-comemage. yml – collation – server = utf8mb4_general_ci. In this way, you just change the server code. All encoding formats can be specified directly using configuration file overrides.

The configuration is as follows:

[mysqld] default-storage-engine=INNODB character-set-server=utf8mb4 collation-server=utf8mb4_general_ci init-connect='SET NAMES utf8mb4' init_connect='SET collation_connection = utf8mb4_general_ci' Bd-address = 0.0.0.0; bd-address = 0.0.0.0Copy the code

Note: mysql 5.7.19 configuration file mode is successful, 5.7.4, 5.7.17 failed, please refer to.

Web wait for mysql to start and then continue

The mysql container cannot accept database links before it is started. During web startup initialization, if the database is not started properly, the Web container will fail to be started and exit. We can add detection scripts when the Web container is started, and continue after the database is connected.

The script is as follows:

#! /usr/bin/env bash
set -o errexit
set -o pipefail

echo $MYSQL_PASSWORD
echo $MYSQL_DATABASE
echo $MYSQL_HOST
echo $MYSQL_USER
echo $MYSQL_PORT

function mysql_ready(){
python << END
import sys
import pymysql
try:
    conn = pymysql.connect(host="db", port=3306, user="root", passwd="$MYSQL_ROOT_PASSWORD", db='$MYSQL_DATABASE', charset='utf8')
except pymysql.err.OperationalError:
    sys.exit(-1)
sys.exit(0)
END
}

until mysql_ready; do2 > &echo "MySQL is unavailable - sleeping"
  sleep 1
done2 > &echo "MySQL is up - continuing..."

Copy the code

conclusion

Docker-django-demo is used to deploy django-stack services. Docker-django-demo is used to deploy django-stack services.

At the beginning of the article, the deployment mode is not suitable for online production services of large websites, and there are many problems such as high coupling degree and difficult maintenance. However, deploying your own sideproject or test environment is a good idea when hardware resources are limited. In addition to reducing the hassle of setting up the environment, migration is also very convenient.

There are also some cases of how to use Docker in the development environment in the demo project, but I always think docker is more suitable for deployment, and convenient in the development environment is not as flexible and convenient as directly built. Welcome everyone to leave a comment and discuss the use experience of Docker in development and deployment.

reference

  • cookiecutter-django