background

Since the current Flask project involves some Windows-dependent processing, it cannot be migrated to Linux platform. So how to deploy it in Windows environment?

Train of thought

According to the Flask website, due to the poor performance of Flask’s built-in server, the main recommended deployment methods are as follows:

  • mod_wsgi (Apache)

  • A standalone WSGI container

    • Gunicorn
    • Tornado
    • Gevent
  • uWSGI
  • FastCGI
  • CGI

Among the above deployment methods, only Tornado supports deployment under Windows, and it can achieve better results when combined with NGINX. The concurrent evaluation of NGINX and Tornado framework can be referred to.

However, in practice, it has been found that although Tornado has a high stability, deploying FLASK on Tornado does not have an asynchronous effect. In fact, it will run in a single-process block, even if threaded = True is configured in Flask, it will not be multithreaded.

Flask multithreading

Configure enabling multithreading:

# manage.py
from flask_script import Server

server = Server(host="0.0.0.0", threaded=True)

Configure the two test routes in Flask

import time
@main.route('/test')
def maintest():
    return 'hello world'
    
@main.route('/sleep')
def mainsleep():
    time.sleep(60)
    return 'wake up'

First access \sleep with a browser:

Immediately access \test:

It can be seen that the two accesses are handled by different threads, and there will be no blockage.

Tornado + Flask multithreaded case

Using Tornado hosting:

from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from yourapplication import app

http_server = HTTPServer(WSGIContainer(app))
http_server.listen(5000)
IOLoop.instance().start()

First access \sleep with a browser:

Immediately access \test:

It can be found that although Tornado framework supports asynchronous processing, asynchronous processing cannot be achieved due to the fact that background processing is synchronous. If you want background processing to be asynchronous, you need to develop it directly using Tornado.

So why use Tornado to host Flask?

TornadoIs an open source, scalable, non-blocking web server and toolset that drives the
FriendFeed. Because it uses the Epoll model and is non-blocking, it can handle thousands of concurrent fixed connections, which means it is ideal for real-time Web services. Integrating Flask with this service is straightforward

According to the description on the official website, it is also to solve the problem of unstable server of Flask.

Flask performance under high concurrency

Pressure test with Tsung, pressure 500:

Name highest 10sec mean lowest 10sec mean Highest Rate Mean Rate Mean Count
connect 34.30 msec 31.91 msec 506 / sec 356.60 / SEC 33.19 msec 103908
page 0.42 SEC 0.29 SEC 505 / sec 356.32 / SEC 0.39 SEC 103782
request 0.42 SEC 0.29 SEC 505 / sec 356.32 / SEC 0.39 SEC 103782
session 1mn 24sec 10.64 SEC 11.4 / SEC 1.21 / SEC 14.24 SEC 362
Code Highest Rate Mean Rate Total number
200 505 / sec 356.32 / SEC 104792
Name Highest Rate Total number
error_abort 0.5 / SEC 1
error_abort_max_conn_retries 11.7 / SEC 362
error_connect_econnrefused 58.6 / SEC 1667

As you can see, at 500 concurrency, it doesn’t work very well, with a lot of link rejections.

Flask + Nginx performance at high concurrency

  • usetsungPressure test, pressure 500:
Name highest 10sec mean lowest 10sec mean Highest Rate Mean Rate Mean Count
connect 0.20 SEC 30.95 msec 1810.5 / SEC 626.43 / SEC 0.11 SEC 189853
page 0.68 SEC 0.17 SEC 1810.1 / SEC 625.72 / SEC 0.40 SEC 189581
request 0.68 SEC 0.17 SEC 1810.1 / SEC 625.72 / SEC 0.40 SEC 189581
Code Highest Rate Mean Rate Total number
200 906.4 / SEC 196.08 / SEC 60689
502 1443.9 / SEC 430.02 / SEC 129006
Name Highest Rate Total number
error_abort 0.5 / SEC 1

If the situation is similar and the Flask server is relatively stable, try increasing the number of backend Flask servers (with multiple ports) :

python manage.py runserver --port=8001
python manage.py runserver --port=8002
python manage.py runserver --port=8003
python manage.py runserver --port=8004
  • usetsungPressure test, pressure 500,4FlaskServer:
Name highest 10sec mean lowest 10sec mean Highest Rate Mean Rate Mean Count
connect 0.18 SEC 32.57 msec 3510.1 / SEC 639.92 / SEC 0.11 SEC 195154
page 0.49 SEC 85.30 msec 3512.1 / SEC 639.07 / SEC 0.35 SEC 194856
request 0.49 SEC 85.30 msec 3512.1 / SEC 639.07 / SEC 0.35 SEC 194856
Code Highest Rate Mean Rate Total number
200 3510.1 / SEC 639.50 / SEC 194986
Name Highest Rate Total number
error_abort 0.333333333333333 / SEC 1

That’s fine.

  • usetsungPressure test, pressure 1000,4FlaskServer:
Name highest 10sec mean lowest 10sec mean Highest Rate Mean Rate Mean Count
connect 0.20 SEC 32.63 msec 2983.8 / SEC 492.94 / SEC 98.56 msec 150793
page 0.57 SEC 90.00 msec 2976.4 / SEC 491.31 / SEC 0.40 SEC 150275
request 0.57 SEC 90.00 msec 2976.4 / SEC 491.31 / SEC 0.40 SEC 150275
Code Highest Rate Mean Rate Total number
200 2981.4 / SEC 488.92 / SEC 149556
502 92.5 / SEC 4.02 / SEC 925
Name Highest Rate Total number
error_abort 0.333333333333333 / SEC 1

There are some 502 timeout errors starting.

  • usetsungPressure test, pressure 1000,4tornadoServer:
Name highest 10sec mean lowest 10sec mean Highest Rate Mean Rate Mean Count
connect 0.18 SEC 86.24 msec 2052.1 / SEC 693.82 / SEC 0.14 SEC 208786
page 0.52 SEC 0.24 SEC 2060.7 / SEC 693.34 / SEC 0.45 SEC 208606
request 0.52 SEC 0.24 SEC 2060.7 / SEC 693.34 / SEC 0.45 SEC 208606
Code Highest Rate Mean Rate Total number
200 2056.6 / SEC 693.67 / SEC 208703

In the case of 1000 concurrency, whether Tornado hosted Flask is used or not has similar effect.

conclusion

According to the above tests, using the Flask server directly, due to the weak concurrent processing, will have a variety of timeout or connection refused errors. Caching is done with NGINX, and concurrency is provided by increasing the number of back-end servers.

So I finally chose the way of NGINX + 4 Flask servers in the background. Since the total number of users of the Flask project is only a few thousand, the current concurrency situation is very low, so this method is completely satisfactory.

If you are in a larger project with tens of thousands of concurrent projects, it is recommended that you still consider moving to a LiunX environment and deploying it as officially recommended.