Cgi, WSGI, UWSGI, uWSGI, uWSGI, uWSGI, uWSGI, etc. And try using it to run Django and Tornado framework apps.

Write a simple HTTP server

Before we can implement WSGi Server, we need to do some preparatory work. First of all, HTTP server uses HTTP protocol, and HTTP protocol encapsulated in TCP protocol, so to establish an HTTP server we need to establish a TCP server. To use THE TCP protocol we can not implement one of our own, now the more popular solution is to use socket programming, socket has helped us to implement the DETAILS of THE TCP protocol, we can directly use it without caring about the details. Socket programming is language agnostic. Whether bloggers used to write chat rooms in MFC or network latency calculations in C# or HTTP server today, the process is the same:

server

  1. Initialize the socket.

  2. Bind a socket to a port (bind);

  3. Listen port;

  4. Accept connection requests.

  5. Communication (send/recv);

  6. Close connection (close);

client

  1. Initialize the socket.

  2. Make a connect request;

  3. Communication (send/recv);

  4. Close connection (close);

Specific implementation of server:

# coding: Utf-8 # server.py import socket HOST, PORT = ", 8888 Setsockopt (socket.sol_socket, socket.so_reuseaddr, 1) # Listen_socket. Listen (1) print 'Serving HTTP on PORT %s... ' % PORT while True: Accept request client_connection, Client_address = listen_socket.accept() # communication request = client_connection.recv(1024) print Request http_response = """ HTTP/1.1 200 OK Hello, World! Client_connection.sendall (http_response) # Client_connection.close ()Copy the code

The client doesn’t need to be implemented by ourselves. Our browser is just a client. Now run Python server.py and open localhost:8888 in the browser to see hello World! There is no HIN excited about implementing an HTTP server so quickly!

However, it’s not enough for applications like Django to run on our HTTP servers. Now we need to introduce the WSGI specification so that our servers can run on these applications.

Write a standard WSGI Server

If you want your servers and applications to work well together, you need to follow this standard to write your Web apps and Web Servers:

server–middleware–application

application

Application is a standard WSGI app that accepts two parameters environ, start_Response:

Start_response: a method that accepts two parameters: status, response_headers: Return a status code, such as HTTP 200 or 404 response_headers: returns a list of headersCopy the code

Concrete implementation:

def application(environ, start_response):
    status = '200 OK'
    response_headers = [('Content-Type', 'text/plain')]
    start_response(status, response_headers)
    return ['Hello world']
Copy the code

This is a standard WSGI app. Although it may look very different from the Django app and Tornado App we wrote, these apps are actually processed to fit the WSGI standard. More on this later.

server

The wsGi Server implementation is a bit more complicated, so I’ll post my own WSGi Server code first and then explain:

# server.py # coding: utf-8 from __future__ import unicode_literals import socket import StringIO import sys import datetime class WSGIServer(object): socket_family = socket.AF_INET socket_type = socket.SOCK_STREAM request_queue_size = 10 def __init__(self, address): self.socket = socket.socket(self.socket_family, self.socket_type) self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.socket.bind(address) self.socket.listen(self.request_queue_size) host, port = self.socket.getsockname()[:2] self.host = host self.port = port def set_application(self, application): self.application = application def serve_forever(self): while 1: self.connection, client_address = self.socket.accept() self.handle_request() def handle_request(self): self.request_data = self.connection.recv(1024) self.request_lines = self.request_data.splitlines() try: self.get_url_parameter() env = self.get_environ() app_data = self.application(env, self.start_response) self.finish_response(app_data) print '[{0}] "{1}" {2}'.format(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), self.request_lines[0], self.status) except Exception, e: pass def get_url_parameter(self): self.request_dict = {'Path': self.request_lines[0]} for itm in self.request_lines[1:]: if ':' in itm: self.request_dict[itm.split(':')[0]] = itm.split(':')[1] self.request_method, self.path, self.request_version = self.request_dict.get('Path').split() def get_environ(self): env = { 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input': StringIO.StringIO(self.request_data), 'wsgi.errors': sys.stderr, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.run_once': False, 'REQUEST_METHOD': self.request_method, 'PATH_INFO': self.path, 'SERVER_NAME': self.host, 'SERVER_PORT': self.port, 'USER_AGENT': self.request_dict.get('User-Agent') } return env def start_response(self, status, response_headers): Headers = [(' Date and datetime. Datetime. Now (). The strftime (' % a, H: % d % % b Y % % M: % S GMT ')), (' Server ', 'RAPOWSGI0.1'), ] self.headers = response_headers + headers self.status = status def finish_response(self, app_data): try: Response = 'HTTP/1.1 {status}\r\n'. Format (status=self.status) for header in self.headers: response += '{0}: {1}\r\n'.format(*header) response += '\r\n' for data in app_data: response += data self.connection.sendall(response) finally: self.connection.close() if __name__ == '__main__': Port = 8888 if len(sys.argv) < 2: sys.exit(' Please provide a wsGi application available in the format: module name. ') elif len(sys.argv) > 2: port = sys.argv[2] def Generate_server (address, application): server = WSGIServer(address) server.set_application(TestMiddle(application)) return server app_path = sys.argv[1] module, application = app_path.split('.') module = __import__(module) application = getattr(module, application) httpd = generate_server(('', int(port)), application) print 'RAPOWSGI Server Serving HTTP service on port {0}'.format(port) print '{0}'.format(datetime.datetime.now(). strftime('%a, %d %b %Y %H:%M:%S GMT')) httpd.serve_forever()Copy the code

WSGIServer class __init__ initializes socket and server addresses, binds and listens to ports. Second, serve_forever(self): keep running the server; Handle_request (self): Handles requests; Finally, finish_response(self, app_data): Returns the request response. Initialize WSGIServer: server = WSGIServer(address) and set wsGi app to load: Server.set_application (TestMiddle(application)); Httpd.serve_forever () Then, based on the above information, wsGI Server should be like this:

  1. Initialize, set up a socket, bind the listening port;

  2. Set up the loaded Web app;

  3. Start continuously running the server;

  4. Processing access requests (you can add your own processing here, for example I added printing access information, lexicographing access headers, etc.);

  5. Get request information and environment information (get_environ(self));

  6. Run the loaded Web App with environ to get the return message;

  7. Construct the return message header;

  8. Return information;

Once the above process is implemented, a standard WSGi Server is written. On closer inspection, the key to a WSGi Server is to use itenvironGo to the web app and get the results back, which is complementary to the previous application implementation, and then the framework and the server follow the standards, and everyone can work happily together.

Now runpython server.py app.app 8000And then the browser visitslocalhost:8000:



The back-end



The browser

Now that wsGi Server is up and running, let’s take a look at Middleware:

middleware

The Middleware is used to do some processing or validation before the server gets the request data to the application. You can also write it in your server if you want. However, the WSGI specification recommends that this be written in middleware. Here I implement a middleware that checks if the request ‘user-agent’ is a normal browser and then rejects it:

# coding: utf-8
# middleware.py
from __future__ import unicode_literals


class TestMiddle(object):
    def __init__(self, application):
        self.application = application

    def __call__(self, environ, start_response):
        if 'postman' in environ.get('USER_AGENT'):
            start_response('403 Not Allowed', [])
            return ['not allowed!']
        return self.application(environ, start_response)
Copy the code

Init to receive the application, write the processing in the __call__ method, and return the application so that our middleware can be called as a function.

Then introduce middleware:

from middleware import TestMiddle

...

server.set_application(TestMiddle(application))Copy the code

Now restart the server and access the server with Postman:



As you can see, the middleware is working!

Next, let’s talk about Django and Tornado’s support for WSGI:

Django WSGI:

Django WSGI application

Django itself is too complex to use directly on the WSGi Server we wrote, but Django takes this into account and provides WSGIHandler:

class WSGIHandler(base.BaseHandler): request_class = WSGIRequest def __init__(self, *args, **kwargs): super(WSGIHandler, self).__init__(*args, **kwargs) self.load_middleware() def __call__(self, environ, start_response): set_script_prefix(get_script_name(environ)) signals.request_started.send(sender=self.__class__, environ=environ) try: request = self.request_class(environ) except UnicodeDecodeError: logger.warning( 'Bad Request (UnicodeDecodeError)', exc_info=sys.exc_info(), extra={ 'status_code': 400, } ) response = http.HttpResponseBadRequest() else: response = self.get_response(request) response._handler_class = self.__class__ status = '%d %s' % (response.status_code,  response.reason_phrase) response_headers = [(str(k), str(v)) for k, v in response.items()] for c in response.cookies.values(): response_headers.append((str('Set-Cookie'), str(c.output(header='')))) start_response(force_str(status), response_headers) if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'): response = environ['wsgi.file_wrapper'](response.file_to_stream) return responseCopy the code

As you can see, WSGIHandler also wraps Django App into a standard WSGI app using start_response(force_str(status), response_headers) and returns response.

Django WSGI server

Django also implements WSGi Server:

class WSGIServer(simple_server.WSGIServer, object):
    """BaseHTTPServer that implements the Python WSGI protocol"""

    request_queue_size = 10

    def __init__(self, *args, **kwargs):
        if kwargs.pop('ipv6', False):
            self.address_family = socket.AF_INET6
        self.allow_reuse_address = kwargs.pop('allow_reuse_address', True)
        super(WSGIServer, self).__init__(*args, **kwargs)

    def server_bind(self):
        """Override server_bind to store the server name."""
        super(WSGIServer, self).server_bind()
        self.setup_environ()

    def handle_error(self, request, client_address):
        if is_broken_pipe_error():
            logger.info("- Broken pipe from %s\n", client_address)
        else:
            super(WSGIServer, self).handle_error(request, client_address)Copy the code

Wsgiref.simple_server. WSGIServer:

class WSGIServer(HTTPServer): """BaseHTTPServer that implements the Python WSGI protocol""" application = None def server_bind(self): """Override server_bind to store the server name.""" HTTPServer.server_bind(self) self.setup_environ() def setup_environ(self): # Set up base environment env = self.base_environ = {} env['SERVER_NAME'] = self.server_name env['GATEWAY_INTERFACE'] = 'CGI / 1.1' env = [' SERVER_PORT] STR (self. SERVER_PORT) env = [' REMOTE_HOST '] 'env = [' CONTENT_LENGTH'] 'env = [' SCRIPT_NAME']  '' def get_app(self): return self.application def set_app(self,application): self.application = applicationCopy the code

As you can see, this is pretty much the same wsGi Server that we implemented.

Tornado WSGI

Tornado implements event pool operations, TCP Server, and HTTP Server directly from the bottom with epoll itself, so it is a completely different asynchronous framework, but Tornado also provides support for WSGI. But there is no way to use Tornado asynchronism in this case.

Tornado doesn’t provide WSGI support so much as wsGI compatibility. Tornado provides two ways:

WSGIContainer

Other applications should be run in Tornado Server, and Tornado provides WSGIContainer. Wsgi is mainly discussed here today, so tornado code will not be analyzed here. Tornado source code analysis will be done later.

WSGIAdapter

Tornado application should run on WSGI Server, tornado provides WSGIAdapter:

class WSGIAdapter(object): def __init__(self, application): if isinstance(application, WSGIApplication): self.application = lambda request: web.Application.__call__( application, request) else: self.application = application def __call__(self, environ, start_response): method = environ["REQUEST_METHOD"] uri = urllib_parse.quote(from_wsgi_str(environ.get("SCRIPT_NAME", ""))) uri += urllib_parse.quote(from_wsgi_str(environ.get("PATH_INFO", ""))) if environ.get("QUERY_STRING"): uri += "?" + environ["QUERY_STRING"] headers = httputil.HTTPHeaders() if environ.get("CONTENT_TYPE"): headers["Content-Type"] = environ["CONTENT_TYPE"] if environ.get("CONTENT_LENGTH"): headers["Content-Length"] = environ["CONTENT_LENGTH"] for key in environ: if key.startswith("HTTP_"): headers[key[5:].replace("_", "-")] = environ[key] if headers.get("Content-Length"): body = environ["wsgi.input"].read( int(headers["Content-Length"])) else: body = b"" protocol = environ["wsgi.url_scheme"] remote_ip = environ.get("REMOTE_ADDR", "") if environ.get("HTTP_HOST"): host = environ["HTTP_HOST"] else: host = environ["SERVER_NAME"] connection = _WSGIConnection(method, start_response, _WSGIRequestContext(remote_ip, Protocol) request = httputil. HTTPServerRequest (method, uri, "HTTP / 1.1," headers = headers, body = body, host = host, connection=connection) request._parse_body() self.application(request) if connection._error: raise connection._error if not connection._finished: raise Exception("request did not finish synchronously") return connection._write_bufferCopy the code

It can be seen that Tornado also changed its application to the standard WSGI APP using the process mentioned above. Finally, let’s try to run Tornado App on our own server:

# coding: utf-8
# tornado_wsgi.py

from __future__ import unicode_literals

import datetime
import tornado.web
import tornado.wsgi

from middleware import TestMiddle
from server import WSGIServer


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("this is a tornado wsgi application")


if __name__ == "__main__":
    application = tornado.web.Application([
        (r"/", MainHandler),
    ])
    wsgi_app = tornado.wsgi.WSGIAdapter(application)
    server = WSGIServer(('', 9090))
    server.set_application(TestMiddle(wsgi_app))
    print 'RAPOWSGI Server Serving HTTP service on port {0}'.format(9090)
    print '{0}'.format(datetime.datetime.now().
                       strftime('%a, %d %b %Y %H:%M:%S GMT'))
    server.serve_forever()Copy the code

Run:python tornado_wsgi.py, open the browser:localhost:9090And the middleware is working fine:

Simple_wsgi_server Let’s Build A Web Server

The original address

Author: rapospectre