WTF, after writing a blog all morning, accidentally shut it down, it seems that I need to make a draft box function. Rewrite it:

Cgi, WSGI, UWSGI, uWSGI, uWSGI, uWSGI, uWSGI, etc. And try using it to run Django and Tornado framework apps.

  • directory

  • Write a simple HTTP server
    • server
    • client
  • Write a standard WSGI Server
    • application
    • server
    • middleware
  • Django WSGI:
    • Django WSGI application
    • Django WSGI server
  • Tornado WSGI
    • WSGIContainer
    • WSGIAdapter

    directory


    Write a simple HTTP server

    Before we can implement WSGi Server, we need to do some preparatory work. First of all, HTTP server uses HTTP protocol, and HTTP protocol encapsulated in TCP protocol, so to establish an HTTP server we need to establish a TCP server. To use THE TCP protocol we can not implement one of our own, now the more popular solution is to use socket programming, socket has helped us to implement the DETAILS of THE TCP protocol, we can directly use it without caring about the details. Socket programming is language agnostic. Whether bloggers used to write chat rooms in MFC or network latency calculations in C# or HTTP server today, the process is the same:

    server

    1. Initialize the socket.
    2. Bind a socket to a port (bind);
    3. Listen port;
    4. Accept connection requests.
    5. Communication (send/recv);
    6. Close connection (close);

    client

    1. Initialize the socket.
    2. Make a connect request;
    3. Communication (send/recv);
    4. Close connection (close);

    Specific implementation of server:

    # coding: Utf-8 # server.py import socket HOST, PORT = ", 8888 Setsockopt (socket.sol_socket, socket.so_reuseaddr, 1) # Listen_socket. Listen (1) print 'Serving HTTP on PORT %s... ' % PORT while True: Accept request client_connection, Client_address = listen_socket.accept() # communication request = client_connection.recv(1024) print Request http_response = """ HTTP/1.1 200 OK Hello, World! Client_connection.sendall (http_response) # Client_connection.close ()Copy the code

    The client doesn’t need to be implemented by ourselves. Our browser is just a client. Now run Python server.py and open localhost:8888 in the browser to see hello World! There is no HIN excited about implementing an HTTP server so quickly!

    However, it’s not enough for applications like Django to run on our HTTP servers. Now we need to introduce the WSGI specification so that our servers can run on these applications.

    Write a standard WSGI Server

    If you want your servers and applications to work well together, you need to follow this standard to write your Web apps and Web Servers:

    Server, middleware, application

    application

    Application is a standard WSGI app that accepts two parameters environ, start_Response:

    Start_response: a method that accepts two parameters: status, response_headers: Return a status code, such as HTTP 200 or 404 response_headers: returns a list of headersCopy the code

    Concrete implementation:

    
    

    This is a standard WSGI app. Although it may look very different from the Django app and Tornado App we wrote, these apps are actually processed to fit the WSGI standard. More on this later.

    server

    The wsGi Server implementation is a bit more complicated, so I’ll post my own WSGi Server code first and then explain:

    # server.py # coding: utf-8 from __future__ import unicode_literals import socket import StringIO import sys import datetime class WSGIServer(object): socket_family = socket.AF_INET socket_type = socket.SOCK_STREAM request_queue_size = 10 def __init__(self, address): self.socket = socket.socket(self.socket_family, self.socket_type) self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) self.socket.bind(address) self.socket.listen(self.request_queue_size) host, port = self.socket.getsockname()[:2] self.host = host self.port = port def set_application(self, application): self.application = application def serve_forever(self): while 1: self.connection, client_address = self.socket.accept() self.handle_request() def handle_request(self): self.request_data = self.connection.recv(1024) self.request_lines = self.request_data.splitlines() try: self.get_url_parameter() env = self.get_environ() app_data = self.application(env, self.start_response) self.finish_response(app_data) print '[{0}] "{1}" {2}'.format(datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S'), self.request_lines[0], self.status) except Exception, e: pass def get_url_parameter(self): self.request_dict = {'Path': self.request_lines[0]} for itm in self.request_lines[1:]: if ':' in itm: self.request_dict[itm.split(':')[0]] = itm.split(':')[1] self.request_method, self.path, self.request_version = self.request_dict.get('Path').split() def get_environ(self): env = { 'wsgi.version': (1, 0), 'wsgi.url_scheme': 'http', 'wsgi.input': StringIO.StringIO(self.request_data), 'wsgi.errors': sys.stderr, 'wsgi.multithread': False, 'wsgi.multiprocess': False, 'wsgi.run_once': False, 'REQUEST_METHOD': self.request_method, 'PATH_INFO': self.path, 'SERVER_NAME': self.host, 'SERVER_PORT': self.port, 'USER_AGENT': self.request_dict.get('User-Agent') } return env def start_response(self, status, response_headers): Headers = [(' Date and datetime. Datetime. Now (). The strftime (' % a, H: % d % % b Y % % M: % S GMT ')), (' Server ', 'RAPOWSGI0.1'), ] self.headers = response_headers + headers self.status = status def finish_response(self, app_data): try: Response = 'HTTP/1.1 {status}\r\n'. Format (status=self.status) for header in self.headers: response += '{0}: {1}\r\n'.format(*header) response += '\r\n' for data in app_data: response += data self.connection.sendall(response) finally: self.connection.close() if __name__ == '__main__': Port = 8888 if len(sys.argv) < 2: sys.exit(' Please provide a wsGi application available in the format: module name. ') elif len(sys.argv) > 2: port = sys.argv[2] def Generate_server (address, application): server = WSGIServer(address) server.set_application(TestMiddle(application)) return server app_path = sys.argv[1] module, application = app_path.split('.') module = __import__(module) application = getattr(module, application) httpd = generate_server(('', int(port)), application) print 'RAPOWSGI Server Serving HTTP service on port {0}'.format(port) print '{0}'.format(datetime.datetime.now(). strftime('%a, %d %b %Y %H:%M:%S GMT')) httpd.serve_forever()Copy the code

    WSGIServer class __init__ initializes socket and server addresses, binds and listens to ports. Second, serve_forever(self): keep running the server; Handle_request (self): Handles requests; Finally, finish_response(self, app_data): Returns the request response. Initialize WSGIServer: server = WSGIServer(address) and set wsGi app to load: Server.set_application (TestMiddle(application)); Httpd.serve_forever () Then, based on the above information, wsGI Server should be like this:

    1. Initialize, set up a socket, bind the listening port;
    2. Set up the loaded Web app;
    3. Start continuously running the server;
    4. Processing access requests (you can add your own processing here, for example I added printing access information, lexicographing access headers, etc.);
    5. Get request information and environment information (get_environ(self));
    6. withenvironRun the loaded Web app and get the return message;
    7. Construct the return message header;
    8. Return information;

    Once the above process is implemented, a standard WSGi Server is written. On closer inspection, the key to a WSGi Server is to use itenvironGo to the web app and get the results back, which is complementary to the previous application implementation, and then the framework and the server follow the standards, and everyone can work happily together.

    Now runpython server.py app.app 8000And then the browser visitslocalhost:8000:



    The back-end



    The browser

    Now that wsGi Server is up and running, let's take a look at Middleware:

    middleware

    The Middleware is used to do some processing or validation before the server gets the request data to the application. You can also write it in your server if you want. However, the WSGI specification recommends writing this in middleware. Here I implement a middleware that checks if the request 'user-agent' is a normal browser and then rejects it:

    # coding: utf-8
    # middleware.py
    from __future__ import unicode_literals
    class TestMiddle(object):
        def __init__(self, application):
            self.application = application
        def __call__(self, environ, start_response):
            if 'postman' in environ.get('USER_AGENT'):
                start_response('403 Not Allowed', [])
                return ['not allowed!']
            return self.application(environ, start_response)Copy the code

    Init to receive the application, write the processing in the __call__ method, and return the application so that our middleware can be called as a function.

    Then introduce middleware:

    from middleware import TestMiddle
    ...
    server.set_application(TestMiddle(application))Copy the code

    Now restart the server and access the server with Postman:



    As you can see, the middleware is working!

    Next, let's talk about Django and Tornado's support for WSGI:

    Django WSGI:

    Django WSGI application

    Django itself is too complex to use directly on the WSGi Server we wrote, but Django takes this into account and provides WSGIHandler:

    class WSGIHandler(base.BaseHandler): request_class = WSGIRequest def __init__(self, *args, **kwargs): super(WSGIHandler, self).__init__(*args, **kwargs) self.load_middleware() def __call__(self, environ, start_response): set_script_prefix(get_script_name(environ)) signals.request_started.send(sender=self.__class__, environ=environ) try: request = self.request_class(environ) except UnicodeDecodeError: logger.warning( 'Bad Request (UnicodeDecodeError)', exc_info=sys.exc_info(), extra={ 'status_code': 400, } ) response = http.HttpResponseBadRequest() else: response = self.get_response(request) response._handler_class = self.__class__ status = '%d %s' % (response.status_code,  response.reason_phrase) response_headers = [(str(k), str(v)) for k, v in response.items()] for c in response.cookies.values(): response_headers.append((str('Set-Cookie'), str(c.output(header='')))) start_response(force_str(status), response_headers) if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'): Response = environ['wsgi.file_wrapper'](response.file_to_stream) return response Here WSGIHandler also uses start_response(force_str(status), response_headers) to wrap Django app into a standard WSGI app and return response. WSGIServer(Simple_server.wsgiserver, Object): """BaseHTTPServer that implements the Python WSGI protocol""" request_queue_size = 10 def __init__(self, *args, **kwargs): if kwargs.pop('ipv6', False): self.address_family = socket.AF_INET6 self.allow_reuse_address = kwargs.pop('allow_reuse_address', True) super(WSGIServer, self).__init__(*args, **kwargs) def server_bind(self): """Override server_bind to store the server name.""" super(WSGIServer, self).server_bind() self.setup_environ() def handle_error(self, request, client_address): if is_broken_pipe_error(): logger.info("- Broken pipe from %s\n", client_address) else: super(WSGIServer, self).handle_error(request, client_address)Copy the code

    As you can see, WSGIHandler also wraps Django App into a standard WSGI app using start_response(force_str(status), response_headers) and returns response.

    Django WSGI server

    Django also implements WSGi Server:

    class WSGIServer(HTTPServer): """BaseHTTPServer that implements the Python WSGI protocol""" application = None def server_bind(self): """Override server_bind to store the server name.""" HTTPServer.server_bind(self) self.setup_environ() def setup_environ(self): # Set up base environment env = self.base_environ = {} env['SERVER_NAME'] = self.server_name env['GATEWAY_INTERFACE'] = 'CGI / 1.1' env = [' SERVER_PORT] STR (self. SERVER_PORT) env = [' REMOTE_HOST '] 'env = [' CONTENT_LENGTH'] 'env = [' SCRIPT_NAME']  '' def get_app(self): return self.application def set_app(self,application): self.application = applicationCopy the code

    Wsgiref.simple_server. WSGIServer:

    class WSGIServer(HTTPServer): """BaseHTTPServer that implements the Python WSGI protocol""" application = None def server_bind(self): """Override server_bind to store the server name.""" HTTPServer.server_bind(self) self.setup_environ() def setup_environ(self): # Set up base environment env = self.base_environ = {} env['SERVER_NAME'] = self.server_name env['GATEWAY_INTERFACE'] = 'CGI / 1.1' env = [' SERVER_PORT] STR (self. SERVER_PORT) env = [' REMOTE_HOST '] 'env = [' CONTENT_LENGTH'] 'env = [' SCRIPT_NAME']  '' def get_app(self): return self.application def set_app(self,application): self.application = applicationCopy the code

    As you can see, this is pretty much the same wsGi Server that we implemented.

    Tornado WSGI

    Tornado implements event pool operations, TCP Server, and HTTP Server directly from the bottom with epoll itself, so it is a completely different asynchronous framework, but Tornado also provides support for WSGI. But there is no way to use Tornado asynchronism in this case.

    Tornado doesn't provide WSGI support so much as wsGI compatibility. Tornado provides two ways:

    WSGIContainer

    Other applications should be run in Tornado Server, and Tornado provides WSGIContainer. Wsgi is mainly discussed here today, so tornado code will not be analyzed here. Tornado source code analysis will be done later.

    WSGIAdapter

    Tornado application should run on WSGI Server, tornado provides WSGIAdapter:

    class WSGIAdapter(object): def __init__(self, application): if isinstance(application, WSGIApplication): self.application = lambda request: web.Application.__call__( application, request) else: self.application = application def __call__(self, environ, start_response): method = environ["REQUEST_METHOD"] uri = urllib_parse.quote(from_wsgi_str(environ.get("SCRIPT_NAME", ""))) uri += urllib_parse.quote(from_wsgi_str(environ.get("PATH_INFO", ""))) if environ.get("QUERY_STRING"): uri += "?" + environ["QUERY_STRING"] headers = httputil.HTTPHeaders() if environ.get("CONTENT_TYPE"): headers["Content-Type"] = environ["CONTENT_TYPE"] if environ.get("CONTENT_LENGTH"): headers["Content-Length"] = environ["CONTENT_LENGTH"] for key in environ: if key.startswith("HTTP_"): headers[key[5:].replace("_", "-")] = environ[key] if headers.get("Content-Length"): body = environ["wsgi.input"].read( int(headers["Content-Length"])) else: body = b"" protocol = environ["wsgi.url_scheme"] remote_ip = environ.get("REMOTE_ADDR", "") if environ.get("HTTP_HOST"): host = environ["HTTP_HOST"] else: host = environ["SERVER_NAME"] connection = _WSGIConnection(method, start_response, _WSGIRequestContext(remote_ip, Protocol) request = httputil. HTTPServerRequest (method, uri, "HTTP / 1.1," headers = headers, body = body, host = host, connection=connection) request._parse_body() self.application(request) if connection._error: raise connection._error if not connection._finished: raise Exception("request did not finish synchronously") return connection._write_bufferCopy the code

    It can be seen that Tornado also changed its application to the standard WSGI APP using the process mentioned above. Finally, let's try to run Tornado App on our own server:

    # coding: utf-8
    # tornado_wsgi.py
    from __future__ import unicode_literals
    import datetime
    import tornado.web
    import tornado.wsgi
    from middleware import TestMiddle
    from server import WSGIServer
    class MainHandler(tornado.web.RequestHandler):
        def get(self):
            self.write("this is a tornado wsgi application")
    if __name__ == "__main__":
        application = tornado.web.Application([
            (r"/", MainHandler),
        ])
        wsgi_app = tornado.wsgi.WSGIAdapter(application)
        server = WSGIServer(('', 9090))
        server.set_application(TestMiddle(wsgi_app))
        print 'RAPOWSGI Server Serving HTTP service on port {0}'.format(9090)
        print '{0}'.format(datetime.datetime.now().
                           strftime('%a, %d %b %Y %H:%M:%S GMT'))
        server.serve_forever()Copy the code

    Run:python tornado_wsgi.py, open the browser:localhost:9090And the middleware is working fine:

    Simple_wsgi_server Let's Build A Web Server

    The original address

    Author: rapospectre