Why did I create turbo, the rear wheel

March sand original link

Turbo is not so much a framework as a back-end solution.

Background to the birth of Turbo

Tornado is an asynchronous, non-blocking Web server that is also a Web framework that is simple but fully functional.

The former company’s technology stack is Tornado, mongodb, Redis and OpenResty. The largest service serves millions of independent users every day. Most of the early projects were completely constructed in tornado’s most primitive way. A main file contains hundreds of routes, and all the handlers are distributed under dozens of large and small folders. The basic file structure of the project includes:

• An abstract DAL layer (data abstraction)
• Database Connection (DB)
• The configuration (conf)
• Tools (utils)
• Cache (cache)
• Third-party Libraries (Lib)

As a project grows in code, multiple feature developments often occur at the same time, and developers often have to be very careful about code changes. Even so, conflicts often occur, and the code becomes less and less maintainable and robust.

Due to the rapid business growth of start-up companies, multiple business lines go hand in hand, and the development team often has to maintain many projects. Some projects have a short life cycle and are finished in one activity; some projects have to be completed with the completion of external cooperation; some projects serve new products and need to be built quickly. Different projects are created by different developers. The basic way to create projects is to copy old projects, which are primitive and error-prone. Differences in project structure between different projects As the business changes, the differences become more and more prominent, and there is more and more repetitive functional and tool code.

What features does Turbo have

• Project building tool, one click to generate project directory and APP structure
• Session provides flexible interfaces and allows you to customize storage modes
• Simple mongodb ORM, very simple and even crude compared to Mongoose, but sufficient
• Rich and highly uniform directory division, no matter how big the project
• Code pluggable, easy to transplant, easy to maintain
• It makes it easy to implement RESTFul apis
• Flexible route organization structure facilitates route reconstruction and search

The key to build

Session

With its own Session, the storage mode can be changed freely, and the flexibility is strong.

Simple mongodb ORM

Rich directory structure partition

The role and hierarchy of each directory structure is strictly defined, providing a variety of directory divisions required by the average large project

├ ─ ─ the README. Md ├ ─ ─ app - server ├ ─ ─ the config ├ ─ ─ the db ├ ─ ─ helpers ├ ─ ─ lib ├ ─ ─ models ├ ─ ─ script ├ ─ ─ service ├ ─ ─ store ├ ─ ─ a task ├ ─ ─ utilsCopy the code

The code is pluggable and portable

Apps ├ ─ ─ just set py ├ ─ ─ app │ ├ ─ ─ just set py │ ├ ─ ─ app. Py │ ├ ─ ─ base. Py │ ├ ─ ─ setting. Py ├ ─ ─ base. Py ├ ─ ─ Settings. PyCopy the code

RESTFul api

All methods supported by the HTTP protocol can be case-sensitive in Turbo to implement normal page render and interface apis

def get(self):
    pass
    def GET(self):
    self._data = {'username': 'zhyq0826'}
    Copy the code

Flexible routing

register.register_group_urls('', [
    ('/', app.HomeHandler),
    ('/index', app.HomeHandler, 'index'),
    ('/hello', app.HomeHandler, 'home'),
    ])
register.register_url('/v1/hello', app.ApiHandler)
Copy the code

Integrate many useful features

The practical development of mongodb and Tornado has encountered various problems, so Turbo itself has integrated many small functions during the development process, which is very practical.

1. You can use the command line to generate indexes for all collections

1
Copy the code

turbo-admin index 
Copy the code

2. The state and event distribution mechanism similar to Flux can be very convenient to manage and change the state of global variables. This feature can be used for request statistics, function call statistics and other aspects

from turbo.flux import Mutation, register, State
mutation = Mutation(__file__)
state = State(__file__)
def increase_rank(rank):
    return rank+1
    def dec_rank(rank):
    return rank-1
    from turbo.flux import Mutation, register, dispatch, register_dispatch
import mutation_types
def increase(rank):
    pass
    def decrease(rank):
    return dispatch('user', mutation_types.DECREASE, rank)
    def inc_qps():
    pass
    Copy the code

3. The mongodb read and write call hook. With this hook, the status of the mongodb read and write call can be counted

def collection_method_call(turbo_connect_ins, name):
    def outwrapper(func):
        def wrapper(*args, **kwargs):
            if name in turbo_connect_ins._write_operators:
                turbo_connect_ins._model_ins.write_action_call(name, *args, **kwargs)
            if name in turbo_connect_ins._read_operators:
                turbo_connect_ins._model_ins.read_action_call(name, *args, **kwargs)
            return func(*args, **kwargs)
        return wrapper
    return outwrapper
    class Model(object):
    def write_action_call(self, name, *args, **kwargs):
        """        execute when write action occurs, note: in this method write action must be called asynchronously        """
        pass
    def read_action_call(self, name, *args, **kwargs):
        """        execute when read action occurs, note: in this method read action must be called asynchronously        """
        pass
        Copy the code

4. The logger name can be generated based on the path where the log file resides, and the logger path, level, and name can be customized

from turbo.util import getLogger
logger = getLogger(__file__)
logger = getLogger('feed', log_level=logging.DEBUG, log_path='/var/log/feed.log')
Copy the code

5. String mongodb data

from turbo.util import to_str, to_list_str, to_dict_str, json_encode
data  = json_encode(to_str(db.collection.find()))
Copy the code

6. Fast parameter extraction and conversion

_get_params = {
    'need': [
        ('skip', int),
        ('limit', int),
    ],
    'option': [
        ('did', basestring, None)
    ]
    }
Copy the code

You can call self.parameters to extract the parameters. If the required parameters do not exist or the conversion is wrong, the value is None. Optional parameters can specify default values.

assert self.parameters['skip'] == 0
assert self.parameters['did'] == None
Copy the code

The wheel Turbo was built in time