Avenue to Jane, back to nature.

preface

On the eve of this post, some of you are asking the following questions:

  1. What about performance?

  2. Is it easy to learn?

  3. Where are the development goals?

  4. How to respond to problems?

  5. What are the advantages over lua open source projects in the industry?

And so on, the above problems will be introduced in this article.

The cause of the CF

Let’s talk about feelings first! I believe that practitioners in every industry are more or less had a dream, this dream is called: “I will develop a XXX”! In fact, the author was the same.

Whenever I am working late at night, reading documents, debugging, I will always find some framework or beginner demo from years or more ago. For example: Tinyhttp, the source of the link is a mirror of some students fork.

Every time I see these things, I will arouse a little bit of dying enthusiasm in my heart. Maybe this is the last desire for technology?

Before you start creating a project, you ask yourself, “Should I do it?” Can you stick with it? Maybe being sprayed is a kind of hope?

After answering these questions in my mind, I created the project at the end of 2018.

Tell the truth! The hardest part of a web development framework is not implementing a feature, but building the wheel from scratch.

As a web development framework, the two most important features are definitely needed! Timer library, event driver library. How to choose? There are two options: libev/libuv.

Libev is mature, stable, lightweight, UniX-like, and easy to embed; Libuv is better than Libev, adding many features (thread pooling, signaling, synchronization, locking, etc.), better packaging, and added Windows support;Copy the code

Judging from the selection of CF framework at the beginning of development, Libuv is definitely the optimal solution at present. But the author chose Libev. From then on, the hard road of bottom development began.

First, the authors don’t let users do actual business development in C/C++! This can lead to high development costs and learning costs for users, and it is particularly important to choose a good scripting language.

The author is somewhat familiar with Lua, so he chose Lua as the business scripting language. As for the advantages of Lua, there are plenty of articles on the Internet praising it.

Now that the scripting language has been chosen, start writing code! Let’s Lua.

The way to write CF

1. The network layer

First, let’s take a look at the API code C encapsulates for Lua calls:

LUAMOD_API int
luaopen_tcp(lua_State *L){
	luaL_checkversion(L);
	/* Add SSL support */
    SSL_library_init();
    SSL_load_error_strings();
    // CRYPTO_set_mem_functions(xmalloc, xrealloc, xfree);
    // OpenSSL_add_ssl_algorithms();
	/* Add SSL support */
	luaL_newmetatable(L, "__TCP__");
	lua_pushstring (L, "__index");
	lua_pushvalue(L, 2 -);
	lua_rawset(L, - 3);
    lua_pushliteral(L, "__mode");
    lua_pushliteral(L, "kv");
    lua_rawset(L, - 3);
	luaL_Reg tcp_libs[] = {
		{"read", tcp_read},
		{"write", tcp_write},
		{"ssl_read", tcp_sslread},
		{"ssl_write", tcp_sslwrite},
		{"stop", tcp_stop},
		{"start", tcp_start},
		{"close", tcp_close},
		{"listen", tcp_listen},
		{"connect", tcp_connect},
		{"ssl_connect", tcp_sslconnect},
		{"new", tcp_new},
		{"new_ssl", ssl_new},
		{"free_ssl", ssl_free},
		{"new_server_fd", new_server_fd},
		{"new_client_fd", new_client_fd},
		{NULL.NULL}}; luaL_setfuncs(L, tcp_libs,0);
	luaL_newlib(L, tcp_libs);
	return 1;
}
Copy the code

The above is the TCP C code fragment, interested in reading the source partners please click here;

It is well known that Lua does not have native sockets, so it is up to the framework writers to abstract the underlying logic and re-implement an API.

Anyone can simply package the Lua C library, and it’s not that hard. But our goal is to make the underlying synchronous blocking Socket hook non-blocking, where the difficulty comes!

As we all know, Libev is an event-driven network library based on the React model. All business logic after registered events is triggered in the form of callback. Wouldn’t that be node-lua code? (smile)

At this point, the author came up with an idea to solve the problem! The execution process is as follows:

  1. The C API is called to register the callback event every time something synchronization needs to be done.
  1. Creates a Lua coroutine save context for all currently registered events and cedes execution rights to the current coroutine.
  1. After the registration event is triggered, the C API recovery coroutine is called to resume execution.

In simple terms, the asynchronous callback logic at the C level is encapsulated as synchronous non-blocking at the Lua level, ensuring that threads are not blocked due to IO problems.

The following provides a socket synchronization non-blocking pseudo-code for your reference:

function TCP:recv(bytes)
    local current_co = co_self()
    self.read_co = read_ev(function(a)
    -- do action
    -- stop timer_ev
    -- wakeup(current_co) resumes execution
    end)
   self.timer_co =  self.timer_ev(function(a)
    -- do action
    -- stop read_ev
    -- wakeup(current_co) resumes execution
    end)
    tcp_start(io, EV_READ, self.read_co)
    timer_start(timer, 3Seconds timeout, self.timer_co)return co_yield() -- Delegating executive power
end
Copy the code

A Lua version of the Socket EV_READ pseudocode roughly processing flow above, to see the actual processing logic see here.

Similarly, the Socket write/connect/listen API can be copied directly (UDP is also similar).

Observant users may notice that the code registers both Socket and Timer events, and that non-blocking Socket operations do not resolve read and connect timeouts. So the CF framework simply encapsulates thoroughly.

At this point, the Socket is already a basic hook and encapsulation completed. Now you can start writing application layer protocols.

2. Application layer protocol

Now that sockets are finally working properly, there’s a new problem.

Libev doesn’t come with asynchronous DNS

DNS also need to use their own encapsulation, this pit is really filled very uncomfortable! Fortunately, there are predecessors on the network to achieve the Lua version of asynchronous DNS, the author slightly understand after borrowing over encapsulation internal use.

So CF also has a deeply customized asynchronous DNS library! (Not perfect, but good enough)

Whether a network library is popular basically depends on ecology. Then the protocol layer wheel has to be built again:

  1. HTTPD with HTTPC
  2. mail
  3. mysql
  4. redis
  5. mqtt
  6. websocket

Some of the agreements were borrowed and adapted from your predecessors. Simple agreements took 1-2 hours to write by hand.

3. Encapsulation and ease of use

In order not to make the API so closed and improve the availability of CF, the author decided to make mysql and Redis preliminary encapsulation.

Encapsulation includes common features such as connection pooling, object-oriented operations, no need to manually manage the session lifecycle, and so on. Simplify programming overhead to improve development efficiency.

As for internal sockets, let the framework solve the problem of releasing files to ensure that the number of file descriptions can be used normally.

CF is what?

If you had the patience to follow the first part of the introduction, then you should have a general understanding of CF.

Cf full name: CoreFramework, is a Libev based Lua network development framework. In its internal implementation of a variety of network protocols and third-party libraries to help users to carry out rapid development of project prototype.

Cf uses a separate front and back end solution for HTTPD usage, implementing only basic View routing and not supporting REST style API routing. Although this may lead to a lot of criticism.

Cf HTTPD embedded WebSocket support, convenient users can reuse ports at the same time can enjoy the fun of writing long connections.

For more information, please refer to the project address Wiki

What can CF do?

  • Swarm/Kubernetes microservices scenario based on container technology; – recommend

  • Front-end agent layer of game server; – recommend

  • Cloud servers with insufficient memory /CPU resources; – recommend

  • Stateless clusters with high performance requirements; – recommend

  • Massive Websocket Agent cluster; – recommend

What technology stack does CF use?

Transport layer: TCP/UDP

Session layer: SSL Client is supported

The protocol layer: DNS/webocket/HTTP/MQTT/redis/mysql/SMTP

Tool library: Timer/TASK

Third party libraries: Libev, OpenSSL /libressl, LuA-5.3, Jemalloc/TCMALloc (optional)

How do I install CF?

Cf currently supports most Unix like operating systems. The author is developing on Mac, so Mac support is necessary.

/usr/local/lib = /usr/local/lib = /usr/local/lib

At the same time, the author also sweet for everyone to do a simple dockerfile. file in the project root directory, you can download directly to use.

If you don’t want to create a Dockerfile, you can also use the Docker command to pull the author to create an image for the Docker hub. Candymi /cfweb

For details and usage instructions, see Docker Installation and Compilation Installation

How does CF work?

A test run

bash#: ./cfadmin

The background

bash#: ./cfadmin

exit

killall cfadmin

ctrl + c

Where is the document?

The author wrote a thoughtful and detailed document for everyone, in order to get everyone’s likes and attention.

The author also prepared sufficient Chinese and English annotations for students who like to read the source code, which is convenient for everyone to quickly understand the working mode of CF (Chinese/English annotations combined with easy to understand some exclusive vocabulary).

Answer the previous question:

Q. What is the performance?

A. The performance is not bad, but please test the specific value by yourself.

Q. Is it easy to learn?

A. Learn lua in one hour -> CF in one hour

Q. What was the original intention of the project?

A. Actually, it’s already been answered.

Q. What are the development goals?

A. Wiki has TODO items

Q. How do you feedback problems?

A. There are Q & A items in Wiki

Q. What are the advantages of lua open source projects in the industry?

A. CF changes user habits more deeply than other Lua development projects! Simplify the difficulty of getting started the framework, the framework is transparent black box. No need to learn complex design patterns and concepts.

Q. What is the development philosophy of CF?

A. THE goal of THE CF project is not to compete, but to understand that simplicity is beautiful. When you get used to it, maybe you become addicted to it.

Use the sample

  • Container deployment

    • Dockerfile quickly builds the development environment

    • Cf K8S example

  • Initialize and use the CF Web

  • Cf Web Websocket Application Guide

  • Cf Web Middleware Development Guide

Wonderful capture

hope

You may be using other development frameworks, but that doesn’t stop you from pushing CF.

Maybe you’re trying it out, but that doesn’t stop you from communicating your ideas to the author.

Maybe you are making fun of its shortcomings, please make fun of it.

Documents and Addresses

Project documentation

The project address