Nginx module systems: Prequel
It has been 17 years since the first version of Nginx. In this decade, Nginx has been holding the throne of the first tier of Web services software. The rich module system behind it, like the Mongolian horseman holding the banner of Nginx, has swept over half of the global market in this field.
However, we know that software compilation is very time consuming and not very good for “fast marching.” How does Nginx maintain a relatively fast iterative process other than heap hardware? Behind this module system and dynamic module design can not be neglected.
In this first article, we will talk about some of the Nginx modules outside of the book, hoping to give you a more three-dimensional view of the Nginx module.
Writing in the front
In 2015, Nginx lead engineer Ruslan Ermilov shared a powerpoint presentation that outlined the evolution of the Nginx module system and how to develop dynamic modules. Subsequently, in Nginx 1.19.7, the code repository also formally added dynamic module compilation and related code.
A year later, in late October 2016, An article from the official Nginx tech blog, “Compiling third-party Dynamic Modules for Nginx and Nginx Plus,” officially announces the new version 1.9.11 for Nginx Officially supports the three-party dynamic module function.
Since then, building dynamic modules has not only been the prerogative of other open-source “Nginx-like” software in the community, but has also enabled faster development and iteration using official Nginx releases.
In the fall of 2012, the Tengine team released version 1.4.0 supporting this feature, four years before the official Nginx team’s DSO feature, We don’t know how much time the intra-related developers in Nginx wasted in the long on-screen compilation process. But we do know that Alibaba Cloud’s business growth during this period was triple digit growth, thanks to Tengine, which provides a large number of “application features” as the seven-tier traffic entry point for all alibaba businesses.
But better late than never, right?
Before we talk about dynamic modules, we first need to understand its module system.
Nginx module system
The module system of Nginx is mainly divided into “core function module” and “dynamic module”, among which the dynamic module is divided into official module and tripartite module. In a sense, the Nginx module is an application of AOP ideas.
Nginx internal module system
In addition to the large number of third-party modules, Nginx has been able to maintain a high level of exposure to the present day, as well as the stable quality of internal modules.
Are responsible for the internal general data type, data object, memory management, file management, hash check, network communication, lock, network connection, log management, timer “core module”; The “event module” is responsible for using the optimal solution of different event schemes in different operating systems; The “HTTP module” that is responsible for the facade and provides high-performance Web services; The “flow module” that provides the general TCP proxy service function; Responsible for “mail function” of mail service agent function; And “OS modules” to resolve system differences. Up to now, these modules still keep a relatively high frequency of small iterations, continuous optimization.
These modules are part of Nginx, compiled and delivered with binary software packages across borders and operating systems, to multinational companies in several countries, to your home router, and even to the industrial computer in the elevator.
If Nginx is missing any of these modules, or if the version of the module is not the same as the Nginx main file, the software will either stop working or fail to provide high-quality service capabilities. This can be called a “child module”, which is a standard static module.
As mentioned earlier, software compilation is time-consuming, and the more files/features, the more dependencies, the longer it takes. When the original version of Nginx appeared, there were only thirty modules including the three-party modules. But today, there are more than 150 three-party modules in the official module documentation alone, and many modules are scattered in various open-source code platforms and personal blogs.
If these modules are not rebuilt and compiled with Nginx as the Nginx iteration progresses, the original functionality will fail or the software will fail. The lengthy compilation process is undoubtedly an unnecessary burden for module maintainers, especially the open source authors who power love.
So once Tengine/Openresty launched the Lua module, it attracted a large number of “just-in-need” users and significantly reduced development time and learning costs while maintaining similar Nginx performance.
Support for “tripartite dynamic modules”, the external modules mentioned below, was introduced in 2016.
Nginx external module
When it comes to Nginx external modules, there are actually two types of modules. One is the ones that have been stripped out by the authorities in order to keep the packages with Nginx core functions small and efficient, but not everyone needs them. The other is functionality provided entirely by community users, some commercial companies. Earlier, there was an official page dedicated to displaying dynamic modules. If you’re interested, you can visit the page for a more intuitive view.
It can be said that the former also belongs to the “close son” camp, but “often live in school” not often at home, the latter is only the honorary son, we generally call “dry son”.
This type of module can be compiled separately, which is at least an order of magnitude shorter than building with the mother of Nginx, and can be used under the same VERSION of Nginx, arbitrarily combined modules, both in the distribution of experience, and the use of experience has a huge “evolution”.
After a qualitative improvement in engineering efficiency, Nginx officially launched more and more software features, the official blog post at the end of the same year mentioned the strategic product Nginx Plus, almost all of the main functions are implemented by dynamic modules: Brotli, Cookie-Flag, Encrypted-Session, FIPS Status Check, GeoIP, GeoIP2, Headers More, HTTP Substitutions Filter, Image-Filter, Lua, NJS Scripting Language, NGINX ModSecurity WAF, NGINX Developer Kit, OpenTracing, Phusion Passenger Open Source, Perl, Prometheus-Njs, RTMP, SET-misC, SPNEGO, XSLT…
Being able to load external modules dynamically is great, but there are many problems and challenges. Although every modern operating system provides a system interface similar to Dlopen’s dynamic link library, engineers who have ever played with “dynamic” applications know that there is a pit in this. The common problem of Dependency Hell is similar to this Bug. Some loading failure scene detection and corresponding processing; Or a variety of cross-platform compilation errors, but if you’re patient, you can find more on the official problem tracker.
Nginx engineers have to do a lot more “customer service” explaining, answering questions and fixing problems than they would have without dynamic modules, but the benefits are real: the software ecosystem is more open and compilation efficiency is improved. In five years, the stock price of F5 company increased from $100 at the beginning to nearly doubled. To some extent, this proves the old saying “it is better to be a channel than a product, better to be a platform than a channel, and better to be an ecology than a platform”.
The last
Observation Nginx and some other course based on open source software has grown up, will find many good software is similar, with some open standard and ability to access, allow the community to contribute, and then into a certain function or other areas, constantly module function optimization, finally formed a relatively complete ecological, Subdivide the scene and can play a very deep situation.
In the next article, we will continue to talk about another Nginx module, how to write and compile an Nginx module.
–EOF
We have a little group of people who like to do things.
In the case of no advertisement, we will talk about software and hardware, HomeLab and programming problems together, and also share some information of technical salon irregularly in the group.
Like to toss small partners welcome to scan code to add friends. (Please indicate the source and purpose, and note the real name, otherwise it will not pass the audit)
All this stuff about getting into groups
This article is published under a SIGNATURE 4.0 International (CC BY 4.0) license. Signature 4.0 International (CC BY 4.0)
Author: Su Yang
Creation time: March 05, 2021 statistical word count: 4013 words reading time: 9 minutes to read this article links: soulteary.com/2021/03/05/…