This book not only explains the principle of high performance distributed real-time processing system programming, especially memory resource management, coding solutions, concurrent and asynchronous processing, threading model, the difference between batch and real-time processing, message queue, dynamic loading and so on are introduced in detail. The architecture and internal implementation of real-time processing system are analyzed in detail. Finally, how to implement Hurricane, a distributed real-time processing system, is analyzed in detail.
This book is a great read for big data developers and architects, while addressing many of the problems of performance optimization.
This book altogether 17 chapters: everybody can look carefully, can absorb all words, affirmation meeting a step.
Everyone forward attention xiaobian, private xiaobian “learn” to take away “distributed real-time processing system principle, architecture and implementation” it ~~~
Chapter 1 overview of distributed computing
This chapter mainly introduces some basic concepts of distributed system, and some important knowledge points needed to develop real-time processing system, including distributed concept, distributed system characteristics, specific types of distributed storage system and product examples. The general distributed computing system is briefly introduced. By comparing the traditional batch solution (Apache Hadoop and Apache Spark) with Apache Storm’s stream computing model, the concept of real-time processing system is finally introduced, which is also the focus of this book.
Everyone forward attention xiaobian, private xiaobian “learn” to take away “distributed real-time processing system principle, architecture and implementation” it ~~~
Chapter 2 Distributed system communication basics
The previous chapter introduced what distributed systems are and their classification and composition.
It is important to emphasize that a distributed system consists of a variety of nodes that perform their respective functions to fulfill the business logic requirements of the entire system. So naturally, these nodes need to communicate with each other.
The mainstream protocol family used in the Internet is TCP/IP protocol family, which is a multi-level, multi-protocol communication framework.
This chapter will give an overview of TCP/IP protocol family, and emphatically explain IP protocol, TCP protocol and HTTP protocol (including the concept of Restful API), because these are the knowledge points that must be mastered for the subsequent development of distributed real-time processing system. We will use Socket in the development program to communicate. It also provides flexible Restful apis for users to use.
Network communication protocols cover a wide range of topics, each of which requires us to spend a lot of time and energy to learn and research.
This chapter presents the most common and straightforward concepts to the reader, supplemented by some quick practices to help you quickly and flexibly grasp the knowledge of network communication.
In addition, RFC documents are listed where appropriate, and the reader can refer to the RFC document number for more detailed network protocol details.
Everyone forward attention xiaobian, private xiaobian “learn” to take away “distributed real-time processing system principle, architecture and implementation” it ~~~
Chapter 3 high level abstraction of communication system
The foundation of distributed system communication technology should be network communication. In the last chapter, we explained the basic concept of network and Socket, directly used the Socket at the bottom to complete a simple bulletin board program, and designed the corresponding application layer protocol.
However, readers will find that it is very inefficient to use sockets directly to implement network services: they need to design their own protocols, they need to encode and decode their own data transmission, and they need to develop their own servers. So unless there are specific reasons, such as performance issues, security issues, or legacy systems, we don’t use sockets directly when developing real-world applications.
In the computer world, there are two ways to solve all complexity problems: layering and encapsulation. In order to control the complexity of network communication programs, many high level abstractions of network communication have been proposed. This chapter will introduce some popular and widely used high level abstractions. These high-level abstractions can greatly simplify our application development process and improve application development efficiency.
Everyone forward attention xiaobian, private xiaobian “learn” to take away “distributed real-time processing system principle, architecture and implementation” it ~~~
Because the content is too much, xiaobian will not be introduced, we take the hand of careful thinking, repeated deliberation, although the truth is in the hands of a few people, but anyone may become a minority. Come on!
