This article is by the original author “crossoverJie (Chen Jie)” original share, instant messenger network included changes and revisions, thanks to the original author.
1, the introduction
There are also several other purposes:
- 1) The server can take the initiative to close the channel and make it offline when detecting that the heartbeat of a client is not coming.
- 2) If the client detects that the heartbeat of a server is delayed, it can reconnect to obtain a new connection.
Pick up the keyboard: Develop a distributed IM system freehand with me
CIM source code address:
- Main image: github.com/crossoverJi…
- Standby image: github.com/52im/cim
For network programming basics, read the following:
- TCP/IP detail – Chapter 11 UDP: User Datagram Protocol
- Chapter 17: TCP: Transmission Control Protocol
- Chapter 18: Establishment and Termination of TCP Connections
- TCP/IP Detail – Chapter 21: TCP Timeouts and Retransmission
- “Understanding TCP in Depth (PART 1) : Theoretical Foundations” (recommended)
- Network Programming Lazy Introduction part 1: Quick Understanding of Network Communication Protocols part 1
- “Network Programming Lazy Introduction ii: A Quick Understanding of Network Communication Protocols (Part II)”
For more information about the Netty framework, read the following:
- Netty source code online read version (recommended)
- Netty API Documentation Online (recommended)
- Getting Started: The most Thorough Analysis of Netty high-performance Principles and Frameworks so far
- For Beginners: Learning Methods and Advanced Strategies for Netty, the Java High-performance NIO Framework
- The Difference between Java NIO and Classic IO in one Minute
- The greatest Introduction to Java NIO ever: For those worried about getting started and giving up, read this!
2. About the author
- CrossoverJie (Chen Jie) : Born in 1990, graduated from Chongqing Institute of Information Engineering, and now works for Chongqing Zhubajie Network Co., LTD.
- The author’s blog:crossoverjie.top
- The author’s Github:Github.com/crossoverJi…
Other articles by the author:
- Pick up the Keyboard and do it: Develop a Distributed IM System with my Hands
- Tech Dry Goods: Designing a Million-dollar Push System from Scratch
3. Related articles
The last expression you see in Internet heartbeat is the last expression you see in Internet heartbeat.
- Why Does TCP – based MOBILE IM Still Need heartbeat Keepalive mechanism?
- “Wechat team original sharing: Android version of wechat backstage Combat Sharing (Network Protection)”
- Mobile IM Practice: Implementing intelligent Heartbeat Mechanism of wechat on Android
- Mobile IM Practice: Analysis of Heartbeat Strategy of WhatsApp, Line and wechat
- Understanding the Mechanism of Network Heartbeat Packet in Instant Messaging Applications: Functions, Principles, Implementation ideas, etc.
- “Sharing of Rongyun Technology: Practice of Network Link Preservation Technology of Rongyun Android IM Products”
Touppercase’s Practice on Keeping the Internet heartbeat alive:
- MobileIMSDK — an open source, original mobile im framework with complete heartbeat logic and code implementation
- Discussion on the design and Implementation of an Android IM Intelligent Heartbeat Algorithm (with sample code)
- “Hand to hand teach you to use Netty to realize the heartbeat mechanism of network communication program, disconnection reconnection mechanism”
- For beginners: Developing an IM Server from Scratch (Netty based, complete source code available)
- Pick up the Keyboard and do it: Develop a Distributed IM System with my Hands
- From the development of IM is so difficult? Hand to hand teach you from a Simple Andriod version OF IM (source)
4. Heartbeat implementation mode
The heartbeat actually works in two ways:
- 1) TCP protocol implementation (Keepalive mechanism, see TCP/IP Volume 1: Protocol – Chapter 23 TCP Keepalive Timer);
- 2) The application layer itself.
Why does TCP – based mobile IM still need the heartbeat keepalive mechanism?
So what we are talking about here is the implementation of the application layer:
5. The client automatically reconnects
A normal implementation would be:
- 1) Start a scheduled task and send heartbeat packets periodically.
- 2) Update the local time after receiving the response from the server;
- 3) Another scheduled task periodically checks whether the “local time” exceeds the threshold;
- 4) If the number exceeds the threshold, the server is considered faulty and needs to be reconnected.
Netty
IdleStateHandler
Take a look at the implementation in CIM:
Let’s start with this example:
The pseudocode is as follows:
@Override
public void process(ChannelHandlerContext ctx) throws Exception {
long heartBeatTime = appConfiguration.getHeartBeatTime() * 1000;
Long lastReadTime = NettyAttrUtil.getReaderTime(ctx.channel());
long now = System.currentTimeMillis();
if (lastReadTime != null && now - lastReadTime > heartBeatTime){
reconnect();
}
}Copy the code
6. IdleStateHandler error
IdleStateHandler
Let’s assume the following scenario:
- 1) The client connects to the server through login and maintains a long connection. When everything is normal, both sides send heartbeat packets to maintain connection;
- 2) When the server suddenly becomes down, the ideal situation is that the client does not receive the response from the server so that userEventTriggered executes the scheduled task.
- 3) Determine the current time – UpdateWriteTime > threshold for reconnection.
ChannelInbound
7, reliable implementation
The CIM starts a scheduled task when the client senses a network disconnection:
The reason for not starting the client is to save a bit of thread consumption. Network problems are inevitable, but enabling them when needed saves resources.
In the meantime, verify the effect:
Start the two servers, and then start the client to connect to one server and maintain the long connection. When a service is suddenly shut down manually, the client can automatically reconnect to the available service node.
After the client is started, the server also receives a normal ping message:
Run the :info command to check the connection status of the current client and find that the client is connected to port 9000.
At this point I close the node on the connection:
kill -9 2142Copy the code
8. The server automatically removes offline clients
Myths about sending messages:
IsSuccess cannot be used as a criterion for sending a message successfully:
I’m not alone in having the same misunderstanding as I did before, but here’s Netty’s official reply:
Related issue:
Github.com/netty/netty…
cim
Dubbo
So here’s an experiment:
9. Summary of this paper
In this way, the two requirements of the beginning of the article are realized:
- 1) The server can take the initiative to close the channel and make it offline when detecting that the heartbeat of a client is not coming.
- 2) If the client detects that the heartbeat of a server is delayed, it can reconnect to obtain a new connection.
All the relevant code for this article is here, if you are interested, you can check it out for yourself:
- Main image: github.com/crossoverJi…
- Standby image: github.com/52im/cim
Appendix: Summary of more references
[1] IM code Practices (for beginners) :
Is it so hard to develop IM yourself? Hand-in-hand teach you from a simple Andriod version OF IM (source code)
Discussion on the design and Implementation of an INTELLIGENT IM Heartbeat Algorithm on Android terminal
Teach you to use Netty to realize the heartbeat mechanism of network communication procedures, disconnection reconnection mechanism
Netty security in Detail: Introduction to the principles, code demo (Part 1)
Netty security in Detail: Introduction to the principles, code demo (Part 2)
Cracked version of wechat local database (including iOS and Android), only for study and research
MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA MINA
Lightweight im framework MobileIMSDK iOS source (open source version)
Open source IM project “Mogujie TeamTalk” before May 2015 Uncut version complete code
Cracked version of wechat local database (including iOS and Android), only for study and research
NIO framework introduction (a) : Server based on Netty4 UDP two-way communication Demo Demo
NIO framework introduction (ii) : Server based on MINA2 UDP bidirectional communication Demo Demo
NIO framework introduction (iii) : iOS and MINA2, Netty4 cross-platform UDP bidirectional communication practice
NIO framework introduction (4) : Android and MINA2, Netty4 cross-platform UDP two-way communication actual combat
For IM image compression Android tool class source, the effect can be comparable to wechat
High imitation Android version of mobile QQ can drag not reading small bubble source
Node.js +socket.io
Android chat interface source code: to achieve the chat bubble, expression icon (can turn the page)
High imitation Android version of mobile QQ home page side slider menu source
Open source Libco library: the foundation of background framework supporting 800 million users of wechat
Share Java AMR audio file merge source code, the most complete network
Android source obfuscation tool: AndResGuard
A complete Android push Demo based on MQTT communication protocol
Android version of high imitation wechat chat interface source code
High imitation mobile QQ Android version lock screen chat message reminder function
High imitation iOS version of mobile QQ recording and amplitude animation complete implementation
Comment and Reply functions in Android social applications
Implementation of @person function in Android IM application: Imitation of Weibo, QQ, wechat, zero intrusion, high extensibility
Imitated wechat IM chat time display format (including iOS/Android/Web implementation)
Android version copy wechat circle of friends picture drag return effect
For beginners: Developing an IM server from scratch (based on Netty, with complete source code)
Pick up the keyboard: Develop a distributed IM system freehand with me
Correctly understand IM long connection heartbeat and reconnection mechanism, and start to implement (complete IM source code)
More of the same…
[2] Network programming basics:
TCP/IP, rounding
Chapter 11 ·UDP: User datagram protocol
TCP/IP, rounding
Chapter 17: TCP: Transmission control Protocol
TCP/IP, rounding
Chapter 18: Establishment and termination of TCP connections
TCP/IP, rounding
Chapter 21. TCP Timeouts and retransmission
Once upon a time in technology: TCP/IP changed the world
Easy to Understand – In-depth understanding of TCP (part 1) : Theoretical basis
Easy to Understand – In-depth understanding of TCP (part 2) : RTT, sliding window, and congestion handling
Theory classic: TCP protocol three handshake and four wave process in detail
Connection between theory and practice: Wireshark captures packets to analyze TCP three-way handshake and four-way wave
Protocol Diagram of Computer Network Communication (Chinese Edition)
What is the maximum size of a packet in UDP?
P2P technology details (a) : NAT details – detailed principle, P2P introduction
(2) : P2P NAT traversal (hole) solution
P2P technology details (three) : P2P technology STUN, TURN, ICE details
Easy to Understand: Quickly understand the PRINCIPLE of NAT penetration in P2P technology
High performance network programming (a) : the number of concurrent TCP connections can be a single server
High performance Network programming (II) : The last 10 years, the famous C10K concurrent connection problem
High performance Network Programming (III) : In the next 10 years, it is time to consider C10M concurrency
High performance Network programming (IV) : Theoretical exploration of high performance network applications from C10K to C10M
High performance network programming (5) : read the I/O model of high performance network programming
High performance network programming (6) : understand the threading model of high performance network programming
Unknown Network Programming (I) : A Brief Analysis of the DIFFICULT Problems in TCP Protocol (Part I)
Unknown Network Programming (II) : A Brief Analysis of the DIFFICULT Problems in TCP Protocol (Part II)
Why TIME_WAIT and CLOSE_WAIT when closing TCP connections
Unknown network programming (iv) : In-depth analysis of TCP abnormal shutdown
Network programming unknown (5) : UDP connectivity and load balancing
Unknown network programming (6) : Understand UDP in depth and use it well
Unknown network programming (7) : How to make unreliable UDP reliable?
Hidden Network programming (8) : Deep decryption of HTTP from the data transport layer
Unknown network programming (9) : Theory and practice, comprehensive in-depth understanding of DNS
Quick Understanding of Network Communication Protocols (Part 1)
A Quick Understanding of Network Protocols (Part 2)
Network programming lazy introduction (three) : a quick understanding of the TCP protocol is enough
Network programming lazy starter (4) : Quickly understand the difference between TCP and UDP
Why UDP sometimes has an advantage over TCP
Network programming lazy entry (six) : the history of the most popular hub, switch, router function principle entry
Network programming lazy introduction (7) : simple, comprehensive understanding of HTTP protocol
Network programming lazy entry (eight) : teach you to write based on TCP Socket long connection
Network programming lazy introduction (nine) : popular explanation, IP address, why use MAC address?
Technology Literacy: A new generation of UDP-based low latency Network Transport layer protocol – QUIC in detail
Making the Internet faster: A new generation of QUIC protocols shared in Tencent’s technology practices
Summary of optimization means of modern mobile terminal network short connection: request speed, weak network adaptation, security guarantee
Let’s talk about long connections in iOS network programming
Mobile IM Developers must read (1) : Easy to understand the “weak” and “slow” mobile web
Mobile IM Developers must read (ii) : Summary of the most comprehensive mobile weak Network optimization methods ever
IPv6 Technology details: Basic Concepts, Application Status, Technical Practice (Part I)
IPv6 Technology details: Basic Concepts, Application status, Technical Practice (Part 2)
From HTTP/0.9 to HTTP/2: Understand the history and design of the HTTP protocol
Brain-disabled network programming introduction (a) : follow the animation to learn TCP three handshakes and four waves
What are we reading and writing when we read and write sockets?
Brain-disabled network programming introduction (3) : HTTP protocol must know must know some knowledge
Quick Understanding of HTTP/2 Server Push
Introduction to brain-damaged Network programming (5) : Ping command used every day, what is it?
Introduction to Network Programming (6) : What are public IP and internal IP? What the hell is NAT?
Take the network access layer design of online game server as an example to understand the technical challenges of real-time communication
To the next level: The network basics that a good Android programmer must know
Comprehensive understanding of mobile DNS domain name hijacking and other miscellaneous problems: technical principles, root causes, solutions, etc
Mobile DNS optimization practice of Meitu App: THE HTTPS request time is reduced by nearly half
Android programmers must know the network communication transport layer protocols – UDP and TCP
Introduction to Zero-Base Communication Technology for IM Developers (PART I) : 100 Years of Development of Communication Switching Technology (Part I)
Introduction to Zero-Base Communication Technology for IM Developers (II) : 100 Years of Development of Communication Switching Technology (II)
Introduction to Zero-base Communication Technology for IM Developers (III) : The Century-old changes of Chinese communication mode
Introduction to Zero-Base Communication technology for IM Developers (iv) : The evolution of mobile phones, the most comprehensive history of mobile terminals in history
Introduction to Zero-base Communication Technology for IM Developers (5) : 1G to 5G, 30 years of mobile communication technology evolution
Introduction to Zero-base Communication technology for IM developers (6) : Mobile terminal connector – “base station” technology
Introduction to Zero-Base Communication Technology for IM Developers (7) : The Swift Horse of Mobile Terminal — “Electromagnetic Wave”
Introduction to Zero-based Communication technology for IM Developers (eight) : Zero-based, the strongest “antenna” principle in history literacy
Introduction to Zero-Base Communication Technology for IM Developers (9) : “Core Network” — the Backbone of Wireless Communication Network
Introduction to Zero-Base Communication Technology for IM Developers (10) : Zero-base, the strongest 5G technology literacy in history
Why is WiFi signal bad? A text is understood!
Introduction to Basic Communication technology for IM Developers (12) : Networking? Network down? A text is understood!
Introduction to Zero-Base Communication Technology for IM Developers (13) : Why cell phone Reception is Poor? A text is understood!
How Hard is wireless On high-speed Trains? A text is understood!
Introduction to Zero-base Communication technology for IM Developers (15) : Understanding location technology, one article is enough
Baidu APP mobile terminal network in-depth optimization practice sharing (I) : DNS optimization chapter
Baidu APP mobile terminal network in-depth optimization practice sharing (II) : Network connection optimization
Baidu APP mobile terminal network in-depth optimization practice sharing (III) : mobile terminal weak network optimization chapter
Technology master Chen Shuo’s share: from shallow to deep, network programming learning experience dry summary
Can ruin your interview: Do you know how many HTTP requests can be made over a TCP connection?
Zhihu technology sharing: Zhihu high performance long connection gateway technology practice of ten million level concurrent
More of the same…
[3] NIO asynchronous network programming Data:
The principle of Java new generation network programming model AIO and the introduction of Linux system AIO
11 Questions and answers about “Why choose Netty”
Open Source NIO Framework gossip – MINA or Netty came first?
Netty or Mina: A Deep Study and Comparison
Netty or Mina: An In-depth Study and Comparison (II)
NIO framework introduction (a) : server based on Netty4 UDP bidirectional communication Demo
NIO framework introduction (ii) : Server based on MINA2 UDP bidirectional communication Demo
NIO framework introduction (3) : iOS and MINA2, Netty4 cross-platform UDP two-way communication actual combat
NIO framework introduction (four) : Android and MINA2, Netty4 cross-platform UDP two-way communication actual combat
Netty 4.x learning (a) : ByteBuf detail explanation
Netty 4.x learning (2) : Channel and Pipeline details
Netty 4.x learning (3) : Details of the threading model
Apache Mina Framework Advanced Part 1: IoFilter details
Apache Mina Framework Advanced Part ii: IoHandler details
MINA2 Thread Principle Summary (including simple test example)
Apache MINA2.0 Development Guide
MINA, Netty source code (online reading version) has been collated and released
Solve the problem of TCP sticky packet and missing packet in MINA data transmission (source code)
Resolve the coexistence of multiple instances of Filter of the same type in Mina
Practice summary: Netty3.x upgrade netty4. x encountered those pits (threads)
Practice summary: Netty3.x vs. Netty4.x threading model
Netty security in Detail: Introduction to the principles, code demo (Part 1)
Netty security in Detail: Introduction to the principles, code demo (Part 2)
Explain the elegant exit mechanism and principle of Netty
NIO framework details: Netty’s high-performance approach
Twitter: How to Use Netty 4 to reduce JVM GC overhead
Absolute Dry goods: Technical essentials of Netty based mass access push service
Netty dry goods sharing: Jingdong Jingmai production grade TCP gateway technology practice summary
Beginner: by far the most thorough Netty high-performance principle and framework architecture analysis
For beginners: Learning methods and advanced strategies for Netty, the Java High-performance NIO framework
No buts.! Learn the difference between Java NIO and classic IO in one minute
The most powerful Introduction to Java NIO ever: Worried about getting started and giving up, please read this article!
Teach you to use Netty to realize the heartbeat mechanism of network communication procedures, disconnection reconnection mechanism
More of the same…