This article from the “paste” share, the original title “real-time message push collation”, has been optimized and changed.
1, in front
For developers familiar with Web IM technology, we review the underlying communication technology of Web IM, from short polling, long polling, to later SSE and WebSocket, the threshold to use is getting lower and lower (the early long polling Comet technology is actually hack, the threshold to use is not low), the technology is getting more and more advanced, The experience of instant messaging on the web is getting better as a result.
But last week, while editing the third article in my IM Scan Login technology series, it occurred to me that these so-called “old technologies” of web-based instant messaging are not useless compared to the popular WebSocket. In the case of scanning login in IM, short polling technology is perfectly suitable, and there is no need to beat the WebSocket.
Therefore, most of the time there is no need to blindly pursue new technology, relative to the application scenario is the best. While WebSocket is important for developers of im and push messaging technologies, it’s still helpful to know about the “old” technologies of Web messaging, such as short polling and long polling, which is why this article is important to share.
Learning and communication:
Im/Push Technology Development Exchange 5 groups: 215477170 [Recommended]
Introduction to Mobile IM Development: One Entry is Enough: Developing Mobile IM from Zero
Open source IM framework source:
https://github.com/JackJiang2…
(synchronous published in this article: http://www.52im.net/thread-35…
2
[1] The evolution of Web communication mode: from Ajax, JSONP to SSE, Websocket [3] Web instant messaging technology inventory: Short polling, Comet, Websocket, SSE
3. Introduction to the text
For instant messaging systems such as IM/ push messaging, the key is the ability to communicate in real time.
On the face of it, “real-time communication” means:
1) The client can proactively send data to the server at any time; 2) The server can notify the client in real time when the client’s concerns change. Compared with the traditional C/S request model, “real-time communication” does not need to send a request subjectively to get the content they care about, but is “pushed” by the server side.
Note: the word “push” above is in quotes. In fact, several existing technical implementations do not really push actively on the server side, but create an illusion of “real-time communication” through certain means.
In terms of existing technologies, there are mainly the following categories:
1) Client Polling: Short Polling in the traditional sense; 2) Server-side Polling: Long Polling; 3) One-way Server push: server-sent Events (SSE); 4) Full-duplex communication: WebSocket. The following text will be aimed at these several technical solutions, for you one by one.
4. This article is provided with Demo and code
In order to help readers better understand the content of this article, the author specially wrote a more complete Demo, the Demo will be a simple chat room example to achieve the above four technical ways (the code has some bugs, mainly for demonstration use, don’t mind).
Complete Demo source package download:
(please download from synchronous link attachments: http://www.52im.net/thread-35…
Demo running effect (GIF) :
If you are interested, you can download and study by yourself.
5. Understand Short Polling
The implementation principle of short polling:
1) The client sends a request to the server, the server returns data, and then the client processes the data returned by the server; 2) The client continues to send requests to the server and repeat the above steps. If you do not want to put too much pressure on the server, a request interval is generally set. The logic is shown in the figure below:
Advantages of using short polling: there is no additional development cost for the base, request the data, parse it, respond, and so on.
The drawbacks are obvious:
1) Constantly sending and closing requests will put a lot of pressure on the server, because opening the Http connection itself is a resource-intensive thing; 2) The polling interval is not easy to control. If the real-time requirement is high, it is obvious that short polling has obvious disadvantages. If the interval is set too long, the message will be delayed; if the interval is set too short, the server will be stressed. Short polling client code implementation (excerpt) :
var ShortPollingNotification = {
datasInterval: null,
subscribe: function() {
this.datasInterval = setInterval(function() {
Request.getDatas().then(function(res) {
window.ChatroomDOM.renderData(res);
});
}, TIMEOUT);
return this.unsubscribe;
},
unsubscribe: function() {
this.datasInterval && clearInterval(this.datasInterval);
}
}
PS: Complete code, please see “4, this article with Demo and code” section.
The operating effect of the corresponding Demo in this paper is as follows (GIF) :
Here are the corresponding requests, notice that the number of requests in the lower left corner is constantly changing:
In the figure above, sending a request every 1s looks good, but setting the timeout value to 5s makes it less effective. As shown in the figure below.
Demo running effect when setting timeout value to 5s (GIF) :
6. Understand Long Polling
6.1 Basic Principles The basic principles of long polling are as follows:
1) The client sends a request and the server holds the request; 2) It will not return data until the listening content changes, and the connection will be disconnected (or if the request is not returned within a certain period of time, the connection will be automatically disconnected due to timeout); 3) The client continues to send the request and repeat the above steps. The logic is shown in the figure below:
Long polling is an improved version of short polling: it reduces the overhead of the client initiating an Http connection, and instead, the server proactively determines whether the content it cares about has changed.
So the nature of polling doesn’t change much, but it does:
1) The polling for content changes is changed from the client to the server (the client will send the request again after the connection is broken, which greatly reduces the number of connections initiated compared to short polling); 2) The client only makes changes when the data is changed. In contrast to short polling, it does not receive all the data. 6.2 Code implementation of long polling customer code implementation (excerpt) :
/ / the client
var LongPollingNotification = {
/ /... subscribe: function() { var that = this; Request.getv2datas (this.getKey(),{timeout: 10000}).then(function(res) {var data = res.data; window.ChatroomDOM.renderData(res); // After the data is successfully obtained, the request will be sent again that subscribe(); }). Catch (function(error) {// timeout will also send a request that. Subscribe (); }); return this.unsubscribe; } / /...}
The author uses express, default does not support hold request, so use an Express-Longpoll library to achieve.
The following is a native non-library implementation (this is just an introduction to the principle). The overall idea is that if the server supports holding the request, it will poll itself for a certain period of time, and then decide whether to return new data by comparing key values.
Here are the specific ideas:
1) The first time the client returns with an empty key value, this time it immediately returns with new content. The server returns the computed contentKey to the client. 2) The client then sends a second request, taking the first return contentKey as the key value, and makes the next round of comparison; 3) If the two key values are the same, the request will be held, internal polling will be conducted, and if there is new content or client timeout, the connection will be disconnected; 4) Repeat the above steps. The code is as follows:
// Server side
router.get(‘/v2/datas’, function(req, res) {
const key = _.get(req.query, ‘key’, ”);
let contentKey = chatRoom.getContentKey();
while(key === contentKey) {
sleep.sleep(5);
contentKey = chatRoom.getContentKey();
}
const connectors = chatRoom.getConnectors();
const messages = chatRoom.getMessages();
res.json({
code: 200,
data: { connectors: connectors, messages: messages, key: contentKey },
});
});
Here’s an implementation snippet with Express-LongPoll:
// mini-chatroom/public/javascripts/server/longPolling.js
function pushDataToClient(key, longpoll) {
var contentKey = chatRoom.getContentKey();
if(key ! == contentKey) {
var connectors = chatRoom.getConnectors();
var messages = chatRoom.getMessages();
long poll.publish(
'/v2/datas',
{
code: 200,
data: {connectors: connectors, messages: messages, key: contentKey},
}
);
}
}
long poll.create(“/v2/datas”, function(req, res, next) {
key = _.get(req.query, ‘key’, ”);
pushDataToClient(key, longpoll);
next();
});
intervalId = setInterval(function() {
pushDataToClient(key, longpoll);
}, LONG_POLLING_TIMEOUT);
PS: Complete code, please see “4, this article with Demo and code” section.
To facilitate the demonstration, I changed the timeout of the client request to 4s. Note the following screenshot:
As you can see, there are two ways to disconnect, either as a timeout or as a request to return data.
6.3 Long Polling Mode Based on Iframe This is another implementation scheme of long polling technology.
The specific principle of the scheme is as follows:
1) Embed an iframe in the page that points to the polling server address, and then place an execute function in the parent page, such as execute(data); ; 3) Through the sent script, the method in the parent page is actively executed to achieve the effect of push. Due to lack of space, I will not give an in-depth introduction here. If you are interested, please read the section “3.3.2 Iframe Based Data Flow” in the article “Beginner’s Post: The Most Comprehensive Explanation of the Principles of Instant Messaging Technology on the Web”.
7, What is Server-Sent Events (SSE)
7.1 Basic Introduction From the perspective of pure technology: In the short polling and long polling technologies described in the previous two sections, the server cannot proactively push messages to the client. In both cases, the client proactively requests the server to obtain the latest data.
SSE, which is described in this section, is a technology that actively pushes messages from the server.
SSE is essentially an HTTP long connection, but instead of sending a one-time data packet to the client, it sends a stream in the format of text/event-stream. So the client will not close the connection and will wait for a new data stream from the server, such as video playback.
In simple terms, SSE is:
1) SSE uses HTTP protocol, which is supported by existing server software. WebSocket is a stand-alone protocol. 2) SSE is lightweight and easy to use; The WebSocket protocol is relatively complex. 3) SSE supports disconnection by default, WebSocket needs to be implemented. 4) SSE is generally only used to transmit text, binary data needs to be encoded after transmission, WebSocket default support to transmit binary data. 5) SSE supports custom message types sent. The technical principle of SSE is shown in the following figure:
Basic usage of SSE can see SSE API documentation, the address is: https://developer.mozilla.org/en… – $_server sent_events.
Most modern browsers currently support SSE except IE and earlier versions:
(Above: https://caniuse.com/?search=S…
7.2 Code implementation // client
var SSENotification = {
source: null,
subscribe: function() {
if('EventSource'inwindow) {
this.source = newEventSource('/sse');
this.source.addEventListener('message', function(res) {
const d = res.data;
window.ChatroomDOM.renderData(JSON.parse(d));
});
}
return this.unsubscribe;
},
unsubscribe: function() {
this.source && this.source.close();
}
}
// Server side
router.get(‘/sse’, function(req, res) {
const connectors = chatRoom.getConnectors();
const messages = chatRoom.getMessages();
const response = { code: 200, data: { connectors: connectors, messages: messages } };
res.writeHead(200, {
"Content-Type":"text/event-stream",
"Cache-Control":"no-cache",
"Connection":"keep-alive",
"Access-Control-Allow-Origin": '*',
});
res.write(“retry: 10000\n”);
res.write(“data: “+ JSON.stringify(response) + “\n\n”);
var unsubscribe = Event.subscribe(function() {
const connectors = chatRoom.getConnectors();
const messages = chatRoom.getMessages();
const response = { code: 200, data: { connectors: connectors, messages: messages } };
res.write("data: "+ JSON.stringify(response) + "\n\n");
});
req.connection.addListener(“close”, function() {
unsubscribe();
}, false);
});
Here’s what the console looks like, noting the response type:
Note in the details to see the request type, and the EventStream message type:
PS: SSE more detailed information is not here, interested students can read “SSE technology in detail: a new HTML5 server push event technology”, “using WebSocket and SSE technology to achieve Web end message push”.
What is WebSocket
8.1 Basic Introduction to PS: This section is based on “Web Im Practices: How to Make WebSocket Connections Faster”. 3. Get to know WebSocket quickly.
WebSocket was born in 2008, became an international standard in 2011, and is now supported by all browsers (see Quick Start: WebSocket Tutorial for Beginners). It is a brand new application layer protocol, is specially designed for web client and server side of the real full-duplex communication protocol, can be likened to HTTP protocol to understand webSocket protocol.
(Picture from “WebSocket In Detail (4) : Probe into the Relationship between HTTP and WebSocket (Part 1)”)
Their differences:
1) HTTP protocol identifier is HTTP, WebSocket is WS; 2) HTTP request can only be initiated by the client, the server can not actively push messages to the client, but WebSocket can; 3) HTTP requests have the same origin restriction, and communication between different sources needs to cross domains, while WebSocket does not have the same origin restriction. Their similarities:
1) Both are communication protocols at the application layer; 2) The default port is the same, 80 or 443; 3) Both can be used for communication between the browser and the server; 4) Both are based on TCP. A diagram of the two and TCP:
(Image from Quick Start: WebSocket Tutorial)
For more information on the relationship between Http and WebSocket, read:
WebSocket in Detail (4) : Probing into the Relationship between HTTP and WebSocket (Part 1)
WebSocket in Detail (5) : The Relationship between HTTP and WebSocket (Part 2)
The relationship between WebSocket and Socket can be read in detail: “WebSocket in detail (six) : Probing the relationship between WebSocket and Socket”.
8.2 Technical Features WebSocket technical features are summarized as follows:
1) Two-way communication, designed to reduce the overhead of the number of HTTP connections in traditional polling; 2) Based on THE TCP protocol, the handshake phase uses HTTP protocol, so the handshake is not easy to mask, can pass a variety of HTTP proxy servers; 3) Good compatibility with HTTP, can also use port 80 and 443; 4) No same-origin restriction, the client can communicate with any server; 5) Can send text, can send binary data; 6) The protocol identifier is WS (if encrypted, it is WSS), and the server URL is URL. The technical principle of WebSocket is shown as follows:
Knowledge about the WebSocket API, will no longer be explained here, can refer to: https://developer.mozilla.org…
8.3 Browser Compatibility WebSocket is compatible with all modern browsers.
(Above: https://caniuse.com/mdn-api_w…
The author uses socket. IO here, which is based on WebSocket encapsulation, providing client and server support.
/ / the client
var WebsocketNotification = {
// …
subscribe: function(args) {
var connector = args[1];
this.socket = io();
this.socket.emit('register', connector);
this.socket.on('register done', function() {
window.ChatroomDOM.renderAfterRegister();
});
this.socket.on('data', function(res) {
window.ChatroomDOM.renderData(res);
});
this.socket.on('disconnect', function() {
window.ChatroomDOM.renderAfterLogout();
});
}
// …
}
// Server side
var io = socketIo(httpServer);
io.on(‘connection’, (socket) => {
socket.on(‘register’, function(connector) {
chatRoom.onConnect(connector);
io.emit('register done');
var data = chatRoom.getDatas();
io.emit('data', { data });
});
socket.on(‘chat’, function(message) {
chatRoom.receive(message);
var data = chatRoom.getDatas();
io.emit('data', { data });
});
});
PS: Complete code, please see “4, this article with Demo and code” section.
The response format is as follows:
With the increasing popularity of HTML5, WebSocket applications are becoming more and more popular. Learning materials about WebSocket are easily found on the Internet. This article will not go into depth on this topic due to the limited space.
If you want to learn more about all aspects of WebSocket, the following articles are worth reading:
“Novice quick start: WebSocket concise tutorial” “WebSocket in detail (a) : a preliminary understanding of WebSocket technology” “WebSocket in detail (two) : technical principles, code demonstration and application case” “WebSocket in detail (three) : In-depth WebSocket communication protocol details “WebSocket detailed (4) : Probe into the relationship between HTTP and WebSocket (part 1)” WebSocket detailed (5) : Probe into the relationship between HTTP and WebSocket (part II) “WebSocket detailed explanation (six) : Probe into the relationship between WebSocket and Socket” theory and practice: From zero understanding of WebSocket communication principle, protocol format, security “micro channel small program how to use WebSocket to achieve a long connection (including complete source code)” “Eight questions WebSocket protocol: for you to quickly answer WebSocket hot questions” “Web side instant communication practice dry products: How to make WebSocket connections faster? WebSocket from Beginner to Master in half an hour! “WebSocket core entry: 200 lines of code, teach you unarmed a WebSocket server” “long connection gateway technology topic (4) : IQiyi WebSocket real-time push gateway technology practice”
9. Summary of this paper
The cost of short polling and long polling is relatively simple, and they are suitable for some message push with low real-time requirements. In the scenario with high real-time requirements, there will be delay and more pressure on the server.
SSE can only be a server-side push message, so it is suitable for projects that do not require two-way communication.
At present, WebSocket has relatively low implementation cost and is suitable for duplex communication. It is more practical for projects requiring high real-time performance for multiplayer online.
This post has been posted on the “Im Technosphere” official account.
▲ The link of this article on the official account is: click here to enter. The synchronous publish link is:http://www.52im.net/thread-35…
