High performance IOT server implementation path

The market dynamic

The iot market saw an unexpected acceleration in 1Q / 2Q 2018 and brought the total number of iot devices in use to 7B. This is one of many findings in IoT Analytics’ latest “State of Things and Short-term Outlook” update.

Companies across the board, particularly iot software, cloud and services companies, beat revenue expectations by a wide margin. Microsoft Azure and Amazon AWS grew 93% and 49% respectively (over the past 12 months), with their iot component contributing significantly to growth. But smaller businesses like C3IoT have also reported a 60 percent increase in revenue this year.

Number of connected devices worldwide: 17B

There are now more than 17 billion connected devices and 7 billion iot devices in use (this figure does not include smartphones, tablets, laptops or landlines).

Number of iot devices in the world: 7B

Global connectivity growth is primarily driven by iot devices — both on the consumer side (e.g. smart home) and on the enterprise/B2B side (e.g. connected machines). The number of active iot devices is expected to rise to 10 billion by 2020 and 22 billion by 2025. These iot devices include all active connections, regardless of devices purchased in the past but no longer used.

From a device connection point of view, the dynamics are very large

Wireless Personal Network (WPAN) :

The largest number of iot devices are connected via short-range technology (WPAN), typically within a maximum range of no more than 100 meters. These include Bluetooth-connected devices, such as headphones, as well as Zigbee and Z-wave connected devices, most of which can be found in smart homes, for example, to connect to smoke alarms or thermostats.
Wireless Local area Network (WLAN)

Another category is wireless lans, covering connections up to 1 kilometer long. Wi-fi is the most common standard in this category and has seen huge growth, mainly through the use of home assistants, smart TVS and smart speakers, but also increasingly used in industrial environments such as industry (although it continues to play a secondary role) in these Settings compared to other technologies).
Low Power Wide Area Network (LPWAN)

Much of the future growth in the number of iot devices is expected to come from low-power wans. By 2025, more than 2 billion devices are expected to be connected via LPWAN. The technology promises extremely high battery life and a maximum communication range of more than 20 km and is used by three major competing standards Sigfox, Lora and NB-iot. These standards are currently being promoted worldwide with over 25 million devices now connected, most of which are smart meters.
The cable

When they think of the Internet of Things, few think of wired connections. However, wired device connections are still the cheapest and most reliable option in many Settings. Especially in industrial environments, fieldbus and Ethernet technologies use wired connections to a large extent and are expected to continue to do so in the coming years.
Cellular/M2M

2G, 3G and 4G technologies have long been the only options for connecting remote devices. As LPWA and 5G gain momentum, these traditional cellular standards are expected to lose ground to new technologies as they offer more lucrative opportunities for many end users.
5G

In the United States, the first pre-standard 5G networks will offer fixed wireless Access (FWA) service to residential and small business customers by the end of this year. While more use cases will be targeted after final standards are approved in 2020, we should see the first adopters next year and do expect to grow rapidly from there.

5G is a wildcard. The technology, still under development in 2018, promises a new era of connectivity with its huge bandwidth and extremely low latency, and is now being heavily promoted by the government, especially China. In an effort to shift the balance of technological innovation away from the US and Europe and towards China, the Chinese government views 5G adoption as a competitive asset.
Wireless Neighborhood Network (WNAN)

Wireless Neighborhood networks (WNAN) lie between WLAN and remote technologies such as cellular in terms of communication range. Typical proponents of this technology include mesh networks such as Wi-Sun or JupiterMesh. In some cases, the technology is used as an alternative to LPWA/Cellular (e.g., in utility domain networks) and in other cases as a complementary element (e.g., for metering at depths where nothing else reaches).
other

Other technologies such as satellites and unclassified proprietary networks will continue to play a role in the iot, albeit to a lesser extent than other technologies.

Global Iot market: $15.1 billion in 2018, projected to grow to $1.567 billion by 2025

Sentiment and the short-term outlook are very positive. As more and more data moves to the cloud, new iot applications enter the market and analytics become increasingly important, software and platforms will continue to drive the market. The 12-month outlook for connectivity and hardware remains stable as an increase in the number of iot devices drives sales, but hardware margins generally decline. Some hardware specialists (such as NVIDIA) don’t seem to need to commoditize hardware. With LPWA technology just coming to the global market and 5G not yet ready on a large scale, a gap in connected players is expected to emerge in 2019 and years after.

The global iot market — end-user spending on iot solutions — is expected to grow 37% from 2017 to $15.1 billion. These estimates have been revised upwards due to the market acceleration of iot (as described above) and are now projected to reach a total market of $156.7 billion by 2025.

Current Internet of Things market trends

Today’s market environment is extremely dynamic and dozens of trends can be observed, including edge to convergence, TSN connectivity and Internet of Things and blockchain experiments. For this version of the IoT Status Report, the IoT Analytics analyst team compiled 50 such insights.

Insight 32, for example, highlights that cloud vendors are increasingly making their own cloud-ready hardware to improve interoperability and performance between iot devices and data stored and analyzed in public or private clouds. Leading iot cloud providers Microsoft, Amazon and Google have all recently announced their own hardware.

The following is an excerpt from the report that highlights and provides evidence for this specific trend. It remains to be seen if these shifts from cloud to hardware pose a threat to some existing hardware players, such as gateway providers or chip manufacturers. Iot analytics will continue to monitor this and other trends.

Research methods and further information

All insights and charts in this article are from the “Internet of Things and Short Term Outlook 2018” report, a 56-page in-depth market report available to iot Analytics subscribers. The purpose of the report is to inform our subscribers of what is happening in the marketplace so that they remain at the forefront of innovation and ultimately make the right decisions for their companies.

The report includes 50 insights into the current market environment as well as market segmentation by industry, technology stack and geography. By leading more than 100 expert interviews, attending 16 industry conferences, listening to various briefings and analyst calls, the analyst team has found every insight and example point over the past six months.

IOT Technology snooping

As a developer, the author is not an investor or a pioneer. It doesn’t really matter what the details are. What I care about is how to use technology to implement or simulate an IOT server that supports millions of links. It is not rigorous, just for your reference.

Required technology:

MQTT

MQTT was developed in 1999 by Andy Stanford-Clark (IBM) and Arlen Nipper (Eurotech, now Cirrus Link) to monitor oil pipelines crossing the desert. The goal was to have a protocol that was bandwidth-efficient and used very little battery power because the devices were connected via satellite links, which were very expensive at the time. ** This protocol uses a publish/subscribe architecture in contrast to HTTP and its request/response paradigm. ** Publish/subscribe is event-driven and can push messages to clients. The central point of communication is the MQTT broker, which is responsible for scheduling all messages between senders and legitimate recipients. Each client that publishes a message to the broker includes a topic in the message. The ** topic is the routing information for the proxy. ** Each client that wants to receive messages subscribes to a topic, and the broker passes all messages with matching topics to the client. As a result, customers don’t have to know each other; they communicate only by subject. The architecture supports highly scalable solutions that are independent of data producers and data consumers.

MQTT publish/subscribe architecture

The difference with HTTP is that the client does not have to extract the required information, but the proxy pushes the information to the client in the case of new content. Therefore, each MQTT client has a permanently open TCP connection with the broker. If this connection is broken in any case, the MQTT broker can buffer all messages and send them to the client when it comes back online. As mentioned earlier, the core concept used in MQTT to dispatch messages is topics. A topic is a simple string that can have more hierarchical levels, separated by slashes. An example topic for sending temperature data for a living room could be house/living room/temperature. On the one hand, the client can subscribe to the exact topic, or on the other hand use wildcards. A subscription to house / + / temperature will cause all messages to be sent to the previously mentioned subject house/living room/temperature and any subject that has arbitrary values in the place of the living room, such as house/kitchen/temperature. The plus sign is a single-level wildcard, allowing only arbitrary values in one hierarchy. If you need to subscribe to multiple levels, such as the entire subtree, there is also a multilevel wildcard (#). It allows subscriptions to all levels of the underlying hierarchy. Like house / # subscribe to all topics that start with house.

Netty

Netty is an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers and clients.

Netty is a NIO client server framework that allows you to quickly and easily develop network applications such as protocol servers and clients. It greatly simplifies and simplifies network programming such as TCP and UDP socket servers.

“Fast and easy” does not mean that the final application will suffer from maintainability or performance issues. Netty is well designed with rich protocols such as FTP, SMTP, HTTP and a variety of binary and text-based traditional protocols. As a result, Netty has managed to find a way to achieve ease of development, performance, stability, and flexibility without compromising.

This can be taken a look at my blog and github website, relatively simple, god ignored. Continuously updated… !

Blog garden source: www.cnblogs.com/sanshengshu…

github.com/sanshengshu…

Sanshengshui.github. IO /

Kafka

ApacheKafka® is a distributed streaming media platform. What exactly does that mean?

Streaming media platforms have three key functions:

Publish and subscribe record flows, similar to message queues or enterprise messaging systems.
Store record streams in a fault-tolerant persistent manner.
Record the processing flow as it occurs.

Kafka is typically used in two broad categories of applications:

Build a real-time streaming data pipeline that reliably retrieves data between systems or applications
Build real-time flow applications that transform or respond to data flows

To understand how Kafka does these things, let’s dive into Kafka’s capabilities.

Redis
MySQL

Technical outline:

other

That concludes the IOT market and technology simulation.

Original is not easy, if you feel good, I hope to give a recommendation! Your support is the biggest motivation for my writing!

The following will gradually use Netty to implement MQTT protocol IOT server.

Author: Mu Shuwei