According to a study by olin School of Business at The University of Washington, 40 percent of the fortune 500 will disappear in 10 years.

In the past, enterprises paid attention to every little helps

Hard work, brick by brick, from the budding to the industry pioneer, eventually changed the building from the ground up.

Now the wise builders turn to swim. Riding the Internet wave and watching the next wave emerge, they are waiting for the underlying Technology of new IT (Intelligence Technology) to mature.

As astute sharks, leaders sniff left and right and smell blood in the water. As they figure out how to remake inefficient companies to provide consumers with everything better, faster and cheaper, the old systems, the self-proclaimed monopolists, will collapse.

There is a consensus among industry insiders familiar with the frontier of technology that if AI, robot butlers, brain-computer interfaces and smart wearables are to be regarded as the babies with great talent and potential, they must be fed with high-quality, processed data — especially real-time data.

Applications driven by real-time data processing tend to upend most people’s perceptions of efficiency. For example, in the IoT field, real-time device data analysis can detect system failures within 1 second and make maintenance predictions at the same time; In the financial field, the data of trading system not only need to be presented to customers in real time, but also need to be submitted to automatic trading system for processing as soon as possible. In IT field, the log collection and processing of software system is very important for service abnormal alarm and fault detection. In the field of traffic, real-time traffic data has been used to optimize traffic command through processing.

The most representative one is e-commerce. Platforms that have sprung up due to new retail hold massive user transaction data and backstage commodity update data. Such data has high requirements for real-time performance: transaction processing, backstage presentation, quick response after payment, and settlement of disputes… They’re all supported by servers. Especially now discount, promotion, second kill, haggling and other marketing models emerge in an endless stream. If there is a problem with the system, the e-commerce platform will lose traffic and suffer huge losses. Therefore, access to information, monitoring and real-time processing of commodity transaction data is particularly important for such enterprises.

In addition, in the era of big data, the sales end of enterprises will gradually move towards intelligent recommendation system. Traditional recommendation systems use periodic data analysis to update the model. Due to its regularly updated characteristics, the recommendation model cannot keep real-time, and the recommendation results are not accurate enough for users’ current behaviors. In such a scenario, real-time data analysis is the equivalent of a shopping guide who sees through the consumer at a glance.

In the new IT wave, real-time data collection and data-based analysis and processing have become the general trend. However, with more and more kinds of data, faster and faster generation speed and more and more data, many enterprises are facing the limitation of technology and equipment, and the technical barrier of real-time data processing has surfaced. To sum up, the main difficulties are:

1. Very large amount of data; The system requirements are very high. When processing data, the system requirements for real-time processing are much higher than those for offline systems.

2. The scale of real-time processing system cannot keep up with the demand of business growth; Much real-time data, such as k-line analysis in finance, requires specialized sequential database techniques that are not widely available.

3. It is not easy to build open source components by ourselves, such as Kafka, Storm and Hbase. The deployment, operation and maintenance of Hadoop open source components require a lot of money and manpower.

Fundamentally, the three most fundamental requirements of real-time data processing are: data access, real-time analysis and processing of data, and data storage. In view of the growing demand for real-time data processing on the cloud, Huawei Cloud EI Service Product Department has developed “three senders” for real-time data processing:

Data Access Service (DIS), Cloud Stream Service (CS) and Table Storage Service (CloudTable).

• Data Access Service (DIS) is a fully hosted real-time data access service provided by Huawei Cloud. DIS provides flexible data collection, efficient data transmission, real-time data distribution capabilities, so that users can easily build real-time data-based analysis and applications.

• Real-time streaming Computing Service (CS) is a real-time streaming big data analysis service, fully hosting computing resources and Serverless experience, real-time execution of jobs, providing intelligent streaming computing platform with low latency and high throughput.

• Table Storage Service (CloudTable) is based on Apache

HBase is a distributed, scalable, and fully managed NoSQL data storage service that provides millisecond random read/write capabilities and is applicable to massive structured and semi-structured data storage and query applications. OpenTSDB and GeoMesa will also provide time-series database capabilities and spatio-temporal big data query and analysis capabilities.

He dabing responded to Anil on weibo

Menon: “Who judges Huawei’s ability to innovate? It should be the market and customers.” At present, there are many benchmark customers using the “Three Musketeers” service to process real-time data on Huawei cloud. A gas group, through the use of DIS,

CS and CloudTable built a new nationwide patrol system, which reduced the end-to-end query performance of the real-time patrol monitoring system from tens of seconds to less than seconds. The real-time trading system of a financial start-up makes full use of OpenTSDB capability built into CloudTable to refresh the k-line market of 15 time Windows in real time. In a network retail real-time public opinion system, the cost performance of DIS access data far exceeds that of offline self-built systems, and CS SQL programming is used to easily complete data cleaning.

It only takes a few years for the Internet to grow from barbarism to deep cultivation. As the market matures, how to provide better service and faster data decision-making has become the key point of competition. As one of the key technologies, real-time data processing is widely popular in the industry. It has become the consensus of various enterprises to exploit the value of “thermal data” to the maximum by using technology as potential energy.

We live in a time of constant disintegration, and that’s a good thing.

Industries will be disrupted and big companies will collapse.

How many enterprises, investors and entrepreneurs, unaware of the arrival of the new IT era, only in the Internet +O2O 9.6 million square kilometers of the bed of HIGH infatuation, thousands of sail has passed.

Jump on the real-time data processing bandwagon.