Before we study big data, the first thing we need to know is:
1. What is big data?
2. What does big data do?
3. What is the employment situation like in the field of big data employment?
4. After making clear the above three points, we can start to learn big data
To determine the learning route, zero programming basic white how to learn?
It will take you about 20 minutes to read this article carefully
Many beginners, the concept of big data are ambiguous, what big data is, what to do, to learn, which according to the routes to learn, what learned where aspects of development, want to understand, want to learn the students 72268 o258, welcome to join big data have a lot of dry goods (zero foundation as well as the advanced practical classic) to share with everyone, And there are senior big data lecturers graduated from Tsinghua University to give free lectures to share the most complete domestic high-end practical learning process system of big data
First: What is big data and what does it do
The rapid development of modern science and technology, on the one hand, brings convenience to people’s life; On the other hand, it also gives people more and more impact on their work and life. The coming high-tech such as the Internet of Things, artificial intelligence, big data, cloud computing and intelligent hardware will further overturn people’s traditional way of life and work, so it is necessary for us to know and understand them.
Big Data definition:
According to the definition given by the National Institute of Standards and Technology (NIST), big data refers to data with large quantity, fast acquisition speed or various forms, which is difficult to be effectively analyzed by traditional relational data analysis methods, or requires large-scale horizontal expansion to be efficiently processed.
Meaning of Big Data:
Big data is a new data management technology that helps enterprises build disruptive advantages by leveraging massive data assets to gain real-time and accurate insight into dynamic changes in unknown logical fields and rapidly reshape business processes, organizations and industries:
(1) Insight into the unknown: Diversified data enables enterprises to leverage a wider range of data to support more dimensions of enterprise analysis needs, not limited to the analysis of known facts, thus increasing strategic insight;
(2) Optimization process: dynamic analysis changes can enable enterprises to monitor and analyze the deficiencies of the business process in real time, and constantly optimize the business process;
③ Real-time response: data can be accessed in real-time analysis to accelerate the speed of enterprise access to information and analysis, so as to make users more sensitive to the changes in the market.
Technical support of big data:
1. The store
(1) The decrease of storage cost, the decrease of storage cost, also changed people’s view of data, more willing to save 1 year, 2 years or even longer historical data, with the precipitation of historical data, can be compared to find the correlation and value between data;
(2) The best infrastructure for big data can be built because of the falling cost of storage;
2. Calculate
The computing speed is getting faster and faster. Massive data will go through storage, cleaning, mining, analysis and other links from original data source to value generation. If the computing speed is not fast enough, many things cannot be achieved.
3. Smart
The biggest value brought by big data is “intelligence”. Big data makes machines intelligent, while ARTIFICIAL intelligence further improves their ability to process and understand data.
Several commonly used functions of big data:
1. The tracking
The Internet and the Internet of Things are recording all the time. Big data can track and trace any record to form a real historical track. Tracking is the starting point for many big data applications, including consumer purchase behavior, purchase preferences, means of payment, search and browsing history, location information, and more.
2. Identify
On the basis of comprehensive tracking of various factors, accurate recognition can be realized through positioning, comparison and screening, especially for speech, image and video recognition, so that the content of analysis can be greatly enriched and the results obtained are more accurate.
3. The portrait
Through the tracking, identification and matching of different data sources of the same subject, a more three-dimensional characterization and a more comprehensive understanding can be formed. For consumer portraits, advertisements and products can be pushed accurately; Enterprise portrait, can accurately judge its credit and risk.
4. Prompt
On the basis of historical track, identification and portrait, it predicts the future trend and the possibility of repeated occurrence, and gives hints and warnings when some indicators show expected changes or exceed expected changes. In the past, there were also statistics-based forecasts. Big data greatly enriched the forecasting methods and had profound significance for the establishment of risk control models.
5. Matching Accurate tracking and identification in the mass of information, using relevance, proximity and other screening and comparison, to achieve product tie-in and supply and demand matching more efficiently. The big data matching function is the foundation of new business models of sharing economy such as Internet car booking, rental housing and finance.
6. Optimize
According to the given principle of shortest distance and lowest cost, various algorithms are used to optimize the configuration of paths and resources. For enterprises, improve service level and internal efficiency; For the public sector, saving public resources and improving public service capacity.
Second: What is the future of big data?
In recent years, the rapid development of mass data has become a hot topic of concern in the industry, academia and the world.
McKinsey & Company is a famous management consulting firm. Its data has permeated various industries and business areas, and has become an important factor of production. The U.S. government announced a $200 million big data research and development initiative in 2012.
Ownership and control of data will become a new focus of contention between countries and companies. Big data is becoming a new hot spot after cloud computing. The era of big data has come, and there are huge business opportunities hidden behind big data. A number of well-known companies, including IBM, Microsoft, Google and Amazon, have driven the development of the gold market. Domestic companies are also seeing a gold rush.
For example, Alibaba actively builds a data loop to collect and share the underlying architecture. Huawei is providing a professional and stable IT infrastructure platform for large-scale data mining and analysis. The Internet’s big data collection center collected more than 2 petabytes of data.
Tencent is using user relationship data and social data to return data to wechat e-commerce products in Qzone. Zte has launched an efficient data center overall service solution with ICT services as the core.
Dawning China Science and Technology introduced XDATA big data machine. Digital China has launched a smart city strategy. Research and development in the field of big data processing in business analysis. Use the resources accumulated in social security industry to build an intelligent medical platform.
Gao DE and Ali will cooperate in map search, product commercialization, data sharing, cloud computing and other areas. Gao De, as a content provider for the development of geographic information systems for map navigation, is now trying to use big data to inform government decision-making. For countries, big data is the new oil of the future. Corporate big data is the blue ocean of their dreams.
For those of you who live in the era of big data, if you don’t know big data, you really need to leave. First, we need to know what is big data? Customer service companies that sell pizza are using big data. Accurately analyze their customers’ favorite flavors, varieties and consumption habits.
Why does customer service in this paragraph sell so precisely to the average user? This is because they have a magical sales system behind them, isn’t it a system? Is there anything like that? There! Not only did he identify customers over the phone, he learned about their blood pressure, cholesterol and health care systems, based on their records in the central library, they recommended a healthy low-fat pizza to the customer, but also incidentally to the elderly mother of healthy people.
When a customer makes a payment, the system obtains the customer’s credit history. When the customer delivers, the system will locate the person on the motorcycle, deliver pizza cash, and ride the motorcycle himself.
Mobile Internet is a hot topic at the moment, and its biggest influence is not only people, but also the exponential growth of data compared to the past. That trend is likely to continue as more devices become connected. What the data is, it’s not just 0,1, it’s the world’s perception of itself, of human behavior. When we complain that the devices around us are not smart enough, in fact, these devices know too little about us. When enough user behaviors are observed and analyzed by them, they will become more and more intelligent and humanized, and this is the real trend of big data.
From the present point of view: No data, No learning. No data, No intelligence. Data is the blood. No algorithm can get rid of the data. Machine learning and statistics never worry about too much data, only too little.
So you don’t have to worry about not being able to extract value from the data, it’s how they extract it (what algorithm) and what value they extract from it (what their output is).
So the big data craze is not going away. It’s going to be more and more important in the future as we get smarter, as more sensors, as more websites collect more and more of what people do, as more and more electronic ways of expressing the world.
But it will slowly back to the background, a group of pure hype it, not reasonable use of its value of enterprises out, with the emergence of new enterprises, the real play of its value.
After reading the above, you will no longer hesitate to learn Java, Python or big data.
Third: How to learn big data based on zero?
Many beginners, the concept of big data is vague, what is big data, what can be done, when learning, according to what line to learn, after learning to which aspect of development, want to have a deeper understanding, want to learn students welcome to join, there are a lot of dry goods (zero basis and advanced classical combat) to share with you, And there are senior big data lecturers graduated from Tsinghua University to give free lectures to share the most complete domestic high-end practical learning process system of big data
The study of big data development focuses on mastering basic knowledge and practical application. Reasonable arrangement of basic knowledge learning can achieve twice the result with half the effort. The following are the classic learning routes of big data development:
1. Basic introduction to big data, such as JavaSe, MySQL, Linux, HTML, CSS, JS.
2. Big data Hadoop basics, such as Data Introduction, Hadoop framework, HDFS distributed file system, MapReduce computing model.
3. Offline analysis of big data, such as Hive Data Warehouse, Sqoop, and Azkaban.
4. Real-time computing of big data, such as Zookeeper, HBase, Redis, Kudu, Storm, and Kafka.
5.Spark data calculation, such as Scala, RDD, Mahout, and Python.
The above technologies are from the basic to the advanced, in fact, it is not so difficult, persist, I believe that we will be able to learn, the salary of the big data industry is still very high, it is doomed to pay. At the same time, the learning system chart I summarized is more intuitive and systematic than the written description. It was compiled by several front-line Internet employees who have been working in the big data industry for a long time. They are interested in saving the HIGH-DEFINITION map and studying it later.
But before you learn, it’s important to know which talents are needed in the data age, as follows:
1. Big data system R&D Engineer:
Responsible for the research and development of big data system, including large-scale unstructured data business model construction, big data storage, database construction, database architecture optimization, database center design, etc. Meanwhile, I am also responsible for the daily operation of data cluster and system monitoring.
2. Big data application development Engineer:
Responsible for building big data application platforms and developing analytical applications, and developing various applications and industry solutions based on big data technology. To extract data from different sources, conversion and import the data warehouse to meet the needs of the enterprise, the data in a distributed and heterogeneous data sources such as relational data and graphic data files extracted into a temporary middle layer after cleaning, conversion, integration, finally is loaded into the data warehouse, as the foundation of on-line analytical processing, data mining, data to create the conditions for extraction of various types of needs.
Big data analyst
Engage in data mining, use algorithms to solve and analyze problems, let the data reveal the truth, and promote the continuous update of data solutions.
4. Data visualization engineer
Responsible for the application of graphical tools and means in the collected high-quality data, clearly reveal the complex information in the data, visualize it, and help users to better develop big data applications.
5, data security research and development talents
Responsible for the internal large server, storage, data security management, and network, information security project planning, design and implementation.
6. Zero-based learning route is shown as follows: