I’ve been doing big data for 6 years, and today I’m going to talk about my technology growth history. High salary, many opportunities, and big gaps make big data a hot property in the development circle. At the same time, during my two years as an official account, I have witnessed too many people “from entry to give up”, and some people have not even entered the door of big data. Which one are you? I have done big data in small and medium-sized enterprises for a period of time, but I only do a small piece of work in the whole process of big data, and I have no idea about the whole process and how to select the type. Moreover, the data level of the company is not enough, so it is difficult to move to a big factory.

I studied big data by myself for a while, but I only learned superficial things. There was no database to simulate storage and calculation, and I only dared to write “understand” certain technology on my resume. Finally, I couldn’t even find a job.

Want to switch to big data, suffering into the line no door……

The above situations are caused by the fact that we have not experienced real projects or received systematic training. The salary of doing big data is high, but the threshold is also high, because no matter what level you are, you should have used the technology stack required. Otherwise, it is difficult to enter small and medium-sized enterprises, let alone big factories.

So how do you learn big data? Today, I would like to talk with you about my learning path and method.

Stage 1: Master Java Web data visualization. You need to master Java server-side technology, front-end visualization technology, and database technology. This stage is mainly to reserve pre-big data skills. Of course, you can work as a data visualization engineer, but you can’t really start big data.

Stage 2: Learn about the Hadoop core and ecosphere technology stack. This part covers a lot of technologies, such as HDFS distributed storage, MapReduce, Zookeeper, Kafka and so on. After mastering these technologies, I can be engaged in some big data positions such as ETL engineer, but my knowledge reserve is not complete enough.

In stage 3, work out the computing engine and analysis algorithm. I suggest that both Spark and Flink can be used proficiently. Although Spark is still used by some enterprises, Flink will definitely become the mainstream in the future. With this knowledge, you will have a relatively complete set of big data skills that will enable you to take on some high-paying jobs, such as big data r&d engineer, recommendation system engineer, user portrait engineer, etc.

I have collected some internal data of big data industry, including big data engineer manual, big data development learning roadmap, as well as the real interview questions of Meituan, Byte and other big factories. Students who want to get them can scan the code for free.