The application and development of big data is biased towards the bottom layer, which is difficult to learn and involves a wide range of technologies, which restricts the popularization of big data. Now a technology is needed to encapsulate some common and reused basic codes and algorithms in big data development into class libraries, so as to reduce the learning threshold of big data, reduce the difficulty of development and improve the development efficiency of big data projects. There are three applications of big data in work: business related, such as user portrait and risk control; Related to decision making, data science, knowledge of statistics, algorithms, which is the domain of the data scientist; Related to engineering, how to implement, how to implement, what business problems to solve, that’s the job of data engineers.
