2021 Big data field high-quality creation blog, take you from entry to mastery, the blog updates every day, gradually improve the big data knowledge system of the article, help you more efficient learning.
Those who are interested in big data can follow the wechat official account: Sanbang Big Data
Hadoop is introduced
Hadoop is an open source software framework in the Java language, which is owned by Apache. Hadoop is a software platform for developing and running large-scale data processing. Allows distributed processing of large data sets on large clusters of computers using a simple programming model.
In a narrow sense, Hadoop refers to Apache, an open source framework whose core components are:
Distributed file system (HDFS) : Provides massive data storage
MAPREDUCE (distributed computing programming framework) : Solves massive data computing
YARN (a framework for job scheduling and cluster resource management) : Resolves resource task scheduling
Broadly speaking, Hadoop usually refers to a broader concept called the Hadoop ecosystem.
Hadoop today has grown into a large system, and as the ecosystem has grown, more and more projects have emerged, some of which are not led by Apache, that complement Hadoop or are higher level abstractions. Such as:
| The framework | use |
|---|---|
| HDFS | Distributed file system |
| MapReduce | Distributed computing program development framework |
| ZooKeeper | Distributed coordination service base component |
| HIVE | Distributed data warehouse based on HADOOP, providing SQL based query data operation |
| FLUME | Log data collection framework |
| oozie | Workflow Scheduling framework |
| Sqoop | Data import and export tool (for example between mysql and HDFS) |
| Impala | Real-time SQL query analysis based on Hive |
| Mahout | Machine learning algorithm library based on distributed computing framework such as MapReduce/Spark/Flink |
- 📢 Blog homepage: lansonli.blog.csdn.net
- 📢 Welcome to like 👍 collect ⭐ message 📝 if there is any error please correct!
- 📢 this post was originally written by Lansonli and originally published on the CSDN blog 🙉
- 📢 Big data series will be updated every day. When you stop to rest, don’t forget that others are still running. I hope you can seize the time to learn and strive for a better life ✨