2021 Quality creation blog in the field of big data, take you from the entry to the master, the blog is updated every day, gradually improve the knowledge system of big data articles, to help you learn more efficiently.
Those who are interested in big data can follow the wechat public account sanbang Big Data
directory
Hadoop architecture
1. Introduction of x version architecture model
2. Introduction of x version architecture model
Type 1: NameNode and ResourceManager single-node architecture model
Second: NameNode high availability and ResourceManager single-node architecture model
Third, NameNode single-node and ResourceManager ha architecture model
Fourth, NameNode and ResourceManager HA architecture model
3. Introduction of x version architecture model
Hadoop architecture
1. Introduction of x version architecture model
File system core modules:
NameNode: The primary node in a cluster. It manages metadata (file size, file location, and file permissions), which is used to manage various data in the cluster
SecondaryNameNode: Used to manage metadata information in Hadoop
DataNode: Secondary nodes in a cluster that are used to store data in the cluster
Data computing core module:
JobTracker: receives computing request tasks from users and assigns tasks to secondary nodes
TaskTracker: Performs tasks assigned by the master JobTracker
2. Introduction of x version architecture model
Type 1: NameNode and ResourceManager single-node architecture model
File system core modules:
NameNode: Primary node in a cluster. It is used to manage data in the cluster
SecondaryNameNode: Used to manage metadata information in Hadoop
DataNode: Secondary nodes in a cluster that are used to store data in the cluster
Data computing core module:
ResourceManager: receives computing request tasks from users and allocates cluster resources
NodeManager: Is responsible for executing tasks assigned by the master APPmaster node
Second: NameNode high availability and ResourceManager single-node architecture model
File system core modules:
NameNode: primary node in a cluster. It is used to manage various data in the cluster. Two NameNode nodes are in high availability state
DataNode: Secondary nodes in a cluster that are used to store data in the cluster
JournalNode: Manages file system metadata information
Data computing core module:
ResourceManager: Receives computing request tasks from users, allocates cluster resources, and divides computing tasks
NodeManager: Performs tasks assigned by the active ResourceManager node
Third, NameNode single-node and ResourceManager ha architecture model
File system core modules:
NameNode: Primary node in a cluster. It is used to manage data in the cluster
SecondaryNameNode: Used to manage metadata information in Hadoop
DataNode: Secondary nodes in a cluster that are used to store data in the cluster
Data computing core module:
ResourceManager: Receives computing requests from users, allocates cluster resources, and divides computing tasks. Zookeeper implements high availability of ResourceManager
NodeManager: Performs tasks assigned by the active ResourceManager node
Fourth, NameNode and ResourceManager HA architecture model
File system core modules:
NameNode: primary node in a cluster. It is used to manage various data in the cluster. Generally, two nodes are used to implement HA
JournalNode: metadata information management process, usually an odd number
DataNode: secondary nodes used to store data
Data computing core module:
ResourceManager: the primary node of the Yarn platform. It is used to receive various tasks. Two types of ResourceManager are deployed to ensure high availability
NodeManager: secondary node of the Yarn platform. It processes tasks allocated by ResourceManager
3.x Introduction to the version architecture model
The basic architecture of Hadoop3.x is similar to that of Hadoop2.x, but hadoop3. x has added many new features, such as support for multiple Namenodes and optimization of HDFS and MapReduce.
- 📢 : lansonli.blog.csdn.net
- 📢 welcome to like 👍 collect ⭐ message 📝 if there is an error please correct!
- 📢 this article was originally written by Lansonli and originally appeared on CSDN blog 🙉
- 📢 big data series of articles will be updated every day, stop to rest do not forget that others are still running, I hope that we seize the time to learn, strive for a better life ✨