Hbase and Hadoop are also classified into single-server, pseudo-distributed, and fully distributed cluster versions. This document describes how to set up a fully distributed cluster environment.

Hbase depends on the Hadoop environment. Before building HaBase, you need to build a full Cluster environment of Hadoop. Therefore, before reading this article, you need to read my previous article on Hadoop Distributed Cluster Building. In this document, hbase zooKeeper is used instead of an independent ZooKeeper.

Environment to prepare

  • Hbase package: http://mirror.bit.edu.cn/apache/hbase/1.3.1/hbase-1.3.1-bin.tar.gz
  • The Hadoop cluster environment is set up

Install the hbase

After the hadoop-master installation is configured, copy to the slave node

Wget http://mirror.bit.edu.cn/apache/hbase/1.3.1/hbase-1.3.1-bin.tar.gz # extract tar - XZVF hbase - 1.3.1 - bin. Tar. Gz - C /usr/local/ # rename mv hbase-1.3.1 hbaseCopy the code

Configure the environment variable vim /etc/profile

Export HBASE_HOME=/usr/local/hbase export PATH=$HBASE_HOME/bin:$PATH source /etc/profileCopy the code

Example Change the system variable ulimit

ulimit -n 10240
Copy the code

The configuration file

Hbase-related configuration includes hbase-env.sh, hbase-site. XML, and regionServers, which are stored in the /usr/local/hbase-/ conf directory:

Configure hbase – env. Sh

Vim hbase-env.sh # Export JAVA_HOME=/usr/lib/ JVM/jre-1.7.0-openJDk.x86_64 export HBASE_CLASSPATH=/usr/local/ hbase.conf Zookeeper is managed by hbase itself. No separate ZooKeeper is required. Export HBASE_MANAGES_ZK=true export HBASE_HOME=/usr/local/hbase export HADOOP_HOME=/usr/local/hadoop # hbase log directory export HBASE_LOG_DIR=/usr/local/hbase/logsCopy the code

Configure hbase – site. XML

<configuration>
	<property>
		<name>hbase.rootdir</name>
		<value>hdfs://hadoop-master:9000/hbase</value>
	</property>
	<property>
		<name>hbase.cluster.distributed</name>
		<value>true</value>
	</property>
	<property>
		<name>hbase.master</name>
		<value>hadoop-master:60000</value>
	</property>
	<property>
		<name>hbase.zookeeper.quorum</name>
		<value>hadoop-master,hadoop-slave1,hadoop-slave2,hadoop-slave3</value>
	</property>
</configuration>
Copy the code

Configuration regionservers

vim /usr/local/hbase/conf/regionservers
hadoop-master
hadoop-slave1
hadoop-slave2
hadoop-slave3
Copy the code

Copy hbase to a secondary node

scp -r /usr/local/hbase hadoop-slave1:/usr/local/
scp -r /usr/local/hbase hadoop-slave2:/usr/local/
scp -r /usr/local/hbase hadoop-slave3:/usr/local/
Copy the code

Start the hbase

The startup is performed only on the master node

~/hbase/bin/start-hbase.sh
Copy the code

List of processes on master and slave after startup

Information in master

[hadoop@master ~]$JPS 6225 JPS 2897 SecondaryNameNode hadoop process 2710 NameNode Hadoop Master process 3035 ResourceManager # Hadoop process 5471 HMaster # hbase Master process 2543 HQuorumPeer # ZooKeeper processCopy the code

Information in salve

[hadoop@slave1 ~]$JPS 4689 JPS 2533 HQuorumPeer # ZooKeeper process 2589 DataNode # Hadoop slave process 4143 HRegionServer # hbase Slave processCopy the code

Hbase relies on Hadoop. Therefore, hbase must be started and stopped in sequence

If an independent ZooKeeper is installed

Start sequence: Hadoop > ZooKeeper > hbase Stop sequence: hbase > ZooKeeper > Hadoop

Use the built-in ZooKeeper

Start sequence: Hadoop -> hbase Stop sequence: hbase-> Hadoop

Restart hbase

~/hbase/bin/stop-hbase.sh
~/hadoop/sbin/stop-all.sh
~/hadoop/sbin/start-all.sh
~/hbase/bin/start-hbase.sh
Copy the code

Error handling

During the build process, an error was reported with the following error message:

Unhandled: org, apache hadoop. Hbase. ClockOutOfSyncException: Server hadoop - slave3, 16020150526553, 33 under Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): Org, apache hadoop, hbase. ClockOutOfSyncException: Server hadoop - slave3, 33, 16020150526553 has had rejected; Reported time is too far out of sync with master. Time difference of 77348ms > max allowed of 30000msCopy the code

The primary node failed to connect to the secondary node. There may be two reasons. First, the time of the Linux server is inconsistent. Second, the connection time is too long due to other network reasons.

Solution:

The first reason is to change the server time to be consistent. The final solution is: set a time to use NTP to synchronize time from a server

0 */1 * * * /usr/sbin/ntpdate 192.168.0.12; /sbin/hwclock -wCopy the code

Performed manually

/usr/sbin/ntpdate 192.168.0.12Copy the code

Second, you can change the default maximum hbase link duration to be longer.

Add the connection duration attribute to the HBase configuration file hbase-siter. XML

<property>
    <name>hbase.master.maxclockskew</name>
    <value>120000</value>
 </property>
Copy the code

Reference:

Hbase 1.0.1 Distributed Cluster Construction in centos 6.4

The original:Hbase distributed cluster construction