In order to learn more about hive data warehouse, here is the hive installation and deployment tutorial. Subsequent articles will explain the principles and use of Hive in detail. Prerequisites for Hive installation are as follows: 1. Install the Hadoop cluster of the corresponding version and start the HDFS and YARN service of Hadoop. 2. The MySQL service is installed and started

1. Install mysql as user root

  • In CentOS 7, switch to user root and install mysql
  • CentOS 7 has MariaDB installed by default. MariaDB is a branch of MySQL. However, MySQL will still be installed, and MariaDB will be overwritten when it is installed

2. Installation of mysql

2.1 Download and install the official mysql YUM source

Install to the third node node03

  • Run the following command in the/KKB /soft directory on the CentOS 7 server as user root
  • Switch to user root
[hadoop@node03 ~]$ su root
Copy the code
  • Go to the/KKB /soft directory and install the WGET software
[root@node03 hadoop]# cd /kkb/soft/
[root@node03 soft]# yum -y install wget
Copy the code

The Installed! “, indicating that the installation is successful

  • Run the wget command to download the MYSQL RPM package
[root@node03 soft]# wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
Copy the code

-i Specifies the input file

-c indicates resumable data

Install mysql 2.2

[root@node03 soft]# yum -y install mysql57-community-release-el7-10.noarch.rpm
Copy the code
  • Installing mysql Server may take some time and requires an online download, depending on network speed. Then install; Once installed, the previous Mariadb will be overwritten
[root@node03 soft]# yum -y install mysql-community-server
Copy the code

3. Set the mysql

3.1 the mysql service

  • Start the MySQL service first
[root@node03 soft]# systemctl start mysqld.service
Copy the code
  • Check the mysql startup status
[root@node03 soft]# systemctl status mysqld.service
Copy the code

In the following figure, active (running) indicates that the mysql service has been started

3.2 Changing a Password

  • If you want to log in to MySQL, you need to find the temporary password of user root

To find the temporary password in the log file, run the following command

[root@node03 hadoop]# grep "password" /var/log/mysqld.log
Copy the code
  • You can check that my temporary password is

Note: == Different people’s temporary password is different, according to their actual situation ==

Copy the code

  • Log in to the mysql client using a temporary password
[root@node03 hadoop]# mysql -uroot -p
Copy the code
  • Set the password policy to LOW, which checks only the password length
set global validate_password_policy=LOW;
Copy the code

Query OK indicates that the SQL statement is successfully executed

  • Set the minimum password length
set global validate_password_length=6;
Copy the code

  • Change the local login password of user root to 123456
ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
Copy the code
  • Enable the remote connection permission for mysql
grant all privileges  on  *.* to 'root'@The '%' identified by '123456' with grant option;
flush privileges;
Copy the code
  • If you no longer need to use the mysql command line, exit
Copy the code

2. Install and deploy Hive

Note Hive is a tool for building data warehouses. You only need to install hive on one server rather than multiple servers.

The following uses node03 as an example.

Use == Hadoop common user == operation

1.1 Prerequisites

  • Build a three-node Hadoop cluster;
  • MySQL server installed on node03

1.2 Preparing installation packages

  • To download the Hive installation package, visit the hive official website:

    • Archive.cloudera.com/cdh5/cdh/5/…
I am here ready to install packages available for download, or you can pay attention to my public reply hive for installing mysql - connector jar package links: https://pan.baidu.com/s/1PDyYDXX-PIohe-m7pk4GLA password: HFTP hive installation package links: https://pan.baidu.com/s/1GI3mLcvobzGQbOcwruwFzA password: jignCopy the code
  • Planning an Installation Directory

    • /kkb/install
  • Upload the installation package to/KKB /soft on the node03 server

1.3 unzip

  • Decompress the installation package to the specified directory/KKB /install
[hadoop@node03 ~]$ cd/ KKB /soft/ [hadoop@node03 soft]$tar -xzvf hive-1.1.0-cDH5.14.2.tar. gz -c/KKB /install/Copy the code

1.4 Modifying the Configuration File

  • Modify == configuration file hive-env.sh==

    • Go to the conf folder in the Hive installation directory
[hadoop@node03 soft]$ cdKKB/install/hive - 1.1.0 - cdh5.14.2 / conf /Copy the code
  • Rename the hive – env. Sh. The template
[hadoop@node03 conf]$ mv hive-env.sh.template hive-env.sh
Copy the code
  • Modify the hive – env. Sh
[hadoop@node03 conf]$ vim hive-env.sh 
Copy the code
  • Modify the values of HADOOP_HOME and HIVE_CONF_DIR in this file as follows (according to the actual situation of your machine)
Configure the HADOOP_HOME path
exportHADOOP_HOME = / KKB/install/hadoop - server - cdh5.14.2 /Configure the HIVE_CONF_DIR path
exportHIVE_CONF_DIR = / KKB/install/hive - 1.1.0 - cdh5.14.2 / confCopy the code

Modify == configuration file hive-site. XML ==

  • Conf does not have this file by default. Vim can create this file
[hadoop@node03 conf]$ vim hive-site.xml
Copy the code
  • The file contents are as follows

                <value>jdbc:mysql://node03:3306/hive? createDatabaseIfNotExist=true&amp;characterEncoding=latin1&amp;useSSL=false</value>

Copy the code

Modify == log configuration file hive-log4j.properties==

  • Create a hive log storage directory
[hadoop@node03 conf]$mkdir -p/KKB /install/hive-1.1.0-cdh5.14.2/logs/Copy the code
  • Rename the generated hive-log4j.properties file
[hadoop@node03 conf]$ pwdKKB/install/hive - 1.1.0 - cdh5.14.2 / conf/hadoop @ node03 conf] $mv hive - log4j. The properties, the template hive - log4j. Properties [hadoop@node03 conf]$ vim hive-log4j.properties# modify file
Copy the code
  • Modify the value of the hive.log.dir property of this file
# Change the following content, set our hive log file store path, easy troubleshootingHive. Log. Dir = / KKB/install/hive - 1.1.0 - cdh5.14.2 / logs /Copy the code

1.5 Copying the mysql Driver Package

  • Upload the mysql driver package, such as mysql-connector-java-5.1.38.jar, to the/KKB /soft directory

  • If ==, you need to upload the mysql driver package to the Hive lib directory because metadata is read and written to the mysql database during Hive running

[hadoop@node03 ~]$ cd/ KKB /soft/ [hadoop@node03 soft]$cp mysql-connector-java-5.1.38.jar/KKB /install/hive-1.1.0-cdh5.14.2/lib/Copy the code

1.6 Configuring Hive Environment Variables

  • Switch to user root
[hadoop@node03 soft]$ su root
Copy the code
  • Open the/etc/profilefile
[root@node03 soft]# vim /etc/profile
Copy the code
  • At the end of the article
exportHIVE_HOME = / KKB/install/hive - 1.1.0 - cdh5.14.2export PATH=$PATH:$HIVE_HOME/bin
Copy the code
  • Switch back to the Hadoop user and source
[root@node03 soft]# su hadoop
[hadoop@node03 soft]$ source /etc/profile
Copy the code

1.7 Verifying the Installation

  • == The Hadoop cluster is started ==
  • == The mysql service is started ==
  • Start the Hive CLI client in any directory on Node03
[hadoop@node03 ~]$ hive
Copy the code
  • See what databases are available
show databases;
Copy the code

  • The Hive installation is successful
  • Exit the cli
Copy the code


Now that hive is installed, you can enjoy learning about Hive. For more dry goods, please pay attention to my personal public account, pay attention to receive benefits