@TOC

preface

In order to learn more about hive data warehouse, here is the hive installation and deployment tutorial. Subsequent articles will explain the principles and use of Hive in detail. Prerequisites for Hive installation are as follows: 1. Install the Hadoop cluster of the corresponding version and start the HDFS and YARN service of Hadoop. 2. The MySQL service is installed and started

1. Install mysql as user root

In CentOS 7, switch to user root and install mysql
CentOS 7 has MariaDB installed by default. MariaDB is a branch of MySQL. However, MySQL will still be installed, and MariaDB will be overwritten when it is installed

2. Installation of mysql

2.1 Download and install the official mysql YUM source

Install to the third node node03

Run the following command in the/KKB /soft directory on the CentOS 7 server as user root
Switch to user root

[hadoop@node03 ~]$ su root
Copy the code

Go to the/KKB /soft directory and install the WGET software

[root@node03 hadoop]# cd /kkb/soft/
[root@node03 soft]# yum -y install wget
Copy the code

The Installed! “, indicating that the installation is successful

Run the wget command to download the MYSQL RPM package

[root@node03 soft]# wget -i -c http://dev.mysql.com/get/mysql57-community-release-el7-10.noarch.rpm
Copy the code

-i Specifies the input file

-c indicates resumable data

Install mysql 2.2

[root@node03 soft]# yum -y install mysql57-community-release-el7-10.noarch.rpm
Copy the code

Installing mysql Server may take some time and requires an online download, depending on network speed. Then install; Once installed, the previous Mariadb will be overwritten

[root@node03 soft]# yum -y install mysql-community-server
Copy the code

3. Set the mysql

3.1 the mysql service

Start the MySQL service first

[root@node03 soft]# systemctl start mysqld.service
Copy the code

Check the mysql startup status

[root@node03 soft]# systemctl status mysqld.service
Copy the code

In the following figure, active (running) indicates that the mysql service has been started

3.2 Changing a Password

If you want to log in to MySQL, you need to find the temporary password of user root

To find the temporary password in the log file, run the following command

[root@node03 hadoop]# grep "password" /var/log/mysqld.log
Copy the code

You can check that my temporary password is

Note: == Different people’s temporary password is different, according to their actual situation ==

fHy3Su:&REkh
Copy the code

[root@node03 hadoop]# mysql -uroot -p
Copy the code

Set the password policy to LOW, which checks only the password length

set global validate_password_policy=LOW;
Copy the code

Query OK indicates that the SQL statement is successfully executed

Set the minimum password length

set global validate_password_length=6;
Copy the code

Change the local login password of user root to 123456

ALTER USER 'root'@'localhost' IDENTIFIED BY '123456';
Copy the code

Enable the remote connection permission for mysql

grant all privileges  on  *.* to 'root'@The '%' identified by '123456' with grant option;
flush privileges;
Copy the code

If you no longer need to use the mysql command line, exit

exit
Copy the code

2. Install and deploy Hive

Note Hive is a tool for building data warehouses. You only need to install hive on one server rather than multiple servers.

The following uses node03 as an example.

Use == Hadoop common user == operation

1.1 Prerequisites

Build a three-node Hadoop cluster;
MySQL server installed on node03

1.2 Preparing installation packages

To download the Hive installation package, visit the hive official website:
- Archive.cloudera.com/cdh5/cdh/5/…

I am here ready to install packages available for download, or you can pay attention to my public reply hive for installing mysql - connector jar package links: https://pan.baidu.com/s/1PDyYDXX-PIohe-m7pk4GLA password: HFTP hive installation package links: https://pan.baidu.com/s/1GI3mLcvobzGQbOcwruwFzA password: jignCopy the code

Planning an Installation Directory
- /kkb/install
Upload the installation package to/KKB /soft on the node03 server

1.3 unzip

Decompress the installation package to the specified directory/KKB /install

[hadoop@node03 ~]$ cd/ KKB /soft/ [hadoop@node03 soft]$tar -xzvf hive-1.1.0-cDH5.14.2.tar. gz -c/KKB /install/Copy the code

1.4 Modifying the Configuration File

Modify == configuration file hive-env.sh==
- Go to the conf folder in the Hive installation directory

[hadoop@node03 soft]$ cdKKB/install/hive - 1.1.0 - cdh5.14.2 / conf /Copy the code

Rename the hive – env. Sh. The template

[hadoop@node03 conf]$ mv hive-env.sh.template hive-env.sh
Copy the code

Modify the hive – env. Sh

[hadoop@node03 conf]$ vim hive-env.sh 
Copy the code

Modify the values of HADOOP_HOME and HIVE_CONF_DIR in this file as follows (according to the actual situation of your machine)

Configure the HADOOP_HOME path
exportHADOOP_HOME = / KKB/install/hadoop - server - cdh5.14.2 /Configure the HIVE_CONF_DIR path
exportHIVE_CONF_DIR = / KKB/install/hive - 1.1.0 - cdh5.14.2 / confCopy the code

Modify == configuration file hive-site. XML ==

Conf does not have this file by default. Vim can create this file

[hadoop@node03 conf]$ vim hive-site.xml
Copy the code

The file contents are as follows


      
<configuration>
        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://node03:3306/hive? createDatabaseIfNotExist=true&amp;characterEncoding=latin1&amp;useSSL=false</value>
        </property>

        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <value>com.mysql.jdbc.Driver</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <value>root</value>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <value>123456</value>
        </property>
        <property>
                <name>hive.cli.print.current.db</name>
                <value>true</value>
        </property>
        <property>
                <name>hive.cli.print.header</name>
            <value>true</value>
        </property>
    	<property>
                <name>hive.server2.thrift.bind.host</name>
                <value>node03</value>
        </property>
</configuration>
Copy the code

Modify == log configuration file hive-log4j.properties==

Create a hive log storage directory

[hadoop@node03 conf]$mkdir -p/KKB /install/hive-1.1.0-cdh5.14.2/logs/Copy the code

Rename the generated hive-log4j.properties file

[hadoop@node03 conf]$ pwdKKB/install/hive - 1.1.0 - cdh5.14.2 / conf/hadoop @ node03 conf] $mv hive - log4j. The properties, the template hive - log4j. Properties [hadoop@node03 conf]$ vim hive-log4j.properties# modify file
Copy the code

Modify the value of the hive.log.dir property of this file

# Change the following content, set our hive log file store path, easy troubleshootingHive. Log. Dir = / KKB/install/hive - 1.1.0 - cdh5.14.2 / logs /Copy the code

1.5 Copying the mysql Driver Package

Upload the mysql driver package, such as mysql-connector-java-5.1.38.jar, to the/KKB /soft directory
If ==, you need to upload the mysql driver package to the Hive lib directory because metadata is read and written to the mysql database during Hive running

[hadoop@node03 ~]$ cd/ KKB /soft/ [hadoop@node03 soft]$cp mysql-connector-java-5.1.38.jar/KKB /install/hive-1.1.0-cdh5.14.2/lib/Copy the code

1.6 Configuring Hive Environment Variables

Switch to user root

[hadoop@node03 soft]$ su root
Password:
Copy the code

Open the/etc/profilefile

[root@node03 soft]# vim /etc/profile
Copy the code

At the end of the article

exportHIVE_HOME = / KKB/install/hive - 1.1.0 - cdh5.14.2export PATH=$PATH:$HIVE_HOME/bin
Copy the code

Switch back to the Hadoop user and source

[root@node03 soft]# su hadoop
[hadoop@node03 soft]$ source /etc/profile
Copy the code

1.7 Verifying the Installation

== The Hadoop cluster is started ==
== The mysql service is started ==
Start the Hive CLI client in any directory on Node03

[hadoop@node03 ~]$ hive
Copy the code

See what databases are available

show databases;
Copy the code

The Hive installation is successful
Exit the cli

quit;
Copy the code

conclusion

Now that hive is installed, you can enjoy learning about Hive. For more dry goods, please pay attention to my personal public account, pay attention to receive benefits

mo4tech.com (Moment For Technology) is a global community with thousands techies from across the global hang out!Passionate technologists, be it gadget freaks, tech enthusiasts, coders, technopreneurs, or CIOs, you would find them all here.

Hive Installation and Deployment

preface

1. Install mysql as user root

2. Installation of mysql

2.1 Download and install the official mysql YUM source

Install mysql 2.2

3. Set the mysql

3.1 the mysql service

3.2 Changing a Password

2. Install and deploy Hive

1.1 Prerequisites

1.2 Preparing installation packages

1.3 unzip

1.4 Modifying the Configuration File

1.5 Copying the mysql Driver Package

1.6 Configuring Hive Environment Variables

1.7 Verifying the Installation

conclusion

Hive Installation and Deployment

preface

1. Install mysql as user root

2. Installation of mysql

2.1 Download and install the official mysql YUM source

Install mysql 2.2

3. Set the mysql

3.1 the mysql service

3.2 Changing a Password

2. Install and deploy Hive

1.1 Prerequisites

1.2 Preparing installation packages

1.3 unzip

1.4 Modifying the Configuration File

1.5 Copying the mysql Driver Package

1.6 Configuring Hive Environment Variables

1.7 Verifying the Installation

conclusion

Related Posts

Use Python to generate your own special QR codes

(1) Pointer constants, constant Pointers

Distributed transaction: from theory to practice (3) | 🏆 fifth edition essay technology projects