Introduction to environment:





1 Sqoop1.99.6 installation

1.1 Sqoop1.99.6 installation

First of all, from the website (…). Download sqoop-1.99.6-bin-hadoop200.tar.gz and decompress it in /usr/local/sqoop.

Add the following configuration to /etc/profile

export SQOOP_HOME=/usr/local/sqoopexport CATALINA_HOME=
S Q O O P H O M E / s e r v e r e x p o r t L O G D I R = SQOOP_HOME/serverexport LOGDIR=


Enter/usr/local/sqoop/server/conf/directory, modify sqoop. Properties (hadoop installation directory) org. Apache. Sqoop.. Log4j appenders. File. The file = / usr/local/sqoop/logs/sqoop log# org. Apache. Sqoop.. Log4j appenders. Fi le.File=@LOGDIR@/sqoop.log#org.apache.sqoop.auditlogger.default.file=@LOGDIR@/default.auditorg.apache.sqoop.auditlogger. default.file=/usr/local/sqoop/logs/default.audit

Modify the catalina. The properties

common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina .home}/.. /lib/*.jar,/usr/local/hadoop/share/hadoop/common/*.jar,/usr/local/hadoop/share/hadoop/common/lib/*.jar,/usr/local/hadoop /share/hadoop/hdfs/*.jar,/usr/local/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/hadoop/share/hadoop/mapreduce/*.jar,/u sr/local/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/hadoop/share/hadoop/tools/*.jar,/usr/local/hadoop/share/hado op/tools/lib/*.jar,/usr/local/hadoop/share/hadoop/yarn/*.jar,/usr/local/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/ha doop/share/hadoop/httpfs/tomcat/lib/*.jarCopy the code

Copy the mysql connector – Java – 5.1.39 – bin. The jar to/usr/local/sqoop/server/lib directory. Create a new lib directory under /usr/local/sqoop/.

1.2 Sqoop1.99.6 start

Go to the /usr/local/sqoop/bin directory and run the server start

Run the following command to enter the command line interface (CLI) : client

Setting the Server

Set server – host localhost – port 12000 – webapp sqoop

Viewing version Information

show connector

Establish connections to MySQL and Hadoop respectively.

Viewing Connection Information


Creating job Tasks


See the…

The job is successfully started if the following information is displayed:

1.3 mysql data table

In the exported MySQL table, the user table is located under databases:test.



1.4 Viewing information on Hadoop

Warning: Make sure your Hadoop installation is correct!!

The following two pages appear.

In /sqoop there are the following

These two files are generated TXT files, open to view

It is found that the data is exactly the same as the data in MySQL.

Note: If the following problems occur during the import process, please check the sqoop.log file under /usr/local/sqoop/log/file, according to the corresponding error to solve the problem.

Exception has occurred during processing command 
Exception:org.apache.sqoop.common.SqoopExceptionMessage: CLIENT_0001:Server has returned exception
Copy the code