This few days began to do new and old cluster migration, today began to hive all data migration, mainly table meta information and table data. The meta information of the table is stored in mysql, which is not on the same server as hive, so we don’t have to worry about it. However, for table data, we will migrate the HDFS directory to the past. We used the fast distributed copy of Hadoop. During the process, we will encounter some problems about how to avoid going outside the Internet. When migrating hive today, I transferred hive from the old cluster to SCP. Because the user did not switch over, the following problems occurred:
Logging initialized using configuration in file:/usr/local/hive-0.13.1/conf/hive-log4j.properties
Exception in thread “main” java.lang.RuntimeException: java.io.IOException: Permission denied
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:330)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Permission denied
at java.io.UnixFileSystem.createFileExclusively(Native Method)
at java.io.File.createNewFile(File.java:1006)
at java.io.File.createTempFile(File.java:1989)
at org.apache.hadoop.hive.ql.session.SessionState.createTempFile(SessionState.java:432)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:328)
… 7 more
The problem was solved after the corresponding file permissions were given to the user executing the file. A lesson here is that we must pay attention to the permissions when doing the migration, otherwise the trouble sometimes occurs.
Properties does not have the read permission. 2. Create temporary files that require the write permission of /usr/local/hive-0.13.1/tmp
Properties file. 2. Change the read and write permissions of /usr/local/hive-0.13.1/tmp