The introduction

Hello everyone, I am ChinaManor, which literally translates to Chinese code farmer. I hope I can become a pathfinder on the road of national rejuvenation, a ploughman in the field of big data, an ordinary but not mediocre person. I have been studying big data for almost a year. Recently, I have been organizing my notes on big data learning. This series is about some must-know and must-know knowledge of big data.

How to customize UDF in Hive

UDF Development Example Simple UDF example

Step 1: Create a Maven Java project and import the JAR package

<repositories>
    <repository>
        <id>cloudera</id>
 <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
        <version>2.6. 0-cdh514.. 0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.hive</groupId>
        <artifactId>hive-exec</artifactId>
        <version>1.1. 0-cdh514.. 0</version>
    </dependency>
</dependencies>
<build>
<plugins>
    <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.0</version>
        <configuration>
            <source>1.8</source>
            <target>1.8</target>
            <encoding>UTF-8</encoding>
        </configuration>
    </plugin>
     <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-shade-plugin</artifactId>
         <version>2.2</version>
         <executions>
             <execution>
                 <phase>package</phase>
                 <goals>
                     <goal>shade</goal>
                 </goals>
                 <configuration>
                     <filters>
                         <filter>
                             <artifact>*:*</artifact>
                             <excludes>
                                 <exclude>META-INF/*.SF 
      
       META-INF/*.DSA
       
      
       META-INF/*/
      RSA</exclude>
                             </excludes>
                         </filter>
                     </filters>
                 </configuration>
             </execution>
         </executions>
     </plugin>
</plugins>
</build>


Copy the code

Step 2: Develop Java classes that inherit UDF and override the Evaluate method

public class ItcastUDF extends UDF { public Text evaluate(final Text s) { if (null == s) { return null; } // Return uppercase Text new Text(s.tostring ().toupperCase ()); }}Copy the code

Step 3: Package our project and upload it to the Hive lib directory

Step 4: Add our JAR package and rename our JAR package name

cd /export/servers/hive-1.1. 0-cdh514.. 0/lib
mv original-day_06_hive_udf-1.0-SNAPSHOT.jar udf.jar
Copy the code

Hive client adds our JAR package

add jar /export/servers/hive-1.1. 0-cdh514.. 0/lib/udf.jar;
Copy the code

Step 5: Set the function to associate with our custom function to create temporary functions

create temporary function tolowercase as 'cn.itcast.udf.ItcastUDF';
Copy the code

Delete temporary functions

drop temporary function tolowercase
Copy the code

Creating a permanent function

create function tolowercase1 as 'cn.itcast.udf.ItcastUDF';
Copy the code

Delete permanent functions

drop   function tolowercase1;
Copy the code

Step 6: Use custom functions

select tolowercase('abc');
Copy the code

conclusion

The above is the project selection of some reference answers, I wish you read after their own harvest, if there is a harvest might as well pay attention to ~ we next see 👋 the above article to help you? If there is, might as well a key three connect ~~