Set up the development environment
You can create a Hive UDF in a development environment using IntelliJ, for example, and build the UDF. You define the Cloudera Maven Repository in your POM, which accesses necessary JARS hadoop-common-<version>.jar and hive-exec-<version>.jar that your define as dependencies.
- Open IntelliJ and create a new Maven-based project. Click Create New Project. Select Maven and the supported Java version as the Project SDK. Click Next.
-
Add archetype information.
For example:
- GroupId: com.mycompany.hiveudf
- ArtifactId: hiveudf
-
Click Next and Finish.
The generated pom.xml appears in
sample-hiveudf
. -
To the
pom.xml
, add properties to facilitate versioning.For example:<properties> <hadoop.version>TBD</hadoop.version> <hive.version>TBD</hive.version> </properties>
-
In the pom.xml, define the repositories.
Use internal repositories if you do not have internet access.
<repositories> <repository> <releases> <enabled>true</enabled> <updatePolicy>always</updatePolicy> <checksumPolicy>warn</checksumPolicy> </releases> <snapshots> <enabled>false</enabled> <updatePolicy>never</updatePolicy> <checksumPolicy>fail</checksumPolicy> </snapshots> <id>HDPReleases</id> <name>HDP Releases</name> <url>http://repo.hortonworks.com/content/repositories/releases/</url> <layout>default</layout> </repository> <repository> <id>public.repo.hortonworks.com</id> <name>Public Hortonworks Maven Repo</name> <url>http://repo.hortonworks.com/content/groups/public/</url> <snapshots> <enabled>false</enabled> </snapshots> </repository> <repository> <id>repository.cloudera.com</id> <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url> </repository> </repositories>
-
Define dependencies.
For example:
<dependencies> <dependency> <groupId>org.apache.hive</groupId> <artifactId>hive-exec</artifactId> <version>${hive.version}</version> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>${hadoop.version}</version> </dependency> </dependencies>