Configure Phoenix-Spark connector using Cloudera Manager
When using the Phoenix-Spark connector, you need the Spark connector JAR file. You can find this JAR file in the following location: /opt/cloudera/parcels/CDH/lib/phoenix_connectors
- Before you can use Phoenix-Spark connector for your Spark applications, you must
configure your Maven settings to have a repository that points to the repository
at https://repository.cloudera.com/artifactory/public/org/apache/phoenix/phoenix5-spark/ and use the dependency:
<dependency> <groupId>org.apache.phoenix</groupId> <artifactId>phoenix5-spark</artifactId> <version>[***VERSION EXAMPLE: 6.0.0.7.1.6.0-297***]</version> <scope>provided</scope> </dependency>
- Enable your IDE by adding the following dependency to your
build:
<dependency> <groupId>org.apache.phoenix</groupId> <artifactId>phoenix5-spark</artifactId> <version>[***VERSION EXAMPLE: 6.0.0.7.1.6.0-297***]</version> <scope>provided</scope> </dependency>
- Build a Spark application using the Phoenix-Spark connector with the dependencies that are present in the connector.
- Build a Spark application using the dependencies that you provide when you run
your application. Use the
--jars
/opt/cloudera/parcels/CDH/lib/phoenix_connectors/phoenix5-spark-[***VERSION***]-shaded.jar parameter when running thespark-submit
command.