Livy to CDP

How to migrate Apache Livy data to CDP.

CDP Livy Versions

  • Livy has been upgraded to version 0.7 to in CDP Private Cloud Base 7.1.x.
  • CDP Public Cloud includes a DataHub Data Engineering template that contains with Livy and Spark.

Features

  • JDBC/Thrift Server support has been added to Livy. This is an improvement over Spark Thrift Server, which is neither secure nor fault-tolerant.
  • The Thrift server is disabled by default in Livy, but you can use Cloudera Manager to enable the Thrift Server.

Usage

  • Use the HWC .jar and configuration when accessing managed tables.
  • Writes to managed tables via the Livy Thrift Server are not currently supported.