Cloudera Data Science Workbench on HDP has the following platform requirements.
Cloudera Data Science Workbench 1.8.0 (and higher)
Hortonworks Data Platform 2.6.5, 3.1.0, 3.1.5
- Apache Spark 2 - Use the version of Spark that ships with the version of HDP you are running. Refer the HDP component version lists for details: HDP 3.1.0, HDP 3.1.5.
- Spark 1.x is not supported.
- Apache Hive - Dataframe-based tables (such as tables from PySpark, sparklyr, or Scala) require Hive to be installed on your CDH cluster because of a dependency on the Hive Metastore.