Managing Spark

Apache Spark is a general framework for distributed computing that offers high performance for both batch and interactive processing.

To run applications distributed across a cluster, Spark requires a cluster manager. In CDH 6, Cloudera supports only the YARN cluster manager. When run on YARN, Spark application processes are managed by the YARN ResourceManager and NodeManager roles. Spark Standalone is no longer supported.

In CDH 6, Cloudera only supports running Spark applications on a YARN cluster manager. The Spark Standalone cluster manager is not supported.

Related Information

Spark Guide
Monitoring Spark Applications
Tuning Apache Spark Applications
Spark Authentication
Cloudera Spark forum
Apache Spark documentation

This section describes how to manage Spark services.

Continue reading:

Managing Spark Using Cloudera Manager
Managing the Spark History Server

Categories: Administrators | Spark | All Categories

Managing Solr

Managing Spark Using Cloudera Manager