Chapter 8. Developing Spark Applications
Apache Spark enables you to quickly develop applications and process jobs. It is designed for fast application development and processing. Spark Core is the underlying execution engine; other services, such as Spark SQL, MLlib, and Spark Streaming, are built on top of the Spark Core.
Depending on your use case, you can extend your use of Spark into several domains, including the following described in this chapter:
Spark DataFrames
Spark SQL
Calling Hive user-defined functions from Spark SQL
Spark Streaming
Accessing HBase tables, HDFS files, and ORC data (Hive)
Using custom libraries
Additional resources:
To get started with Spark, see the Apache Spark Quick Start and the Spark 1.6.3 and Spark 2.0 overviews.
For more information about application development, see the Apache Spark Programming Guide.
For more information about using Livy to submit Spark jobs, see Submitting Spark Applications Through Livy.