Developing Spark Applications

When you are ready to move beyond running core Spark applications in an interactive shell, you need best practices for building, packaging, and configuring applications and using the more advanced APIs. This section describes:

• How to develop, package, and run Spark applications.
• Aspects of using Spark APIs beyond core Spark.
• How to access data stored in various file formats, such as Parquet and Avro.
• How to access data stored in cloud storage systems, such as Amazon S3.
• Best practices in building and configuring Spark applications.