Livy API Reference for Batch Jobs
GET /batches returns all active batch sessions.
| Request Parameters | Description | Type |
|---|---|---|
| from | Starting index used to fetch sessions | int |
| size | Number of sessions to fetch | int |
| Response | Description | Type |
|---|---|---|
| from | Starting index of fetched sessions | int |
| total | Number of sessions fetched | int |
| sessions | List of active batch sessions | list |
GET /batches/{batchId} returns the batch session information as a
batch object.
GET /batches/{batchId}/state returns the state of batch session:
| Response | Description | Type |
|---|---|---|
| id | A non-negative integer that represents a specific batch session | int |
| state | The current state of batch session | string |
GET /batches/{batchId}/log retrieves log records for the specified batch
session.
| Request Parameters | Description | Type |
|---|---|---|
| from | Offset into log file | int |
| size | Max number of log lines to return | int |
| Response | Description | Type |
|---|---|---|
| id | A non-negative integer that represents a specific batch session | int |
| from | Offset from start of the log file | int |
| size | Number of log records returned | int |
| log | Log records | list of strings |
POST /batches creates a new batch environment and runs a specified
application:
| Request Body | Description | Type |
|---|---|---|
| file | File containing the application to run (required) | path |
| proxyUser | User ID to impersonate when running the job | string |
| className | Application Java or Spark main class | string |
| args | Command line arguments for the application | list of strings |
| jars | Jar files to be used in this session | list of strings |
| pyFiles | Python files to be used in this session | list of strings |
| files | Other files to be used in this session | list of strings |
| driverMemory | Amount of memory to use for the driver process | string |
| driverCores | Number of cores to use for the driver process | int |
| executorMemory | Amount of memory to use for each executor process | string |
| executorCores | Number of cores to use for each executor | int |
| numExecutors | Number of executors to launch for this session | int |
| archives | Archives to be used in this session | list of strings |
| queue | The name of the YARN queue to which the job should be submitted | string |
| name | Name of this session | string |
| conf | Spark configuration properties | Map of key=val |
| Response | Description | Type |
|---|---|---|
| batch object (for more information see "Livy Batch Object") | The created batch object | batch object |
DELETE /batches/{batchId} terminates the Batch job.