Livy API reference for batch jobs
GET /batches
returns all active batch sessions.
Request Parameters | Description | Type |
---|---|---|
from | Starting index used to fetch sessions | int |
size | Number of sessions to fetch | int |
Response | Description | Type |
---|---|---|
from | Starting index of fetched sessions | int |
total | Number of sessions fetched | int |
sessions | List of active batch sessions | list |
GET /batches/{batchId}
returns the batch session
information as a batch
object.
GET /batches/{batchId}/state
returns the state of batch
session:
Response | Description | Type |
---|---|---|
id | A non-negative integer that represents a specific batch session | int |
state | The current state of batch session | string |
GET /batches/{batchId}/log
retrieves log records for
the specified batch session.
Request Parameters | Description | Type |
---|---|---|
from | Offset into log file | int |
size | Max number of log lines to return | int |
Response | Description | Type |
---|---|---|
id | A non-negative integer that represents a specific batch session | int |
from | Offset from start of the log file | int |
size | Number of log records returned | int |
log | Log records | list of strings |
POST /batches
creates a new batch environment and runs
a specified application:
Request Body | Description | Type |
---|---|---|
file | File containing the application to run (required) | path |
proxyUser | User ID to impersonate when running the job | string |
className | Application Java or Spark main class | string |
args | Command line arguments for the application | list of strings |
jars | Jar files to be used in this session | list of strings |
pyFiles | Python files to be used in this session | list of strings |
files | Other files to be used in this session | list of strings |
driverMemory | Amount of memory to use for the driver process | string |
driverCores | Number of cores to use for the driver process | int |
executorMemory | Amount of memory to use for each executor process | string |
executorCores | Number of cores to use for each executor | int |
numExecutors | Number of executors to launch for this session | int |
archives | Archives to be used in this session | list of strings |
queue | The name of the YARN queue to which the job should be submitted | string |
name | Name of this session | string |
conf | Spark configuration properties | Map of key=val |
Response | Description | Type |
---|---|---|
batch object (for more information see "Livy Batch Object") | The created batch object |
batch object |
DELETE /batches/{batchId}
terminates the Batch job.