HCatalog
 
PDF -icon
PDF

POST mapreduce/jar

Description

Creates and queues a standard Hadoop MapReduce job.

URL

http://www.myserver.com/templeton/v1/mapreduce/jar

Parameters

NameDescriptionRequired?Default
jar Name of the jar file for Map Reduce to use. Required None
class Name of the class for Map Reduce to use. Required None
libjars Comma separated jar files to include in the classpath. Optional None
files Comma separated files to be copied to the map reduce cluster Optional None
arg Set a program argument. Optional None
define Set an Hadoop configuration variable using the syntax define=NAME=VALUE Optional None
statusdir A directory where Templeton will write the status of the Map Reduce job. If provided, it is the caller's responsibility to remove this directory when done. Optional None
callback Define a URL to be called upon job completion. You may embed a specific job ID into this URL using $jobId. This tag will be replaced in the callback URL with this job's job ID. Optional None

Results

NameDescription
id A string containing the job ID similar to "job_201110132141_0001".
info A JSON object containing the information returned when the job was queued. See the Hadoop documentation (Class TaskController) for more information.

Example

Code and Data Setup 

% hadoop fs -put wordcount.jar .
% hadoop fs -put transform.jar .

% hadoop fs -ls .
Found 2 items
-rw-r--r--   1 ctdean supergroup         23 2011-11-11 13:29 /user/ctdean/wordcount.jar
-rw-r--r--   1 ctdean supergroup         28 2011-11-11 13:29 /user/ctdean/transform.jar

Curl Command 

% curl -s -d user.name=ctdean \
       -d jar=wordcount.jar \
       -d class=org.myorg.WordCount \
       -d libjars=transform.jar \
       -d arg=wordcount/input \
       -d arg=wordcount/output \
       'http://localhost:50111/templeton/v1/mapreduce/jar'

JSON Output 

{
 "id": "job_201111121211_0001",
 "info": {
          "stdout": "templeton-job-id:job_201111121211_0001
                    ",
          "stderr": "",
          "exitcode": 0
         }
}