Create LZO files

You can output LZO files directly from a Hive query.

Create LZO files as the output of the Hive query.

Use lzo command utility or your custom Java to generate lzo.index for the .lzo files.

Prefix the query string with these parameters:

Table 1. Hive LZO Query String Parameters and Values
Parameter	Value
SET mapreduce.output.fileoutputformat.compress.codec	com.hadoop.compression.lzo.LzoCodec
SET hive.exec.compress.output	true
SET mapreduce.output.fileoutputformat.compress	true

hive -e "SET
                  mapreduce.output.fileoutputformat.compress.codec=com.hadoop.compression.lzo.LzoCodec;SET
                  hive.exec.compress.output=true;SET
                  mapreduce.output.fileoutputformat.compress=true;"