11. Configuring LZO Compression

LZO compression is a lossless data compression library favoring speed over compression ratio; LZO compression is recommended for temporary tables. You can enable LZO compression for HDP to optimize Hive query speed.

LZO compression is not enabled automatically. To enable it, perform the following steps on each node in your cluster:

Copy the hadoop-lzo.jar file from your installation zip package to %HADOOP_COMMON_HOME%\share\hadoop\common.
Copy gplcompression.dll and lzo2.dllfrom your installation zip package to the same bin folder as hadoop.dll.

Ensure that the following configuration properties are set in core-site.xml:

  <property>
    <name>io.compression.codecs</name>
<value>org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec,com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
  </property>
  <property>
    <name>io.compression.codec.lzo.class</name>
    <value>com.hadoop.compression.lzo.LzoCodec</value>
  </property>

Legal notices