Home Forums HDP on Linux – Installation Native snappy library not available

This topic contains 2 replies, has 2 voices, and was last updated by  Sasha J 1 year, 1 month ago.

  • Creator
    Topic
  • #28052

    dgreenshtein
    Member

    I am using HDP 1.3, installed snappy library using command “yum install snappy snappy-devel”
    Also I am using jdk 1.6_45 (32bit), so link “ln -sf /usr/lib64/libsnappy.so /usr/lib/hadoop/lib/native/Linux-i386-32/” was created.

    Folder /usr/lib/hadoop/lib/native/Linux-i386-32 contains:
    lrwxrwxrwx 1 root root 23 Jun 23 17:24 libsnappy.so -> /usr/lib64/libsnappy.so

    When I am running example: “/usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop/hadoop-examples.jar terasort /test/10gsort/input /test/10gsort/output”

    I get exception:
    13/06/23 17:43:30 INFO mapred.JobClient: Task Id : attempt_201306231742_0001_m_000000_0, Status : FAILED
    java.lang.RuntimeException: native snappy library not available
    at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
    at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
    at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
    at org.apache.hadoop.mapred.IFile$Writer.(IFile.java:102)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1411)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1302)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    Please assist.

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #28185

    Sasha J
    Moderator

    64-bit JDK is prerequisite.

    Thank you!
    Sasha

    Collapse
    #28101

    dgreenshtein
    Member

    After jdk 64 bit installation test application working fine.

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)