HDP on Linux – Installation Forum

Native snappy library not available

  • #28052

    I am using HDP 1.3, installed snappy library using command “yum install snappy snappy-devel”
    Also I am using jdk 1.6_45 (32bit), so link “ln -sf /usr/lib64/libsnappy.so /usr/lib/hadoop/lib/native/Linux-i386-32/” was created.

    Folder /usr/lib/hadoop/lib/native/Linux-i386-32 contains:
    lrwxrwxrwx 1 root root 23 Jun 23 17:24 libsnappy.so -> /usr/lib64/libsnappy.so

    When I am running example: “/usr/lib/hadoop/bin/hadoop jar /usr/lib/hadoop/hadoop-examples.jar terasort /test/10gsort/input /test/10gsort/output”

    I get exception:
    13/06/23 17:43:30 INFO mapred.JobClient: Task Id : attempt_201306231742_0001_m_000000_0, Status : FAILED
    java.lang.RuntimeException: native snappy library not available
    at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
    at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
    at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
    at org.apache.hadoop.mapred.IFile$Writer.(IFile.java:102)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1411)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1302)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:365)
    at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Unknown Source)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.mapred.Child.main(Child.java:249)

    Please assist.

to create new topics or reply. | New User Registration

  • Author
  • #28101

    After jdk 64 bit installation test application working fine.

    Sasha J

    64-bit JDK is prerequisite.

    Thank you!

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.