Home Forums HBase MapReduce job over HBase fails

This topic contains 7 replies, has 5 voices, and was last updated by  tedr 1 year, 6 months ago.

  • Creator
    Topic
  • #15814

    Hi,

    I am running MapReduce job over HBase-0.94.3 and Hadoop-1.0.3 cluster of 3 machines but it gives me error as Null pointer exception. Caused by ReverseDNS.

    When I tried to run MapReduce job over Hadoop it gets executed perfectly.

    How to resolve it?

    Thanks in advance.

Viewing 7 replies - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #17245

    tedr
    Member

    Hi Bushan,

    Actually both files are used, DNS is usually controlled by a server on a different box than yours, and the /etc/hosts file is, of course the file on your machine. Entries in the /etc/hosts file will take precedence over those on the DNS server, though this can be changed. So if you make your entry there it should work.

    Thanks,
    Ted.

    Collapse
    #16162

    Thanks Larry.

    How to use only /etc/hosts instaed of DND? Is their any MapReduce programs which says about using only one of them?

    Collapse
    #16071

    Larry Liu
    Moderator

    Do you use DNS or /etc/hosts?
    If you use dns server, please set up reverse DNS for each node.

    Here is a guide how to set up DNS record in BIND. I assume you use BIND on linux

    http://www.philchen.com/2007/04/04/configuring-reverse-dns

    Thanks

    Larry

    Collapse
    #16061

    Is it isue of version of HBase?

    If its issue of forward and ReverseDNS, how to set it?

    Collapse
    #16060

    I am using Hadoop-1.0.3 and HBase-0.94.3. I am able to run MapReduce job over Hadoop-1.0.3. But I got following error while running MapReduce job over HBase-0.94.3.
    Exception in thread “main” java.lang.NullPointerException
    at org.apache.hadoop.net.DNS.reverseDns(DNS.java:72)
    at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.reverseDNS(TableInputFormatBase.java:218)
    at org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.getSplits(TableInputFormatBase.java:183)
    at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
    at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
    at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
    at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:416)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
    at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
    at org.apache.hadoop.hbase.mapreduce.CopyTable.main(CopyTable.java:237)

    Is MapReduce job runs over HBase-0.94.3 and Hadoop-1.0.3?

    ( Ref : HBASE-7693 )

    Collapse
    #16049

    Yi Zhang
    Moderator

    Hi Bhusahn,

    Do you have reverse DNS set up for HBase? In earlier versions of HBase, both for forward and reverse DNS resolving must work. I am not sure about the version 0.94.3.

    Happy Hadoop,
    Yi

    Collapse
    #16019

    tedr
    Moderator

    Hi Bhusahn,

    Could you post the relevant parts of the logs where the error shows?

    Thanks,
    Ted.

    Collapse
Viewing 7 replies - 1 through 7 (of 7 total)