Home Forums HDFS MapReduce Debugging

This topic contains 4 replies, has 4 voices, and was last updated by  tedr 8 months, 3 weeks ago.

  • Creator
    Topic
  • #29758

    I am new to hadoop. I need to debug sample mapreduce code using eclipse.
    For this I have edited hadoop-env.sh with the follwing line
    HADOOP_OPTS=”-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=50002″

    and in my eclipse Remote Debug configuration wizard,I was given
    host:10.0.2.23
    port:50002

    But while running my code in debug mode, I am getting the following error

    Listening for transport dt_socket at address: 50002
    13/07/22 23:18:13 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 time(s).
    13/07/22 23:18:14 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 time(s).
    13/07/22 23:18:15 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 2 time(s).
    13/07/22 23:18:16 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 3 time(s).
    13/07/22 23:18:17 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 4 time(s).
    13/07/22 23:18:18 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 5 time(s).
    13/07/22 23:18:19 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 6 time(s).
    13/07/22 23:18:20 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 7 time(s).
    13/07/22 23:18:21 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 8 time(s).
    13/07/22 23:18:22 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 9 time(s).
    Exception in thread “main” java.lang.RuntimeException: java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: Connection refused
    at org.apache.hadoop.mapred.JobConf.getWorkingDirectory(JobConf.java:546)
    at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:318)
    at WordCount1.main(WordCount1.java:66)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
    Caused by: java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: Connection refused
    at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
    at org.apache.hadoop.ipc.Client.call(Client.java:1075)

Viewing 4 replies - 1 through 4 (of 4 total)

The topic ‘MapReduce Debugging’ is closed to new replies.

  • Author
    Replies
  • #29864

    tedr
    Moderator

    Hi Mullangi,

    Thanks for letting us know that you can now debug your code.

    Thanks,
    Ted.

    Collapse
    #29832

    Thanks to all for responding to me.
    I have solved the issue by updating $HADOOP_HOME/conf/core-site.xml with the following entries

    fs.default.name
    file:///

    and $HADOOP_HOME/conf/mapred-site.xml:

    mapred.job.tracker
    local

    Now I am able to debug the code.

    Collapse
    #29802

    Akki Sharma
    Moderator

    Hello Mullangi,

    I presume you have a single node installation and also your Eclipse remote debugger is running fine.

    Where are you running your NameNode? It seems that the MapReduce process is not able to communicate to the NameNode.

    Best Regards,
    Akki

    Collapse
    #29800

    abdelrahman
    Moderator

    Hi Mullangi,

    How is your day so far? Have you tried to ssh forward the port to your local host port 9000? What is the documentation link which you are following?

    Thanks
    -Abdelrahman

    Collapse
Viewing 4 replies - 1 through 4 (of 4 total)