Home Forums Sqoop HDP 1.2.0 Issue

This topic contains 2 replies, has 2 voices, and was last updated by  Mahesh Balakrishnan 9 months, 3 weeks ago.

  • Creator
    Topic
  • #46215

    sonali
    Participant

    Hi,
    I am using HDP 1.2.0 to save data coming from flume.
    Data is getting saved for some hours in hdfs and after some time getting error like “””””input-data/FlumeData.1388227106936.tmp could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1637)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:757)
    at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)””””””

    Then I checked my DATANODE LOG and i found its stopped and trying to restart datanode and time of restarting Incompatible namespaceIDs is the error. So my 1st doubt is “datanode is getting restarted by itself within some period or what”?

    I am using two virtual box with centos 6.3 and each box has 7 GB RAM & one box is for flume and namenode and other is only for datanode.

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #49326

    Hi Sonali,
    The error you are getting is related to HDFS and not necessarily flume. It seems you cannot put any data into HDFS. I would suggest to try and just use hadoop fs commands to put files into the hdfs where flume is running from as a test. Lastly, it would be best to move this topic into the HDFS forums.

    Regards,
    Robert

    Collapse
    #46221

    sonali
    Participant

    NAMENODE LOG:

    Number of transactions: 3755 Total time for transactions(ms): 174Number of transactions batched in Syncs: 7 Number of syncs: 2184 SyncTimes(ms): 1296
    2013-12-29 23:12:51,328 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* allocateBlock: /user/sonali/input-data/activity-analysis/2013/12/29/17/42/FlumeData.1388338930935.tmp. blk_2850247969211670571_1547
    2013-12-29 23:12:51,357 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* fsync: /user/sonali/input-data/activity-analysis/2013/12/29/17/42/FlumeData.1388338930935.tmp for DFSClient_NONMAPREDUCE_466199001_29
    2013-12-29 23:13:16,555 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Not able to place enough replicas, still in need of 1
    2013-12-29 23:13:16,555 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sonali cause:java.io.IOException: File /user/sonali/input-data/activity-analysis/2013/12/29/17/43/FlumeData.1388338996369.tmp could only be replicated to 0 nodes, instead of 1
    2013-12-29 23:13:16,555 INFO org.apache.hadoop.ipc.Server: IPC Server handler 12 on 8019, call addBlock(/user/sonali/input-data/activity-analysis/2013/12/29/17/43/FlumeData.1388338996369.tmp, DFSClient_NONMAPREDUCE_466199001_29, null) from 10.1.1.119:36504: error: java.io.IOException: File /user/sonali/input-data/activity-analysis/2013/12/29/17/43/FlumeData.1388338996369.tmp could only be replicated to 0 nodes, instead of 1
    java.io.IOException: File /user/sonali/input-data/activity-analysis/2013/12/29/17/43/FlumeData.1388338996369.tmp could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1637)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:757)
    at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1393)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1389)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)