Home Forums Hortonworks Sandbox Issue connecting Talend studio with Sandbox

This topic contains 9 replies, has 7 voices, and was last updated by  janhess 3 months ago.

  • Creator
    Topic
  • #29739

    Hi Team,
    I have tried connecting Talend with Sandbox based on the documentation provided in the HortonWorks. I have just tried using the tHDFSPut on a sample job created . Once i execute the job as per the documentation it is expected to create a file under “user/hue” directory . When i executed the job it created a file in the user/hue directory with 0 byte size. The talend execution console shows the following exception. Please help.
    Exception
    ======================
    Starting job SampleJob2 at 23:19 20/07/2013.
    [statistics] connecting to socket on port 3635
    [statistics] connected
    13/07/20 23:20:27 INFO hdfs.DFSClient: Exception in createBlockOutputStream 10.0.2.15:50010 java.net.ConnectException: Connection timed out: no further information
    13/07/20 23:20:27 INFO hdfs.DFSClient: Abandoning block blk_-3874851192982451386_1670
    13/07/20 23:20:27 INFO hdfs.DFSClient: Excluding datanode 10.0.2.15:50010
    13/07/20 23:20:27 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hue/data.txt could only be replicated to 0 nodes, instead of 1
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:785)
    at sun.reflect.GeneratedMethodAccessor21.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1444)
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1440)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1438)
    at org.apache.hadoop.ipc.Client.call(Client.java:1066)
    at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
    at $Proxy1.addBlock(Unknown Source)

Viewing 9 replies - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #47030

    janhess
    Participant

    Same problem. Does anyone have a solution to this?

    Collapse
    #41685

    I have the same issue, Talend is creating a 0 byte file in the hdfs

    Collapse
    #38439

    I tried using sandbox HDP 1.3 with talend studio to write a file into hadoop . I was using hortworks data platform 1.0 library from the drop down in the talend studio connection as I was not able to see another library version of horton works. For HDP 1.3 . When it tried executing the file was generated inside hadoop folder but the file was empty and. An exception was thrown in talend studio log . As per the tutorial the file was suppose to generate with some rows entered .

    Does anyone successfully excecuted this scenario ? I used hdfs:// 127.0.0.1:50010 / to connect to hadoop. I
    is thre any library available for talend studio to execute against HDP 1.3 .?

    regards
    Arun

    Collapse
    #36605

    Kai Waehner
    Member

    Btw: Now, I am using Virtual Box image. Before, I used a VMware image which I converted to Parallels…

    Collapse
    #36604

    Kai Waehner
    Member

    Same problem here with Sandbox 1.3. Disc is not full. Hortonworks tutorials work perfectly.

    I had it working with Talend and Sandbox 1.2. Tried with same configuration here. I only changed the IP addresses (was 10.37.129.3 in 1.2, now it is 127.0.0.1 in 1.3)

    Probably, this is a configuration issue ?!

    Collapse
    #33683

    Robert
    Participant

    Hi guys,
    I am just wondering if from your machines you can telnet to 10.0.2.15:50010. I assume from the hue interface all is working with uploading files to hdfs. Can you guys verify the port is accessible from your client machine?

    Regards,
    Robert

    Collapse
    #33037

    Rupesh Dandekar
    Participant

    I’m getting the same error. HDFS is not full and working fine with tutorials.

    Collapse
    #29838

    Hi Sasha,

    Thanks for your analysis and response.
    I have verified the disk and its not full. All the input files used for tutorials where cleared from user directory before attempting to connect Sanbox with talend studio . Also the Sanbox was restarted multiple times.
    Could you please let me know how to verify or confirming that the data node in a sandbox environment is running or not. Only during the launch and booting of sandbox environment I could find that servers and nodes are started and running. Once it’s started I am not able to find any options in any dashboards to verify the status of datanode.
    I tried with the basic tutorials provided along with the sandbox which are working fine. So I think the datanodes are functioning properly. When I tried connecting the sandbox with talend studio I was getting the exception in talend log console as mentioned in my earlier posting. Please let me know any additional information I could share from my end that could be helpfull for resolving this issue.

    Awaiting your response . Thank you.
    Regards
    Arun

    Collapse
    #29774

    Sasha J
    Moderator

    THis error means that you have problems with HDFS.
    Most likely DataNode process is not running or disk is full.
    HDFS can not create file.
    Restart your Sandbox and make sure you did not fill up it’s disk.

    Thank you!
    Sasha

    Collapse
Viewing 9 replies - 1 through 9 (of 9 total)