Home Forums Hortonworks Sandbox What is Templeton configuration? (connecting via Talend Studio)

Tagged: 

This topic contains 7 replies, has 3 voices, and was last updated by  Larry Liu 10 months, 3 weeks ago.

  • Creator
    Topic
  • #25580

    Kai Waehner
    Member

    HDP seems to work fine. I can run PIG, HIVE, etc. I use Talend Studio.

    I just cannot run HCatalog examples. I think my Templeton configuration is wrong. My host IP is correct (IP of sandbox). My port is 50111. Maybe this is wrong? I cannot access http://“IP-of-HDP”:50111 via webbrowser.

    What is the correct configuration of Templeton? Can I find out this information from HDP vm or within web application?

    Here is my error when running your Talend example job. It already dies when trying to create a HCatalog database:

    [statistics] connecting to socket on port 3386
    [statistics] connected
    2013-05-15 16:09:53.314 java[5945:120b] Unable to load realm info from SCDynamicStore
    Exception in component tHCatalogOperation_1
    2013-05-15 16:10:08|CGgn8K|APRIL_2013_LOCAL_PROJECT|hcatalog_load|java||Failed|Job execution error|Error 500 (Internal Server Error): We received an unexpected result.
    —>Failure calling hcat: create database talend location ‘/user/hdp/test/usersdb’;
    [statistics] disconnected
    java.lang.Exception: Error 500 (Internal Server Error): We received an unexpected result.
    —>Failure calling hcat: create database talend location ‘/user/hdp/test/usersdb’;
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.tHCatalogOperation_1Process(hcatalog_load.java:1696)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.tHDFSDelete_1Process(hcatalog_load.java:1533)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.runJobInTOS(hcatalog_load.java:6402)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.main(hcatalog_load.java:6236)
    job failed
    java.lang.Exception: Error 500 (Internal Server Error): We received an unexpected result.
    —>Failure calling hcat: create database talend location ‘/user/hdp/test/usersdb’;
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.tHCatalogOperation_1Process(hcatalog_load.java:1696)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.tHDFSDelete_1Process(hcatalog_load.java:1533)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.runJobInTOS(hcatalog_load.java:6402)
    at april_2013_local_project.hcatalog_load_0_1.hcatalog_load.main(hcatalog_load.java:6236)

    Thanks for help.
    Kai

Viewing 7 replies - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #26436

    Larry Liu
    Moderator

    Hi, Kai

    The database location is configured in the hive-site.xml.

    hive.metastore.warehouse.dir
    /apps/hive/warehouse

    Larry

    Collapse
    #26411

    Kai Waehner
    Member

    Ok guys. Thanks for your help. Problem solved. The last error was some “strange” status of my environment. I used a new Talend Studio and restarted everything. Now, it works.

    Just one last (newbie) question: Why is database location /apps/hive/warehouse? in HDFS, the data is here: / user/ hdp/ test/ data (as I defined it this way in Talend’s components? Can you please explain me why I nee an additional database location? What is it? Probably, I have to read some getting started articles about Hive, right ?!

    Collapse
    #26331

    Kai Waehner
    Member

    Thanks, “/apps/hive/warehouse” seems to work.

    Though, now the second tHCatalogOperation (for creating the Table) does not work.

    Operation: Table (Drop if exists and create)
    Database: “talend” (which I created before)
    Table: “users”
    Username: “sandbox” (same as before)

    Can you maybe send me your working Talend job via email? I can then adjust it with my configuration. That would be easier to solve the problem…
    Thanks.

    ######

    Here is the error (connection to database not working) ?!

    Starting job hcatalog_load at 08:03 27/05/2013.

    [statistics] connecting to socket on port 3706
    [statistics] connected
    Exception in component connectionStatsLogs
    com.mysql.jdbc.CommunicationsException: Communications link failure due to underlying exception:

    ** BEGIN NESTED EXCEPTION **

    java.net.SocketException
    MESSAGE: java.net.ConnectException: Connection refused

    STACKTRACE:

    java.net.SocketException: java.net.ConnectException: Connection refused
    at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:156)
    at com.mysql.jdbc.MysqlIO.(MysqlIO.java:284)
    at com.mysql.jdbc.Connection.createNewIO(Connection.java:2569)
    at com.mysql.jdbc.Connection.(Connection.java:1485)
    at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
    at java.sql.DriverManager.getConnection(DriverManager.java:579)
    at java.sql.DriverManager.getConnection(DriverManager.java:221)
    at routines.system.SharedDBConnection.getConnection(SharedDBConnection.java:63)
    at routines.system.SharedDBConnection.getDBConnection(SharedDBConnection.java:131)
    at demo_project_5_2_1.hcatalog_load_0_1.hcatalog_load.connectionStatsLogsProcess(hcatalog_load.java:6698)
    2013-05-27 08:03:45|pCCmWu|DEMO_PROJECT_5_2_1|hcatalog_load|java||Failed|Job execution error|Communications link failure due to underlying exception:

    ** BEGIN NESTED EXCEPTION **

    java.net.SocketException
    MESSAGE: java.net.ConnectException: Connection refused

    Collapse
    #25933

    Seth Lyubich
    Keymaster

    Hi Kai,

    I just tried and I was able to create database using Talend when I chose database location as /apps/hive/warehouse.

    Hope this helps.

    Thanks
    Seth

    Collapse
    #25929

    Seth Lyubich
    Keymaster

    Hi Kai,

    I have not tried Talend with Sandbox and I am not sure if this is supported but I think default database location is /apps/hive/warehouse.

    Can you please try to change database location to ‘/apps/hive/warehouse’ and try again?

    Thanks,
    Seth

    Collapse
    #25678

    Kai Waehner
    Member

    Thanks for quick answer.

    My Templeton status is ok. I am trying to run your example Talend job “hcatalog_load”. Maybe I have done some misconfiguration here.

    tHDFSDelete works, then tHCatalogOperation fails with the above error message, I cannot create the database. So connection via HCatalog does not work.

    Do you see some misconfiguration?

    Distro: Hortonworks V1
    Templeton hostname: “IP of VM”
    Templeton port: 50111
    Operation on “Database”: “Create”
    username: sandbox
    database location: /user/hdp/test/usersdb
    database description: “”
    die on error: true

    Collapse
    #25646

    Seth Lyubich
    Keymaster

    Hi Kai,

    Tempeton/WebHcat should run on port 50111. It does not have a web ui, but you can test if Templeton responds with the command below using you browser:

    http://ip-of-sandbox:50111/templeton/v1/status .

    If it responds you should see something like below:

    {“status”:”ok”,”version”:”v1″}

    Can you please test this? If you still have an issue can you please let us know which example you are using and we will try to reproduce.

    Hope this helps,
    Thanks,
    Seth

    Collapse
Viewing 7 replies - 1 through 7 (of 7 total)