Home Forums Hive / HCatalog Parallel ODBC queries to Hive

Tagged: , ,

This topic contains 5 replies, has 3 voices, and was last updated by  Tian An Koh 1 year, 2 months ago.

  • Creator
    Topic
  • #25332

    We’re implementing Proof of Concept project on Hive and ODBC connectivity.
    We’re running Hive with Hiveserver2. We’ve established connection to Hive from MS SQL 2012 sp1 via linked server, which uses System DSN (ODBC datasource) based on Hortonworks ODBC driver v1.2.0.1005 (64 bit).
    We’ve created a database view, which uses OPENQUERY to access Hive and run command.

    Everything appears to be OK until we launch parallel selects from the view. The first view returns data, while others crash with errors:

    OLE DB provider “MSDASQL” for linked server “hortonhive” returned message “[Hortonworks][Hardy] (35) Error from Hive: error code: ‘0’ error message: ‘java.io.IOException: java.lang.ArrayIndexOutOfBoundsException’.”.
    Msg 7330, Level 16, State 2, Line 1
    Cannot fetch a row from OLE DB provider “MSDASQL” for linked server “hortonhive”.

Viewing 5 replies - 1 through 5 (of 5 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #33757

    Tian An Koh
    Member

    Hi,

    Can i ask if you are to use openquery, how will a hive insert/load query look like?

    Sorry i’m new to Hive too and am also doing some research on this technology.

    I did something similiar using barebones Hive on Ubuntu. I’ve posted the same question here

    http://stackoverflow.com/questions/18501954/hive-insert-with-openquery

    Hoping someone can shed some light on it

    Thank you
    Koh

    Collapse
    #25553

    Further more (thanks to Brock Noland):
    The patch in https://issues.apache.org/jira/browse/HIVE-4516 is much simpler and fixes the same issue. HIVE-4516 will be included in the 4.3.0 release.

    Collapse
    #25552

    I’ve located the source of the problem. It’s not ODBC drivers issue. My Hive table has TIMESTAMP and it’s ToString is not thread safe. Some conditions are triggered while fetching and it causes crash in Hive.
    The problem is described and patch is provided here:

    https://issues.apache.org/jira/browse/HIVE-4220

    Collapse
    #25508

    Yi Zhang
    Moderator

    Hi Ramunas,

    Can you give us sample schema of the view?

    Could you help us see the problem by posting the hiveserver2 log (/var/log/hive/hiveserver2.log on the hive server node) and the log4j logs for the user (/tmp/$user/hive.log on the hive client node) when the problem happens? Any other sterr/stout messages would be helpful too.

    Thanks,
    Yi

    Collapse
    #25357

    in addition: it appears that crash occurs after map/reduce finishes and second query is about to return result set.

    Collapse
Viewing 5 replies - 1 through 5 (of 5 total)