Home Forums Hive / HCatalog HiveMetastore: Unable to Make JDBC connection

This topic contains 9 replies, has 2 voices, and was last updated by  Jayashankar VS 1 year, 1 month ago.

  • Creator
    Topic
  • #30899

    Jayashankar VS
    Participant

    Team,
    I am validating the hive installation by starting Hive Metastore service using the below command, but it says jdbc driver is not found. Please note that I’ve already installed mysql-java connector and copied the connector to /usr/lib/hive/lib/mysql-connector-java.jar, still for some reason the connector is not being picked up. I even tried setting the connector path explicitly in the hive.env.sh file and OS env path, but the issue persist. Please suggest.

    -rw-r–r– 1 root root 538667 Aug 6 14:34 /usr/lib/hive/lib/mysql-connector-java.jar

    Login as $HIVE_USER
nohup hive –service metastore>$HIVE_LOG_DIR/hive.out 2>$HIVE_LOG_DIR/hive. log &
    bash-3.2$ ./bin/hive
    WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
    Logging initialized using configuration in jar:file:/usr/lib/hive/lib/hive-common-0.10.0.21.jar!/hive-log4j.properties
    Hive history file=/tmp/hive/hive_job_log_hive_201308061714_797728604.txt
    hive> show tables;
    FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
    hive>

    WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
    javax.jdo.JDOFatalInternalException: Error creating transactional connection factory
    at org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:558)
    at org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:604)

    Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the “DBCP” plugin to create a ConnectionPool gave an error : The specified datastore driver (“com.mysql.jdbc.driver”) was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
    at org.datanucleus.store.rdbms.ConnectionFactoryImpl.initialiseDataSources(ConnectionFactoryImpl.java:202)
    at org.datanucleus.store.rdbms.ConnectionFactoryImpl.(ConnectionFactoryImpl.java:95)-rw-r–r– 1 root root 538667 Aug 6 14:34 /usr/lib/hive/lib/mysql-connector-java.jar

    Thanks,
    Jay

Viewing 9 replies - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #31208

    Jayashankar VS
    Participant

    Thank you so much and apologies for wasting your time. It was such simple fix and not sure how I missed it, it was a careless mistake from my side. I had the documentation on my side but rather than copying & pasting the driver class name I typed it myself , though I did run through the documentaion again failed to catch this up.

    Thanks again for pointing this out this silly mistake and sorry again for wasting your valuable time.

    Thanks,
    Jay

    Collapse
    #31161

    Sasha J
    Moderator

    javax.jdo.option.ConnectionDriverName value should be: com.mysql.jdbc.Driver

    Driver, which CAPITAL “D”.

    Please, fix your hive-site.xml

    Sasha

    Collapse
    #31116

    Jayashankar VS
    Participant

    Let me know if you need more information, hope you got a chance to review the config file.

    Collapse
    #30997

    Jayashankar VS
    Participant

    And finally here is hive.env.sh, let me know if you need further details:

    if [ "$SERVICE" = "cli" ]; then
    if [ -z "$DEBUG" ]; then
    export HADOOP_OPTS=”$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:+UseParNewGC -XX:-UseGCOverheadLimit”
    else
    export HADOOP_OPTS=”$HADOOP_OPTS -XX:NewRatio=12 -Xms10m -XX:MaxHeapFreeRatio=40 -XX:MinHeapFreeRatio=15 -XX:-UseGCOverheadLimit”
    fi
    fi

    # The heap size of the jvm stared by hive shell script can be controlled via:
    HBASE_CONF_DIR=${HBASE_CONF_DIR:-/etc/hbase/conf}
    export HADOOP_HEAPSIZE=1024
    # set hadoop HADOOP_CLIENT_OPTS so that hadoop does not override it
    export HADOOP_CLIENT_OPTS=”-Xmx${HADOOP_HEAPSIZE}m $HADOOP_CLIENT_OPTS”

    # Larger heap size may be required when running queries over large number of files or partitions.
    # By default hive shell scripts use a heap size of 256 (MB). Larger heap size would also be
    # appropriate for hive server (hwi etc).

    # Set HADOOP_HOME to point to a specific hadoop install directory
    HADOOP_HOME=${HADOOP_HOME:-/etc/hadoop/conf}

    # Hive Configuration Directory can be controlled by:
    export HIVE_CONF_DIR=/etc/hive/conf
    export JAVA_HOME=/usr/java/default
    # Folder containing extra ibraries required for hive compilation/execution can be controlled by:
    export HIVE_AUX_JARS_PATH=/usr/lib/hcatalog/share/hcatalog/hcatalog-core.jar
    “hive-env.sh” 55L, 2650C

    Collapse
    #30996

    Jayashankar VS
    Participant

    Here is the second split of hive-site.xml

    hive.metastore.uris
    thrift://hadoopnn:9083
    URI for client to contact metastore server

    hadoop.clientside.fs.operations
    true
    FS operations are owned by client

    hive.metastore.client.socket.timeout
    60
    MetaStore Client socket timeout in seconds

    hadoop.proxyuser.HTTP.groups
    hadoop
    Proxy group for Hadoop.

    hadoop.proxyuser.HTTP.hosts
    *
    Proxy host for Hadoop.

    hive.security.authorization.manager
    org.apache.hcatalog.security.HdfsAuthorizationProvider

    the hive client authorization manager class name.
    The user defined authorization class should implement interface
    org.apache.hadoop.hive.ql.security.authorization.HiveAuthorizationProvider.

    hive.server2.enable.doAs
    true

    fs.hdfs.impl.disable.cache
    true

    Collapse
    #30995

    Jayashankar VS
    Participant

    I am pasting you the file hve-site.xml in 2 splits, here it is:

    javax.jdo.option.ConnectionURL
    jdbc:mysql://hadoopnn:3306/hive?createDatabaseIfNotExist=true
    JDBC connect string for a JDBC metastore

    hive.metastore.execute.setugi
    true

    javax.jdo.option.ConnectionDriverName
    com.mysql.jdbc.driver
    Driver class name for a JDBC metastore

    javax.jdo.option.ConnectionUserName
    hive
    username to use against metastore database

    javax.jdo.option.ConnectionPassword
    password
    password to use against metastore database

    hive.metastore.warehouse.dir
    /apps/hive/warehouse
    location of default database for the warehouse

    hive.metastore.sasl.enabled
    false
    If true, the metastore thrift interface will be secured with SASL.
    Clients must authenticate with Kerberos.

    hive.metastore.kerberos.keytab.file
    /etc/security/keytabs/hive.service.keytab
    The path to the Kerberos Keytab file containing the metastore
    thrift server’s service principal.

    hive.metastore.kerberos.principal
    hive/_HOST@EXAMPLE.COM
    The service principal for the metastore thrift server. The special
    string _HOST will be replaced automatically with the correct host name.

    hive.server2.authentication
    NONE

    hive.server2.authentication.kerberos.principal
    hive/_HOST@EXAMPLE.COM

    hive.server2.authentication
    NONE

    hive.server2.authentication.kerberos.principal
    hive/_HOST@EXAMPLE.COM

    hive.server2.authentication.kerberos.keytab
    /etc/security/keytabs/hive.service.keytab
    </property

    hive.metastore.cache.pinobjtypes
    Table,Database,Type,FieldSchema,Order
    List of comma separated metastore object types that should be pinned in the cache

    Collapse
    #30993

    Jayashankar VS
    Participant

    Please specify a mail ID where I could mail you the attachment.

    Collapse
    #30992

    Jayashankar VS
    Participant

    Not sure how to post the complete file(hive-site.xml), pls specify a mail ID where I could send the mail.
    javax.jdo.option.ConnectionURL
    jdbc:mysql://hadoopnn:3306/hive?createDatabaseIfNotExist=true
    JDBC connect string for a JDBC metastore

    hive.metastore.execute.setugi
    true

    javax.jdo.option.ConnectionDriverName
    com.mysql.jdbc.driver
    Driver class name for a JDBC metastore

    javax.jdo.option.ConnectionUserName
    hive
    username to use against metastore database

    javax.jdo.option.ConnectionPassword
    password
    password to use against metastore database

    hive.metastore.warehouse.dir
    /apps/hive/warehouse
    location of default database for the warehouse

    hive.metastore.sasl.enabled
    false
    If true, the metastore thrift interface will be secured with SASL.
    Clients must authenticate with Kerberos.

    hive.metastore.kerberos.keytab.file
    /etc/security/keytabs/hive.service.keytab
    The path to the Kerberos Keytab file containing the metastore
    thrift server’s service principal.

    hive.metastore.kerberos.principal
    hive/_HOST@EXAMPLE.COM
    The service principal for the metastore thrift server. The special
    string _HOST will be replaced automatically with the correct host name.

    hive.server2.authentication
    NONE

    hive.server2.authentication.kerberos.principal
    hive/_HOST@EXAMPLE.COM

    hive.server2.authentication
    NONE

    hive.server2.authentication.kerberos.principal
    hive/_HOST@EXAMPLE.COM

    hive.server2.authentication.kerberos.keytab
    /etc/security/keytabs/hive.service.keytab
    </property

    hive.metastore.cache.pinobjtypes
    Table,Database,Type,FieldSchema,Order
    List of comma separated metastore object types that should be pinned in the cache

    hive.metastore.uris
    thrift://hadoopnn:9083
    URI for client to contact metastore server
    </p

    Collapse
    #30946

    Sasha J
    Moderator

    looks like a problems with hive configuration…
    Can you post you hove config file?

    Sasha

    Collapse
Viewing 9 replies - 1 through 9 (of 9 total)