Home Forums HDP on Windows – Installation Sqoop Import Error

This topic contains 9 replies, has 2 voices, and was last updated by  Seth Lyubich 6 months, 1 week ago.

  • Creator
    Topic
  • #46246

    Manish Sharma
    Participant

    Hi All,

    I am working on HDP 1.3.0 for Windows Server, configured one master and two slave nodes including HDP client. I want to import mysql data HDFS all time its showing me an error, I have mentioned that below. Could any please help me out, How I will be able to resolve this issue.

    Thanks
    MANISH

    Error which I am getting
    ============================================================================================================================
    c:\hdp\hadoop\sqoop-1.4.3.1.3.0.0-0380\bin>sqoop import –connect “jdbc:mysql://xxx.xxx.x.xx:xxxx;username=root;password=;database=mysql” -table u
    –target-dir /user/Administrator/tmp/
    ============================================================================================================================
    14/01/01 12:41:49 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
    14/01/01 12:41:49 INFO tool.CodeGenTool: Beginning code generation
    14/01/01 12:41:49 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Drive
    java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
    at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:716)
    at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
    at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
    at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
    at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
    at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1264)
    at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1081)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
    =============================================================================================================================

Viewing 9 replies - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #47355

    Seth Lyubich
    Keymaster

    Hi Manish,

    I think your other post has similar issue. I think there are two things missing here:

    1. Java connector needs to be installed, not just moved to sqoop/lib. If you can’t install the connector you need to copy .jar file that we can send you if needed.

    2. The error message seems to complain about Java version. “… It seems as though you are running sqoop with a JRE.”

    Can you please try following command?

    C:\Java\jdk1.7.0_51>set | findstr JAVA
    JAVA_HOME=C:\Java\jdk1.7.0_51

    You should see JDK location.

    Hope this helps,

    Thanks,
    Seth

    Collapse
    #47324

    Manish Sharma
    Participant

    Hi Seth,

    Thanks for Responding, What I did I just want to share it with you,I have copied mysql-connector-java-3.0.17-ga-bin to sqoop lib folder.I am using HDP 1.3.0 for Windows.

    I am getting following Error.

    14/01/23 15:41:00 ERROR orm.CompilationManager: It seems as though you are running sqoop with a JRE.
    14/01/23 15:41:00 ERROR orm.CompilationManager: Sqoop requires a JDK that can compile Java code.
    14/01/23 15:41:00 ERROR orm.CompilationManager: Please install a JDK and set $JAVA_HOME to use it.
    14/01/23 15:41:00 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not start Java compiler.

    Below is the command which I have used: sqoop import –connect jdbc:mysql://192.168.xxx.xx:3306/test –username root -table users

    Please tell me what I am doing wrong, and If possible then please tell me the steps.

    Thanks
    MANISH

    Collapse
    #47279

    Seth Lyubich
    Keymaster

    Hi Manish,

    From the error message looks like driver is not loading:

    Could not load db driver class: com.mysql.jdbc…

    Can you please verify that the driver is installed?

    Hope this helps.

    Thanks,
    Seth

    Collapse
    #47244

    Manish Sharma
    Participant

    Hi Seth,

    Is HDP 1.3.0 SQOOP for windows supports connector for My SQL. I am not getting any way to import MySQL data to Hadoop Cluster (means HDP 1.3.0 for windows).

    Thanks
    MANISH

    Collapse
    #47243

    Manish Sharma
    Participant

    Hi Seth,

    The Error which I am getting I have mentioned Below.

    ============================================================================================================================
    c:\hdp\hadoop\sqoop-1.4.3.1.3.0.0-0380\bin>sqoop import –connect “jdbc:mysql://xxx.xxx.x.xx:xxxx;username=root;password=;database=mysql” -table u
    –target-dir /user/Administrator/tmp/
    ============================================================================================================================
    14/01/01 12:41:49 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
    14/01/01 12:41:49 INFO tool.CodeGenTool: Beginning code generation
    14/01/01 12:41:49 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Drive
    java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
    at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:716)
    at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:605)
    at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:628)
    at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:235)
    at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:219)
    at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:347)
    at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1264)
    at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1081)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
    =============================================================================================================================

    Thanks
    MANISH

    Collapse
    #47234

    Seth Lyubich
    Keymaster

    Hi Manish,

    Can you please let us know which error are you getting?

    Thanks,
    Seth

    Collapse
    #46766

    Manish Sharma
    Participant

    Hi Seth,

    I have downloaded the MySQL connector but placed it to sqoop/lib folder, But its not working, Could you please tell me the steps how I can import mysql data to HDFS using HDP 1.3 for windows.

    Thanks
    MANISH

    Collapse
    #46752

    Manish Sharma
    Participant

    Hi,

    Thanks for your response, could you please tell me the steps how to use sqoop with HDP 1.3 on windows. I am not getting any document related to this, Actually my requirement is to transfer MY SQL data to Hadoop cluster.

    Thanks
    MANISH

    Collapse
    #46738

    Seth Lyubich
    Keymaster

    Hi Manish,

    Can you please try downloading MySQL driver from link below and try again?

    http://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-java-gpl-5.1.28.msi

    Thanks,
    Seth

    Collapse
Viewing 9 replies - 1 through 9 (of 9 total)