Sqoop Forum

Error in running sqoop from a Java code

  • #47768
    Aysan Rasooli
    Participant

    Hi,
    I have a java code which runs sqoop to import data from Postgresql. When I run it on eclipse I get the following error. I am using CentOS, and I have already installed JDK, but still getting this error. Could you please let me know how can I solve the problem?

    Here is the error:

    :40,943 WARN [main] sqoop.ConnFactory (ConnFactory.java:loadManagersFromConfDir(277)) – $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
    2014-01-29 16:07:40,984 INFO [main] manager.SqlManager (SqlManager.java:initOptionDefaults(95)) – Using default fetchSize of 1000
    2014-01-29 16:07:40,985 INFO [main] tool.CodeGenTool (CodeGenTool.java:generateORM(78)) – Beginning code generation
    2014-01-29 16:07:41,118 INFO [main] manager.SqlManager (SqlManager.java:execute(618)) – Executing SQL statement: SELECT t.* FROM “two” AS t LIMIT 1
    2014-01-29 16:07:41,150 INFO [main] orm.CompilationManager (CompilationManager.java:findHadoopCoreJar(92)) – HADOOP_MAPRED_HOME is /usr/lib/hadoop/etc/hadoop
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(173)) – It seems as though you are running sqoop with a JRE.
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(174)) – Sqoop requires a JDK that can compile Java code.
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(175)) – Please install a JDK and set $JAVA_HOME to use it.
    2014-01-29 16:07:41,154 ERROR [main] tool.ImportTool (ImportTool.java:run(484)) – Encountered IOException running import job: java.io.IOException: Could not start Java compiler.
    at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:176)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at sqoopJob.main(sqoopJob.java:61)

    Thanks,
    Aysan

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #47799
    Robert Molina
    Moderator

    Hi Aysan,
    It seems to state you don’t have the JDK installed. Can you verifying your java home is pointed to the JDK?

    Regards,
    Robert

    #47806
    Aysan Rasooli
    Participant

    Hi Robert,

    I set the JAVA_HOME as an environment variable in my eclipse when I am running the code. I also set it in ~/.bash_profile but still getting this error. Shall I set it anywhere else?

    Thanks,
    Aysan

    #47947
    Dave
    Moderator

    Hi Aysan,

    If you run the command:

    javac

    does it provide an output?

    Thanks

    Dave

    #48090
    Aysan Rasooli
    Participant

    Hi Dave,

    Thanks for the help! It worked. The problem was that the eclipse was using the old setting for the java home, which was pointing to jar location.

    Thanks,
    Aysan

    #48140
    Sourabh Potnis
    Participant

    Hi,
    Even I am trying to run Sqoop from a Java program using Sqoop.runTool(str), to export data to Teradat from HDFS.

    While compiling I have added following Jars:
    /etc/hadoop/conf:/usr/lib/sqoop/sqoop-1.4.4.2.0.6.0-76.jar, all jars in/usr/lib/sqoop/lib, /usr/lib/hadoop/hadoop-common-2.2.0.2.0.6.0-76.jar, all jars in /usr/lib/hadoop/lib, /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.2.0.2.0.6.0-76.jar, all jars in /usr/lib/hadoop-mapreduce, all jars in /usr/lib/hadoop-mapreduce/lib

    But when running the jar, getting following error:
    Exception in thread “main” java.lang.NoSuchFieldError: IBM_JAVA
    at org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:303)
    at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:348)
    at org.apache.hadoop.mapreduce.JobContext.<init>(JobContext.java:80)
    at org.apache.hadoop.mapreduce.Job.<init>(Job.java:100)
    at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:382)
    at org.apache.sqoop.teradata.TeradataConnManager.exportTable(TeradataConnManager.java:109)
    at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
    at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
    at SqoopTest.exportHDFSToSQL(SqoopTest.java:20)
    at SqoopTest.main(SqoopTest.java:46)

    (From a command line Sqoop export is working fine.)

    Thanks,
    Sourabh

    #49594

    Hi Saurabh, I see that you have posted the same on a different thread, I will close this thread and you can continue to use the other thread.

    -Mahesh

The topic ‘Error in running sqoop from a Java code’ is closed to new replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.