Home Forums Sqoop Error in running sqoop from a Java code

This topic contains 6 replies, has 5 voices, and was last updated by  Mahesh Balakrishnan 4 months, 2 weeks ago.

  • Creator
    Topic
  • #47768

    Aysan Rasooli
    Participant

    Hi,
    I have a java code which runs sqoop to import data from Postgresql. When I run it on eclipse I get the following error. I am using CentOS, and I have already installed JDK, but still getting this error. Could you please let me know how can I solve the problem?

    Here is the error:

    :40,943 WARN [main] sqoop.ConnFactory (ConnFactory.java:loadManagersFromConfDir(277)) – $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
    2014-01-29 16:07:40,984 INFO [main] manager.SqlManager (SqlManager.java:initOptionDefaults(95)) – Using default fetchSize of 1000
    2014-01-29 16:07:40,985 INFO [main] tool.CodeGenTool (CodeGenTool.java:generateORM(78)) – Beginning code generation
    2014-01-29 16:07:41,118 INFO [main] manager.SqlManager (SqlManager.java:execute(618)) – Executing SQL statement: SELECT t.* FROM “two” AS t LIMIT 1
    2014-01-29 16:07:41,150 INFO [main] orm.CompilationManager (CompilationManager.java:findHadoopCoreJar(92)) – HADOOP_MAPRED_HOME is /usr/lib/hadoop/etc/hadoop
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(173)) – It seems as though you are running sqoop with a JRE.
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(174)) – Sqoop requires a JDK that can compile Java code.
    2014-01-29 16:07:41,153 ERROR [main] orm.CompilationManager (CompilationManager.java:compile(175)) – Please install a JDK and set $JAVA_HOME to use it.
    2014-01-29 16:07:41,154 ERROR [main] tool.ImportTool (ImportTool.java:run(484)) – Encountered IOException running import job: java.io.IOException: Could not start Java compiler.
    at org.apache.sqoop.orm.CompilationManager.compile(CompilationManager.java:176)
    at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:83)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at sqoopJob.main(sqoopJob.java:61)

    Thanks,
    Aysan

Viewing 6 replies - 1 through 6 (of 6 total)

The topic ‘Error in running sqoop from a Java code’ is closed to new replies.

  • Author
    Replies
  • #49594

    Hi Saurabh, I see that you have posted the same on a different thread, I will close this thread and you can continue to use the other thread.

    -Mahesh

    Collapse
    #48140

    Sourabh Potnis
    Participant

    Hi,
    Even I am trying to run Sqoop from a Java program using Sqoop.runTool(str), to export data to Teradat from HDFS.

    While compiling I have added following Jars:
    /etc/hadoop/conf:/usr/lib/sqoop/sqoop-1.4.4.2.0.6.0-76.jar, all jars in/usr/lib/sqoop/lib, /usr/lib/hadoop/hadoop-common-2.2.0.2.0.6.0-76.jar, all jars in /usr/lib/hadoop/lib, /usr/lib/hadoop-mapreduce/hadoop-mapreduce-client-core-2.2.0.2.0.6.0-76.jar, all jars in /usr/lib/hadoop-mapreduce, all jars in /usr/lib/hadoop-mapreduce/lib

    But when running the jar, getting following error:
    Exception in thread “main” java.lang.NoSuchFieldError: IBM_JAVA
    at org.apache.hadoop.security.UserGroupInformation.getOSLoginModuleName(UserGroupInformation.java:303)
    at org.apache.hadoop.security.UserGroupInformation.<clinit>(UserGroupInformation.java:348)
    at org.apache.hadoop.mapreduce.JobContext.<init>(JobContext.java:80)
    at org.apache.hadoop.mapreduce.Job.<init>(Job.java:100)
    at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:382)
    at org.apache.sqoop.teradata.TeradataConnManager.exportTable(TeradataConnManager.java:109)
    at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
    at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
    at SqoopTest.exportHDFSToSQL(SqoopTest.java:20)
    at SqoopTest.main(SqoopTest.java:46)

    (From a command line Sqoop export is working fine.)

    Thanks,
    Sourabh

    Collapse
    #48090

    Aysan Rasooli
    Participant

    Hi Dave,

    Thanks for the help! It worked. The problem was that the eclipse was using the old setting for the java home, which was pointing to jar location.

    Thanks,
    Aysan

    Collapse
    #47947

    Dave
    Moderator

    Hi Aysan,

    If you run the command:

    javac

    does it provide an output?

    Thanks

    Dave

    Collapse
    #47806

    Aysan Rasooli
    Participant

    Hi Robert,

    I set the JAVA_HOME as an environment variable in my eclipse when I am running the code. I also set it in ~/.bash_profile but still getting this error. Shall I set it anywhere else?

    Thanks,
    Aysan

    Collapse
    #47799

    Robert Molina
    Moderator

    Hi Aysan,
    It seems to state you don’t have the JDK installed. Can you verifying your java home is pointed to the JDK?

    Regards,
    Robert

    Collapse
Viewing 6 replies - 1 through 6 (of 6 total)