Sqoop job through oozie

to create new topics or reply. | New User Registration

This topic contains 5 replies, has 5 voices, and was last updated by  Bijoy Kumar Khandelwal 10 months, 2 weeks ago.

  • Creator
  • #56348

    Colman Madden

    I have created a sqoop job called TeamMemsImportJob which basically pulls data from sql server into hive.
    I can execute the sqoop job through the unix command line by running the following command:

    sqoop job –exec TeamMemsImportJob

    If I create an oozie job with the actual scoop import command in it, it runs through fine.
    However if I create the oozie job and run the sqoop job through it, I get the following error:

    oozie job -config TeamMemsImportJob.properties -run

    >>> Invoking Sqoop command line now >>>

    4273 [main] WARN org.apache.sqoop.tool.SqoopTool – $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
    4329 [main] INFO org.apache.sqoop.Sqoop – Running Sqoop version:
    5172 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage – Cannot restore job: TeamMemsImportJob
    5172 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage – (No such job)
    5172 [main] ERROR org.apache.sqoop.tool.JobTool – I/O error performing job operation: java.io.IOException: Cannot restore missing job TeamMemsImportJob
    at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.read(HsqldbJobStorage.java:256)
    at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:198)

    it looks as if it cannot find the job. However I can see the job as below

    [root@sandbox ~]# sqoop job –list
    Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
    Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    14/06/25 08:12:08 INFO sqoop.Sqoop: Running Sqoop version:
    Available jobs:

    Can someone please help me out with this?


Viewing 5 replies - 1 through 5 (of 5 total)

You must be to reply to this topic. | Create Account

  • Author
  • #58694

    The metastore for sqoop is hsqldb. Your hsqldb is started locally where sqoop runs. You have to start it remotley so that it is accessible across the cluster. Sqoop-sit.xml needs changes.


    Aparna Nayak

    Hello Diwakar,
    Thank you for your response. I tried that. Beginning I did not get any examples. Later I found.


    Diwakar Dhanuskodi

    Did you try out the sqoop sample code availabe in oozie example directories.


    Aparna Nayak

    I have executed sqoop import tables from sql server to hdfs and vice versa. How I can schedule the same with oozie? Do I get step by step procedure anywhere?


    Daro Sdf

    Have you found solution ? because i have the same problem.

Viewing 5 replies - 1 through 5 (of 5 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.