The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Oozie Forum

Sqoop job through oozie

  • #56348
    Colman Madden
    Participant

    I have created a sqoop job called TeamMemsImportJob which basically pulls data from sql server into hive.
    I can execute the sqoop job through the unix command line by running the following command:

    sqoop job –exec TeamMemsImportJob

    If I create an oozie job with the actual scoop import command in it, it runs through fine.
    However if I create the oozie job and run the sqoop job through it, I get the following error:

    oozie job -config TeamMemsImportJob.properties -run

    >>> Invoking Sqoop command line now >>>

    4273 [main] WARN org.apache.sqoop.tool.SqoopTool – $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
    4329 [main] INFO org.apache.sqoop.Sqoop – Running Sqoop version: 1.4.4.2.1.1.0-385
    5172 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage – Cannot restore job: TeamMemsImportJob
    5172 [main] ERROR org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage – (No such job)
    5172 [main] ERROR org.apache.sqoop.tool.JobTool – I/O error performing job operation: java.io.IOException: Cannot restore missing job TeamMemsImportJob
    at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.read(HsqldbJobStorage.java:256)
    at org.apache.sqoop.tool.JobTool.execJob(JobTool.java:198)

    it looks as if it cannot find the job. However I can see the job as below

    [root@sandbox ~]# sqoop job –list
    Warning: /usr/lib/sqoop/../accumulo does not exist! Accumulo imports will fail.
    Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    14/06/25 08:12:08 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.1.1.0-385
    Available jobs:
    TeamMemsImportJob

    Can someone please help me out with this?

    Thanks,
    Colman

  • Author
    Replies
  • #57069
    Daro Sdf
    Participant

    Have you found solution ? because i have the same problem.

    #57323
    Aparna Nayak
    Participant

    Hello,
    I have executed sqoop import tables from sql server to hdfs and vice versa. How I can schedule the same with oozie? Do I get step by step procedure anywhere?

    #57784
    Diwakar Dhanuskodi
    Participant

    Did you try out the sqoop sample code availabe in oozie example directories.

    #57786
    Aparna Nayak
    Participant

    Hello Diwakar,
    Thank you for your response. I tried that. Beginning I did not get any examples. Later I found.

    #58694

    The metastore for sqoop is hsqldb. Your hsqldb is started locally where sqoop runs. You have to start it remotley so that it is accessible across the cluster. Sqoop-sit.xml needs changes.

The forum ‘Oozie’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.