The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Sqoop Forum

sqoop export problem

  • #55519
    D D
    Participant

    Hello
    we are having trouble doing sqoop export from hcatalog table to a mysql table, seems like there is some haoop version incompatibility when generating sqoop job, does anyone have a clue for me on this ?

    sqoop export -D mapreduce.job.queuename=q_restitution –connect jdbc:mysql://xx.xx.xx.xx/test –username hive –password hive –table ihm_ano –hcatalog-database ‘hive_temp’ –hcatalog-table ‘ihm_ano’ –verbose

    14/06/10 14:56:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.4.2.0.6.1-102
    ..
    14/06/10 14:56:57 INFO client.RMProxy: Connecting to ResourceManager at x.x.x.x/xx.xx.xx.xx:8050
    14/06/10 14:56:57 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 55683 for u__etl on xx.xx.xx.xx:8020
    14/06/10 14:56:57 INFO security.TokenCache: Got dt for hdfs://x.x.x.x:8020; Kind: HDFS_DELEGATION_TOKEN, Service: xx.xx.xx.xx:8020, Ident: (HDFS_DELEGATION_TOKEN token 55683 for u__etl)
    14/06/10 14:57:01 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/u__etl/.staging/job_1400766215563_0542
    14/06/10 14:57:01 DEBUG util.ClassLoaderStack: Restoring classloader: sun.misc.Launcher$AppClassLoader@63c78e57
    <b>Exception in thread “main” java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
    at org.apache.hcatalog.mapreduce.HCatBaseInputFormat.getSplits(HCatBaseInputFormat.java:101)</b>
    at org.apache.sqoop.mapreduce.hcat.SqoopHCatExportFormat.getSplits(SqoopHCatExportFormat.java:56)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491)
    at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
    at org.apache.sqoop.mapreduce.ExportJobBase.doSubmitJob(ExportJobBase.java:296)
    at org.apache.sqoop.mapreduce.ExportJobBase.runJob(ExportJobBase.java:273)
    at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:405)
    at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.java:828)
    at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:81)
    at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:100)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.ru

  • Author
    Replies
  • #55637

    Hi,

    Can you share the output of ‘rpm -qa | grep hadoop’ and ‘rpm -qa | grep mapreduce’?

    -Mahesh

    #55767
    D D
    Participant

    Hi,

    here are the output

    rpm -qa | grep hadoop

    hadoop-client-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-mapreduce-historyserver-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-lzo-0.5.0-1.x86_64
    hadoop-yarn-resourcemanager-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-yarn-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-libhdfs-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-lzo-native-0.5.0-1.x86_64
    hadoop-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-mapreduce-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-hdfs-2.2.0.2.0.6.0-102.el6.x86_64

    rpm -qa | grep mapreduce

    hadoop-mapreduce-historyserver-2.2.0.2.0.6.0-102.el6.x86_64
    hadoop-mapreduce-2.2.0.2.0.6.0-102.el6.x86_64

The forum ‘Sqoop’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.