The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Sqoop Forum

Sqoop export FROM HDFS into SAP HANA

  • #45808
    Felix Bekcer

    Hi everybody,

    i’m working now with hadoop and sap hana for around 3 month and i stick heavily.
    My Hadoop System is importing and converting logfiles via pig script and generates a result, which i finally want to export to sap hana appliance.

    I wanted to use sqoop for this task, but at the moment, i’m wondering about the error message and the behavior of the system.
    To get handy with sqoop, i generated a test file with some simple input like this:


    On the other side, i generated a table in sap hana with a text field column.
    I used this sqoop command to successfully export the hdfs test file into sap hana:

    sqoop export –connect jdbc:sap://saphana.XXX.XXX:30115/ –driver –table XXX.XXX–username XXX –password XXX –export-dir /user/tom/test

    And it worked. I saw 1 3 and 5 on the sap hana side.
    Then i tried to export much more data, but till in the simple format like:


    And then i got some error messages, which dont have something to do with the change of my test file:

    15:18:44 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1422) | Task Id : attempt_201312180839_0052_m_000000_0, Status : FAILED SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near “,”: line 1 col 35 (at pos 35)

    Could someone help me with this dirty problem?


  • Author
  • #45891
    Felix Bekcer

    curiously sqoop exported the first two lines to sap hana and then crashes.

    15:18:19 DEBUG – org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(76) | Generated splits:
    15:18:19 DEBUG – org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(78) | Paths:/user/tom/part2:0+3 Locations:localhost.localdomain:;
    15:18:19 DEBUG – org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(78) | Paths:/user/tom/part2:3+3 Locations:localhost.localdomain:;
    15:18:19 DEBUG – org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(78) | Paths:/user/tom/part2:6+3 Locations:localhost.localdomain:;
    15:18:19 DEBUG – org.apache.sqoop.mapreduce.ExportInputFormat.getSplits(78) | Paths:/user/tom/part2:9+2,/user/tom/part2:11+3 Locations:localhost.localdomain:;
    15:18:19 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1380) | Running job: job_201312180839_0052
    15:18:20 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1393) | map 0% reduce 0%
    15:18:42 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1393) | map 25% reduce 0%
    15:18:43 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1393) | map 75% reduce 0%
    15:18:44 INFO – org.apache.hadoop.mapred.JobClient.monitorAndPrintJob(1422) | Task Id : attempt_201312180839_0052_m_000000_0, Status : FAILED SAP DBTech JDBC: [257]: sql syntax error: incorrect syntax near “,”: line 1 col 35 (at pos 35)
    at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(
    at org.apache.hadoop.mapred.MapTask.runNewMapper(

    My first idea was the autocommit feature, but i debugged in and checked it. it is false.
    i tried several text file values but the only thing that fixes the crash was to cut all values out expect the first two lines.

    Felix Bekcer

    couldn’t nobody help me? i’m still sticking at this point.


    Felix Bekcer

    [root@localhost attempt_201312270310_0023_m_000000_0]# cat syslog
    2013-12-28 15:33:18,150 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop library
    2013-12-28 15:33:18,703 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0
    2013-12-28 15:33:18,711 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2830ae41
    2013-12-28 15:33:18,979 INFO org.apache.hadoop.mapred.MapTask: Processing split: Paths:/user/tom/part2:0+10
    2013-12-28 15:33:19,706 INFO com.hadoop.compression.lzo.GPLNativeCodeLoader: Loaded native gpl library
    2013-12-28 15:33:19,710 INFO com.hadoop.compression.lzo.LzoCodec: Successfully loaded & initialized native-lzo library [hadoop-lzo rev cf4e7cbf8ed0f0622504d008101c2729dc0c9ff3]
    2013-12-28 15:33:19,717 WARN Snappy native library is available
    2013-12-28 15:33:19,717 INFO Snappy native library loaded
    2013-12-28 15:33:19,748 DEBUG org.apache.sqoop.mapreduce.AutoProgressMapper: Instructing auto-progress thread to quit.
    2013-12-28 15:33:19,748 DEBUG org.apache.sqoop.mapreduce.AutoProgressMapper: Waiting for progress thread shutdown…
    2013-12-28 15:33:19,751 INFO org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
    2013-12-28 15:33:19,753 DEBUG org.apache.sqoop.mapreduce.AutoProgressMapper: Progress thread shutdown detected.
    2013-12-28 15:33:19,898 INFO org.apache.hadoop.mapred.MapTask: Ignoring exception during close for org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector@525c7734 SAP DBTech JDBC: Object is closed:[ID 220714].
    at org.apache.sqoop.mapreduce.AsyncSqlRecordWriter.close(
    at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.close(
    at org.apache.hadoop.mapred.MapTask.closeQuietly(
    at org.apache.hadoop.mapred.MapTask.runNewMapper(
    at org.apache.hadoop.mapred.Child$
    at Method)
    at org.apache.hadoop.mapred.Child.main(
    Caused by: SAP DBTech JDBC: Object is closed:[ID 220714].

    Vick R

    I was looking at this thread. Although I can’t help right now,but I have a question regarding what drivers to use to connect HDFS to Hana. I am familiair with SAP SDA but when I tried to look at SAP Marketplace, I was unable to find the correct drivers.From you message above, it looks like you are using JDBC drivers? Are there are specific ones and how to get those?


    Felix Bekcer

    Hi Vick R,

    here you can find the client, studio and driver.

    Have fun 😉

    Vick R

    Thanks Felix and Have a Happy New Year

    Koelli Mungee

    Hi Suerte,

    We apologize for the delay. Assuming you are still seeing this problem, would it be possible to send in the full stack trace from the error. Is there a difference between the first two lines and the rest of the lines that are not being exported. Is there a timeout on the SAP Hana side you could be hitting?


The forum ‘Sqoop’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.