Home Forums Sqoop Sqoop Incremental import failing pulling data from SQL Server

This topic contains 8 replies, has 3 voices, and was last updated by  Robert Molina 1 year ago.

  • Creator
    Topic
  • #39169

    Rick Buchan
    Member

    In the 2.0 Sandbox, we are getting an error attempting to execute a sqoop incremental import using the lastmodified parameter when pulling data from SQL Server. The same command works fine using the append parameter.

    13/10/02 11:56:55 WARN manager.SqlManager: SQL exception accessing current timestamp: com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near ‘)’.
    13/10/02 11:56:55 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Could not get current time from database

    Here is the command which was issued:

    [root@sandbox ~]# sqoop import –connect “jdbc:sqlserver://00.00.00.00:14433;database=xxxxxx” –username xxxxx –password xxxxxx –driver com.microsoft.sqlserver.jdbc.SQLServerDriver –table “dbo.xxxxxx” –target-dir “xxxxxxx” –incremental “lastmodified” –check-column “LAST_UPD” –last-value “2013-09-20 04:02:03″ –split-by ROW_ID –m 1

    Log also included this statement:
    13/10/02 11:56:46 WARN sqoop.ConnFactory: Parameter –driver is set to an explicit driver however appropriate connection manager is not being set (via –connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.

    Is there a way around this error?

Viewing 8 replies - 1 through 8 (of 8 total)

The topic ‘Sqoop Incremental import failing pulling data from SQL Server’ is closed to new replies.

  • Author
    Replies
  • #39418

    Robert Molina
    Moderator

    Hi Rick,
    What happens if you add the –connection-manager as well?

    Regards,
    Robert

    Collapse
    #39416

    Rick Buchan
    Member

    Hi Koelli,

    I’ve also tried using the DB name associated with the table ‘common.dbo.W_SBL_S_LST_OF_VAL’ but was still getting the invalid object name.

    When I remove both common and dbo and just use the table name (and still without the –driver option), I get a different error related to the ImportTool:

    Yes, I do have the jdbc driver in /usr/lib/sqoop/lib

    When I tried running the command without the –driver and with the db name included in the — table option, I was still getting an Invalid Option Name error. I was getting the same error, though, directly in SQL Server Management Studio. So, I tried without the db name and without the table owner of dbo. (which worked in Management Studio). Using this combination, I got a error with the ImportTool:

    13/10/04 16:34:49 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM [W_SBL_S_LST_OF_VAL] AS t WHERE 1=0
    13/10/04 16:34:49 ERROR tool.ImportTool: Imported Failed: Attempted to generate class with no columns!

    Collapse
    #39413

    Rick Buchan
    Member

    Hi Koelli,

    Got a new error message after removing the –driver parameter:

    13/10/04 11:48:45 ERROR manager.SqlManager: Error executing statement: com.microsoft.sqlserve.jdbc.SQLServerException: Invalid object name ‘dbo.W_SBL_S_LST_OF_VAL’.
    com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name ‘dbo.W_SBL_S_LST_OF_VAL’

    Note: we tried removing the –driver option on the command to do the full extract, and we get the same error. But, with the –driver option, the full extract is working OK.

    Thanks,
    Rick

    Collapse
    #39412

    Koelli Mungee
    Moderator

    Thanks Rick, can you let us know if it worked without the -driver parameter?

    Regards
    Koelli

    Collapse
    #39410

    Rick Buchan
    Member

    Thanks Koelli.

    The generated SQL does not show the syntax being used in SQL Server to check the timestamp:

    13/10/04 11:18:43 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM dbo.W_S_LST_OF_VAL AS t WHERE 1=0
    13/10/04 11:18:43 ERROR manager.SqlManager: SQL exception accessing current timestamp: com.microsoft.sqlserver.jdbc.SQLServerException: Incorrect syntax near ‘)’.

    From a post on the Cloudera site, it looks like this was an issue in version 1.2 of Sqoop, but that it should be fixed in version 1.4:

    https://issues.cloudera.org/browse/SQOOP-229?page=com.atlassian.jira.plugin.system.issuetabpanels%3Achangehistory-tabpanel

    Collapse
    #39406

    Koelli Mungee
    Moderator

    Hi Rick,

    Can you please try the same command without the -driver parameter and that should work.

    Regards
    Koelli

    Collapse
    #39402

    Koelli Mungee
    Moderator

    Rick,

    Can you get us the SQL that is running as well so we can take a look?

    Thanks,
    Koelli

    Collapse
    #39401

    Koelli Mungee
    Moderator

    Hi Rick,

    We are going to test this in-house and get back to you.

    Regards
    Koelli

    Collapse
Viewing 8 replies - 1 through 8 (of 8 total)