Deleting data in MySql using Hive/Pig

to create new topics or reply. | New User Registration

This topic contains 1 reply, has 2 voices, and was last updated by  Thejas Nair 1 year, 5 months ago.

  • Creator
    Topic
  • #44575

    Can we delete data in a external database(MySql etc) from Hadoop using Hive/Pig? I am trying to do Archival usecase as a POC for using Hadoop in our technology stack. I have to store data older than 6years into a hive table(incrementel load) and delete same data from the source table(in MySql). Kindly help on this, I am pity frustrated searching the web and getting resulta for importing data into Hadoop everytime??????

Viewing 1 replies (of 1 total)

You must be to reply to this topic. | Create Account

  • Author
    Replies
  • #44614

    Thejas Nair
    Participant

    Pig or Hive won’t be the right tool for deleting data from mysql. You are better off triggering a separate mysql sql query for that.
    You can take a look at project falcon – http://hortonworks.com/blog/project-falcon-tackling-hadoop-data-lifecycle-management-via-community-driven-open-source/ . It might help with this use case.

    Collapse
Viewing 1 replies (of 1 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.