Hive / HCatalog Forum

Hive query creates temp folders

  • #78830
    Igor Kuzmenko

    Hello, i’m using insertion in table using this query:

    from temp_cdr
    insert into table data_cdr partition(dt)
    select temp_cdr.*, cast(concat(substring(endtime, 7, 4), substring(endtime, 4, 2), substring(endtime, 1, 2)) as int) as dt;

    Query works fine creating missing partitions and also creating folder .hive-staging_hive_2015-09-21_17-16-57_118_6915829821044291073-1 in table directory. It seems like some sort of temp folder.

    Since I’m going to use such insertions a lot, there would be many of those temp dirs. Why hive creating it without deletion. Is it realy temp dir? Can I remove it without consequences.

to create new topics or reply. | New User Registration

  • Author
  • #78832
    Thejas Nair

    Do you have HDFS encryption enabled or are you using ACID ?
    What version of HDP is this ?

    Igor Kuzmenko

    I’m using HDP 2.3. HDFS encription disabled.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.