HBase Forum

When hfiles are written to hdfs

  • #57194
    Craig Clarks

    Hi ,

    When hfiles are written to hdfs? .

    My understanding is all the minor compaction and major compaction are local to hbase ( regional servers). When this hfiles will be written to hdfs?

    Question 2. Does regional servers should always reside where datanodes are location?


to create new topics or reply. | New User Registration

  • Author
  • #57246
    SambaSivaRao Y

    Hi Craig,

    As of I know Hfiles are files for storing the data into Hbase.
    If your question is about data commit into Hbase, Can you share your sample code which you wrote to write the Hfiles and to commit ?


    Craig Clarks

    Hi Siva,

    what you send is correct – Hfiles are files for storing the data into Hbase.

    My understand was, whenever mem-store fills, it will spill to hfiles. These hfiles will be subject to minor and major compactions, subsequently. Does the journey of hfiles ends at major compaction?

    or will these major compaction files will be moved to hdfs ?

    or hfiles are directly written to hdfs from memstore ? In this case all the hfiles have default replication of hdfs?

    Some how I am not clear on the documentation and expecting a good explanation from this froum


You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.