Home Forums Hortonworks Sandbox Hadoop fs

This topic contains 4 replies, has 2 voices, and was last updated by  Jan Leonhard 5 months, 3 weeks ago.

  • Creator
    Topic
  • #55985

    Jan Leonhard
    Participant

    Hi!

    I’ve created a Hive Table EDW_RETAIL…

    Added some data by

    LOAD DATA INPATH ‘/user/hue/BACK_130530_0809_EDW_RETAIL_BEWEGUNGEN_PART53.DEL’ INTO TABLE default.edw_retail_bewegungen;
    LOAD DATA INPATH ‘/user/hue/BACK_130530_0809_EDW_RETAIL_BEWEGUNGEN_PART54.DEL’ INTO TABLE default.edw_retail_bewegungen;
    LOAD DATA INPATH ‘/user/hue/BACK_130530_0809_EDW_RETAIL_BEWEGUNGEN_PART55.DEL’ INTO TABLE default.edw_retail_bewegungen;
    … etc.

    Worked fine – content grows by every file added.

    So i tried to find out, how hdfs/HIVE manages the table in the hadoop file system.

    but when i try this on the console:

    hadoop fs -ls

    the command is executed successfully, but i receive no output.

    Any advice how i can find out, if the table is stored in one hadoop file?

    best regards

    jan

Viewing 4 replies - 1 through 4 (of 4 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #56839

    Jan Leonhard
    Participant

    Hello again :)

    thanx again! That worked fine, too.

    i now found that a hive table does not refer to its own hadoop file, so my table is stored like this:

    Permission Owner Group Size Replication Block Size Name
    -rwxrwxrwx hue hdfs 17.78 KB 1 32.63 MB BACK_130530_0756_EDW_RETAIL_BEWEGUNGEN_PART51.DEL
    -rwxr-xr-x hue hue 3.21 MB 1 32.63 MB BACK_130530_0804_EDW_RETAIL_BEWEGUNGEN_PART52.DEL
    -rwxr-xr-x hue hue 30.06 MB 1 32.63 MB BACK_130530_0809_EDW_RETAIL_BEWEGUNGEN_PART53.DEL
    -rwxr-xr-x hue hue 39.41 MB 1 32.63 MB BACK_130530_0813_EDW_RETAIL_BEWEGUNGEN_PART54.DEL
    -rwxr-xr-x hue hue 23.78 MB 1 32.63 MB BACK_130530_0819_EDW_RETAIL_BEWEGUNGEN_PART55.DEL

    so it is stored in 6 Blocks, although 3 would be enough.

    Why is it, that a hive table does not generate a new hadoop file, so that it inserts new data at the end of the file? I might have read that this is possible in Hadoop 2.0. ISn’t it?

    highest regards

    /jan

    Collapse
    #56809

    iandr413
    Moderator

    HI Jan,
    Running hadoop fs -ls with no path will assume the home directory of the user you are logged in as. For example, if I am in as root, it would show me the contest of /user/root

    To get the highest level you would need to execute hadoop fs -ls /

    Ian

    Collapse
    #56808

    Jan Leonhard
    Participant

    Hi Ian!

    That worked fine! Thank you very much!

    Why don’t i get a result without giving a path? Shouldn’t i get the contents of the highest level?

    best wishes

    jan

    Collapse
    #56614

    iandr413
    Moderator

    Hi Jan,
    Can you try the following hadoop fs -ls /user/hue?

    Ian

    Collapse
Viewing 4 replies - 1 through 4 (of 4 total)