Home Forums HDFS handle HDFS file

This topic contains 1 reply, has 2 voices, and was last updated by  Koelli Mungee 3 months, 2 weeks ago.

  • Creator
    Topic
  • #52078

    easyoups
    Participant

    Hi everybody

    I am facing a issue and I desesperately look for the solution since days…
    I try to write a HDFS file with HDP 2.0. However, despite I define a resource pointing to my Hadoop configuration, I am still writing in local filesystem.

    Has someone an idea for my mistake ?
    Here the part of my code that does not work as I want…


    // define variables for write and check destination
    Configuration config = new Configuration();
    config.addResource(new Path("/etc/hadoop/conf/core-site.xml"));
    config.addResource(new Path("/etc/hadoop/conf/hdfs-site.xml"));
    FileSystem hdfs = FileSystem.get(config);
    Path file = new Path(destFile);
    // Check if the file already exists
    if ((hdfs.exists(file))) {
    // file exists... exit
    System.out.println("File " + file + " exists");
    return;
    }
    OutputStream outStream = hdfs.create(file,
    new Progressable() {
    public void progress() {
    System.out.println("...");
    }
    }
    );

    Thank in advance…

Viewing 1 replies (of 1 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #53612

    Koelli Mungee
    Moderator

    Hi there,

    Can you try an example like this?

    Configuration conf = new Configuration(); Path dir = new Path(“results”);
    FileSystem fs = FileSystem.get(conf);
    if(!fs.exists(dir)) { fs.mkdirs(dir);
    }

    Let me know if this helps to get started,

    -Koelli

    Collapse
Viewing 1 replies (of 1 total)