handle HDFS file

to create new topics or reply. | New User Registration

This topic contains 1 reply, has 2 voices, and was last updated by  Koelli Mungee 1 year, 2 months ago.

  • Creator
  • #52078


    Hi everybody

    I am facing a issue and I desesperately look for the solution since days…
    I try to write a HDFS file with HDP 2.0. However, despite I define a resource pointing to my Hadoop configuration, I am still writing in local filesystem.

    Has someone an idea for my mistake ?
    Here the part of my code that does not work as I want…

    // define variables for write and check destination
    Configuration config = new Configuration();
    config.addResource(new Path("/etc/hadoop/conf/core-site.xml"));
    config.addResource(new Path("/etc/hadoop/conf/hdfs-site.xml"));
    FileSystem hdfs = FileSystem.get(config);
    Path file = new Path(destFile);
    // Check if the file already exists
    if ((hdfs.exists(file))) {
    // file exists... exit
    System.out.println("File " + file + " exists");
    OutputStream outStream = hdfs.create(file,
    new Progressable() {
    public void progress() {

    Thank in advance…

Viewing 1 replies (of 1 total)

You must be to reply to this topic. | Create Account

  • Author
  • #53612

    Koelli Mungee

    Hi there,

    Can you try an example like this?

    Configuration conf = new Configuration(); Path dir = new Path(“results”);
    FileSystem fs = FileSystem.get(conf);
    if(!fs.exists(dir)) { fs.mkdirs(dir);

    Let me know if this helps to get started,


Viewing 1 replies (of 1 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.