Home Forums HDFS No FileSystem for scheme: hdfs

This topic contains 1 reply, has 2 voices, and was last updated by  easyoups 4 months, 3 weeks ago.

  • Creator
    Topic
  • #51883

    Param Dutta
    Participant

    Hi,
    I have a test program that uses FileSystem api to CRUD files on hdfs. This works well with HDP1.3, but when I use it with HDP2.0 (sandbox), I get the exception “java.io.IOException: No FileSystem for scheme: hdfs”. Since there is no hadoop-core.jar anymore, I have tried different combinations of jars to make it work. Currently, I have the following (and some of the related dependencies not listed here), but I can’t make it work yet :
    hadoop-annotations-2.2.0.jar
    hadoop-mapreduce-client-core-2.2.0.jar
    hadoop-yarn-common-2.2.0.jar
    hadoop-auth-2.2.0.jar
    hadoop-yarn-api-2.2.0.jar
    hadoop-yarn-server-common-2.2.0.jar
    hadoop-common-2.2.0.jar

    Thanks in advance for any help.

    Best,
    Param

    java.io.IOException: No FileSystem for scheme: hdfs
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:166)

Viewing 1 replies (of 1 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #52187

    easyoups
    Participant

    Hi,

    I met the same error. The reason was that some jar were missing in my CLASSPATH (hadoop-hdfs’s ones).
    Hope this will help you…


    export CLASSPATH=.:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-hdfs/lib/*:/usr/share/java/*

    Collapse
Viewing 1 replies (of 1 total)