HDFS Forum

No FileSystem for scheme: hdfs

  • #51883
    Param Dutta
    Participant

    Hi,
    I have a test program that uses FileSystem api to CRUD files on hdfs. This works well with HDP1.3, but when I use it with HDP2.0 (sandbox), I get the exception “java.io.IOException: No FileSystem for scheme: hdfs”. Since there is no hadoop-core.jar anymore, I have tried different combinations of jars to make it work. Currently, I have the following (and some of the related dependencies not listed here), but I can’t make it work yet :
    hadoop-annotations-2.2.0.jar
    hadoop-mapreduce-client-core-2.2.0.jar
    hadoop-yarn-common-2.2.0.jar
    hadoop-auth-2.2.0.jar
    hadoop-yarn-api-2.2.0.jar
    hadoop-yarn-server-common-2.2.0.jar
    hadoop-common-2.2.0.jar

    Thanks in advance for any help.

    Best,
    Param

    java.io.IOException: No FileSystem for scheme: hdfs
    at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2421)
    at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2428)
    at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:88)
    at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2467)
    at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2449)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:367)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:166)

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #52187
    easyoups
    Participant

    Hi,

    I met the same error. The reason was that some jar were missing in my CLASSPATH (hadoop-hdfs’s ones).
    Hope this will help you…


    export CLASSPATH=.:/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop-hdfs/*:/usr/lib/hadoop-hdfs/lib/*:/usr/share/java/*

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.