WebHDFS error when upload file

to create new topics or reply. | New User Registration

Tagged: ,

This topic contains 6 replies, has 4 voices, and was last updated by  tedr 1 year, 10 months ago.

  • Creator
  • #24880


    I’m getting the following error message when I upload a File to HDFS using WebHDFS that has spaces on it, (no spaces works great):

    I’m basically using URLEncode for the path of the HTTP request, the Namenode seems to accept this but getting error when make the second Request (PUT) to the Datanode from the initial redirect of the Namenode.

    Am I missing something?

    2013-05-08 09:57:52,989 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs cause:java.net.URISyntaxException: Illegal character in path at index 51: webhdfs://HadoopMaster.demo.vm:50070/usr/UT/File1MB (1).txt
    2013-05-08 09:57:52,989 WARN org.apache.hadoop.hdfs.web.resources.ExceptionHandler: INTERNAL_SERVER_ERROR
    java.lang.reflect.UndeclaredThrowableException: Unknown exception in doAs
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1191)
    at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods.put(DatanodeWebHdfsMethods.java:168)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    Caused by: java.security.PrivilegedActionException: java.net.URISyntaxException: Illegal character in path at index 51: webhdfs://HadoopMaster.demo.vm:50070/usr/UT/File1MB (1).txt
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:396)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1178)
    … 40 more
    Caused by: java.net.URISyntaxException: Illegal character in path at index 51: webhdfs://HadoopMaster.demo.vm:50070/usr/UT/File1MB (1).txt
    at java.net.URI$Parser.fail(URI.java:2810)
    at java.net.URI$Parser.checkChars(URI.java:2983)
    at java.net.URI$Parser.parseHierarchical(URI.java:3067)
    at java.net.URI$Parser.parse(URI.java:3015)
    at java.net.URI.(URI.java:577)
    at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:198)
    at org.apache.hadoop.hdfs.server.datanode.web.resources.DatanodeWebHdfsMethods$1.run(DatanodeWebHdfsMethods.java:168)
    … 43 more

Viewing 6 replies - 1 through 6 (of 6 total)

You must be to reply to this topic. | Create Account

  • Author
  • #25374


    Hi Ramiro,

    Yes we will log an issue for this, after making sure it’s not already there. Also I will continue to look for a workaround and will get back to you when I find one.



    Hi tedr,

    I have tried escaping spaces with ‘\’, and it looks like the file name now is correct on the HDFS, however I got the file in a sub folder and I still have an Internal Server error response from the HDFS, but there is no error on the NameNode.log.

    for example
    http://Namenode:50070/usr/UT/File1MB 1.txt – > http://Namenode:50070/usr/UT/File1MB\ 1.txt
    and I ended up having the file under a File1MB subfolder with a file name 1.txt

    # hadoop fs -ls /usr/UT/File1MB/
    Found 1 items
    -rw-r–r– 3 hdfs hdfs 1050626 2013-05-10 13:30 /usr/UT/File1MB/ 1.txt.tmp

    However I think the escape shouldn’t be done by the client, but the Hadoop name node, since HTTP web request have standards and RFC when it comes to URI and URL encoding handling .

    So my question for you guys is:
    Are you guys going to create a Issue so it can be fix in future versions? is there is a workaround I can use.

    Let me know and also let me know there is anything else I can help. cheers.



    Hi Ramiro,

    Have you tried escaping the spaces with ‘\’ ?



    Hi Seth, yes i can read the file using command line, the file seems to be saved correctly, I can even download it using webHDFS, it just the error code from the request that comes back. Not sure what is happening, also not sure if the file integrity is correct among the cluster. I test with a small file.


    Seth Lyubich

    Hi Ramiro,

    We were able to reproduce this. What is interesting is that we can actually see and read file from command line even though WebHDFS exception happened during put. Can you please check to see if you actually can read file from command line after this exception occurred?



    Here is the Request/Response sequence:
    PUT http://hadoopmaster.demo.vm:50070/webhdfs/v1/usr/UT/File1MB%20(1).txt?user.name=hdfs&op=CREATE HTTP/1.1
    Host: hadoopmaster.demo.vm:50070
    Content-Length: 0

    Content-Type: application/octet-stream
    Expires: Thu, 01-Jan-1970 00:00:00 GMT
    Set-Cookie: hadoop.auth=”u=hdfs&p=hdfs&t=simple&e=1368061072186&s=S8h6fTDohs5HZ8Uanm/ghaFGavs=”;Path=/
    Location: http://HadoopNode1.demo.vm:50075/webhdfs/v1/usr/UT/File1MB%20(1).txt?op=CREATE&user.name=hdfs&overwrite=false
    Content-Length: 0
    Server: Jetty(6.1.26)
    PUT http://hadoopnode1.demo.vm:50075/webhdfs/v1/usr/UT/File1MB%20(1).txt?op=CREATE&user.name=hdfs&overwrite=false HTTP/1.1
    Host: hadoopnode1.demo.vm:50075
    Content-Length: 1050626
    Expect: 100-continue

    HTTP/1.1 500 Internal Server Error
    Content-Type: application/json
    Transfer-Encoding: chunked
    Server: Jetty(6.1.26)

    {“RemoteException”:{“exception”:”UndeclaredThrowableException”,”javaClassName”:”java.lang.reflect.UndeclaredThrowableException”,”message”:”Unknown exception in doAs”}}

Viewing 6 replies - 1 through 6 (of 6 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.