The Hortonworks Community Connection is now live. A completely rebuilt Q&A forum, Knowledge Base, Code Hub and more, backed by the experts in the industry.

You will be redirected here in 10 seconds. If your are not redirected, click here to visit the new site.

The legacy Hortonworks Forum is now closed. You can view a read-only version of the former site by clicking here. The site will be taken offline on January 31,2016

Hortonworks Sandbox Forum

upload files from local computer to sandbox

  • #38709
    Matt Workman
    Participant

    Hi, I am new to Hadoop and HDP so please forgive my basic question. I am trying to load a text file from my local computer to the HDP 1.3 Sandbox VM. I am trying to use PIG to do this. Here is the command:

    fs -copyFromLocal file://c:\temp\REST_Log.txt hdfs://sandbox:8020/user/hue/

    The error I am getting is here:

    copyFromLocal: Can not create a Path from an empty string

    So my question is how can I format my command to make this work? Is PIG the right thing to use to do this? Is there some other way I can accomplish this?

    My end goal is to create a script that will pull CSV files from a network path and load them into HDFS on my sandbox.

    Thanks for your help and time!!

    Matt

  • Author
    Replies
  • #38906
    Dave
    Moderator

    Hi Matt,

    CopyFromLocal is not going to work here as this is direct from the FileSystem. You would need to write a java programme which calls HDFS so you can run this locally (along with having the JAR files locally)
    You’re easiest way would be to write a pig script as you have done which loads the files from a directory and sftp the files from your local desktop to there.

    I hope this makes sense,

    Thanks

    Dave

The forum ‘Hortonworks Sandbox’ is closed to new topics and replies.

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.