HDP on Windows – Installation Forum

Launch From Windows To Linux Cluster

  • #53780
    Ian Cadieu
    Participant

    I’m trying to get some examples running. I got HDP2.2 installed and can submit jobs but they all fail on container startup with bash code 127 (command not found). When inspecting the generated launch_container.sh script, it looks like {{JAVA_HOME}} isn’t getting replaced properly. I’ve tried everything I can think of. Any thoughts? What do I need to do in order to make it set this variable?

    Hadoop Command (with simplified map script, but even this reproduces the error):
    jar hadoop-streaming-2.3.0-cdh5.0.0.jar -input “/data/test.csv” -output “/output.txt” -mapper “cat launch_container.sh” -numReduceTasks 0 -verbose
    Snippet from Launch_Container.sh:
    exec /bin/bash -c “{{JAVA_HOME}}/bin/java -Dlog4…
    Error from log file:
    /bin/bash: {{JAVA_HOME}}/bin/java: No such file or directory

to create new topics or reply. | New User Registration

  • Author
    Replies
  • #53946
    Dave
    Moderator

    Hi Ian,

    Do you mean HDP 2.1.2 ?
    Have you set JAVA_HOME correctly to C:\java in your system environment variables and added java to the PATH ?
    Also hadoop-streaming-2.3.0-cdh5.0.0.jar is a Cloudera jar.
    Did you install HDP or CDH?

    Thanks

    Dave

    #53992
    Ian Cadieu
    Participant

    So I got this working and figured it out. The issue is you need Yarn 2.4 running on the server for it to work properly (pretty much any distribution I think).

    I had been trying with the cloudera streaming jar, and the hortonworks streaming jar thinking it’s a client side problem, but it actually required a server side upgrade to be compliant. Debugging container startup issues are a bit of a nightmare though.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.