Get fresh updates from Hortonworks by email

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

Sign up for the Developers Newsletter

Once a month, receive latest insights, trends, analytics information and knowledge of Big Data.

cta

Get Started

cloud

Ready to Get Started?

Download sandbox

How can we help you?

* I understand I can unsubscribe at any time. I also acknowledge the additional information found in Hortonworks Privacy Policy.
closeClose button
January 21, 2014
prev slideNext slide

How To Install Hadoop on Windows with HDP 2.0

Installing the Hortonworks Data Platform 2.0 for Windows is straightforward. Lets take a look at how to install a one node cluster on your Windows Server 2012 R2 machine.

To start, download the HDP 2.0 for Windows package. The package is under 1 GB, and will take a few moments to download depending on your internet speed. Documentation for installing a single node instance is located here. This blog post will guide you through that instruction set to get you going with HDP 2.0 for Windows!

Here’s an outline of the process you’ll work through to deploy:

  • Install the prerequisites
  • Deploy HDP on your single node machine
  • Start the services
  • Run smoke tests to validate the install

Install the Pre-requisites

You’ll now install Java, Python, and MSFT C++ run time. Windows Server 2012 already has the up to date .NET runtime, so you can skip that step.

Let’s download the C++ run time, and install that by double clicking the downloaded MSI.

Download Python 2.7.x, and double click the downloaded MSI to install the package.

Once you’ve installed, you’ll need to ensure HDP can find Python – by updating the PATH System Environment variable.

Go to Computer > Properties > Advanced System Settings > Environment variables. Then append the install path to Python, for example C:Python27, to this path after a ‘;’:

()

Verify your path is setup by entering a new Powershell or Command Prompt and typing: python, which should run the python interpreter. Type quit() to exit.

Setup Java, which you can get here. You will also need to setup JAVA_HOME, which Hadoop requires. Make sure to install Java to somewhere without a space in the path – “Program Files” will not work!

To setup JAVA_HOME, in Explorer > right click Computer > Properties > Advanced System Settings > Environment variables. Then setup a new System variable called JAVA_HOME that points to your Java install (in this case, C:javajdk1.6.0_31).

Install the MSI package

Now we have all the pre-requisites installed. The next step is to install the HDP 2.0 for Windows package.

Extract the MSI from the zip package you downloaded earlier. Open a Powershell prompt in Administrator (“Run as Administrator”) mode, and execute the MSI through this command:

   > msiexec /i "hdp-2.0.6.0.winpkg.msi"

The HDP Setup window appears pre-populated with the host name of the server, as well as default installation parameters. Now, complete the form with your parameters:

  • Set the Hadoop User Password. This enables you to log in as the administrative user and perform administrative actions. This must match your local Windows Server password requirements. We recommend a strong pasword. Note the password you set – we’ll use this later.
  • Check ‘Delete Existing HDP Data’. This ensures that HDFS will be formatted and ready to use after you install.
  • Check ‘Install HDP Additional Components’. Select this check box to install Zookeeper, Flume, and HBase as HDP services deployed to the single node server.
  • Set the Hive and Oozie database credentials. Set ‘hive’ for all Hive Metastore entries, and ‘oozie’ for all Oozie Metastore entries.
  • Select DERBY, and not MSSQL, as the DB Flavor in the dropdown selection. This will setup HDP to use an embedded Derby database, which is ideal for the evaluation single node scenario.

When you have finished setting the installation parameters, click ‘Install’ to install HDP.

The HDP Setup window will close, and a progress indicator will be displayed while the installer is running. The installation will take a few minutes – disregard the progress bar expected time display.

The MSI installer window will display an info prompt when the installation is finished and successful.

Start the services and run a jobs

Once the install is successful, you will start the HDP services on the single node.

Open a command prompt, and navigate to the HDP install directory. By default, the location is “C:hdp”, unless you set a different location:

   > cd C:hdp

   > start_local_hdp_services

Validate the install by running the full suite of smoke tests. It’s easiest to run the smoke tests as the HDP super user: ‘hadoop’.

In a command prompt, switch to using the ‘hadoop’ user:

   > runas /user:hadoop cmd

When prompted, enter the password you had set up during install.

Run the provided smoke tests as the hadoop user to verify that the HDP 2.0 services work as expected:

   > cd C:hdp

   > Run-SmokeTests hadoop

This will fire up a Mapreduce job on your freshly set up cluster. If it fails the first time, try running it again with the same command Run-SmokeTests hadoop.

Congratulations, you are now Hadooping on Windows!

If you’d like to learn more about Hadoop, check out the Hortonworks Sandbox, a virtual machine for you to learn Hadoop and sign up for our free ‘Learn Hadoop in 2 Weeks‘ guided tutorial series.

Tags:

Comments

Antonio Paternina says:
Your comment is awaiting moderation.

I followed the steps in windows server 2012, but I get the installation inition has’ installation failed ‘and I downloaded the 2.0 hdp two occasions thinking it was the executable, but keeps coming’ installation ‘failed’.

Lucho F says:
Your comment is awaiting moderation.

Hello, I’m trying to install hadoop 2.0 in a windows 2012 DC virtual machine but I got all the time problems with hadoop user password so I can’t proceed with installation.

I have found this piece of code in the hadoop lo file. It seems that hadoop user does not meet the requirements to be accepted, but I’m using capital letters and number and my password is more than 6 letter so I don’t know what is happening. When I try one more time I got the same problem. It seems that the first password that I’ve entered with no capital letters and numbers has been saved somewhere.

Could you please help me!?

CREATE-USER: Creating user hadoop
CREATE-USER: Setting password for hadoop
CREATE-USER FAILURE: Exception calling “SetInfo” with “0” argument(s): “The password does not meet the password policy requirements. Check the minimum password length, password complexity and password history requirements.

At C:HadoopInstallFilesHadoopPackageshdp-2.0.6.0-winpkgscriptscreate_hadoop_user.ps1:88 char:9
+ $user.SetInfo()
+ ~~~~~~~~~~~~~~~

Best regards,
Lucho

Lucho F says:
Your comment is awaiting moderation.

Hello, I was struggling with the installation of the package and I found an error when the instalation package tries to install OOZIE. This below is the error message
OOZIE: Copy Oozie additional Jars
OOZIE: Calling Setup script to add Hadoop libs to the generated oozie.war file
OOZIE FAILURE: Unable to execute ‘jar xvf’. Error (1)
At D:hdpoozie-4.0.0.2.0.6.0-0009oozie-win-distrobinoozie-setup.ps1:129 char:9
+ throw “Unable to execute ‘jar xvf’. Error ($LASTEXITCODE)”
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Could you please let me know what should I do to fix it?
Thanks in advanced for your help!
Lucho Farje

Jan says:
Your comment is awaiting moderation.

If your locale uses , instead of . in numbers, you have to change the Get-TimeStamp function in Run-SmokeTests.ps1 to:

function Get-TimeStamp
{
(Get-Date -UFormat(‘%s’)) -Replace(“.”,””) -Replace(“,”,””)
}

@Rohit, you should integrate this by default.

Mike says:
Your comment is awaiting moderation.

How can I add another node to this singlenode install? I cannot a find set of instructions on how to add a node to the cluster on widnows?

Sven Bleckwedel says:
Your comment is awaiting moderation.

Hello,

> The MSI installer window will display an info prompt when the installation is finished and successful

Well, something gone wrong when trying HDP Sandbox, only because I tried to run the WMWare player appliance, without success (I´m trying to download it again). Tried to install the HDP for Windows, without success either…

Please advice,
Sven

David Goyal says:
Your comment is awaiting moderation.

Need help. I have tried installing and uninstalling HDP 2.0 more than 20 times during last 5 days. So far it installs successfully but SmokeTests fail. I used administrative access for installation.

1. I am on Windows 7, JDK 1.7.0_02 & phthon 27.
2. hdp-2.0.6.0.winpkg.install.log shows successful installation and does not show any errors.
3. Firewall is off for everything
4. I am on private network
5. Only Smoke tests passed: Sqoop, Oozie
6. Extracts of SmokeTest results..
14/02/07 10:57:01 WARN util.NativeCodeLoader: Unable to load native-hadoop libra
ry for your platform… using builtin-java classes where applicable
copyFromLocal: Call From MALT-7539/192.168.2.7 to MALT-7539:8020 failed on conne
ction exception: java.net.ConnectException: Connection refused: no further infor
mation; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
Run-HadoopSmokeTest : Error copying the input file for the Hadoop smoke test
At line:1 char:20
+ Run-HadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest

14/02/07 10:57:06 WARN util.NativeCodeLoader: Unable to load native-hadoop libra
ry for your platform… using builtin-java classes where applicable
14/02/07 10:57:06 INFO client.RMProxy: Connecting to ResourceManager at MALT-753
9/192.168.2.7:8032

Kevin Iddles says:
Your comment is awaiting moderation.

Nice guide. I think you should make it clear that JDK not JRE is required for the install to complete.

I had the same error described at https://hortonworks.com/community/forums/topic/hdp20-install-fails-with-no-logs/

Now to test !

Martin Jensen says:
Your comment is awaiting moderation.

Worth noting that the JAVA_HOME can not contains spaces and the default path that the java sdk installer uses is “c:program filesjava…”.

Mark Granger says:
Your comment is awaiting moderation.

Hi there. I followed all the instructions to the letter and the install still says: “Installation failed. Please see installation log for details: <> This is great but no log is being created in the destination so my troubleshooting options are very limited. BTW: I’m using Win Server 2012 R2.

jay says:
Your comment is awaiting moderation.

Can I Install this software in windows 7 (home premium) machine.?

Anoop Rathi says:
Your comment is awaiting moderation.

Try removing the space in the folder name. It worked for me. For Program Files, it was only taking “Files” and searching for JDK.

Forrest says:
Your comment is awaiting moderation.

I’m having trouble post-install starting the services: NameNode and DataNode wouldn’t start.

Here’s the logs: https://gist.github.com/anonymous/9f62d7805711f3915bbe.
The main error is java.net.URISyntaxException: Illegal character in opaque part at index 2: C:LabAppshortondatahdfsdn

Is this a known issue?

Abdul Khan says:
Your comment is awaiting moderation.

I am trying to install HDP on windows but It keeps giving me an error message on the last step that the Password doesnt match the system requirements defined in system policies.

Tom says:

Windows 7 Ultimate, 64-bit. JDK 1.8.0.11, Python34 – installation fails, no log written, no information in the system log (beyond the advise to check the non-existent log file.
To me, free was too expensive this time, wasting about a day to troubleshoot.

David says:
Your comment is awaiting moderation.

Required java version is 2.7.x

David says:
Your comment is awaiting moderation.

sorry 🙂 required Python version is 2.7

dalal says:

I installed the Hortonworks Data Platform 2.0 for Windows. The tutorial stops at Run-SmokeTests hadoop. After that there is no information / guide to what we can do on this windows installation .All tutorials are related to virtual sandbox. Is it possible to provide tutorials for Hortonworks Data Platform 2.0 for Windows similar to what you have for virtual sandbox https://hortonworks.com/hadoop-tutorial/hello-world-an-introduction-to-hadoop-hcatalog-hive-and-pig/ .

Disha says:

I am getting a pop-up saying “JAVA_HOME” must be set even when I have set it with a path that has no spaces.
Please help!!

VU NGUYEN TRAN says:
Your comment is awaiting moderation.

You have to set JAVA_HOME at Command line : Syntax setx /M JAVA_HOME java path

Martin Serrano says:
Your comment is awaiting moderation.

I had the same issue and realized that it was because I had JAVA_HOME set as a user environment variable rather than as a system environment variable.

Disha says:

I am getting a pop-up saying “JAVA_HOME” must be set even when I have set it with a path that has no spaces.
Please help!!

Juan Luis Rivero says:

Hello guys,

I am trying to install hdp 2.3.1.0 for windows in a single node. I have already installed Python under Anaconda pck. Anaconda directory is in PATH env. variable. If I run python from a cmd command, python 2.7.7 executes and point to anaconda from any directory I run. Also I have installed Java.

The fact is that I get an error: No python executable found in path. I am lost…. What could be happened?

Thanks a lot.

Regards. JL

Amitesh says:
Your comment is awaiting moderation.

Check this out!
http://mybigdataconcepts.blogspot.in/2014/09/hadoop-setup-and-understanding.html

nomad says:

Exactly where did we download a .zip file? I didn’t see one, also can’t find “hdp-2.0.6.0.winpkg.msi”
Nothing is working for me or the others here. What a poor tutorial and waste of time

Neeraj says:
Your comment is awaiting moderation.

Please try this http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.5-Win/index.html

David says:
Your comment is awaiting moderation.

In your browser go to downloads (ctrl+j)

Viral Parikh says:
Your comment is awaiting moderation.

Is there a similar thing for Mac OS? I want to install single node on my Mac. I dont want to use the Sandbox VM.

Pramod AG says:
Your comment is awaiting moderation.

Getting below errors while running smoke test
—————————————————————————————————
C:hdp>Run-SmokeTests hadoop
Hadoop smoke test – wordcount using hadoop.cmd file
DEPRECATED: Use of this script to execute hdfs command is deprecated.
Instead use the hdfs command for it.
copyFromLocal: Call From andromeda/192.168.1.34 to ANDROMEDA:8020 failed on conn
ection exception: java.net.ConnectException: Connection refused: no further info
rmation; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
Run-HadoopSmokeTest : Error copying the input file for the Hadoop smoke test
At line:1 char:20
+ Run-hadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest
————————————————————–
——
Run-HadoopSmokeTest : Hadoop Smoke Test: FAILED
At line:1 char:20
+ Run-hadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest

Prasanna Reddy says:
Your comment is awaiting moderation.

even I installed the Hortonworks Data Platform 2.0 for Windows. The tutorial stops at Run-SmokeTests hadoop. After that what I need to do?There is no information regarding windows installation.help me out.

aman says:
Your comment is awaiting moderation.

Does this work for windows 8.1?
Has anyone tried it yet?

Rajalingam says:

Hi guys , I also tried and getting errors like path for python not able to find .
So anybody have updated on that.

tarun mirani says:

hey rohit,
thank you for your detailed post.

i have installed everything successfully. but after that can you tell me the working of this ? how should i make use of this ?
actually i need to make this hadoop working on my computer lab of my college. so i need some Real details.

Alshafaraz Gazi says:

The hdp file size for windows is not less then 1GB its 5.8GB..

srp says:

will this work for windows 10

Phani says:

Getting the following error on trying to run the smoketests as mentioned in the page.

16/11/11 10:58:10 INFO mapreduce.Job: map 0% reduce 0%
16/11/11 10:58:10 INFO mapreduce.Job: Job job_1477935038448_0015 failed with sta
te FAILED due to: Application application_1477935038448_0015 failed 2 times due
to AM Container for appattempt_1477935038448_0015_000002 exited with exitCode:
-1000 due to: Could not find any valid local directory for nmPrivate/container_1
477935038448_0015_02_000001.tokens
.Failing this attempt.. Failing the application.
16/11/11 10:58:10 INFO mapreduce.Job: Counters: 0
Run-HadoopSmokeTest : Hadoop Smoke Test: FAILED
At line:1 char:20
+ Run-hadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest

Please let me know how can I have this resolved.

I have Windows 2008 Operating System in which I tried above

Leave a Reply

Your email address will not be published. Required fields are marked *

If you have specific technical questions, please post them in the Forums