Installation signals successful but nothing installed

Hi,
I’m trying to install HDP under Windows Server 2012 x64. I followed the instructions and the installation signals that everything is successful but when I look at the target folder, I just find an empty one.
Here’s my configuration file and the last few lines of my log.

#Log directory
HDP_LOG_DIR=c:\hadoop\logs

#Data directory
HDP_DATA_DIR=c:\hadoop\data

#Hosts
NAMENODE_HOST=localhost
SECONDARY_NAMENODE_HOST=localhost
JOBTRACKER_HOST=localhost
HIVE_SERVER_HOST=localhost
OOZIE_SERVER_HOST=localhost
TEMPLETON_HOST=localhost
SLAVE_HOSTS=localhost

#Database host
# DB_FLAVOR=derby
# DB_HOSTNAME=DB_myHostName

#Hive properties
# HIVE_DB_NAME=hive
# HIVE_DB_USERNAME=hive
# HIVE_DB_PASSWORD=hive

#Oozie properties
OOZIE_DB_NAME=oozie
OOZIE_DB_USERNAME=oozie
OOZIE_DB_PASSWORD=oozie

MSI (s) (F8:C8) [12:25:30:430]: Custom Action Manager thread ending.
MSI (c) (1C:B0) [12:25:30:430]: Back from server. Return value: 0
MSI (c) (1C:B0) [12:25:30:430]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied. Counter after decrement: -1
MSI (c) (1C:B0) [12:25:30:430]: PROPERTY CHANGE: Deleting SECONDSEQUENCE property. Its current value is ’1′.
Action ended 12:25:30: ExecuteAction. Return value 1.
Action ended 12:25:30: INSTALL. Return value 1.
=== Logging stopped: 3/5/2013 12:25:30 ===
MSI (c) (1C:B0) [12:25:30:430]: Note: 1: 1707
MSI (c) (1C:B0) [12:25:30:430]: Note: 1: 2262 2: Error 3: -2147287038
MSI (c) (1C:B0) [12:25:30:430]: Note: 1: 2262 2: Error 3: -2147287038
MSI (c) (1C:B0) [12:25:30:430]: Product: Hortonworks Data Platform 1.1.0 for Windows — Installation completed successfully.

MSI (c) (1C:B0) [12:25:30:430]: Windows Installer installed the product. Product Name: Hortonworks Data Platform 1.1.0 for Windows. Product Version: 1.1.0. Product Language: 1033. Manufacturer: Hortonworks. Installation success or error status: 0.

MSI (c) (1C:B0) [12:25:30:430]: Grabbed execution mutex.
MSI (c) (1C:B0) [12:25:30:430]: Cleaning up uninstalled install packages, if any exist
MSI (c) (1C:B0) [12:25:30:430]: MainEngineThread is returning 0
=== Verbose logging stopped: 3/5/2013 12:25:30 ===

Topic

[tedrMember]

Hi Santosh,

It would appear that since you are getting a WriteErrorException as the reason for failure on these tests that you may not have the correct permissions. Are you running the smoke test script as an administrator?

Thanks,
Ted.

[Santosh UmaraniMember]

HDP got installed successfully with the services running up and running. After running Run -SmokeTests command, I’m getting the following error. No update in log file.

C:\>C:\hdp\hadoop\Run-SmokeTests.cmd
Hadoop smoke test – wordcount using hadoop.cmd file
The system cannot find the path specified.
Run-HadoopSmokeTest : Error copying the input file for the Hadoop smoke test
At line:1 char:20
+ Run-HadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest

The system cannot find the path specified.
Run-HadoopSmokeTest : Hadoop Smoke Test: FAILED
At line:1 char:20
+ Run-HadoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HadoopSmokeTest

Pig smoke test – wordcount using hadoop.cmd file
The system cannot find the path specified.
The system cannot find the path specified.
Run-PigSmokeTest : Pig Smoke Test: FAILED
At line:1 char:17
+ Run-PigSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-PigSmokeTest

Hcatalog smoke test – show tables, create table, and drop table
Running hcat command: show tables
python: can't open file '\bin\hcat.py': [Errno 2] No such file or directory
Run-HCatalogSmokeTest : HCatalog Smoke Test: FAILED
At line:1 char:22
+ Run-HCatalogSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HCatalogSmokeTest

2
Hive smoke test – create table and describe table
Running hive command: create external table if not exists hivesmoke ( foo INT, b
ar STRING)
The system cannot find the path specified.
Run-HiveSmokeTest : Hive Smoke Test: FAILED
At line:1 char:18
+ Run-HiveSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-HiveSmokeTest

3
Sqoop smoke test – version
The system cannot find the path specified.
Run-SqoopSmokeTest : Sqoop Smoke Test: FAILED
At line:1 char:19
+ Run-SqoopSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-SqoopSmokeTest

Oozie smoke test – status
The system cannot find the path specified.
Run-OozieSmokeTest : Oozie Smoke Test: FAILED
At line:1 char:19
+ Run-OozieSmokeTest <<<<
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
tion
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
n,Run-OozieSmokeTest

Thanks,
Santosh

[Santosh UmaraniMember]

Thanks Ted for the response.

In most of the places Error 3: -2147287038 is occurring. In one place the following error is occurring:

CAQuietExec: powershell.exe : Error resolving IPAddress for host ‘HIVE_SERVER_HOST’
CAQuietExec: At C:\HadoopInstallFiles\HadoopSetupTools\winpkg.ps1:118 char:23
CAQuietExec: + $out = powershell.exe <<<&1
CAQuietExec: + CategoryInfo : NotSpecified: (Error resolving…VE_SERVER_HOST’
CAQuietExec: :String) , RemoteException
CAQuietExec: + FullyQualifiedErrorId : NativeCommandError
CAQuietExec:
CAQuietExec: At C:\HadoopInstallFiles\HadoopPackages\hdp-1.1.0-winpkg\scripts\InstallApi.psm
CAQuietExec: 1:102 char:14
CAQuietExec: + throw <<<< "Error resolving IPAddress for host '$hostname'"
CAQuietExec: + CategoryInfo : OperationStopped: (Error resolving…VE_SERVER_H
CAQuietExec: OST':String) , RuntimeException
CAQuietExec: + FullyQualifiedErrorId : Error resolving IPAddress for host 'HIVE_SERVER_
CAQuietExec: HOST'
CAQuietExec:
CAQuietExec: WINPKG: rd /s /q C:\HadoopInstallFiles\HadoopPackages\hdp-1.1.0-winpkg
CAQuietExec:
CAQuietExec: WINPKG: Finished processing C:\HadoopInstallFiles\HadoopSetupTools\\..\HadoopPackages\hdp-1.1.0-winpkg.zip
MSI (s) (D8:60) [23:35:16:189]: Executing op: ActionStart(Name=WriteEnvironmentStrings,Description=Updating environment strings,Template=Name: [1], Value: [2], Action [3])

Please do let me know if you found any solution for this.

Thanks,
Santosh

[tedrMember]

Hi Santosh,

Can you post a few snippets from your install log? Specifically ab out 10 lines after each time it has the word “error”, but not those that have just “Error: 3 -12324565667″ – the lines with error 3 can be ignored. This will help us determine what might have gone wrong with your installation.

Thanks,
Ted.

[Santosh UmaraniMember]

Hi,
Even I’m facing the same problem. No error after installation and I’m using IPV4. Here is my clusterproperties file:

HDP_LOG_DIR=C:\hadoop\logs
HDP_DATA_DIR=C:\hdp\data
NAMENODE_HOST=NODE1.TECTestLab.local
SECONDARY_NAMENODE_HOST=NODE1.TECTestLab.local
JOBTRACKER_HOST=NODE1.TECTestLab.local
HIVE_SERVER_HOST=NODE1.TECTestLab.local
OOZIE_SERVER_HOST=NODE1.TECTestLab.local
TEMPLETON_HOST=NODE1.TECTestLab.local
SLAVE_HOSTS=NODE1.TECTestLab.local
DB_FLAVOR=derby
DB_HOSTNAME=HIVE_SERVER_HOST
DB_PORT=1527
HIVE_DB_NAME=hivedb
HIVE_DB_USERNAME=hive_user
HIVE_DB_PASSWORD=hive_pass
OOZIE_DB_NAME=ooziedb
OOZIE_DB_USERNAME=ooziedb_user
OOZIE_DB_PASSWORD=ooziedb_pass

Please do let me know if I’m doing any mistake over here. I’m using only single node.

Thanks,
Santosh

[SefMember]

Hi Jim,

Regarding your last question, running IPv6 should not be a problem, but you should make sure that all hosts in the cluster resolve to IPv4 adresses. You can try ‘ping -a ‘ command and see what ip you get. If you get IPv6 address, then you should switch to IPv4 address.

Please let me know if this is helpful. Also can you please provide output from you hosts file, output from ‘ping -a’ command, and other logs that could be useful.

Thanks,
Seth

[Jim CrossMember]

Does the installation fail with IPv6 or does running it fail?

Just to be clear my problem now is installation.

[SefMember]

Hi Jim,

Do you have anyway to try with IPv4? Depending on the server version you might need to disable IPv6 in the registry to make sure that host names resolve to IPv4..

Please let us know if we can assist you with anything.

Thanks,
Seth

[Larry LiuModerator]

Hi, Jim

We completely understand your frustration. This HDP 1.1 for windows is beta. We are aware of these issues and addressing them in GA release. With your involvement, we can make this product better.

Thanks

Larry

[Jim CrossMember]

Not really sure but I don’t want to mess around with my network configuration just to try this out. I have already had to modify quite bit.

There are problems with spaces the Java path too which the installation ought to be able to handle since the default location for Java installation is in the “Program Files” directory.

Frankly almost everything I try with hadoop doesn’t work. I went through a cygwin attempt with the code from Apache and it doesn’t work either. Since this is written in Java I can’t see why the software is so poorly architected that it can’t run seamlessly on Windows even if performance is superior on other platforms. The documentation is very fragmentary. I was hoping this going to address that but it seems like it has too many problems too.

[Anthony MattasMember]

Well are you running full IPV6 or IPV6/IPV4 mixed mode. You can force it to use IPV4 if you modify your hosts file from what I understand.

[Jim CrossMember]

Yes. I have log.

I can upload it if you like but I get the impression it isn’t going to work anyway with ipv6.

[Anthony MattasMember]

Do you have the full log?

[Jim CrossMember]

Same problem and I am using ipv6.

I guess I am out of luck.

[yi zhangMember]

Hi Ahmed,

Yes, it is a known issue that the installer log doesn’t reflect the true status of installation. This has been logged as a bug.

A few things in your clusterproperties.txt:

1. is your server named ‘localhost’? it should use the server’s FQDN
2. DB section is configured in clusterproperties. these are necessary for hive

Also check if the server is using ipv6 or ipv4. ipv6 is not supported in current version.

Thanks,
Yi.

[Author]

Replies