Home Forums HDP on Windows – Installation WriteErrorException when running Smoke Test

This topic contains 11 replies, has 8 voices, and was last updated by  Iain Sproat 6 months, 3 weeks ago.

Viewing 11 replies - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #50775

    Iain Sproat
    Participant

    Ignore my previous comment – I’ve now spotted the error in the log file as well. d’oh!

    Turns out the root cause of the problem is due to the Python versions in Templeton. This can be resolved
    with this fix.

    Remember to stop and restart all the services for the fix to take hold.

    Collapse
    #50773

    Iain Sproat
    Participant

    I’ve run into the same problem.

    Strangely, I did not get any errors when I redirected the output into a text file (in a folder I know I have suitable permissions for).

    Run-SmokeTests > C:\Hadoop\SmokeTests.log

    It’s a workaround that works for me, but doesn’t solve the root of the problem

    Collapse
    #48330

    Martin Meier
    Participant

    We are getting the same problem here, tried it on 2 different machines, already tried it with Admin rights for the hadoop user, deactivating IPv6, same result.

    Could anyone solve this issue? Thanks!

    Collapse
    #38271

    Seth Lyubich
    Keymaster
    Collapse
    #37814

    Elias Hortas
    Member

    I am experiencing the same issue.

    In Namenode log I found:
    2013-09-24 17:21:18,599 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory E:\Software\HDP2\data\hdfs\nn does not exist
    2013-09-24 17:21:18,601 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
    org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory E:\Software\HDP2\data\hdfs\nn is in an inconsistent state: storage directory does not exist or is not accessible.
    And I confirm the namenode dir “nn” does not exist in the path indicated above. Btw, my hadoop accound has full rights to the hadoop directory (and subdirectories and folders).

    Datanode log is this (over and over):

    2013-09-24 18:24:47,574 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: COEMAXQUA01/170.251.74.159:8020. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
    2013-09-24 18:24:48,576 INFO org.apache.hadoop.ipc.RPC: Server at COEMAXQUA01/170.251.74.159:8020 not available yet, Zzzzz…

    Any help welcome!

    Collapse
    #36120

    Monika Chugh
    Member

    Hi

    I am getting the same error while running the smoke tests. All services are up and running and I did execute the Run-SmokeTests command from HADOOP_HOME directory itself. Would appreciate if anyone can help.

    Collapse
    #35633

    Seth Lyubich
    Keymaster

    Hi John/Adam,

    Can you please check that you setup security on all your machines correctly? Please verify using instruction here:

    http://docs.hortonworks.com/HDPDocuments/HDP1/HDP-Win-1.3.0/bk_installing_hdp_for_windows/content/win-getting-ready-5-5.html

    Please make sure that you list all nodes in the cluster using FQDN hostnames in the commands and that you run with Administrator rights.

    Also please check DataNode and NameNode logs. If you see error – java.io.IOException: An existing connection was forcibly closed by the remote host, than it is possible this issue is related to security settings.

    Please let me know if this is helpful.

    Thanks,
    Seth

    Collapse
    #34123

    John Bunch
    Member

    Same problem using single-node installation. Installation process does not report an error, and all services confirmed to be running. Run-Smoketests.cmd fails with above message.

    Collapse
    #34106

    John Bunch
    Member

    This is the same error I’m getting. I think this is related to a Powershell remote command security issue. Have you tried a single machine installation? I’m going to try that next to see if it eliminates the problem.

    Collapse
    #33922

    I am also getting these failures. Can someone please tell me what you would need to help troubleshoot. I have two machines and they are identical (i set them up in parallel. I’m not sure what the problem is. I can’t find the difference.

    C:\hdp\hadoop>Run-SmokeTests.cmd
    Hadoop smoke test – wordcount using hadoop.cmd file
    The system cannot find the path specified.
    Run-HadoopSmokeTest : Error copying the input file for the Hadoop smoke test
    At line:1 char:20
    + Run-HadoopSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-HadoopSmokeTest

    The system cannot find the path specified.
    Run-HadoopSmokeTest : Hadoop Smoke Test: FAILED
    At line:1 char:20
    + Run-HadoopSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-HadoopSmokeTest

    Pig smoke test – wordcount using hadoop.cmd file
    The system cannot find the path specified.
    The system cannot find the path specified.
    Run-PigSmokeTest : Pig Smoke Test: FAILED
    At line:1 char:17
    + Run-PigSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-PigSmokeTest

    Hcatalog smoke test – show tables, create table, and drop table
    Running hcat command: show tables
    python: can't open file '\bin\hcat.py': [Errno 2] No such file or directory
    Run-HCatalogSmokeTest : HCatalog Smoke Test: FAILED
    At line:1 char:22
    + Run-HCatalogSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-HCatalogSmokeTest

    2
    Hive smoke test – drop table, create table and describe table
    Running hive command: drop table if exists hivesmoke
    The system cannot find the path specified.
    Run-HiveSmokeTest : Hive Smoke Test: FAILED
    At line:1 char:18
    + Run-HiveSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-HiveSmokeTest

    3
    HiveServer2 smoke test – drop table, create table and describe table
    The system cannot find the path specified.
    Run-HiveServer2SmokeTest : HiveServer2 Smoke Test: FAILED
    At line:1 char:25
    + Run-HiveServer2SmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorExceptio
    n,Run-HiveServer2SmokeTest

    3
    Sqoop smoke test – version
    The system cannot find the path specified.
    Run-SqoopSmokeTest : Sqoop Smoke Test: FAILED
    At line:1 char:19
    + Run-SqoopSmokeTest <<<<
    + CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorExcep
    tion
    + Fully

    Collapse
    #32300

    Seth Lyubich
    Keymaster

    Hi,

    Can you please check that you running smoke test per instructions?

    cd %HADOOP_NODE_INSTALL_ROOT%
    Run-SmokeTests

    Also, can you please check to make sure that all hadoop services are up?

    Hope this helps.

    Thanks,
    Seth

    Collapse
Viewing 11 replies - 1 through 11 (of 11 total)