Home Forums HDP on Windows – Installation HDP 1-.3 FlumeAgent and Hbase Services Will not Start

Tagged: ,

This topic contains 23 replies, has 9 voices, and was last updated by  Roshan Naik 1 year, 2 months ago.

  • Creator
    Topic
  • #32531

    Pavan Keerthi
    Participant

    HI have successfully installed HDP 1.3 on Windows Server 2012 .When I start services the following services fail with the below same error for all failed services on Event Log

    Failed_Start flumeagent master regionserver rest thrift thrift2

    Error #1
    Source : .NET Runtime
    Application: flumeagent.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an unhandled exception.
    Exception Info: System.IO.FileNotFoundException
    Stack:
    at HadoopServiceHost.ServiceHost.InitInternal(System.String)
    at HadoopServiceHost.ServiceHost..ctor()
    at HadoopServiceHost.Program.Main()

    Error #2
    Source : Application Error
    Faulting application name: flumeagent.exe, version: 1.0.0.0, time stamp: 0x51fdd974
    Faulting module name: KERNELBASE.dll, version: 6.2.9200.16451, time stamp: 0x50988aa6
    Exception code: 0xe0434352
    Fault offset: 0x000000000003811c
    Faulting process id: 0x1434
    Faulting application start time: 0x01ce9b5b0058cb7e
    Faulting application path: C:\HDP\flume-1.3.1.1.3.0.0-0380\bin\flumeagent.exe
    Faulting module path: C:\Windows\system32\KERNELBASE.dll
    Report Id: 3e0b997e-074e-11e3-93fb-080027c00f6c
    Faulting package full name:
    Faulting package-relative application ID:

Viewing 23 replies - 1 through 23 (of 23 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #38707

    Roshan Naik
    Member

    To start flume please ensure the following:
    – You need to create a flume.conf based on your needs. Since the purpose of Flume is to transfer event data from point A to point B, it does not make sense to have a predefined flume config to be shipped as part of HDP.
    – On Windows : HDP Flume installation creates a flume service (see windows service manager). This service is turned off by default (for same reason mentioned above). Assuming Flume is installed in c:\hdp\flume, create a c:\hdp\flume\conf\flume.conf to suit your needs and start the Flume service. The Windows service will automatically pick up the.

    NOTE:
    The flume windows service basically starts flume agent as follows:
    flume-ng.cmd agent -f path/conf/flume.conf -n agent

    So ensure that your agent name inside flume.conf is ‘agent’

    Collapse
    #36135

    Ambrose Wong
    Member

    Just FYI for Zookeeper zkServer startup crashes with Invalid Config error in zkServer.err.log:

    1. In zoo.cfg, find the dataDir configuration and make sure the path exists. Manually create the path if it does not exist. Default is c:\hadoop\zkdata.
    2. In that folder, create a text file named myid. In the file, put a number that corresponds to the server number configured in zoo.cfg. For example, on a server named server1, if you have server.1=server1:2888:3888 in zoo.cfg, since server.1 indicates “1” is the ID, put “1” in the myid file.

    Collapse
    #36127

    Ambrose Wong
    Member

    In C:\hdp\hadoop\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd comment out line 161, the “set PATH=%PATH%;%HADOOP_HOME%\bin” line (add “REM ” at the beginning of the line). And re-run the following commands (change your path as necessary):

    c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd --service master start > "c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\master.xml
    c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd --service regionserver start > "c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\regionserver.xml
    c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd --service rest start > "c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\rest.xml
    c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd --service thrift start > "c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\thrift.xml
    c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd --service thrift2 start > "c:\hdp\hadoop\\hbase-0.94.6.1.3.0.0-0380\bin\thrift2.xml

    Collapse
    #33693

    Seth Lyubich
    Keymaster

    Hi Dave,

    Thanks for letting us know.

    Thanks,
    Seth

    Collapse
    #33588

    Dave Warner
    Participant

    An alternative to removing all PATH entries with “Program Files (x86)” is to replace them with the Windows short name for the directory, Progra~2.

    Then reinstall HDP 1.3 to get the proper XML files.

    Collapse
    #33518

    Pavan Keerthi
    Participant

    Failed Start: is more like label telling nothing failed to start.

    My smoke tests all ran succesfully

    Collapse
    #33516

    Eric Xu
    Member

    Hi Field and Pavan,
    Although all the services can run successfully, I still got failed_start in the end, just like you had.
    I cannot run the smoke_test also.

    Hope someone can give some advice.

    Collapse
    #33468

    Field Wachi
    Participant

    hi all
    I have the same error for this issue and I’ve tried Eric step’s which is the correct answer.
    Status of implementation as following
    PS C:\hdp\hadoop> .\start_local_hdp_services.cmd
    starting namenode
    starting secondarynamenode
    starting datanode
    starting jobtracker
    starting historyserver
    starting tasktracker
    starting zkServer
    starting master
    starting regionserver
    starting hwi
    starting hiveserver
    starting hiveserver2
    starting metastore
    starting derbyserver
    starting templeton
    starting oozieservice
    Sent all start commands.
    total services
    16
    running services
    16
    not yet running services
    0
    Failed_Start

    Thanks Eric

    Collapse
    #33372

    Pavan Keerthi
    Participant

    I can confirm Eric’s steps resolved the problem.Here is what I did

    1) Uninstalled HDP 1.3 from server
    2) Removed all old files from install folders
    3) From my Environmental Variable Path I removed all paths which has like C:\Program files (x86)\SQL
    4) Reinstalled HDP 1.3

    This time master.xml regionserver.xml all got xml populated properly and services started fine.

    I also noticed 1 more problem with Flume.It was failing because .trace showed it was expecting flumeagnet.xml in bin folder where as deployment has file by name flumeservice.xml.I changed the filename to flumeagent.xml and this got resolved

    Finally all 19 services started successfully

    Thanks Eric

    Collapse
    #33334

    Eric Xu
    Member

    some software installed in the program file(86). SQL server is one of them.

    Collapse
    #33301

    Seth Lyubich
    Keymaster

    Hi Eric,

    Can you please let us know which environmental variables had “()”?

    Thanks,
    Seth

    Collapse
    #33096

    Eric Xu
    Member

    I also post the log of namenode.

    this is a piece of the error in it.

    2013-08-23 14:27:31,097 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hadoop cause:java.io.IOException: File /mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1
    2013-08-23 14:27:31,097 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 8020, call addBlock(/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_-1839200336_1, null) from 169.254.53.78:53147: error: java.io.IOException: File /mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1

    Collapse
    #33087

    Eric Xu
    Member

    I tried to delete all the path with “()” in the environment variable.
    and finally get all the service started successfully.
    However, PS C:\hadoop\hdp> ./start_local_hdp_services
    starting namenode
    starting secondarynamenode
    starting datanode
    starting jobtracker
    starting historyserver
    starting tasktracker
    starting zkServer
    starting master
    starting regionserver
    starting hwi
    starting hiveserver
    starting hiveserver2
    starting metastore
    starting derbyserver
    starting templeton
    starting oozieservice
    Sent all start commands.
    total services
    16
    running services
    16
    not yet running services
    0
    Failed_Start

    I already upload the new install.log to where you provided.

    Hope for your further advice.

    Collapse
    #33085

    Seth Lyubich
    Keymaster

    Thanks for your feedback Eric,

    Can you please upload us full installation log as well as file called hdp-1.3.0.0.winpkg.install.log located in C:\HadoopInstallFiles\HadoopSetupTools? You can upload the file here (you can use web browser):

    http://ftp.support.hortonworks.com

    username: dropoff
    password: horton

    Thanks,
    Seth

    Collapse
    #33081

    Eric Xu
    Member

    this is the installation log.

    HBASE: Copying configuration for master regionserver rest thrift thrift2
    HBASE: Creating service config C:\hadoop\hdp\\hbase-0.94.6.1.3.0.0-0380\bin\master.xml
    HBASE: C:\hadoop\hdp\\hbase-0.94.6.1.3.0.0-0380\bin\hbase.cmd –service master start > “C:\hadoop\hdp\\hbase-0.94.6.1.3.0.0-0380\bin\master.xml”
    HBASE-CMD FAILURE: \Microsoft was unexpected at this time.

    Collapse
    #33071

    Eric Xu
    Member

    I was so excited to re-install it following the steps you provided three times.
    However, I still failed.
    The master.xml and regionserver.xml are always blank.

    Do you have any advice?

    Collapse
    #33041

    Aaa Aaa
    Member

    thank Sef for my help.
    This problem has been solved.
    I performed according to the following :
    – Re-download the installer
    – Uninstall HDP
    – delete all files in HDP and data directories
    – Re-run the installer DESTROY_DATA=”yes”

    From Powershell with Administrator privileges, execute the following commands:
    cd c:\hdp\hadoop
    start_local_hdp_services
    ———————————
    All services already successfully started !
    very good thanks Sef

    Collapse
    #33021

    Seth Lyubich
    Keymaster

    Hi all,

    Thanks for helping to look at this issue. I was able to reproduce the issue when I removed the content of master.xml file. Can you please try to re-install HDP and see if this issue is reproducible? Please make sure to do the following:

    – Re-download the installer
    – Uninstall HDP
    – delete all files in HDP and data directories
    – Re-run the installer

    Note: you can set DESTROY_DATA=YES in order to avoid manual execution Namenode format command.

    Please let me know if this helps to resolve the issue.

    Thanks,
    Seth

    Collapse
    #33011

    Seth Lyubich
    Keymaster

    Hi all,

    I was not able to reproduce the issue. I set DESTROY_DATA=NO and executed ‘hadoop namenode -format’ command manually as was discussed in this thread. Can you please provide me with HBase master logs (service which is not starting for few people) and we will investigate further. Following are the files that will be useful for investigation.

    HBase-master-*.log
    master.trace.log

    Please provide me with anything else which you think might be useful.

    You can upload logs here:

    ftp http://ftp.support.hortonworks.com
    username: dropoff
    password: horton

    Thanks,
    Seth

    Collapse
    #32880

    Aaa Aaa
    Member

    I have encountered this problem.
    I also successfully installed HDP 1.3 on Windows
    namenode,datanode,hive,and other services already successfully started.
    but Failed_Start master regionserver
    ——————————–
    Here is the error message in master.trace.log:
    HadoopServiceTraceSource Information: 0 : Loading service xml: C:\hdp\hadoop\hbase-0.94.6.1.3.0.0-0380\bin\master.xml
    DateTime=2013-08-22T02:34:23.9268430Z
    Timestamp=604004639813
    HadoopServiceTraceSource Error: 0 : Failed to parse the service xml with exceptionSystem.Xml.XmlException:The root element is missing.
    at System.Xml.XmlTextReaderImpl.ThrowWithoutLineInfo(String res)
    at System.Xml.XmlTextReaderImpl.ParseDocumentContent()
    at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean preserveWhitespace)
    at System.Xml.XmlDocument.Load(XmlReader reader)
    at System.Xml.XmlDocument.Load(String filename)
    at HadoopServiceHost.ServiceHost.InitInternal(String fileName)
    DateTime=2013-08-22T02:34:23.9894222Z
    Timestamp=604005210775
    ——————————————————
    master.xml file has no content, how should I do it?

    Collapse
    #32867

    Seth Lyubich
    Keymaster

    Hi Pavan,

    Can you please check your .wrapper file or component .log file to see if there is anything obviously wrong with command that tries to start the service?

    Thanks,
    Seth

    Collapse
    #32570

    Pavan Keerthi
    Participant

    Hi,

    Yes Name node and other services run just fine and I can see from Dashboard

    The only services that fail are
    flumeagent
    master
    regionserver
    rest
    thrift
    thrift2

    Collapse
    #32549

    Rohit Bakhshi
    Moderator

    Hi Pavan,

    Did the NameNode service start successfully? Without the NameNode service running, the HBase Master and RegionServer services will not start.

    If the NameNode service is not starting, please try this:
    Could you please try this:
    1. Open the “Hadoop Command Line” Command Prompt shortcut.
    2. Run the following command that sets up the NameNode directories: “hadoop namenode -format”

    Then start all services with the same script you used above.

    Collapse
Viewing 23 replies - 1 through 23 (of 23 total)