Home Forums HDP on Linux – Installation Ambari Stuck after Login

This topic contains 76 replies, has 6 voices, and was last updated by  Srimanth Gunturi 2 weeks, 1 day ago.

  • Creator
    Topic
  • #33261

    Ray Roberts
    Participant

    I have Ambari 1.2.4.9-1 installed and was working fine. I now have an issue where I go to the Ambari webpage and it is stuck on ‘Loading’.

    I’ve tried restarting both the agent and server with no success.

    I see this in the ambari-server.log:

    10:32:44,598 INFO AmbariLocalUserDetailsService:62 – Loading user by name: admin
    10:32:45,110 INFO HeartBeatHandler:108 – Received heartbeat from host, hostname=xxxxxxxx.domain.com, currentResponseId=2, receivedResponseId
    =2
    10:32:45,110 INFO AgentResource:109 – Sending heartbeat response with response id 3
    10:32:45,210 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,392 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,935 ERROR ReadHandler:82 – Caught a runtime exception executing a query
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10

    Any ideas what is causing this?

Viewing 30 replies - 31 through 60 (of 76 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #43122

    Ray Roberts
    Participant

    Sorry, I’ve been absent from this conversation for awhile, so let me jump back in………

    After reading through the thread, I tried going to http://ambari-server:8080/api/v1/clusters/clustername/services/NAGIOS

    I got…………Parent Cluster resource doesn’t exist. Cluster not found, clusterName=clustername

    So, I guess this means I didn’t install nagios, which I recall is correct.

    Any other ideas why it would hang with ‘loading’ after logging in?

    -Ray

    Collapse
    #43068

    Srimanth Gunturi
    Participant

    The HTTPD server on the Nagios machine is not providing the “http://hostname/ambarinagios/nagios/nagios_alerts.php” page.

    Please check that the HTTPD server on that machine is using the “/etc/httpd/conf.d/hdp_mon_nagios_addons.conf” file, as that is the file which maps the “/ambarinagios” to local HDP install. Once this is fixed I think we should be in better shape.

    Collapse
    #43026

    Josh Hansen
    Member

    On the machine that ran the installer, the command you gave returns no output. On the other machine in the cluster, it gives the following:
    hdp_mon_nagios_addons-1.4.1.25-1.el6.noarch

    So not the version you specified.

    Collapse
    #42712

    Srimanth Gunturi
    Participant

    Can you please check if the RPM “hdp_mon_nagios_addons-1.4.2.218-1.el6.noarch” is installed on the Nagios server machine?
    `rpm -qa | grep hdp_mon_nagios_addons`
    Regards,
    Srimanth

    Collapse
    #42676

    Josh Hansen
    Member

    Okay, I did that and it’s basically the same result:
    HTTP ERROR: 404

    Problem accessing /ambarinagios/nagios/nagios_alerts.php. Reason:

    Not Found

    Powered by Jetty://

    Collapse
    #42631

    Srimanth Gunturi
    Participant

    Actually this forums software changed the URL badly.
    Please replace “hostname” in “http://hostname/ambarinagios/nagios/nagios_alerts.php?q1=alerts&alert_type=all”

    Collapse
    #42630

    Josh Hansen
    Member

    I take it that by “replace nagiosserver with correct host/ip” you meant “replace ambarinagios with correct host/ip” ?

    Collapse
    #42629

    Josh Hansen
    Member

    The output of that URL is:

    HTTP ERROR: 404

    Problem accessing /nagios/nagios_alerts.php. Reason:

    Not Found

    Powered by Jetty://

    Collapse
    #42385

    Srimanth Gunturi
    Participant

    OK, so Nagios is installed. Can you now please provide the exact content of the below URL (replace nagiosserver with correct host/ip)?

    http:///ambarinagios/nagios/nagios_alerts.php?q1=alerts&alert_type=all

    Collapse
    #42299

    Josh Hansen
    Member

    (Note that port 8123 is being mapped to 8080)

    Collapse
    #42298

    Josh Hansen
    Member

    Yes, I did check both Nagios and Ganglia in Ambari during install. The URL you listed gives this result:
    {
    “href” : “http://localhost:8123/api/v1/clusters/research/services/NAGIOS”,
    “ServiceInfo” : {
    “cluster_name” : “research”,
    “service_name” : “NAGIOS”,
    “state” : “STARTED”,
    “desired_configs” : { }
    },
    “components” : [
    {
    "href" : "http://localhost:8123/api/v1/clusters/research/services/NAGIOS/components/NAGIOS_SERVER",
    "ServiceComponentInfo" : {
    "cluster_name" : "research",
    "component_name" : "NAGIOS_SERVER",
    "service_name" : "NAGIOS"
    }
    }
    ]
    }

    I’m not that familiar with Nagios, but I don’t see any process running that seems to correspond to it.

    Collapse
    #42127

    Srimanth Gunturi
    Participant

    Hi Josh,
    I might have misunderstood your statement
    >> “It should be noted that I only selected the Nagios and Ganglia options because I thought Ambari would fully configure them for me”

    During install, you had both Nagios and Ganglia checkboxes checked? If they were checked/selected, then Nagios and Ganglia will be installed by Ambari. If they were not checked, they will not be installed.

    To verify if Ambari installed Nagios, can you please look at output of http://ambari-server:8080/api/v1/clusters/clustername/services/NAGIOS . If you see output status of 404, then Nagios is not installed. If meaningful content is generated, then Ambari installed/monitors Nagios service.
    Regards,
    Srimanth

    Collapse
    #42110

    Josh Hansen
    Member

    As I mentioned below, I never installed Nagios as the installer seemed to imply that this would be done for me.

    Collapse
    #42100

    Srimanth Gunturi
    Participant

    Hi Josh,
    From the ambari-server machine, can you please provide response of following URL of Nagios server machine?

    http://nagiosserver/ambarinagios/nagios/nagios_alerts.php?q1=alerts&alert_type=all

    We want to make sure valid JSON is being provided to Ambari server by Nagios.
    Regards,
    Srimanth

    Collapse
    #42067

    Josh Hansen
    Member

    Any news?

    Collapse
    #41840

    Josh Hansen
    Member

    I just emailed the log with debug enabled

    Collapse
    #41838

    Sid Wagle
    Participant

    Could you turn on the debug logging on the ambari server and post the findings ?

    Edit the /etc/ambari-server/conf/log4j.properties.
    Change, log4j.rootLogger=INFO,file
    To, log4j.rootLogger=DEBUG,file

    Restart the ambari server.

    The /var/log/ambari-server/ambari-server.log should provide a lot more details.
    Note: You could backup current file, stop server, delete the file and then capture the exclusive debug logs and then turn off debugging thereafter.

    Collapse
    #41832

    Josh Hansen
    Member

    The log file is sent

    Collapse
    #41829

    Srimanth Gunturi
    Participant

    Please email to srimanth@hortonworks.com
    Regards,
    Srimanth

    Collapse
    #41828

    Josh Hansen
    Member

    I would love to provide that log for you. Apparently I can’t attach files in this forum, and the compose box is limited to 3000 characters, so maybe email is best. Where can I send it?

    Collapse
    #41827

    Josh Hansen
    Member

    BTW, that last stack trace was truncated. But those exceptions haven’t happened recently. Now it’s just heartbeats along with this:

    10:56:19,958 INFO [qtp626385835-2392] GangliaPropertyProvider:439 – Ganglia server component is not live

    Collapse
    #41826

    Srimanth Gunturi
    Participant

    Hi Josh,
    The problem is that an important call fails:
    ——-
    Failed to load resource: the server responded with a status of 500 (java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 6) http://localhost:8123/api/v1/clusters/research/services?fields=components/S…/flume/flume,components/host_components/metrics/yarn/Queue&_=1382721085709
    ——-

    We are trying to determine why that call failed. If possible, can you please provide or email the ambari server log (/var/log/ambari-server/ambari-server.log)?
    Regards,
    Srimanth

    Collapse
    #41824

    Josh Hansen
    Member

    In the ambari log, I got some exceptions like this about an hour ago:
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 6
    at com.google.gson.Gson.fromJson(Gson.java:806)
    at com.google.gson.Gson.fromJson(Gson.java:761)
    at com.google.gson.Gson.fromJson(Gson.java:710)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.getHttpResponse(HttpProxyPropertyProvider.java:133)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.populateResources(HttpProxyPropertyProvider.java:109)
    at org.apache.ambari.server.controller.internal.HostComponentResourceProvider.getResources(HostComponentResourceProvider.java:195)
    at org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:119)
    at org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:94)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:149)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:169)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:169)
    at org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:61)
    at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:107)
    at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:80)
    at org.apache.ambari.server.api.services.ServiceService.getServices(ServiceService.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
    at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
    at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
    at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:134)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
    at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
    at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)

    Collapse
    #41823

    Josh Hansen
    Member

    And finally, this part gets logged to the web console over and over and over again:

    ============== ajax ============== background_operations.get_most_recent undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters/research/requests?to=end&page_size=10&fields=*,tasks/Tasks/* ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    ============== ajax ============== background_operations.get_most_recent undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters/research/requests?to=end&page_size=10&fields=*,tasks/Tasks/* ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    No Nagios URL provided. cluster_controller.js:239

    It should be noted that I only selected the Nagios and Ganglia options because I thought Ambari would fully configure them for me. Apparently this isn’t the case!

    Collapse
    #41822

    Josh Hansen
    Member

    Here’s some more:
    ============== ajax ============== background_operations.get_most_recent undefined ajax.js:1182
    STATEMANAGER: Entering root.main.index ember-latest.js:15815
    STATEMANAGER: Entering root.main.dashboard ember-latest.js:15815
    Before load: App.Dataset.find().content : dataset_mapper.js:184
    After load: App.Dataset.find().content : 11,12,13 dataset_mapper.js:186
    Before load: App.DataSetJob.find().content : dataset_mapper.js:207
    After load: App.DataSetJob.find().content : 14,15,16,17 dataset_mapper.js:209
    Status code 200: Success. statusCodes.js:22
    TRACE: The url is: /api/v1/services/AMBARI/components/AMBARI_SERVER ajax.js:1211
    loading ambari properties cluster_controller.js:403
    Status code 200: Success. statusCodes.js:22
    TRACE: The url is: /api/v1/clusters/research/requests?to=end&page_size=10&fields=*,tasks/Tasks/* ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    Failed to load resource: the server responded with a status of 500 (java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 6) http://localhost:8123/api/v1/clusters/research/services?fields=components/S…/flume/flume,components/host_components/metrics/yarn/Queue&_=1382721085709
    ============== ajax ============== background_operations.get_most_recent undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters/research/requests?to=end&page_size=10&fields=*,tasks/Tasks/* ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    in status mapper status_mapper.js:35
    Uncaught TypeError: Cannot read property ‘mastersRunning’ of undefined status_mapper.js:292
    ============== ajax ============== background_operations.get_most_recent undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters/research/requests?to=end&page_size=10&fields=*,tasks/Tasks/* ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    No Nagios URL provided. cluster_controller.js:239
    Failed to load resource: the server responded with a status of 500 (java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 6) http://localhost:8123/api/v1/clusters/research/services?fields=components/S…/flume/flume,components/host_components/metrics/yarn/Queue&_=1382721100999

    Collapse
    #41821

    Josh Hansen
    Member

    Just installed HDP 2.0 on CentOS 6.4 and am having what seems like the same issue. Here’s a bunch of output from Chrome’s web console:

    CLDR.pluralForm not found. Em.I18n will not support count-based inflection. ember-i18n-1.4.1.js:28
    ‘window.webkitStorageInfo’ is deprecated. Please use ‘navigator.webkitTemporaryStorage’ or ‘navigator.webkitPersistentStorage’ instead.
    STATEMANAGER: Entering root ember-latest.js:15815
    2
    TRACE: Entering db:getUser function db.js:412
    Administrator logged in router.js:349
    No Nagios URL provided. cluster_controller.js:239
    STATEMANAGER: Sending event ‘navigateAway’ to state root. ember-latest.js:15578
    STATEMANAGER: Sending event ‘unroutePath’ to state root. ember-latest.js:15578
    STATEMANAGER: Sending event ‘routePath’ to state root. ember-latest.js:15578
    STATEMANAGER: Entering root.index ember-latest.js:15815
    STATEMANAGER: Entering root.login ember-latest.js:15815
    Trace: Entering db:getAuthenticated function db.js:424
    ============== ajax ============== router.authentication undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    /login:connectOutlet router.js:380
    Trace: Entering db:getLoginName function db.js:418
    Trace: Entering db:getWizardCurrentStep function for installer db.js:477
    getWizardCurrentStep: loginName=admin, currentStep=0 router.js:74
    returning currentStep=0 router.js:78
    currentStep is: 0 router.js:381
    Trace: Entering db:getAuthenticated function db.js:424
    ============== ajax ============== router.authentication undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    authenticated is: true router.js:382
    after initialize initialize.js:55
    TRACE: app.js-> localStorage:Ambari.authenticated=null initialize.js:56
    Trace: Entering db:getLoginName function db.js:418
    TRACE: app.js-> localStorage:currentStep=null initialize.js:57
    TRACE: app.js-> router.authenticated=true initialize.js:58
    2
    Trace: Entering db:getLoginName function db.js:418
    admin already authenticated. Redirecting… router.js:372
    Status code 200: Success. statusCodes.js:22
    STATEMANAGER: Entering root.main ember-latest.js:15815
    in /main:enter main.js:26
    Trace: Entering db:getAuthenticated function db.js:424
    ============== ajax ============== router.authentication undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    ============== ajax ============== cluster.load_cluster_name undefined ajax.js:1182
    TRACE: The url is: /api/v1/clusters ajax.js:1211
    Status code 200: Success. statusCodes.js:22
    ============== ajax ============== ambari.service undefined ajax.js:1182
    No Nagios URL provided. cluster_controller.js:239

    Collapse
    #40879

    Srimanth Gunturi
    Participant

    Hi Ray,
    The ambari-server and nagios-server are on same host or different hosts?
    Also, is this an upgraded Ambari server?
    I was hoping if you can attach or email the ambari server log file? (/var/log/ambari-server/ambari-server.log)
    Regards,
    Srimanth

    Collapse
    #40860

    Ray Roberts
    Participant

    Srimanth,

    I installed it during the Ambari installation, however, I don’t start the service.

    This is what I see when going to the link you have below:

    {“alerts”:[],”hostcounts”:{“up_hosts”:”0″,”down_hosts”:”0″},”servicestates”:{“PUPPET”:”0″}}

    -Ray

    Collapse
    #40787

    Srimanth Gunturi
    Participant

    Hi Ray,
    Knowing Nagios is installed helps. When you say that you do not use it, did you kill the process, or stop the service? Or just leave it alone as it was installed?
    On the Nagios server machine please try to access http://nagiosserver/ambarinagios/nagios/nagios_alerts.php?q1=alerts&alert_type=all . If you do not see JSON, that is a problem.
    Regards,
    Srimanth

    Collapse
    #40777

    Ray Roberts
    Participant

    Srimanth,

    It is thrown right after I login …. http://server:8080/#/main/dashboard

    As far as Nagios, after looking further, it is installed, however, I don’t use it.

    I received the following when trying to go to server:8080/api/v1/clusters/staging/services?fields=components/*:
    “{
    “status” : 400,
    “message” : “The properties [] specified in the request or predicate are not supported for the resource type Component.”
    }”

    Thanks,

    -Ray

    Collapse
Viewing 30 replies - 31 through 60 (of 76 total)