Home Forums HDP on Linux – Installation Ambari Stuck after Login

This topic contains 76 replies, has 6 voices, and was last updated by  Srimanth Gunturi 4 months, 2 weeks ago.

  • Creator
    Topic
  • #33261

    Ray Roberts
    Participant

    I have Ambari 1.2.4.9-1 installed and was working fine. I now have an issue where I go to the Ambari webpage and it is stuck on ‘Loading’.

    I’ve tried restarting both the agent and server with no success.

    I see this in the ambari-server.log:

    10:32:44,598 INFO AmbariLocalUserDetailsService:62 – Loading user by name: admin
    10:32:45,110 INFO HeartBeatHandler:108 – Received heartbeat from host, hostname=xxxxxxxx.domain.com, currentResponseId=2, receivedResponseId
    =2
    10:32:45,110 INFO AgentResource:109 – Sending heartbeat response with response id 3
    10:32:45,210 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,392 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,935 ERROR ReadHandler:82 – Caught a runtime exception executing a query
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10

    Any ideas what is causing this?

Viewing 16 replies - 61 through 76 (of 76 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #40775

    Srimanth Gunturi
    Participant

    Hi Ray,
    A very important call for the Web UI (the JS exception you provided on 8/26/2013) is failing. Due to this UI is unable to load to completion.

    The browser is asking Ambari API for installed services along with status of components. One of those components is Nagios Server. At this point the exception “java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10″ is thrown.

    You mentioned that you did not even install Nagios? Can you please provide the full URL the browser throws exception on?
    Also, can you please check if NAGIOS_SERVER is in http://server:8080/api/v1/clusters/staging/services?fields=components/*
    Regards,
    Srimanth

    Collapse
    #40690

    Ray Roberts
    Participant

    Dave,

    This is actually running on one server. We are only using it as a proof of concept. (I’ve installed HDP on single nodes for demo purposes numerous times and never had this issue.)

    postgres is running.

    Srimanth,

    I’m not using Nagios in this instance. If I was, I don’t see how that would cause the behavior of not bringing up the Ambari GUI after login.

    Thanks,

    -Ray

    Collapse
    #40270

    Srimanth Gunturi
    Participant

    Hi Ray,
    Ambari in the background seems to be reaching for the Nagios server. It is not getting JSON back when expected.
    Can you please check you can access Nagios web-site from your Ambari server machine?
    Maybe there is a firewall in between, or Nagios server is down etc.
    Regards,
    Srimanth

    Collapse
    #40075

    Dave
    Moderator

    Hi Ray,

    How many nodes are you running?
    Lets say you have server1.my.domain and server2.my.domain

    server1 etc/hosts should have server2 in it with the IP

    Also I would check the tcpstack to ensure communication is solid over port 8440/1

    Also ensure that postgres is running, you can log.

    Thanks

    Dave

    Collapse
    #39977

    Ray Roberts
    Participant

    So,

    I never resolved this issue.

    I’ve been starting/stoping HDP services manually, which is needles to say, getting old :)

    Its not an issues with conflicting ports, i’ve changed which port Ambari is using numerous times.

    Also, it used to “work”(not hang after logging in) for a short amount of time, after a reboot, however, it no longer works.

    The ambari-server.log shows:

    11:34:16,217 INFO AmbariLocalUserDetailsService:62 – Loading user by name: admin
    11:34:16,776 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    11:34:17,008 ERROR ReadHandler:82 – Caught a runtime exception executing a query
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10
    at com.google.gson.Gson.fromJson(Gson.java:806)
    at com.google.gson.Gson.fromJson(Gson.java:761)
    at com.google.gson.Gson.fromJson(Gson.java:710)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.getHttpResponse(HttpProxyPropertyProvider.java:116)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.populateResources(HttpProxyPropertyProvider.java:104)
    at org.apache.ambari.server.controller.internal.HostComponentResourceProvider.getResources(HostComponentResourceProvider.java:190)
    at org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:101)

    Any help would be appreciated.

    -Ray

    Collapse
    #33555

    Robert
    Participant

    Hi Ray,
    Based on your responses, the problem goes away if you restart the box? Hmm, maybe there is some service that runs on that machine that is potentially causing a conflict with ambari-server. Can you confirm if you have some other applications that you run that maybe you can turn off?

    Regards,
    Robert

    Collapse
    #33547

    Ray Roberts
    Participant

    Dave,

    Sorry, I just realized I didn’t answer your question fully.

    I do see file:/usr/lib/ambari-server/ambari-server-1.2.4.9.jar_ambari-server_url=jdbc:postgresql://localhost/ambari_user=ambari login successful’

    in the ambari-server.out. However, I don’t see it when I hit ‘Sign In’, I see it after I start ambari-server……before I’ve logged in via the web page.

    -Ray

    Collapse
    #33423

    Ray Roberts
    Participant

    I’m using CentOS 6.4.

    tailing the log shows a bunch of these:

    SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10
    at com.google.gson.Gson.fromJson(Gson.java:806)
    at com.google.gson.Gson.fromJson(Gson.java:761)
    at com.google.gson.Gson.fromJson(Gson.java:710)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.getHttpResponse(HttpProxyPropertyProvider.java:116)
    at org.apache.ambari.server.controller.internal.HttpProxyPropertyProvider.populateResources(HttpProxyPropertyProvider.java:104)
    at org.apache.ambari.server.controller.internal.HostComponentResourceProvider.getResources(HostComponentResourceProvider.java:190)
    at org.apache.ambari.server.controller.internal.ClusterControllerImpl.getResources(ClusterControllerImpl.java:101)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:141)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:156)
    at org.apache.ambari.server.api.query.QueryImpl.execute(QueryImpl.java:156)
    at org.apache.ambari.server.api.handlers.ReadHandler.handleRequest(ReadHandler.java:60)
    at org.apache.ambari.server.api.services.BaseRequest.process(BaseRequest.java:100)
    at org.apache.ambari.server.api.services.BaseService.handleRequest(BaseService.java:80)
    at org.apache.ambari.server.api.services.ServiceService.getServices(ServiceService.java:77)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
    at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)

    Collapse
    #33422

    Dave
    Moderator

    Hi Ray,

    What OS did you install this on? I have 1.3.0 installed on CentOS 6 and another installation of 1.3.2 and do not experience this issue.
    If you tail the ambari-server.out log do you see anything which looks like the following when you hit “Sign In” ?

    file:/usr/lib/ambari-server/ambari-server-1.2.3.7.jar_ambari-server_url=jdbc:postgresql://localhost/ambari_user=ambari-server login successful

    Thanks

    Dave

    Collapse
    #33418

    Ray Roberts
    Participant

    Dave,

    Yes, I installed via HDP 1.3.0.

    Both files begin with {

    I wonder if upgrading to 1.3.2 would help. I don’t like that fix, however, I’m currently unable to doing anything via Ambari unless I reboot the system.

    -Ray

    Collapse
    #33409

    Dave
    Moderator

    Hi Ray,

    Did you install Ambari via the HDP installation or was it a standalone implementation?
    It looks like an issue with the JSON query either on the user login (which doesn’t seem it, as you actually ‘login’) but it is possibly with identifying the host.
    Could you check the /usr/lib/anmbari-server/web/data/clusters/cluster.json and /usr/lib/ambari-server/web/data/hosts/hosts70.json

    These should both start with {

    Thanks

    Dave

    Collapse
    #33406

    Ray Roberts
    Participant

    Dave,

    I’m not seeing anything obvious with netstat, I even changed the port being used by ambari but got the same result.

    I’m seeing the following errors in the ambari-server.log and ambari-server.out:

    ambari-server.log:
    ERROR ReadHandler:82 – Caught a runtime exception executing a query
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10

    ambari-server.out:
    SEVERE: The RuntimeException could not be mapped to a response, re-throwing to the HTTP container
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10

    -Ray

    Collapse
    #33400

    Dave
    Moderator

    Hi Ray,

    If you run a netstat -an do you see anything which may be using a lot of ports on the machine?

    Also what does the ambari-server log show (/var/log/ambari-server/ambari-server.log /.out)

    Thanks

    Dave

    Collapse
    #33396

    Ray Roberts
    Participant

    Dave,

    No changes.

    Yes, postgresql is up.

    I recently did a clean install of HDP, not sure what happened. I can fix the issue by restarting the machine, however, thats not a good longterm solution.

    Thanks,

    -Ray

    Collapse
    #33324

    Dave
    Moderator

    Hi Ray,

    What was the last change that was made on the machine? Have any environment variables been added / modified? Is the postgresql database up and running?

    Thanks

    Dave

    Collapse
    #33263

    Ray Roberts
    Participant

    This may help as well. I’m seeing this in the console in Chrome:

    GET http://hostname:8080/api/v1/clusters/staging/services?fields=components/Servic…nents/host_components,components/host_components/HostRoles&_=1377539100602 500 (java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10) app.js:66668
    App.HttpClient.Em.Object.create.request app.js:66668
    request app.js:66718
    App.HttpClient.Em.Object.create.get app.js:66729
    newFunc vendor.js:2595
    App.UpdateController.Em.Controller.extend.updateServiceMetric app.js:1163
    App.ClusterController.Em.Controller.extend.loadClusterData app.js:1052
    App.MainController.Em.Controller.extend.initialize app.js:1634
    module.exports.Em.Route.extend.enter app.js:33804
    newFunc vendor.js:2595
    Ember.State.Ember.Object.extend.trigger vendor.js:16430
    newFunc vendor.js:2595
    (anonymous function) vendor.js:17462
    Ember.StateManager.Ember.State.extend.enterState vendor.js:17460
    Ember.StateManager.Ember.State.extend.transitionTo vendor.js:17328
    Ember.Router.Ember.StateManager.extend.transitionTo vendor.js:18474
    newFunc vendor.js:2595
    Ember._ResolvedState.Ember.Object.extend.transition vendor.js:17525
    (anonymous function) vendor.js:17877
    runTransition vendor.js:17877
    Ember.Routable.Ember.Mixin.create.routePath vendor.js:17888
    Ember.StateManager.Ember.State.extend.sendRecursively vendor.js:17225
    Ember.StateManager.Ember.State.extend.send vendor.js:17210
    Ember.Router.Ember.StateManager.extend.route vendor.js:18497
    Ember.Application.Ember.Namespace.extend.startRouting vendor.js:11931
    Ember.Application.Ember.Namespace.extend.initialize vendor.js:11897
    window.require.define.initialize app.js:26278
    initModule vendor.js:46
    require vendor.js:55
    (anonymous function) stage:8085/:31
    o vendor.js:91
    p.fireWith vendor.js:91
    e.extend.ready vendor.js:91
    c.addEventListener.B vendor.js:91
    Uncaught SyntaxError: Unexpected token < vendor.js:91
    e.extend.parseJSON vendor.js:91
    App.HttpClient.Em.Object.create.defaultErrorHandler app.js:66641
    (anonymous function)

    Collapse
Viewing 16 replies - 61 through 76 (of 76 total)