Home Forums HDP on Linux – Installation Ambari Stuck after Login

This topic contains 76 replies, has 6 voices, and was last updated by  Srimanth Gunturi 3 months, 2 weeks ago.

  • Creator
    Topic
  • #33261

    Ray Roberts
    Participant

    I have Ambari 1.2.4.9-1 installed and was working fine. I now have an issue where I go to the Ambari webpage and it is stuck on ‘Loading’.

    I’ve tried restarting both the agent and server with no success.

    I see this in the ambari-server.log:

    10:32:44,598 INFO AmbariLocalUserDetailsService:62 – Loading user by name: admin
    10:32:45,110 INFO HeartBeatHandler:108 – Received heartbeat from host, hostname=xxxxxxxx.domain.com, currentResponseId=2, receivedResponseId
    =2
    10:32:45,110 INFO AgentResource:109 – Sending heartbeat response with response id 3
    10:32:45,210 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,392 INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    10:32:45,935 ERROR ReadHandler:82 – Caught a runtime exception executing a query
    com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 10

    Any ideas what is causing this?

Viewing 30 replies - 1 through 30 (of 76 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #51339

    Srimanth Gunturi
    Participant

    Hi Ray,
    Just by looking at the exceptions I am not able to determine who is setting the incorrect value for ‘ambari’ key of Local Storage.
    This needs to be debugged in the JavaScript debugger of Chrome as to which line sets this incorrect value.
    Once the incorrect value is set, UI trips at login trying to determine user.
    Regards,
    Srimanth

    Collapse
    #51338

    Ray Roberts
    Participant

    So, I’ve tried upgrading to Ambari 1.5.0 with no change in behavior.

    I’d really like to resolve this issue, however, I’m running out of time…..I’ve got angry developers breathing down my neck :) …..So, I’m going to blow it up and redeploy HDP. Luckily, this instance doesn’t have valuable data resident in HDFS, so theres not much of a loss. We mainly use this instance for test and integration cycles for our developers to run their code against.

    Thanks for all your help, I appreciate it.

    -Ray

    Collapse
    #51271

    Ray Roberts
    Participant

    Srimanth,

    Was the info in below post what you needed?

    -Ray

    Collapse
    #51110

    Ray Roberts
    Participant

    Srimanth,

    Nope, it just displays the login prompt again.

    Here is what I see under ‘ambari’:

    {“haStatus”:”false”,”secureUserInfo”:[{"id":"puppet var","name":"hdfs_user","value":"hdfs"},{"id":"puppet var","name":"yarn_user","value":"yarn"},{"id":"puppet var","name":"mapred_user","value":"mapred"},{"id":"puppet var","name":"hbase_user","value":"hbase"},{"id":"puppet var","name":"hive_user","value":"hive"},{"id":"puppet var","name":"proxyuser_group","value":"users"},{"id":"puppet var","name":"smokeuser","value":"ambari-qa"},{"id":"puppet var","name":"zk_user","value":"zookeeper"},{"id":"puppet var","name":"oozie_user","value":"oozie"},{"id":"puppet var","name":"nagios_user","value":"nagios"},{"id":"puppet var","name":"user_group","value":"hadoop"}],”status”:null,”currentStep”:1}

    -Ray

    Collapse
    #51109

    Srimanth Gunturi
    Participant

    Hi Ray,
    That is good. If you then do a Ctrl+Shift+R or Command+Shift+R to refresh browser, you are not able to login?
    If so, can you provide the JSON 'ambari' value in Local Resources?
    Regards,
    Srimanth

    Collapse
    #51108

    Ray Roberts
    Participant

    Srimanth,

    Here is what I see under ‘Resources’ tab after logging in and running ‘App.db.cleanUp():

    {“app”:{“loginName”:”",”authenticated”:false,”configs”:[],”tables”:{“filterConditions”:{},”displayLength”:{},”startIndex”:{},”sortingConditions”:{}}},”Installer”:{},”AddHost”:{},”AddService”:{},”StackUpgrade”:{},”ReassignMaster”:{},”AddSecurity”:{},”HighAvailabilityWizard”:{},”RollbackHighAvailabilityWizard”:{}}

    -Ray

    Collapse
    #50740

    Srimanth Gunturi
    Participant

    Hi Ray,
    After you ran the ‘App.db.cleanUp()’ did the value for ‘ambari’ change so as to have ‘app’ key in it?
    If you could provide the sanitized value that is set after ‘App.db.cleanUp()’ that would be helpful.
    Regards,
    Srimanth

    Collapse
    #50701

    Ray Roberts
    Participant

    Srimanth,

    Here is what I see:

    Uncaught TypeError: Cannot set property ‘user’ of undefined db.js:143
    TRACE: The url is: /api/v1/services/AMBARI/components/AMBARI_SERVER ajax.js:1362
    loading ambari properties cluster_controller.js:444
    Status code 200: Success. statusCodes.js:22
    App.serviceMapper execution time: 0.132ms service_mapper.js:38
    App.statusMapper execution time: 99.965ms status_mapper.js:118
    App.serviceMetricsMapper execution time: 13.667ms service_metrics_mapper.js:254
    UpdateServiceMetric() Finished in:106 ms update_controller.js:180
    App.componentConfigMapper execution time: 5.405ms component_config_mapper.js:59
    App.hostsMapper execution time: 16.594ms hosts_mapper.js:156
    App.alertsMapper execution time: 97.844ms alerts_mapper.js:92
    App.db.cleanUp();
    TRACE: Entering db:cleanup function db.js:71
    In cleanup./.. db.js:73
    undefined

    Collapse
    #50698

    Srimanth Gunturi
    Participant

    Hi Ray,
    The data you have as value for ‘ambari’ key is not in the format we expect.
    Can you please switch to the Console tab and run the command “App.db.cleanUp();”. Following this please verify the ‘ambari value has changed to something which has “app”:{} in it? If not, please provide the value you see immediately after running the command.
    Regards,
    Srimanth

    Collapse
    #50447

    Ray Roberts
    Participant

    Srimanth,

    It still shows the same behavior in incognito mode. Here is the value under the key ambari:

    {“haStatus”:”false”,”secureUserInfo”:[{"id":"puppet var","name":"hdfs_user","value":"hdfs"},{"id":"puppet var","name":"yarn_user","value":"yarn"},{"id":"puppet var","name":"mapred_user","value":"mapred"},{"id":"puppet var","name":"hbase_user","value":"hbase"},{"id":"puppet var","name":"hive_user","value":"hive"},{"id":"puppet var","name":"proxyuser_group","value":"users"},{"id":"puppet var","name":"smokeuser","value":"ambari-qa"},{"id":"puppet var","name":"zk_user","value":"zookeeper"},{"id":"puppet var","name":"oozie_user","value":"oozie"},{"id":"puppet var","name":"nagios_user","value":"nagios"},{"id":"puppet var","name":"user_group","value":"hadoop"}],”status”:null,”currentStep”:1}

    Collapse
    #50419

    Srimanth Gunturi
    Participant

    Hi Ray,
    Thanks for the console/network output – it clears some things from our end.

    You mentioned sometime back that this happens on all browsers (Chrome/FF/IE) – can you please confirm that is still true.
    Did you try accessing Ambari UI from Chrome’s Incognito mode (File > New Incognito Window)?
    If the same problem persists, can you please open the Chrome JavaScript Console again and go to the ‘Resources’ tab. Under the ‘Local Resources’ entry you should see your server URL. Select your server URL to see key/values being stored locally. You should see a key named ‘ambari’ and a JSON value starting with the below.

    {
    "app" : {
    ...
    }
    }

    Can you please verify that you see the above?
    If not, please provide details on where things dont match.
    Regards,
    Srimanth

    Collapse
    #50418

    Ray Roberts
    Participant

    Srimanth,

    Thanks.

    So, in the console I see the following:

    CLDR.pluralForm not found. Em.I18n will not support count-based inflection. ember-i18n-1.4.1.js:28
    ‘window.webkitStorageInfo’ is deprecated. Please use ‘navigator.webkitTemporaryStorage’ or ‘navigator.webkitPersistentStorage’ instead. ember-latest.js:4848
    STATEMANAGER: Entering root ember-latest.js:15815
    TRACE: Entering db:getUser function db.js:516
    Uncaught TypeError: Cannot read property ‘user’ of undefined db.js:518
    App.db.getUser db.js:518
    App.Router.Em.Router.extend.initAdmin router.js:343
    App.Router.Em.Router.extend.root.Em.Route.extend.enter router.js:356
    newFunc ember-latest.js:949
    Ember.State.Ember.Object.extend.trigger ember-latest.js:14784
    newFunc ember-latest.js:949
    (anonymous function) ember-latest.js:15816
    Ember.StateManager.Ember.State.extend.enterState ember-latest.js:15814
    Ember.StateManager.Ember.State.extend.transitionTo ember-latest.js:15682
    Ember.Router.Ember.StateManager.extend.transitionTo ember-latest.js:16828
    newFunc ember-latest.js:949
    Ember.StateManager.Ember.State.extend.init ember-latest.js:15510
    newFunc ember-latest.js:949
    Ember.Router.Ember.StateManager.extend.init ember-latest.js:16949
    newFunc ember-latest.js:949
    Class ember-latest.js:8259
    Ember.Mixin.create.create ember-latest.js:8398
    Ember.Application.Ember.Namespace.extend.initialize ember-latest.js:10226
    (anonymous function) initialize.js:55
    initModule vendor.js:46
    require vendor.js:55
    (anonymous function) (index):31
    o jquery-1.7.2.min.js:2
    p.fireWith jquery-1.7.2.min.js:2
    e.extend.ready jquery-1.7.2.min.js:2
    c.addEventListener.B jquery-1.7.2.min.js:2
    event.returnValue is deprecated. Please use the standard event.preventDefault() instead. jquery-1.7.2.min.js:3

    Everything in the network tab seems ok.

    Collapse
    #50383

    Srimanth Gunturi
    Participant

    Hi Ray,
    From the browser can you please follow the instructions in the page below and provide additional details regarding the issue?
    https://cwiki.apache.org/confluence/display/AMBARI/Troubleshoot+Ambari+UI+issues
    Regards,
    Srimanth

    Collapse
    #50354

    Ray Roberts
    Participant

    So, I found this while poking in the postgresql db(ambari).

    When I do: select * from user_roles;
    I see 2 roles, user and admin. Also, they both have the userid of 1. Not sure if this is a problem or not.

    When I do the same command on my other cluster where Ambari is working fine, I only get one user role: admin

    Could this be a problem?

    Collapse
    #50321

    Ray Roberts
    Participant

    Ok, so I think I may have found something…..

    I have two clusters running HDP 1.3.2 with Ambari 1.4.4. One cluster is working and the other is experiencing the issues with Ambari I’ve noted below.

    On the cluster where Ambari is working, when I tail /var/log/ambari-server/ambari-server.log while logging into Ambari, I see the following:

    INFO AmbariLocalUserDetailsService:62 – Loading user by name: admin
    INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    INFO PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    INFO PersistKeyValueService:82 – Looking for keyName user-pref-admin-dashboard

    However, on the cluster where Ambari is not working and just displaying ‘Loading’ after I Iog in, I see the following in it’s ambari-server.log:

    INFO [qtp626967026-36] AmbariLocalUserDetailsService:62 – Loading user by name: admin
    INFO [qtp626967026-33] PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS
    INFO [qtp626967026-33] PersistKeyValueService:82 – Looking for keyName CLUSTER_CURRENT_STATUS

    Its missing the ‘Looking for keyName user-pref-admin-dashboard’ section of the log that shows up in the log of the working cluster.

    Not sure why, but at least I have something that is different from the working cluster.

    Collapse
    #50304

    Ray Roberts
    Participant

    Sid,
    Thanks for the reply.

    I have tried stopping nagios and restarting ambari, however, I get the same behavior…….ambari login page loads, then I login, and then it just sits at ‘Loading’.

    I have upgraded to 1.4.4, this is what has caused this behavior.

    Its really unfortunate, its in a development environment, so not a huge deal to blow it up, however I hate these types of fixes. It makes me think twice before putting Ambari into production.

    Collapse
    #50303

    Sid Wagle
    Participant

    Hi Robert,

    Have you tried stopping Nagios (do not start it) and then restart Ambari ?

    Another suggestion: If you upgrade to Ambari-1.4.4, this should problem should go away (AMBARI-3846. Was fixed in 1.4.2)

    Best Regards,
    Sid

    Collapse
    #50302

    Ray Roberts
    Participant

    Well,

    I’ve tried everything I could think of, however, I’m not getting anywhere.

    Unless someone else has another solution, I’m going to blow up Ambari and reinstall it.

    I’d love to know what is causing this issue though.

    Collapse
    #50032

    Ray Roberts
    Participant

    I’ve also noticed that in my ambari.properties file, the ‘ambari-server.user’ is set to ambari-server, however, on another cluster that is running Ambari 1.4.1, its set to root.

    I’ve tried changing this, however, I still get the same behavior as below.

    Collapse
    #49971

    Ray Roberts
    Participant

    ….I forgot to expand the error below:

    Uncaught TypeError: Cannot read property ‘user’ of undefined db.js:518
    App.db.getUser db.js:518
    App.Router.Em.Router.extend.initAdmin router.js:343
    App.Router.Em.Router.extend.root.Em.Route.extend.enter router.js:356
    newFunc ember-latest.js:949
    Ember.State.Ember.Object.extend.trigger ember-latest.js:14784
    newFunc ember-latest.js:949
    (anonymous function) ember-latest.js:15816
    Ember.StateManager.Ember.State.extend.enterState ember-latest.js:15814
    Ember.StateManager.Ember.State.extend.transitionTo ember-latest.js:15682
    Ember.Router.Ember.StateManager.extend.transitionTo ember-latest.js:16828
    newFunc ember-latest.js:949
    Ember.StateManager.Ember.State.extend.init ember-latest.js:15510
    newFunc ember-latest.js:949
    Ember.Router.Ember.StateManager.extend.init ember-latest.js:16949
    newFunc ember-latest.js:949
    Class ember-latest.js:8259
    Ember.Mixin.create.create ember-latest.js:8398
    Ember.Application.Ember.Namespace.extend.initialize ember-latest.js:10226
    (anonymous function) initialize.js:55
    initModule vendor.js:46
    require vendor.js:55
    (anonymous function) (index):31
    o jquery-1.7.2.min.js:2
    p.fireWith jquery-1.7.2.min.js:2
    e.extend.ready jquery-1.7.2.min.js:2
    c.addEventListener.B

    Collapse
    #49970

    Ray Roberts
    Participant

    Srimanth,

    Thanks for the response.

    I have tried refreshing and even using different browsers (Safari/Chrom/IE) and I get the same behavior.
    Its weird though, sometimes I’ll get a login window and I login, however, it just sits with the loading bar. Other times I don’t get a login window, and it just says loading int the upper left corner of the browser….no matter which browser I use.

    Here is the console from Chrome:
    CLDR.pluralForm not found. Em.I18n will not support count-based inflection. ember-i18n-1.4.1.js:28
    ‘window.webkitStorageInfo’ is deprecated. Please use ‘navigator.webkitTemporaryStorage’ or ‘navigator.webkitPersistentStorage’ instead.
    STATEMANAGER: Entering root ember-latest.js:15815
    TRACE: Entering db:getUser function db.js:516
    Uncaught TypeError: Cannot read property ‘user’ of undefined db.js:518
    event.returnValue is deprecated. Please use the standard event.preventDefault() instead.

    Collapse
    #49945

    Srimanth Gunturi
    Participant

    Ray, after upgrading to Ambari 1.4.4 did you do a refresh of browser content (Ctrl+Shift+R) so that latest UI was used?
    We fixed a similar issue in Ambari 1.4.2 – https://issues.apache.org/jira/browse/AMBARI-3846 where bad Nagios JSON broke the UI.

    If you are still hitting the problem, can you please send the following information when the problem happens
    - In Chrome console if there is an exception stack trace please provide it.
    - In Chrome console there will be a Network tab showing various server calls. If there is a server call which fails (shown in red, or return code other than 20x), please provide the URL and response from server for that call.
    - If server call from above did occur, please send me the server logs.

    Collapse
    #49893

    Ray Roberts
    Participant

    Was there any solution to this issue? I’ve just upgraded to Ambari 1.4.4 and I’m now experiencing this issue once again.

    I log in and it just sits with ‘Loading’ being displayed.

    When I look at the console within Chrome as I log in to ambari, I notice this error: ‘Uncaught TypeError: Cannot set property ‘user’ of undefined’

    Any ideas?

    Collapse
    #43345

    Josh Hansen
    Member

    @Srimanth, here’s the contents of /etc/httpd/conf.d/hdp_mon_nagios_addons.conf:
    Alias /ambarinagios /usr/share/hdp
    <Directory /usr/share/hdp>
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
    </Directory>

    Collapse
    #43215

    Ray Roberts
    Participant

    Srimanth,

    I get:

    Problem accessing /ambarinagios/nagios/nagios_alerts.php. Reason:

    Not Found

    Collapse
    #43152

    Srimanth Gunturi
    Participant

    Ray, do you get any content trying to access “http://hostname/ambarinagios/nagios/nagios_alerts.php?q1=alerts&alert_type=all” (replace hostname to be Nagios machine name) from the ambari-server machine. If so, please provide the sanitized content. It will help us find out why there are GSON errors.

    Collapse
    #43150

    Ray Roberts
    Participant

    I have the following in my /etc/httpd/conf.d/hdp_mon_nagios_addon.conf:

    Alias /ambarinagios /usr/share/hdp

    Options None
    AllowOverride None
    Order allow,deny
    Allow from all

    Collapse
    #43149

    Ray Roberts
    Participant

    Srimanth,

    What do you mean by …..”Please check that the HTTPD server on that machine is using the “/etc/httpd/conf.d/hdp_mon_nagios_addons.conf” file, as that is the file which maps the “/ambarinagios” to local HDP install. Once this is fixed I think we should be in better shape.”

    Collapse
    #43138

    Ray Roberts
    Participant

    Wow, yes. I totally missed that.

    So, using the correct cluster name, it does return the following:

    ServiceInfo” : {
    “cluster_name” : “ClusterA”,
    “service_name” : “NAGIOS”,
    “state” : “STARTED”,
    “desired_configs…………

    Collapse
    #43131

    Srimanth Gunturi
    Participant

    Hi Ray, In the API call did you replace “clustername” with your specific cluster name (provided during install)?

    Collapse
Viewing 30 replies - 1 through 30 (of 76 total)