Home Forums HDFS HDFS HA from client side

This topic contains 2 replies, has 2 voices, and was last updated by  Pavel Hladik 7 months ago.

  • Creator
    Topic
  • #47154

    Pavel Hladik
    Participant

    Hi,

    We have fully working HDP 2.0.9 stack on our test environment with NameNode HA. If first of NN is in active state and will fall the other will take his place and will change state from standby to active.

    My question is regarding to HA of NN from client side. I mean if we have client host with client libraries and accessing first of NN, we have to detect first NN has fall and switch to second NN manually? Is there any proxy or something like that? Is there automatic process that will let client host know about stop using fallen NN and start using the second one?

    Thanks for answer.

    Pavel

Viewing 2 replies - 1 through 2 (of 2 total)

You must be logged in to reply to this topic.

  • Author
    Replies
  • #47157

    Pavel Hladik
    Participant

    Thanks for quick answer.

    Collapse
    #47156

    Jing Zhao
    Participant

    Currently the DFSClient in Hadoop has this functionality. Basically a DFSClient instance will load the host/port information of both NN from the configuration. When it sends a RPC request, it will first try one of the NNs (specifically, the first one specified in the configuration). Then if the NN is actually in standby state, a StandbyException will be sent from the NN to the client. The client will then automatically failover to the other NN and sends the request there. This client failover process can also happen when the first NN fails and the client receives exceptions like ConnectException.

    This client failover mechanism has also been implemented in webhdfs.

    Collapse
Viewing 2 replies - 1 through 2 (of 2 total)