HDFS Forum

HDFS HA from client side

  • #47154
    Pavel Hladik


    We have fully working HDP 2.0.9 stack on our test environment with NameNode HA. If first of NN is in active state and will fall the other will take his place and will change state from standby to active.

    My question is regarding to HA of NN from client side. I mean if we have client host with client libraries and accessing first of NN, we have to detect first NN has fall and switch to second NN manually? Is there any proxy or something like that? Is there automatic process that will let client host know about stop using fallen NN and start using the second one?

    Thanks for answer.


to create new topics or reply. | New User Registration

  • Author
  • #47156
    Jing Zhao

    Currently the DFSClient in Hadoop has this functionality. Basically a DFSClient instance will load the host/port information of both NN from the configuration. When it sends a RPC request, it will first try one of the NNs (specifically, the first one specified in the configuration). Then if the NN is actually in standby state, a StandbyException will be sent from the NN to the client. The client will then automatically failover to the other NN and sends the request there. This client failover process can also happen when the first NN fails and the client receives exceptions like ConnectException.

    This client failover mechanism has also been implemented in webhdfs.

    Pavel Hladik

    Thanks for quick answer.

You must be to reply to this topic. | Create Account

Support from the Experts

A HDP Support Subscription connects you experts with deep experience running Apache Hadoop in production, at-scale on the most demanding workloads.

Enterprise Support »

Become HDP Certified

Real world training designed by the core architects of Hadoop. Scenario-based training courses are available in-classroom or online from anywhere in the world

Training »

Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.