zookeeper node failure causes regionserver shutdown?

to create new topics or reply. | New User Registration

This topic contains 2 replies, has 3 voices, and was last updated by  Sanjeev T 2 months ago.

  • Creator
  • #45572

    David Nguyen

    I have a regionserver that is running hdfs datanode + hbase regionserver only and the regionserver shutdown right when one of the zookeeper nodes in the cluster went offline. This is what is in the regionserver log around that time:

    2013-12-16 02:44:16,268 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to sac-prod1-ops-hbase-06.unix.newokl.com/, initiating session
    2013-12-16 02:44:16,274 INFO org.apache.zookeeper.ClientCnxn: Unable to reconnect to ZooKeeper service, session 0x442d9dd48ef0002 has expired, closing socket connection
    2013-12-16 02:44:16,276 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server sac-prod1-ops-hbase-02.unix.newokl.com,60020,1386634675805: regionserver:60020-0x442d9dd48ef0002 regionserver:60020-0x442d9dd48ef0002 received expired from ZooKeeper, aborting
    org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired
    at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.connectionEvent(ZooKeeperWatcher.java:363)
    at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:282)
    at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:519)
    at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
    2013-12-16 02:44:16,278 FATAL org.apache.hadoop.hbase.regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
    2013-12-16 02:44:16,280 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Dump of metrics: requestsPerSecond=0, ……..
    2013-12-16 02:44:16,284 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: STOPPED: regionserver:60020-0x442d9dd48ef0002 regionserver:60020-0x442d9dd48ef0002 received expired from ZooKeeper, aborting

    I’m running HDP 1.3.3 (hbase- on centos6. Has anybody seen this one before?

Viewing 2 replies - 1 through 2 (of 2 total)

You must be to reply to this topic. | Create Account

  • Author
  • #72289

    Sanjeev T

    how many zookeeper nodes are there in quorum.



    Hi David,

    Any errors you see in the Zookeeper logs?


Viewing 2 replies - 1 through 2 (of 2 total)
Hortonworks Data Platform
The Hortonworks Data Platform is a 100% open source distribution of Apache Hadoop that is truly enterprise grade having been built, tested and hardened with enterprise rigor.
Get started with Sandbox
Hortonworks Sandbox is a self-contained virtual machine with Apache Hadoop pre-configured alongside a set of hands-on, step-by-step Hadoop tutorials.
Modern Data Architecture
Tackle the challenges of big data. Hadoop integrates with existing EDW, RDBMS and MPP systems to deliver lower cost, higher capacity infrastructure.