- 浏览: 1774777 次
- 性别:
- 来自: 北京
文章分类
最新评论
-
奔跑的小牛:
例子都打不开
如何使用JVisualVM进行性能分析 -
蜗牛coder:
好东西[color=blue][/color]
Lucene学习:全文检索的基本原理 -
lovesunweina:
不在haoop中是在linux系统中,映射IP的时候,不能使用 ...
java.io.IOException: Incomplete HDFS URI, no host -
evening_xxxy:
挺好的, 谢谢分享
如何利用 JConsole观察分析Java程序的运行,进行排错调优 -
di1984HIT:
学习了~~~
ant使用ssh和linux交互 如:上传文件
Hbase不稳定,分析日志发现,归纳总结,目前发现共存在两个问题,一个就是上篇博客提到的问题,还有个问题就是zookeeper的问题
我的异常输出为:
2010-10-28 00:36:49,573 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x9d2be33dbe860005 to sun.nio.ch.SelectionKeyImpl@6655bb93
java.io.IOException: Read error rc = -1 java.nio.DirectByteBuffer[pos=0 lim=4 cap=4]
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:701)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-10-28 00:36:49,602 WARN org.apache.zookeeper.ClientCnxn: Ignoring exception during shutdown input
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownInput(SocketChannelImpl.java:658)
at sun.nio.ch.SocketAdaptor.shutdownInput(SocketAdaptor.java:378)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:999)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-10-28 00:36:49,602 WARN org.apache.zookeeper.ClientCnxn: Ignoring exception during shutdown output
java.net.SocketException: Transport endpoint is not connected
at sun.nio.ch.SocketChannelImpl.shutdown(Native Method)
at sun.nio.ch.SocketChannelImpl.shutdownOutput(SocketChannelImpl.java:669)
at sun.nio.ch.SocketAdaptor.shutdownOutput(SocketAdaptor.java:386)
at org.apache.zookeeper.ClientCnxn$SendThread.cleanup(ClientCnxn.java:1004)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:970)
2010-10-28 00:36:49,622 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: Attempt=1
org.apache.hadoop.hbase.Leases$LeaseStillHeldException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:94)
at org.apache.hadoop.hbase.RemoteExceptionHandler.checkThrowable(RemoteExceptionHandler.java:48)
at org.apache.hadoop.hbase.RemoteExceptionHandler.checkIOException(RemoteExceptionHandler.java:66)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:549)
at java.lang.Thread.run(Thread.java:636)
2010-10-28 00:36:49,703 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: Disconnected, type: None, path: null
2010-10-28 00:36:50,505 INFO org.apache.zookeeper.ClientCnxn: Attempting connection to server /192.168.5.151:2181
2010-10-28 00:36:50,505 INFO org.apache.zookeeper.ClientCnxn: Priming connection to java.nio.channels.SocketChannel[connected local=/192.168.5.156:49407 remote=/192.168.5.151:2181]
2010-10-28 00:36:50,506 INFO org.apache.zookeeper.ClientCnxn: Server connection successful
2010-10-28 00:36:50,507 WARN org.apache.zookeeper.ClientCnxn: Exception closing session 0x9d2be33dbe860005 to sun.nio.ch.SelectionKeyImpl@335819e4
java.io.IOException: Session Expired
at org.apache.zookeeper.ClientCnxn$SendThread.readConnectResult(ClientCnxn.java:589)
at org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:709)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:945)
2010-10-28 00:36:50,507 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: Got ZooKeeper event, state: Expired, type: None, path: null
2010-10-28 00:36:50,507 ERROR org.apache.hadoop.hbase.regionserver.HRegionServer: ZooKeeper session expired
然后,区域节点就会exit!
对了,介绍个好地方哦:http://wiki.apache.org/hadoop/Hbase/Troubleshooting
这里会有些你碰到的问题,对了还个汇总的地方,也介绍下:http://bbs.hadoopor.com/thread-71-1-1.html
说说解决方法:
设置zookeeper的过期时间长一点,默认的过期时间(zookeeper.session.timeout )是60秒,参看这里: http://hbase.apache.org/docs/r0.20.6/hbase-conf.html
他和另外个因素(hbase.zookeeper.property.tickTime )配合使用。
我设置如下:
<property> <name>zookeeper.session.timeout</name> <value>90000</value> </property> <property> <name>hbase.zookeeper.property.tickTime</name> <value>9000</value> </property>
在段时间内,没有发现再出问题,不知道是否根解了。
另外注意细读理解这里的列出来的几点:
Resolution
- Make sure you give plenty of RAM (in hbase-env.sh), the default of 1GB won't be able to sustain long running imports.
- Make sure you don't swap, the JVM never behaves well under swapping.
- Make sure you are not CPU starving the region server thread. For example, if you are running a mapreduce job using 6 CPU-intensive tasks on a machine with 4 cores, you are probably starving the region server enough to create longer garbage collection pauses.
- If you wish to increase the session timeout, add the following to your hbase-site.xml to increase the timeout from the default of 60 seconds to 120 seconds.
<property> <name>zookeeper.session.timeout</name> <value>1200000</value> </property> <property> <name>hbase.zookeeper.property.tickTime</name> <value>6000</value> </property>
- Be aware that setting a higher timeout means that the regions served by a failed region server will take at least that amount of time to be transfered to another region server. For a production system serving live requests, we would instead recommend setting it lower than 1 minute and over-provision your cluster in order the lower the memory load on each machines (hence having less garbage to collect per machine).
-
If this is happening during an upload which only happens once (like initially loading all your data into HBase), consider importing into HFiles directly .
-
HBase ships with some GC tuning, for more information see Performance Tuning .
发表评论
-
HBase配置LZO压缩
2011-07-10 22:40 6103系统: gentoo HDFS: hadoop:hado ... -
HBase RegionServer 退出 ( ZooKeeper session expired)
2011-04-23 08:32 8988RegionServer 由于 ZooKeeper sessi ... -
HBase迁移数据方案1(两个集群不能通信)
2011-03-30 18:23 3805前一篇文章里面介绍了 两个可以直接通信的集群之间很容易拷贝数据 ... -
HBase如何迁移数据
2011-03-10 13:42 6447HBase如何迁移数据?这里有个方案:http://blog. ... -
HBase如何存取多个版本的值
2011-03-07 16:11 27156HBase如何存取多个版本 ... -
HBase简介(很好的梳理资料)
2011-01-30 10:18 130540一、 简介 history s ... -
Google_三大论文中文版(Bigtable、 GFS、 Google MapReduce)
2010-11-28 16:30 22131做个中文版下载源: http://dl.iteye.c ... -
hadoop主节点(NameNode)备份策略以及恢复方法
2010-11-11 19:35 27710一、dits和fsimage 首先要提到 ... -
Bad connect ack with firstBadLink
2010-10-25 13:20 8276hbase报的错误,经过分析是Hadoop不能写入数据了。可恶 ... -
hbase0.20.支持多个主节点容灾切换功能(只激活当前某个节点,其他节点备份)
2010-09-09 14:53 2823http://wiki.apache.org/hadoop/H ... -
java.io.IOException: Incomplete HDFS URI, no host
2010-09-07 08:31 16097ERROR org.apache.hadoop.hdfs.se ... -
升级hadoop0.20.2到hadoop-0.21.0
2010-09-05 11:52 7711按照新的文档来 更新配置: http://hadoop.apa ... -
hadoop-hdfs启动又自动退出的问题
2010-05-20 10:45 6052hadoop-hdfs启动又自动退出的问题,折腾了我1天时间啊 ... -
在windows平台下Eclipse调试Hadoop/Nutch
2010-04-29 14:34 3256即让碰到这个问题说明 准备工作都做好了,软件包,环境什么的这里 ... -
Hadoop运行mapreduce实例时,抛出错误 All datanodes xxx.xxx.xxx.xxx:xxx are bad. Aborting…
2010-04-29 14:26 6358Hadoop运行mapreduce实例时,抛出错误 All d ... -
cygwin 添加用户
2010-04-13 17:48 7340http://hi.baidu.com/skychen1900 ... -
nutch总体输入输出流程图解析
2010-04-12 16:58 2418附件里面有word文档,请下 ... -
解析hadoop框架下的Map-Reduce job的输出格式的实现
2010-04-10 18:34 10083Hadoop 其实并非一个单纯用于存储的分布式文 ... -
nutch分布式搭建
2010-04-06 17:54 6780如何在eclipse中跑nutch :http://jiaj ... -
解析Nutch插件系统
2010-03-31 16:31 6478nutch系统架构的一个亮点就是插件,借鉴这个架构我们 ...
相关推荐
第二课:zookeeper客户端使用与集群特性.docx
ZKInspector:zookeeper节点管理工具,可以很方便地增删改查ZK集群当前的节点状态
第二课:zookeeper客户端使用与集群特性.md
linux环境下:zookeeper3.5.7二进制安装包
面试官:Zookeeper集群怎么搭建?.doc
Maven坐标:org.apache.zookeeper:zookeeper:3.4.6; 标签:apache、zookeeper、jar包、java、API文档、中文版; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,...
Maven坐标:org.apache.zookeeper:zookeeper:3.4.9; 标签:apache、zookeeper、中文文档、jar包、java; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,文档中...
Maven坐标:org.apache.zookeeper:zookeeper:3.4.10; 标签:apache、zookeeper、中文文档、jar包、java; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,文档中...
二、zookeeper概述:概述:ZooKeeper 是一个分布式的,开放源码的分布式应用程序协调服务,是 Google 的 Chubby 一个开源的实现 三、
使用Storm实时处理交通大数据(数据源:kafka,集群管理:zookeeper).zip
Maven坐标:org.apache.zookeeper:zookeeper:3.4.14; 标签:apache、zookeeper、中英对照文档、jar包、java; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,...
赠送jar包:zookeeper-3.3.3.jar; 赠送原API文档:zookeeper-3.3.3-javadoc.jar; 赠送源代码:zookeeper-3.3.3-sources.jar; 包含翻译后的API文档:zookeeper-3.3.3-javadoc-API文档-中文(简体)版.zip 对应...
赠送jar包:zookeeper-3.4.6.jar; 赠送原API文档:zookeeper-3.4.6-javadoc.jar; 赠送源代码:zookeeper-3.4.6-sources.jar; 包含翻译后的API文档:zookeeper-3.4.6-javadoc-API文档-中文(简体)-英语-对照版....
赠送jar包:zookeeper-3.3.3.jar 赠送原API文档:zookeeper-3.3.3-javadoc.jar 赠送源代码:zookeeper-3.3.3-sources.jar 包含翻译后的API文档:zookeeper-3.3.3-javadoc-API文档-中文(简体)-英语-对照版.zip ...
Maven坐标:org.apache.zookeeper:zookeeper:3.4.10; 标签:apache、zookeeper、中英对照文档、jar包、java; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,...
Maven坐标:org.apache.zookeeper:zookeeper:3.4.9; 标签:apache、zookeeper、中英对照文档、jar包、java; 使用方法:解压翻译后的API文档,用浏览器打开“index.html”文件,即可纵览文档内容。 人性化翻译,...
面试官:Zookeeper怎么解决读写、双写并发不一致问题,以及共享锁的实现原理?.doc
-- 此rar文件包含两个ZooKeeper版本 一个3.4.6、一个3.4.14.无需积分可以直接下载,希望能够帮助到你。 zookeeper-3.4.14.tar.gz :2019-04-01 14:45 发布 zookeeper-3.4.6.tar.gz :2017-10-04 10:47 发布 ...
Zookeeper是Hadoop的一个子项目,它是分布式系统中的协调系统,可提供的服务主要有:配置服务、名字服务、分布式同步、组服务等。
华为大数据认证,协调服务组件Zookeeper