转自:http://3199782.blog.51cto.com/3189782/651889
HTable是HBase提供的一个主要客户端接口, 通过它可以实现与HBase集群连接, 进而实现CRUD等一系列功能。
1. HTable如何实现与HBase集群的连接?
HTable是如何知道HBase集群中的HMaser和HRegionServer, 答案是通过Zookeeper, zookeeper是一个分布式协调系统, HMaser,HRegionServer在启动后会自动在zookeeper中注册自己的地址。
HTable实例构造后会查找zookeeper配置, 与zookeeper取的连接, 进而获取HMaser,HRegionServer信息
加载zookeeper配置流程如下:
通过org.apache.hadoop.hbase.zookeeper.ZKConfig 类来加载zookeeper配置:
public static Properties makeZKProps(Configuration conf) { // First check if there is a zoo.cfg in the CLASSPATH. If so, simply read // it and grab its configuration properties. ClassLoader cl = HQuorumPeer.class.getClassLoader(); // zoo.cfg必须在CLASSPATH路径下 final InputStream inputStream = cl.getResourceAsStream(HConstants.ZOOKEEPER_CONFIG_NAME); // 优先使用zoo.cfg配置 if (inputStream != null) { try { return parseZooCfg(conf, inputStream); } catch (IOException e) { LOG.warn("Cannot read " + HConstants.ZOOKEEPER_CONFIG_NAME + ", loading from XML files", e); } } // 在zoo.cfg配置加载失败的情况下再使用hbase的配置文件中指定的配置项 // Otherwise, use the configuration options from HBase's XML files. Properties zkProperties = new Properties(); // Directly map all of the hbase.zookeeper.property.KEY properties. for (Entry<String, String> entry : conf) { String key = entry.getKey(); if (key.startsWith(ZK_CFG_PROPERTY)) { String zkKey = key.substring(ZK_CFG_PROPERTY_SIZE); String value = entry.getValue(); // If the value has variables substitutions, need to do a get. if (value.contains(VARIABLE_START)) { value = conf.get(key); } zkProperties.put(zkKey, value); } } // If clientPort is not set, assign the default if (zkProperties.getProperty(ZK_CLIENT_PORT_KEY) == null) { zkProperties.put(ZK_CLIENT_PORT_KEY, HConstants.DEFAULT_ZOOKEPER_CLIENT_PORT); } // Create the server.X properties. int peerPort = conf.getInt("hbase.zookeeper.peerport", 2888); int leaderPort = conf.getInt("hbase.zookeeper.leaderport", 3888); final String[] serverHosts = conf.getStrings(HConstants.ZOOKEEPER_QUORUM, "localhost"); for (int i = 0; i < serverHosts.length; ++i) { String serverHost = serverHosts[i]; String address = serverHost + ":" + peerPort + ":" + leaderPort; String key = "server." + i; zkProperties.put(key, address); } return zkProperties; }
加载过程主要是先从CLASSPATH中寻找zoo.cfg的配置文件,
没有再使用hbase的配置项,有一个优先级关系, 需要注意, 否则,在zoo.cfg配置文件存在的情况下, 再怎么修改hbase配置文件否无济于事.
2.
连上zookeeper后会监控哪些节点?
HTable连上zookeeper后就会监控一些节点, 以期获取HMaster, HRegionServer等信息, 通过org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher类的setNodeNames方法可以看出:
/** * Set the local variable node names using the specified configuration. */ private void setNodeNames(Configuration conf) { baseZNode = conf.get(HConstants.ZOOKEEPER_ZNODE_PARENT, HConstants.DEFAULT_ZOOKEEPER_ZNODE_PARENT); rootServerZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.rootserver", "root-region-server")); rsZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.rs", "rs")); masterAddressZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.master", "master")); clusterStateZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.state", "shutdown")); assignmentZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.unassigned", "unassigned")); tableZNode = ZKUtil.joinZNode(baseZNode, conf.get("zookeeper.znode.tableEnableDisable", "table")); }
监控的节点有:
/hbase/master
/hbase/root-region-server
/hbase/rs
/hbase/shutdown
/hbase/unassigned
/hbase/table
接下来验证下zookeeper中是否生成了这些节点,
zkCli -server 192.168.203.129:2181
- [zk: 192.168.203.129:2181(CONNECTED) 0]
- [zk: 192.168.203.129:2181(CONNECTED) 0] ls /
- [hbase, zookeeper]
- [zk: 192.168.203.129:2181(CONNECTED) 1] ls /hbase
- [root-region-server, rs, table, unassigned, master, shutdown]
- [zk: 192.168.203.129:2181(CONNECTED) 2]
通过zookeeper客户端可以确认zookeeper中确实创建了上述六个节点
本文出自 “炽天使” 博客,请务必保留此出处http://3199782.blog.51cto.com/3189782/651889