HBase1.4.13完全分布式安装

版本选择

首先check版本要求！否则后续会遇到难以预料的“惊喜”。以下截图为2020年9月信息，可能随时更新，最新信息自行查询官网标准。

JDK版本：

Hadoop版本：

本机已安装Hadoop2.7.7&JDK1.8，于是选择HBase-1.4.x进行安装。从镜像源中下载hbase-1.4.13-bin.tar.gz。

HBase1.4.13安装

先用scp从网关把压缩包传到Master服务器上：

scp hbase-1.4.13-bin.tar.gz lpj@cpu-node0:/home/lpj/

解压并重命名：

tar -zxf hbase-1.4.13-bin.tar.gz -C /home/lpj/
mv hbase-1.4.13 hbase1.4

配置环境变量，将hbase下的bin目录添加到path中，这样，启动hbase就无需到/usr/local/hbase目录下，大大地方便了hbase的使用。编辑~/.bashrc文件，先前在安装Hadoop时已经引入过PATH，直接在export PATH这行追加/home/lpj/hbase1.4/bin，这里的“：”是分隔符：

export PATH=$PATH:/home/lpj/hadoop2.7/bin:/home/lpj/hadoop2.7/sbin:/home/lpj/hbase1.4/bin

编辑完成后，用source ~/.bashrc使上述配置在当前终端立即生效。用hbase version查看HBase版本，确定HBase安装成功：

[lpj@cpu-node0 ~]$ hbase version
HBase 1.4.13
Source code repository git://Sakthis-MacBook-Pro-2.local/Users/sakthi/dev/hbase revision=38bf65a22b7e9320f07aeb27677e4533b9a77ef4
Compiled by sakthi on Sun Feb 23 02:06:36 PST 2020
From source with checksum cfb98e5fbeeca2068278ea88175d751b

HBase配置

配置hbase-env.sh

修改conf文件夹里的hbase-env.sh文件：

# The java implementation to use.  Java 1.8+ required.
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk

# Extra Java CLASSPATH elements.  Optional.
export HBASE_CLASSPATH=/home/lpj/hbase1.4/conf

# Configure PermSize. Only needed in JDK7. You can safely remove it for JDK8+
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m -XX:ReservedCodeCacheSize=256m"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -XX:PermSize=128m -XX:MaxPermSize=128m -XX:ReservedCodeCacheSize=256m"

# Tell HBase whether it should manage it's own instance of ZooKeeper or not.
export HBASE_MANAGES_ZK=false

配置hbase-site.xml

修改conf文件夹里的hbase-site.xml文件：

<configuration>
  <property>
         <name>hbase.rootdir</name>
         <value>hdfs://cpu-node0:9000/hbase</value>
  </property>
  <property>
         <name>hbase.cluster.distributed</name>
         <value>true</value>
         <description>The mode the cluster will be in. Possible values are
      false: standalone and pseudo-distributed setups with managed Zookeeper
      true: fully-distributed with unmanaged Zookeeper Quorum (see hbase-env.sh)
        </description>
  </property>
  <property>
        <name>hbase.zookeeper.quorum</name>
        <value>cpu-node0,cpu-node3</value>
  </property>
  <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>3384</value>
  </property>
  <property>
        <name>hbase.zookeeper.property.dataDir</name>
        <value>/home/lpj/zookeeper3.4/zkdata</value>
  </property>
  <property>
        <name>hbase.zookeeper.peerport</name>
        <value>2878</value>
  </property>
  <property>
        <name>hbase.zookeeper.leaderport</name>
        <value>3878</value>
  </property>
  <property>
        <name>hbase.tmp.dir</name>
        <value>/home/lpj/hbase1.4/tmp/hbase</value>
  </property>
  <property>
        <name>dfs.replication</name>
        <value>1</value>
  </property>
  <property>
        <name>hbase.regionserver.port</name>
        <value>16021</value>
  </property>    
  <property>
        <name>hbase.regionserver.info.port</name>
        <value>16031</value>
  </property>  
</configuration>

注意：

• hbase.rootdir值是根据%HADOOP_HOME%/etc/hadoop下配置文件core-site.xml中fs.default.name的值加上/hbase.
• hbase.zookeeper.property.dataDir的值是ZooKeeper的zoo.cfg文件里的dataDir的值，hbase.zookeeper.property.clientPort的值是同一个文件中clientPort的值。
• hbase.zookeeper.peerport和hbase.zookeeper.leaderport分别是zoo.cfg里server.x的ip地址后面的两个端口，如果每个节点设置得不一样的话，hbase-site.xml复制过去之后记得修改。

配置regionservers

在conf/regionservers中把从节点写进去：

cpu-node3

配置backup-masters

新建一个backup-masters文件并写入备份节点：

cpu-node0

分发并同步安装包

scp -r hbase1.4 lpj@cpu-node3:/home/lpj/

Start & Check

在Master节点上运行启动脚本start-hbase.sh：

[lpj@cpu-node0 ~]$ start-hbase.sh
running master, logging to /home/lpj/hbase1.4/bin/../logs/hbase-lpj-master-cpu-node0.hustlab.out
cpu-node3: running regionserver, logging to /home/lpj/hbase1.4/bin/../logs/hbase-lpj-regionserver-cpu-node3.hustlab.out
cpu-node0: master running as process 28346. Stop it first.

然后用jps分别在主从节点上检查，如果Master上有HMaster、Slave上有HRegionServer即为启动成功：

[lpj@cpu-node0 ~]$ jps
28657 Jps
10293 QuorumPeerMain
9495 SecondaryNameNode
28346 HMaster
10172 JobHistoryServer
9149 NameNode
9791 ResourceManager

[lpj@cpu-node3 ~]$ jps
22833 QuorumPeerMain
22338 DataNode
23861 HRegionServer
22486 NodeManager
24189 Jps

如何区分启动的ZooKeeper是HBase自带的还是外部的：若jps中是QuorumPeerMain则为外部自行安装的，HQuorumPeerMain则为HBase自带的。

HBase Shell实践

用hbase shell进入shell界面：

[lpj@cpu-node0 ~]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/lpj/hbase1.4/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/lpj/hadoop2.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 1.4.13, r38bf65a22b7e9320f07aeb27677e4533b9a77ef4, Sun Feb 23 02:06:36 PST 2020

hbase(main):001:0>

简单操作一下：

hbase(main):001:0> list
TABLE
0 row(s) in 0.2460 seconds

=> []
hbase(main):002:0> help
HBase Shell, version 1.4.13, r38bf65a22b7e9320f07aeb27677e4533b9a77ef4, Sun Feb 23 02:06:36 PST 2020
Type 'help "COMMAND"', (e.g. 'help "get"' -- the quotes are necessary) for help on a specific command.
Commands are grouped. Type 'help "COMMAND_GROUP"', (e.g. 'help "general"') for help on a command group.

COMMAND GROUPS:
  Group name: general
  Commands: processlist, status, table_help, version, whoami

  Group name: ddl
  Commands: alter, alter_async, alter_status, create, describe, disable, disable_all, drop, drop_all, enable, enable_all, exists, get_table, is_disabled, is_enabled, list, list_regions, locate_region, show_filters

  Group name: namespace
  Commands: alter_namespace, create_namespace, describe_namespace, drop_namespace, list_namespace, list_namespace_tables

  Group name: dml
  Commands: append, count, delete, deleteall, get, get_counter, get_splits, incr, put, scan, truncate, truncate_preserve

  Group name: tools
  Commands: assign, balance_switch, balancer, balancer_enabled, catalogjanitor_enabled, catalogjanitor_run, catalogjanitor_switch, cleaner_chore_enabled, cleaner_chore_run, cleaner_chore_switch, clear_deadservers, close_region, compact, compact_rs, compaction_state, flush, is_in_maintenance_mode, list_deadservers, major_compact, merge_region, move, normalize, normalizer_enabled, normalizer_switch, split, splitormerge_enabled, splitormerge_switch, trace, unassign, wal_roll, zk_dump

  Group name: replication
  Commands: add_peer, append_peer_tableCFs, disable_peer, disable_table_replication, enable_peer, enable_table_replication, get_peer_config, list_peer_configs, list_peers, list_replicated_tables, remove_peer, remove_peer_tableCFs, set_peer_bandwidth, set_peer_tableCFs, show_peer_tableCFs, update_peer_config

  Group name: snapshots
  Commands: clone_snapshot, delete_all_snapshot, delete_snapshot, delete_table_snapshots, list_snapshots, list_table_snapshots, restore_snapshot, snapshot

  Group name: configuration
  Commands: update_all_config, update_config

  Group name: quotas
  Commands: list_quotas, set_quota

  Group name: security
  Commands: grant, list_security_capabilities, revoke, user_permission

  Group name: procedures
  Commands: abort_procedure, list_procedures

  Group name: visibility labels
  Commands: add_labels, clear_auths, get_auths, list_labels, set_auths, set_visibility

  Group name: rsgroup
  Commands: add_rsgroup, balance_rsgroup, get_rsgroup, get_server_rsgroup, get_table_rsgroup, list_rsgroups, move_servers_rsgroup, move_servers_tables_rsgroup, move_tables_rsgroup, remove_rsgroup, remove_servers_rsgroup

SHELL USAGE:
Quote all names in HBase Shell such as table and column names.  Commas delimit
command parameters.  Type <RETURN> after entering a command to run it.
Dictionaries of configuration used in the creation and alteration of tables are
Ruby Hashes. They look like this:

  {'key1' => 'value1', 'key2' => 'value2', ...}

and are opened and closed with curley-braces.  Key/values are delimited by the
'=>' character combination.  Usually keys are predefined constants such as
NAME, VERSIONS, COMPRESSION, etc.  Constants do not need to be quoted.  Type
'Object.constants' to see a (messy) list of all constants in the environment.

If you are using binary keys or values and need to enter them in the shell, use
double-quote'd hexadecimal representation. For example:

  hbase> get 't1', "key\x03\x3f\xcd"
  hbase> get 't1', "key\003\023\011"
  hbase> put 't1', "test\xef\xff", 'f1:', "\x01\x33\x40"

The HBase shell is the (J)Ruby IRB with the above HBase-specific commands added.
For more on the HBase Shell, see http://hbase.apache.org/book.html
hbase(main):003:0> exit

Reference

• 林子雨HBase教程（不过主要讲了单机和伪分布模式）
• CSDN博文：Hadoop+hbase+zookeeper完全分布式搭建
• Apache HBase官方文档（特别是7.Default Configuration和8.Example Configurations，参考意义较大）