Hbase的安装及简单使用

/ hadoopBig DataHbaseDatabase技术 / 没有评论 / 4371浏览

alt 最近在学习使用Hbase,从安装到使用简单操作了一遍。安装Hbase的前提是已经安装配置好了Hadoop和zookeeper。这里就不在细讲了,后面可能会在补充一下。

安装Hbase

  1. 下载Hbase
wget http://mirrors.hust.edu.cn/apache/hbase/stable/hbase-1.2.6.1-bin.tar.gz
tar -xzvf hbase-1.2.6.1-bin.tar.gz

在单机模式下配置HBase

  1. 在hbase-env.sh配置JAVA_HOME环境变量
cd /home/aozhang/Documents/company/app/hbase-1.2.6.1/conf
vim hbase-env.sh
export JAVA_HOME=/home/aozhang/Documents/company/app/jdk1.8.0_181
  1. Edit conf/hbase-site.xml
<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>file:///home/aozhang/Documents/company/data/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/home/aozhang/Documents/company/data/hbase/zookeeper</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
    <description>
      Controls whether HBase will check for stream capabilities (hflush/hsync).

      Disable this if you intend to run on LocalFileSystem, denoted by a rootdir
      with the 'file://' scheme, but be mindful of the NOTE below.

      WARNING: Setting this to false blinds you to potential data loss and
      inconsistent system state in the event of process and/or node failures. If
      HBase is complaining of an inability to use hsync or hflush it's most
      likely not a false positive.
    </description>
  </property>
</configuration>
  1. 启动HBASE
cd /home/aozhang/Documents/company/app/hbase-1.2.6.1/bin
./start-hbase.sh
  1. 访问页面是否正常

ubuntu16搭建OpenTSDB中的大坑hosts文件

127.0.1.1	aozhang-Latitude-5290
192.168.31.111	aozhang-Latitude-5290

hbase server 断电后发现Master启动不了

查看master相关日志,删除日志中报错数据

cd ../data/hbase/WALs/
rm -rf aozhang-latitude-5290,16201,1532424355724-splitting

HBase导入创建表脚本报错:Compression algorithm 'lzo' previously failed test.

Type "exit<RETURN>" to leave the HBase Shell
Version 1.2.6.1, rUnknown, Sun Jun  3 23:19:26 CDT 2018

create 'tsdb-uid',
  {NAME => 'id', COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'},
  {NAME => 'name', COMPRESSION => 'LZO', BLOOMFILTER => 'ROW'}

ERROR: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.RuntimeException: java.lang.ClassNotFoundException: com.hadoop.compression.lzo.LzoCodec Set hbase.table.sanity.checks to false at conf or table descriptor if you want to bypass sanity checks
	at org.apache.hadoop.hbase.master.HMaster.warnOrThrowExceptionForFailure(HMaster.java:1754)
<property>  
  <name>hbase.table.sanity.checks</name>  
  <value>false</value>  
</property>  

为OpenTSDB导入脚本(需导入代码版本对应的脚本)

/home/aozhang/Documents/company/app/hbase-1.2.6.1/create_table.sh

启动OpenTSDB报错

2018-07-27 21:00:10.616  WARN 3973 --- [e I/O Worker #1] org.hbase.async.HBaseClient              : Probe Exists(table="tsdb-uid", key=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60], family=null, qualifiers=null, attempt=0, region=null) failed

org.hbase.async.NonRecoverableException: Too many attempts: Exists(table="tsdb-uid", key=[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 58, 65, 115, 121, 110, 99, 72, 66, 97, 115, 101, 126, 112, 114, 111, 98, 101, 126, 60, 59, 95, 60], family=null, qualifiers=null, attempt=11, region=null)
hbase(main):001:0> list
TABLE                                          
tsdb                                                        
tsdb-meta                                                            
tsdb-tree                                                           
tsdb-uid 
    cd /home/aozhang/Documents/company/app/opentsdb-2.3.1
    env COMPRESSION=NONE HBASE_HOME=/home/aozhang/Documents/company/app/hbase-1.2.6.1 ./src/create_table.sh

HBase是什么?

HBase的存储机制

Hbase操作

  1. 启动 HBase Shell
/home/aozhang/Documents/company/app/hbase-1.2.6.1/bin/hbase shell
#查看所有表
hbase(main):001:0> list
TABLE  
tsdb  
tsdb-meta 
tsdb-tree 
tsdb-uid 
4 row(s) in 0.0300 seconds
=> ["tsdb", "tsdb-meta", "tsdb-tree", "tsdb-uid"]
# 提供HBase的状态,例如,服务器的数量。
hbase(main):002:0> status
1 active master, 0 backup masters, 1 servers, 0 dead, 6.0000 average load
  1. 一系列命令
list :查看有哪些表
status:命令返回包括在系统上运行的服务器的细节和系统的状态
version:该命令返回HBase系统使用的版本
table_help:此命令将引导如何使用表引用的命令。下面给出的是使用这个命令的语法。
whoami:该命令返回HBase用户详细信息。如果执行这个命令,返回当前HBase用户

HBase在表中操作的命令

数据操纵语言

HBaseAdmin类

HBaseAdmin是一个类表示管理。这个类属于org.apache.hadoop.hbase.client包。使用这个类,可以执行管理员任务。使用Connection.getAdmin()方法来获取管理员的实例。

  1. 创建一个新的表
void createTable(HTableDescriptor desc)
  1. 创建一个新表使用一组初始指定的分割键限定空区域
void createTable(HTableDescriptor desc, byte[][] splitKeys)
  1. 从表中删除列
void deleteColumn(byte[] tableName, String