学步园

hive+mysql

<property>    
        <name>hive.metastore.local</name>    
        <value>true</value>    
        <description>controls whether to connect to remove metastore server or open a new metastore server in Hive Client JVM</description>  
</property>  
<property>    
        <name>javax.jdo.option.ConnectionURL</name>    
        <value>jdbc:mysql://hive-mysql:3306/hive?createDatabaseIfNotExist=true</value>    
        <description>JDBC connect string for a JDBC metastore</description>  
</property>  
<property>    
        <name>javax.jdo.option.ConnectionDriverName</name>    
        <value>com.mysql.jdbc.Driver</value>    
        <description>Driver class name for a JDBC metastore</description>  
</property>  
<property>    
        <name>javax.jdo.option.ConnectionUserName</name>    
        <value>hive</value>    
        <description>username to use against metastore database</description>  
</property>  
<property>    
        <name>javax.jdo.option.ConnectionPassword</name>    
        <value>hivepasswd</value>    
        <description>password to use against metastore database</description>  
</property>  

添加jdbc的jar包 到  hive/lib

各种异常

hadoop+hbase

FATAL master.HMaster: Unhandled exception. Starting shutdown.org.apache.hadoop.ipc.RPC$VersionMismatch: Protocol org.apache.hadoop.hdfs.protocol.ClientProtocol version mismatch. (client = 42, server = 41)

把hadoop目录里的hadoop-0.20.2-core.jar复制到hbase的lib

创建hive+hbase表

CREATE TABLE test(uid int, name string,login_time int) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name,cf1:login_time") TBLPROPERTIES ("hbase.table.name" = "test");

创建分区表

CREATE TABLE test(uid int, name string,login_time int)

PARTITIONED BY(year String, month String, day String)

STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,cf1:name,cf1:login_time") TBLPROPERTIES ("hbase.table.name" = "test");

创建临时表

   create table tmp_table(key int,value string) ROW FORMAT DELIMITED
  FIELDS TERMINATED BY '\t';
从文本导入数据

LOAD DATA LOCAL INPATH '/work/hive/examples/files/kv1.txt' OVERWRITE INTO TABLE tmp_table;

从临时表导入数据

    INSERT OVERWRITE TABLE hbase_table_1 SELECT * FROM tmp_table;

从临时表导入到分区表

insert overwrite table test partition(year=2011,month=07,day=11) select * from tmp_login_table;

查询，可以指定不同的分区查询

 select count(*) from test where year='2011';   

select count(*) from test where year='2011' and month='07' and day='11';