catalog节点
yum install -y impala-server impala impala-state-store impala-catalog
worker 节点
yum install -y impala-server impala
配置
打开 “短路读取” 和 打开”块位置跟踪”
所谓的短路读取,就是允许impala把一些信息存储在本地磁盘上,可以加快计算的速度。
修改 /etc/hadoop/conf/hdfs-site.xml 增加或者修改以下项
<!-- for impala -->
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/var/run/hdfs-sockets/dn._PORT</value>
</property>
<property>
<name>dfs.client.file-block-storage-locations.timeout.millis</name>
<value>10000</value>
</property>
创建相关文件夹
mkdir -p /var/run/hdfs-sockets/ && chown -R hdfs.hdfs /var/run/hdfs-sockets/
mkdir -p /data/log/impala && chown -R impala.impala /data/log/impala
usermod -a -G hadoop impala
usermod -a -G hdfs impala
配置 impala 参数 /etc/default/impala
IMPALA_CATALOG_SERVICE_HOST=namenode
IMPALA_STATE_STORE_HOST=namenode
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/data/log/impala
IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} "
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
-log_dir=${IMPALA_LOG_DIR} \
-catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
-state_store_port=${IMPALA_STATE_STORE_PORT} \
-use_statestore \
-state_store_host=${IMPALA_STATE_STORE_HOST} \
-be_port=${IMPALA_BACKEND_PORT}"
ENABLE_CORE_DUMPS=true
# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
IMPALA_HOME=/usr/lib/impala
HADOOP_HOME=/usr/lib/hadoop
HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
IMPALA_CONF_DIR=/etc/impala/conf
HADOOP_CONF_DIR=/etc/hadoop/conf
HIVE_CONF_DIR=/etc/hive/conf
# HBASE_CONF_DIR=/etc/impala/conf
启动
sudoserviceimpala−state−storestartsudo service impala-catalog start
work节点
$ sudo service impala-server start
参考文章
impala-shell需要在python2.6 环境进行安装(centos6的默认环境),然后随便选择一个impala-server 进行连接。
impala-shell -i ${impalad_host}
版权声明:本文为wankunde原创文章,遵循CC 4.0 BY-SA版权协议,转载请附上原文出处链接和本声明。