Redis集群（cluster）模式部署
Redis Cluster 是社区版推出的 Redis 分布式集群解决方案，主要解决 Redis分布式方面的需求，比如，当遇到单机内存，并发和流量等瓶颈的时候，Redis Cluster 能起到很好的负载均衡的目的。 Redis Cluster 集群节点最小配置 6 个节点以上(3 主 3从)，其中主节点提供读写操作，从节点作为备用节点，不提供请求，只作为故障转移使用。 Redis Cluster采用虚拟槽分区，所有的键根据哈希函数映射到 0～16383 个整数槽内，每个节点负责维护一部分槽以及槽所印映射的键值数据。Redis Cluster 集群模式通常具有 高可用、可扩展性、分布式、容错 等特性。
优点：
无中心架构;
数据按照 slot 存储分布在多个节点，节点间数据共享，可动态调整数据分布;
可扩展性：可线性扩展到 1000 多个节点，节点可动态添加或删除;
高可用性：部分节点不可用时，集群仍可用。通过增加 Slave 做 standby 数据副本，能够实现故障自动 failover，节点之间通过 gossip 协议交换状态信息，用投票机制完成 Slave 到 Master 的角色提升;
降低运维成本，提高系统的扩展性和可用性。
缺点：
Client 实现复杂，驱动要求实现 Smart Client，缓存 slots mapping 信息并及时更新，提高了开发难度，客户端的不成熟影响业务的稳定性。目前仅 JedisCluster 相对成熟，异常处理部分还不完善，比如常见的“max redirect exception”。
节点会因为某些原因发生阻塞(阻塞时间大于 clutser-node-timeout)，被判断下线，这种 failover 是没有必要的。
数据通过异步复制，不保证数据的强一致性。
多个业务使用同一套集群时，无法根据统计区分冷热数据，资源隔离性较差，容易出现相互影响的情况。
Slave 在集群中充当“冷备”，不能缓解读压力，当然可以通过 SDK 的合理设计来提高 Slave 资源的利用率。
Key 批量操作限制，如使用 mset、mget 目前只支持具有相同 slot 值的 Key 执行批量操作。对于映射为不同 slot 值的 Key 由于 Keys 不支持跨 slot 查询，所以执行 mset、mget、sunion 等操作支持不友好。
Key 事务操作支持有限，只支持多 key 在同一节点上的事务操作，当多个 Key 分布于不同的节点上时无法使用事务功能。
Key 作为数据分区的最小粒度，不能将一个很大的键值对象如 hash、list 等映射到不同的节点。
不支持多数据库空间，单机下的 redis 可以支持到 16 个数据库，集群模式下只能使用 1 个数据库空间，即 db 0。
复制结构只支持一层，从节点只能复制主节点，不支持嵌套树状复制结构。
避免产生 hot-key，导致主库节点成为系统的短板。
避免产生 big-key，导致网卡撑爆、慢查询等。
重试时间应该大于 cluster-node-time 时间。
Redis Cluster 不建议使用 pipeline 和 multi-keys 操作，减少 max redirect 产生的场景。
摘自：Redis集群的5种使用方式，各自优缺点分析
环境
node-1 192.168.186.31
node-2 192.168.186.32
node-3 192.168.186.33
(在每一台机器上部署两个redis实例，以满足至少六个redis的要求)
在每一台机器上部署两个redis实例
[root@node-1 ~]# tar xf redis-4.0.6.tar.gz -C /opt/
[root@node-1 ~]# cd /opt/
[root@node-1 ~]# mv redis-4.0.10/ redis-7000/
[root@node-1 ~]# cd redis-7000/
[root@node-1 redis-7000]# make MALLOC=libc
#编译完成即可，不同机器可能会缺少一些环境，根据报错安装相应的环境即可
#创建多实例
[root@node-1 redis-7000]# cp -r redis-7000/ redis-7001/
#修改配置文件
[root@node-1 redis-7000]# vim /opt/redis-7000/redis.conf
主要修改如下：
bind 0.0.0.0
protected-mode no
port 7000
daemonize yes
cluster-enabled yes
cluster-config-file nodes.conf #生成集群后，会保留集群的信息到这个文件
cluster-node-timeout 5000
#其他配置文件也是一样，注意修改端口即可，这里部署的是 每台机器的两个redis实例分别都是7000 和 7001 端口。
#改完配置文件即可启动所有服务
[root@node-1 redis-7000]# /opt/redis-7000/src/redis-server /opt/redis-7000/redis.conf
[root@node-1 redis-7001]# /opt/redis-7001/src/redis-server /opt/redis-7001/redis.conf
#其他两台服务器也是一模一样的操作，可以直接把这台机器上的目录scp到其他机器相应的位置即可启动。
使用 redis-trib.rb创建集群
方法一：
部署ruby环境
[root@node-1 src]# yum -y install ruby ruby-devel rubygems rpm-build
[root@node-1 src]# curl -L get.rvm.io | bash -s stable #有可能会连接失败，多试几次。
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 194 100 194 0 0 204 0 --:--:-- --:--:-- --:--:-- 204
100 24535 100 24535 0 0 5537 0 0:00:04 0:00:04 --:--:-- 25138
Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: directory `/root/.gnupg' created
gpg: new configuration file `/root/.gnupg/gpg.conf' created
gpg: WARNING: options in `/root/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/root/.gnupg/pubring.gpg' created
gpg: Signature made Thu 26 Mar 2020 05:58:42 AM CST using RSA key ID 39499BDB
gpg: Can't check signature: No public key
GPG signature verification failed for '/usr/local/rvm/archives/rvm-1.29.10.tgz' - 'https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc'! Try to install GPG v2 and then fetch the public key:
gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
or if it fails:
command curl -sSL https://rvm.io/mpapis.asc | gpg2 --import -
command curl -sSL https://rvm.io/pkuczynski.asc | gpg2 --import -
In case of further problems with validation please refer to https://rvm.io/rvm/security
#根据提示执行上面的命令
[root@node-1 src]# gpg2 --keyserver hkp://pool.sks-keyservers.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: requesting key D39DC0E3 from hkp server pool.sks-keyservers.net
gpg: requesting key 39499BDB from hkp server pool.sks-keyservers.net
gpg: /root/.gnupg/trustdb.gpg: trustdb created
gpg: key D39DC0E3: public key "Michal Papis (RVM signing) <mpapis@gmail.com>" imported
gpg: key 39499BDB: public key "Piotr Kuczynski <piotr.kuczynski@gmail.com>" imported
gpg: no ultimately trusted keys found
gpg: Total number processed: 2
gpg: imported: 2 (RSA: 2)
#再次执行
[root@node-1 src]# curl -L get.rvm.io | bash -s stable
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 194 100 194 0 0 220 0 --:--:-- --:--:-- --:--:-- 220
100 24535 100 24535 0 0 13675 0 0:00:01 0:00:01 --:--:-- 2994k
Downloading https://github.com/rvm/rvm/archive/1.29.10.tar.gz
Downloading https://github.com/rvm/rvm/releases/download/1.29.10/1.29.10.tar.gz.asc
gpg: Signature made Thu 26 Mar 2020 05:58:42 AM CST using RSA key ID 39499BDB
gpg: Good signature from "Piotr Kuczynski <piotr.kuczynski@gmail.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 7D2B AF1C F37B 13E2 069D 6956 105B D0E7 3949 9BDB
GPG verified '/usr/local/rvm/archives/rvm-1.29.10.tgz'
Creating group 'rvm'
Installing RVM to /usr/local/rvm/
Installation of RVM in /usr/local/rvm/ is almost complete:
* First you need to add all users that will be using rvm to 'rvm' group,
and logout - login again, anyone using rvm will be operating with `umask u=rwx,g=rwx,o=rx`.
* To start using RVM you need to run `source /etc/profile.d/rvm.sh`
in all your open shell windows, in rare cases you need to reopen all shell windows.
* Please do NOT forget to add your users to the rvm group.
The installer no longer auto-adds root or users to the rvm group. Admins must do this.
Also, please note that group memberships are ONLY evaluated at login time.
This means that users must log out then back in before group membership takes effect!
Thanks for installing RVM ?
Please consider donating to our open collective to help us maintain RVM.
? Donate: https://opencollective.com/rvm/donate
[root@node-1 src]# source /usr/local/rvm/scripts/rvm
[root@node-1 src]# rvm list known
# MRI Rubies
[ruby-]1.8.6[-p420]
[ruby-]1.8.7[-head] # security released on head
[ruby-]1.9.1[-p431]
。。。
[root@node-1 src]# rvm install 2.4.0 #安装过程会有点慢
Searching for binary rubies, this might take some time.
No binary rubies available for: centos/7/x86_64/ruby-2.4.0.
Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.
Checking requirements for centos.
Installing requirements for centos.
Installing required packages: autoconf, automake, bison, gcc-c++, libffi-devel, libtool, readline-devel, sqlite-devel, zlib-devel, openssl-devel.......................
Requirements installation successful.
Installing Ruby from source to: /usr/local/rvm/rubies/ruby-2.4.0, this may take a while depending on your cpu(s)...
ruby-2.4.0 - #downloading ruby-2.4.0, this may take a while depending on your connection...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.9M 100 11.9M 0 0 1949k 0 0:00:06 0:00:06 --:--:-- 2834k
ruby-2.4.0 - #extracting ruby-2.4.0 to /usr/local/rvm/src/ruby-2.4.0.....
ruby-2.4.0 - #applying patch /usr/local/rvm/patches/ruby/2.4.0/ruby_2_4_gcc7.patch.
ruby-2.4.0 - #applying patch /usr/local/rvm/patches/ruby/2.4.0/random_c_using_NR_prefix.patch.
ruby-2.4.0 - #configuring..................................................................
ruby-2.4.0 - #post-configuration..
ruby-2.4.0 - #compiling.................................................................................................................
ruby-2.4.0 - #installing...........................
ruby-2.4.0 - #making binaries executable..
ruby-2.4.0 - #downloading rubygems-3.0.8
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 867k 100 867k 0 0 712k 0 0:00:01 0:00:01 --:--:-- 712k
No checksum for downloaded archive, recording checksum in user configuration.
ruby-2.4.0 - #extracting rubygems-3.0.8.....
ruby-2.4.0 - #removing old rubygems........
ruby-2.4.0 - #installing rubygems-3.0.8................................................................
ruby-2.4.0 - #gemset created /usr/local/rvm/gems/ruby-2.4.0@global
ruby-2.4.0 - #importing gemset /usr/local/rvm/gemsets/global.gems................................................................
ruby-2.4.0 - #generating global wrappers.......
ruby-2.4.0 - #gemset created /usr/local/rvm/gems/ruby-2.4.0
ruby-2.4.0 - #importing gemsetfile /usr/local/rvm/gemsets/default.gems evaluated to empty gem list
ruby-2.4.0 - #generating default wrappers.......
ruby-2.4.0 - #adjusting #shebangs for (gem irb erb ri rdoc testrb rake).
Install of ruby-2.4.0 - #complete
Please be aware that you just installed a ruby that requires 2 patches just to be compiled on an up to date linux system.
This may have known and unaccounted for security vulnerabilities.
Please consider upgrading to ruby-2.7.0 which will have all of the latest security patches.
Ruby was built without documentation, to build it run: rvm docs generate-ri
[root@node-1 src]# rvm use 2.4.0
Using /usr/local/rvm/gems/ruby-2.4.0
[root@node-1 src]# rvm remove 2.0.0
ruby-2.0.0-p648 - #already gone
Using /usr/local/rvm/gems/ruby-2.4.0
[root@node-1 src]# ruby --version
ruby 2.4.0p0 (2016-12-24 revision 57164) [x86_64-linux]
[root@node-1 src]# gem install redis
Fetching redis-4.2.5.gem
Successfully installed redis-4.2.5
Parsing documentation for redis-4.2.5
Installing ri documentation for redis-4.2.5
Done installing documentation for redis after 1 seconds
1 gem installed
redis-trib.rb是ruby脚本，它依赖ruby环境。
[root@node-1 src]# ./redis-trib.rb create --replicas 1 192.168.186.31:7000 192.168.186.31:7001 192.168.186.32:7000 192.168.186.32:7001 192.168.186.33:7000 192.168.186.33:7001
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.186.31:7000
192.168.186.32:7000
192.168.186.33:7000
Adding replica 192.168.186.32:7001 to 192.168.186.31:7000
Adding replica 192.168.186.33:7001 to 192.168.186.32:7000
Adding replica 192.168.186.31:7001 to 192.168.186.33:7000
M: da6e64979e85968e5366a0a1d0a7217e1afd8f21 192.168.186.31:7000
slots:0-5460 (5461 slots) master
S: 3b4913918ca8b0e172b8d6853c4a5b1a96368bda 192.168.186.31:7001
replicates 7206336b7c65f02712c029420dd7833dc5e500cb
M: 292d31d46e7120c5dba499ffcd896cfe53849acc 192.168.186.32:7000
slots:5461-10922 (5462 slots) master
S: af94ed450d080f44dc0e704c2c8e4c207e0eba0c 192.168.186.32:7001
replicates da6e64979e85968e5366a0a1d0a7217e1afd8f21
M: 7206336b7c65f02712c029420dd7833dc5e500cb 192.168.186.33:7000
slots:10923-16383 (5461 slots) master
S: f4ec122664feacab585290d3f6005406496c97e8 192.168.186.33:7001
replicates 292d31d46e7120c5dba499ffcd896cfe53849acc
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join..
>>> Performing Cluster Check (using node 192.168.186.31:7000)
M: da6e64979e85968e5366a0a1d0a7217e1afd8f21 192.168.186.31:7000
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 3b4913918ca8b0e172b8d6853c4a5b1a96368bda 192.168.186.31:7001
slots: (0 slots) slave
replicates 7206336b7c65f02712c029420dd7833dc5e500cb
S: af94ed450d080f44dc0e704c2c8e4c207e0eba0c 192.168.186.32:7001
slots: (0 slots) slave
replicates da6e64979e85968e5366a0a1d0a7217e1afd8f21
S: f4ec122664feacab585290d3f6005406496c97e8 192.168.186.33:7001
slots: (0 slots) slave
replicates 292d31d46e7120c5dba499ffcd896cfe53849acc
M: 292d31d46e7120c5dba499ffcd896cfe53849acc 192.168.186.32:7000
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 7206336b7c65f02712c029420dd7833dc5e500cb 192.168.186.33:7000
slots:10923-16383 (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
根据上面的信息，已经可用清晰看到成功创建了3主3从的cluster集群
方法二：
在更高的版本中，已经不使用redis-trib.rb来创建集群，而是直接使用redis-client命令
```bash
[root@redis-5-1 src]# ./redis-trib.rb create --replicas 1 192.168.186.21:7000 192.168.186.21:7001 192.168.186.22:7000 192.168.186.22:7001 192.168.186.23:7000 192.168.186.23:7001
WARNING: redis-trib.rb is not longer available!
You should use redis-cli instead.
All commands and features belonging to redis-trib.rb have been moved
to redis-cli.
In order to use them you should call redis-cli with the --cluster
option followed by the subcommand name, arguments and options.
Use the following syntax:
redis-cli --cluster SUBCOMMAND [ARGUMENTS] [OPTIONS]
Example:
redis-cli --cluster create 192.168.186.21:7000 192.168.186.21:7001 192.168.186.22:7000 192.168.186.22:7001 192.168.186.23:7000 192.168.186.23:7001 --cluster-replicas 1
To get help about all subcommands, type:
redis-cli --cluster help
可以看到以上建议，直接复制命令使用：
[root@redis-5-1 src]# ./redis-cli --cluster create 192.168.186.21:7000 192.168.186.21:7001 192.168.186.22:7000 192.168.186.22:7001 192.168.186.23:7000 192.168.186.23:7001 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.186.22:7001 to 192.168.186.21:7000
Adding replica 192.168.186.23:7001 to 192.168.186.22:7000
Adding replica 192.168.186.21:7001 to 192.168.186.23:7000
M: f360fbbd4aa3f134f5b09032a6fd6eab724e3883 192.168.186.21:7000
slots:[0-5460] (5461 slots) master
S: 656157fd9981346c3e3fab52e1be6a215c92094c 192.168.186.21:7001
replicates 61415628f5ef7b23fc806d5832de210eddaa1a46
M: bee0adf62b77fe0b6421efc1fef7fde0a49735b3 192.168.186.22:7000
slots:[5461-10922] (5462 slots) master
S: 20b4b6a0e9e2409e8cafdf2f3b7ff503cc830b0c 192.168.186.22:7001
replicates f360fbbd4aa3f134f5b09032a6fd6eab724e3883
M: 61415628f5ef7b23fc806d5832de210eddaa1a46 192.168.186.23:7000
slots:[10923-16383] (5461 slots) master
S: 8612ea8df467a1f216b7b661fc846c959e90da55 192.168.186.23:7001
replicates bee0adf62b77fe0b6421efc1fef7fde0a49735b3
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 192.168.186.21:7000)
M: f360fbbd4aa3f134f5b09032a6fd6eab724e3883 192.168.186.21:7000
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: 61415628f5ef7b23fc806d5832de210eddaa1a46 192.168.186.23:7000
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
S: 20b4b6a0e9e2409e8cafdf2f3b7ff503cc830b0c 192.168.186.22:7001
slots: (0 slots) slave
replicates f360fbbd4aa3f134f5b09032a6fd6eab724e3883
S: 8612ea8df467a1f216b7b661fc846c959e90da55 192.168.186.23:7001
slots: (0 slots) slave
replicates bee0adf62b77fe0b6421efc1fef7fde0a49735b3
S: 656157fd9981346c3e3fab52e1be6a215c92094c 192.168.186.21:7001
slots: (0 slots) slave
replicates 61415628f5ef7b23fc806d5832de210eddaa1a46
M: bee0adf62b77fe0b6421efc1fef7fde0a49735b3 192.168.186.22:7000
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
登录验证
（随意登录一个reids实例进行验证）
[root@node-1 src]# ./redis-cli -p 7000
127.0.0.1:7000> cluster nodes
da6e64979e85968e5366a0a1d0a7217e1afd8f21 192.168.186.31:7000@17000 myself,master - 0 1606805839000 1 connected 0-5460
3b4913918ca8b0e172b8d6853c4a5b1a96368bda 192.168.186.31:7001@17001 slave 7206336b7c65f02712c029420dd7833dc5e500cb 0 1606805840000 5 connected
af94ed450d080f44dc0e704c2c8e4c207e0eba0c 192.168.186.32:7001@17001 slave da6e64979e85968e5366a0a1d0a7217e1afd8f21 0 1606805839586 4 connected
f4ec122664feacab585290d3f6005406496c97e8 192.168.186.33:7001@17001 slave 292d31d46e7120c5dba499ffcd896cfe53849acc 0 1606805840000 6 connected
292d31d46e7120c5dba499ffcd896cfe53849acc 192.168.186.32:7000@17000 master - 0 1606805840996 3 connected 5461-10922
7206336b7c65f02712c029420dd7833dc5e500cb 192.168.186.33:7000@17000 master - 0 1606805840000 5 connected 10923-16383
127.0.0.1:7000> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:204
cluster_stats_messages_pong_sent:207
cluster_stats_messages_sent:411
cluster_stats_messages_ping_received:202
cluster_stats_messages_pong_received:204
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:411
127.0.0.1:7000> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=192.168.186.32,port=7001,state=online,offset=182,lag=1
master_replid:6d704587e982c37025de8086fb42123af45e31a4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:182
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:182
集群状况正常，部署完毕！
版权声明：本文为rookie23rook原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接和本声明。