redis集群分片存储
文章目录
- redis集群分片存储
- 为什么要分片存储
- 官方集群方案
- 搭建集群
- 集群关心的问题
- Java客户端代码
- 非官方集群方案
redis集群分片存储
为什么要分片存储
假设公司用户有3千万,用户基本信息缓存到redis中,需要内存10G,如何设计redis的缓存架构?
- 3千万用户,各种业务场景对用户信息的访问量很大,单台redis实例的读写瓶颈凸显。
- 单redis实例管理10G内存,必然影响处理效率。
- redis的内存需求可能超过机器的最大内存,一台机器不够用。
因此就有了分片存储的技术。
官方集群方案
redis cluster是Redis的分布式集群解决方案,在3.0版本推出后有效地解决了redis分布式方面的需求,实现了数据在多个redis节点之间自动分片,故障自动转移,扩容机制等功能。
搭建集群
环境信息
centos6,redis5。
集群的信息
我会在192.168.1.120,192.168.1.122,192.168.1.125这三台机器上分别启动两个redis实例,端口分别是6379和6380。
搭建集群前确保每台机器上都已经安装过redis,我的每台机器上的redis安装目录为:/var/redis-5.0.5。
每台机器上准备两个redis配置文件,一个redis_6379.conf,一个redis_6380.conf。
# 配置文件进行了精简,完整配置可自行和官方提供的完整conf文件进行对照。端口号自行对应修改 #后台启动的意思 daemonize yes #端口号 port 6381 # IP绑定,redis不建议对公网开放,直接绑定0.0.0.0没毛病 bind 0.0.0.0 # redis数据文件存放的目录 dir /usr/local/redis/data # 开启AOF appendonly yes# 开启集群 cluster-enabled yes # 会自动生成在上面配置的dir目录下 cluster-config-file nodes-6381.conf cluster-node-timeout 5000 # 这个文件会自动生成 pidfile /var/run/redis_6381.pid
启动6个redis实例,在三台机器的redis的安装目录的src下,分别运行如下两条命令。
./redis-server ../redis_6379.conf ./redis-server ../redis_6380.conf
使用ps -ef | grep redis命令查看集群是否启动正常。
创建cluster,在redis安装目录的src下执行下面的命令。
./redis-cli --cluster create 192.168.1.120:6379 192.168.1.120:6380 192.168.1.122:6379 192.168.1.122:6380 192.168.1.125:6379 192.168.1.125:6380 --cluster-replicas 1
在已经使用安装并使用过的redis服务器上运行此命令可能会抛出如下错误。参考https://blog.csdn.net/XIANZHIXIANZHIXIAN/article/details/82777767解决。
[ERR] Node 192.168.1.122:6379 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
运行成功后会让你确认集群信息。
[root@spark src]# ./redis-cli --cluster create 192.168.1.120:6379 192.168.1.120:6380 192.168.1.122:6379 192.168.1.122:6380 192.168.1.125:6379 192.168.1.125:6380 --cluster-replicas 1 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 192.168.1.122:6380 to 192.168.1.120:6379 Adding replica 192.168.1.125:6380 to 192.168.1.122:6379 Adding replica 192.168.1.120:6380 to 192.168.1.125:6379 M: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots:[0-5460] (5461 slots) master S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380replicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[5461-10922] (5462 slots) master S: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380replicates fd36242496e7a04d883857891ce425a0fc948783 M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380replicates 77e054f92028a383b69147f0c21e680ae664bb6b Can I set the above configuration? (type 'yes' to accept): y [root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6379 cluster nodes fd36242496e7a04d883857891ce425a0fc948783 :6379@16379 myself,master - 0 0 0 connected [root@spark src]# ./redis-cli -h 192.168.1.120 -p 6379 192.168.1.120:6379> cluster nodes fd36242496e7a04d883857891ce425a0fc948783 :6379@16379 myself,master - 0 0 0 connected 192.168.1.120:6379> [root@spark src]# ./redis-cli --cluster create 192.168.1.120:6379 192.168.1.120:6380 192.168.1.122:6379 192.168.1.122:6380 192.168.1.125:6379 192.168.1.125:6380 --cluster-replicas 1 >>> Performing hash slots allocation on 6 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 Adding replica 192.168.1.122:6380 to 192.168.1.120:6379 Adding replica 192.168.1.125:6380 to 192.168.1.122:6379 Adding replica 192.168.1.120:6380 to 192.168.1.125:6379 M: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots:[0-5460] (5461 slots) master S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380replicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[5461-10922] (5462 slots) master S: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380replicates fd36242496e7a04d883857891ce425a0fc948783 M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380replicates 77e054f92028a383b69147f0c21e680ae664bb6b Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ... >>> Performing Cluster Check (using node 192.168.1.120:6379) M: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots:[0-5460] (5461 slots) master1 additional replica(s) M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master1 additional replica(s) S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380slots: (0 slots) slavereplicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380slots: (0 slots) slavereplicates 77e054f92028a383b69147f0c21e680ae664bb6b S: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380slots: (0 slots) slavereplicates fd36242496e7a04d883857891ce425a0fc948783 M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[5461-10922] (5462 slots) master1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
集群检验和测试
查看所有节点信息。
[root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6379 cluster nodes ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379@16379 master - 0 1561421002080 5 connected 10923-16383 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380@16380 slave ace69b140e315ce90ed6c19de1aaec73032b1ca6 0 1561421000463 5 connected fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379@16379 myself,master - 0 1561421000000 1 connected 0-5460 a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380@16380 slave 77e054f92028a383b69147f0c21e680ae664bb6b 0 1561421000160 6 connected 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380@16380 slave fd36242496e7a04d883857891ce425a0fc948783 0 1561420999891 4 connected 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379@16379 master - 0 1561421001081 3 connected 5461-10922# 开启客户端不加-c选项,如果当前数据不应该在此实例上会返回MOVED ./redis-cli -h 192.168.1.120 -p 6379 192.168.1.120:6379> set a 1 (error) MOVED 15495 192.168.1.125:6379 192.168.1.120:6379> # 开启的话会为当前的数据自动跳转到对应的实例 [root@spark src]# ./redis-cli -h 192.168.1.120 -p 6379 192.168.1.120:6379> set a 1 (error) MOVED 15495 192.168.1.125:6379 192.168.1.120:6379> [root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6379 192.168.1.120:6379> set a 1 -> Redirected to slot [15495] located at 192.168.1.125:6379 OK 192.168.1.125:6379> get a "1" 192.168.1.125:6379> set hello tony -> Redirected to slot [866] located at 192.168.1.120:6379 OK 192.168.1.120:6379> get a -> Redirected to slot [15495] located at 192.168.1.125:6379 "1" 192.168.1.125:6379> cluster keyslot a (integer) 15495 192.168.1.125:6379>
集群slot数量整理reshard
# 有时根据集群中的机器的配置,我们可能希望有些性能高的机器上的槽多一些 # 就可以使用reshard,下面我们从192.168.1.120服务器移动1000个slot到192.168.1.122 # 这台服务器上 ./redis-cli --cluster reshard 192.168.1.120:6379 --cluster-from fd36242496e7a04d883857891ce425a0fc948783 --cluster-to 77e054f92028a383b69147f0c21e680ae664bb6b --cluster-slots 1000 --cluster-yes# 运行后会有如下输出 Moving slot 714 from 192.168.1.120:6379 to 192.168.1.122:6379: Moving slot 715 from 192.168.1.120:6379 to 192.168.1.122:6379: # 重新检查集群 ./redis-cli --cluster check 192.168.1.120:6379M: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots:[1000-5460] (4461 slots) master1 additional replica(s) M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master1 additional replica(s) S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380slots: (0 slots) slavereplicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380slots: (0 slots) slavereplicates 77e054f92028a383b69147f0c21e680ae664bb6b S: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380slots: (0 slots) slavereplicates fd36242496e7a04d883857891ce425a0fc948783 M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[0-999],[5461-10922] (6462 slots) master1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. # 可以看出从192.168.1.120移动了1000个slot到192.168.1.122
测试自动故障转移
# cluster集群不保证数据一致,数据也可能丢失 # 首先是运行客户端不断的写入或读取数据,以便能够发现问题 # 然后是模拟节点故障:找一个主节点关闭,主从故障切换的过程中,这个时间端的操作,客户端而言,只能是失败 # 官方描述 https://redis.io/topics/cluster-spec There is always a window of time when it is possible to lose writes during partitions. 分区的时间窗口内总是有可能丢失写操作。# 将192.168.1.120:6379这个实例停掉,192.168.1.122:6380成为一个master./redis-cli -c -h 192.168.1.120 -p 6379 192.168.1.120:6379> shutdown not connected> set a 1 [expiration EX seconds|PX milliseconds] [NX|XX] [root@spark src]# ./redis-cli --cluster check 192.168.1.120:6380 Could not connect to Redis at 192.168.1.120:6379: Connection refused 192.168.1.125:6379 (ace69b14...) -> 1 keys | 5461 slots | 1 slaves. 192.168.1.122:6380 (1892d627...) -> 0 keys | 4461 slots | 0 slaves. 192.168.1.122:6379 (77e054f9...) -> 1 keys | 6462 slots | 1 slaves. [OK] 2 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 192.168.1.120:6380) S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380slots: (0 slots) slavereplicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master1 additional replica(s) M: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380slots:[1000-5460] (4461 slots) master M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[0-999],[5461-10922] (6462 slots) master1 additional replica(s) S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380slots: (0 slots) slavereplicates 77e054f92028a383b69147f0c21e680ae664bb6b [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. [root@spark src]# # 查看此时的集群信息,192.168.1.120:6379下线了 [root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6380 192.168.1.120:6380> cluster nodes fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379@16379 master,fail - 1561422523649 1561422523348 1 disconnected ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379@16379 master - 0 1561422691644 5 connected 10923-16383 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380@16380 master - 0 1561422693669 8 connected 1000-5460 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379@16379 master - 0 1561422693570 7 connected 0-999 5461-10922 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380@16380 myself,slave ace69b140e315ce90ed6c19de1aaec73032b1ca6 0 1561422687000 2 connected a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380@16380 slave 77e054f92028a383b69147f0c21e680ae664bb6b 0 1561422692148 7 connected 192.168.1.120:6380> # 如果再把192.168.1.120:6379启动起来,其会议salve的身份加入集群 [root@spark src]# ./redis-cli --cluster check 192.168.1.120:6380 192.168.1.125:6379 (ace69b14...) -> 1 keys | 5461 slots | 1 slaves. 192.168.1.122:6380 (1892d627...) -> 0 keys | 4461 slots | 1 slaves. 192.168.1.122:6379 (77e054f9...) -> 1 keys | 6462 slots | 1 slaves. [OK] 2 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 192.168.1.120:6380) S: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380slots: (0 slots) slavereplicates ace69b140e315ce90ed6c19de1aaec73032b1ca6 S: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots: (0 slots) slavereplicates 1892d6272a759f454548ce97b0d09c580f687ef3 M: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots:[10923-16383] (5461 slots) master1 additional replica(s) M: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380slots:[1000-5460] (4461 slots) master1 additional replica(s) M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[0-999],[5461-10922] (6462 slots) master1 additional replica(s) S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380slots: (0 slots) slavereplicates 77e054f92028a383b69147f0c21e680ae664bb6b [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
手动故障转移
# 可能某个节点需要维护(机器下线,硬件升级,系统版本调整等场景),需要手动的实现转移 # 在slave节点上执行命令 CLUSTER FAILOVER # 相比较于自动故障转移而言,手动故障转移不会丢失数据
# 上面我们手动将192.168.1.120:6379上的redis服务停掉再起来后,它以slave的身份加入 # 集群,现在我们想让它重新成为master [root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6379 # 查看当前集群信息 192.168.1.120:6379> cluster nodes 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380@16380 master - 0 1561423733576 8 connected 1000-5460 ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379@16379 slave 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 0 1561423734623 9 connected a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380@16380 slave 77e054f92028a383b69147f0c21e680ae664bb6b 0 1561423734084 7 connected 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380@16380 master - 0 1561423735107 9 connected 10923-16383 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379@16379 master - 0 1561423734620 7 connected 0-999 5461-10922 fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379@16379 myself,slave 1892d6272a759f454548ce97b0d09c580f687ef3 0 1561423733000 1 connected # 执行手动故障转移 192.168.1.120:6379> cluster failover OK # 检查手动故障转移效果,发现192.168.1.120:6379再次成为master 192.168.1.120:6379> cluster nodes 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380@16380 slave fd36242496e7a04d883857891ce425a0fc948783 0 1561423750936 10 connected ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379@16379 slave 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 0 1561423751000 9 connected a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380@16380 slave 77e054f92028a383b69147f0c21e680ae664bb6b 0 1561423750000 7 connected 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380@16380 master - 0 1561423751955 9 connected 10923-16383 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379@16379 master - 0 1561423750432 7 connected 0-999 5461-10922 fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379@16379 myself,master - 0 1561423751000 10 connected 1000-5460 192.168.1.120:6379>
扩容
# 我们再在192.168.1.120这台服务器上准备一个redis配置文件,redis_6381.conf # 启动新节点 ./redis-server ../redis_6381.conf# 以master身份加入集群 ./redis-cli --cluster add-node 192.168.1.120:6381 192.168.1.120:6379 >>> Adding node 192.168.1.120:6381 to cluster 192.168.1.120:6379 >>> Performing Cluster Check (using node 192.168.1.120:6379) M: fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379slots:[1000-5460] (4461 slots) master1 additional replica(s) S: 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380slots: (0 slots) slavereplicates fd36242496e7a04d883857891ce425a0fc948783 S: ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379slots: (0 slots) slavereplicates 9ffd15c5674f4ff06ff4dca186fbf919a992c87b S: a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380slots: (0 slots) slavereplicates 77e054f92028a383b69147f0c21e680ae664bb6b M: 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380slots:[10923-16383] (5461 slots) master1 additional replica(s) M: 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379slots:[0-999],[5461-10922] (6462 slots) master1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 192.168.1.120:6381 to make it join the cluster. [OK] New node added correctly. # 查看此时的集群信息,新创建的redis实例以master身份加入集群 [root@spark src]# ./redis-cli -c -h 192.168.1.120 -p 6379 192.168.1.120:6379> cluster nodes 12593a9cfbdffd272547926b625acb09d306c7fc 192.168.1.120:6381@16381 master - 0 1561424739585 0 connected 1892d6272a759f454548ce97b0d09c580f687ef3 192.168.1.122:6380@16380 slave fd36242496e7a04d883857891ce425a0fc948783 0 1561424739488 10 connected ace69b140e315ce90ed6c19de1aaec73032b1ca6 192.168.1.125:6379@16379 slave 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 0 1561424740528 9 connected a4c630716cd013da8a2e2ce89a26127c57ff356e 192.168.1.125:6380@16380 slave 77e054f92028a383b69147f0c21e680ae664bb6b 0 1561424740529 7 connected 9ffd15c5674f4ff06ff4dca186fbf919a992c87b 192.168.1.120:6380@16380 master - 0 1561424740628 9 connected 10923-16383 77e054f92028a383b69147f0c21e680ae664bb6b 192.168.1.122:6379@16379 master - 0 1561424740019 7 connected 0-999 5461-10922 fd36242496e7a04d883857891ce425a0fc948783 192.168.1.120:6379@16379 myself,master - 0 1561424738000 10 connected 1000-5460 192.168.1.120:6379> # 本质就是发送一个新节点通过CLUSTER MEET命令加入集群 # 注意新加入的节点是没有分配slot的,它不会存储数据,我们使用之前需要给它分配slot # 参考第8点# 以slave身份加入集群 ./redis-cli --cluster add-node 192.168.1.120:6381 192.168.1.120:6379 --cluster-slave # 还可以手动指定要以slave身份加入集群的节点的master节点,否则就选择一个slave数量 # 最少的master ./redis-cli --cluster add-node 192.168.1.120:6381 192.168.1.120:6379 --cluster-slave --cluster-master-id <node-id> # 将空master转换为slave cluster replicate <master-node-id>
缩容
# 注意:删除master的时候要把数据清空或者分配给其他主节点 # 下面我们将新加入的192.168.1.120:6381节点删除 [root@spark src]# ./redis-cli --cluster del-node 192.168.1.120:6381 12593a9cfbdffd272547926b625acb09d306c7fc >>> Removing node 12593a9cfbdffd272547926b625acb09d306c7fc from cluster 192.168.1.120:6381 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node. # 查看6381对应的redis进程是否还在 [root@spark src]# ps -ef | grep redis root 17475 17274 0 07:53 pts/1 00:00:34 ./redis-server 192.168.1.120:6380 [cluster] root 17503 17228 0 08:33 pts/0 00:00:18 ./redis-server 192.168.1.120:6379 [cluster] root 17571 17303 0 09:17 pts/3 00:00:00 grep redis [root@spark src]#
集群关心的问题
增加了slot槽位的计算,是不是比单机性能差?
共16384个槽位,slots槽计算方式公开的,HASH_SLOT = CRC(key) mod 16384。为了避免每次都需要服务器计算重定向,优秀的java客户端都实现了本地计算,并且缓存服务器slots分配,有变动时再更新本地内容,从而避免了多次重定向带来的性能损耗。
redis集群大小,到底可以装多少数据?
理论是可以做到16384个槽,每个槽对应一个实例,但是redis官方建议是最大1000个实例。
集群节点间是怎么通信的?
每个redis集群节点都有一个额外的TCP端口,每个节点使用TCP连接与其他节点连接。检测和故障转移这些步骤基本和哨兵模式类似。
ask和moved重定向的区别。
重定向包括两种情况:
- 若确定slot不属于当前节点,redis会返回moved。
- 若当前redis节点正在处理slot迁移,则代表此处请求对应的key暂时不在此节点,返回ask,告诉客户端本次请求重定向。
数据倾斜和访问倾斜的问题。
倾斜导致集群中部分节点数据多,压力大。解决方案分为前期和后期:
前期是业务层面提前预测,哪些是热点key,在设计的过程中规避。
后期是slot迁移,尽量将压力分摊(slot调整有自动reblance,reshard和手动)。
slot手动迁移怎么做?
- 在迁移目的节点执行cluster setslot IMPORTING 命令,指明需要迁移的slot和迁移源节点。
- 在迁移源节点执行cluster setslot MIGRATING 命令,指明需要迁移的slot和迁移目的节点。
- 在迁移源节点执行cluster getkeysinslot获取该slot的key列表。
- 在迁移源节点对每个key执行migrate命令,该命令会同步把该key迁移到目的节点。
- 在迁移源节点返回执行cluster getkeysinslot命令,直到该slot的列表为空。
- 在迁移源节点和目的节点执行cluster setslot NODE ,完成迁移操作。
节点之间会交换信息,传递的消息包括槽的信息,带来带宽消耗。避免使用大的集群,可以分多个集群。
Pub/Sub发布订阅机制,对集群内任意一个节点执行publish发布消息,这个消息会在集群中进行传播,其他节点接收到发布的消息。
读写分离。
- redis-cluster默认所有从节点上的读写,都会重定向到key对接槽的主节点上。
- 可以通过readonly设置当前连接可读,通过readwrite取消当前连接的可读状态。
- 主从节点依然存在数据不一致的问题。
Java客户端代码
配置Bean。
@Configuration // 在cluster环境下生效 @Profile("a7_cluster") class ClusterAppConfig {@Beanpublic JedisConnectionFactory redisConnectionFactory() {System.out.println("加载cluster环境下的redis client配置");RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(Arrays.asList("192.168.1.120:6379","192.168.1.120:6380","192.168.1.122:6379","192.168.1.122:6380","192.168.1.125:6379","192.168.1.125:6380"));// 自适应集群变化return new JedisConnectionFactory(redisClusterConfiguration);} }
业务service。
@Service @Profile("a7_cluster") public class ClusterService {@Autowiredprivate StringRedisTemplate template;public void set(String userId, String userInfo) {template.opsForValue().set(userId, userInfo);} }
单元测试
@RunWith(SpringJUnit4ClassRunner.class) @ContextConfiguration("classpath:applicationContext.xml") @ActiveProfiles("a7_cluster") // 设置profile // 集群对于客户端而言,基本是无感知的 public class ClusterServiceTests {@AutowiredClusterService clusterService;@Testpublic void setTest() {clusterService.set("tony", "hahhhhh");clusterService.set("a", "1");clusterService.set("foo", "bar");}// 测试cluster集群故障时的反应@Testpublic void failoverTest() {while (true) {try {long i = System.currentTimeMillis();clusterService.set("tony", i + "");// delay 10msTimeUnit.MILLISECONDS.sleep(10);} catch (Exception e) {System.out.println(e.getMessage());}}} }运行setTest后,它实际上存储在192.168.1.120:6380 ./redis-cli -c -h 192.168.1.120 -p 6379 192.168.1.120:6379> get tony -> Redirected to slot [14405] located at 192.168.1.120:6380 "hahhhhh" 192.168.1.120:6380> get tongy -> Redirected to slot [9429] located at 192.168.1.122:6379 (nil) 192.168.1.122:6379> get tony -> Redirected to slot [14405] located at 192.168.1.120:6380 "hahhhhh" 192.168.1.120:6380运行failoverTest后,停掉192.168.1.120:6380 192.168.1.120:6380> shutdown not connected> 对客户端无影响
非官方集群方案
- Codis,豌豆荚开源。
- twemproxy,推特开源。
redis集群分片存储相关推荐
- Redis集群分片存储原理图
- Redis集群分片原理
转载:http://blog.csdn.net/shmnh/article/details/72868328 Redis集群的目的是实现数据的横向伸缩,把一块数据分片保存到多个机器,可以横向扩展数据库 ...
- redis 集群 分片 扩容_Redis高可用之集群实现原理
概要:本文主要用于介绍Redis集群实现的原理,以及集群中的主从切换.副本漂移.分片迁移的原理 1 Redis集群实现的原理 集群是用来提供横向扩展能力,即当数据量增多以后,通过增加服务节点就可以扩展 ...
- Redis集群搭建(Cluster 集群模式,分片集群)
Redis集群搭建Cluster 集群模式 引言 Redis的集群介绍 搭建方法 创建集群 其他服务的搭建 引言 redis相信大家不陌生,是最常用的缓存解决方案了.但是,在服务开发中,单机都会存在单 ...
- 一致性hash算法和redis集群动态数据存储
记录:对一致性Hash算法,Java代码实现的深入研究链接地址: http://www.cnblogs.com/xrq730/p/5186728.html 全部来自: https://mp.weixi ...
- Redis 集群方案
根据一些测试整理出来的一份方案: 1. Redis 性能 对于redis 的一些简单测试,仅供参考: 测试环境:Redhat6.2 , Xeon E5520(4核)*2/8G,1000M网卡 Redi ...
- 如何从0到1构建一个稳定、高性能的Redis集群?(附16张图解)
阅读本文大约需要 13 分钟. 大家好,我是 Kaito. 这篇文章我想和你聊一聊 Redis 的架构演化之路. 现如今 Redis 变得越来越流行,几乎在很多项目中都要被用到,不知道你在使用 Red ...
- Redis集群——利用Gearman在Lnmp架构中做MySQL的缓存服务器
一.概述 Redis的集群主要是使用切片技术来搭建的,简单来说就是把所有KEY分散存放到不同的redis节点上(不要把鸡蛋都放在一个篮子里). 1. 集群基本原理 Redis集群中内置了16384个槽 ...
- 阿里云Redis集群版简要介绍
阿里云云数据库Redis集群版火热抢购,详情参见>> 产品简介 云数据库 Redis 提供集群版实例,轻松突破 Redis 自身单线程瓶颈,可极大满足对于 Redis 大容量或高性能的业务 ...
最新文章
- datasnap的客户端回传机制
- 公告三大“罪状”,无人驾驶公司Roadstar联合创始人被罢免
- C# 线程安全与 lock锁
- 首次安装mysql57需要注意的问题
- python常用内置函数可以操作字符串_Python常用内置函数(二)
- NET问答: 如何在 ASP.NET Core 的 .json 文件中读取 AppSettings ?
- 防止事件导致的oncreate的多次调用
- Linux专题_行编辑
- 北京,公交遭贼后…………[Teaks]
- star cd linux安装,STAR-CD的Linux版安装详细过程
- html foreach 行 间隔,颜色,javascript简单实现表格行间隔显示颜色并高亮显示
- 【史上最全的PLC源码】2978个PLC应用例程合集
- 【毕业设计】深度学习指纹识别算法研究与实现 - python opencv
- NeoKylin(中标麒麟)6.5 更新yum源
- Invalid or unexpected token 报错
- abb机器人工具坐标系设定方式_实用 | ABB机器人如何设定工具坐标系?
- java string 加法_java 对String 加法计算
- 远程桌面 - linux
- win10网络不出现计算机列表,win10笔记本WiFi网络列表不显示内容的解决方法
- lightgbm处理类别特征