LVS(DR)+Keepalived 安装
参照：http://wwwfastwebcc.blog.51cto.com/blog/862520/476473
该文章的内容已经以doc格式发布外连接：
我要下载
准备工作
硬件环境
Linux (CentOS 5.5)服务器4台。注意hostname 不要一样。
软件环境
Linux 2.6.xx内核开发包的安装，该步很重要，本人在安装过程中就是因为没有安装该包导致近一周都在原地打转转，且相关的文章看了N多，最后在相关网友(运维研究群中start)的帮助下才知道是该原因
Yum install kernel-devel
Ln –s /usr/src/kernels/2.6.xx/usr/src/linux
其它软件
Yum install gcc openssl-devel popt-develwget tcpdump sendmail httpd
关掉防火干墙或添加VRRP通讯支持
1） Service iptables stop
2） iptables -A INPUT -d 224.0.0.18 -j ACCEPT
service iptablessave
关掉selinux
1） 不重启服务器直接关掉
Setenforce 0
2） 修改selinux的配置
Vi/etc/sysconfig/selinux
SELINUX=enforcingè SELINUX=disabled
reboot
IP分配情况
Lvs_master: 192.168.0.251
Lvs_backup:192.168.0.250
Vip:192.168.0.2
Real_ip1: 192.168.0.10
Real_ip2: 192.168.0.11
安装ipvsadm
Yum install ipvsadm
查看是否安装成功
Ipvsamd –ln 显示下面内容
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:Port Scheduler Flags
->RemoteAddress:Port ForwardWeight ActiveConn InActConn
安装Keepalived
a) 下载keepalived
Wget http://www.keepalived.org/software/keepalived-1.2.2.tar.gz
Tar –zxvf keepalived-1.2.2.tar.gz
Cd keepalived-1.2.2
b) 编译及安装
./configure –prefix=/usr/apps/keepalived--with-kernel-dir=/usr/src/linux[MS1]
c) 后期准备
Cp/usr/apps/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
Cp/usr/apps/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
Cp/usr/apps/keepalived/sbin/keepalived /usr/sbin/
Mkdir –p/etc/keepalived
Cp /usr/apps/keepalived/etc/keepalived/keepalived.conf/etc/keepalived
Service keepalivedstart|stop|restart
LVS(DR)+Keepalived实验
Keepalived配置
Vi /e tc/keepalived.conf
Lvs_master:
正文内容：
!Configuration File for keepalived
global_defs{
notification_email {
xx@139.com
}
notification_email_from xx@qq.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instanceVI_1 {
#(主服务器 MASTER 备用服务器用BACKUP)
state MASTER
interface eth0
virtual_router_id 51
#(主服务器的值大于备用服务器的值，即备用服务器用100)
priority 102
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.0.2
}
}
virtual_server192.168.0.2 80 {
delay_loop 6
#lb_algo wlc
lb_algo rr
lb_kind DR
# persistence_timeout 50
protocol TCP
real_server 192.168.0.10 80 {
weight 5
TCP_CHECK {
connect_timeout 10
connect_port 80
}
}
real_server 192.168.0.11 80 {
weight 5
TCP_CHECK {
connect_timeout 10
#nb_get_retry 3
#delay_before_retry 3
connect_port 80
}
}
}
Lvs_backup:
即把上面的复制一份，同时作两处修改
1) state MASTER --à state BACKUP
2) priority 102 ---à priority 100
真实服务器（节点服务器）配置
禁止响应 ARP 请求
1. 方法一：
a) 修改sysctl.conf
#vi /etc/sysctl.conf
net.ipv4.conf.lo.arp_ignore= 1
net.ipv4.conf.lo.arp_announce= 2
net.ipv4.conf.all.arp_ignore= 1
net.ipv4.conf.all.arp_announce= 2
net.ipv4.ip_forward= 1
b) 在本地回环地址上邦定 VIP ,达到 DR 转发的目的。并添加到 /etc/rc.local 里
代码:
#/sbin/ifconfiglo:0 10.10.11.120 broadcast 10.10.11.120netmask 255.255.255.255 up
#/sbin/routeadd -host 10.10.11.120 dev lo:
2. 方法二：
使用下面的脚本：
Vi/usr/bin/lvs_real
#!/bin/bash
#description: Config realserver
VIP=192.168.0.2
#下面一句在centos6.0平台上面不可执行
./etc/rc.d/init.d/functions
case"$1" in
start)
/sbin/ifconfig lo:0 $VIP netmask255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $VIP >/dev/null 2>&1
echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
#endreal_lvs
开始测试
1. 开启禁止arp请求
在两台真实服务器（192.168.0.10/11）下面执行
Chmod a+x/usr/bin/lvs_real
/usr/bin/lvs_realstart
然后使用ip a可以查看192.168.0.2已经加在了回环设备上面（红色粗体）
#p a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdiscnoqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet127.0.0.1/8 scope host lo
inet 192.168.0.2/32brd 192.168.0.2 scope global lo:0
inet6::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu1500 qdisc pfifo_fast qlen 1000
link/ether12:34:56:78:90:ac brd ff:ff:ff:ff:ff:ff
inet192.168.0.10/24 brd 192.168.0.255 scope global eth0
inet6fe80::1034:56ff:fe78:90ac/64scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0brd 0.0.0.0
如果想手工打开可以执行 ifconfig lo:0 down
2. 开始lvs_master/backup服务
在mastet/backup两台lvs服务器上面执行
Service keepalived start
在lvs_master 查看虚ip是否绑定成功(红色地方)
#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdiscnoqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet127.0.0.1/8 scope host lo
inet6::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu1500 qdisc pfifo_fast qlen 1000
link/ether00:0c:29:08:f6:03 brdff:ff:ff:ff:ff:ff
inet 192.168.0.251/24brd 192.168.0.255 scope global eth0
inet 192.168.0.2/32 scope global eth0
inet6fe80::20c:29ff:fe08:f603/64scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0brd 0.0.0.0
查看虚拟服务器的路由表是否存在
#ipvsadm –Ln
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:Port SchedulerFlags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.2:80 rr
-> 192.168.0.11:80 Route 5 0 0
-> 192.168.0.10:80 Route 5 0 0
3. ab 测试lvs是否生效
a) 执行指令
#ab -n 100 -c 50 http://192.168.0.2/index.php
This is ApacheBench, Version 2.0.40-dev<$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd,http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation,http://www.apache.org/
Benchmarking 192.168.0.2 (be patient).....done
Server Software: Apache/2.2.3
Server Hostname: 192.168.0.2
Server Port: 80
Document Path: /index.php
Document Length: 13 bytes
Concurrency Level: 50
Time taken for tests: 0.78254 seconds
Complete requests: 100
Failed requests: 0
Write errors: 0
Total transferred: 27400 bytes
HTML transferred: 1300 bytes
Requests per second: 1277.89 [#/sec] (mean)
Time per request: 39.127 [ms] (mean)
Time per request: 0.783 [ms] (mean, across all concurrentrequests)
Transfer rate: 332.25 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.7 0 3
Processing: 0 1 1.0 1 5
Waiting: 0 0 1.1 0 4
Total: 0 1 1.3 1 5
Percentage of the requests served within a certaintime (ms)
50% 1
66% 2
75% 2
80% 2
90% 4
95% 4
98% 5
99% 5
100% 5 (longest request)
b)查看lvs_master的执行结果
#ipvsadm -Ln
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:Port SchedulerFlags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.2:80 rr
->192.168.0.11:80 Route 5 0 50
->192.168.0.10:80 Route 5 0 50
4. 测试LVS主备服务器双机热备效果
1) 停用lvs_master和查看vip
#servicekeepalived stop
#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdiscnoqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet127.0.0.1/8 scope host lo
inet6 ::1/128scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu1500 qdisc pfifo_fast qlen 1000
link/ether00:0c:29:08:f6:03 brdff:ff:ff:ff:ff:ff
inet192.168.0.251/24 brd 192.168.0.255 scope global eth0
inet6fe80::20c:29ff:fe08:f603/64scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480 qdisc noop
link/sit 0.0.0.0brd 0.0.0.0
2) 在lvs_backup上面查看接管vip的情况(看红色字体说明接管成功)
#ip a
1: lo: <LOOPBACK,UP,LOWER_UP>mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 12:34:56:78:90:ab brd ff:ff:ff:ff:ff:ff
inet 192.168.0.250/24 brd 192.168.0.255 scope global eth0
inet192.168.0.2/32 scope global eth0
inet6 fe80::1034:56ff:fe78:90ab/64 scope link
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
3) 在lvs_master(192.168.0.251)上面压力测试
#ab -n 100 -c 50 http://192.168.0.2/index.php
4) 在lvs_backup(192.168.0.250)上面查看LVS(DR)分配结果
#ipvsadm –Ln
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:Port SchedulerFlags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.0.2:80 rr
->192.168.0.11:80 Route 5 0 51
->192.168.0.10:80 Route 5 0 50
5. 测试节点宕机
在192.168.0.10上面执行 service httpd stop
在lvs上面用 ipvsadm –Ln查看可以发现节点 192.168.0.10已经被LVS踢除
同样在执行service httpd start后用ipvsadm –Ln可以发现虚拟服务路由表中已经加入了该节点，在此不再多试。
总结：
1. 多看keepalived的日志
查看方法 tail /var/log/message
2. 本机邮件报警还有问题和sendmail服务有关系。
3. 一定要安装linux内核开发包（yum install kernel-devel,ln -s /usr/src/kernerls/2.6.xxx /usr/src/linux)
[MS1]可以省去，默认为此路径