手头有三台机器，领导要求做成高可用的网站，一台做数据库肯定是没错的，剩下两台机器只能做web了。
原本打算用nginx+keepalived，keepalived做高可用没问题，一主一从，但是从机完全standby，资源有些浪费。于是想到了负载均衡集群。考虑到nginx又做负载均衡，又做web，怕管理上麻烦，负载均衡就用了haproxy做，并且haproxy的健康检查非常到位。而nginx的健康检查实际上是假的，应该较故障转移，因为一旦timeout过期了，流量仍然会分过去，发现不通后再进行故障转移，这会导致网页间歇性加载缓慢。
本文做法的好处是，从服务器平时也能分担主服务器的流量，任何一个机器宕机，web服务都可以正常访问。
环境：CentOS6.4 x86_64
VIP 192.168.122.10
web01 192.168.122.11
web02 192.168.122.12
1、安装软件
yum install haproxy keepalived nginx php-fpm php-gd php-mysql php-xml php-cli -y
chkconfig keepalived on
chkconfig haproxy on
chkconfig nginx on
chkconfig php-fpm on
chkconfig iptables off #防火墙暂时不配置
2、系统和内核设置
sed -i ‘s/enforcing/disabled/g' /etc/sysconfig/selinux
sysctl -w "net.ipv4.ip_nonlocal_bind=1"
echo "net.ipv4.ip_nonlocal_bind=1">>/etc/sysctl.conf
上面这个内核参数，可以允许服务器监听在一个不存在的地址上。
3、配置keepalived(主从配置略有不同)
配置主服务器web01
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id web01
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
preempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.122.10 label eth0:1
}
}
配置从服务器
global_defs {
notification_email {
root@localhost
}
notification_email_from keepalived02@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id web02
}
vrrp_instance VI_1 {
state BACKUP
interface eth0
virtual_router_id 51
priority 90
advert_int 1
#preempt
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.122.10 label eth0:1
}
}
4、配置haproxy(主从服务器配置完全相同，监听地址192.168.122.10：80)
global
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
listen status
bind *:10086
stats uri /haproxy-status
stats auth admin：123456
stats hide-version
frontend haproxy-nlb
bind 192.168.122.10:80
default_backend nginx-web
backend nginx-web
option httpchk HEAD /check.txt HTTP/1.0
balance roundrobin
server web01 192.168.122.11:80 weight 3 check inter 5s rise 2 fall 3
server web02 192.168.122.12:80 weight 3 check inter 5s rise 2 fall 3
5、配置nginx(除了监听地址不同，其余全一样)
主服务器监听192.168.122.11：80
从服务器监听192.168.122.12：80
user nginx;
worker_processes 2;
worker_rlimit_nofile 65535;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
google_perftools_profiles /tmp/tcmalloc;
events {
use epoll;
worker_connections 2048;
}
http {
include /etc/nginx/mime.types;
include /etc/nginx/naxsi_core.rules;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
#access_log /var/log/nginx/access.log main;
sendfile on;
server_tokens off;
#tcp_nopush on;
keepalive_timeout 65;
gzip on;
gzip_static on;
gzip_disable "msie6";
gzip_http_version 1.1;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript application/javascript text/x$
gzip_buffers 16 8k;
client_max_body_size 20m;
client_body_buffer_size 128k;
server {
listen 192.168.122.11:8080;
server_name localhost;
root /usr/share/nginx/html;
index index.html index.htm index.php;
#charset koi8-r;
access_log /var/log/nginx/host.access.log main;
location / {
include /etc/nginx/naxsi_conf ;
if (!-e $request_filename) {
rewrite ^/(.*)$ /index.php?q=$1 last;
}
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
#
location ~ \.php$ {
#try_files $uri=404;
fastcgi_pass 127.0.0.1:9000;
#fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_buffer_size 128k;
fastcgi_buffers 256 16k;
fastcgi_busy_buffers_size 256k;
fastcgi_temp_file_write_size 256k;
fastcgi_read_timeout 240;
include fastcgi_params;
}
if ($fastcgi_script_name ~ \..*\/.*php) {
return 403;
}
# deny access to hiden file . (filename begin with ".")
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
# deny access to bakup file .(any filename end with "~" )
location ~ ~$ {
access_log off;
log_not_found off;
deny all;
}
# cache p_w_picpath file
location ~* \.(jpg|jpeg|gif|png|css|js|ico|xml|swf)$ {
expires 1d;
}
# don't log robots and favion
location= /robots.txt { access_log off; log_not_found off; }
location= /favicon.ico { access_log off; log_not_found off; }
# deny access to .htaccess files, if Apache's document root
# concurs with nginx's one
#
location ~ /\.ht {
deny all;
}
}
}
6、配置php-fpm
sed -i ‘s/apache/nginx/g’ /etc/php-fpm.d/www.conf
7、其他
给web01加上检测页面
echo web01>/usr/share/nginx/html/check.txt
给web02加上检测页面
echo web02>/usr/share/nginx/html/check.txt
8、重启两台机器
9、验证
由于是轮询，我们很容易检测负载均衡(最好不要用浏览器，浏览器有缓存)
$ for n in {1..10};do curl http://192.168.122.10/check.txt;done
web02
web01
web02
web01
web02
web01
web02
web01
web02
web01
OK，大功告成！