JAVA和Nginx 教程大全

网站首页 > 精选教程 正文

搭建MariaDB高可用集群 == 【篇二】 == nginx集群

wys521 2024-10-05 04:11:55 精选教程 19 ℃ 0 评论

篇二:nginx集群 【共四篇】

搭建 MariaDB高可用数据库集群 == 基于mariadb + galera + keepalived + nginx


一、内容介绍:

(1) 本章目的:

通过nginx集群实现代理转发高可用,任何一个节点故障,自动漂移到其他节点。

(2) 实验环境:

物理机:VMware 虚拟机 + CentOS 7.9 x64

nginx: nginx-1.23.3

(3) 机器信息

物理机1 192.168.8.35 CNT7XMDBD01(master-01) vip = 192.168.8.31

物理机2 192.168.8.36 CNT7XMDBD02(master-02) vip = 192.168.8.32

物理机3 192.168.8.37 CNT7XMDBD03(master-03) vip = 192.168.8.33


二、安装步骤:

在集群三个节点,安装nginx,编辑nginx.conf配置参数, 然后注册和启动nginx服务。

检查确认nginx集群正常,做故障模拟测试:任何n-1个节点故障,都不影响nginx集群转发的正常服务。


三、步骤细节:

步骤 step 1:

安装准备工作,安装依赖软件,给nginx源码编译使用

# 安装依赖软件(编译nginx源码使用)
yum install -y gcc pcre-devel zlib-devel openssl openssl-devel

# 检查安装结果
yum info gcc # 其他组件略
gcc --version

下载niginx源代码,根据实际需要选择版本,这里选择最新版本1.23.3

# 下载地址 == nginx 历史版本
# http://nginx.org/download/
# 下载源代码
yum install -y wget && \
mkdir -p /home/soft/nginx/ && \
wget -L http://nginx.org/download/nginx-1.23.3.tar.gz  -O /home/soft/nginx/nginx-1.23.3.tar.gz


步骤 step 2:

解压源代码之后,准备编译nginx

# 解压源代码
mkdir -p /home/app/nginx/ && \
mkdir -p /home/soft/nginx/ && \
tar -zxvf /home/soft/nginx/nginx-1.23.3.tar.gz -C /home/soft/nginx/
cd /home/soft/nginx/nginx-1.23.3/
# 查看当前nginx安装包拥有的默认安装的模块清单
cat ./auto/options | grep YES
mkdir /home/soft/nginx/nginx-1.23.3/nginx-prefix
# 配置编译参数:下面有两种选择
# --with-stream:开启 4 层透明转发(TCP Proxy)功能;
# --without-xxx:关闭所有其他功能,这样生成的动态链接二进制程序依赖最小

# 方式1: 启用全部功能 = all feature
./configure --prefix=$(pwd)/nginx-prefix \
 --with-stream \
 --with-select_module --with-poll_module --with-threads --with-file-aio

# 方式2: 启用部分功能 = TCP四层协议 + HTTP 七层协议  === 【推荐】
./configure  --prefix=$(pwd)/nginx-prefix \
 --with-stream \
 --with-select_module --with-poll_module --with-threads --with-file-aio \
 --with-http_ssl_module --with-stream_ssl_module \
 --with-http_v2_module --with-http_realip_module --with-http_gzip_static_module \
 --with-stream_realip_module 

开始编译源代码,然后安装编译文件

# 编译和安装:
cd /home/soft/nginx/nginx-1.23.3/
make && make install
# 输出结果,如下:
## 部分内容,省略
##         || cp -R html '/home/soft/nginx/nginx-1.23.3/nginx-prefix'
## test -d '/home/soft/nginx/nginx-1.23.3/nginx-prefix/logs' \
##         || mkdir -p '/home/soft/nginx/nginx-1.23.3/nginx-prefix/logs'
## make[1]: Leaving directory `/home/soft/nginx/nginx-1.23.3'
## [root@CNT7XMDBD03 nginx-1.23.3]#

检查编译文件是否OK

ls -al /home/soft/nginx/nginx-1.23.3/nginx-prefix/
cd /home/soft/nginx/nginx-1.23.3/nginx-prefix/
./nginx --version
# 输出结果,如下:
## [root@CNT7XMDBD03 nginx-1.23.3]# ls -al /home/soft/nginx/nginx-1.23.3/nginx-prefix/
## total 4
## drwxr-xr-x.  6 root root   54 Feb 15 10:56 .
## drwxr-xr-x. 10 1001 1001  206 Feb 15 10:54 ..
## drwxr-xr-x.  2 root root 4096 Feb 15 10:56 conf
## drwxr-xr-x.  2 root root   40 Feb 15 10:56 html
## drwxr-xr-x.  2 root root    6 Feb 15 10:56 logs
## drwxr-xr-x.  2 root root   19 Feb 15 10:56 sbin
## [root@CNT7XMDBD03 nginx-1.23.3]#
## [root@CNT7XMDBD03 nginx-prefix]# ./sbin/nginx -v
## nginx version: nginx/1.23.3
## [root@CNT7XMDBD03 nginx-prefix]# 

查看 nginx 动态链接的库

ldd ./sbin/nginx
# 输出结果,如下:
## [root@CNT7XMDBD03 nginx-prefix]# ldd ./sbin/nginx
##         linux-vdso.so.1 =>  (0x00007ffd9e7a6000)
##         libdl.so.2 => /lib64/libdl.so.2 (0x00007f57240d6000)
##         libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f5723eba000)
##         libssl.so.10 => /lib64/libssl.so.10 (0x00007f5723c48000)
##         libcrypto.so.10 => /lib64/libcrypto.so.10 (0x00007f57237e5000)
##         libc.so.6 => /lib64/libc.so.6 (0x00007f5723417000)
##         /lib64/ld-linux-x86-64.so.2 (0x00007f57242da000)
##         libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f57231ca000)
##         libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f5722ee1000)
##         libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f5722cdd000)
##         libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f5722aaa000)
##         libz.so.1 => /lib64/libz.so.1 (0x00007f5722894000)
##         libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f5722684000)
##         libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f5722480000)
##         libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f5722266000)
##         libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f572203f000)
##         libpcre.so.1 => /lib64/libpcre.so.1 (0x00007f5721ddd000)
## [root@CNT7XMDBD03 nginx-prefix]# 


步骤 step 3:

安装nginx

设置/查询系统的最大文件打开数

# 设置/查询系统的最大文件打开数,这个和nginx支持的最大连接数有关
vi /etc/systemd/system.conf
# ----------------------------------编辑参数,如下------------------ begin
# 然后配置和启用如下两个参数
DefaultLimitNOFILE=1000000
DefaultLimitNPROC=1000000
# ----------------------------------编辑参数,如下------------------ end
# 最后,记得保存重启电脑,才能生效参数

# 检查是否生效,执行如下命令
ulimit -a
# 输出结果,如下:
## [root@CNT7XMDBD03 /]# ulimit -a
## core file size          (blocks, -c) 0
## data seg size           (kbytes, -d) unlimited
## scheduling priority             (-e) 0
## file size               (blocks, -f) unlimited
## pending signals                 (-i) 31117
## max locked memory       (kbytes, -l) 64
## max memory size         (kbytes, -m) unlimited
## open files                      (-n) 1000000
## pipe size            (512 bytes, -p) 8
## POSIX message queues     (bytes, -q) 819200
## real-time priority              (-r) 0
## stack size              (kbytes, -s) 8192
## cpu time               (seconds, -t) unlimited
## max user processes              (-u) 31117
## virtual memory          (kbytes, -v) unlimited
## file locks                      (-x) unlimited
## [root@CNT7XMDBD03 /]# 

复制文件到安装目录

cp -rf  /home/soft/nginx/nginx-1.23.3/nginx-prefix/*    /home/app/nginx/
ls -al /home/app/nginx/
# 输出结果,如下:
## [root@CNT7XMDBD03 /]# cp -rf  /home/soft/nginx/nginx-1.23.3/nginx-prefix/*    /home/app/nginx/
## [root@CNT7XMDBD03 /]# ls -al /home/app/nginx/
## total 4
## drwxr-xr-x. 6 root root   54 Feb 15 11:16 .
## drwxr-xr-x. 3 root root   19 Feb 15 10:43 ..
## drwxr-xr-x. 2 root root 4096 Feb 15 11:16 conf
## drwxr-xr-x. 2 root root   40 Feb 15 11:16 html
## drwxr-xr-x. 2 root root    6 Feb 15 11:16 logs
## drwxr-xr-x. 2 root root   19 Feb 15 11:16 sbin
## [root@CNT7XMDBD03 /]# 

修改nginx配置文件

# 修改nginx配置文件
# 备份配置文件
mkdir -p /home/app/nginx/conf_bak/
cp -rf /home/app/nginx/conf/*   /home/app/nginx/conf_bak/

# 创建所有站点的配置子目录集合的根目录
mkdir -p  /home/app/nginx/vsites_conf/


下面,是具体nginx站点配置文件的修改步骤,分为主站点配置和子站点配置。

1. 主站点 == 执行范围:所有nginx机器节点

# 1. 主站点 == 执行范围:所有nginx机器节点
# 创建配置文件
# 备注:注意文件编码格式为 "UTF-8 无BOM编码"
vi /home/app/nginx/conf/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
# user  nobody;
worker_processes 2;
worker_cpu_affinity 01 10;

error_log  logs/error.log;
# error_log  logs/error.log  notice;
# error_log  logs/error.log  info;

pid        logs/nginx.pid;
events {
    use epoll;  # use [ kqueue | rtsig | epoll | /dev/poll | select | poll ]
    worker_connections  1000000;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  ' -  [] "" '
                      '  "" '
                      '"" ""';

    access_log  logs/access.log;
    error_log  logs/error.log;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    gzip  on;

    server {
        listen       8080;
        server_name  localhost;

        charset utf-8;
        access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        error_page  404              /50x.html;
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
	
    # add by kevin, 2023-02-15
    # import other website dnsname mapping 
    include /home/app/nginx/web_conf/*/*.conf;
}

stream {
    # add by kevin, 2023-02-15
    # import other website dnsname mapping 
    include /home/app/nginx/app_conf/*/*.conf;
}
#----------------------------------- 编辑内容,如下 -----------------------------begin


2. 子站点== 执行范围:所有nginx机器节点

mkdir -p /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/ && \
mkdir -p /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/
touch /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf
touch /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf


2.1 nginx-1 节点

创建配置文件 : tcp

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf

vi  /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
    # mariadb proxy
    upstream mariadb-cluster-sit.dev10w.cn { 
        hash $remote_addr consistent;
        server 192.168.8.35:3306;
        server 192.168.8.36:3306;
        server 192.168.8.37:3306;
    }
    server {
        # listen 13306;
        listen 192.168.8.31:13306; # vip-1 地址
        proxy_connect_timeout 60s;					# 连接超时时间
        proxy_timeout 300s;							# 端口保持时间
        proxy_pass mariadb-cluster-sit.dev10w.cn;
    }
#----------------------------------- 编辑内容,如下 -----------------------------end

创建配置文件 : http

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf

vi /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
	# haproxy proxy
	upstream haproxy-cluster-sit.dev10w.cn {                                                         
        server 192.168.8.35:8010;
        server 192.168.8.36:8010;
		server 192.168.8.37:8010;
        keepalive 60000;
    }
	server {
		#listen    80;
		listen     192.168.8.31:80;  # vip-1 地址
		server_name             haproxy-cluster-sit.dev10w.cn;
		
		location / {
			access_log			logs/haproxy-cluster-sit.access.log main;
			error_log 			logs/haproxy-cluster-sit.error.log;
			proxy_pass			http://haproxy-cluster-sit.dev10w.cn;
			proxy_set_header 	Host $host:$server_port;
			proxy_set_header 	X-Forwarded-For $proxy_add_x_forwarded_for;
        	proxy_set_header 	Host $host;
        	proxy_set_header 	Host $upstream_http_name;
		}
	}
#----------------------------------- 编辑内容,如下 -----------------------------end


2.2 nginx-2 节点

创建配置文件 : tcp

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf

vi  /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
    # mariadb proxy
    upstream mariadb-cluster-sit.dev10w.cn { 
        hash $remote_addr consistent;
        server 192.168.8.35:3306;
        server 192.168.8.36:3306;
        server 192.168.8.37:3306;
    }
    server {
        # listen 13306;
        listen 192.168.8.32:13306; # vip-2 地址
        proxy_connect_timeout 60s;					# 连接超时时间
        proxy_timeout 300s;							# 端口保持时间
        proxy_pass mariadb-cluster-sit.dev10w.cn;
    }
#----------------------------------- 编辑内容,如下 -----------------------------end

创建配置文件 : http

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf

vi /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
	# haproxy proxy
	upstream haproxy-cluster-sit.dev10w.cn {                                                         
        server 192.168.8.35:8010;
        server 192.168.8.36:8010;
		server 192.168.8.37:8010;
        keepalive 60000;
    }
	server {
		#listen    80;
		listen     192.168.8.32:80; # vip-2 地址
		server_name             haproxy-cluster-sit.dev10w.cn;
		
		location / {
			access_log			logs/haproxy-cluster-sit.access.log main;
			error_log 			logs/haproxy-cluster-sit.error.log;
			proxy_pass			http://haproxy-cluster-sit.dev10w.cn;
			proxy_set_header 	Host $host:$server_port;
			proxy_set_header 	X-Forwarded-For $proxy_add_x_forwarded_for;
        	proxy_set_header 	Host $host;
        	proxy_set_header 	Host $upstream_http_name;
		}
	}
#----------------------------------- 编辑内容,如下 -----------------------------end


2.3 nginx-3 节点

创建配置文件 : tcp

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf

vi  /home/app/nginx/app_conf/mariadb-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
    # mariadb proxy
    upstream mariadb-cluster-sit.dev10w.cn { 
        hash $remote_addr consistent;
        server 192.168.8.35:3306;
        server 192.168.8.36:3306;
        server 192.168.8.37:3306;
    }
    server {
        # listen 13306;
        listen 192.168.8.33:13306; # vip-3 地址
        proxy_connect_timeout 60s;					# 连接超时时间
        proxy_timeout 300s;							# 端口保持时间
        proxy_pass mariadb-cluster-sit.dev10w.cn;
    }
#----------------------------------- 编辑内容,如下 -----------------------------end

创建配置文件 : http

备注:注意文件编码格式为 "UTF-8 无BOM编码"

路径 = /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf

vi /home/app/nginx/web_conf/haproxy-cluster-sit.dev10w.cn/nginx.conf
#----------------------------------- 编辑内容,如下 -----------------------------begin
	# haproxy proxy
	upstream haproxy-cluster-sit.dev10w.cn {                                                         
        server 192.168.8.35:8010;
        server 192.168.8.36:8010;
		server 192.168.8.37:8010;
        keepalive 60000;
    }
	server {
		#listen    80;
		listen     192.168.8.33:80; # vip-3 地址
		server_name             haproxy-cluster-sit.dev10w.cn;
		
		location / {
			access_log			logs/haproxy-cluster-sit.access.log main;
			error_log 			logs/haproxy-cluster-sit.error.log;
			proxy_pass			http://haproxy-cluster-sit.dev10w.cn;
			proxy_set_header 	Host $host:$server_port;
			proxy_set_header 	X-Forwarded-For $proxy_add_x_forwarded_for;
        	proxy_set_header 	Host $host;
        	proxy_set_header 	Host $upstream_http_name;
		}
	}
#----------------------------------- 编辑内容,如下 -----------------------------end


步骤 step 5:

注册nginx服务

首先,创建配置文件

# 创建配置文件
cat <<EOF> /home/app/nginx/conf/nginx-env.conf
NGINX_HOME="/home/app/nginx"
NGINX_CONF="/home/app/nginx/conf/nginx.conf"
EOF

然后,创建系统服务文件

创建服务文件
cat <<"EOF"> /etc/systemd/system/nginx.service
[Unit]
Description=Nginx Load Balancer
After=network.target
After=network-online.target
Wants=network-online.target

[Service]
Type=forking
WorkingDirectory=/home/app/nginx
EnvironmentFile=/home/app/nginx/conf/nginx-env.conf
ExecStart=/home/app/nginx/sbin/nginx -c "${NGINX_CONF}" -p "${NGINX_HOME}"
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

注册为系统服务

# 注册系统服务
systemctl enable nginx && \
systemctl daemon-reload && \
systemctl restart nginx

检查服务是否启动成功

# 查看启动端口
# 启动端口,如下:80 和 13306
netstat -nltp 
ps -ef | grep nginx
# 输出结果,如下:
## [root@CNT7XMDBD01 ~]# netstat -nltp 
## Active Internet connections (only servers)
## Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
## tcp        0      0 192.168.8.31:80         0.0.0.0:*               LISTEN      110781/haproxy      
## tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      900/sshd            
## tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN      1299/master         
## tcp        0      0 192.168.8.31:13306      0.0.0.0:*               LISTEN      110781/haproxy      
## tcp6       0      0 :::22                   :::*                    LISTEN      900/sshd            
## [root@CNT7XMDBD01 ~]# ps -ef | grep haproxy
## root       7163   3459  0 11:01 pts/0    00:00:00 grep --color=auto haproxy
## root     110779      1  0 10:46 ?        00:00:00 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
## haproxy  110780 110779  0 10:46 ?        00:00:00 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## haproxy  110781 110780  0 10:46 ?        00:00:00 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## [root@CNT7XMDBD01 ~]# 


# 查看启动日志
systemctl status -l nginx
journalctl -u nginx -n 20
# 输出结果,如下:
## [root@CNT7XMDBD01 ~]# systemctl status -l haproxy
## ● haproxy.service - HAProxy Load Balancer
##    Loaded: loaded (/usr/lib/systemd/system/haproxy.service; enabled; vendor preset: disabled)
##    Active: active (running) since Tue 2023-02-14 10:46:40 CST; 13min ago
##  Main PID: 110779 (haproxy-systemd)
##    CGroup: /system.slice/haproxy.service
##            ├─110779 /usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid
##            ├─110780 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
##            └─110781 /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## 
## Feb 14 10:46:40 CNT7XMDBD01 systemd[1]: Started HAProxy Load Balancer.
## Feb 14 10:46:40 CNT7XMDBD01 haproxy-systemd-wrapper[110779]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## [root@CNT7XMDBD01 ~]# 
## [root@CNT7XMDBD01 ~]# journalctl -u haproxy -n 10
## -- Logs begin at Tue 2023-02-14 09:47:23 CST, end at Tue 2023-02-14 11:00:16 CST. --
## Feb 14 10:28:10 CNT7XMDBD01 haproxy-systemd-wrapper[53450]: haproxy-systemd-wrapper: exit, haproxy RC=0
## Feb 14 10:28:10 CNT7XMDBD01 systemd[1]: Stopped HAProxy Load Balancer.
## Feb 14 10:28:26 CNT7XMDBD01 systemd[1]: Started HAProxy Load Balancer.
## Feb 14 10:28:26 CNT7XMDBD01 haproxy-systemd-wrapper[77083]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## Feb 14 10:40:25 CNT7XMDBD01 systemd[1]: Stopping HAProxy Load Balancer...
## Feb 14 10:40:25 CNT7XMDBD01 haproxy-systemd-wrapper[77083]: haproxy-systemd-wrapper: SIGTERM -> 77085.
## Feb 14 10:40:25 CNT7XMDBD01 haproxy-systemd-wrapper[77083]: haproxy-systemd-wrapper: exit, haproxy RC=0
## Feb 14 10:40:25 CNT7XMDBD01 systemd[1]: Stopped HAProxy Load Balancer.
## Feb 14 10:46:40 CNT7XMDBD01 systemd[1]: Started HAProxy Load Balancer.
## Feb 14 10:46:40 CNT7XMDBD01 haproxy-systemd-wrapper[110779]: haproxy-systemd-wrapper: executing /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -Ds
## [root@CNT7XMDBD01 ~]# 


最后,测试nginx转发tcp协议和端口的方法,在后面第三章介绍。

Tags:

本文暂时没有评论,来添加一个吧(●'◡'●)

欢迎 发表评论:

最近发表
标签列表