搜尋此網誌

2015年7月15日 星期三

CentOS7 - 使用rvm安裝ruby環境

下載套件

[root@jyc-blog ~]# yum install gcc-c++ patch readline readline-devel zlib zlib-devel libyaml-devel libffi-devel openssl-devel make bzip2 autoconf automake libtool bison iconv-devel -y


安裝 RVM ( Ruby Version Manager )

下列指令可以安裝最新版的RVM,這兩行指令會自動下載所需的檔案並自動安裝: 
[root@jyc-blog ~]# gpg2 --keyserver hkp://keys.gnupg.net --recv-keys D39DC0E3

[root@jyc-blog ~]# curl -L get.rvm.io | bash -s stable


設定RVM環境

安裝完RVM後,要先輸入下列的指令設定RVM的環境: 

[root@jyc-blog ~]# source /etc/profile.d/rvm.sh

安裝Ruby

RVM可以安裝多個Ruby版本在同一台機器上,輸入下列的指令安裝你所需的Ruby版本 :

[root@jyc-blog ~]#  rvm install 2.2.2

或是你也可以安裝其它版本

[root@jyc-blog ~]# rvm install 1.9.3

設定預設要使用的Ruby版本


[root@jyc-blog ~]#  rvm use 2.2.2 --default 


查看目前Ruby的版本


[root@jyc-blog ~]#  ruby --version
ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux]



2015年7月9日 星期四

CentOS 7 設定PAM透過資料庫(MySQL、MariaDB)認證使用者身份 - pam_mysql & libnss_mysql


環境準備

在開始安裝之前,請先完成下列的步驟:
1. 安裝EPEL Repository,安裝步驟請參考這篇文章:
http://jyc-blog.blogspot.tw/2015/07/centos7-epel-repository.html

2. 關閉SELinux


下載套件

[root@jyc-blog ~]# yum groupinstall "Development Tools" -y
[root@jyc-blog ~]# yum install cyrus-sasl openssl mariadb mariadb-devel mariadb-server pam-devel libnss-mysql -y


編譯pam_mysql

pam_mysql的tarball檔可以到它的官方網站下載:

[root@jyc-blog ~]# cd /usr/local/src
[root@jyc-blog src]# wget http://prdownloads.sourceforge.net/pam-mysql/pam_mysql-0.7RC1.tar.gz
[root@jyc-blog src]# tar -zxvf pam_mysql-0.7RC1.tar.gz
[root@jyc-blog src]# cd pam_mysql-0.7RC1/
[root@jyc-blog src]# ./configure --with-pam-mods-dir=/usr/lib64/security  #指定pam module要存放到哪一個目錄底下
...
...省略...
...
config.status: creating Makefile
config.status: creating pam_mysql.spec
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing default-1 commands

[root@jyc-blog src]# make install

#確認pam_mysql.so安裝成功
[root@jyc-blog src]# ls /usr/lib64/security/pam_mysql.so
/usr/lib64/security/pam_mysql.so


設定mysql

[root@jyc-blog ~]# systemctl start mariadb
[root@jyc-blog ~]# systemctl enable mariadb
[root@jyc-blog ~]# mysql_secure_installation

/usr/bin/mysql_secure_installation: line 379: find_mysql_client: command not found

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none): <ENTER>
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? [Y/n] Y
New password: PASSWORD
Re-enter new password: PASSWORD
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] Y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] Y
 ... Success!

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] Y
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] Y
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!



設定libnss-mysql

libnss-mysql提供三個設定檔的範例:

complex: 
/usr/share/doc/libnss-mysql-1.5/sample/complex/libnss-mysql-root.cfg
/usr/share/doc/libnss-mysql-1.5/sample/complex/libnss-mysql.cfg
/usr/share/doc/libnss-mysql-1.5/sample/complex/sample_database.sql

linux:
/usr/share/doc/libnss-mysql-1.5/sample/linux/libnss-mysql-root.cfg
/usr/share/doc/libnss-mysql-1.5/sample/linux/libnss-mysql.cfg
/usr/share/doc/libnss-mysql-1.5/sample/linux/sample_database.sql

minimal:
/usr/share/doc/libnss-mysql-1.5/sample/minimal/libnss-mysql-root.cfg
/usr/share/doc/libnss-mysql-1.5/sample/minimal/libnss-mysql.cfg
/usr/share/doc/libnss-mysql-1.5/sample/minimal/sample_database.sql

這三種範例之間的差異只有資料庫欄位的多寡,complex的欄位最多,因此可以設定的使用者資訊也最多。
minimal則只有username、password、group、uid和gid這些欄位。
linux的欄位就跟/etc/passwd的欄位設定一樣,此為套件預設值。


# 先複製一份sample sql,有一些地方需要修改
[root@jyc-blog ~]# cp /usr/share/doc/libnss-mysql-1.5/sample/linux/sample_database.sql /root/linux_sample_database.sql

#將TYPE=MyISAM從sample檔刪除,否則在匯入時會出現Syntax Error
[root@jyc-blog ~]# sed -i "s:TYPE=MyISAM::g" linux_sample_database.sql

[root@jyc-blog ~]# vim /root/linux_sample_database.sql

# line 23  設定資料庫名稱 ,如果有變動連下面有資料庫名稱的部份也要跟著修改
create database auth;
use auth;

# line 62 - 67 這部份是建立新使用者的sql範例
INSERT INTO users (username,gecos,homedir,password)
    VALUES ('jychen', 'Jheng-Yu Chen', '/home/jychen', ENCRYPT('jychen'));
INSERT INTO groups (name)
    VALUES ('jychen');
INSERT INTO grouplist (gid,username)
    VALUES (5000,'jychen');

# line 70 - 93 comment out. 這部份新增的資料庫使用者,只能查詢不能寫入,但我希望能透過passwd指令修改使用者的密碼,因此後面的使用者設定都使用root身份
#GRANT USAGE ON *.* TO `nss-root`@`localhost` IDENTIFIED BY 'rootpass';
#GRANT USAGE ON *.* TO `nss-user`@`localhost` IDENTIFIED BY 'userpass';
#
#GRANT Select (`username`, `uid`, `gid`, `gecos`, `homedir`, `shell`, `password`,
#              `lstchg`, `min`, `max`, `warn`, `inact`, `expire`, `flag`)
#             ON `auth`.`users`
#             TO 'nss-root'@'localhost';
#GRANT Select (`name`, `password`, `gid`)
#             ON `auth`.`groups`
#             TO 'nss-root'@'localhost';
#
#GRANT Select (`username`, `uid`, `gid`, `gecos`, `homedir`, `shell`)
#             ON `auth`.`users`
#             TO 'nss-user'@'localhost';
#GRANT Select (`name`, `password`, `gid`)
#             ON `auth`.`groups`
#             TO 'nss-user'@'localhost';
#
#GRANT Select (`username`, `gid`)
#             ON `auth`.`grouplist`
#             TO 'nss-user'@'localhost';
#GRANT Select (`username`, `gid`)
#             ON `auth`.`grouplist`
#             TO 'nss-root'@'localhost';



到目前為止,我們已經把資料庫和要匯入的sql檔案給設定好了,接下就把sql檔匯入資料庫 :

[root@jyc-blog ~]# mysql -u root -pPASSWORD < linux_sample_database.sql

#查看是否有匯入成功
[root@jyc-blog ~]# mysql -u root -pPASSWORD -e "use auth; select * from users;"
+----------+------+------+---------------+--------------+-----------+---------------+--------+-----+-------+------+-------+--------+------+
| username | uid  | gid  | gecos      | homedir      | shell     | passwor   | lstchg | min | max   | warn | inact | expire | flag |
+----------+------+------+---------------+--------------+-----------+---------------+--------+-----+-------+------+-------+--------+------+
| jychen   | 5000 | 5000 | Jheng-Yu Chen | /home/jychen | /bin/bash | 9OrIPVOVR0zk. | 1 | 0 | 99999 |    0 | 0 | -1 | 0 |
+----------+------+------+---------------+--------------+-----------+---------------+--------+-----+-------+------+-------+--------+------+



資料庫匯入完成後,再來就要修改libnss-mysql的設定檔,因為預設已經將設定檔放在/etc目錄底下,我們只需要修改這些設定檔就可以了:

[root@jyc-blog ~]# vim /etc/libnss-mysql.cfg

# line 34 - 39 modify
host        localhost
database    auth
username    root
password    PASSWORD
socket      /var/lib/mysql/mysql.sock
port        3306


[root@jyc-blog ~]# vim /etc/libnss-mysql-root.cfg

# line 1 modify
username    root
password    PASSWORD


再來要修改Name Service Switch的設定檔,讓它能夠讀取MySQL的認證資料:

[root@jyc-blog ~]# vim /etc/nsswitch.conf

# line 33 - 35 modify
passwd:     files sss mysql
shadow:     files sss mysql
group:      files sss mysql



設定pam_mysql

詳細的設定可以參考官方的Document:
[root@jyc-blog ~]# vim /etc/pam_mysql.conf

#add following
users.host=/var/lib/mysql/mysql.sock
users.db_user=root
users.db_passwd=PASSWORD
users.database=auth
users.table=users
users.user_column=username
users.password_column=password
users.password_crypt=1
verbose=0


修改PAM


[root@jyc-blog ~]# vim /etc/pam.d/system-auth

#插入下面五行(綠色字)
auth        required      pam_env.so
auth        sufficient    pam_unix.so nullok try_first_pass
auth        requisite     pam_succeed_if.so uid >= 1000 quiet_success
auth        sufficient  pam_mysql.so    config_file=/etc/pam_mysql.conf
auth        required      pam_deny.so

account     required      pam_unix.so
account     sufficient    pam_localuser.so
account     sufficient    pam_succeed_if.so uid < 1000 quiet
account     sufficient  pam_mysql.so    config_file=/etc/pam_mysql.conf
account     required      pam_permit.so

password    requisite     pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type=
password    sufficient    pam_unix.so sha512 shadow nullok try_first_pass use_authtok
password     sufficient  pam_mysql.so    config_file=/etc/pam_mysql.conf
password    required      pam_deny.so

session     optional      pam_keyinit.so revoke
session     required      pam_limits.so
-session     optional      pam_systemd.so
#因為是透過資料庫新增使用者,所以需要手動建立使用者的家目錄,但自己手動建立還要設定權限等等,因此這邊以PAM的方式,讓使用者在登入的同時自動建立家目錄:
session     optional      pam_mkhomedir.so umask=0077
session     [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
session     sufficient  pam_mysql.so    config_file=/etc/pam_mysql.conf
session     required      pam_unix.so




測試

[root@jyc-blog ~]# id jychen
uid=5000(jychen) gid=5000(jychen) groups=5000(jychen)

[root@jyc-blog ~]# ssh jychen@127.0.0.1
The authenticity of host '127.0.0.1 (127.0.0.1)' can't be established.
ECDSA key fingerprint is 3e:95:18:bb:65:98:ff:4b:cb:15:34:ee:b1:f7:22:3f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '127.0.0.1' (ECDSA) to the list of known hosts.
jychen@127.0.0.1's password:  PASSWORD
Creating directory '/home/jychen'.
Last login: Thu Jul  9 23:52:36 2015 from localhost
[jychen@jyc-blog ~]$ exit


#用passwd指令修改密碼
[root@jyc-blog ~]# mysql -u root -pPASSWORD -e "use auth ; select password from users where uid=5000"
+---------------------+
|       password       |
+---------------------+
| 1uQQ2ugLqZa.A |
+---------------------+

[root@jyc-blog ~]#  passwd jychen
Changing password for user jychen.
New password: PASSWORD
Retype new password: PASSWORD
passwd: all authentication tokens updated successfully.

[root@jyc-blog ~]# mysql -u root -pPASSWORD -e "use auth ; select password from users where uid=5000"
+-------------------------------------------------------+
|                              password                             |
+-------------------------------------------------------+
| $1$WRJqJIP1$U3nAswi3F6RJtGtGD0OyW. |
+-------------------------------------------------------+
#從第一次和第二次Select出來的password資料,可以看出密碼有被修改成功。

Enjoy!






可能會遇到的錯誤

[root@jyc-blog pam_mysql-0.7RC1]# ./configure --with-pam-mods-dir=/usr/lib64/security
...
...省略...
...
checking if /usr /usr/local /usr/mysql /opt/mysql is a mysql_config script... no
checking mysql_config availability in /usr/bin... no
checking mysqlclient availability in /usr/lib... no
checking mysqlclient availability in /usr/lib/mysql... no
checking mysql headers availability in /usr/include... no
checking mysql headers availability in /usr/include/mysql... no
checking mysql_config availability in /usr/local/bin... no
checking mysqlclient availability in /usr/local/lib... no
checking mysqlclient availability in /usr/local/lib/mysql... no
checking mysql headers availability in /usr/local/include... no
checking mysql headers availability in /usr/local/include/mysql... no
checking mysql_config availability in /usr/mysql/bin... no
checking mysqlclient availability in /usr/mysql/lib... no
checking mysqlclient availability in /usr/mysql/lib/mysql... no
checking mysql headers availability in /usr/mysql/include... no
checking mysql headers availability in /usr/mysql/include/mysql... no
checking mysql_config availability in /opt/mysql/bin... no
checking mysqlclient availability in /opt/mysql/lib... no
checking mysqlclient availability in /opt/mysql/lib/mysql... no
checking mysql headers availability in /opt/mysql/include... no
checking mysql headers availability in /opt/mysql/include/mysql... no
configure: error: Cannot locate mysql client library. Please check your mysql installation.

#解決方式 : 安裝mariadb-devel
[root@jyc-blog pam_mysql-0.7RC1]# yum install mariadb-devel -y



[root@jyc-blog pam_mysql-0.7RC1]# ./configure --with-pam-mods-dir=/usr/lib64/security
...
...省略...
...
configure: error: Cannot find pam headers. Please check if your system is ready for pam module development.

#解決方式 : 安裝pam-devel
[root@jyc-blog pam_mysql-0.7RC1]# yum install pam-devel -y



[root@jyc-blog pam_mysql-0.7RC1]# ./configure
...
...省略...
...
configure: error: Cannot find pam headers. Please check if your system is ready for pam module development.

#解決方式 : 指定pam module的存放目錄
[root@jyc-blog pam_mysql-0.7RC1]# ./configure --with-pam-mods-dir=/usr/lib64/security



CentOS7 安裝 EPEL Repository

 Red Hat 系列的Distribution官方的套件庫,有很多常用到的套件都沒有收錄在裡面,因此這篇文章介紹如何安裝EPEL的套件庫。

EPEL(Extra Packages for Enterprise Linux)是一個由 Fedora Special Interest Group 社群所維護的套件庫,這個套件庫可用於Red Hat系列的Distribution,如RHEL、CentOS、Scientific。

下載RPM

EPEL套件庫的RPM可以到這個網址下載:

並瀏覽到如下圖的區塊,可以看到有三個連結可以點選,分別支援RHEL 7、6和5,本文章所用的系統是CentOS7 ,因此要下載「epel-release for EL7」。


安裝epel-release

[root@jyc-blog ~]# rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Retrieving https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
warning: /var/tmp/rpm-tmp.FNRwTD: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
1:epel-release-7-5                 ################################# [100%]



測試

[root@jyc-blog ~]# yum update

Loaded plugins: fastestmirror
epel/x86_64/metalink                                     | 3.2 kB     00:00
epel                                                     | 4.4 kB     00:00
(1/2): epel/x86_64/group_gz                                | 169 kB   00:00
(2/2): epel/x86_64/primary_db                              | 3.6 MB   00:00
(1/2): epel/x86_64/updateinfo                              | 430 kB   00:00
(2/2): epel/x86_64/pkgtags                                 | 1.6 MB   00:00
^C



2015年7月4日 星期六

CentOS 7 架設LVS-DR + Keepalived + ProFTPd (2 Nodes)

當伺服器有很多台的時候,架設LVS Cluster當然沒有什麼問題,但是當你的伺服器只有兩台或三台的時候,還要把其中一台拿來當LVS Router就有點浪費了,何況只有一台LVS Router還會有單點失效(SPoF)的問題,因此這篇文章就是教大家如何在伺服器只有兩台的情況下,還能提供具有負載平衡與備援機制的服務。

架構




環境準備

我的VirtualBox版本是4.3.28,兩台主機都要準備兩張網卡。

介面卡1: 在「附加到(A):」欄位選擇「僅限主機介面卡」,並新增一個新的172.24.10.0網段的網路。
介面卡1是設定LVS時會使用到的網路卡,但僅限主機介面卡不能連線到網際網路,所以要再準備另一張網路卡,用於連線到網際網路下載所需要的套件。

介面卡2 : 請選擇任何一種網路,只要能連線到網際網路下載套件就可以,LVS的設定不會用到這個網段。

LVS不支援NetworkManager,因此必須將NetworkManager停用,以手動的方式修改設定檔如下圖:

兩台虛擬機的網路設定都相同,只需要將IP Address改掉即可。


ProFTPd的設定請參考這篇文章: http://jyc-blog.blogspot.tw/2015/07/centos-7-proftpd-ftpftpssftp.html


安裝套件

############lvs_active##############
[root@lvs_active ~]# yum install keepalived ipvsadm -y


############lvs_backup##############
[root@lvs_backup ~]# yum install keepalived ipvsadm -y




設定LVS與Keepalived (lvs_active)


############lvs_active##############
#要建一個全新的設定檔,將原本的keepalived設定檔重新命名
[root@lvs_active ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.original

#建立設定檔
[root@lvs_active ~]# vim /etc/keepalived/keepalived.conf

#add following
global_defs {
        notification_email {
                root@localhost
        }
        notification_email_from root@localhost
        smtp_server 127.0.0.1
        smtp_connect_timeout 60
}
vrrp_instance RH_1 {
        state MASTER
        interface enp0s3
        virtual_router_id 50
        priority 100
        advert_int 1
        authentication {
                auth_type PASS
                auth_pass password123
        }
        virtual_ipaddress {
                172.24.10.100
        }
}
virtual_server 172.24.10.100 21 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.101 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.102 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}
virtual_server 172.24.10.100 2221 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.101 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.102 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}

[root@lvs_active ~]# systemctl start keepalived.service
[root@lvs_active ~]# systemctl enable keepalived.service

[root@lvs_active ~]# vim /etc/sysctl.conf

#add following
net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.conf.enp0s3.arp_ignore = 1

net.ipv4.conf.enp0s3.arp_announce = 2

[root@lvs_active ~]# sysctl -p

#載入LVS的FTP module
[root@lvs_active ~]# modprobe ip_vs_ftp

#設定開機自動載入Kernel module
[root@lvs_active ~]# vim /etc/modules-load.d/ip_vs_ftp.conf

#add following
ip_vs_ftp



設定LVS與Keepalived (lvs_backup)


############lvs_backup##############
[root@lvs_backup ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.original

#建立設定檔
[root@lvs_backup ~]# vim /etc/keepalived/keepalived.conf

#add following
global_defs {
        notification_email {
                root@localhost
        }
        notification_email_from root@localhost
        smtp_server 127.0.0.1
        smtp_connect_timeout 60
}
vrrp_instance RH_1 {
        state BACKUP
        interface enp0s3
        virtual_router_id 50
        priority 80                #Backup node的priority值要比Active node的值小
        advert_int 1
        authentication {
                auth_type PASS
                auth_pass password123
        }
        virtual_ipaddress {
                172.24.10.100
        }
}
virtual_server 172.24.10.100 21 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.101 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.102 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}
virtual_server 172.24.10.100 2221 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.101 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.102 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}

[root@lvs_backup ~]# systemctl start keepalived.service
[root@lvs_backup ~]# systemctl enable keepalived.service

[root@lvs_backup ~]# vim /etc/sysctl.conf

#add following
net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.conf.enp0s3.arp_ignore = 1

net.ipv4.conf.enp0s3.arp_announce = 2

[root@lvs_backup ~]# sysctl -p

#載入LVS的FTP module
[root@lvs_backup ~]# modprobe ip_vs_ftp

#設定開機自動載入Kernel module
[root@lvs_backup ~]# vim /etc/modules-load.d/ip_vs_ftp.conf

#add following
ip_vs_ftp



設定防火牆 (lvs_active)


[root@lvs_active ~]# systemctl start iptables
[root@lvs_active ~]# systemctl enable iptables
[root@lvs_active ~]# vim iptables.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 21 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 40000:45000 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 2221 -j MARK --set-mark 2221
iptables-save > /etc/sysconfig/iptables


[root@lvs_active ~]# sh iptables.sh

#載入iptables的FTP模組
[root@lvs_active ~]# vim /etc/sysconfig/iptables-config

#line 6 modify
IPTABLES_MODULES="ip_nat_ftp ip_conntrack_ftp"

[root@lvs_active ~]# systemctl restart iptables.service


設定防火牆 (lvs_backup)


[root@lvs_backup ~]# systemctl start iptables
[root@lvs_backup ~]# systemctl enable iptables
[root@lvs_backup ~]# vim iptables.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 21 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 40000:45000 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 2221 -j MARK --set-mark 2221
#下面三行Redirect Rule如果也設定在lvs_active,則Client連線到VIP(172.24.10.100)的時候,不會透過LVS轉發封包,這會導致Client只能夠跟lvs_active連線,這就失去負載平衡的功能了。
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 21 -j REDIRECT
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 2221 -j REDIRECT

iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 40000:45000 -j REDIRECT
iptables-save > /etc/sysconfig/iptables


[root@lvs_backup ~]# sh iptables.sh

#載入iptables的FTP模組

[root@lvs_backup ~]# vim /etc/sysconfig/iptables-config

#line 6 modify
IPTABLES_MODULES="ip_nat_ftp ip_conntrack_ftp"

[root@lvs_backup ~]# systemctl restart iptables.service


查看連線狀態:


如果在測試的時候發現一直連到同一台Real Server的話,這是正常的,只要多換幾個IP試試就可以了,因為在設定的時候我加入了「persistence_timeout」與「firewall mark」,因此同一個IP連線的時候會被導向同一台Real Server。

2015年7月2日 星期四

CentOS 7 架設LVS-DR + Keepalived + ProFTPd (4 Nodes)

架構




環境準備

我的VirtualBox版本是4.3.28,四台主機都要準備兩張網卡。

介面卡1: 在「附加到(A):」欄位選擇「僅限主機介面卡」,並新增一個新的172.24.10.0網段的網路。
介面卡1是設定LVS時會使用到的網路卡,但僅限主機介面卡不能連線到網際網路,所以要再準備另一張網路卡,用於連線到網際網路下載所需要的套件。

介面卡2 : 請選擇任何一種網路,只要能連線到網際網路下載套件就可以,LVS的設定不會用到這個網段。

LVS不支援NetworkManager,因此必須將NetworkManager停用,以手動的方式修改設定檔如下圖:

四台虛擬機的網路設定都相同,只需要將IP Address改掉即可。


ProFTPd的設定請參考這篇文章: http://jyc-blog.blogspot.tw/2015/07/centos-7-proftpd-ftpftpssftp.html


安裝套件

############lvs_active##############
[root@lvs_active ~]# yum install keepalived ipvsadm -y


############lvs_backup##############
[root@lvs_backup ~]# yum install keepalived ipvsadm -y


############proftpd1##############
[root@proftpd1 ~]# yum install arptables -y


############proftpd2##############
[root@proftpd2 ~]# yum install arptables -y




設定LVS與Keepalived


############lvs_active##############
#要建一個全新的設定檔,將原本的keepalived設定檔重新命名
[root@lvs_active ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.original

#建立設定檔
[root@lvs_active ~]# vim /etc/keepalived/keepalived.conf

#add following
global_defs {
        notification_email {
                root@localhost
        }
        notification_email_from root@localhost
        smtp_server 127.0.0.1
        smtp_connect_timeout 60
}
vrrp_instance RH_1 {
        state MASTER
        interface enp0s3
        virtual_router_id 50
        priority 100
        advert_int 1
        authentication {
                auth_type PASS
                auth_pass password123
        }
        virtual_ipaddress {
                172.24.10.100
        }
}
virtual_server 172.24.10.100 21 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.103 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.104 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}
virtual_server 172.24.10.100 2221 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.103 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.104 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}

[root@lvs_active ~]# systemctl start keepalived.service
[root@lvs_active ~]# systemctl enable keepalived.service

[root@lvs_active ~]# vim /etc/sysctl.conf

#add following
net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

[root@lvs_active ~]# sysctl -p

#載入LVS的FTP module
[root@lvs_active ~]# modprobe ip_vs_ftp

#設定開機自動載入Kernel module
[root@lvs_active ~]# vim /etc/modules-load.d/ip_vs_ftp.conf

#add following
ip_vs_ftp



############lvs_backup##############
[root@lvs_backup ~]# mv /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.original

#建立設定檔
[root@lvs_backup ~]# vim /etc/keepalived/keepalived.conf

#add following
global_defs {
        notification_email {
                root@localhost
        }
        notification_email_from root@localhost
        smtp_server 127.0.0.1
        smtp_connect_timeout 60
}
vrrp_instance RH_1 {
        state BACKUP
        interface enp0s3
        virtual_router_id 50
        priority 80                #Backup node的priority值要比Active node的值小
        advert_int 1
        authentication {
                auth_type PASS
                auth_pass password123
        }
        virtual_ipaddress {
                172.24.10.100
        }
}
virtual_server 172.24.10.100 21 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.103 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.104 21 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}
virtual_server 172.24.10.100 2221 {
        delay_loop 10
        lb_algo rr
        lb_kind DR
        persistence_timeout 9600
        protocol TCP
        real_server 172.24.10.103 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
        real_server 172.24.10.104 2221 {
                weight 1
                TCP_CHECK {
                        connect_timeout 10
                }
        }
}

[root@lvs_backup ~]# systemctl start keepalived.service
[root@lvs_backup ~]# systemctl enable keepalived.service

[root@lvs_backup ~]# vim /etc/sysctl.conf

#add following
net.ipv4.ip_forward = 1
net.ipv4.ip_nonlocal_bind = 1

[root@lvs_backup ~]# sysctl -p

#載入LVS的FTP module
[root@lvs_backup ~]# modprobe ip_vs_ftp

#設定開機自動載入Kernel module
[root@lvs_backup ~]# vim /etc/modules-load.d/ip_vs_ftp.conf

#add following
ip_vs_ftp



設定防火牆



############lvs_active##############
[root@lvs_active ~]# systemctl start iptables
[root@lvs_active ~]# systemctl enable iptables
[root@lvs_active ~]# vim iptables.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 21 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 40000:45000 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 2221 -j MARK --set-mark 2221
iptables-save > /etc/sysconfig/iptables


[root@lvs_active ~]# sh iptables.sh
[root@lvs_active ~]# systemctl restart iptables.service




############lvs_backup##############
[root@lvs_backup ~]# systemctl start iptables
[root@lvs_backup ~]# systemctl enable iptables
[root@lvs_backup ~]# vim iptables.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 21 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 40000:45000 -j MARK --set-mark 21
iptables -t mangle -A PREROUTING -p tcp -d 172.24.10.100/32 --dport 2221 -j MARK --set-mark 2221
iptables-save > /etc/sysconfig/iptables


[root@lvs_backup ~]# sh iptables.sh
[root@lvs_backup ~]# systemctl restart iptables.service 



設定Real Server

############proftpd1############
[root@proftpd1 ~]# systemctl start arptables.service
[root@proftpd1 ~]# systemctl enable arptables.service
[root@proftpd1 ~]# vim arptable.sh

#add following
arptables -F
arptables -X
arptables -P INPUT ACCEPT
arptables -P OUTPUT ACCEPT
arptables -P FORWARD ACCEPT
arptables -A INPUT -d 172.24.10.100 -j DROP
arptables -A OUTPUT -s 172.24.10.100 -j mangle --mangle-ip-s 172.24.10.103
arptables-save > /etc/sysconfig/arptables

[root@proftpd1 ~]# sh arptable.sh
[root@proftpd1 ~]# systemctl restart arptables.service

[root@proftpd1 ~]# systemctl start iptables.service
[root@proftpd1 ~]# systemctl enable iptables.service
[root@proftpd1 ~]# vim iptable.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 21 -j REDIRECT
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 2221 -j REDIRECT
iptables-save > /etc/sysconfig/iptables

[root@proftpd1 ~]# sh iptable.sh

#載入iptables的FTP模組
[root@proftpd1 ~]# vim /etc/sysconfig/iptables-config

#line 6 modify
IPTABLES_MODULES="ip_nat_ftp ip_conntrack_ftp"

[root@proftpd1 ~]# systemctl restart iptables.service

[root@proftpd1 ~]# vim /etc/sysctl.conf

#add following
net.ipv4.conf.enp0s3.arp_ignore = 1
net.ipv4.conf.enp0s3.arp_announce = 2

[root@proftpd1 ~]# sysctl -p

############proftpd2############
[root@proftpd2 ~]# systemctl start arptables.service
[root@proftpd2 ~]# systemctl enable arptables.service
[root@proftpd2 ~]# vim arptable.sh

#add following
arptables -F
arptables -X
arptables -P INPUT ACCEPT
arptables -P OUTPUT ACCEPT
arptables -P FORWARD ACCEPT
arptables -A INPUT -d 172.24.10.100 -j DROP
arptables -A OUTPUT -s 172.24.10.100 -j mangle --mangle-ip-s 172.24.10.104
arptables-save > /etc/sysconfig/arptables

[root@proftpd2 ~]# sh arptable.sh
[root@proftpd2 ~]# systemctl restart arptables.service

[root@proftpd2 ~]# systemctl start iptables.service
[root@proftpd2 ~]# systemctl enable iptables.service
[root@proftpd2 ~]# vim iptable.sh

#add following
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
iptables -P INPUT ACCEPT
iptables -P FORWARD ACCEPT
iptables -P OUTPUT ACCEPT
iptables -P INPUT DROP
iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -p vrrp -j ACCEPT
iptables -A INPUT -s 172.24.10.0/24 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 21 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp --dport 2221 -j ACCEPT
iptables -A INPUT -p tcp -m state --state NEW -m tcp -m multiport --dports 40000:45000 -j ACCEPT
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 21 -j REDIRECT
iptables -t nat -A PREROUTING -p tcp -d 172.24.10.100 --dport 2221 -j REDIRECT
iptables-save > /etc/sysconfig/iptables

[root@proftpd2 ~]# sh iptable.sh

#載入iptables的FTP模組
[root@proftpd2 ~]# vim /etc/sysconfig/iptables-config

#line 6 modify
IPTABLES_MODULES="ip_nat_ftp ip_conntrack_ftp"

[root@proftpd2 ~]# systemctl restart iptables.service

[root@proftpd2 ~]# vim /etc/sysctl.conf

#add following
net.ipv4.conf.enp0s3.arp_ignore = 1
net.ipv4.conf.enp0s3.arp_announce = 2

[root@proftpd2 ~]# sysctl -p



查看連線狀態:

如果在測試的時候發現一直連到同一台Real Server的話,這是正常的,只要多換幾個IP試試就可以了,因為在設定的時候我加入了「persistence_timeout」與「firewall mark」,因此同一個IP連線的時候會被導向同一台Real Server。