설치 전 개요
- 네트워크 구성
VM | Public IP | Private IP | Virtual IP | Netmask | Gateway | DNS Server | SCAN IP |
RAC1 | 192.168.20.241 | 10.0.1.101 | 192.168.20.243 | 255.255.255.0 | 192.168.20.254 | 127.0.0.1 | 192.168.20.245 |
RAC2 | 192.168.20.242 | 10.0.1.102 | 192.168.20.244 | 255.255.255.0 | 192.168.20.254 | 127.0.0.1 |
- 서버
VM | Host Name | Memory | Net Adapter 1 | Net Adapter 2 | 구성 방법 | 비고 |
RAC1 | rac1 | 10GB | 어댑터에 브리지 | 호스트 전용 어댑터 | 리눅스 설치 | dnsmasq를 사용하여 DNS 구성 |
RAC2 | rac2 | 10GB | 어댑터에 브리지 | 호스트 전용 어댑터 | RAC1 복제 | dnsmasq를 사용하여 DNS 구성 |
- 스토리지
파일 이름용량타입용도비고
파일 이름 | 용량 | 타입 | 용도 | 비고 |
RAC1.vdi | 100GB | Dynamic / Normal | RAC1 노드 스토리지 | 디스크 사용량을 최소화하기 위해 Dynamic으로 생성합니다. |
RAC2.vdi | 100GB | Dynamic / Normal | RAC2 노드 스토리지 | vboxmanage clonemedium을 이용하여 RAC1.vdi 파일을 복제합니다. |
CRS1.vdi | 1GB | Fixed / Shareable | CRS / Vote 저장 영역 | 공유(shareable) 디스크로 사용하기 위해서는 Fixed Size로 생성해야 합니다. |
CRS2.vdi | 1GB | Fixed / Shareable | ||
CRS3.vdi | 1GB | Fixed / Shareable | ||
GIMR1.vdi | 40GB | Fixed / Shareable | Grid Infrastructure Management Repository | |
GIMR2.vdi | 40GB | Fixed / Shareable | ||
GIMR3.vdi | 40GB | Fixed / Shareable | ||
DATA1.vdi | 50GB | Fixed / Shareable | DATA 영역 | |
DATA2.vdi | 50GB | Fixed / Shareable | ||
FRA1.vdi | 50GB | Fixed / Shareable | Fast Recovery Area |
VM 구성
- VM 생성
rac1 환경설정
- Linux 패키지 추가 설치
rpm -ivh ksh-20120801-26.el7.x86_64.rpm
rpm -ivh libaio-devel-0.3.109-13.el7.x86_64.rpm
- 오라클 라이브러리 설치
yum install -y binutils compat-libcap1 compat-libstdc++-33 \
gcc gcc-c++ glibc glibc-devel ksh libaio libaio-devel \
libX11 libXau libXi libXtst libgcc libstdc++ libstdc++-devel \
make sysstat unixODBC unixODBC-devel
yum install -y smartmontools net-tools
yum install java-1.8.0-openjdk-devel -y
yum install -y dnsmasq
yum install -y bind-utils
yum install -y nfs-utils
* connot find a valid baseurl for repo: base/7/x86_64 오류 시 아래 더보기 링크 참조
- 불필요한 프로세스 중지
# 방화벽 중지
systemctl stop firewalld
systemctl disable firewalld
# 블루투스 중지
systemctl stop bluetooth
systemctl disable bluetooth
# 시간 동기화 중지
systemctl stop chronyd
systemctl disable chronyd
mv /etc/chrony.conf /etc/chrony.conf.bak
# NTP 중지
systemctl stop ntpdate
systemctl disable ntpdate
# DNS 중지
systemctl stop avahi-daemon.socket
systemctl stop avahi-daemon
systemctl disable avahi-daemon
# 가상 시스템 관리 중지
virsh net-destroy default
virsh net-undefine default
- hosts파일 수정
[root@localhost ~]# vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
### Public
192.168.20.241 rac1
192.168.20.242 rac2
### Private
10.0.1.101 rac1-priv
10.0.1.102 rac2-priv
### Virtual
192.168.20.243 rac1-vip
192.168.20.244 rac2-vip
### DNS
192.168.20.245 rac-scan
- selinux 파일 수정
[root@localhost ~]# vi /etc/selinux/config
SELINUX=disabled
- resolv.conf 확인 및 dnsmasq 활성화
[root@rac1 ~]# vi /etc/resolv.conf
# Generated by NetworkManager
nameserver 10.0.1.101
nameserver 8.8.8.8
[root@rac1 ~]# systemctl start dnsmasq
[root@rac1 ~]# systemctl enable dnsmasq
[root@rac1 ~]# nslookup rac-scan
Server: 10.0.1.101
Address: 10.0.1.101#53
Name: rac-scan
Address: 192.168.20.245
- group 및 user 등록, 설정
### 그룹 등록 (필요에 따라 생략 가능)
groupadd oper
groupadd backupdba
groupadd dgdba
groupadd kmdba
groupadd racdba
groupadd asmadmin
groupadd asmdba
groupadd asmoper
groupadd vboxsf
groupadd oinstall
groupadd grid
groupadd dba
### 유저 그룹 설정 변경
useradd -g oinstall -G dba,racdba,asmadmin,asmdba,asmoper,vboxsf grid
useradd -g oinstall -G dba,racdba,asmadmin,asmdba,asmoper,vboxsf oracle
usermod -g oinstall -G dba,oper,backupdba,dgdba,kmdba,racdba,asmadmin,asmdba,vboxsf oracle
### 유저 암호 변경
passwd grid
passwd oracle
- 디렉토리 설치
mkdir -p /u01/app/12.2.0/grid
mkdir -p /u01/app/oracle/product/12.2.0/db_1
chown -R grid:oinstall /u01
chown -R oracle:oinstall /u01/app/oracle/product
chmod -R 775 /u01
- 커널 파라미터값 설정
[root@rac1 ~]# vi /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
# 아래 항목들 추가
kernel.sem = 1000 32000 1000 1000
kernel.shmmni = 16384
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 9000 65500
fs.file-max = 16777216
#fs.inode-max=67108864
fs.aio-max-nr = 1048576
- 유저 리소스 제한 설정
[root@rac1 ~]# vi /etc/security/limits.conf
# 아래 항목들 추가
oracle soft nproc 2047
oracle hard nproc 16384
oracle soft nofile 1024
oracle hard nofile 65536
oracle soft stack 10240
grid soft nproc 2047
grid hard nproc 16384
grid soft nofile 1024
grid hard nofile 65536
grid soft stack 10240
- grid 유저 .bash_profile 수정
export TMP=/tmp
export TMPDIR=$TMP
export EDITOR=vi
export ORACLE_HOSTNAME=rac1 # 호스트명에 맞게 변경
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.2.0/grid
export DB_HOME=$ORACLE_BASE/product/12.2.0/db_1
export ORACLE_HOME=$GRID_HOME
export ORACLE_SID=+ASM1 # 2번 노드 +ASM2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$GRID_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_KOREA.AL32UTF8
PS1='[$ORACLE_SID:$PWD]> '
- oracle 유저 .bash_profile 수정
export TMP=/tmp
export TMPDIR=$TMP
export EDITOR=vi
export ORACLE_HOSTNAME=rac1 # 호스트명에 맞게 변경
export ORACLE_UNQNAME=racdb
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.2.0/grid
export DB_HOME=$ORACLE_BASE/product/12.2.0/db_1
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=racdb1 # 2번 노드 racdb2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$GRID_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_KOREA.AL32UTF8
PS1='[$ORACLE_SID:$PWD]> '
공유 스토리지
- 공유 스토리지 구성(Host PC의 cmd에서 진행)
cd C:\Program Files\Oracle\VirtualBox
vboxmanage createmedium --filename D:\VM\storage\CRS1.vdi --size 1024 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\CRS2.vdi --size 1024 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\CRS3.vdi --size 1024 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\GIMR1.vdi --size 40960 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\GIMR2.vdi --size 40960 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\GIMR3.vdi --size 40960 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\DATA1.vdi --size 51200 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\DATA2.vdi --size 51200 --format VDI --variant Fixed
vboxmanage createmedium --filename D:\VM\storage\FRA1.vdi --size 51200 --format VDI --variant Fixed
- 가상 디스크 공유타입으로 변경 (Host PC의 cmd에서 진행)
cd C:\Program Files\Oracle\VirtualBox
vboxmanage modifymedium D:\VM\storage\CRS1.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\CRS2.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\CRS3.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\GIMR1.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\GIMR2.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\GIMR3.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\DATA1.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\DATA2.vdi --type shareable
vboxmanage modifymedium D:\VM\storage\FRA1.vdi --type shareable
- 공유 디스크 추가
rac2 노드 생성
- 1번 노드 이미지 복제
cd C:\Program Files\Oracle\VirtualBox
vboxmanage clonemedium D:\VM\rac1\RAC1.vdi D:\VM\RAC2.vdi
- 노드2 vm 만들기
- 노드2 네트워크 설정
[root@rac1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-enp0s3
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp0s3
UUID=649cb03f-d73c-4fac-8f92-81aac74b52af
DEVICE=enp0s3
ONBOOT=yes
IPADDR=192.168.20.242 # RAC2 - Public IP로 수정
PREFIX=24
GATEWAY=192.168.20.254
DNS1=8.8.8.8
[root@rac1 ~]# vi /etc/sysconfig/network-scripts/ifcfg-enp0s8
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=no
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp0s8
UUID=1f288ca1-83c2-452b-a23f-7157b42969be
DEVICE=enp0s8
ONBOOT=yes
IPADDR=10.0.1.102 # RAC2 - Private IP로 수정
PREFIX=24
[root@rac1 ~]# systemctl restart network
[root@rac1 ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:6c:29:61 brd ff:ff:ff:ff:ff:ff
inet 192.168.20.242/24 brd 192.168.20.255 scope global noprefixroute enp0s3
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe6c:2961/64 scope link
valid_lft forever preferred_lft forever
3: enp0s8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:3c:4c:57 brd ff:ff:ff:ff:ff:ff
inet 10.0.1.102/24 brd 10.0.1.255 scope global noprefixroute enp0s8
valid_lft forever preferred_lft forever
inet6 fe80::a00:27ff:fe3c:4c57/64 scope link
valid_lft forever preferred_lft forever
- RAC2 hostname 변경
hostnamectl status
hostnamectl set-hostname rac2
hostnamectl status
- grid 계정 .bash_profile 수정
[+ASM1:/home/grid]> vi .bash_profile
export PATH
export TMP=/tmp
export TMPDIR=$TMP
export EDITOR=vi
export ORACLE_HOSTNAME=rac2 # 호스트명에 맞게 변경
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.2.0/grid
export DB_HOME=$ORACLE_BASE/product/12.2.0/db_1
export ORACLE_HOME=$GRID_HOME
export ORACLE_SID=+ASM2 # 2번 노드 +ASM2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$GRID_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_KOREA.AL32UTF8
PS1='[$ORACLE_SID:$PWD]> '
- Oracle 계정 .bash_profile 수정
[root@rac2 ~]# su - oracle
export PATH
export TMP=/tmp
export TMPDIR=$TMP
export EDITOR=vi
export ORACLE_HOSTNAME=rac2 # 호스트명에 맞게 변경
export ORACLE_UNQNAME=racdb
export ORACLE_BASE=/u01/app/oracle
export GRID_HOME=/u01/app/12.2.0/grid
export DB_HOME=$ORACLE_BASE/product/12.2.0/db_1
export ORACLE_HOME=$DB_HOME
export ORACLE_SID=racdb2 # 2번 노드 racdb2
export ORACLE_TERM=xterm
export BASE_PATH=/usr/sbin:$PATH
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$GRID_HOME/bin:$BASE_PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib
export CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib
export NLS_LANG=AMERICAN_KOREA.AL32UTF8
PS1='[$ORACLE_SID:$PWD]> '