Describe 首頁/work/QH-htrac-VM-replace-disk here.
Contents
数据库存储迁移流程
明确DATABASE-NAME数据库存储迁移需求
将现有的EMC存储替换为PowerStore X1000存储，采用asm rebalance技术进行在线迁移。
前期准备工作
查看ASM磁盘组信息
- login in as grid user
$ asmcmd lsdg
[grid@htrac1 ~]$ asmcmd lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 1048576 624628 183534 0 183534 0 N DATA/ MOUNTED EXTERN N 512 4096 1048576 102398 27343 0 27343 0 N RECO/ MOUNTED NORMAL N 512 4096 1048576 15345 14419 5115 4652 0 Y SYSTEM/
查看ASM 磁盘空间使用情况
- login in as grid user
$ asmcmd lsdsk -k
Total_MB Free_MB OS_MB Name Failgroup Failgroup_Type Library Label UDID Product Redund Path
307196 90267 307196 DATA1 DATA1 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) DATA1 UNKNOWN ORCL:DATA1
112635 33092 112635 DATA2 DATA2 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) DATA2 UNKNOWN ORCL:DATA2
204797 60175 204797 DATA3 DATA3 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) DATA3 UNKNOWN ORCL:DATA3
5115 4807 5115 OCR1 OCR1 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) OCR1 UNKNOWN ORCL:OCR1
5115 4805 5115 OCR2 OCR2 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) OCR2 UNKNOWN ORCL:OCR2
5115 4807 5115 OCR3 OCR3 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) OCR3 UNKNOWN ORCL:OCR3
102398 27343 102398 RECO1 RECO1 REGULAR ASM Library - Generic Linux, version 2.0.4 (KABI_V2) RECO1 UNKNOWN ORCL:RECO1$ sqlplus / as sysasm
SQL> set linesize 300 column path format a50 column NAME format a15; select GROUP_NUMBER,DISK_NUMBER,HEADER_STATUS,MODE_STATUS,MOUNT_STATUS,STATE,total_mb,free_mb,NAME,PATH from v$asm_disk; GROUP_NUMBER DISK_NUMBER HEADER_STATUS MODE_STATUS MOUNT_STATUS TOTAL_MB FREE_MB NAME PATH ------------ ----------- ------------------------ -------------- -------------- ---------- ---------- ---------------- -------------------- 2 0 MEMBER ONLINE CACHED 30719690267 DATA1 ORCL:DATA1 1 0 MEMBER ONLINE CACHED 5115 4807 OCR1 ORCL:OCR1 1 1 MEMBER ONLINE CACHED 5115 4805 OCR2 ORCL:OCR2 1 2 MEMBER ONLINE CACHED 5115 4807 OCR3 ORCL:OCR3 3 0 MEMBER ONLINE CACHED 10239827343 RECO1 ORCL:RECO1 2 1 MEMBER ONLINE CACHED 11263533092 DATA2 ORCL:DATA2 2 2 MEMBER ONLINE CACHED 20479760175 DATA3 ORCL:DATA3 7 rows selected.
检查多路径状态
multipath –ll
DATABASE-NAME数据库存储迁移步骤
查看磁盘的分区和挂载情况
lsblk
确认新挂载的磁盘状态及链路正常
multipath -ll | grep -A 7 'X1000'
X1000-DATA02 (3600a098000f63f48000001c75c7db889) dm-29 NETAPP,INF-01-00 size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 7:0:9:3 sdy 65:128 active ready running | `- 8:0:8:3 sdah 66:16 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 7:0:8:3 sdp 8:240 active ready running `- 8:0:9:3 sdap 66:144 active ready running
配置多路径
- /etc/multipath.conf
# 2023-02-25 added For X1000
multipath {
alias X1000-DATA01
wwid 3600a098000e3c7e6000002635c7db855
}- /etc/multipath/bindings
X1000-DATA01 3600A098000E3C7E6000002635C7DB855
多路径配置生效
service multipathd reload
udev 磁盘绑定
cat /etc/udev/rules.d/99-oracle-asmdevices.rules
# 2023-02-25 added for X1000
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000f63f48000001cb5c7db88e",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/X1000-DATA06p1"
udev 规则生效
优先采用使用Redhat udev 规则生效推荐的方法操作
udevadm control --reload-rules udevadm trigger --type=devices --action=change ... .. .
将磁盘X1000-DATA01p1、X1000-DATA02p1、X1000-DATA03p1、X1000-DATA04p1加入到 DATAVG
确认asm能够识别到udev规则生效后的磁盘
- login in as grid user
$ sqlplus / as sysasm
SQL> select GROUP_NUMBER,DISK_NUMBER,HEADER_STATUS,MODE_STATUS,MOUNT_STATUS,STATE,total_mb,free_mb,NAME,PATH from v$asm_disk;
首先在DATAVG中加入X1000存储的DATA LUN
alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA01p1' rebalance power 8;
alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA02p1' rebalance power 12;
SQL> select * from v$asm_operation;
GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
2 REBAL RUN 11 11 11870 851620 37158 22alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA03p1' rebalance power 12;
alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA04p1' rebalance power 12;
SQL> alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA05p1','/dev/oracleasm/disks/X1000-DATA06p1' rebalance power 12; Diskgroup altered. $ tail -f /u01/app/grid/diag/asm/+asm/+ASM/trace/alert.log SQL> alter diskgroup DATAVG add disk '/dev/oracleasm/disks/X1000-DATA05p1','/dev/oracleasm/disks/X1000-DATA06p1' rebalance power 12
备注：尽量在业务不繁忙的情况下操作;
然后在REDOVG中加入X1000存储的REDO LUN
alter diskgroup REDOVG add disk '/dev/oracleasm/disks/X1000-REDO01p1' rebalance power 8;
SQL> select * from v$asm_operation;
GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
2 REBAL RUN 11 11 1082386 1082386 0 0
3 REBAL WAIT 8
$ tail -f alert_+ASM.logalter diskgroup REDOVG add disk '/dev/oracleasm/disks/X1000-REDO02p1' rebalance power 5;
查看rebalance情况与日志对比
- login in as grid user
使用以下命令查看观察:当没有输出时则表明reblance完成。
select * from v$asm_operation;
* 以下为加第一块时的输出，看EST_MINUTES大约需要29分钟。
SQL> select * from v$asm_operation;
GROUP_NUMBER OPERA STAT POWER ACTUAL SOFAR EST_WORK EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
2 REBAL RUN 8 8 33691 830701 26839 29$ asmcmd lsdsk -k
[grid@MCPOS-HOST-01 ~]$ asmcmd lsdsk -k
$ asmcmd -p lsop Group_Name Dsk_Num State Power EST_WORK EST_RATE EST_TIME DATAVG REBAL RUN 8 830701 0 0 $ tail -f alert.log
备注：确认所有磁盘rebalance完成后，再进行下一步删盘操作； 尽量在业务不繁忙的情况下操作;
逐次剔除旧存储AFF8040对应的asm disk
DROP AFF8040 REDOVG MEMBERs
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup REDOVG drop disk REDOVG_0002 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup REDOVG drop disk REDOVG_0003 rebalance power 11; EOF
DROP AFF8040 DATAVG MEMBERs
RETVAL=`sqlplus -s '/ as sysasm' << "EOF" select * from v$asm_operation; EXIT; EOF` if [ -z "$RETVAL" ]; then echo "No rows returned from database" exit 0 else echo $RETVAL echo "DROP FAILED" fi
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0003 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0004 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0005 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0006 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0007 rebalance power 11; EOF
sqlplus -s '/ as sysasm' << "EOF" alter diskgroup DATAVG drop disk DATAVG_0008 rebalance power 11; EOF
- select * from v$asm_operation;
查看rebalance情况与asm日志对比
$ asmcmd -p lsop $ tail -f alert.log
删除ARCHVG磁盘组
sqlplus / as sysasm SQL> alter diskgroup ARCHVG dismount; SQL> drop diskgroup ARCHVG force including contents;