Carpoly

mcposdb数据库存储迁移流程

明确mcposdb数据库存储迁移需求

将现有的AFF8040存储替换为EF570存储，采用asm rebalance技术进行在线迁移。

前期准备工作

查看ASM磁盘组信息

$ asmcmd lsdg

State    Type    Rebal  Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512   4096  1048576   5812280   665666                0          665666              0             N  DATAVG/
MOUNTED  EXTERN  N         512   4096  1048576    204796   161066                0          161066              0             N  REDOVG/

查看ASM 磁盘空间使用情况

$ asmcmd lsdsk -k

Total_MB  Free_MB    OS_MB  Name         Failgroup    Failgroup_Type  Library  Label  UDID  Product  Redund   Path
 1048586   120095  1048586  DATAVG_0003  DATAVG_0003  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA01p1
 1048586   120095  1048586  DATAVG_0004  DATAVG_0004  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA02p1
 1048586   120094  1048586  DATAVG_0005  DATAVG_0005  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA03p1
 1048586   120092  1048586  DATAVG_0006  DATAVG_0006  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA04p1
  808968    92644   808968  DATAVG_0007  DATAVG_0007  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA05p1
  808968    92646   808968  DATAVG_0008  DATAVG_0008  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA06p1
  102398    80534   102398  REDOVG_0002  REDOVG_0002  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFREDO01p1
  102398    80532   102398  REDOVG_0003  REDOVG_0003  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFREDO02p1

$ sqlplus / as sysasm

SQL> set linesize 300
column path format a50
column NAME format a15;
select GROUP_NUMBER,DISK_NUMBER,HEADER_STATUS,MODE_STATUS,MOUNT_STATUS,STATE,total_mb,free_mb,NAME,PATH from v$asm_disk;

GROUP_NUMBER DISK_NUMBER HEADER_STATU MODE_ST MOUNT_S STATE      TOTAL_MB    FREE_MB NAME            PATH
------------ ----------- ------------ ------- ------- -------- ---------- ---------- --------------- ----------------------------------------
           0           0 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                 /dev/oracleasm/disks/EF570-REDO01p1
           0           1 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                 /dev/oracleasm/disks/EF570-DATA05p1
           0           2 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                 /dev/oracleasm/disks/EF570-DATA03p1
           0           3 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                 /dev/oracleasm/disks/EF570-DATA01p1
           3           2 MEMBER       ONLINE  CACHED  NORMAL       102398      80534 REDOVG_0002     /dev/oracleasm/disks/AFFREDO01p1
           2           4 MEMBER       ONLINE  CACHED  NORMAL      1048586     120095 DATAVG_0004     /dev/oracleasm/disks/AFFDATA02p1
           2           8 MEMBER       ONLINE  CACHED  NORMAL       808968      92646 DATAVG_0008     /dev/oracleasm/disks/AFFDATA06p1
           2           6 MEMBER       ONLINE  CACHED  NORMAL      1048586     120092 DATAVG_0006     /dev/oracleasm/disks/AFFDATA04p1
           2           7 MEMBER       ONLINE  CACHED  NORMAL       808968      92644 DATAVG_0007     /dev/oracleasm/disks/AFFDATA05p1
           3           3 MEMBER       ONLINE  CACHED  NORMAL       102398      80532 REDOVG_0003     /dev/oracleasm/disks/AFFREDO02p1
           2           5 MEMBER       ONLINE  CACHED  NORMAL      1048586     120094 DATAVG_0005     /dev/oracleasm/disks/AFFDATA03p1

GROUP_NUMBER DISK_NUMBER HEADER_STATU MODE_ST MOUNT_S STATE      TOTAL_MB    FREE_MB NAME            PATH
------------ ----------- ------------ ------- ------- -------- ---------- ---------- --------------- ----------------------------------------
           2           3 MEMBER       ONLINE  CACHED  NORMAL      1048586     120095 DATAVG_0003     /dev/oracleasm/disks/AFFDATA01p1
12 rows selected.

检查多路径状态

multipath –ll

mcposdb数据库存储迁移步骤

查看磁盘的分区和挂载情况

lsblk lsblk -o +fstype,label,uuid,partuuid

确认新挂载的磁盘状态及链路正常

multipath -ll | grep -A 7 'EF570'

EF570-DATA02 (3600a098000f63f48000001c75c7db889) dm-29 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:9:3  sdy  65:128  active ready running
| `- 8:0:8:3  sdah 66:16   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:8:3  sdp  8:240   active ready running
  `- 8:0:9:3  sdap 66:144  active ready running
--
EF570-REDO02 (3600a098000f63f48000001cd5c7db891) dm-33 NETAPP,INF-01-00
size=100G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:9:9  sdad 65:208  active ready running
| `- 8:0:8:9  sdam 66:96   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:8:9  sdv  65:80   active ready running
  `- 8:0:9:9  sdav 66:240  active ready running
EF570-DATA01 (3600a098000e3c7e6000002635c7db855) dm-12 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:8:1  sdn  8:208   active ready running
| `- 8:0:9:1  sdan 66:112  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:9:1  sdw  65:96   active ready running
  `- 8:0:8:1  sdae 65:224  active ready running
--
EF570-REDO01 (3600a098000e3c7e60000026a5c7db868) dm-25 NETAPP,INF-01-00
size=100G features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:8:8  sdu  65:64   active ready running
| `- 8:0:9:8  sdat 66:208  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:9:8  sdac 65:192  active ready running
  `- 8:0:8:8  sdal 66:80   active ready running
--
EF570-DATA06 (3600a098000f63f48000001cb5c7db88e) dm-32 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:9:6  sdab 65:176  active ready running
| `- 8:0:8:6  sdak 66:64   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:8:6  sdt  65:48   active ready running
  `- 8:0:9:6  sdas 66:192  active ready running
--
EF570-DATA05 (3600a098000e3c7e6000002685c7db865) dm-18 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:8:5  sds  65:32   active ready running
| `- 8:0:9:5  sdar 66:176  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:9:5  sdaa 65:160  active ready running
  `- 8:0:8:5  sdaj 66:48   active ready running
EF570-DATA04 (3600a098000f63f48000001c95c7db88b) dm-31 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:9:4  sdz  65:144  active ready running
| `- 8:0:8:4  sdai 66:32   active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:8:4  sdr  65:16   active ready running
  `- 8:0:9:4  sdaq 66:160  active ready running
--
EF570-DATA03 (3600a098000e3c7e6000002665c7db862) dm-16 NETAPP,INF-01-00
size=1.0T features='4 queue_if_no_path pg_init_retries 50 retain_attached_hw_handle' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 7:0:8:2  sdo  8:224   active ready running
| `- 8:0:9:2  sdao 66:128  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 7:0:9:2  sdx  65:112  active ready running
  `- 8:0:8:2  sdag 66:0    active ready running

配置多路径

/etc/multipath.conf

# 20190305 added For EF570
multipath {
        alias EF570-DATA01
        wwid 3600a098000e3c7e6000002635c7db855
}
multipath {
        alias EF570-DATA02
        wwid 3600a098000f63f48000001c75c7db889
}
multipath {
        alias EF570-DATA03
        wwid 3600a098000e3c7e6000002665c7db862
}
multipath {
        alias EF570-DATA04
        wwid 3600a098000f63f48000001c95c7db88b
}
multipath {
        alias EF570-DATA05
        wwid 3600a098000e3c7e6000002685c7db865
}
multipath {
        alias EF570-DATA06
        wwid 3600a098000f63f48000001cb5c7db88e
}
multipath {
        alias EF570-REDO01
        wwid 3600a098000e3c7e60000026a5c7db868
}
multipath {
        alias EF570-REDO02
        wwid 3600a098000f63f48000001cd5c7db891
}

/etc/multipath/bindings

EF570-DATA01 3600A098000E3C7E6000002635C7DB855
EF570-DATA02 3600A098000F63F48000001C75C7DB889
EF570-DATA03 3600A098000E3C7E6000002665C7DB862
EF570-DATA04 3600A098000F63F48000001C95C7DB88B
EF570-DATA05 3600A098000E3C7E6000002685C7DB865
EF570-DATA06 3600A098000F63F48000001CB5C7DB88E
EF570-REDO01 3600A098000E3C7E60000026A5C7DB868
EF570-REDO02 3600A098000F63F48000001CD5C7DB891

多路径配置生效

service multipathd reload

udev 磁盘绑定

for DISK in find /dev/ -name "sd?1" ; do echo udevadm info --query=all --name=$DISK | grep -i "DEVLINKS=" | awk -F"=" '{ print "ACTION==\"add|change\" ENV{DEVLINKS}==\""$2"\",OWNER=\"grid\",GROUP=\"asmadmin\",MODE=\"0660\"" }',SYMLINK+=\"oracleasm/\"; done

cat /etc/udev/rules.d/99-oracle-asmdevices.rules

# 20190305 added for EF570
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000f63f48000001cb5c7db88e",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA06p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000e3c7e6000002685c7db865",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA05p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000f63f48000001c95c7db88b",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA04p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000e3c7e6000002665c7db862",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA03p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000f63f48000001c75c7db889",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA02p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000e3c7e6000002635c7db855",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-DATA01p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000f63f48000001cd5c7db891",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-REDO02p1"
KERNEL=="dm-*",ENV{DM_UUID}=="part1-mpath-3600a098000e3c7e60000026a5c7db868",OWNER="grid",GROUP="asmadmin",MODE="0660",NAME+="oracleasm/disks/EF570-REDO01p1"

udev 规则生效

优先采用使用Redhat udev 规则生效推荐的方法操作

udevadm control --reload-rules
udevadm trigger --type=devices --action=change

Mar 11 11:00:30 MCPOS-HOST-01 kernel: [22546324.993657] oracle: sending ioctl 2285 to a partition!
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062904] scsi_io_completion: 46 callbacks suppressed
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062924] sd 7:0:6:2: [sdek] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062936] sd 7:0:6:2: [sdek] tag#0 Sense Key : Illegal Request [current]
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062941] sd 7:0:6:2: [sdek] tag#0 Add. Sense: Logical unit not supported
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062946] sd 7:0:6:2: [sdek] tag#0 CDB: Read(10) 28 00 6f 41 3f 80 00 00 08 00
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062948] blk_update_request: 46 callbacks suppressed
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.062951] blk_update_request: I/O error, dev sdek, sector 1866547072
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063353] sd 7:0:6:2: [sdek] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063359] sd 7:0:6:2: [sdek] tag#0 Sense Key : Illegal Request [current]
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063362] sd 7:0:6:2: [sdek] tag#0 Add. Sense: Logical unit not supported
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063366] sd 7:0:6:2: [sdek] tag#0 CDB: Read(10) 28 00 6f 41 3f 80 00 00 08 00
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063368] blk_update_request: I/O error, dev sdek, sector 1866547072
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063606] buffer_io_error: 30 callbacks suppressed
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063609] Buffer I/O error on dev sdek, logical block 233318384, async page read
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063885] sd 7:0:6:5: [sden] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063889] sd 7:0:6:5: [sden] tag#0 Sense Key : Illegal Request [current]
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063893] sd 7:0:6:5: [sden] tag#0 Add. Sense: Logical unit not supported
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063897] sd 7:0:6:5: [sden] tag#0 CDB: Read(10) 28 00 6f 41 3f 80 00 00 08 00
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.063900] blk_update_request: I/O error, dev sden, sector 1866547072
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.064125] sd 7:0:6:5: [sden] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Mar 11 11:10:22 MCPOS-HOST-01 kernel: [22546917.064128] sd 7:0:6:5: [sden] tag#0 Sense Key : Illegal Request [current]
...
..
.

将磁盘EF570-DATA01p1、EF570-DATA02p1、EF570-DATA03p1、EF570-DATA04p1加入到 DATAVG

确认asm能够识别到udev规则生效后的磁盘

$ sqlplus / as sysasm

SQL> select GROUP_NUMBER,DISK_NUMBER,HEADER_STATUS,MODE_STATUS,MOUNT_STATUS,STATE,total_mb,free_mb,NAME,PATH from v$asm_disk;

GROUP_NUMBER DISK_NUMBER HEADER_STATU MODE_ST MOUNT_S STATE      TOTAL_MB    FREE_MB NAME                                PATH
------------ ----------- ------------ ------- ------- -------- ---------- ---------- ----------------------------------- ----------------------------------------
           0           0 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-REDO01p1
           0           1 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA05p1
           0           7 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-REDO02p1
           0           6 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA04p1
           0           5 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA02p1
           0           4 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA06p1
           0           3 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA01p1
           0           2 CANDIDATE    ONLINE  CLOSED  NORMAL            0          0                                     /dev/oracleasm/disks/EF570-DATA03p1
           3           2 MEMBER       ONLINE  CACHED  NORMAL       102398      80534 REDOVG_0002                         /dev/oracleasm/disks/AFFREDO01p1
           2           4 MEMBER       ONLINE  CACHED  NORMAL      1048586     120095 DATAVG_0004                         /dev/oracleasm/disks/AFFDATA02p1
           2           8 MEMBER       ONLINE  CACHED  NORMAL       808968      92646 DATAVG_0008                         /dev/oracleasm/disks/AFFDATA06p1

GROUP_NUMBER DISK_NUMBER HEADER_STATU MODE_ST MOUNT_S STATE      TOTAL_MB    FREE_MB NAME                                PATH
------------ ----------- ------------ ------- ------- -------- ---------- ---------- ----------------------------------- ----------------------------------------
           2           6 MEMBER       ONLINE  CACHED  NORMAL      1048586     120092 DATAVG_0006                         /dev/oracleasm/disks/AFFDATA04p1
           2           7 MEMBER       ONLINE  CACHED  NORMAL       808968      92644 DATAVG_0007                         /dev/oracleasm/disks/AFFDATA05p1
           3           3 MEMBER       ONLINE  CACHED  NORMAL       102398      80532 REDOVG_0003                         /dev/oracleasm/disks/AFFREDO02p1
           2           5 MEMBER       ONLINE  CACHED  NORMAL      1048586     120094 DATAVG_0005                         /dev/oracleasm/disks/AFFDATA03p1
           2           3 MEMBER       ONLINE  CACHED  NORMAL      1048586     120095 DATAVG_0003                         /dev/oracleasm/disks/AFFDATA01p1

16 rows selected.

首先在DATAVG中加入EF570存储的DATA LUN

alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA01p1' rebalance power 8;

alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA02p1' rebalance power 12;
SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
           2 REBAL RUN          11         11      11870     851620      37158          22

alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA03p1' rebalance power 12;

alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA04p1' rebalance power 12;

SQL> alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA05p1','/dev/oracleasm/disks/EF570-DATA06p1' rebalance power 12;
Diskgroup altered.

$ tail -f /u01/app/grid/diag/asm/+asm/+ASM/trace/alert.log
SQL> alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA05p1','/dev/oracleasm/disks/EF570-DATA06p1' rebalance power 12
Mon Mar 11 13:44:17 2019
NOTE: stopping process ARB0
NOTE: rebalance interrupted for group 2/0x1ee96165 (DATAVG)
NOTE: Assigning number (2,10) to disk (/dev/oracleasm/disks/EF570-DATA05p1)
NOTE: Assigning number (2,11) to disk (/dev/oracleasm/disks/EF570-DATA06p1)
NOTE: requesting all-instance membership refresh for group=2
NOTE: initializing header on grp 2 disk DATAVG_0010
NOTE: initializing header on grp 2 disk DATAVG_0011
NOTE: requesting all-instance disk validation for group=2
NOTE: skipping rediscovery for group 2/0x1ee96165 (DATAVG) on local instance.
NOTE: requesting all-instance disk validation for group=2
NOTE: skipping rediscovery for group 2/0x1ee96165 (DATAVG) on local instance.
NOTE: initiating PST update: grp = 2
Mon Mar 11 13:44:21 2019
GMON updating group 2 at 104 for pid 27, osid 87060
NOTE: PST update grp = 2 completed successfully
WARNING: diskgroup compatibility limits power to 11
NOTE: membership refresh pending for group 2/0x1ee96165 (DATAVG)
GMON querying group 2 at 105 for pid 13, osid 15904
NOTE: cache opening disk 10 of grp 2: DATAVG_0010 path:/dev/oracleasm/disks/EF570-DATA05p1
NOTE: cache opening disk 11 of grp 2: DATAVG_0011 path:/dev/oracleasm/disks/EF570-DATA06p1
GMON querying group 2 at 106 for pid 13, osid 15904
SUCCESS: refreshed membership for 2/0x1ee96165 (DATAVG)
NOTE: starting rebalance of group 2/0x1ee96165 (DATAVG) at power 11
SUCCESS: alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA05p1','/dev/oracleasm/disks/EF570-DATA06p1' rebalance power 12
Starting background process ARB0
Mon Mar 11 13:44:24 2019
ARB0 started with pid=33, OS id=39931
NOTE: assigning ARB0 to group 2/0x1ee96165 (DATAVG) with 11 parallel I/Os
Mon Mar 11 13:44:27 2019
NOTE: Attempting voting file refresh on diskgroup DATAVG
cellip.ora not found.

WARNING的原因是COMPATIBLE.ASM参数为11.2.0.0。

https://docs.oracle.com/cd/E18283_01/server.112/e17110/initparams012.htm#

备注：尽量在业务不繁忙的情况下操作;

然后在REDOVG中加入EF570存储的REDO LUN

alter diskgroup REDOVG add disk '/dev/oracleasm/disks/EF570-REDO01p1' rebalance power 8;

SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
           2 REBAL RUN          11         11    1082386    1082386          0           0
           3 REBAL WAIT          8

$ tail -f alert_+ASM.log
Mon Mar 11 14:28:09 2019
GMON updating group 3 at 108 for pid 27, osid 87060
NOTE: PST update grp = 3 completed successfully
NOTE: membership refresh pending for group 3/0x1ee96166 (REDOVG)
GMON querying group 3 at 109 for pid 13, osid 15904
NOTE: cache opening disk 0 of grp 3: REDOVG_0000 path:/dev/oracleasm/disks/EF570-REDO01p1
GMON querying group 3 at 110 for pid 13, osid 15904
SUCCESS: refreshed membership for 3/0x1ee96166 (REDOVG)
SUCCESS: alter diskgroup REDOVG add disk '/dev/oracleasm/disks/EF570-REDO01p1' rebalance power 8
NOTE: Attempting voting file refresh on diskgroup REDOVG


Mon Mar 11 14:32:23 2019
NOTE: stopping process ARB0
SUCCESS: rebalance completed for group 2/0x1ee96165 (DATAVG)
NOTE: ASM did background COD recovery for group 3/0x1ee96166 (REDOVG)
NOTE: starting rebalance of group 3/0x1ee96166 (REDOVG) at power 8
Starting background process ARB0
Mon Mar 11 14:32:24 2019
ARB0 started with pid=33, OS id=112768
NOTE: assigning ARB0 to group 3/0x1ee96166 (REDOVG) with 8 parallel I/Os
cellip.ora not found.
Mon Mar 11 14:32:57 2019
NOTE: requesting all-instance membership refresh for group=3
Mon Mar 11 14:33:00 2019
NOTE: membership refresh pending for group 3/0x1ee96166 (REDOVG)
Mon Mar 11 14:33:03 2019
GMON querying group 3 at 111 for pid 13, osid 15904
SUCCESS: refreshed membership for 3/0x1ee96166 (REDOVG)
NOTE: Attempting voting file refresh on diskgroup REDOVG
Mon Mar 11 14:33:18 2019
NOTE: stopping process ARB0
SUCCESS: rebalance completed for group 3/0x1ee96166 (REDOVG)

alter diskgroup REDOVG add disk '/dev/oracleasm/disks/EF570-REDO02p1' rebalance power 5;

查看rebalance情况与日志对比

使用以下命令查看观察:当没有输出时则表明reblance完成。

select * from v$asm_operation;
  * 以下为加第一块时的输出，看EST_MINUTES大约需要29分钟。
SQL> select * from v$asm_operation;

GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
           2 REBAL RUN           8          8      33691     830701      26839          29

$ asmcmd lsdsk -k

[grid@MCPOS-HOST-01 ~]$ asmcmd lsdsk -k
Total_MB  Free_MB    OS_MB  Name         Failgroup    Failgroup_Type  Library  Label  UDID  Product  Redund   Path
 1048586   602676  1048586  DATAVG_0003  DATAVG_0003  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA01p1
 1048586   602675  1048586  DATAVG_0004  DATAVG_0004  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA02p1
 1048586   602675  1048586  DATAVG_0005  DATAVG_0005  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA03p1
 1048586   602676  1048586  DATAVG_0006  DATAVG_0006  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA04p1
  808968   464955   808968  DATAVG_0007  DATAVG_0007  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA05p1
  808968   464955   808968  DATAVG_0008  DATAVG_0008  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFDATA06p1
  102398    91463   102398  REDOVG_0002  REDOVG_0002  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFREDO01p1
  102398    91463   102398  REDOVG_0003  REDOVG_0003  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/AFFREDO02p1
 1048574   602671  1048574  DATAVG_0000  DATAVG_0000  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA01p1
 1048574   602672  1048574  DATAVG_0001  DATAVG_0001  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA02p1
 1048574   602672  1048574  DATAVG_0002  DATAVG_0002  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA03p1
 1048574   602668  1048574  DATAVG_0009  DATAVG_0009  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA04p1
 1048574   602874  1048574  DATAVG_0010  DATAVG_0010  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA05p1
 1048574   602875  1048574  DATAVG_0011  DATAVG_0011  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-DATA06p1
  102398    91464   102398  REDOVG_0000  REDOVG_0000  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-REDO01p1
  102398    91468   102398  REDOVG_0001  REDOVG_0001  REGULAR         System                         UNKNOWN  /dev/oracleasm/disks/EF570-REDO02p1

$ asmcmd -p lsop
Group_Name  Dsk_Num  State  Power  EST_WORK  EST_RATE  EST_TIME
DATAVG      REBAL    RUN    8      830701    0         0

$ tail -f alert.log
SQL> alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA01p1' rebalance power 8 
NOTE: Assigning number (2,0) to disk (/dev/oracleasm/disks/EF570-DATA01p1)
NOTE: requesting all-instance membership refresh for group=2
NOTE: initializing header on grp 2 disk DATAVG_0000
NOTE: requesting all-instance disk validation for group=2
Mon Mar 11 11:42:05 2019
NOTE: skipping rediscovery for group 2/0x1ee96165 (DATAVG) on local instance.
NOTE: requesting all-instance disk validation for group=2
NOTE: skipping rediscovery for group 2/0x1ee96165 (DATAVG) on local instance.
NOTE: initiating PST update: grp = 2
Mon Mar 11 11:42:05 2019
GMON updating group 2 at 88 for pid 27, osid 87060
NOTE: PST update grp = 2 completed successfully 
NOTE: membership refresh pending for group 2/0x1ee96165 (DATAVG)
GMON querying group 2 at 89 for pid 13, osid 15904
NOTE: cache opening disk 0 of grp 2: DATAVG_0000 path:/dev/oracleasm/disks/EF570-DATA01p1
GMON querying group 2 at 90 for pid 13, osid 15904
SUCCESS: refreshed membership for 2/0x1ee96165 (DATAVG)
NOTE: starting rebalance of group 2/0x1ee96165 (DATAVG) at power 8
SUCCESS: alter diskgroup DATAVG add disk '/dev/oracleasm/disks/EF570-DATA01p1' rebalance power 8
Starting background process ARB0
Mon Mar 11 11:42:08 2019
ARB0 started with pid=36, OS id=8405 
NOTE: assigning ARB0 to group 2/0x1ee96165 (DATAVG) with 8 parallel I/Os
NOTE: Attempting voting file refresh on diskgroup DATAVG
cellip.ora not found.
Mon Mar 11 12:12:08 2019
NOTE: requesting all-instance membership refresh for group=2
Mon Mar 11 12:12:11 2019
NOTE: membership refresh pending for group 2/0x1ee96165 (DATAVG)
Mon Mar 11 12:12:14 2019
GMON querying group 2 at 91 for pid 13, osid 15904
SUCCESS: refreshed membership for 2/0x1ee96165 (DATAVG)
NOTE: Attempting voting file refresh on diskgroup DATAVG
[grid@MCPOS-HOST-01 trace]$ tail -f alert_+ASM.log 
NOTE: Attempting voting file refresh on diskgroup DATAVG
cellip.ora not found.
Mon Mar 11 12:12:08 2019
NOTE: requesting all-instance membership refresh for group=2
Mon Mar 11 12:12:11 2019
NOTE: membership refresh pending for group 2/0x1ee96165 (DATAVG)
Mon Mar 11 12:12:14 2019
GMON querying group 2 at 91 for pid 13, osid 15904
SUCCESS: refreshed membership for 2/0x1ee96165 (DATAVG)
NOTE: Attempting voting file refresh on diskgroup DATAVG

备注：确认所有磁盘rebalance完成后，再进行下一步删盘操作；尽量在业务不繁忙的情况下操作;

逐次剔除旧存储AFF8040对应的asm disk

DROP AFF8040 REDOVG MEMBERs

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup REDOVG drop disk REDOVG_0002 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup REDOVG drop disk REDOVG_0003 rebalance power 11;
EOF

DROP AFF8040 DATAVG MEMBERs

RETVAL=`sqlplus -s '/ as sysasm' << "EOF"
select * from v$asm_operation;
EXIT;
EOF`

if [ -z "$RETVAL" ]; then
  echo "No rows returned from database"
  exit 0
else
  echo $RETVAL
  echo "DROP FAILED"
fi

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0003 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0004 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0005 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0006 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0007 rebalance power 11;
EOF

sqlplus -s '/ as sysasm' << "EOF"
alter diskgroup DATAVG drop disk DATAVG_0008 rebalance power 11;
EOF

select * from v$asm_operation;

查看rebalance情况与asm日志对比

$ asmcmd -p lsop
$ tail -f alert.log

删除ARCHVG磁盘组

sqlplus / as sysasm
SQL> alter diskgroup ARCHVG dismount;
SQL> drop diskgroup ARCHVG force including contents;

当主库发生故障时切换到备库，采用DataGuard 主备切换方式

Pre-Switchover Checks

The following query at the standby verifies that managed recovery is running:

SQL> SELECT PROCESS FROM V$MANAGED_STANDBY WHERE PROCESS LIKE 'MRP%';

The following query at the Primary verifies that recovery is running with “REAL TIME APPLY” option.

SQL> SELECT RECOVERY_MODE FROM V$ARCHIVE_DEST_STATUS WHERE DEST_ID=2;

RECOVERY_MODE
-----------------------
MANAGED REAL TIME APPLY

If managed standby recovery is not running or not started with real-time apply, restart managed recovery with real-time apply enabled:

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;

Verify Primary and Standby tempfiles match and all datafiles are ONLINE

SQL> SELECT TMP.NAME FILENAME, BYTES, TS.NAME TABLESPACE FROM V$TEMPFILE TMP, V$TABLESPACE TS WHERE TMP.TS#=TS.TS#;

On the target standby:

SQL> SELECT NAME FROM V$DATAFILE WHERE STATUS=OFFLINE';
no rows selected

If there are any OFFLINE datafiles, and these are needed after switchover, bring them ONLINE:

SQL> ALTER DATABASE DATAFILE ‘datafile-name’ ONLINE;

switch over切换（先主库端）

Turn on Data Guard tracing on primary and standby

  * Capture the current value on both the primary and the target physical standby databases
SQL> SHOW PARAMETER log_archive_trace

  * Set Data Guard trace level to 8191 on both the primary and the target physical standby databases
SQL> ALTER SYSTEM SET log_archive_trace=8191;

SQL> SHOW PARAMETER background_dump_dest

  * Tail the alert logs.
> tail –f <background_dump_dest location>/alert*

主库端操作

select database_role,switchover_status from v$database; //查看数据角色及切换状态
select protection_mode,protection_level from v$database; //确认保护模式和级别

alter database commit to switchover to physical standby with session shutdown; //切换为备库

startup mount; //开启数据库
alter database recover managed standby database using current logfile disconnect from session; //开启日志应用

备库端操作（后备库端）

Verify that the standby database can be switched to the primary role

select database_role,switchover_status from v$database; //查看角色及状态

alter database commit to switchover to primary with session shutdown;  //切换为主库
alter database open;

désert/workarea/jbl/Replace-ERP-ASM-disks (last edited 2020-08-16 09:49:22 by merlyn)