ORA-15032-15063
ORACLE_BASE value has been saved for future startups ORA-15032: not all alterations performed ORA-15017: diskgroup "QDY_FRA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "QDY_FRA" ORA-15017: diskgroup "QDY_DATA" cannot be mounted ORA-15063: ASM discovered an insufficient number of disks for diskgroup "QDY_DATA" ERROR: ALTER DISKGROUP ALL MOUNT
SQL> create pfile='/tmp/init-asm.ora' from spfile'
SQL> !cat /tmp/init-asm.ora
+ASM2.__oracle_base='/oracle/app/grid'#ORACLE_BASE set from in memory value
+ASM1.asm_diskgroups='CRS','DATA','FRA'#Manual Mount
+ASM2.asm_diskgroups='CRS','QDY_DATA','QDY_FRA'#Manual Mount
*.asm_power_limit=1
*.diagnostic_dest='/oracle/app/grid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'
SQL> show parameter spfile;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +CRS/txld-cluster/asmparameter
file/registry.253.1025031469
SQL> shutdown immediate;
ASM diskgroups dismounted
ASM instance shutdown
SQL> startup pfile='/tmp/init-asm.ora';
ASM instance started
Total System Global Area 1135747072 bytes
Fixed Size 2260728 bytes
Variable Size 1108320520 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
SQL> show parameter diskgroup;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string QDY_DATA, QDY_FRA, FRA, DATA,
CRS
SQL> ALTER SYSTEM SET ASM_DISKGROUPS='CRS','DATA','FRA' scope=both;
System altered.
SQL> show parameter diskgroup;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskgroups string CRS, DATA, FRA
asm copy
for i in $(asmcmd ls fra/CARPOLY/ARCHIVELOG/2019_12_29); do asmcmd cp +fra/CARPOLY/ARCHIVELOG/2019_12_29/$i /mnt/backup/cloud1 done
- Records all DISKs WWID(scsi_id)
for DISK in `find /dev/ -name "sd*" | sort`; do WWID=`/sbin/scsi_id --whitelisted --replace-whitespace --device=$DISK`; echo $DISK " WWID is " $WWID; done | sort -k 4 -n
for DISK in `find /dev/ -name "sd*" | sort`; do WWID=`/sbin/scsi_id --whitelisted --replace-whitespace --device=$DISK`; echo $DISK",WWN is," $WWID; done | sort -k 4 -n
moving from external redundancy +OCRVOTE diskgroup to normal redundancy +CRS diskgroup
1. create new diskgroup +CRS as desired 2. $ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 5e391d339a594fc7bf11f726f9375095 (ORCL:ASMDG02) [+OCRVOTE] Located 1 voting disk(s). 3. $ crsctl replace votedisk +CRS Successful addition of voting disk 941236c324454fc0bfe182bd6ebbcbff. Successful addition of voting disk 07d2464674ac4fabbf27f3132d8448b0. Successful addition of voting disk 9761ccf221524f66bff0766ad5721239. Successful deletion of voting disk 5e391d339a594fc7bf11f726f9375095. Successfully replaced voting disk group with +CRS. CRS-4266: Voting file(s) successfully replaced 4. $ crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 941236c324454fc0bfe182bd6ebbcbff (ORCL:CRSD1) [CRS] 2. ONLINE 07d2464674ac4fabbf27f3132d8448b0 (ORCL:CRSD2) [CRS] 3. ONLINE 9761ccf221524f66bff0766ad5721239 (ORCL:CRSD3) [CRS] Located 3 voting disk(s).
For ASMLIB disk
- Add the device mappings to the device mapper disks.
kpartx -a /dev/mapper/DATA05-TEST
Find disk UUID
udevadm info --query=all --name=$DISK | grep -i dm_uuid
Add disk to ASM diskgroup
$ sqlplus / as sysasm
set linesize 300
column path format a50
select GROUP_NUMBER,MOUNT_STATUS,STATE,NAME,FAILGROUP,LABEL,PATH from v$asm_disk;
GROUP_NUMBER MOUNT_S STATE NAME FAILGROUP LABEL PATH
------------ ------- -------- ------------------------------ ------------------------------ ------------------------------- --------------------------------------------------
0 CLOSED NORMAL DATA03 ORCL:DATA03
0 CLOSED NORMAL DATA04 ORCL:DATA04
0 CLOSED NORMAL DATA05 ORCL:DATA05
0 CLOSED NORMAL DATA06 ORCL:DATA06
0 CLOSED NORMAL DATA07 ORCL:DATA07
0 CLOSED NORMAL DATA08 ORCL:DATA08
1 CACHED NORMAL CRS01 CRS01 CRS01 ORCL:CRS01
1 CACHED NORMAL CRS02 CRS02 CRS02 ORCL:CRS02
1 CACHED NORMAL CRS03 CRS03 CRS03 ORCL:CRS03
2 CACHED NORMAL DATA01 DATA01 DATA01 ORCL:DATA01
3 CACHED NORMAL DATA02 DATA02 DATA02 ORCL:DATA02
11 rows selected.
SQL> select
mount_status,
header_status,
mode_status,
state,
total_mb,
free_mb,
name,
path,
label
from
v$asm_disk;
alter diskgroup DATA add disk '/dev/DATA05-TEST' rebalance power 6;
SQL> select GROUP_NUMBER,MOUNT_STATUS,STATE,NAME,FAILGROUP,LABEL,PATH from v$asm_disk;
GROUP_NUMBER MOUNT_S STATE NAME FAILGROUP LABEL PATH
------------ ------- -------- ------------------------------ ------------------------------ ------------------------------- --------------------------------------------------
0 CLOSED NORMAL CRS03 ORCL:CRS03
1 CACHED NORMAL CRS01 CRS01 CRS01 ORCL:CRS01
1 CACHED NORMAL CRS02 CRS02 CRS02 ORCL:CRS02
2 CACHED NORMAL DATA01 DATA01 DATA01 ORCL:DATA01
2 CACHED NORMAL DATA02 DATA02 DATA02 ORCL:DATA02
2 CACHED NORMAL DATA03 DATA03 DATA03 ORCL:DATA03
3 CACHED NORMAL DATA04 DATA04 DATA04 ORCL:DATA04
3 CACHED NORMAL DATA05 DATA05 DATA05 ORCL:DATA05
3 CACHED NORMAL DATA06 DATA06 DATA06 ORCL:DATA06
9 rows selected.
SQL> alter diskgroup CRS add disk 'ORCL:CRS03' rebalance power 6;
Diskgroup altered.
SQL> exit
Disconnected from Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
txld11220<2019-11-26 00:03:36> ~
(reverse-i-search)`i': cd /soft/gr^C/
txld11220<2019-11-26 00:03:40> ~
grid>$ asmcmd lsdg --discovery
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL Y 512 4096 1048576 3069 2143 1023 560 0 Y CRS/
MOUNTED EXTERN N 512 4096 1048576 307197 305459 0 305459 0 N QDY_DATA/
MOUNTED EXTERN N 512 4096 1048576 307197 306597 0 306597 0 N QDY_FRA/- 使用以下命令查看观察:当没有输出时则表明reblance完成。
select * from v$asm_operation;
Drop disk to ASM diskgroup
$ sqlplus / as sysasm SQL> set linesize 300 SQL> column path format a50 SQL> alter diskgroup DATA drop disk 'DATA_0003' rebalance power 6;
- 查看drop disk的状态，直到DROPING状态变更为NORMAL.
SQL> select GROUP_NUMBER,MOUNT_STATUS,STATE,NAME,FAILGROUP,LABEL,PATH from v$asm_disk;
GROUP_NUMBER MOUNT_S STATE NAME FAILGROUP LABEL PATH
------------ ------- -------- ------------------------------ ------------------------------ ------------------------------- ------------------------------
0 CLOSED NORMAL /dev/DATA04-TEST
2 CACHED NORMAL DATA_0000 DATA_0000 /dev/DATA01
1 CACHED NORMAL CRS_0002 CRS_0002 /dev/CRS03
2 CACHED NORMAL DATA_0002 DATA_0002 /dev/DATA03
2 CACHED NORMAL DATA_0001 DATA_0001 /dev/DATA02
1 CACHED NORMAL CRS_0000 CRS_0000 /dev/CRS01
1 CACHED NORMAL CRS_0001 CRS_0001 /dev/CRS02
2 CACHED NORMAL DATA_0004 DATA_0004 /dev/DATA5-NEW- 或者使用以下命令查看观察:当没有输出时则表明reblance完成。
select * from v$asm_operation;
delete disk from VM
txld11215<2019-11-21 16:10:59> ~ root># oracleasm querydisk -p DATA01 Disk "DATA01" is a valid ASM disk /dev/sde1: LABEL="DATA01" TYPE="oracleasm" txld11215<2019-11-21 16:11:01> ~ root># udevadm info --query=all --name=/dev/sde1 | grep PATH E: DEVPATH=/devices/pci0000:00/0000:00:18.0/0000:1b:00.0/host3/target3:0:3/3:0:3:0/block/sde/sde1 E: ID_PATH=pci-0000:1b:00.0-scsi-0:0:3:0 E: ID_PATH_TAG=pci-0000_1b_00_0-scsi-0_0_3_0 * ===> scsi1-3
SQL> select GROUP_NUMBER,MOUNT_STATUS,STATE,NAME,FAILGROUP,LABEL,PATH from v$asm_disk;
GROUP_NUMBER MOUNT_S STATE NAME FAILGROUP LABEL PATH
------------ ------- -------- ------------------------------ ------------------------------ ------------------------------- --------------------------------------------------
1 CACHED NORMAL CRS01 CRS01 CRS01 ORCL:CRS01
1 CACHED NORMAL CRS02 CRS02 CRS02 ORCL:CRS02
1 CACHED NORMAL CRS03 CRS03 CRS03 ORCL:CRS03
2 CACHED NORMAL DATA01 DATA01 DATA01 ORCL:DATA01
3 CACHED NORMAL DATA02 DATA02 DATA02 ORCL:DATA02
delete resource from grid
grid>$ crsctl status resource -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
ora.DATA.dg
OFFLINE OFFLINE txld11215
ONLINE OFFLINE txld11216
ora.LISTENER.lsnr
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
ora.QDY_DATA.dg
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
ora.QDY_FRA.dg
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
ora.asm
ONLINE ONLINE txld11215 Started
ONLINE ONLINE txld11216 Started
ora.gsd
OFFLINE OFFLINE txld11215
OFFLINE OFFLINE txld11216
ora.net1.network
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
ora.ons
ONLINE ONLINE txld11215
ONLINE ONLINE txld11216
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE txld11216
ora.carpoly.db
1 ONLINE ONLINE txld11215 Open
2 ONLINE ONLINE txld11216 Open
ora.cvu
1 ONLINE ONLINE txld11216
ora.oc4j
$ crsctl stop resource ora.DATA.dg
# /oracle/app/11.2.0/grid/bin/crsctl delete resource ora.DATA.dg
Change Ownershipt of ASM disk
1)Stop CRS ( or ASM instance ) from all nodes crsctl stop crs -f 2)Stop oracleasm # /sbin/service oracleasm stop 3)Reconfigure ASMlib with the correct group ( ex. asmdba ) # /usr/sbin/oracleasm configure -i 4)Startup ASMlib # /sbin/service oracleasm start 5)Start crs crsctl start crs https://support.oracle.com/epmos/faces/DocumentDisplay?_adf.ctrl-state=jjye3icdw_302&id=1171674.1