Contents
磁盘规划。
SERVICE NUMBER: 1462279 Huawei_S5600T 存储，共4个盘柜，每个盘柜24个硬盘。 每个硬盘容量为2TB。裸容量为96*2=192TB，實際顯示為179TB。
計劃一
每个盘柜创建二组RAID6+一个热备盘： 當硬盤數為12塊時，RAID6实际可用18.3TB左右。 當硬盤數為11塊時，RAID6实际可用16.5TB左右。
計劃二
盤櫃1為三個raid1+0，0-3為一個(3679GB)，4-11為一個(7450GB)， 11-23為一個(11175GB,16個30GB,3個100G,4個200G,一個500GB，一個3T,剩下6T單獨）。 該盤櫃沒有熱備盤。 餘下的三個盤櫃全部做raid6，每個raid抽取這三個盤櫃的前4個硬盤構建raid6， 其結果為5個12塊硬盤的raid6(18625GB,3個6T的盤)，最後的那個raid6為9塊硬盤(13037GB)。 每個盤櫃的最後一個盤為熱備盤。 實際可用容量為： 18625*5+3679+7450+11175+13037 = 128466 GB
實際結果
盤櫃1為三個raid1+0，0-3為一個(3679GB,2T+1.6T)，4-11為一個(7450GB,4T+3.5T)，11-23為一個(11175GB,16個30GB,3個100G,4個200G,一個500GB，一個3T,剩下6T單獨）。該盤櫃沒有熱備盤。 剩下的每個個盤櫃分別做兩組RAID6: 當硬盤數為12塊時，RAID6实际可用18.3TB左右(分成三個6TB的盤X3）。 當硬盤數為11塊時，RAID6实际可用16.5TB左右(分成三個6TB的盤X3）。
问题
1) 4个黄色硬盘的作用为，存储操作系统预留，占用15G空间，最大充许坏掉三块。 S5600T 充许坏4块，因为控制器上面有个大约50G的ROM盘，可以管理存储。
2) 熱備盤作為全局模式存在。
- RAID6 12块硬盘的情况下，热备盘替换并同步需要的时间尚。
当坏掉两块的时候，再坏掉一块，会引起业务中断，但RAID中数据不会丢失。
- RAID5 12块硬盘(2TB)的情况下，热备盘替换并同步需要的时间，至少需要大约3个小时,
24块硬盘的情况下，需要的时间翻倍。
管理信息
存儲管理地址
型號: Huawei S5600T IP: 12.33.68.250/24 IP: 12.33.68.251/24 Gateway: 12.33.68.254 USER/PASSWD: admin/Admin@storage
光纖交換機
型號: IBM 2498_B24 IP: 12.33.68.252/24 USER/PASSWD: admin/
安裝多路徑軟件UltraPath
在AIX中安裝
查看版本
# oslevel -r 6100-07
查看FC硬件信息
# lsdev -Cc adapter | grep fc fcs0 Available 02-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) fcs1 Available 02-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) fcs2 Available 08-00 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) fcs3 Available 08-01 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
查看FC model
# lscfg -vpl fcs0 | grep -i model
Model: 00E0806
# lscfg -vpl fcs0
fcs0 U497B.001.DBJ7088-P2-C1-T1 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03)
Part Number.................00E0806
Serial Number...............1A235010DA
Manufacturer................001A
EC Level.................... D77161
Customer Card ID Number.....577D
FRU Number..................00E0806
Device Specific.(ZM)........3
Network Address.............10000090FA0D0AEA
ROS Level and ID............02782037
Device Specific.(Z0)........31004549
Device Specific.(Z1)........00000000
Device Specific.(Z2)........00000000
Device Specific.(Z3)........09030909
Device Specific.(Z4)........FF781150
Device Specific.(Z5)........02782037
Device Specific.(Z6)........07732037
Device Specific.(Z7)........0B7C2037
Device Specific.(Z8)........20000120FA0D0AEA
Device Specific.(Z9)........US2.00X7
Device Specific.(ZA)........U2D2.00X7
Device Specific.(ZB)........U3K2.00X7
Device Specific.(ZC)........00000000
Hardware Location Code......U497B.001.DBJ7088-P2-C1-T1
PLATFORM SPECIFIC
Name: fibre-channel
Model: 00E0806
Node: fibre-channel@0
Device Type: fcp
Physical Location: U497B.001.DBJ7088-P2-C1-T1
重設FC設備
rmdev -dl fcs0 -R rmdev -dl fcs1 -R rmdev -dl fcs2 -R rmdev -dl fcs3 -R
上傳安裝文件
scp OR ftp
設置HBA卡屬性（可選）
#
# chdev -l fcs0 -a fc_err_recov=fast_fail
Method error (/usr/lib/methods/chggen):
0514-017 The following attributes are not valid for the
specified device:
fc_err_recov
# chdev -l fcs0 -a dyntrk=yes
Method error (/usr/lib/methods/chggen):
0514-017 The following attributes are not valid for the
specified device:
dyntrk
# sh install.sh
NOTICE :For properties of SAN HBA,please set dyntrk to yes and set fc_err_recov to fast_fail.
if not,the function of fail_over may be invalid.
WARNING 1 :The HBA model fcs0 is 00E0806,it has not been validate.
which may cause the function of UltraPath software is invalid.
WARNING 2 :The HBA model fcs1 is 00E0806,it has not been validate.
which may cause the function of UltraPath software is invalid.
WARNING 3 :The HBA model fcs2 is 00E0806,it has not been validate.
which may cause the function of UltraPath software is invalid.
WARNING 4 :The HBA model fcs3 is 00E0806,it has not been validate.
which may cause the function of UltraPath software is invalid.
4 warnings exist during installing the UltraPath software,are you sure continue(Y/y)or(N/n)? : -->
y
...
..
.
Installing Software...
+-----------------------------------------------------------------------------+
installp: APPLYING software for:
UltraPath-5.01.005.AIX6.1.ppc_64.rte 5.1.5.1
NOTICE :For properties of SAN HBA,please set dyntrk to yes and set fc_err_recov to fast_fail.if not,the function of
fail_over may be invalid.
+============================= A T T E N T I O N =============================+
You must restart your system for the configuration changes made to UltraPath to take effect.
+=============================================================================+
Finished processing all filesets. (Total time: 14 secs).
0503-409 installp: bosboot verification starting...
installp: bosboot verification completed.
0503-408 installp: bosboot process starting...
trustchk: Verification of attributes failed: /usr/sbin/bootinfo
: accessauths
bosboot: Boot image is 53276 512 byte blocks.
0503-292 This update will not fully take effect until after a
system reboot.
* * * A T T E N T I O N * * *
System boot image has been updated. You should reboot the
system as soon as possible to properly integrate the changes
and to avoid disruption of current functionality.
installp: bosboot process completed.
+-----------------------------------------------------------------------------+
Summaries:
+-----------------------------------------------------------------------------+
Installation Summary
--------------------
Name Level Part Event Result
-------------------------------------------------------------------------------
UltraPath-5.01.005.AIX6.1.p 5.1.5.1 USR APPLY SUCCESS
Begin to modify queue_depth and max_transfer...
The PowerPC model is newer than PowerPc_Power4:PowerPc_Power7, queue_depth and max_transfer will be modified.
查看設備屬性
# lsattr -l hdisk0 -E # lsattr -l hdisk0 -E PCM PCM/friend/scsiscsd Path Control Module False algorithm fail_over Algorithm True dist_err_pcnt 0 Distributed Error Percentage True dist_tw_width 50 Distributed Error Sample Time True hcheck_interval 0 Health Check Interval True hcheck_mode nonactive Health Check Mode True max_coalesce 0x10000 Maximum Coalesce Size True max_transfer 0x100000 Maximum TRANSFER Size True pvid 00f7e8fa07cf26950000000000000000 Physical volume identifier False queue_depth 16 Queue DEPTH True reserve_policy no_reserve Reserve Policy True size_in_mb 300000 Size in Megabytes False unique_id 281135000039438286D0809MBF2300RC03IBMsas Unique device identifier False ww_id 5000039438286d08 World Wide Identifier False # lsattr -l hdisk0 -E -a ww_id ww_id 5000039438286d08 World Wide Identifier False
修改設備屬性
# lsattr -E -a reserve_policy -F value -l hdisk44 no_reserve ''reserve_policy'' meanings: '''single_path''' SCSI2 reserve polices. The virtual disk is exclusively availabe to one host, and the other one has no rights to accessing or controlling it. '''no_reserve''' SCSI2 reserve polices. The virtual disk is shared by the two hosts. '''PR_exclusive''' SCSI2 reserve polices. The virtual disk is excusively available to on host, and the other on has no right to accessing or controlling it. '''PR_shared''' SCSI2 reserve polices. The virtual disk is shared by the two hosts. e:w # chdev -a algorithm=fail_over -l hdisk44 # prtconf | grep -i fc + fcs2 U497B.001.DBJ7088-P2-C5-T1 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) * fcnet2 U497B.001.DBJ7088-P2-C5-T1 Fibre Channel Network Protocol Device + fscsi2 U497B.001.DBJ7088-P2-C5-T1 FC SCSI I/O Controller Protocol Device + fcs3 U497B.001.DBJ7088-P2-C5-T2 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) * fcnet3 U497B.001.DBJ7088-P2-C5-T2 Fibre Channel Network Protocol Device + fscsi3 U497B.001.DBJ7088-P2-C5-T2 FC SCSI I/O Controller Protocol Device + fcs0 U497B.001.DBJ7088-P2-C1-T1 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) * fcnet0 U497B.001.DBJ7088-P2-C1-T1 Fibre Channel Network Protocol Device + fscsi0 U497B.001.DBJ7088-P2-C1-T1 FC SCSI I/O Controller Protocol Device + fcs1 U497B.001.DBJ7088-P2-C1-T2 8Gb PCI Express Dual Port FC Adapter (df1000f114108a03) * fcnet1 U497B.001.DBJ7088-P2-C1-T2 Fibre Channel Network Protocol Device + fscsi1 U497B.001.DBJ7088-P2-C1-T2 FC SCSI I/O Controller Protocol Device # prtconf | grep -i hdisk hdisk0 active 558 493 111..96..63..111..112 hdisk1 active 558 447 111..50..63..111..112 + hdisk1 U497B.001.DBJ7088-P2-C9-D4 SAS Disk Drive (300000 MB) + hdisk0 U497B.001.DBJ7088-P2-C9-D1 SAS Disk Drive (300000 MB)
重啟AIX主機使UltraPath生效
# upadm show version Version : V100R005 UltraPath for AIX : 5.01.005 # upadm show option Show option: vismaxnode = 8 failover = on failback_interval = 10 # upadm show daemon Daemon autoSetPri is running. #
LUN映射
在Huaiwei OceaStor ISM管理界面中的 SAN服务->映射->启动器 查看HBA卡的WWN号码，与AIX机器对应。
AIX01
# lscfg -vpl fcs1 | grep -i net
Network Address.............10000090FA0D0AEB
# lscfg -vpl fcs3 | grep -i net
Network Address.............10000090FA0D0913AIX02
# lscfg -vpl fcs1 | grep -i net
Network Address.............10000090FA0D0A6F
# lscfg -vpl fcs3 | grep -i net
Network Address.............10000090FA0D7521在存储管理软件上把lun映射到AIX主机。
扫描一哈
# cfgmgr # prtconf | grep 'FC Disk' * hdisk2 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L0 Huawei S5600T FC Disk Drive * hdisk3 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L1000000000000 Huawei S5600T FC Disk Drive * hdisk4 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L2000000000000 Huawei S5600T FC Disk Drive * hdisk5 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L3000000000000 Huawei S5600T FC Disk Drive * hdisk6 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L4000000000000 Huawei S5600T FC Disk Drive * hdisk7 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L5000000000000 Huawei S5600T FC Disk Drive * hdisk8 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L6000000000000 Huawei S5600T FC Disk Drive * hdisk9 U497B.001.DBJ7088-P2-C1-T2-W201B0022A10D4D28-L7000000000000 Huawei S5600T FC Disk Drive
查看hdisk lun WWN
# upadm show lun Vendor of /dev/hdisk0 is not HUAWEI, HUASY, SYMANTEC or HS Vendor of /dev/hdisk1 is not HUAWEI, HUASY, SYMANTEC or HS ----------------------------------------------------------------------------------------------------------------- Device Name: Lun Name: Vendor ID: Type: Serial Number: Device WWN: ----------------------------------------------------------------------------------------------------------------- /dev/hdisk2 LUN_027 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd39c0000001a /dev/hdisk3 LUN_028 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd3e50000001b /dev/hdisk4 LUN_029 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd4210000001c /dev/hdisk5 LUN_030 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd4690000001d /dev/hdisk6 LUN_031 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd4ad0000001e /dev/hdisk7 LUN_032 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd4ec0000001f /dev/hdisk8 LUN_033 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd52700000020 /dev/hdisk9 LUN_034 HUAWEI S5600T 1T50214955 60022a11000d4d28036cd57900000021 /dev/hdisk10 LUN_001 HUAWEI S5600T 1T50214955 60022a11000d4d280355aaf900000000 /dev/hdisk11 LUN_002 HUAWEI S5600T 1T50214955 60022a11000d4d280355ab3900000001 /dev/hdisk12 LUN_003 HUAWEI S5600T 1T50214955 60022a11000d4d280355c68900000002 /dev/hdisk13 LUN_004 HUAWEI S5600T 1T50214955 60022a11000d4d280355ea2a00000003 /dev/hdisk14 LUN_005 HUAWEI S5600T 1T50214955 60022a11000d4d280355ea6900000004 /dev/hdisk15 LUN_006 HUAWEI S5600T 1T50214955 60022a11000d4d280356186d00000005 /dev/hdisk16 LUN_007 HUAWEI S5600T 1T50214955 60022a11000d4d2803566f3700000006 /dev/hdisk17 LUN_008 HUAWEI S5600T 1T50214955 60022a11000d4d2803566f7900000007 /dev/hdisk18 LUN_009 HUAWEI S5600T 1T50214955 60022a11000d4d280356931900000008 /dev/hdisk19 LUN_010 HUAWEI S5600T 1T50214955 60022a11000d4d280356ad2b00000009 /dev/hdisk20 LUN_011 HUAWEI S5600T 1T50214955 60022a11000d4d280356ad700000000a /dev/hdisk21 LUN_012 HUAWEI S5600T 1T50214955 60022a11000d4d280356c9f90000000b /dev/hdisk22 LUN_013 HUAWEI S5600T 1T50214955 60022a11000d4d280356e4400000000c /dev/hdisk23 LUN_014 HUAWEI S5600T 1T50214955 60022a11000d4d280356e4790000000d /dev/hdisk24 LUN_015 HUAWEI S5600T 1T50214955 60022a11000d4d2803571c3f0000000e /dev/hdisk25 LUN_016 HUAWEI S5600T 1T50214955 60022a11000d4d28035733220000000f /dev/hdisk26 LUN_017 HUAWEI S5600T 1T50214955 60022a11000d4d280357336700000010 /dev/hdisk27 LUN_018 HUAWEI S5600T 1T50214955 60022a11000d4d2803574c5d00000011 /dev/hdisk28 LUN_019 HUAWEI S5600T 1T50214955 60022a11000d4d28036bfce300000012 /dev/hdisk29 LUN_020 HUAWEI S5600T 1T50214955 60022a11000d4d28036bfd5a00000013 /dev/hdisk30 LUN_021 HUAWEI S5600T 1T50214955 60022a11000d4d28036bfda800000014 /dev/hdisk31 LUN_022 HUAWEI S5600T 1T50214955 60022a11000d4d28036c223800000015 /dev/hdisk32 LUN_023 HUAWEI S5600T 1T50214955 60022a11000d4d28036c229500000016 /dev/hdisk33 LUN_024 HUAWEI S5600T 1T50214955 60022a11000d4d28036c22da00000017 /dev/hdisk34 LUN_025 HUAWEI S5600T 1T50214955 60022a11000d4d28036c231a00000018 /dev/hdisk35 LUN_026 HUAWEI S5600T 1T50214955 60022a11000d4d28036c435300000019 /dev/hdisk36 LUN_043 HUAWEI S5600T 1T50214955 60022a11000d4d28036d12b60000002a /dev/hdisk37 LUN_044 HUAWEI S5600T 1T50214955 60022a11000d4d28036eaa9c0000002b /dev/hdisk38 LUN_045 HUAWEI S5600T 1T50214955 60022a11000d4d28036fbc110000002c /dev/hdisk39 LUN_046 HUAWEI S5600T 1T50214955 60022a11000d4d28036fe8030000002d /dev/hdisk40 LUN_047 HUAWEI S5600T 1T50214955 60022a11000d4d2803700c600000002e /dev/hdisk41 LUN_048 HUAWEI S5600T 1T50214955 60022a11000d4d2804c138f20000002f /dev/hdisk42 LUN_049 HUAWEI S5600T 1T50214955 60022a11000d4d2804c1929100000030 /dev/hdisk43 LUN_050 HUAWEI S5600T 1T50214955 60022a11000d4d2804c192e900000031 /dev/hdisk44 LUN_051 HUAWEI S5600T 1T50214955 60022a11000d4d2804c1932b00000032 /dev/hdisk45 LUN_052 HUAWEI S5600T 1T50214955 60022a11000d4d2804c1938000000033 /dev/hdisk46 LUN_053 HUAWEI S5600T 1T50214955 60022a11000d4d2804c193bc00000034 /dev/hdisk47 LUN_054 HUAWEI S5600T 1T50214955 60022a11000d4d2804c193fa00000035 /dev/hdisk48 LUN_055 HUAWEI S5600T 1T50214955 60022a11000d4d2804c1944600000036 /dev/hdisk49 LUN_056 HUAWEI S5600T 1T50214955 60022a11000d4d2804c1949400000037 /dev/hdisk50 LUN_057 HUAWEI S5600T 1T50214955 60022a11000d4d2804cf03e40000003