#acl merlyn:read,write All:read
<<TableOfContents()>>
= lab =
http://labondemand.netapp.com/

https://library.netapp.com/ecmdocs/ECMP1136573/html/frameset.html


== System Performance Modeler ==
https://spm.netapp.com

== UP tool ==
  *  UP Supported Components: cDOT & 7-Mode Storage Controllers, Windows, VMware, Linux, HPUX, AIX, Cisco, Brocade
https://up.netapp.com

== top 10 qa ==
https://kb.netapp.com/support/index?page=content&id=4010151

== how to find lun wwid  ==
http://www.voleg.info/netapp_serial_to_wwid_converter.html

== data ontap simulator requirements ==
http://mysupport.netapp.com/now/cgi-bin/simulator 
http://mysupport.netapp.com/now/cgi-bin/simulatorlic8.cgi/download/tools/simulator/ontap/8.1.3/vsim_netapp-7m.tgz

=== adding disks to a data ontap ===
http://community.netapp.com/t5/developer-network-articles-and-resources/adding-disks-to-a-data-ontap-8-1-1-simulator/ta-p/86240

http://kabri.uk/2013/11/04/how-to-increase-the-usable-disk-size-in-the-netapp-ontap-8-x-7-mode-simulator/
== SNAPSHOT ==
{{{
Cluster1::> run -node Cluster1-0
    Cluster1-01 Cluster1-02 
Cluster1::> run -node Cluster1-02
Type 'exit' or 'Ctrl-D' to return to the CLI
Cluster1-02> snap delete -a -f vol0
Deleted vol0 snapshot hourly.5.
Deleted vol0 snapshot hourly.4.
Deleted vol0 snapshot nightly.1.
Deleted vol0 snapshot hourly.3.
Deleted vol0 snapshot hourly.2.
Deleted vol0 snapshot hourly.1.
Deleted vol0 snapshot hourly.0.
Deleted vol0 snapshot nightly.0.
Cluster1-02> snap sched vol0 0 0 0
Cluster1-02> snap autodelete vol0 on
snap autodelete: snap autodelete enabled
Cluster1-02> snap autodelete vol0 target_free_space 35
snap autodelete: snap autodelete configuration options set
Cluster1-02> snap autodelete vol0
snapshot autodelete settings for vol0:
state				: on
commitment			: try
trigger				: volume
target_free_space		: 35%
delete_order			: oldest_first
defer_delete			: user_created
prefix				: (not specified)
destroy_list			: none 
}}}

= troubleshooting =
== Changing the disk shelf ID ==
{{{
Steps
1. Turn on the power to the disk shelf if it is not already on.
2. Remove the left end cap to locate the button near the shelf LEDs.
3. Change the first number of the shelf ID by pressing and holding the orange button until the first
number on the digital display blinks, which can take up to three seconds.
Note: If the ID takes longer than three seconds to blink, press the button again, making sure to
press it in all the way.
This activates the disk shelf ID programming mode.
4. Press the button to advance the number until you reach the desired number from 0 to 9.
The first number continues to blink.
5. Change the second number of the shelf ID by pressing and holding the button until the second
number on the digital display blinks, which can take up to three seconds.
The first number on the digital display stops blinking.
6. Press the button to advance the number until you reach the desired number from 1 to 9.
The second number continues to blink.
7. Lock in the desired number and exit the programming mode by pressing and holding the button
until the second number stops blinking, which can take up to three seconds.
Both numbers on the digital display start blinking and the amber LED on the operator display
panel illuminates after about five seconds, alerting you that the pending disk shelf ID has not yet
8. Power cycle the disk shelf to make the shelf ID take effect.
}}}
== collection autosupport log ==
  * Display the health status of the system, which reflects the overall status of individual health monitors	
{{{
system health status show
}}}
  
  * Display the health status of subsystems for which health monitoring is available	
{{{
system health subsystem show
}}}

  * node run -node local sysconfig -a

{{{
收集ASUP log，执行下面的命令就可以了，log就会产生。
system node autosupport invoke -node * -type all -uri file:///mroot/etc/log/case_number-asup.7z

存储的管理IP后面跟 /SPI，然后输入admin账号密码
https://x.x.x.x/spi/

log的路径是
/etc/log/case_number-asup.7z
}}}

== System Fault  Led is on (FAS 2240-2) ==
{{{
We opened a case for this issue. Support sent to me the 7-step process below to resolve the issue with the system fault LEDs:

    1.  At the Data ONTAP CLI prompt, issue the “halt –s” command. Please note that this will cause an HA takeover in an HA configuration.

    2.  Wait for the controller to shut down and power off.

    3.  On the console port (not RSH, SSH, etc.), issue a “^G”(ctrl G)to switch the console to the SP.

    4.     Log in to the SP.

    5.  At the SP prompt, issue the “system power on” command.

    6.  Issue a “^D”(ctrl D)to get back to the Data ONTAP console.

    7.     Once the BIOS / Loader have booted, issue the “boot_ontap” command.

    8. Issue Giveback after Takeover.

After following the 7-step process, the issue has been solved

Regards,
}}}
http://vipulvajpayeestorage.blogspot.com/2012/04/netapp-fas-2240-storage-showing-pcm.html

https://community.netapp.com/t5/OnCommand-Storage-Management-Software-Discussions/System-Fault-Led-is-on-FAS-2240-2/td-p/79624


== too many users logged in!  please try again later. ==
http://nayabrasools.blogspot.com/2013/08/netapp-too-many-users-logged-in-please.html
{{{
可能是其他人有登入，則請其他人先退出
或是登入後因為網路不穩，造成連線不正常中斷，此時就必須要等timeout時間一到，才可以再登入
另外可以嘗試透過rsh重啟telnet 或是ssh服務來解決這個問題
首先確定netapp 的rsh 是否有開啟
找一台client測試以下指令
client# rsh filer_ip -l root@password df
如果有正常回應的話
輸入
client# rsh filer_ip -l root@password "options ssh.enable off"
client# rsh filer_ip -l root@password "options ssh.enable on"
或是
client# rsh filer_ip -l root@password "options telnet.enable off"
client# rsh filer_ip -l root@password "options telnet.enable on"
即可
}}}
rsh 192.168.0.192 -l root:xx sysconfig -a
rsh <filername> -l root:<password> logout telnet

==  fri dec 25 15:03:29 cst [ur-2240a:sis.changelog.full:warning]: sis change logging metafile for volume vol1_a is full. ==


== oncommand system manager error 500 connection refused ==
 * http://admincapital.com/2015/02/oncommand-system-manager-error-500-connection-refused/
tls is disabled by default on the storage controllers. it must be explicitly enabled
to enable tls, run the following command via the shell:

options tls.enable on

that should do it.
== aggr ? ==
for volumes - you can't have a flexible volume without an aggregate.  flex vols
    are logical, aggregates are physical.  you layer one or more flex vols on
    top (in side) of an aggregate

== how to monitor netapp storage systems using snmp ==
https://kb.netapp.com/support/index?page=content&id=1011225&pmv=print&impressions=false

== netapp best practice ==
as the best practice, netapp now recommends to set fractional reserve and snap reserve for your volumes to 0%. don’t forget about that, if you want to save more storage space:
{{{
    > vol options “targetvol” fractional_reserve 0
    > snap reserve “targetvol” 0

disable snapshots if you don’t use them:

    > snap sched “targetvol” 0
}}}

= how to =
== initialize setting ==
http://www.getshifting.com/wiki/netappchangeip

== how to add disk shelf to exist netapp ==

http://community.netapp.com/t5/fas-and-v-series-storage-systems-discussions/ds2246-sas-install-fas2240-2-config-check/td-p/55471

http://community.netapp.com/t5/fas-and-v-series-storage-systems-discussions/ds2246-sas-install-fas2240-2-config-check/m-p/55487#m3904

http://www.cosonok.com/2013/03/how-to-hot-add-ds4243-shelf-quick-guide.html

 how to add a new disk shelf to a netapp filer ??

 assumption:- filer has already one disk shelf connected this is the new one
 which has to be added and cabled.


 cables required:- two sas cables , two acp (cat6 ethernet ) cables if any


1. rack the shelf

2. connect power cables & turn the shelf on (wait about 30 seconds)

3. on the front of the shelf, change the shelf id to something unique. (try
and keep it sequential with the shelf that's already there)  do not use "00"

4. wait about 45 seconds and then restart the shelf and make sure the
new shelf id came up.

5. ssh into both filers and do "options disk.auto_assign off"

6. do a "storage show disk -p" on both controllers and copy the
contents into a txt file and save it. (run this command again and make
sure all your disks are still there)

7. do a "options autosupport.doit "before shelf add"" on both
controllers since i'm scared your doing this via community forums.
("before shelf add" is in quotes on the command)

8. now for the fun part, take a sas cable and plug one end into the
top circle of the old shelf and plug it in the bottom square of the
         new shelf

9. now take another sas cable and plug one end into the bottom circle
of the old shelf and plug it into the top square of the new shelf

10. assign your disks and don't forget to turn back on disk auto assign
when you're done.

11. run "storage show disk -p" and make sure all of your old disks are
there.

(shelves at the end of a stack are always reversed, this is a special
circumstance and you will not cable all shelves like this in the future.
http://nayabrasools.blogspot.com/2012/10/how-to-add-new-disk-shelf-to-netapp.html


== data ontap basic command please? ==

  this post contains the list of commands that will be most used
and will come handy when managing or monitoring or troubleshooting a netapp
filer.
{{{
    sysconfig -a : shows hardware
    configuration with more verbose information
    sysconfig -d :
    shows information of the disk attached to the filer
    version : shows the
    netapp ontap os version.
    uptime : shows the
    filer uptime
    dns info : this
    shows the dns resolvers, the no of hits and misses and other info
    nis info : this
    shows the nis domain name, yp servers etc.
    rdfile : like
    "cat" in linux, used to read contents of text files/
    wrfile :
    creates/overwrites a file. similar to "cat > filename" in linux
    aggr status : shows
    the aggregate status
    aggr status -r :
    shows the raid configuration, reconstruction information of the disks in filer
    aggr show_space :
    shows the disk usage of the aggreate, wafl reserve, overheads etc.
    vol status : shows
    the volume information
    vol status -s :
    displays the spare disks on the filer
    vol status -f :
    displays the failed disks on the filer
    vol status -r :
    shows the raid configuration, reconstruction information of the disks
    df -h : displays
    volume disk usage
    df -i : shows the inode
    counts of all the volumes
    df -ah : shows
    "df" information of the aggregate
    license :
    displays/add/removes license on a netapp filer
    maxfiles : displays
    and adds more inodes to a volume
    aggr create :
    creates aggregate
    vol create
    <volname> <aggrname> <size> : creates volume in an aggregate
    vol offline
    <volname> : offlines a volume
    vol online
    <volname> : onlines a volume
    vol destroy
    <volname> : destroys and removes an volume
    vol size
    <volname> [+|-]<size> : resize a volume in netapp filer
    vol options : displays/changes
    volume options in a netapp filer
    qtree create
    <qtree-path> : creates qtree
    qtree status :
    displays the status of qtrees
    quota on : enables
    quota on a netapp filer
    quota off :
    disables quota
    quota resize :
    resizes quota
    quota report :
    reports the quota and usage
    snap list :
    displays all snapshots on a volume
    snap create
    <volname> <snapname> : create snapshot
    snap sched
    <volname> <schedule> : schedule snapshot creation
    snap reserve
    <volname> <percentage> : display/set snapshot reserve space in volume
    /etc/exports : file
    that manages the nfs exports
    rdfile /etc/exports
    : read the nfs exports file
    wrfile /etc/exports
    : write to nfs exports file
    exportfs -a :
    exports all the filesystems listed in /etc/exports
    cifs setup : setup
    cifs
    cifs shares : create/displays
    cifs shares
    cifs access :
    changes access of cifs shares
    lun create :
    creates iscsi or fcp luns on a netapp filer
    lun map : maps lun
    to an igroup
    lun show : show all
    the luns on a filer
    igroup create :
    creates netapp igroup
    lun stats : show lun
    i/o statistics
    disk show : shows
    all the disk on the filer
    disk zero spares :
    zeros the spare disks
    disk_fw_update :
    upgrades the disk firmware on all disks
    options :
    display/set options on netapp filer
    options nfs :
    display/set nfs options
    options timed :
    display/set ntp options on netapp.
    options autosupport
    : display/set autosupport options
    options cifs :
    display/set cifs options
    options tcp :
    display/set tcp options
    options net :
    display/set network options
    ndmpcopy
    <src-path> <dst-path> : initiates ndmpcopy
    ndmpd status :
    displays status of ndmpd
    ndmpd killall :
    terminates all the ndmpd processes.
    ifconfig :
    displays/sets ip address on a network/vif interface
    vif create :
    creates a vif (bonding/trunking/teaming)
    vif status :
    displays status of a vif
    netstat : displays
    network statistics
    sysstat -us 1 : begins a 1
    second sample of the filer's current utilization (crtl - c to end)
    nfsstat : shows nfs
    statistics
    nfsstat -l :
    displays nfs stats per client
    nfs_hist : displays
    nfs historgram
    statit : beings/ends a
    performance workload sampling [-b starts / -e ends]
    stats : displays
    stats for every counter on netapp. read stats man page for more info
    ifstat : displays
    network interface stats
    qtree stats :
    displays i/o stats of qtree
    environment : display
    environment status on shelves and chassis of the filer
    storage show
    <disk|shelf|adapter> : shows storage component details
    snapmirror
    intialize : initialize a snapmirror relation
    snapmirror update :
    manually update snapmirror relation
    snapmirror resync :
    resyns a broken snapmirror
    snapmirror quiesce
    : quiesces a snapmirror bond
    snapmirror break :
    breakes a snapmirror relation
    snapmirror abort :
    abort a running snapmirror
    snapmirror status :
    shows snapmirror status
    lock status -h :
    displays locks held by filer
    sm_mon : manage the
    locks
    storage download
    shelf : installs the shelf firmware
    software get :
    download the netapp os software
    software install :
    installs os
    download : updates
    the installed os
    cf status :
    displays cluster status
    cf takeover : takes
    over the cluster partner
    cf giveback : gives
    back control to the cluster partner
    reboot : reboots a
    filer
}}}

== pocket guide for netapp commands ==

this post contains the list of commands that will be most used
and will come handy when managing or monitoring or troubleshooting a netapp
filer.
{{{
    sysconfig -a : shows hardware
    configuration with more verbose information
    sysconfig -d :
    shows information of the disk attached to the filer
    version : shows the
    netapp ontap os version.
    uptime : shows the
    filer uptime
    dns info : this
    shows the dns resolvers, the no of hits and misses and other info
    nis info : this
    shows the nis domain name, yp servers etc.
    rdfile : like
    "cat" in linux, used to read contents of text files/
    wrfile :
    creates/overwrites a file. similar to "cat > filename" in linux
    aggr status : shows
    the aggregate status
    aggr status -r :
    shows the raid configuration, reconstruction information of the disks in filer
    aggr show_space :
    shows the disk usage of the aggreate, wafl reserve, overheads etc.
    vol status : shows
    the volume information
    vol status -s :
    displays the spare disks on the filer
    vol status -f :
    displays the failed disks on the filer
    vol status -r :
    shows the raid configuration, reconstruction information of the disks
    df -h : displays
    volume disk usage
    df -i : shows the inode
    counts of all the volumes
    df -ah : shows
    "df" information of the aggregate
    license :
    displays/add/removes license on a netapp filer
    maxfiles : displays
    and adds more inodes to a volume
    aggr create :
    creates aggregate
    vol create
    <volname> <aggrname> <size> : creates volume in an aggregate
    vol offline
    <volname> : offlines a volume
    vol online
    <volname> : onlines a volume
    vol destroy
    <volname> : destroys and removes an volume
    vol size
    <volname> [+|-]<size> : resize a volume in netapp filer
    vol options : displays/changes
    volume options in a netapp filer
    qtree create
    <qtree-path> : creates qtree
    qtree status :
    displays the status of qtrees
    quota on : enables
    quota on a netapp filer
    quota off :
    disables quota
    quota resize :
    resizes quota
    quota report :
    reports the quota and usage
    snap list :
    displays all snapshots on a volume
    snap create
    <volname> <snapname> : create snapshot
    snap sched
    <volname> <schedule> : schedule snapshot creation
    snap reserve
    <volname> <percentage> : display/set snapshot reserve space in volume
    /etc/exports : file
    that manages the nfs exports
    rdfile /etc/exports
    : read the nfs exports file
    wrfile /etc/exports
    : write to nfs exports file
    exportfs -a :
    exports all the filesystems listed in /etc/exports
    cifs setup : setup
    cifs
    cifs shares : create/displays
    cifs shares
    cifs access :
    changes access of cifs shares
    lun create :
    creates iscsi or fcp luns on a netapp filer
    lun map : maps lun
    to an igroup
    lun show : show all
    the luns on a filer
    igroup create :
    creates netapp igroup
    lun stats : show lun
    i/o statistics
    disk show : shows
    all the disk on the filer
    disk zero spares :
    zeros the spare disks
    disk_fw_update :
    upgrades the disk firmware on all disks
    options :
    display/set options on netapp filer
    options nfs :
    display/set nfs options
    options timed :
    display/set ntp options on netapp.
    options autosupport
    : display/set autosupport options
    options cifs :
    display/set cifs options
    options tcp :
    display/set tcp options
    options net :
    display/set network options
    ndmpcopy
    <src-path> <dst-path> : initiates ndmpcopy
    ndmpd status :
    displays status of ndmpd
    ndmpd killall :
    terminates all the ndmpd processes.
    ifconfig :
    displays/sets ip address on a network/vif interface
    vif create :
    creates a vif (bonding/trunking/teaming)
    vif status :
    displays status of a vif
    netstat : displays
    network statistics
    sysstat -us 1 : begins a 1
    second sample of the filer's current utilization (crtl - c to end)
    nfsstat : shows nfs
    statistics
    nfsstat -l :
    displays nfs stats per client
    nfs_hist : displays
    nfs historgram
    statit : beings/ends a
    performance workload sampling [-b starts / -e ends]
    stats : displays
    stats for every counter on netapp. read stats man page for more info
    ifstat : displays
    network interface stats
    qtree stats :
    displays i/o stats of qtree
    environment : display
    environment status on shelves and chassis of the filer
    storage show
    <disk|shelf|adapter> : shows storage component details
    snapmirror
    intialize : initialize a snapmirror relation
    snapmirror update :
    manually update snapmirror relation
    snapmirror resync :
    resyns a broken snapmirror
    snapmirror quiesce
    : quiesces a snapmirror bond
    snapmirror break :
    breakes a snapmirror relation
    snapmirror abort :
    abort a running snapmirror
    snapmirror status :
    shows snapmirror status
    lock status -h :
    displays locks held by filer
    sm_mon : manage the
    locks
    storage download
    shelf : installs the shelf firmware
    software get :
    download the netapp os software
    software install :
    installs os
    download : updates
    the installed os
    cf status :
    displays cluster status
    cf takeover : takes
    over the cluster partner
    cf giveback : gives
    back control to the cluster partner
    reboot : reboots a
    filer
}}}

vperumal former netapp employee
http://community.netapp.com/t5/data-ontap-discussions/data-ontap-basic-command-please/td-p/59641


== netapp basic how to ==
=== lun create & map ===
1, fcp wwpn-alias set alias_name wwpn

2,igroup create -i -t windows_2008 win-group0 alias_name,1,2,3
{{{
igroup create -f -t aix ibm_p720 ibm_p720a ibm_p720b
igroup create -f -t windows image 198.0.0.10
igroup add initialtor_group nodename/alias_name
}}}

3, n6060a> 
{{{
vol create ibm_p720 aggr0a 330g
vol create image aggr0a 2448g
vol create data aggr0a 900g

vol create chjbak aggr0b 900g

snap reserve ibm_p720 0
}}}

4, lun create -s size -t ostype lun_path  (lun show)
{{{
lun create -s 5g -t aix /vol/ibm_p720/crs01
lun create -s 5g -t aix /vol/ibm_p720/crs02
lun create -s 5g -t aix /vol/ibm_p720/crs03
lun create -s 100g -t aix /vol/ibm_p720/data01
lun create -s 100g -t aix /vol/ibm_p720/data02
lun create -s 100g -t aix /vol/ibm_p720/data03

lun create -s 898g -t windows /vol/chjbak/chjbak01

lun create -s 2048g -t windows_2008 /vol/image/image01
lun create -s 400g -t windows_2008 /vol/image/image02
}}}

5, lun map lun_path initialtor_group/alias_name 
{{{
n6060a> fcp wwpn-alias show
wwpn                alias
----                -----
10:00:00:90:fa:09:34:87     198.0.0.221
10:00:00:90:fa:09:37:de     198.0.0.222
10:00:00:90:fa:09:35:b6     198.0.0.223
10:00:00:00:c9:88:9b:4b     ibm_p720a
10:00:00:00:c9:7a:14:35     ibm_p720b
10:00:00:00:c9:e0:b0:12     198.0.0.10
21:00:00:1b:32:90:4e:63     198.0.0.152

lun map /vol/ibm_p720/crs01 ibm_p720
lun map /vol/ibm_p720/crs02 ibm_p720
lun map /vol/ibm_p720/crs03 ibm_p720
lun map /vol/ibm_p720/data01 ibm_p720
lun map /vol/ibm_p720/data02 ibm_p720
lun map /vol/ibm_p720/data03 ibm_p720
lun map /vol/image/image01 198.0.0.10
lun map /vol/image/image02 198.0.0.10
}}}



=== notes ===

    sysconfig -a : shows hardware

    aggr status 
    aggr status -r 
    aggr show_space 
    vol status -s 
    vol status -r 
    lun stats 




== test failover works ==
1, cf takeover

2, cf status
filer x has taken over filer y.

3, cf status
filer x is ready for giveback.
partner ifconfig -a
partner vfiler status

4, vol status
   aggr status
n6060b(takeover) partner
   vol status
   aggr status


cf giveback
cf giveback -f

== simulator download ==
http://mysupport.netapp.com/now/cgi-bin/simulator

== 7 mode vs cluster mode ==

最开始的ｄata ontap 借鉴了最早的开源unix之一的bsd net/2许多代码，包括 tcp/ip
堆栈、启动代码、设备驱动等，后来的data
ontap也从其他的开源unix借鉴了大量的代码。至于命令行接口是netapp仿unix独立开发的,另外wafl文件系统与raid代码及磁盘子系统为netapp自己开发的所以与unix的完全不同。

一开始dataontap没有什么mode之分的，netapp收购了spinnaker之后基于freebsd把dataontap与spinnaker的软件重新整合出另一个叫data
ontap gx的操作系统来，当data
ontap更新至8.0版本后，netapp统一了二者的版本号,并把原生的dataontap称为7 mode,
后来的data ontap gx则称为 cluster mode,简称为c mode。

由于新的dataontap cluster mode功能一直不够完善，netapp一直同时开发着7 mode
和cluster mode两个操作系统，不过随着netapp大力推广和开发cluster
mode，相信很快就会停止开发7 mode只有cluste mode了。目前（data ontap
8.1.1）二者功能上的区别如下：

cluster-mode only

−clustered scaleout (24-nas: 4-san)
（多节点集群，如果提供san则一个集群最多4个节点）

−namespace （统一命名空间）

−nondisruptive operations （节点间平滑迁移volume或lun）

−management as single system （统一管理整个集群）

−scalable and integrated multi-tenancy （可扩展及多租户）

−nfs v4, v4.1 (pnfs); smb 2.0, 2.1

−onboard antivirus

 
{{{
 7-mode only

 −snaplock®

 −snapvault® and ossv

 −qtree and synchronous snapmirror

 −metrocluster™

 −vfiler®

 −flexshare®

 −ipv6, http, ftp, sftp, tftp

  

  both 7-mode and cluster-mode

  −unified architecture

  −storage efficiency features 和（重复数据删除与压缩）

  −snapshot™ copies and asynchronous volume snapmirror®

  −intelligent caching with flash cache
}}}

== oncommand ==
http://mysupport.netapp.com/now/cgi-bin/software/
http://community.netapp.com/t5/oncommand-storage-management-software-discussions/operations-manager-core-free-license-available/td-p/61964

== how to determine the disk and shelf firmware version on a controller and verify it is up to date ==

https://kb.netapp.com/index?page=content&id=1010762&actp=list_popular
https://kb.netapp.com/support/index?page=content&id=1011673&actp=list_recent

== iops calc ==
the shot analysis of netapp fas8040 (2-node, c-mode) spc-1 results | storage
news:
    http://www.stornews.com/?page_id=53

    http://www.stornews.com/?p=390

    http://www.hudop.com/tools/iopse.php

== perfstat ==
 * enable rsh
{{{
options rsh.enable on
}}}

{{{
Hi,

These are the commands I use to troubleshoot performance issues:

always start with "priv set diag"

sysstat -M -i 5

--> already explained

sysstat -x 5

--> shows you the different I/O ("Disk util" is an importend one and also "CPty" see
http://now.netapp.com/NOW/knowledge/docs/ontap/rel707/html/ontap/cmdref/man1/na_sysstat.1.html)

lun stats -i 5

--> shows you the read / write / latency's of luns

stats show lun

--> shows you detailed info of every lun (you will want to capture this in an output file)

stats show volume

--> same as lun but now for the volumes (you will want to capture this in an output file)

reallocate status

--> shows if any reallocation jobs are running (walf scan status shows you even more info)

If this is not enough you can get some info with statit

a "statit -b" will start the data collection (wait a few minutes)

a "statit -e" will stop the collection and will give you the result. (you will want this to capture in an
output file)
}}}

https://latx.netapp.com/summary/id/323525

== upgrade shelf disk firmware ==
http://mysupport.netapp.com/now/cgi-bin/shelffwdnld.cgi/download/tools/diskshelf/bin/all_shelf_fw#7dot

== re: understanding aggregate and lun ==
http://community.netapp.com/t5/fas-and-v-series-storage-systems-discussions/understanding-aggregate-and-lun/td-p/23326/page/3

== netapp ds4243, ds4246 and ds2246 sas storage shelf experiences a spurious id change ==
generally it requires either a power cycle of the shelf or a new esh module. either one requires a downtime.

netapp ds4243, ds4246 and ds2246 sas storage shelf experiences a spurious id change (kb alerts) - kamazoy knowledge books - it support in birmingham:

== netapp knowledgebase - how to power down and power up the controllers in a 7-mode ha-pair: ==
https://www.google.com.sb/url?sa=t&rct=j&q=&esrc=s&source=web&cd=4&cad=rja&uact=8&ved=0cdcqfjadahukewjy8jt7okjhahvf0h4khcv9dci&url=https%3a%2f%2fkb.netapp.com%2fsupport%2findex%3fpage%3dcontent%26id%3d1010084&ei=frtnvfk5nswke8x7tzam&usg=afqjcngezjtw7gv4gkafcw3rdvl4o0bhkg&bvm=bv.99804247,d.cww

== flash pool setting ==
aggr create aggr_hybrid -t raid_dp -t sas -n 18 -r 18
aggr option aggr_hybrid hybrid_enabled on
aggr add aggr_hybrid -t ssd

priority hybrid-cache set aggr_hybrid read-cache=random-read
priority hybrid-cache set aggr_hybrid write-cache=random-write

== differences between netapp flash cache and flash pool ==
https://bitpushr.wordpress.com/2013/07/30/differences-between-netapp-flash-cache-and-flash-pool/

= performance tunning =
== calculating queue depth ==
https://library.netapp.com/ecmdocs/ecmp12458217/html/guid-a055b184-0876-4376-9c75-35fe8c9be832.html

== san configuration limits ==
https://library.netapp.com/ecmdocs/ecmp1196793/html/guid-f664a5db-cab3-43fb-9e0a-6e61aeaf8466.html

== how to create and use flash pool ==
flash pool is a new feature in data ontap 8.1.1. basically, it is a pool of solid state disks (ssd) plus either sata or sas disks (but not both sata and sas in the same pool). 
=== reate a flash pool ===
 flash pool is implemented in the data ontap aggregate layer. to construct a flash pool, you use the follow three steps and in that order.
  1. create a 64-bit aggregate using either sata or sas hard drives as usual (see figure 1)
* aggr create fpaggr -b 64 -t raid_dp -t sata -r 16 16

  2. enable the aggregate option hybrid_enabled as shown in figure 2
* aggr options fpaggr hybrid_enabled on

  3. add ssds into the aggregate created in step 1 above (figure 3)
* aggr add fpaggr -t ssd 6

=== use the flash pool ===
vol create myvol fpaggr 1t

=== ur setting ===
{{{
aggr create aggr_hybrid -t raid_dp -t sas -n 18 -r 18
aggr option aggr_hybrid hybrid_enabled on
aggr add aggr_hybrid -t ssd

priority hybrid-cache set aggr_hybrid read-cache=random-read
priority hybrid-cache set aggr_hybrid write-cache=random-write
}}}

=== differences between netapp flash cache and flash pool ===
https://bitpushr.wordpress.com/2013/07/30/differences-between-netapp-flash-cache-and-flash-pool/

= Troubleshooting =
== received NHT health trigger ==
Disk 0a.14.23 received NHT health trigger (0x1  0xb 0x5d 0x10)

=== solution ===
disk maint start -d 3a.14.13

This will copy the data to a spare, then run tests

if ok, it will make the disk a spare, if not it will fail it.

* In the end the disk failed and was replaced.

aggr status -r aggrXX

* Once the disk copy is complete, run:

disk maint status


=== Replace disk ===

displays the failed disks on the filer:

vol status -r

Yes, you are correct, if disk is in showing in failed disk, then its good to remove the disk. If you failed it manully then you can again unfail the disk. below is the setps.

  1. failed the disk (disk failed <disk name>)

  2. remove the failed disk ( disk remove <disk name>)

  3. if you want this disk again which you failed manully ( use the command disk unfail <disk name>)

  4. If physicall want to see which disk then umber the light (blink_on <disk name>)

== aggr status ==
    Find the disk ID of the failed disk by entering the following command: 
      * aggr status -f
    If the disk does not appear in the list of failed disks, it might be partially failed, with a Rapid RAID Recovery in process. In this case, you should wait until the disk is present in the list of failed disks (which means that the Rapid RAID Recovery process is complete) before removing the disk.

    Determine the physical location of the disk you want to remove from the output of the aggr status -f command. The location is shown in the columns labeled HA, SHELF, and BAY.

    Remove the disk from the disk shelf, following the instructions in the hardware guide for your disk shelf model.

== FCP wwn ==
{{{
# storage port show -node clusterB-02 -port 2a
# storage bridge show

grep -B 13 'Switch Port' SYSCONFIG-A.txt | egrep 'FC Node Name|FC Port Name|FC Host Adapter|Fibre Channel Target Host Adapter|FCVI Host Adapter|Switch Port|FC Nodename|FC Portname' 
$ pwd
/mnt/play/Downloads/Oupai-logs/10.10.98.215
}}}