Contents
- VSAN ready
- VMware vSAN Best Practices
- memory consumed
- This procedure must be repeated on all participating vSAN nodes. LAG configuration can also be interrogated using command line, i.e. esxcli
- VSAN how to
- RVC how to
- Supported network topologies for VSAN stretched cluster
- 故障域构造
- About VSAN capacity and VMDK placement
- 删除磁盘原有分区
- MTU setting
- 请遵循以下准则
- Network Misconfiguration Status in a Virtual SAN Cluster
- Changing the multicast address used for a VMware Virtual SAN Cluster (2075451)
- Using tcpdump-uw to collect packet traces to troubleshoot network issues
- Generate Multicast traffic
- Monitor VSAN VMKernel Port network traffic
-
Troubleshooting
- Delete an uplink from a DVPort on a DVSwitch
- Gather special VM logs
- iperf test
- Monitor VM performance status
- show vds switch
- find disk
- export VDS configuration
- Collect logs
- vsan info
- 手动平衡
- vsan check
-
Disk/Storage
- 此命令对确定以下有关磁盘的信息非常有用
- SSD Permanent disk failure
- There is no more space for virtual disk
- host cannot communicate with all other nodes in the virtual san enabled cluster
- Registration/unregistration of a VASA vendor provider on a Virtual SAN host fails
- Issues information is not available at this time.
- java.lang.RuntimeException
- VM虚拟机无法ping通网关
- H3C 6800 交换机端口处理down状态
- No space left on device
- System logs are stored on non-persistent storage
- How to Delete VSAN Datastore
- VSAN upload problem
- Remove a disk group FROM a host
- Shutdown VSAN cluster
- Remove a host FROM VSAN VMware KB: Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x/6.0 hosts
- Note: Run this command to permanently turn off the service :
- Remove failure Cache disk from vSAN
- Removing Host FROM VSAN
- Add host to VSAN cluster
- RVC observer monitor
- I created a folder on my VSAN datastore, but how do I delete it
- Unmounting an NFS datastore fails with the error: Sysinfo set operation
- Performance test
- HP
- Oracle RAC setting
- vid svid
- HDD claim to SSD
VSAN ready
VMware vSAN Best Practices
- Always make sure that the hardware you are using is listed in the VMware Compatibility Guide. You do not want to be in a scenario where you don’t get support because you aren’t running on the right hardware.
- In addition to hardware, also make sure that you run the supported software, driver and firmware versions in your cluster. You can again refer to the VMware Compatibility Guide to find out what those versions are.
- Keep your environment updated. This is not only a vSAN best practice but a general rule of thumb. You should always keep your environment patched.
- Similarly configured and sized ESXi hosts should be used to form a vSAN cluster. This should be done to ensure that there is an even balance of virtual machine storage components across all the hosts and their disks.
- Design your cluster for growth. This is very important. Although scaling up and scaling out a vSAN cluster isn’t difficult, you should always plan for growth. For eg., If you know that you will be adding 3 additional capacity drives to your hosts in the next year, then make sure you select a Cache drive that would be able to serve your eventual capacity tier.
- Having minimum 4 nodes in your vSAN cluster would give you higher availability when compared to running a 3 node cluster(supported configuration).
- When you size your vSAN cluster to support the number of VMs that you want to deploy, also make sure to size the ESXi hosts appropriately. You don’t want to end up in a scenario where you have TBs of space available, but not enough memory to support additional VMs.
- Always enable vSphere HA. Keep in mind, that to enable vSAN, you will have to disable HA. Remember to turn it back on.
- To avoid inconsistent performance, make sure not to use hybrid and all-flash disk groups as part of the same cluster.
- Ensure that there are enough hosts in the cluster to accommodate the desired number of failures to tolerate. Use the formula 2n + 1, where n is the number of failures to tolerate. So if you want to tolerate 2 failures, you should at least 5 hosts in your cluster. The maximum number of failures that vSAN can tolerate is 3 so you will need a minimum of 7 hosts in that scenario.
- For Hybrid configurations, you can use both 1G and 10G NICs on your host. In case of 1G NICs, vSAN needs a dedicated NIC.
- For All-Flash configurations, vSAN needs at least 10GB of network connectivity between hosts.
- If you use NIC-Teaming, then vSAN will only use it for high availability, and not for the aggregated bandwidth.
- vSAN works with or without Jumbo Frames, without a considerable performance impact. So follow your standard practices for Jumbo Frames implementation.
- Multiple smaller disk groups are recommended vs single larger disk group as multiple smaller groups allow for a higher cache to capacity ratio, thus leading to an accelerated performance of virtual machines.
- Consider the cost parameter when buying a PCIe device for the cache tier. Check if you can get the required performance using SSD devices in multiple smaller disk groups.
- The Cache tier should be sized to be at least 10% of the capacity consumed by virtual machines.
- Allow for 30% slack space when designing capacity, this is because vSAN will start automatic rebalancing when a disk reaches 80% threshold which generates rebuild traffic on the cluster.
- If virtual machine snapshots are going to be used heavily in hybrid configurations, then increase the minimum cache to capacity ratio from 10% to 15%.
- Multiple storage I/O controllers per host can help eliminate single points of failures and also improve performance.
- Choose storage I/O controllers that have as large a queue depth as possible. Having larger queue depths will improve virtual machine I/O performance.
- Remember to account for vSAN filesystem overhead when sizing the capacity tier.
- All VMs deployed on vSAN will be thin provisioned, so keep an eye on the capacity tier.
- vSAN adds around 10% overhead on host CPU, so keep that in mind when sizing your ESXi hosts.
- When deploying large vSAN clusters, use fault domains as a way to avoid single rack failures impacting all replicas belonging to a virtual machine.
memory consumed
https://kb.vmware.com/s/article/2113954 Example 4: Three disk groups per host, dedup configuration: Formula: HOST_FOOTPRINT + NumDiskGroups * (DISKGROUP_FIXED_FOOTPRINT + DISKGROUP_SCALABLE_FOOTPRINT + CacheSize * CACHE_DISK_FOOTPRINT + NumCapacityDisks * CAPACITY_DISK_FOOTPRINT) Example: 7100 + 3 * (1360 + 120 + 1310 + 600 * 20 + 3 * 160) = 52910 MB
This procedure must be repeated on all participating vSAN nodes. LAG configuration can also be interrogated using command line, i.e. esxcli
[root@vSAN-ESXi02-2:~] esxcli network vswitch dvs vmware lacp config get DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan $ ansible MLS-VSAN01 -m shell -a 'esxcli network vswitch dvs vmware lacp config get' 10.97.2.2 | SUCCESS | rc=0 >> DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan 10.97.2.3 | SUCCESS | rc=0 >> DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan 10.97.2.5 | SUCCESS | rc=0 >> DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan 10.97.2.4 | SUCCESS | rc=0 >> DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan 10.97.2.1 | SUCCESS | rc=0 >> DVS Name LAG Name LAG ID NICs Enabled Mode Load balance ---------- -------- -------- ------------- ------- ------ -------------------------- VM-DSwitch lag1 38482354 vmnic2,vmnic3 true Active Src and dst ip, port, vlan
update HCL
http://partnerweb.vmware.com/service/vsan/all.json.gz
VSAN how to
http://www.davidhill.co/2013/08/vmware-vsan-how-to-configure/
http://www.yellow-bricks.com/2013/09/16/frequently-asked-questions-virtual-san-vsan/
Docker Container for the Ruby vSphere Console (RVC)
docker pull lamw/rvc docker run --rm -it lamw/rvc docker run --rm -it -p 80:8010 lamw/rvc
rvm install 1.9.2 gem install rvc gem install ffi
http://www.virtuallyghetto.com/2015/11/docker-container-for-the-ruby-vsphere-console-rvc.html http://tsmith.co/2014/vsan-installation/
RVC how to
- 100.11.4.1
docker run --rm -it -p 8011:8010 lamw/rvc rvc administrator@vsphere.local@100.12.1.10 vsan.observer --run-webserver --force XX/XX/TEST_Cluster
Enter URL: https://100.11.4.1:8011
Step 1: Enable the VSAN service on a VMKernel port.
* Create your virtual switches, and create at least 1 VMKernel port that will be * vmk-vsanused for VSAN traffic. * Be sure to keep the Switch names the same across hosts! * Edit the VMKernel port and check the box for Virtual SAN Traffic. * Save the port settings, repeat for each host. * Time Saver: Use host profiles!
Step 2: Enable VSAN on the Cluster
- Select your cluster, and choose the Manage tab, and the select General under Virtual SAN.
- Edit, and check the box to Turn ON Virtual SAN.
- Choose your setting for “Add disks to storage”
- Manual – You will select each disk that will be a part of the Virtual SAN
- Automatic – VSAN will select all eligible disks for you and add them
- Click OK
Step 3: Add Disks to Disk Groups
Since I chose Manual mode, I will need to add my disks into Disk Groups. A Disk Group is a collection of 1 SSD, and multiple HDD drives. You can have multiple disk groups per host if capacity allows.
- Still in Virtual SAN settings under the cluster, select the Disk Management section.
- Click on the Claim Disks button, and select the drives for use in Virtual SAN.
Alternatively, select each host, and manually create Disk Groups per host.
Step 4: Start building Virtual Machines!
Yes, it is that easy. You now have a datastore called vsanDatastore.
My DL360 G6 server can also access this datastore, since I enabled VSAN on the VMKernel port groups, even though it’s not providing any resources to the VSAN cluster.
Supported network topologies for VSAN stretched cluster
http://cormachogan.com/2015/09/10/supported-network-topologies-for-vsan-stretched-cluster/
故障域构造
您必须至少定义三个故障域，每个故障域可能包含一个或多个主机。故障域定义必须确认可能代表潜在故障域的物理硬件构造，如单个计算机柜。 如果可能，请使用至少四个故障域。使用三个故障域时，不允许使用特定撤出模式，Virtual SAN 也无法在故障发生后重新保护数据。在这种情况下，您需要一个使用三域配置时无法提供的备用容量故障域用于重新构建。 如果启用故障域，Virtual SAN 将根据故障域而不是单个主机应用活动虚拟机存储策略。 根据计划分配给虚拟机的存储策略中规定的“允许的故障数”属性，计算群集中的故障域数目。 number of fault domains = 2 * number of failures to tolerate + 1 如果主机不是故障域成员，Virtual SAN 会将其解析为单独的域。 http://cormachogan.com/2015/04/20/vsan-6-0-part-8-fault-domains/ If rack 1 fails (containing host 1), do I still have a full copy of the data? The answer is Yes. If rack 2 fails (containing host 2), do I still have a full copy of the data? The answer is Yes. If rack 3 fails (containing hosts 3 & 4), do I still have a full copy of the data? The answer is still Yes.
About VSAN capacity and VMDK placement
http://www.viktorious.nl/2013/10/10/vsan-capacity-vmdk-placement/
http://cormachogan.com/2013/09/04/vsan-part-4-understanding-objects-and-components/
删除磁盘原有分区
使用partedUtil工具 首先确认哪些磁盘被确认出来。
esxcli storage core device list # 获取 ID partedUtil get /vmfs/xxx/xxx 1 2048 3xxx 0 0 # 删除分区 partedUtil delete /vmfs/xxx/xxx 1
[root@ESXP-D15U13-INI-04:~] esxcli storage core device list | grep naa.518564421 39de002 naa.51856442139de002 Display Name: Local HUAWEI Disk (naa.51856442139de002) Devfs Path: /vmfs/devices/disks/naa.51856442139de002 [root@ESXP-D15U13-INI-04:~] [root@ESXP-D15U13-INI-04:~] partedUtil get /vmfs/devices/disks/naa.51856442139de 002 Error: The primary GPT table on '/dev/disks/naa.51856442139de002' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (6251233968) AlternateLBA (6251233967) LastUsableLBA (6251233934) 389121 255 63 6251233968 1 64 8191 0 128 5 8224 520191 0 0 6 520224 1032191 0 0 7 1032224 1257471 0 0 8 1257504 1843199 0 0 9 1843200 7086079 0 0 2 7086080 15472639 0 0 [root@ESXP-D15U13-INI-04:~] partedUtil delete /vmfs/devices/disks/naa.5185644213 9de002 1 Error: The primary GPT table on '/dev/disks/naa.51856442139de002' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (6251233968) AlternateLBA (6251233967) LastUsableLBA (6251233934) [root@ESXP-D15U13-INI-04:~] partedUtil get /vmfs/devices/disks/naa.51856442139de 002 389121 255 63 6251233968 5 8224 520191 0 0 6 520224 1032191 0 0 7 1032224 1257471 0 0 8 1257504 1843199 0 0 9 1843200 7086079 0 0 2 7086080 15472639 0 0 [root@ESXP-D15U13-INI-04:~] partedUtil get /vmfs/devices/disks/naa.51856442139de002 389121 255 63 6251233968 [root@ESXP-D15U16-INI-05:~] esxcli storage core device list | grep naa.51856442139ba002 naa.51856442139ba002 Display Name: Local HUAWEI Disk (naa.51856442139ba002) Devfs Path: /vmfs/devices/disks/naa.51856442139ba002 [root@ESXP-D15U16-INI-05:~] [root@ESXP-D15U16-INI-05:~] partedUtil get /vmfs/devices/disks/naa.51856442139ba002 Error: The primary GPT table on '/dev/disks/naa.51856442139ba002' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (6251233968) AlternateLBA (6251233967) LastUsableLBA (6251233934) 389121 255 63 6251233968 1 64 8191 0 128 5 8224 520191 0 0 6 520224 1032191 0 0 7 1032224 1257471 0 0 8 1257504 1843199 0 0 9 1843200 7086079 0 0 2 7086080 15472639 0 0 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 5 Error: The primary GPT table on '/dev/disks/naa.51856442139ba002' is OK, but secondary is corrupt. Fix secondary table? This will move secondary at the end in case it is not at the end already. It will also set LastUsableLBA to use all the space at the end. diskSize (6251233968) AlternateLBA (6251233967) LastUsableLBA (6251233934) [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 6 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 7 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 8 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 9 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 2 [root@ESXP-D15U16-INI-05:~] partedUtil delete /vmfs/devices/disks/naa.51856442139ba002 1 [root@ESXP-D15U16-INI-05:~] partedUtil get /vmfs/devices/disks/naa.51856442139ba002 389121 255 63 6251233968 Well done!
MTU setting
Dell switches a MTU 9000 is actually (9 * 1024) = 9216. Ugh, so now I have set
- 9000 MTU on the VSAN VMkernel on the vSphere host.
9000 MTU on the VSAN VLAN on the Dell PowerConnect
9216 MTU on the VSAN Physical Interface the Dell PowerConnect
BAM! The VSAN datastore is rocking and I can now write to it. However, this scenario posed some reflection on how MTU actually impacts VSAN. I did some further testing and came up with the follow conclusions:
- If any host in a VSAN cluster has a mismatched MTU size, NOTHING can write to the VSAN datastore. Even if one host with the wrong MTU is set then it will prevent VSAN from working.
Even with mismatched MTU’s when one verifies the Network Status (vCenter > Virtual SAN > General) it will show Normal. However, this doesn’t verify MTU, just IP connectivity. To test if the MTU is correct then use the MTU of the VSAN VMkernel’s MTU size and issue a vmkping -s <VSANvmkernel_mtu_setting> <Other-VSANvmkernel-interfaces-in-cluster>
In conclusion, would I advise configuring Jumbo Frames with VSAN? No. Unless you’re the type who prefers all risk and no reward…
http://flcloudlabs.com/vsan-and-mtu/
https://communities.vmware.com/message/2455828
请遵循以下准则
Virtual SAN 需要一个专用 1 Gb 网络。最佳做法是使用 10 Gb 网络。 在每台主机上，可至少将一个物理 1 Gb 以太网网卡专用于 Virtual SAN。还可以置备另外一个物理网卡作为故障切换网卡。 可以在每个主机上使用 vSphere 标准交换机，或者可以将环境配置为使用 vSphere Distributed Switch。 为每个用于 Virtual SAN 的网络配置一个已激活 Virtual SAN 端口属性的 VMkernel 端口组。 为每个端口组使用相同的 Virtual SAN 网络标签，并确保这些标签在所有主机上一致。 使用巨帧以实现最佳性能。 Virtual SAN 支持 IP 哈希负载平衡，但无法保证所有配置的性能都有提升。当除 Virtual SAN 以外还有众多 IP 哈希使用者时，可以从 IP 哈希中获益。这种情况下，IP 哈希将执行负载平衡。但是，如果 Virtual SAN 是唯一的使用者，则可能看不到什么变化。这一规则特别适用于 1G 环境。例如，如果您将四个设置了 IP 哈希的 1G 物理适配器用于 Virtual SAN，实际能够使用的可能不超过 1G。对于我们目前支持的所有网卡成组策略来说，这一点也同样适用。有关网卡成组的详细信息，请参见《vSphere NetworkingvSphere 网络》指南的“网络连接策略”部分。 Virtual SAN 不支持同一子网上有多个 VMkernel 适配器用于负载平衡。但是支持多个 VMkernel 适配器位于不同网络的情况，如 VLAN 或单独的物理结构。 您应该将所有参与 Virtual SAN 的主机连接到已启用多播（IGMP 侦听）的单个 L2 网络。如果参与 Virtual SAN 的主机跨越多个交换机乃至 L3 边界，必须确保将网络正确配置为启用多播连接。如果您的网络环境需要，或者如果您在同一 L2 网络中运行多个 Virtual SAN 群集，则可以更改多播地址的默认设置。
http://www.tomsitpro.com/articles/essential-virtual-san-vsan-book-excerpt,2-888.html
Network Misconfiguration Status in a Virtual SAN Cluster
After you enable Virtual SAN on a cluster, the datastore is not assembled correctly because of a detected network misconfiguration. Problem
After you enable Virtual SAN on a cluster, on the Summary tab for the cluster the Network Status for Virtual SAN appears as Misconfiguration detected. Cause
One or more members of the cluster cannot communicate because of either of the following reasons:
- A host in the cluster does not have a VMkernel adapter for Virtual SAN.
- The hosts cannot connect each other in the network.
- Multicast is not enabled on the physical switch.
How do you resolve it? Well, a number of our VSAN beta customers discussed some options on the community, and these were the recommendations:
- Option 1 – Disable IGMP Snooping. Now this will allow all multicast traffic through, but if the only traffic is VSAN, then this should be a negligible amount of traffic and should be safe to use.
- Option 2 – Configure IGMP snooping querier. If there is other multicast traffic and you are concerned that disabling IGMP snooping might open the network up to a flood of multicast traffic, then this is a preferred option
Changing the multicast address used for a VMware Virtual SAN Cluster (2075451)
Purpose
This article provides steps to change the multicast address for each VMware Virtual SAN cluster. If there are multiple Virtual SAN clusters on the same Layer 2 network, each host receives all multicast messages. In order to reduce the amount of multicast traffic for each VSAN cluster, it is necessary to change the multicast address for each VMware Virtual SAN cluster.
Warning: If you change the multicast address on an active Virtual SAN cluster, it can lead to network partitioning until all of the ESXi hosts in the cluster are on the same multicast network. It is recommended to organize downtime before making this change.
Resolution
In order to change the multicast address for VMware Virtual SAN, perform these steps on each ESXi host within the Virtual SAN Cluster.
To change the multicast address on an ESXi 5.5/6.0 host configured for Virtual SAN:
- Open an SSH connection to the ESXi host and log in as root. For more information, see Using ESXi Shell in ESXi 5.x (2004746). Identify the VMkernel interface configured for Virtual SAN. To identify the VMkernel interface, run this command on the ESXi hosts: esxcli vsan network list You see output similar to: Interface
VmkNic Name: vmk1 IP Protocol: IPv4 Interface UUID: 28b52f53-69c1-c193-eabe-005056885a94 Agent Group Multicast Address: 224.2.3.4 Agent Group Multicast Port: 23451 Master Group Multicast Address: 224.1.2.3 Master Group Multicast Port: 12345 Multicast TTL: 5
esxcli vsan network ipv4 set -i <vmkernel interface> -d <multicast agent group address> -u <multicast master group address> For example, to set the Master Group Multicast address to 224.2.3.5 and the Agent Group Multicast Address to 224.2.3.6 , run this command on each ESXi host for this particular VSAN cluster: esxcli vsan network ipv4 set -i vmk1 -d 224.2.3.6 -u 224.2.3.5
Using tcpdump-uw to collect packet traces to troubleshoot network issues
Usage: tcpdump-uw tcpdump-uw -i = interface -n = no IP or Port name resolution -s0 = Collect entire packet -t = no timestamp -c = number of frames to capture
- tcpdump-uw -i vmk1 -n -s0 -t -c 20 udp port 12345
- tcpdump-uw -i vmk1 -n -s0 -t -c 20 udp port 23451
- tcpdump-uw -i vmk2 udp port 23451 -v
Generate Multicast traffic
nc -uz <destination-ip> <destination-port>
Monitor VSAN VMKernel Port network traffic
esxcli network ip connection list
Troubleshooting
Delete an uplink from a DVPort on a DVSwitch
esxcfg-vswitch -l esxcfg-vswitch -Q vmnic1 -V 20 vDS-DCGZLG02-P-INT-vSAN01
Gather special VM logs
[root@txvsan12:~] vm-support --listvms /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/e850d55d-88b8-42b3-4d82-e4434b91e7ae/TXLA100240_iuap.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/4d83c25d-20b2-67f0-f6cf-e4434b91e7ae/txld11210.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/aa4de65d-7404-f7b6-628e-e4434b91f114/BIRAC-txld11294.vmx (Registered) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/896ade5d-089b-aadb-2be8-e4434b91e7e6/txld10072.vmx (Registered) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/fdd1dd5d-1ed2-44be-8bdc-e4434b91e7e6/TXLD100132.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/1668ca5d-9416-88f7-cd1e-e4434b91f114/VMware vCenter Server Appliance-6.7-u3a.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/a0c6d85d-567e-8c83-0d30-e4434b91e5e2/TXWA11010.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/6b38dd5d-7e61-b956-09c0-e4434b91e7e6/txld11273.vmx (Registered) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/6719e65d-3eef-c223-b785-e4434b91f114/TXWA11003.vmx (Running) /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/23acf45d-8277-aad8-83af-e4434b91e7ae/TXLA100212_NC65.vmx (Running) [root@txvsan12:/vmfs/volumes/0b2f831a-ab08744e/192.168.201.12] vm-support --vm /vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/6b38dd5d-7e61-b956-09c0-e4434b91e7e6/txld11273.vmx -w /vmfs/volumes/FAS252_VSANLOG_DATA
iperf test
/usr/lib/vmware/vsan/bin/iperf.copy -s [IPERF-SERVER-IP] /usr/lib/vmware/vsan/bin/iperf.copy -c [IPERF-Client-IP]
Monitor VM performance status
rvc> vsan.vm_perf_stats ~/vms/Oracle --interval 10 --show-objects *通过了解衡量指标之间的关系，以下计算可能会有所帮助： IOPS = (MBps 吞吐量/每 IO KB) * 1024 MBps = (IOPS * 每 IO KB)/1024
show vds switch
/bin/net-dvs -l | grep -i mtu -B 15
esxcli network vswitch dvs vmware list
[root@localhost:~] esxcli network vswitch dvs vmware list
T-DSwitch
Name: T-DSwitch
VDS ID: 50 02 eb 73 02 f6 36 77-67 07 2c 40 34 0b 7c 99
Class: cswitch
Num Ports: 5106
Used Ports: 1
Configured Ports: 512
MTU: 1500
CDP Status: listen
Beacon Timeout: -1
Uplinks:
VMware Branded: true
DVPort:
Client:
DVPortgroup ID: dvportgroup-2990
In Use: false
Port ID: 64
Client:
DVPortgroup ID: dvportgroup-2990
In Use: false
Port ID: 65
find disk
vdq -i | grep naa | awk -F"\"" '{ print $2 }' | tee | sort > /tmp/1
esxcli storage core device list | grep ^naa. | sort > /tmp/2
diff -u /tmp/1 /tmp/2
export VDS configuration
Note: For vCenter Server 6.x, right-click the distributed switch and click Settings > Export Configuration.
Collect logs
- RVC log
rvc -c "vsan.support_information 1" -c "quit" administrator@vsphere.local@localhost> /tmp/rvc1.log
vsan info
rvc > vsan.check_state * 此命令执行以下 3 项检查： 检查不可访问的 Virtual SAN 对象 检查无效/不可访问的虚拟机 检查 VC/hostd/vmx 不同步的虚拟机 rvc> vsan.cluster_info 0 rvc> vsan.whatif_host_failures
手动平衡
rvc> vsan.proactive_rebalance -s 0 rvc> vsan.proactive_rebalance_info 0
vsan check
# Disk quere depth esxcfg-info -s | grep "==+SCSI Interface" | -A 18
$ esxcli vsan health cluster list $ esxcli vsan debug controller list
vSAN 6.6:
vsan.cluster.unicastagent clear
vsan.debug.controller list
vsan.debug.disk list
vsan.debug.disk.summary get
vsan.debug.limit get
vsan.debug.object.health.summary get
vsan.debug.object list
vsan.debug.resync list
vsan.debug.resync.summary get
vsan.debug.vmdk list
vsan.resync.bandwidth get
vsan.resync.throttle get
vsan.resync.throttle setrvc > vsan.check_state rvc > vsan.vm_object_info rvc > vsan.check_lists 0
Disk/Storage
esxcli core storage device list esxcli storage core device list –d naa.600508b1001c626dcb42716218d73319 # vdp -iH
此命令对确定以下有关磁盘的信息非常有用
- 磁盘上的组件数量（对于 SSD，始终为 0）
- 磁盘总容量
- 消耗的磁盘百分比
- 磁盘的健康状况
- 磁盘格式的版本
/ie-vcsa-03.ie.local/vsan-dc/computers> vsan.disks_stats 0
SSD Permanent disk failure
+----------------------+-------------+-------+------+------------+---------+----------+-------------+ | naa.500003965c8995f8 | 100.12.1.22 | SSD | 0 | 372.61 GB | 70.00 % | 0.00 % | FAILED (v2) | | naa.5000c5008e9caa53 | 100.12.1.22 | MD | 18 | 1106.62 GB | 72.26 % | 3.55 % | FAILED (v2) | | naa.5000c5008e9fa32f | 100.12.1.22 | MD | 24 | 1106.62 GB | 72.17 % | 13.38 % | FAILED (v2) | | naa.5000c5008e9ec9f7 | 100.12.1.22 | MD | 19 | 1106.62 GB | 51.11 % | 4.98 % | FAILED (v2) | | naa.5000c5008e9ee5ef | 100.12.1.22 | MD | 20 | 1106.62 GB | 59.04 % | 4.12 % | FAILED (v2) | +----------------------+-------------+-------+------+------------+---------+----------+-------------+ | naa.500003965c899624 | 100.12.1.22 | SSD | 0 | 372.61 GB | 70.00 % | 0.00 % | FAILED (v2) | | naa.5000c5008eaa0ec3 | 100.12.1.22 | MD | 18 | 1106.62 GB | 67.22 % | 8.11 % | FAILED (v2) | | naa.5000c5008ea9d7ff | 100.12.1.22 | MD | 22 | 1106.62 GB | 48.36 % | 21.94 % | FAILED (v2) | | naa.5000c5008e9f98f3 | 100.12.1.22 | MD | 17 | 1106.62 GB | 65.91 % | 13.23 % | FAILED (v2) | | naa.5000c5008e9cb727 | 100.12.1.22 | MD | 23 | 1106.62 GB | 49.88 % | 8.32 % | FAILED (v2) | +----------------------+-------------+-------+------+------------+---------+----------+-------------+
[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device list | grep 5f8
naa.500003965c8995f8
Display Name: TOSHIBA Serial Attached SCSI Disk (naa.500003965c8995f8)
Devfs Path: /vmfs/devices/disks/naa.500003965c8995f8
Other UIDs: vml.0200000000500003965c8995f850583032534d- show disk status
[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device smart get naa.500003965c8995f8
Error: Unknown command or namespace storage core device smart get naa.500003965c8995f8
[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device smart get -d naa.500003965c8995f8
Parameter Value Threshold Worst
---------------------------- ----- --------- -----
Health Status OK N/A N/A
Media Wearout Indicator N/A N/A N/A
Write Error Count 0 N/A N/A
Read Error Count 0 N/A N/A
Power-on Hours N/A N/A N/A
Power Cycle Count 0 N/A N/A
Reallocated Sector Count N/A N/A N/A
Raw Read Error Rate N/A N/A N/A
Drive Temperature 29 N/A N/A
Driver Rated Max Temperature N/A N/A N/A
Write Sectors TOT Count N/A N/A N/A
Read Sectors TOT Count N/A N/A N/A
Initial Bad Block Count N/A N/A N/A
[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000]- show cluster status
[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli vsan cluster get Cluster Information Enabled: true Current Local Time: 2017-04-10T07:30:09Z Local Node UUID: 562e59c2-4667-26fe-4a47-246e9601c430 Local Node Type: NORMAL Local Node State: AGENT Local Node Health State: HEALTHY Sub-Cluster Master UUID: 5657370c-efa5-6206-0af9-246e9601d388 Sub-Cluster Backup UUID: 562e5da5-7841-732c-05a3-246e9601cd58 Sub-Cluster UUID: 52541b50-5a76-7701-6f36-d587dbae5691 Sub-Cluster Membership Entry Revision: 109 Sub-Cluster Member Count: 10 Sub-Cluster Member UUIDs: 5657370c-efa5-6206-0af9-246e9601d388, 562e5da5-7841-732c-05a3-246e9601cd58, 562e6190-eac3-0a40-dbc0-246e9601cee0, 5639e982-1f2e-840c-57ae-246e9601d120, 562e4827-7827-cee2-500f-246e9601c348, 5659155e-bc17-5972-59cf-246e9601c918, 57c99c2f-3ddc-ccc6-5499-ac6175745062, 57c9a7c9-0878-6eb2-365d-ac6175745060, 5639eddd-2c0f-c7d3-2c3d-246e9601d6a8, 562e59c2-4667-26fe-4a47-246e9601c430 Sub-Cluster Membership UUID: c55d9c56-1732-6bdb-787c-246e9601d388
http://www.mrvsan.com/using-the-error-injection-command-to-test-a-disk-failure/
There is no more space for virtual disk
http://virtualinator.com/2016/04/page/2/
http://cormachogan.com/2016/04/19/recovering-full-vsan-datastore-scenario/
host cannot communicate with all other nodes in the virtual san enabled cluster
https://communities.vmware.com/thread/495882?start=15&tstart=0 http://www.tomsitpro.com/articles/essential-virtual-san-vsan-book-excerpt,2-888.html
- 当我随便找了个节点，执行如下命令重启相关服务之后就OK了。
- [root@ESXi03:~] services.sh restart
Registration/unregistration of a VASA vendor provider on a Virtual SAN host fails
Issues information is not available at this time.
- i had same problem, but after reinstalling web client it solved !
https://communities.vmware.com/thread/500032?start=0&tstart=0
java.lang.RuntimeException
jvm 1 | 2015/12/18 16:53:02 | java.lang.RuntimeException: com.vmware.vim.vmomi.core.exception.UnmarshallException: local name and field name mismatch: 'namespace' - 'capabilityMetadataPerCategory'
VMware CASE sulotions AS following? http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113435
VM虚拟机无法ping通网关
现象
- 虚拟机网络无法跟网关相通。
- 但跟同网段的其它虚拟机能通。其它虚拟机也能与它互通。
解决方法
- 删除当前网卡，重新创建一张网卡。
* esxcli network diag ping -I vmk1 -H vSAN-IP
H3C 6800 交换机端口处理down状态
现象
- 从ESXi主机上查看esxcli network nic list可以看到端口down。
解决方法
- 在H3C 6800交换机上把对应的端口shutdown之后再 no shut就OK了。
http://www.virten.net/2014/01/manage-vsan-with-rvc-part-4-troubleshooting/
No space left on device
http://www.m80arm.co.uk/2013/12/ha-issues-with-vsan-beta-refresh.html
stat -f /vsanDatastore
* The no space left on the device error confused me so in order to rectify the issue I tried the following:
- Increases the size of the VMDK used for the ESXi installation from 2GB to 5GB
- Inflated the ESXi installation VMDK so it was Thick Eager Zero'd just in case this was causing any strange issues
- Rebuild the nested environment manually (not using the .ova supplied by William Lam)
System logs are stored on non-persistent storage
http://cormachogan.com/2015/02/24/vsan-considerations-when-booting-from-usbsd/ To verify the location:
Browse to the host in the vSphere Web Client navigator.
- Click the Manage tab, then click Settings.
- Under System, click Advanced System Settings.
- Ensure that Syslog.global.logDir points to a persistent location.
If the field Syslog.global.logDir is empty or points to a scratch partition, make sure that the field ScratchConfig.CurrentScratchLocation shows a location on persistent storage.
Note: You must reboot the host for the changes to take effect.
Note: To log to a datastore, the Syslog.global.logDir entry should be in the format of [Datastorename]/foldername. To log to the scratch partition set in the ScratchConfig.CurrentScratchLocation, the format is blank or []/foldername.
The Solution is
- create NFS server (VM) and disk.
- mount NFS on ESXi cluster
- setting Syslog.global.logDir as [NFS4VSAN-LOG]/ESXi-IP-ADDR-DIR
How to Delete VSAN Datastore
http://www.vladan.fr/how-to-delete-vsan-datastore/
- Evacuate all the VMs out of the VSAN datastore.
- Turn OFF VMware HA.
- Delete all VSAN disk groups Individually
Once done (for each of the hosts participating in the VSAN cluster), the local disks are available to be re-used….
Deactivate VSAN cluster. At the Cluster level > Manage > Virtual SAN > General
- Re-enable VMware HA.
VSAN upload problem
2015-11-19T19:05:08.31Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] In do_GET: ('127.0.0.1', 36677)
2015-11-19T19:05:08.31Z WARNING vsan-health[Thread-7] [VsanHealthServer::do_GET] do_GET: isStringResponse = True
2015-11-19T19:05:08.32Z INFO vsan-health[Thread-7] [VsanHealthServer::log_message] ('127.0.0.1', 36677) - - "GET /vsanHealth/health HTTP/1.1" 200 -
2015-11-19T19:05:08.32Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] Done do_Get: ('127.0.0.1', 36677) (took 0.0)
2015-11-19T19:05:35.507Z WARNING vsan-health[Thread-1] [VsanPyVmomiProfiler::InvokeMethod] Invoke: mo=ServiceInstance, info=CurrentTime
2015-11-19T19:05:38.475Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] In do_GET: ('127.0.0.1', 36677)
2015-11-19T19:05:38.475Z WARNING vsan-health[Thread-7] [VsanHealthServer::do_GET] do_GET: isStringResponse = True
2015-11-19T19:05:38.475Z INFO vsan-health[Thread-7] [VsanHealthServer::log_message] ('127.0.0.1', 36677) - - "GET /vsanHealth/health HTTP/1.1" 200 -
2015-11-19T19:05:38.476Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] Done do_Get: ('127.0.0.1', 36677) (took 0.0)
Remove a disk group FROM a host
Entering Maintenance Mode is done by selecting the correct ESXi host and then clicking on the maintenance mode icon in the Disk Management section on Virtual SAN in the vSphere web client (third icon from the left):
Shutdown VSAN cluster
To recap, if shutting down the whole of the VSAN cluster, use maintenance mode for the hosts, and do not move VMs or migrate any data.
Remove a host FROM VSAN VMware KB: Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x/6.0 hosts
If you are not using VSAN:
- If VSAN is not used in your ESXi host, run this command to stop the vsantraces:
#/etc/init.d/vsantraced stop
Perform a Refresh for Storage.
Unmount the datastore.
Run this command to start the service:
#/etc/init.d/vsantraced start
[root@ESXi-B02:~] esxcli storage filesystem list
Mount Point Volume Name UUID Mounted Type Size Free
---------------------------------------------------- ----------------- -------------------------------------- ------- ------ ------------- -------------
/vmfs/volumes/562a5aa6-f20c535a-1171-246e9601d388 ESXi-B02_SASRAID5 562a5aa6-f20c535a-1171-246e9601d388 true VMFS-5 5997921828864 5717964619776
/vmfs/volumes/562a4405-a83996b6-4fa5-246e9601d388 562a4405-a83996b6-4fa5-246e9601d388 true vfat 299712512 83386368
/vmfs/volumes/93df10e1-58e61d4c-8c05-c5b28ac43c65 93df10e1-58e61d4c-8c05-c5b28ac43c65 true vfat 261853184 79302656
/vmfs/volumes/5f7ee1e6-998a02ee-20c2-d9c3f71404fd 5f7ee1e6-998a02ee-20c2-d9c3f71404fd true vfat 261853184 92418048
/vmfs/volumes/vsan:52541b505a767701-6f36d587dbae5691 vsanDatastore vsan:52541b505a767701-6f36d587dbae5691 true vsan 0 0
[root@ESXi-B02:~] esxcli storage filesystem unmount -u vsan:52541b505a767701-6f36d587dbae5691
No volume with uuid 'vsan:52541b505a767701-6f36d587dbae5691' was found
[root@ESXi-B02:~] grep -n -i vsan /etc/vmware/esx.conf
351:/adv/VSAN/LicensedFeatures = "allflash,stretchedcluster"
366:/firewall/services/vsanvp/allowedall = "true"
367:/firewall/services/vsanvp/enabled = "true"
457:/vsan/faultDomainVersion = "2"
458:/vsan/faultDomainName = ""
459:/vsan/autoClaimStorage = "false"
460:/vsan/enabled = "true"
461:/vsan/subClusterUuid = "52541b50-5a76-7701-6f36-d587dbae5691"
462:/vsan/datastoreName = "vsanDatastore"
463:/vsan/checksumEnabled = "false"
[root@ESXi-B01:~] grep -n -i vsan /etc/vmware/esx.conf
90:/firewall/services/vsanvp/enabled = "true"
91:/firewall/services/vsanvp/allowedall = "true"
112:/adv/VSAN/LicensedFeatures = "allflash,stretchedcluster"
531:/vsan/faultDomainVersion = "2"
532:/vsan/faultDomainName = ""
533:/vsan/autoClaimStorage = "false"
534:/vsan/enabled = "false"
535:/vsan/subClusterUuid = "52541b50-5a76-7701-6f36-d587dbae5691"
536:/vsan/datastoreName = "vsanDatastore"
537:/vsan/checksumEnabled = "false"
In ESXi-B02 the /vsan/enabled = "True" !!!!
So change "True" to "false", then restart services, but didn't work.http://ambitech.blogspot.in/2015/07/unable-to-add-standalone-host-already.html
So here were few hosts whose IP addresses were messed up that what it was in the host file (or in your case it might be DNS) and i corrected them via vCLI. After that i tried to add them but it was always throwing this error saying the ip already exists even though i was trying to add it using the hostname. Then i just checked the vCenter server database---->Table---->dbo.vpx.host. ---Right click and select 1000 rows and there they were, the ip address, the hostname. These are not in the vcenter anymore but they are in the vcenter server database. I wanted to delete these entries here and then try to re add but my colleague ravi suggested an alternative. I cleaned off the hostfile for these 2 clusters where these hosts (in maintenance mode) were there and then remove the clusters in which these hosts were supposed to be there. Automagically the stale entries in the database were gone. Now I was able to peacefully add the hosts back to the vcenter and the clusters in which they reside. you might have to remove the entries for these hosts in the DNS and then do the same to get around it.
Note: Run this command to permanently turn off the service :
- chkconfig vsantraced off
If you are using VSAN:
If you are using VSAN on the ESXi host, run this command to change the VSAN trace location:
# esxcli vsan trace set -p datastore_name
Unmount the datastore.
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2069171
Remove failure Cache disk from vSAN
# to show your disks. esxcli vsan storage list OR vdq -i # Find each orphaned disks VSAN UUID by matching the Device ID. esxcli vsan storage remove -u [VSAN UUID] # for each disk. # DONE! # Now go back into web client and recreate your disk group.
Removing Host FROM VSAN
method 1
- Place host in maintenance mode
- Delete disk group when “maintenance mode” is completed
- Move host out of the cluster
- Remove the VSAN VMkernel (not a requirement, but I prefer to clean things up)
http://www.yellow-bricks.com/2014/01/14/remove-host-virtual-san-cluster/
method 2
- Removing a VMware Virtual SAN-enabled cluster and detaching the VMware ESXi host from the cluster
- Log in to each ESXi host's console, see Using ESXi Shell in ESXi 5.x and 6.0 (2004746).
- Place the host in maintenance mode and select the full data migration option.
- To obtain information about the VSAN storage, run this command:
- # esxcli vsan storage list
- To disable automated mode, run this command:
- # esxcli vsan storage automode set --enabled false
- To remove the specific SSD disk used in VSAN configuration, run this command:
- # esxcli vsan storage remove -s [SSD-DEVICE-ID]
- To obtain information about the VSAN cluster for the ESXi host, run this command:
- # esxcli vsan cluster get
- To disconnect the host from the VSAN cluster, run this command:
- # esxcli vsan cluster leave
- To find any left-over configuration, run this command:
- # esxcli vsan cluster get
Note: The command should not return anything if VSAN cluster information gets removed. You can also verify that the vsanDatastore will disappear from the Configuration > Storage option, too.
- # esxcli vsan cluster get
- From the Hosts tab of the cluster, ensure no ESXi host is part of the VSAN cluster. After confirmation, right-click on the cluster and then click Remove.
You can now add the ESXi host to any other non-VSAN cluster.
Add host to VSAN cluster
esxcli vsan cluster get esxcli vsan cluster leave esxcli vsan cluster get ... Sub-Cluster UUID: 523d154a-3198-d9bb-2829-bf1e4b8cf1b0 ...
Then add it to VSAN Cluster !!!
esxcli vsan cluster join -u UUID...AS ABOVE
RVC observer monitor
/100.12.1.10/YXJK-Datacenter> vsan.observer 1/TEST_Cluster/ -r --force 2015-12-10 13:21:19 +0000: Spawning HTTPS server 2015-12-10 13:21:19 +0000: Using certificate file: /etc/vmware-vpx/ssl/rui.crt 2015-12-10 13:21:19 +0000: Using private key file: /etc/vmware-vpx/ssl/rui.key [2015-12-10 13:21:19] INFO WEBrick 1.3.1 [2015-12-10 13:21:19] INFO ruby 1.9.2 (2011-07-09) [x86_64-linux] [2015-12-10 13:21:19] WARN TCPServer Error: Address already in use - bind(2) [2015-12-10 13:21:19] INFO Certificate: ...
I created a folder on my VSAN datastore, but how do I delete it
- change directory to /vmfs/volumes/vsanDatastore
- run “ls -l” in /vmfs/volumes/vsanDatastore to identify the folder you want to delete
run “/usr/lib/vmware/osfs/bin/osfs-rmdir <name-of-the-folder>” to delete the folder
Unmounting an NFS datastore fails with the error: Sysinfo set operation
ssh root@100.12.6.13 -C 'services.sh restart'
Then unmount NFS datastore
Performance test
https://www.thomas-krenn.com/en/wiki/Linux_I/O_Performance_Tests_using_dd
HP
- We did find an easier way to make these changes to the HBAs. When installing ESXi with the HP Custom ISO, the hpssacli utility can be used to make changes to the HBA modes.
- So we simply installed ESXi on all hosts, then simply executed the following command to change the HBA mode :
http://pcdsolutions.com/vmware-vsan-hp-dl380-gen9-p440-hbas/
esxcli hpssacli cmd -q "controller slot=0 modify hbamode=on forced" esxcli hpssacli cmd -q "controller slot=1 modify hbamode=on forced" esxcli hpssacli cmd -q "controller slot=2 modify hbamode=on forced"
Where slot=x = the adapter instance. slot=0 = embedded HBA, slot=1 = first PCI HBA etc..
A reboot is required after these changes are made. Once rebooted, you can validate the HBA mode with the following command (look for HBA Mode)
esxcli hpssacli cmd -q "controller all show config"
Oracle RAC setting
[root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] vmkfstools -t0 txld10056_1.vmdk Mapping for file txld10056_1.vmdk (1073741824 bytes in size): [ 0: 536870912] --> [NOMP -- :( 0 --> 536870912)] [ 536870912: 536870912] --> [NOMP -- :( 0 --> 536870912)] [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] vmkfstools -w^Cxld10056_1.vmdk [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] pwd /vmfs/volumes/vsanDatastore/txld10056 [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] vmkfstools -w `pwd`/txld10056_1.vmdk All data on '/vmfs/volumes/vsanDatastore/txld10056/txld10056_1.vmdk' will be overwritten with zeros. vmfsDisk: 1, rdmDisk: 0, blockSize: 1048576 Zeroing: 100% done. [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] grep -i 'displayName' txld10056.vmx displayName = "TEST-txld10056" [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] vmkfstools -t0 txld10056_1.vmdk Mapping for file txld10056_1.vmdk (1073741824 bytes in size): [ 0: 536870912] --> [VMFS -- VMFS:00000000-00000000-0000-000000000000:( 0 --> 536870912)] [ 536870912: 536870912] --> [VMFS -- VMFS:00000000-00000000-0000-000000000000:( 536870912 --> 1073741824)] [root@vSAN-ESXi01:/vmfs/volumes/vsan:52b0cd830511f56d-e1a4e2e642cbc250/5abcde5d-8e3d-6358-d5ed-e4434b91e7ae] vmkfstools -t0 txld10056_1.vmdk Mapping for file txld10056_1.vmdk (1073741824 bytes in size): [ 0: 536870912] --> [VMFS -- VMFS:00000000-00000000-0000-000000000000:( 0 --> 536870912)] [ 536870912: 536870912] --> [VMFS -- VMFS:00000000-00000000-0000-000000000000:( 536870912 --> 1073741824)]
vid svid
esxcfg-scsidevs -a && vmkchdev -l | grep -i hba
[root@ESXP-D15U07-INI-02:~] vmkchdev -l | head 0000:00:00.0 8086:2020 8086:0000 vmkernel PCIe RP[0000:00:00.0] 0000:00:04.0 8086:2021 19e5:4001 vmkernel 0000:00:04.1 8086:2021 19e5:4001 vmkernel 0000:00:04.2 8086:2021 19e5:4001 vmkernel 0000:00:04.3 8086:2021 19e5:4001 vmkernel 0000:00:04.4 8086:2021 19e5:4001 vmkernel 0000:00:04.5 8086:2021 19e5:4001 vmkernel 0000:00:04.6 8086:2021 19e5:4001 vmkernel 0000:00:04.7 8086:2021 19e5:4001 vmkernel 0000:00:05.0 8086:2024 8086:0000 vmkernel esxcfg-scsidevs -a && vmkchdev -l | grep -i hba
HDD claim to SSD
brightmoon ~ # ansible SYSTEC-vSAN -m raw -a 'esxcli storage nmp satp rule add -s VMW_SATP_LOCAL -d mpx.vmhba1:C0:T0:L0 -o enable_ssd' [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details 192.168.26.52 | CHANGED | rc=0 >> Shared connection to 192.168.26.52 closed. 192.168.26.53 | CHANGED | rc=0 >> Shared connection to 192.168.26.53 closed. 192.168.26.54 | CHANGED | rc=0 >> Shared connection to 192.168.26.54 closed. 192.168.26.51 | CHANGED | rc=0 >> Shared connection to 192.168.26.51 closed. brightmoon ~ # ansible SYSTEC-vSAN -m raw -a 'esxcli storage core claiming reclaim -d mpx.vmhba1:C0:T0:L0' [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details 192.168.26.52 | CHANGED | rc=0 >> Shared connection to 192.168.26.52 closed. 192.168.26.51 | CHANGED | rc=0 >> Shared connection to 192.168.26.51 closed. 192.168.26.53 | CHANGED | rc=0 >> Shared connection to 192.168.26.53 closed. 192.168.26.54 | CHANGED | rc=0 >> Shared connection to 192.168.26.54 closed.