Diff for "désert/VMware/VSAN"

VSAN how to

• http://www.davidhill.co/2013/08/vmware-vsan-how-to-configure/

• http://vsphere-land.com/vsphere-links/vsan-links.html

• http://www.yellow-bricks.com/2013/09/16/frequently-asked-questions-virtual-san-vsan/

Docker Container for the Ruby vSphere Console (RVC)

docker pull lamw/rvc
docker run --rm -it lamw/rvc
docker run --rm -it -p 80:8010 lamw/rvc

rvm install 1.9.2
gem install rvc
gem install ffi

http://www.virtuallyghetto.com/2015/11/docker-container-for-the-ruby-vsphere-console-rvc.html http://tsmith.co/2014/vsan-installation/

RVC how to

• 100.11.4.1

docker run --rm -it -p 8011:8010 lamw/rvc
rvc administrator@vsphere.local@100.12.1.10
vsan.observer --run-webserver --force XX/XX/TEST_Cluster

• Enter URL: https://100.11.4.1:8011

Step 1: Enable the VSAN service on a VMKernel port.

* Create your virtual switches, and create at least 1 VMKernel port that will be * vmk-vsanused for VSAN traffic. * Be sure to keep the Switch names the same across hosts! * Edit the VMKernel port and check the box for Virtual SAN Traffic. * Save the port settings, repeat for each host. * Time Saver: Use host profiles!

Step 2: Enable VSAN on the Cluster

• Select your cluster, and choose the Manage tab, and the select General under Virtual SAN.
• Edit, and check the box to Turn ON Virtual SAN.
• Choose your setting for “Add disks to storage”
• Manual – You will select each disk that will be a part of the Virtual SAN
• Automatic – VSAN will select all eligible disks for you and add them
• Click OK

Step 3: Add Disks to Disk Groups

Since I chose Manual mode, I will need to add my disks into Disk Groups. A Disk Group is a collection of 1 SSD, and multiple HDD drives. You can have multiple disk groups per host if capacity allows.

• Still in Virtual SAN settings under the cluster, select the Disk Management section.
• Click on the Claim Disks button, and select the drives for use in Virtual SAN.

Alternatively, select each host, and manually create Disk Groups per host.

Step 4: Start building Virtual Machines!

Yes, it is that easy. You now have a datastore called vsanDatastore.

My DL360 G6 server can also access this datastore, since I enabled VSAN on the VMKernel port groups, even though it’s not providing any resources to the VSAN cluster.

Supported network topologies for VSAN stretched cluster

http://cormachogan.com/2015/09/10/supported-network-topologies-for-vsan-stretched-cluster/

故障域构造

您必须至少定义三个故障域，每个故障域可能包含一个或多个主机。故障域定义必须确认可能代表潜在故障域的物理硬件构造，如单个计算机柜。

如果可能，请使用至少四个故障域。使用三个故障域时，不允许使用特定撤出模式，Virtual SAN 也无法在故障发生后重新保护数据。在这种情况下，您需要一个使用三域配置时无法提供的备用容量故障域用于重新构建。

如果启用故障域，Virtual SAN 将根据故障域而不是单个主机应用活动虚拟机存储策略。

根据计划分配给虚拟机的存储策略中规定的“允许的故障数”属性，计算群集中的故障域数目。

number of fault domains = 2 * number of failures to tolerate + 1

如果主机不是故障域成员，Virtual SAN 会将其解析为单独的域。 

http://cormachogan.com/2015/04/20/vsan-6-0-part-8-fault-domains/

If rack 1 fails (containing host 1), do I still have a full copy of the data? The answer is Yes.

If rack 2 fails (containing host 2), do I still have a full copy of the data? The answer is Yes.

If rack 3 fails (containing hosts 3 & 4), do I still have a full copy of the data? The answer is still Yes.

About VSAN capacity and VMDK placement

• http://www.viktorious.nl/2013/10/10/vsan-capacity-vmdk-placement/

• http://cormachogan.com/2013/09/04/vsan-part-4-understanding-objects-and-components/

删除磁盘原有分区

使用partedUtil工具 首先确认哪些磁盘被确认出来。

esxcli storage core device list
# 获取 ID
partedUtil get /vmfs/xxx/xxx
1 2048 3xxx 0 0

# 删除分区
partedUtil delete /vmfs/xxx/xxx 1

MTU setting

Dell switches a MTU 9000 is actually (9 * 1024) = 9216. Ugh, so now I have set

• 9000 MTU on the VSAN VMkernel on the vSphere host.

• 9000 MTU on the VSAN VLAN on the Dell PowerConnect

• 9216 MTU on the VSAN Physical Interface the Dell PowerConnect

BAM! The VSAN datastore is rocking and I can now write to it. However, this scenario posed some reflection on how MTU actually impacts VSAN. I did some further testing and came up with the follow conclusions:

• If any host in a VSAN cluster has a mismatched MTU size, NOTHING can write to the VSAN datastore. Even if one host with the wrong MTU is set then it will prevent VSAN from working.

  • Even with mismatched MTU’s when one verifies the Network Status (vCenter > Virtual SAN > General) it will show Normal. However, this doesn’t verify MTU, just IP connectivity. To test if the MTU is correct then use the MTU of the VSAN VMkernel’s MTU size and issue a vmkping -s <VSANvmkernel_mtu_setting> <Other-VSANvmkernel-interfaces-in-cluster>

  VSAN performances about the same with or without Jumbo Frames configured.

In conclusion, would I advise configuring Jumbo Frames with VSAN? No. Unless you’re the type who prefers all risk and no reward…

http://flcloudlabs.com/vsan-and-mtu/

https://communities.vmware.com/message/2455828

请遵循以下准则

Virtual SAN 需要一个专用 1 Gb 网络。最佳做法是使用 10 Gb 网络。

在每台主机上，可至少将一个物理 1 Gb 以太网网卡专用于 Virtual SAN。还可以置备另外一个物理网卡作为故障切换网卡。

可以在每个主机上使用 vSphere 标准交换机，或者可以将环境配置为使用 vSphere Distributed Switch。

为每个用于 Virtual SAN 的网络配置一个已激活 Virtual SAN 端口属性的 VMkernel 端口组。

为每个端口组使用相同的 Virtual SAN 网络标签，并确保这些标签在所有主机上一致。

使用巨帧以实现最佳性能。

Virtual SAN 支持 IP 哈希负载平衡，但无法保证所有配置的性能都有提升。当除 Virtual SAN 以外还有众多 IP 哈希使用者时，可以从 IP 哈希中获益。这种情况下，IP 哈希将执行负载平衡。但是，如果 Virtual SAN 是唯一的使用者，则可能看不到什么变化。这一规则特别适用于 1G 环境。例如，如果您将四个设置了 IP 哈希的 1G 物理适配器用于 Virtual SAN，实际能够使用的可能不超过 1G。对于我们目前支持的所有网卡成组策略来说，这一点也同样适用。有关网卡成组的详细信息，请参见《vSphere NetworkingvSphere 网络》指南的“网络连接策略”部分。

Virtual SAN 不支持同一子网上有多个 VMkernel 适配器用于负载平衡。但是支持多个 VMkernel 适配器位于不同网络的情况，如 VLAN 或单独的物理结构。

您应该将所有参与 Virtual SAN 的主机连接到已启用多播（IGMP 侦听）的单个 L2 网络。如果参与 Virtual SAN 的主机跨越多个交换机乃至 L3 边界，必须确保将网络正确配置为启用多播连接。如果您的网络环境需要，或者如果您在同一 L2 网络中运行多个 Virtual SAN 群集，则可以更改多播地址的默认设置。 

http://www.tomsitpro.com/articles/essential-virtual-san-vsan-book-excerpt,2-888.html

http://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vsphere.virtualsan.doc%2FGUID-D52F00FF-CA2C-4DDD-B76B-B8BF211BB0E8.html

Network Misconfiguration Status in a Virtual SAN Cluster

After you enable Virtual SAN on a cluster, the datastore is not assembled correctly because of a detected network misconfiguration. Problem

After you enable Virtual SAN on a cluster, on the Summary tab for the cluster the Network Status for Virtual SAN appears as Misconfiguration detected. Cause

One or more members of the cluster cannot communicate because of either of the following reasons:

• A host in the cluster does not have a VMkernel adapter for Virtual SAN.
• The hosts cannot connect each other in the network.
• Multicast is not enabled on the physical switch.

How do you resolve it? Well, a number of our VSAN beta customers discussed some options on the community, and these were the recommendations:

• Option 1 – Disable IGMP Snooping. Now this will allow all multicast traffic through, but if the only traffic is VSAN, then this should be a negligible amount of traffic and should be safe to use.
• Option 2 – Configure IGMP snooping querier. If there is other multicast traffic and you are concerned that disabling IGMP snooping might open the network up to a flood of multicast traffic, then this is a preferred option

http://cormachogan.com/2014/01/21/vsan-part-15-multicast-requirement-for-networking-misconfiguration-detected/

Changing the multicast address used for a VMware Virtual SAN Cluster (2075451)

Purpose

This article provides steps to change the multicast address for each VMware Virtual SAN cluster. If there are multiple Virtual SAN clusters on the same Layer 2 network, each host receives all multicast messages. In order to reduce the amount of multicast traffic for each VSAN cluster, it is necessary to change the multicast address for each VMware Virtual SAN cluster.

Warning: If you change the multicast address on an active Virtual SAN cluster, it can lead to network partitioning until all of the ESXi hosts in the cluster are on the same multicast network. It is recommended to organize downtime before making this change.

Resolution

In order to change the multicast address for VMware Virtual SAN, perform these steps on each ESXi host within the Virtual SAN Cluster.

To change the multicast address on an ESXi 5.5/6.0 host configured for Virtual SAN:

• Open an SSH connection to the ESXi host and log in as root. For more information, see Using ESXi Shell in ESXi 5.x (2004746). Identify the VMkernel interface configured for Virtual SAN. To identify the VMkernel interface, run this command on the ESXi hosts: esxcli vsan network list You see output similar to: Interface

  • VmkNic Name: vmk1 IP Protocol: IPv4 Interface UUID: 28b52f53-69c1-c193-eabe-005056885a94 Agent Group Multicast Address: 224.2.3.4 Agent Group Multicast Port: 23451 Master Group Multicast Address: 224.1.2.3 Master Group Multicast Port: 12345 Multicast TTL: 5

  To change the multicast address on each ESXi host in the cluster, run this command:

  esxcli vsan network ipv4 set -i <vmkernel interface> -d <multicast agent group address> -u <multicast master group address> For example, to set the Master Group Multicast address to 224.2.3.5 and the Agent Group Multicast Address to 224.2.3.6 , run this command on each ESXi host for this particular VSAN cluster: esxcli vsan network ipv4 set -i vmk1 -d 224.2.3.6 -u 224.2.3.5

Using tcpdump-uw to collect packet traces to troubleshoot network issues

Usage: tcpdump-uw
tcpdump-uw
-i = interface
-n = no IP or Port name resolution
-s0 = Collect entire packet
-t = no timestamp
-c = number of frames to capture

• tcpdump-uw -i vmk1 -n -s0 -t -c 20 udp port 12345
• tcpdump-uw -i vmk1 -n -s0 -t -c 20 udp port 23451

Generate Multicast traffic

nc -uz <destination-ip> <destination-port>

Monitor VSAN VMKernel Port network traffic

esxcli network ip connection list

Troubleshooting

SSD Permanent disk failure

+----------------------+-------------+-------+------+------------+---------+----------+-------------+
| naa.500003965c8995f8 | 100.12.1.22 | SSD   | 0    | 372.61 GB  | 70.00 % | 0.00 %   | FAILED (v2) |
| naa.5000c5008e9caa53 | 100.12.1.22 | MD    | 18   | 1106.62 GB | 72.26 % | 3.55 %   | FAILED (v2) |
| naa.5000c5008e9fa32f | 100.12.1.22 | MD    | 24   | 1106.62 GB | 72.17 % | 13.38 %  | FAILED (v2) |
| naa.5000c5008e9ec9f7 | 100.12.1.22 | MD    | 19   | 1106.62 GB | 51.11 % | 4.98 %   | FAILED (v2) |
| naa.5000c5008e9ee5ef | 100.12.1.22 | MD    | 20   | 1106.62 GB | 59.04 % | 4.12 %   | FAILED (v2) |
+----------------------+-------------+-------+------+------------+---------+----------+-------------+
| naa.500003965c899624 | 100.12.1.22 | SSD   | 0    | 372.61 GB  | 70.00 % | 0.00 %   | FAILED (v2) |
| naa.5000c5008eaa0ec3 | 100.12.1.22 | MD    | 18   | 1106.62 GB | 67.22 % | 8.11 %   | FAILED (v2) |
| naa.5000c5008ea9d7ff | 100.12.1.22 | MD    | 22   | 1106.62 GB | 48.36 % | 21.94 %  | FAILED (v2) |
| naa.5000c5008e9f98f3 | 100.12.1.22 | MD    | 17   | 1106.62 GB | 65.91 % | 13.23 %  | FAILED (v2) |
| naa.5000c5008e9cb727 | 100.12.1.22 | MD    | 23   | 1106.62 GB | 49.88 % | 8.32 %   | FAILED (v2) |
+----------------------+-------------+-------+------+------------+---------+----------+-------------+

[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device list | grep 5f8
naa.500003965c8995f8
   Display Name: TOSHIBA Serial Attached SCSI Disk (naa.500003965c8995f8)
      Devfs Path: /vmfs/devices/disks/naa.500003965c8995f8
         Other UIDs: vml.0200000000500003965c8995f850583032534d

• show disk status

[root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device smart get naa.500003965c8995f8
         Error: Unknown command or namespace storage core device smart get naa.500003965c8995f8

         [root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000] esxcli storage core device smart get -d naa.500003965c8995f8
         Parameter                     Value  Threshold  Worst
         ----------------------------  -----  ---------  -----
         Health Status                 OK     N/A        N/A
         Media Wearout Indicator       N/A    N/A        N/A
         Write Error Count             0      N/A        N/A
         Read Error Count              0      N/A        N/A
         Power-on Hours                N/A    N/A        N/A
         Power Cycle Count             0      N/A        N/A
         Reallocated Sector Count      N/A    N/A        N/A
         Raw Read Error Rate           N/A    N/A        N/A
         Drive Temperature             29     N/A        N/A
         Driver Rated Max Temperature  N/A    N/A        N/A
         Write Sectors TOT Count       N/A    N/A        N/A
         Read Sectors TOT Count        N/A    N/A        N/A
         Initial Bad Block Count       N/A    N/A        N/A
         [root@ESXi04:/vmfs/volumes/df7a5342-5b24f8ad-0000-000000000000]

http://www.mrvsan.com/using-the-error-injection-command-to-test-a-disk-failure/

There is no more space for virtual disk

http://virtualinator.com/2016/04/page/2/

http://cormachogan.com/2016/04/19/recovering-full-vsan-datastore-scenario/

host cannot communicate with all other nodes in the virtual san enabled cluster

https://communities.vmware.com/thread/495882?start=15&tstart=0 http://www.tomsitpro.com/articles/essential-virtual-san-vsan-book-excerpt,2-888.html

1. 当我随便找了个节点，执行如下命令重启相关服务之后就OK了。
2. [root@ESXi03:~] services.sh restart

Registration/unregistration of a VASA vendor provider on a Virtual SAN host fails

• https://communities.vmware.com/thread/520441?start=0&tstart=0

Issues information is not available at this time.

• i had same problem, but after reinstalling web client it solved !

https://communities.vmware.com/thread/500032?start=0&tstart=0

java.lang.RuntimeException

jvm 1 | 2015/12/18 16:53:02 | java.lang.RuntimeException: com.vmware.vim.vmomi.core.exception.UnmarshallException: local name and field name mismatch: 'namespace' - 'capabilityMetadataPerCategory'

VMware CASE sulotions AS following? http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2113435

VM虚拟机无法ping通网关

现象

1. 虚拟机网络无法跟网关相通。
2. 但跟同网段的其它虚拟机能通。其它虚拟机也能与它互通。

解决方法

1. 删除当前网卡，重新创建一张网卡。

H3C 6800 交换机端口处理down状态

现象

1. 从ESXi主机上查看esxcli network nic list可以看到端口down。

解决方法

1. 在H3C 6800交换机上把对应的端口shutdown之后再 no shut就OK了。

http://www.virten.net/2014/01/manage-vsan-with-rvc-part-4-troubleshooting/

No space left on device

http://www.m80arm.co.uk/2013/12/ha-issues-with-vsan-beta-refresh.html

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1007638

 stat -f /vsanDatastore

* The no space left on the device error confused me so in order to rectify the issue I tried the following:

1. Increases the size of the VMDK used for the ESXi installation from 2GB to 5GB
2. Inflated the ESXi installation VMDK so it was Thick Eager Zero'd just in case this was causing any strange issues
3. Rebuild the nested environment manually (not using the .ova supplied by William Lam)

System logs are stored on non-persistent storage

http://cormachogan.com/2015/02/24/vsan-considerations-when-booting-from-usbsd/ To verify the location:

Browse to the host in the vSphere Web Client navigator.

• Click the Manage tab, then click Settings.
• Under System, click Advanced System Settings.
• Ensure that Syslog.global.logDir points to a persistent location.

• If the field Syslog.global.logDir is empty or points to a scratch partition, make sure that the field ScratchConfig.CurrentScratchLocation shows a location on persistent storage.

Note: You must reboot the host for the changes to take effect.

Note: To log to a datastore, the Syslog.global.logDir entry should be in the format of [Datastorename]/foldername. To log to the scratch partition set in the ScratchConfig.CurrentScratchLocation, the format is blank or []/foldername.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2032823

The Solution is

1. create NFS server (VM) and disk.
2. mount NFS on ESXi cluster
3. setting Syslog.global.logDir as [NFS4VSAN-LOG]/ESXi-IP-ADDR-DIR

How to Delete VSAN Datastore

http://www.vladan.fr/how-to-delete-vsan-datastore/

• Evacuate all the VMs out of the VSAN datastore.
• Turn OFF VMware HA.
• Delete all VSAN disk groups Individually

Once done (for each of the hosts participating in the VSAN cluster), the local disks are available to be re-used….

• Deactivate VSAN cluster. At the Cluster level > Manage > Virtual SAN > General

• Re-enable VMware HA.

VSAN upload problem

2015-11-19T19:05:08.31Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] In do_GET: ('127.0.0.1', 36677)
2015-11-19T19:05:08.31Z WARNING vsan-health[Thread-7] [VsanHealthServer::do_GET] do_GET: isStringResponse = True
2015-11-19T19:05:08.32Z INFO vsan-health[Thread-7] [VsanHealthServer::log_message] ('127.0.0.1', 36677) - - "GET /vsanHealth/health HTTP/1.1" 200 -
2015-11-19T19:05:08.32Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] Done do_Get: ('127.0.0.1', 36677) (took 0.0)
2015-11-19T19:05:35.507Z WARNING vsan-health[Thread-1] [VsanPyVmomiProfiler::InvokeMethod] Invoke: mo=ServiceInstance, info=CurrentTime
2015-11-19T19:05:38.475Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] In do_GET: ('127.0.0.1', 36677)
2015-11-19T19:05:38.475Z WARNING vsan-health[Thread-7] [VsanHealthServer::do_GET] do_GET: isStringResponse = True
2015-11-19T19:05:38.475Z INFO vsan-health[Thread-7] [VsanHealthServer::log_message] ('127.0.0.1', 36677) - - "GET /vsanHealth/health HTTP/1.1" 200 -
2015-11-19T19:05:38.476Z DEBUG vsan-health[Thread-7] [VsanHealthServer::do_GET] Done do_Get: ('127.0.0.1', 36677) (took 0.0)

Remove a disk group FROM a host

Entering Maintenance Mode is done by selecting the correct ESXi host and then clicking on the maintenance mode icon in the Disk Management section on Virtual SAN in the vSphere web client (third icon from the left):

Shutdown VSAN cluster

To recap, if shutting down the whole of the VSAN cluster, use maintenance mode for the hosts, and do not move VMs or migrate any data.

Remove a host FROM VSAN VMware KB: Unmounting a LUN or detaching a datastore/storage device from multiple VMware ESXi 5.x/6.0 hosts

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2004605

If you are not using VSAN:

• If VSAN is not used in your ESXi host, run this command to stop the vsantraces:

    #/etc/init.d/vsantraced stop

    Perform a Refresh for Storage.
    Unmount the datastore.
    Run this command to start the service:

    #/etc/init.d/vsantraced start

    [root@ESXi-B02:~] esxcli storage filesystem list
    Mount Point                                           Volume Name        UUID                                    Mounted  Type             Size           Free
    ----------------------------------------------------  -----------------  --------------------------------------  -------  ------  -------------  -------------
    /vmfs/volumes/562a5aa6-f20c535a-1171-246e9601d388     ESXi-B02_SASRAID5  562a5aa6-f20c535a-1171-246e9601d388        true  VMFS-5  5997921828864  5717964619776
    /vmfs/volumes/562a4405-a83996b6-4fa5-246e9601d388                        562a4405-a83996b6-4fa5-246e9601d388        true  vfat        299712512       83386368
    /vmfs/volumes/93df10e1-58e61d4c-8c05-c5b28ac43c65                        93df10e1-58e61d4c-8c05-c5b28ac43c65        true  vfat        261853184       79302656
    /vmfs/volumes/5f7ee1e6-998a02ee-20c2-d9c3f71404fd                        5f7ee1e6-998a02ee-20c2-d9c3f71404fd        true  vfat        261853184       92418048
    /vmfs/volumes/vsan:52541b505a767701-6f36d587dbae5691  vsanDatastore      vsan:52541b505a767701-6f36d587dbae5691     true  vsan                0              0

[root@ESXi-B02:~] esxcli storage filesystem unmount -u vsan:52541b505a767701-6f36d587dbae5691
No volume with uuid 'vsan:52541b505a767701-6f36d587dbae5691' was found

[root@ESXi-B02:~] grep -n -i vsan /etc/vmware/esx.conf 
351:/adv/VSAN/LicensedFeatures = "allflash,stretchedcluster"
366:/firewall/services/vsanvp/allowedall = "true"
367:/firewall/services/vsanvp/enabled = "true"
457:/vsan/faultDomainVersion = "2"
458:/vsan/faultDomainName = ""
459:/vsan/autoClaimStorage = "false"
460:/vsan/enabled = "true"
461:/vsan/subClusterUuid = "52541b50-5a76-7701-6f36-d587dbae5691"
462:/vsan/datastoreName = "vsanDatastore"
463:/vsan/checksumEnabled = "false"

[root@ESXi-B01:~]  grep -n -i vsan /etc/vmware/esx.conf 
90:/firewall/services/vsanvp/enabled = "true"
91:/firewall/services/vsanvp/allowedall = "true"
112:/adv/VSAN/LicensedFeatures = "allflash,stretchedcluster"
531:/vsan/faultDomainVersion = "2"
532:/vsan/faultDomainName = ""
533:/vsan/autoClaimStorage = "false"
534:/vsan/enabled = "false"
535:/vsan/subClusterUuid = "52541b50-5a76-7701-6f36-d587dbae5691"
536:/vsan/datastoreName = "vsanDatastore"
537:/vsan/checksumEnabled = "false"

In ESXi-B02 the /vsan/enabled = "True" !!!!
So change "True" to "false", then restart services, but didn't work.

http://ambitech.blogspot.in/2015/07/unable-to-add-standalone-host-already.html

 So here were few hosts whose IP addresses were messed up that what it was in the host file (or in your case it might be DNS) and i corrected them via vCLI. After that i tried to add them but it was always throwing this error saying the ip already exists even though i was trying to add it using the hostname.
 Then i just checked the vCenter server database---->Table---->dbo.vpx.host.  ---Right click and select 1000 rows and there they were, the ip address, the hostname. These are not in the vcenter anymore but they are in the vcenter server database.
 I wanted to delete these entries here and then try to re add but my colleague ravi suggested an alternative.
 I cleaned off the hostfile for these 2 clusters where these hosts (in maintenance mode) were there and then remove the clusters in which these hosts were supposed to be there. Automagically the stale entries in the database were gone. Now I was able to peacefully add the hosts back to the vcenter and the clusters in which they reside.
 you might have to remove the entries for these hosts in the DNS and then do the same to get around it.

Note: Run this command to permanently turn off the service :

• chkconfig vsantraced off

If you are using VSAN:

• If you are using VSAN on the ESXi host, run this command to change the VSAN trace location: # esxcli vsan trace set -p datastore_name Unmount the datastore.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2069171

Removing Host FROM VSAN

method 1

• Place host in maintenance mode
• Delete disk group when “maintenance mode” is completed
• Move host out of the cluster
• Remove the VSAN VMkernel (not a requirement, but I prefer to clean things up)

  http://www.yellow-bricks.com/2014/01/14/remove-host-virtual-san-cluster/

method 2

Removing a VMware Virtual SAN-enabled cluster and detaching the VMware ESXi host from the cluster == http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2072347

1. Log in to each ESXi host's console, see Using ESXi Shell in ESXi 5.x and 6.0 (2004746).
2. Place the host in maintenance mode and select the full data migration option.
3. To obtain information about the VSAN storage, run this command:
   • # esxcli vsan storage list
4. To disable automated mode, run this command:
   • # esxcli vsan storage automode set --enabled false
5. To remove the specific SSD disk used in VSAN configuration, run this command:
   • # esxcli vsan storage remove -s [SSD-DEVICE-ID]
6. To obtain information about the VSAN cluster for the ESXi host, run this command:
   • # esxcli vsan cluster get
7. To disconnect the host from the VSAN cluster, run this command:
   • # esxcli vsan cluster leave
8. To find any left-over configuration, run this command:
   • # esxcli vsan cluster get

     • Note: The command should not return anything if VSAN cluster information gets removed. You can also verify that the vsanDatastore will disappear from the Configuration > Storage option, too.

9. From the Hosts tab of the cluster, ensure no ESXi host is part of the VSAN cluster. After confirmation, right-click on the cluster and then click Remove.

You can now add the ESXi host to any other non-VSAN cluster.

Add host to VSAN cluster

esxcli vsan cluster get
esxcli vsan cluster leave
esxcli vsan cluster get
...
Sub-Cluster UUID: 523d154a-3198-d9bb-2829-bf1e4b8cf1b0
...

Then add it to VSAN Cluster !!!

esxcli vsan cluster join -u UUID...AS ABOVE

RVC observer monitor

/100.12.1.10/YXJK-Datacenter> vsan.observer 1/TEST_Cluster/ -r --force
2015-12-10 13:21:19 +0000: Spawning HTTPS server
2015-12-10 13:21:19 +0000: Using certificate file: /etc/vmware-vpx/ssl/rui.crt
2015-12-10 13:21:19 +0000: Using private key file: /etc/vmware-vpx/ssl/rui.key
[2015-12-10 13:21:19] INFO  WEBrick 1.3.1
[2015-12-10 13:21:19] INFO  ruby 1.9.2 (2011-07-09) [x86_64-linux]
[2015-12-10 13:21:19] WARN  TCPServer Error: Address already in use - bind(2)
[2015-12-10 13:21:19] INFO  
Certificate:
...

I created a folder on my VSAN datastore, but how do I delete it

• change directory to /vmfs/volumes/vsanDatastore
• run “ls -l” in /vmfs/volumes/vsanDatastore to identify the folder you want to delete

• run “/usr/lib/vmware/osfs/bin/osfs-rmdir <name-of-the-folder>” to delete the folder

Unmounting an NFS datastore fails with the error: Sysinfo set operation

ssh root@100.12.6.13 -C 'services.sh restart'

Then unmount NFS datastore

Performance test

https://www.thomas-krenn.com/en/wiki/Linux_I/O_Performance_Tests_using_dd