ESXi

Contents

  1. ESXi
    1. To monitor the VMDK snapshot and base disks which are currently being updated, run this watch command
    2. Edit hardware properties of the specified OVF or OVA
    3. Optimization
      1. IO control
      2. Enabling hot add memory & CPU will disable vNUMU.
    4. Get hardware info
    5. EVC and CPU Compatibility FAQ (1005764)
    6. Manger host
      1. To power on a virtual machine from the command line
    7. Show FC WWN
    8. VSAN
    9. Virtual Machine Disks consolidation is needed
    10. Deploying a vApp failed with the error “Cannot initialize property “ | VMware Arena
    11. SSLLoadSharedLibraries: Failed to load OpenSSL libraries. libdir is /usr/lib/vmware
    12. 在主机上找到已弃用的VMFS卷
    13. How to Fix Snapshot consolidation needed fails with a lock message?
      1. Method – 1
      2. Method – 2 (Best Option to fix the issue)
      3. Method – 3 (Good Option to fix – but you need extra space on datastore)
    14. P2V
      1. Windows P2V error "fault.ManagedDiskOpenFault.summary"
      2. Network card configuration missing after P2V using VMware Converter
    15. Linux workstation
    16. kill cancel task
    17. Clearing partitioning information in ESXi using the DD utility
    18. rescan datastores
    19. Troubleshooting
    20. Useful SMART Information
    21. 关于snapshot快照整合问题
      1. vCenter Server Appliance 6.0 services fail due to 100% usage of /storage/seat disk
      2. Increasing the disk space for the VMware vCenter Server Appliance in vSphere 6.0
      3. Import OVF Template - Unsupported hardware family 'vmx-06'
      4. frequent error message 'nmp_ThrottleLogForDevice:3178' in vmkernel.log
      5. Unable to connect to the MKS: Internal error
      6. Insufficient resources to satisfy vSphere HA failover level
    22. Agent can't send heartbeats
    23. IMPI log full
    24. vMotion about
      1. How is disk data migration via vMotion different from a Storage vMotion?
    25. How-to rename VMNIC in VMware ESXi
    26. To restart a vCenter Server and/or Platform Services Controller service using the command-line:
    27. Backing up and restoring the vCenter Server Appliance vPostgres database
    28. ESXi web client Embedded Host Client – VMware Labs:
    29. How to quickly check ESXi logs from web browser
    30. Create a VMDK Larger than 16TB
    31. Convert vmdk to raw
    32. UUID
    33. References

To monitor the VMDK snapshot and base disks which are currently being updated, run this watch command

  cd datastore...

watch -d 'ls -luth | grep -E "delta|flat|sesparse"'

where:

-d highlights the differences between successive updates

In addition, run this command to monitor the time stamp update of the base disks to confirm if the process is working. The result lists the files by their modification date.

ls -lrth |grep -E "flat|delta|sesparse"

Edit hardware properties of the specified OVF or OVA

Since COT is written in Python, it can be installed like any other Python package using the pip tool. For most users this is the recommended installation method.

http://cot.readthedocs.io/en/latest/installation.html

Optimization

IO control

Enabling hot add memory & CPU will disable vNUMU.

vNUMA (virtual NUMA) is a memory-access optimization method for VMware virtual machines (VMs) that helps prevent memory-bandwidth bottlenecks.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008205

http://www.virtualinsanity.com/index.php/2010/03/16/performance-troubleshooting-vmware-vsphere-storage/

https://vikernel.wordpress.com/2014/06/27/troubleshooting-storage-performance/

Get hardware info

smbiosDump | egrep '(Product|Serial)' 
lspci -p

EVC and CPU Compatibility FAQ (1005764)

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005764#Which%20CPUs%20are%20compatible%20with%20each%20EVC%20mode?

http://www.virten.net/2013/04/intel-cpu-evc-matrix/

Manger host

To power on a virtual machine from the command line

List the inventory ID of the virtual machine with the command:

vim-cmd vmsvc/getallvms |grep <vm name>

Check the power state of the virtual machine with the command:

vim-cmd vmsvc/power.getstate <vmid>

Power-on the virtual machine with the command:

vim-cmd vmsvc/power.on <vmid> 

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1038043

Show FC WWN

[root@localhost:~] esxcli storage san fc list
Adapter: vmhba2
Port ID: 010B00
Node Name: 20:00:00:90:fa:86:71:cc
Port Name: 10:00:00:90:fa:86:71:cc
Speed: 8 Gbps
Port Type: NPort
Port State: ONLINE

Adapter: vmhba3
Port ID: 010B00
Node Name: 20:00:00:90:fa:82:8d:f4
Port Name: 10:00:00:90:fa:82:8d:f4
Speed: 8 Gbps
Port Type: NPort
Port State: ONLINE

VSAN

http://www.settlersoman.com/how-to-configure-vmware-virtual-san-vsan/

http://flcloudlabs.com/vsan-and-mtu/

http://www.vladan.fr/infiniband-in-the-lab-the-missing-piece-for-vsan/

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2078224

https://vsantco.vmware.com/vsan/SI/SIEV === VSAN Performance test == https://blogs.vmware.com/vsphere/2014/03/supercharge-virtual-san-cluster-2-million-iops.html

Virtual Machine Disks consolidation is needed

https://communities.vmware.com/message/2540110 http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1002310

Deploying a vApp failed with the error “Cannot initialize property “ | VMware Arena

http://www.vmwarearena.com/2014/02/deploying-vapp-failed-with-the-error-cannot-initialize-property.html

SSLLoadSharedLibraries: Failed to load OpenSSL libraries. libdir is /usr/lib/vmware

Creating the symlink with ln -sf /opt/vmware/lib/vmware/ /usr/lib64/vmware fixes the issue.

在主机上找到已弃用的VMFS卷

这是一个在VMware ESXi6.X上的BUG。

https://kb.vmware.com/s/article/2109735

https://kb.vmware.com/s/article/1003490?CoveoV2.CoveoLightningApex.getInitializationData=1&r=2&ui-communities-components-aura-components-forceCommunity-seoAssistant.SeoAssistant.getSeoData=1&other.KM_Utility.getArticleDetails=1&other.KM_Utility.getArticleMetadata=2&other.KM_Utility.getUrl=1&other.KM_Utility.getUser=1&other.KM_Utility.getAllTranslatedLanguages=2&ui-comm-runtime-components-aura-components-siteforce-qb.Quarterback.validateRoute=1

To Resolve is: Restarting the Management agents on an ESXi

/etc/init.d/hostd restart

/etc/init.d/vpxa restart

OR alternatively:

[root@GRGESXi01:~] services.sh restart
services.sh: stop /etc/opt/init.d/vmware-fdm
Running vmware-fdm stop
Stopping vmware-fdm:Setting the memory limit for fdm resource pool on this host to
    0 MB
    success

    services.sh: stopped /etc/opt/init.d/vmware-fdm
    services.sh: stop /etc/init.d/vmtoolsd
    Running vmtoolsd stop
    vmtoolsd is not running
    Errors: 
    Sysinfo error on operation returned status : Not found. Please see the VMkernel log for detailed error information
    services.sh: stopped /etc/init.d/vmtoolsd
    services.sh: stop /etc/init.d/vmsyslogd
    Running vmsyslogd stop
    Stopping vmsyslogd service using init is not supported
    services.sh: stopped /etc/init.d/vmsyslogd
    services.sh: stop /etc/init.d/xorg
    Running xorg stop
    services.sh: stopped /etc/init.d/xorg
    services.sh: stop /etc/init.d/snmpd
    Running snmpd stop
    root: snmpd is not running.
    services.sh: stopped /etc/init.d/snmpd
    services.sh: stop /etc/init.d/wsman
    Running wsman stop
    Stopping openwsmand
    services.sh: stopped /etc/init.d/wsman
    services.sh: stop /etc/init.d/sfcbd
    Running sfcbd stop
    This operation is not supported.
    Please use /etc/init.d/sfcbd-watchdog stop
    services.sh: stopped /etc/init.d/sfcbd
    services.sh: stop /etc/init.d/sfcbd-watchdog
    Running sfcbd-watchdog stop
    Getting Exclusive access, please wait...
    Exclusive access granted.
    services.sh: stopped /etc/init.d/sfcbd-watchdog
    services.sh: stop /etc/init.d/vpxa
    Running vpxa stop
    watchdog-vpxa: Terminating watchdog process with PID 34992
    vpxa stopped.
    services.sh: stopped /etc/init.d/vpxa
    services.sh: stop /etc/init.d/vobd
    Running vobd stop
    watchdog-vobd: Terminating watchdog process with PID 33247
    vobd stopped
    services.sh: stopped /etc/init.d/vobd
    services.sh: stop /etc/init.d/memscrubd
    Running memscrubd stop
    memscrubd is not running
    services.sh: stopped /etc/init.d/memscrubd
    services.sh: stop /etc/init.d/lacp
    Running lacp stop
    watchdog-net-lacp: Terminating watchdog process with PID 33478
    services.sh: stopped /etc/init.d/lacp
    services.sh: stop /etc/init.d/nscd
    Running nscd stop
    watchdog-nscd: Terminating watchdog process with PID 34535
    services.sh: stopped /etc/init.d/nscd
    services.sh: stop /etc/init.d/dcbd
    Running dcbd stop
    watchdog-dcbd: Terminating watchdog process with PID 34502
    services.sh: stopped /etc/init.d/dcbd
    services.sh: stop /etc/init.d/smartd
    Running smartd stop
    watchdog-smartd: Terminating watchdog process with PID 34496
    smartd stopped
    services.sh: stopped /etc/init.d/smartd
    services.sh: stop /etc/init.d/cdp
    Running cdp stop
    watchdog-cdp: Terminating watchdog process with PID 34530
    services.sh: stopped /etc/init.d/cdp
    services.sh: stop /etc/init.d/slpd
    Running slpd stop
    Stopping slpd
    services.sh: stopped /etc/init.d/slpd
    services.sh: stop /etc/init.d/rabbitmqproxy
    Running rabbitmqproxy stop
    rabbitmqproxy is not running
    services.sh: stopped /etc/init.d/rabbitmqproxy
    services.sh: stop /etc/init.d/hostd
    Running hostd stop
    watchdog-hostd: Terminating watchdog process with PID 34365
    hostd stopped.
    services.sh: stopped /etc/init.d/hostd
    services.sh: stop /etc/init.d/lbtd
    Running lbtd stop
    watchdog-net-lbt: Terminating watchdog process with PID 34103
    net-lbt stopped
    services.sh: stopped /etc/init.d/lbtd
    services.sh: stop /etc/init.d/sdrsInjector
    Running sdrsInjector stop
    watchdog-sdrsInjector: Terminating watchdog process with PID 34045
    sdrsInjector stopped
    services.sh: stopped /etc/init.d/sdrsInjector
    services.sh: stop /etc/init.d/rhttpproxy
    Running rhttpproxy stop
    watchdog-rhttpproxy: Terminating watchdog process with PID 34159
    VMware HTTP reverse proxy stopped.
    services.sh: stopped /etc/init.d/rhttpproxy
    services.sh: stop /etc/init.d/vvold
    Running vvold stop
    /etc/init.d/vvold stop, called by pid 583428
    vvold is not running.
    services.sh: stopped /etc/init.d/vvold
    services.sh: stop /etc/init.d/nfcd
    Running nfcd stop
    nfcd is not running
    services.sh: stopped /etc/init.d/nfcd
    services.sh: stop /etc/init.d/storageRM
    Running storageRM stop
    watchdog-storageRM: Terminating watchdog process with PID 34052
    storageRM stopped
    services.sh: stopped /etc/init.d/storageRM
    services.sh: stop /etc/init.d/vmfstraced
    Running vmfstraced stop
    watchdog-vmfstracegd: PID file /var/run/vmware/watchdog-vmfstracegd.PID does not exist
    watchdog-vmfstracegd: Unable to terminate watchdog: No running watchdog process for vmfstracegd
    vmfstracegd is not running
    Failed to clear vmfstracegd memory reservation
    services.sh: stopped /etc/init.d/vmfstraced
    services.sh: stop /etc/init.d/sensord
    Running sensord stop
    watchdog-sensord: Terminating watchdog process with PID 34111
    sensord stopped
    services.sh: stopped /etc/init.d/sensord
    services.sh: stop /etc/init.d/swapobjd
    Running swapobjd stop
    watchdog-swapobjd: Terminating watchdog process with PID 33900
    Waiting for process to terminate...
    swapobjd stopped
    services.sh: stopped /etc/init.d/swapobjd
    services.sh: stop /etc/init.d/usbarbitrator
    Running usbarbitrator stop
    watchdog-usbarbitrator: Terminating watchdog process with PID 33932
    usbarbitrator stopped
    services.sh: stopped /etc/init.d/usbarbitrator
    services.sh: stop /etc/init.d/DCUI
    Running DCUI stop
    Disabling DCUI logins
    VobUserLib_Init failed with -1
    services.sh: stopped /etc/init.d/DCUI
    services.sh: stop /etc/init.d/SSH
    Running SSH stop
    SSH login disabled
    VobUserLib_Init failed with -1
    Connect to localhost failed: Connection failure.
    Errors: 
    Invalid operation requested: This ruleset is required and connot be disabled
    services.sh: stopped /etc/init.d/SSH
    services.sh: stop /etc/init.d/vsantraced
    Running vsantraced stop
    watchdog-vsantraced: PID file /var/run/vmware/watchdog-vsantraced.PID does not exist
    watchdog-vsantraced: Unable to terminate watchdog: No running watchdog process for vsantraced
    vsantraced is not running
    watchdog-vsantracedUrgen: PID file /var/run/vmware/watchdog-vsantracedUrgen.PID does not exist
    watchdog-vsantracedUrgen: Unable to terminate watchdog: No running watchdog process for vsantracedUrgen
    vsantracedUrgen is not running
    Persisting traces to /scratch/vsantraces
    cp: can't stat '/var/log/vsantraces/*.index': No such file or directory
    ls: /var/log/vsantraces/*.gz: No such file or directory
    Failed to clear vsantraced memory reservation
    services.sh: stopped /etc/init.d/vsantraced
    services.sh: stop /etc/init.d/vsandevicemonitord
    Running vsandevicemonitord stop
    watchdog-VSANDeviceMonitor: PID file /var/run/vmware/watchdog-VSANDeviceMonitor.PID does not exist
    watchdog-VSANDeviceMonitor: Unable to terminate watchdog: No running watchdog process for VSANDeviceMonitor
    /usr/lib/vmware/vsan/bin/VSANDeviceMonitor.py is not running
    Failed to clear  memory reservation
    services.sh: stopped /etc/init.d/vsandevicemonitord
    services.sh: stop /etc/init.d/vsanObserver
    Running vsanObserver stop
    services.sh: stopped /etc/init.d/vsanObserver
    services.sh: start /etc/init.d/vsanObserver
    services.sh: start /etc/init.d/vsandevicemonitord
    services.sh: start /etc/init.d/vsantraced
    Running vsandevicemonitord start
    Running vsanObserver start
    Running vsantraced start
    services.sh: started /etc/init.d/vsanObserver
    VSAN Cluster not configured, not starting VSANDeviceMonitor
    VSAN Cluster not configured, not starting vsantraced
    services.sh: started /etc/init.d/vsandevicemonitord
    services.sh: started /etc/init.d/vsantraced
    services.sh: start /etc/init.d/SSH
    services.sh: start /etc/init.d/DCUI
    Running DCUI start
    Running SSH start
    Enabling DCUI login: runlevel = 
    VobUserLib_Init failed with -1
    services.sh: started /etc/init.d/DCUI
    Connect to localhost failed: Connection failure.
    SSH login enabled
    VobUserLib_Init failed with -1
    services.sh: started /etc/init.d/SSH
    services.sh: start /etc/init.d/usbarbitrator
    services.sh: start /etc/init.d/swapobjd
    Running usbarbitrator start
    Running swapobjd start
    swapobjd started
    services.sh: started /etc/init.d/swapobjd
    usbarbitrator started
    services.sh: started /etc/init.d/usbarbitrator
    services.sh: start /etc/init.d/sensord
    services.sh: start /etc/init.d/vmfstraced
    services.sh: start /etc/init.d/storageRM
    services.sh: start /etc/init.d/nfcd
    services.sh: start /etc/init.d/vvold
    services.sh: start /etc/init.d/rhttpproxy
    services.sh: start /etc/init.d/sdrsInjector
    services.sh: start /etc/init.d/lbtd
    services.sh: start /etc/init.d/hostd
    services.sh: start /etc/init.d/rabbitmqproxy
    Running vmfstraced start
    Running sensord start
    Running rhttpproxy start
    Running vvold start
    Running sdrsInjector start
    Running storageRM start
    Running lbtd start
    Running hostd start
    Running rabbitmqproxy start
    Running nfcd start
    /etc/init.d/vvold start, called by pid 583971
    VMFS Global Tracing is not enabled.
    services.sh: started /etc/init.d/vmfstraced
    nfcd started
    sensord started
    services.sh: started /etc/init.d/sensord
    services.sh: started /etc/init.d/nfcd
    net-lbt started
    sdrsInjector started
    services.sh: started /etc/init.d/sdrsInjector
    services.sh: started /etc/init.d/lbtd
    storageRM started
    services.sh: started /etc/init.d/storageRM
    vvold max reserve memory set to 200
    VMware HTTP reverse proxy started.
    services.sh: started /etc/init.d/rhttpproxy
    Ramdisk 'hostd' with estimated size of 750MB already exists
    rabbitmqproxy not up, probably not unconfigured yet.
    hostd started.
    services.sh: started /etc/init.d/hostd
    services.sh: started /etc/init.d/rabbitmqproxy
    WaitVvoldToComeUp /var/run/vmware/.vmware-vvol.started created
    vvold stopped after start!
    /var/run/vmware/.vmware-vvol.started is not created
    Successfully cleared vvold memory reservation
    services.sh: started /etc/init.d/vvold
    services.sh: start /etc/init.d/slpd
    Running slpd start
    Starting slpd
    services.sh: started /etc/init.d/slpd
    services.sh: start /etc/init.d/cdp
    services.sh: start /etc/init.d/smartd
    services.sh: start /etc/init.d/dcbd
    services.sh: start /etc/init.d/nscd
    services.sh: start /etc/init.d/lacp
    services.sh: start /etc/init.d/memscrubd
    Running smartd start
    Running dcbd start
    Running lacp start
    Running nscd start
    Running memscrubd start
    Running cdp start
    The checkPages boot option is FALSE, hence memscrubd could not be started.
    services.sh: started /etc/init.d/memscrubd
    nscd started
    services.sh: started /etc/init.d/nscd
    smartd started
    services.sh: started /etc/init.d/smartd
    LACP daemon started
    dcbd started
    services.sh: started /etc/init.d/dcbd
    services.sh: started /etc/init.d/lacp
    cdp started
    services.sh: started /etc/init.d/cdp
    services.sh: start /etc/init.d/vobd
    Running vobd start
    vobd started
    services.sh: started /etc/init.d/vobd
    services.sh: start /etc/init.d/vpxa
    Running vpxa start
    services.sh: started /etc/init.d/vpxa
    services.sh: start /etc/init.d/sfcbd-watchdog
    Running sfcbd-watchdog start
    Getting Exclusive access, please wait...
    Exclusive access granted.
    services.sh: started /etc/init.d/sfcbd-watchdog
    services.sh: start /etc/init.d/sfcbd
    services.sh: start /etc/init.d/wsman
    services.sh: start /etc/init.d/snmpd
    Running snmpd start
    Running sfcbd start
    Running wsman start
    Starting openwsmand
    This operation is not supported.
    Please use /etc/init.d/sfcbd-watchdog start
    services.sh: started /etc/init.d/sfcbd
    services.sh: started /etc/init.d/wsman
    root: snmpd has not been enabled.
    services.sh: started /etc/init.d/snmpd
    services.sh: start /etc/init.d/xorg
    services.sh: start /etc/init.d/vmsyslogd
    services.sh: start /etc/init.d/vmtoolsd
    services.sh: start /etc/opt/init.d/vmware-fdm
    Running vmware-fdm start
    Running vmtoolsd start
    Running vmsyslogd start
    Running xorg start
    Starting vmware-fdm:services.sh: started /etc/init.d/vmtoolsd
    vmsyslogd is running.
    services.sh: started /etc/init.d/vmsyslogd
    Setting the memory limit for fdm resource pool on this host to
            250 MB
            services.sh: started /etc/init.d/xorg
            success

            services.sh: started /etc/opt/init.d/vmware-fdm

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2115558

How to Fix Snapshot consolidation needed fails with a lock message?

There are few ways which you can opt for to fix this issue – you need to follow the below steps choose your methods.

Method – 1

Note the path of the disk files before removing or unlinking them from VM

You need to unlink your disk files /vmdk files from virtual machine (Do Not Delete the Disks – Only remove them from Virtual Machine)

Note: You VM must be Power off state when you do the complete activity

Once disks are unlinked from the VM – then you need to run Consolidate

Re-add disks from the datastore and power on your VM – Error message will be solved

–some peoples do also have success by restating the ESX management service from CLI “services.sh restart” but I don’t think this will help long term resolution.

Method – 2 (Best Option to fix the issue)

In this step you can solve the issue online – this means Virtual Machine will be stay powered on during this activity.

You need right click on VM and click Migrate – you need choose migrate disks to another available datastore – this will clear lock on disk files of the virtual machine.

Once storage migration is complete then you can run the consolation and it will fix the issue.

Method – 3 (Good Option to fix – but you need extra space on datastore)

For this option you need to power off you virtual machine and then we will clone the vmdk and that will consolidate the all disks files which are linked to the main disk files of the VM.

Login to ESX / ESXi Server from ssh console to run the below command

From the ESX/ESXi terminal:

vmkfstools -i input output -d format -a adaptertype

From the ESX/ESXi terminal:

vmkfstools -i “/vmfs/volumes/Datastore/examplevm/examplevm.vmdk” “/vmfs/volumes/Datastore 2/newexamplevm/newexamplevm.vmdk” -d thin -a buslogic

Warning: If virtual machine snapshots or delta disks are present, ensure that the source chosen is the current snapshot delta disk. Failing to do so results in an outdated destination copy. However, when using the command, the corresponding descriptor file for the latest delta disk needs to be used when cloning the disk.

For example to clone and/or convert virtual machine disk with snapshot (otherwise known as a delta disk), examplevm-000001.vmdk and like above, from Datastore to Datastore 2 , with a thin-provisioned destination copy, run this command:

From the ESX/ESXi terminal:

vmkfstools -i “/vmfs/volumes/Datastore/examplevm/examplevm-000001.vmdk” “/vmfs/volumes/Datastore 2/newexamplevm/newexamplevm.vmdk” -d thin -a buslogic

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2003638

http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=10051

P2V

Windows P2V error "fault.ManagedDiskOpenFault.summary"

https://communities.vmware.com/thread/479441?start=0&tstart=0 http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1019690 https://communities.vmware.com/thread/479441?start=0&tstart=0

Network card configuration missing after P2V using VMware Converter

The steps are:

Click Start, click Run, type cmd.exe, and then press ENTER.
Type set devmgr_show_nonpresent_devices=1, and then press ENTER.
Type Start DEVMGMT.MSC, and then press ENTER.
Click View, and then click Show Hidden Devices.
Expand the Network adapters tree.
Right-click the dimmed network adapter, and then click Uninstall.

Finally I configured the static IP on the NIC, and all was well.

http://thebackroomtech.com/2009/10/01/network-card-configuration-missing-after-p2v-using-vmware-converter/

Linux workstation

/etc/init.d/vmware restart
* Starting VMware USB Arbitrator ...    [ !! ]
* Starting VMware services ...          [ ok ]
Failed to initialize                    [ !! ]
* ERROR: vmware failed to start

brightmoon vmware # touch /tmp/x
brightmoon vmware # vmware-networks --migrate-network-settings /tmp/x
Restored network settings
brightmoon vmware # less /tmp/x
brightmoon vmware # rm /var/log/vmware/vmware-usbarb-22* -vf
removed ‘/var/log/vmware/vmware-usbarb-22412.log’
removed ‘/var/log/vmware/vmware-usbarb-22669.log’
removed ‘/var/log/vmware/vmware-usbarb-22877.log’
removed ‘/var/log/vmware/vmware-usbarb-22911.log’
removed ‘/var/log/vmware/vmware-usbarb-22945.log’
brightmoon vmware # vmware-networks --migrate-network-settings /tmp/x
Stopped all configured services on all networks
Restored network settings
brightmoon vmware # /etc/init.d/vmware restart
 * Starting VMware USB Arbitrator ... [ !! ]
 * Starting VMware services ...       [ ok ]
Started Bridge networking on vmnet0
Started all configured services on all networks[ ok ]

kill cancel task

[root@GRGESXi02:~] esxcli vm process list
Redhat_env6.0_landrayOAweixin_89
   World ID: 36079
   Process ID: 0
   VMX Cartel ID: 36077
   UUID: 42 02 ea ad 2a 98 03 07-0e e9 1a 82 36 47 36 6c
   Display Name: Redhat_env6.0_landrayOAweixin_89
   Config File: /vmfs/volumes/5618b866-7b466cdb-3be0-40f2e9638ca0/redhat_ent6.0_lanling_134/redhat_ent6.0_lanling_134.vmx

CentOS_v7%2f64_landrayOAkk_90
   World ID: 36087
   Process ID: 0
   VMX Cartel ID: 36086
   UUID: 42 0b ba 88 cf af 21 4e-60 8f 2f d5 90 32 f4 a9
   Display Name: CentOS_v7%2f64_landrayOAkk_90
   Config File: /vmfs/volumes/5618b866-7b466cdb-3be0-40f2e9638ca0/Redhat_90/Redhat_90.vmx

vSphere Data Protection 6.1
   World ID: 41570
   Process ID: 0
   VMX Cartel ID: 41569
   UUID: 42 0b eb 56 13 69 1c 40-ec fd e1 b6 7e 76 71 e9
   Display Name: vSphere Data Protection 6.1
   Config File: /vmfs/volumes/5625a51a-38b5b428-f929-40f2e9639878/vSphere Data Protection 6.1/vSphere Data Protection 6.1.vmx
[root@GRGESXi02:~] esxcli vm process kill -t soft -w 46015

Clearing partitioning information in ESXi using the DD utility

fdisk -l "/vmfs/devices/disks/DeviceName"

dd if=/dev/zero of="/vmfs/devices/disks/DeviceName" bs=512 count=34 conv=notrunc

dd if=/dev/zero of="/vmfs/devices/disks/DeviceName" bs=512 count=34 seek=SeekOffset conv=notrunc

rescan datastores

esxcli storage core adapter rescan --all

Troubleshooting

Useful SMART Information

Parameter                    Value Threshold Worst 
---------------------------- ----- --------- -----
Health Status                 OK     N/A       N/A
Media Wearout Indicator       N/A    N/A       N/A
Write Error Count             N/A    N/A       N/A
Read Error Count              114    6         100
Power-on Hours                90     0         90
Power Cycle Count             100    20        100
Reallocated Sector Count      2      36        2
Raw Read Error Rate           114    6         100
Drive Temperature             33     0         53
Driver Rated Max Temperature  67     45        47
Write Sectors TOT Count       200    0         200
Read Sectors TOT Count        N/A    N/A       N/A 
Initial Bad Block Count       100    99        10

https://cormachogan.com/2014/02/25/a-list-of-esxcli-storage-commands-you-cant-live-without/

关于snapshot快照整合问题

执行快照时,虚拟机上正在执行的其他活动可能会影响恢复到该快照的快照过程。从存储角度而言,执行快照的最佳时间是在未产生大量 I/O 负载时。从服务角度而言,执行快照的最佳时间是在虚拟机中没有应用程序与其他计算机通信时。如果虚拟机正在与其他计算机通信,特别是在生产环境中,最可能出现问题。例如,如果执行快照时虚拟机正在从网络中的服务器下载文件,虚拟机会继续下载文件,同时就其进度与服务器进行通信。如果恢复为此快照,虚拟机与服务器之间的通信将受到干扰,且文件传输失败。

内存快照

    生成快照的默认选择。捕获虚拟机内存状况时,快照会保留虚拟机的实时状况。当您要创建某一精确时间点的快照(例如升级仍在运行的软件)时,内存快照非常有用。生成了内存快照后,如果升级未按预期完成,或软件不符合您的预期,则可将虚拟机恢复到其以前的状态。

    捕获内存状况时,虚拟机的文件无需静默。如果未捕获内存状况,快照就不会保存虚拟机的实时状况,除非静默磁盘,否则磁盘就是崩溃一致的。

静默快照

    当静默虚拟机时,VMware Tools 会静默虚拟机的文件系统。静默操作可确保快照磁盘表示客户机文件系统的一致状况。静默快照适用于自动备份或定期备份。例如,您启动了快照,但不清楚虚拟机中正在发生的事情,并且需要多个最近备份用于恢复(而不是捕获内存状况),则可以静默文件。

    如果虚拟机电源已关闭或 VMware Tools 不可用,则 Quiesce 参数将不可用。您无法静默包含大容量磁盘的虚拟机。

vCenter Server Appliance 6.0 services fail due to 100% usage of /storage/seat disk

cd /opt/vmware/vpostgres/9.3/bin
./psql -d VCDB -U postgres
select event_type, host_name, count(*) from vpx_event group by host_name, event_type having count(*) > 10 order by count(*) desc;

truncate table vpx_event cascade;
truncate table vpx_event_arg cascade;

Increasing the disk space for the VMware vCenter Server Appliance in vSphere 6.0

Disk  VMDK)      Default Size  
 (Tiny VCSA)     Mount Point     Purpose
 VMDK1   12GB   / (10GB)
/boot (132MB)
SWAP (1GB)       Boot directory where the kernel images and boot load configurations are stored
 VMDK2   1.3GB  /tmp     Temporary directory used to store temporary files generated or used by services from vCenter Server
 VMDK3   25GB   SWAP     Swap directory used when the system is out of memory to swap to disk
 VMDK4   25GB   /storage/core    Core directory where core dumps from VPXD process from the vCenter Server are stored
 VMDK5   10GB   /storage/log     Log directory where vCenter Server stores all logs for the environment
 VMDK6   10GB   /storage/db      VMware Postgres database storage location
 VMDK7   5GB    /storage/dblog   VMware Postgres database logging location
 VMDK8   10GB   /storage/seat    Stats, Events, Alarms and Tasks (SEAT) directory for VMware Postgres
 VMDK9   1GB    /storage/netdump         VMware Netdump collector repository that stores ESXi dumps
 VMDK10          10GB   /storage/autodeploy      VMware Auto Deploy repository that stores the thinpackages used for stateless booting of ESXi hosts
 VMDK11          5GB    /storage/invsvc          VMware Inventory Service directory where the xDB, Inventory Service bootstrap configuration file, and tomcat configuration files reside

SSH login in
shell.set --enabled true

vpxd_servicecfg storage lvm autogrow

 * When the grow operation is successful, you see output similar to:

VC_CFG_RESULT=0

Import OVF Template - Unsupported hardware family 'vmx-06'

To avoid wasting time extracting/rebuilding with OVF tool you can just edit the OVF file, change the line that says "VMX-XX" (in my case was 10) to read "07" or whatever version works with your ESXI. Then use a hashing utility to calculate the SHA-1 of the edited OVF file and replace the corresponding SHA-1 hash in the ".mf" file.

frequent error message 'nmp_ThrottleLogForDevice:3178' in vmkernel.log

http://buildvirtual.net/analyzing-esxi-log-files-to-identify-storage-and-multipathing-problems/

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1029039

I found some very useful information to interpret the error message here: Analyzing ESXi Log Files to Identify Storage and Multipathing Problems

Based on this article I will interpret you error message, hopefully this will help some others. In your message we find the code H:0x0 D:0x2 P:0x0, this means the error is related to the Device (H: host, D: device, P:plugin status). On the vmware site (vSphere Documentation Center) we find the information required to understand the code 0x2, what means CHECK CONDITION. In the article you also find some references to t10.org about how to interpret the Valid sense data: 0x5 0x24 0x0, which is also in the log message. http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.troubleshooting.doc_50%2FGUID-E8304C8B-E2EA-459E-A545-8531C1BF12B0.html

Based on the following ref: SCSI Sense Keys we know that an ILLEGAL REQUEST was send to your storage device. But how bad is this? http://www.t10.org/lists/2sensekey.htm

Unable to connect to the MKS: Internal error

Worked on this part time for weeks. The shutdown/remove/re-add was all it took. Doesn't explain what happened, but at least I can get back in.

Met same issue on ESXi 6.0 after i added host to VC server. All VMs refused to open console with this error.

I tried to re-add first VM, and it works.

Then i shutdown second VM, open config and increase Video Memory from 4 MB up to 64MB. And it helps too. So it can be some bug with video resolution error.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=749640

Insufficient resources to satisfy vSphere HA failover level

http://www.orionscache.com/2014/08/fixing-insufficient-resources-error-in-vsphere-ha-high-availability/

https://support.software.dell.com/zh-cn/vranger/kb/127660

http://geekswing.com/geek/vmware-cpu-and-ram-reservations-fixing-insufficient-resources-to-satisfy-configured-failover-level-for-ha/

Agent can't send heartbeats

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2091510

IMPI log full

localcli hardware ipmi sel clear

vMotion about

http://frankdenneman.nl/2012/09/07/vsphere-5-1-vmotion-deepdive/

How is disk data migration via vMotion different from a Storage vMotion?

The main difference between vMotion and Storage vMotion is that vMotion does not “touch” the storage subsystem for copy operations of non-shared datastores, but transfers the disk data via an Ethernet network. Due to the possibilities of longer distances and higher latency, disk data is transferred asynchronously. To cope with higher latencies, a lot of changes were made to the buffer structure of the vMotion process. However if vMotion detects that the Guest OS issues I/O faster than the network transfer rate, or that the destination datastore is not keeping up with the incoming changes, vMotion can switch to synchronous mirror mode to ensure the correctness of data.

How-to rename VMNIC in VMware ESXi

http://www.vladan.fr/how-to-rename-vmnic-in-vmware-esxi/

To restart a vCenter Server and/or Platform Services Controller service using the command-line:

To restart a vCenter Server and/or Platform Services Controller service using the command-line:

Backing up and restoring the vCenter Server Appliance vPostgres database

ESXi web client Embedded Host Client – VMware Labs:

https://labs.vmware.com/flings/esxi-embedded-host-client?ClickID=dxroxcxz0cbwwz2ns2hoyocthh0xchokzybk

# Install
scp /path/to/downloaded/vib/esxui.vib root@<esxip>:/tmp
ssh root@<esxip> esxcli software vib install -v /tmp/esxui.vib
# Upgrade 
# ssh root@<esxip> esxcli software vib update -v /tmp/esxui.vib

# Removal
# ssh root@ esxcli software vib remove -n esx-ui

How to quickly check ESXi logs from web browser

http://IP_of_Your_ESXi/host

Create a VMDK Larger than 16TB

vmkfstools -c <size> <newvmdk> -a lsilogic -d thin

Example (creates a 25T thin provisioned vmdk named "testvm2.vmdk" with a lsilogic adapter) : # vmkfstools -c 25t testvm2.vmdk -a lsilogic -d thin #

Convert vmdk to raw

qemu-img convert -f vmdk disk.vmdk -O raw disk.raw

# Know the disk size:
qemu-img info -f raw disk.raw 
image: disk.raw
file format: raw
virtual size: 7.8G (8388608000 bytes)
disk size: 2.5G

UUID

This tells you which VMFS volumes are mapped to which physical disk device:

# esxcfg-scsidevs -m

References

http://www.virtuallyghetto.com/2014/12/handy-vcsa-vcenter-server-appliance-operational-kb-resources.html

désert/VMware (last edited 2019-03-24 04:21:37 by merlyn)