Cross vCenter vMotion

https://labs.vmware.com/flings/cross-vcenter-workload-migration-utility

brightmoon /tmp/xvm-1.0 # java -jar xvm-1.0.jar

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v1.5.1.RELEASE)

15:38:32 INFO  Starting ApiController on brightmoon with PID 2888 (/tmp/xvm-1.0/xvm-1.0.jar started by root in /tmp/xvm-1.0)
15:38:32 DEBUG Running with Spring Boot v1.5.1.RELEASE, Spring v4.3.6.RELEASE
15:38:32 INFO  No active profile set, falling back to default profiles: default
15:38:33 INFO  HV000001: Hibernate Validator 5.3.4.Final
15:38:34 INFO  Starting service Tomcat
15:38:34 INFO  Starting Servlet Engine: Apache Tomcat/8.5.11
15:38:34 INFO  Initializing Spring embedded WebApplicationContext
15:38:35 INFO  Context refreshed
15:38:35 INFO  Found 1 custom documentation plugin(s)
15:38:35 INFO  Scanning for api listing references
15:38:35 INFO  Initializing ProtocolHandler ["http-nio-8080"]
15:38:35 INFO  Starting ProtocolHandler [http-nio-8080]
15:38:35 INFO  Using a shared selector for servlet write/read
15:38:35 INFO  Started ApiController in 3.578 seconds (JVM running for 4.142)
15:38:35 INFO  Cross vCenter Workload Migration Utility Initialized!
9915:38:49 INFO  Initializing Spring FrameworkServlet 'dispatcherServlet'

http://127.0.0.1:8080/

Unregister PNID

root@photon-machine [ ~ ]# cmsso-util unregister --node-pnid 192.168.16.26 --username administrator@vsphere.local
Password:
2018-02-07T10:19:56.061Z   Running command: ['/usr/lib/vmware-vmafd/bin/dir-cli', 'service', 'list', '--login', 'administrator@vsphere.local']
2018-02-07T10:19:56.104Z   Done running command

Reconfigure a Standalone vCenter Server with an Embedded Platform Services Controller to a vCenter Server with an External Platform Services Controller

https://docs.vmware.com/en/VMware-vSphere/6.0/com.vmware.vsphere.install.doc/GUID-EA6CD275-DD0A-4ADC-9512-B2A006DE00F1.html

Deploy or install the external Platform Services Controller instance as a replication partner of the existing embedded Platform Services Controller instance in the same vCenter Single Sign-On site.

Log in to the vCenter Server instance with an embedded Platform Services Controller

Verify that all Platform Services Controller services are running

Run the service-control --status --all command.

The Platform Services Controller services that must be running are VMware License Service, VMware Identity Management Service, VMware Security Token Service, VMware Certificate Service, and VMware Directory Service.

Configure Replication Agreement Between All External Platform Services Controller Instances

/usr/lib/vmware-vmdir/bin/vdcrepadmin -f showpartners -h psc_fqdn_or_static_ip -u administrator 

If there is an external Platform Services Controller instance that is not in replication agreement with another external Platform Services Controller instance, run the vdcrepadmin command with the createagreement parameter against this Platform Services Controller instance to join it to another external Platform Services Controller instance.

If you are using a connection to a vCenter Server Appliance or Platform Services Controller appliance, run the following command. If the vCenter Server with an embedded Platform Services Controller instance and the external Platform Services Controller instance are not direct replication partners, create such a replication agreement. For a vCenter Server Appliance with an embedded Platform Services Controller, from the appliance Bash shell, run the following command.

/usr/lib/vmware-vmdir/bin/vdcrepadmin -f createagreement -h localhost -H psc_fqdn_or_static_ip -u administrator
/usr/lib/vmware-vmdir/bin/vdcrepadmin -f createagreement -2 -h psc_fqdn_or_static_ip -H partner_psc_fqdn_or_static_ip -u administrator 

Repeat Step above against each external Platform Services Controller instance that is not in replication agreement with another external Platform Services Controller instance.

Results

The vCenter Server instances with an embedded Platform Services Controller are demoted, and the vCenter Server instances are redirected to the external Platform Services Controller instances.

MAC Address Management During Migration Between vCenter Server Systems

When you move a virtual machine between vCenter Server instances, the environment specifically handles MAC address migration to avoid address duplication and loss of data in the network.

In an environment with multiple vCenter Server instances, when a virtual machine is migrated, its MAC addresses are transferred to the target vCenter Server. The source vCenter Server adds the MAC addresses to a black list so that it does not assign them to newly created virtual machines.

To reclaim unused MAC addresses from the black list, contact VMware Technical Support for assistance.

Run the cmsso-util reconfigure command

cmsso-util reconfigure --repoint-psc psc_fqdn_or_static_ip --username username --domain-name domain_name --passwd password [--dc-port port_number] 

Here, psc_fqdn_or_static_ip is the system name used to identify the external Platform Services Controller instance. This system name must be an FQDN or a static IP address.

Repoint an External Deployment to an Alternate Platform Services Controller in the Same Site (Intra-Site)

Repoint an External Deployment to an Same Platform Services Controller in the Same Site (Intra-Site)

root@photon-machine [ ~ ]# cmsso-util repoint --repoint-psc 10.97.2.252
Validating Provided Configuration ...
Error: The provided Platform Services Controller(PSC) 10.97.2.252 is already the current active PSC of this vCenter Server

root@photon-machine [ ~ ]# cmsso-util unregister  --username administrator@vsphere.local --node-pnid 10.97.2.252
Password:
This command is supported only on PSC and vCenter with embedded PSC nodes.

Oracle

使用脚本创建本地或远程 Oracle 数据库

 CREATE SMALLFILE TABLESPACE "VPX" DATAFILE '/data/vcdb/vpx01.dbf'
 SIZE 1G AUTOEXTEND ON NEXT 10M MAXSIZE UNLIMITED LOGGING EXTENT MANAGEMENT LOCAL SEGMENT 
 SPACE MANAGEMENT AUTO;

配置 Oracle 数据库用户

 CREATE USER "VSAN" PROFILE "DEFAULT" IDENTIFIED BY "oracle" DEFAULT TABLESPACE
 "VPX" ACCOUNT UNLOCK;
 grant connect to VSAN;
 grant resource to VSAN;
 grant create view to VSAN;
 grant create sequence to VSAN;
 grant create table to VSAN;
 grant create materialized view to VSAN;
 grant execute on dbms_lock to VSAN;
 grant execute on dbms_job to VSAN;
 grant select on dba_lock to VSAN;
 grant select on dba_tablespaces to VSAN;
 grant select on dba_temp_files to VSAN;
 grant select on dba_data_files to VSAN;
 grant select on v_$session to VSAN;
 grant unlimited tablespace to VSAN;

 alter user "VSAN" quota unlimited on "VPX";
  1. 在成功安装了具有 Oracle 数据库的 vCenter Server 之后,您可以撤销下列特权。

 revoke select on dba_tablespaces from VSAN;
 revoke select on dba_temp_files from VSAN;
 revoke select on dba_data_files from VSAN;

运行以下 SQL 命令向 vCenter Server 数据库用户授予其他权限

grant select on v_$system_event to VSAN;
grant select on v_$sysmetric_history to VSAN;
grant select on v_$sysstat to VSAN;
grant select on dba_data_files to VSAN;
grant select on v_$loghist to VSAN;

vCenter 数据库监控已启用。

@/home/oracle/dbschema/VCDB_oracle.sql
@/home/oracle/dbschema/VCDB_views_oracle.sql
@/home/oracle/dbschema/insert_stats_proc_oracle.sql
@/home/oracle/dbschema/load_stats_proc_oracle.sql
@/home/oracle/dbschema/purge_stat2_proc_oracle.sql
@/home/oracle/dbschema/purge_stat3_proc_oracle.sql
@/home/oracle/dbschema/purge_usage_stats_proc_oracle.sql
@/home/oracle/dbschema/stats_rollup1_proc_oracle.sql
@/home/oracle/dbschema/stats_rollup2_proc_oracle.sql
@/home/oracle/dbschema/stats_rollup3_proc_oracle.sql
@/home/oracle/dbschema/cleanup_events_oracle.sql
@/home/oracle/dbschema/delete_stats_proc_oracle.sql
@/home/oracle/dbschema/load_usage_stats_proc_oracle.sql
@/home/oracle/dbschema/TopN_DB_oracle.sql
@/home/oracle/dbschema/calc_topn1_proc_oracle.sql
@/home/oracle/dbschema/calc_topn2_proc_oracle.sql
@/home/oracle/dbschema/calc_topn3_proc_oracle.sql
@/home/oracle/dbschema/calc_topn4_proc_oracle.sql
@/home/oracle/dbschema/clear_topn1_proc_oracle.sql
@/home/oracle/dbschema/clear_topn2_proc_oracle.sql
@/home/oracle/dbschema/clear_topn3_proc_oracle.sql
@/home/oracle/dbschema/clear_topn4_proc_oracle.sql
@/home/oracle/dbschema/rule_topn1_proc_oracle.sql
@/home/oracle/dbschema/rule_topn2_proc_oracle.sql
@/home/oracle/dbschema/rule_topn3_proc_oracle.sql
@/home/oracle/dbschema/rule_topn4_proc_oracle.sql
@/home/oracle/dbschema/process_license_snapshot_oracle.sql
@/home/oracle/dbschema/l_purge_stat2_proc_oracle.sql
@/home/oracle/dbschema/l_purge_stat3_proc_oracle.sql
@/home/oracle/dbschema/l_stats_rollup1_proc_oracle.sql
@/home/oracle/dbschema/l_stats_rollup2_proc_oracle.sql
@/home/oracle/dbschema/l_stats_rollup3_proc_oracle.sql
@/home/oracle/dbschema/job_dbm_performance_data_oracle.sql
@/home/oracle/dbschema/process_performance_data_oracle.sql

@/home/oracle/dbschema/job_schedule1_oracle.sql
@/home/oracle/dbschema/job_schedule2_oracle.sql
@/home/oracle/dbschema/job_schedule3_oracle.sql 
@/home/oracle/dbschema/job_cleanup_events_oracle.sql
@/home/oracle/dbschema/job_topn_past_day_oracle.sql
@/home/oracle/dbschema/job_topn_past_week_oracle.sql
@/home/oracle/dbschema/job_topn_past_month_oracle.sql
@/home/oracle/dbschema/job_topn_past_year_oracle.sql

drop vpx

SQL> drop user vpxadmin cascade;

User dropped.

SQL> DROP TABLESPACE vpx INCLUDING CONTENTS AND DATAFILES;

Tablespace dropped.

upgrade vCenter

Command> com.vmware.software-packages staged --url http://xxx/
Command> com.vmware.software-packages install --staged

How to Upgrade from VCSA 5.5 to 6.0

https://pubs.vmware.com/vsphere-60/index.jsp?topic=%2Fcom.vmware.vsphere.upgrade.doc%2FGUID-5FCA78EC-8637-43A4-8B28-24624E4D5EBA.html

  1. Do a backup or create a snapshot of your existing VCSA.
  2. Verify that the clocks of all machines on the vSphere network are synced.
  3. Verify that the ESXi host on which you deploy the vCenter Server Appliance is not in lockdown or maintenance mode.

FQDN issue

    vCenterServer FQDN vcsa55.lab.local does not match DNS servers “localhost.localdom,localhost” and ip addresses “10.10.7.151” from VC certificate
        Examine the VC certificate and make sure it is valid and point to vCenterServer FQDN.
  1. Log in to the source vCenter Server Appliance Web interface at https://Source_vCenter_Server_Appliance_FQDN:5480/.

  2. Click the Admin tab.
  3. Regenerate certificates:
    • vCenter Server 5.1: Select Toggle certificate setting so that the Certificate regeneration enabled displays Yes. vCenter Server 5.5: Select Yes under Certificate regeneration enabled.
  4. Click Submit.
  5. Reboot the vCenter Server Appliance.
  6. After the vCenter Server Appliance reboots, confirm that the Certificate regeneration enabled option is set to disabled.

    https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2110772

    https://communities.vmware.com/thread/528020?start=0&tstart=0

vCenter Server Downgrade options

http://www.vladan.fr/esxi-6-0-downgrade-options-lab-time/

Upgrade vCenter Server Appliance 6.0 U2 to 6.5

http://www.jonkensy.com/upgrade-vcenter-server-appliance-6-0-u2-to-6-5/

http://www.virtualizationhowto.com/2016/11/upgrade-vmware-vcenter-vcsa-appliance-6-0-to-6-5/

Troubleshooting

On my vCenter Server Appliance VM, I get "soft lockup - CPU## stuck for ##s!" message.

I've tried back the vcsa on the hdd. No good.

addition or reconfiguration of network adapters attached to non-ephemeral distributed virtual port groups

  1. reinstall VCSA
  2. recreate VDS or VSS switch
  3. migrate vmnicS&PortGroup one by one host.

Error naming or renaming a VM file

 vmodl.fault.SystemError:
 --> Result:
 --> (vmodl.fault.SystemError) {
   -->    faultCause = (vmodl.MethodFault) null, 
   -->    reason = "Error naming or renaming a VM file.", 
   -->    msg = ""
   --> }
   --> Args:
   --> 
   2016-09-12T01:30:09.059Z info vpxd[7F4E7A6B7700] [Originator@6876 sub=vpxLro opID=CA18011C-00000115-8d] [VpxLRO] -- FINISH task-4554
   2016-09-12T01:30:09.059Z info vpxd[7F4E7A6B7700] [Originator@6876 sub=Default opID=CA18011C-00000115-8d] [VpxLRO] -- ERROR task-4554 -- vm-48 -- vim.VirtualMachine.reloca
   te: vmodl.fault.SystemError:

Collecting diagnostic information for VMware vCenter Server 4.x, 5.x and 6.0 (1011641)

Collecting diagnostic information from vCenter 6.0 Server Appliance or external Platform Service Controller using a web browser

 Open a web browser and navigate to https://vCenter_server_FQDN:443/appliance/support-bundle
 When prompted enter the root credentials and click Enter.
 The download will begin automatically as vm-support.tgz. 

Quick Tip – How to quickly find the release & build number on VCSA

vcenter:/var/log/vmware/vapi/endpoint # vpxd -v
VMware VirtualCenter 6.0.0 build-4541948

VMware vCenter 6 Web Console doesn’t load or very slow

vSphere Web client unresponsive in vCenter Server

"back-end property provider" error while adding new datastore in vSphere Web Client

vcenter:/var/log/vmware/vapi/endpoint # tail  /var/log/vmware/vsphere-client/logs/vsphere_client_virgo.log 
[2017-12-07T08:29:22.440Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.541Z] [INFO ] console-message-pool-2464204  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.541Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.665Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.679Z] [INFO ] console-message-pool-2464204  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.871Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:22.929Z] [INFO ] console-message-pool-2464204  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:24.506Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:29:53.211Z] [INFO ] console-message-pool-2709139  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 
[2017-12-07T08:30:07.019Z] [INFO ] console-message-pool-2464204  c.vmware.vise.vim.commons.mks.tomcat.RemoteConsoleMessageInbound  Encountered EOF character, sleeping for 100 ms. 

[2017-12-07T08:59:07.471Z] [WARN ] http-bio-9090-exec-383388    70806777 107443 204980 com.vmware.vise.data.query.profiling.ProfilingUtil                The Data Service detected slow execution:
The query batch execution took too long: 258983 milliseconds.



[2017-12-07T08:59:07.476Z] [ERROR] http-bio-9090-exec-383388    70806777 107443 204980 com.vmware.vise.data.query.impl.DataServiceImpl                   Error occurred while executing query:
QuerySpec
   QueryName: dam-auto-generated: LicensingGlobalDataRetriever:dr-7
   ResourceSpec
      Constraint: ObjectIdentityConstraint
         TargetType: LicensingGlobalData
         Target: urn:vri:LicensingGlobalData:licensing_service_version1
      PropertySpec[1]
         ProviderType: LicensingGlobalData
         Relationship:
         Properties[1]
            PropertyName: licensingGlobalData
   ResultSpec:
      Offset: 0
      MaxResultCount: -1
      OrderingCriteria
         OrderPropertySpec[1]
            ProviderType: Object
            Relationship: null
            SortType: ASCENDING
            OrderingProperties[1]
               PropertyName: id
      Facets: null
   Options:
      REDUCE_QUERIES=true
 com.vmware.vise.data.query.DataServiceException: The query execution timed out because of a back-end data adapter 'com.vmware.license.client.cis.adapter.LicensingGlobalDataProviderAdapterImpl' which took more than 120 seconds.
        at com.vmware.vise.data.query.impl.DataAdapterUtil.processDataAdapterTaskException(DataAdapterUtil.java:154)
        at com.vmware.vise.data.query.impl.DataAdapterUtil.executeAdapterTasks(DataAdapterUtil.java:120)
        at com.vmware.vise.data.query.impl.DataAdapterRetriever.runDataTasks(DataAdapterRetriever.java:385)
        at com.vmware.vise.data.query.impl.DataAdapterRetriever.invokeDataAdapters(DataAdapterRetriever.java:334)
        at com.vmware.vise.data.query.impl.DataAdapterRetriever.getResultsFromDataAdapters(DataAdapterRetriever.java:79)
        at com.vmware.vise.data.query.impl.DataManager.getResultsFromDataAdapters(DataManager.java:178)
        at com.vmware.vise.data.query.impl.DataServiceImpl.getResults(DataServiceImpl.java:349)
        at com.vmware.vise.data.query.impl.DataServiceImpl.getData(DataServiceImpl.java:215)
        at sun.reflect.GeneratedMethodAccessor308.invoke(Unknown Source)
...

[2017-12-07T08:59:07.711Z] [ERROR] data-service-pool-8130405    70806778 107443 204980 c.v.v.client.sso.admin.impl.PrincipalManagerPropertyProvider      Not able to password expiration info for current user com.vmware.vsphere.client.sso.admin.exception.SsoBackendException: A vCenter Single Sign-On service error occurred
        at com.vmware.vsphere.client.sso.admin.exception.FallbackExceptionTransformer.transform(FallbackExceptionTransformer.java:30)
        at com.vmware.vsphere.client.sso.admin.exception.BackendExceptionTransformer.transformException(BackendExceptionTransformer.java:96)
        at com.vmware.vsphere.client.sso.admin.impl.AdminServiceBase.transformBackendException(AdminServiceBase.java:94)
        at com.vmware.vsphere.client.sso.admin.impl.PrincipalManagementServiceImpl.getSelfPasswordExpiration(PrincipalManagementServiceImpl.java:518)
        at com.vmware.vsphere.client.sso.admin.impl.PrincipalManagerPropertyProvider.getSelfPasswordExpiration(PrincipalManagerPropertyProvider.java:127)
        at sun.reflect.GeneratedMethodAccessor2666.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at com.vmware.vise.data.query.impl.ServicePropertyProviderAdapter.invokeMethod(ServicePropertyProviderAdapter.java:285)
        at com.vmware.vise.data.query.impl.ServicePropertyProviderAdapter.getProperties(ServicePropertyProviderAdapter.java:127)
        at com.vmware.vise.data.query.impl.DataManager.getDataFromPropertyProvider(DataManager.java:1403)
        at com.vmware.vise.data.query.impl.DataManager.getResultFromPropertyProvider(DataManager.java:1375)
        at com.vmware.vise.data.query.impl.DataManager.access$000(DataManager.java:79)
        at com.vmware.vise.data.query.impl.DataManager$1.call(DataManager.java:884)
        at com.vmware.vise.data.query.impl.DataManager$1.call(DataManager.java:880)
        at com.vmware.vise.util.concurrent.ExecutorUtil$3.call(ExecutorUtil.java:630)
        at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingCallable.call(ExecutorUtil.java:984)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

vcenter:/var/log/vmware/vapi/endpoint # less endpoint.log
        at com.vmware.vapi.internal.core.abort.AbortHandleImpl.abort(AbortHandleImpl.java:39)
        at com.vmware.vapi.endpoint.api.TimedApiProvider$1.run(TimedApiProvider.java:58)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(Unknown Source)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
2017-12-07T08:28:26.107Z | INFO  | state-manager1            | MetadataSynchronizationBuilder | rebuild
2017-12-07T08:28:26.127Z | INFO  | state-manager1            | MetadataSynchronizationBuilder | rebuild finished.
2017-12-07T08:28:26.200Z | INFO  | state-manager1            | DefaultStateManager            | lock
2017-12-07T08:28:26.200Z | INFO  | state-manager1            | DefaultStateManager            | State changed.
2017-12-07T08:28:26.200Z | INFO  | state-manager1            | DefaultStateManager            | unlock
2017-12-07T08:28:28.670Z | ERROR | jetty-default-157583      | ServletHelper                  | SSO verification failed; client:10.170.1.14
com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil$SsoAuthException: java.lang.NumberFormatException: null
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:194)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeader(SsoOverRestVerifierUtil.java:143)
        at com.vmware.cis.cm.common.endpoint.ServletHelper.getVerifiedInputStream(ServletHelper.java:267)
        at com.vmware.cis.cm.common.endpoint.ServletHelper.requestPrologue(ServletHelper.java:180)
        at com.vmware.cis.cm.common.endpoint.HealthStatusServlet.doGet(HealthStatusServlet.java:105)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at com.vmware.vapi.endpoint.common.ProxyServlet.service(ProxyServlet.java:52)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
        at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1669)
        at org.eclipse.jetty.servlets.DoSFilter.doFilterChain(DoSFilter.java:470)
        at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:322)
        at org.eclipse.jetty.servlets.DoSFilter.doFilter(DoSFilter.java:292)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at com.vmware.vapi.endpoint.http.RequestSizeFilter.doFilter(RequestSizeFilter.java:59)
        at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
        at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
        at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
        at org.eclipse.jetty.server.Server.handle(Server.java:497)
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
        at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NumberFormatException: null
        at java.lang.Integer.parseInt(Unknown Source)
        at java.lang.Integer.parseInt(Unknown Source)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.parseSecurityHeader(SsoOverRestVerifierUtil.java:72)
        at com.vmware.cis.services.common.sso.SsoOverRestVerifierUtil.verifySecurityHeaderImpl(SsoOverRestVerifierUtil.java:171)
        ... 34 more