Todays post I would like to share yet another wired experience during the VRSLCM deployment with VCF DNS , Reverse looking configuration of , AVN all was good but it failed to deploy the vRSLCM 8.6.2 the VM keeps rebooting . Lets look into the logs and how we fixed the issue.
Error from the SDDC GUI Sub Task
Issue : VRSLCM_OVF_DEPLOYMENT_FAILED vRSLCM OVF deployment failed, check if the appliance booted properly.
After the VCF bringup we downloaded the vRSLCM 8.6.2 image from the depot . Configuration of AVN , DNS reverse and forward was done along with the MTU. The ovf deployment of VRSLCM failed and the VM keeps reboot .
Since we are doing the deployment of VRSLCM from VCF , ssh to the SDDC manager and check the /var/log/vmware/vcf/domainmanager/domainmanage.log
domainmanager.2022-07-08.0.log:
LocalProcess INFO: 2022-07-08 13:33:08 - Powering on VM: bnc-B1c1-vrslcm
LocalProcess INFO: 2022-07-08 13:33:08 -
LocalProcess INFO: 2022-07-08 13:33:08 - Task progress: 0%
LocalProcess INFO: 2022-07-08 13:33:09 - Task progress: 1%
LocalProcess INFO: 2022-07-08 13:33:09 - Task progress: 19%
LocalProcess INFO: 2022-07-08 13:33:09 - Task progress: 37%
LocalProcess INFO: 2022-07-08 13:33:09 - Task progress: 64%
LocalProcess INFO: 2022-07-08 13:33:09 - Task progress: 91%
LocalProcess INFO: 2022-07-08 13:33:09 - Task Completed
LocalProcess INFO: 2022-07-08 13:34:29 - Waiting for IP address...
LocalProcess INFO: 2022-07-08 13:34:29 - Received IP address: 192.168.40.10
LocalProcess INFO: 2022-07-08 13:34:29 - Warning:
LocalProcess INFO: 2022-07-08 13:34:29 - - The manifest is present but user flag causing to skip it
LocalProcess INFO: 2022-07-08 13:34:29 - Completed successfully
2022-07-08T13:34:29.576+0000 DEBUG [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.e.s.c.util.LocalProcessService,dm-exec-5] Execute method with timeout period completed.
2022-07-08T13:34:29.576+0000 INFO [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.e.s.c.v.v.d.BaseVrslcmDeployer,dm-exec-5] vRSLCM VM bnc-B1c1-vrslcm with IP address 192.168.40.10 and hostname bnc-B1c1-vrslcm.vmware.com is successfully deployed.
From the above events the VM bnc-B1c1-vrslcm got deployed and we can see the VM form the VC , when open the console it keep rebooting and the boot configuration is not yet completed.
After the VM got deployed it was trying to download the certs from /opt/vmware/vcf/domainmanager/certs/cacerts and import it but it failed. this was our issue is.
Downloading certs bnc-B1c1-vrslcm.vmware.com, /opt/vmware/vcf/domainmanager/certs/cacerts, changeit
2022-07-08T13:34:29.582+0000 WARN [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.v.s.config.TrustAllTrustManager,dm-exec-5] Trusting server
2022-07-08T13:34:29.626+0000 INFO [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.e.s.c.v.v.d.Vrslcm8Deployer,dm-exec-5] vRSLCM service still not deployed: Failed to import trusted certificate
2022-07-08T13:49:31.075+0000 ERROR [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.e.s.v.v.DeployVrslcmContract,dm-exec-5] vRSLCM deployment failed!
2022-07-08T13:49:31.075+0000 ERROR [vcf_dm,6d373dfb74974fcd,0a3d] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-5] [IK8NDM] VRSLCM_OVF_DEPLOYMENT_FAILED vRSLCM OVF deployment failed, check if the appliance booted properly.
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: vRSLCM OVF deployment failed, check if the appliance booted properly.
When we check the opt/vmware/vcf/domainmanager/certs/ we found the file size was 0 probably got corrupted .
Cause : The failure was down to the cacerts file located at opt/vmware/vcf/domainmanager/certs/ on the SDDC Manager. We believe that most likely this file was corrupt and therefore when it copied the certs from the vRSLCM VM it was not able to write the file to here.
Workaround :
Snapshot the SDDC Manager .
Removed the file cacerts file from the path (opt/vmware/vcf/domainmanager/certs/ )
After a few minutes the deployment was completed successfully.
Comments