Issue:
PowerChute VM fails to start with the ESXi host after power is restored.
Product Line:
PowerChute Network Shutdown v5.0.
Environment:
PowerChute Network Shutdown configured with vSAN/VMWare or any HCI support
Cause:
Per VMware recommendations, HA is disabled during the shutdown process. This will prevent HA from re-starting the PowerChute VM.
VMware Auto-start policy cannot be used in HCI environments where there is a dependency on cluster services being available before VMs can be powered on. If Hosts are in maintenance mode on start-up this will also prevent Auto-start policy from powering on the PowerChute VM.
Solution:
1. Disable the option to turn off HA in PowerChute. This will allow HA to re-start the PowerChute VM. NOTE: This will not work in a VxRail configuration because the PowerChute VM is shut down.
2. There are two scripts compressed as FAQ000258432_Scripts.zip attached to this KBase that may be used to power on the PowerChute VM.
The 1st script (migrate_PowerChute.ps1) is a powershell script that migrates PowerChute to the 1st host in the cluster as part of the shutdown sequence. This is run from the PowerChute Appliance.
Change the values in the lines highlighted in Bold/Italic below for your environment.
#!/usr/bin/env pwsh
$server = "<vcenter_ip_address>" #"provide Vcenter server IP/hostname"
$username = "<vcenter_user>" #"provide username to access vCenter"
$password = "<vcenter password>" #"Provide Password to access vCenter server"
$targetHost = "<target_host>" #”Provide hostname or IP as is appears in vCenter inventory
$PowerChuteVM = "<PowerChute VM Name>" # “Provide the name of the PowerChute VM
$env:HOME = '/root'
Set-PowerCLIConfiguration -Scope User -ParticipateInCEIP $false -Confirm:$false
Connect-VIServer $server -Protocol https -User $username -password $password
$pcns_vm = Get-VM -Name $PowerChuteVM
Move-VM $pcns_vm -Destination $targetHost
Copy the migrate_PowerChute.ps1 script to the /opt/APC/PowerChute/user_files folder on the PowerChute Appliance via scp.
Run the command chmod +x migrate_PowerChute.ps1 to grant execute permissions.
Configure PowerChute to run the script using the Run command action for an event on the Event Configuration page or the Run Command feature on the Shutdown Settings page.
NOTE: By default, PowerCLI will not connect to a vCenter Server that uses a self-signed or untrusted SSL certificate.
Download the vCenter Server Root certificates:
NOTE: If the .zip opens in the browser right, right-click inside the page and select save as. Save the file as a compressed file.
Extract the contents of the zip and copy the Linux certificates (lin folder) to the PowerChute appliance via SCP.
Copy all the certificates (*.0 extension) to /etc/pki/ca-trust/source/anchors folder and run the command “update-ca-trust”
Confirm that the connection to vCenter Server works in powercli without any certificate errors:
Pwsh
Connect-ViServer -Server <vCenter Server> (enter your credentials when prompted)
The migration script needs to be executed when vCenter Server is available. In a VxRail configuration, a shutdown command file can be used. Ensure you allow enough time for the script to complete and PowerChute VM to shut down before the Network Management Card sends the Cluster Shutdown request to VxRail Manager.
In an environment where vCenter Server is deployed as a VM and being shut down you can run the migration script via an event driven command file.
The 2nd script is a python script that is executed on the 1st ESXi host in the cluster on start-up – this waits until the PowerChute VM can be found on the host (Cluster services are running again) and then powers it on.
Edit start_pcns.py enter the name of your PowerChute VM at the bottom of the file:
# Usage example
pcns_vm_name = "<PowerChute VM Name>"
pcns_vm_id = get_world_id(pcns_vm_name)
print(f"World ID {pcns_vm_id}")
wait_until_powered_on(pcns_vm_id)
Do not modify any other lines in this file.
On the 1st ESXi host in the cluster create a new folder called scripts on a local disk that has persistent storage and upload python script.
For example, using the vSphere Web UI:
Log into vCenter Server.
Click on the 1st node in the cluster.
Select Datastores tab.
Click on the local datastore.
In the datastore view click on New Folder:
Enter a name for the new folder e.g. scripts and Click OK
Select the new folder and click Upload Files
Browse to the location of start_pcns.py and upload the file.
NOTE: The upload may fail if the VMware Root CA certs have not been imported to your browser.
Please refer to https://kb.vmware.com/article/2147256 for instructions on how to address this.
Select the scripts folder in the file-hand tree view and confirm that the file has ben uploaded:
Click on the Summary to view the path to the Datastore:
SSH to the ESXi host.
Run the following command:
chmod +x /vmfs/volumes/<datastore id>/scripts/start_pcns.py
We then need to ensure that the script runs at start-up when the host boots. To do this we can add a line to /etc/rc.local.d/local.sh file.
Run the command vi /etc/rc.local.d/local.sh
Before the line “exit 0” add the line python /vmfs/volumes/<datastore id>/scripts/start_pcns.py &
NOTE: The & symbol is added so that the python script runs in the background and allows the local.sh script to continue.
Save the file and exit the SSH session.
PowerChute VM fails to start with the ESXi host after power is restored.
Product Line:
PowerChute Network Shutdown v5.0.
Environment:
PowerChute Network Shutdown configured with vSAN/VMWare or any HCI support
Cause:
Per VMware recommendations, HA is disabled during the shutdown process. This will prevent HA from re-starting the PowerChute VM.
VMware Auto-start policy cannot be used in HCI environments where there is a dependency on cluster services being available before VMs can be powered on. If Hosts are in maintenance mode on start-up this will also prevent Auto-start policy from powering on the PowerChute VM.
Solution:
1. Disable the option to turn off HA in PowerChute. This will allow HA to re-start the PowerChute VM. NOTE: This will not work in a VxRail configuration because the PowerChute VM is shut down.
2. There are two scripts compressed as FAQ000258432_Scripts.zip attached to this KBase that may be used to power on the PowerChute VM.
The 1st script (migrate_PowerChute.ps1) is a powershell script that migrates PowerChute to the 1st host in the cluster as part of the shutdown sequence. This is run from the PowerChute Appliance.
Change the values in the lines highlighted in Bold/Italic below for your environment.
#!/usr/bin/env pwsh
$server = "<vcenter_ip_address>" #"provide Vcenter server IP/hostname"
$username = "<vcenter_user>" #"provide username to access vCenter"
$password = "<vcenter password>" #"Provide Password to access vCenter server"
$targetHost = "<target_host>" #”Provide hostname or IP as is appears in vCenter inventory
$PowerChuteVM = "<PowerChute VM Name>" # “Provide the name of the PowerChute VM
$env:HOME = '/root'
Set-PowerCLIConfiguration -Scope User -ParticipateInCEIP $false -Confirm:$false
Connect-VIServer $server -Protocol https -User $username -password $password
$pcns_vm = Get-VM -Name $PowerChuteVM
Move-VM $pcns_vm -Destination $targetHost
Copy the migrate_PowerChute.ps1 script to the /opt/APC/PowerChute/user_files folder on the PowerChute Appliance via scp.
Run the command chmod +x migrate_PowerChute.ps1 to grant execute permissions.
Configure PowerChute to run the script using the Run command action for an event on the Event Configuration page or the Run Command feature on the Shutdown Settings page.
NOTE: By default, PowerCLI will not connect to a vCenter Server that uses a self-signed or untrusted SSL certificate.
Download the vCenter Server Root certificates:
NOTE: If the .zip opens in the browser right, right-click inside the page and select save as. Save the file as a compressed file.
Extract the contents of the zip and copy the Linux certificates (lin folder) to the PowerChute appliance via SCP.
Copy all the certificates (*.0 extension) to /etc/pki/ca-trust/source/anchors folder and run the command “update-ca-trust”
Confirm that the connection to vCenter Server works in powercli without any certificate errors:
Pwsh
Connect-ViServer -Server <vCenter Server> (enter your credentials when prompted)
The migration script needs to be executed when vCenter Server is available. In a VxRail configuration, a shutdown command file can be used. Ensure you allow enough time for the script to complete and PowerChute VM to shut down before the Network Management Card sends the Cluster Shutdown request to VxRail Manager.
In an environment where vCenter Server is deployed as a VM and being shut down you can run the migration script via an event driven command file.
The 2nd script is a python script that is executed on the 1st ESXi host in the cluster on start-up – this waits until the PowerChute VM can be found on the host (Cluster services are running again) and then powers it on.
Edit start_pcns.py enter the name of your PowerChute VM at the bottom of the file:
# Usage example
pcns_vm_name = "<PowerChute VM Name>"
pcns_vm_id = get_world_id(pcns_vm_name)
print(f"World ID {pcns_vm_id}")
wait_until_powered_on(pcns_vm_id)
Do not modify any other lines in this file.
On the 1st ESXi host in the cluster create a new folder called scripts on a local disk that has persistent storage and upload python script.
For example, using the vSphere Web UI:
Log into vCenter Server.
Click on the 1st node in the cluster.
Select Datastores tab.
Click on the local datastore.
In the datastore view click on New Folder:
Enter a name for the new folder e.g. scripts and Click OK
Select the new folder and click Upload Files
Browse to the location of start_pcns.py and upload the file.
NOTE: The upload may fail if the VMware Root CA certs have not been imported to your browser.
Please refer to https://kb.vmware.com/article/2147256 for instructions on how to address this.
Select the scripts folder in the file-hand tree view and confirm that the file has ben uploaded:
Click on the Summary to view the path to the Datastore:
SSH to the ESXi host.
Run the following command:
chmod +x /vmfs/volumes/<datastore id>/scripts/start_pcns.py
We then need to ensure that the script runs at start-up when the host boots. To do this we can add a line to /etc/rc.local.d/local.sh file.
Run the command vi /etc/rc.local.d/local.sh
Before the line “exit 0” add the line python /vmfs/volumes/<datastore id>/scripts/start_pcns.py &
NOTE: The & symbol is added so that the python script runs in the background and allows the local.sh script to continue.
Save the file and exit the SSH session.