vCenter Server Appliance services not available

VMware vCenter Server Appliance 5.5 had a technical issue. View Manager dashboard reports ‘The service is not working properly’.  The vSphere Client cannot connect to the appliance. This happened all of a sudden after months of having no issue at all.

Likely an upgrade from vCenter Server Appliance 5.5 Update 2 to vCenter Server Appliance 6.0 was done a while ago. This went all fine.

Troubleshooting the appliance is much more difficult than troubleshooting the vCenter Server running on Windows. This blogpost will provide some info. There are a couple of good VMware KB articles about the appliance.

Log in to the appliance using a SSH (Putty.exe for example) or directly into the console using root account.

 

This VMware KB article provides information about checking status of services. This KB article has an overview of the service names.

Type the command:

service vmware-vpxd status

and

service vmware-vpostgres status

All services should be running. If you notice the Postgres database is not running check the diskspace with command

df -h 

If the available diskspace for the /storage/db is 0% there is your issue. Check the folder where logs are saved. This is /storage/db/vpostgres/pg_log

You see a lot of logs here. The content of the log is allmost all the same. This line saying “there is already a transaction in progress”.

That is the problem!

The problem is caused by a bug in  vCenter Server Appliance 5.5 Update 2. The bug has been resolved in Update 3. However if you upgrade from Update 2 to vCenter Server Appliance 6.0 the bug will still be there!

The bug is that Postgres writes each and every warning in the logfile. The logs will grow very rapidly. So we need to adjust the level of logging.

A detailled description of how to adjust is described in this VMware KB.

This is a summary:

We change the level of logging  by editing the /storage/db/vpostgres/postgresql.conf file. Make sure to make a copy of this file first.

Run this command to stop the VMware VirtualCenter Server service:

service vmware-vpxd stop

Run this command to stop the embedded vCenter Server database service:

service vmware-vpostgres stop

Then we change the log level.

cd /storage/db/vpostgres

vi postgresql.conf

using the downarrow on your keyboard  go to a line saying log_min_messages = warning

Position the cursor next to the =  character. Press i key for inserting text  and type ‘error’. Delete the word ‘warning’.

The line now reads:

log_min_messages = error
Then press Escape key on your keyboard. Then :wq to save the file

We then need to remove the logfiles.

Remove all but the two newest log files from the /storage/db/vpostgres/pg_log file using the rm command:

For example:

rm /storage/db/vpostgres/pg_log/postgressql-YYYY-MM-DD_xxxxxx.log

Run this command to start the embedded vCenter Server database service:

service vmware-vpostgres start

Run this command to start the VMware VirtualCenter Server service:

service vmware-vpxd start
Check if all services start. You now can connect again to vCenter and View Manager should show a green square showing it can connect to vCenter.

Add a Comment

Your email address will not be published. Required fields are marked *

Current ye@r *