Using procmon to troubleshoot "Vsphere client stopped working" crash (System.NullReferenceException)

Hello,

Below steps taken to troubleshoot issue I encountered on Windows 7 x64 launching Vsphere client (v4.0).

I used Vsphere quite exstensively for past 2 years and today for whatever reason it started failing to launch crashing with NullReferenceException below

I installed some Microsoft patches previous day as well as increased UAC prompt from “Never Notify” level to one notch up. There were no other changes to system.

I uninstalled patches but this did not fix the issue. I reinstalled Vmware vsphere client and this did not fix it either.
Now where sysinternal tools comes to rescue.
First I downloaded DbgView.Exe (http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx) which allows to see debug output from system. Below is snapshot what was captured during crash.

[5368] [        :Error   :W: 1] 2013-09-11 10:45:49.767  A fatal unhandled error occurred in vi client 
[5368] [        :Error   :W: 1] 2013-09-11 10:45:49.786  System.NullReferenceException: Object reference not set to an instance of an object. 
[5368]    at VirtualInfrastructure.Utils.Log.Init(String logFileName, Nullable`1 traceFlags, Boolean clearLog, Boolean setAppDomainVars) 
[5368]    at VpxClient.Program.InitLogFile() 
[5368]    at VpxClient.Program.Main(String[] commandline) 

We have a stack trace from application which points to some sort File Access/Permission error.

Now comes second tool from SysInternals called procmon.exe (http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx) which listens to configured ETW events. For my case I needed just file events for knows process vpxclient.exe. So below is what was captured

There were 2100 events preceeding crash and you need to filter those to find a curlpit. In my case I made filter for Result Column to “Access Denied”

It’s clear what the issue at this point. VpxClient trying to read some sort initialization file and get’s access denied. At this point I remembered that I changed UAC prompt the day before but did not launch VpxClient with “run as Administrator” and hence the issue since previously I did not have to do that since it was set to “Never Notify” before.

Advertisements

Extending system drive in Windows 2008

Below steps which needs to be taken to extend system (or any other drive for that matter) on Windows 2008. No reboot is required for this method.

Initial configuration is below

1. Add additional drive. No reboot is required if it’s virtual OS. Initialize the drive with MBR partition.

2. Choose “Add Mirror” for D drive and select new disk as destination drive.
3. After mirroring is complete, choose “break mirror”. This shall assign D drive letter to volume on new disk

4. Delete volume which is left from old drive on original disk.

Add caption

5. Choose extend volume from OS drive. This will extend volume with no reboot.

Final config is below

Extending system partition on Windows 2003

Below is set of steps which can be taken to extend system partition on Windows 2003. It will require 2 downloads so if you don’t have Internet connection from target computer download them separately.

Steps:

This is starting configuration. Basic disk with 2 partitions.

1. Add new HDD to target computer (virtual or physical one). Virtual one obviously would not require reboot and will show up in disk management.

2. Initialize new Disk
3. Convert new disks to Dynamic
4. Mirror second partition to new drive

5. After mirroring is complete. Right click on partition of old drive and select “Break Mirror”. This shall result in disk with old letter (D in this case) to be on secondary disk and old disk getting new drive letter (F in this case)
6. Delete extra disk (F). You will notice that you can not extend system partion message if you would try to extend it.
7. To use Dell extend partition tool we have to convert disk back to Basic. Hex Disk Editor from link in the start of the article would have to be used. Go to options and uncheck “Open Disk as readonly by default” and then go to Extras/Open Disk… and open Physical Disks/Hard Disk 1
Find row 1C0 and change all numbers in column 02 from  42 to 07 (that’s the difference between dynamic and basic disk believe it or not). First disk has 2 partitions so there are 2 numbers 42 and both needs to be changed to 07.
Schedule to run “chkdsk c: /f” on next reboot and start a reboot
8. After reboot you will see something like below. Old disk will marked as offline. Delete volume on that disk and remove it after that. Also delete extra volume at the end of disk 1 to have empty space to expand to.
9. Extend partition with disk tool from Dell at the start of the article. (Do not use “extend” option in disk menu since it’s not actually extending disk in Windows 2003 but creates a new volume and spans them accross partitions). 
You shall have system volume extended now.