I had the same Windows Home Server backup issue on one of my systems running Windows Vista (and later on Windows 7 RC1). Nothing I found in any of the related forum posts helped.
After taking a closer look at the installed VSS components on the problematic system using 'vssadmin', it turned out that there seemed to be a "leftover" VSS provider from a previous (properly uninstalled) Norton Ghost. That's at least the only somewhat plausible explanation I can come up with.
First, take a look at the providers in the system:
C:\Windows\system32>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Symantec Software VSS Provider'
Provider type: Software
Provider Id: {262b716e-bb23-41b5-aaef-e2c15e767167}
Version: 1.0
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
The 'Symantec Software VSS Provider' was suspicious. So, I simply went into regedit, searched for its name, and removed the two related provider entries.
Another 'vssadmin list providers' confirmed that the Symantec provider entry was indeed gone:
C:\Windows\system32>vssadmin list providers
vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001-2005 Microsoft Corp.
Provider name: 'Microsoft Software Shadow Copy provider 1.0'
Provider type: System
Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
Version: 1.0.0.7
After that, my Windows Home Server backups worked like a charm again. I am not saying that you will see the same Symantec VSS provider, however, my point is that it probably does not hurt to take a peek at your providers, and see if there is anything questionable to be seen.
