AFAIK Server 2008 is not a supported WHS client. There's some things you can try though:
1. In IE browser go to http://yourservername:55000
2. Try right click setup.exe and select "Run as Administrator"
3. Run the following command in an elevated command prompt (Start, All Programs, Accessories, right-click Command Prompt and select “Run as administrator”).
msiexec /i "\\YOURHOMESERVERNAME_HERE\Software\Home Server Connector Software\whsconnector.msi" WHSMSI="RUNSETUP"
Don't have server 2008, so can't test any of this