This is related to multiple bindings, i solved a case in which the user was using an 80 port in his crm web binding when the user had the 5555 but changed it to 80, worked in url in IE, but oddly outlook didn't like it, it was tryin to connect to the original
port but the user had 80 ports in the default website as well, causing conflict, once 5555 was put in the binding for the crm, the test connection succeeded, but the user also had to type in the hostname:5555 from the server originally which was different
from what he was using in IE. COnfusing yes.. just leave ports alone in bindings.
IFD can help change it,but the user didnt have ifd so the 5555 but this KB is related to the issue http://support.microsoft.com/kb/2498357