Answered by:
Crawling bug by msnbot?

Question
-
Hi all,
Maybe someone can shed some constructive light on the following problem I have with msnbot failing to crawl one of my websites - namely - http://www.dovada.com.au/
This site in webmaster tools shows the site address as http://www..dovada.com.au/ (notice the error - two dots between the www and dovada www..dovada
When the msnbot attempts to crawl the website it uses this http://www..dovada.com.au/ faulty address and of course shows a report under crawl issues shown below
"Live Search encountered a "404 File Not Found" HTTP status code when last attempting to download these URLs"
This problem will not go away - It seems that this faulty address is never deleted by the update software and is always called upon and used from somewhere within the msnbot crawling system.
Every attempt to contact support results in a failure to recognise the problem exists.
It appears that the leading periods (two dots) are ignored by some sections of the update software and not in other update areas thus creating the address confusion.
Webmaster tools lists over 7600 backlinks into the site but cannot list any Outbound links (because it cannot find the site of course) -I have attempted to and I believe successfully supplied the sitemap and site identification keys but it seems to only fail when the msnbot attempts to crawl the site for indexing as it calls on this faulty address.
Here is the only msn indexing:
http://search.live.com/results.aspx?q=site%3Adovada.com.au&go=&form=QBRE
Can anbody suggest a solution to this problem as the site has been not indexed by msn for nearly 12 months now.
Please be constructive in your advise and thanks in advance.
David
Tuesday, September 23, 2008 6:10 AM
Answers
-
Hi David,
Sorry that you have not had a good experience with our customer support. I've seen this issue a few times in the past. I'll send this to the right people, but please give it about two maybe three weeks to fully correct itself. If it takes longer, please post back to this thread and I will follow up.
Thanks,
Brett
Wednesday, September 24, 2008 4:35 PM
All replies
-
Is it possible for moderators of this forum to report this bug to the indexing software engineers?Wednesday, September 24, 2008 7:05 AM
-
This ".." problem was mentioned on another post, but danged if I can find it. And I cannot recall how it was resolved. I believe the person who reported it was asked to request that the domain name be removed. Sorry, this isn't much help, but I'll keep looking for the post.
... Duane
Wednesday, September 24, 2008 11:10 AM -
There is no way we can ask for the faulty domain name to be removed - The msnbot constantly tries to assign this faulty address to www.dovada.com.au because of thousands of incoming links it finds on websites everywhere.
It then tries to index us using the ".." faulty address that it retains within the msn indexing system.
We the public cannot delete website names even faulty ones which we have described above.
The msn indexing software has a bug which needs to be attended to so that it can distinguish between a address that is malformed i.e. ".." with the two periods and then discard this from the index thus leaving the index clean for further re-indexing.
The msn indexing software will never be able to drop the affending address while it continues to find the domain dovada.com.au on the internet. Asking for the address to be manually removed will not fix the problem, the msn indexing software must be capable of finding the error itself. (What if there is a link out there which somebody has used which is malformed and msnbot finds that again the problem then returns).
Can the fault be brought to the attention of your indexing software engineers to obtain a permanent fix.
This Forum is the only hope we have, as attempts to contact support result in unhelpfull comments like I need to get quality links to my website or similar unrealistic unhelpful comments. The webmaster area has always confirmed the the domain name dovada.com.au is not blocked.
The faulty domain entry returned using http://search.live.com/results.aspx?q=site%3Adovada.com.au&go=&form=QBRE confirms this when it shows the indexed domain entry of http://www..dovada.com.au/ and is permanently shown indexed faulty.
Note that the double period occurrs just before the domain name - this is why it is not picked up as being invalid in the domain name, it is only when the domain address is completely formed that the error is occurring.
Please help
David
Wednesday, September 24, 2008 12:27 PM -
Hi David,
Sorry that you have not had a good experience with our customer support. I've seen this issue a few times in the past. I'll send this to the right people, but please give it about two maybe three weeks to fully correct itself. If it takes longer, please post back to this thread and I will follow up.
Thanks,
Brett
Wednesday, September 24, 2008 4:35 PM -
Thank you Brett,
This problem has taken a lot of time and expense by us to track down and we feel extremely gratefull for you passing this problem on to right people within msn.
We will wait in anticipation for the problem to become resolved, it seems by what you say, that other people may also be experiencing similar problems and this problem being fixed should help them too.
Thanks again
David
Thursday, September 25, 2008 12:39 AM -
Hi Brett,
At last, seems like the faulty address with the two periods has finally dissappeard from the msn index, we are just waiting now for the correct web address to be indexed.
cpanel finally showed us the site was crawled as shown below:
MSNBot 93+7 1.96 MB 06 Oct 2008 - 09:43 Could you please advise me if there are any other problems I need to attend to in the meantime.
According to webmaster.live tools there is no blockage and as yet no indexing. Approximatly how long would it take for us to expect to be fully re-indexed?
Thanks again for your help.
David
Wednesday, October 8, 2008 12:15 PM