We're trying to figure out if the log results are indicating a potential problem which may be causing the indexing issues.
There are three pages in particular that seem to be causing a problem all have - in the urls. First is our leeds promotional products page which we've organized fairly well. Watching the msn bot it comes in to that page and won't scan beyond the few items on the page before ignoring the rest. It doesn't do this with other category pages with only a single page and no subs like our columbia jacket page. That's a single page with only single page urls that appear, there are no asp.net driven paging in place. Lastly it seems to choke entirely on some categories like our bags category. I'll try to get up more detailed logs tommorrow as the bot is actively scanning the site now. What we're wondering is whether or not the navigation of our site is the problem even though it's asp.net driven.
