none
Recent problems with My Threads and Alerts pages RRS feed

  • Question

  • This site appeared to go down hard around 2pm PDT yesterday afternoon.  A couple of hours before that happened, I noticed that My Threads page stopped updating automatically.  After the crash, My Threads and Alerts produced an almost instant Error 500.  Most forums worked without problem, except for the C# General forum.  This morning, My Threads is working but not without problems.  I'm getting an almost instant "There was an error" when I turn on the Unread filter.  The page is unusable without that filter.

    What's going on?

    Hans Passant.
    Tuesday, July 29, 2008 10:53 AM

Answers

  • Currently MyThreads works for me by listing out my threads, but now paginating past a certain point throws a server error. :-(
    William Wegerson (www.OmegaCoder.Com)
    Monday, August 4, 2008 6:53 PM

All replies

  • I am simply getting an "Error 500" when I access "My Threads" which happens btw on IE and Firefox. 
    William Wegerson (www.OmegaCoder.Com)
    Tuesday, July 29, 2008 12:35 PM
  •   I don't know of any event @ 2pm yesterday. Usually when the site is in a bad state, I get a lot of email.. ;)
    Matt Fraser, STO Forums Software Developer
    Tuesday, July 29, 2008 4:51 PM
  • So are you saying you are not getting error 500 currently? I still cannot access My Threads... 
    William Wegerson (www.OmegaCoder.Com)
    Tuesday, July 29, 2008 6:40 PM
  • My threads is working fine for me.
    Matt Fraser, STO Forums Software Developer
    Tuesday, July 29, 2008 8:16 PM
  • I am still getting this error.
    David Morton - Consultant - Catapult Systems - Houston
    Tuesday, July 29, 2008 8:30 PM
  • And I still can't use the Unread filter.  Tell us what you need to know to troubleshoot this problem.  All I've got is "There was an error".  Resetting the microsoft.com cookies had no effect.
    Hans Passant.
    Tuesday, July 29, 2008 9:26 PM
  • nobugz said:

    And I still can't use the Unread filter.  Tell us what you need to know to troubleshoot this problem.  All I've got is "There was an error".  Resetting the microsoft.com cookies had no effect.


    Hans Passant.



    OK, that was going to be my first question... Do you have Fiddler? If so, could you see which server you're hitting? I'm concerned there's one misbehaving server in the pool. Hans, feel free to email me. My display name is my Microsoft email address.

    Thanks
    Matt
    Matt Fraser, STO Forums Software Developer
    Tuesday, July 29, 2008 10:46 PM
  • Okay, I've got a good working theory.  The people you've heard from are all contributors to the C# General forum.  In addition, C# General was the forum that gave me Error 500.  Right now, when I navigate through the My Threads pages with the Unread filter turned off, I get "There was an error" when I try to bring up page #3.  Every other page works.

    The theory: there's a thread in C# General that cases some kind of server in your system to throw an exception.  That thread was on page #1 late yesterday afternoon, giving me Error 500 when trying to visit C# General.  I more than likely contributed to that thread and got that same thread on page #1 in my My Threads page, causing Error 500 there as well.  This morning, I got enough other threads on my first My Threads page to be able to bring it up again.  But as soon as I turn the Unread filter back on, that thread pops right back up on page #1 (I cannot read it), now transmorphing Error 500 into "There was an error".

    If my theory is correct, Bill and David probably contributed to that thread as well.  It's quite common that the three of us post to a C# General thread.  Eventually, they'll get to where I am at now, getting enough other threads to push the rogue thread to their 2nd page and turning Error 500 into "There was an error".

    The trick is to find that rogue thread.  I went through all my alerts received around the time of the first signs of trouble, no hit.  No great surprise, you alert generator is probably dying as well.  From looking at the time stamps of the last thread on my page #2 and the first one on page #4, the last post to the thread should be between 11:07am and 9:58pm CDT yesterday.  And is likely to have me, Bill and David as contributors.

    Well, sounds like a good theory to me.  Fix it soon please.


    Hans Passant.
    • Edited by nobugz Tuesday, July 29, 2008 11:49 PM I was still on page #3
    Tuesday, July 29, 2008 11:30 PM
  • mattfras said:

    OK, that was going to be my first question... Do you have Fiddler?


    No.  Monitoring the Ajax traffic with Firebug has been useless before FF3 disabled it.  "There was an error".

    Do you have an error log?

    Hans Passant.
    Tuesday, July 29, 2008 11:47 PM
  • Okay, I figured out a workaround.  I marked all threads as read in C# General.  Now I can turn my Unread filter back on.  With it off, I still get "There was an error" on page #3, makes sense.

    Any progress getting better error messages?  "GetUpdates failed" doesn't count.

    Hans Passant.
    Wednesday, July 30, 2008 12:43 AM
  • nobugz said:

    mattfras said:

    OK, that was going to be my first question... Do you have Fiddler?


    No.  Monitoring the Ajax traffic with Firebug has been useless before FF3 disabled it.  "There was an error".

    Do you have an error log?

    Hans Passant.



    Yes, but unfortunately it's spread across several servers (and I think I can only see the last hour). That's why I was hoping you could narrow down the server for me with Fiddler. I've glaced at a could logs and haven't seen anything unusual.

    Better user error messages is one of my tasks for the release after the coming one...
    Matt Fraser, STO Forums Software Developer
    Wednesday, July 30, 2008 2:38 AM
  • You can only see the last hour?  How many errors do you get in an hour?  It is probably your back-end, the errors where completely consistent no matter how often I tried to refresh the page.

    If you need by account name and password to login to get a repro, please send me an email.  You'll find it in my profile on the old web site, |Monkeytail| = @

    Hans Passant.
    Wednesday, July 30, 2008 3:03 AM
  • One other question, does the getupdates call fail quick or like after 30 seconds or so?
    Rob J, Forums Dev
    Wednesday, July 30, 2008 3:39 AM
  • Very quick, within a second or two.  Same with the original Error 500.  Isn't it "GetThreads" that runs when you change the page?  It is not a "GetUpdates failed" error message.
    Hans Passant.
    Wednesday, July 30, 2008 3:59 AM
  • GetUpdates() is the general call that's made to retrieve the list of threads on the threads page for any state change (paging, filter, sort...)

    The fact that it fails quick at least indicates its not a sql timeout, so that narrows it down some. Still the strange part is that it seems to have some dependency on user. The main thing that comes to mind there is possibly a case where an extra row is in the association table between you and the thread. Which could cause a multi-row return where only a single is expected, but thats pure speculation... (and I'd think we'd have a unique index there to prevent it)

    Rob J, Forums Dev
    Wednesday, July 30, 2008 4:19 AM
  • Well, since it is a My Threads and Alerts page failure, the user dependency is almost automatic.  I found an easy repro for you.  Browse Users + Search, "OmegaMan".  You'll get his profile, click on OmegaMan's Threads.  Whammo, Error 500.  Same for David M Morton, dies on page #2.  You should see me die on page #3 or 4.  Come to think of it, the links to the user profiles are right here in this page.

    Run a query for all three users on those bad pages, the evil thread should pop right out.


    Hans Passant.
    • Edited by nobugz Wednesday, July 30, 2008 4:48 AM query note
    Wednesday, July 30, 2008 4:43 AM
  • nobugz said:

    Well, since it is a My Threads and Alerts page failure, the user dependency is almost automatic.  I found an easy repro for you.  Browse Users + Search, "OmegaMan".  You'll get his profile, click on OmegaMan's Threads.  Whammo, Error 500.  Same for David M Morton, dies on page #2.  You should see me die on page #3 or 4.  Come to think of it, the links to the user profiles are right here in this page.

    Run a query for all three users on those bad pages, the evil thread should pop right out.


    Hans Passant.



    Thanks Hans. That should help us immensely. I'll try and track it down tomorrow.
    Matt Fraser, STO Forums Software Developer
    Wednesday, July 30, 2008 7:24 AM
  • In browsing the C# General forum, I can go back more than a week when I filter by answered questions, so its not an answered one. But if I change the filter to unanswered and browse back more than a day, roughly page three with the final post viewed being about 1pm yesterday is the killer and I get a red "Unable to process request" at the top of the page and it quits.

    Deleted Threads have no effect.

    William Wegerson (www.OmegaCoder.Com)
    • Edited by OmegaMan Wednesday, July 30, 2008 11:14 AM This space for rent.
    Wednesday, July 30, 2008 11:09 AM
  • There appears to have been an update lauched yesterday. I'm guessing the issues  here are as a result of the upgrade. 
    Thomas Lee
    Wednesday, July 30, 2008 12:37 PM
  • Everything seems to be working okay for me now.

    On another note, it'd be REALLY neat to be able to create a personal page that combines the threads from several forums into one place.  Since many of us are regulars on several different forums, it'd be a great help to only have to have one browser window or tab open, instead of having to open each forum in a different place. 

    I know I should have created a new thread here, but since we seem to have the developers' attention... you know. :)
    David Morton - Consultant - Catapult Systems - Houston
    Wednesday, July 30, 2008 1:59 PM
  • Check again.  Click page #2 in your My Threads page to see the "There was an error" message.  And yes, start your own thread.
    Hans Passant.
    Wednesday, July 30, 2008 4:38 PM
  • Well, I'll be.  I retract my previous statement.  This is still broken.
    David Morton - Consultant - Catapult Systems - Houston
    Wednesday, July 30, 2008 4:41 PM
  • Thomas Lee said:

    There appears to have been an update lauched yesterday. I'm guessing the issues  here are as a result of the upgrade. 


    Thomas Lee



    As far as I know, there has been no deployment since Thurs July 24.
    Matt Fraser, STO Forums Software Developer
    • Edited by mattfras1 Wednesday, July 30, 2008 5:05 PM grammar
    Wednesday, July 30, 2008 5:04 PM
  • I'm in the process of restoring a production backup on my dev box now. Hopefully takign the intersection of people having issues with "My Threads" will shed some light on the problem.

    Thanks
    Matt
    Matt Fraser, STO Forums Software Developer
    Wednesday, July 30, 2008 5:07 PM
  • mattfras said:

    I'm in the process of restoring a production backup on my dev box now. Hopefully takign the intersection of people having issues with "My Threads" will shed some light on the problem.

    Thanks
    Matt


    Matt Fraser, STO Forums Software Developer



    The problem SEEMS to be with the "returning the key of a Dictionary" thread in the C# General Forum. This seems consistant with who was having problems witht he "My Threads" listing and some people having issue in the C# General Forum. I'm still investigating as to WHY it's a problem.

    Matt
    Matt Fraser, STO Forums Software Developer
    Wednesday, July 30, 2008 6:11 PM
  • Yes, that's the one.
    Hans Passant.
    Wednesday, July 30, 2008 6:34 PM
  • I appreciate being the poster child for this issue, but if there is not a quick fix; could someone at least remove the offending post from my queue? I have numerous threads that have gone ragged because I can't access what  I have....

    Thanks

    William Wegerson (www.OmegaCoder.Com)
    • Edited by OmegaMan Thursday, July 31, 2008 6:12 PM left out a key word in a sentance
    Thursday, July 31, 2008 6:11 PM
  • Same for me as well.
    David Morton - Consultant - Catapult Systems - Houston
    Thursday, July 31, 2008 6:17 PM
  • OmegaMan said:

    I appreciate being the poster child for this issue, but if there is not a quick fix; could someone at least remove the offending post from my queue? I have numerous threads that have gone ragged because I can't access what  I have....

    Thanks


    William Wegerson (www.OmegaCoder.Com)



    Well, I can't just delete it from your queue. That would basically require deleting your post from the thread...

    The good news is that I've found the issue. Trust me, it was a high priority of mine since it's basically breaking the C# general forum right now. We're working on a fix right now, and I'll push to get it deployed ASAP, but I'm not sure how quickly I can fasttrack it thru Ops...

    Thanks for your patience.
    Matt
    Matt Fraser, STO Forums Software Developer
    Thursday, July 31, 2008 11:50 PM
  • Sounds good.  But, please, don't keep it secret what was actually wrong with either the thread or the code.  Troubleshooting these kinds of problems without having any decent error message or insight in how the system is put together is unreasonably difficult.
    Hans Passant.
    Friday, August 1, 2008 12:03 AM
  • nobugz said:

    Sounds good.  But, please, don't keep it secret what was actually wrong with either the thread or the code.  Troubleshooting these kinds of problems without having any decent error message or insight in how the system is put together is unreasonably difficult.


    Hans Passant.



    Actually, I'd like to thank you. It was actually your research that got me headed in the right direction.

    The particular problem here relates to the fact that a user exists across multiple systems. Somehow, the user that made the last reply to the thread in question did so without having a complete profile (a condition that should not exist). So, whe the thread tried to load (and, in particular, the information for the "Last reply") is was unable to load the user and error'd out.

    Needless to say, posts should not be made by incomplete users, and once we've fixed this message (likely by assigning it to a generic user) we'll investigate a) how this coul have happened and b) making the code more robost, even though it SHOULDN'T happen.

    Matt
    Matt Fraser, STO Forums Software Developer
    Friday, August 1, 2008 12:11 AM
  • Thanks.
    Hans Passant.
    Friday, August 1, 2008 1:39 AM
  • Currently MyThreads works for me by listing out my threads, but now paginating past a certain point throws a server error. :-(
    William Wegerson (www.OmegaCoder.Com)
    Monday, August 4, 2008 6:53 PM
  • The damaged thread is still not fixed.
    Hans Passant.
    Monday, August 4, 2008 7:03 PM
  • This error should be fixed now. Thanks!

    Ed Price (a.k.a User Ed), SQL Server Experience Program Manager (Blog, Twitter, Wiki)

    Monday, February 20, 2012 3:05 AM
    Owner