none
Dismal submit response times RRS feed

  • Question

  • After the last maintenance shutdown, submitting a post got noticeably faster.  Couple of seconds, tops.  That stopped being that way a couple of days ago.  I just timed it, 45 seconds.  On a Saturday.  Everything else seems about at normal speed, the usual 10 seconds or so response.

    Something is wrong, no idea what.  Back during the 2.x days, these kind of delays were always a preview of major problems down the road.

    This was a message to the forums support team, not Mike Walsh.



    Hans Passant.
    Sunday, November 2, 2008 1:21 AM

Answers

  • Not sure what is going on, but the delays are gone this morning.  Did you do something on your end?
    Hans Passant.
    Wednesday, November 19, 2008 11:13 AM
  • No code changes on our end. Keep an eye on it and let us know if you see any major fluctuations.
    Rob J, Forums Dev
    Wednesday, November 19, 2008 6:06 PM
  • Hans,

    If the issue occurs again, I'd like to find if there's any specific time you see the slow reponse times.
    STO Forums Test Lead
    Thursday, November 20, 2008 6:17 PM
  • Hi All,

    I'm verifying this issue while submitting a new thread and reply. It seems to be fast for me (couple of seconds posting to the sandbox forum). Hans, I'm wondering if there's a specific pattern or not. Which Forum/s (common) have you submitted a post (new thread or reply) where you see response time issues?
    STO Test Lead
    Wednesday, November 5, 2008 3:07 PM
  • Thanks Hans.

    I/we need to look more on this since I can't repro the problem and I don't see any patterns as well. I would suggest running a tool at the background such as Fiddler. 

    You can use Fiddler to see which dependent requests are going through and verify. 

    Let us know how it goes. 

     


    STO Test Lead
    Wednesday, November 5, 2008 6:59 PM
  • I can confirm Hans' statements.

    I mostly post in the SharePoint forums and the times taken to submit a post vary seemingly without any logic, but most of the time there is very definitiely a clear "d***, this is taking a long time" feeling before many submits go through.

    I have a vague feeling that submits are faster early 7-9 Finnish time = 9pm - 11 pm Redmond time than later in the day but that's just a feeling and when I submit this one (just before 7) there's bound to be a pause for thought before the submit goes through.  (Not much later: there wasn't!)

    The same goes for Moves; changes of thread type; deletions of threads but these are never as slow as submits can be.

    Has it occured to you that the sandbox might not be the best place to test this ?
    Thursday, November 6, 2008 4:54 AM
  • Yes. I also submitted a new thread and reply on a heavily used Forum such as Visual C# General and didn't get long submit response times. I'll submit a bug on this so we can track it and investigate further.

    Thanks for the info Mike.


    STO Forums Test Lead
    Thursday, November 6, 2008 5:50 AM
  • I'm kind of suspecting it's a variety of DB related issues, and the reason some people experience it to a greater degree than others is data volume.

    Our next release is targeted for this coming Monday. I think it will help with some, but not all, of these issues. Forums data will be sharded amongst multiple servers, thereby decreasing total db volume and load. This should help some with the Mark All Read issue and to a lesser extent the Posting time issue. I've got other ideas for Mark All Read that we just didn't have time for with this release that will be A LOT more backend friendly than it's current implementation.

    I'm thinking that the post issue you and some others experience will be most improved by the moving of Points Recalc to a backend/async operation instead of inline as it is now. This is also part of the forthcoming drop.

     
    Rob J, Forums Dev
    Thursday, November 13, 2008 5:53 PM

All replies

  • >This was a message to the forums support team, not Mike Walsh.

    Will you stop writing that. It's very offensive and completely unnecessary.


    Sunday, November 2, 2008 8:20 AM
  • Hans:

    Yes, these delays are extremely annoying, and seem much worse than previously.

    Maybe your post was missing a smiley at the end?


    David Wilkinson | Visual C++ MVP
    Sunday, November 2, 2008 6:20 PM
  • Hi All,

    I'm verifying this issue while submitting a new thread and reply. It seems to be fast for me (couple of seconds posting to the sandbox forum). Hans, I'm wondering if there's a specific pattern or not. Which Forum/s (common) have you submitted a post (new thread or reply) where you see response time issues?
    STO Test Lead
    Wednesday, November 5, 2008 3:07 PM
  • Hi Don,

    I'm not seeing any kind of pattern.  Since I started this thread, I'm seeing variability in response times the past two days.  It was consistently very slow late last week and this weekend, now it is usually very slow, sometimes fast (~10 sec).  Across multiple forums, no pattern.  Other than the potential for something very wrong on your end of the wire, it is also very awkward baby-sitting the glob of tabs I need to keep open to get posts submitted.

    Sorry, I wish I had better diagnostics.

    Hans Passant.
    Wednesday, November 5, 2008 3:24 PM
  • Thanks Hans.

    I/we need to look more on this since I can't repro the problem and I don't see any patterns as well. I would suggest running a tool at the background such as Fiddler. 

    You can use Fiddler to see which dependent requests are going through and verify. 

    Let us know how it goes. 

     


    STO Test Lead
    Wednesday, November 5, 2008 6:59 PM
  • I can confirm Hans' statements.

    I mostly post in the SharePoint forums and the times taken to submit a post vary seemingly without any logic, but most of the time there is very definitiely a clear "d***, this is taking a long time" feeling before many submits go through.

    I have a vague feeling that submits are faster early 7-9 Finnish time = 9pm - 11 pm Redmond time than later in the day but that's just a feeling and when I submit this one (just before 7) there's bound to be a pause for thought before the submit goes through.  (Not much later: there wasn't!)

    The same goes for Moves; changes of thread type; deletions of threads but these are never as slow as submits can be.

    Has it occured to you that the sandbox might not be the best place to test this ?
    Thursday, November 6, 2008 4:54 AM
  • Yes. I also submitted a new thread and reply on a heavily used Forum such as Visual C# General and didn't get long submit response times. I'll submit a bug on this so we can track it and investigate further.

    Thanks for the info Mike.


    STO Forums Test Lead
    Thursday, November 6, 2008 5:50 AM
  • The delays are steadily increasing.  I'm clocking around 55 seconds average now.  One notable outlier was a mind-numbing two minutes and 25 seconds.  Sunday morning.  Too bad you can't repro.  Nothing visible by watching traffic, SubmitReply's POST just doesn't get a timely response.  It looks like delays at your back-end, not the IIS servers.

    As long as only 3 posters see these delays, there's probably not a lot of reason to consider this a wide-spread problem.

    Hans Passant.
    Sunday, November 9, 2008 2:44 PM
  • if you are using fiddler or a like tool, could you pick up the server name you are hitting? Possible we have a bad server in the farm...
    Matt Fraser, STO Forums Software Developer
    Monday, November 10, 2008 6:51 AM
  • Not untypical, there are now additional problems.  I can no longer use the "Mark All Read" button.  It trundles for about 35 seconds, then spits out "There was an error".  Smells like a database update query timeout.
    Hans Passant.
    Thursday, November 13, 2008 1:54 PM
  • I'm kind of suspecting it's a variety of DB related issues, and the reason some people experience it to a greater degree than others is data volume.

    Our next release is targeted for this coming Monday. I think it will help with some, but not all, of these issues. Forums data will be sharded amongst multiple servers, thereby decreasing total db volume and load. This should help some with the Mark All Read issue and to a lesser extent the Posting time issue. I've got other ideas for Mark All Read that we just didn't have time for with this release that will be A LOT more backend friendly than it's current implementation.

    I'm thinking that the post issue you and some others experience will be most improved by the moving of Points Recalc to a backend/async operation instead of inline as it is now. This is also part of the forthcoming drop.

     
    Rob J, Forums Dev
    Thursday, November 13, 2008 5:53 PM
  • Hmya, I tend to be on the leading edge of these kind of DB problems.  It was the same back with Forums 2.x.  I have a rather large number of posts, 13,000+ on this site, many more page views.  I imagine update queries start to grind down with such large result sets.  The only other users we heard from about this problem are also frequent posters.  This will get a lot worse when the remaining 2.x forums are moved, it will add another 8,000 or so posts to my total.  Eek.

    This sounds like it ought to be easy to test.  Just make sure you use a user account with a large number of posts.  And comparable SQL index fragmentation, I guess.

    Hans Passant.
    Thursday, November 13, 2008 6:37 PM
  • nobugz said:

    Not untypical, there are now additional problems.  I can no longer use the "Mark All Read" button.  It trundles for about 35 seconds, then spits out "There was an error".  Smells like a database update query timeout.


    Hans Passant.



    Ha!  Okay... Last time it was only me.  Nobody ever responded to my thread, and I forgot how I fixed the problem.  This time I'm not alone.  Good to know.  I can stop restarting and clearing my cache and resetting my IE settings, etc. 

    It's still very frustrating nonetheless. :(  Must I read every thread?
    David Morton - http://blog.davemorton.net/
    Friday, November 14, 2008 3:18 PM
  • Four.  Maybe we should add recog points instead.
    Hans Passant.
    Saturday, November 15, 2008 1:32 AM
  • David M Morton said:

    nobugz said:

    Not untypical, there are now additional problems.  I can no longer use the "Mark All Read" button.  It trundles for about 35 seconds, then spits out "There was an error".  Smells like a database update query timeout.


    Hans Passant.



    Ha!  Okay... Last time it was only me.  Nobody ever responded to my thread, and I forgot how I fixed the problem.  This time I'm not alone.  Good to know.  I can stop restarting and clearing my cache and resetting my IE settings, etc. 

    It's still very frustrating nonetheless. :(  Must I read every thread?
    David Morton - http://blog.davemorton.net/



    David, I just replied to your other thread.
    STO Forums Test Lead
    Tuesday, November 18, 2008 5:44 AM
  • It solved the "Mark All Read" error.  Submitting is still very slow.
    Hans Passant.
    Tuesday, November 18, 2008 6:08 AM
  • Not sure what is going on, but the delays are gone this morning.  Did you do something on your end?
    Hans Passant.
    Wednesday, November 19, 2008 11:13 AM
  • No code changes on our end. Keep an eye on it and let us know if you see any major fluctuations.
    Rob J, Forums Dev
    Wednesday, November 19, 2008 6:06 PM
  • Hans,

    If the issue occurs again, I'd like to find if there's any specific time you see the slow reponse times.
    STO Forums Test Lead
    Thursday, November 20, 2008 6:17 PM