none
Application threading question RRS feed

  • General discussion

  • Someone asked the following question on the Speech Server Beta discussion list:

    "My workflow application needs to hit a backend (make an HttpWebRequest) which could take a long time to return. After 5 seconds if the backend hasn’t returned, the app needs to play a message like, ‘we’re still looking up…’. The way I was thinking of implementing this was to create a separate thread to go do the backend access. The main application thread would create this thread, start it, then Join to it, with a, say, 5 second timeout. If it comes out of the join because of timeout, it plays the ‘we’re still looking up’ message and goes back and joins the backend thread again. If the backend thread returns, the main thread comes out of the join and give the results. Can you see any issues with doing it this way? Is having this extra thread going to break the application? I don’t understand the MSS threading model enough to know if this would be problematic."

     

    Since the beta discussion list is having problems at the moment, here is my reply:

    This is not a good way to achieve this.  If I understand correctly, you're going to block the application thread and also create a 2nd thread which itself will be blocked most of the time.  This is then run on a server, with multiple application instances at a time.  Generally speaking it is a bad idea to block threads on a server as this limits scalability and creating and destroying lots of threads is expensive

    As far as Speech Server is concerned, you won't receive any events (eg. disconnect) whilst the application thread is blocked.  There may also be other consequences, depending on what event you block on (eg. we may not be able to actually start playing your prompt until you return - that's a guess, I don't know off-hand what any consequences might be). 

    A much better approach would be to use System.Net.WebClient, using one of the async overloads (DownloadStringAsync, DownloadDataAsync or DownloadFileAsync).  If you invoke one of these methods from the application thread, the callback should come back on the application thread.

    If you want more control over the request, certainly use HttpWebRequest - but use BeginGetResponse instead of GetResponse so that the request goes asynchronously.  With HttpWebRequest, the callback won't be on the application thread. That's ok; you can still invoke Speech Server APIs, but your code must be robust to the fact that the application thread could be executing it in parallel (eg. firing OnClosed).  It would be safer to invoke IApplicationHost.QueueEvent from the HttpWebRequest callback in order to get onto the application thread, and then you don't have to worry about multi-threaded issues. 

    If this is gobble-di-gook, feel free to ask, and I can try to explain in more detail

    Tuesday, May 8, 2007 7:11 PM

All replies

  • I think I understand your response, but it doesn't really tell me how to handle my situation.

    First, if I don't block the applicaiton thread, then what does it do for the 5 seconds that I wait before checking to see of my web request has come back? If I just played a 5 second prompt of silence, do I run into the same issues as if I blocked the thread; that is, I am still tying up a speech resources (the appl thread is not free to work for any other applicaiton during this time - I think), and I will I still not see events until the prompt finishes playing?

    And if I use the asynch HttpWebRequest (I'm not sure if the WebClient will work. I need to do more research into how it works), then I have the issue of how do I synchronize the response with my application (the main applicaton thread) since it comes back in a separate thread? What would your suggestion be for synchronizing?

    Remember the main thread is basically giving the backend request 5 seconds to complete (or less; as soon as it some back, we want to continue withe the app), and if it's hasn't returned, then it plays a prompt saying please wait longer, then it waits another 5 seconds before finally continuing down an 'error' path. If at any point in all this waiting the response comes back, the applicaiton needs to proceed.

    Maybe I just need to understand what my options are for the main thread to wait for 5 seconds and still capture events and still not bind up speech server resources. And then understand how I can synch up my asynch response with the main thread.

    Thanks for the help Anthony.

    Wednesday, May 9, 2007 12:14 AM
  • The "application thread" or "serialized callback thread" is a concept not a physical thread.  When there is an event to be raised to the application, SpeechServer grabs a thread from the threadpool, executes the event handlers, then returns the thread to the threadpool once the handlers have completed.  There is a separate serialized callback thread for each application instance, so blocking the thread in one instance won't affect other instances (unless the threadpool runs out of threads), however it will stop stuff happening for this application instance, as Speech Server does use that same thread internally. 

    Regarding playing silence - this is generally considered to be a bad idea because the user may think that the line has gone dead.  Instead, look at the ThinkingSounds sample application. This shows how to repeatedly play a short prompt whilst some background operation is executing. Replace the BackgroundOperation with the one I give below, and you'll see how to perform an async web request.

    Playing a prompt does not block a thread.  The underlying implementation of a Prompt activity is to invoke ISynthesizer.SpeakAsync, then return.  The activity resumes processing when the SpeakCompleted event is raised.

     

    Code Snippet

    //--------------------------------------------------------------------

    //

    // Copyright (c) 2006 Microsoft Corporation. All rights reserved.

    //

    //--------------------------------------------------------------------

    using System;

    using System.Diagnostics;

    using System.Net;

    using System.Threading;

    using Microsoft.SpeechServer;

     

    namespace ThinkingSounds

    {

    ///

    /// BackgroundOperation - Simulate a background operation using a

    /// separate thread.

    ///

    public class BackgroundOperation

    {

    private bool _operationComplete;

    private WebRequest _httpRequest;

    private IApplicationHost _applicationHost;

     

    // Property exposing operation state.

    ///

    /// OperationComplete - Indicates whether or not the thread

    /// has completed the operation.

    ///

    public bool OperationComplete

    {

    get { return _operationComplete; }

    }

     

    ///

    /// Constructor - Start the http download.

    ///

    public BackgroundOperation(IApplicationHost applicationHost)

    {

    _applicationHost = applicationHost;

    _httpRequest = HttpWebRequest.Create("http://localhost/");

    _httpRequest.BeginGetResponse(GetResponseCallback, null);

    }

    private void GetResponseCallback(IAsyncResult ar)

    {

    Debug.Assert(!_applicationHost.IsSerializedCallbackThread,

    "This won't be executed on the application's serialized callback thread");

    _applicationHost.QueueEvent(new AsyncCallback(SerializedGetResponseCallback), ar);

    }

    private void SerializedGetResponseCallback(IAsyncResult ar)

    {

    Debug.Assert(_applicationHost.IsSerializedCallbackThread,

    "This should be executed on the application's serialized callback thread");

    // Get the response with EndGetResponse

    // HttpWebResponse httpResponse = (HttpWebReponse)_httpRequest.EndGetResponse(ar);

    _operationComplete = true;

    }

    }

    }

     

     

    In ThinkingSoundsWorkflow.ExecuteOperation, create BackgroundOperation as:

    Code Snippet

    this.BackgroundOP = new BackgroundOperation(this.ApplicationHost);

     

    Wednesday, May 9, 2007 1:41 PM
  • Thank you for the help Anthony,

    that was very helpful.

    Wednesday, May 9, 2007 2:36 PM
  • No problem.  BTW note that QueueEvent will throw InvalidOperationException if the application instance has completed (eg. due to a far-end disconnect).  Therefore you should wrap the call to QueueEvent in a try...catch (InvalidOperationException) { <dispose the HttpRequest> }

    Wednesday, May 9, 2007 2:59 PM
  • I implemented this and was testing it and got some unexpected behavior. Maybe you can explain why it would behave this way. So the app basically makes the backend call assynchronously as your example shows. Then it goes in a loop playing a short wait message (just like the thinking sounds app). That loop keeps checking to see if we got a response. The short wait is like music, but after 5 seconds I then play a "Please keep waiting..." message, so it's a little more complicated than thinking sounds. To test out the intermediate 5 second prompt, I wanted to delay the http request, so I put a sleep call in the callback for the httpWebRequest, thinking that this should only hold up the thread that is servicing the assynchronous response to the http request. But what appears to happen, is it holds up the main applicaiton thread that is in a loop saying the wait message. So the scenario is hte app makes hte assync request then start looping playing the wait message. That all keep workiung until we get the http response and hit my sleep call. Then the main app thread stops playing that prompt. I would have thought that the thread I am blocking, the on that is servicing the http assync response was separate from teh thread running hte speech app, but it doesn't behave that way. Can you think of what would be causing this behavior? Is this only a funciton of running this in a developement (debugger) environment? I think the app does indeed work with this technique and I probably can do testing by causing this delay in other ways, but I wanted to understand what was happening with threads in this scenario. Let me know if this scenario is not clear.

    Note you can't really compare this to the thinking sounds implementation because it spins off it's own thread (not doing assync calls), and in this scenario everything works as you would expect.

    Thanks for your insight.

    Thursday, May 10, 2007 3:49 PM
  • Did you put the sleep in SerializedGetResponseCallback or in GetResponseCallback?  If you put it in the former, then this behaviour is expected, because that's the whole point of QueueEvent, to put SerializedGetResponseCallback onto the application thread.  If you put it in the latter, then this behaviour is not expected.
    Thursday, May 10, 2007 6:39 PM
  • I put it in the GetResponseCallback. That should be executed in another thread I would think, so I doesn't seem like I should see the application get held up like it does.

    When I test the app by delaying the http request by other means (external to the application), everything indeed works as you would expect. There's something going on with teh handling of the response of the assynchronous web request. It's almost as if once the response comes back, the thread that made the assychronous thread gets held up until the response is handled by whatever thread is handling it.

    Thursday, May 10, 2007 7:13 PM
  • I have tried to repro your problem by updating ThinkingSounds.  I have replaced the current whileThinking which just plays the thinking prompt with one which contains an ifElse within the while loop.  The code condition for the ifElse is as follows:

     

    Code Snippet

    private void BackgroundOp5Sec(object sender, ConditionalEventArgs conditionalEventArgs)

    {

    TimeSpan timeTaken = DateTime.Now - BackgroundOP.StartTime;

    if (Math.Truncate(timeTaken.TotalSeconds) % 5 == 0) { conditionalEventArgs.Result = true; }

    else { conditionalEventArgs.Result = false; }

    }

     

    which accesses the following property on BackgroundOperation:

     

    Code Snippet

    public DateTime StartTime = DateTime.Now;

     

    One branch of the ifElse plays a "Please keep waiting" prompt and the other branch plays the thinking sound.  This should be similar to what you're doing. 

     

    With this application, I do not observe the behaviour that you describe.  I have a Thread.Sleep(10000) prior to QueueEvent in

    GetResponseCallback, and the "Please keep waiting" prompt plays twice and the thinking sound in between.  I am running this on XP; I have also tried something similar on W2k3.  The only suggestion I have is to pause the process in a debugger & see what's going on - eg. are you sleeping within a lock?  I presume not, since the purpose of usng QueueEvent is to avoid the necessity for using locks.

    Thursday, May 10, 2007 8:37 PM
  • Hey Anthony,

    Thanks for the followup.

    I was just doing some more testing and it seems to be working as we expect now. Yea, I know you're saying, 'yea sure...', thinking I was doing something stupid and don't want to admit it, and maybe I was, but I still don't know what that might have been. The only thing I can think of is when I stop the code in the debugger (with a breakpoint), that appears to prevent the main application thread from running, but when I put a Thread.Sleep() call to do the delay it works as expected, so maybe I was stopping the callback thread in the debugger and thinking I had stopped it using the sleep. I know I had tried both methods before and I am pretty sure that the sleep method was holding the main thread up, but I either I was mistaken or there was something going on that isn't there anymore. In any event, it 'calms my soul' when things work as you expect them to.

    Thanks again for the help and sorry for the extra work you went through.

    Thursday, May 10, 2007 10:08 PM
  • When you hit a breakpoint in the debugger, all threads in the process are suspended.  If you want to Freeze only one thread, you can do so with the threads window (under Debug->Windows).
    Friday, May 11, 2007 12:11 PM