locked
Load testing tools? RRS feed

  • Question

  • Hi!

     

    Is there any way to perform load tests besides having a lot of people calling the application? Could I have one application calling another or are there any tools to do this?

     

    I would like to reproduce some high load problems we had today and cannot go on asking everybody at the office make calls...

     

    Thanks for your help!

    / Markus

    Wednesday, May 9, 2007 6:48 PM

Answers

  • It is possible to automate load testing of applications.  However you do it, you need some form of anti-app which plays prompts in response to the application's prompts such that the application follows a pre-determined path through the dialog.  You'll need to use multiple pre-determined paths to give good coverage of the application under test.  The anti-app will need to validate each prompt to verify that the correct dialog is being followed.

    It is fairly easy to configure an outbound application ("anti-app") on 1 Speech Server to call an inbound application on another Speech Server, so writing the anti-app as a Speech Server application is perfectly possible.  You should consider how the anti-app will do it's verification; it may be benficial to use a modified version of the application under test that prefixes each prompt with some DTMF tones so that the anti-app can do DTMF recognition instead of Speech reco.  You also need to ensure that the anti-app is realistic in how quickly it responds to prompts, in order to simulate expected caller behaviour.

    There are also standard load testing tools available; a search for "IVR Load Testing" might be a good place to start.

     

    Your idea of splitting your application up & stress testing parts is valid, but you do need to stress the system as a whole because the speech recognition itself should be the most resource intensive part. 

     

    Whenever you do load testing, you need to measure the performance of the system under test.  There are various useful performance counters, eg. the Process, Processor, SpeechService and .NET CLR Memory performance objects.  You should also ensure that Speech Server trace logging is enabled appropriately.  More information is generally better, though sampling the audio will allow you to compromise log size vs. ability to check the quality of the audio.

    Thursday, May 10, 2007 10:25 AM

All replies

  • Giving it some more thought, load testing a speech application isn't that easy (at least until I build the Extreme Talkalot SpeechTool...), so I'm thinking about creating an ordinary asp.net application that uses a workflow that performs similar actions (database lookups, etc) as my speech application does.

    This way I can use an ordinary web stress tool to test this site, tune settings, test some more and the result should at least be relevant for the web part of the speech application.

     

    How have you tested your applications?

     

    Thanks for your help!

    / Markus

    Thursday, May 10, 2007 6:50 AM
  • It is possible to automate load testing of applications.  However you do it, you need some form of anti-app which plays prompts in response to the application's prompts such that the application follows a pre-determined path through the dialog.  You'll need to use multiple pre-determined paths to give good coverage of the application under test.  The anti-app will need to validate each prompt to verify that the correct dialog is being followed.

    It is fairly easy to configure an outbound application ("anti-app") on 1 Speech Server to call an inbound application on another Speech Server, so writing the anti-app as a Speech Server application is perfectly possible.  You should consider how the anti-app will do it's verification; it may be benficial to use a modified version of the application under test that prefixes each prompt with some DTMF tones so that the anti-app can do DTMF recognition instead of Speech reco.  You also need to ensure that the anti-app is realistic in how quickly it responds to prompts, in order to simulate expected caller behaviour.

    There are also standard load testing tools available; a search for "IVR Load Testing" might be a good place to start.

     

    Your idea of splitting your application up & stress testing parts is valid, but you do need to stress the system as a whole because the speech recognition itself should be the most resource intensive part. 

     

    Whenever you do load testing, you need to measure the performance of the system under test.  There are various useful performance counters, eg. the Process, Processor, SpeechService and .NET CLR Memory performance objects.  You should also ensure that Speech Server trace logging is enabled appropriately.  More information is generally better, though sampling the audio will allow you to compromise log size vs. ability to check the quality of the audio.

    Thursday, May 10, 2007 10:25 AM
  • Hi Markus,
    Have you tried Cyara Solutions?  They offer intelligent load testing which allows you to pump thousands of calls to load test your application!  http://www.cyarasolutions.com
    Alok
    Sunday, August 12, 2007 6:19 AM
  • Try Jigsaw from www.gtn-tech.com. You can simulate high load environment for troubleshoot with a few key stroke.
    Monday, July 13, 2009 9:49 PM
  •  

    Disclaimer: I work for Nu Echo Inc.

     

    A bit of late but you might want to try out NuBot IVR Testing Platform. The NuBot platform SaaS helps you in automating IVR application testing. The SaaS (Software as a service) model gives you the ability to define your test scenarios locally on your workstation using a free Eclipse plug-in and let you launch test over our remote platform as you see fit, in complete autonomy without worrying about the telephony infrastructure. The platform calls your test application through the public network and interacts with it, following your test scenario.

    Contact me if you need further information.

    Regards


    Thursday, July 15, 2010 3:51 PM