Using Infer.Net models in an ASP.Net web service RRS feed

  • Question

  • I'm building an ASP.Net Web API 2 web service in Azure to give access to an Infer.Net naive Bayes model. There are two modes for starting up the model: building the model from scratch or loading the last-saved state of the model. The latter is faster than the former, but neither is near-instantaneous. Web service calls need to be immediate and near-stateless. The two ways I can imagine doing that association are to use ASP.Net's application state http://msdn.microsoft.com/en-us/library/ms178594.aspx or to use an Azure Worker Role and communicate between the web service methods and the running worker role using queues.

    What is the best practice for associating the running model with its web service API?

    Monday, November 24, 2014 3:34 PM

All replies

  • Does the latency hit come from model compilation? If so, you should consider using pre-compiled models.
    Monday, November 24, 2014 5:26 PM
  • Thanks Yordan.

    Is then firing up the precompiled model for each web service API call the preferred route, or is it better to have the model running independently from, and called by, the ASP.Net code?

    Monday, November 24, 2014 5:40 PM
  • Hi Tim,

    I don't think that the Infer.NET's pre-compiled algorithms are different from any other library in this regard. Note that after pre-compilation you'll end up with a couple of C# classes which (together with the Infer.NET runtime) you can load in your favourite way. I guess the REST layer would be the more principle way to go, especially if you have to retrain on the background. But I wouldn't go that far if I were building a simple demo app which calls into a pre-trained model (not to be confused with a pre-compiled algorithm!).

    My concern was rather about what you said regarding the approach to running the model - that it's not near-instantaneous. Is this still true after using pre-compiled algorithms? Is it inference that's slow? Or the network call? Or the deserialization of the model?


    Tuesday, November 25, 2014 12:24 AM
  • Thanks Yordan.

    I'll do some performance analysis as the model grows and if there is a problem I'll open up another thread.

    I'd still like to bottom out my understanding of using Infer.Net in a web service. The application I'm working on is a voice based 20 Questions game. At first it will just be the two strong development team using it and so the concurrent load will be slight. But if later it received more attention I can imagine the web service managing multiple concurrent games. Each of those games will want to update the beliefs in the model. One of the stated advantages of pre-compiled models is "to run multi-threaded inference. Including the compiled algorithm directly allows different instances of the compiled algorithm to be used in each thread separately without recompilation." What happens in this multi-threaded environment if each model want to update the beliefs?

    Tuesday, November 25, 2014 10:26 AM
  • Hi Tim,

    If you want to have multiple concurrently played games to update that same model, then it would be better to have an Azure Worker Role which runs in the background and trains the model. Submit the games (/training data) to the worker role for each game in a queue. Then, when the worker role runs, it'll simply pull the data from its queue and train on it. All of your instances should pull out the latest model from the worker role.

    The machine running the training will be quite loaded with work. Here are some pointers on how to scale this up. I guess you won't need to go that far in the beginning though. You should also never need to go that far if online learning is an option for you. To perform online learning simply plug in the old posteriors as the new priors and train on the new batch of data pulled from the queue.


    Wednesday, November 26, 2014 6:24 PM