locked
On learning parameters from data (Migrated from community.research.microsoft.com) RRS feed

  • Question

  • Henry Hoo posted on 02-08-2011 4:24 AM

    Thanks for providing such powerful tool for the comunity !

    I have a question on learning parameters (CPTs)  from data. As far as I know, all the tutorials or examples of Infer.net defines CPD first. So I am wondering if there's any way to learn parameters from data? 

    I have a database table which contains data with multiple columns (attributes). In BNT, I can define a Dirichlet prior for each column and then, given the data, I can sequentially update the CPT by just run a function called "bayes_update_params" (data as multinomial distribution). Note that, I define the structure of bayesian network first in adjacent matrix.

    Does Infer.net has same function on updating parameters from data? Could you give me any examples?

    Thanks very much !

     

     

    Friday, June 3, 2011 6:26 PM

Answers

  • John Guiver replied on 03-02-2011 12:19 PM

    Just to keep the community informed - there has been ongoing discussion on the performance issue via infersup@microsoft.com. Here is a summary:

    1. There is a performance bug in the way switch and case statements are handled which is significant for very large value ranges. This has been addressed for switch statements and will be available the next time we do a release.
    2. At query time, the CPTs can be directly observed using the CPT posterior means rather than observing the CPT prior using the full CPT posteriors; this has a significant impact as the latter choice means that the CPTs continue to learn at query-time also.
    3. At query time, many fewer message passing iterations are needed - the default of 50 can be reduced to 10, say.

    John

    Friday, June 3, 2011 6:28 PM

All replies

  • John Guiver replied on 02-08-2011 12:32 PM

    Hi Henry

    Yes - you can do this in Infer.NET without any problem, and can define much more general models. I can help you with the code, but before I do that, can you give me a concrete toy problem including model structure and data so (a) we can make sure that there is no misunderstanding in what you want to do, and (b) you can compare results with your existing models. Easiest just paste the data structures (C# preferably) into a post.

    John

    Friday, June 3, 2011 6:26 PM
  • Henry Hoo replied on 02-08-2011 1:47 PM

    Hi John,

    Thanks very much for your reply.  Here's the toy example. Suppose we have following Car table with 6 columns:

    Model

    Make

    Type

    Powertrain

    Engine

    MDX

    Acura

    SUV

    AWD

    3.7L V6

    TL

    Acura

    Sedan

    FWD

    3.5L V6

    RDX

    Acura

    SUV

    FWD

    2.3L I4

    TL

    Acura

    Sedan

    FWD

    3.5L V6

    TSX

    Acura

    Sedan

    FWD

    2.4L I4

    TSX

    Acura

    Sedan

    FWD

    2.4L I4

    TL

    Acura

    Sedan

    FWD

    3.5L V6

    TL

    Acura

    Sedan

    FWD

    3.5L V6

    RDX

    Acura

    SUV

    FWD

    2.3L I4

    TSX

    Acura

    Sedan

    FWD

    2.4L I4

    TSX

    Acura

    Sedan

    FWD

    2.4L I4

    MDX

    Acura

    SUV

    4X4

    3.5L V6

    From this table, we have learnt a Bayes net with structure as:

    Powertrain---> Engine---> Type
                                 ^
                                 |
                                 |

                Make-->Model          

    (Engine is determined by both Powertrain and Model)              

     

    Then, in BNT (Sorry, I am also new to C#, so I use Matlab code instead, hope you can understand), we assign a Dirichlet prior for each column, such as:

    bayesnet.CPD{'Model'} = tabular_CPD(bayesnet, 'Model', 'prior_type', 'dirichlet', 'dirichlet_type', 'unif'); for each column

    and we update bayes net by: bayesnet = bayes_update_params(bayesnet, cases); where cases are from the Car table by following commands in BNT:

    data = textread('datafile.csv', '', 'delimiter', ',');

    ncases = size(data, 1); 

    cases = cell(N, ncases);  

    cases(:, :) = num2cell(data');

     

    Finally, I can do inference. For example, given the evidence that Model='MDX', what 's the probability: P(Type|Model='MDX')? I can get an array of results (different types with different probabilities, such as SUV: 0.68), from which I pick the maximum one.


     Now, I am moving to C# and Infer.NET which I believe is much faster. Could you tell me what should I do in Infer.NET for above example, step by step?

    Thanks again for your help !

     

     

     

    Friday, June 3, 2011 6:26 PM
  • John Guiver replied on 02-10-2011 5:18 AM

    Hi Henry

    Below is an example model class to solve your problem along with a test program. You can drop it straight into a console application in Visual Studio, though making sure you add references to the Infer.NET libraries. Rather than give you the simplest possible code I have implemented an efficient well-structured version which hopefully will allow you to do some larger scale experiments. Note that Infer.NET has a model compilation step which, when you first run inference, converts your model into a piece of inference code; subsequent queries will then use the generated code to run very efficiently.

    As part of the efficient implementation, the prior and posterior distributions over the conditional probability tables are represented as distributions over array variables rather than .NET arrays of Dirichlet distributions - these are the DirichletArray and DirichletArray2D aliases at the top of the code. You can recover the .NET arrays by using the Distribution.ToArray<Dirichlet[]> and Distribution.ToArray<Dirichlet[,]> methods respectively.

    I have not put in any code to read in data as this is not part of Infer.NET.

    I have put comments in the code which are hopefully reasonably self explanatory, especially if you additionally invest time in reading the Infer.NET user guide.

    John

    using System;

    using System.Collections.Generic;

    using System.Linq;

    using System.Text;

    using MicrosoftResearch.Infer.Models;

    using MicrosoftResearch.Infer.Maths;

    using MicrosoftResearch.Infer.Distributions;

    using MicrosoftResearch.Infer;

    using MicrosoftResearch.Infer.Collections;

     

     

    namespace BayesNet

    {

      // Aliases for distributions over 1-D and 2-D probability tables

      using DirichletArray = DistributionRefArray<Dirichlet, Vector>;

      using DirichletArray2D = DistributionRefArray2D<Dirichlet, Vector>;

      public class CarBayesNet

      {

        // Model variables

        public Variable<int> NumCases = Variable.New<int>();

        public VariableArray<int> CarModel;

        public VariableArray<int> CarType;

        public VariableArray<int> PowerTrain;

        public VariableArray<int> EngineType;

        // Probability tables and conditional probability tables

        public Variable<Vector> PT_M;

        public Variable<Vector> PT_P;

        public VariableArray2D<Vector> CPT_E;

        public VariableArray<Vector> CPT_T;

        // Prior distributions for the probability tables - make these variable so we can

        // change at run-time without model recompilation

        public Variable<Dirichlet> PT_M_Prior = null;

        public Variable<Dirichlet> PT_P_Prior = null;

        public Variable<DirichletArray2D> CPT_E_Prior = null;

        public Variable<DirichletArray> CPT_T_Prior = null;

        // Inferred posterior distributions over the probability tables

        public Dirichlet PT_M_Posterior = null;

        public Dirichlet PT_P_Posterior = null;

        public DirichletArray2D CPT_E_Posterior = null;

        public DirichletArray CPT_T_Posterior = null;

        // Inference engine

        public InferenceEngine InfEngine = new InferenceEngine() { ShowProgress = false };

     

        public void CreateModel(int numModels, int numTypes, int numPowerTrains, int numEngines)

        {

          NumCases = Variable.New<int>(); // Set this at run time

          // The ranges and value ranges

          Range n = new Range(NumCases);

          Range m = new Range(numModels);

          Range t = new Range(numTypes);

          Range p = new Range(numPowerTrains);

          Range e = new Range(numEngines);

     

          // Define the priors as variables - these will be set (observed) at run time

          PT_M_Prior = Variable.New<Dirichlet>();

          PT_P_Prior = Variable.New<Dirichlet>();

          CPT_E_Prior = Variable.New<DirichletArray2D>();

          CPT_T_Prior = Variable.New<DirichletArray>();

          // Probability table and conditional probability table variables (parameters)

          PT_M = Variable<Vector>.Random(PT_M_Prior);

          PT_P = Variable<Vector>.Random(PT_P_Prior);

          CPT_E = Variable.Array<Vector>(m, p);

          CPT_E.SetTo(Variable<Vector[,]>.Random(CPT_E_Prior));

          CPT_T = Variable.Array<Vector>(e);

          CPT_T.SetTo(Variable<Vector[]>.Random(CPT_T_Prior));

          // Set the value ranges for the probability tables

          PT_M.SetValueRange(m);

          PT_P.SetValueRange(p);

          CPT_E.SetValueRange(e);

          CPT_T.SetValueRange(t);

          // Describe the structure

          CarModel = Variable.Array<int>(n);

          CarModel[n] = Variable.Discrete(PT_M).ForEach(n);

          PowerTrain = Variable.Array<int>(n);

          PowerTrain[n] = Variable.Discrete(PT_P).ForEach(n);

          EngineType = AddChildFromTwoParents(CarModel, PowerTrain, CPT_E);

          CarType = AddChildFromOneParent(EngineType, CPT_T);

        }

        // Infer the probability tables and conditional probability tables

        public void InferParameters(int[] modelData, int[] typeData, int[] ptData, int[] engineData)

        {

          // Set the observations:

          NumCases.ObservedValue = modelData.Length;

          CarModel.ObservedValue = modelData;

          CarType.ObservedValue = typeData;

          PowerTrain.ObservedValue = ptData;

          EngineType.ObservedValue = engineData;

     

          // Set the uniform priors for the probability tables

          int numModels = CarModel.GetValueRange().SizeAsInt;

          int numPowerTrains = PowerTrain.GetValueRange().SizeAsInt;

          int numEngines = EngineType.GetValueRange().SizeAsInt;

          int numTypes = CarType.GetValueRange().SizeAsInt;

     

          Dirichlet[,] cpt_E_Prior = new Dirichlet[numModels, numPowerTrains];

          Dirichlet[] cpt_T_Prior = new Dirichlet[numEngines];

          for (int i = 0; i < numModels; i++)

            for (int j = 0; j < numPowerTrains; j++)

              cpt_E_Prior[i, j] = Dirichlet.Uniform(numEngines);

          for (int i = 0; i < numEngines; i++)

            cpt_T_Prior[i] = Dirichlet.Uniform(numTypes);

          PT_M_Prior.ObservedValue = Dirichlet.Uniform(numModels);

          PT_P_Prior.ObservedValue = Dirichlet.Uniform(numPowerTrains);

          CPT_E_Prior.ObservedValue = (DirichletArray2D)Distribution<Vector>.Array(cpt_E_Prior);

          CPT_T_Prior.ObservedValue = (DirichletArray)Distribution<Vector>.Array(cpt_T_Prior);

     

          // Run the inference

          InfEngine.InferAll(PT_M, PT_P, CPT_E, CPT_T);

          PT_M_Posterior = InfEngine.Infer<Dirichlet>(PT_M);

          PT_P_Posterior = InfEngine.Infer<Dirichlet>(PT_P);

          CPT_E_Posterior = InfEngine.Infer<DirichletArray2D>(CPT_E);

          CPT_T_Posterior = InfEngine.Infer<DirichletArray>(CPT_T);

        }

        // Query the distribution over car type conditioned on each of a set of model types

        public Discrete[] QueryCarTypesFromModelTypes(int[] models)

        {

          NumCases.ObservedValue = models.Length;

          CarModel.ObservedValue = models;

          CarType.ClearObservedValue();

          PowerTrain.ClearObservedValue();

          EngineType.ClearObservedValue();

     

          PT_M_Prior.ObservedValue = PT_M_Posterior;

          PT_P_Prior.ObservedValue = PT_P_Posterior;

          CPT_E_Prior.ObservedValue = CPT_E_Posterior;

          CPT_T_Prior.ObservedValue = CPT_T_Posterior;

     

          // Run the inference

          return InfEngine.Infer<Discrete[]>(CarType);

        }

        // Model code for adding a child from a single parent

        public static VariableArray<int> AddChildFromOneParent(

          VariableArray<int> parent, VariableArray<Vector> cpt)

        {

          var d = parent.Range;

          // data range

          var child = Variable.Array<int>(d);

          using (Variable.ForEach(d))

          using (Variable.Switch(parent[d]))

            child[d] = Variable.Discrete(cpt[parent[d]]);

          return child;

        }

        // Model code for adding a child from two parents

        public static VariableArray<int> AddChildFromTwoParents(

          VariableArray<int> parent1, VariableArray<int> parent2, VariableArray2D<Vector> cpt)

        {

          var d = parent1.Range;

          // data range

          var child = Variable.Array<int>(d);

          using (Variable.ForEach(d))

          using (Variable.Switch(parent1[d]))

          using (Variable.Switch(parent2[d]))

            child[d] = Variable.Discrete(cpt[parent1[d], parent2[d]]);

          return child;

        }

      }

     

      class Program

      {

        static void Main(string[] args)

        {

          int[] modelData = new int[] { 0, 1, 2, 1, 3, 3, 1, 1, 2, 3, 3, 0 };

          int[] typeData = new int[] { 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0 };

          int[] ptData = new int[] { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2 };

          int[] engineData = new int[] { 0, 1, 2, 1, 3, 3, 1, 1, 2, 3, 3, 1 };

          int numModels = 4;

          int numTypes = 2;

          int numPowerTrains = 3;

          int numEngines = 4;

     

          // Learn the parameters

          CarBayesNet bayesNet = new CarBayesNet();

          bayesNet.CreateModel(numModels, numTypes, numPowerTrains, numEngines);

          bayesNet.InferParameters(modelData, typeData, ptData, engineData);

     

          // Print out the PT distributions

          Console.WriteLine("\nCar model PT");

          Console.WriteLine(bayesNet.PT_M_Posterior);

          Console.WriteLine("\nPower Train PT");

          Console.WriteLine(bayesNet.PT_P_Posterior);

          Console.WriteLine("\nEngine CPT");

          Console.WriteLine(bayesNet.CPT_E_Posterior);

          Console.WriteLine("\nType CPT:");

          Console.WriteLine(bayesNet.CPT_T_Posterior);

     

          // Query the model

          int[] modelTypes = System.Linq.Enumerable.Range(0, numModels).ToArray();

          var carTypesFromModelTypes = bayesNet.QueryCarTypesFromModelTypes(modelTypes);

          Console.WriteLine("\nDistributions for car types from model types");

     

          for (int i = 0; i < numModels; i++)

            Console.WriteLine("Probability of type, given model {0}: {1}", i, carTypesFromModelTypes[i]);

        }

      }

    }

     

     

     

    Friday, June 3, 2011 6:26 PM
  • minka replied on 02-10-2011 6:05 AM

    Note that a somewhat cleaner approach to declaring the CPT parameters is to use VariableArrays of Dirichlet rather than DirichletArray.  However, as John points out, this is slightly less efficient. Here is the code when using this method:

     

    using System;

    using System.Collections.Generic;

    using System.Linq;

    using System.Text;

    using MicrosoftResearch.Infer.Models;

    using MicrosoftResearch.Infer.Maths;

    using MicrosoftResearch.Infer.Distributions;

    using MicrosoftResearch.Infer;

    using MicrosoftResearch.Infer.Collections;

     

     

    namespace BayesNet

    {

      public class CarBayesNet

      {

        // Model variables

        public Variable<int> NumCases = Variable.New<int>();

        public VariableArray<int> CarModel;

        public VariableArray<int> CarType;

        public VariableArray<int> PowerTrain;

        public VariableArray<int> EngineType;

        // Probability tables and conditional probability tables

        public Variable<Vector> PT_M;

        public Variable<Vector> PT_P;

        public VariableArray2D<Vector> CPT_E;

        public VariableArray<Vector> CPT_T;

        // Prior distributions for the probability tables - make these variable so we can

        // change at run-time without model recompilation

        public Variable<Dirichlet> PT_M_Prior = null;

        public Variable<Dirichlet> PT_P_Prior = null;

        public VariableArray2D<Dirichlet> CPT_E_Prior = null;

        public VariableArray<Dirichlet> CPT_T_Prior = null;

        // Inferred posterior distributions over the probability tables

        public Dirichlet PT_M_Posterior = null;

        public Dirichlet PT_P_Posterior = null;

        public Dirichlet[,] CPT_E_Posterior = null;

        public Dirichlet[] CPT_T_Posterior = null;

        // Inference engine

        public InferenceEngine InfEngine = new InferenceEngine() { ShowProgress = false };

     

        public void CreateModel(int numModels, int numTypes, int numPowerTrains, int numEngines)

        {

          NumCases = Variable.New<int>(); // Set this at run time

          // The ranges and value ranges

          Range n = new Range(NumCases);

          Range m = new Range(numModels);

          Range t = new Range(numTypes);

          Range p = new Range(numPowerTrains);

          Range e = new Range(numEngines);

     

          // Define the priors as variables - these will be set (observed) at run time

          PT_M_Prior = Variable.New<Dirichlet>();

          PT_P_Prior = Variable.New<Dirichlet>();

          CPT_E_Prior = Variable.Array<Dirichlet>(m,p);

          CPT_T_Prior = Variable.Array<Dirichlet>(e);

          // Probability table and conditional probability table variables (parameters)

          PT_M = Variable<Vector>.Random(PT_M_Prior);

          PT_P = Variable<Vector>.Random(PT_P_Prior);

          CPT_E = Variable.Array<Vector>(m, p);

          CPT_E[m,p] = Variable<Vector>.Random(CPT_E_Prior[m,p]);

          CPT_T = Variable.Array<Vector>(e);

          CPT_T[e] = Variable<Vector>.Random(CPT_T_Prior[e]);

          // Set the value ranges for the probability tables

          PT_M.SetValueRange(m);

          PT_P.SetValueRange(p);

          CPT_E.SetValueRange(e);

          CPT_T.SetValueRange(t);

          // Describe the structure

          CarModel = Variable.Array<int>(n);

          CarModel[n] = Variable.Discrete(PT_M).ForEach(n);

          PowerTrain = Variable.Array<int>(n);

          PowerTrain[n] = Variable.Discrete(PT_P).ForEach(n);

          EngineType = AddChildFromTwoParents(CarModel, PowerTrain, CPT_E);

          CarType = AddChildFromOneParent(EngineType, CPT_T);

        }

        // Infer the probability tables and conditional probability tables

        public void InferParameters(int[] modelData, int[] typeData, int[] ptData, int[] engineData)

        {

          // Set the observations:

          NumCases.ObservedValue = modelData.Length;

          CarModel.ObservedValue = modelData;

          CarType.ObservedValue = typeData;

          PowerTrain.ObservedValue = ptData;

          EngineType.ObservedValue = engineData;

     

          // Set the uniform priors for the probability tables

          int numModels = CarModel.GetValueRange().SizeAsInt;

          int numPowerTrains = PowerTrain.GetValueRange().SizeAsInt;

          int numEngines = EngineType.GetValueRange().SizeAsInt;

          int numTypes = CarType.GetValueRange().SizeAsInt;

     

          Dirichlet[,] cpt_E_Prior = new Dirichlet[numModels, numPowerTrains];

          Dirichlet[] cpt_T_Prior = new Dirichlet[numEngines];

          for (int i = 0; i < numModels; i++)

            for (int j = 0; j < numPowerTrains; j++)

              cpt_E_Prior[i, j] = Dirichlet.Uniform(numEngines);

          for (int i = 0; i < numEngines; i++)

            cpt_T_Prior[i] = Dirichlet.Uniform(numTypes);

          PT_M_Prior.ObservedValue = Dirichlet.Uniform(numModels);

          PT_P_Prior.ObservedValue = Dirichlet.Uniform(numPowerTrains);

          CPT_E_Prior.ObservedValue = cpt_E_Prior;

          CPT_T_Prior.ObservedValue = cpt_T_Prior;

     

          // Run the inference

          InfEngine.InferAll(PT_M, PT_P, CPT_E, CPT_T);

          PT_M_Posterior = InfEngine.Infer<Dirichlet>(PT_M);

          PT_P_Posterior = InfEngine.Infer<Dirichlet>(PT_P);

          CPT_E_Posterior = InfEngine.Infer<Dirichlet[,]>(CPT_E);

          CPT_T_Posterior = InfEngine.Infer<Dirichlet[]>(CPT_T);

        }

        // Query the distribution over car type conditioned on each of a set of model types

        public Discrete[] QueryCarTypesFromModelTypes(int[] models)

        {

          NumCases.ObservedValue = models.Length;

          CarModel.ObservedValue = models;

          CarType.ClearObservedValue();

          PowerTrain.ClearObservedValue();

          EngineType.ClearObservedValue();

     

          PT_M_Prior.ObservedValue = PT_M_Posterior;

          PT_P_Prior.ObservedValue = PT_P_Posterior;

          CPT_E_Prior.ObservedValue = CPT_E_Posterior;

          CPT_T_Prior.ObservedValue = CPT_T_Posterior;

     

          // Run the inference

          return InfEngine.Infer<Discrete[]>(CarType);

        }

        // Model code for adding a child from a single parent

        public static VariableArray<int> AddChildFromOneParent(

          VariableArray<int> parent, VariableArray<Vector> cpt)

        {

          var d = parent.Range;

          // data range

          var child = Variable.Array<int>(d);

          using (Variable.ForEach(d))

          using (Variable.Switch(parent[d]))

            child[d] = Variable.Discrete(cpt[parent[d]]);

          return child;

        }

        // Model code for adding a child from two parents

        public static VariableArray<int> AddChildFromTwoParents(

          VariableArray<int> parent1, VariableArray<int> parent2, VariableArray2D<Vector> cpt)

        {

          var d = parent1.Range;

          // data range

          var child = Variable.Array<int>(d);

          using (Variable.ForEach(d))

          using (Variable.Switch(parent1[d]))

          using (Variable.Switch(parent2[d]))

            child[d] = Variable.Discrete(cpt[parent1[d], parent2[d]]);

          return child;

        }

      }

     

      class Program

      {

        static void Main(string[] args)

        {

          int[] modelData = new int[] { 0, 1, 2, 1, 3, 3, 1, 1, 2, 3, 3, 0 };

          int[] typeData = new int[] { 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 1, 0 };

          int[] ptData = new int[] { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2 };

          int[] engineData = new int[] { 0, 1, 2, 1, 3, 3, 1, 1, 2, 3, 3, 1 };

          int numModels = 4;

          int numTypes = 2;

          int numPowerTrains = 3;

          int numEngines = 4;

     

          // Learn the parameters

          CarBayesNet bayesNet = new CarBayesNet();

          bayesNet.CreateModel(numModels, numTypes, numPowerTrains, numEngines);

          bayesNet.InferParameters(modelData, typeData, ptData, engineData);

     

          // Print out the PT distributions

          Console.WriteLine("\nCar model PT");

          Console.WriteLine(bayesNet.PT_M_Posterior);

          Console.WriteLine("\nPower Train PT");

          Console.WriteLine(bayesNet.PT_P_Posterior);

          Console.WriteLine("\nEngine CPT");

          Console.WriteLine(Distribution<Vector>.Array(bayesNet.CPT_E_Posterior));

          Console.WriteLine("\nType CPT:");

          Console.WriteLine(Distribution<Vector>.Array(bayesNet.CPT_T_Posterior));

     

          // Query the model

          int[] modelTypes = System.Linq.Enumerable.Range(0, numModels).ToArray();

          var carTypesFromModelTypes = bayesNet.QueryCarTypesFromModelTypes(modelTypes);

          Console.WriteLine("\nDistributions for car types from model types");

     

          for (int i = 0; i < numModels; i++)

            Console.WriteLine("Probability of type, given model {0}: {1}", i, carTypesFromModelTypes[i]);

        }

      }

    }

     

     

     

    • Proposed as answer by rohitrnathan Monday, July 4, 2011 6:07 AM
    • Unproposed as answer by rohitrnathan Monday, July 4, 2011 6:07 AM
    Friday, June 3, 2011 6:27 PM
  • sushovande replied on 02-11-2011 3:08 PM

    John & Minka

    I am Henry's partner in this project. The code sample helped a lot. We now have a wroking Bayes Net Model over 9 variables and 5000 input tuples working as expected :) Thank you very much.

    We find that it's a bit slower than we thought, we are currently exploring the tips in this article to see if that helps or not.

    Friday, June 3, 2011 6:27 PM
  • minka replied on 02-12-2011 3:33 AM

    Try using the ShowProgress and ShowTiming options on the InferenceEngine to see where the slowness is.  The code we sent has ShowProgress turned off.

    Friday, June 3, 2011 6:27 PM
  • Henry Hoo replied on 02-15-2011 3:38 AM

    Hi John & Tom,

    Thanks very much for helping us ! The code is exactly what we want. It works well and can handle somewhat large-scale data. The only problem is for building CPT, current our program takes about 5 min while BNT use 1.2s. We would take a look at this issue. 

    Friday, June 3, 2011 6:27 PM
  • forceamp replied on 02-16-2011 11:28 AM

    I am very interested in whether you are able to close the performance gap between Infer.net and BNT.

    Please keep us informed on your efforts.

    Thanks.

    Friday, June 3, 2011 6:27 PM
  • John Guiver replied on 02-21-2011 5:03 AM

    Hi Henry

    Please can you provide more information? The generated code should be very fast. First we need to understand what is taking the time. If the model is getting compiled each time the code is run then this will slow things down.

    Could you give us more information. As Tom says, switch on the show progress flag to see whether it is compiler or runtime. Also please send the code and the experiments you are doing to infersup@microsoft.com or at least give details that will enable us to see what's going on.

    Finally, if BNT only takes 1.2s why were you looking for something faster?

    John

    Friday, June 3, 2011 6:27 PM
  • Henry Hoo replied on 02-21-2011 2:54 PM

    HI John,

    Sorry for recent silence as we were working on a paper deadline last week....We will test the program using Tom's suggestion and will report our progress here. We will also send our code (and also BNT code) and running example to infersup@microsoft.com so you can do further analysis later this week.

    One of the reasons we chose Infer.NET over BNT was the learning and inference is a intermediate part of our project, it receives parameters (e.g., evidence name, value...etc) from a module and return probabilities to other modules. In this case, we cannot learn the probability in advance and pre-store it. Originally, we wrote Inference code in BNT and other modules were written in Java to call the BNT function. However, as you might know, calling Matlab function in Java is unreliable, slow and not well supported by Matlab, especially when we don't have the JABuilder (a build-in toolbox for Java connection in Matlab, which is for commercial use only).

    We found Infer.NET perfectly fits our requirements (except structure learning as we have to learn the BN structure using other programs). So we rewrote our program in C# and found it works.The only concern is about speed but we can live with that for running experiment. For further use, we are seeking to accelerate it.

    Thanks for providing such powerful tool to the community! We appreciate your time and help !

    Friday, June 3, 2011 6:27 PM
  • John Guiver replied on 02-22-2011 3:56 AM

    OK. Thanks Henry.

    One other point to consider as you extend your model. The conditional 2-D CPTs in the example code we sent are rectangular, but in some cases you may want to make them jagged - i.e. the set of choices may depend on the condition. This can be easily done using jagged arrays - i.e. use VariableArray<VariableArray<Vector>,Vector[][]> rather than VariableArray2D<Vector>. Similarly you can extend to three parent nodes.

    John

    Friday, June 3, 2011 6:27 PM
  • John Guiver replied on 03-02-2011 12:19 PM

    Just to keep the community informed - there has been ongoing discussion on the performance issue via infersup@microsoft.com. Here is a summary:

    1. There is a performance bug in the way switch and case statements are handled which is significant for very large value ranges. This has been addressed for switch statements and will be available the next time we do a release.
    2. At query time, the CPTs can be directly observed using the CPT posterior means rather than observing the CPT prior using the full CPT posteriors; this has a significant impact as the latter choice means that the CPTs continue to learn at query-time also.
    3. At query time, many fewer message passing iterations are needed - the default of 50 can be reduced to 10, say.

    John

    Friday, June 3, 2011 6:28 PM
  • Hi

    I tried to debug the problem. I found that whenever I have a node with two children, the dirichlet distribution is not updating properly.

    In the above case, Node Body has 2 parents - Model and Year. The Dirichlet distribution corresponding to Model = Tucson and Year = 2005 says Body = SUV is 2 despite the count being 198 in the training data.

    I do not have this problem when I have a node with only 1 or no parents.

    I have posted the same issue in this thread.

    Thanks,

    Rohit

    Monday, July 4, 2011 6:12 AM