locked
Creating my own distribution (Migrated from community.research.microsoft.com) RRS feed

  • Question

  • JaviOliver posted on 03-21-2011 6:41 AM

    Dear Tom,

     I want to approximate the posterior distribution from a quite complicated Likelihood and its observations: p(y,x) = ∏<sub> </sub>t<sub>i</sub>(x), approximated by q(x) = ∏<sub> </sub>t’<sub>i</sub>(x), where each t’<sub>i</sub>(x) is Gaussian.

     I expect q(x) to be a MoG that approximates my distribution.

     My question is:

    To get this approximated distribution q(x), shall I start from the example of Mixture of Gaussians, then create my own distribution type, and then create the data variable using MyDistribution instead of doing data[n] = Variable.VectorGaussianFromMeanAndPrecision(means[z[n]], precs[z[n]]);

     

    Is it correct?  Sorry if I did a mess…

     

    Friday, June 3, 2011 6:39 PM

Answers

  • JaviOliver replied on 03-23-2011 9:46 AM

    Ok, now I understand it. Thanks for the reply!

    Friday, June 3, 2011 6:39 PM

All replies

  • minka replied on 03-21-2011 9:02 AM

    Something is wrong with your description because if each t'_i(x) is Gaussian, then q(x) must be Gaussian by definition.  As for the Mixture of Gaussian example, you haven't said which variable is going to be 'x'.  Is 'data' going to be 'x'? 

    Friday, June 3, 2011 6:39 PM
  • JaviOliver replied on 03-22-2011 6:46 AM

    Sorry, I did not explain it well...

    I have a Likelihood function with a complex form, with parameters w1, w2, w3.  L(w1, w2, w3, obs)

    I want to infer the posterior distribution for these parameters, defined as VectorGaussian, in the region of the observations using EP. The approximate function must be characterised as a MoG in 3D space.

    Would the attached figure correctly describe the problem? In that case, I should create my own distribution type for the data, right?  Once the distribution is created, would the procedure be as in the example of Bayesian Mixture of Gaussians?

     

    // Create a variable array which will hold the data

    Range n = new Range(100).Named("n");

    VariableArray<Vector> data = Variable.Array<Vector>(n).Named("x");

     

    // Create latent indicator variable for each data point

    VariableArray<int> z = Variable.Array<int>(n).Named("z");

     

    // The mixture of Gaussians model

    using (Variable.ForEach(n))

    {

      z[n] = Variable.Discrete(weights);

      using (Variable.Switch(z[n]))

      {

     data[n] = Variable.VectorMyDistributionFromParameters(w1[z[n]], w2[z[n]], w3[z[n]])

      }

    }

     

    Thanks a lot.

     

     

    Friday, June 3, 2011 6:39 PM
  • minka replied on 03-22-2011 8:22 AM

    It sounds like you want to take a likelihood function of arbitrary form L(w1,w2,w3,obs) and approximate it by a MoG likelihood L2(w1,w2,w3,obs) according to the figure.  Neither EP nor Infer.NET provides this capability.  Infer.NET always uses a fully factorized approximation and this one is not fully factorized.  You could write a custom distribution type for the variables (w1,w2,w3) jointly but this wouldn’t look like your code since your code is written in the modelling language.  Distribution types are implemented as classes that implement a set of interfaces.  On top of that, since EP doesn’t support this distribution you would have to write new message functions to manipulate these distributions.  See the user guide for instructions on this.

    Friday, June 3, 2011 6:39 PM
  • JaviOliver replied on 03-23-2011 9:46 AM

    Ok, now I understand it. Thanks for the reply!

    Friday, June 3, 2011 6:39 PM