Answered by:
Creating my own distribution (Migrated from community.research.microsoft.com)
Question

JaviOliver posted on 03212011 6:41 AM
Dear Tom,
I want to approximate the posterior distribution from a quite complicated Likelihood and its observations: p(y,x) = ∏<sub> </sub>t<sub>i</sub>(x), approximated by q(x) = ∏<sub> </sub>t’<sub>i</sub>(x), where each t’<sub>i</sub>(x) is Gaussian.
I expect q(x) to be a MoG that approximates my distribution.
My question is:
To get this approximated distribution q(x), shall I start from the example of Mixture of Gaussians, then create my own distribution type, and then create the data variable using MyDistribution instead of doing data[n] = Variable.VectorGaussianFromMeanAndPrecision(means[z[n]], precs[z[n]]);
Is it correct? Sorry if I did a mess…
Friday, June 3, 2011 6:39 PM
Answers

JaviOliver replied on 03232011 9:46 AM
Ok, now I understand it. Thanks for the reply!
 Marked as answer by Microsoft Research Friday, June 3, 2011 6:39 PM
Friday, June 3, 2011 6:39 PM
All replies

minka replied on 03212011 9:02 AM
Something is wrong with your description because if each t'_i(x) is Gaussian, then q(x) must be Gaussian by definition. As for the Mixture of Gaussian example, you haven't said which variable is going to be 'x'. Is 'data' going to be 'x'?
Friday, June 3, 2011 6:39 PM 
JaviOliver replied on 03222011 6:46 AM
Sorry, I did not explain it well...
I have a Likelihood function with a complex form, with parameters w1, w2, w3. L(w1, w2, w3, obs)
I want to infer the posterior distribution for these parameters, defined as VectorGaussian, in the region of the observations using EP. The approximate function must be characterised as a MoG in 3D space.
Would the attached figure correctly describe the problem? In that case, I should create my own distribution type for the data, right? Once the distribution is created, would the procedure be as in the example of Bayesian Mixture of Gaussians?
// Create a variable array which will hold the data
Range n = new Range(100).Named("n");
VariableArray<Vector> data = Variable.Array<Vector>(n).Named("x");
// Create latent indicator variable for each data point
VariableArray<int> z = Variable.Array<int>(n).Named("z");
// The mixture of Gaussians model
using (Variable.ForEach(n))
{
z[n] = Variable.Discrete(weights);
using (Variable.Switch(z[n]))
{
data[n] = Variable.VectorMyDistributionFromParameters(w1[z[n]], w2[z[n]], w3[z[n]])
}
}
Thanks a lot.
Friday, June 3, 2011 6:39 PM 
minka replied on 03222011 8:22 AM
It sounds like you want to take a likelihood function of arbitrary form L(w1,w2,w3,obs) and approximate it by a MoG likelihood L2(w1,w2,w3,obs) according to the figure. Neither EP nor Infer.NET provides this capability. Infer.NET always uses a fully factorized approximation and this one is not fully factorized. You could write a custom distribution type for the variables (w1,w2,w3) jointly but this wouldn’t look like your code since your code is written in the modelling language. Distribution types are implemented as classes that implement a set of interfaces. On top of that, since EP doesn’t support this distribution you would have to write new message functions to manipulate these distributions. See the user guide for instructions on this.
Friday, June 3, 2011 6:39 PM 
JaviOliver replied on 03232011 9:46 AM
Ok, now I understand it. Thanks for the reply!
 Marked as answer by Microsoft Research Friday, June 3, 2011 6:39 PM
Friday, June 3, 2011 6:39 PM