Fully factorized approximations<p>Dear All,</p>
<p>I'm pretty new to Infer.NET and graphical models in general so excuse me if the question might be trivial or if I got it wrong to start with. In the end what I would like to do is learn a linear-chain CRF for some sort of sequence labeling task given data.
</p>
<p>Let's assume I have T outcome variables Y_i, i=1..T and for each Y_i an input variable X_i which I observe. What I am interested in is the MAP assignment of Y given X, so argmax(Y) P(Y|X). From what I understood of Infer.NET so far it makes use of a so-called
"fully factorized approximation" which means I can use the inference engine to query each P(Y_i | X_i) and the joint CPD will then be the product of the CPDs of the individual variables. Or I could simply use each individual CPD to chose my MAP assignment
for that position and repeat that for the whole sequence. But doesn't that imply that all Y_i's are independent of each other which is exactly what I am trying to prevent in the first place by using a graphical model with edges between adjacent Y_i's? This
has caused me some headache so any hint would be appreciated.</p>
<p>Further, in the end I would like to make advantage of the fact that I model my problem as a discriminative model, not generative, meaning that I might want to include rich features where each Y_i not only depends on its X_i, but potentially on all or a large
subset of members of X. Is there an explicit way of modeling that in Infer.NET or is it simply a question of not querying P(X), i.e. only asking questions about an unnormalized measure P'(Y|X)? How do I tell Infer.NET that I only would like the unnormalized
measure? I hope that makes sense.</p>
<p>Thanks a lot in advance,</p>
<p>Roland</p>
<p></p>© 2009 Microsoft Corporation. All rights reserved.Sun, 06 May 2012 21:08:11 Zc434657c-fa75-4941-a409-dad62494f25b- https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#c434657c-fa75-4941-a409-dad62494f25bhttps://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#c434657c-fa75-4941-a409-dad62494f25bUlchuchuhttps://social.microsoft.com:443/profile/ulchuchu/?type=forumFully factorized approximations<p>Dear All,</p>
<p>I'm pretty new to Infer.NET and graphical models in general so excuse me if the question might be trivial or if I got it wrong to start with. In the end what I would like to do is learn a linear-chain CRF for some sort of sequence labeling task given data.
</p>
<p>Let's assume I have T outcome variables Y_i, i=1..T and for each Y_i an input variable X_i which I observe. What I am interested in is the MAP assignment of Y given X, so argmax(Y) P(Y|X). From what I understood of Infer.NET so far it makes use of a so-called
"fully factorized approximation" which means I can use the inference engine to query each P(Y_i | X_i) and the joint CPD will then be the product of the CPDs of the individual variables. Or I could simply use each individual CPD to chose my MAP assignment
for that position and repeat that for the whole sequence. But doesn't that imply that all Y_i's are independent of each other which is exactly what I am trying to prevent in the first place by using a graphical model with edges between adjacent Y_i's? This
has caused me some headache so any hint would be appreciated.</p>
<p>Further, in the end I would like to make advantage of the fact that I model my problem as a discriminative model, not generative, meaning that I might want to include rich features where each Y_i not only depends on its X_i, but potentially on all or a large
subset of members of X. Is there an explicit way of modeling that in Infer.NET or is it simply a question of not querying P(X), i.e. only asking questions about an unnormalized measure P'(Y|X)? How do I tell Infer.NET that I only would like the unnormalized
measure? I hope that makes sense.</p>
<p>Thanks a lot in advance,</p>
<p>Roland</p>
<p></p>Wed, 02 May 2012 12:18:28 Z2012-05-02T12:18:28Z
- https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#cdf977fe-7899-4aa6-a12e-089787be0436https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#cdf977fe-7899-4aa6-a12e-089787be0436Tom Minkahttps://social.microsoft.com:443/profile/tom%20minka/?type=forumFully factorized approximationsInfer.NET only allows inferring the marginal distribution of a single variable at a time and therefore it cannot compute a joint MAP assignment.Fri, 04 May 2012 17:04:34 Z2012-05-04T17:04:34Z
- https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#8c78c3e6-ffb4-4b3a-8cdc-7332568ed8abhttps://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#8c78c3e6-ffb4-4b3a-8cdc-7332568ed8abUlchuchuhttps://social.microsoft.com:443/profile/ulchuchu/?type=forumFully factorized approximations<p>Well if that was a design choice, is querying the marginals typically sufficient for most real-world applications? I've been dealing mostly with HMMs in the past and I realize that working on the marginals only to determine an optimal state path can give
similar results to say the Viterbi path, but still there are cases where the joint assignment is what you are really looking for. Any thoughts on that?</p>Sat, 05 May 2012 09:40:14 Z2012-05-05T09:40:14Z
- https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#6fb3892b-e65c-4371-a7f6-8c69c0dcb8ddhttps://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#6fb3892b-e65c-4371-a7f6-8c69c0dcb8ddTom Minkahttps://social.microsoft.com:443/profile/tom%20minka/?type=forumFully factorized approximationsQuerying the marginals is typically sufficient for real-world applications. See the thread about
<a href="http://social.microsoft.com:80/Forums/en-US/infer.net/thread/b6bb7e7d-ee2c-4096-9b61-380abc105cb8">
Papers in which Infer.net was used.</a> Of course there are cases where the joint assignment is needed but since we have limited development resources it is a question of priority.Sun, 06 May 2012 12:23:16 Z2012-05-06T12:23:16Z
- https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#6cb7e98c-bf87-4cdd-b11a-c39f456400b3https://social.microsoft.com/Forums/en-US/c434657c-fa75-4941-a409-dad62494f25b/fully-factorized-approximations?forum=infer.net#6cb7e98c-bf87-4cdd-b11a-c39f456400b3Ulchuchuhttps://social.microsoft.com:443/profile/ulchuchu/?type=forumFully factorized approximationsThanks, Tom!Sun, 06 May 2012 21:08:11 Z2012-05-06T21:08:11Z