Fully factorized approximations

# Fully factorized approximations

• Wednesday, May 02, 2012 12:18 PM

Dear All,

I'm pretty new to Infer.NET and graphical models in general so excuse me if the question might be trivial or if I got it wrong to start with. In the end what I would like to do is learn a linear-chain CRF for some sort of sequence labeling task given data.

Let's assume I have T outcome variables Y_i, i=1..T and for each Y_i an input variable X_i which I observe. What I am interested in is the MAP assignment of Y given X, so argmax(Y) P(Y|X). From what I understood of Infer.NET so far it makes use of a so-called "fully factorized approximation" which means I can use the inference engine to query each P(Y_i | X_i) and the joint CPD will then be the product of the CPDs of the individual variables. Or I could simply use each individual CPD to chose my MAP assignment for that position and repeat that for the whole sequence. But doesn't that imply that all Y_i's are independent of each other which is exactly what I am trying to prevent in the first place by using a graphical model with edges between adjacent Y_i's? This has caused me some headache so any hint would be appreciated.

Further, in the end I would like to make advantage of the fact that I model my problem as a discriminative model, not generative, meaning that I might want to include rich features where each Y_i not only depends on its X_i, but potentially on all or a large subset of members of X. Is there an explicit way of modeling that in Infer.NET or is it simply a question of not querying P(X), i.e. only asking questions about an unnormalized measure P'(Y|X)? How do I tell Infer.NET that I only would like the unnormalized measure? I hope that makes sense.

Roland

### All Replies

• Friday, May 04, 2012 5:04 PM

Infer.NET only allows inferring the marginal distribution of a single variable at a time and therefore it cannot compute a joint MAP assignment.
• Saturday, May 05, 2012 9:40 AM

Well if that was a design choice, is querying the marginals typically sufficient for most real-world applications? I've been dealing mostly with HMMs in the past and I realize that working on the marginals only to determine an optimal state path can give similar results to say the Viterbi path, but still there are cases where the joint assignment is what you are really looking for. Any thoughts on that?

• Sunday, May 06, 2012 12:23 PM

Querying the marginals is typically sufficient for real-world applications.  See the thread about Papers in which Infer.net was used.  Of course there are cases where the joint assignment is needed but since we have limited development resources it is a question of priority.
• Sunday, May 06, 2012 9:08 PM

Thanks, Tom!