How to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>rivuletish posted on 12-06-2010 8:44 AM</b></p>
<p></p>
<p>suppose the model is </p>
<p>p(x)p(y|x)</p>
<p>x is a Gaussian variable. However, in practice x is discrete, say, it is discrete pixel position.</p>
<p>y is the observed variable, say it is an image of an apple, p(y|x) would be the probability of seeing the apple at pixel location x. </p>
<p> </p>
<p>I don't know the form of p(y|x). All I have is a classifier to evaluate the probability of seeing the apple at position x, that is p(y|x) at disrete x.</p>
<p> </p>
<p>How to model this case? If x is modeled as discrete variable, in the case of large image, the sample size of x is large. Then it is computationaly inefficient.</p>
<p></p>
<p>BTW, can I use Infer.Net to do MAP inference or selecting the most probable joint configuration?</p>© 2009 Microsoft Corporation. All rights reserved.Fri, 03 Jun 2011 18:09:45 Z6a1225ff-337f-42bf-8789-3e2f97ff27e1- https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#6a1225ff-337f-42bf-8789-3e2f97ff27e1https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#6a1225ff-337f-42bf-8789-3e2f97ff27e1Microsoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumHow to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>rivuletish posted on 12-06-2010 8:44 AM</b></p>
<p></p>
<p>suppose the model is </p>
<p>p(x)p(y|x)</p>
<p>x is a Gaussian variable. However, in practice x is discrete, say, it is discrete pixel position.</p>
<p>y is the observed variable, say it is an image of an apple, p(y|x) would be the probability of seeing the apple at pixel location x. </p>
<p> </p>
<p>I don't know the form of p(y|x). All I have is a classifier to evaluate the probability of seeing the apple at position x, that is p(y|x) at disrete x.</p>
<p> </p>
<p>How to model this case? If x is modeled as discrete variable, in the case of large image, the sample size of x is large. Then it is computationaly inefficient.</p>
<p></p>
<p>BTW, can I use Infer.Net to do MAP inference or selecting the most probable joint configuration?</p>Fri, 03 Jun 2011 18:09:19 Z2011-06-03T18:09:19Z
- https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#fb7e9df6-f13c-40ce-8190-b143615b23f0https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#fb7e9df6-f13c-40ce-8190-b143615b23f0Microsoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumHow to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>jwinn replied on 12-07-2010 11:26 AM</b></p>
<p>I'm not sure I quite understand your situation - do you have an existing classifier giving p(y|x) or are you trying to build one in Infer.NET?</p>
<p>If x is a discrete variable with large cardinality, you may be able to handle this efficiently in Infer.NET by using
<a href="http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Using%20sparse%20messages.aspx" title="sparse messages">
sparse messages</a>.</p>
<p>For MAP inference, Infer.NET has very limited support for max product belief propagation - otherwise you can just pick the mode of the returned marginals. There is no support for finding the most probable joint configuration, although I can think of
ways you could do it using Infer.NET as a subroutine. Alternatively, Gibbs sampling will returns a (joint) sample which may be sufficient for your application. Our philosophy is that it is generally preferable to do inference/sampling rather than
optimisation since the inherent uncertainty in the solution is retained. </p>
<p>Best</p>
<p>John W.</p>Fri, 03 Jun 2011 18:09:21 Z2011-06-03T18:09:21Z
- https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#6d583c5d-144a-40de-bb01-ca4dd484e0afhttps://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#6d583c5d-144a-40de-bb01-ca4dd484e0afMicrosoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumHow to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>rivuletish replied on 12-07-2010 8:19 PM</b></p>
<p>Sorry I didn't make myself clear.</p>
<p>My question comes from part model of objects. Let us assume the object consists of two related parts, A and B. And x and y are their positions. y~Gaussian(y| x, Sigma). We want to recognize the two parts from an image of the object.</p>
<p>The model would be p(x) p(y|x) p(A|x) p(B|y). The factor graph is B------[p(B|y)]----y------[Gau(y|x,Sigma)]---x---[p(A|x)]----A (p(x) is omitted in the graph, and [. ] represent factors)</p>
<p>Because modelling p(A|x) and p(B|y) is hard, we can use existing part classifiers to calculate them given that the image is observed. These p(A|x) and p(B|y) are accurate, but they work reasonably in practice.</p>
<p>So the message sent from B to y is known at discrete y. But Gau(y|x,Sigma) is a continuous function. How to calculate the message from y to x in Infer.Net.</p>
<p> </p>Fri, 03 Jun 2011 18:09:28 Z2011-06-03T18:09:28Z
- https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#a72763cf-ac9f-4c31-aac4-274d0821901chttps://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#a72763cf-ac9f-4c31-aac4-274d0821901cMicrosoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumHow to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>jwinn replied on 12-09-2010 4:17 AM</b></p>
<p>Your question is partly an Infer.NET question and partly a research question.</p>
<p>The naive answer is that you can use a giant discrete factor that simply evaluates the Gaussian function for each discretised value of x and y. In Infer.NET, this could be implemented as a
<a href="http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/How%20to%20add%20a%20new%20factor%20and%20message%20operators.aspx" title="custom factor">
custom factor</a>. However, such a factor would take O(N^2) to evaluate a message, where N is the cardinality of x and y. More efficient methods of evaluation of such factors have been proposed - see, for example:<br/>
<strong> Efficient Belief Propagation for Early Vision<br/>
</strong> Pedro F. Felzenszwalb and Daniel P. Huttenlocher<br/>
International Journal of Computer Vision, Vol. 70, No. 1, October 2006<br/>
which suggests distance transform methods for max product belief propagation and also mentions sum-product options.<br/>
There is example code implementing the max product methods in this paper in the supplied factor source code file Undirected.cs, as described on
<a href="http://research.microsoft.com/en-us/um/cambridge/projects/infernet/docs/Undirected%20factors.aspx" title="this page">
this page</a>.</p>
<p>Another alternative would be to transform your discretised representation into a Gaussian (e.g. by moment matching), but then you would lose desirable properties of the discrete distribution such as the ability to capture multiple modes.</p>
<p>In general, I would say that finding a really good solution to this problem is still an open research question.</p>
<p>Best</p>
<p>John W.</p>Fri, 03 Jun 2011 18:09:31 Z2011-06-03T18:09:31Z
- https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#648598dc-180d-4200-9878-c248fbf838a3https://social.microsoft.com/Forums/en-US/6a1225ff-337f-42bf-8789-3e2f97ff27e1/how-to-model-a-continuous-variable-evaluated-at-discrete-values-migrated-from?forum=infer.net#648598dc-180d-4200-9878-c248fbf838a3Microsoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumHow to model a continuous variable evaluated at discrete values? (Migrated from community.research.microsoft.com)<p><b>rivuletish replied on 12-11-2010 8:04 AM</b></p>
<p>Hi John, </p>
<p>Thank you very much for your reply. You mentioned that the discretised representation can be fitted as a Gaussian. Maybe mixture of Gaussian distribution would be better.
</p>
<p>BTW, I read the following paper. In the case y~Gaussian(x,Sigma), "Message passing can be performed exhaustively and efﬁciently with convolutions." I am not sure what general conditions are required for the convolution trick to be applied. Maybe
Infer.Net can take this into account. </p>
<p>Ramanan, D. "<b>Learning to Parse Images of Articulated Bodies.</b>" <i>Neural Info. Proc. Systems</i>(NIPS) To appear. Dec 2006.</p>
<p>Have a nice weekend.</p>
<p>Kevn</p>Fri, 03 Jun 2011 18:09:44 Z2011-06-03T18:09:44Z