XOR inference (Migrated from community.research.microsoft.com)<p><b>redstr posted on 12-06-2009 1:36 PM</b></p>
<p>Hello All,</p>
<p>I tried a simple XOR setup with observed result, as follows:</p>
<p> Variable<bool> x = Variable.Bernoulli(0.5), y = Variable.Bernoulli(0.5);<br/>
var xt = x & (~y);<br/>
var yt = (~x) & y;<br/>
var z = yt | xt;<br/>
z.ObservedValue = true;<br/>
InferenceEngine ie = new InferenceEngine();<br/>
Console.WriteLine("x={0}", ie.Infer(x));</p>
<p>I get:</p>
<p>x=Bernoulli(0)</p>
<p>Which is no good, since it really should be Bernoulli(0.5). Gibbs sampler can't infer this, complaining that the model has zero probability. I suppose I'm doing something wrong, but still this indicates some hidden, non-obvious limitations as to
what can be done. Could somebody explain why I get this result and what is the correct setup for such discrete problems?</p>
<p>Also, is it possible to get the value of a deterministic variable? For example if I specify</p>
<p> var t = ~z;</p>
<p>in my example, I can't simply infer it, and there is no obvious way to retrieve its value.</p>
<p>BTW, OpenID login to the forum doesn't work for me. And how do you post colored text from Visual Studio here?</p>
<p>Thanks!</p>حقوق النشر © محفوظة لشركة Microsoft Corporation. جميع الحقوق محفوظة.Fri, 03 Jun 2011 17:27:43 Zffaa6ffd-498e-41e8-8787-011b16be348f- https://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#ffaa6ffd-498e-41e8-8787-011b16be348fhttps://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#ffaa6ffd-498e-41e8-8787-011b16be348fMicrosoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumXOR inference (Migrated from community.research.microsoft.com)<p><b>redstr posted on 12-06-2009 1:36 PM</b></p>
<p>Hello All,</p>
<p>I tried a simple XOR setup with observed result, as follows:</p>
<p> Variable<bool> x = Variable.Bernoulli(0.5), y = Variable.Bernoulli(0.5);<br/>
var xt = x & (~y);<br/>
var yt = (~x) & y;<br/>
var z = yt | xt;<br/>
z.ObservedValue = true;<br/>
InferenceEngine ie = new InferenceEngine();<br/>
Console.WriteLine("x={0}", ie.Infer(x));</p>
<p>I get:</p>
<p>x=Bernoulli(0)</p>
<p>Which is no good, since it really should be Bernoulli(0.5). Gibbs sampler can't infer this, complaining that the model has zero probability. I suppose I'm doing something wrong, but still this indicates some hidden, non-obvious limitations as to
what can be done. Could somebody explain why I get this result and what is the correct setup for such discrete problems?</p>
<p>Also, is it possible to get the value of a deterministic variable? For example if I specify</p>
<p> var t = ~z;</p>
<p>in my example, I can't simply infer it, and there is no obvious way to retrieve its value.</p>
<p>BTW, OpenID login to the forum doesn't work for me. And how do you post colored text from Visual Studio here?</p>
<p>Thanks!</p>Fri, 03 Jun 2011 17:27:20 Z2011-06-03T17:27:20Z
- https://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#422dc28c-b813-41cd-9074-5ebed04153d3https://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#422dc28c-b813-41cd-9074-5ebed04153d3Microsoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumXOR inference (Migrated from community.research.microsoft.com)<p><b>minka replied on 12-07-2009 3:31 PM</b></p>
<p>Yes, the behavior of approximate inference can often be non-obvious! In this case, the Bernoulli(0) comes from the way you have defined z. Infer.NET performs inference by constructing a factor graph from the exact sets of operations used
in your program. In your case, the operations are 'not' (twice), 'and' (twice) and 'or'. Because x is used twice in defining z, the resulting factor graph has a loop. Also, the loop consists entirely
of deterministic operations. This is a particularly bad case for all of the inference algorithms in Infer.NET. VMP and Gibbs sampling cannot run at all in this situation and throw errors (admittedly, the error from Gibbs could be more helpful).
Expectation Propagation (the default algorithm) can be executed in this situation, but the result is far from the exact marginal.</p>
<p>For this particular problem, there is a simple fix. You can change your definition of z to involve fewer operations. Here is an equivalent definition of z using one operation:</p>
<p>z = (x != y);</p>
<p>Using this definition gives the exact marginal. So the moral of the story is to simplify your programs as much as possible. Not only will they run faster, but also give more accurate results.</p>
<p>Your other question about inferring t = ~z is not possible in the current release of Infer.NET. Basically Infer.NET assumes that you want to infer a random variable. For a deterministic variable, you don't really need
Infer.NET since C# can already compute the value. However, we could add this functionality in the future.</p>Fri, 03 Jun 2011 17:27:30 Z2011-06-03T17:27:30Z
- https://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#331778d9-54ee-4499-95d6-cb2015b6f4ffhttps://social.microsoft.com/Forums/ar-SA/ffaa6ffd-498e-41e8-8787-011b16be348f/xor-inference-migrated-from-communityresearchmicrosoftcom?forum=infer.net#331778d9-54ee-4499-95d6-cb2015b6f4ffMicrosoft Researchhttps://social.microsoft.com:443/profile/microsoft%20research/?type=forumXOR inference (Migrated from community.research.microsoft.com)<p><b>redstr replied on 12-07-2009 4:29 PM</b></p>
<p>Thank you, that was helpful! Your suggestion worked, except that the Gibbs Sampling algorithm still gives me Bernoulli(0) or Bernoulli(1) depending on its mood.</p>
<p>The loop with deterministic variables was the root of the problem in this case, you say. Is there a more or less systematic description of what kind of models may cause problems for these approximate inference algorithms?</p>
<p>Inferring deterministic variables can make sense in a scenario where there may be observations assigned to different variables. Then some of the variables may become deterministic and thus "hidden". It was the case with the XOR problem, where I
put a statement to print values of all variables at the end.</p>Fri, 03 Jun 2011 17:27:34 Z2011-06-03T17:27:34Z