# Bayesian CRR (Migrated from community.research.microsoft.com) • ### Question

• Ali AlKahki posted on 02-24-2011 4:50 AM

I am trying to implement a simple full Bayesian CRF as follwoing

double[] obsImage = new double[] { 0, 0, 1,0, 0, 1, 0, 1, 0, 1 };
bool[] trueImage = new bool[] { true, true, false, true, true, false, true, false, true, false };
int numPixels = 10;
Variable<double> probXEqualX =  Variable.Beta(1, 1);
var y = new Variable<Vector>[numPixels];
var x = new Variable<bool>[numPixels];
Variable<Vector> w = Variable.Random(new VectorGaussian(Vector.Zero(2),PositiveDefiniteMatrix.Identity(2)));
for (int i = 0; i < numPixels; i++)
{
x[i] = Variable.New<bool>();
y[i] = Vector.FromArray(new double[] { 1, obsImage[i] });
double noise = .1;
Variable.ConstrainEqual(x[i]==((Variable.InnerProduct(w, y[i])) > 0),x[i]==(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(w, y[i]), noise) > 0));//, Variable.Bernoulli(probYEqualX));

}
for (int i = 0; i < numPixels - 1; i++)
{

{

Variable.ConstrainEqual(x[i] == x[i + 1], Variable.Bernoulli(probXEqualX));
}
}

InferenceEngine engine = new InferenceEngine();

for (int i = 0; i < numPixels; i++)
{

x[i].ObservedValue = trueImage[i];
}

Console.WriteLine(engine.Infer<VectorGaussian>(w));
Console.WriteLine(engine.Infer<Beta>(probXEqualX));

which gives me :

Compiling model...done.
Iterating:
.........|.........|.........|.........|.........| 50
VectorGaussian(0 0, 1 0)
0 1
Compiling model...done.
Iterating:
.........|.........|.........|.........|.........| 50
Beta(3,8)[mean=0.2727]
Press any key to continue . . .

I can not figure out what is wrong in this code. Any help please ?

thank you

Friday, June 3, 2011 6:20 PM

• minka replied on 02-27-2011 3:48 PM

Try engine.Infer<Bernoulli[]>(TestOut);

Friday, June 3, 2011 6:21 PM

### All replies

• Ali AlKahki replied on 02-24-2011 5:00 AM

in the title it is written CRR and I meant CRF

Friday, June 3, 2011 6:20 PM
• minka replied on 02-24-2011 5:38 AM

This line of code looks weird:

Variable.ConstrainEqual(x[i]==((Variable.InnerProduct(w, y[i])) > 0),x[i]==(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(w, y[i]), noise) > 0));//, Variable.Bernoulli(probYEqualX));

I think you meant something else.  Perhaps this:

Variable.ConstrainEqual(x[i]==(Variable.GaussianFromMeanAndVariance(Variable.InnerProduct(w, y[i]), noise) > 0), Variable.Bernoulli(probYEqualX));

Friday, June 3, 2011 6:20 PM
• Ali AlKahki replied on 02-24-2011 6:26 AM

thank you tom. I tried the line you wrote before and it did not work also. the same problem exists which is I can not learn w vector.

Friday, June 3, 2011 6:21 PM
• Ali AlKahki replied on 02-24-2011 6:52 AM

i tried also to comment the line

Variable.ConstrainEqual(x[i] == x[i + 1], Variable.Bernoulli(probXEqualX));

which removes the dependencies between output nodes and reduce the problem to traditional BPM but modeled with soft constrains but the problem still exists.

Friday, June 3, 2011 6:21 PM
• minka replied on 02-24-2011 7:40 AM

What prior did you put on probYEqualX?  If you used Beta(1,1) then you have a symmetry-breaking issue, because w and -w are both equally likely.  You need to use an asymmetric prior or initializer for probYEqualX.

Friday, June 3, 2011 6:21 PM
• Ali AlKahki replied on 02-27-2011 5:15 AM

Thank you Tom. your Advice worked great with me but now I came to prediction stage and wrote something like the following:

Beta XXPosterior = (engine.Infer<Beta>(probXEqualX));
Beta YXPosterior = (engine.Infer<Beta>(probYEqualX));
VectorGaussian wPosterior = (engine.Infer<VectorGaussian>(w));

VariableArray<bool> TestOut = Variable.Array<bool>(new Range(numPixels)).Named("out");
Range jj = TestOut.Range;
double[] NewImage = new double[] { 0, 0, 1, 0, 0, 1, 0, 1, 0, 1 };
Vector[] TestVector = new Vector[numPixels];
for (int i = 0; i < TestVector.Length; i++)
TestVector[i] = Vector.FromArray(1, NewImage[i]);

VariableArray<Vector> TestInput = Variable.Observed(TestVector, jj);
Variable.ConstrainEqualRandom(TestOut[jj] = (Variable.InnerProduct(Variable.Random(wPosterior), TestInput[jj]) > 0), new Bernoulli(1.0 * YXPosterior.TrueCount / (YXPosterior.TrueCount + YXPosterior.FalseCount)));
for (int i = 0; i < numPixels - 1; i++)
Variable.ConstrainEqualRandom(TestOut[i] == TestOut[i + 1],  new Bernoulli(1.0*XXPosterior.TrueCount/(XXPosterior.TrueCount+ XXPosterior.FalseCount)));
engine.ShowFactorGraph = true;
for (int i = 0; i < numPixels; i++)
Console.WriteLine(engine.Infer<Bernoulli>(TestOut));

and got exception "This class was not built to infer out_vint19_" on the line Console.WriteLine(engine.Infer<Bernoulli>(TestOut));

Friday, June 3, 2011 6:21 PM
• minka replied on 02-27-2011 3:48 PM

Try engine.Infer<Bernoulli[]>(TestOut);

Friday, June 3, 2011 6:21 PM