Exception in BPM: The matrix is not positive definite. (Migrated from community.research.microsoft.com)

• Question

• shankarooni posted on 01-24-2010 3:46 AM

I am following the BPM example in tutorial  (to classify willbuy based on age and salary)

My version of the problem has about 100 features and they are all boolean (expressed as 0 or 1).

On some dataset it works fine, but on some it throws a PositiveDefiniteMatrixException and returns this error:

The matrix is not positive definite.

at line:

VectorGaussian wPosterior = engine.Infer<VectorGaussian>(w);

This is one array of my  data:

0000000000000000000000000000000000000000000000000000000100

all arrays are similar the  data 2Darray in:

public void BayesPointMachine(double[][] data, Variable<Vector> w, VariableArray<bool> y )

The algorithm i am using is EP

What went wrong?

Friday, June 3, 2011 5:32 PM

• shankarooni replied on 01-27-2010 7:35 AM

Thank you John! That  solved the problem!

No need to apologize, we are still in Beta :)

I can wait to see the final release. Its great work.

Thanks

Friday, June 3, 2011 5:33 PM

All replies

• shankarooni replied on 01-24-2010 5:52 AM

I just read about  Positive definite matrix in wikipeida...I see it is not related to infer.net, just removed the zeros and replaced them with 1...and 1 with 2... and error is gone...

Thanks...

Friday, June 3, 2011 5:33 PM
• shankarooni replied on 01-24-2010 6:06 AM

The error is back on different data sample

Friday, June 3, 2011 5:33 PM
• jwinn replied on 01-25-2010 4:41 AM

Apologies, the tutorial is somewhat simplified and assumes the data is linearly separable - if this is not the case, you will get the PositiveDefiniteMatrixException.  To fix the problem, you need to add noise to the model as follows:

To add noise, rather than doing IsPositive directly on the result of the inner product, instead make the inner product the mean of a Gaussian whose variance is some noise parameter and find out if that ‘IsPositive’. So:

y[j]=Variable.IsPositive(Variable.GaussianFromMeanAndVariance
(Variable.InnerProduct(w, x[j]), noise))

The variable 'noise' can be a constant double (defining the variance of the noise), or you could give it a Gamma prior and learn it. A particular noise level will set the ‘tolerance’ of the classifier to examples which are not linearly separable (i.e. training examples which are incorrectly classified) and will also affect the confidence of the predictions. If you’re not going to learn it, then you should tune it to get the best results on your problem.

The tutorial will be updated to include noise on the next release. Apologies again for this omission,

Best,
John W.

Friday, June 3, 2011 5:33 PM
• jwinn replied on 01-25-2010 4:43 AM

I should add that the Image Classifier example (source included in the Infer.NET install) is an example of a BPM with a noise term that you could use for reference.

Friday, June 3, 2011 5:33 PM
• shankarooni replied on 01-27-2010 7:35 AM

Thank you John! That  solved the problem!

No need to apologize, we are still in Beta :)

I can wait to see the final release. Its great work.

Thanks

Friday, June 3, 2011 5:33 PM