locked
How to use non-Conjugate Prior distribution in the modeling RRS feed

  • Question

  • Hello, infer.net people,

             Is it possible to  use non-Conjugate Prior distribution in the Infer.net?  For example, I want to model the mean of a Gaussian using Gamma distribution in the below example, and it shows compile fail no matter what algorithm is used.  

    Thank you very much.

               double[] data = new double[100];
                double shape = 2;
                double rate = 0.5;
                for (int i = 0; i < data.Length; i++) data[i] = Rand.Gamma(shape) / rate+Rand.Normal(0,0.01);
                var rateRandV = Variable.Random<double>(Gamma.Uniform());// Variable.GammaFromMeanAndVariance(rate * 2, 10);
                for (int i = 0; i < data.Length; i++)
                {
                    Variable<double> x_t = Variable.GammaFromShapeAndRate(shape, rateRandV);
                    Variable<double> x = Variable.GaussianFromMeanAndVariance(x_t, 0.0001);
                    x.ObservedValue = data[i];
                }
                InferenceEngine engine = new InferenceEngine(new VariationalMessagePassing());
                Console.WriteLine("rate=" + engine.Infer(rateRandV));

    Tuesday, October 28, 2014 3:35 PM

Answers

  • If you just want a distribution that is positive with a long tail, you can use a Gaussian distribution for the mean together with ConstrainPositive.
    • Marked as answer by Zhizhuo Monday, November 3, 2014 5:46 PM
    Wednesday, October 29, 2014 3:26 PM
    Owner

All replies

  • You can use non-conjugate priors in cases where they are implemented.  In this case, a Gaussian whose mean is Gamma-distributed has not been implemented in the framework yet.  There is a mechanism described in the user guide for extending the framework.  Is there a particular reason why you need this?
    Tuesday, October 28, 2014 6:50 PM
    Owner
  • Hi, Tom,

    Thank you very much again. The reason I need that is because my real data looks like that way, the distribution generally follow gamma distribution (positive + long tail), but with gaussain noise for each point.

    I will look at the user guide to see how to implement that.

    Thanks again. 

    Wednesday, October 29, 2014 2:56 PM
  • If you just want a distribution that is positive with a long tail, you can use a Gaussian distribution for the mean together with ConstrainPositive.
    • Marked as answer by Zhizhuo Monday, November 3, 2014 5:46 PM
    Wednesday, October 29, 2014 3:26 PM
    Owner
  • Hi Tom,

    Is it at all possible to learn a Gamma distribution directly, and if so, what distribution do the prior values have to be, what factor must be used, and what algorithm supports it?

    I've tried to peruse the factor manager, and tried GammaFromShapeAndScale, GammaFromShapeAndRate, and GammaFromMeanAndVariance with different combinations of Gamma-distributed prior parameters, but haven't been able to get anything to run. GammaFromShapeAndRate gets the furthest - it compiles with VMP, but throws warnings of "Experimental" band during the compilation process and eventually Gamma(NaN, NaN) during inference.

    Thanks,

    Andrew

    Monday, December 29, 2014 6:20 AM
  • Andrew, please start a new thread for this question and provide more details about what model you used with VMP.
    Monday, December 29, 2014 4:19 PM
    Owner
  • Sure. I've made a pretty detailed post in this thread.
    Monday, December 29, 2014 7:46 PM