locked
Errors in independent variables - Bayesian Regression example RRS feed

  • Question

  • Hi. I was wondering how can this error in independent variables modeled in Bayesian Regression example (as explained here: http://en.wikipedia.org/wiki/Errors-in-variables_models).

    Is there any way to define x (which is currently observed) as a noisy variable in infer.net?

    public static void runBayesianRegressionArray()
            {
                Console.WriteLine("Bayesian regression example : using variableArray ");
                Random rnd = new Random();
                
                int numData = 3;
                int numFeature = 3;
                double[][] input = new double[numData][];
                for (int i = 0; i < input.Length; i++)
                {
                    input[i] = new double[numFeature + 1];
                    for (int j = 0; j < input[i].Length - 1; j++)
                    {
                        input[i][j] = rnd.Next(1, 100);
                    }
                    input[i][input[i].Length - 1] = 1;
                }
    
                Range dataRange = new Range(input.Length).Named("dataRange"); ;
                Range featureRange = new Range(numFeature + 1).Named("featureRange");
                VariableArray<VariableArray<double>, double[][]> x = Variable.Constant(input, dataRange, featureRange);
                VariableArray<double> w = Variable.Array<double>(featureRange).Named("w");
                using (Variable.ForEach(featureRange))
                {
                    w[featureRange] = Variable.GaussianFromMeanAndVariance(1, 10000000);
                }
    
                VariableArray<double> y = Variable.Array<double>(dataRange).Named("y");
                VariableArray<VariableArray<double>, double[][]> tmpMat = Variable.Array(Variable.Array<double>(featureRange), dataRange).Named("tmpMat");
    
                using (Variable.ForEach(dataRange)) tmpMat[dataRange][featureRange].SetTo(w[featureRange] * x[dataRange][featureRange]);
                using (Variable.ForEach(dataRange))
                {
                    y[dataRange] .SetTo( Variable.GaussianFromMeanAndVariance(Variable<double>.Sum(tmpMat[dataRange]).Named("sum"), 1.0));
                }
    
                double[] yObserved = new double[numData];
                for (int i = 0; i < numData; i++)
                {
                    yObserved[i] = 5 * input[i][0] + 17 * input[i][1] + Gaussian.FromMeanAndVariance(0, 1).Sample();
                }
                
                y.ObservedValue = yObserved;
                dataRange.AddAttribute(new Sequential());
                InferenceEngine engine = new InferenceEngine(new ExpectationPropagation());
                engine.NumberOfIterations = 50;
    
    			Gaussian[] postW = engine.Infer<Gaussian[]>(w);
                Gaussian[][] postTmp = engine.Infer<Gaussian[][]>(tmpMat);
                Gaussian[][] postX = engine.Infer<Gaussian[][]>(x);
    
                for (int i = 0; i < postW.Length; i++)
                {
                    Console.WriteLine(string.Format("postW[{0}]={1}", i, postW[i]));
                }            
    
                Console.ReadLine();
            }

    Monday, January 26, 2015 9:06 PM

All replies

  • Sure, just give it a distribution instead of making it constant.
    Tuesday, January 27, 2015 7:36 PM
    Owner
  • Thank you Tom. You mean variables with certain distributions? Then I guess the distribution is limited to what can be multiplied by w (like Gaussian - becuase here w is also a Variable.Gaussian) in infer.NET.

    If I'm not wrong, I think in this case each variable in x, instead of being observed, should be for example a variable defined as Variable.GaussianFromMeanAndVariance(m, v) , which m is the value that we used as observed in the first example (noiseless x).

    But another issue is how to initialize the different priors for this jagged array. If it is defined on dataRange and featureRange, how can it be defined on different priors?

    And won't defining x as a variable, result in x values to be learned also (which is not of interest)?

    • Edited by Capli19 Wednesday, January 28, 2015 2:29 PM
    Wednesday, January 28, 2015 2:12 PM