locked
problem using "Variable.If(variable[variableRange])" inside a loop RRS feed

  • Question

  • Hello. I have a model. The input is a vector of attributes. Inner product of this vector by a weight vecotr, gives the output (one scalar). The aim is to perform an feature selection on input features. The weights are chosen by a Bernoulli distributed variable (for each feature) from a spike or slab distribution.

    The problem is that on my loop over data points (dataRange), I can't apply a Variable.If() on my gamma variable which itself is a variableArray defined on featureRange.

    Thanks.

    The code is as below:

    using System; using System.Collections.Generic; using System.Linq; using System.Text; using MicrosoftResearch.Infer.Models; using MicrosoftResearch.Infer; using MicrosoftResearch.Infer.Distributions; using MicrosoftResearch.Infer.Maths; namespace sparseModelTest { class Program { static int dataSize = 100, dataRange = 20; static int featureSize = 10; static void Main(string[] args) { double Rho = 0.5; Range featureRange = new Range(featureSize).Named("featureRange"); Range dataRange = new Range(dataSize).Named("dataRange"); VariableArray<VariableArray<double>, double[][]> w = Variable.Array(Variable.Array<double>(featureRange),dataRange); VariableArray<Gaussian> spike = Variable.Array<Gaussian>(featureRange).Named("spike"); VariableArray<Gaussian> slab = Variable.Array<Gaussian>(featureRange).Named("slab"); VariableArray<double> output = Variable.Array<double>(dataRange).Named("output"); VariableArray<VariableArray<double>, double[][]> input = Variable.Array(Variable.Array<double>(featureRange), dataRange); VariableArray<bool> gamma = Variable.Array<bool>(featureRange).Named("gamma"); gamma[featureRange] = Variable.Bernoulli(Rho).ForEach(featureRange); using(Variable.ForEach(featureRange)) spike[featureRange] = Gaussian.FromMeanAndVariance(0, 0); using (Variable.ForEach(featureRange)) slab[featureRange] = Gaussian.FromMeanAndVariance(0, 1); using (Variable.ForEach(featureRange)) { using (Variable.If(gamma[featureRange])) { // spike is the true distribution for w w[dataRange][featureRange] = Variable<double>.Random(spike[featureRange]).ForEach(dataRange); } using (Variable.IfNot(gamma[featureRange])) { // slab is the true distribution for w w[dataRange][featureRange] = Variable<double>.Random(slab[featureRange]).ForEach(dataRange); } } using(Variable.ForEach(dataRange)) { var tmp = Variable<double>.Array(featureRange); tmp[featureRange] = w[dataRange][featureRange] * input[dataRange][featureRange]; output[dataRange] = Variable<double>.Sum(tmp); }//dataRange GenerateData(100); output.ObservedValue = outputData; input.ObservedValue = inputData; InferenceEngine ie = new InferenceEngine(); var gammaPosterior = ie.Infer<Bernoulli[]>(gamma); var spikePosterior = ie.Infer<Gaussian[]>(spike); var slabPosterior = ie.Infer<Gaussian[]>(slab); for (int i = 0; i < gammaPosterior.Length; i++) { Console.WriteLine("gammaPosterior" + "(" + i + ")=" + gammaPosterior[i]); Console.WriteLine("spikePosterior" + "(" + i + ")=" + spikePosterior[i]); Console.WriteLine("slabPosterior" + "(" + i + ")=" + slabPosterior[i]); } } public static void writeArr(Gaussian[] array, string name/*, StreamWriter resultFile*/) { // Console.WriteLine(); for (int i = 0; i < array.Length; i++) { Console.WriteLine(name + "(" + i + ")=" + array[i]); } } public static void writeArr(Bernoulli[] array, string name/*, StreamWriter resultFile*/) { // Console.WriteLine(); for (int i = 0; i < array.Length; i++) { Console.WriteLine(name + "(" + i + ")=" + array[i]); } } static double[][] inputData = new double[100][]; static double[] outputData = new double[100]; public static void GenerateData(int nData) { Gaussian spikeDist = Gaussian.FromMeanAndPrecision(1, 0.3); Gaussian spikeDist2 = Gaussian.FromMeanAndPrecision(2, 0.3); Gaussian spikeDist3 = Gaussian.FromMeanAndPrecision(3, 0.3); Gaussian slabDist = Gaussian.FromMeanAndPrecision(0,0); inputData = new double[nData][]; outputData = new double[nData]; Rand.Restart(12347); for (int i = 0; i < dataSize; i++) { inputData[i] = new double[featureSize]; for (int j = 0; j < featureSize; j++) { inputData[i][j] = Rand.Int(dataRange); }

    // the only features attended are feature at position 0, 1 and 2.

    // the other features have no effect on ouput outputData[i] = spikeDist.Sample() * inputData[i][0] + spikeDist2.Sample() * inputData[i][1] + spikeDist3.Sample() * inputData[i][2]; } } } }




    • Edited by Capli19 Monday, October 27, 2014 10:10 AM
    Sunday, October 26, 2014 10:45 PM

Answers

  • This is a bug.  Thanks for reporting this.  You can work around it by using a ForEach block for dataRange:

                using (Variable.ForEach(featureRange))
                {
                    using (Variable.If(gamma[featureRange]))
                    {
                        using (Variable.ForEach(dataRange))
                        {
                            // spike is the true distribution for w
                            w[dataRange][featureRange] = Variable<double>.Random(spike[featureRange]);
                        }
                    }
                    using (Variable.IfNot(gamma[featureRange]))
                    {
                        using (Variable.ForEach(dataRange))
                        {
                            // slab is the true distribution for w
                            w[dataRange][featureRange] = Variable<double>.Random(slab[featureRange]);
                        }
                    }
                }
    

    By the way, this code at the end won't work:

                var spikePosterior = ie.Infer<Gaussian[]>(spike);
                var slabPosterior = ie.Infer<Gaussian[]>(slab);
    

    Since spike and slab have type VariableArray<Gaussian[]>, you can't infer them as type Gaussian[].  You can only infer them as a distribution over Gaussian[].

    • Marked as answer by Capli19 Monday, October 27, 2014 9:49 PM
    Monday, October 27, 2014 1:05 PM
    Owner
  • ie.Infer<PointMass<Gaussian[]>>(spike);

    should work.
    • Marked as answer by Capli19 Tuesday, October 28, 2014 7:13 PM
    Tuesday, October 28, 2014 6:16 PM
    Owner

All replies

  • This is a bug.  Thanks for reporting this.  You can work around it by using a ForEach block for dataRange:

                using (Variable.ForEach(featureRange))
                {
                    using (Variable.If(gamma[featureRange]))
                    {
                        using (Variable.ForEach(dataRange))
                        {
                            // spike is the true distribution for w
                            w[dataRange][featureRange] = Variable<double>.Random(spike[featureRange]);
                        }
                    }
                    using (Variable.IfNot(gamma[featureRange]))
                    {
                        using (Variable.ForEach(dataRange))
                        {
                            // slab is the true distribution for w
                            w[dataRange][featureRange] = Variable<double>.Random(slab[featureRange]);
                        }
                    }
                }
    

    By the way, this code at the end won't work:

                var spikePosterior = ie.Infer<Gaussian[]>(spike);
                var slabPosterior = ie.Infer<Gaussian[]>(slab);
    

    Since spike and slab have type VariableArray<Gaussian[]>, you can't infer them as type Gaussian[].  You can only infer them as a distribution over Gaussian[].

    • Marked as answer by Capli19 Monday, October 27, 2014 9:49 PM
    Monday, October 27, 2014 1:05 PM
    Owner
  • Oh I'm glad it was useful :) Thanks for your reply.
    Monday, October 27, 2014 9:48 PM
  • Tom, I understood your point about inference of Gaussian type variables that cannot be Gaussian itself. but I don't know what type can I use here? Can you give a clue?
    Tuesday, October 28, 2014 4:33 PM
  • ie.Infer<PointMass<Gaussian[]>>(spike);

    should work.
    • Marked as answer by Capli19 Tuesday, October 28, 2014 7:13 PM
    Tuesday, October 28, 2014 6:16 PM
    Owner
  • This bug is now fixed in Infer.NET 2.6.
    Saturday, November 29, 2014 6:07 PM
    Owner