locked
Problem with algorithm convergence RRS feed

  • Question

  • Hi everybody,

    I have a network consisting of N level of nodes.The weights (Wij) is the same in all levels. So I have just one Wij matrix to be learned by observing data from this N level. I faced different problems in running this model.

    The first was a problem with mean and variance values selected for variables. I got an error:

    ** The distribution is improper (Gaussian(m/v=-6.115e+13, 1/v=-5.498e+11)). Cannot compute expectations.

    Then I thought the variance values has some problem here. I changed my variable distributions from "Variable.GaussianFromMeanAndVariance(1, 0.002)"  to "Variable.GaussianFromMeanAndVariance(0,1)" just to check what will happen (the first one was not logical with variance of 0.002 in fact). The error message I get is this then:

    I get this warning:   [922] GaussianProductOp.ProductAverageConditional(vdouble223_B[geneRange], beta_uses_F[geneRange][13], vdouble222_F[geneRange]) has quality band Experimental which is less than t
    he recommended quality band (Preview)

    And this errror finally after some iterations: Not converging for n=0,x=NaN

    Then I changed it to "Variable.GaussianFromMeanAndPrecision(0,1)".

    Again I get this warning:   [931] GaussianProductOp.ProductAverageConditional(vdouble4371_B[geneRange], alpha_uses_F[geneRange][310], timeSeriesModel_item310_uses_F[1][geneRange]) has quality band Experimen
    tal which is less than the recommended quality band (Preview)

    And this error: The distribution is improper (Gaussian(m/v=-50, 1/v=-2500)). Cannot compute expectations.

    Now my question is that, when I don't have a good estimate of my priors how can I handle these errors? Should I use "Variable.GaussianFromMeanAndPrecision( .... )" and not "Variable.GaussianFromMeanAndVariance(...)" ?

    Thanks a lot in advance.

    == my code: ** I have a loop for batch learning which is removed from this code.

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using MicrosoftResearch.Infer;
    using MicrosoftResearch.Infer.Distributions;
    using MicrosoftResearch.Infer.Maths;
    using MicrosoftResearch.Infer.Models;
    using MicrosoftResearch.Infer.Factors;
    using System.IO;
    
    namespace InferGeneNet
    {
        public class PertNetModel
        {
            public InferenceEngine inferenceEngine;
    
    
            protected VariableArray<double> alpha;
            protected VariableArray<double> beta;
            public VariableArray<VariableArray<double>, double[][]> wMean;
            public VariableArray<VariableArray<double>, double[][]> wVariance;
            public VariableArray<VariableArray<double>, double[][]> w;
            public VariableArray<VariableArray<double>, double[][]> timeSeriesData;
            public Variable<int> NumGenes;
            public Variable<int> NumLayers;
    
            protected VariableArray<Gaussian> alphaPrior;
            protected VariableArray<Gaussian> betaPrior;
            protected VariableArray<VariableArray<Gaussian>, Gaussian[][]> wMeanPrior;
            protected VariableArray<VariableArray<Gamma>, Gamma[][]> wVariancePrior;
    
    
            
    
            public virtual void InferModel( )
            {
                var data = DataReader.ReadData(@"W:\data\abidata\MD_MCF7_main.csv", 13);
                Tuple<string[], int[]> information = data.Item1;
                double[][] dataMatrix = data.Item2;//trainingData;//data.Item2;
                string[] perturbations = information.Item1;
                int[] timepoints = information.Item2;
                int numGenes = dataMatrix[0].Length;
    
                NumGenes = Variable.New<int>();
                NumGenes.ObservedValue = numGenes;
    
                NumLayers = Variable.New<int>();
                NumLayers.ObservedValue = dataMatrix.Length;
    
                Range geneRange = new Range(NumGenes).Named("geneRange");
                Range geneWeightRange = new Range(NumGenes - 1).Named("geneWeightRange");
                Range layerRange = new Range(NumLayers);
    
                wVariance = Variable.Array(Variable.Array<double>(geneWeightRange), geneRange).Named("wVariance");
                wVariance[geneRange][geneWeightRange] = Variable.GammaFromShapeAndRate(1.0, 1.0).ForEach(geneRange, geneWeightRange);
    
                timeSeriesData = Variable.Array(Variable.Array<double>(geneRange), layerRange).Named("timeSeriesModel");
                timeSeriesData[0][geneRange] = Variable.GaussianFromMeanAndPrecision(0,1).ForEach(geneRange);
    
                var indicesArray = Variable.Array(Variable.Array<int>(geneWeightRange), geneRange).Named("indicesArray");
                            
                int[][] indices = new int[NumGenes.ObservedValue][];
                for (int i = 0; i < NumGenes.ObservedValue; i++)
                {
                    indices[i] = new int[NumGenes.ObservedValue - 1];
                    int j = 0, ind = 0;
                    while (j < NumGenes.ObservedValue)
                    {
                        if (i == j) { j++; continue; }
                        indices[i][ind] = j;
                        j++;
                        ind++;
                    }
                }
               // for (int i = 0; i < NumGenes.ObservedValue; i++) for (int j = 0; j < NumGenes.ObservedValue - 1; j++) { Console.WriteLine(indices[i][j]); }
                indicesArray.ObservedValue = indices;
    
                alpha = Variable.Array<double>(geneRange).Named("alpha");
                alpha[geneRange] = Variable.GaussianFromMeanAndPrecision(0,1).ForEach(geneRange);
                beta = Variable.Array<double>(geneRange).Named("beta");
                beta[geneRange] = Variable.GaussianFromMeanAndPrecision(0,1).ForEach(geneRange);
                
    
                w = Variable.Array(Variable.Array<double>(geneWeightRange), geneRange).Named("w");
                w[geneRange][geneWeightRange] = Variable.GaussianFromMeanAndPrecision(0, wVariance[geneRange][geneWeightRange]);
                
    
                VariableArray<double> genesubarray = Variable.Array<double>(geneWeightRange).Named("genesubarray");
    
                for (int layer = 0; layer < NumLayers.ObservedValue - 1; layer++ )
                {
                    using (ForEachBlock firstBlock = Variable.ForEach(geneRange))
                    {
                        Variable<double> selfEffect = -(alpha[geneRange] * timeSeriesData[layer][geneRange]);
                        VariableArray<double> weightSum = Variable.Array<double>(geneWeightRange);
                        Console.WriteLine("summing " + geneRange + " + " + geneWeightRange);
                        genesubarray = Variable.Subarray(timeSeriesData[layer], indicesArray[geneRange]);
                        weightSum[geneWeightRange] = w[geneRange][geneWeightRange] * genesubarray[geneWeightRange];//genesT1[geneRange2]
                        Variable<double> othersEffect = beta[geneRange] * Variable.Sum(weightSum);// Variable.Logistic(Variable.Sum(weightSum));// Variable.Sum(weightSum); //Variable.Logistic(innerproduct);
                        timeSeriesData[layer+1][geneRange] = selfEffect + othersEffect;
                    }
                }//for each time layer
    
                if (inferenceEngine == null)
                {
                    inferenceEngine = new InferenceEngine();
                }
    
                /* VariableArray<VariableArray<Gaussian>, Gaussian[][]>*/ 
                var wMsg = Variable.Array(Variable.Array<Gaussian>(geneWeightRange), geneRange).Named("wMsg");
                wMsg[geneRange][geneWeightRange] = Variable.Observed(Gaussian.Uniform()).ForEach(geneRange, geneWeightRange);
    
                Variable.ConstrainEqualRandom(w[geneRange][geneWeightRange], wMsg[geneRange][geneWeightRange]);
                w.AddAttribute(QueryTypes.Marginal);
                w.AddAttribute(QueryTypes.MarginalDividedByPrior);
    
                /*double[][] trainingData = new double[2][];
                trainingData[0] = dataMatrix[0];
                trainingData[1] = dataMatrix[1];
                timeSeriesData[0].ObservedValue = trainingData[0];
                timeSeriesData[1].ObservedValue = trainingData[1];*/
    
                Console.WriteLine("First w {0} \n\t*****\t\n", inferenceEngine.Infer<Gaussian[][]>(w));
                
                var wExpected = inferenceEngine.Infer<Gaussian[][]>(w);
    
                Console.WriteLine("\n\n ** type = " + wExpected.GetType()+"\n\n");
                writeFile(wExpected, "wExpected1.txt");
    
                Gaussian[][] wMarginal = null;// = Gaussian.Uniform();
    
                timeSeriesData.ObservedValue = dataMatrix;
                wMarginal = inferenceEngine.Infer<Gaussian[][]>(w);
                //Console.WriteLine("w after batch {0} = \n{1}", i, wMarginal);
                // writeFile(wMarginal, "wMarginal"+i+j);
                wMsg.ObservedValue = inferenceEngine.Infer<Gaussian[][]>(w, QueryTypes.MarginalDividedByPrior);
    
                /*
                for (int i = 2; i < NumLayers.ObservedValue - 1; i++)
                {
                    double[] newData = dataMatrix[i];
                    timeSeriesData[i].ObservedValue = newData;
                    wMarginal = inferenceEngine.Infer<Gaussian[][]>(w);
                    Console.WriteLine("w after batch {0} = \n{1}", i, wMarginal);
                    // writeFile(wMarginal, "wMarginal"+i+j);
                    wMsg.ObservedValue = inferenceEngine.Infer<Gaussian[][]>(w, QueryTypes.MarginalDividedByPrior);
                    
                }*/
    
                
                Console.WriteLine("\n\nExpected   -------- after:\n {0}", wExpected);
                writeFile(wExpected, "wExpectedfinal.txt");
                Console.WriteLine("\n\nActual: {0}", wMarginal);
                writeFile(wExpected, "wMarginal.txt");
                ////////////// end
    
    
            }
    
    
            public void writeFile(Gaussian[][] matrix, string fileName) {
                fileName = "W:\\data\\abidata\\" + fileName;
                using (TextWriter tw = new StreamWriter(fileName))
                {
                    for (int i = 0; i < matrix.Length; i++)
                    {
                        for (int j = 0; j < matrix[0].Length; j++)
                        {
                            tw.Write(matrix[i][j] + " ");
                        }
                        tw.WriteLine();
                    }
                    tw.Close();
                }
                
            }
    
            
    
        }
    
        
    }


    • Edited by RazinR Tuesday, June 17, 2014 3:01 PM
    Tuesday, June 17, 2014 2:48 PM

All replies

  • You are using a product between random variables which has only Experimental support.  Try using the following option (see the Recommender System example):

    engine.Compiler.GivePriorityTo(typeof(GaussianProductOp_SHG09));
    

    Tuesday, June 17, 2014 5:26 PM
    Owner
  • Thank you Tom.

    And the problem of "Not converging for n=0,x=NaN" or "Quadrature found zero variance" is caused by wrong selection of priors? Or problems in data. I mean when I set this option for compiler and I still get these errors. 

    Thanks you in advance.





    • Edited by RazinR Wednesday, June 18, 2014 1:30 PM
    Wednesday, June 18, 2014 10:01 AM
  • A shot in the dark - try adding:

    geneRange.AddAttribute(new Sequential())
    Thursday, June 19, 2014 9:01 AM